Wednesday, October 21, 2009

Optimistic Locking With @Version

The datastore does optimistic locking, so if you update an entity inside a transaction and another request commits an update to either that same entity or some other entity in the same entity group before you commit your transaction, you will get an exception.  This keeps your data from getting stomped on when there are concurrent requests.  However, in the web-app world where clients can and typically do maintain state across requests, there is a far more likely scenario in which your data can get stomped on.  Consider the following scenario involving a bug tracking system:

Time 1: User A brings up the details page for Bug 838.
Time 2: User B brings up the details page for Bug 838.
Time 3: User A assigns Bug 838 to Jim.
Time 4: User B assigns Bug 838 to Sally.

Even though User A and User B updated Bug 838 at different times, User A's update was effectively stomped on by User B.  Sure, the fact that Bug 838 was assigned to Jim at some point may be in the history for the Bug, we can't change the fact that User B made a decision to assign Bug 838 based on out-of-date information.  In a bug tracking system this might not be such a big deal, but if you're doing something like compensation planning you'd much rather have your users receive an exception when they make an update based on out-of-date information.  Fortunately this is easy to implement using JPA/JDO on top of App Engine.  Let's use a Person object with a 'salary' property as an example.

JPA:
@Entity
public class Person {
    @Id
    @GeneratedValue(strategy=GenerationType.IDENTITY)
    private Long id;

    private int salary;

    @Version
    private long version;

    // ...getters and setters
}

public void updateSalary(EntityManager em, Person p, int newSalary) {
    em.getTransaction().begin();
    try {
        p.setSalary(newSalary);
        p = em.merge(p);
        em.getTransaction().commit();
    } catch (RollbackException e) {
        if (e.getCause() instanceof OptimisticLockException) {
            handleVersionConflict(e.getCause(), p);
        } else {
            throw e;
        }
    } finally {
        if (em.getTransaction().isActive()) {
            em.getTransaction().rollback();
        }
    }
}

JDO:

@PersistenceCapable(identityType=IdentityType.APPLICATION)
@Version(strategy=VersionStrategy.VERSION_NUMBER)
public class Person {
    @PrimaryKey
    @Persistent(valueStrategy=IdGeneratorStrategy.IDENTITY)
    private Long id;

    private int salary;


    // ...getters and setters
}

public void updateSalary(PersistenceManager pm, Person p, int newSalary) {
    pm.currentTransaction().begin();
    try {
        p.setSalary(newSalary);
        pm.makePersistent(p);
        pm.currentTransaction().commit();
    } catch (JDOOptimisticVerificationException e) {
        handleVersionConflict(e, p);
    } finally {        
        if (pm.currentTransaction().isActive()) {
            pm.currentTransaction().rollback();
        }
    }
}

If you declare a Version field on your model object, JDO/JPA will compare the value of that field on the instance you are updating with the value of that field in the datastore.  If the version numbers are equal the version number will be incremented and your model object will be persisted.   If the version numbers are not equal the appropriate exception will be thrown.  In the above examples I've caught this exception in the update code itself, but this is really just to illustrate what's going on.  In practice I would most likely let this exception propagate out of the update method and handle it at a higher level, perhaps even as part of a generic exception handler in my presentation code.

Note that there is a performance cost to using this feature.  The datastore does not support updates with predicates the way relational databases do ('update person where version = 3') so in order to perform the version comparison it needs to do an additional get() for the entity that corresponds to your model object.  You'll have to decide for yourself whether or not the cost of the additional fetch is worthwhile.

Now, if you've looked at both the JPA and JDO examples you may have noticed that JPA requires the declaration of an explicit field annotated with @Version while JDO accepts an @Version annotation on the class without an explicit field.  If you are using JDO and would like the version stored in an explicit field you can use a DataNucleus-specific extension to tell JDO which field to use.  You can see an example of this here.

Using JPA and JDO's built-in versioning checks is an easy way to help users of your app make decisions based on up-to-date information.  Try it out!

142 comments:

  1. Great tips! I still scratching my head on how to solve this kind of problem using memcache, reader writer lock...

    ReplyDelete
  2. this blog is a great initiative, thanks for the tips!

    ReplyDelete
  3. when Time
    3: User A assigns Bug 838 to Jim.

    Time 4: User B assigns Bug 838 to Sally.

    when user B do merge() there is no need to do explicit checking of version number already changed by userA before calling merge() ?

    ReplyDelete
  4. Sorry, I don't understand your question.

    ReplyDelete
  5. @Max, may i know how to test for optismic lock?

    I loaded same entity record on 2 separate browser window then press submit (hibernate template.merge), version number incremented for both browser window, but never caught any problem with optimistic lock..

    ReplyDelete
  6. You'll want to make sure that you're submitting the version number of the object when it was read along with the data you plan to update and then ensure that the object you're updating has the version number that was submitted.

    You ca also see a unit-test of this functionality here:
    http://code.google.com/p/datanucleus-appengine/source/browse/trunk/tests/org/datanucleus/store/appengine/JPAUpdateTest.java#131

    The key is that the version number in the datastore doesn't match the version number in the object being updated.

    ReplyDelete
  7. Thanks for your post, and many thanks for your work !

    But doing a second query to check the last version number is not suffisant to guarantee Optimistic Locking (OL). The time between the second query and the moment where the entity is really stored can be arbitrary long, so another write can happens during this time. I think the test is not working for testing OL with concurrent updates.

    So does GAE datastore has built in OL ?
    As far as I explored the API, it seems that it isn't. So if such feature does not come, we will unfortunately never be able to do real OL.

    ReplyDelete
  8. Ok GAE datastore has build in OL. It throws ConcurrentModificationException when an object is modified concurrently by another transaction.
    So the actual implementation of the @Version should work. Nice job !

    ReplyDelete
  9. That's right, the GAE datastore has built-in optimistic locking for txns. Just remember that the locking is done per entity-group, not per record.

    ReplyDelete
  10. Thanks for your post.

    I tried implementing @version to my app which uses GWT for a client. The problem is I can not seem to make the locking work.

    SCENARIO:

    From list, the user selects a object for update. Update event passes the updated object to a service method (which updates data).

    The method doesn't throw an exception and it allows update.

    As a workaround, I am explicitly comparing the version currently at hand and the one currently stored.

    What is the best way to implement locking (when using GWT). Thanks a lot?

    ReplyDelete
  11. If the version of the object you're updating doesn't match the version in the datastore you should get an exception. Can you boil the problem down to a simple unit test with your model object? You fetch your object, then update the version directly in the datastore using the low-level api, then try to update your object.

    ReplyDelete
  12. This comment has been removed by a blog administrator.

    ReplyDelete
  13. This comment has been removed by a blog administrator.

    ReplyDelete
  14. This comment has been removed by a blog administrator.

    ReplyDelete
  15. Hi Max

    Your blog is a real treasure when starting to program with the gae datastore.

    I could not set up a unit test where I get an exception when testing the @version using JDO and GAE SDK 1.3.2

    I have the following scenario:
    I persisted person it with salary 10
    read it into result1
    read it into result2
    re-persisted result1 with 20
    re-persisted result2 with 30
    I expected the 2nd re-persist to fail but it did not and the final result when rereading it was 30.

    1) Where do I make wrong assumptions?
    2) Does anyone have a simple JDO unit test for the @Version?
    3) I thought I could implement uniqueness through this version handling, I would like to make sure that a person can only be created once by using a explicit primary key (not IdGeneratorStrategy.IDENTITY) and versions. Is that possible? I could not test that yet since I was not able to set up any unit test for the @version mechanism. :'(

    Thanks a lot,
    Claus

    ReplyDelete
  16. I just made a test using "explicit" versions...

    @PersistenceCapable(identityType = IdentityType.APPLICATION)
    @Version(strategy = VersionStrategy.VERSION_NUMBER, extensions = {@Extension(vendorName = "datanucleus", key = "field-name", value = "versionField")})

    the versionField (Long) is set to 1 when persiting a new object but never changes in my scenario (she post before).

    Even setting this field explicitly to 5 does not throw an exception.

    What do I miss?

    ReplyDelete
  17. @Version is not a good mechanism for enforcing uniqueness, it only detects changes in long-running transactions. If you want to make sure a person can only be created once then using a named Key, where you provide the name, is the correct approach. You'll need to perform a transactional read before you attempt to create the person, but this is pretty straightforward:

    1) beginTxn();
    2) Person p = lookupPersonByName("x");
    3) if (p != null) {
    4) throw new PersonAlreadyExists("x");
    5) }
    6) createNewPerson(p);
    7) commitTxn();

    if a person with uniquely identified by "x" gets created by a different thread in between lines 2 and 7 you'll receive an exception when you attempt to commit due to a concurrent modification of the entity group.

    For @Version, here's a test that you may find useful:
    http://code.google.com/p/datanucleus-appengine/source/browse/trunk/tests/org/datanucleus/store/appengine/JDOUpdateTest.java#111

    Hope this helps,
    Max

    ReplyDelete
  18. Hell Max,

    Yes, this will help ... I just could not imagine that there is such a thing as read-locks in the datastore that is why I did not investigate further on the transactions.

    Thanks for the test ... I will give it a try but I was told a well that I can NOT relay on the functionality of the local version.

    Thanks a lot,
    Claus

    ReplyDelete
  19. The best celebration of new year, I think to celebrate in New York city because new York city is one of the famous and biggest towns in the world. It is large city thanksgiving day gif clipart
    funny thanksgiving wishes
    beautiful christmas images 2016
    new year 2017 images and also has a significant population.

    ReplyDelete
  20. seo is good process for rankingWhat is seo process What is pbn networksDigital Marketing Career earn money online by Digital MarketingWhat is seo scope of seoHow to convert traffic into leads
    Digital Marketing course in faridabad

    ReplyDelete
  21. Happy Valentine Day greetings and sms for boyfriend/girlfriend
    Happy Valentine Day 2019 wishes for girlfriend/boyfriend

    Happy Valentine Day Celebration 2019 Facebook & whatsapp sms

    Best SEO training in Faridabad Learn and become seo expert with us

    happy rose day 2019 wishes images quotes messages for whatsapp, facebook and twitter

    happy rose day 2019 wishes for facebook ,whatsapp and instagram

    ReplyDelete
  22. This comment has been removed by the author.

    ReplyDelete
  23. Valentine Week List 2019  love day in February

    Happy Anniversary Wishes Happy Anniversary Messages for Husband/ Wife

    Best digital marketing course in Faridabad Webhelpy is the best digital marketing training provider

    Happy Maha Shivratri Date Maha Shiv Ratri Wishes Images 2019 pooja vidhi

    SEO training in Faridabad Webhelpy is the best place to learn SEO

    ReplyDelete
  24. gst Advisors in Faridabad  to get more details in indirect tax
    Professional Tax Consultant in Faridabad
    digital book on gst ebook 
    gst consultants in india
    gst consultancy in india
    indirect tax consultancy in india to make the law more clear
    indirect tax consultancy in faridabad
    Professional Tax Consultant in Faridabad best tax consultant
    gst ebook To make the law more clear, we provide you with the elaborative interpretation of the law in this GST E-Book.

    ReplyDelete
  25. gst Advisors in Faridabad  to get more details in indirect tax
    Professional Tax Consultant in Faridabad
    digital book on gst ebook 
    gst consultants in india
    gst consultancy in india
    indirect tax consultancy in india to make the law more clear
    indirect tax consultancy in faridabad
    Professional Tax Consultant in Faridabad best tax consultant
    gst ebook To make the law more clear, we provide you with the elaborative interpretation of the law in this GST E-Book.

    ReplyDelete
  26. gst Advisors in Faridabad  to get more details in indirect tax
    Professional Tax Consultant in Faridabad
    digital book on gst ebook 
    gst consultants in india
    gst consultancy in india
    indirect tax consultancy in india to make the law more clear
    indirect tax consultancy in faridabad
    Professional Tax Consultant in Faridabad best tax consultant
    gst ebook To make the law more clear, we provide you with the elaborative interpretation of the law in this GST E-Book.

    ReplyDelete
  27. gst Advisors in Faridabad  to get more details in indirect tax
    Professional Tax Consultant in Faridabad
    digital book on gst ebook 
    gst consultants in india
    gst consultancy in india
    indirect tax consultancy in india to make the law more clear
    indirect tax consultancy in faridabad
    Professional Tax Consultant in Faridabad best tax consultant
    gst ebook To make the law more clear, we provide you with the elaborative interpretation of the law in this GST E-Book.

    ReplyDelete
  28. gst Advisors in Faridabad  to get more details in indirect tax
    Professional Tax Consultant in Faridabad
    digital book on gst ebook 
    gst consultants in india
    gst consultancy in india
    indirect tax consultancy in india to make the law more clear
    indirect tax consultancy in faridabad
    Professional Tax Consultant in Faridabad best tax consultant
    gst ebook To make the law more clear, we provide you with the elaborative interpretation of the law in this GST E-Book.

    ReplyDelete
  29. Easter Festival 2019 - Easter Dresses Gift For Girls best dresses gift for baby girls, best dresses gift for girls

    Happy Easter Festival - Decorating Ideas and Tips decorating ideas and tips for celebrating Easter festival

    Easter Celebration - Best Dresses Ideas For Women best dresses gifts ideas for women

    Happy Easter Festival Wishes 2019 - family, parents, friends Send wishes, quotes and greetings to your family, friends and relatives with images.

    ReplyDelete
  30. Easter Festival 2019 - Easter Dresses Gift For Girls best dresses gift for baby girls, best dresses gift for girls

    Happy Easter Festival - Decorating Ideas and Tips decorating ideas and tips for celebrating Easter festival

    Easter Celebration - Best Dresses Ideas For Women best dresses gifts ideas for women

    Happy Easter Festival Wishes 2019 - family, parents, friends Send wishes, quotes and greetings to your family, friends and relatives with images.

    ReplyDelete
  31. The Tango Yogi a youtube channel

    https://www.youtube.com/channel/UCU9LIxJGG8B09ZVnShbNs2Q

    ReplyDelete
  32. gst  best gst consultancygst Advisors in Faridabad  to get more details in indirect taxProfessional Tax Consultant in Faridabaddigital book on gst ebook gst consultants in indiagst consultancy in indiaindirect tax consultancy in india to make the law more clearindirect tax consultancy in faridabadProfessional Tax Consultant in Faridabad best tax consultantgst ebook To make the law more clear, we provide you with the elaborative interpretation of the law in this GST E-Book.buy gst ebookonline buy gst ebookgst ebook software gst book online gst ebookonline buy gst book  

    ReplyDelete
  33. Earn Money with Google   Earn Money online from google without investment

    Earn Money Without Investment Earn Money without any investments from google

    Earn Money Online For students without any investments make money in free time.

    Earn Money Form Home without Investment  Do part time job for earning extra income

    earn money with google without investment    earn money only through google without any investment

    Earn Money online through google Earn extra money on google without payment

    google earn money from home  You can earn money from home without any investment in your free time

    How can I earn money online    Extra income with google from home

    Digital Marketing Course in delhi  Best digital marketing course in Faridabad
    Digital Marketing Course in delhi NCR   Best Institute for Digital Marketing course in delhi Ncr
    Digital Marketing Institute in Delhi   Best Digital Marketing training provider in Delhi
    Digital Marketing Institute in Faridabad   Learn digital marketing from best institute of Faridabad
    best digital marketing course in delhi  Grow your career in digital marketing with the help of professional trainer
    Best digital marketing course in Faridabad  Best Institute for Digital Marketing course in delhi Ncr
    Best Digital Marketing Institute in Delhi NCR   Best Digital Marketing training provider in Delhi
    Best Institute for Digital Marketing course in Delhi  Best digital marketing course in Faridabad

    ReplyDelete
  34. Seo Course in Delhi  Seo training provider in Delhi
    Seo Course in Faridabad  best Seo course in Faridabad
    Seo Training Institute in Delhi Best Seo training provider in Delhi NCR
    Seo Training Institute in Faridabad  Learn Seo from best institute of Faridabad
    Seo Company In Faridabad      Do internship or job in Faridabad
    Seo Training Institute  Seo Training course from Delhi NCR
    Seo Training in Delhi NCR       Seo training provider in Delhi

    What is Artificial Intelligence    What is Artificial Intelligence - scope and benefits of AI.

    Scope Of Artificial intelligence     Scope of Artificial intelligence and career in AI.

    Scope in AI     What is Artificial Intelligence - scope and benefits of AI

    Career in Artificial Intelligence   Scope of Artificial intelligence and career in AI.

    What is AI    What is Artificial Intelligence - scope and benefits of AI.

    web development training institute  Website designing course in Faridabad

    website development faridabad   Website development training  institute in Delhi NCR

    web designing company in faridabad Website development course in Faridabad

    website designing in faridabad Website designing course in Faridabad

    ReplyDelete
  35. Google adword interview questions Some questions for google adword  for clearing interviews

    adword interview question Interview questions for google adword

    Interview question for google adword  Some questions for google adword  for clearing interviews

    Google Adsense Training in Faridabad  Best training institute for google adsense in faridabad

    PPC training in Faridabad    Google Adsense training institute in Delhi NCR

    Google Adsense Training Institute in Delhi NCR Best training institute for google adsense in Delhi Ncr

    Earn Money from Home  earn money only through digital marketing without any investment

    Earn money online from google  Earn extra money in free time

    Earn Money Online without any investments make money in free time.

    Earn money with Digital Marketing Earn extra money without payment.

    ReplyDelete
  36. Thank you so much for providing individuals with such a wonderful chance to read articles and post posts from here. It is often very nice and packed with amusement for me personally and my office acquaintances to visit your web site particularly thrice weekly to study the latest guides you have. And of course, we are certainly amazed with all the incredible inspiring ideas served by you. Some two points in this post are without a doubt the very best we've had. private jet price in india

    ReplyDelete
  37. Thank you so much for providing individuals with such a wonderful chance to read articles and post posts from here. It is often very nice and packed with amusement for me personally and my office acquaintances to visit your web site particularly thrice weekly to study the latest guides you have. And of course, we are certainly amazed with all the incredible inspiring ideas served by you. Some two points in this post are without a doubt the very best we've had. [url=https://www.thewayanadresorts.com/rented-two-wheelers-wayanad.html]rented two wheelers wayanad[/url]

    ReplyDelete
  38. Thank you so much for providing individuals with such a wonderful chance to read articles and post posts from here. It is often very nice and packed with amusement for me personally and my office acquaintances to visit your web site particularly thrice weekly to study the latest guides you have. And of course, we are certainly amazed with all the incredible inspiring ideas served by you. Some two points in this post are without a doubt the very best we've had. rented two wheelers wayanad

    ReplyDelete
  39. gst  best gst consultancygst Advisors in Faridabad  to get more details in indirect taxProfessional Tax Consultant in Faridabaddigital book on gst ebook gst consultants in indiagst consultancy in indiaindirect tax consultancy in india to make the law more clearindirect tax consultancy in faridabadProfessional Tax Consultant in Faridabad best tax consultantgst ebook To make the law more clear, we provide you with the elaborative interpretation of the law in this GST E-Book.buy gst ebookonline buy gst ebookgst ebook software gst book online gst ebookonline buy gst book  

    ReplyDelete
  40. Reiki first degree training in Faridabad | Vedic Seeds Astro Vastu Reiki first degree training in Faridabad | Vedic Seeds Astro Vastu – Reiki is helpful alternate medical therapy that help to recover

    ReplyDelete
  41. Reiki second degree training in Faridabad | Vedic Seeds Astro Vastu Reiki first degree training in Faridabad | Vedic Seeds Astro Vastu – Reiki is helpful alternate medical therapy that help to recover

    ReplyDelete
  42. Reiki first degree training in Faridabad | Vedic Seeds Astro Vastu Reiki first degree training in Faridabad | Vedic Seeds Astro Vastu – Reiki is helpful alternate medical therapy that help to recover

    Reiki second degree training in Faridabad

    ReplyDelete
  43. Vedic seeds provide you all services remedies to help in your trouble. ``Do you karma make your future`` We believe in results and benefits which you get.If you have questions about your future, the answers are in the palm of your hands! We are with you now and after your work done, for your services.To get to the questions that arise in everyday get in here to introduce you to vedic Seeds Astro Vastu, who is one of astrologer in Faridabad in the service of society...... http://www.vedicseedsastrovastu.com/

    ReplyDelete
  44. Vedic seeds provide you all services remedies to help in your trouble. ``Do you karma make your future`` We believe in results and benefits which you get.If you have questions about your future, the answers are in the palm of your hands! We are with you now and after your work done, for your services.To get to the questions that arise in everyday get in here to introduce you to vedic Seeds Astro Vastu, who is one of astrologer in Faridabad in the service of society more information ......... http://www.vedicseedsastrovastu.com/

    ReplyDelete
  45. Vedic Seeds Astro Vastu a top astrologer deals in providing career astrology based services and advice along with astro insights. Vedic seeds provide you all services remedies to help in your trouble. Do your karma make your future.We believe in results and benefits which you get.
    To Get more information visit at:www.vedicseedsastrovastu.com

    ReplyDelete
  46. gst  best gst consultancygst Advisors in Faridabad  to get more details in indirect taxProfessional Tax Consultant in Faridabaddigital book on gst ebook gst consultants in indiagst consultancy in indiaindirect tax consultancy in india to make the law more clearindirect tax consultancy in faridabadProfessional Tax Consultant in Faridabad best tax consultantgst ebook To make the law more clear, we provide you with the elaborative interpretation of the law in this GST E-Book.buy gst ebookonline buy gst ebookgst ebook software gst book online gst ebookonline buy gst book  

    ReplyDelete
  47. Best GST consultancy Lawcrux advisors are the best GST consultancy in India
    gst Advisors in Faridabad  to get more details in indirect tax
    Professional Tax Consultant in Faridabad
    digital book on gst ebook 
    gst consultants in india
    gst consultancy in india
    indirect tax consultancy in india to make the law more clear
    indirect tax consultancy in Faridabad
    Professional Tax Consultant in Faridabad best tax consultant
    gst ebook To make the law more clear, we provide you with the elaborative interpretation of the law in this GST E-Book.
    Best GST ebook which is based on indirect taxes
    online buy gst ebook FTP, Fema, Custom, Service Tax, Excise
    GST ebook software for professionals
    GST Ebook best ebook based on indirect taxation
    online GST ebook on GST
    online buy GST book

    ReplyDelete
  48. It is really a nice and helpful piece of information. I’m glad that you shared this helpful info with us. Please keep us informed like this. Thank you for sharing. Private Jet

    ReplyDelete