Skip to content

Datastore - Inconsistent behavior of transactions between Datastore and Firestore in Datastore Mode #3582

Description

@sai-pullabhotla

I'm seeing inconsistency in transactions behavior between Cloud Datastore and Cloud Firestore running in Datastore mode. The sample code below runs fine when working with Cloud Datastore, but fails when working with Cloud Firestore running in Datastore mode.

import com.google.cloud.datastore.Datastore;
import com.google.cloud.datastore.DatastoreOptions;
import com.google.cloud.datastore.FullEntity;
import com.google.cloud.datastore.IncompleteKey;
import com.google.cloud.datastore.Transaction;

public class TransactionTest {

  public static void main(String[] args) {
    //change it to a project ID that has plain old Datastore and then try again 
    //with a different project that has Firestore running in Datastore mode. 
    final String projectId = "xxx-xxx";
    Datastore datastore = DatastoreOptions.newBuilder().setProjectId(projectId).build()
        .getService();
    IncompleteKey incompleteKey = datastore.newKeyFactory().setKind("TransactionTest").newKey();
    FullEntity<IncompleteKey> fullEntity = FullEntity.newBuilder().setKey(incompleteKey)
        .set("name", "Bob").build();
    com.google.cloud.datastore.Entity entity = datastore.add(fullEntity);

    Transaction t1 = datastore.newTransaction();
    try {
      com.google.cloud.datastore.Entity entity2 = t1.get(entity.getKey());
      update(entity, datastore);
      t1.commit();
      System.out.println("t1 is committed successfully");
    } finally {
      if (t1.isActive()) {
        t1.rollback();
      }
    }

  }

  private static void update(com.google.cloud.datastore.Entity entity, Datastore datastore) {
    Transaction t2 = datastore.newTransaction();
    try {
      t2.update(entity);
      t2.commit();
      System.out.println("t2 is committed successfully");
    } finally {
      if (t2.isActive()) {
        t2.rollback();
      }
    }
  }

}

Running this code against good old Datastore completes normally and I get the below printed to the console:

t2 is committed successfully
t1 is committed successfully

However, running this against a different project_id, which has Firestore running in Datastore mode, produces the below exception, after running for about a minute:

Exception in thread "main" com.google.cloud.datastore.DatastoreException: The referenced transaction has expired or is no longer valid.
	at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.translate(HttpDatastoreRpc.java:129)
	at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.translate(HttpDatastoreRpc.java:114)
	at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.rollback(HttpDatastoreRpc.java:173)
	at com.google.cloud.datastore.DatastoreImpl$6.call(DatastoreImpl.java:530)
	at com.google.cloud.datastore.DatastoreImpl$6.call(DatastoreImpl.java:527)
	at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:89)
	at com.google.cloud.RetryHelper.run(RetryHelper.java:74)
	at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:51)
	at com.google.cloud.datastore.DatastoreImpl.rollback(DatastoreImpl.java:527)
	at com.google.cloud.datastore.DatastoreImpl.rollbackTransaction(DatastoreImpl.java:522)
	at com.google.cloud.datastore.TransactionImpl.rollback(TransactionImpl.java:120)
	at com.jmethods.catatumbo.TransactionTest.main(TransactionTest.java:30)
Caused by: com.google.datastore.v1.client.DatastoreException: The referenced transaction has expired or is no longer valid., code=INVALID_ARGUMENT
	at com.google.datastore.v1.client.RemoteRpc.makeException(RemoteRpc.java:226)
	at com.google.datastore.v1.client.RemoteRpc.makeException(RemoteRpc.java:275)
	at com.google.datastore.v1.client.RemoteRpc.call(RemoteRpc.java:186)
	at com.google.datastore.v1.client.Datastore.rollback(Datastore.java:111)
	at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.rollback(HttpDatastoreRpc.java:171)
	... 9 more

Any idea why this difference? Which one is behaving correctly, Datastore or Firestore in Datastore mode?

Activity

  1. added
    type: questionRequest for information or clarification. Not an issue.
    api: datastoreIssues related to the Datastore API.
    priority: p2Moderately-important priority. Fix may not be included in next release.
    api: firestoreIssues related to the Firestore API.
    on Aug 20, 2018
  2. removed
    🚨 criticalP0 critical issue. Requires immediate fix
    priority: p2Moderately-important priority. Fix may not be included in next release.
    on Feb 19, 2019
  3. athakor commented on Aug 13, 2019

    @athakor
    Contributor

    @sai-pullabhotla

    As per the google documentation Firestore in datastore mode has all the existing functionality of google cloud datastore with extra features.Currently you are getting DatastoreException "The referenced transaction has expired or is no longer valid." just because of the entity that you are using for transactions.

    For the transaction t2 you change the nature of entity t2.update(entity) and then commit transaction t2. Transaction t1 also refer same entity t1.get(entity.getKey()) and while committing t1 you will get Datastore exception.Could you please check method level comment of get(Key key) method.

    For the transaction T1 could you please modified your code something like below because it's working fine in both project cloud datastore and firestore in datastore mode.

    try {
          update(entity, datastore);
          com.google.cloud.datastore.Entity entity2 = t1.get(entity.getKey());
          t1.commit();
          System.out.println("t1 is committed successfully");
        } finally {
          if (t1.isActive()) {
            t1.rollback();
          }
        }
    
  4. athakor commented on Sep 11, 2019

    @athakor
    Contributor

    closing this issue for now due to staleness and if above reply didn't help feel free to reopen it.

  5. self-assigned this
    on Sep 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

api: datastoreIssues related to the Datastore API.api: firestoreIssues related to the Firestore API.type: questionRequest for information or clarification. Not an issue.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions