Unable to upload new object when CMEK is enabled for a bucket.

[frankyn]

Tracking issue for a customer.

The issue is the Python library does a metadata get on an Object which includes kmsKeyName version resource ID metadata.
If you use the same Blob instance to perform an upload the Python library will use the kmsKeyName version resource ID instead of the kmsKeyName resource ID.
Cloud Storage API expects the kmsKeyName kmsKeyName without version information.

Here's an example for illustration:
kmsKeyName version resource ID:

projects/project-id/us/keyRings/testmrbucket/cryptoKeys/testkey/cryptoKeyVersions/1

kmsKeyName resource ID:

projects/project-id/locations/us/keyRings/testmrbucket/cryptoKeys/testkey

Reproduction

from google.cloud import storage
 
bucket_name = 'your-bucket-name'
blob_name = 'your-object-name'
 
client = storage.Client()
bucket = client.bucket(bucket_name)

# Creates a random encrypted blob.
blob = bucket.blob(blob_name)
blob.upload_from_string("oldcontent")
blob.upload_from_string("newcontent", if_generation_match=blob.generation)

Workaround

from google.cloud import storage
 
bucket_name = 'your-bucket-name'
blob_name = 'your-object-name'
 
client = storage.Client()
bucket = client.bucket(bucket_name)

# Creates a random encrypted blob.
blob = bucket.blob(blob_name)
blob.upload_from_string("oldcontent")
# Store the generation
saved_generation = blob.generation

# Get a new instance of Blob to unset value of kmsKeyName
blob = bucket.blob(blob_name)
blob.upload_from_string("newcontent", if_generation_match=saved_generation)

Potential Fix:

• blob.upload_from_* should only use kmsKeyName if it doesn't end with a version.

[HemangChothani]

@frankyn May i look in to it?

[frankyn]

I'm available through chat for the next few hours if you'd like to discuss this more. Thanks @HemangChothani