gsutil cp succeeds but upload_from_filename() method from storage client fails

I’ve gotten to the bottom of this problem. It was twofold.

  1. I discovered that the code would succeed if the storage object destination_blob_name did not already exist. If the object did exist then the error explained above would occur.
  2. Permission for $ACCOUNT to access the bucket was provided by a custom role that was applied to the bucket. When I added permission storage.objects.get & storage.objects.delete to that custom role then my code succeeded, even if the object already existed.

I’ve discovered that storage.objects.get & storage.objects.delete are described as

  • Read object data and metadata, excluding ACLs
  • Delete objects

https://cloud.google.com/storage/docs/access-control/iam-permissions#object_permissions

The learning I’ve taken away from this is that under certain circumstances (e.g. when the destination object already exists) then gsutil cp requires different permissions to google.cloud.storage.blob.upload_from_filename(). If anyone out there can elucidate the difference between those two operations then I’d love to understand it better.

CLICK HERE to find out more related problems solutions.

Leave a Comment

Your email address will not be published.

Scroll to Top