Reading using open api fails with 416 error code in google-resumable-media

[amardeep]

• OS type and version: Ubuntu
• Python version: 3.7.5
• pip version: 9.0.1
• google-cloud-storage version: 1.37.0
• google-resumable-media version: 1.2.0

While trying to copy a gcs file using blob.open method to read and write, it fails at the last read step with the following error:

google.resumable_media.common.InvalidResponse: ('Request failed with status code', 416, 'Expected one of', <HTTP
Status.OK: 200>, <HTTPStatus.PARTIAL_CONTENT: 206>)

Code example

In the snippet below i am using shutil.copyfileobj, but same error happens if I i loop and call read with a chunk size.

import shutil

from google.cloud import storage

client = storage.Client()
bucket: storage.Bucket = client.bucket("gcp-public-data-landsat")
blob = bucket.get_blob("index.csv.gz")
src = blob.open("rb", chunk_size=40 * 1024 * 1024)
dest = open("index.csv.gz", "wb")
shutil.copyfileobj(src, dest, 40 * 1024 * 1024)

Stack trace

Original exception was:
Traceback (most recent call last):
  File "/home/amardeeps/.local/lib/python3.7/site-packages/google/cloud/storage/client.py", line 728, in downloa
d_blob_to_file
    checksum=checksum,
  File "/home/amardeeps/.local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 961, in _do_downl
oad
    response = download.consume(transport, timeout=timeout)
  File "/home/amardeeps/.local/lib/python3.7/site-packages/google/resumable_media/requests/download.py", line 16
8, in consume
    self._process_response(result)
  File "/home/amardeeps/.local/lib/python3.7/site-packages/google/resumable_media/_download.py", line 186, in _p
rocess_response
    response, _ACCEPTABLE_STATUS_CODES, self._get_status_code
  File "/home/amardeeps/.local/lib/python3.7/site-packages/google/resumable_media/_helpers.py", line 104, in req
uire_status_code
    *status_codes
google.resumable_media.common.InvalidResponse: ('Request failed with status code', 416, 'Expected one of', <HTTP
Status.OK: 200>, <HTTPStatus.PARTIAL_CONTENT: 206>)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "storage-test.py", line 10, in <module>
    shutil.copyfileobj(src, dest, 40 * 1024 * 1024)
  File "/usr/lib/python3.7/shutil.py", line 79, in copyfileobj
    buf = fsrc.read(length)
  File "/home/amardeeps/.local/lib/python3.7/site-packages/google/cloud/storage/fileio.py", line 97, in read
    start=fetch_start, end=fetch_end, **self._download_kwargs
  File "/home/amardeeps/.local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 1307, in download
_as_bytes
    checksum=checksum,
  File "/home/amardeeps/.local/lib/python3.7/site-packages/google/cloud/storage/client.py", line 731, in downloa
d_blob_to_file
    _raise_from_invalid_response(exc)
  File "/home/amardeeps/.local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 4061, in _raise_f
rom_invalid_response
    raise exceptions.from_http_status(response.status_code, message, response=response)
google.api_core.exceptions.RequestRangeNotSatisfiable: 416 GET https://storage.googleapis.com/download/storage/v
1/b/gcp-public-data-landsat/o/index.csv.gz?generation=1616911592995216&alt=media: Request range not satisfiable:
 ('Request failed with status code', 416, 'Expected one of', <HTTPStatus.OK: 200>, <HTTPStatus.PARTIAL_CONTENT: 
206>)