chore(migration): Migrate code from googleapis/python-storage into packages/google-cloud-storage#16192
chore(migration): Migrate code from googleapis/python-storage into packages/google-cloud-storage#16192chalmerlowe wants to merge 1238 commits intomainfrom
Conversation
* chore: update secret manager in kokoro * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
#1353) * test: test universe domain client only in prod * unflake hmac snippet test
Source-Link: googleapis/synthtool@71a7297 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:e8dcfd7cbfd8beac3a3ff8d3f3185287ea0625d859168cc80faccfc9a7a00455 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: cojenco <cathyo@google.com>
Co-authored-by: Holt Skinner <13262395+holtskinner@users.noreply.github.com> Co-authored-by: cojenco <cathyo@google.com>
* chore(deps): update all dependencies * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
…nloading using transfer_manager (#1357) * fix: do not spam the log with checksum related INFO messages when downloading using transfer_manager `download_chunks_concurrently` function does not allow to set `checksum` field in `download_kwargs`. It also does not set it on its own so it takes the default value of `"md5"` (see `Blob._prep_and_do_download`). Because ranged downloads do not return checksums it results in a lot of INFO messages (tens/hundreds): ``` INFO google.resumable_media._helpers - No MD5 checksum was returned from the service while downloading ... (which happens for composite objects), so client-side content integrity checking is not being performed. ``` To fix it set the `checksum` field to `None` which means no checksum checking for individual chunks. Note that `transfer_manager` has its own checksum checking logic (enabled by `crc32c_checksum` argument) * fix tests
* feat: add support for restore token * add unit tests coverage * update docstrings * fix docs
* feat: IAM signBlob retries * support universe domain and update tests * update test credentials * use ud signing bucket fixture
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Integrate the google-resumable-media library into python-storage. --------- Co-authored-by: cojenco <cathyo@google.com>
…ring()` to `from_uri()` (#1335)
#1403) Source-Link: googleapis/synthtool@de3def6 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:a1c5112b81d645f5bbc4d4bbc99d7dcb5089a52216c0e3fb1203a0eeabadd7d5 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces significant updates to the Google Cloud Storage library, including the addition of new experimental asyncio support, updated Kokoro build configurations, and refined metadata. My review identified several critical issues, including missing module imports in experimental files, a typo in an error message, and inconsistencies between the documentation and the actual Python version requirements specified in setup.py. I have also recommended removing unreachable version-check code that conflicts with the new minimum Python 3.10 requirement.
| import warnings | ||
|
|
||
| # Import everything from the new stable module | ||
| from google.cloud.storage.grpc_client import * # noqa |
There was a problem hiding this comment.
This file attempts to import from google.cloud.storage.grpc_client, but this file does not seem to be included in the pull request. This will cause an ImportError at runtime. Please ensure the target module file is also added.
| import warnings | ||
|
|
||
| # Import everything from the new stable module | ||
| from google.cloud.storage.asyncio.retry.writes_resumption_strategy import * # noqa |
There was a problem hiding this comment.
This file attempts to import from google.cloud.storage.asyncio.retry.writes_resumption_strategy, but this file does not seem to be included in the pull request. This will cause an ImportError at runtime. Please ensure the target module file is also added.
| if isinstance(credentials, auth_credentials.AnonymousCredentials): | ||
| if client_options is None or client_options.api_endpoint is None: | ||
| raise ValueError( | ||
| "Either client_options or `client_option.api_endpoint` is None. Please provide api_endpoint when `AnonymousCredentials` is used " |
There was a problem hiding this comment.
There's a typo in the error message string. client_option.api_endpoint should be client_options.api_endpoint. If this error is raised, it will cause a NameError due to the typo.
| "Either client_options or `client_option.api_endpoint` is None. Please provide api_endpoint when `AnonymousCredentials` is used " | |
| "Either client_options or client_options.api_endpoint is None. Please provide api_endpoint when AnonymousCredentials is used " |
| "api_id": "storage.googleapis.com", | ||
| "requires_billing": true, | ||
| "default_version": "v2", | ||
| "codeowner_team": "@googleapis/cloud-sdk-python-team @googleapis/gcs-team @googleapis/gcs-fs", |
There was a problem hiding this comment.
There appears to be a typo in the codeowner_team value. It's specified as @googleapis/gcs-fs, but based on the .github/CODEOWNERS file, it should likely be @googleapis/gcs-fs-team.
| "codeowner_team": "@googleapis/cloud-sdk-python-team @googleapis/gcs-team @googleapis/gcs-fs", | |
| "codeowner_team": "@googleapis/cloud-sdk-python-team @googleapis/gcs-team @googleapis/gcs-fs-team", |
| - The feature must work fully on the following CPython versions: | ||
| 3.7, 3.8, 3.9, 3.10, 3.11, 3.12, 3.13 and 3.14 on both UNIX and Windows. |
There was a problem hiding this comment.
The documentation states that Python 3.7, 3.8, and 3.9 are supported, but setup.py specifies python_requires='>=3.10'. Please update this documentation, as well as README.rst, to reflect that the minimum supported Python version is 3.10. Additionally, when a release introduces breaking changes in environment requirements, such as dropping support for specific Python versions, prefer a minor version bump over a patch version bump.
References
- When a release introduces breaking changes in environment requirements, such as dropping support for specific Python versions, prefer a minor version bump over a patch version bump to maintain the ability to provide patches for the previous minor version.
| Our client libraries are compatible with all current `active`_ and `maintenance`_ versions of | ||
| Python. | ||
|
|
||
| Python >= 3.7 |
There was a problem hiding this comment.
The supported Python version listed here (>= 3.7) is inconsistent with setup.py, which specifies python_requires='>=3.10'. Please update the documentation to reflect the actual minimum supported version. Per repository guidelines, dropping support for Python versions should be handled with a minor version bump.
References
- When a release introduces breaking changes in environment requirements, such as dropping support for specific Python versions, prefer a minor version bump over a patch version bump to maintain the ability to provide patches for the previous minor version.
| if sys.version_info < (3, 9): | ||
| warnings.warn( | ||
| "You are using a non-supported Python version " | ||
| + f"({_py_version_str}). Google will not post any further " | ||
| + f"updates to {_package_label} supporting this Python version. " | ||
| + "Please upgrade to the latest Python version, or at " | ||
| + f"least to Python 3.9, and then update {_package_label}.", | ||
| FutureWarning, | ||
| ) | ||
| if sys.version_info[:2] == (3, 9): | ||
| warnings.warn( | ||
| f"You are using a Python version ({_py_version_str}) " | ||
| + f"which Google will stop supporting in {_package_label} in " | ||
| + "January 2026. Please " | ||
| + "upgrade to the latest Python version, or at " | ||
| + "least to Python 3.10, before then, and " | ||
| + f"then update {_package_label}.", | ||
| FutureWarning, | ||
| ) |
There was a problem hiding this comment.
This block of code checks for Python versions less than 3.10. However, setup.py requires Python >= 3.10, which makes this code unreachable. It should be removed to improve code clarity.
…with handcrafted version
20 participants
See #10956.
This PR should be merged with a merge-commit, not a squash-commit, in order to preserve the git history.