build: fix dd-trace-py git overrides and filter publish layer check#824
Open
joeyzhao2018 wants to merge 2 commits intomainfrom
Open
build: fix dd-trace-py git overrides and filter publish layer check#824joeyzhao2018 wants to merge 2 commits intomainfrom
joeyzhao2018 wants to merge 2 commits intomainfrom
Conversation
- build_layers.sh: force the `ddtrace` package name when overriding via
DD_TRACE_COMMIT or DD_TRACE_COMMIT_BRANCH. Git source builds always
produce the `ddtrace` package; the `ddtrace_serverless` variant only
exists as pre-built S3 wheels from dd-trace-py CI, so the previous
`${wheel_basename}` substitution caused pip to reject the install
with a name/metadata mismatch.
- publish_layers.sh: only require layer zips for the layers in LAYERS,
rather than unconditionally requiring all 14 entries in LAYER_PATHS.
When LAYERS is unset it still defaults to all layers, so release
pipelines that build everything are unaffected.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
|
I'm not sure if I'm mixing up two things here, but problem 1 might be getting solved by DataDog/dd-trace-py#17964? |
Contributor
Author
@rithikanarayan Yeah, probably. We can close this pr if that one get merged and working |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two small build-script fixes I hit while trying to build a single layer from a dd-trace-py feature branch:
scripts/build_layers.sh—DD_TRACE_COMMITandDD_TRACE_COMMIT_BRANCHwere broken. Both paths build dd-trace-py from a git checkout, but the script wroteddtrace_serverless = { git = ... }intopyproject.toml. Source checkouts publish project metadata asddtrace, notddtrace_serverless— theddtrace_serverlessvariant only exists as pre-built wheels from dd-trace-py's CI (consumed viaUPSTREAM_PIPELINE_ID). pip rejected the install with:Fix: hard-code
ddtraceas the package name in the two git-override branches.DD_TRACE_WHEELandUPSTREAM_PIPELINE_IDpaths are unchanged, so the release pipeline is unaffected.scripts/publish_layers.sh— the pre-flight check looped over the full hard-codedLAYER_PATHS(14 entries) and bailed if any zip was missing, even whenLAYERSwas set to a single layer. So you couldn't publish one layer without first building all 14 zips. Moved the check to afterLAYERSis resolved, and only verify files for layers we'll actually publish. WhenLAYERSis unset it still resolves to all available layers, so the default behavior (and the release pipeline that builds all layers) is unchanged.Backwards compatibility
DD_TRACE_WHEEL/UPSTREAM_PIPELINE_ID/ no-override default → unchanged.LAYERSunset → still requires all 14 zips, identical to before.Test plan
DD_TRACE_COMMIT_BRANCH=<branch> PYTHON_VERSION=12 ./scripts/build_layers.shproducesdatadog_lambda_py-{amd64,arm64}-3.12.zip.VERSION=N REGIONS=us-east-2 LAYERS=Datadog-Python312 ./scripts/publish_layers.shproceeds past the existence check with only the 3.12 amd64 zip built.UPSTREAM_PIPELINE_ID=..., noLAYERSfilter) continue to work.