[experiment] Parallelize recovery phone functional tests for PR workflow#20687
Draft
nshirley wants to merge 2 commits into
Draft
[experiment] Parallelize recovery phone functional tests for PR workflow#20687nshirley wants to merge 2 commits into
nshirley wants to merge 2 commits into
Conversation
Because: - Recovery phone (#phone) tests run serially on CI node 0, making it the long pole while the other parallel nodes finish and idle. - Several phone tests were not tagged #phone, so they ran unserialized against the shared test number — a latent smoke flakiness source. This commit: - Adds a phoneTestMode() helper gated on PLAYWRIGHT_PHONE_PARALLEL; locally codes are read from Redis per-uid so phone tests can run in parallel. - Tags the stray phone tests (replace2fa, cms-2fa, totp) with #phone so they serialize correctly for stage/prod smoke. - Adds a parallelize_phone_tests param to the PR functional job that runs #phone tests in the parallel timings split and raises the per-number registration cap. Experimental — not for merge.
Because: - The PR phone-test parallelization is experimental and may hit phone-number conflicts or Twilio rate limits; we need a fast, obvious way to disable it. This commit: - Promotes the PR parallelize_phone_tests job arg to a top-level pipeline parameter (default true) documented alongside the other enable_* toggles. - Flipping the default to false reverts PR functional runs to the previous serial-on-node-0 behavior with no other changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Recovery phone (
#phone) functional tests run serially on CI node 0, making it the long pole while the other parallel nodes finish and idle. This experiment runs them in the parallel timings split for the PR workflow only (codes are read from Redis per-uid locally, so parallel runs don't collide), and tags three stray phone tests that were running unserialized against the shared test number.Goal: measure the functional-test wall-clock delta vs baseline, and surface any intra-file ordering or Twilio test-credential throttling issues under parallelism.
Smoke / nightly / tag workflows keep the existing serial-on-node-0 behavior.