Speed up Codecov CI ~38% with parallel surefire forks#17697
Merged
Conversation
The codecov job currently runs ~66 min, ~51 min of which is the DataNode
unit tests on a single JVM at a time. The datanode pom sets
reuseForks=false (fresh JVM per test class) and surefire defaults to
forkCount=1, so on the 4-core runner only one test class runs at any
moment. The parent pom already configures per-fork working directories
(fork_${surefire.forkNumber}), so the project is plumbed for parallel
forks but never enables them.
Pass -DforkCount=2 to the codecov maven invocation so two test JVMs run
in parallel, cutting the DataNode UT phase roughly in half without
changing test isolation semantics.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #17697 +/- ##
============================================
+ Coverage 40.38% 40.40% +0.01%
- Complexity 2574 2575 +1
============================================
Files 5179 5179
Lines 349628 349628
Branches 44683 44683
============================================
+ Hits 141206 141253 +47
+ Misses 208422 208375 -47 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
JackieTien97
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
codecovjob in the Sonar-Codecov workflow currently runs ~66 minutes, of which ~51 minutes is the DataNode unit tests. Two factors stack to make these tests fully serial:iotdb-core/datanode/pom.xmlsets<reuseForks>false</reuseForks>(fresh JVM per test class).forkCount=1is never overridden, so only one JVM exists at a time.On a 4-vCPU
ubuntu-latestrunner this means at most one test class runs at any moment, paying full JVM startup + JaCoCo instrumentation per class.The parent
pom.xmlalready sets<workingDirectory>${project.build.directory}/fork_${surefire.forkNumber}</workingDirectory>, so per-fork filesystem isolation is configured but unused. This PR passes-DforkCount=4to the codecovmvninvocation so four test JVMs run in parallel.Measured impact (this PR's draft runs)
Net: 66 → 41 min (–38%). forkCount=4 saturates the 4-vCPU runner; the floor is now set by the long-tail tests (slowest single class is 127s).
Memory budget on the 16 GB runner: 4 × 1024m heap + maven + jacoco ≈ 5 GB, well within limits.
Test plan