Skip to content

fix(benchmarks): improve reviewed Rust benchmark snapshots#833

Merged
mstykow merged 15 commits intomainfrom
fix/compare-benchmark-snapshots
May 2, 2026
Merged

fix(benchmarks): improve reviewed Rust benchmark snapshots#833
mstykow merged 15 commits intomainfrom
fix/compare-benchmark-snapshots

Conversation

@mstykow
Copy link
Copy Markdown
Owner

@mstykow mstykow commented May 1, 2026

Summary

  • improve Rust benchmark-review outcomes by fixing scanner behavior around dual-license README handling, weak author noise, bounded extracted-text recovery, and workspace-root Cargo lockfile identity
  • keep the compare workflow generic by removing compare-only URL collapsing and avoiding ecosystem-specific package-data equivalence rules in the benchmark compare path
  • add reviewed docs/BENCHMARKS.md snapshots for Amanieu/atomic-rs, marshallpierce/rust-base64, and rustcrypto/aeads, then regenerate the benchmark chart and headline stats

Scope and exclusions

  • Included:
    • generic scanner fixes for dual-license README notices, weak author noise, binary copyright preservation, extracted-text recovery, and Cargo workspace lockfile identity
    • compare-path cleanup so benchmark review stays generic and literal where it should, instead of hiding URL or package-identity differences
    • reviewed benchmark rows for atomic-rs, rust-base64, and aeads
  • Explicit exclusions:
    • no ecosystem-specific compare exception framework was introduced; compare stays generic and the remaining reviewed differences are expressed in benchmark notes instead

Dual-license handling details

  • README dual-license notices such as licensed under either of ... at your option and dual-licensed under ... are now treated as true alternative-license signals instead of being flattened into conjunctive output.
  • On the detection side, the scanner now combines the substantive README matches with OR semantics when the surrounding notice text clearly describes an alternative-license choice, while still keeping genuinely supplemental material such as warranty-disclaimer fragments separate.
  • On the result-shaping side, the branch adds joined-expression selection logic so that when a README produces both singleton fragments like MIT / Apache-2.0 and a covering joined expression like Apache-2.0 OR MIT, Provenant prefers the joined expression instead of surfacing a noisier or accidentally conjunctive end-state.
  • The README/license-processing path also stops injecting Apache-specific badge or markdown-hint augmentation into files that already contain one of these explicit dual-license notices, because that hinting was one of the main reasons the original scans drifted toward AND-shaped output.
  • Redundant README conjunctive detections are pruned more locally now: the scanner only drops overlapping conjunctive detections that are subsumed by the stronger alternative-license notice, rather than broadly suppressing unrelated detections elsewhere in the same README.
  • End-to-end, that means the benchmarked Rust repositories in this PR keep the intended repository semantics — for example Apache-2.0 OR MIT in atomic-rs, rust-base64, and the member crates inside aeads — instead of reporting the pair as a conjunction or surfacing extra singleton leftovers as the primary expression.

Follow-up work

  • Created or intentionally deferred:
    • if additional benchmark targets surface the same kinds of reviewed differences, prefer scanner fixes or benchmark-note framing first; only introduce compare normalization when it is clearly generic and semantically lossless

@mstykow mstykow changed the title fix(compare): normalize Rust benchmark verification snapshots fix(benchmarks): improve reviewed Rust benchmark snapshots May 1, 2026
mstykow and others added 12 commits May 1, 2026 22:59
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
@mstykow mstykow force-pushed the fix/compare-benchmark-snapshots branch from 4cb846a to 0297e7f Compare May 1, 2026 21:10
mstykow and others added 3 commits May 2, 2026 00:05
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
@mstykow mstykow merged commit f3c180e into main May 2, 2026
15 checks passed
@mstykow mstykow deleted the fix/compare-benchmark-snapshots branch May 2, 2026 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant