test(awk): add rewritten oracle harness#233
Conversation
558c2bb to
6cd7980
Compare
| @@ -0,0 +1,155 @@ | |||
| --- | |||
There was a problem hiding this comment.
Topic from slack:
I'm wondering if could just compare to mawk already available in debian:bookworm-slim that we already use:
docker run --rm debian:bookworm-slim bash -c 'awk -W version'
random-funcs: srandom/random
regex-funcs: internal
compiled limits:
sprintf buffer 8192
maximum-integer 2147483647
mawk 1.3.4 20200120
Copyright 2008-2019,2020, Thomas E. Dickey
Copyright 1991-1996,2014, Michael D. Brennan
There was a problem hiding this comment.
also:
I think it would be nice to make it work for scenarios tests "bash" comparison mode
which means possible installing gawk in debian:bookworm-slim or another image
| @@ -0,0 +1,155 @@ | |||
| --- | |||
| name: implement-awk | |||
There was a problem hiding this comment.
Topic 2 from slack:
I'm not fully sure if we should just use the upstream tests suites as it,
we might have/want behaviour that slight differ from upstream awk
features that we don't implement
features we differ on purpose
also, beside the previous point, I believe, it's not ideal I believe to have this kind of dependency on upstream repo:
- slower CI
- less predictability, more flaky
- more complex setup
the good thing about replicating the tests as our own scenario tests is that:
- we have full control for divergent behaviour
- we can easily add more tests (if we use upstream, we can add our own scenarios tests too, but it's harder to track wasn't missing or not, and have 2 sources of tests)
There was a problem hiding this comment.
Yeah, agreed that we shouldn’t run the upstream suites as-is or make them the main CI contract.
We do have harness filters, though, so I think the intended use is a curated allowlist/smoke profile: run only known-relevant upstream cases for the rshell awk subset, then translate any important discoveries into first-class tests/scenarios/ coverage.
So the split I’d suggest is:
tests/scenarios/stay authoritative for normal CI and intentional rshell behavior.- The upstream harness runs filtered cases only, either opt-in, periodic, or as a small curated CI job.
- We do not depend on the moving/full upstream suite for merge correctness.
- Any upstream failure we decide matters becomes an original rshell scenario test.
That gives us the benefit of gawk coverage without the full cost/flakiness/control problem
cf898c3 to
d8e1cb4
Compare
1e3539e to
985be3d
Compare
Summary
Adds the rewritten AWK scenario harness and pinned GNU awk 5.4.0 oracle that future rshell awk implementation PRs build on. The harness keeps implementation gating opt-in through
enabled.txt, runs rshell through an awk-shaped adapter, and preserves upstream provenance without requiring live upstream test fetches.tools/awk-harnesswith pinned GNU awk install support and thershell-awkadapter.implement-awkskill and compatibility guidance for future work.Validation
bash -n tools/awk-harness/*.sh tools/awk-harness/rshell-awkgo test ./tests -run TestAwkScenarioMetadata -count=1make test_awk_rewritten