AuditEnvelope v1 — unified abstract audit message format (Phases B + C + F)

[hanwencheng]

Context

PR #95 (issue #82 ERC-7730 + EIP-712 sign) added signed_intent_text + signed_intent_hash to the audit-row schema, but only for typed-data signs. AuditEnvelope v1 is the canonical abstract format every audit-producing surface emits going forward.

Phase A — the canonical schema + non-break design — landed in PR #95 under arch.md §15.3a. This issue tracks the implementation phases B / C / F.

2026-06-11 sync to current design (checkbox audit against main @ b75597c): Phases B + C are fully shipped; Phase F's data-plane families (cred/memory/config) are live via #229 / PR #261. Deviations from the original plan are called out inline below. Remaining work: the control-plane emit sites (sign / scope / device — in progress), then email / K3 / payments / K10-rotate.

Phase B — worker + core migration ✅ SHIPPED

• AuditEnvelope v1 struct in agentkeys-core (audit/mod.rs, CBOR-canonical per RFC 8949 §4.2.1 in audit/cbor.rs).
• AuditOpKind u8 + per-kind op_body schemas (op_kind.rs + bodies.rs + TypedAuditBody) — every byte in the canonical table has a typed body, including the config family 80–82 added by Config data class + lazy, config-driven memory list (Phases 1–5) #201/Worker-side durable audit for memory + cred data-plane ops (store/fetch) #229.
• GET /v1/audit/envelope/<hash> on agentkeys-worker-audit (returns canonical CBOR, application/cbor).
• V1/V2 coexistence — shipped with inverse route naming vs the original plan: the legacy 5-field shape stays at POST /v1/audit/append; the envelope shape landed at POST /v1/audit/append/v2. Same one-cycle compatibility property (old callers keep working untouched), simpler migration.
• Workers emit AuditEnvelope — cred + memory + config (Worker-side durable audit for memory + cred data-plane ops (store/fetch) #229 / PR feat: #229 worker-side durable audit for memory + cred + config data-plane ops #261): one envelope per store/fetch/teardown, after cap-verify, before the success response, via the shared AuditEmitter. Responses carry the audit_envelope_hash receipt; AGENTKEYS_WORKER_REQUIRE_AUDIT=1 fails closed (staged rollout, default best-effort).
• envelope_hash = keccak256(canonical_cbor(envelope)).
• CBOR encode/decode roundtrip tests on canonical fixtures per op_kind (+ the agentkeys-core: AuditEnvelope v1 cross-language CBOR vector exporter #137 cross-language vectors; unknown-byte tolerance test).

Phase C — contract revision ✅ SHIPPED (prod); test-stack redeploy pending

• appendV2(operatorOmni, actorOmni, opKind, envelopeHash) in CredentialAudit.sol — additive, v1 retained. Design refinement: V2 is event-only (no on-chain entry/root storage; the indexed event log is the canonical history — entry position derivable from block number + log index).
• appendRootV2(operatorOmni, merkleRoot, opKindBitmap, entryCount) — opKindBitmap: bytes32, bit N = op_kind N; gated to the operator master wallet like V1 appendRoot. The audit worker queues V2 envelope hashes per operator and flush returns the appendRootV2 inputs (Worker-side durable audit for memory + cred data-plane ops (store/fetch) #229).
• AuditAppendedV2 + AuditRootAppendedV2 events with indexed opKind topic.
• Forge tests covering both shapes coexisting (AgentKeysV1.t.sol).
• Heima Mainnet prod: live in the 0.3 contract set (deployed 2026-06-09 via heima-bring-up.sh). ⚠️ The CI/test stack's CredentialAudit predates the V2 surface — appendRootV2 anchors soft-skip there by design until its next contract-set redeploy (VERSION bump ceremony).
• Contract registry updated — addresses live in the chain profile heima.json (contract_set_version 0.3); deployed-contracts.md carries the prose (the doc no longer holds an address table).

Phase F — extend op_kind coverage

All bytes were pre-claimed in the Phase A table, so the per-row work is: emit-site wiring + the 3-test ritual (arch.md row already exists).

Live (data plane, #229 / PR #261):

• CredStore (0) + CredFetch (1) + CredTeardown (2) — credentials worker.
• MemoryPut (10) + MemoryGet (11) + MemoryTeardown (12) — memory worker.
• ConfigPut (80) + ConfigGet (81) + ConfigTeardown (82) — config worker (family 80–89 claimed via the §15.3b ritual; not in this issue's original table).

In progress (control plane — PR forthcoming from this issue):

• SignEip191 (20) — emitted by the CLI sign orchestrator (agentkeys signer sign) after the signer returns. Emit-site deviation: the original table said "signer via daemon callback"; the CLI command is the component that orchestrates the sign and (for 712) holds the ERC-7730 intent text, so it emits.
• SignEip712 (21) — same site (agentkeys signer sign-typed-data); carries intent_text + intent_commitment from PR issue #82: ERC-7730 clear-signing + EIP-712 typed-data sign (v2-aligned) #95's ERC-7730 preview when --preview-7730 is active.
• ScopeGrant (40) + ScopeRevoke (41) — broker submit relay decodes the landed executeBatch calldata after the UserOp receipt confirms: setScope with services → ScopeGrant (body carries the FULL replacement set — set-replace semantics per Permissions panel READ/R+W toggle is local-only — never commits setScope on chain (header claims it does) #248); empty services → ScopeRevoke. Covers both the accept batch and /v1/scope/submit re-grants. Schema alignment: the op_body rows for 40/41 predate the Migrate master authority to an ERC-4337 P-256 smart-account (resolves §11 gating findings) #164/On-chain K11 gate for agent accept (passkey-account master UserOp) #225 set-replace setScope and are updated to the on-chain truth in the same PR (bytes never emitted before — not a break).
• DeviceAdd (50) + DeviceRevoke (51) — same broker decode path: registerAgentDevice → DeviceAdd (role_bits = ROLE_CAP_MINT, attestation hash zero for agent binds), revokeAgentDevice → DeviceRevoke (covers /v1/revoke/submit unpair).

Remaining (each a follow-up PR):

• K10Rotate (52) — blocked: no runtime K10 rotation flow exists yet.
• PaymentEscrowRedeem (30) + PaymentDirect (31) — blocked on the §15.5 payment-service worker.
• EmailSend (60) + EmailReceive (61) — email-service worker emit (worker exists; wire AuditEmitter like Worker-side durable audit for memory + cred data-plane ops (store/fetch) #229).
• K3EpochAdvance (70) — advanceEpoch() is operator-script-driven today; the emit site lands with whatever runtime component calls it.

Open design item (deferred from #229): audit-worker-initiated on-chain submission of appendRootV2 (tier-A relay). Today flush returns the inputs and the operator master submits (worker smoke + harness do this). True worker-initiated anchoring needs a master-delegated relay wallet or a contract-side allowance — design TBD here.

Non-break invariants (per arch.md §15.3a)

Unchanged — every PR landing a new op_kind MUST preserve all 8 invariants (open u8 enum, stable envelope-level fields, version gated on envelope-level breakage only, explorer Unknown(byte) fallback, opaque op_body passthrough, op-kind-agnostic contract, canonical table append-only in arch.md, 3 tests per op_kind).

Companion issue

Explorer-side work (phases D/E) tracked at litentry/subscan-essentials#12.

Acceptance

• Phase B + C ship.
• Phase F has ≥3 op_kinds wired end-to-end (9 data-plane op_kinds live with appendRootV2 batching, asserted by harness/v2-stage3-demo.sh steps 11-12 + heima-worker-smoke.sh).
• The recommended control-plane trio (SignEip712, ScopeGrant, DeviceAdd) wired — in progress (see above).
• Explorer renders the wired op_kinds; others render via the Unknown(byte) fallback (subscan-essentials#12).

Related: #82 (closed by #95), #90 (Stage 2 hardening), #91 (worker hardening), #201 (config data class), #229 (worker durable audit, PR #261).

[hanwencheng]

Phase B partially landed in PR #95

Commit 7738166. What shipped:

• ✅ `agentkeys-core::audit` module — `AuditEnvelope` struct, `AuditOpKind` open-enum (18 variants), `AuditResult`, canonical CBOR codec (deterministic per RFC 8949 §4.2.1), `envelope_hash()` = keccak256(canonical_cbor), `commit_intent()` helper.
• ✅ Per-op_kind typed body schemas in `audit::bodies` for all 18 canonical op_kinds.
• ✅ Worker V2 endpoints:
  • `POST /v1/audit/append/v2` — accept envelope JSON, store canonical CBOR, return `envelope_hash`.
  • `GET /v1/audit/envelope/:hash` — return canonical CBOR (200 application/cbor) or 404.
• ✅ V1 endpoints retained so existing callers keep working through the migration cycle.
• ✅ 17 audit-module unit tests + 7 V2 worker integration tests. Cargo test --workspace clean.

What's left in Phase B (will land in follow-up PRs against this issue):

• Migrate credentials-service emit site from `/v1/audit/append` (V1) to `/v1/audit/append/v2` (V2).
• Migrate memory-service emit site to V2.
• Persistent envelope storage (S3 `audit/envelopes/.cbor`) — in-memory storage is sufficient for v0 since chain commitment is the durability mechanism.

Phase C (contract revision) and Phase F (new op_kind coverage) remain as documented above.

[hanwencheng]

Phase D — sunset legacy SQLite audit, wire real EvmAuditAnchor

Folding scope picked up while resolving codex review on PR #96 (commit 4243434 retired the mock-server audit_log table that duplicated this story dev-side). The remaining piece is at the broker.

Current state (broker, today)

Three audit write paths coexist, all SQLite-backed:

Path	Where	Status
state.audit.record_mint(...) → legacy AuditLog SQLite	crates/agentkeys-broker-server/src/audit.rs	written from /v1/mint-oidc-jwt + /v1/grant/create
state.registry.audit → SqliteAnchor (plugin_mint_log table)	plugins/audit/sqlite.rs:113	written by the multi-anchor coordinator
state.registry.audit → EvmStubAnchor	plugins/audit/evm.rs:86	STUB only — evm.rs:10: "unit-test-only fixture that simulates the EVM dependency. Production uses the eventual EvmAuditAnchor (deferred…)"

Net result: every mint writes the same row twice to local SQLite and increments an in-memory counter pretending to be a chain anchor. Zero on-chain audit on prod today.

What Phase D delivers

The Phase B/C work in this issue lands the data-plane: AuditEnvelope v1 serializer, worker /v1/audit/append v2 endpoint, CredentialAudit.appendV2 + appendRootV2. Phase D wires the broker to actually use it:

• Replace EvmStubAnchor with EvmAuditAnchor that posts to the audit-worker's new POST /v1/audit/append (envelope-shape) instead of fabricating receipts in-memory. Worker is the broker's submission endpoint, NOT direct chain RPC — keeps the broker pure-policy + lets the worker batch via appendRootV2.
• Demote SqliteAnchor to confirmation cache, not primary durability. Today's plugin_mint_log is the audit ledger; post-Phase-D it becomes a local index pointing at on-chain envelope hashes. Schema gains chain_tx_hash NOT NULL + envelope_hash NOT NULL; the row is written AFTER the chain anchor returns, not before.
• Retire state.audit (legacy AuditLog). Its only callers are /v1/mint-oidc-jwt (oidc.rs:92,115) and /v1/grant/create. Migrate them to the plugin chain (state.registry.audit) so there's one write path, not two. Delete audit.rs::AuditLog + MintRecord + MintOutcome after callers move.
• Tighten the durability invariant. plan §2 rule 7 said "no credential released unless every configured audit anchor wrote Ok" — today that's vacuously true because EvmStubAnchor.anchor() always returns Ok. Phase D makes the multi-anchor coordinator actually short-circuit on a real chain submit failure; add the regression test that proves a worker /v1/audit/append → 5xx blocks the mint response from going out.
• Operator config — gate behind BROKER_AUDIT_ANCHORS=evm_worker (vs current default sqlite,evm_testnet). One config flip, one rollback path.

Sequencing

Phase D requires Phase C contract revision (appendV2) to be deployed on Heima Mainnet first, otherwise the worker has no appendV2 selector to call into. Sequence:

1. Phase B (envelope codec + worker endpoints) ← in-flight on claude/gallant-ride-cec4d7
2. Phase C (contract appendV2 + redeploy)
3. Phase D (this comment) — real EvmAuditAnchor in broker, SQLite demoted to cache, legacy AuditLog removed
4. Phase F (op_kind expansion) — strictly orthogonal to D, can run in parallel

Why this fits inside #97 instead of its own issue

D is the load-bearing answer to "why does SQLite still exist after the on-chain story shipped?" Splitting it into a separate issue invites the same kind of multi-quarter drift that left EvmStubAnchor un-replaced for two contract redeploys. The envelope schema (B) + contract surface (C) + broker wiring (D) are the same architectural unit; treat them as one.

Acceptance

• BROKER_AUDIT_ANCHORS=evm_worker boots clean; /readyz reports tier2/audit_worker_reachable.
• Every mint flow under harness/v2-stage*-demo.sh results in a CredentialAudit.AuditAppendedV2 event on Heima Mainnet matching the local plugin_mint_log row's envelope_hash.
• worker /v1/audit/append → 5xx blocks /v1/mint-oidc-jwt from returning 200 (durability invariant regression test).
• crates/agentkeys-broker-server/src/audit.rs deleted; zero callers of state.audit remain.
• EvmStubAnchor deleted entirely (not just #[cfg(test)]-gated) — it was always a stand-in for the real thing, and once EvmAuditAnchor lands there's no test value in keeping a no-op alongside.