feat(ai-openrouter): surface per-request cost on RUN_FINISHED#654
Conversation
OpenRouter reports the actual cost of each request inline on the chat response. Forward it on the terminal RUN_FINISHED event as usage.cost, with OpenRouter's per-request breakdown under usage.costDetails. This is the cost OpenRouter itself reports, so it accounts for routing, fallback providers, BYOK, and cached-token pricing rather than being computed from token counts. Both the Chat Completions (openRouterText) and Responses (openRouterResponsesText) adapters populate it. A shared UsageTotals type in @tanstack/ai carries the optional cost/costDetails fields, so they're also available on the middleware onUsage and onFinish hooks. Cost-detail keys are normalized to camelCase so the SDK-parsed and raw fallback paths stay consistent. The fields are optional and additive; adapters that don't report cost are unaffected.
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughSurfaces OpenRouter-reported per-request USD cost on terminal RUN_FINISHED.usage (usage.cost and usage.costDetails), adds shared UsageTotals/UsageCostBreakdown types, implements extractUsageCost and integrates it into OpenRouter adapters, and adds unit, adapter, and e2e tests plus docs/changeset/README updates. ChangesOpenRouter Cost Tracking
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
testing/e2e/src/routes/api.openrouter-cost.ts (1)
3-3: ⚡ Quick winUse the tree-shakeable adapter entrypoint here.
This route imports the provider adapter from the package root instead of the
/adapterssubpath required by the repo conventions.Suggested fix
-import { createOpenRouterText } from '`@tanstack/ai-openrouter`' +import { createOpenRouterText } from '`@tanstack/ai-openrouter/adapters`'As per coding guidelines, "Import tree-shakeable provider adapters from /adapters subpath (e.g.,
@tanstack/ai-openai/adapters) rather than default imports".🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@testing/e2e/src/routes/api.openrouter-cost.ts` at line 3, The import in this route uses the package root; replace the default import of createOpenRouterText from '`@tanstack/ai-openrouter`' with the tree-shakeable adapter entrypoint under the adapters subpath (e.g., import createOpenRouterText from '`@tanstack/ai-openrouter/adapters`'), so update the import statement referencing createOpenRouterText to use the /adapters subpath per repo conventions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@testing/e2e/global-setup.ts`:
- Around line 322-326: The POST handler that matches the pathname check
(req.method !== 'POST' || !pathname.startsWith('/v1/chat/completions')) must
fully consume/drain the request body before sending the SSE response to avoid
leaving unread bytes on keep-alive sockets; update the code path that proceeds
after that check to asynchronously read the request stream to completion (e.g.,
await reading req.body or piping the request until end) before writing the SSE
response so sockets are clean for connection reuse.
---
Nitpick comments:
In `@testing/e2e/src/routes/api.openrouter-cost.ts`:
- Line 3: The import in this route uses the package root; replace the default
import of createOpenRouterText from '`@tanstack/ai-openrouter`' with the
tree-shakeable adapter entrypoint under the adapters subpath (e.g., import
createOpenRouterText from '`@tanstack/ai-openrouter/adapters`'), so update the
import statement referencing createOpenRouterText to use the /adapters subpath
per repo conventions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 39a08319-43e6-44b2-885d-8abebbfe59a7
📒 Files selected for processing (16)
.changeset/openrouter-cost-tracking.mdREADME.mddocs/adapters/openrouter.mdpackages/ai-openrouter/src/adapters/cost.tspackages/ai-openrouter/src/adapters/responses-text.tspackages/ai-openrouter/src/adapters/text.tspackages/ai-openrouter/tests/cost.test.tspackages/ai-openrouter/tests/openrouter-adapter.test.tspackages/ai-openrouter/tests/openrouter-responses-adapter.test.tspackages/ai/src/activities/chat/middleware/types.tspackages/ai/src/types.tspackages/ai/tests/usage-cost-types.test.tstesting/e2e/global-setup.tstesting/e2e/src/routeTree.gen.tstesting/e2e/src/routes/api.openrouter-cost.tstesting/e2e/tests/openrouter-cost.spec.ts
Match the other aimock mounts (drainBody) so the keep-alive socket has no unread request bytes before the SSE response is written.
|
Thanks @coderabbitai 🐰 — addressed in 6ca2e89:
|
|
Tip For best results, initiate chat on the files or code changes.
🐰 ✨ ✏️ Learnings added
|
| /** Provider-reported cost for the request, when available. */ | ||
| cost?: number | ||
| /** Provider-specific cost breakdown, when available. */ | ||
| costDetails?: Record<string, number | null | undefined> |
There was a problem hiding this comment.
I'm going to give this the actual openrouter shape. If another provider supports cost details like this, we can look at the shape then. No one else does at this point
Replace the open `Record<string, number | null | undefined>` shape on `UsageTotals.costDetails` with a typed `UsageCostDetails` interface enumerating the five fields OpenRouter actually reports across its Chat Completions and Responses endpoints. Consumers get autocomplete on the breakdown and a typo on a key becomes a compile error; unknown keys are dropped at extraction time so the public surface stays closed. The extractor swaps generic `toCamelCase` for a snake↔camel allowlist keyed on the known fields, and treats `null` as absent rather than forwarding it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The @tanstack/ai changes are additive type plumbing only (new exported UsageTotals/UsageCostDetails interfaces, optional fields on existing shapes) — no runtime change, no new callable surface. The feature lives in @tanstack/ai-openrouter. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
tombeckenham
left a comment
tombeckenham
left a comment
There was a problem hiding this comment.
I'm creating a type UsageCostBreakdown as a vendor independent cost breakdown type. I did a bit of research on how many others return cost data - no one we currently support - but thought I'd keep the shape generic and mould the openrouter inference cost data into it. it's almost the same. Does this work for you?
…ral shape UsageCostBreakdown becomes three concrete fields (upstreamCost, upstreamInputCost, upstreamOutputCost) that every adapter maps its provider-specific wire keys onto at extraction time. Consumer code reads the same fields regardless of which gateway populated them, so swapping adapters is a one-line change with no consumer rewrites. The OpenRouter adapter collapses its two endpoint naming styles (Chat Completions' prompt/completions and Responses' input/output) onto the same canonical input/output split — they bill against the same tokens. Replaces the prior declaration-merging approach, which leaked OpenRouter vocabulary into every consumer site that read costDetails. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Yes, this works. Thank you. |
|
View your CI Pipeline Execution ↗ for commit 1d51862 ☁️ Nx Cloud last updated this comment at |
|
View your CI Pipeline Execution ↗ for commit 1d51862
☁️ Nx Cloud last updated this comment at |
@tanstack/ai
@tanstack/ai-anthropic
@tanstack/ai-client
@tanstack/ai-code-mode
@tanstack/ai-code-mode-skills
@tanstack/ai-devtools-core
@tanstack/ai-elevenlabs
@tanstack/ai-event-client
@tanstack/ai-fal
@tanstack/ai-gemini
@tanstack/ai-grok
@tanstack/ai-groq
@tanstack/ai-isolate-cloudflare
@tanstack/ai-isolate-node
@tanstack/ai-isolate-quickjs
@tanstack/ai-ollama
@tanstack/ai-openai
@tanstack/ai-openrouter
@tanstack/ai-preact
@tanstack/ai-react
@tanstack/ai-react-ui
@tanstack/ai-solid
@tanstack/ai-solid-ui
@tanstack/ai-svelte
@tanstack/ai-utils
@tanstack/ai-vue
@tanstack/ai-vue-ui
@tanstack/openai-base
@tanstack/preact-ai-devtools
@tanstack/react-ai-devtools
@tanstack/solid-ai-devtools
commit: |
tombeckenham
left a comment
tombeckenham
left a comment
There was a problem hiding this comment.
All good. Approving
2 participants
Changes
Closes #468.
OpenRouter returns the actual per-request cost inline on the chat response. This surfaces it on the terminal
RUN_FINISHEDevent asusage.cost, with OpenRouter's per-request breakdown underusage.costDetails. Both theopenRouterText(Chat Completions) andopenRouterResponsesText(Responses) adapters populate it.@tanstack/aigains a sharedUsageTotalstype with optionalcost/costDetailsfields.RunFinishedEvent.usage, the middlewareUsageInfo(onUsage), andFinishInfo.usage(onFinish) all use it, so cost can be read without casts and on the middleware hooks. Cost is the value OpenRouter itself reports — not computed locally from token counts — so it accounts for routing, fallback providers, BYOK, and cached-token pricing. Cost-detail keys are normalized to camelCase so the SDK-parsed and raw-fallback paths stay consistent. The fields are optional and additive; adapters that don't report cost are unaffected.This supersedes the earlier #469, which is closed. That approach cloned the SSE stream and reconstructed cost through a custom
CostStore/HTTP-client hook. With@openrouter/sdkparsingusage.cost/usage.cost_detailsnatively, this version is purely additive plumbing — no stream cloning, no provider-specific machinery.Test plan
cost.test.ts(theextractUsageCosthelper —cost === 0, negatives,null, non-finite, snake/camel key normalization), cost suites in both adapter test files (including a raw/UNKNOWNresponse.completedfallback case), andusage-cost-types.test.tslocking the types.openrouter-cost.spec.tsdriving a deterministic OpenRouter cost mount end-to-end.pnpm test:prgreen across 31 projects; full E2E suite green (224 passed) locally.✅ Checklist
pnpm run test:pr.Release Impact
Summary by CodeRabbit
New Features
Documentation
Types
Tests