Skip to content

vincentkoc/tokenjuice

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

210 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

tokenjuice banner

tokenjuice πŸ§ƒ

lean output compaction for terminal-heavy agent workflows.

what is tokenjuice?

tokenjuice is a deterministic output compactor for terminal-heavy agent workflows. agents and harnesses run noisy commands like git status, pnpm test, docker build, rg, or pnpm --help; tokenjuice keeps the command semantics untouched, observes the output after execution, and returns a smaller payload built from rule-driven reducers instead of dumping the whole wall of terminal text back into context.

the point is leverage: less transcript waste, fewer useless reruns, and cleaner handoff between tools without making the shell magical. raw output stays available only when you explicitly ask for it through --raw / --full or opt-in artifact storage, rules stay inspectable JSON instead of LLM vibes, and host integrations stay thin wrappers around the same core reducer instead of becoming one-off adapter logic.

host integrations

supported integrations:

Logo Client Install Hook file
Claude Claude Code tokenjuice install claude-code ~/.claude/settings.json
CodeBuddy CodeBuddy tokenjuice install codebuddy ~/.codebuddy/settings.json
Codex Codex CLI tokenjuice install codex ~/.codex/hooks.json
Cursor Cursor tokenjuice install cursor ~/.cursor/hooks.json
Droid Droid (Factory CLI) tokenjuice install droid ~/.factory/settings.json
GitHub Copilot CLI GitHub Copilot CLI tokenjuice install copilot-cli ~/.copilot/hooks/tokenjuice-cli.json
OpenClaw OpenClaw openclaw config set plugins.entries.tokenjuice.enabled true ~/.openclaw/openclaw.json
OpenCode OpenCode tokenjuice install opencode ~/.config/opencode/plugins/tokenjuice.js
pi pi tokenjuice install pi ~/.pi/agent/extensions/tokenjuice.js
VS Code Copilot VS Code Copilot Chat tokenjuice install vscode-copilot ~/.copilot/hooks/tokenjuice-vscode.json

beta integrations:

Logo Client Install Hook file
Aider Aider tokenjuice install aider CONVENTIONS.tokenjuice.md
Avante Avante.nvim tokenjuice install avante avante.md
Cline Cline tokenjuice install cline ~/Documents/Cline/Hooks/tokenjuice-post-tool-use
Continue Continue tokenjuice install continue .continue/rules/tokenjuice.md
Gemini Gemini CLI tokenjuice install gemini-cli ~/.gemini/settings.json
Junie Junie tokenjuice install junie .junie/AGENTS.md
OpenHands OpenHands tokenjuice install openhands .openhands/hooks.json
Zed Zed tokenjuice install zed .rules

install

npm install -g tokenjuice
# or
pnpm add -g tokenjuice
# or
yarn global add tokenjuice
# or
brew tap vincentkoc/tap
brew install tokenjuice

then:

tokenjuice --help
tokenjuice --version
tokenjuice install [aider|avante|codex|claude-code|cline|codebuddy|continue|cursor|droid|gemini-cli|junie|openhands|pi|opencode|vscode-copilot|copilot-cli|zed]
tokenjuice uninstall [aider|avante|codex|cline|continue|droid|gemini-cli|junie|openhands|opencode|vscode-copilot|copilot-cli|zed]

OpenClaw support is bundled on the OpenClaw side. Do not run tokenjuice install openclaw; enable the bundled plugin instead:

openclaw config set plugins.entries.tokenjuice.enabled true

this requires OpenClaw 2026.4.22 or newer.

commands

tokenjuice --help
tokenjuice --version
tokenjuice reduce [file]
tokenjuice reduce-json [file]
tokenjuice wrap -- <command> [args...]
tokenjuice wrap --raw -- <command> [args...]
tokenjuice wrap --store -- <command> [args...]
tokenjuice install [aider|avante|codex|claude-code|cline|codebuddy|continue|cursor|droid|gemini-cli|junie|openhands|pi|opencode|vscode-copilot|copilot-cli|zed]
tokenjuice install [aider|avante|codex|claude-code|cline|codebuddy|continue|cursor|droid|gemini-cli|junie|openhands|pi|opencode|vscode-copilot|copilot-cli|zed] --local
tokenjuice uninstall [aider|avante|codex|cline|continue|droid|gemini-cli|junie|openhands|opencode|vscode-copilot|copilot-cli|zed]
tokenjuice ls
tokenjuice cat <artifact-id>
tokenjuice verify
tokenjuice discover
tokenjuice doctor
tokenjuice doctor hooks
tokenjuice doctor pi
tokenjuice doctor opencode
tokenjuice stats
tokenjuice stats --timezone utc

overview

tokenjuice has three surfaces. reduce compacts text that already exists, wrap runs a command and compacts the observed output, and reduce-json gives host adapters a stable machine protocol. host integrations are intentionally thin: they install a hook, extension, rule, or guidance file; call the shared compactor; and return compacted context through the host's native surface. use tokenjuice doctor hooks to check installed wiring, tokenjuice doctor <host> for one integration, and tokenjuice install <host> --local when validating the current repo build before release.

the reduction engine is rule-driven. built-in JSON rules live in src/rules, user overrides live in ~/.config/tokenjuice/rules, and project overrides live in .tokenjuice/rules; later layers override earlier ones by rule id. rules classify command output, normalize lines, keep or drop patterns, count facts, and retain deterministic head/tail slices. host adapters also apply a narrow safe-inventory policy: exact file-content reads stay raw, standalone repository inventory commands can compact, and unsafe mixed command sequences stay raw.

when a reducer gets it wrong or the task needs untouched bytes, use the explicit bypass:

tokenjuice wrap --raw -- pnpm --help
tokenjuice wrap --full -- git status

useful maintenance commands:

tokenjuice verify --fixtures
tokenjuice discover
tokenjuice doctor hooks
tokenjuice stats --timezone utc

adapter JSON

reduce-json is the machine-facing adapter command. it reads JSON from stdin or a file and always writes JSON to stdout; see the spec for envelope options and adapter behavior.

direct payload:

{
  "toolName": "exec",
  "command": "pnpm test",
  "argv": ["pnpm", "test"],
  "combinedText": "RUN  v3.2.4 /repo\n...",
  "exitCode": 1
}

docs

status

usable foundation for token reduction with diagnostics and a growing reducer set, now focused on deeper coverage and tuning.

πŸ’™ built by Vincent Koc.

About

πŸ§ƒ Token weight loss. Lean output compaction for terminal-heavy agent workflows. Works as a native CLI tool or as an extension to popular coding and agent frameworks.

Topics

Resources

License

Security policy

Stars

Watchers

Forks

Sponsor this project

 

Packages

 
 
 

Contributors