Skip to content

Add rare disease information triage cookbook example#211

Merged
JackChen-me merged 1 commit intoJackChen-me:mainfrom
oooooowoooooo:rare-disease-information-triage
May 7, 2026
Merged

Add rare disease information triage cookbook example#211
JackChen-me merged 1 commit intoJackChen-me:mainfrom
oooooowoooooo:rare-disease-information-triage

Conversation

@oooooowoooooo
Copy link
Copy Markdown
Contributor

@oooooowoooooo oooooowoooooo commented May 7, 2026

What

Adds a rare disease information triage cookbook example demonstrating source-isolated evidence audits and downstream conflict/safety arbitration.

Closes #210.

Why

This example shows a healthcare-domain multi-agent workflow where agents do not role-play clinicians or provide diagnosis. Instead, each agent audits one clearly flagged MOCK evidence source, and a downstream arbiter resolves evidence conflicts and safety boundaries.

The seeded conflict is intentional:

  • Web/commercial content maps symptom similarity directly to one rare disease and promotes a commercial test.
  • The official guideline / expert-consensus source keeps the differential broad and requires specialist evaluation.
  • The gene-phenotype evidence is weak or uncertain for the claimed disease.
  • The safety policy forbids diagnosis, treatment advice, dosing, or commercial recommendation.

The arbiter should return conflicting_evidence, misleading_or_commercial, or needs_specialist_review, not a diagnosis.

What it demonstrates

  • Source-isolated audits over different MOCK evidence sources
  • Patient-reported symptom summary audit
  • Nonprofit / patient-advocacy education audit
  • Official guideline / expert-consensus style audit
  • Gene-phenotype evidence audit
  • Web/forum/commercial claim audit
  • Medical safety policy audit
  • Downstream arbiter that receives only structured audit outputs
  • Runtime conflict detection between simplified patient-facing content, broad guideline differential, weak genetics evidence, and over-specific commercial web claims
  • Safety-boundary enforcement that prevents diagnosis, treatment advice, dosing, or commercial recommendation

Mock source policy

All fixtures are clearly flagged as MOCK sources. They are shaped like realistic source types but do not contain real patient data and do not provide medical diagnosis or treatment advice.

Files added

examples/cookbook/rare-disease-information-triage.ts

examples/fixtures/rare-disease-information-triage/
  patient-symptom-summary.json
  nonprofit-patient-education.md
  official-guideline-excerpt.md
  gene-phenotype-evidence.json
  web-claims-snippets.json
  medical-safety-policy.json

Validation

Local validation passed:

npm run lint
npx tsx examples/cookbook/rare-disease-information-triage.ts

Runtime assertions passed:

PASS: decision should reflect conflict/safety concern
PASS: patient-facing diagnosis should not be allowed
PASS: diagnosis should not be provided
PASS: conflicts should be surfaced
PASS: unsafe elements should be surfaced

Checklist

  • npm run lint passes
  • npm test passes
  • Added/updated tests for changed behavior — N/A, cookbook example only
  • No new runtime dependencies

Copy link
Copy Markdown
Owner

Main blocker is source isolation: the shared buildAgent() helper currently calls registerBuiltInTools(registry), which gives each audit agent file and shell tools, so isolation is prompt-only. Since the host already reads each fixture into the prompt, these agents can use an empty registry or tools: [] to make the example match #210.

Also add the cookbook row to examples/README.md so the new example is discoverable.

@oooooowoooooo oooooowoooooo force-pushed the rare-disease-information-triage branch from 6c4783a to 74a81cb Compare May 7, 2026 08:35
@oooooowoooooo oooooowoooooo force-pushed the rare-disease-information-triage branch from 74a81cb to 2b39151 Compare May 7, 2026 08:53
Copy link
Copy Markdown
Owner

@JackChen-me JackChen-me left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@JackChen-me JackChen-me merged commit 0bcba82 into JackChen-me:main May 7, 2026
4 checks passed
@oooooowoooooo
Copy link
Copy Markdown
Contributor Author

Updated.

Changes made:

  • Removed built-in tool registration from the cookbook example.
  • Added tools: [] through the shared provider config.
  • Audit agents now receive fixture content only through prompts and run with an empty tool registry.
  • Added the cookbook row to examples/README.md.
  • Normalized README encoding after adding the cookbook row.

Re-ran locally:

  • npm run lint
  • npx tsx examples/cookbook/rare-disease-information-triage.ts

Runtime assertions passed:

  • decision should reflect conflict/safety concern
  • patient-facing diagnosis should not be allowed
  • diagnosis should not be provided
  • conflicts should be surfaced
  • unsafe elements should be surfaced

Note: I did not mark npm test as passing locally. The full test suite fails in this Windows/WSL environment on existing bash built-in tool tests because /bin/bash is unavailable, which appears unrelated to this cookbook/README-only change.

@JackChen-me
Copy link
Copy Markdown
Owner

Got it. I verified npm test passes on my side before merging, so the WSL /bin/bash failure sounds environment-specific. Merged with the source isolation and README fixes included.
@oooooowoooooo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add rare disease information triage cookbook example

2 participants