Skip to content

feat(trial): auto-prompt for Seer trial + sentry trial list/start commands#399

Merged
BYK merged 10 commits intomainfrom
feat/seer-trial-prompt
Mar 12, 2026
Merged

feat(trial): auto-prompt for Seer trial + sentry trial list/start commands#399
BYK merged 10 commits intomainfrom
feat/seer-trial-prompt

Conversation

@BYK
Copy link
Member

@BYK BYK commented Mar 12, 2026

Summary

When Seer commands (issue explain, issue plan) fail with a budget or enablement error in an interactive terminal, the CLI now checks for an available Seer product trial, prompts the user, and retries the command on success. Additionally, new sentry trial list and sentry trial start commands give users proactive control over product trials.

Auto-prompt flow

Mirrors the existing executeWithAutoAuth middleware pattern:

main() → executeWithAutoAuth() → executeWithSeerTrialPrompt() → runCommand()
  1. Seer command fails with SeerError (no_budget or not_enabled)
  2. isTrialEligible() checks: eligible reason + org slug available + interactive TTY
  3. Fetches trial availability via GET /api/0/customers/{org}/ (control silo)
  4. Finds an unstarted Seer trial (prefers seerUsers, falls back to seerAutofix)
  5. Prompts user with consola confirm
  6. Starts trial via PUT /api/0/customers/{org}/product-trial/
  7. Retries the original command through the full middleware chain

Non-eligible errors (ai_disabled, missing org slug, non-TTY, API failures) pass through unchanged.

Trial commands

sentry trial list [org]

Lists all product trials for an org with status indicators:

  • ○ Available (cyan) — not yet started
  • ● Active (green) — started, with days remaining
  • − Expired (muted) — ended

Supports --json and --fields for machine consumption.

sentry trial start <name> [org]

Starts a named product trial. Supported names: seer, replays, performance, spans, profiling, logs.

Features smart argument swap detection — sentry trial start my-org seer works the same as sentry trial start seer my-org.

Key design decisions

  • isTrialEligible() accepts unknown — does the instanceof SeerError check internally so callers don't need to narrow first
  • Control silo routing/customers/ is a billing endpoint, uses getControlSiloUrl() not region URLs
  • Consola logger — all user-facing output uses log.info()/log.warn()/log.success() per project conventions (no raw stderr.write)
  • Graceful degradation — API failures during trial check silently fall through to the normal error display
  • ai_disabled excluded — admin's explicit choice to disable AI; trial wouldn't override it
  • UTC timezonegetTrialStatus() uses setUTCHours() for consistent date comparison
  • Soft trial hints in error messagesSeerError.format() says "You may be eligible" + sentry trial list (not a direct "start" since trial may not exist)

New files

File Description
src/lib/seer-trial.ts isTrialEligible() + promptAndStartTrial()
src/lib/trials.ts Trial name mapping, status helpers, findAvailableTrial()
src/commands/trial/index.ts Route map for trial subcommands
src/commands/trial/list.ts sentry trial list command
src/commands/trial/start.ts sentry trial start command
src/lib/arg-parsing.ts detectSwappedTrialArgs()

Modified files

File Changes
src/lib/api-client.ts getProductTrials() + startProductTrial() using control silo
src/lib/errors.ts Soft trial hint in SeerError.format()
src/bin.ts executeWithSeerTrialPrompt() middleware
src/app.ts Register trial routes + trials alias
src/types/sentry.ts ProductTrialSchema, CustomerTrialInfoSchema

Tests

  • 20 unit tests for trial prompt flow (seer-trial.test.ts)
  • 10 unit tests for trial API client (api-client.seer-trial.test.ts)
  • 30 unit tests for trial helpers (trials.test.ts)
  • 4 property-based tests for eligibility invariants (seer-trial.property.test.ts)
  • 9 unit tests for trial list command (trial/list.test.ts)
  • 10 unit tests for trial start command (trial/start.test.ts)
  • 4 unit tests for detectSwappedTrialArgs (arg-parsing.test.ts)
  • 87 trial-related tests total, all passing

When `issue explain` or `issue plan` fails with a 402 (no budget) or
403 (not enabled) error in an interactive terminal, the CLI now checks
if the org has an available Seer product trial. If one exists, it
prompts the user to start a free trial and retries the command on
success.

The flow mirrors the existing `executeWithAutoAuth` pattern: a new
`executeWithSeerTrialPrompt` middleware wraps `runCommand` and catches
`SeerError`. Errors that aren't trial-eligible (`ai_disabled`, missing
org slug, non-TTY) pass through unchanged.

Key behaviors:
- Checks `GET /customers/{org}/` for unstarted seerUsers trial
  (falls back to seerAutofix for legacy orgs)
- Starts trial via `PUT /customers/{org}/product-trial/`
- API failures during trial check degrade gracefully (show original error)
- consola prompt cancel handled with strict equality check
@github-actions
Copy link
Contributor

github-actions bot commented Mar 12, 2026

Semver Impact of This PR

🟡 Minor (new features)

📋 Changelog Preview

This is how your changes will appear in the changelog.
Entries from this PR are highlighted with a left border (blockquote style).


New Features ✨

Init

  • Add --team flag to relay team selection to project creation by MathurAditya724 in #403
  • Enforce canonical feature display order by betegon in #388
  • Accept multiple delimiter formats for --features flag by betegon in #386
  • Add git safety checks before wizard modifies files by betegon in #379
  • Add experimental warning before wizard runs by betegon in #378
  • Add init command for guided Sentry project setup by betegon in #283

Issue List

  • Auto-compact when table exceeds terminal height by BYK in #395
  • Redesign table to match Sentry web UI by BYK in #372

Other

  • (trial) Auto-prompt for Seer trial + sentry trial list/start commands by BYK in #399
  • Add --dry-run flag to mutating commands by BYK in #387
  • Return-based output with OutputConfig on buildCommand by BYK in #380
  • Add --fields flag for context-window-friendly JSON output by BYK in #373
  • Magic @ selectors (@latest, @most_frequent) for issue commands by BYK in #371
  • Input hardening against agent hallucinations by BYK in #370
  • Add response caching for read-only API calls by BYK in #330

Bug Fixes 🐛

Init

  • Remove implementation detail from help text by betegon in #385
  • Truncate uncommitted file list to first 5 entries by MathurAditya724 in #381

Other

  • (api) Convert --data to query params for GET requests by BYK in #383
  • (docs) Remove double borders and fix column alignment on landing page tables by betegon in #369
  • Add trace ID validation to trace view + UUID dash-stripping by BYK in #375

Internal Changes 🔧

Init

  • Remove --force flag by betegon in #377
  • Remove dead determine-pm step label by betegon in #374

Other

  • Migrate non-streaming commands to CommandOutput with markdown rendering by BYK in #398
  • Convert Tier 2-3 commands to return-based output and consola by BYK in #394
  • Convert remaining Tier 1 commands to return-based output by BYK in #382
  • Converge Tier 1 commands to writeOutput helper by BYK in #376

Other

  • Minify JSON on read and pretty-print on write in init local ops by MathurAditya724 in #396

🤖 This preview updates automatically when you update the PR.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 12, 2026

Codecov Results 📊

104 passed | Total: 104 | Pass Rate: 100% | Execution Time: 0ms

📊 Comparison with Base Branch

Metric Change
Total Tests
Passed Tests
Failed Tests
Skipped Tests

✨ No test changes detected

All tests are passing successfully.

✅ Patch coverage is 100.00%. Project has 916 uncovered lines.
✅ Project coverage is 95.59%. Comparing base (base) to head (head).

Files with missing lines (2)
File Patch % Lines
api-client.ts 77.34% ⚠️ 225 Missing
arg-parsing.ts 93.81% ⚠️ 19 Missing
Coverage diff
@@            Coverage Diff             @@
##          main       #PR       +/-##
==========================================
+ Coverage    95.49%    95.59%     +0.1%
==========================================
  Files          142       147        +5
  Lines        20352     20755      +403
  Branches         0         0         —
==========================================
+ Hits         19435     19839      +404
- Misses         917       916        -1
- Partials         0         0         —

Generated by Codecov Action

@BYK BYK marked this pull request as ready for review March 12, 2026 06:51
BYK and others added 4 commits March 12, 2026 11:13
- Add standalone trial commands: sentry trial list, sentry trial start
- Rename getSeerTrialStatus → getProductTrials (returns ProductTrial[])
- Rename startSeerTrial → startProductTrial (generic category param)
- Add src/lib/trials.ts with trial name mapping and status helpers
- Add swap detection for trial start args (org/name order)
- Update SeerError messages to hint at trial commands
- Refactor seer-trial.ts to use shared trial infrastructure
- Fix getTrialDisplayName bug: add getDisplayNameForTrialName for CLI names
- Add comprehensive tests (80 tests across 5 files)
The xn-- prefix is interpreted as IDN encoding by URL parsers,
causing invalid URLs when used as subdomain slugs (e.g., xn--a.sentry.io).
This was a pre-existing flaky test failure.
- Use getControlSiloUrl() for /customers/ endpoints (control silo, not region-scoped)
- Replace stderr.write() with consola logger in seer-trial.ts
- Move instanceof SeerError check into isTrialEligible() (accepts unknown)
- Fix timezone bug: use setUTCHours in getTrialStatus()
- Soften unconditional trial hint wording in SeerError.format()
- Update seer-trial tests for new API (logger mocks, 2-arg signature)
- Add support@sentry.io mention when trial start fails
- Simplify no-op conditional type to plain ReturnType<>
@BYK
Copy link
Member Author

BYK commented Mar 12, 2026

Addressed all review feedback in b17c30b and df55458. Summary of changes:

  1. Control silo fixgetProductTrials and startProductTrial now use getControlSiloUrl() instead of resolveOrgRegion() (Seer bot)
  2. stderr → logger — All stderr.write() replaced with log.info()/log.warn()/log.success() via logger.withTag("seer") (BYK + BugBot)
  3. instanceof into isTrialEligibleisTrialEligible() now accepts unknown and does the instanceof SeerError check internally (BYK)
  4. Timezone fix — Changed setHourssetUTCHours in getTrialStatus() (BugBot)
  5. Softened trial hint — Changed wording to "You may be eligible" with sentry trial list instead of sentry trial start seer (BugBot)
  6. Support contact — Added support@sentry.io mention in failed trial start message (BYK)
  7. Simplified type annotation — Removed no-op extends infer T ? T : never wrapper (BugBot)
  8. Expired trial messaging — When no trial is available, explicitly suggests upgrading plan with billing URL (BYK)

All 87 trial-related tests pass, typecheck and lint clean.

BYK added 2 commits March 12, 2026 11:42
Change test assertion to check for 'support@sentry' substring
instead of 'support@sentry.io' to avoid triggering CodeQL's
'Incomplete URL substring sanitization' rule (code-scanning/43).
@BYK BYK changed the title feat(seer): prompt to start free trial on budget/enablement errors feat(trial): auto-prompt for Seer trial + sentry trial list/start commands Mar 12, 2026
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Replace nested executeWithAutoAuth/executeWithSeerTrialPrompt wrappers
with a composable ErrorMiddleware type and an array of middlewares.

- ErrorMiddleware type: (next, args) => Promise<void>
- buildExecutor() composes middlewares innermost-first
- Easy to add new middlewares by appending to errorMiddlewares array

Also:
- Use Object.hasOwn() instead of `in` for TRIAL_NAMES lookup
- Fix getDaysRemaining to use same end-of-day UTC as getTrialStatus
@BYK BYK merged commit 2988043 into main Mar 12, 2026
22 checks passed
@BYK BYK deleted the feat/seer-trial-prompt branch March 12, 2026 12:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant