diff --git a/.chalk/plans/deploy-langsmith-cloud.md b/.chalk/plans/deploy-langsmith-cloud.md new file mode 100644 index 0000000..785a3de --- /dev/null +++ b/.chalk/plans/deploy-langsmith-cloud.md @@ -0,0 +1,235 @@ +# Deployment Migration — LangSmith Cloud (LangGraph Platform) + +## Context + +Migrating from a fully self-hosted Render deployment (both agent + frontend) to a split topology: **frontend on Render, agent on LangGraph Platform (LangSmith Cloud)**. The codebase changes are on `feat/deploy-langsmith-cloud` (commit `0d3a6e6`). This plan covers the safe rollout path: branch deploy to dev, verify, merge to main, promote to production. + +### What changed (code) + +- `render.yaml` — Agent service removed; `LANGGRAPH_DEPLOYMENT_URL` switched from `fromService` to `sync: false` +- `apps/agent/main.py` — `LANGGRAPH_CLOUD` detection fixed (truthy string bug), startup log added +- `apps/agent/langgraph.json` — Already correct: `"sample_agent": "./main.py:agent"`, no `.env` ref +- `docs/deployment.md` — Rewritten for split deployment with `langgraph deploy` CLI +- `.env.example` — Noted `LANGSMITH_API_KEY` needed on frontend + +### Prerequisites + +- [ ] LangSmith account (Plus plan or higher) +- [ ] LangSmith API key (`lsv2_...`) — obtain from https://smith.langchain.com/settings +- [ ] `langgraph` CLI installed: `pip install langgraph-cli` +- [ ] Docker installed and running (Apple Silicon: ensure Buildx is available) +- [ ] `OPENAI_API_KEY` ready for agent env vars +- [ ] Render dashboard access for the frontend service + +--- + +## Phase 1: Deploy Branch to Development (LangSmith Cloud) + +### 1.1 Create development deployment on LangGraph Platform + +```bash +cd apps/agent +langgraph deploy \ + --name open-generative-ui-agent-dev \ + --deployment-type dev +``` + +- The CLI builds a Docker image from `langgraph.json` and pushes to the managed registry +- Use `--verbose` if the build fails to see Docker output +- Apple Silicon: the CLI uses Buildx to cross-compile to `linux/amd64` + +### 1.2 Configure agent env vars in LangSmith dashboard + +Navigate to the deployment in the LangSmith dashboard and set: + +| Variable | Value | +|----------|-------| +| `OPENAI_API_KEY` | Your OpenAI key | +| `LANGGRAPH_CLOUD` | `true` | +| `LLM_MODEL` | `gpt-5.4-2026-03-05` (or preferred model) | +| `LANGCHAIN_TRACING_V2` | `true` | +| `LANGCHAIN_PROJECT` | `open-generative-ui-dev` | + +### 1.3 Note the deployment URL + +After the deployment is live, note the URL: +``` +https://.default.us.langgraph.app +``` + +### 1.4 Point local frontend at dev deployment + +Test locally before touching Render: + +```bash +LANGGRAPH_DEPLOYMENT_URL=https://.default.us.langgraph.app \ +LANGSMITH_API_KEY=lsv2_... \ +pnpm dev:app +``` + +--- + +## Phase 2: Verify Development Deployment + +### 2.1 Smoke tests + +- [ ] Frontend loads at `http://localhost:3000` +- [ ] Chat input accepts a message and gets a response +- [ ] Agent can add/update/complete todos (state sync works) +- [ ] Generative UI renders (widgetRenderer, charts) +- [ ] No checkpointer warnings in agent logs (check via `langgraph deploy logs`) + +### 2.2 Persistence check + +- [ ] Send a message, note the thread ID +- [ ] Refresh the page — conversation state should persist (Postgres-backed) +- [ ] This is the key improvement over BoundedMemorySaver (in-memory, lost on restart) + +### 2.3 Tracing check + +- [ ] Open LangSmith dashboard → project `open-generative-ui-dev` +- [ ] Verify traces appear for each agent invocation +- [ ] Check for errors or unexpected latency in traces + +### 2.4 Auth check + +- [ ] Confirm requests without `x-api-key` header are rejected by the platform +- [ ] Confirm requests with valid `LANGSMITH_API_KEY` succeed + +### 2.5 Edge cases + +- [ ] Rapid successive messages (rate limiting on frontend if enabled) +- [ ] Long-running agent response (streaming works end-to-end) +- [ ] Empty/malformed input handling + +--- + +## Phase 3: Deploy Frontend Branch to Render (Optional) + +If you want to test the full split deployment before merging: + +### 3.1 Push branch and create preview + +Render supports branch-based preview environments. Push the branch: + +```bash +git push origin feat/deploy-langsmith-cloud +``` + +In Render dashboard, create a preview environment or manually set env vars on a staging service: + +| Variable | Value | +|----------|-------| +| `LANGGRAPH_DEPLOYMENT_URL` | `https://.default.us.langgraph.app` (dev deployment) | +| `LANGSMITH_API_KEY` | `lsv2_...` | + +### 3.2 Verify on Render + +- [ ] Frontend health check passes: `GET /api/health` → 200 +- [ ] Chat works end-to-end through Render → LangGraph Platform +- [ ] No CORS or network errors in browser console + +--- + +## Phase 4: Merge to Main + +### 4.1 PR and merge + +```bash +gh pr create \ + --title "feat: split deployment — frontend on Render, agent on LangGraph Platform" \ + --base main \ + --head feat/deploy-langsmith-cloud +``` + +After review/approval: +```bash +gh pr merge --squash +``` + +### 4.2 Verify CI passes + +- [ ] CI smoke tests pass (build + lint + startup check) +- [ ] No regressions on the frontend build + +--- + +## Phase 5: Deploy Main to Production (LangSmith Cloud) + +### 5.1 Create production deployment + +```bash +cd apps/agent +langgraph deploy \ + --name open-generative-ui-agent \ + --deployment-type prod +``` + +Or update existing deployment: +```bash +langgraph deploy \ + --name open-generative-ui-agent \ + --deployment-id +``` + +### 5.2 Configure production env vars in LangSmith dashboard + +| Variable | Value | +|----------|-------| +| `OPENAI_API_KEY` | Production OpenAI key | +| `LANGGRAPH_CLOUD` | `true` | +| `LLM_MODEL` | `gpt-5.4-2026-03-05` | +| `LANGCHAIN_TRACING_V2` | `true` | +| `LANGCHAIN_PROJECT` | `open-generative-ui` | + +### 5.3 Note production deployment URL + +``` +https://.default.us.langgraph.app +``` + +### 5.4 Update Render production frontend + +In the Render dashboard, update the production frontend env vars: + +| Variable | Value | +|----------|-------| +| `LANGGRAPH_DEPLOYMENT_URL` | `https://.default.us.langgraph.app` | +| `LANGSMITH_API_KEY` | Production LangSmith API key | + +Trigger a redeploy (or it will auto-deploy from the main branch merge). + +### 5.5 Production verification + +- [ ] `GET /api/health` → 200 +- [ ] Chat works end-to-end +- [ ] Todos persist across page refreshes +- [ ] Traces appear in LangSmith project `open-generative-ui` +- [ ] No errors in `langgraph deploy logs` +- [ ] Generative UI (widgets, charts) renders correctly + +--- + +## Rollback + +### If agent deployment fails + +The previous Render agent service is still defined in the `main` branch prior to merge. If the LangGraph Platform deployment is broken: + +1. Revert the `render.yaml` change (restore agent service block) +2. Revert `LANGGRAPH_DEPLOYMENT_URL` to `fromService` in Render dashboard +3. Redeploy on Render + +### If frontend deployment fails + +The frontend changes are minimal (no code changes to `route.ts`). Rolling back just means pointing `LANGGRAPH_DEPLOYMENT_URL` back to the old agent URL in the Render dashboard. + +--- + +## Post-Migration Cleanup + +- [ ] Delete the dev deployment: `langgraph deploy delete ` +- [ ] Remove old Render agent service if it was kept as fallback +- [ ] Confirm LangSmith tracing project is receiving data +- [ ] Update any team runbooks or onboarding docs referencing the old Render agent +- [ ] Consider enabling `RATE_LIMIT_ENABLED=true` on the Render frontend diff --git a/.env.example b/.env.example index 9f6d59f..55a3da6 100644 --- a/.env.example +++ b/.env.example @@ -4,6 +4,13 @@ OPENAI_API_KEY= # Recommended: gpt-5.4, gpt-5.4-pro, claude-opus-4-6, gemini-3.1-pro LLM_MODEL=gpt-5.4-2026-03-05 +# LangGraph Platform (LangSmith Cloud) — only needed for cloud deployment +# LANGGRAPH_CLOUD=true +# LANGCHAIN_TRACING_V2=true +# LANGCHAIN_PROJECT=open-generative-ui +# LANGSMITH_API_KEY= +# ^ Also required on the frontend for cloud deployments (sent as x-api-key header) + # Rate limiting (per IP) — disabled by default RATE_LIMIT_ENABLED=false RATE_LIMIT_WINDOW_MS=60000 diff --git a/.gitignore b/.gitignore index b352c66..85f9e0a 100644 --- a/.gitignore +++ b/.gitignore @@ -63,3 +63,7 @@ bun.lockb # References .references + +# Auto-generated by Chalk +.chalk/context/ +.chalk/skills/*.enabled diff --git a/apps/agent/langgraph.json b/apps/agent/langgraph.json index 804b1ea..f1028e4 100644 --- a/apps/agent/langgraph.json +++ b/apps/agent/langgraph.json @@ -4,7 +4,6 @@ "dependencies": ["."], "package_manager": "uv", "graphs": { - "sample_agent": "./main.py:graph" - }, - "env": "../../.env" + "sample_agent": "./main.py:agent" + } } diff --git a/apps/agent/main.py b/apps/agent/main.py index 4283880..b6852c9 100644 --- a/apps/agent/main.py +++ b/apps/agent/main.py @@ -3,6 +3,7 @@ It defines the workflow graph, state, tools, nodes and edges. """ +import logging import os import warnings from pathlib import Path @@ -22,13 +23,23 @@ load_dotenv() +logger = logging.getLogger(__name__) + +# LangGraph Platform provides a managed Postgres-backed checkpointer, +# so BoundedMemorySaver is only needed for self-hosted / local dev. +_on_langgraph_platform = os.environ.get("LANGGRAPH_CLOUD", "").lower() == "true" +checkpointer = None if _on_langgraph_platform else BoundedMemorySaver(max_threads=200) + +if _on_langgraph_platform: + logger.info("LANGGRAPH_CLOUD=true — using platform-managed checkpointer") + agent = create_deep_agent( model=ChatOpenAI(model=os.environ.get("LLM_MODEL", "gpt-5.4-2026-03-05")), tools=[query_data, plan_visualization, *todo_tools, generate_form], middleware=[CopilotKitMiddleware()], context_schema=AgentState, skills=[str(Path(__file__).parent / "skills")], - checkpointer=BoundedMemorySaver(max_threads=200), + checkpointer=checkpointer, system_prompt=""" You are a helpful assistant that helps users understand CopilotKit and LangGraph used together. diff --git a/apps/app/src/app/api/copilotkit/route.ts b/apps/app/src/app/api/copilotkit/route.ts index bfbdd8e..d40af22 100644 --- a/apps/app/src/app/api/copilotkit/route.ts +++ b/apps/app/src/app/api/copilotkit/route.ts @@ -3,7 +3,7 @@ import { ExperimentalEmptyAdapter, copilotRuntimeNextJSAppRouterEndpoint, } from "@copilotkit/runtime"; -import { LangGraphHttpAgent } from "@copilotkit/runtime/langgraph"; +import { LangGraphAgent, LangGraphHttpAgent } from "@copilotkit/runtime/langgraph"; import { NextRequest } from "next/server"; // Simple in-memory sliding-window rate limiter (per IP) @@ -44,9 +44,18 @@ const deploymentUrl = !raw : `http://${raw}`; // 1. Define the agent connection to LangGraph -const defaultAgent = new LangGraphHttpAgent({ - url: deploymentUrl, -}); +// LangGraphAgent talks the native LangGraph SDK API (for LangGraph Platform / Cloud). +// LangGraphHttpAgent talks the AG-UI HTTP protocol (for self-hosted FastAPI server). +const usePlatform = !!process.env.LANGSMITH_API_KEY; +const defaultAgent = usePlatform + ? new LangGraphAgent({ + deploymentUrl: deploymentUrl, + graphId: "sample_agent", + langsmithApiKey: process.env.LANGSMITH_API_KEY, + }) + : new LangGraphHttpAgent({ + url: deploymentUrl, + }); // 3. Define the route and CopilotRuntime for the agent export const POST = async (req: NextRequest) => { diff --git a/docs/deployment.md b/docs/deployment.md index 9d19ea4..9ffdedd 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -1,52 +1,127 @@ # Deployment -## Render +## Render (Frontend Only) -The project includes a `render.yaml` for one-click deployment to [Render](https://render.com/). +The project includes a `render.yaml` that deploys the **frontend** to [Render](https://render.com/). +The agent is deployed separately — see [LangGraph Platform](#langgraph-platform-langsmith-cloud) below. -### Services +### Frontend Service (Node) -**Agent** (Python): -- Runtime: Python 3.12.6 -- Build: `pip install uv && uv sync` -- Start: `uv run uvicorn main:app --host 0.0.0.0 --port $PORT` -- Health check: `GET /health` -- Root directory: `apps/agent` - -**Frontend** (Node): - Runtime: Node 22 - Build: `corepack enable && pnpm install --no-frozen-lockfile && pnpm --filter @repo/app build` - Start: `pnpm --filter @repo/app start` - Health check: `GET /api/health` -- Root directory: (repo root) ### Environment Variables -| Variable | Service | Required | Notes | -|----------|---------|----------|-------| -| `OPENAI_API_KEY` | Agent | Yes | Your OpenAI API key | -| `LLM_MODEL` | Agent | No | Defaults to `gpt-5.4-2026-03-05` | -| `LANGSMITH_API_KEY` | Agent | No | For LangSmith tracing | -| `LANGGRAPH_DEPLOYMENT_URL` | Frontend | Auto | Injected from agent service via `fromService` | -| `SKIP_INSTALL_DEPS` | Frontend | No | Set to `true` to skip redundant installs | - -### Auto-Scaling +| Variable | Required | Notes | +|----------|----------|-------| +| `LANGGRAPH_DEPLOYMENT_URL` | Yes | URL of your LangGraph agent (e.g., `https://.default.us.langgraph.app`) | +| `LANGSMITH_API_KEY` | Yes* | Required when agent is on LangGraph Platform (sent as `x-api-key`) | +| `RATE_LIMIT_ENABLED` | No | Set to `true` to enable per-IP rate limiting | +| `RATE_LIMIT_WINDOW_MS` | No | Rate limit window in ms (default: 60000) | +| `RATE_LIMIT_MAX` | No | Max requests per window (default: 40) | +| `SKIP_INSTALL_DEPS` | No | Set to `true` to skip redundant installs | -Both services are configured with: -- Min instances: 1 -- Max instances: 3 -- Memory target: 80% -- CPU target: 70% +\*Not required if the agent is self-hosted on a private network without auth. ### Deploy 1. Fork the repository 2. Create a new **Blueprint** on Render 3. Connect your forked repo -4. Add `OPENAI_API_KEY` as a secret -5. Deploy +4. In the Render dashboard, set `LANGGRAPH_DEPLOYMENT_URL` to your agent's URL +5. Set `LANGSMITH_API_KEY` to your LangSmith API key +6. Deploy + +Render reads `render.yaml` and creates the frontend service. + +## LangGraph Platform (LangSmith Cloud) + +The agent can be deployed as a managed service on [LangGraph Platform](https://docs.langchain.com/langsmith/cli#deploy) using the `langgraph deploy` CLI. The platform provides built-in Postgres persistence, tracing, streaming, and auto-scaling. + +### Prerequisites + +- A [LangSmith](https://smith.langchain.com) account (Plus plan or higher) +- LangSmith API key (`LANGSMITH_API_KEY`) +- The `langgraph` CLI: `pip install langgraph-cli` +- Docker installed and running (Apple Silicon users need Docker Buildx) + +### Deploy + +```bash +cd apps/agent +langgraph deploy --name open-generative-ui-agent +``` + +The CLI reads `langgraph.json`, builds a Docker image, and pushes it to the managed registry. Auth is via `LANGSMITH_API_KEY` env var or `--api-key` flag. + +After deployment, configure environment variables in the [LangSmith dashboard](https://smith.langchain.com): + +| Variable | Required | Notes | +|----------|----------|-------| +| `OPENAI_API_KEY` | Yes | Your OpenAI API key | +| `LANGGRAPH_CLOUD` | Yes | Set to `true` — skips local checkpointer in favor of platform-managed Postgres | +| `LLM_MODEL` | No | Defaults to `gpt-5.4-2026-03-05` | +| `LANGCHAIN_TRACING_V2` | No | Set to `true` for built-in tracing | +| `LANGCHAIN_PROJECT` | No | Project name for organizing traces | + +Note the deployment URL (e.g., `https://.default.us.langgraph.app`). + +### Other CLI Commands + +| Command | Description | +|---------|-------------| +| `langgraph deploy list` | List all deployments | +| `langgraph deploy logs` | Fetch runtime or build logs | +| `langgraph deploy revisions list ` | Show revision history | +| `langgraph dev` | Local dev server (no Docker) | + +## Split Deployment: Render + LangGraph Platform + +The recommended production setup runs the frontend on Render and the agent on LangGraph Platform. This gives you managed scaling and Postgres persistence for the agent, with familiar Node hosting for the frontend. + +### Architecture + +``` +┌──────────────────────┐ HTTPS + x-api-key ┌─────────────────────────┐ +│ Render (Frontend) │ ──────────────────────────▶ │ LangGraph Platform │ +│ Next.js on Node 22 │ │ (LangSmith Cloud) │ +│ /api/copilotkit │ ◀────────────────────────── │ Python agent │ +└──────────────────────┘ SSE / streaming └─────────────────────────┘ +``` + +### Step-by-Step + +1. **Deploy the agent** on LangGraph Platform: + ```bash + cd apps/agent + langgraph deploy --name open-generative-ui-agent + ``` + Then set `OPENAI_API_KEY` and `LANGGRAPH_CLOUD=true` in the LangSmith dashboard. + Note the deployment URL. + +2. **Deploy the frontend** on Render following the [Render](#render-frontend-only) section. + In the Render dashboard, set: + - `LANGGRAPH_DEPLOYMENT_URL` = your LangGraph Platform URL (e.g., `https://.default.us.langgraph.app`) + - `LANGSMITH_API_KEY` = your LangSmith API key (starts with `lsv2_`) + +3. **Verify** by hitting `GET /api/health` on the frontend and sending a test message through the UI. + +### How Auth Works + +The frontend's API route (`/api/copilotkit`) reads `LANGSMITH_API_KEY` from the environment and sends it as an `x-api-key` header on every request to the agent. LangGraph Platform validates this key. The API key is never exposed to the browser — it stays server-side in the Next.js API route. + +### Self-Hosted vs. LangGraph Platform -Render reads `render.yaml` and creates both services. The frontend automatically gets the agent URL via service discovery. +| Concern | Self-Hosted | LangGraph Platform | +|---------|-------------|--------------------| +| Checkpointer | BoundedMemorySaver (in-memory) | Managed Postgres (automatic) | +| HTTP serving | FastAPI + uvicorn | Platform-managed | +| Health checks | `/health` endpoint | Platform-managed | +| Tracing | Optional (LANGSMITH_API_KEY) | Built-in | +| Scaling | Manual / render.yaml | Platform-managed | +| Auth | None (private network) | LANGSMITH_API_KEY required | ## General Deployment diff --git a/render.yaml b/render.yaml index 48a151c..65bbc1e 100644 --- a/render.yaml +++ b/render.yaml @@ -1,31 +1,4 @@ services: - # ── Agent (LangGraph Python) — native Python with FastAPI + uvicorn ── - - type: web - name: open-generative-ui-agent - runtime: python - plan: starter - rootDir: apps/agent - buildCommand: pip install uv && uv sync - startCommand: uv run uvicorn main:app --host 0.0.0.0 --port $PORT - healthCheckPath: /health - scaling: - minInstances: 1 - maxInstances: 3 - targetMemoryPercent: 80 - targetCPUPercent: 70 - envVars: - - key: PYTHON_VERSION - value: "3.12.6" - - key: OPENAI_API_KEY - sync: false - - key: LANGSMITH_API_KEY - sync: false - - key: LLM_MODEL - value: gpt-5.4-2026-03-05 - buildFilter: - paths: - - apps/agent/** - # ── Frontend (Next.js) — native Node runtime ── - type: web name: open-generative-ui-app @@ -45,10 +18,7 @@ services: - key: SKIP_INSTALL_DEPS value: "true" - key: LANGGRAPH_DEPLOYMENT_URL - fromService: - name: open-generative-ui-agent - type: web - property: hostport + sync: false - key: LANGSMITH_API_KEY sync: false - key: RATE_LIMIT_ENABLED