This is not a demo. It is running on this site right now. Open the chat bubble on any page and ask about my work. Live metrics are on the /ops dashboard, and the source lives in the worker/ folder of this site's repository.
What it is
A production RAG chatbot that answers questions about me, grounded in this site's content: bio, experience, projects, publications, and the full text of every write-up. It is agentic: Claude holds a search_knowledge_base tool and decides per question whether to search or answer directly, then streams the answer token by token. The whole backend is one Cloudflare Worker at the edge, and the project covers the full lifecycle of an LLM system: retrieval design, evaluation, cost and latency tracking, and deployment.
Architecture
One Worker, four Cloudflare primitives (Workers, Workers AI, Vectorize, D1) and the Anthropic API. The dashed path is the no-search branch.
How a request flows
Each /chat request goes to Claude with a persona prompt and the search tool. Greetings and core facts are answered directly; anything deeper triggers a search, and the Worker feeds the results back and streams the grounded answer as Server-Sent Events. Retrieved text is wrapped as untrusted data so nothing inside a document can inject instructions, inputs are capped, CORS is locked to this origin, and the API key never leaves the Worker.
Hybrid retrieval
Two searches run in parallel: semantic (the query embedded with bge-base-en-v1.5 and matched by similarity in Vectorize) and keyword (BM25 via SQLite FTS5 in D1), which catches exact names that embeddings blur. The rankings are fused with Reciprocal Rank Fusion into 12 candidates, and a cheap Claude Haiku pass reranks them down to the 6 chunks the answer is grounded on. A slot is always reserved for the best project, about, and bio chunk so identity facts are never crowded out.
Design decisions
The small, critical facts (projects, publication, roles, contact) are pinned straight into the system prompt; retrieval is reserved for the ~350 chunks of write-up and project content that do not fit. That split came from a real failure: when full write-up bodies were first ingested, they crowded the projects and publication chunks out of the results entirely. A golden-question eval now guards that seam, failing below 90% retrieval recall and reporting every run to the /ops RAG tab.
The corpus itself is curated from the site: bio, project pages (parsed from the built HTML), and every write-up, split into overlapping chunks. npm run ingest purges both stores, then re-embeds and upserts everything so the semantic and keyword indexes stay in lockstep.
Cheatsheet
The whole system at a glance:
| Stage | Tech / model | Runs on |
|---|---|---|
| Embeddings | bge-base-en-v1.5 (768-dim) | Workers AI |
| Semantic search | Cosine similarity, top 12 | Cloudflare Vectorize |
| Keyword search | SQLite FTS5 (BM25) | Cloudflare D1 |
| Fusion | Reciprocal Rank Fusion (k = 60) | Worker |
| Rerank | Claude Haiku 4.5 → top 6 chunks | Anthropic API |
| Search decision + generation | Claude Haiku 4.5 with tool use | Anthropic API |
| Streaming | Server-Sent Events | Worker → browser |
| Tracing | Per-request traces; Langfuse spans | D1 + Langfuse |
| Eval gate | Golden-set recall ≥ 0.9 | npm run eval → /ops |
| Chat widget | React, on every page | GitHub Pages |
Deployment
The site is a Next.js static export on GitHub Pages; the backend deploys with one wrangler deploy. Secrets live in the Worker, CORS is pinned to the production origin, and there is no server to patch and no idle cost.
Observability
Every request is traced to D1: tokens, cost per pipeline stage, latency, retrieval quality, and the sources each answer cited. That feeds the live /ops dashboard, including the latest eval run with its PASS/FAIL badge.
Langfuse tracing
/ops shows the aggregate picture; Langfuse shows what happened inside one request: decision, rerank, and generation spans with per-span tokens, cost, and latency, plus retrieval stats attached to the trace. Events are posted straight to the Langfuse API and flushed with ctx.waitUntil(), so tracing adds zero latency for the visitor and no-ops entirely if the keys are unset.
