Projects
Flagship first. Every claim below is backed by tests, deploy evidence, or benchmark data in the repos.
Verifier-backed codebase onboarding copilot. Point it at an unfamiliar repository and it maps the architecture, explains entry paths from AST evidence, and — unlike most code-explanation tools — shows its uncertainty instead of hiding it.
- Evidence-first pipeline: a LangGraph Supervisor routes each question to role-contracted worker agents (architect_mapper, entry_explainer, verifier) that emit typed ClaimPackets with per-claim provenance.
- Every claim gets a label:
verified(AST/repo/test evidence),unverified(no grounding — shown, not hidden), orcontradicted(a failing test challenges it, and the final prose is rewritten). - Hardened NL→symbol resolution: backticked/dotted symbols, CLI entry points from pyproject.toml, module-behavior questions — ambiguity resolves to "refuse" rather than "guess."
- Production-shaped: auth + encrypted key storage, SQLite/Postgres run stores, rate limiting, sandboxed test-runner worker, readiness probes, full trace metadata schema — deployed publicly on Railway with recorded smoke evidence.
Open-source evaluation framework for LLM agents, shipped with a Supervisor-vs-ReAct benchmark: same model, same five MCP tools, 40 tasks across 10 Python OSS repos — so the comparison isolates orchestration, not tooling.
- Headline result: the Supervisor architecture used ~12× fewer tokens (396k vs 4.8M) and completed all 40 tasks with zero errors, while the ReAct baseline failed 6/40 by blowing past its recursion limit.
- Honest reporting: ReAct scored higher raw answer quality on tasks it finished (factual 0.70 vs 0.48) — the report says so, because a benchmark that hides the trade-off is worthless.
- Four metrics: routing accuracy, LLM-as-judge factual correctness with variance-gated self-consistency, citation grounding (anti-hallucination symbol resolver), and verification rate from real pytest execution.
- Run/score split: expensive agent runs persist to JSONL; a mid-analysis resolver bug was fixed and re-scored offline for free — citation score corrected 0.37 → 0.80 without re-running a single agent.
Three self-authored, deterministic MCP servers that form wayfinder's fact layer. None of them contains an LLM — if a symbol doesn't exist, they return a structured not-found instead of inventing an answer.
- mcp-repo-mapper: repository structure, language breakdown, Python dependency graph, circular-dependency detection, framework detection, ranked entry-point candidates.
- mcp-ast-explorer: LibCST-based symbol layer — definitions, signatures, references, call chains, class hierarchies. Refuses ambiguous bare names rather than guessing.
- mcp-test-runner: bounded pytest/Jest execution (timeouts, CPU/memory limits, shell=False), normalized JSON result parsing, coverage summaries — the layer that turns claims into verdicts.
- Designed as a stack: structure → symbols → execution mirrors how an engineer actually reads a new codebase.