I ship AI products to production.

Proves a prompt change is safe before it ships.

Calvis runs an AI copilot for security guards. Working onsite with their CTO, I built the eval harness that decides whether a prompt change ships, by replaying real shifts against the old and new prompt on the same model.

• —The dataset is recorded guard shifts, replayed turn by turn and as full shifts. A recorded tool result is served only on an exact match of tool and input, so a changed prompt can't borrow the old run's answers.
• —Every change names what must improve, what must stay true and what must never happen, scored on behavior and tool calls rather than prose, over three runs. One edit took claim verification from 67–75% to 100% without losing a reply or an escalation.
• —It fixes itself. Point it at a shift and it finds what the copilot got wrong, writes a one-file prompt fix, and keeps it only if every safety check still passes. A human just approves the merge.

Push an agent. Get an API, an MCP server and a playground.

A deployment platform for AI agents. One git push ships an agent as an API, an MCP server, a CLI and a shareable playground.

• —An agent is a folder with a manifest. The platform builds it into an immutable artifact and serves it over HTTP, SSE streaming, a CLI and an auto-generated MCP server from one registry.
• —Agent code runs in an E2B sandbox, but model calls, traces and tools go back through a bridge to the platform gateway, which is the only path to an LLM and enforces each org's monthly budget.
• —Deploys come from a signed GitHub webhook with PR preview URLs. Eval cases, including LLM-as-judge, run on every deploy and block a regression, and long runs are durable on Inngest and can pause for human approval.

Ask your monitoring data what went wrong.

Built for BizScout: a live API monitor with a Claude analyst inside. Ask it why latency spiked, and it writes its own incident report when response times double.

• —A scheduler posts randomized payloads to httpbin, stores every result in Postgres through Prisma, and pushes new rows to the dashboard over Socket.IO.
• —The chat streams over SSE and can query the data with tools, up to three rounds per question. Every 60 seconds a monitor looks for responses slower than twice the rolling average and has Claude file a report through a forced tool call, so the output is always structured before it's saved.
• —Costs stay predictable: a token count before every call, an 8k input budget, a shared hourly rate limit, and a cache keyed to a fingerprint of the data, so answers go stale the moment new pings land.