Thoughts on AI, engineering, and building things that matter.
How a one-program-per-row Triton softmax gets within 17% of the memory roofline on H100 — the bandwidth math, the measured numbers, and the shapes where it honestly loses to torch.softmax.
I rebuilt vLLM's two core ideas — a block-paged KV cache and iteration-level scheduling — in a from-scratch engine. Here's the design logic, the ownership contract that prevents KV-block leaks, and the preemption policy that fell out of it.
OpenEvidence's moat isn't better models. It's that doctors share things with it they'd never tell ChatGPT — and that made me rethink where my own product's value actually comes from.
Sub-agents aren't about doing more things at once. They're about keeping your context clean.
When execution costs approach zero, the scarce input becomes real-world signal. Not more brainstorming. Actual conversations with actual people.