DraftPrivate AI / Systems research
Serving 500B+ parameter mixture-of-experts models on a workstation
Expert streaming, speculative decoding, and the practical limits of single-machine inference.
Field notes
Drafts from the systems we build and the questions we study. These titles are in preparation; no articles are published here yet.
Expert streaming, speculative decoding, and the practical limits of single-machine inference.
Scheduling, memory pressure, and the design choices behind multi-model serving.
Where dispatch, service records, billing, and reconciliation meet.
How to make human responsibility and release decisions visible in AI-native delivery.
A conversation with the people doing the work
Describe the system and the decision you need to make.