Modal
Member of Technical Staff - Research, Inference
Full-timeNot specifiedNew YorkNot disclosedApply by 3 Sept 2026
Overview
We're building it at Modal. Every era of computing brought new workloads that previous infrastructure couldn't support: mainframes, databases, and the cloud. Each time, the company that rebuilt the layer underneath defined the decade.
What you'll do
- We're building a platform that covers the whole life of an LLM, train it, deploy it, observe it, and inference is where teams feel the difference every day.
- We already run elastic inference, sandboxes, distributed volumes, and multi-node training, and we control the infrastructure underneath, so the serving stack is ours to shape rather than something we resell.
- You will do hands-on inference research at Modal, working with the research lead to pick high-impact bets and owning them end to end.
- The bets that matter most are the ones that move cost per token and tail latency on the workloads our customers actually run.
- Own end-to-end inference research bets: speculative decoding, disaggregated prefill/decode, quantization (FP8, INT4), KV-cache and memory management, autoscaling for spiky serverless traffic, and whatever else the research agenda calls for.
- Train custom speculators against real production traffic and feed what you learn back into target models, acceptance length is the metric that decides the win.
- Work directly with customers alongside our Forward Deployed Engineers to deploy and tune models, and bring what you learn back into the research.
- Carry and expand collaborations with outside research labs, for example:
Requirements
- Practical experience with Node, Go, LLM, RAG.
- Relevant academic or project background for a Member of Technical Staff - Research, Inference role.
- Strong written and verbal communication in English.
- Comfortable working on-site in New York.
Skills
NodeGoLLMRAG