Senior AI Builder

Overview

Replit is the agentic software creation platform that enables anyone to build applications using natural language. With millions of users worldwide, Replit is democratizing software development by removing traditional barriers to application creation. The AI Studio is the team responsible for designing, building, and operationalizing internal software platforms across every functional department of the company, inclu

What you'll do

• Design, architect, and ship custom AI-powered internal platforms that serve specific operational needs across Replit’s functional departments, including but not limited to People Operations, Customer Support, Sales, Recruiting, Marketing, Finance, and Executive Operations.
• Translate ambiguous, often non-technical business requirements into technical specifications, system architectures, and shipped software products.
• Own platform development end-to-end, including front-end interfaces, back-end services, data models, third-party integrations, and AI model orchestration.
• Applied AI and Large Language Model Engineering
• Architect multi-stage AI pipelines incorporating large language models, including prompt engineering, tool-call architecture, structured output generation, retrieval-augmented generation, and agentic workflows.
• Conduct model selection across multiple frontier and open-source providers, evaluating models against cost per query, latency, output quality, reliability, and task-specific performance.
• Design and implement evaluation frameworks for AI features, including test datasets, scoring rubrics, and regression tests, so that AI-driven workflows stay reliable in production.
• Optimize AI systems for cost and performance, including the appropriate use of lighter-weight models for qualification passes, batching strategies for high-throughput workloads, and caching architectures.

Requirements

• Own platform development end-to-end, including front-end interfaces, back-end services, data models, third-party integrations, and AI model orchestration.
• Applied AI and Large Language Model Engineering
• Architect multi-stage AI pipelines incorporating large language models, including prompt engineering, tool-call architecture, structured output generation, retrieval-augmented generation, and agentic workflows.
• Conduct model selection across multiple frontier and open-source providers, evaluating models against cost per query, latency, output quality, reliability, and task-specific performance.
• Design and implement evaluation frameworks for AI features, including test datasets, scoring rubrics, and regression tests, so that AI-driven workflows stay reliable in production.
• Optimize AI systems for cost and performance, including the appropriate use of lighter-weight models for qualification passes, batching strategies for high-throughput workloads, and caching architectures.
• Business Understanding and Stakeholder Partnership
• Treat each internal department as its own startup: understand its goals, its constraints, and the metrics that define whether it is succeeding.

Skills