Senior Rust Developer

Our client is building an open-source platform designed to optimize LLM (Large Language Model) applications by creating a feedback loop that transforms production data into smarter, faster, and cheaper models. Their platform enables:

• Model Gateway Integration: Easily integrate our model gateway.
• Metrics & Feedback: Send metrics or feedback for continuous model improvement.
• LLM Optimization: Optimize prompts, models, and inference strategies.
• Continuous Improvement: Watch your LLMs improve over time through real-time data feedback.

Their platform unifies key aspects of LLM development:

• Inference: One API for all LLMs, with <1ms P99 overhead.
• Observability: Inference and feedback are seamlessly integrated with your database.
• Optimization: From prompt engineering to fine-tuning and reinforcement learning (RL).
• Experimentation: Built-in A/B testing, routing, and fallbacks for model experiments.

They're early in our journey, having started in stealth mode in February, completing a successful technical pilot over the summer, and recently launching as an open-source project in September. With backing from top-tier investors like FirstMark, Bessemer, and Bedrock, we have a strong runway to focus on our long-term vision and open-source goals.

Role Overview:

They are looking for a Founding Member of Technical Staff with a strong background in systems engineering (especially Rust) to join our core technical team. As part of this early-stage, high-impact role, you’ll have the opportunity to shape the technical direction of the platform, building from the ground up.

You’ll work on our model gateway, which is written in Rust and serves as the backbone of the project. Your work will range from inference-time optimizations (e.g., dynamic in-context learning) to advanced experimentation (e.g., built-in Statsig functionality).

Key Responsibilities:

• Drive Technical Projects: Lead large, complex projects from inception to deployment, including building out critical systems infrastructure.
• Optimize LLMs: Work on the optimization of LLM inference, model architectures, and fine-tuning strategies.
• Backend Systems Expertise: Leverage your deep technical knowledge in backend systems and Rust to shape the architecture of our platform.
• Technical Leadership: Provide guidance and mentorship across the engineering team, fostering a culture of technical excellence.
• Cross-Functional Impact: Contribute to both the backend and broader tech stack while expanding your expertise in machine learning, systems engineering, and beyond.

What We’re Looking For:

• Strong Technical Background: You've tackled tough technical challenges and are comfortable taking large projects from ideation through deployment.
• Experience in Backend / Systems Engineering: Expertise in backend systems, especially with Rust. Experience optimizing high-performance systems and working with complex architectures.
• Growth Mindset: You’re excited to continuously learn and expand your skill set, particularly in machine learning and LLM optimization. You’re open to contributing across the tech stack.
• In-Person Collaboration: You’re based in New York City and comfortable working in the office five days a week, contributing to a highly collaborative environment.

What They Offer:

• Competitive Compensation: We offer a competitive salary, equity, and benefits, reflecting the value you bring to the team.
• Open-Source Focus: The vast majority of your work will be open-source and public, making an immediate impact in the community.
• Learning & Growth Opportunities: While you'll bring a strong backend/systems engineering background, you’ll have the chance to expand your knowledge and skills into areas like machine learning and reinforcement learning.
• Collaborative, Technical Team: Join a small, highly technical team in-person, where you’ll play a central role in shaping the vision, culture, and engineering practices of the project.
• Best-in-Class Investors: Backed by leading investors like FirstMark, Bessemer, and Bedrock, we have years of runway, giving us the flexibility to focus on our long-term mission.

Team & Culture:

Their team is composed of passionate, driven engineers and researchers who are eager to tackle the challenges of the future of LLMs. You’ll work alongside leaders in the field with diverse expertise — from systems engineering to reinforcement learning — and help shape our open-source project from the ground up.

If you’re a skilled backend engineer with a passion for systems, a curiosity about LLMs, and an appetite for building impactful, open-source projects, we’d love to meet you!