AI Engineer - New York

Distyl AI develops AI native technologies for humans & AI to collaborate to power the operations of the Global Fortune 1000.

In just 24 months, we’ve rapidly grown to partner with some of the world’s largest enterprises—including F100 telecom, healthcare, manufacturing, insurance, and retail companies—delivering multiple AI deployments with $100M impact. Our platform, Distillery, along with our team of AI Engineers, Researchers, and Strategists, is pioneering AI-native systems of work, solving the most complex, high-stakes challenges at scale.

Distyl is founded and led by proven leaders from companies like Palantir, Apple, and top national laboratories. We work in deep partnership with OpenAI, jointly going-to-market at the largest enterprises and collaborating evaluating and testing the latest models. Backed by Lightspeed, Khosla, Coatue, industry leaders like Nat Friedman (former GitHub CEO), as well as board members of over 20 F500s, Distyl is building the future of AI-powered enterprise operations.

What We Are Looking For

As an AI Engineer at Distyl, you’ll be responsible for designing, implementing, and deploying AI-powered systems into production for some of the most iconic institutions in the world. You’ll partner closely with customers to understand their most critical challenges and deliver innovative, reliable solutions that harness the power of Large Language Models (LLMs). You’ll also contribute to the evolution of our internal platform, Distillery, shaping how enterprise-grade LLM applications are built and deployed.

You’ll be expected to bring technical depth, a builder’s mindset, and an obsession with quality and performance. You will work at the cutting edge of LLMs, evaluation models, designing responsible AI solutions, and implementing them into customer environments with speed and precision. You’ll also help evolve Distyl’s engineering best practices, bringing the discipline needed to scale.

This role requires strong software engineering skills, deep familiarity with machine learning workflows, and a bias toward practical, high impact solutions.

Key Responsibilities

• LLM Systems Engineering: Design and implement robust LLM-powered systems that deliver real business value. This includes model evaluation, prompt design, agent logic, pilot development, and production deployment.
• Customer Collaboration: Work closely with customer stakeholders to understand business pain points and technical constraints, then translate these into technical architectures and applications.
• Platform Development: Contribute to the development of Distillery, our internal platform for LLM application development, by building reusable components, tools, and infrastructure to support the broader engineering team.
• Production Quality: Ensure that deployed systems are observable, maintainable, and meet rigorous performance and reliability standards.
• AI Evaluation: Apply a scientific and thoughtful approach to model vetting and evaluation, accounting for accuracy, explainability, cost, latency, and robustness.
• Continuous Improvement: Drive improvements in system architecture, development workflows, and deployment processes across teams.
• Tech Leadership: Mentor peers and contribute to a culture of technical excellence, collaboration, and curiosity.
  
  

What We Require

• 3 years of experience as a software or machine learning engineer.
• Strong programming skills and experience with production software development.
• Expertise in machine learning frameworks (e.g., PyTorch, Transformers, Scikit-learn), data libraries (e.g., NumPy, Pandas), and ML operations.
• Strong understanding of AI/ML evaluation and how to operationalize responsible, reliable systems.
• Experience with cloud infrastructure and modern deployment practices (e.g., CI/CD, Git, Docker, AWS/GCP/Azure).
• Bachelor’s degree in Computer Science, Mathematics, or a related technical field.
• Strong communication and collaboration skills, with the ability to work cross-functionally with customers and internal teams.
• Passion for AI and a pragmatic approach to solving real-world problems.
  
  

What We Offer

• Competitive salary and benefits package, including equity options, medical/dental/vision covered at 100% for you and your dependents, 401K plan, and perks such as commuter benefits and lunch provided in office.
• Opportunity to be part of a mission-oriented company to create practical adoption during the biggest revolution in human productivity.
• Opportunity to work on cutting-edge AI technologies and make a real impact on the adoption of LLMs in the enterprise space.
• Collaborative and supportive work environment, with a strong focus on personal and professional growth.
• Exposure to a diverse range of real-world industries and projects, providing unique challenges and learning opportunities.
  
  

Note: Distyl is a hybrid working environment and requires in-office collaboration 3 days a week. We have offices in San Francisco, CA and New York, NY, and this role is available in either location.