Demo

Engineering Manager, Inference Engine

The Rundown
San Francisco, CA Full Time
POSTED ON 3/11/2025
AVAILABLE BEFORE 6/12/2025

About the Team

The Inference Engine team builds and scales the inference infrastructure powering OpenAI's research and production models. Our mission is to ensure high availability, performant, and cost-effective utilization of AI models at scale. We manage all OpenAI GPU-related code and infrastructure, enabling seamless execution of AI workloads.

About the Role

We are seeking an experienced engineering manager to lead our inference engine team, overseeing key technical designs and operations to deliver high-performance inference capabilities.

In this role, you will :

  • Lead and scale a team of engineers across multiple domains in the model inference.
  • Define and execute a technical roadmap to optimize inference engine performance, flexibility and scalability.
  • Define and achieve goals to efficiently serve AI models with an emphasis on GPU workload optimization, and inference performance.
  • Collaborate with stakeholders to align technical solutions meeting research and business goals.
  • Provide strong technical leadership, driving best practices in execution and collaboration.

You might thrive in this role if you :

  • Are highly technical.
  • Have 10 years of experience in ML model training, serving, infrastructure or Performance Engineering, with 5 years in management.
  • Possess deep expertise in LLM inference systems and distributed computing.
  • Have experience in GPU kernel programming or optimization.
  • Have experience in close collaboration with ML researchers.
  • Nice to have) Have experience in performance engineering.
  • Thrive in fast-paced environments with evolving priorities and ambitious goals, with the ability to work closely with technical and research stakeholders.
  • Have experience leading high performing teams with a focus on scaling and fostering a culture of inclusion and performance.
  • About OpenAI

    OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.

    J-18808-Ljbffr

    If your compensation planning software is too rigid to deploy winning incentive strategies, it’s time to find an adaptable solution. Compensation Planning
    Enhance your organization's compensation strategy with salary data sets that HR and team managers can use to pay your staff right. Surveys & Data Sets

    What is the career path for a Engineering Manager, Inference Engine?

    Sign up to receive alerts about other jobs on the Engineering Manager, Inference Engine career path by checking the boxes next to the positions that interest you.
    Income Estimation: 
    $151,448 - $188,145
    Income Estimation: 
    $203,425 - $249,816
    Income Estimation: 
    $213,375 - $267,876
    Income Estimation: 
    $190,687 - $235,769
    Income Estimation: 
    $151,448 - $188,145
    Income Estimation: 
    $203,425 - $249,816
    Income Estimation: 
    $213,375 - $267,876
    Income Estimation: 
    $190,687 - $235,769
    Income Estimation: 
    $85,996 - $102,718
    Income Estimation: 
    $111,859 - $131,446
    Income Estimation: 
    $110,457 - $133,106
    Income Estimation: 
    $105,809 - $128,724
    Income Estimation: 
    $122,763 - $145,698
    Income Estimation: 
    $105,809 - $128,724
    Income Estimation: 
    $136,611 - $163,397
    Income Estimation: 
    $135,163 - $163,519
    Income Estimation: 
    $131,953 - $159,624
    Income Estimation: 
    $150,859 - $181,127
    Income Estimation: 
    $190,687 - $235,769
    Income Estimation: 
    $218,238 - $263,470
    Income Estimation: 
    $213,354 - $274,761
    View Core, Job Family, and Industry Job Skills and Competency Data for more than 15,000 Job Titles Skills Library

    Job openings at The Rundown

    The Rundown
    Hired Organization Address Seattle, WA Full Time
    As an Applied Machine Learning Engineer, you’ll research and utilize established and state-of-the-art machine learning (...
    The Rundown
    Hired Organization Address Washington, DC Full Time
    About the Team OpenAI’s Global Affairs team leads efforts to build trust and understanding of OpenAI’s work among policy...
    The Rundown
    Hired Organization Address Washington, DC Full Time
    About Shield AI Founded in 2015, Shield AI is a venture-backed defense technology company focused on protecting service ...

    Not the job you're looking for? Here are some other Engineering Manager, Inference Engine jobs in the San Francisco, CA area that may be a better fit.

    Engineering Manager, Production Inference

    OpenAI, San Francisco, CA

    AI Assistant is available now!

    Feel free to start your new journey!