Demo

Product Manager II - LLM Evaluations

Datadog
New York, NY Full Time
POSTED ON 2/27/2025
AVAILABLE BEFORE 5/10/2025

We are seeking a highly motivated and experienced Product Manager to lead our new Large Language Model Observability product. This is an exciting opportunity to shape the future of observability in a dynamic and innovative environment in a nascent market.

As the Product Manager for LLM Observability, you will work with engineers and designers to build a product that our customers love and will use everyday to monitor and understand how well their LLM-based applications are working. We are looking for a passionate, hands-on product manager who will dive into conversations with customers, deeply understand their daily workflows and pains and ship a product that is integral to their daily workflows for monitoring and troubleshooting AI / ML model performance. Working with this team is an opportunity to drive impact in the rapidly growing field of AI evaluations and collaborating with the best Applied Scientists in the industry.

At Datadog, we place value in our office culture - the relationships and collaboration it builds and the creativity it brings to the table. We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them.

What You'll Do

  • Lead the development and execution of the roadmap for a large portion of our LLM Observability product
  • Work closely with engineers and designers to define the product feature, functionality as well as requirements for backend systems and user experience
  • Run beta programs with customers collecting feedback on the feature, functionality and usability of the core product offering
  • Partner with our AI research team and applied scientists to conceptualize, test and productize new ideas
  • Breakdown complexity and ambiguity into prioritized set of problems for Engineering to solve
  • Responsible for taking this product to the public, including sales and marketing enablement in partnership with marketing teams
  • Use your understanding of ML / AI or a relevant technical space understand end user workflows and translate that into product and design requirements

Who You Are

  • Expertise in AI / ML either as a practitioner (Data scientist) or a Product Manager
  • 3 years of PM experience or experience in the ML / AI space
  • Working knowledge of ML, data platforms, data engineering (e.g. key LLM / ML frameworks such as Langchain,, CrewAI, pyTorch etc.)
  • Comfortable diving into the code and running small LLM experiments when needed!
  • Strong, inherent understanding of customer workflows and challenges with MLOps tooling
  • A strong inclination towards taking initiative
  • Preferred : previous 0 to 1 product experience
  • Benefits & Growth :

  • New hire stock equity (RSUs) and employee stock purchase plan (ESPP)
  • Continuous professional development, product training, and career pathing
  • Intra-departmental mentor and buddy program for in-house networking
  • An inclusive company culture, ability to join our Community Guilds
  • Access to Inclusion Talks, our Internal panel discussions
  • Free, global Spring Health benefits for employees and dependents age 6
  • Competitive global benefits
  • Benefits and Growth listed above may vary based on the country of your employment and the nature of your employment with Datadog.

    Datadog offers a competitive salary and equity package, and may include variable compensation. Actual compensation is based on factors such as the candidate's skills, qualifications, and experience. In addition, Datadog offers a wide range of best in class, comprehensive and inclusive employee benefits for this role including healthcare, dental, parental planning, and mental health benefits, a 401(k) plan and match, paid time off, fitness reimbursements, and a discounted employee stock purchase plan.

    The reasonably estimated yearly salary for this role at Datadog is :

    149,000-$190,000 USD

    About Datadog :

    Datadog (NASDAQ : DDOG) is a global SaaS business, delivering a rare combination of growth and profitability. We are on a mission to break down silos and solve complexity in the cloud age by enabling digital transformation, cloud migration, and infrastructure monitoring of our customers' entire technology stacks. Built by engineers, for engineers, Datadog is used by organizations of all sizes across a wide range of industries. Together, we champion professional development, diversity of thought, innovation, and work excellence to empower continuous growth. Join the pack and become part of a collaborative, pragmatic, and thoughtful people-first community where we solve tough problems, take smart risks, and celebrate one another. Learn more about #DatadogLife on Instagram, LinkedIn, and Datadog Learning Center.

    Equal Opportunity at Datadog :

    Datadog is an Affirmative Action and Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. Here are our Candidate Legal Notices for your reference.

    Your Privacy :

    Any information you submit to Datadog as part of your application will be processed in accordance with Datadog's Applicant and Candidate Privacy Notice.

    Salary : $149,000 - $190,000

    If your compensation planning software is too rigid to deploy winning incentive strategies, it’s time to find an adaptable solution. Compensation Planning
    Enhance your organization's compensation strategy with salary data sets that HR and team managers can use to pay your staff right. Surveys & Data Sets

    What is the career path for a Product Manager II - LLM Evaluations?

    Sign up to receive alerts about other jobs on the Product Manager II - LLM Evaluations career path by checking the boxes next to the positions that interest you.
    Income Estimation: 
    $95,865 - $120,012
    Income Estimation: 
    $123,272 - $153,570
    Income Estimation: 
    $95,865 - $120,012
    Income Estimation: 
    $123,272 - $153,570
    Income Estimation: 
    $147,514 - $184,796
    Income Estimation: 
    $182,852 - $230,675
    Income Estimation: 
    $124,417 - $155,140
    Income Estimation: 
    $147,514 - $184,796
    Income Estimation: 
    $123,272 - $153,570
    Income Estimation: 
    $150,776 - $185,671
    View Core, Job Family, and Industry Job Skills and Competency Data for more than 15,000 Job Titles Skills Library

    Job openings at Datadog

    Datadog
    Hired Organization Address Denmark, IA Full Time
    No one likes dealing with errors. Error Tracking reduces noise by identifying errors that matter and equip developers to...
    Datadog
    Hired Organization Address New York, NY Full Time
    At Datadog, we're on a mission to build the best platform in the world for engineers to understand and scale their syste...
    Datadog
    Hired Organization Address New York, NY Full Time
    Datadog is building a modern AIOps solution to help users accelerate issue discovery and remediation in their system. A ...
    Datadog
    Hired Organization Address New York, NY Full Time
    We're on a mission to build the best platform in the world for engineers to understand and scale their systems, applicat...

    Not the job you're looking for? Here are some other Product Manager II - LLM Evaluations jobs in the New York, NY area that may be a better fit.

    Technical Product Manager, Prescient LLM

    1000 Genentech, Inc., New York, NY

    AI Assistant is available now!

    Feel free to start your new journey!