Demo

Software Engineer, Core Engine

Eventualcomputing
San Francisco, CA Full Time
POSTED ON 1/15/2025
AVAILABLE BEFORE 4/11/2025

About Eventual

Eventual is a data platform that helps data scientists and engineers build data applications across ETL, analytics, and ML / AI.

Is your CV ready If so, and you are confident this is the role for you, make sure to apply asap.

OUR PRODUCT IS OPEN-SOURCE AND USED AT ENTERPRISE SCALE

Our distributed data engine Daft is open-sourced and runs on 800k CPU cores daily. This is more compute than Frontier, the world's largest supercomputer!

Daft is used at leading AI / ML companies such as Amazon, TogetherAI, EssentialAI, CloudKitchens, and more. It makes ML / AI workloads easy and performant to run alongside traditional relational tabular workloads.

Today's “Big Data” data tooling (Spark, Trino, Snowflake) was built for a world of tabular data analytics. They do not generalize well to the needs of modern ML / AI data workloads. We built Daft to be the successor to these Big Data technologies along these core principles :

  • Python-native : Python is the native language of ML / AI and most of data engineering today.
  • First-Class Local Development UX : Interactive development in a local Python notebook or script is where the magic happens.
  • Multimodal Data Support : Modern workloads require support for operations on complex types such as long-form text, images, tensors, and more.
  • Heterogeneous Compute (GPUs) : GPUs are a requirement for workloads that perform model batch inference as part of the overall query.

Key Responsibilities :

As a Software Engineer on the Core Engine team, you will build key capabilities for the Daft distributed data engine. You will be working on core architectural design and implementation of various components in Daft including :

  • Planning / Query Optimizer : Intelligently optimize users’ workloads with modern database techniques.
  • Execution Engine : Improve memory stability through the use of streaming computation and more efficient data structures.
  • Distributed Scheduler : Improve Daft’s resource utilization, task scheduling, and fault tolerance.
  • Storage : Improve Daft integrations with modern data lake technologies such as Apache Parquet, Apache Iceberg, and Delta Lake.
  • Our goal is to build the world’s best open-source distributed query engine, becoming the leading framework for data engineering and analytics. We are a young startup - so be prepared to wear many hats such as tinkering with infrastructure, talking to customers, and participating heavily in the core design process of our product!

    What we look for :

    We are looking for a candidate with a strong foundation in systems programming and ideally experience with building distributed data systems or databases (e.g., Hadoop, Spark, Dask, Ray, BigQuery, PostgreSQL, etc.)

    Our ideal candidate has :

  • 3 years of experience working with distributed data systems (query planning, optimizations, workload pipelining, scheduling, networking, fault tolerance, etc.).
  • Strong fundamentals in systems programming (e.g., C , Rust, C) and Linux.
  • Familiarity and experience with cloud technologies (e.g., AWS S3, etc.).
  • Most importantly, we are looking for someone who works well in small, focused teams with fast iterations and lots of autonomy. If you are passionate, intellectually curious, and excited to build the next generation of distributed data technologies, we want you on the team!

    Benefits and Remote Work

    We are believers in both having the flexibility of remote work but also the importance of in-person work, especially at the earliest stages of a startup. We have a flexible hybrid approach to in-person work with at least 3 days of in-person work typically from Monday - Wednesday at our office in San Francisco.

    We believe in providing employees with best-in-class compensation and benefits including meal allowances, comprehensive health coverage including medical, dental, vision, and more.

    About the interviewINTRODUCTORY CALL [15M]

    A short phone screen over video call with one of our co-founders for us to get acquainted, understand your aspirations, and evaluate if there is a good fit in terms of the type of role you are looking for.

    TECHNICAL PHONE SCREEN [1 HR]

    A technical phone screen question over video call to understand your technical abilities.

    TECHNICAL INTERVIEW PANEL [4 HR]

    Technical interviews with the rest of the Eventual team with questions to further understand your technical strengths, weaknesses, and experiences.

    MEET THE TEAM

    As many chats as necessary to get to know us - come have a coffee with our co-founders and existing team members to understand who we are and our goals, motivations, and ambitions.

    We look forward to meeting you!

    WE'RE GROWING - COME GROW WITH US!

    We are well funded by investors such as YCombinator, Caffeinated Capital, Array.vc, and top angels in the valley from Databricks, Meta, and Lyft.

    Our team has deep expertise in high-performance computing, big data technologies, cloud infrastructure, and machine learning. Our team members have previously worked in top technology companies such as Amazon, Databricks, Tesla, and Lyft.

    We are looking for exceptional individuals with a passion for technology and a strong sense of intellectual curiosity. If that sounds like you, please reach out even if you don't see a specific role listed that matches your skillsets - we'd love to chat!

    J-18808-Ljbffr

    If your compensation planning software is too rigid to deploy winning incentive strategies, it’s time to find an adaptable solution. Compensation Planning
    Enhance your organization's compensation strategy with salary data sets that HR and team managers can use to pay your staff right. Surveys & Data Sets

    What is the career path for a Software Engineer, Core Engine?

    Sign up to receive alerts about other jobs on the Software Engineer, Core Engine career path by checking the boxes next to the positions that interest you.
    Income Estimation: 
    $123,167 - $152,295
    Income Estimation: 
    $146,673 - $180,130
    Income Estimation: 
    $97,257 - $120,701
    Income Estimation: 
    $123,167 - $152,295
    Income Estimation: 
    $123,167 - $152,295
    Income Estimation: 
    $146,673 - $180,130
    Income Estimation: 
    $146,673 - $180,130
    Income Estimation: 
    $176,149 - $220,529
    Income Estimation: 
    $77,657 - $95,021
    Income Estimation: 
    $97,257 - $120,701
    View Core, Job Family, and Industry Job Skills and Competency Data for more than 15,000 Job Titles Skills Library

    Job openings at Eventualcomputing

    Eventualcomputing
    Hired Organization Address San Francisco, CA Full Time
    Key Responsibilities As a Senior Software Engineer, you will be an early member of the Eventual team with primary respon...
    Eventualcomputing
    Hired Organization Address San Francisco, CA Full Time
    About Eventual Eventual is a data platform that helps data scientists and engineers build data applications across ETL, ...
    Eventualcomputing
    Hired Organization Address San Francisco, CA Full Time
    Key Responsibilities As a Fullstack Engineer, you will be an early member of the Eventual team with primary responsibili...

    Not the job you're looking for? Here are some other Software Engineer, Core Engine jobs in the San Francisco, CA area that may be a better fit.

    Software Engineer, Core Engine

    Eventual, San Francisco, CA

    Software Engineer - Core Systems

    Health Plan of San Mateo, South San Francisco, CA

    AI Assistant is available now!

    Feel free to start your new journey!