Demo

Databricks Lead Data Engineer

K2 Insurance Services, LLC
Poway, CA Full Time
POSTED ON 2/23/2025
AVAILABLE BEFORE 5/21/2025

Databricks Lead Data Engineer

Full Time

Remote - San Diego, CA, US

Salary Range : $200,000 To $260,000 Annually

Databricks Lead Data Engineer K2 Insurance Services

K2 Insurance Services, LLC seeks a full-time Lead Databricks Data Engineer to spearhead our data engineering initiatives, focusing on designing, building, and optimizing scalable data solutions using Databricks. As a leader, you will supervise a team of data engineers, collaborate with cross-functional stakeholders, and define best practices to unlock the full potential of our data.

K2 Insurance Services offers the opportunity to join an established company in growth mode. Our compensation program includes competitive pay; bonus plan; medical, dental, and vision insurance; paid time-off in year of hire; and 401(k) with employer match.

Salary Range : $200,000 To $260,000 USD per year

Key Responsibilities :

  • Leadership & Strategy :

Supervise and mentor a team of data engineers in implementing Databricks-based ETL / ELT solutions.

  • Define and drive the data engineering strategy, ensuring alignment with business goals.
  • IT Product Owner role for organizational data warehouse project.
  • Databricks Expertise :
  • Design and develop scalable data pipelines and ETL / ELT processes using Databricks.

  • Optimize and tune Spark jobs for performance and efficiency.
  • Develop and enforce best practices for Databricks cluster management and data security.
  • Data Architecture :
  • Build and maintain robust data models to support analytics and reporting needs.

  • Integrate Databricks with various data sources (cloud storage, databases, APIs).
  • Implement Delta Lake for reliable, scalable, and performant data lakes.
  • Collaboration :
  • Work closely with data scientists, analysts, and stakeholders to deliver actionable insights.

  • Act as the technical liaison between the data engineering team and other stakeholders.
  • Innovation & Optimization :
  • Stay updated on the latest Databricks and Azure features to drive continuous improvements.

  • Automate repetitive processes and streamline data workflows.
  • Qualifications :

  • Proven experience (10 years) in data engineering, with 2 years as a team supervisor or equivalent role.
  • Expertise in Databricks, including Spark, Delta Lake, Delta Live Tables, and MLFlow.
  • Proficiency in Python, SQL, and at least one cloud platform (Azure, AWS, or GCP).
  • Hands-on experience with data lakes, data warehouses, and big data technologies.
  • Strong understanding of CI / CD pipelines and infrastructure as code (IaC).
  • Excellent problem-solving and communication skills.
  • Prior insurance industry experience is a strong plus.
  • Preferred Skills :

  • Familiarity with Databricks Unity Catalog and Lakehouse architecture.
  • Experience with workflow orchestration tools.
  • Knowledge of machine learning pipelines and integrations.
  • Certifications in Databricks, Azure, AWS, or GCP.
  • Previous IT product owner role.
  • Compensation details : 200000-260000 Yearly Salary

    PI36dc60f22a6e-26289-36571660

    Salary : $200,000 - $260,000

    If your compensation planning software is too rigid to deploy winning incentive strategies, it’s time to find an adaptable solution. Compensation Planning
    Enhance your organization's compensation strategy with salary data sets that HR and team managers can use to pay your staff right. Surveys & Data Sets

    What is the career path for a Databricks Lead Data Engineer?

    Sign up to receive alerts about other jobs on the Databricks Lead Data Engineer career path by checking the boxes next to the positions that interest you.
    Income Estimation: 
    $92,929 - $122,443
    Income Estimation: 
    $122,257 - $154,284
    Income Estimation: 
    $92,929 - $122,443
    Income Estimation: 
    $122,257 - $154,284
    Income Estimation: 
    $71,122 - $96,652
    Income Estimation: 
    $92,929 - $122,443
    Income Estimation: 
    $122,257 - $154,284
    Income Estimation: 
    $143,391 - $179,890
    View Core, Job Family, and Industry Job Skills and Competency Data for more than 15,000 Job Titles Skills Library

    Job openings at K2 Insurance Services, LLC

    K2 Insurance Services, LLC
    Hired Organization Address Alpine, CA Full Time
    Databricks Lead Data Engineer Full Time Remote - San Diego, CA, US Salary Range : $200,000 To $260,000 Annually Databric...
    K2 Insurance Services, LLC
    Hired Organization Address Bonita, CA Full Time
    Databricks Lead Data Engineer Full Time Remote - San Diego, CA, US Salary Range : $200,000 To $260,000 Annually Databric...
    K2 Insurance Services, LLC
    Hired Organization Address San Marcos, CA Full Time
    Databricks Lead Data Engineer Full Time Remote - San Diego, CA, US Salary Range : $200,000 To $260,000 Annually Databric...
    K2 Insurance Services, LLC
    Hired Organization Address Imperial, CA Full Time
    Databricks Lead Data Engineer Full Time Remote - San Diego, CA, US Salary Range : $200,000 To $260,000 Annually Databric...

    Not the job you're looking for? Here are some other Databricks Lead Data Engineer jobs in the Poway, CA area that may be a better fit.

    Databricks Lead Data Engineer

    K2 Insurance Services, LLC, Bonita, CA

    Databricks Lead Data Engineer

    K2 Insurance Services, LLC, Imperial, CA

    AI Assistant is available now!

    Feel free to start your new journey!