Demo
Salary Resume Critique Job Openings

Senior Data Engineer - Airflow Implementation

Bayforce
Denver, CO Full Time
POSTED ON 1/22/2025
AVAILABLE BEFORE 4/20/2025

This is a contract role with possibility of conversion to full time

Okay with remote but prefer someone who can come in onsite intially

NO OPT / H1 / VENDOR CANDIDATES

Position Summary

We are seeking an experienced Senior Data Engineer to spearhead the implementation of Airflow (Astronomer version) on Azure Cloud in a highly scalable and performance-driven environment. This role requires a hands-on expert with proven experience designing, building, and optimizing data workflows, as well as managing cloud-based DevOps systems, specifically with Airflow, Databricks, and Azure.

The ideal candidate will possess deep technical knowledge in data engineering, demonstrate leadership in designing and implementing large-scale data systems, and excel in documentation and process standardization. This is a pivotal role in ensuring the successful migration and optimization of existing workflows while creating a robust foundation for future data engineering projects.

Key Responsibilities

Airflow Implementation & Optimization

  • Architect, design, and implement Airflow (Astronomer) ecosystems from the ground up in Azure.
  • Lead efforts to establish best practices for Airflow, including branching strategies, deployment pipelines, and repository configuration.
  • Migrate existing orchestration workflows from Databricks to Airflow, ensuring seamless integration and enhanced performance.

Data Workflow Development & Optimization

  • Build, manage, and optimize scalable data workflows using Databricks PySpark , and SQL to handle large-scale datasets efficiently.
  • Collaborate with stakeholders to design and maintain operational data systems that meet performance, scalability, and availability standards.
  • Continuously monitor and fine-tune data processes to improve resource utilization and minimize workflow delays.

    • DevOps & Cloud Environment Management

  • Work directly with environment managers to configure, monitor, and optimize development, testing, and production environments in Azure.
  • Design cost-efficient, high-performing Azure cloud infrastructure tailored to project requirements.
  • Ensure alignment with CI / CD best practices and version control strategies for a seamless development lifecycle.

    • Documentation & Best Practices

  • Develop comprehensive design documents for review and approval by key stakeholders.
  • Standardize coding, testing, deployment, and documentation practices to ensure high-quality deliverables.
  • Mentor team members on implementing and maintaining robust data engineering standards.

    • Minimum Qualifications

    Experience

  • 5 years of experience in data engineering with a focus on large-scale environments and complex workflows.
  • Proven expertise in implementing Airflow and migrating workflows from Databricks.

    • Technical Proficiencies

  • Databricks

    • Extensive hands-on experience in building and managing pipelines.

  • Airflow

    • Demonstrated ability to set up and optimize Airflow environments, job scheduling, and automation.

  • Azure

    • Strong experience configuring and managing cloud environments on Azure.

  • Snowflake

    • Expertise in data warehousing with Snowflake.

  • PySpark & Python

    • Advanced proficiency in building and optimizing distributed data operations.

    Mastery in querying, managing, and transforming data.

  • DevOps

    • Experience with environment configuration, cost optimization, and cloud-based deployment pipelines.

    Education

  • Bachelor’s degree in Computer Science, Information Systems, or related field, or equivalent experience.

    • Preferred Skills

  • Experience with AWS or GCP alongside Azure.
  • Hands-on experience with Data Lakes / Delta Lake for large-scale data storage.
  • Familiarity with serverless technologies and event-driven data workflows.
  • Proficiency in CI / CD pipelines and tools specific to data engineering.
  • Exposure to additional tools in the big data ecosystem.

    • If your compensation planning software is too rigid to deploy winning incentive strategies, it’s time to find an adaptable solution. Compensation Planning
    Enhance your organization's compensation strategy with salary data sets that HR and team managers can use to pay your staff right. Surveys & Data Sets

    What is the career path for a Senior Data Engineer - Airflow Implementation?

    Sign up to receive alerts about other jobs on the Senior Data Engineer - Airflow Implementation career path by checking the boxes next to the positions that interest you.
    Income Estimation: 
    $122,257 - $154,284
    Income Estimation: 
    $143,391 - $179,890
    Income Estimation: 
    $122,257 - $154,284
    Income Estimation: 
    $143,391 - $179,890
    Income Estimation: 
    $71,122 - $96,652
    Income Estimation: 
    $92,929 - $122,443
    Income Estimation: 
    $92,929 - $122,443
    Income Estimation: 
    $122,257 - $154,284
    Employees: Get a Salary Increase
    View Core, Job Family, and Industry Job Skills and Competency Data for more than 15,000 Job Titles Skills Library

    Job openings at Bayforce

    Demand Planning Manager
    Bayforce
    Hired Organization Address Detroit, MI Contractor
    Description We are seeking a dedicated professional to take ownership of capacity and resource planning, with a focus on...
    SAP EWM Configuration Consultant
    Bayforce
    Hired Organization Address Mundelein, IL Full Time
    NO THIRD PARTIES SAP EWM Configuration Consultant 6 Months Hybrid Schedule Role Description : A direct client is looking...
    Director of IT Supply Chain & Logistics Systems
    Bayforce
    Hired Organization Address Forest, IL Full Time
    NO H1 / OPT / VENDOR CANDIDATES MUST BE LOCAL OR OPEN TO RELOCATE TO LAKE FOREST, ILLINOIS THIS IS A CONTRACT TO HIRE PO...
    Product Owner - Intrader
    Bayforce
    Hired Organization Address Buffalo, NY Contractor
    No C2C / OPT / H1 MUST HAVE SOME EXPERIENCE WITH INTRADER PREFERABALY BANKING / FINANCIAL BAKGROUND A Product Owner is r...
    View More Jobs at Bayforce

    Not the job you're looking for? Here are some other Senior Data Engineer - Airflow Implementation jobs in the Denver, CO area that may be a better fit.

    Senior Voice and Data Engineer

    MDS (Micro-Data Systems), Denver, CO

    MDS (Micro-Data Systems) | Senior Network Engineer

    MDS (Micro-Data Systems), Denver, CO

    View More Jobs

    AI Assistant is available now!

    Feel free to start your new journey!