Lead Data Engineer

Want to learn more about this role and Jobot? Click our Jobot logo and follow our LinkedIn page!

Job details

Remote - Lead Data Engineer - up to $190K base - join a team building systems to make data driven business decisions

This Jobot Job is hosted by Lucas Watson

Are you a fit? Easy Apply now by clicking the "Easy Apply" button and sending us your resume.

Salary $150,000 - $190,000 per year

A Bit About Us

Our client, in the financial services industry, is seeking a Lead Data Engineer to join their team. This is a full-time, direct hire, remote role that can pay $150-190K base salary plus benefits, depending on experience.



Why join us?


This role is ideal for someone who thrives in a dynamic, fast-paced environment, enjoys solving complex data problems, and is passionate about driving innovation in data engineering. If you're looking to make an impact on the financial landscape with cutting-edge data solutions this could be for you!

Job Details

Core Responsibilities

• Lead the design and implementation of end-to-end data pipelines, from extraction (API, scraping, pyodbc) to cleansing/transformation (Python, TSQL) and loading into SQL databases or data lakes.
• Oversee the development of robust data architectures that support efficient querying and analytics, ensuring high-performance and scalable data workflows.
• Collaborate with data scientists, software developers, business intelligence teams, and stakeholders to develop and deploy data solutions that meet business needs.
• Ensure smooth coordination between engineering and other teams to translate business requirements into technical solutions.
• Guide the development of data models and business schemas, ensuring that they are optimized for both relational (3NF) and dimensional (Kimball) architectures.
• Lead the creation of scalable, reliable data models and optimize them for performance and usability.
• Develop and maintain the infrastructure for large-scale data solutions, leveraging cloud platforms (e.g., Azure) and containerization technologies (e.g., Docker).
• Lead the use of modern data platforms such as Snowflake and Fabric, ensuring their effective use in large-scale data solutions.
• Manage and optimize data pipelines using tools such as Apache Airflow, Prefect, DBT, and SSIS, ensuring that all stages of the pipeline (ETL) are efficient, scalable, and reliable.
• Ensure robust testing, monitoring, and validation of all data systems and pipelines.
• Drive continuous improvement in data engineering processes and practices, ensuring they remain cutting-edge, efficient, and aligned with industry best practices.
• Foster a culture of clean code, best practices, and rigorous testing across the team.
• Strong experience with data pipeline design and implementation, including data extraction, transformation, and loading (ETL) processes.
• Proficiency in SQL (Postgres, SQL Server) and experience with modern data warehouse solutions (e.g., Snowflake, Fabric).
• Expertise in Python for data engineering tasks, including data manipulation (Pandas, NumPy) and workflow management (Dask, PySpark, FastAPI).
• Solid knowledge of cloud platforms (Azure, AWS) and big data technologies (Hadoop, Spark).
• Hands-on experience with Docker, Kubernetes, and containerized environments.
• Strong understanding of dimensional modeling (Kimball), relational database design (3NF), and best practices in data architecture.
• Experience with API development, including building and managing API integrations.
• Proficiency with orchestration tools like Prefect or Airflow for workflow management.
• Strong focus on testing and validation, ensuring that all data systems meet reliability and performance standards.
  
  

Experience & Qualifications

• 5 years of experience in data engineering roles, with a proven track record of developing and maintaining data pipelines and architectures.
• Experience working with large-scale data platforms and cloud environments.
• Strong background in relational databases, dimensional data modeling, and cloud-native solutions.
• Familiarity with data engineering tools such as Apache Airflow, Prefect, and cloud storage platforms.
• Excellent problem-solving skills, with the ability to navigate complex technical challenges.
  
  

Interested in hearing more? Easy Apply now by clicking the "Easy Apply" button.

Want to learn more about this role and Jobot?

Click our Jobot logo and follow our LinkedIn page!