Data Engineering Team Lead

Job Overview

We are seeking an experienced Data Engineering Team Lead to lead our data integration and pipeline initiatives within our Data Management Platform. This role requires a subject matter expert in Snowflake and dbt with a proven track record of developing efficient data models, building robust integrations, and managing end-to-end data pipelines. The ideal candidate will possess a blend of technical expertise and strong business analysis skills to translate abstract requirements into clear, actionable tasks for a team of data engineers. This position demands a leader who can organize work effectively to meet project milestones and inspire a team to excel in a dynamic, data-driven environment.

In this role, you will dedicate 80% of your time to technical leadership and solution architecture, and 20% to operational management and stakeholder engagement.

Key Responsibilities

80% Technical Leadership and Solution Architecture

• Data Integration & Pipeline Development:
• Leverage advanced knowledge in Snowflake to design and implement high-performance data integrations and data model pipelines that serve as the backbone for our data management initiatives.
• Utilize dbt for efficient data transformation and modeling.
• Solution Guidance and Debugging:
• Lead the troubleshooting and optimization of complex data engineering solutions.
• Provide technical guidance to team members to overcome challenges.
• Technology Selection and Best Practices:
• Evaluate and select appropriate technologies to solve data problems.
• Establish and enforce industry best practices and coding standards.
• CI/CD Pipeline Development:
• Design, build, and maintain robust CI/CD pipelines for data applications.
• Automate testing and deployment processes to enhance efficiency.
• Code Reviews:
• Conduct thorough code reviews to ensure code quality and compliance with standards.
• Mentor team members on coding best practices and efficient techniques.
• Solution Planning and Architecture:
• Plan and design scalable, reliable architectures for data products.
• Collaborate with cross-functional teams to align solutions with business objectives.
• Technical Support and Mentoring:
• Answer complex technical questions and provide expertise in SQL and Python.
• Mentor and develop the technical skills of team members.
• Promote a culture of continuous learning and improvement.

20% Operational Management and Stakeholder Engagement

• Business Analysis & Requirements Translation:
• Collaborate closely with stakeholders to understand high-level objectives.
• Distill abstract requirements and transform them into actionable work elements for the engineering team.
• Project Planning & Execution:
• Organize and structure work into realistic, achievable tasks and timelines to ensure deliverables are met consistently.
• Establish clear goals and benchmarks that enable team productivity and on-time project delivery.
• Manage the Jira board to track project progress, tasks, and deadlines.
• Facilitate daily stand-up meetings and ensure effective team communication.
• Team Leadership & Mentorship:
• Lead a team of data engineers with a focus on professional development and operational excellence.
• Provide mentorship, foster a collaborative environment, and inspire the team to excel.
• Strategic Collaboration:
• Partner with cross-functional teams, including Product, Security, and Operations, to align data engineering efforts with broader company goals.
• Ensure the seamless integration of our platform with existing products and systems.
• Financial Management:
• Oversee finances and administrative accounts for tools like Snowflake and dbt.
• Manage budgets, licensing, subscriptions, and vendor relationships.
• Stakeholder Engagement:
• Serve as the first point of contact in stakeholder meetings.
• Translate technical concepts for non-technical stakeholders and gather requirements.
• Quality & Compliance:
• Ensure that all data engineering efforts adhere to the latest industry standards for data quality, security, and privacy.
• Regularly review and audit processes to uphold high standards of data integrity and reliability.
• Administrative Oversight:
• Ensure compliance with organizational policies and procedures.
• Handle administrative tasks related to vendor accounts and services.

Qualifications

• Education:
• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
• Experience:
• 5 years of experience in data engineering or related technical roles.
• 2 years in a leadership or managerial position.
• Proven track record of developing efficient data models, building robust integrations, and managing end-to-end data pipelines.
• Technical Expertise:
• Extensive experience with Snowflake, including advanced data modeling, pipeline creation, and integration design.
• Proficiency in dbt for data transformation and modeling.
• Proficiency in SQL and Python is required.
• Experience with CI/CD tools and building deployment pipelines.
• Strong understanding of data architecture and data warehousing concepts.
• Analytical Mindset:
• Strong business analysis skills with the ability to break down abstract requirements into concrete, executable tasks.
• Project Management:
• Demonstrated experience in structuring and prioritizing tasks to meet project milestones.
• Familiarity with project management tools like Jira.
• Tools and Technologies:
• Experience with data platforms such as Snowflake and dbt.
• Familiarity with version control systems like Git.

Skills and Competencies

• Leadership Skills:
• Strong ability to lead, mentor, and develop technical teams.
• Excellent problem-solving and decision-making capabilities.
• Ability to inspire and motivate a team in a dynamic, data-driven environment.
• Technical Acumen:
• Deep understanding of data engineering principles and best practices.
• Ability to architect complex data solutions and oversee their implementation.
• Communication Skills:
• Excellent verbal and written communication skills.
• Ability to effectively communicate with both technical and non-technical stakeholders.
• Organizational Skills:
• Strong project management skills with attention to detail.
• Ability to manage multiple priorities and projects simultaneously.
• Strategic Collaboration:
• Proven ability to collaborate with cross-functional teams to achieve strategic objectives.
• Quality & Compliance:
• Commitment to maintaining high standards of data integrity, security, and privacy.
• Financial Management:
• Experience in managing budgets and vendor relationships.
• Ability to optimize costs related to data platforms and tools.