What are the responsibilities and job description for the Lead Data Engineer position at Sparity?
Required Skills:
- Extensive experience with data pipeline creation and optimization
- Proficiency in Databricks, Azure, and AWS environments
- Strong knowledge of DevOps practices and principles
- Expertise in code repository management, branching strategies, and versioning
- Advanced troubleshooting skills for pipeline, application, and environment issues
- Solution design capabilities for complex data engineering problems
- Familiarity with release processes and deployment strategies
Required Experience:
- 5 years of experience in data engineering or related field
- Proven track record of leading data engineering teams and projects
Additional Requirements:
- Excellent communication skills, both written and verbal
- Ability to coordinate with cross-functional teams
- Experience in mentoring and guiding junior engineers
- Strong problem-solving and analytical skills
Responsibilities:
- Lead the design and implementation of data pipelines using best practices and available templates
- Oversee and enforce policies for code repositories, branch management, and versioning
- Develop and implement release processes and deployment strategies
- Lead troubleshooting efforts for complex pipeline, application, and environment issues
- Collaborate with the System Team to enhance pipeline capabilities and resolve critical issues
- Provide technical leadership and mentorship to the data engineering team
Salary : $98,500 - $129,800