What are the responsibilities and job description for the Lead Data Engineer position at TrueSkilla?
Job Title: Lead Data Engineer
Location: Hartford, CT (2 days a week)
Duration: 12 months
Type: Only W2
Minimum experience - 8 years
About the Role
We are looking for a Lead Data Engineer to design and build data pipelines, focusing on Kafka streaming and GCP ingestion. This role involves leading a team, building scalable solutions, and ensuring data reliability and performance.
Key Responsibilities
- Build Data Pipelines:
- Create streaming data pipelines using Kafka and tools like Kafka Connect.
- Develop ingestion frameworks using Google Cloud Platform (GCP) tools such as BigQuery, Dataflow, and Pub/Sub.
- Team Leadership:
- Guide and mentor a team of data engineers.
- Collaborate with other teams to understand and deliver on data requirements.
- Optimize Performance:
- Ensure data pipelines are reliable, scalable, and cost-effective.
- Troubleshoot issues and improve system performance.
- Stay Updated:
- Keep up with new tools and trends to improve systems and processes.
Required Skills
- Apache Kafka: Experience with Kafka Streams, Connect, and schema registries.
- Google Cloud Platform (GCP): Hands-on with BigQuery, Dataflow, Pub/Sub, and Cloud Storage.
- Programming: Proficiency in Python,.
- Data Experience: Strong understanding of ETL pipelines, data modeling, and stream processing.
- Leadership: Proven ability to lead and mentor a team.
Salary : $50 - $53
Lead Data Engineer - AWS/Full Stack
The Hartford -
Hartford, CT
Data Engineer
Robert Half -
Simsbury, CT
Data Engineer
Twenty Ideas -
Hartford, CT
