What are the responsibilities and job description for the Lead Data Engineer position at Brooksource?
Lead Data Engineer
Contract
On-site 3 days a week
Bloomfield, CT
Overview
Brooksource is looking for an experienced Data Engineer to lead the design, development, and implementation of cutting-edge data solutions. This is a high-impact role for a skilled engineer who thrives in a fast-paced environment and is passionate about building scalable, high-performance platforms using modern data technologies.
As a Lead Data Engineer, you will spearhead initiatives that power real-time analytics and large-scale data processing. You’ll collaborate with cross-functional teams – product managers, architects, and engineers – to drive innovative solutions and ensure the delivery of robust, secure, and maintainable data products.
Key Responsibilities
- Lead architecture and development of scalable data solutions using AWS services
- Implement and manage Open Search engine for optimized data retrieval and search experiences
- Leverage Mustache for templating and dynamic query generation
- Develop and maintain high-performance data pipelines using Apache Spark
- Write efficient, clean code in Scala and/or Python
- Utilize Databricks for advanced data processing, transformation, and analytics workflows
- Collaborate across teams to translate business requirements into technical solutions
- Mentor junior engineers, promote best practices, and contribute to a culture of engineering excellence
- Drive quality through code reviews, testing, and continuous integration/deployment
Qualifications
- 8 years of experience in a Data Engineer role
- Advanced experience with Databricks for unified analytics and scalable compute
- Proficient in Mustache for logic-less templating
- Strong command of functional or object-oriented programming in Scala and Python
- Extensive experience with Apache Spark for distributed data processing
- Proficient in Open Search for search and log analytics in cloud environments
- Strong understanding of data architecture, security, and performance optimization
- Excellent communication and leadership skills
- Ability to work independently and as part of a team
- Ability to work in a fast-paced, dynamic environment
Eight Eleven Group (Brooksource) provides equal employment opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, national origin, age, sex, citizenship, disability, genetic information, sexual orientation, gender identity, marital status, amnesty or status as a covered veteran in accordance with applicable federal, state, and local laws.
