What are the responsibilities and job description for the Lead Data Engineer position at Orion Innovation?
Orion Innovation is a premier, award-winning, global business and technology services firm. Orion delivers game-changing business transformation and product development rooted in digital strategy, experience design, and engineering, with a unique combination of agility, scale, and maturity. We work with a wide range of clients across many industries including financial services, professional services, telecommunications and media, consumer products, automotive, industrial automation, professional sports and entertainment, life sciences, ecommerce, and education.
Lead Data Engineer
A large-scale project that would process 100s TB of ERP data and other relevant data. Auditors would use this processed information to opine on the client engagements and provide value-added services to their clients. The project currently uses Azure data technologies and Databricks.
About the Job: We are looking for an experienced professional with at least 6 years of hands-on experience with Databricks technology.
What Candidates Will Do
Candidate Privacy Policy
Orion Systems Integrators, LLC And Its Subsidiaries And Its Affiliates (collectively, “Orion,” “we” Or “us”) Are Committed To Protecting Your Privacy. This Candidate Privacy Policy (orioninc.com) (“Notice”) Explains
Lead Data Engineer
A large-scale project that would process 100s TB of ERP data and other relevant data. Auditors would use this processed information to opine on the client engagements and provide value-added services to their clients. The project currently uses Azure data technologies and Databricks.
About the Job: We are looking for an experienced professional with at least 6 years of hands-on experience with Databricks technology.
What Candidates Will Do
- Design, build, and manage scalable data architectures using Databricks and Azure data technologies.
- Architect and implement modern data lakehouse solutions, ensuring seamless data integration, storage, and processing.
- Collaborate with cross-functional teams to gather and translate business requirements into effective technical designs.
- Ensure data quality and governance practices are implemented throughout the data lifecycle.
- Optimize data workflows for performance, reliability, and security using Azure Synapse, Data Factory, and Databricks.
- Develop and enforce best practices in data modeling, pipeline design, and storage architecture.
- Conduct regular assessments of data systems to identify and address performance bottlenecks or areas for improvement.
- 10 Years of total IT experience in Data engineering and Data warehouse project development.
- 6 years of hands-on experience with Azure Databricks and expertise in PySpark & Python development.
- Proven expertise in designing and managing scalable data architectures.
- 6 years’ experience with Azure Synapse, Data Factory, and other Azure data technologies.
- 8 years’ experience in data modeling,
- 6 Years experience with Data pipelines design and implementation, and cloud storage architecture.
- Deep understanding of data quality and governance practices, and hands-on with Data quality and governance using Unity catalog or azure purview.
- Ability to collaborate with cross-functional teams and translate business requirements into technical designs.
- Familiarity with other modern data processing tools and technologies.
- Previous experience in a similar role working with large-scale data systems.
Candidate Privacy Policy
Orion Systems Integrators, LLC And Its Subsidiaries And Its Affiliates (collectively, “Orion,” “we” Or “us”) Are Committed To Protecting Your Privacy. This Candidate Privacy Policy (orioninc.com) (“Notice”) Explains
- What information we collect during our application and recruitment process and why we collect it;
- How we handle that information; and
- How to access and update that information.
