What are the responsibilities and job description for the Cloud Data Engineering Lead position at Jobleads-US?
**Job Title:** Cloud Data Engineering Lead
About Jobleads-US:
We are seeking an experienced Azure Databricks Architect to lead the design, implementation, and optimization of our enterprise-wide data platform.
Responsibilities:
- Create a scalable, secure, and high-performance data ecosystem that enables advanced analytics, machine learning, and business intelligence capabilities.
- Align solutions following operating procedures and Architecture principles and work closely with internal technical teams.
- Design and architect comprehensive data solutions using Azure Databricks, Azure Data Lake, Azure Event Hub, and related Azure data services.
Requirements:
- Develop and implement data Lakehouse architecture patterns that support both batch and real-time data processing.
- Create robust data ingestion pipelines leveraging Azure Databricks, Azure Data Factory, and other ETL technologies.
- Establish best practices for data governance, security, and compliance across the data platform.
- Create and maintain infrastructure-as-code (IaC) templates for reproducible and scalable data platform deployments.
- Provide architectural guidance and mentorship to data engineering and analytics teams.
- Ensure alignment of data platform architecture with overall enterprise architecture and business objectives.
- Foster collaboration with cross-functional teams including Development, Operations, Compliance, Security, etc.
- Execute end-to-end testing for various non-functional requirements.
- Resolve design and development issues in creative and effective ways.
- Research, evaluate, and incorporate new technology and tools to enhance product development and platform operations.
