What are the responsibilities and job description for the Lead Data Engineer position at Incedo Inc.?
Lead Data Engineer
Role Overview :
We are seeking a Lead Data Engineer with expertise in data modeling , Data Vault architecture , and cloud data warehouse (CDW) architecture utilizing the Kimball model . The ideal candidate will have hands-on experience with data fabric technologies (e.g., Databricks), Spark , and Python on Azure Cloud . This role involves designing, implementing, and optimizing data pipelines and data platforms to support business intelligence and advanced analytics.
Key Responsibilities :
- Design and implement scalable and efficient data models, including Data Vault and Kimball dimensional modeling for cloud-based data warehouses.
- Architect and manage Azure Cloud -based data infrastructure, ensuring scalability, security, and performance.
- Develop and optimize data pipelines using Databricks , Spark , and other data fabric tools to process large volumes of data.
- Collaborate with data scientists, analysts, and business stakeholders to understand data needs and deliver high-quality solutions.
- Implement best practices for data engineering, including data quality, governance, and lifecycle management.
- Build and maintain infrastructure-as-code using Terraform or other DevOps tools.
- Create reusable data engineering components for ingestion, transformation, and orchestration.
- Troubleshoot, optimize, and maintain existing data systems to meet evolving business needs.
Required Skills and Experience :
Educational Qualifications :
