What are the responsibilities and job description for the Tech Lead - Databricks Data Engineer position at PamTen?
Primary skills : Advanced SQL, Azure Databricks, PySpark Azure Data Factory, Azure Datalake.
Secondary skills : Azure SQL, Azure Synapse.
About the job
We are seeking a Tech Lead - Databricks Data Engineer with experience in designing and developing data pipelines using Azure Databricks, Data Factory, and Datalake. The role involves managing large volumes of data, building complex ETL solutions, and working closely with business teams to deliver robust data transformations and analytics solutions.
Responsibilities :
- Design and develop ETL pipelines using ADF for data ingestion and transformation.
- Collaborate with Azure stack modules like Data Lakes and SQL DW to build robust data solutions.
- Write SQL, Python, and PySpark code for efficient data processing and transformation.
- Understand and translate business requirements into technical designs.
- Develop mapping documents and transformation rules as per project scope.
- Communicate project status with stakeholders, ensuring smooth project execution.
Requirements Must have :
