What are the responsibilities and job description for the Azure Data Engineer Lead position at Aptino?
Job Details
Job Description:
We are seeking a highly skilled Senior Data Engineer with extensive experience in Python, PySpark, SQL, and Microsoft Azure Cloud Services. The ideal candidate will have a strong background in data warehousing, big data technologies, and cloud data engineering, along with a proven ability to lead development teams and drive high-quality project deliverables.
Key Responsibilities:
Develop and maintain scalable data pipelines using Python, PySpark, and SQL.
Design, implement, and optimize data warehouses and data lake architectures on Azure Cloud.
Build and manage ETL/ELT workflows using tools like Azure Data Factory (ADF), Informatica, or Talend.
Work with Azure Synapse, Azure SQL, Amazon Redshift, Snowflake, or Google BigQuery to support data analytics and reporting.
Develop dimensional data models and schema designs for optimized query performance.
Leverage Databricks (preferable) or similar platforms for big data processing.
Implement best practices in Medallion Architecture for data organization and management.
Utilize Azure Data Lake Storage, Cosmos DB, and Blob Storage for efficient data storage solutions.
Collaborate with cross-functional teams to solve complex data challenges and drive insights.
Provide technical leadership, mentor junior engineers, and ensure project success.
Required Skills & Experience:
12 years of experience in SQL, schema design, and dimensional data modeling.
5 years of experience in modern data engineering, data warehousing, and Azure cloud technologies.
Strong hands-on expertise in Python, PySpark, and SQL.
Proficiency in Azure Cloud Services, including Azure Synapse, Azure SQL Database, Azure Data Factory, and Azure Data Lake Storage.
Experience with big data platforms (Databricks preferred).
Solid understanding of ETL/ELT processes and tools such as ADF, Informatica, Talend.
Excellent problem-solving, analytical skills, and leadership abilities.
Preferred Qualifications:
Experience implementing Medallion Architecture for data organization.
Familiarity with Azure Cosmos DB and Blob Storage.
Strong ability to lead development teams and drive project execution.
