Data Engineer

Position : Data Engineer

Location : Pittsburgh PA (W2 ROLE)

Experience : - 9 years

Skillsets : Data Bricks Development, Informatica Data Quality, Informatica Development, Azure Data Factory

• Over 8 years of experience in data engineering, with expertise in Azure Databricks, MSSQL, LakeFlow, Python and supporting Azure technology.
• Design, build, test, and maintain highly scalable data management systems using Azure Databricks.
• Create data processing pipelines utilizing Databricks Notebooks, Spark SQL.
• Integrate Azure Databricks with other Azure services like Azure Data Lake Storage, Azure SQL Data Warehouse.
• Design and implement robust ETL pipelines using Databricks, ensuring data quality and integrity.
• Design and implement effective data models, schemas and data governance using the Databricks environment.
• Develop and optimize PySpark / Python code for data processing tasks.
• Assist stakeholders with data-related technical issues and support their data infrastructure needs.
• Develop and maintain documentation for data pipeline architecture, development processes, and data governance.
• Data Warehousing : In-depth knowledge of data warehousing concepts, architecture, and implementation, including experience with various data warehouse platforms.
• Data Quality implement data quality rules using Databricks and external platforms like IDQ.
• Extremely strong organizational and analytical skills with strong attention to detail
• Strong track record of excellent results delivered to internal and external clients.
• Excellent problem-solving skills, with ability to work independently or as part of team.
• Strong communication and interpersonal skills, with ability to effectively engage with both technical and non-technical stakeholders.
• Able to work independently without the needs for close supervision and collaboratively as part of cross-team efforts.