What are the responsibilities and job description for the Azure Data Engineer (Local Applicants ONLY) position at A.C.Coy Company?
Applicants MUST be local to Pittsburgh, PA
No 3rd Parties/Sub Vendors
Location: Hybrid - 3 days/week on-site in Pittsburgh, PA (candidates must be local to Pittsburgh, PA)
Job Type: Full Time/Permanent
Work Authorization: U.S. Citizens Only
Overview:
The A.C.Coy has an immediate need for an Azure Data Engineer for a full-time opportunity.
Responsibilities:
- Automate data pipeline monitoring, error handling, and performance tuning to ensure data integrity and availability
- Collaborating with the Digital Data Solutions Team or external consultants as needed
- Design, develop, and implement scalable and high-performance data solutions on the Azure platform, specifically architecting data pipelines and ETL/ELT processes utilizing Azure Data Factory and Azure Databricks to extract, transform, and load data from various sources into Azure data storage solutions
- Optimize data storage and retrieval mechanisms to ensure efficient processing of large datasets, particularly focusing on the capabilities of Databricks
- Develop and maintain data models and schemas to support data analysis and reporting requirements
- Implement data security and privacy measures to protect sensitive data
- Develop and maintain data ingestion processes to acquire data from various sources, including APIs, databases, and streaming platforms
- Implement data quality checks and validation procedures to ensure the accuracy and integrity of data
- Design and implement data transformation and enrichment processes using Azure Databricks, Azure Data Factory, and other relevant tools
- Document technical design, deployment processes, and best practices for data engineering solutions
- Implement data partitioning, indexing, and optimization techniques to improve query performance
- Collaborate with data scientists and analysts to operationalize and deploy machine learning models on Azure, utilizing Databricks for model training and deployment
- Configure and manage data storage options, including Azure Blob Storage, Azure Data Lake Storage, and Databricks LakeHouse
- Collaborate with DevOps teams to deploy and manage data engineering solutions in Azure
Education: Bachelor’s degree in Computer Science, Computer Engineering, or another technical field
Qualifications:
- 3-5 years' experience with Azure data services, including Azure Data Lake Storage, Azure SQL Database, Azure Data Factory, and Azure Databricks. Understanding of Azure Synapse Data Warehouse preferred
- Strong experience in data engineering and architecture, with a focus on Azure cloud technologies
- Proficiency in data-centric programming languages, including SQL, Python, and PowerShell
- Knowledge of data integration techniques and tools, such as ETL/ELT processes, data pipelines, and data warehousing concepts
- Familiarity with data modeling and schema design principles
- Understanding of data security and privacy regulations, including GDPR and CCPA
- Azure certifications, such as Microsoft Certified: Enterprise Data Engineer Associate, are a plus
Salary : $100,000 - $150,000
