What are the responsibilities and job description for the Azure Data Factory Engineer/Developer with PostgreSQL position at SANS?
Job Details
Job Title: Azure Data Factory Developer with PostgreSQL Location : Weehawken, NJ - 5 Day onsite mandatory
We are seeking a skilled Azure Data Factory Engineer/Developer with strong PostgreSQL knowledge to design, develop, and maintain data pipelines for extracting, transforming, and loading (ETL) data from various sources, including PostgreSQL databases, into Azure data platforms
- Design, Develop, and Maintain ADF Pipelines:
- Design, develop, and maintain robust and efficient ADF pipelines for data ingestion, transformation, and loading from PostgreSQL databases and other sources.
- Optimize ADF pipelines for performance, scalability, and reliability.
- Develop and implement data quality checks and validation rules within ADF pipelines.
- PostgreSQL Expertise:
- Proficiently use SQL to query, manipulate, and extract data from PostgreSQL databases.
- Understand PostgreSQL database architecture, performance tuning, and best practices.
- Troubleshoot and resolve issues related to PostgreSQL connectivity, performance, and data integrity.
- Data Integration and Transformation:
- Design and implement data transformation logic within ADF pipelines, leveraging various ADF activities and connectors.
- Develop and maintain data mappings and transformations to ensure data consistency and quality.
- Collaborate with data engineers and business stakeholders to understand data requirements and implement solutions.
- Monitoring and Optimization:
- Monitor the performance and health of ADF pipelines and PostgreSQL databases.
- Identify and resolve performance bottlenecks and data quality issues.
- Implement and maintain monitoring and alerting mechanisms for ADF pipelines and PostgreSQL databases.
- Skills:
- Azure Data Factory (ADF): Strong experience in designing, developing, and maintaining ADF pipelines.
- PostgreSQL: Extensive knowledge of PostgreSQL database architecture, SQL, and performance tuning.
- SQL: Proficient in writing and optimizing SQL queries.
- Data Modeling: Experience in designing and implementing data models.
- ETL/Data Integration: Experience with ETL processes and data integration tools.
- Cloud Computing: Familiarity with Azure cloud services and technologies.
- Problem-Solving: Strong analytical and problem-solving skills.
- Communication: Excellent communication and collaboration skills.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
