What are the responsibilities and job description for the Senior Data Engineer position at Cynet Systems?
Job Details
We are looking for Senior Data Engineer for our client in Minneapolis, MN
Job Title: Senior Data Engineer
Job Type: Contract
Job Description:
Job Description:
Responsibilities:
- Design, develop, and optimize ETL pipelines using PySpark, Databricks, and Azure Data Services.
- Develop and deploy scalable data models, transformations, and solutions using Python and SQL.
- Manage and orchestrate workflows using Azure Logic Apps and Azure CLI.
- Implement best practices for data governance, security, and monitoring.
- Collaborate with data scientists, business analysts, and other developers to understand and meet data processing needs.
- Ensure high performance, reliability, and scalability of data platforms and solutions.
- Automate data pipelines, deployments, and processes to increase system efficiency.
- Troubleshoot and resolve issues with data integrity, performance, and availability.
- Stay up-to-date with the latest tools, frameworks, and practices in data engineering and cloud services.
- 5 years of experience in software or data engineering.
- Strong programming skills in Python and distributed data processing with PySpark.
- Proficiency in SQL for data manipulation, optimization, and analysis.
- Hands-on experience with Azure services, including:
- Logic Apps.
- Azure CLI.
- Azure Storage (Blob, Data Lake).
- Expertise in Databricks for data integration, transformation, and analysis.
- Experience in designing and managing ETL pipelines and data warehouses.
- Knowledge of cloud infrastructure and containerization concepts.
- Strong problem-solving and debugging skills.
- Ability to work in an agile environment with cross-functional teams.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
Senior Cloud Engineer
NTT DATA Group Corporation -
Prairie, MN
Senior Data Manager
Professional Data Analysts, Inc. -
Lakeville, MN
Senior Data Engineer
3M -
Saint Paul, MN
