What are the responsibilities and job description for the Azure Data Engineer position at Compunnel?
Project Summary :
This project involves building AC360 data pipelines in Azure using Spark technologies. As part of this initiative, the development of Zafin data pipelines in Azure is also included.
Key Responsibilities :
1. Identify Requirements and Develop Solutions
Identify requirements for RMS replacement and develop solutions in Azure.
Re-engineer solutions to retrofit the rules with AC360 in Azure.
2. Data Pipeline Management
Register data pipelines in Azure, onboard datasets, and enable data processing.
Build Airflow DAGs and ensure scheduling is integrated.
3. DevOps and Production Management
Apply a DevOps mindset, take ownership of production success, and optimize operations via automation, active alerting, and self-healing.
Lead the resolution of production issues to ensure high availability and performance.
4. Code Development and Review
Develop high-quality code and define best engineering practices.
Perform peer code reviews to ensure successful deliverables with engineering excellence.
5. Documentation
Document engineering artifacts such as technical design documents, flowcharts, system context diagrams, and code comments.
6. Collaboration
Collaborate cross-functionally with product owners, data scientists, business users, project managers, and other engineers to achieve elegant solutions.
Work with other engineers on the team to elevate technology and consistently apply best practices.
7. End-to-End Development and Testing
Execute true end-to-end development and testing of software products in an agile lifecycle.
Partner with key technical and business stakeholders to ensure the quality of the final product through manual and automated tests.
Support the new software as it is moved to production.
Required Skills / Technologies :
Mainframe COBOL : Good knowledge to understand the rules in Strata rules engine.
Spark-Scala or Spark-SQL, Spark-Py : Minimum 5 years of experience.
Keep a pulse on the job market with advanced job matching technology.
If your compensation planning software is too rigid to deploy winning incentive strategies, it’s time to find an adaptable solution.
Compensation Planning
Enhance your organization's compensation strategy with salary data sets that HR and team managers can use to pay your staff right.
Surveys & Data Sets
What is the career path for a Azure Data Engineer?
Sign up to receive alerts about other jobs on the Azure Data Engineer career path by checking the boxes next to the positions that interest you.
