What are the responsibilities and job description for the Sr. Data Engineer position at Integrated Resources?
The Sr. data engineer will play a pivotal role in designing, building and operationalizing data pipelines that feed the enterprise data ecosystem for a variety of multi-faceted analytics created by data consumers. The data engineer needs to guarantee compliance with data governance and data security requirements while creating, improving and operationalizing integrated and reusable data pipelines. The data engineer will be measured on their ability to create repeatable, efficient, high-quality code, automation-friendly processes, and delivering work products quickly.
Responsibilities :
- Lead construction, maintenance, & optimization of data pipelines.
- Drive Automation through effective metadata management, using innovative and modern tools, techniques, and architectures to improve productivity.
- Lead renovation of the data management infrastructure to drive automation in data integration and management.
- Collaborate with multiple Analytics Center of Excellence teams and EIM team in refining their data requirements for various DnA initiatives and data consumption requirements.
- Identify gaps in new data initiatives and how to address new data requirements.
- Transfer knowledge of data and / or domain understanding in addressing new data requirements.
- Propose appropriate (and innovative) data ingestion, preparation, integration and operationalization techniques to optimize data requirements.
- Work with data governance teams to ensure data scientists and consumers use corresponding data responsibly through data governance and compliance initiatives.
- Design and implement data quality monitoring systems, which includes source-to-target data validations as well as anomaly detection
Requirements :
Preferred skills :
