What are the responsibilities and job description for the Data Engineer position at EDI Staffing?
SUMMARY
The Data Engineer is responsible for designing, developing, and maintaining data pipelines for data consumption, analytics, and AI. This includes overseeing the data lifecycle, from ingestion to quality assurance. The Data Engineer is a member of a product team and will collaborate with cross functional teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering data-driven decision making across the organization
The Products and Services department is a data and technology organization responsible for leveraging data, artificial intelligence and cloud technologies to enable the organization to fulfill its mission to protect and strengthen the municipal bond market, enabling access to capital, economic growth, and societal progress in tens of thousands of communities across the country.
ESSENTIAL DUTIES AND RESPONSIBILITIES
- Maintain optimal data pipeline architecture.
- ssemble structured, semi-structure and unstructured data sets to meet functional / non-functional business requirements.
- Identify, and implement internal process improvements : automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. DataOps and / or DevOps.
- Contribute to the infrastructure required for optimal data extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS 'big data' technologies.
- Utilize analytics tools that utilize the data pipeline to provide actionable insights into market insights and transparency, operational efficiency and other key business performance metrics.
- Work with stakeholders including the Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
- Keep data secure across regional boundaries through multiple data centers and AWS regions.
- Create data tools for analytics and data scientist team members that assist in building and optimizing data products.
- Support and deploy machine learning models to production.
- Ensure data quality and consistency across various sources.
- Other duties as assigned.
EDUCATION / QUALIFICATIONS
