Senior Data Engineer

*** W2 Contract Only – No C2C – No 3rd Parties ***

*** Candidates MUST be located within specified areas ***

Position: Senior Software Engineer – Data Engineering

Location: Greenwood Village, CO; Chicago, IL or Minneapolis, MN

Setting: Colorado-based candidates will need to be on-site as needed, typically 1 day/week; Illinois and Minnesota-based candidates will work remotely

Contract Length: 6 months, contract-to-hire

Overview: As a Senior Software Engineer – Data Engineering, you will play a pivotal role in building and scaling our data platform on AWS. You will work closely with data architects, analysts, and engineers to design, develop, and maintain robust data pipelines that extract, transform, and load (ETL) data from various sources into our data lake/warehouse.

Top Skills needed: 7 years of experience as a Data Engineer (or 5 years of experience in software engineering or DevOps) with a strong focus on cloud-based environments and problem-solving, analytical, communication, and collaboration skills. Certifications in AWS cloud architecture or data engineering are desired.

Key Responsibilities:

Data Pipelines and Automation:

• Design, develop, and maintain scalable data pipelines and ETL solutions using AWS services like AWS Glue, AWS Lambda, AWS EKS, Amazon S3, Amazon Redshift, and Amazon EMR.
• Develop and maintain data infrastructure, including data lakes, data warehouses, and data marts. Integrate data products with the broader data platform, including data lakes, data warehouses, and analytics tools.
• Design and develop data-sharing mechanisms and APIs to enable data exchange between teams.
• Develop, configure, and manage CI/CD pipelines using tools such as Github Actions.
• Automate and streamline deployment and testing processes to enhance solution delivery speed and quality.
• Implement pipeline monitoring and alerting using AWS CloudWatch, Prometheus, and other monitoring tools.

Data Integration:

• Collaborate with data engineers to design and implement data integration strategies to extract, transform, and load (ETL) data from various sources into the cloud data platform.
• Collaborate with data architects and analysts to understand their data needs and requirements.
• Assist in migrating existing data warehouses to AWS, including planning, execution, and optimization.
• Optimize data pipelines for performance, scalability, and cost-efficiency.
• Implement best practices for indexing, partitioning, and clustering to improve query performance.
• Troubleshoot and resolve data- and ETL-related issues.

Security and Governance:

• Implement robust security measures to protect sensitive data, including encryption, access controls, and data masking.
• Develop and enforce data governance policies to ensure data quality, consistency, and compliance with regulations.
• Implement data lineage and provenance to track data movement and transformations.
• Technical Leadership:
• Provide technical guidance and mentorship to data engineers, analysts, and scientists.
• Collaborate with business stakeholders to understand data requirements and translate them into technical solutions.

Required Skills and Experience:

• Bachelor’s degree in Computer Science, Software Engineering, Data Engineering, or a related field.
• 7 years of experience as a Data Engineer (or 5 years of experience in software engineering or DevOps) with a strong focus on cloud-based environments.
• Strong proficiency in Terraform, SQL and Python.
• Extensive experience with AWS Cloud services such as AWS Glue, AWS Lambda, EKS, Amazon S3, Amazon Redshift, Amazon EMR, and AWS Kinesis.
• Knowledge of cloud-native technologies and serverless computing.
• Knowledge of data warehousing/data lake concepts and best practices.
• Understanding of data modeling and data architecture.
• Strong problem-solving and analytical skills.
• Strong communication and collaboration skills.

Desired Skills:

• Hands-on experience in developing data ETL tools, frameworks, or data APIs.
• Experience with data visualization tools (e.g., Quick Sight, Tableau, Power BI).
• Experience with machine learning and artificial intelligence.
• Certifications in AWS data architecture or cloud architecture.
• Experience with event-driven architectures and serverless computing.
• Knowledge of domain-driven design and microservices patterns.
• Experience with data governance and data quality frameworks.
• Certifications related to AWS or data engineering.