What are the responsibilities and job description for the Mid Level Data Engineer position at Compsciprep LLC?
Job Details
Junior Data Engineer
Job Description:
Cloud Infrastructure Management: Design, deploy, and manage AWS cloud environments using services such as Step Functions, EMR,EC2, S3, RDS, and Lambda.
Infrastructure as Code (IaC): Develop and maintain infrastructure automation scripts using Terraform and AWS CloudFormation.
Data Pipeline Development: Build and optimize data pipelines for ETL processes using Informatica, ensuring efficient data flow and transformation.
Version Control: Utilize GitLab and Bitbucket for source code management and collaboration with development teams.
Scripting: Write and maintain Unix shell scripts to automate tasks and improve operational efficiency.
Programming: Develop applications and data processing scripts using Java or Python to support data integration.
Database Management: Write and optimize SQL and Oracle queries for data extraction, reporting, and analysis.
Documentation: Maintain comprehensive documentation of systems, processes, and changes.
Qualifications:
AWS Knowledge: Strong knowledge of AWS services and best practices, particularly for data storage and processing.
Version Control: Experience with GitLab and Bitbucket.
Infrastructure as Code: Proficiency in Terraform and AWS CloudFormation.
Data Engineering Skills: Experience with ETL processes, and data warehousing concepts.
Programming Skills: Strong programming knowledge in Java or Python for automation and data processing tasks.