Data Engineer with Python

Education: Masters preferred in Engineering. Minimum Bachelor s degree required

Required skills: AWS, DATA ANALYTICS, GIT, PYTHON, SQL

Additional skills:

Minimum degree required: Bachelor's Degree

• 8 years of diverse Technology experience with a minimum of 5 years of experience in Software development
• Bachelor s in computer science or equivalent trade
• Experience with AWS capabilities Glue, DynamoDB, Lambda, Redshift, elastic search
• Experience with API
• Software development experience with Python, PySpark and Apache Spark Proficiency in SQL, relational and non-relational databases, query optimization and data modelling
• Strong knowledge of Data Integration (e.g. Streaming, Batch, Error and Replay) and data analysis techniques
• Experience with GitHub, Jenkins, and Terraform
• Experience with Teradata (Vantage) or any RDBMS system/ ETL Tools
• Good experience on designing and developing data pipelines for data ingestion and transformation using Spark
• Excellent in trouble shooting the performance and data skew issues
• Working knowledge on the implementation of data lake ETL using AWS glue, Databricks etc.
• Experience with large scale distributed relational and NoSQL database systems
• Excellent communication skills
• Provide recommendations and design optimal configurations for large-scale deployments
• Superb analytical and technical skills coupled with the ability to drive project deliverables
• Knowledge in computer science, a problem-solving mentality, and a desire to learn
• Comfortable working within a technical capacity across many of the sectors including core development on platform, data processing, and report generation