What are the responsibilities and job description for the Data Engineer IV (Senior level) position at MAPFRE?
Job Summary
- Augment and maintain the existing repositories and data structures within AWS (used to process and store large amounts of data from unrelated sources)
- Experience with several formats and means for data ingestion. Including, data types (structured, semi-structured, and unstructured), and sources (on premise, and in the cloud) using the most appropriate techniques in each case
- Continue to expand and enhance the model, utilizing best practices, in regards to the organization of data and the various relationships
- Optimize existing and future models for fast and scalable queries (while maintaining performance and related price thresholds)
- Work with the team to define, construct, and maintain self-service dashboards for the Business and Advanced Analytics teams within PowerBI
- Implement scalable and flexible, high performance data pipelines for AWS to support analytics
- Develop and maintain data maps and their relationships
- Generate associated technical documentation including follow-up reports
- Work with Data Governance to implement quality rules and data governance measures (data dictionary, metadata, traceability, ...)
- Propose improvements and actions based on provided results
- Communicate results effectively with required teams
Knowledge, Skills and Abilities:
- Bachelor's Degree with 6 years of Data Engineering experience.
- Advanced knowledge and experience using Python, Airflow, Spark, AWS, and Snowflake
- Database architectures: SQL, NoSQL, graph databases
- CI/CD and Orchestration: Jira, Jenkins, Bit Bucket, Terraform, and Airflow
- Past experience with data modeling tools, ETL tools (e.g. Informatica Power Center)
- Computer languages, data query and transformation tools: AWS Athena, Jupyter notebooks, Spark, Pyspark, Python, and EMR Studio
- Algorithm analysis (for working with our Data Scientists)
- Understanding of multidimensional modeling for quantitative and fact related data storage
- OS: Linux, and MS Windows
- Code IDE: Microsoft Visual Code, Jupyter notebooks
- Artificial intelligence, machine learning, and deep learning are a plus
Data Engineer
Comcast -
Worcester, MA
Data Engineer
Rose International -
Worcester, MA
Lead Data Path Engineer
Dell Technologies -
Hopkinton, MA
