What are the responsibilities and job description for the Data Engineer. position at VeriiPro?
Responsibilities
- Build, maintain, and optimize data pipelines using Apache Spark.
- Consult and mentor other data professionals on best practices and standards.
- Define and document the company's data assets in collaboration with stakeholders.
- Migrate data platform to Chase's environment.
- Partner with business analysts and solutions architects to develop technical architectures for enterprise projects.
- Develop libraries to standardize data processing methods.
- Foster a culture of continuous learning and knowledge sharing within the team.
- Demonstrate expertise in AWS tools (e.g., EMR, Glue), including their advantages and limitations.
- Implement automation for applicable processes to enhance efficiency.
Requirements
- 8 years of experience in a data engineering role.
- Proficient in Python (or similar) and SQL.
- Strong experience in building data pipelines with Apache Spark.
- Excellent verbal and written communication skills.
- Strong analytical and problem-solving abilities.
- Familiarity with relational databases, NoSQL databases, and cloud object storage.
- Proven experience in building data processing infrastructure on AWS.
- Bachelor's Degree in Computer Science, Programming, or a related field preferred.
