What are the responsibilities and job description for the Principle Data Engineer position at EXL Service?
Required Skills:
Python, Pyspark/Spark, SQL, Data Lake (ON Prem or cloud) AWS, Snowflake.
Job Summary:
The Data Engineer’s role is to play a lead role in developing a high-performance data platform, integrating data from a variety of internal and external sources, in order to support data and analytics activities. This is a technical role that involves defining changes to the warehouse data model and building scalable and efficient processes to populate or modify warehouse data. The successful candidate will have hands-on data processing and data modeling experience in cloud and on-prem environments.
Responsibilities:
Be a technical lead in the development of high-volume platforms to drive decisions and have a tangible beneficial impact on our clients and on business results.
Design and implement efficient data pipelines (ETLs) in order to integrate data from a variety of sources into Data Warehouse.
Design and implement data model changes that align with warehouse standards.
Design and implement backfill or other warehouse data management processes.
Develop and execute testing strategies to ensure high quality warehouse data.
Provide documentation, training, and consulting for data warehouse users.
Perform requirement and data analysis in order to support warehouse project definition.
Provide input and feedback to support continuous improvement in team processes.
Experience in working both, On-Prem and Cloud (AWS preferred).
Responsible for leading the team onsite and off shore with technical leadership and guidance.
Qualifications:
5 years in a Data Engineering role
7 years hands on experience with SQL:
Ability to write/ interpret SQL and Complex Joins/ Queries
Execution plan and SQL optimization (Oracle SQL Profiler)
o 3 years coding experience (Python and/or PySpark).
o 3 years hands on experience with big data and cloud technologies (Snowflake, EMR, Redshift, or similar technologies) is highly preferred
Schema design and architecture on snowflake
Architecture and design experience with AWS cloud
AWS services expertise: S3, RDS, EC2, ETL services (Glue, Kinesis)
Consumption layer design experience for reporting and dashboard
Expert level understanding and experience of ETL fundamentals and building efficient data pipelines.
3 years at least - Enterprise GitHub – branch, release, DevOps, CI/CD pipeline.
Team player, Strong communication and collaboration skills.
Experience with Agile methodologies.
Master’s Degree (or a B.S. degree with relevant industry experience) in math, statistics, computer science, or equivalent technical field.
