Data Engineer - Operations

Senior Data Engineer Group: Enterprise Technology Group

Division: ETG Architecture & Development

I. SCOPE OF JOB: Under the direction and minimal guidance of the Sr Mgr/Mgr, Data Engineering designs, develops, tests, and maintains the scripts and jobs that are required to extract, transform, clean, and move data and metadata so that it can be loaded into a data warehouse, data mart, or operational data store. Mentors less experienced developers and serves as a project technical data lead for data projects. Trains, coaches and mentors team members.

Ii. Preparation Required

• Bachelor’s degree in Computer Science, MIS or related field with 8 years of experience or Master’s with 6 years of experience or an equivalent combination of education and experience is required.
• Experience required in data warehousing or operational data stores, relational database management systems or multi-dimensional database management systems.
• Post graduate degree or coursework a plus.
  
  

Iii. Customer Focus & Impact

It is our goal to exceed our customer’s expectations for the products we market and the services we provide. This must be accomplished for the Company to be successful. Every employee must understand this goal and how their job assists the Company in its achievement. Further, the accuracy and timeliness of the information produced will have a direct impact on critical corporate decisions on products we market and customers we market to.

For This Position This Means

• Focus: To ensure accurate, timely and actionable information is produced by utilizing the appropriate data warehousing architecture and implementing appropriate data marts, as well as supporting structured/semi-structured file transformation.
• Standards: Ensures data warehousing and operational functionality is consistent with the overall Corporate Data architecture.
  
  

IV. CONDITIONS OF WORK AND QUANTIFIABLE DATA:

• Physical Environment: Standard office environment; approximate PC use (over 90%).
• Travel: Travels out of town to attend seminars, workshops, etc. (1 to 2 times per year).
• Work Schedule: Standard work schedule (40.0 hours per week). Will be required to be on-call evenings and weekends on an assigned basis. May need to work in excess of standard work schedule.
  
  

V. JOB QUALIFICATIONS (Knowledge, Skills, Abilities And Personal Characteristics)

• Strong experience in data engineering with a total of 8 years’ experience using ETL tools such as Informatica, DataStage or ELT tools such as DBT.
• 1 - 4 years of Snowflake experience preferred.
• 1 years of experience developing or deploying data solutions in any cloud such as AWS, Google or Azure preferred.
• 5 - 6 years of demonstrated knowledge of designing dimensional modeling or relational data modeling or 5-6 years of demonstrated experience using Data Warehousing data population techniques.
• 6 - 8 years of strong knowledge of source to target mappings.
• 6 - 8 years of strong relational database knowledge; possesses a broad understanding of the current and prospective data architecture, and database performance tuning.
• 4 - 6 years’ experience with programming languages such as Python or Shell scripting is essential.
• 6 - 8 years of SQL programming.
• Strong verbal and written communication skills. Has ability to establish and maintain effective working relationships with external and internal personnel. Must be customer focused and excel in coordination of problem resolution.
• Flexibility to work in a changing, fast-paced environment and work with a sense of urgency and accountability.
• Experience with Agile methodologies is desirable.
• Rotational On-Call support is expected.
• Ability to effectively provide work direction to team members and delegate tasks; ability to train, mentor and provide constructive feedback to team members to meet or exceed performance in their job responsibilities.
• Demonstrated project management skills are desirable.
  
  

VI. SUMMARY OF DUTIES:Rank/% JOB DUTY

1/15�TA ANALYSIS

• Maps source system data to data warehouse models or other file transformations (source to target mappings).
• Works with program delivery and business team members to understand source systems data and data quality issues/performs data quality analysis.
• Defines and captures rules associated with ETL processes.
• STANDARDS: Data quality issues and risks are identified and mitigated early in the development cycle.
  
  

2/25% DATA DESIGN

• Creates ETL processes / Data pipelines and is responsible for creating and maintaining High Level Design documents.
• Employs cost effective techniques in design of ETL processes / Data pipelines to meet immediate and future business needs.
• Conducts Data pipeline design / sizing estimates for data warehousing projects or other file transformations
• Designs data solutions to satisfy business information needs and receive design acceptance.
• STANDARDS: ETL processes comply with standards and architecture.
  
  

3/30% ETL DEVELOPMENT AND TESTING

• Develops complex ETL processes.
• Creates ETL processes that supports business functionality, performs well, and is easily supported.
• Creates scripts when necessary.
• Follows ETL standards, goals, and objectives.
• Works with teammates to define and resolve complex data warehouse or file transformation project issues.
• Adapts extraction, transformation, and load (ETL) processes to accommodate changes in source systems and new business user requirements
• Completes work according to the iteration plan or release plan.
• Participates in ETL Code Reviews, receives acceptance of code by peers at review. Responsible for code reviews for other developers.
• Tests (unit level, integration level, system level, and user acceptance level) complex ETL processes / Data pipelines.
• Works with testing team in order to assist them with their testing processes.
• Assists with testing upgrades; identifies errors or inconsistencies with vendors’ software applied to Company’s data environment.
• STANDARDS: ETL processes /Data pipelines are developed according to project schedule with a high level of quality.
  
  

4/20% ETL CHANGE/CONFIGURATION MANAGEMENT AND PRODUCTION SUPPORT

• Follows Change Management/Configuration Management processes for the deployment of code into test and production environments.
• Troubleshoots data problems. Identifies, coordinates and helps resolve the root cause of day to day data production issues.
• Supports client areas in their analytical functions; corrects errors/resolves problems associated with data warehouse processes and other file transformations.
• Provides rotational on-call support, responding quickly to production issues.
• STANDARDS: ETL code is moved according to the standard process and all development audit points are delivered. Production systems are given the priority needed to keep them stable and issues are resolved quickly.
  
  

5/10% TRAINS, COACHES AND MENTORS TEAM MEMBERS

• Works collaboratively with the application development resources, all data team resources, systems analysts, and with client areas.
• Leads teammates to define and resolve complex data warehouse or file transformation production issues.
• Develops relationships though a team environment by creating vertical and horizontal communication avenues to ensure cooperative efforts are established.
• Train, mentor and coaches staff to perform their responsibilities by providing processes, tools and deliverables related to their job. Acts as a subject matter expert to the team.
• Assists team members with complex and critical tasks and determines the impact to existing systems. Makes formal recommendations to manager.
• Ensures alignment between corporate and ETG goals, Business Council prioritization, and the work that his/her team performs.
• STANDARDS: Team relationships are essential in a fast-paced, Agile environment. Verbal and written communication skills assist in these team relationships. Coaching and mentoring is provided in such a way that encourages long term learning. Simply providing solutions to problems must be balanced with providing enough direction to enable peers to discover solutions on their own. In addition, project quality and schedule must also be considered in this balance.
  
  

Performs other duties as assigned.