What are the responsibilities and job description for the Senior Data Engineer position at C&G Consulting Services, Inc?
This job is a Hybrid role located in Jupiter FL. Open to Relocation.
Job Description:
Summary of Job Functions
- Collaborate effectively with Stakeholders, Project Managers, Software Engineers, Data Analysts, QA Analysts, DBAs, and Data Engineers
- Build and maintain data pipelines based on functional and non-functional requirements
- Proactively seek out information and overcome obstacles to deliver projects efficiently
- Ensure that data pipelines incorporate best practices related to performance, scaling, extensibility, fault tolerance, instrumentation, and maintainability
- Ensure that data pipelines are kept simple and not overly engineered
- Produce and maintain design and operational documentation
- Analyze complex data problems and engineer elegant solutions
- Stay abreast of emerging technologies and make relevant recommendations
- Upgrade existing data models and pipelines leveraging newer features and techniques
- Work in a Kanban environment
- Mentor less experienced data engineers
- Participate in engineering standards and best practices evolution
- Participate in an on-call rotation
- Lead investigations to troubleshoot pipeline issues
Minimum Requirements
10 years with hands-on data engineering and deep knowledge of data architecture fundamentals including:
- Extensive experience building ETL/ELT pipelines from a variety of data sources
- Broad experience with database and associated technologies for moving and storage of data
- Experience evaluating and selecting tools and technologies based on fitness for purpose and stakeholder requirements
- Deep knowledge and experience with concepts such as performance tuning, execution plan analysis, and index optimization in at least one database technology (e.g., T-SQL, PL-SQL, NoSQL, etc.)
- Extensive experience creating and maintaining a wide variety of database objects including, tables, views, functions, stored procedures, etc.
- Comprehensive experience with data modeling indexes, temporal tables, CLR, etc.
- Deep understanding of the development of data pipelines using multiple external data sources and transport mechanisms
- Strong initiative, collaboration, accountability, impartiality, and communication
- Strong analytical skills, a real passion for working with data and strong interest in solving data problems
- Strong track record for judging core requirements and meeting deadlines
- Experience managing master data
- Experience writing C#, PowerShell, and Python
- Experience with Git source control
- Experience working in a Kanban SDLC and a strong understanding of traditional Kanban SDLC workflows
- Experience with deploying changes through segregated Development, QA, UAT and Production SDLC stages
- Experience owning mission-critical processes
- Bachelor’s degree in Computer Science, Information Systems, or related disciplines
- Ability to work onsite in our Jupiter, FL office
Preferred Skills and Previous Experience
- Python programming using libraries such as Pandas, Numpy, csv, Traceback, JSON, PyODBC, Math
- Experience with source code branching and pull requests / code reviews
- Experience with AWS
- Experience working with clinical and/or consumer healthcare data
- Experience with tools such as Red Gate, Grafana, OpsGenie and JAMS
- MS/PhD in Computer Science, Information Systems, or related disciplines