What are the responsibilities and job description for the DHTS Developer position at Duke Health?
At Duke Health, we're driven by a commitment to compassionate care that changes the lives of patients, their loved ones, and the greater community. No matter where your talents lie, join us and discover how we can advance health together.
Primary Role
The Data Engineer will be responsible for designing, developing, and optimizing data pipelines, manage servers, and integrate APIs. They will collaborate closely with data scientists and analysts to meet data requirements and ensure robust data management solutions.
Work Performed
Level 1
- Individual contributor works under the direct supervision of level 3 or above and collaborates as a member of a team
- Basic understanding of local databases (Clarity; Caboodle) and underlying tables pertaining to role/domain
- Basic understanding of ETL technologies and scripting languages and terminology related to role
- Basic understanding of cloud data engineering technologies, including storage accounts, data pipelines, data lakes, structured vs. unstructured data, etc.
- Ability to design, construct and test data management systems under guidance of senior team members
- Assists in server administration tasks, ensuring system performance and availability
- Basic knowledge of SQL and relational databases
- Follows all policies and procedures set forth by DUHS/DHTS/ACE as pertains to role
- Responds to customer requests for assistance as assigned and prioritized by manager
- Participates in code review and learning best practices in data engineering
- Maintains ongoing communications (timely, clear, concise, and accurate) with manager keeping them apprised of request status
- Maintains appropriate project documentation as assigned such as GITLAB or data dictionary
Level 2
- Meets all Competencies in Level 1 AND:
- Works with new hires/Level I team members to train on use of tools and technologies to accomplish job duties
- Demonstrates advanced knowledge of the assigned domain/role. Understands data, workflows, and tools related to role.
- Demonstrates advanced knowledge of, and experience with, cloud data engineering concepts, including (but not limited to) Microsoft Azure Lakehouses and Delta Tables, Azure Data Factory, Azure Synapse, and medallion-style data architecture.
- Demonstrates ability to work with stakeholders to understand business needs and perform business analysis independently >75% of the time.
- Develop technical requirements from business requirements independently >75% of the time.
- Can write/modify accurate SQL queries independently > 90% of the time
- Ability to optimize SQL queries, stored procedures and scripts for performance reliability
- Can write complex SQL inclusion/exclusion criteria for cohort development
- Manages and optimizes server infrastructure for performance and reliability
- Troubleshoot and resolve complex data pipeline, server and API issues
- Ability to locate data elements independently > 75% of the time
- Knowledge/utilization of basic data design and infrastructure
- Participates in team code review
- Actively participates/contributes to database QA/submit requests for new database tables/attributes
- Actively seeks out participation in Special Projects and is able to effectively accomplish project assignments with minimal supervision
- Ability to manage multiple projects to completion in a timely manor
- Ability to develop and present topics as assigned/needed by customer and/or ACE team
- Active participation in Duke Analytic Community or other networking forums applicable to job functions
Level 3
- Meets all Competencies in Level 2 AND:
- Demonstrates expertise in systems and software related to project area to accomplish job duties (e.g. SSIS, Azure Data Factory, etc..)
- Resolves escalated issues/incidents to ensure high availability and performance of ETL processes
- Architects and designs robust data solutions
- Expert knowledge and experience with Microsoft Azure Data Engineering tools, concepts, and industry best practices.
- Expert locating data elements as required. Assists team in locating data elements as needed.
- Works with team members to train on advanced skills in using tools and technologies to accomplish job duties
- Leads the development of complex data pipelines and API integration, ensuring scalability and efficiency
- Defines standards and best practices for data pipeline design, development and maintenance
- Demonstrates ability to work independently to solve problems and complete assigned work >95% of the time
- Demonstrates ability to work with customers to understand business need or problem to address independently >95% of the time
- Leads Projects with minimal supervision
- Change agent with regard to team processes, workflows when issues/gaps are found
- Collaborates with data architect on design and implementation of data solutions (lakehouse, data warehouse, etc.)
- Serves as moderator, leader or channel owner for a forum such as Duke Analytics Community or other professional networking organization
Required Qualifications at this Level
Education/Training
Bachelor's degree in a related field, or four years of equivalent technical experience required.
LICENSURE/CERTIFICATION:
None required. Preferred certification in development technologies, as appropriate for the position.
Experience
Level 1 - No experience required beyond the minimum education (or equivalency) requirement.
Level 2 - Two years of directly related experience is required.
Level 3 - Four years of directly related experience is required.
Duke is an Affirmative Action/Equal Opportunity Employer committed to providing employment opportunity without regard to an individual's age, color, disability, gender, gender expression, gender identity, genetic information, national origin, race, religion, sex, sexual orientation, or veteran status.
Duke aspires to create a community built on collaboration, innovation, creativity, and belonging. Our collective success depends on the robust exchange of ideas—an exchange that is best when the rich diversity of our perspectives, backgrounds, and experiences flourishes. To achieve this exchange, it is essential that all members of the community feel secure and welcome, that the contributions of all individuals are respected, and that all voices are heard. All members of our community have a responsibility to uphold these values.
Essential Physical Job Functions: Certain jobs at Duke University and Duke University Health System may include essentialjob functions that require specific physical and/or mental abilities. Additional information and provision for requests for reasonable accommodation will be provided by each hiring department.
