What are the responsibilities and job description for the Azure Databricks Architect Sr Data Engineer with Healthcare Background position at Artifact Global Corporation?
Overall Responsibility :
Responsible for developing implementing and operating stable scalable low cost solutions to source data from client systems into the Data Lake data warehouse and enduser facing BI applications. Responsible for ingestion transformation and integration of data to provide a platform that supports data analysis and enrichment as well as making data operationally available for analysis. The data Engineer will be a data pipeline builder data wrangler and support application developers database architects data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture and is consistent throughout our ongoing projects.
Essential Functions :
- Build and maintain scalable automated data pipelines. Support critical data pipelines with a highly scalable distributed architecture including data ingestion (streaming events and batch) data integration data curation. Deploy automate maintain and manage Azure cloudbased production system to ensure the availability performance scalability and security of productions systems. Good architectural understanding to build and ensure customer success when building new solutions and migrating existing data applications on Azure platform.
- Conduct full technical discovery identifying pain points business and technical requirements as is and to be scenarios.
- Design and arrangement of scalable highly attainable and fault tolerant systems on Azure platform.
- Ownership and responsibility for endtoend design and development testing release of key components.
- Understand and implement best practices in management of data including master data reference data metadata data quality and lineage. Experience with code versioning tools and a command of configuration management concepts and tools CICD including DevOps. Other duties as assigned.
Experience :
Other Skills :
Key Skills
Fund Management,Drafting,End User Support,Infrastructure,Airlines,Catia
Employment Type : Full Time
Vacancy : 1
