Data MLOps Engineer

Role : Azure Senior Data Lead with Azure Data Factory (ADF)Azure DatabricksSQLOracle PL / SQLPython

Location : Remote

• Experience working within the Azure ecosystem including Azure AI Search Azure Storage Blob Azure Postgres and understanding how to leverage them for data processing storage and analytics tasks.
• Ability to preprocess and clean large datasets efficiently using Azure Tools / Python and other data manipulation tools. Experience with techniques such as data normalization feature engineering and data augmentation is preferred.
• To have background in Data Science / MLOps and proficiency in DevOps CI / CD Azure Cloud computing and Model monitoring.
• Expertise in working with healthcare data standards (ex. HIPAA and FHIR) sensitive data and data masking techniques to mask personally identifiable information (PII) and protected health information (PHI) is essential.
• Indepth knowledge of search algorithms indexing techniques and retrieval models for effective information retrieval tasks. Familiarity with search platforms like Elasticsearch or Azure AI Search is a must.
• Familiarity with chunking techniques and working with vectors and vector databases like Pinecone.
• Ability to design develop and maintain scalable data pipelines for ingesting processing and transforming large volumes of structured and unstructured data.
• Experience with implementing best practices for data storage retrieval and access control to ensure data integrity security and compliance with regulatory requirements.
• Be able to implement efficient data processing workflows to support the training and evaluation of solutions using large language models ensuring reliability scalability and performance.
• Ability to proactively identify and address issues related to data quality pipeline failures or resource contention ensuring minimal disruption to systems.
• Experience with large language model frameworks such as Langchain and know how to integrate them into data pipelines for natural language processing tasks.
• Experience working within the snowflake ecosystem.
• Knowledge of cloud computing principles and experience in deploying scaling and monitoring AI solutions on cloud platforms like Snowflake Azure AWS.
• Ability to communicate complex technical concepts effectively to technical and nontechnical stakeholders and collaborate with crossfunctional teams.
• Analytical mindset with a keen attention to detail coupled with the ability to solve complex problems efficiently.
• Knowledge of cloud cost management principles and best practices to optimize cloud resource usage and minimize costs.

Job Summary

The Senior Technical Lead will be responsible for leading the technical aspects of projects related to Azure Data Factory (ADF) Azure Databricks SQL Oracle PL / SQL and Python. The role involves designing developing and maintaining data pipelines and ETL processes while ensuring high performance and scalability of data solutions. (1.) Key Responsibilities

1. Lead the endtoend technical implementation of data projects using azure data factory azure databricks sql oracle pl / sql and python.

2. Design and develop efficient and reliable etl processes for large datasets.

3. Collaborate with cross functional teams to understand business requirements and translate them into technical solutions.

4. Optimize data workflows troubleshoot issues and ensure data quality and integrity.

5. Implement best practices for data security governance and compliance.

6. Provide technical guidance mentoring and support to junior team members.

7. Stay uptodate with the latest trends and technologies in data engineering and analytics.

Skill Requirements

1. Proficiency in azure data factory (adf) and azure databricks for data integration and processing.

2. Strong skills in writing complex sql queries for data manipulation and analysis.

3. Experience with oracle pl / sql for database development and management.

4. Proficient in python programming language for automation and scripting tasks.

5. Solid understanding of data warehousing concepts etl processes and data modeling.

6. Ability to work in a fast paced environment and manage multiple priorities effectively.

7. Excellent problemsolving skills and attention to detail.

8. Strong communication and interpersonal skills for effective collaboration with team members and stakeholders.

Certifications : Relevant certifications in Azure Data Factory Azure Databricks SQL Oracle or Python would be a plus.

Key Skills

Apache Hive,S3,Hadoop,Redshift,Spark,AWS,Apache Pig,NoSQL,Big Data,Data Warehouse,Kafka,Scala

Employment Type : Full Time

Vacancy : 1