What are the responsibilities and job description for the Data Engineer position at STIAOS Technologies?
Required :
8 Years of hands on experience on below skillset
1. Snowflake
2. Azure Databricks
3. Azure Data Factory
4. Pyspark
5. Python
6. SQL
Role and Responsibilities :
Design, implement, and manage Azure cloud solutions for various projects.
- Develop and maintain data pipelines using Databricks (PySpark) for data ingestion, processing, and analysis.
- Configure and manage Azure Data Factory (ADF) to orchestrate data workflows and ETL processes.
- Implement and optimize Azure Data Lake Storage (ADLS) for efficient data storage and retrieval.
- Collaborate with cross-functional teams to design and implement data warehouse solutions.
- Utilize Git for version control and collaboration on codebase.
- Monitor, troubleshoot, and optimize data processes for performance and reliability.
- Implement security best practices and manage access controls using Azure Active Directory (AAD).
- Document technical designs, processes, and procedures.
- Stay updated with the latest Azure cloud technologies and best practices.
Requirements :
