What are the responsibilities and job description for the Data Architect - Microsoft Fabric & Azure Databricks position at Apollo ITS?
Job Title : Data Architect - Microsoft Fabric & Azure Databricks
Location : Atlanta, GA
Duration : 6 Months
Interview Type : Either Web Cam or In Person
Work Arrangement : Hybrid
Job Description :
The client is seeking an experienced Data Architect to design and implement enterprise data solutions using Microsoft Fabric and Azure Databricks for integration with state-level systems. This role will focus on creating scalable data architecture that enables seamless data flow between IES Gateway and our analytics platform. The ideal candidate will have deep expertise in modern data architecture, with specific experience in Microsoft's data platform and Delta Lake architecture.
Work Location & Attendance Requirements :
- Must be physically located in Georgia
- On-site : Tuesday to Thursday, per manager's discretion
- Mandatory in-person meetings :
- All Hands
- Enterprise Applications
- On-site meetings
- DECAL All Staff
- Work arrangements subject to management's decision
While the intent may be a long-term tenure, this position is subject to annual budget restrictions. The initial contract is through the end of this fiscal year and is anticipated to be renewed July 1st.
Key Responsibilities :
Data Architecture :
Integration Design :
Lakehouse Architecture :
Data Governance :
Implement row-level security :
Pipeline Development :
Performance Optimization :
Security Framework :
Required Qualifications :
Education : Bachelor's degree in computer science or related field.
Experience :
Technical Skills : Microsoft Fabric Expertise :
Data Integration : Combining and cleansing data from various sources.
Data Pipeline Management : Creating, orchestrating, and troubleshooting data pipelines.
Analytics Reporting : Building and delivering detailed reports and dashboards to derive meaningful insights from large datasets.
Data Visualization Techniques : Representing data graphically in impactful and informative ways.
Optimization and Security : Optimizing queries, improving performance, and securing data
Azure Databricks Experience :
Apache Spark Proficiency : Utilizing Spark for large-scale data processing and analytics.
Data Engineering : Building and managing data pipelines, including ETL (Extract, Transform, Load) processes.
Delta Lake : Implementing Delta Lake for data versioning, ACID transactions, and schema enforcement.
Data Analysis and Visualization : Using Databricks notebooks for exploratory data analysis (EDA) and creating visualizations.
Cluster Management : Configuring and managing Databricks clusters for optimized performance. (Ex : autoscaling and automatic termination)
Integration with Azure Services : Integrating Databricks with other Azure services like Azure Data Lake, Azure SQL Database, and Azure Synapse Analytics.
Machine Learning : Developing and deploying machine learning models using Databricks MLflow and other tools.
Data Governance : Implementing data governance practices using Unity Catalog and Microsoft Purview Programming & Query Languages :
SQL : Proficiency in SQL for querying and managing databases, including skills in SELECT statements, JOINs, subqueries, and window functions12.
Python : Using Python for data manipulation, analysis, and scripting, including libraries like Pandas, NumPy, and PySpark
Data Modeling :
Soft Skills :
Preferred Experience :
Certifications (preferred) :
Project-Specific Requirements :
This position requires strong expertise in modern data architecture with specific focus on Microsoft's data platform. The successful candidate will play a crucial role in designing and implementing scalable data solutions that enable efficient data processing and analytics for state-level grant management and reporting systems.
Required / Desired Skills :
