Data Architect (Microsoft Fabric & Azure Databricks)

Job Description

Job Description

Job Title : Data Architect (Microsoft Fabric & Azure Databricks)

Address : Atlanta, Georgia 30334

6 Months Contract Position

Job Description :

The Client is seeking an experienced Data Architect to design and implement enterprise data solutions using Microsoft Fabric and Azure Databricks for integration with state-level systems. This role will focus on creating scalable data architecture that enables seamless data flow between IES Gateway and our analytics platform. The ideal candidate will have deep expertise in modern data architecture, with specific experience in Microsoft's data platform and Delta Lake architecture.

Work Location & Attendance Requirements :

• Must be physically located in Georgia
• On-site : Tuesday to Thursday, per manager's discretion
• Mandatory in-person meetings :
• Work arrangements subject to management's decision
• While the intent may be a long-term tenure, this position is subject to annual budget restrictions. The initial contract is through the end of this fiscal year and is anticipated to be renewed July 1st.

Key Responsibilities :

Data Architecture :

I ntegration Design :

Lakehouse Architecture :

Data Governance :

Implement row-level security :

Pipeline Development :

Performance Optimization :

Security Framework :

Required Qualifications :

6 years of experience in data architecture and engineering.

2 years hands-on experience with Azure Databricks and Spark.

Recent experience with Microsoft Fabric platform.

Technical Skills :

Microsoft Fabric Expertise :

Azure Databricks Experience : Apache Spark Proficiency : Utilizing Spark for large-scale data processing and analytics.

Data Engineering : Building and managing data pipelines, including ETL (Extract, Transform, Load) processes.

Delta Lake : Implementing Delta Lake for data versioning, ACID transactions, and schema enforcement.

Data Analysis and Visualization : Using Databricks notebooks for exploratory data analysis (EDA) and creating visualizations.

Cluster Management : Configuring and managing Databricks clusters for optimized performance. (Ex : autoscaling and automatic termination)

Integration with Azure Services : Integrating Databricks with other Azure services like Azure Data Lake, Azure SQL Database, and Azure Synapse Analytics.

Machine Learning : Developing and deploying machine learning models using Databricks MLflow and other tools.

Data Governance : Implementing data governance practices using Unity Catalog and Microsoft Purview

Programming & Query Languages :

SQL : Proficiency in SQL for querying and managing databases, including skills in SELECT statements, JOINs, subqueries, and window functions12.

Python : Using Python for data manipulation, analysis, and scripting, including libraries like Pandas, NumPy, and PySpark

Data Modeling : Dimensional modeling

Real-time data modeling patterns

Soft Skills :

Strong analytical and problem-solving abilities

Excellent communication skills for technical and non-technical audiences

Experience working with government stakeholders

Preferred Experience :

Certifications (preferred) :

Project-Specific Requirements :

Skill Matrix :

Skill Required No. of Years Actual Years of Experience 6 years of experience in data architecture and engineering. 6 Years 2 years hands-on experience with Azure Databricks and Spark. 2 Years Recent experience with Microsoft Fabric platform. 2 Years Azure Databricks Experience 2 Years Proficiency in SQL for querying and managing databases, including skills in SELECT statements, JOINs, subqueries, and window functions12. 3 Years Using Python for data manipulation, analysis, and scripting, including libraries like Pandas, NumPy, and PySpark 3 Years

About Us :

InterSources Inc. is a Small, Woman, and Minority-Owned Business Enterprise, ISO / IEC 27001, SOC 2 Type 2 certified company with massive 18 years of diversified experience in providing IT Consulting Services, Artificial Intelligence, Data Analysis, Application Development, Cloud Services, Cybersecurity, Digital Marketing, ERP Management, Custom Software Development, Web Development, UI / UX Design, System Integration, QA Support etc. We make reasonable accommodations for clients and employees, and we do not discriminate based on any protected attribute including race, religion, color, national origin, gender sexual orientation, gender identity, age, or marital status. We also are a Google Cloud and Oracle partner company.