Implement modern data lake house architecture using Delta Lake, ensuring data reliability and performance.
Data Governance :
Establish data governance frameworks incorporating Microsoft Purview for data quality, lineage, and compliance.
Implement row-level security.
Configure Microsoft Purview policies.
Establish data masking for sensitive information.
Design audit logging mechanisms.
Pipeline Development :
Design scalable data pipelines using Azure Data bricks for ETL / ELT processes and real-time data integration.
Performance Optimization :
Implement performance tuning strategies for large-scale data processing and analytics workloads.
Optimize Spark configurations.
Implement partitioning strategies.
Design caching mechanisms.
Establish monitoring frameworks.
Security Framework :
Design and implement security patterns aligned with federal and state requirements for sensitive data handling.
Required Qualifications :
Bachelor's degree in computer science or related field.
Experience :
6 years of experience in data architecture and engineering.
2 years hands-on experience with Azure Data bricks and Spark.
Recent experience with Microsoft Fabric platform.
Technical Skills :
Microsoft Fabric Expertise :
Data Integration.
Combining and cleansing data from various sources.
Data Pipeline Management.
Creating, orchestrating, and troubleshooting data pipelines.
nalytics Reporting :
Building and delivering detailed reports and dashboards to derive meaningful insights from large datasets.
Data Visualization Techniques : Representing data graphically in impactful and informative ways.
Optimization and Security : Optimizing queries, improving performance, and securing data
zure Databricks Experience :
pache Spark Proficiency Utilizing Spark for large-scale data processing and analytics.
Data Engineering : Building and managing data pipelines, including ETL (Extract, Transform, Load) processes.
Delta Lake : Implementing Delta Lake for data versioning, ACID transactions, and schema enforcement.
Data Analysis and Visualization : Using Databricks notebooks for exploratory data analysis (EDA) and creating visualizations.
Cluster Management : Configuring and managing Data bricks clusters for optimized performance.
Integration with Azure Services : Integrating Data bricks with other Azure services like Azure Data Lake, Azure SQL Database, and Azure Synapse Analytics.
Machine Learning : Developing and deploying machine learning models using Databricks MLflow and other tools.
Data Governance : Implementing data governance practices using Unity Catalog and Microsoft Purview.
Programming and Query Languages :
SQL : Proficiency in SQL for querying and managing databases, including skills in SELECT statements, JOINs, subqueries, and window functions12.
Python : Using Python for data manipulation, analysis, and scripting, including libraries like Pandas, NumPy, and PySpark.
Data Modeling :
Dimensional modeling.
Real-time data modeling patterns.
Soft Skills :
Strong analytical and problem-solving abilities.
Excellent communication skills for technical and non-technical audiences.
Experience working with government stakeholders.
Preferred Experience :
zure DevOps.
Infrastructure as Code (Terraform).
CI / CD for data pipelines.
Data mesh architecture.
Certifications (Preferred) :
Microsoft Azure Data Engineer Associate.
Data bricks Data Engineer Professional.
Microsoft Fabric certifications (as they become available).
Project-Specific Requirements :
Experience designing data architectures for grant management systems.
Knowledge of federal / state compliance requirements for data handling.
Understanding of financial data processing requirements.
Experience with real-time integration patterns.
This position requires strong expertise in modern data architecture with specific focus on Microsoft's data platform.
The successful candidate will play a crucial role in designing and implementing scalable data solutions that enable efficient data processing and analytics for state-level grant management and reporting systems.
Keep a pulse on the job market with advanced job matching technology.
If your compensation planning software is too rigid to deploy winning incentive strategies, it’s time to find an adaptable solution.
Compensation Planning
Enhance your organization's compensation strategy with salary data sets that HR and team managers can use to pay your staff right.
Surveys & Data Sets
What is the career path for a Data Architect?
Sign up to receive alerts about other jobs on the Data Architect career path by checking the boxes next to the positions that interest you.
