What are the responsibilities and job description for the Senior Azure Data Architect position at NextGen IT Inc.?
Job Details
Sr. Data Architect (Databricks, Azure Data Lake, Python)
Duration: Long term contract
Location: Greenfield, Indian
Mandatory Cloud/Data/Solution Architecture addressing various patterns around Data management , security, deployment, costing, governance, modelling, observability, Finops, well architected frameworks etc
Key Responsibilities: Design, develop, and maintain Databricks Lakehouse solutions sourcing from cloud platforms such as Azure Synapse and Google Cloud Platform.
Implement and manage DevOps and CICD workflows using tools like GitHub.
Apply best practices in test-driven development, code review, branching strategies, and deployment processes.
Build, manage, and optimize Python packages using tools like setup, poetry, wheels, and artifact registries.
Develop and optimize data pipelines and workflows in Databricks, utilizing PySpark and Databricks Asset Bundles.
Manage and query SQL databases (Unity Catalog, SQL Server, Hive, Postgres).
Implement orchestration solutions using Databricks Workflows, Airflow, and Dagster.
Work with event-driven architectures using Kafka, Azure Event Hub, and Google C4 Cloud Pub/Sub.
Develop and maintain Change Data Capture (CDC) solutions using tools like Debezium.
Extensive experience in design and implementation of data migration projects specifically involving Azure Synapse and Databricks Lakehouse
Manage cloud storage solutions, including Azure Data Lake Storage and Google Cloud Storage.
Configure and manage identity and access solutions using Azure Active Directory, including AD Groups, Service Principals, and Managed Identities.
Primary (Must have skills) *
Python package builds(setup,poetry, wheels,artifact registries)
Specific technologies
Databricks(PySpark,Databricks Asset Bundles)
Open File Formats (Delta/Parquet/Iceberg/etc )
SQL Databases (Unity Catalog, SQL Server, Hive, Postgres)
Orchestration Tools(Databricks Workflows, Airflow, Dagster)
Azure Data Lake Storage
Azure Active Directory (AD groups, Service Principles,Managed Identities) Kafka, Azure Event Hub, Cloud Pub/Sub
Change Data Capture (Debizum)
Google Cloud Storage
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
