What are the responsibilities and job description for the Senior Azure Databricks Administrator position at Vish Consulting Services, Inc.?
Job Details
Position: Senior Azure Databricks Administrator (Urgent - 1 interview scheduled)
Location: Cincinnati, OH
Duration: 12 Months
Job Description:
Databricks Platform Administration
- Deploy, configure, and maintain Azure Databricks workspaces
- Manage clusters, pools, and jobs for optimal performance.
- Handle Databricks Runtime upgrades and compatibility testing.
- Configure Databricks notebooks, Repos, and Delta Lake for efficient workflows.
Security & Access Control:
- Implement Identity and Access Management (IAM) using Azure AD / Azure IAM / SCIM.
- Set up and enforce Role-Based Access Control (RBAC) and network security.
- Ensure compliance with data governance policies (Unity Catalog, creation of Catalogs, schema etc.).
- Applying grants of catalogs and schemas
- Manage personal access tokens (PATs) and OAuth-based authentication.
Performance Monitoring & Optimization:
- Monitor cluster performance, auto-scaling, and cost optimization.
- Analyze and fine-tune Spark jobs to improve execution efficiency.
- Use Databricks SQL for querying and performance diagnostics.
Integration & Automation:
- Integrate Databricks with Azure Data Factory, Synapse, Snowflake, and other data sources.
- Automate cluster management using Terraform.
- Implement CI/CD pipelines for Databricks notebooks using GitHub or Azure DevOps.
Troubleshooting & Support:
- Investigate and resolve job failures, cluster crashes, and query performance issues.
- Provide L2/L3 support for Databricks infrastructure issues.
- Collaborate with Databricks support for bug fixes and escalations
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
