What are the responsibilities and job description for the Databricks Platform Administrator position at Derex Technologies Inc?
Company Description
Derex Technologies Inc specializes in providing IT consulting, staffing solutions and software services. Globally headquartered in Harrison New Jersey since 1996 Derex delivers the highest quality technology professionals and an array of customized IT talent solutions designed to improve productivity and drive results to global clients throughout North America.
With over two decades of unparalleled experience, Derex provides supports to its clientele, across such industries as Systems Integration, Banking and Finance, Telecommunications, Pharmaceutical and Life Sciences, Energy, Healthcare, Technology, Transportation, and local and federal Government agencies.
Job Description
Position: Databricks Platform Administrator
Location: Cincinnati, OH (Remote)
Duration: 12 months
Job Description:
Databricks Platform Administration:
- Deploy, configure, and maintain Azure Databricks workspaces
- Manage clusters, pools, and jobs for optimal performance.
- Handle Databricks Runtime upgrades and compatibility testing.
- Configure Databricks notebooks, Repos, and Delta Lake for efficient workflows.
Security & Access Control:
- Implement Identity and Access Management (IAM) using Azure AD / Azure IAM / SCIM.
- Set up and enforce Role-Based Access Control (RBAC) and network security.
- Ensure compliance with data governance policies (Unity Catalog, creation of Catalogs, schema etc.).
- Applying grants of catalogs and schemas
- Manage personal access tokens (PATs) and OAuth-based authentication.
Performance Monitoring & Optimization:
- Monitor cluster performance, auto-scaling, and cost optimization.
- Analyze and fine-tune Spark jobs to improve execution efficiency.
- Use Databricks SQL for querying and performance diagnostics.
Integration & Automation:
- Integrate Databricks with Azure Data Factory, Synapse, Snowflake, Kafka, and other data sources.
- Automate cluster management using Terraform.
- Implement CI/CD pipelines for Databricks notebooks using GitHub or Azure DevOps.
Troubleshooting & Support:
- Investigate and resolve job failures, cluster crashes, and query performance issues.
- Provide L2/L3 support for Databricks infrastructure issues.
- Collaborate with Databricks support for bug fixes and escalations.
Regards,
Manoj
Derex Technologies INC
Contact : 973-834-5005 Ext 206
Additional Information
All your information will be kept confidential according to EEO guidelines.
