What are the responsibilities and job description for the Databricks Administrator - Onsite position at DHANUTEK?
Position Title: Databricks administrator
Work location: Dallas, TX/LA
Duration: 12 plus months contract with possible extension
Job duties:
• Responsibilities will include designing, implementing, and maintaining the Databricks platform, and providing operational support
• Operational support responsibilities include platform set-up and configuration, workspace administration, resource monitoring, providing technical support to data engineering, Data Science/ML, and Application/integration teams, performing restores/recoveries, troubleshooting service issues, determining the root causes of issues, and resolving issues
• Assist in the deployment and configuration of Databricks environments, ensuring alignment with best practices for security and performance
• Responsible for the administration, configuration, and optimization of the Databricks platform to enable data analytics, machine learning, and data engineering activities within the organization
• Install, configure, and maintain Databricks clusters and workspaces
• Maintain Platform currency with security, compliance, and patching best practices
• Monitor and manage cluster performance, resource utilization, platform costs, and troubleshoot issues to ensure optimal performance
• Implement and manage access controls and security policies to protect sensitive data
• Manage schema data with Unity Catalog - create, configure, catalog, external storage, and access permissions
• Manage AWS resources related to Databricks and networking configurations
Technical Competencies:
• Strong understanding of data engineering needs
• Strong understanding of aws infrastructure required by Databricks
• Experience with compliance audits/audit documentation
• Experience using Databricks APIs, system tables for creating admin related queries and dashboards
• Understanding of the Databricks workspace and development environment
• Strong debugging skills
• Proficiency in pySpark, Python, SQL and aws
Skills :
4 years - Database admin
2 years - Python, Pyspark
