What are the responsibilities and job description for the Databricks Subject Matter Expert position at Vigna Solutions Inc.?
Job Details
Hello,
We are looking for,
Position: Databricks Subject Matter Expert (SME) with expertise in Data Profiling
Location: Dallas, TX (Onsite)
Duration: 9-12 Months contract with possible extension.
Job Description
Key Responsibilities:
- Act as a Databricks SME, providing guidance on best practices, performance tuning, and optimization.
- Develop and maintain data models (conceptual, logical, and physical) to support business intelligence and analytical needs.
- Conduct data profiling, data quality analysis, and anomaly detection to ensure clean, reliable, and well-structured data.
- Design, implement, and optimize ETL/ELT pipelines within Databricks, using Apache Spark, Delta Lake, and cloud storage.
- Advanced knowledge of SQL, PySpark, and other scripting languages used within Databricks.
- Experience in Data Governance and Data Quality practices..
- Proficiency in Python, Scala, or other programming languages commonly used in Databricks.
- Work with cloud-based data architectures, such as Azure Databricks, AWS Databricks, or Google Cloud Platform Dataproc, and integrate with data lakes and data warehouses.
- Stay updated with emerging trends in big data, data engineering, and Databricks to drive continuous improvement.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
