Senior Data Engineer

Job Title: Data Engineer / Data Architect – Databricks & Unity Catalog

Cleveland, Ohio or Alpharetta, Georgia

FTE Permanent

Job Summary:

We are looking for a skilled Sr Data Engineer /with expertise in Databricks and Unity Catalog to design, implement, and manage scalable data solutions.

Key Responsibilities:

• Design and implement scalable data pipelines and ETL workflows using Databricks.
• Implement Unity Catalog for data governance, access control, and metadata management across multiple workspaces.
• Develop Delta Lake architectures for optimized data storage and retrieval.
• Establish best practices for data security, compliance, and lineage tracking in Unity Catalog.
• Optimize data lakehouse architecture for performance and cost efficiency.
• Collaborate with data scientists, engineers, and business teams to support analytical workloads.
• Monitor and troubleshoot Databricks clusters, performance tuning, and cost management.
• Implement data quality frameworks and observability solutions to maintain high data integrity.
• Work with Azure/AWS/GCP cloud environments to deploy and manage data solutions.

Required Skills & Qualifications:

• 8-19 years of experience in data engineering, data architecture, or cloud data solutions.
• Strong hands-on experience with Databricks and Unity Catalog.
• Expertise in PySpark, Scala, or SQL for data processing.
• Deep understanding of Delta Lake, Lakehouse architecture, and data partitioning strategies.
• Experience with RBAC, ABAC, and access control mechanisms within Unity Catalog.
• Knowledge of data governance, compliance standards (GDPR, HIPAA, etc.), and audit logging.
• Familiarity with cloud platforms (Azure, AWS, or GCP) and their respective data services.
• Strong understanding of CI/CD pipelines, DevOps, and Infrastructure as Code (IaC).
• Experience integrating BI tools (Tableau, Power BI, Looker) and ML frameworks is a plus.
• Excellent problem-solving, communication, and collaboration skills