Sr. Databricks Engineer (Databricks + Spark + Azure) :: Fulltime :: Onsite @Dallas,TX

Primary Skills

Databricks

PySpark

ETL

DWH

Responsibilities

Manage end to end delivery by Investigating problem areas, working cross-functionally with product manager & other stakeholders

Follow the Agile development methodology; think strategically and execute methodically

Develop and manage capacity and growth projection forecasts of the environment within budgets

Create and maintain optimal data pipeline architecture, assemble large, complex data sets that meet functional / non-functional business requirements

Drive optimization, testing and tooling to improve quality of solutions

Manage teams that build and operate high volume distributed systems in a SaaS environment

Great at devising efficient processes that increase velocity and quality

Desired Experience/ Skills:

Strong experience building and leading highly complex, technical engineering teams.

Strong hands-on experience in Data-bricks

Experience managing distributed teams preferred.

Strong technical experience in large distributed systems, Data Warehousing, Data Lake at scale Project management skills: financial/budget management, scheduling and resource management experience with medium and large-scale projects

Comfortable working with ambiguity and multiple stakeholders.

Comfortable working cross functionality with product management and directly with customers; ability to deeply understand product and customer personas.

Architecture Design Experience for Cloud and Non-cloud platforms

Expertise on Azure Cloud platform

Knowledge on orchestrating workloads on cloud

Ability to set and lead the technical vision while balancing business drivers

Strong experience with PySpark, Python programming

Proficiency with APIs, containerization and orchestration is a plus

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.