Azure Databricks Engineer

Looking for a skilled Azure Databricks Engineer to join our team. This role requires a combination of technical expertise in data processing and analysis, programming, and a deep understanding of the Azure ecosystem. If you are passionate about turning data into actionable insights and have a knack for optimizing data systems, read on for the qualifications and responsibilities.

Responsibilities

• Develop and maintain scalable data pipelines within Azure Databricks
• Design and implement data modeling, data warehousing, and data lakes using Azure services
• Utilize PowerBI for creating insightful dashboards and interactive reports
• Write SQL queries and scripts for data manipulation and retrieval
• Ensure data quality and consistency across customer's and other data sources
• Optimize data processing performance within Azure Databricks
• Collaborate with cross-functional teams to gather and understand data requirements

Qualifications

Proficiency in PowerBI

• Proficiency in DAX (Data Analysis Expressions)

• Skilled in PowerQuery for data transformation

• Experience in creating robust dashboards and reports

• Strong background in data visualization techniques and tools

• In-depth knowledge of SQL, including Data Query Language (DQL), Data Manipulation Language (DML), and Data Definition Language (DDL)
• Experience with SQL Server database management
• Proficient with PySpark
• Knowledge of Apache Spark and its core components
• Ability to write and optimize Spark SQL queries
• Experience with Spark DataFrame and Spark Streaming
• Familiarity with Databricks Unity Catalog
• Experience with Azure DevOps
• Azure Bicep experience is a plus
• Expertise in Python with C#, R or Scala experience as an added advantage

Education

Bachelor s degree in Computer Science, Engineering, or a related field.