Azure Data Engineer (Snowflake experience ideal!)

Overview

We are seeking a highly skilled and experienced Data Engineer to join Global Logistics client. The ideal candidate will be a senior engineer with design experience and the potential to grow into a lead role!

We are looking for someone possessing a strong technical foundation in data engineering and analytics within Azure environments. This role involves hands-on development, data modeling, architecture, and the implementation of ETL/ELT processes, with a focus on optimizing data warehousing and analytics solutions. The successful candidate will play a crucial role in understanding business needs, building out data models informed by the business, and establishing a robust data warehouse to support reporting and analytics.

Key Responsibilities

• Design, develop, and maintain Azure Database technologies and environments.
• Utilize Databricks for data warehousing, including hands-on development and some design
• Implement automation for CI/CD pipelines, DevOps validations, and notebook-based pipelines using Python/Jupyter and PySpark.
• Use PowerBI for reporting and analytics.
• Develop and validate ETL/ELT processes, ensuring data quality and integrity.
• Design and build data models from scratch, including entity-relationship diagrams (ERDs) and relational data models to represent business concepts effectively.
• Engage with stakeholders to establish validation strategies and automate the validation process, enhancing data security and compliance.

Required Experience:

• Extensive depth of experience designing and developing within Azure Databricks or Snowflake
• Proficient in other Azure data technologies, including MS SQL Database, Azure SQL Database, Azure Synapse, Azure Data Warehouse, and Azure Data Factory.
• Expertise in writing complex SQL scripts from scratch
• Experience with ELT/ETL processes using Databricks.
• Experience with PowerBI and reporting
• Experience implementing data governance and metadata management.
• Knowledge of data integration strategies, performance optimization, and practical application of ETL/ELT methodologies.
• Familiarity with data processing and analytics using Databricks and Azure Synapse, including real-time data processing.

Other desired skills:

• Strong project leadership skills with a focus on ownership and delivery of projects.
• Ability to work efficiently in a lean team, wearing multiple hats and driving projects forward without needing extensive guidance.
• Excellent communication skills, with the ability to understand business needs and offer suggestions
• A proactive approach to problem-solving, with a recognition of the value of version control and eagerness to contribute to the team's success.