Data Engineer

Please, no third parties. Permanent residents only.
This is a direct-hire or contract-to-hire position.

Main Duties & Responsibilities:
- Develop and sustain efficient, scalable data pipelines leveraging Microsoft Azure tools, SQL, and ETL technologies to support analytics and machine learning workflows.
- Manage structured and unstructured datasets from a variety of internal and external systems.
- Utilize Azure-based services such as Azure Data Factory, Azure Synapse, and Azure Storage to create robust, cloud-centric data solutions.
- Optimize data processes for cost-effectiveness, speed, and reliability.
- Partner with key stakeholders to define schemas, partitioning strategies, and data structures tailored for analytics and AI initiatives.
- Enforce best practices for data governance, including metadata management, security policies, and compliance measures.
- Monitor data pipelines and databases for bottlenecks, diagnosing and resolving issues to enhance system performance.
- Implement strategies to reduce data processing latency and improve throughput.
- Work alongside machine learning engineers to identify data requirements, define validation rules, and assist in feature engineering.
- Engage in Agile development practices, contributing to sprint planning, backlog refinement, and iterative improvements.
- Maintain thorough documentation detailing data flows, transformation logic, and architectural decisions.
- Clearly convey technical insights to business teams, ensuring alignment between data strategies and organizational goals.

Skills & Requirements:
- 3 years of experience in data engineering or a similar field, including a heavy focus on Microsoft cloud solutions.
- Strong command of Microsoft technologies, ETL/ELT strategies, and data processing techniques.
- Advanced SQL skills, including query optimization, performance tuning, and database modeling.
- Hands-on experience with Azure data services such as Data Factory, Databricks, and Synapse, or equivalent cloud-based solutions.
- Familiarity with distributed data systems, data partitioning methodologies, and data lake architectures.
- Proficiency in Agile development environments and integrating data solutions into CI/CD pipelines.
- Experience with version control systems such as Git.
- Excellent communication skills, with the ability to articulate technical concepts to non-technical stakeholders.
- Team-oriented mindset with a strong focus on customer needs.
- Experience with real-time data streaming technologies (e.g., Kafka).
- Exposure to MLOps and automated model deployment pipelines.
- Bachelors Degree or higher in Computer Science or related field is preferred.


tags: senior cloud data engineer, senior data engineer, senior azure data engineer, senior business intelligence developer, senior bi developer, senior datawarehouse developer, senior data warehouse developer, senior database engineer, senior sql developer