What are the responsibilities and job description for the Senior Data Engineer position at Smart Synergies?
Job Details
General Summary
As a Senior Data Engineer, you will design, build, and maintain the data infrastructure that powers the client's analytics and data science initiatives. This includes developing and maintaining data pipelines, data warehouses, and data lakes using the latest Microsoft technologies. The Senior Data Engineer will also be responsible for implementing practices to ensure compliance with data privacy and security policies.
Principal Duties and Responsibilities
1. Design, build, and maintain scalable and efficient data pipelines for acquiring, processing, and storing structured and unstructured data
2. Design, build, and maintain data lake and data warehouse solutions in accordance with the architecture and standards
3. Create and maintain data models that capture the organization's data requirements and enable efficient data retrieval and analysis
4. Design, build, and maintain practices to ensure compliance with data privacy and data governance policies
5. Continuous improvement of data engineering processes and development practices
Specific Functional Work Experience Required
1. 7 years of experience in a data engineering role or similar job-related experience
2. Proven record of accomplishment designing and implementing complex data integration solutions using Microsoft Azure technologies
3. In-depth knowledge and hands-on experience with Azure Data Factory, Azure Synapse Analytics, Azure SQL Database, Microsoft Fabric, Azure Cloud Storage, and Azure Cosmos DB
4. Hands-on Lakehouse development experience using Delta Lake
5. Extensive experience in dimensional and relational data modeling, database design, and optimization
6. Expert-level SQL and T-SQL skills and experience
7. Hands-on experience using the Spark data processing framework
8. Hands-on experience automating processes with a programming language such as Python and PowerShell
9. Practical experience implementing data quality, data privacy, and data security measures
10. Hands-on experience using Microsoft Power BI
Required Knowledge, Skills, or Abilities
1. Excellent documentation and communication skills, with the ability to articulate complex technical concepts clearly
2. Continuous learning mindset and commitment to stay current with advances in Microsoft data and analytics technologies
3. Proven ability to work collaboratively in a team environment and drive successful data engineering initiatives
4. Familiarity with Agile development methodologies and experience working in Agile teams
5. Ability to adapt to changing project requirements and priorities, deliver results in iterative cycles, and participate in sprint planning and reviews
6. Strong leadership abilities, with the capacity to provide technical mentorship and guidance to junior team members
