What are the responsibilities and job description for the Senior Azure Data Engineer position at K2 Partnering Solutions, Inc.?
Job Details
Position: Senior Azure Data Engineer
Location: Andover, MA (Hybrid role which requires working 2 days a week from office)
Duration: 6 Months Contract
Responsibilities:
Design and implement highly available and scalable Azure cloud data solutions that meet business requirements.
Collaborate with cross-functional teams to design and implement solutions that meet business requirements.
Participates in the testing process through test review and analysis.
Developing and maintaining data lake and data warehouse schematics, layouts, architectures and relational/non-relational databases for data access and Advanced Analytics.
Required Qualifications:
At least 10 years of experience in data architecture, engineering, or related field
Strong experience with Azure cloud technologies, including Azure Data Factory, IoT hub, Event hub, Cosmos DB, Azure Databricks/Apache Spark, Apache Airflow, Apache Flink.
Knowledge of streaming data platforms and real-time analytics.
Experience with designing and implementing cost-effective large-scale data platforms from real-time/batch ingestion to delivering analytics feedback and reporting to consumers.
Understanding different file format (parquet, ORC etc.) and table format (iceberg, delta, etc.).
Experience with data transformation and manipulation using Azure Databricks or similar tools.
Experience working in an Agile development environment.
Familiar with a variety of the field s concepts, practices, and procedures and a strong focus on data integrity.
Demonstrates strong written and oral communications skills. Ability, desire, and focus to meet deadlines Demonstrates ability to work with all levels of individuals.
Demonstrates organizational skills while working on multiple projects; and communicate effectively within the team.
Working knowledge of CI/CD tools and concepts - Azure Pipelines, GitHub Workflow, Terraform
Solid scripting experience with Python, and/or PowerShell.
Advanced troubleshooting skills to drive to root cause.
Ability to adapt quickly and positively to change.
Ability to manage ambiguity and solve undefined problems.
Nice to Have:
Experience on data integrations with API's.
Exposure Service Oriented Architecture.
Familiarity with GDPR, CCPA, and other data privacy regulations.
Exposure to Microsoft Fabric
Ability to guide or lead junior resources to get desired project results.
Independent and able to manage and prioritize workload.
Good working knowledge with Azure Infrastructure, security, token management.
Good knowledge of Timeseries databases (for example; InfluxDB, TimescaleDB).
Education
Bachelor s degree in computer science, Applied Mathematics, Data Science, or Machine Learning.
