Azure Data Engineer

Role: Sr. Data Engineer

Location: Washington, DC

Duration: Long Term

Hybrid Onsite: 4 days onsite per week from Day 1

Overview:

A professional in data modeling and synthesis is responsible for designing, implementing, and maintaining data structures that support organizational data needs.

This role encompasses various tasks, including connecting multiple data sources, creating data synthesis flows, and working with Big Data(Databricks and PySpark), AI, and Gen AI technologies to drive data-driven insights and automation.

Key Responsibilities:

• Data Modeling: Design and implement conceptual, logical, and physical data models to support business requirements.
• Data Synthesis: Develop processes to synthesize data from various sources, ensuring that it is clean, accurate, and ready for analysis.
• Connecting Data Sources: Integrate multiple data sources into a unified system, facilitating seamless data flow and accessibility.
• Data Flow Creation: Create and manage workflows that define how data moves through systems, ensuring efficient processing and storage.
• Big Data Experience: Utilize big data technologies (like Databricks, and PySpark) to handle large volumes of structured and unstructured data.
• Databricks & PySpark Implementation: Build scalable ETL pipelines using Databricks and PySpark, optimizing data transformations and processing for analytics.
• Collaboration: Work closely with data architects, analysts, and IT teams to align data strategies with business objectives.
• AI & Gen AI Exposure:
• Leverage AI/ML models to enhance data processing, transformation, and analysis.
• Work with Gen AI frameworks (like OpenAI, Hugging Face, or LangChain) for data-driven automation and insights generation.
• Implement AI-powered data pipelines to optimize data quality, enrichment, and predictive analytics.

Required Skills

• Technical Proficiency: Strong knowledge of SQL, Databricks, and PySpark, along with experience in data modeling tools (e.g., Microsoft Visio).
• Data Architecture Understanding: Familiarity with relational databases, big data frameworks and ETL processes.
• Analytical Skills: Ability to analyze complex datasets and derive actionable insights.
• Communication Skills: Excellent verbal and written communication skills to convey technical concepts to non-technical stakeholders.
• Problem-Solving Abilities: Strong troubleshooting skills to identify and resolve data-related issues efficiently.

“Mindlance is an Equal Opportunity Employer and does not discriminate in employment on the basis of – Minority/Gender/Disability/Religion/LGBTQI/Age/Veterans.”