What are the responsibilities and job description for the Snowflake Data Engineer position at Compu-Vision Consulting?
Job tile : Snowflake Data Engineer
Duration : 5 Months ( 1day / week in office - preferably Tuesday )
Location : Boston, MA(Hybrid)
Summary :
The Client is looking for an experienced Snowflake Data Engineer to lead the implementation of Division of Housing Stabilization data in our new Snowflake data warehouse. The Snowflake Data Engineer plays a critical role in designing, building, and supporting the components of the agency's data warehouse. This position involves working with various stakeholders to ensure accurate and efficient data management, reporting, and analytics. As a Snowflake Data Engineer, you will collaborate closely with the technology group, the agency data team, agency program staff and external organizations that assist in delivery of thier programs. The Data Engineer will report to clients Director of Information Technology Development
Responsibilities :
The Snowflake Data Engineer is responsible for :
o Design, develop, and maintain complex snowflake data warehouses, including data modeling, ETL processes, and data quality control.
o Ensure data integrity, quality, and security across all data warehousing activities.
o Develop and maintain data pipelines to integrate data from various internal and external sources.
o Develop and maintain data pipelines using tools for data transformation, testing, and deployment. Ensure data pipelines are scalable, reliable, and efficient.
o Write Python code to automate data processing, transformation, and loading tasks, including data ingestion, data quality control, and data visualization.
o Collaborate with data analysts and business stakeholders to understand data requirements and develop data solutions that meet business needs.
o Develop and maintain data visualizations and reports using Snowflake's built-in visualization tools or third-party tools like Tableau.
o Ensure data quality, integrity, and security by developing data validation rules, data cleansing processes, and data access controls.
o Optimize snowflake data warehouse performance, scalability, and reliability, including monitoring and troubleshooting data issues.
o Develop and maintain technical documentation, including data dictionary, data flow diagrams, and code comments.
o Participate in code reviews and provide feedback to ensure high-quality code and adherence to coding standards.
o Work closely with the agency's data team and the overall state technology team to ensure alignment on data strategy and technology standards.
o Communicate complex data findings to non-technical stakeholders in a clear and concise manner
o Provide technical guidance and mentorship to junior analysts and other team members
Qualifications :
o 8 years of experience in Snowflake data warehouse development and management, data engineering field.
o Strong understanding of Snowflake architecture, data modeling, and data warehousing concepts.
o Proficiency with SQL, including snowflake's SQL dialect and complex query writing for performance optimization.
o Experience with Snowflake features, including cloning, time travel, data sharing, and micro-partitioning.
o Strong understanding of Snowflake's architecture, including virtual warehouses, storage, and compute separation.
o Expertise in Snowflake Role-Based Access Control (RBAC), data masking, and encryption.
o Proficiency with data integration tools and ETL / ELT processes.
o Experience in Python or similar scripting languages for automation and data processing.
o Knowledge of data security and compliance standards as they relate to Snowflake.
o Experience with cloud platforms like AWS, Azure.
o SnowPro Advanced certification is a plus.
o Knowledge of data governance frameworks and tools.
o Strong experience integrating Snowflake with Tableau, Salesforce, third party APIs etc.
o Strong analytical, problem-solving, and communication skills
o bility to work collaboratively in a cross-functional team environment
o Excellent verbal and written communication skills with project teams and business teams
o Team-oriented attitude and the proven ability to collaborate at all levels of an organization
Desired Skills / Experience :
- SnowPro Advanced Architect or SnowPro Data Engineering certification is preferred.
- Experience in building complex data pipelines in an multi cloud environment.
- Experience implementing data masking, and aggregation to reduce privacy risks
- Experience integrating data from Salesforce solutions
- Knowledge of CI / CD practices and tools.
- Familiarity JIRA or other project management tools
- Experience using GitHub
- Experience working on projects in the affordable housing sector or other public benefit programs
- Familiarity with a variety of software development methodologies such as agile, waterfall, and a strong understanding of the suitability of these methodologies for certain development environments and types of pplications
- Experience with advanced Microsoft Office, pivot tables, charts, Visio, data analytics and JIRA or other project management tools
- Experience with SQL a plus
