Data Engineer

Position Summary

We are seeking a Data Engineer who is passionate about creating highly scalable and efficient data solutions that drive key business insights. As a Data Engineer, you will work with cutting-edge technologies in a cross-functional team to design, build, and optimize data architectures that empower data-driven innovation across the organization.

This role requires someone who thrives in an agile, fast-paced environment and has a deep understanding of building data solutions that evolves with rapid business growth and change.

Responsibilities

Build and Optimize Data Pipelines: Analyze and organize raw data from various internal and external sources. Design, develop, and manage scalable, high-performance and resilient data pipelines that process, transform, and make data accessible for real-time analytics and reporting.

Data Infrastructure Development: Design, build, and maintain data infrastructure to ensure data is reliable, accessible, and secure for various applications and stakeholders. Implement lake-house architecture.

Data Integration: Analyze business use case, perform data profiling and implement data transformations required for curating raw data into AI/ML and BI ready data.

Streaming & Batch Processing: Build streaming and batch data pipelines that are reusable, fault-tolerant and scalable.

Data Quality & Governance: Implement data integrity validations and data quality checks for ensuring accuracy, consistency, and completeness across all datasets.

Collaboration & Cross-Functional Partnerships: Collaborate with platform engineers, data analysts, platform engineers and business stakeholders to define and implement data requirements and deliver end-to-end solutions.

Data Warehouse Design: Design, implement, and optimize data lake, data warehouse and data mart to meet analytics and reporting needs.  

Innovation & Automation: Identify opportunities to automate data workflows, streamline processes, and introduce best-in-class tools and frameworks that enhance productivity and efficiency.

Performance Tuning: Continuously monitor and tune data pipelines and infrastructure for optimal performance, scalability, and cost-efficiency.

Qualifications

Experience: 4 years of experience as a Data Engineer or in a similar role, with a proven track record of building and maintaining data lake and warehouse solutions at scale.

Programming Skills: Expert proficiency in SQL, Python, DAX, Power Query, Scala or similar language for data processing and automation.

Data Pipeline: Experience in building scalable and resilient ELT data pipelines using modern ELT tools like Azure Data Factory, Fivetran or similar. Experience with event driven data processing using Snowflake Task, Stream and Dynamic Table. Knowledge of streaming platforms like Kafka, Spark Streaming, Kinesis or similar.

Data Lake & Warehouse: Experience with cloud-based data platforms like Snowflake. Advanced proficiency with SQL, writing complex queries and optimizing performance. Familiar with dimensional data model or data vault 2.0 design techniques.

Data Governance: Knowledge of data governance practices, including RBAC, RLS, data masking, data lineage and compliance with regulatory standards (e.g., GDPR, HIPAA).

Automation & DevOps: Familiarity with CI/CD practices, infrastructure as code (Terraform), and containerization (Docker, Kubernetes) to automate and manage data infrastructure.

Agile Methodologies: Experience working in an agile pod or scrum team with the ability to iterate and deliver rapidly.

Cultural Fit: You are comfortable working in a rapidly changing environment with ambiguous requirements. You are nimble and take intelligent risks.

Preferred Qualifications

Machine Learning: Hands-on experience with Snowpark or PySpark for machine learning and AI workflows.

Data Visualization: Experience with using and administrating data visualization tools like PowerBI, Sigma or similar.

Company Overview

Vista is a leading global investment firm that exclusively invests in enterprise software, data and technology-enabled organizations across private equity, permanent capital, credit and public equity strategies, bringing an approach that prioritizes creating enduring market value for the benefit of its global ecosystem of investors, companies, customers and employees. Vista’s investments are anchored by a sizable long-term capital base, experience in structuring technology-oriented transactions and proven, flexible management techniques that drive sustainable growth. Vista believes the transformative power of technology is the key to an even better future – a healthier planet, a smarter economy, a diverse and inclusive community and a broader path to prosperity. Further information is available at vistaequitypartners.com. Follow Vista on LinkedIn, @Vista Equity Partners, and on X, @Vista_Equity.