What are the responsibilities and job description for the Principal Data Engineer position at OneTrust?
The Challenge
As a Principal Data Engineer, you will join our innovative enterprise data team. This team is dedicated to enabling trusted and secure data for our Marketing, Sales, Finance, Product, and Customer Experience teams. Your work will empower these teams to explore data and take actions that set us apart from our competition. You will design and develop robust data pipelines, build dimensional and semantic models, enable and manage our data catalog, and ensure security and compliance across all data and reporting solutions. Join us in supporting the growth of OneTrust through cutting-edge data solutions.
Your Mission
Inspire our Data Engineering and Insights team, drive innovation in data ingestion, transformation and management within our secure Business Technology environments.
- Driving technical conversations with business stakeholders, data architects. business intelligence, and systems team to support critical reporting and analytics needs.
- Defining and establishing scalable data models and supporting the tools and methods of populating data into various data lakes. data warehouses, data marts or data repositories.
- Designing, building, and monitoring data pipelines that meet current business requirements but can scale gracefully for the future.
- Building scalable semantic models and data solutions that will serve as analytics data products for self-service analytics.
- Providing operational support and incident/problem management, including logging, instrumentation, analysis, and implementation of process improvements.
- Coordinating with other IT teams to configure and implement security, monitoring, tooling, and related capabilities.
- Work effectively using scrum with multiple team members to deliver value effectively
You Are
This hands-on technical role demands excellent knowledge and can demonstrate best practices in the industry.
- Bachelor's degree or master's degree in computer science, Engineering or related field
- 8 years of experience in Technology, Data Engineering field in architecting, developing end-to-end scalable data applications and data pipelines; work with product manager and scrum master's and assist team with the story creations and sprint deliverables.
- 8 years of experience working in enterprise data warehouse solutions and platforms, and working knowledge of different databases (e.g., Snowflake, Databricks, MongoDB) and Storage Solutions
- 6 years of experience designing, implementing automated data solutions using Snowflake/Databricks Lakehouse; building workflows, developing, and orchestrating data pipelines using modern technologies
- 6 years of experience implementing data solutions by adopting, recommending and improving data security, auditing, change management, automation and CICD processes.
- 6 years of experience working with software engineering best practices across the development lifecycle, including agile methodologies, coding standards, code reviews, source management, build processes and testing
- You are a SQL expert. You thoroughly understand aggregate functions, window functions, UDFs, self-joins, partitioning and clustering approaches to run performant queries
- You are an Engineer. You thoroughly understand object-oriented and procedural programming paradigms (like Python) and manipulation of various data formats for extraction and transformation.
Extra Awesome
- Prior experience with full software development lifecycle, continuous integration and continuous deployment, release and change management
- Prior experience in designing data models with Salesforce, Workday, Marketo, Gainsight and other applications.
- Prior experience working with Snowflake, Databricks, AWS (S3), MongoDB, and other modern database technologies.
- Prior experience working with data integration, data ingestion and data orchestration frameworks using Fivetran, Mulesoft, Apache Spark, Apache Kafka and Apache Airflow.
- Prior experience working with data exploration and data transformation frameworks using DBT, HEX, Coalesce, SQL and Python.
- AWS or Databricks or Snowflake certification is preferred
- Problem-Solving Skills: Data Engineers look at an issue that needs to be solved and come up with solutions quickly.
- Growth Learner: Desire to continue to learn about the future architecture of data technologies.
