What are the responsibilities and job description for the Sr Data Engineer position at Pipe Recruit?
Job Title: Sr. Data Engineer
Location: Glendale, CA Onsite
Job Type: C2C
Job Responsibilities:
- Maintain, update, and expand existing core data platform pipelines.
- Build and maintain APIs to expose data to downstream applications.
- Develop real-time streaming data pipelines.
- Work with technologies including Airflow, Spark, Databricks, Delta Lake, and Snowflake.
- Collaborate with product managers, architects, and other engineers to ensure platform success.
- Develop and document internal/external standards, best practices, and pipeline configurations.
- Ensure high operational efficiency and data accuracy to meet SLAs and reliability standards for stakeholders (Engineering, Data Science, Operations, and Analytics teams).
Required Qualifications:
- 7 years of data engineering experience developing large-scale data pipelines.
- Proficiency in at least one major programming language (Python, Java, or Scala).
- Hands-on experience with distributed processing systems such as Apache Spark.
- Production-level experience with data pipeline orchestration tools like Apache Airflow.
- Expertise in at least one Massively Parallel Processing (MPP) or cloud database technology (Snowflake, Databricks, BigQuery).
- Experience in data modeling and API development using GraphQL.
- Advanced understanding of OLTP vs. OLAP environments.
Preferred Qualifications (Nice-to-Have):
- Experience with Graph Databases.
- Real-time event streaming expertise.
Salary : $121,300 - $145,600
