What are the responsibilities and job description for the Data Engineer (Python / Google Cloud Platform) [Local candidates preferred] position at Xoriant Corporation?
Job Details
Job Description:
As a Data Engineer, you will be responsible for designing, building, maintaining, monitoring and troubleshooting large-scale data pipelines. You will own a mix of batch-job and event-driven data processing applications with strict SLAs around processing latencies and accuracy. You will work with Data Scientists, Product Managers, Machine Learning Engineers and Platform/Service Engineers to build robust, fault-tolerant, large-scale data processing applications including datalakes and warehouses.
Responsibilities:
- Experience building robust large-scale data pipelines and storage solutions using Python, Pyspark, Databricks, Google/Azure clouds, Apache Airflow, and other relevant tools.
- Experience with frameworks like Apache Airflow, Apache Kafka/Streams etc.
- Experience designing and managing data warehouses and data lakes like Big Query, GCS, Delta lake. Reporting/Dashboarding tools like Google looker, Tableau, MS Power BI. Familiarity with automated ML frameworks like Vertex.ai or Element.
- Experience with data-quality and pipeline-health monitoring and alerting solutions like Prometheus, Grafana, Splunk etc. Good understanding of DevOps and MLOps practices.
- Solid understanding of data engineering, data analysis, and data visualization disciplines.
Others:
- Bachelor s degree in Computer Science or equivalent is required. Strong expertise in data analysis, data visualization, and machine learning algorithms.
- Position requires candidate to work from our Sunnyvale or Bentonville offices at least 2 days of the week. Strong communication skills to effectively convey complex technical concepts to non-technical stakeholders.
- Must be able to deliver with little to no supervision or support, as this will be a new team we are building.