What are the responsibilities and job description for the Walmart Senior Data Engineer - Sunnyvale - sql scala apache kafka Agriculture position at Data Engineer Jobs?
What you'll do...
Position : Senior Data Engineer
Make sure to apply quickly in order to maximise your chances of being considered for an interview Read the complete job description below.
Job Location : 680 W. California Avenue, Sunnyvale, CA 94086
Duties :
- Build scalable, high performance and robust Java, Scala, and Python applications.
- Build complex large scale data pipelines using cutting-edge big data technologies such as Spark, Kafka, Apache Airflow, Apache NiFi, Hadoop, and Hive which are used for customer interactions and customer transactions.
- Responsible for designing, developing, and maintaining ETL data processing pipelines on the Google Cloud Platform for customer profile and engagement data sets.
- Discover batch data from external sources to ingest it into the team's data pipeline and store it in the organization's HDFS space.
- Work on user engagement data such as clicks, email interactions and engagements, and push notification engagements.
- Collect, clean, organize and analyze big data of user interactions with platforms such as mobile app and website and store the data in Hive tables to assist in marketing.
- Work on multiple projects to ensure that the collected customer interaction data meets data quality standards and rules.
- Collect and visualize data metrics using Grafana, Scala, and MySQL.
- Configure, maintain, and optimize GCP cloud compute infrastructure including migration of workflows to reduce cloud costs.
- Implement CICD (Continuous Integration and Continuous Development) for the platform using Jenkins, GitHub, and the GCP console.
- Actively work with product teams to plan out development cycle for new features via JIRA tickets.
- Help analyze A / B test results and plan out a test-driven development approach as per AGILE methodology.
- Participate in the discovery phase and build high level designs.
- Evaluate and fine-tune systems for speed, robustness and cost efficiency.
- Troubleshoot production issues as necessary by responding to pager duty alerts triggered by Xmatters.
- Participate in high-impact tech initiatives from end-to-end, covering all stages of the software development lifecycle and all the components of the solution.
- Apply security patches, upgrading versions, renewing certificates, and managing the change management process.
- Provide new developers with training, a system overview, and deep dive presentations for knowledge transfer.
Minimum education and experience required :
Master's degree or equivalent in Computer Science, Engineering (Any), or related field and 1 year of experience in large scale enterprise software development environment; OR Bachelor's degree or equivalent in Computer Science, Engineering (Any), or related field and 4 years of experience in large scale enterprise software development environment.
Skills required :
Salary Range : $174,518 / year to $234,000 / year. Additional compensation includes annual or quarterly performance incentives. Additional compensation for certain positions may also include : Regional Pay Zone (RPZ) (based on location) and Stock equity incentives.
Benefits :
Eligibility requirements apply to some benefits and may depend on your job classification and length of employment. Benefits are subject to change and may be subject to a specific plan or program terms.
J-18808-Ljbffr
Salary : $174,518 - $234,000
