Data Engineer

Data Engineer
Location-San Francisco, CA / 1-2 days onsite(Candidates Local to CA Only).
Job Type-Long Term
Direct client

W2 Role

Interview Mode:
1. Coding Test will be required on SQL , Python/Pyspark/Scala.
2. Internal Video Call Interview.
3. Client Video Call.

12 Years IT Experience Required.

Required Skills & Experience:
Hands-on code mindset with deep understanding in technologies / skillset and an ability to understand larger picture.
Sound knowledge to understand Architectural Patterns, best practices and Non[1]Functional Requirements
Overall, 5 years of experience in heavy volume data processing, data platform, data lake, big data, data warehouse, or equivalent.
5 years of experience with strong proficiency in Python and Spark (must-have).
3 years of hands-on experience in ETL workflows using Spark, Scala and Python.
3 years of experience with large-scale data loads, feature extraction, and data processing pipelines in different modes near real time, batch, realtime.
Solid understanding of data quality, data accuracy concepts and practices.
2 years of experience working with Python deep learning libraries like PyTorch, Tensorflow, Keras. Experience integrating with various data stores, including: SQL/NoSQL databases o In-memory stores like Redis o Data lakes (e.g., Delta Lake).

• 
  Required: Experience withDatabricks or similar data lake / data platform.
  Requred:
  NoSQL/ SQL
  Plus:Java and Spring Boot experience
  Familiarity with notebook-based environments such as Jupyter Notebook.
  Adaptability: Must be open to learning new technologies and approaches.
  Initiative: Ability to take ownership of tasks, learn independently, and innovate.
  With technology landscape changing rapidly, ability and willingness to learn new technologies as needed and produce results on job.
  Preferred Skills:
• StrongJava skills (preferable)
  Ability to pivot from conventional approaches and develop creative solutions