What are the responsibilities and job description for the Data Engineer position at ShiftCode Analytics?
Interview : Vide
Visa : USC, GC, GC EAD, H4, L2
This is hybrid from day-1.
Must have strong experience with Scala.
Description :
- Experience dealing with large volumes of data, from various sources, both structured and unstructured.
- Ability to triage and talk through performance / scaling issues of dealing with data at scale.
- Good understanding of how data will be read (file formats, partitioning, bucketing).
- Extensive experience writing testable jobs using Spark (or equivalent) framework.
- Programming & Scripting Languages : Java EE, Scala, Spark, SQL, Bash.
- Web services & API standards : REST, OAuth, JSON.
- Software Architectures (micro-services, event driven, peer-to-peer).
- Application Security.
- Asynchronous Pub-Sub and Point to Point Messaging Systems.
- Advantage, if you have experience working in ETL and Hadoop Ecosystem : HBase, Solr, Spark Streaming, Kudu, Spring Boot, Spring Context, Spring Data Rest, General Cloudera experience.
- Streaming within the Hadoop ecosystem is a plus.
