Bigdata Engineer

Job Description

Position Details:

Job Title: Bigdata Engineer

Location: Tampa, FL

Duration: 12 Months Contract to hire

Job Responsibilities:

Principal Responsibilities

·        Design interfaces to the data warehouses/data storages and machine learning/Big Data

·        applications using open source tools such as Scala, Java, Python, Perl and shell scripting.

·        Design and create data pipelines to maintain stable dataflow to the machine learning models –

·        both in batch mode and near real-time mode.

·        Interface with Engineering/Operations/System Admin/Data Scientist teams to ensure data

·        pipelines and processes fit within the production framework.

·        Ensure that tools and environments adhere to strict security protocols.

·        Deploy the machine learning model and serve its outputs as RESTful API calls.

·        Understand the business needs in close collaborations with subject matter experts (SMEs)

·        and Data Scientists to do efficient feature engineering for machine learning models.

·        Maintain the code and libraries in code repository.

·        Work with system administration team to proactively resolve issues/install tools and libraries

·        on the AWS platform.

·        Research and come up with architecture and solutions most appropriate for problems at hand.

·        Maintain and improve tools to assist Analytics in ETL, retrospective testing, efficiency,

·        repeatability, and R&D.

·        Lead by example regarding software best practices, including code style and architecture,

·        documentation, source control, and testing.

·        Support the Chief Data Scientist/Data Scientists/Big Data Engineers in creating new and novel

·        approaches to solve challenging problems using Machine Learning, Big Data and Cloud

·        technologies.

·        Handle ADHOC requirements to create reports for the end users.

Required Skills

·        Strong skills with Apache Spark (Spark SQL) and SCALA with at least 2 years of experience.

·        Understanding of AWS Big Data components and tools.

·        Strong Java skills with experience in web services and web development is required.

·        Hands on experience with model deployment.

·        Hands on experience in application deployment on Docker and/or Kubernetes or other similar technology.

·        Linux scripting is a plus.

·        Fundamental understanding of AWS cloud components.

·        2 years of experience in data ingesting, cleansing/processing, storing and querying large datasets

·        2 years of experience in engineering large-scale data solutions with Java/Tomcat/ SQL/Linux

·        Experience working in a data intensive role including the extraction of data (db/web/api/etc.), transformation and loading (ETL)

·        Exposure with structured and/or unstructured data contents

·        Experience with data cleansing/preparation on Hadoop/Apache Spark Ecosystem – MapReduce/Hive/HBase/Spark SQL

·        Experience with distributed streaming tools like Apache KAFKA.

·        Experience with multiple file formats (Parquet, Avro, OCR)

·        Knowledge in AGILE development cycle.

·        Efficient coding skills to enhance the performance/cost savings of the job running on AWS platform.

·        Experience in building stable, scalable, and high-speed live streams of data and serving web platforms

·        Enthusiastic self-starter with ability to work in a team environment.

·        Graduate (MS) or Undergraduate degree in Computer Science/ Engineering/relevant field

Nice to have:

·        Strong Software development experience

·        Machine Learning model deployment experience

·        Ability to write custom Map/Reduce programs to clean/prepare complex data

·        Familiarity with Streaming data processing - Experience with distributed real time computation system like Apache STORM/Apache Spark Streaming.

Additional Information

All your information will be kept confidential according to EEO guidelines.