What are the responsibilities and job description for the Data Engineer position at Software Resources, Inc.?
Software Resources has an immediate job opportunity for a Data Engineer with a major corporation in San Francisco, CA (hybrid).
Duration: 9 month(s)
Pay Rate: $75 - $83/hr DOE
Focus on SQL Knowledge and workflow management (airflow tool) and Machine Learning Knowledge specifically feature. Understanding data inputs into the model and feature analysis. Feature Engineering. Training data set composition.
Department Overview:
Applied Machine Learning Engineers, Data Engineers, and Data Scientists on the Streaming Machine Learning and Innovation team develop and maintain recommendation and personalization algorithms for Streaming's suite of streaming video apps. As a member of this team you will collaborate across Engineering, Product, and Data teams to apply machine learning methods to meet strategic product personalization goals, explore innovative, cutting edge techniques that can be applied to recommendations, and constantly seek ways to optimize operational processes.
Our team is…
• A group of engineers and data scientists with diverse expertise delivering solutions together.
• Collaborative and dynamic.
• Embracing agile practices.
• Using continuous integration/automated testing.
• Led by startup veterans.
Required Qualifications:
2 years of data engineering experience
Deep knowledge of the Python data ecosystem
Great coding and problem-solving skills
Experience in building large datasets and scalable services
Experience deploying and running services in AWS and in engineering big-data
solutions using technologies like Databricks, EMR, S3, Spark, and Docker
Excellent communication and people engagement skills
Preferred Qualifications:
Experience with Pyspark to perform data transformations, feature engineering, data analysis, and ML training data generation
Experience building streaming pipelines using Kafka, Spark, or Flink
ML algorithmic and systems knowledge/experience
Responsibilities:
Partner with technical and non-technical colleagues to understand ML algorithm feature, data, and workflow requirements.
Implement necessary feature analysis, new features, training datasets and related workflows and monitoring metrics and services
Work with Engineering teams to collect required data from internal and external systems
Develop and maintain ETL routines using orchestration tools such as Airflow
Collaborate with machine learning practitioners to design and build model and data forward
solutions
Deploy scalable streaming and batch data pipelines to support petabyte scale datasets
Enforce common data design patterns to increase code maintainability
Create ETL architecture designs and conduct reviews
Perform ad hoc data analysis and maintenance as necessary
Partner with team leads to identify, design, and implement internal process
improvements
Drive and maintain a culture of quality, innovation, and experimentation
Work in an Agile environment that focuses on collaboration and teamwork
Required Education:
Bachelor's degree or relevant years of work experience
About Software Resources
Software Resources, founded more than 3 decades ago, is a trusted staffing partner specializing in Technology (IT, Creative, & Marketing), Finance, & Accounting placements. We work with companies nationwide that need skilled professionals and connect them with candidates like you looking for the right opportunity.
We are always looking for professionals at all experience levels and in various specialties to meet our clients' hiring needs. As a member of our team, you will have access to benefits including medical, dental, and vision coverage, a 401(k) with company match, short-term disability, life insurance with AD&D.
If this position fits interests you, we encourage you to apply. You can view all our job openings at Software Resources Careers.
Duration: 9 month(s)
Pay Rate: $75 - $83/hr DOE
Focus on SQL Knowledge and workflow management (airflow tool) and Machine Learning Knowledge specifically feature. Understanding data inputs into the model and feature analysis. Feature Engineering. Training data set composition.
Department Overview:
Applied Machine Learning Engineers, Data Engineers, and Data Scientists on the Streaming Machine Learning and Innovation team develop and maintain recommendation and personalization algorithms for Streaming's suite of streaming video apps. As a member of this team you will collaborate across Engineering, Product, and Data teams to apply machine learning methods to meet strategic product personalization goals, explore innovative, cutting edge techniques that can be applied to recommendations, and constantly seek ways to optimize operational processes.
Our team is…
• A group of engineers and data scientists with diverse expertise delivering solutions together.
• Collaborative and dynamic.
• Embracing agile practices.
• Using continuous integration/automated testing.
• Led by startup veterans.
Required Qualifications:
2 years of data engineering experience
Deep knowledge of the Python data ecosystem
Great coding and problem-solving skills
Experience in building large datasets and scalable services
Experience deploying and running services in AWS and in engineering big-data
solutions using technologies like Databricks, EMR, S3, Spark, and Docker
Excellent communication and people engagement skills
Preferred Qualifications:
Experience with Pyspark to perform data transformations, feature engineering, data analysis, and ML training data generation
Experience building streaming pipelines using Kafka, Spark, or Flink
ML algorithmic and systems knowledge/experience
Responsibilities:
Partner with technical and non-technical colleagues to understand ML algorithm feature, data, and workflow requirements.
Implement necessary feature analysis, new features, training datasets and related workflows and monitoring metrics and services
Work with Engineering teams to collect required data from internal and external systems
Develop and maintain ETL routines using orchestration tools such as Airflow
Collaborate with machine learning practitioners to design and build model and data forward
solutions
Deploy scalable streaming and batch data pipelines to support petabyte scale datasets
Enforce common data design patterns to increase code maintainability
Create ETL architecture designs and conduct reviews
Perform ad hoc data analysis and maintenance as necessary
Partner with team leads to identify, design, and implement internal process
improvements
Drive and maintain a culture of quality, innovation, and experimentation
Work in an Agile environment that focuses on collaboration and teamwork
Required Education:
Bachelor's degree or relevant years of work experience
About Software Resources
Software Resources, founded more than 3 decades ago, is a trusted staffing partner specializing in Technology (IT, Creative, & Marketing), Finance, & Accounting placements. We work with companies nationwide that need skilled professionals and connect them with candidates like you looking for the right opportunity.
We are always looking for professionals at all experience levels and in various specialties to meet our clients' hiring needs. As a member of our team, you will have access to benefits including medical, dental, and vision coverage, a 401(k) with company match, short-term disability, life insurance with AD&D.
If this position fits interests you, we encourage you to apply. You can view all our job openings at Software Resources Careers.
Salary : $75 - $83
