What are the responsibilities and job description for the Platform Engineer position at MOTOR Information Systems?
Job Description
MOTOR Information Systems, an operating group of Hearst, is actively building out its AI team. We are looking for people with a proven track record, who are willing to experiment with new ideas, invest time in them, fail-fast, and move on if they don't work out. We want team members who have shown a consistent interest in continuous learning, especially in Cloud Technologies, who are aware of, and follow the latest and best technologies and trends, can learn by themselves, are self-motivated and value self-directed initiative with technology and AI exploration.
Summary
As a Platform Engineer, your primary focus will be on establishing a Unified Data Platform. You will be responsible for designing, developing, and maintaining Kubernetes clusters, data lakes, data pipelines and platforms that fulfill the analytics and business intelligence requirements of our clients. Utilizing advanced technologies and tools, including Spark, Kafka, AWS, Azure, and Kubernetes, you will tackle large-scale and intricate data challenges. Additionally, you will collaborate with full stack developers, data scientists, analysts, and stakeholders to guarantee data quality, reliability, and usability. Proficiency in handling massive datasets is essential.
Main Responsibilities
MOTOR Information Systems, an operating group of Hearst, is actively building out its AI team. We are looking for people with a proven track record, who are willing to experiment with new ideas, invest time in them, fail-fast, and move on if they don't work out. We want team members who have shown a consistent interest in continuous learning, especially in Cloud Technologies, who are aware of, and follow the latest and best technologies and trends, can learn by themselves, are self-motivated and value self-directed initiative with technology and AI exploration.
Summary
As a Platform Engineer, your primary focus will be on establishing a Unified Data Platform. You will be responsible for designing, developing, and maintaining Kubernetes clusters, data lakes, data pipelines and platforms that fulfill the analytics and business intelligence requirements of our clients. Utilizing advanced technologies and tools, including Spark, Kafka, AWS, Azure, and Kubernetes, you will tackle large-scale and intricate data challenges. Additionally, you will collaborate with full stack developers, data scientists, analysts, and stakeholders to guarantee data quality, reliability, and usability. Proficiency in handling massive datasets is essential.
Main Responsibilities
- Infrastructure Development and Maintenance: Design, develop, and manage scalable, reliable and secure cloud infrastructures using Kubernetes, Terraform and/ or CDK
- Automate Processes: Create and manage automated pipelines for data extraction and processing using modern tools and best practices.
- Optimize Platforms: Enhance the performance, security, and reliability of data systems using AWS and DevOps best practices.
- Collaborate Across Teams: Work closely with developers, data scientists, and stakeholders to ensure high-quality, reliable data solutions.
- Research and Innovate: Continuously evaluate and integrate new technologies to improve existing data solutions.
- Bachelor's degree or higher in Computer Science, Engineering, Mathematics, or a related field
- At least 4 years of experience in data engineering or a similar role (previous DBA experience is a plus)
- Experience with CI/CD, containerization (ex: docker, K8s) and orchestration (ex: Airflow)
- Experience deploying data infrastructure on cloud platforms (AWS preferred)
- Experience working with infrastructure as code
- Experience with big data frameworks and tools, such as Spark, Hadoop, Kafka and Hive
- Proficient in SQL and at least one programming language, such as Python, Go or Java
- Experience building production systems with more modern ETL, ELT and data systems, such as AWS Glue, Databricks, Snowflake, Elastic, and Azure Cognitive Search
- Experience with microservices
- Excellent communication, collaboration, and problem-solving skills