What are the responsibilities and job description for the AI Big Data Engineer position at Hearst Television Inc?
Job Description
MOTOR Information Systems, an operating group of Hearst, is actively building out its AI team. We are looking for people with a proven track record, who are willing to experiment with new ideas, invest time in them, fail-fast, and move on if they don't work out. We want team members who have shown a consistent interest in continuous learning, especially in Cloud Technologies, who are aware of, and follow the latest and best generative AI technologies and trends, can learn by themselves, are self-motivated and value self-directed initiative with technology and AI exploration.
Summary
As a Data Engineer, your primary focus will be on establishing a Unified Data Platform. You will be responsible for designing, developing, and maintaining data pipelines, data lakes, and platforms that fulfill the analytics and business intelligence requirements of our clients. Utilizing advanced technologies and tools, including Spark, Kafka, AWS, Azure, and Kubernetes, you will tackle large-scale and intricate data challenges. Additionally, you will collaborate with full stack developers, data scientists, analysts, and stakeholders to guarantee data quality, reliability, and usability. Proficiency in handling massive datasets is essential.
Main Responsibilities
- Build automated pipelines to extract and process data from a variety of legacy platforms (predominantly SQL Server), e.g., in stored procedures, Glue processing, etc.
- Implement data-related business logic on modern data platforms, such as AWS Glue, Databricks, and Azure using best practices and industry standards.
- Create vector databases, data marts and the data models to support them
- Optimize and monitor the performance, reliability, and security of data systems and processes.
- Integrate and transform data from (or to) various sources and formats, such as structured, unstructured, streaming, and batch.
- Develop and maintain data quality checks, tests, and documentation.
- Support data analysis, reporting, and visualization using tools such as SQL, Python, Tableau and Quicksight
- Research and evaluate new data technologies and trends to improve data solutions and existing capabilities.
Qualifications And Skills
EEO EMPLOYER
