What are the responsibilities and job description for the Data Engineer position at Inficare?
Job Position : Data Engineer
Job Location : Bentonville, AR
Job duration : Contract
Responsibilities :
- Design, develop, and maintain robust and scalable ETL workflows and data pipelines using tools like Hive, Spark, and Airflow.
- Implement and manage data storage and processing solutions using Apache Hudi and BigQuery.
- Develop and optimize data pipelines for structured and unstructured data in GCP environments, leveraging GCS for data storage.
- Write clean, maintainable, and efficient code in Scala and Python to process and transform data.
- Ensure data quality, integrity, and consistency by implementing appropriate data validation and monitoring techniques.
- Work with cross-functional teams to understand business requirements and deliver data solutions that drive insights and decision-making.
- Troubleshoot and resolve performance and scalability issues in data processing and pipelines.
- Stay updated with the latest developments in big data technologies and tools and incorporate them into the workflow as appropriate.
Required Skills and Qualifications :
Must have skills.
