What are the responsibilities and job description for the Senior Data Engineer / Data Engineering Architect position at DESIGNLIBRO INC?
Job Description
Job Description
Benefits :Responsibilities
- Design robust and scalable data system architecture to collect, process, and store large volumes of data from IoT devices.
- Design and build data systems that support real-time, customer facing data applications, ensuring data integrity and efficient retrieval processes.
- Design and build a data validation framework to guarantee the accuracy and consistency of data across all stages of the pipeline, ensuring reliable insights and decision-making.
- Implement data architectures to continuously improve AI models through automated feedback loops and data collection processes.
- Optimize data processing workflows for AIoT tasks.
- Collaborate with product owners to translate data requirements into actionable data engineering solutions.
- Collaborate with other engineers, product owners to solve challenging problems.
- Evangelize software engineering best practices and lead by example.
- Bachelor's degree in Computer Science, Data Science, Engineering, or a related field.
- 5 years of proven experience in data engineering, with a strong focus on data warehousing and data system architecture.
- Expertise in design and build IoT data systems.
- Expertise in architecting that power high-volume consumer application.
- Expertise in big data and streaming technologies (Python, Java, Kafka, Spark,Spark Streaming).
- Expertise in moderen database technology (MongoDB, ElasticSearch, StarRocks, S3, Snowflake, Bigquery etc).
- Experience with data annotation tools and processes for machine learning datasets.
- Familiarity with MLOps practices and tools for managing AI model lifecycles.
- Knowledge about deploying systems into a production Cloud Native Environment (AWS or similar).
- Experience in Java Spring Framework is a plus
- Experience with video data handling and image processing is a plus
Qualifications
