What are the responsibilities and job description for the Mid-Level Data Engineer position at VivSoft Technologies?
About the company:
At VivSoft, we aim to solve complex federal problems using emerging and open technologies in a collaborative and rewarding environment. VivSoft is a diverse team of strategists, engineers, designers, and creators experienced in building high performance effective softwares, with impactful organizational design and organizational dynamics for software delivery. We build secure Software Factories based on DoD reference designs and NIST Frameworks for Cloud and DevSecOps. These factories deliver AI/ML Applications, Data Science Platforms, Blockchain and Microservices for DoD, Healthcare and Civilian Agencies
Job Summary:
The Mid-Level Data Engineer will play a crucial role in the DoD team, focusing on building and maintaining data pipelines and architectures that support the development of AI-enabled applications and analytics solutions. This position is essential for ensuring the effective integration and management of data across various platforms to enhance decision-making capabilities within the Department of Defense (DoD).
\n- Data Pipeline Development: Design, implement, and optimize data pipelines to ingest, process, and store diverse datasets from multiple sources, ensuring high availability and reliability.
- Data Integration: Collaborate with data scientists, analysts, and other engineers to integrate data from various systems into a cohesive architecture that supports advanced analytics and machine learning initiatives.
- Performance Monitoring: Monitor data processing workflows to ensure optimal performance and troubleshoot any issues that arise in the data pipeline.
- Data Quality Assurance: Implement data quality checks and validation processes to maintain the integrity and accuracy of data used in analytics.
- Documentation: Develop and maintain comprehensive documentation for data architectures, processes, and workflows to facilitate knowledge sharing within the team.
- Clearance Requirements: Active Top Secret (TS) clearance with Sensitive Compartmented Information (SCI) eligibility
- Education: Bachelor’s degree in computer science, Data Engineering, Information Technology, or a related field; master’s degree preferred.
- Minimum of 5 years of experience in data engineering or a related field with a focus on building data pipelines and architectures.
- Proven experience with big data technologies such as Apache Spark, Hadoop, or similar frameworks.
- Familiarity with cloud platforms (e.g., AWS, Azure) for data storage and processing is advantageous.
- Proficiency in programming languages such as Python, Java, or Scala; experience with SQL for database management.
- Strong understanding of data modeling concepts and best practices for managing large datasets.
- Excellent problem-solving skills with the ability to analyze complex data issues.
