What are the responsibilities and job description for the Data Engineer - Mid position at Base-2 Solutions, LLC?
Job Description
As a Data Engineer/Integrator on the Program, your role entails collaborating closely with a team of developers to fulfill data integration requirements. This involves writing and maintaining code using an Extract-Transform-Load (ETL) platform to ensure data is transformed into suitable formats as defined by IC ITE initiatives. You will interface with external teams and systems, employing various protocols including HTML and SFTP to collect data efficiently. Your responsibilities include enhancing the ETL platform by adding features to shorten timelines for future data integration efforts. Beyond coding tasks, you'll develop and maintain software, ensuring seamless integration into a fully functional system. Collaboration with external teams will be necessary to validate data ingest processes. Finally, you'll be responsible for providing comprehensive documentation covering system architecture, development, and any enhancements made throughout the process.
Required Skills and Qualifications
- Active Top Secret/SCI Clearance with a Polygraph.
- Linux/Unix experience.
- Experience with Databricks or similar tools.
- Experience with Advana or Similar Tool.
- Object-Oriented programming language experience.
- Possess strong verbal and written communication skills.
- Strong Experience with data warehousing and data lakes.
- Self-starter with a willingness to learn new skills.
Desired Skills and Qualifications
- Expertise in data ingestion, data transformation (ETL) and data modeling.
- Experience with Ruby, Java, or Python.
- One year working with RESTful web services.
- Experience with code development, deployment, versioning.
- Working in cloud architecture with AWS EC2, RDS, S3, VPC, ElasticSearch.
- Experience with XML data formats including processing and translation with XSLT and XPath.
- Any experience leading a team.
Education and Experience
- 9 years of professional experience with a Bachelor's degree or equivalent.
- 7 years of professional experience with a Master's degree or equivalent.
- 5 years of professional experience with a PhD or equivalent.
- Additional experience will be considered in lieu of a degree.
