What are the responsibilities and job description for the Data Engineer - McLean VA position at Janus Soft Inc?
Description
Develop new tools, code, and services to execute data engineering activities. These activities include tasks such as the movement of structured and unstructured data using approved methods and executing data ingestion activities for storing data at a local or enterprise level. The responsibilities also involve viewing data in its source format and developing code to format data that supports exploration.
Additionally, the role requires analyzing source data formats and collaborating with data scientists and partners to determine the formats and transformations that best meet mission objectives. You will need to develop code and tools to provide one-time and ongoing data formatting and transformations into enterprise or boutique data models, implement existing ETL code and best practices / standards, and develop an ETL Code Transition Plan.
Documentation for each project, including ETL mappings, code use guides, code location, and access instructions, must be developed and delivered. Furthermore, the role involves facilitating code reviews and providing consulting services to support data transport, ingestion, conditioning, access, and management.
Required :
- Extract, Transform, and Load (ETL) tools and processes.
- Python
- APIs
- Linux
- Geospatial tools / data Desired :
- Docker
- Jenkins
- Hadoop / Spark
- Kibana
- Kafka
- NiFi
- ElasticSearch
