What are the responsibilities and job description for the Software Developer(ETL/Data) with TS/SCI FSP position at 3M Consultancy?
Job Title: Software Developer.
Location: Herndon/Ashburn, VA.
Duration: Full-Time.
NEED TS/SCI FSP Clearance.
Our client is looking for a highly-skilled Software Developer (Systems Software), to work within. The successful candidate will manage the identity data applications that enable exploitation of data and mission-focused collection.
Candidate must be a U.S. Citizen, located in the Washington D.C./Baltimore area and possess an active TS SCI clearance w/ full scope poly.
Primary Responsibilities:
• Develop new software for cloud based ETL tools to support data ingestion.
• Maintain existing and new ETL code baselines.
• Support the Ingestion and ETL team.
• Develop and enhance Ingestion and ETL process tools.
• Develop and enhance utilities for the Ingestion and ETL process.
• Develop code and scripts using Java, JavaScript, Bash, Python and Scala.
• Correct production data at the request of the Data Governance Team and other users.
• Assist with identifying, classifying, and verifying incoming dataset types
• Develop new processes to ETL data.
• Enhance existing processes used to ETL data.
• Attend daily standup scrum meetings to ensure team is focused on current priorities.
• Develop code scripts and software modules on an as needed basis to efficiently triage and convert datasets that arrive in non-standard formats to fit a de-normalized row and column data model.
• Initiate and monitor processes for consolidating, moving, and validating data between customer systems using Linux, spark, yarn, and other processes.
• Convert data between multiple file formats and file systems to include .tar, .zip, parquet, Hadoop, Linux, Windows, and AWS.
• Responsible for storing, standardizing, extracting, referencing, and validating images and image references.
• Document present and future process flows using tools such as PowerPoint and Visio.
• Utilize Linux, Bash, excel, and SQL-like tools such as Postgres or Impala to respond to requests for data dissemination, processing history and metrics.
• Respond to high-priority data processing and data mining requests that may require creative or unorthodox approaches for extracting or identifying critical data.
• Assist other groups or developers in moving and tailoring data to assure that it can be processed correctly and efficiently.
Required (Minimum) Qualifications:
Demonstrated experience utilizing big data technologies such as SPARK, IMPALA, and PYSPARK.
Demonstrated experience using ETL tools such as Pentaho and Kylo.
Demonstrated experience extracting, transforming and loading data from disparate structured data formats into enriched, query-friendly structured data in indexed files.
Demonstrated experience performing data quality analysis.
Demonstrated experience gathering information to determine data sources.
Demonstrated experience with SQL and stored procedure languages such as PL/SQL.
Demonstrated experience with Hbase.
Demonstrated experience with Regular Expressions (RegEx).
Demonstrated experience with Hadoop.
Demonstrated experience developing ETL design documentation including source and target mapping and data dictionary information.
Demonstrated experience with Linux operating systems.
Demonstrated experience programming using languages such as Java, JavaScript, Bash, Python and Scala.
Preferred Qualifications:
Demonstrated experience working with data in foreign language (non-English) scripts such as Unicode.
Demonstrated experience writing scripts to update and transform data.
Demonstrated experience working with large-volume data.
Demonstrated experience working with data sensitivity and handling restrictions.
Demonstrated experience with Postgres database.
Required (Minimum):
Bachelor’s degree in Computer Science, Information Systems, Engineering, Business, or a scientific or technical discipline plus a minimum of 8 years of relevant experience.
Salary : $100
