Big Data Engineer

Job Description

We are looking to fill a Long-term Contract role as Big Data Engineer in Chesterfield, VA

Title: Big Data Engineer

Mode of Interview: In person

Work Arrangement: Onsite

Applicants must have a minimum of 5 years of relevant experience with the following:

• Must have experience in development of applications using Spark and Scala, Python languages on Hadoop
• A solid hands on experience and understanding of architecting, designing and operationalization of large scale data and analytics solutions on Snowflake Cloud Data Warehouse
• Cloud engineer/development experience on AWS/EMR cluster: Converting Spark application base on Hadoop to Snowflake.
• Proven experience on Hadoop/Big Data migration to Snowflake: Having Spark and Spark SQL experience is critical to know what Snowflake connectors used in the conversion.
• Expected to have experience in using Lambda functions and code development in Python
• Developing ETL pipelines in and out of data warehouse using combination of Python and Snowflakes SnowSQL Writing SQL queries against Snowflake.
• Developing scripts Unix, Python etc. to do Extract, Load and Transform data.
• Working knowledge of AWS Redshift Provide production support for Data Warehouse issues such data load problems, transformation translation problems. Translate requirements for BI and Reporting to Database design and reporting design
• Proven experience in architecting and implementing very large scale data intelligence solutions around Snowflake Data Warehouse
• Understanding data transformation and translation requirements and which tools to leverage to get the job done.
• Understanding data pipelines and modern ways of automating data pipeline using cloud based Testing and clearly document implementations