What are the responsibilities and job description for the Data Engineer II position at VDart Inc?
Role: Data Engineer II
Location: San Diego, CA Onsite
Duration: 6 Months
Job Description:
Responsibilities:
Assembling large, complex sets of data that meet non-functional and functional business requirements Identifying, designing and implementing internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery.
Qualifications:
Ability to build and optimize data sets, 'big data' data pipelines and architectures Ability to perform root cause analysis on external and internal processes and data to identify opportunities for improvement and answer questions Excellent analytic skills associated with working on unstructured datasets Ability to build processes that support data transformation, workload management, data structures, dependency and metadata Building required infrastructure for optimal extraction, transformation and loading of data from various data sources using AWS and SQL technologies Building analytical tools to utilize the data pipeline, providing actionable insight into key business performance metrics including operational efficiency and customer acquisition Working with stakeholders including data, design, product and executive teams and assisting them with data-related technical issues Working with stakeholders including the Executive, Product, Data and Design teams to support their data infrastructure needs while assisting with data-related technical issues
In addition to these the best candidates will be have:
- Decent data analysis skills, including the ability to programmatically extract, categorize, and search the log data.
- Skills using REST APIs from some programming language (doesn't really matter which one), because we need to update/delete the GH data, and it has to be done programmatically, because there's far too much of it to do manually.
- Good communication skills, because it's important to design a plan of attack, document it, and review it with stakeholders and security before starting the effort of fixing the data.
- Understanding how to assess and communicate the various risks.
