What are the responsibilities and job description for the Data Engineer position at AMS Staffing, Inc.?
Job Description
Job Description
Job Title : Data Engineer
Division : Manager, Software Development
Location : Washington D.C. (hybrid 2 days onsite, 3 days WFH)
Salary : $110K - $120K Bonus, Excellent Benefits!
Type : Full-Time / Permanent (Exempt)
Sponsorship / C2C : No
Referral Fee : $500
The Data Engineer is responsible for designing, developing, and maintaining data pipelines for data consumption, analytics, and AI. This includes overseeing the data lifecycle, from ingestion to quality assurance. The Data Engineer is a member of product team and will collaborate with cross functional teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering data-driven decision making across the organization.
The Products and Services department is a data and technology organization responsible for leveraging data, artificial intelligence and cloud technologies to enable the organization to fulfill its mission to protect and strengthen the municipal bond market, enabling access to capital, economic growth, and societal progress in tens of thousands of communities across the country.
ESSENTIAL DUTIES AND RESPONSIBILITIES
Maintain optimal data pipeline architecture.
Assemble structured, semi-structure and unstructured data sets to meet functional / non-functional business requirements.
Identify, and implement internal process improvements : automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. DataOps and / or DevOps.
Contribute to the infrastructure required for optimal data extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS bigdata' technologies.
Utilize analytics tools that utilize the data pipeline to provide actionable insights into market insights and transparency, operational efficiency and other key business performance metrics.
Work with stakeholders including the Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
Keep data secure across regional boundaries through multiple data centers and AWS regions.
Create data tools for analytics and data scientist team members that assist in building and optimizing data products.
Support and deploy machine learning models to production.
Ensure data quality and consistency across various sources.
Other duties as assigned.
EDUCATION / QUALIFICATIONS
3-4years of experience in a Data Engineer role. Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
Experience optimizing big data data pipelines, architectures and data sets.
Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
Experience working with structured datasets.
Experience working with unstructured data.
Build processes supporting data transformation, data structures, metadata, dependency and workload management.
A successful history of manipulating, processing and extracting value from large, disconnected datasets.
Working knowledge of message queuing, stream processing, and highly scalable big data data stores.
Experience with AWS cloud services : S3, Athena, Aurora / Postgres, RDS, Redshift, Glue, Lambda, SQS, SNS
Experience with relational SQL and NoSQL databases, including Postgres and DynamoDB.
Experience with scripting languages : Python, R, etc.
Experience in debugging and performance tuning of spark jobs.
dataengineer
postgres
Salary : $110,000 - $120,000
