Senior Data Engineer

MWBE Assignment 7-858-0557A - MyCity ODA Senior Data Engineer

You are invited to respond to the following RFx:

• RFx name : 85825Y0990-7-858-0557A - MWBE MyCity ODA Senior Data Engineer - PR3
• Agency : DEPARTMENT OF INFORMATION TECHNOLOGY AND TELECOMMUNICATIONS
• EPIN : 85825Y0990
• Main commodity : IT Consulting
• Other Commodity/ies:
• Round # : 1
• Due Date : 2024-12-11

Job Description
Justification : The NYC Office of Technology and Innovation has been tasked with recruiting a Programmer III, Senior Data Engineer. MyCity Portal will streamline the public's interaction with NYC government in applying for services and benefits. The Portal is envisioned to create a unified digital platform to easily search, find, and apply for services and benefits. It will allow users to create an online profile that will provide status updates and notifications. This platform will enable information sharing and streamline business processes between agencies to improve customer experience.

The MWBE resource will support the Office of Data Analytics (ODA) with NYC's Office of Technology and Innovation (OTI) implement its data management strategy for MyCity. This will include developing pipelines from MyCity and related data sources to OTI's various analytic and data integration platforms. Assignment Title: 7-858-0557A MyCity ODA Senior Data Engineer Labor Category Programmer 3 Work Location: Select one and provide required details: On Site
Work Location: Hybrid: Work location & Remote ( X days in office/ X days remote) X Remote: Monday through Friday; 9 am to 6 pm
Scheduled Work Hours: Normal business hours Monday-Friday 40 hours/week (not including mandatory unpaid meal break after 6 hours of work). Projected Assignment Start Date: 12/30/2024 Projected Assignment End Date: 12/28/2025 Total Estimated Assignment Hours: 2080 Hours (1 Year) Max Hour Rate $ 157.32
Note: Normal Business Hours, Monday through Friday (not including a mandatory unpaid meal break after 6 hours of work), 40 work hours per week. If the consultant works more than 40 hours per week, the consultant must request overtime in the Agency's timekeeping system and the project manager must approve those hours worked above the weekly maximum.

SCOPE OF SERVICES
OTI's Office of Data Analytics (ODA) works with City agencies and their data to help serve New Yorkers more equitably and effectively. This work takes three main forms: analyzing data to improve City agency operations; making Open Data more accessible to all New Yorkers; and advancing citywide data infrastructure, integration, and sharing.

ODA seeks a Data Engineer role to ensure the efficient and successful implementation and support of complex data engineering solutions for City agencies. This resource should demonstrate a solid understanding of industry-standard implementation methodologies using data engineering technologies, tools, and processes.

TASKS:

• Create and maintain optimal data pipeline architecture that is coherent and scalable, based on best practices of integrating data into a consolidated repository.
• Perform the technical design, development, and component testing of repository changes.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer engagement and experience, operational efficiency, and other key business performance metrics.
• Build the infrastructure required for optimal extraction, transformation, and loading (ETL) of data from a wide variety of data sources using SQL, cloud, and 'big data' technologies.
• Develop ETLs to move data securely from source to target systems.
• Create, update, and maintain system documentation.
• Develop new or build against existing APIs for data access or landing data as output for further downstream consumption in the appropriate target data store.
• Perform special projects and initiatives as assigned.

MANDATORY SKILLS/EXPERIENCE
Note: Candidates who do not have the mandatory skills will not be considered

• At least 8 years of experience in writing SQL.
• A minimum of 8 years of experience in copying, transferring, manipulating, and automating data operations that were manual processes.
• Experience with tools and components of data architecture such as Informatica Power Center, IICS, SSIS, or similar ETL tools.
• Experience working with Amazon Web Services or Microsoft Azure cloud computing platform and services.
• In-depth knowledge of SQL and other database solutions.
• Experience with data warehousing (Snowflake, Redshift etc.).
• Knowledge of modeling database schemas for large datasets.
• Experience developing cloud-ready applications.
• Experience working with programming languages like Python, Java, and Perl

DESIRABLE SKILLS/EXPERIENCE:

• Hands on experience developing Microsoft PowerBI solutions.
• 5 years hands-on experience in development with the suite of tools from Informatica PowerCenter and B2B Data Transformation.
• Experience using Oracle 10g/11g, SQL Server and/or a database appliance.
• Knowledge of metadata-driven enterprise reporting platforms.

SPECIAL REQUIREMENTS: N/A