Staff Engr Software

Description : Temp to perm

Bill rate $130

Remote

About the role :

As the Senior Software Engineer you will lead a team of data engineers in designing building and maintaining highperformance software system to manage analytical data pipelines that fuel the organization s data strategy using software engineering best practices. Beyond technical expertise you will also serve as a change leader guiding teams through adopting new tools technologies and workflows to improve data management and processing.

This position requires extensive handson data system design and coding experience as well as the development of modern data pipelines (AWS Step functions Prefect Airflow Luigi Python Spark SQL) and associated code in AWS.

You will work closely with stakeholders across the business to understand their data needs ensure scalability and foster a culture of innovation and learning within the data engineering team and beyond.

Key Responsibilities :

Be responsible for the overall architecture of a specific module within a product (e.g. Dataingestion nearrealtimedataprocessor etc.) perform design and assist implementation considering system characteristics to produce optimal performance reliability and maintainability.

Provide technical guidance to team members ensuring they are working towards the products architectural goals.

Create and manage RFCs (Request for Comments) and ADRs (Architecture Decision Records) Design notes and technical documentation for your module following the architecture governance processes.

Lead a team of data engineers providing mentorship setting priorities and ensuring alignment with business goals.

Architect design and build scalable data pipelines for processing large volumes of structured and unstructured data from various sources.

Collaborate with software engineers architects and product teams to design and implement systems that enable realtime and batch data processing at scale.

Be the goto person for PySparkbased solutions ensuring optimal performance and reliability for distributed data processing.

Ensure that data engineering systems adhere to the best data security privacy and governance practices in line with industry standards.

Perform code reviews for the product ensuring adherence to company coding standards and best practices.

Develop and implement monitoring and alerting systems to ensure timely detection and resolution of data pipeline failures and performance bottlenecks.

Act as a champion for new technologies helping ease transitions and addressing concerns or resistance from team members.

Ideal Candidate :

Experience leading a data engineering team with a strong focus on software engineering principles such as KISS DRY YAGNI etc.

Candidate MUST have experience in owning large complex system architecture and handson experience designing and implementing data pipelines across largescale systems.

Experience implementing and optimizing data pipelines with AWS is a must.

Production delivery experience in Cloudbased PaaS Big Data related technologies (EMR Snowflake Data bricks etc.)

Experienced in multiple Cloud PaaS persistence technologies and indepth knowledge of cloud based ETL offerings and orchestration technologies (AWS Step Function Airflow etc.)

Experienced in streambased and batch processing applying modern technologies

Working experience with distributed file systems (S3 HDFC ADLS) table formats (HUDI Iceberg) and various open file formats (JSON Parquet Csv etc.)

Strong programming experience in PySpark SQL Python etc.

Database design skills including normalization / denormalization and data warehouse design

Knowledge and understanding of relevant legal and regulatory requirements such as SOX PCI HIPAA Data Protection

Experience in the healthcare industry a plus

A collaborative and informative mentality is a must!

Toolset :

AWS preferably AWS certified Data Engineer and AWS certified Solutions Architect.

Proficiency in at least one programming language C# GoLang JavaScript or ReactJs

Spark / Python / SQL

Snowflake / Databricks / Synapse / MS SQL Server

ETL / Orchestration Tools (Step Function DBT etc.)

ML / Notebooks

Education and experience required

Bachelors or Master s in Computer Science Information Systems or an engineering field or relevant experience.

10 years of related experience in developing data solutions and data movement.

This role can be REMOTE

Industry Title

Additional Details

• Exempt / NonExempt Status : NonExempt
• Industry Title :
• Blank Template Professional
• Job Group : 4
• Project Name : Flex
• Contract to hire position : Yes
• Shift : N / A
• Screening Requirements : N / A

Salary : $130