Demo

PySpark Technical Lead

The Hartford
Chicago, IL Full Time
POSTED ON 1/22/2025
AVAILABLE BEFORE 4/20/2025

Staff Data Engineer - GE07CE

We’re determined to make a difference and are proud to be an insurance company that goes well beyond coverages and policies. Working here means having every opportunity to achieve your goals – and to help others accomplish theirs, too. Join our team as we help shape the future.

Join a fast-paced and talented team to deliver Data Engineering capabilities for The Hartford’s Commercial Data Science Data Delivery. You will have an opportunity to engage in enabling well architected cloud-based data solutions for Entity Resolution using emerging technologies such as AWS and Snowflake.

Make your application after reading the following skill and qualification requirements for this position.

This role will have a Hybrid work arrangement, with the expectation of working in an office (Hartford, CT, Chicago, IL, Columbus, OH, Charlotte, NC) 3 days a week (Tuesday through Thursday).

Responsibilities :

  • Accountable as the subject matter expert and / or technical lead for a large-scale data products. Drive End-to-End solution delivery involving multiple platforms and technologies with medium to large complexity or oversee certain parts of very large complex implementations, leveraging ELT solutions to acquire, integrate, and operationalize data.
  • Partner with architects and stakeholders to influence and implement the vision of the pipeline and data product architecture while safeguarding the integrity and scalability of the environment.
  • Articulate risks and tradeoffs of technology solutions to senior leaders with translations as needed for business leaders.
  • Accountable for data pipeline and product physical solution designs across teams as well as tool recommendations.
  • Accountable for Data Engineering Practices across all the teams involved.
  • Implement and utilize leading big data methodologies (AWS, Hadoop / EMR, Spark, Kafka, Snowflake and Talend) with cloud / on premise hybrid hosting solutions, on a multi-team / product level.

Knowledge, Skills, and Abilities :

  • Strong Technical Knowledge (Cloud data pipelines and data consumption products).
  • Leader and a team player with a transformation mindset.
  • Ability to lead successfully in a lean, agile, and fast-paced organization, leveraging Scaled Agile principles and ways of working.
  • Guides team to mature Code quality management, DataOps principles, automated testing, and environment management practices to deliver incremental customer value.
  • Qualifications :

  • 5 years in large scale big data engineering experience and designing best practices in Programming, SDLC practices, Distributed systems, Data warehousing solutions SQL and NoSQL, ETL tools, CICD, Cloud Technologies (AWS / AZURE), Python / Spark, Datamesh and Datalake, Data Fabric.
  • 3 years of operating in a technical leadership capacity for 2 teams.
  • 3 years of developing and operating production workloads in cloud infrastructure (AWS, Azure, etc).
  • Exposure to AWS best practices.
  • Knowledge of core functional components / services of AWS – compute, storage, Edge, Database, Migration and Transfer, Networking, and Governance.
  • Candidate must be authorized to work in the US without company sponsorship. The company will not support the STEM OPT I-983 Training Plan endorsement for this position.
  • Certifications / Licenses (as applicable)

  • Cloud certifications preferred.
  • Compensation

    The listed annualized base pay range is primarily based on analysis of similar positions in the external market. Actual base pay could vary and may be above or below the listed range based on factors including but not limited to performance, proficiency and demonstration of competencies required for the role. The base pay is just one component of The Hartford’s total compensation package for employees. Other rewards may include short-term or annual bonuses, long-term incentives, and on-the-spot recognition. The annualized base pay range for this role is : $123,280 - $184,920.

    Equal Opportunity Employer / Females / Minorities / Veterans / Disability / Sexual Orientation / Gender Identity or Expression / Religion / Age

    About Us | Culture & Employee Insights | Diversity, Equity and Inclusion | Benefits

    J-18808-Ljbffr

    Salary : $123,280 - $184,920

    If your compensation planning software is too rigid to deploy winning incentive strategies, it’s time to find an adaptable solution. Compensation Planning
    Enhance your organization's compensation strategy with salary data sets that HR and team managers can use to pay your staff right. Surveys & Data Sets

    What is the career path for a PySpark Technical Lead?

    Sign up to receive alerts about other jobs on the PySpark Technical Lead career path by checking the boxes next to the positions that interest you.
    Income Estimation: 
    $96,448 - $126,269
    Income Estimation: 
    $121,314 - $156,976
    Income Estimation: 
    $71,122 - $96,652
    Income Estimation: 
    $92,929 - $122,443
    Income Estimation: 
    $92,929 - $122,443
    Income Estimation: 
    $122,257 - $154,284
    Income Estimation: 
    $122,257 - $154,284
    Income Estimation: 
    $143,391 - $179,890
    View Core, Job Family, and Industry Job Skills and Competency Data for more than 15,000 Job Titles Skills Library

    Job openings at The Hartford

    The Hartford
    Hired Organization Address Lake, FL Full Time
    Medical Case Manager - CT08GE We’re determined to make a difference and are proud to be an insurance company that goes w...
    The Hartford
    Hired Organization Address Hartford, CT Full Time
    Sr Data Scientist - GD07AEWe’re determined to make a difference and are proud to be an insurance company that goes well ...
    The Hartford
    Hired Organization Address Hartford, CT Full Time
    Managing Assistant General Counsel, Latent Property Damage Coverage Unit Managing Assistant General Counsel, Latent Prop...
    The Hartford
    Hired Organization Address Windsor, CT Full Time
    Join our Workers Compensation line of business pricing actuarial team, supporting a $3 B book across both Small (SC) and...

    Not the job you're looking for? Here are some other PySpark Technical Lead jobs in the Chicago, IL area that may be a better fit.

    Pyspark Technical Lead

    Cognizant, Chicago, IL

    PySpark Technical Lead

    The Hartford Financial Services Group, Chicago, IL

    AI Assistant is available now!

    Feel free to start your new journey!