Demo

Sr. Data Engineer

Veracity Software Inc
Charlotte, NC Full Time
POSTED ON 2/27/2025
AVAILABLE BEFORE 4/23/2025
Job Title: Sr. Data Engineer

Duration: 12 Months

Location: Charlotte, NC - Hybrid Role (3 days onsite in a week)

Manager Notes:

  • Team Duties and business impact: The Risk Data Services is a horizontal function within Risk Technology organization and is responsible for delivering data consistently across Risk. The Risk Data Services team is seeking a Lead Software Engineer. Driving strategy for the team for entire platform and help with migration to cloud.
  • Big Data Hub, the tools are used purely for ETL functions, moving and conforming data, not for building web applications. The use of Python and Java is for Data Engineering, we are not seeking Web Developers, back and front end etc.

Requirements:

  • This is a need for someone to build and write the capabilities.
  • Experience building and enhance scala framework and spark experience
  • Java
  • Python

Job Expectations:

  • Design and implement automated spark-based framework to facilitate data ingestion, transformation and consumption.
  • Implement security protocols such as Kerberos Authentication, Encryption of data at rest, data authorization mechanism such as role-based access control using Apache ranger.
  • Design and develop automated testing framework to perform data validation.
  • Enhance existing spark-based frameworks to overcome tool limitations, and/or to add more features based on consumer expectations.
  • Design and build high performing and scalable data pipeline platform using Hadoop, Apache Spark, MongoDB, Kafka and object storage architecture.
  • Work with Infrastructure Engineers and System Administrators as appropriate in designing the big-data infrastructure
  • Collaborate with application partners, Architects, Data Analysts and Modelers to build scalable and performant data solutions.
  • Effectively work in a hybrid environment where legacy ETL and Data Warehouse applications and new big-data applications co-exist
  • Work with Infrastructure Engineers and System Administrators as appropriate in designing the big-data infrastructure.
  • Support ongoing data management efforts for Development, QA and Production environments
  • Provide tool support, help consumers troubleshooting pipeline issues.
  • Utilizes a thorough understanding of available technology, tools, and existing designs.
  • Leverage knowledge of industry trends to build best in class technology to provide competitive advantage.

Required Qualification:

  • 5 years of experience of software engineering experience
  • 5 years of experience delivering complex enterprise wide information technology solutions
  • 5 years of experience delivering ETL, data warehouse and data analytics capabilities on big-data architecture such as Hadoop
  • 5 years of Apache Spark design and development experience using Scala, Java, Python or Data Frames with Resilient Distributed Datasets (RDDs), Parquet or ORC file formats
  • 6 years of ETL (Extract, Transform, Load) Programming experience
  • 2 years of Kafka or equivalent experience
  • 2 years of NoSQL DB like Couchbase/MongoDB experience.
  • 5 experience working with complex SQLs and performance tuning

Desired Qualification:

  • 3 years of Agile experience
  • 2 years of reporting experience, analytics experience or a combination of both
  • 2 years of operational risk or credit risk or compliance domain experience
  • 2 years of experience integrating with RESTful API
  • 2 years of experience with CICD tools.

If your compensation planning software is too rigid to deploy winning incentive strategies, it’s time to find an adaptable solution. Compensation Planning
Enhance your organization's compensation strategy with salary data sets that HR and team managers can use to pay your staff right. Surveys & Data Sets

What is the career path for a Sr. Data Engineer?

Sign up to receive alerts about other jobs on the Sr. Data Engineer career path by checking the boxes next to the positions that interest you.
Income Estimation: 
$122,257 - $154,284
Income Estimation: 
$143,391 - $179,890
Income Estimation: 
$123,167 - $152,295
Income Estimation: 
$146,673 - $180,130
Income Estimation: 
$146,673 - $180,130
Income Estimation: 
$176,149 - $220,529
Income Estimation: 
$92,929 - $122,443
Income Estimation: 
$122,257 - $154,284
Income Estimation: 
$122,257 - $154,284
Income Estimation: 
$143,391 - $179,890
View Core, Job Family, and Industry Job Skills and Competency Data for more than 15,000 Job Titles Skills Library

Job openings at Veracity Software Inc

Veracity Software Inc
Hired Organization Address Chicago, IL Full Time
RN - Emergency (SANE) - Chicago, Illinois Job Type: Full-time or Part Time Experience: 3 - 20 Years Must-Haves : Associa...
Veracity Software Inc
Hired Organization Address Decatur, IL Full Time
Decatur, Illinois Full Time Experience: 3 - 15 Years Must-Haves High school diploma or general education degree (GED) Tw...
Veracity Software Inc
Hired Organization Address Chicago, IL Full Time
RN - ICU (Nights) - Chicago, Illinois Job Type: Full-time or Part Time Experience: 2 - 20 Years Must-Haves: ADN, BSN or ...
Veracity Software Inc
Hired Organization Address Chicago, IL Full Time
CT Technician (Nights) - Chicago, Illinois Job Type: Full-time / Part time Experience: 2 - 20 Years Must-Haves : High sc...

Not the job you're looking for? Here are some other Sr. Data Engineer jobs in the Charlotte, NC area that may be a better fit.

Sr. Chemical Process Engineer

Sr. Chemical Process Engineer, Huntersville, NC

Cyber Security Sr. Engineer

NTT DATA North America, Charlotte, NC

AI Assistant is available now!

Feel free to start your new journey!