Demo
Salary Resume Critique Job Openings

Data Engineer Apache Spark AWS Python

LGL Technologies
Plano, TX Full Time
POSTED ON 2/19/2025
AVAILABLE BEFORE 5/15/2025

Data Engineer (Developer III) Apache Spark AWS Python

Location : Plano TX

Duration : 910 months

Description :

  • Job consists of setting up Change Data Capture (or CDC) for multiple types of databases for the purpose of hydrating a data lake.
  • Debezium or other CDC knowledge required.
  • Along with data hydration job requires knowledge on ETL transformations using Apache spark both streaming and batch processing of data.
  • Engineer needs to know how to work with Apache Spark Data Frames ETL jobs and streaming data pipelines that will orchestrate raw CDC data and transform it into useable and queryable data for analytics.
  • Big Data concepts including performance tuning is a plus.

Skill Set :

  • Java Mid to Senior level experience
  • Python Mid level experience (pyspark)
  • Apache Spark Data Frames Spark SQL Spark Streaming and ETL pipelines
  • Apache Airflow
  • Scala not required but a plus
  • Apache Hudi not required but a plus
  • Apache Griffin not required but a plus

    • AWS Skillset :

  • Extensive knowledge with S3 and S3 operations (CRUD)
  • EMR & EMR Serverless
  • Glue Data Catalog
  • Step Functions
  • MWAA (Managed Workflows Apache Airflow)
  • Lambdas (Python)
  • AWS Batch
  • AWS Deequ not required but a plus

    • Key Skills

    Apache Hive,S3,Hadoop,Redshift,Spark,AWS,Apache Pig,NoSQL,Big Data,Data Warehouse,Kafka,Scala

    Employment Type : Full Time

    Vacancy : 1

    If your compensation planning software is too rigid to deploy winning incentive strategies, it’s time to find an adaptable solution. Compensation Planning
    Enhance your organization's compensation strategy with salary data sets that HR and team managers can use to pay your staff right. Surveys & Data Sets

    What is the career path for a Data Engineer Apache Spark AWS Python?

    Sign up to receive alerts about other jobs on the Data Engineer Apache Spark AWS Python career path by checking the boxes next to the positions that interest you.
    Income Estimation: 
    $92,929 - $122,443
    Income Estimation: 
    $122,257 - $154,284
    Income Estimation: 
    $92,929 - $122,443
    Income Estimation: 
    $122,257 - $154,284
    Income Estimation: 
    $122,257 - $154,284
    Income Estimation: 
    $143,391 - $179,890
    Income Estimation: 
    $71,122 - $96,652
    Income Estimation: 
    $92,929 - $122,443
    Employees: Get a Salary Increase
    View Core, Job Family, and Industry Job Skills and Competency Data for more than 15,000 Job Titles Skills Library

    Job openings at LGL Technologies

    Senior Application Security Engineer
    LGL Technologies
    Hired Organization Address New York, NY Full Time
    Senior Application Security Engineer - C2C Location: New York, NY Duration: 12 Months 35 hour per Week Job Description: ...
    Oracle EBS WMS Cloud - W2
    LGL Technologies
    Hired Organization Address Cincinnati, OH Full Time
    Oracle EBS WMS Cloud - W2 Role Location: Cincinnati, OH Hybrid Ideal candidate must have 12 years experience. Job Summar...
    AWS Architect
    LGL Technologies
    Hired Organization Address Atlanta, GA Full Time
    Job Details Job Title: Sr. Architect - Niche Job ID: 39777 Location: Atlanta, GA 30354 (hybrid) Duration: 12 Contract Ca...
    Senior JavaJ2EE Developer
    LGL Technologies
    Hired Organization Address Mc Lean, VA Full Time
    Senior Java / J2EE Developer Fulltime Role Location : McLean VA (Hybrid) Duration : 12 Months Note : Experince required ...
    View More Jobs at LGL Technologies

    Not the job you're looking for? Here are some other Data Engineer Apache Spark AWS Python jobs in the Plano, TX area that may be a better fit.

    Data Engineer (Apache Spark, AWS, Python)

    LGL Technologies, Plano, TX

    Data Tech Lead (Apache Spark, Pyspark and Databricks Architecture)

    NTT DATA, Plano, TX

    View More Jobs

    AI Assistant is available now!

    Feel free to start your new journey!