161 Big Data Engineer (Various Levels) Jobs in Atlanta, GA
Ricefw Technologies is Hiring a Big Data Engineer (Various Levels) Near Atlanta, GA
Job Posting Title: Big Data EngineerCompanyClient Corporation is Fortune-300 transportation company specializing in freight railroading. We operate approximately 21,000 route miles in 22 states and the District of Columbia, serve every major container port in the eastern United States, and provide efficient connections to other rail carriers. Client has the most extensive intermodal network in the East and is a major transporter of coal and industrial products.
Job DescriptionClient Corporation is currently seeking an experienced Data Engineer Big Data individual for their Midtown office in Atlanta, GA. The successful candidate must have Big Data engineering experience and must demonstrate an affinity for working with others to create successful solutions. Join a smart, highly skilled team with a passion for technology, where you will work on our state of the art Big Data Platforms. They must be a very good communicator, both written and verbal, and have some experience working with business areas to translate their business data needs and data questions into project requirements. The candidate will participate in all phases of the Data Engineering life cycle and will independently and collaboratively write project requirements, architect solutions and perform data ingestion development and support duties.
RequiredSkills and Experience:
- 6 years of overall IT experience
- 3 years of experience with high-velocity high-volume stream processing: Apache Kafka and Spark Streaming
- Experience with real-time data processing and streaming techniques using Spark structured streaming and Kafka
- Deep knowledge of troubleshooting and tuning Spark applications
- 3 years of experience with data ingestion from Message Queues (Tibco, IBM, etc.) and different file formats across different platforms like JSON, XML, CSV
- 3 years of experience with Big Data tools/technologies like Hadoop, Spark, Spark SQL, Kafka, Sqoop, Hive, S3, HDFS, or
- 3 years of experience building, testing, and optimizing 'Big Data' data ingestion pipelines, architectures, and data sets
- 2 years of experience with Python (and/or Scala) and PySpark/Scala-Spark
- 3 years of experience with Cloud platforms e.g. AWS, GCP, etc.
- 3 years of experience with database solutions like Kudu/Impala, or Delta Lake or Snowflake or BigQuery
- 2 years of experience with NoSQL databases, including HBASE and/or Cassandra
- Experience in successfully building and deploying a new data platform on Azure/ AWS
- Experience in Azure / AWS Serverless technologies, like, S3, Kinesis/MSK, lambda, and Glue
- Strong knowledge of Messaging Platforms like Kafka, Amazon MSK & TIBCO EMS or IBM MQ Series
- Experience with Databricks UI, Managing Databricks Notebooks, Delta Lake with Python, Delta Lake with Spark SQL, Delta Live Tables, Unity Catalog
- Knowledge of Unix/Linux platform and shell scripting is a must
- Strong analytical and problem-solving skills
- Strong SQL skills with ability to write intermediate complexity queries
- Strong understanding of Relational & Dimensional modeling
- Experience with GIT code versioning software
- Experience with REST API and Web Services
- Good business analyst and requirements gathering/writing skills
Required Skills : Data Warehouse
Additional Skills : Big DataThis is a high PRIORITY requisition. This is a PROACTIVE requisition
Job Summary
Full Time
$112k-138k (estimate)
06/27/2024
07/25/2024
ricefwtech.com
Okemos, MI
25 - 50
Related Companies
The job skills required for Big Data Engineer (Various Levels) include SQL, Python, Computer Science, Problem Solving, Data Warehouse, etc. Having related job skills and expertise will give you an advantage when applying to be a Big Data Engineer (Various Levels). That makes you unique and can impact how much salary you can get paid. Below are job openings related to skills required by Big Data Engineer (Various Levels). Select any job title you are interested in and start to search job requirements.
The following is the career advancement route for Big Data Engineer (Various Levels) positions, which can be used as a reference in future career path planning. As a Big Data Engineer (Various Levels), it can be promoted into senior positions as a Database Engineer V that are expected to handle more key tasks, people in this role will get a higher salary paid than an ordinary Big Data Engineer (Various Levels). You can explore the career advancement for a Big Data Engineer (Various Levels) below and select your interested title to get hiring information.
Similar Jobs
Popular Articles
