You haven't searched anything yet.
Additional Job Information:
Title : Big Data Engineer
Location : Atlanta, GA
Description:
Big Data Engineers are responsible for architecture, design and build of Big Data applications to support business strategies and deliver business value. A data engineer participates in all phases of the Data Engineering life cycle and will independently and collaboratively write project requirements, architect solutions, and perform data ingestion development and support duties. Building data ingestion pipelines is highly technical and requires advanced skills with computers and proficiency with Big Data tools and technologies such as SQL, Python/Scala, Spark, Spark Structured Streaming, Spark SQL, Kafka, Sqoop, Hive, Kudu, HBASE, Impala, S3, HDFS, or Cloud platforms such as AWS, Google Cloud Platform, Azure, etc. Data Engineers require ability to communicate effectively, both written and verbal, and have extensive experience working with business areas to translate their business data needs and data questions into project requirements.
Job Description
Client Corporation is currently seeking an experienced Data Engineer Big Data individual for their Midtown office in Atlanta, GA. The successful candidate must have Big Data engineering experience and must demonstrate an affinity for working with others to create successful solutions. Join a smart, highly skilled team with a passion for technology, where you will work on our state of the art Big Data Platforms. They must be a very good communicator, both written and verbal, and have some experience working with business areas to translate their business data needs and data questions into project requirements. The candidate will participate in all phases of the Data Engineering life cycle and will independently and collaboratively write project requirements, architect solutions and perform data ingestion development and support duties.
Skills and Experience:
Required :
6 years of overall IT experience
3 years of experience with high-velocity high-volume stream processing: Apache Kafka and Spark Streaming
3 years of experience with data ingestion from Message Queues (Tibco, IBM, etc.) and different file formats across different platforms like JSON, XML, CSV
3 years of experience with Big Data tools/technologies like Hadoop, Spark, Spark SQL, Kafka, Sqoop, Hive, S3, HDFS, or
3 years of experience building, testing, and optimizing Big Data data ingestion pipelines, architectures, and data sets
2 years of experience with Python (and/or Scala) and PySpark/Scala-Spark
3 years of experience with Cloud platforms e.g. AWS, Google Cloud Platform, etc.
3 years of experience with database solutions like Kudu/Impala, or Delta Lake or Snowflake or BigQuery
2 years of experience with NoSQL databases, including HBASE and/or Cassandra
Experience in successfully building and deploying a new data platform on Azure/ AWS
Experience in Azure / AWS Serverless technologies, like, S3, Kinesis/MSK, lambda, and Glue
Strong knowledge of Messaging Platforms like Kafka, Amazon MSK & TIBCO EMS or IBM MQ Series
Experience with Databricks UI, Managing Databricks Notebooks, Delta Lake with Python, Delta Lake with Spark SQL, Delta Live Tables, Unity Catalog
Knowledge of Unix/Linux platform and shell scripting is a must
Strong analytical and problem-solving skills
Preferred (Not Required):
Strong SQL skills with ability to write intermediate complexity queries
Strong understanding of Relational & Dimensional modeling
Experience with GIT code versioning software
Experience with REST API and Web Services
Good business analyst and requirements gathering/writing skills
Qualification:
Bachelor s Degree required. Preferably in Information Systems, Computer Science, Electrical Engineering, Computer Information Systems or related field
For Big Data Engineer position:
4.Experience with Traditional ETL tools and Data Modeling
Best Regards,
Sethu Mathan, MBA
Senior Manager, IT Strategic Solution
Services You Can Trust
M:
E: |
7000 Peachtree Dunwoody Rd | Bldg. 11, Suite 301
Atlanta, GA 30328
Full Time
$112k-137k (estimate)
06/25/2024
07/10/2024
aaaglobaltech.com
Alpharetta, GA
<25
Insurance
The following is the career advancement route for Big Data Engineer positions, which can be used as a reference in future career path planning. As a Big Data Engineer, it can be promoted into senior positions as a Database Engineer V that are expected to handle more key tasks, people in this role will get a higher salary paid than an ordinary Big Data Engineer. You can explore the career advancement for a Big Data Engineer below and select your interested title to get hiring information.