What are the responsibilities and job description for the AWS Spark developer position at Acunor Inc?
Position: AWS Spark Developer
Location: Chalotte, NC/Fort Mill, SC
About the Role
Our client is seeking an experienced AWS Spark Developer to join our dynamic team. The ideal candidate will have a strong background in distributed data processing, cloud technologies, and building scalable data pipelines. This role will focus on designing, implementing, and optimizing big data solutions using AWS and Apache Spark to support business insights and decision-making.
Responsibilities
- Develop and maintain scalable data processing pipelines using Apache Spark on AWS.
- Collaborate with cross-functional teams to gather requirements and deliver end-to-end data solutions.
- Optimize Spark jobs for performance and cost-effectiveness in an AWS environment.
- Implement data ingestion, transformation, and integration workflows using AWS services like S3, EMR, Glue, Lambda, and Athena.
- Ensure data quality and reliability through robust testing and monitoring processes.
- Provide technical guidance and support for troubleshooting and resolving issues in data pipelines.
- Stay updated on new AWS and Spark features and recommend their adoption where beneficial.
Qualifications
Must-Have Skills:
- Proven experience in developing Spark-based applications, including Spark SQL and streaming.
- Hands-on experience with AWS services (e.g., EMR, S3, Glue, Lambda, CloudWatch).
- Strong programming skills in Python, Scala, or Java.
- Experience with data modeling, ETL, and distributed computing concepts.
- Knowledge of SQL and experience working with large-scale data.
- Familiarity with CI/CD pipelines and version control tools like Git.
Preferred Skills:
- Experience with AWS Glue Catalog and data lake architectures.
- Knowledge of data warehousing solutions like Redshift or Snowflake.
- Familiarity with DevOps practices in AWS environments.
- Certification in AWS or Spark is a plus.
Educational Requirements
- Bachelor's degree in Computer Science, Information Technology, or a related field.
- Master's degree preferred.
