What are the responsibilities and job description for the AWS Data Architect position at Happiest Minds Technologies Limited?
Job Details
Job Title: Data Engineer
Job Overview:
We are looking for a highly skilled and experienced Software Engineer with a strong background in Python development. The ideal candidate will have 15 years of hands-on experience in software development, with advanced expertise in Spark and AWS Databricks. The successful candidate will be responsible for designing, developing, and deploying high-performance, scalable software solutions while collaborating with cross-functional teams to meet business objectives.
Key Responsibilities:
- Lead the design and development of large-scale, distributed systems using Python.
- Architect and implement Apache Spark applications for data processing and analysis.
- Utilize AWS Databricks to build and maintain data pipelines, performing real-time and batch data processing.
- Develop, optimize, and maintain scalable cloud-based data infrastructure.
- Work closely with data scientists, analysts, and other software engineers to deliver integrated solutions.
- Implement best practices for software development, ensuring high-quality, maintainable code.
- Troubleshoot, debug, and optimize software applications and data pipelines for performance and reliability.
- Contribute to the continuous improvement of development processes and methodologies.
- Mentor and guide junior engineers, fostering knowledge sharing and growth within the team.
- Engage in peer code reviews to ensure the consistency and quality of the codebase.
- Stay up-to-date with emerging trends and technologies in cloud computing, big data, and software engineering.
Required Skills:
- 15 years of experience in software development with a strong focus on Python.
- Extensive experience in designing and developing Apache Spark applications, especially in big data environments.
- Proficiency in building and managing data pipelines using AWS Databricks.
- Expertise in distributed computing, cloud architecture, and large-scale data processing systems.
- Strong understanding of AWS services, particularly in the context of data engineering and data analytics (e.g., S3, EC2, Lambda, Redshift, etc.).
- Solid experience in data modeling, ETL processes, and data transformations.
- Deep knowledge of SQL for querying and manipulating data.
- Strong troubleshooting, debugging, and optimization skills.
- Ability to work independently and as part of a collaborative team.
Preferred Skills:
- Experience with Apache Kafka, Apache Hadoop, or similar data streaming and processing technologies.
- Familiarity with containerization technologies such as Docker and orchestration tools like Kubernetes.
- Experience in Agile/Scrum methodologies.
- Understanding of machine learning principles and frameworks.
- Strong knowledge of DevOps practices and CI/CD pipelines.
Qualifications:
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- Proven track record of designing and building complex, high-performance software applications.
- 15 years of professional experience in a software engineering role, with a focus on Python, Spark, and cloud-based solutions.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
