What are the responsibilities and job description for the Pyspark Tech Lead position at Themesoft Inc.?
Role : Pyspark Tech Lead
Location : Chicago, Illinois(Onsite)
Responsibilities :
- Lead the design and implementation of data solutions using Cloudera HUE and Cloudera Data Platform- Oversee the integration of Apache Iceberg and Spark into existing data workflows
- Provide technical guidance and mentorship to junior team members on PySpark best practices- Collaborate with cross-functional teams to understand data requirements and deliver scalable solutions
- Ensure data quality and integrity through rigorous testing and validation processes- Develop and maintain documentation for data architectures and workflows
- Optimize data processing pipelines for performance and efficiency
- Conduct code reviews to ensure adherence to coding standards and best practices
- Troubleshoot and resolve complex technical issues related to data processing
- Stay updated with the latest advancements in data technologies and incorporate them into projects
- Drive innovation by exploring new tools and techniques in the research and development domain
- Communicate effectively with stakeholders to gather requirements and provide project updates
- Contribute to the overall success of the company by delivering high-quality data solutions that drive business value
Qualification :
Regards
Praveen Kumar
Talent Acquisition Group – Strategic Recruitment Manager
praveen.r@themesoft.com
