What are the responsibilities and job description for the Data Architect with Pyspark and Cloudera HUE position at Galent?
Job title- Data Architect with Pyspark and Cloudera HUE
Location- Chicago, IL (5 days onsite)
Job Description/ Responsibilities:
We are seeking a Technical Lead with 10 years of experience to join our 1eam. The ideal candidate will have expertise in Cloudera HUE, Cloudera Data Platform, Apache Iceberg, Spark, and Park. This role involves working in a hybrid model with day shifts.
The candidate will play a crucial role in driving our research and development initiatives, leveraging their technical skills to deliver impactful solutions.
Required Skills: Cloudera HEU, Pyspark, Spark, Pyspark Cloudera Data Platform, Apache Iceberg
Responsibilities:-
- Lead the design and implementation of data solutions using Cloudera HUE and Cloudera Data Platform.
- Oversee the integration of Apache Iceberg and Spark into existing data workflows- Provide technical guidance and mentorship to junior team members on Pyspark best practices
- Collaborate with cross-functional teams to understand data requirements and deliver scalable solutions- Ensure data quality and integrity through rigorous testing and validation processes- Develop and maintain
- documentation for data architectures and workflows
- Optimize data processing pipelines for performance and efficiency- Conduct code reviews to ensure adherence to coding standards and best practices
- Troubleshoot and resolve complex technical issues related to data processing
- Stay updated with the latest advancements in data technologies and incorporate them into projects- Drive innovation by exploring new tools and techniques in the research and development domain- Communicate effectively with stakeholders to gather requirements and provide project updates
- Contribute to the overall success of the company by delivering high-quality data solutions that drive business value
Qualifications-
- Possess strong expertise in Cloudera HUE and Cloudera Data Platform- Demonstrate proficiency in Apache Iceberg and Spark
- Have extensive experience with Pyspark for data processing- Show a solid understanding of data architecture and data engineering principles
- Exhibit excellent problem-solving skills and attention to detail
- Have a background i n research and development is a plus- Display strong communication and collaboration skills- Be able to work effectively in a hybrid work model
- Have a proactive approach to learning and staying current with industry trends- Be capable of mentoring and guiding junior team members- Demonstrate the ability to deliver high-quality solutions within deadlines
- Show a commitment to continuous improvement and innovation- Possess a strong sense of ownership and accountability for project outcomes.
