Apache Flink Streaming Developer

Job Summary:

We are seeking a highly skilled and motivated Lead Software Engineer with deep expertise in Apache Flink to lead the development and optimization of our real-time streaming applications. You will be responsible for designing, building, and maintaining scalable and robust data pipelines that process high-volume, high-velocity data. You will provide technical leadership, mentor junior engineers, and drive best practices in stream processing.

Responsibilities:

• Design and Development:
• Architect, design, and implement high-performance, fault-tolerant, and scalable real-time streaming applications using Apache Flink.
• Develop and optimize Flink pipelines for data ingestion, transformation, and analysis.
• Write clean, efficient, and well-documented code.
• Implement and maintain CI/CD pipelines for streaming applications.
• Technical Leadership:
• Lead technical design discussions and code reviews.
• Mentor and guide junior engineers, fostering a collaborative and learning environment.
• Define and enforce coding standards and best practices for stream processing.
• Evaluate and recommend new technologies and tools to improve streaming capabilities.
• Performance and Optimization:
• Monitor and optimize the performance of streaming applications.
• Troubleshoot and resolve complex performance and stability issues.
• Conduct performance testing and benchmarking.
• Tune Flink configurations for optimal resource utilization.
• Collaboration and Communication:
• Work closely with product managers, data scientists, and other stakeholders to understand requirements and deliver solutions.
• Communicate effectively with both technical and non-technical audiences.
• Participate in Agile/Scrum development processes.
• Data Architecture:
• Contribute to the overall data architecture, ensuring seamless integration of streaming data with other data systems.
• Design and implement data models for streaming data.
• Work with message queues like Kafka, RabbitMQ, etc.

Qualifications:

• Required:
• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• 7 years of experience in software development, with a focus on data engineering or stream processing.
• 3 years of hands-on experience with Apache Flink.
• Strong understanding of stream processing concepts and techniques.
• Proficiency in Java or Scala (or other relevant languages).
• Experience with distributed systems and cloud platforms (e.g., AWS, GCP, Azure).
• Experience with message queues (Kafka, RabbitMQ, etc.).
• Experience with building and maintaining CI/CD pipelines.
• Strong problem-solving and analytical skills.
• Excellent communication and collaboration skills.
• Preferred: 1
• Experience with other stream processing frameworks (e.g., Apache Kafka Streams, Apache Spark Streaming).
• Experience with containerization and orchestration technologies (Docker, Kubernetes).
• Experience with database systems (SQL and NoSQL).
• Experience with data warehousing solutions.
• Contributions to open-source projects, particularly Apache Flink.