Apache Flink Streaming Developer/ Lead

Job Description

Apache Flink Streaming Developer/ Lead

Location: Jersey City, NJ (1 Pershing Plaza, Jersey City, NJ) Hybrid Role

Long Term Project

Job Summary:

We are seeking a highly skilled and motivated Lead Software Engineer with deep expertise in Apache Flink to lead the development and optimization of our real-time streaming applications. You will be responsible for designing, building, and maintaining scalable and robust data pipelines that process high-volume, high-velocity data. You will provide technical leadership, mentor junior engineers, and drive best practices in stream processing.

Responsibilities:

• Design and Development:

  • Architect, design, and implement high-performance, fault-tolerant, and scalable real-time streaming applications using Apache Flink.

  • Develop and optimize Flink pipelines for data ingestion, transformation, and analysis.

  • Write clean, efficient, and well-documented code.

  • Implement and maintain CI/CD pipelines for streaming applications.

• Technical Leadership:

  • Lead technical design discussions and code reviews.

  • Mentor and guide junior engineers, fostering a collaborative and learning environment.

  • Define and enforce coding standards and best practices for stream processing.

  • Evaluate and recommend new technologies and tools to improve streaming capabilities.

• Performance and Optimization:

  • Monitor and optimize the performance of streaming applications.

  • Troubleshoot and resolve complex performance and stability issues.

  • Conduct performance testing and benchmarking.

  • Tune Flink configurations for optimal resource utilization.

• Collaboration and Communication:

  • Work closely with product managers, data scientists, and other stakeholders to understand requirements and deliver solutions.

  • Communicate effectively with both technical and non-technical audiences.

  • Participate in Agile/Scrum development processes.

• Data Architecture:

  • Contribute to the overall data architecture, ensuring seamless integration of streaming data with other data systems.

  • Design and implement data models for streaming data.

  • Work with message queues like Kafka, RabbitMQ, etc.

Qualifications:

• Required:

  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

  • 10 years of experience in software development, with a focus on data engineering or stream processing.

  • 3 years of hands-on experience with Apache Flink.

  • Strong understanding of stream processing concepts and techniques.

  • Proficiency in Java or Scala (or other relevant languages).

  • Experience with distributed systems and cloud platforms (e.g., AWS, Google Cloud Platform, Azure).

  • Experience with message queues (Kafka, RabbitMQ, etc.).

  • Experience with building and maintaining CI/CD pipelines.

  • Strong problem-solving and analytical skills.

  • Excellent communication and collaboration skills.

• Preferred: 1

  • Experience with other stream processing frameworks (e.g., Apache Kafka Streams, Apache Spark Streaming).

  • Experience with containerization and orchestration technologies (Docker, Kubernetes).

  • Experience with database systems (SQL and NoSQL).

  • Experience with data warehousing solutions.

  • Contributions to open-source projects, particularly Apache Flink.