Kafka Developer

Who Is InRhythm?

InRhythm is a leading modern product consultancy and digital innovation firm with a mission to make a dent in the digital economy. Founded in 2002, InRhythm is currently engaged by Fortune 50 enterprises to bring their next generation of digital products and platforms to market. InRhythm has helped hundreds of teams launch mission-critical products that have created a positive impact worth billions of dollars.InRhythm’s unique capabilities of Product Innovation and Platform Modernization services are the most sought-after. The InRhythm team of A thought leaders don’t just “get an assignment,” they join the company to do what they love. It’s that passion that has helped us grow rapidly and consistently deliver on our commitment to helping clients develop better, faster, and in rhythm.

We are seeking an experienced Kafka Developer to design, implement, and optimize Apache Kafka clusters for real-time data streaming in a high-performance, scalable, and secure environment. The ideal candidate will have expertise in event-driven architecture, integrating Kafka with existing systems, and ensuring seamless data flow. You will work closely with cross-functional teams to develop efficient streaming applications and maintain a resilient data pipeline.

Key Responsibilities:

• Design, develop, and optimize Kafka clusters for large-scale, real-time data processing.
• Implement event-driven architectures for streaming financial transactions and other critical data.
• Develop and maintain Kafka producer/consumer applications using Java, Spring Boot, and Kafka Streams API.
• Work with Apache Flink for real-time stream processing, implementing Job Manager, Task Manager, and checkpointing.
• Integrate Kafka with Apache NiFi for data ingestion, transformation, and routing.
• Ensure high availability, security, and resiliency of Kafka environments.
• Implement Kafka Connect and Schema Registry for data replication and schema evolution.
• Utilize Prometheus, Grafana, ELK, and Splunk for monitoring Kafka clusters and troubleshooting failures.
• Manage and optimize Kafka performance, including partitioning, leadership, clustering, and load balancing.
• Develop Terraform scripts for provisioning Kafka clusters on AWS, Azure, and Google Cloud Platform (GCP).
• Automate Kafka deployments using Ansible, Docker, and Kubernetes (AKS, EKS, GKE).
• Integrate Kafka with ActiveMQ, Snowflake, Oracle Autonomous Database, and other data storage solutions.
• Implement Kafka security features (SSL/TLS encryption, SASL authentication, and VPC Service Controls on GCP).
• Provide capacity planning, architecture design, and infrastructure optimization for Kafka clusters.
• Troubleshoot and resolve Kafka failures, leveraging Kibana, Elasticsearch, and monitoring tools.
• Collaborate with DevOps, Cloud, and Middleware teams to streamline Kafka operations.
• Develop reprocessing mechanisms for message failures using Kafka offset ID tracking.

Required Skills & Qualifications:

• Strong hands-on experience with Apache Kafka (setup, configuration, scaling, and monitoring).
• Expertise in Kafka core concepts (broker, producer, consumer, partitions, leadership, controller, protocol, clustering).
• Experience with Spring Boot microservices for real-time message processing.
• Proficiency in Apache Flink for stateful stream processing and time-based operations.
• Experience integrating Kafka with Apache NiFi, Hadoop, Spark, and other big data frameworks.
• Familiarity with Kafka security configurations (SSL, SASL, Kerberos).
• Knowledge of Terraform, Ansible, Docker, and Kubernetes for Kafka deployment automation.
• Experience with cloud-based Kafka deployments (AWS MSK, Azure Event Hubs, GCP Pub/Sub).
• Strong understanding of event-driven architecture and data pipeline design.
• Experience with SQL and NoSQL databases (Snowflake, Cassandra, Oracle, PostgreSQL, MongoDB).
• Proficiency in monitoring tools such as Prometheus, Grafana, Confluent Control Center, RTView, Sysdig, and ELK Stack.
• Hands-on experience with schema management using Confluent Schema Registry and AVRO formats.
• Experience with ActiveMQ, RabbitMQ, IBM MQ is a plus.
• Strong troubleshooting, debugging, and performance-tuning skills.
• Excellent problem-solving and analytical thinking abilities.

Preferred Qualifications:

• Certification in Confluent Kafka, AWS, Azure, or GCP.
• Experience in financial services or payment systems using Kafka for transaction processing.
• Familiarity with CI/CD pipelines, Git, Jenkins, and automated testing frameworks.
• Strong scripting skills in Python, Bash, or Groovy.

Why Join Us?

• Work with cutting-edge real-time data streaming technologies.
• Collaborate with a highly skilled and innovative team.
• Competitive salary, benefits, and career growth opportunities.
• Hybrid/Remote work flexibility.

If you are a passionate Kafka Developer looking to work on scalable, real-time streaming applications, we’d love to hear from you! Apply now to join our dynamic team.

What We Do At InRhythm

We bring enterprises' most urgent, important products to market with high-velocity, high- quality and 10x impact. We enable innovative cultures by coaching teams with the right mix and maturity of modern tools, methods, and thought leadership.This is a unique opportunity to get in on the ground floor of an evolving team. InRhythm clients include a broad range of highly visible and recognizable customers, including, but not limited to:

• Goldman Sachs
• Fidelity
• Morgan Stanley
• American Express

Why Work At InRhythm?

We’ve been named an Inc. 5000 Hall of Fame Fastest Growing Company for 9 years, Deloitte Fast 500 company for 5 years, and Consulting Magazine Fastest Growing Company winner several years in a row. If you’re looking forward to working with awesome colleagues in a high- growth environment and tight-knit community, we’re looking forward to hearing from you.

At InRhythm we believe in building a better workplace. When you join our team, you can expect a number of work/life benefits:

• Comprehensive and customizable Medical, Dental, and Vision Plans
• 401(k) Matching
• Paid Parental Leave
• Scalable PTO
• Social and Flexible Work Environment

The base pay offered will take into account internal equity and may vary depending on the candidate’s geographic region, job-related knowledge, skills, and experience among other factors. A bonus and/or long-term incentive units may be provided as part of the compensation package, in addition to the full range of medical, financial, and/or other benefits, dependent on level and position offered