Kafka Architect

Job Summary:

• We are looking for a Kafka Architect with expertise in Event-Driven Architecture (EDA) for Retail to design, evaluate, and implement scalable, real-time event streaming solutions for retail and eCommerce platforms.
• This role requires hands-on experience with Apache Kafka, Kafka Streams, and Kafka Connect to enable seamless data flow across omnichannel retail operations, real-time inventory management, personalized customer experiences, and predictive analytics.

Key Responsibilities:

• Architect and implement real-time event streaming platforms for use cases such as real-time inventory tracking, fraud detection, personalized promotions, and dynamic pricing.
• Evaluate and select Kafka deployment models (self-managed, Confluent, AWS MSK, Azure Event Hubs, etc.) based on business and scalability needs.
• Design high-availability Kafka clusters to handle massive transaction volumes in eCommerce, POS systems, and digital customer engagement.
• Define best practices for topic management, schema evolution (Schema Registry), and partitioning strategies for large-scale retail data flows.
• Enable real-time customer data streaming from multiple sources (POS, eCommerce platforms, loyalty programs, IoT devices).
• Build Kafka-powered data pipelines to integrate with ERP, CRM, WMS, OMS, and personalization engines.
• Develop stream processing solutions using Kafka Streams and ksqlDB for dynamic promotions, order fulfillment, and fraud detection.
• Ensure seamless integration with retail analytics platforms like Snowflake, Databricks, and Google BigQuery.
• Optimize Kafka infrastructure to support seasonal peaks, high traffic surges (e.g., Black Friday, Cyber Monday), and multi-channel order management.
• Implement disaster recovery, failover strategies, and high-availability configurations for mission-critical retail systems.
• Define security policies for data protection, compliance (PCI-DSS), and customer privacy regulations (GDPR, CCPA).
• Establish monitoring, logging, and alerting using tools like Prometheus, Grafana, Splunk, and OpenTelemetry.
• Act as the Kafka SME for retail transformation projects, educating engineering and business teams on event-driven capabilities.
• Collaborate with eCommerce, supply chain, marketing, and data science teams to drive real-time decision-making.
• Partner with cloud architects, DevOps, and cybersecurity teams to ensure a secure, scalable Kafka ecosystem.

Required Skills & Experience:

• 8 years of experience in designing and implementing event-driven architectures, preferably in retail, eCommerce, or omnichannel environments.
• Deep expertise in Apache Kafka (Kafka Streams, Kafka Connect, ksqlDB, Schema Registry).
• Strong knowledge of real-time retail analytics, demand forecasting, and digital customer engagement.
• Experience integrating Kafka with retail POS systems, inventory management, and order fulfillment platforms.
• Hands-on experience with microservices and event-driven patterns (CQRS, SAGA, Event Sourcing).
• Experience working with cloud-based Kafka implementations (AWS MSK, Azure Event Hubs, GCP Pub/Sub).
• Proficiency in real-time inventory tracking, recommendation engines, and fraud detection systems.
• Experience with containerization (Docker, Kubernetes) and CI/CD pipelines for Kafka deployments.
• Strong background in monitoring tools and performance tuning for Kafka clusters.

Preferred Qualifications:

• Kafka certification (Confluent Certified Developer/Administrator).
• Experience with real-time pricing, customer behavior analytics, and personalization engines.
• Knowledge of stream processing frameworks (Flink, Spark Streaming) for advanced retail analytics.
• Experience designing Kafka-based architectures for omnichannel retail platforms.