Lead Kafka Middleware Engineer

***Position is bonus eligible***

Prestigious Financial Institution is currently seeking a Lead Kafka Middleware Engineer with deep expertise in Kafka administration, infrastructure as code (IaC), cloud automation, container orchestration and CI/CD pipelines. The ideal candidate will be responsible for designing, implementing, and maintaining robust and scalable middleware solutions, ensuring high availability, performance, and security. Candidate will play a crucial role in automating infrastructure provisioning, deployments, and operations, enabling our organization to rapidly deliver and scale applications.

Responsibilities:

• Design, implement and manage highly available and scalable Kafka clusters.
• Monitor Kafka performance, troubleshoot issues and optimize configurations.
• Develop and maintain IaC using Ansible and Terraform for infrastructure provisioning and configuration Management.
• Create and maintain reusable IaC modules.
• Design and implement cloud-based infrastructure solutions on AWS and Azure.
• Automate cloud resource provisioning, scaling and management using cloud-native tools and services.
• Deploy and Manage containerized applications using Kubernetes and Rancher
• Troubleshoot container-related issues and optimize container performance.
• Design, implement and maintain CI/CD pipelines using tools like GitHub, Artifactory, Harness and Jenkins.
• Automated the build, test and deployment of middleware components.
• Integrate IaC and container technologies into CI/CD pipelines.
• Document all processes and procedures.
• Work with development teams to ensure smooth deployments.

Qualifications:

• Strong proficiency in IaC tools, specifically Ansible, Terraform and bash scripting.
• Extensive experience with cloud automation and provisioning on AWS and Azure.
• Proficiency in CI/CD tools, including GitHub, Artifactory, Harness and Jenkins.
• Strong scripting skills in languages like Python and Bash.
• Excellent troubleshooting and problem-solving skills
• Understanding of networking principles.
• Experience with monitoring tools like Splunk, Splunk OTEL, Prometheus and Grafana.
• Kafka, Ansible, Terraform, Bash, Kubernetes, Rancher, GitHub, Artifactory, Harness, Jenkins, AWS, Azure, CI/CD, IaC, Automated Cloud Provisioning
• Bachelors degree in Computer Science, Engineering or a related field (or equivalent experience)
• 10 years of experience in infrastructure middleware administration.
• In-depth expertise in Kafka administration, including cluster management, performance tuning, and security.
• AWS Solutions Architect, CKAD or CKA certifications preferred.