Hi
Kafka Engineer (Kafka Admin)
Remote
Long term contract
Expectation:
Approximately 300 Kafka clusters with 1200 brokers & ~10,000 topics with tens of millions of messages flowing thru and span across on-premise and cloud (AWS). It’s pure open source Kafka on bare metal in on-premise and mix of ECS and EKS on AWS cloud. This kafka is super large and can’t down as this will bring down many OTT services operated by Disney & Co.
The desired candidate:
– should be capable of handling such large systems without downtime
– Exhibit lead attitude and take care of the cluster operationally, plan for upgrades, assist teams to use the cluster, performing regular administration/monitoring, etc
– Come up with ideas on optimal use of clusters and brokers to reduce cost and improve performance. Sometimes, clusters and brokers are not properly scoped and there will be lots of wastage. Other times, they are under scoped and causes downtime.
– They have also built an in-house application to manage this massive cluster that’s written in Kotlin (Java cousin) and thus some programming experience is required but they don’t have to be a super coder as in backend engineer.
They need someone with expertise on Open Source Kafka. Confluent Kafka & AWS MSK hides most of the Kafka admin challenges but that’s not what’s required here. They don’t need someone who uses Kafka as a producer or consumer but someone who knows the internals.
Thanks & Regards..
Bhargavi Nanabolu
Talent Acquisition Specialist
Email: Nanabolu.Bhargavi@infoservices.com
Contact:6093781571 EXT:102
From:
NanaboluBhargavi,
Infoservices
nanabolu.bhargavi@infoservices.com
Reply to: nanabolu.bhargavi@infoservices.com