Tagged: Kafka

In a multi-tenant B2B system, customers rarely generate the same amount of traffic. Many send a steady trickle of events; a few contribute a much larger share. A streaming pipeline has to handle both, often with the same set of workers.

In Flink, that imbalance can be easy to miss at first. The cluster may have CPU and memory to spare while a small number of tasks struggle to keep up. Adding workers seems like a reasonable next step, but whether it helps depends on how the work is divided—and which parts can be divided at all.