How HubSpot Uses Apache Kafka Swimlanes for Timely Processing of Workflow Actions

HubSpot adopted routing messages over multiple Kafka topics (called swimlanes) for the same producer to avoid the build-up in the consumer group lag and prioritize the processing of real-time traffic. Using a combination of automatic and manual detection of traffic spikes, the company ensures the majority of customers’s workflows execute without delays.

ai apache apache kafka architecture & design build combination consumer customers detection development event-driven-architecture event stream processing hubspot kafka messages messaging ml & data engineering multiple performance & scalability processing real-time real time data routing scalability the company topics traffic workflow