Tag: kafka

Timeouts in Kafka clients and Kafka Streams

IMPORTANT: This is information is based on Kafka and Kafka Streams 1.0.0. Past or future versions may defer. As with any distributed system, Kafka relies on timeouts to detect failures. Those timeouts can be sent by clients and brokers that want to detect each other unavailability. The following is a description of the configuration values that control timeouts that both brokers and client will use to detect clients not being available. The original design for the Poll() method in the Java consumer tried to kill two birds with one stone: Guarantee consumer liveness Guarantee progress […]

By Javier

Incompatible AVRO schema in Schema Registry

My company uses Apache Kafka as the spine for its next-generation architecture. Kafka is a distributed append-only log that can be used as a pub-sub mechanism. We use Kafka to publish events once business processes have completed successfully, allowing a high degree of decoupling between producers and consumers. These events are encoded using Avro schemas. Avro is a binary serialization format that enables a compact representation of data, much more than, for instance, JSON. Given the high volume of events we publish to kafka, using a compact format is critical. In combination with Avro we […]

By El Javi

By Daniele Zedda • 18 February


By Daniele Zedda • 18 February

Share on