Message Queue Kafka Application Architecture

• Producer—Operates in push mode and sends messages (such as website visits or server logs) to brokers.
• Kafka broker—Stores information as a server. Brokers support horizontal expansion. Generally, the more the number of the broker nodes, the higher the throughput of the cluster.
• Consumer group—Operates in pull mode and subscribes to messages from brokers.
• Zookeeper—Manages the cluster configuration, elects the leader, and balances the load when the consumer group changes.

Diagram of the Kafka Publication and Subscription Relationships

Message Queue Kafka uses Pub/Sub model.

• One consumer group can subscribe to multiple topics, and one topic can be subscribed to by multiple consumer groups at the same time.
• One message on a topic can be consumed by only one consumer in a consumer group, but multiple consumer groups can consume this message at the same time.

Kafka Deployment Architecture

• Kafka management platform—Uses the standard J2EE application, Tomcat deployment, and the PostgreSQL database.
• Kafka broker—The Zookeeper nodes and broker nodes are deployed on demand. The broker cluster uses the master-subordinate model. A minimum of two copies of a message are written to the disk. We recommend that you deploy nodes in different availability zones.
• Client SDK support for multiple languages—Languages such as JAVA, and C language are supported. The client accesses the broker by using the TCP protocol.