This article describes the architecture and operation process of Log Service.

Physical Architecture

Log processing servers

Log processing servers operate in load balancing mode to analyze and process collected logs.

• Log processing servers support the HTTP and REST protocols.
• Ping An Cloud Log Service scales out log processing servers to speed up log processing when the volume of the collected log data increases.
• Log processing servers provide high throughput and low latency.
  • They process logs in asynchronous mode. A single exception request does not affect the processing of other requests.
  • They compress log data locally in a specific format before transmission to save network bandwidth and improve the transmission efficiency.

Search engine servers

Distributed on multiple servers, the search engine servers receive log data from log processing servers, and store the data based on the retention period set for the index. The search engines support log data query.

• High security
  • Each log is stored in three copies.
  • Data integrity can still be guaranteed even if a server goes down.
• High reliability
  • Automatic load balancing ensures that traffic is distributed evenly among servers.
  • Strict tenant restrictions prevent unexpected operations of one tenant from affecting other tenants.
• Horizontal scalability
  • The servers provide fine-grained shard-based storage.
  • When a large amount of data is stored, shards can be dynamically added to increase the processing capacity.

Storage servers

If you enable the archiving feature when configuring a collection policy, the log processing servers will send the collected logs to the Ping An Cloud storage servers.