Log caching enhancement

FortiAnalyzer log caching mechanism in reliable mode is enhanced to prevent Fortigate log loss during connection interruptions.

Log sync logic guarantees that no logs are lost due to connection issues when reliable mode is enabled on the FortiGate device. If connection is lost between the FortiAnalyzer and FortiGate device, logs will be cached and sent to FortiAnalyzer once the connection resumes.

Reliable mode is disabled by default on FortiGate devices.

To configure the FortiGate device:

1. Configure the FortiGate device to send logs to FortiAnalyzer.
2. In the FortiGate CLI, enter the following commands to confirm reliable is enabled:

   config log fortianalyzer2 setting

   show

   For example:

   config log fortianalyzer2 setting

   show

   config log fortianalyzer2 setting

   set status enable

   set server "10.2.169.54"

   set serial "FAZ-VM0000000001"

   set upload-option realtime

   set reliable enable

   end

3. In the FortiGate CLI, enter the following commands to confirm the value of logsync_enabled is 1:

   diagnose test application fgtlogd 1

   For example:

   diagnose test application fgtlogd 1

   faz2: global , enabled

   server=10.2.169.54, realtime=1, ssl=1, state=connected

   server_log_status=Log is allowed.,

   src=, mgmt_name=FGh_Log_root_10.2.169.54, reliable=1, sni_prefix_type=none,

   required_entitlement=none, region=ca-west-1,

   logsync_enabled:1, logsync_conn_id:131071, seq_no:257

   status: ver=6, used_disk=0, total_disk=0, global=0, vfid=0 conn_verified=Y

   SNs: last sn update:2097 seconds ago.

   Sn list:

   (FAZ-VM0000000001,age=2097s) (FAZ-VMJY00000004,age=2097s)

   queue: qlen=0.

   filter: severity=6, sz_exclude_list=0

To confirm cached logs are sent when connection is lost/resumed between FortiGate and FortiAnalyzer:

In this example, the FortiGate device has already been configured according to the steps above. When connection is lost between the FortiGate and FortiAnalyzer device, logs are cached on the FortiGate until connection resumes. Once connection resumes, the cached logs are sent to the FortiAnalyzer.

1. While connection between the FortiGate and FortiAnalyzer is established, check the log sequence number on the OFTP connection.

   In the FortiAnalyzer CLI, enter the following command:

   diagnose test application oftpd 3

   # DEVICE CONN HOSTNAME IP UPTIME IDLETIME #PKTS

   ----------------------------------------------------------------------------------------

   1 FGT40FTK20025663 131071: 257 FortiGate-40F 10.3.169.1 31m14s 4s 620

   The CONN column has been added to record the connection ID and log sequence number. In this example, the connection ID is 131071 and the sequence number is 257.

2. When the connection between the FortiGate and FortiAnalyzer is lost, check the log sequence number on the OFTP connection.

   In the FortiAnalyzer CLI, enter the following command:

   diagnose test application oftpd 3

   # DEVICE CONN HOSTNAME IP UPTIME IDLETIME #PKTS

   ----------------------------------------------------------------------------------------

   1 FGT40FTK20025663 131071: 257 FortiGate-40F 10.3.169.1 35m14s 244s 620

   While the connection is lost, logs generated on the FortiGate device will be stored in its memory queue. The log sequence number on the OFTP connection will not increase. In this example, the log sequence number has remained at 257.

3. When the connection between the FortiGate and FortiAnalyzer device resumes, check logs on the FortiGate device.

   In the FortiGate CLI, enter the following command:

   diagnose test application fgtlogd 41

   cache maximum: 100573388(95MB) objects: 37 used: 25788(0MB) allocated: 29440(0MB)

   VDOM:root

   Memory queue for: global-faz

   queue:

   num:0 size:0(0MB) total size:25788(0MB) max:100573388(95MB) logs:0

   Confirm queue for: global-faz

   queue:

   num:25 size:17382(0MB) total size:25788(0MB) max:100573388(95MB) logs:81

   Memory queue for: global-faz2

   queue:

   num:0 size:0(0MB) total size:25788(0MB) max:100573388(95MB) logs:0

   Confirm queue for: global-faz2

   queue:

   num:12 size:8406(0MB) total size:25788(0MB) max:100573388(95MB) logs:40

   The confirm queue on the FortiGate device shows all the logs that are waiting to be confirmed and cleared. Once the confirm queue displays 0, all of the cached logs have been sent to the FortiAnalyzer device.

4. Once the logs have been confirmed and cleared from the FortiGate device, check the log sequence number on the OFTP connection.

   In the FortiAnalyzer CLI, enter the following command:

   diagnose test application oftpd 3

   # DEVICE CONN HOSTNAME IP UPTIME IDLETIME #PKTS

   ----------------------------------------------------------------------------------------

   1 FGT40FTK20025663 131071: 308 FortiGate-40F 10.3.169.1 36m23s 6s 635

   Once the cached logs have been sent to the FortiAnalyzer device, the log sequence number increases. In this example, the log sequence number has increased to 308.