FortiSentry real-time monitor 7.2.11

FortiSentry is an independent hardware-based security module that actively monitors the FortiOS file system during runtime to detect unauthorized changes and potential threats. Because FortiSentry is an out-of-band and independent module, it is particularly effective against persistent attacks that may have bypassed FortiGate and FortiBIOs verification and scanning.

FortiSentry performs file integrity monitoring: Critical files and their signatures are continuously verified for unauthorized modifications.

FortiSentry is first introduced in the FortiGate-700G model series with compatible hardware.

FortiSentry communication

To illustrate, the FortiSentry hardware module connects to the FortiGate host to monitor its disk partitions. However, it does not initiate connections back to the FortiGate host aside from exposing some key status indicators.

Communication primarily goes through a System Management Controller (SMC) module as follows:

• FortiSentry communicates with SMC through periodic heartbeat messages. These messages report FortiSentry’s operational status and results of integrity checks. Heartbeat messages are sent every 5 minutes to confirm normal operation.

• In the event of a critical failure or integrity violation, FortiSentry sends an emergency alert to the SMC for immediate action.

• SMC can initiate a controlled system reboot to ensure the FortiGate host does not continue operating in a compromised state.

• The FortiGate can also use IPMI commands to query FortiSentry’s status through SMC:

  • Administrators can manually perform a CLI query through the SMC for the current FortiSentry heartbeat message using CLI commands.

  • When an issue is detected, corresponding events are logged automatically.

FortiSentry includes a fail-safe mechanism. If the SMC doesn't receive a heartbeat from FortiSentry within 30 minutes, it assumes FortiSentry is compromised and initiates a pre-defined emergency action. Within the first 30 minutes, missed heartbeats are considered a security notice. After 30 minutes, they are considered an emergency event.

Examples

Example 1: An attacker places a malicious executable file in the /data monitored directory

1. Threat Planted: A rogue executable named decrypt_log is placed in /data monitored directory.

2. Initial Detection: FortiSentry detects the unauthorized file and sends a "Security Notice" to the SMC, flagging "Binary files found in data or data2."

   # diagnose test application ipmc_sensord 7
   ---Current HeartBeat Message---
   Start byte: ff
   total_size: 62
   version: 1
   operation_hb: 18
   running_hb: 18
   comm_event: 1 <-- Security notice
   reason_code: 11 <-- Binary files found in data or data2
   file_name_size: 46
   file_name: /data/lib/decrypt_log#2025-07-10 01:11:18#6027<-- Binary file name is indicated.
   

3. Warning log: FortiOS generates a warning log.

   1: date=2025-07-10 time=13:12:10 eventtime=1752109930316502839 logid="0100022119" type="event" subtype="system" level="warning" vd="root" logdesc="IPMC Event" action="ipmc-event" msg="Warning, but no pending action, pci heartbeat[1,11]"

4. Emergency Alert: FortiSentry escalates the event, sending an "Emergency Event" to the SMC.

   # diagnose test application ipmc_sensord 7
   diagnose test application ipmc_sensord 7
   ---Current HeartBeat Message---
   Start byte: ff
   total_size: 62
   version: 1
   operation_hb: 34
   running_hb: 34
   comm_event: 2 <-- Emergency event
   reason_code: 11 <-- Binary files found in data or data2
   file_name_size: 46
   file_name: /data/lib/decrypt_log#2025-07-10 02:31:25#6027 <-- Binary file name is indicated.
   

5. System Shutdown: The SMC receives the emergency alert, logs the event, and initiates a 10-minute shutdown countdown. FortiOS logs show the remaining time.

   1: date=2025-07-10 time=14:33:10 eventtime=1752114790585383040 logid="0100022118" type="event" subtype="system" level="emergency" vd="root" logdesc="IPMC Event" action="ipmc-event" msg="Emergency reboot, pending action is to reboot in 504 seconds, pci heartbeat[2,11]"
   2: date=2025-07-10 time=14:32:10 eventtime=1752114730573127340 logid="0100022118" type="event" subtype="system" level="emergency" vd="root" logdesc="IPMC Event" action="ipmc-event" msg="Emergency reboot, pending action is to reboot in 564 seconds, pci heartbeat[2,11]"
   

6. Action: After the countdown, the FortiGate reboots.

   The system is going down NOW !!
   The system is halted.
   
   FortiGate-701G (12:53-04.22.2025)
   Ver:07000104
   Serial number:FG7H1GTB25000066
   RAM activation
   CPU(00:00A20F12 178BFBFF): MP initialization
   CPU(01:00A20F12 178BFBFF): MP initialization
   CPU(02:00A20F12 178BFBFF): MP initialization
   

Example 2: critical system file tampered

This example demonstrates the coordinated detection and response of the subsystems when a critical system library, libav.so, is deleted.

1. File Missing: A malicious activity removes the libav.so file from the monitored /data directory.

2. Integrity Check Fails: During its routine scan, FortiSentry's integrity check fails. It sends a "Security Notice" to the SMC, reporting "LIBAV Signature verification fails."

   # diagnose test application ipmc_sensord 7
   diagnose test application ipmc_sensord 7
   ---Current HeartBeat Message---
   Start byte: ff
   total_size: 46
   version: 1
   operation_hb: 6
   running_hb: 6
   comm_event: 1 <-- Security notice
   reason_code: 9 <-- LIBAV Signature verification fails
   file_name_size: 30
   file_name: LIBAV#2025-07-10 22:11:27#6025
   

3. Warning log: FortiOS generates a warning log.

   1: date=2025-07-10 time=14:52:24 eventtime=1752184344670969223 tz="-0700" logid="0100022119" type="event" subtype="system" level="warning" vd="root" logdesc="IPMC monitor warning" action="ipmc-event" msg="Warning, but no pending action, pci heartbeat[1,9]"

4. Emergency Alert: The event is escalated to an "Emergency Event”, and the SMC is notified.

   # diagnose test application ipmc_sensord 7
   
   ---Current HeartBeat Message---
   Start byte: ff
   total_size: 46
   version: 1
   operation_hb: 10
   running_hb: 10
   comm_event: 2 <-- Emergency event
   reason_code: 9 <-- LIBAV Signature verification fails
   file_name_size: 30
   file_name: LIBAV#2025-07-10 22:31:29#6025
   

5. System Shutdown: The SMC initiates the 10-minute shutdown countdown to prevent the system from running in a compromised state.

   1: date=2025-07-10 time=15:33:24 eventtime=1752186804613806543 tz="-0700" logid="0100022118" type="event" subtype="system" level="emergency" vd="root" logdesc="IPMC monitor emergency" action="ipmc-event" msg="Emergency reboot, pending action is to reboot in 486 seconds, pci heartbeat[2,9]"
   2: date=2025-07-10 time=15:32:24 eventtime=1752186744606396369 tz="-0700" logid="0100022118" type="event" subtype="system" level="emergency" vd="root" logdesc="IPMC monitor emergency" action="ipmc-event" msg="Emergency reboot, pending action is to reboot in 546 seconds, pci heartbeat[2,9]"
   

6. Action: After the countdown, FortiGate is rebooted.

   The system is going down NOW !!
   The system is halted.
   FortiGate-701G (12:53-04.22.2025)
   Ver:07000104
   Serial number:FG7H1GTB25000059
   RAM activation
   CPU(00:00A20F12 178BFBFF): MP initialization
   CPU(01:00A20F12 178BFBFF): MP initialization
   CPU(02:00A20F12 178BFBFF): MP initialization
   

Event code reference

Command/Event (comm_event) codes

The comm_event code indicates the overall severity and type of message being sent from FortiSentry to SMC.

Code	Description
0	Heartbeats
1	Security notice
2	Emergency event
3	Link down

Reason (reason_code) codes

The reason_code provides specific, detailed information about the cause of a comm_event, particularly for security notices (1) and emergency events (2).

Code	Description
0	Normal
1	No signature
2	Signature verification fails
3	NON-GA file corrupted
4	GA file corrupted
5	No virtual NVME is mounted
6	No active MBR on FGT[WN1] [MC2]
7	FortiSentry service timeout
8	LIBIPS Signature verification failure
9	LIBAV Signature verification failure
10	DB Signature verification fails
11	Binary files found in data or data2
12	Data mount failure
13	Data2 mount failure
14	Security File system mount failure
15	No shared memory
16	Process, service, or resource initialization