Fortinet white logo
Fortinet white logo

CLI Reference

debug

debug

Use these commands to:

  • show license status and network connectivity statistics with FortiGuard Antispam rating query servers or override servers

  • show license status for FortiMail-VM

  • kill processes or send them other system-level signals

  • get debugging and crash information

Tab-complete is not supported for many of the later arguments, and so you must type the complete argument.

Arguments for each process's commands, such as:

diagnose debug application <process_name> ...

vary by the process. Syntax shows the process named smtpd. For more examples, see Example.

Level of detail (category and/or verbosity) can be set separately for each process, and for the kernel, such as:

diagnose debug application <process_name> level {{0..8} | d | D | l | L | v | V}

diagnose debug cli [{0..8}]

diagnose debug kernel level [{0..8}]

Debug commands produce output only while the related processes are active. Output for some commands is printed to your CLI display until you stop it by pressing Ctrl + C. Otherwise output can be controlled with commands such as:

diagnose debug setting duration <minutes_int>

diagnose debug application <process_name> duration <minutes_int>

diagnose debug application <process_name> output {default | file | stdout | terminal}

(Process-specific level and duration settings override general debug settings.)

Alternatively, you can use the GUI to download debug log files (also called a crash or trace log) and view them in a text editor. See the trace log information in the FortiMail Administration Guide.

Tooltip

Use diagnose debug commands:

  • At less busy times. Debugging is real-time and can be CPU intensive.

  • Via a local console or terminal server. If CPU or network usage is too much during debugging, then network connections such as via SSH may be less responsive. Local access also allows you to reboot FortiMail to immediately disable debugging.

Syntax

diagnose debug rating [<refresh-interval-seconds_int>]

diagnose debug tools vm-print-license

diagnose debug tools process list [<process_name>] [| grep <filter_str>]

diagnose debug tools process signal <signal_int>

diagnose debug setting rotate <files_int> <megabytes_int>

diagnose debug setting duration <minutes_int>

diagnose debug application <process_name> format {default | datetime | file-name | func-name | line-number | millisecond | pid | trace-name}

diagnose debug application <process_name> filter "<filter_str>"

diagnose debug application <process_name> output {default | file | stdout | terminal}

diagnose debug application <process_name> level {{0..8} | d | D | l | L | v | V}

diagnose debug cli [{0..8}]

diagnose debug database [{0..1}]

diagnose debug database general-log

diagnose debug database error-log

diagnose debug database debug-log

diagnose debug kernel display {pretty | <seconds-since-boot_int>}

diagnose debug kernel log enable

diagnose debug kernel level [{0..8}]

diagnose debug enable

diagnose debug application <process_name> enable

diagnose debug application <process_name> duration <minutes_int>

diagnose debug setting level display

diagnose debug setting level clear

diagnose debug setting level save

diagnose debug application <process_name> display

diagnose debug disable

diagnose debug tools crashlog

diagnose debug tools coredump status

diagnose debug tools coredump list [<process_name>]

diagnose debug tools coredump enable

diagnose debug tools coredump upload "<filename_str>" <tftp-server_ipv4> ["<destination-filename_str>"]

diagnose debug tools coredump delete "<filename_str>"

diagnose debug tools coredump clear

Variable

Description

Default

"<filename_str>"

Enter the file name on the FortiMail unit.

<process_name>

Enter the name of a process, such as smtpd or mailstatd.

<signal_int>

Enter a standard POSIX system signal number, such as:

  • 2 (SIGINT) or 15 (SIGTERM) — Quit with an interrupt signal. Normal termination for many processes. Depending on its state, the process may handle the quit gracefully such as saving state and releasing system resources instead of an immediate SIGKILL.

  • 3 (SIGQUIT) — Quit. Also output a core dump file.

  • 6 (SIGABRT) — Quit. Also output a crash log file. Unlike SIGSEGV, processes usually trigger this signal themselves.

  • 9 (SIGKILL) — Force quit.Unlike SIGINT, the process cannot ignore SIGKILL or handle the signal gracefully. Administrators may trigger this signal to terminate zombie processes.

  • 11 (SIGSEGV) — Force quit like SIGKILL. Also output the segmentation fault in the crash log file.

If a process crashes, crash logs and/or core dumps (trace files) should be automatically generated. Do send a signal in that case.

If processes have not crashed, but Fortinet Support requires detailed troubleshooting information, Support may ask you to send signal 6 or 11 to manually trigger a trace log.

<tftp-server_ipv4>

Enter the IP address of a TFTP server.

coredump

Display a kernel core dump file.

crashlog

Display a process crash log.

debug {enable | disable}

Enable or disable output of debug messages in the CLI display.

Note: diagnose debug setting duration <minutes_int> can only be entered while debugging is enabled. When you disable debugging or the duration time elapses, then the settings are reset.

disable

duration <minutes_int>

Enter how long in minutes to record debug information.

30

filter "<filter_str>"

Enter text that you want to find to search and filter output to show only matching lines.

format {default | datetime | file-name | func-name | line-number | millisecond | pid | trace-name}

Select the format of the debug file. default includes both the time and PID.

grep <filter_str>

Enter text that you want to find to search and filter output to show only matching lines.

For example, you could enter grep smtpd to show only lines with the SMTP process name.

level {{0..8} | d | D | l | L | v | V}

Select an amount or category of detail to include in the debug log, either:

  • a number, such as 0 or 8

  • d — Debug messages.

  • D — Verbose debug messages.

  • l — Library level debug messages.

  • L — Verbose library level debug messages.

  • v — Very verbose debug messages.

  • V — Very verbose library level debug messages.

To display the current detail level, enter:

diagnose debug application <process_name> display

(For diagnose debug commands for the kernel, CLI, or database, to display the current level, you omit the level number instead.)

level clear

Resets the saved verbosity level of debug information.

level display

Displays the current and saved verbosity level of debug information.

level save

Saves the current verbosity level of debug information so that it persists after reboot. To undo this, enter diagnose debug settinglevel clear.

output {default | file | stdout | terminal}

Select the display location of the command results, either:

  • default — Both the terminal and a debug log file.

  • file — Debug log file.

  • stdout — Monitor attached to the FortiMail appliance.

  • terminal — Local console or remote terminal emulator.

rating [<refresh-interval-seconds_int>]

Enter how frequently in seconds to refresh the display for the list of FortiGuard Antispam rating query servers.

If you omit the refresh interval, the CLI output gives a list of the servers and then immediately returns you to the command prompt.

0

rotate <files_int> <megabytes_int>

Enter the maximum number and file size in megabytes (MB) of debug files to keep.

10

vm-print-license

Display FortiMail-VM license information. See also system vm.

Example

FortiMail # diag debug rating

System Time:  2025-05-20 16:19:27 EDT (Uptime: 0d 4h 49m)
License      : Contract
Expiration   : Sat Mar  5 19:00:00 2033
Hostname     : N/A (Server Override Enabled)
-=- Server List (Tue May 20 16:19:27 2025) -=-
IP                  Weight    RTT Flags  TZ    Packets  Curr Lost Total Lost
192.168.100.206       -359     67 DI     -8        203          0          1

FortiMail # diag debug tools process list

System Time:  2025-05-20 16:07:12 EDT (Uptime: 0d 4h 37m)
PID                  STARTED   LWP COMMAND         S TTY          TIME COMMAND
1 Tue May 20 11:30:00 2025     1 init            S ?        00:00:01 /init
2 Tue May 20 11:30:00 2025     2 kthreadd        S ?        00:00:00 [kthreadd]
...

FortiMail # diag debug tools process signal 9 2192

System Time:  2025-05-20 16:07:12 EDT (Uptime: 0d 4h 37m)
signal 9 sent to pid 2152

FortiMail # diagnose debug setting level display

System Time:  2025-05-21 14:41:55 EDT (Uptime: 0d 0h 53m)
Current debug levels:
Cli debug level is 1
httpd debug level is: 0(0x0)
 
Persistent debug levels:
Cli debug level is 3

FortiMail # diag debug app httpd trace-log enable
System Time:  2025-06-05 10:04:35 EDT (Uptime: 0d 19h 47m)
This command will cause brief interruption to existing HTTP connections, and the console will be disconnected. Are you sure you want to continue? (y/n)y
FortiMail # diag debug app httpd display trace-log
System Time:  2025-06-05 10:05:40 EDT (Uptime: 0d 19h 48m)
2025-06-05T10:05:33 {"objectID": "SysTimeManual:","reqAction": 1,"nodePermission": 3,"daylight_saving_time": true,"zone": 12,"timezone_offset": 240}
2025-06-05T10:05:33 =========== END OF RESPONSE ===========
2025-06-05T10:05:38 ########### RECEIVED REQUEST ###########
2025-06-05T10:05:38 extraParam=minute&reqObject=SysStatusUsage&reqAction=1&skipTmUp=1
2025-06-05T10:05:38 ----------- END OF REQUEST -----------
2025-06-05T10:05:38 +++++++++++++ RESPONSE +++++++++++++
2025-06-05T10:05:38 {"objectID": "SysStatusUsage:","reqAction": 1,"nodePermission": 3,"cpu": 0,"memory": 63,"log_disk": 0,"mail_disk": 1,"system_load": 15,"active_sessions": 7,"network_usage": 93}
2025-06-05T10:05:38 =========== END OF RESPONSE ===========
^C
FortiMail # diag debug app httpd trace-log disable
System Time:  2025-06-05 10:07:34 EDT (Uptime: 0d 19h 50m)
This command will cause brief interruption to existing HTTP connections, and the console will be disconnected. Are you sure you want to continue? (y/n)y

Related topics

system status

sniffer

test

system vm

debug

debug

Use these commands to:

  • show license status and network connectivity statistics with FortiGuard Antispam rating query servers or override servers

  • show license status for FortiMail-VM

  • kill processes or send them other system-level signals

  • get debugging and crash information

Tab-complete is not supported for many of the later arguments, and so you must type the complete argument.

Arguments for each process's commands, such as:

diagnose debug application <process_name> ...

vary by the process. Syntax shows the process named smtpd. For more examples, see Example.

Level of detail (category and/or verbosity) can be set separately for each process, and for the kernel, such as:

diagnose debug application <process_name> level {{0..8} | d | D | l | L | v | V}

diagnose debug cli [{0..8}]

diagnose debug kernel level [{0..8}]

Debug commands produce output only while the related processes are active. Output for some commands is printed to your CLI display until you stop it by pressing Ctrl + C. Otherwise output can be controlled with commands such as:

diagnose debug setting duration <minutes_int>

diagnose debug application <process_name> duration <minutes_int>

diagnose debug application <process_name> output {default | file | stdout | terminal}

(Process-specific level and duration settings override general debug settings.)

Alternatively, you can use the GUI to download debug log files (also called a crash or trace log) and view them in a text editor. See the trace log information in the FortiMail Administration Guide.

Tooltip

Use diagnose debug commands:

  • At less busy times. Debugging is real-time and can be CPU intensive.

  • Via a local console or terminal server. If CPU or network usage is too much during debugging, then network connections such as via SSH may be less responsive. Local access also allows you to reboot FortiMail to immediately disable debugging.

Syntax

diagnose debug rating [<refresh-interval-seconds_int>]

diagnose debug tools vm-print-license

diagnose debug tools process list [<process_name>] [| grep <filter_str>]

diagnose debug tools process signal <signal_int>

diagnose debug setting rotate <files_int> <megabytes_int>

diagnose debug setting duration <minutes_int>

diagnose debug application <process_name> format {default | datetime | file-name | func-name | line-number | millisecond | pid | trace-name}

diagnose debug application <process_name> filter "<filter_str>"

diagnose debug application <process_name> output {default | file | stdout | terminal}

diagnose debug application <process_name> level {{0..8} | d | D | l | L | v | V}

diagnose debug cli [{0..8}]

diagnose debug database [{0..1}]

diagnose debug database general-log

diagnose debug database error-log

diagnose debug database debug-log

diagnose debug kernel display {pretty | <seconds-since-boot_int>}

diagnose debug kernel log enable

diagnose debug kernel level [{0..8}]

diagnose debug enable

diagnose debug application <process_name> enable

diagnose debug application <process_name> duration <minutes_int>

diagnose debug setting level display

diagnose debug setting level clear

diagnose debug setting level save

diagnose debug application <process_name> display

diagnose debug disable

diagnose debug tools crashlog

diagnose debug tools coredump status

diagnose debug tools coredump list [<process_name>]

diagnose debug tools coredump enable

diagnose debug tools coredump upload "<filename_str>" <tftp-server_ipv4> ["<destination-filename_str>"]

diagnose debug tools coredump delete "<filename_str>"

diagnose debug tools coredump clear

Variable

Description

Default

"<filename_str>"

Enter the file name on the FortiMail unit.

<process_name>

Enter the name of a process, such as smtpd or mailstatd.

<signal_int>

Enter a standard POSIX system signal number, such as:

  • 2 (SIGINT) or 15 (SIGTERM) — Quit with an interrupt signal. Normal termination for many processes. Depending on its state, the process may handle the quit gracefully such as saving state and releasing system resources instead of an immediate SIGKILL.

  • 3 (SIGQUIT) — Quit. Also output a core dump file.

  • 6 (SIGABRT) — Quit. Also output a crash log file. Unlike SIGSEGV, processes usually trigger this signal themselves.

  • 9 (SIGKILL) — Force quit.Unlike SIGINT, the process cannot ignore SIGKILL or handle the signal gracefully. Administrators may trigger this signal to terminate zombie processes.

  • 11 (SIGSEGV) — Force quit like SIGKILL. Also output the segmentation fault in the crash log file.

If a process crashes, crash logs and/or core dumps (trace files) should be automatically generated. Do send a signal in that case.

If processes have not crashed, but Fortinet Support requires detailed troubleshooting information, Support may ask you to send signal 6 or 11 to manually trigger a trace log.

<tftp-server_ipv4>

Enter the IP address of a TFTP server.

coredump

Display a kernel core dump file.

crashlog

Display a process crash log.

debug {enable | disable}

Enable or disable output of debug messages in the CLI display.

Note: diagnose debug setting duration <minutes_int> can only be entered while debugging is enabled. When you disable debugging or the duration time elapses, then the settings are reset.

disable

duration <minutes_int>

Enter how long in minutes to record debug information.

30

filter "<filter_str>"

Enter text that you want to find to search and filter output to show only matching lines.

format {default | datetime | file-name | func-name | line-number | millisecond | pid | trace-name}

Select the format of the debug file. default includes both the time and PID.

grep <filter_str>

Enter text that you want to find to search and filter output to show only matching lines.

For example, you could enter grep smtpd to show only lines with the SMTP process name.

level {{0..8} | d | D | l | L | v | V}

Select an amount or category of detail to include in the debug log, either:

  • a number, such as 0 or 8

  • d — Debug messages.

  • D — Verbose debug messages.

  • l — Library level debug messages.

  • L — Verbose library level debug messages.

  • v — Very verbose debug messages.

  • V — Very verbose library level debug messages.

To display the current detail level, enter:

diagnose debug application <process_name> display

(For diagnose debug commands for the kernel, CLI, or database, to display the current level, you omit the level number instead.)

level clear

Resets the saved verbosity level of debug information.

level display

Displays the current and saved verbosity level of debug information.

level save

Saves the current verbosity level of debug information so that it persists after reboot. To undo this, enter diagnose debug settinglevel clear.

output {default | file | stdout | terminal}

Select the display location of the command results, either:

  • default — Both the terminal and a debug log file.

  • file — Debug log file.

  • stdout — Monitor attached to the FortiMail appliance.

  • terminal — Local console or remote terminal emulator.

rating [<refresh-interval-seconds_int>]

Enter how frequently in seconds to refresh the display for the list of FortiGuard Antispam rating query servers.

If you omit the refresh interval, the CLI output gives a list of the servers and then immediately returns you to the command prompt.

0

rotate <files_int> <megabytes_int>

Enter the maximum number and file size in megabytes (MB) of debug files to keep.

10

vm-print-license

Display FortiMail-VM license information. See also system vm.

Example

FortiMail # diag debug rating

System Time:  2025-05-20 16:19:27 EDT (Uptime: 0d 4h 49m)
License      : Contract
Expiration   : Sat Mar  5 19:00:00 2033
Hostname     : N/A (Server Override Enabled)
-=- Server List (Tue May 20 16:19:27 2025) -=-
IP                  Weight    RTT Flags  TZ    Packets  Curr Lost Total Lost
192.168.100.206       -359     67 DI     -8        203          0          1

FortiMail # diag debug tools process list

System Time:  2025-05-20 16:07:12 EDT (Uptime: 0d 4h 37m)
PID                  STARTED   LWP COMMAND         S TTY          TIME COMMAND
1 Tue May 20 11:30:00 2025     1 init            S ?        00:00:01 /init
2 Tue May 20 11:30:00 2025     2 kthreadd        S ?        00:00:00 [kthreadd]
...

FortiMail # diag debug tools process signal 9 2192

System Time:  2025-05-20 16:07:12 EDT (Uptime: 0d 4h 37m)
signal 9 sent to pid 2152

FortiMail # diagnose debug setting level display

System Time:  2025-05-21 14:41:55 EDT (Uptime: 0d 0h 53m)
Current debug levels:
Cli debug level is 1
httpd debug level is: 0(0x0)
 
Persistent debug levels:
Cli debug level is 3

FortiMail # diag debug app httpd trace-log enable
System Time:  2025-06-05 10:04:35 EDT (Uptime: 0d 19h 47m)
This command will cause brief interruption to existing HTTP connections, and the console will be disconnected. Are you sure you want to continue? (y/n)y
FortiMail # diag debug app httpd display trace-log
System Time:  2025-06-05 10:05:40 EDT (Uptime: 0d 19h 48m)
2025-06-05T10:05:33 {"objectID": "SysTimeManual:","reqAction": 1,"nodePermission": 3,"daylight_saving_time": true,"zone": 12,"timezone_offset": 240}
2025-06-05T10:05:33 =========== END OF RESPONSE ===========
2025-06-05T10:05:38 ########### RECEIVED REQUEST ###########
2025-06-05T10:05:38 extraParam=minute&reqObject=SysStatusUsage&reqAction=1&skipTmUp=1
2025-06-05T10:05:38 ----------- END OF REQUEST -----------
2025-06-05T10:05:38 +++++++++++++ RESPONSE +++++++++++++
2025-06-05T10:05:38 {"objectID": "SysStatusUsage:","reqAction": 1,"nodePermission": 3,"cpu": 0,"memory": 63,"log_disk": 0,"mail_disk": 1,"system_load": 15,"active_sessions": 7,"network_usage": 93}
2025-06-05T10:05:38 =========== END OF RESPONSE ===========
^C
FortiMail # diag debug app httpd trace-log disable
System Time:  2025-06-05 10:07:34 EDT (Uptime: 0d 19h 50m)
This command will cause brief interruption to existing HTTP connections, and the console will be disconnected. Are you sure you want to continue? (y/n)y

Related topics

system status

sniffer

test

system vm