Fortinet black logo

Handbook

CLI HA status

6.0.0
Copy Link
Copy Doc ID 4afb0436-a998-11e9-81a4-00505692583a:79908
Download PDF

CLI HA status

Use the get system ha status command to display information about an HA cluster. The command displays general HA configuration settings. The command also displays information about how the cluster unit that you have logged into is operating in the cluster. You can enter the get system ha status command from the primary or backup units. The output produced by the command is similar for each unit, it shows cluster data as well as data for the FortiGate that you are logged into.

For a virtual cluster configuration, the get system ha status command displays information about how the cluster unit that you have logged into is operating in virtual cluster 1 and virtual cluster 2. For example, if you connect to the cluster unit that is the primary unit for virtual cluster 1 and the subordinate unit for virtual cluster 2, the output of the get system ha status command shows virtual cluster 1 in the work state and virtual cluster 2 in the standby state. The get system ha status command also displays additional information about virtual cluster 1 and virtual cluster 2.

The command includes the following fields.

Field Description
HA Health Status Indicates if all cluster units are operating normally (OK) or if a problem was detected with the cluster. For example, a message similar to ERROR <serial-number> is lost @ <date> <time> appears if one the subordinate units leaves the cluster.
Model The FortiGate model number.
Mode The HA mode of the cluster, for example, HA A-P or HA A-A.
Group The group ID of the cluster.
Debug The debug status of the cluster.
Cluster Uptime The number of days, hours, minutes, and seconds that the cluster has been operating.
Cluster state changed time The date and time at which the FortiGate most recently changed state. For example, the last time the FortiGate joined the cluster or changed from the primary unit to a backup unit, and so on.
Master selected using Shows how the primary unit was selected the last four times that the cluster negotiated. For example, when a cluster first forms, this part of the command output could have one line showing that the primary unit is the cluster unit with the highest uptime. Up to four lines can be included as the cluster negotiates to choose a new primary unit on different occasions. Each line includes a time stamp and the criteria used to select the primary unit.
ses_pickup The status of session pickup: enable or disable.
load_balance The status of the load-balance-all keyword: enable or disable. Active-active clusters only.
load_balance_udp The stats of the load-balance-udp keyword: enable or disable. Available on some FortiGate models. Active-active clusters only.
schedule The active-active load balancing schedule. Active-active clusters only.
override The status of the override option for the current cluster unit: enable or disable.
Configuration Status Shows if the configurations of each of the cluster units are synchronized or not.
System Usage stats Shows how busy each cluster unit is by displaying the number of sessions being processed by the cluster unit, CPU usage, and memory usage.
HBDEV stats Shows the status of each cluster unit's heartbeat interfaces. Includes whether the interfaces are up or down, how much data they have processed, as well as errors found.
Master
Slave
Displays the host name, serial number, and cluster index of the primary unit and the subordinate units. The FortiGate with cluster index 0 is the primary unit and the FortiGates with cluster indexes 1 to 3 are the backup units.

The order in which the cluster units are listed starts with the cluster unit that you are logged into.
number of vcluster The number of virtual clusters. If virtual domains are not enabled, the cluster has one virtual cluster. If virtual domains are enabled the cluster has two virtual clusters.
vcluster 1
vcluster 2
The heartbeat interface IP address of the primary unit in each virtual cluster. If virtual domains are not enabled there is one vcluster and this is the IP address of the primary unit. If virtual domains are enabled then each vcluster line will have an IP address. If the IP addresses are the same then the same FortiGate is the primary unit for both virtual clusters.
vcluster 1
Master
Slave
The HA state (hello, work, or standby) and HA heartbeat IP address of the primary unit. If virtual domains are not enabled, vcluster 1 displays information for the cluster. If virtual domains are enabled, vcluster 1 displays information for virtual cluster 1.

vcluster 1 also lists the primary unit and subordinate units in virtual cluster 1. The list includes the serial number and operating cluster index of each cluster unit in virtual cluster 1. The cluster unit that you have logged into is at the top of the list. The FortiGate in the cluster with the highest serial number always has an operating cluster index of 0. Other FortiGates in the cluster get a higher operating cluster index based in their serial number. When you use the execute ha manage command to log into another FortiGate you use the operating cluster index to specify the FortiGate to log into.

If virtual domains are not enabled and you connect to the primary unit CLI, the HA state of the cluster unit in virtual cluster 1 is work. The display lists the cluster units starting with the primary unit.

If virtual domains are not enabled and you connect to a subordinate unit CLI, the HA state of the cluster unit in virtual cluster 1 is standby. The display lists the cluster units starting with the subordinate unit that you have logged into.

If virtual domains are enabled and you connect to the virtual cluster 1 primary unit CLI, the HA state of the cluster unit in virtual cluster 1 is work. The display lists the cluster units starting with the virtual cluster 1 primary unit.

If virtual domains are enabled and you connect to the virtual cluster 1 subordinate unit CLI, the HA state of the cluster unit in virtual cluster 1 is standby. The display lists the cluster units starting with the subordinate unit that you are logged into.
vcluster 2
Master Slave
vcluster 2 only appears if virtual domains are enabled. vcluster 2 displays the HA state (hello, work, or standby) and HA heartbeat IP address of the cluster unit that you have logged into in virtual cluster 2. The HA heartbeat IP address is 169.254.0.2 if you are logged into the primary unit of virtual cluster 2 and 169.254.0.1 if you are logged into a subordinate unit of virtual cluster 2.

vcluster 2 also lists the primary unit and subordinate units in virtual cluster 2. The list includes the cluster index and serial number of each cluster unit in virtual cluster 2. The cluster unit that you have logged into is at the top of the list.

If you connect to the virtual cluster 2 primary unit CLI, the HA state of the cluster unit in virtual cluster 2 is work. The display lists the cluster units starting with the virtual cluster 2 primary unit.

If you connect to the virtual cluster 2 subordinate unit CLI, the HA state of the cluster unit in virtual cluster 2 is standby. The display lists the cluster units starting with the subordinate unit that you are logged into.

Get system ha status example - two FortiGates in active-passive mode

The following example shows get system ha status output for a cluster of two FortiGate-600Ds operating in active-passive mode. The cluster is healthy and has been running for 3 hours and 26 minutes. Primary unit selection took place once and the cluster has been stable since then.

The following command output was produced by connecting to the primary unit CLI (host name Edge2-Primary).

get system ha status
HA Health Status: OK
Model: FortiGate-600D
Mode: HA A-P
Group: 25
Debug: 0
Cluster Uptime: 0 days 03:26:00
Cluster state change time: 2018-03-06 13:16:33
Master selected using:
    <2018/03/06 13:16:33> FGT6HD3916806098 is selected as the master because it has the largest value of override priority.
    <2018/03/06 12:47:58> FGT6HD3916806070 is selected as the master because it has the largest value of override priority.
    <2018/03/06 12:47:55> FGT6HD3916806098 is selected as the master because it has the largest value of uptime.
    <2018/03/06 12:47:55> FGT6HD3916806098 is selected as the master because it's the only member in the cluster.
ses_pickup: enable, ses_pickup_delay=disable
override: disable
Configuration Status:
    FGT6HD3916806098(updated 1 seconds ago): in-sync
    FGT6HD3916806070(updated 2 seconds ago): in-sync
System Usage stats:
    FGT6HD3916806098(updated 1 seconds ago):
        sessions=141, average-cpu-user/nice/system/idle=0%/0%/0%/100%, memory=34%
    FGT6HD3916806070(updated 2 seconds ago):
        sessions=12, average-cpu-user/nice/system/idle=0%/0%/0%/100%, memory=33%
HBDEV stats:
    FGT6HD3916806098(updated 1 seconds ago):
        port3: physical/1000full, up, rx-bytes/packets/dropped/errors=45437370/71531/0/0, tx=36186194/65035/0/0
        port4: physical/1000full, up, rx-bytes/packets/dropped/errors=27843923/39221/0/0, tx=27510707/39075/0/0
    FGT6HD3916806070(updated 2 seconds ago):
        port3: physical/1000full, up, rx-bytes/packets/dropped/errors=37267057/67136/0/0, tx=46354380/73516/0/0
        port4: physical/1000full, up, rx-bytes/packets/dropped/errors=28294029/40177/0/0, tx=28536766/40208/0/0
Master: Edge2-Primary  , FGT6HD3916806098, cluster index = 0
Slave : Edge2-Backup   , FGT6HD3916806070, cluster index = 1
number of vcluster: 1
vcluster 1: work 169.254.0.1
Master: FGT6HD3916806098, operating cluster index = 0
Slave : FGT6HD3916806070, operating cluster index = 1

The following command output was produced by using execute ha manage 1 to log into the subordinate unit CLI of the cluster shown in the previous example. The host name of the subordinate unit is Edge2-Backup.

get system ha status
HA Health Status: OK
Model: FortiGate-600D
Mode: HA A-P
Group: 25
Debug: 0
Cluster Uptime: 0 days 03:33:04
Cluster state change time: 2018-03-06 13:16:33
Master selected using:
    <2018/03/06 13:16:33> FGT6HD3916806098 is selected as the master because it has the largest value of override priority.
    <2018/03/06 12:47:58> FGT6HD3916806070 is selected as the master because it has the largest value of override priority.
    <2018/03/06 12:47:57> FGT6HD3916806098 is selected as the master because it has the largest value of uptime.
    <2018/03/06 12:47:56> FGT6HD3916806098 is selected as the master because it has the largest value of uptime.
ses_pickup: enable, ses_pickup_delay=disable
override: disable
Configuration Status:
    FGT6HD3916806070(updated 1 seconds ago): in-sync
    FGT6HD3916806098(updated 1 seconds ago): in-sync
System Usage stats:
    FGT6HD3916806070(updated 1 seconds ago):
        sessions=20, average-cpu-user/nice/system/idle=0%/0%/0%/100%, memory=34%
    FGT6HD3916806098(updated 1 seconds ago):
        sessions=163, average-cpu-user/nice/system/idle=0%/0%/0%/100%, memory=34%
HBDEV stats:
    FGT6HD3916806070(updated 1 seconds ago):
        port3: physical/1000full, up, rx-bytes/packets/dropped/errors=40755112/71809/0/0, tx=48104698/76943/0/0
        port4: physical/1000full, up, rx-bytes/packets/dropped/errors=29804904/42302/0/0, tx=30030641/42333/0/0
    FGT6HD3916806098(updated 1 seconds ago):
        port3: physical/1000full, up, rx-bytes/packets/dropped/errors=47188898/74965/0/0, tx=39680065/69723/0/0
        port4: physical/1000full, up, rx-bytes/packets/dropped/errors=29338501/41347/0/0, tx=29022293/41201/0/0
Slave : Edge2-Backup   , FGT6HD3916806070, cluster index = 1
Master: Edge2-Primary  , FGT6HD3916806098, cluster index = 0
number of vcluster: 1
vcluster 1: standby 169.254.0.1
Slave : FGT6HD3916806070, operating cluster index = 1
Master: FGT6HD3916806098, operating cluster index = 0

About the HA operating cluster index and the execute ha manage command

When a cluster starts up, if primary unit select is based on serial number, the FortiGate Cluster Protocol (FGCP) assigns a cluster index and an HA heartbeat IP address to each cluster unit based on the serial number of the cluster unit:

  • The FGCP selects the cluster unit with the highest serial number to become the primary unit. The FGCP assigns a cluster index of 0, an operating cluster index of 0, and an HA heartbeat IP address of 169.254.0.1 to this unit.
  • The FGCP assigns a cluster index of 1, an operating cluster index of 1, and an HA heartbeat IP address of 169.254.0.2 to the cluster unit with the second highest serial number.
  • If the cluster contains more units, the cluster unit with the third highest serial number is assigned a cluster index of 2, and operating cluster index of 2, and an HA heartbeat IP address of 169.254.0.3, and so on.

You can display the cluster index and operating cluster index assigned to each cluster unit using the get system ha status command. When you use the execute ha manage command you select a cluster unit to log into by entering its operating cluster index.

The operating cluster index and HA heartbeat IP address only change if a unit leaves the cluster or if a new unit joins the cluster. When one of these events happens, the FGCP resets the cluster index, operating cluster index, and HA heartbeat IP address of each cluster unit according to serial number in the same way as when the cluster first starts up.

If FortiGates don't leave or join, each cluster unit keeps its assigned operating cluster index, and HA heartbeat IP address since these are based on the FortiGate serial number, even as the units take on different roles in the cluster. After the operating cluster index and HA heartbeat IP addresses are set according to serial number, the FGCP checks other primary unit selection criteria such as device priority and monitored interfaces. Checking these criteria could result in selecting a cluster unit without the highest serial number to operate as the primary unit.

Even if the cluster unit without the highest serial number now becomes the primary unit, the operating cluster indexes and HA heartbeat IP addresses assigned to the individual cluster units do not change. Instead the FGCP changes the cluster index to reflect this role change. The cluster index is always 0 for the primary unit and 1 and higher for the other units in the cluster. By default both sets of cluster indexes are the same. But if primary unit selection selects the cluster unit that does not have the highest serial number to be the primary unit, then this cluster unit is assigned a cluster index of 0.

Using the execute ha manage command

When you use the CLI command execute ha manage <index_integer> to connect to the CLI of another cluster unit, the <index_integer> that you enter is the operating cluster index of the unit that you want to connect to.

Using get system ha status to display cluster indexes

You can display the cluster index assigned to each cluster unit using the CLI command get system ha status. The following example shows the information displayed by the get system ha status command for a cluster consisting of two FortiGates operating in active-passive HA mode with virtual domains not enabled and without virtual clustering.

get system ha status
.
.
.
Master: Edge2-Primary  , FGT6HD3916806098, cluster index = 0
Slave : Edge2-Backup   , FGT6HD3916806070, cluster index = 1
number of vcluster: 1
vcluster 1: work 169.254.0.1
Master: FGT6HD3916806098, operating cluster index = 0
Slave : FGT6HD3916806070, operating cluster index = 1

In this example, the cluster unit with serial number FGT6HD3916806098 has the highest serial number and so has a cluster index and an operating cluster index of 0 and the cluster unit with serial number FGT6HD3916806070 has a cluster index and an operating cluster index of 1. From the CLI of the primary unit of this cluster you can connect to the CLI of the subordinate unit using the following command:

execute ha manage 1

This works because the cluster unit with serial number FGT6HD3916806070 has a cluster index of 1.

The last three lines of the command output display the status of vcluster 1. In a cluster consisting of two cluster units operating without virtual domains enabled, all clustering actually takes place in virtual cluster 1. HA is designed to work this way to support virtual clustering. If this cluster was operating with virtual domains enabled, adding virtual cluster 2 is similar to adding a new copy of virtual cluster 1. Virtual cluster 2 is visible in the get system ha status command output when you add virtual domains to virtual cluster 2.

The HA heartbeat IP address displayed by the command is the HA heartbeat IP address of the cluster unit that is actually operating as the primary unit. For a default configuration, this IP address will always be 169.254.0.1 because the cluster unit with the highest serial number will be the primary unit. This IP address changes if the operating primary unit is not the primary unit with the highest serial number.

Example where the cluster index and operating cluster index do not match

This example shows get system ha status command output for the same cluster. However, in this example the device priority of the cluster unit with the serial number FGT6HD3916806098 is increased to 250. As a result the cluster unit with the lowest serial number becomes the primary unit. This means the cluster index and the operating cluster index of the cluster units do not match.

get system ha status
.
.
.
Master: Edge2-Primary  , FGT6HD3916806098, cluster index = 1
Slave : Edge2-Backup   , FGT6HD3916806070, cluster index = 0
number of vcluster: 1
vcluster 1: work 169.254.0.2
Master: FGT6HD3916806098, operating cluster index = 0
Slave : FGT6HD3916806070, operating cluster index = 1

The actual cluster indexes have not changed but the operating cluster indexes have. Also, the HA heartbeat IP address displayed for vcluster 1 has changed to 169.254.0.2.

Virtual clustering example output

The get system ha status command output is the same if a cluster is operating with virtual clustering turned on but with all virtual domains in virtual cluster 1. The following get system ha status command output example shows the same cluster operating as a virtual cluster with virtual domains in virtual cluster 1 and added to virtual cluster 2. In this example the cluster unit with serial number FG50012204400045 is the primary unit for virtual cluster 1 and the cluster unit with serial number FG50012205400050 is the primary unit for virtual cluster 2.

get system ha status

.

.

.

number of vcluster: 2

vcluster 1: work 169.254.0.2

Master: FG50012205400050, operating cluster index = 1

Slave : FG50012204400045, operating cluster index = 0

vcluster 2: standby 169.254.0.1

Master: FG50012205400050, operating cluster index = 0

Slave : FG50012204400045, operating cluster index = 1

This example shows three sets of indexes. The indexes in lines six and seven are still used by the execute ha manage command. The indexes on lines ten and eleven are for the primary and subordinate units in virtual cluster 1 and the indexes on the last two lines are for virtual cluster 2.

CLI HA status

Use the get system ha status command to display information about an HA cluster. The command displays general HA configuration settings. The command also displays information about how the cluster unit that you have logged into is operating in the cluster. You can enter the get system ha status command from the primary or backup units. The output produced by the command is similar for each unit, it shows cluster data as well as data for the FortiGate that you are logged into.

For a virtual cluster configuration, the get system ha status command displays information about how the cluster unit that you have logged into is operating in virtual cluster 1 and virtual cluster 2. For example, if you connect to the cluster unit that is the primary unit for virtual cluster 1 and the subordinate unit for virtual cluster 2, the output of the get system ha status command shows virtual cluster 1 in the work state and virtual cluster 2 in the standby state. The get system ha status command also displays additional information about virtual cluster 1 and virtual cluster 2.

The command includes the following fields.

Field Description
HA Health Status Indicates if all cluster units are operating normally (OK) or if a problem was detected with the cluster. For example, a message similar to ERROR <serial-number> is lost @ <date> <time> appears if one the subordinate units leaves the cluster.
Model The FortiGate model number.
Mode The HA mode of the cluster, for example, HA A-P or HA A-A.
Group The group ID of the cluster.
Debug The debug status of the cluster.
Cluster Uptime The number of days, hours, minutes, and seconds that the cluster has been operating.
Cluster state changed time The date and time at which the FortiGate most recently changed state. For example, the last time the FortiGate joined the cluster or changed from the primary unit to a backup unit, and so on.
Master selected using Shows how the primary unit was selected the last four times that the cluster negotiated. For example, when a cluster first forms, this part of the command output could have one line showing that the primary unit is the cluster unit with the highest uptime. Up to four lines can be included as the cluster negotiates to choose a new primary unit on different occasions. Each line includes a time stamp and the criteria used to select the primary unit.
ses_pickup The status of session pickup: enable or disable.
load_balance The status of the load-balance-all keyword: enable or disable. Active-active clusters only.
load_balance_udp The stats of the load-balance-udp keyword: enable or disable. Available on some FortiGate models. Active-active clusters only.
schedule The active-active load balancing schedule. Active-active clusters only.
override The status of the override option for the current cluster unit: enable or disable.
Configuration Status Shows if the configurations of each of the cluster units are synchronized or not.
System Usage stats Shows how busy each cluster unit is by displaying the number of sessions being processed by the cluster unit, CPU usage, and memory usage.
HBDEV stats Shows the status of each cluster unit's heartbeat interfaces. Includes whether the interfaces are up or down, how much data they have processed, as well as errors found.
Master
Slave
Displays the host name, serial number, and cluster index of the primary unit and the subordinate units. The FortiGate with cluster index 0 is the primary unit and the FortiGates with cluster indexes 1 to 3 are the backup units.

The order in which the cluster units are listed starts with the cluster unit that you are logged into.
number of vcluster The number of virtual clusters. If virtual domains are not enabled, the cluster has one virtual cluster. If virtual domains are enabled the cluster has two virtual clusters.
vcluster 1
vcluster 2
The heartbeat interface IP address of the primary unit in each virtual cluster. If virtual domains are not enabled there is one vcluster and this is the IP address of the primary unit. If virtual domains are enabled then each vcluster line will have an IP address. If the IP addresses are the same then the same FortiGate is the primary unit for both virtual clusters.
vcluster 1
Master
Slave
The HA state (hello, work, or standby) and HA heartbeat IP address of the primary unit. If virtual domains are not enabled, vcluster 1 displays information for the cluster. If virtual domains are enabled, vcluster 1 displays information for virtual cluster 1.

vcluster 1 also lists the primary unit and subordinate units in virtual cluster 1. The list includes the serial number and operating cluster index of each cluster unit in virtual cluster 1. The cluster unit that you have logged into is at the top of the list. The FortiGate in the cluster with the highest serial number always has an operating cluster index of 0. Other FortiGates in the cluster get a higher operating cluster index based in their serial number. When you use the execute ha manage command to log into another FortiGate you use the operating cluster index to specify the FortiGate to log into.

If virtual domains are not enabled and you connect to the primary unit CLI, the HA state of the cluster unit in virtual cluster 1 is work. The display lists the cluster units starting with the primary unit.

If virtual domains are not enabled and you connect to a subordinate unit CLI, the HA state of the cluster unit in virtual cluster 1 is standby. The display lists the cluster units starting with the subordinate unit that you have logged into.

If virtual domains are enabled and you connect to the virtual cluster 1 primary unit CLI, the HA state of the cluster unit in virtual cluster 1 is work. The display lists the cluster units starting with the virtual cluster 1 primary unit.

If virtual domains are enabled and you connect to the virtual cluster 1 subordinate unit CLI, the HA state of the cluster unit in virtual cluster 1 is standby. The display lists the cluster units starting with the subordinate unit that you are logged into.
vcluster 2
Master Slave
vcluster 2 only appears if virtual domains are enabled. vcluster 2 displays the HA state (hello, work, or standby) and HA heartbeat IP address of the cluster unit that you have logged into in virtual cluster 2. The HA heartbeat IP address is 169.254.0.2 if you are logged into the primary unit of virtual cluster 2 and 169.254.0.1 if you are logged into a subordinate unit of virtual cluster 2.

vcluster 2 also lists the primary unit and subordinate units in virtual cluster 2. The list includes the cluster index and serial number of each cluster unit in virtual cluster 2. The cluster unit that you have logged into is at the top of the list.

If you connect to the virtual cluster 2 primary unit CLI, the HA state of the cluster unit in virtual cluster 2 is work. The display lists the cluster units starting with the virtual cluster 2 primary unit.

If you connect to the virtual cluster 2 subordinate unit CLI, the HA state of the cluster unit in virtual cluster 2 is standby. The display lists the cluster units starting with the subordinate unit that you are logged into.

Get system ha status example - two FortiGates in active-passive mode

The following example shows get system ha status output for a cluster of two FortiGate-600Ds operating in active-passive mode. The cluster is healthy and has been running for 3 hours and 26 minutes. Primary unit selection took place once and the cluster has been stable since then.

The following command output was produced by connecting to the primary unit CLI (host name Edge2-Primary).

get system ha status
HA Health Status: OK
Model: FortiGate-600D
Mode: HA A-P
Group: 25
Debug: 0
Cluster Uptime: 0 days 03:26:00
Cluster state change time: 2018-03-06 13:16:33
Master selected using:
    <2018/03/06 13:16:33> FGT6HD3916806098 is selected as the master because it has the largest value of override priority.
    <2018/03/06 12:47:58> FGT6HD3916806070 is selected as the master because it has the largest value of override priority.
    <2018/03/06 12:47:55> FGT6HD3916806098 is selected as the master because it has the largest value of uptime.
    <2018/03/06 12:47:55> FGT6HD3916806098 is selected as the master because it's the only member in the cluster.
ses_pickup: enable, ses_pickup_delay=disable
override: disable
Configuration Status:
    FGT6HD3916806098(updated 1 seconds ago): in-sync
    FGT6HD3916806070(updated 2 seconds ago): in-sync
System Usage stats:
    FGT6HD3916806098(updated 1 seconds ago):
        sessions=141, average-cpu-user/nice/system/idle=0%/0%/0%/100%, memory=34%
    FGT6HD3916806070(updated 2 seconds ago):
        sessions=12, average-cpu-user/nice/system/idle=0%/0%/0%/100%, memory=33%
HBDEV stats:
    FGT6HD3916806098(updated 1 seconds ago):
        port3: physical/1000full, up, rx-bytes/packets/dropped/errors=45437370/71531/0/0, tx=36186194/65035/0/0
        port4: physical/1000full, up, rx-bytes/packets/dropped/errors=27843923/39221/0/0, tx=27510707/39075/0/0
    FGT6HD3916806070(updated 2 seconds ago):
        port3: physical/1000full, up, rx-bytes/packets/dropped/errors=37267057/67136/0/0, tx=46354380/73516/0/0
        port4: physical/1000full, up, rx-bytes/packets/dropped/errors=28294029/40177/0/0, tx=28536766/40208/0/0
Master: Edge2-Primary  , FGT6HD3916806098, cluster index = 0
Slave : Edge2-Backup   , FGT6HD3916806070, cluster index = 1
number of vcluster: 1
vcluster 1: work 169.254.0.1
Master: FGT6HD3916806098, operating cluster index = 0
Slave : FGT6HD3916806070, operating cluster index = 1

The following command output was produced by using execute ha manage 1 to log into the subordinate unit CLI of the cluster shown in the previous example. The host name of the subordinate unit is Edge2-Backup.

get system ha status
HA Health Status: OK
Model: FortiGate-600D
Mode: HA A-P
Group: 25
Debug: 0
Cluster Uptime: 0 days 03:33:04
Cluster state change time: 2018-03-06 13:16:33
Master selected using:
    <2018/03/06 13:16:33> FGT6HD3916806098 is selected as the master because it has the largest value of override priority.
    <2018/03/06 12:47:58> FGT6HD3916806070 is selected as the master because it has the largest value of override priority.
    <2018/03/06 12:47:57> FGT6HD3916806098 is selected as the master because it has the largest value of uptime.
    <2018/03/06 12:47:56> FGT6HD3916806098 is selected as the master because it has the largest value of uptime.
ses_pickup: enable, ses_pickup_delay=disable
override: disable
Configuration Status:
    FGT6HD3916806070(updated 1 seconds ago): in-sync
    FGT6HD3916806098(updated 1 seconds ago): in-sync
System Usage stats:
    FGT6HD3916806070(updated 1 seconds ago):
        sessions=20, average-cpu-user/nice/system/idle=0%/0%/0%/100%, memory=34%
    FGT6HD3916806098(updated 1 seconds ago):
        sessions=163, average-cpu-user/nice/system/idle=0%/0%/0%/100%, memory=34%
HBDEV stats:
    FGT6HD3916806070(updated 1 seconds ago):
        port3: physical/1000full, up, rx-bytes/packets/dropped/errors=40755112/71809/0/0, tx=48104698/76943/0/0
        port4: physical/1000full, up, rx-bytes/packets/dropped/errors=29804904/42302/0/0, tx=30030641/42333/0/0
    FGT6HD3916806098(updated 1 seconds ago):
        port3: physical/1000full, up, rx-bytes/packets/dropped/errors=47188898/74965/0/0, tx=39680065/69723/0/0
        port4: physical/1000full, up, rx-bytes/packets/dropped/errors=29338501/41347/0/0, tx=29022293/41201/0/0
Slave : Edge2-Backup   , FGT6HD3916806070, cluster index = 1
Master: Edge2-Primary  , FGT6HD3916806098, cluster index = 0
number of vcluster: 1
vcluster 1: standby 169.254.0.1
Slave : FGT6HD3916806070, operating cluster index = 1
Master: FGT6HD3916806098, operating cluster index = 0

About the HA operating cluster index and the execute ha manage command

When a cluster starts up, if primary unit select is based on serial number, the FortiGate Cluster Protocol (FGCP) assigns a cluster index and an HA heartbeat IP address to each cluster unit based on the serial number of the cluster unit:

  • The FGCP selects the cluster unit with the highest serial number to become the primary unit. The FGCP assigns a cluster index of 0, an operating cluster index of 0, and an HA heartbeat IP address of 169.254.0.1 to this unit.
  • The FGCP assigns a cluster index of 1, an operating cluster index of 1, and an HA heartbeat IP address of 169.254.0.2 to the cluster unit with the second highest serial number.
  • If the cluster contains more units, the cluster unit with the third highest serial number is assigned a cluster index of 2, and operating cluster index of 2, and an HA heartbeat IP address of 169.254.0.3, and so on.

You can display the cluster index and operating cluster index assigned to each cluster unit using the get system ha status command. When you use the execute ha manage command you select a cluster unit to log into by entering its operating cluster index.

The operating cluster index and HA heartbeat IP address only change if a unit leaves the cluster or if a new unit joins the cluster. When one of these events happens, the FGCP resets the cluster index, operating cluster index, and HA heartbeat IP address of each cluster unit according to serial number in the same way as when the cluster first starts up.

If FortiGates don't leave or join, each cluster unit keeps its assigned operating cluster index, and HA heartbeat IP address since these are based on the FortiGate serial number, even as the units take on different roles in the cluster. After the operating cluster index and HA heartbeat IP addresses are set according to serial number, the FGCP checks other primary unit selection criteria such as device priority and monitored interfaces. Checking these criteria could result in selecting a cluster unit without the highest serial number to operate as the primary unit.

Even if the cluster unit without the highest serial number now becomes the primary unit, the operating cluster indexes and HA heartbeat IP addresses assigned to the individual cluster units do not change. Instead the FGCP changes the cluster index to reflect this role change. The cluster index is always 0 for the primary unit and 1 and higher for the other units in the cluster. By default both sets of cluster indexes are the same. But if primary unit selection selects the cluster unit that does not have the highest serial number to be the primary unit, then this cluster unit is assigned a cluster index of 0.

Using the execute ha manage command

When you use the CLI command execute ha manage <index_integer> to connect to the CLI of another cluster unit, the <index_integer> that you enter is the operating cluster index of the unit that you want to connect to.

Using get system ha status to display cluster indexes

You can display the cluster index assigned to each cluster unit using the CLI command get system ha status. The following example shows the information displayed by the get system ha status command for a cluster consisting of two FortiGates operating in active-passive HA mode with virtual domains not enabled and without virtual clustering.

get system ha status
.
.
.
Master: Edge2-Primary  , FGT6HD3916806098, cluster index = 0
Slave : Edge2-Backup   , FGT6HD3916806070, cluster index = 1
number of vcluster: 1
vcluster 1: work 169.254.0.1
Master: FGT6HD3916806098, operating cluster index = 0
Slave : FGT6HD3916806070, operating cluster index = 1

In this example, the cluster unit with serial number FGT6HD3916806098 has the highest serial number and so has a cluster index and an operating cluster index of 0 and the cluster unit with serial number FGT6HD3916806070 has a cluster index and an operating cluster index of 1. From the CLI of the primary unit of this cluster you can connect to the CLI of the subordinate unit using the following command:

execute ha manage 1

This works because the cluster unit with serial number FGT6HD3916806070 has a cluster index of 1.

The last three lines of the command output display the status of vcluster 1. In a cluster consisting of two cluster units operating without virtual domains enabled, all clustering actually takes place in virtual cluster 1. HA is designed to work this way to support virtual clustering. If this cluster was operating with virtual domains enabled, adding virtual cluster 2 is similar to adding a new copy of virtual cluster 1. Virtual cluster 2 is visible in the get system ha status command output when you add virtual domains to virtual cluster 2.

The HA heartbeat IP address displayed by the command is the HA heartbeat IP address of the cluster unit that is actually operating as the primary unit. For a default configuration, this IP address will always be 169.254.0.1 because the cluster unit with the highest serial number will be the primary unit. This IP address changes if the operating primary unit is not the primary unit with the highest serial number.

Example where the cluster index and operating cluster index do not match

This example shows get system ha status command output for the same cluster. However, in this example the device priority of the cluster unit with the serial number FGT6HD3916806098 is increased to 250. As a result the cluster unit with the lowest serial number becomes the primary unit. This means the cluster index and the operating cluster index of the cluster units do not match.

get system ha status
.
.
.
Master: Edge2-Primary  , FGT6HD3916806098, cluster index = 1
Slave : Edge2-Backup   , FGT6HD3916806070, cluster index = 0
number of vcluster: 1
vcluster 1: work 169.254.0.2
Master: FGT6HD3916806098, operating cluster index = 0
Slave : FGT6HD3916806070, operating cluster index = 1

The actual cluster indexes have not changed but the operating cluster indexes have. Also, the HA heartbeat IP address displayed for vcluster 1 has changed to 169.254.0.2.

Virtual clustering example output

The get system ha status command output is the same if a cluster is operating with virtual clustering turned on but with all virtual domains in virtual cluster 1. The following get system ha status command output example shows the same cluster operating as a virtual cluster with virtual domains in virtual cluster 1 and added to virtual cluster 2. In this example the cluster unit with serial number FG50012204400045 is the primary unit for virtual cluster 1 and the cluster unit with serial number FG50012205400050 is the primary unit for virtual cluster 2.

get system ha status

.

.

.

number of vcluster: 2

vcluster 1: work 169.254.0.2

Master: FG50012205400050, operating cluster index = 1

Slave : FG50012204400045, operating cluster index = 0

vcluster 2: standby 169.254.0.1

Master: FG50012205400050, operating cluster index = 0

Slave : FG50012204400045, operating cluster index = 1

This example shows three sets of indexes. The indexes in lines six and seven are still used by the execute ha manage command. The indexes on lines ten and eleven are for the primary and subordinate units in virtual cluster 1 and the indexes on the last two lines are for virtual cluster 2.