FortiOS 5.6 Online Help Link FortiOS 5.4 Online Help Link FortiOS 5.2 Online Help Link FortiOS 5.0 Online Help Link
> Chapter 15 - High Availability > FGCP configuration examples and troubleshooting > Setting up two new FortiGates as an FGCP cluster

Home > Online Help

Setting up two new FortiGates as an FGCP cluster

This section describes a simple HA network topology that includes an HA cluster of two FortiGates in NAT/Route mode installed between an internal network and the Internet. The example uses a generic FortiGate with four interfaces named port1, port2, port3 and port4.

Example NAT/Route mode HA network topology

The figure below shows a typical FortiGate HA cluster consisting of two FortiGates (FGT_ha_1 and FGT_ha_2) connected to the same internal (port2) and external (port1) networks.

Example NAT/Route mode HA network topology

Port3 and port4 are used as the heartbeat interfaces. Because the cluster consists of two FortiGates, you can make the connections between the heartbeat interfaces using crossover cables. You could also use switches and regular ethernet cables.

General configuration steps

The section includes GUI and CLI procedures. These procedures assume that the FortiGates are running the same FortiOS firmware build and are set to the factory default configuration.

General configuration steps
  1. Apply licenses to the FortiGates to become the cluster.
  2. Configure the FortiGates for HA operation.
  • Optionally change each unit’s host name.
  • Configure HA.
  1. Connect the cluster to the network.
  2. Confirm that the cluster units are operating as a cluster and add basic configuration settings to the cluster.
  • View cluster status from the GUI or CLI.
  • Add a password for the admin administrative account.
  • Change the IP addresses and netmasks of the internal and external interfaces.
  • Add a default route.

Configuring a NAT/Route mode active-passive cluster of two FortiGates - GUI

Use the following procedures to configure two FortiGates for NAT/Route HA operation using the GUI. These procedures assume you are starting with two FortiGates with factory default settings.

note icon Give each cluster unit a unique host name to make the individual units easier to identify when they are part of a functioning cluster. The default host name is the FortiGate serial number. You may want to change this host name to something more meaningful for your network.
To configure the first FortiGate (host name FGT_ha_1)
  1. Register and apply licenses to the FortiGate. This includes FortiCloud activation and FortiClient licensing, and entering a license key if you purchased more than 10 Virtual Domains (VDOMS). All of the FortiGates in a cluster must have the same level of licensing.
  2. You can also install any third-party certificates on the primary FortiGate before forming the cluster. Once the cluster is formed third-party certificates are synchronized to the backup FortiGate.
    We recommend that you add FortiToken licenses and FortiTokens to the primary unit after the cluster has formed.
  3. On the System Information dashboard widget beside Host Name, select Change.
  4. Enter a new Host Name for this FortiGate.
New Name FGT_ha_1
  1. Select OK.
  2. Go to System > HA and change the following settings:
Mode Active-Passive
Group Name example1.com
Password HA_pass_1
note icon This is the minimum recommended configuration for an active-passive HA cluster. You can also configure other HA options, but if you wait until after the cluster is operating you will only have to configure these options once for the cluster instead of separately for each unit in the cluster.
  1. Select OK.
    The FortiGate negotiates to establish an HA cluster. When you select OK you may temporarily lose connectivity with the FortiGate as the HA cluster negotiates and the FGCP changes the MAC address of the FortiGate interfaces. The MAC addresses of the FortiGate interfaces change to the following virtual MAC addresses:
  • port1 interface virtual MAC: 00-09-0f-09-00-00
  • port2 interface virtual MAC: 00-09-0f-09-00-01
  • port3 interface virtual MAC: 00-09-0f-09-00-02
  • port4 interface virtual MAC: 00-09-0f-09-00-03

To reconnect sooner, you can update the ARP table of your management PC by deleting the ARP table entry for the FortiGate (or just deleting all arp table entries). You may be able to delete the arp table of your management PC from a command prompt using a command similar to arp -d.

To confirm these MAC address changes, you can use the get hardware nic (or diagnose hardware deviceinfo nic) CLI command to view the virtual MAC address of any FortiGate interface. For example, use the following command to view the port1 interface virtual MAC address (MAC) and the port1 permanent MAC address (Permanent_HWaddr):

get hardware nic port1

.

.

.

Current_HWaddr:   00:09:0f:09:00:00

Permanent_HWaddr  02:09:0f:78:18:c9

.

.

.

 

  1. Power off the first FortiGate (FGT_ha_1).
note icon Note the details and format of the output of the get hardware nic command are specific to the interface hardware. Different FortiGate models and even different interfaces in the same FortiGate may have different output.
To configure the second FortiGate (host name FGT_ha_2)
  1. On the System Information dashboard widget, beside Host Name select Change.
  2. Enter a new Host Name for this FortiGate.
New Name FGT_ha_2
  1. Select OK.
  2. Go to System > HA and change the following settings:
Mode Active-Passive
Group Name example1.com
Password HA_pass_1
  1. Select OK.

The FortiGate negotiates to establish an HA cluster. When you select OK you may temporarily lose connectivity with the FortiGate as the HA cluster negotiates and because the FGCP changes the MAC address of the FortiGate interfaces.

To reconnect sooner, you can update the ARP table of your management PC by deleting the ARP table entry for the FortiGate (or just deleting all arp table entries). You may be able to delete the arp table of your management PC from a command prompt using a command similar to arp -d.

  1. Power off the second FortiGate.
To connect the cluster to the network
  1. Connect the port1 interfaces of FGT_ha_1 and FGT_ha_2 to a switch connected to the Internet.
  2. Connect the port2 interfaces of FGT_ha_1 and FGT_ha_2 to a switch connected to the internal network.
  3. Connect the port3 interfaces of FGT_ha_1 and FGT_ha_2 together. You can use a crossover Ethernet cable or regular Ethernet cables and a switch.
  4. Connect the port4 interfaces of the cluster units together. You can use a crossover Ethernet cable or regular Ethernet cables and a switch.
  5. Power on the cluster units.

The units start and negotiate to choose the primary unit and the subordinate unit. This negotiation occurs with no user intervention and normally takes less than a minute.

When negotiation is complete the cluster is ready to be configured for your network.

To view cluster status

Use the following steps to view the cluster dashboard and cluster members list to confirm that the cluster units are operating as a cluster.

note icon Once the cluster is operating, because configuration changes are synchronized to all cluster units, configuring the cluster is the same as configuring an individual FortiGate. You could have performed the following configuration steps separately on each FortiGate before you connected them to form a cluster.
  1. Start Internet Explorer and browse to the address https://192.168.1.99 (remember to include the “s” in https://).
    The FortiGate Login is displayed.
  2. Type admin in the Name field and select Login.
    The FortiGate dashboard is displayed.

The System Information dashboard widget shows the Cluster Name (example1.com) and the host names and serial numbers of the Cluster Members. The Unit Operation widget shows multiple cluster units.

  1. Go to System > HA to view the cluster members list.

The list shows both cluster units, their host names, their roles in the cluster, and their device priorities. You can use this list to confirm that the cluster is operating normally. For example, if the list shows only one cluster unit then the other unit has left the cluster for some reason.

To troubleshoot the cluster configuration

If the cluster members list and the dashboard do not display information for both cluster units, the FortiGates are not functioning as a cluster. See Troubleshooting HA clusters to troubleshoot the cluster.

To add basic configuration settings to the cluster

Use the following steps to configure the cluster to connect to its network. The following are example configuration steps only and do not represent all of the steps required to configure the cluster for a given network.

  1. Log into the cluster GUI.
  2. Go to System > Admin > Administrators.
  3. Edit admin and select Change Password.
  4. Enter and confirm a new password.
  5. Select OK.
  6. Go to System > Network > Interfaces.
  7. Edit the port2 interface and change IP/Netmask to 10.11.101.100/24.
  8. Select OK.
note icon After changing the IP address of the port1 interface you may have to change the IP address of your management computer and then reconnect to the port1 interface using the 172.20.120.141 IP address.
  1. Edit the port1 interface and change IP/Netmask to 172.20.120.141/24.
  2. Select OK.
  3. Go to Router > Static > Static Routes.
  4. Change the default route.
Destination IP/Mask 0.0.0.0/0.0.0.0
Gateway 172.20.120.2
Device port1
Distance 10
  1. Select OK.

Configuring a NAT/Route mode active-passive cluster of two FortiGates - CLI

Use the following procedures to configure two FortiGates for NAT/Route HA operation using the FortiGate CLI. These procedures assume you are starting with two FortiGates with factory default settings.

To configure the first FortiGate (host name FGT_ha_1)
  1. Power on the FortiGate.
  2. Connect a null modem cable to the communications port of the management computer and to the FortiGate Console port.
  3. Start HyperTerminal (or any terminal emulation program), enter a name for the connection, and select OK.
  4. Configure HyperTerminal to connect directly to the communications port on the computer to which you have connected the null modem cable and select OK.
  5. Select the following port settings and select OK.
Bits per second 9600
Data bits 8
Parity None
Stop bits 1
Flow control None
  1. Press Enter to connect to the FortiGate CLI.
    The FortiGate CLI login prompt appears.

If the prompt does not appear, press Enter. If it still does not appear, power off your FortiGate and power it back on. If you are connected, at this stage you will see startup messages that will confirm you are connected. The login prompt will appear after the startup has completed.

  1. Type admin and press Enter twice.
  2. Register and apply licenses to the FortiGate. This includes FortiCloud activation and FortiClient licensing, and entering a license key if you purchased more than 10 Virtual Domains (VDOMS).
  3. You can also install any third-party certificates on the primary FortiGate before forming the cluster. Once the cluster is formed third-party certificates are synchronized to the backup FortiGate.
    FortiToken licenses can be added at any time because they are synchronized to all cluster members.
  4. Change the host name for this FortiGate.

config system global

set hostname FGT_ha_1

end

 

  1. Configure HA settings.

config system ha

set mode a-p

set group-name example1.com

set password HA_pass_1

end

The FortiGate negotiates to establish an HA cluster. You may temporarily lose network connectivity with the FortiGate as the HA cluster negotiates and the FGCP changes the MAC address of the FortiGate interfaces. The MAC addresses of the FortiGate interfaces change to the following virtual MAC addresses:

  • port1 interface virtual MAC: 00-09-0f-09-00-00
  • port2 interface virtual MAC: 00-09-0f-09-00-01
  • port3 interface virtual MAC: 00-09-0f-09-00-02
  • port4 interface virtual MAC: 00-09-0f-09-00-03

To reconnect sooner, you can update the ARP table of your management PC by deleting the ARP table entry for the FortiGate (or just deleting all arp table entries). You may be able to delete the arp table of your management PC from a command prompt using a command similar to arp -d.

To confirm these MAC address changes, you can use the get hardware nic (or diagnose hardware deviceinfo nic) CLI command to view the virtual MAC address of any FortiGate interface. For example, use the following command to view the port1 interface virtual MAC address (MAC) and the port1 permanent MAC address (Permanent_HWaddr):

get hardware nic port1

.

.

.

Current_HAaddr    00:09:0f:09:00:00

Permanent_HWaddr  02:09:0f:78:18:c9

.

.

.

 

  1. Display the HA configuration (optional).

get system ha

group-id : 0

group-name : example1.com

mode : a-p

password : *

hbdev : "port3" 50 "port4" 50

session-sync-dev    :

route-ttl : 10

route-wait : 0

route-hold : 10

sync-config : enable

encryption : disable

authentication : disable

hb-interval : 2

hb-lost-threshold : 20

hello-holddown : 20

arps : 5

arps-interval : 8

session-pickup : disable

update-all-session-timer: disable

session-sync-daemon-number: 1

link-failed-signal : disable

uninterruptible-upgrade: enable

ha-mgmt-status : disable

ha-eth-type : 8890

hc-eth-type : 8891

l2ep-eth-type : 8893

ha-uptime-diff-margin: 300

vcluster2 : disable

vcluster-id : 1

override : disable

priority : 128

slave-switch-standby: disable

minimum-worker-threshold: 1

monitor :

pingserver-monitor-interface:

pingserver-failover-threshold: 0

pingserver-slave-force-reset: enable

pingserver-flip-timeout: 60

vdom : "root"

 

  1. Power off the FortiGate.
To configure the second FortiGate (host name FGT_ha_2)
  1. Power on the FortiGate.
  2. Connect a null modem cable to the communications port of the management computer and to the FortiGate Console port.
  3. Start HyperTerminal, enter a name for the connection, and select OK.
  4. Configure HyperTerminal to connect directly to the communications port on the computer to which you have connected the null modem cable and select OK.
  5. Select the following port settings and select OK.
Bits per second 9600
Data bits 8
Parity None
Stop bits 1
Flow control None
  1. Press Enter to connect to the FortiGate CLI.
    The FortiGate CLI login prompt appears.
  2. Type admin and press Enter twice.
  3. Register and apply licenses to the FortiGate. This includes FortiCloud activation and FortiClient licensing, and entering a license key if you purchased more than 10 Virtual Domains (VDOMS).
  4. You can also install any third-party certificates on the primary FortiGate before forming the cluster. Once the cluster is formed third-party certificates are synchronized to the backup FortiGate.
    FortiToken licenses can be added at any time because they are synchronized to all cluster members.
  5. Change the host name for this FortiGate.

config system global

set hostname FGT_ha_2

end

 

  1. Configure HA settings.

config system ha

set mode a-p

set group-name example1.com

set password HA_pass_1

end

The FortiGate negotiates to establish an HA cluster. You may temporarily lose network connectivity with the FortiGate as the HA cluster negotiates and because the FGCP changes the MAC address of the FortiGate interfaces.

To reconnect sooner, you can update the ARP table of your management PC by deleting the ARP table entry for the FortiGate (or just deleting all arp table entries). You may be able to delete the arp table of your management PC from a command prompt using a command similar to arp -d.

  1. Display the HA configuration (optional).

get system ha

group-id : 0

group-name : example1.com

mode : a-p

password : *

hbdev : "port3" 50 "port4" 50

session-sync-dev    :

route-ttl : 10

route-wait : 0

route-hold : 10

sync-config : enable

encryption : disable

authentication : disable

hb-interval : 2

hb-lost-threshold : 20

hello-holddown : 20

arps : 5

arps-interval : 8

session-pickup : disable

update-all-session-timer: disable

session-sync-daemon-number: 1

link-failed-signal : disable

uninterruptible-upgrade: enable

ha-mgmt-status : disable

ha-eth-type : 8890

hc-eth-type : 8891

l2ep-eth-type : 8893

ha-uptime-diff-margin: 300

vcluster2 : disable

vcluster-id : 1

override : disable

priority : 128

slave-switch-standby: disable

minimum-worker-threshold: 1

monitor :

pingserver-monitor-interface:

pingserver-failover-threshold: 0

pingserver-slave-force-reset: enable

pingserver-flip-timeout: 60

vdom : "root"

 

  1. Power off the FortiGate.
To connect the cluster to the network
  1. Connect the port1 interfaces of FGT_ha_1 and FGT_ha_2 to a switch connected to the Internet.
  2. Connect the port2 interfaces of FGT_ha_1 and FGT_ha_2 to a switch connected to the internal network.
  3. Connect the port3 interfaces of FGT_ha_1 and FGT_ha_2 together. You can use a crossover Ethernet cable or regular Ethernet cables and a switch.
  4. Connect the port4 interfaces of the cluster units together. You can use a crossover Ethernet cable or regular Ethernet cables and a switch.
  5. Power on the cluster units.

The units start and negotiate to choose the primary unit and the subordinate unit. This negotiation occurs with no user intervention and normally takes less than a minute.

When negotiation is complete the cluster is ready to be configured for your network.

To view cluster status

Use the following steps to view cluster status from the CLI.

  1. Determine which cluster unit is the primary unit.
  • Use the null-modem cable and serial connection to re-connect to the CLI of one of the cluster units.
  • Enter the command get system status.
  • If the command output includes Current HA mode: a-p, master, the cluster units are operating as a cluster and you have connected to the primary unit. Continue with Step Setting up two new FortiGates as an FGCP cluster.
  • If the command output includes Current HA mode: a-p, backup, you have connected to a subordinate unit. Connect the null-modem cable to the other cluster unit, which should be the primary unit and continue with Step 2.
note icon If the command output includes Current HA mode: standalone, the cluster unit is not operating in HA mode and you should review your HA configuration.
  1. Enter the following command to confirm the HA configuration of the cluster:
get system ha status
HA Health Status: OK
Model: FortiGate-XXXX
Mode: HA A-P
Group: 0
Debug: 0
Cluster Uptime: 7 days 00:30:26
.
.
.

You can use this command to confirm that the cluster is healthy and operating normally, some information about the cluster configuration, and information about how long the cluster has been operating. Information not shown in this example includes how the primary unit was selected, configuration synchronization status, usage stats for each cluster unit, heartbeat status, and the relative priorities of the cluster units.

.

  1. Check the cluster synchronization status to make sure the primary and backup units have the same configuration. Log into the primary unit CLI and enter this command:

    diag sys ha cluster-csum

    The CLI lists all members' checksums. If both cluster units have identical checksums you can be sure that their configurations are synchronized. If the checksums are different wait a short while and enter the command again. Repeat until the checksums are identical. It may take a while for some parts of the configuration to be synchronized. If the checksums never become identical contact Fortinet support to help troubleshoot the problem.
To troubleshoot the cluster configuration

If the cluster members list and the dashboard do not display information for both cluster units the FortiGates are not functioning as a cluster. See Troubleshooting HA clusters to troubleshoot the cluster.

To add basic configuration settings to the cluster

Use the following steps to add some basic settings to the cluster so that it can connect to the network.

  1. Log into the primary unit CLI.
  2. Add a password for the admin administrative account.

config system admin

edit admin

set password <password_str>

end

 

  1. Configure the port1 and port2 interfaces.

config system interface

edit port1

set ip 172.20.120.141/24

next

edit port2

set ip 10.11.101.100/24

end

 

  1. Add a default route.

config router static

edit 1

set dst 0.0.0.0 0.0.0.0

set gateway 172.20.120.2

set device port1

end