Managing individual cluster units using a reserved out-of-band management interface

Submit Search

Managing individual cluster units using a reserved out-of-band management interface

You can provide direct management access to all cluster units by reserving up to four management interfaces as part of the HA configuration. Once a management interface is reserved, you can configure a different IP address, administrative access and other interface settings for each management interface for each cluster unit. Then by connecting these interfaces of each cluster unit to your network you can manage each cluster unit separately from different IP addresses. Configuration changes to the reserved management interfaces are not synchronized to other cluster units.

You can also configure and in-band management interface for a cluster unit. See Managing individual cluster units using an in-band management IP address.

Reserved management interfaces provide direct management access to each cluster unit and give each cluster unit a different identity on your network. This simplifies using external services, such as SNMP, to separately monitor and manage each cluster unit.

The reserved management interfaces are not assigned HA virtual MAC addresses like other cluster interfaces. Instead the reserved management interfaces retain the permanent hardware address of the physical interface unless you change it using the config system interface command.

Reserved management interfaces and their IP addresses should not be used for managing a cluster using FortiManager. To correctly manage a FortiGate HA cluster with FortiManager use the IP address of one of the cluster unit interfaces.

If you enable SNMP administrative access for a reserved management interface you can use SNMP to monitor each cluster unit using the reserved management interface IP address. To monitor each cluster unit using SNMP, just add the IP address of each cluster unit’s reserved management interface to the SNMP server configuration. You must also enable direct management of cluster members in the cluster SNMP configuration.

If you enable HTTPS or HTTP administrative access for the reserved management interfaces you can connect to the GUI of each cluster unit. Any configuration changes made to any of the cluster units is automatically synchronized to all cluster units. From the subordinate units the GUI has the same features as the primary unit except that unit-specific information is displayed for the subordinate unit, for example:

The Dashboard System Information widget displays the subordinate unit serial number but also displays the same information about the cluster as the primary unit
On the Cluster members list (go to System > HA) you can change the HA configuration of the subordinate unit that you are logged into. For the primary unit and other subordinate units you can change only the host name and device priority.
Log Access displays the logs of the subordinate that you are logged into fist, You use the HA Cluster list to view the log messages of other cluster units including the primary unit.

If you enable SSH or TELNET administrative access for the reserved management interfaces you can connect to the CLI of each cluster unit. The CLI prompt contains the host name of the cluster unit that you have connected to. Any configuration changes made to any of the cluster units is automatically synchronized to all cluster units. You can also use the execute ha manage command to connect to other cluster unit CLIs.

The reserved management interface is available in NAT and in transparent mode. It is also available if the cluster is operating with multiple VDOMs. In transparent mode you cannot normally add an IP address to an interface. However, you can add an IP address to the reserved management interface.

Reserved management interface for FortiSandbox, SNMP, and RADIUS

By default, management services such as SNMP, remote logging, remote authentication and so on use a cluster interface. As a result, communication from each cluster unit comes from a cluster interface instead of from the interface of an individual cluster unit and not from the HA reserved management interface.

If you want to use an HA reserved management interface for these management features you must enter the following command:

config system ha

set ha-direct enable

end

By enabling ha-direct, the following management features will use the HA reserved management interface:

Remote logging (including syslog, FortiAnalyzer, and FortiCloud).
SNMP queries and traps.
Remote authentication and certificate verification.
Communication with FortiSandbox.

This means that individual cluster units send log messages and communicate with FortiSandbox and so on using their HA reserved management interface instead of one of the cluster interfaces. This allows you to manage each cluster unit separately and to separate the management traffic from each cluster unit. This can also be useful if each cluster unit is in a different location.

If you just want to use HA reserved management interfaces for SNMP remote management you can enable ha-direct in the SNMP configuration as shown in the following section.

Configuring the reserved management interface and SNMP remote management of individual cluster units

This example describes how to configure SNMP remote management of individual cluster units using an HA reserved management interface. The configuration consists of two FortiGate-620B units already operating as a cluster. In the example, the port8 interface of each cluster unit is connected to the internal network using the switch and configured as the reserved management interface.

SNMP remote management of individual cluster units

To configure the reserved management interface - GUI

From the GUI you can also configure IPv4 and IPv6 default routes that are only used by the reserved management interface.

Go to System > HA.
Edit the primary unit.
Select Management Interface Reservation and select port8.
Set Gateway to 10.11.101.2.
Set IPv6 Gateway to 2001:db8:0:2::20
Select OK.

To configure the reserved management interface - CLI

From the CLI you can also configure IPv4 and IPv6 default routes that are only used by the reserved management interface.

Log into the CLI of any cluster unit.
Enter the following command to enable the reserved management interface, set port8 as the reserved interface, and add an IPv4 default route of 10.11.101.2 and an IPv6 default route of 2001:db8:0:2::20 for the reserved management interface.

config system ha

set ha-mgmt-status enable

config ha-mgmt-interfaces

edit 1

set interface port8

set gateway 10.11.101.2

set gateway6 2001:db8:0:2::20

end

The reserved management interface default route is not synchronized to other cluster units.

To change the primary unit reserved management interface configuration - GUI

You can change the IP address of the primary unit reserved management interface from the primary unit GUI. Configuration changes to the reserved management interface are not synchronized to other cluster units.

From a PC on the internal network, browse to http://10.11.101.100 and log into the cluster GUI.

This logs you into the primary unit GUI.

You can identify the primary unit from its serial number or host name that appears on the System Information dashboard widget.

Go to Network > Interfaces and edit the port8 interface as follows:

Alias	primary_reserved
IP/Netmask	10.11.101.101/24
Administrative Access	Ping, SSH, HTTPS, SNMP

Select OK.

You can now log into the primary unit GUI by browsing to https://10.11.101.101. You can also log into this primary unit CLI by using an SSH client to connect to 10.11.101.101.

To change subordinate unit reserved management interface configuration - CLI

At this point you cannot connect to the subordinate unit reserved management interface because it does not have an IP address. Instead, this procedure describes connecting to the primary unit CLI and using the execute ha manage command to connect to subordinate unit CLI to change the port8 interface. You can also use a serial connection to the cluster unit CLI. Configuration changes to the reserved management interface are not synchronized to other cluster units.

Connect to the primary unit CLI and use the execute ha manage command to connect to a subordinate unit CLI.

You can identify the subordinate unit from is serial number or host name. The host name appears in the CLI prompt.

Enter the following command to change the port8 IP address to 10.11.101.102 and set management access to HTTPS, ping, SSH, and SNMP.

config system interface

edit port8

set ip 10.11.101.102/24

set allowaccess https ping ssh snmp

end

You can now log into the subordinate unit GUI by browsing to https://10.11.101.102. You can also log into this subordinate unit CLI by using an SSH client to connect to 10.11.101.102.

To configure the cluster for SNMP management using the reserved management interfaces - CLI

This procedure describes how to configure the cluster to allow the SNMP server to get status information from the primary unit and the subordinate unit. The SNMP configuration is synchronized to all cluster units. To support using the reserved management interfaces, you must add at least one HA direct management host to an SNMP community. If your SNMP configuration includes SNMP users with user names and passwords you must also enable HA direct management for SNMP users.

Enter the following command to add an SNMP community called Community and add a host to the community for the reserved management interface of each cluster unit. The host includes the IP address of the SNMP server (10.11.101.20).

config system snmp community

edit 1

set name Community

config hosts

edit 1

set ha-direct enable

set ip 10.11.101.20

end

Enabling ha-direct in non-HA environments makes SNMP unusable.

Enter the following command to add an SNMP user for the reserved management interface.

config system snmp user

edit 1

set ha-direct enable

set notify-hosts 10.11.101.20

end

Configure other settings as required.

To get CPU, memory, and network usage of each cluster unit using the reserved management IP addresses

From the command line of an SNMP manager, you can use the following SNMP commands to get CPU, memory and network usage information for each cluster unit. In the examples, the community name is Community. The commands use the MIB field names and OIDs listed below.

Enter the following commands to get CPU, memory and network usage information for the primary unit with reserved management IP address 10.11.101.101 using the MIB fields:

snmpget -v2c -c Community 10.11.101.101 fgHaStatsCpuUsage

snmpget -v2c -c Community 10.11.101.101 fgHaStatsMemUsage

snmpget -v2c -c Community 10.11.101.101 fgHaStatsNetUsage

Enter the following commands to get CPU, memory and network usage information for the primary unit with reserved management IP address 10.11.101.101 using the OIDs:

snmpget -v2c -c Community 10.11.101.101 1.3.6.1.4.1.12356.101.13.2.1.1.3.1

snmpget -v2c -c Community 10.11.101.101 1.3.6.1.4.1.12356.101.13.2.1.1.4.1

snmpget -v2c -c Community 10.11.101.101 1.3.6.1.4.1.12356.101.13.2.1.1.5.1

Enter the following commands to get CPU, memory and network usage information for the subordinate unit with reserved management IP address 10.11.101.102 using the MIB fields:

snmpget -v2c -c Community 10.11.101.102 fgHaStatsCpuUsage

snmpget -v2c -c Community 10.11.101.102 fgHaStatsMemUsage

snmpget -v2c -c Community 10.11.101.102 fgHaStatsNetUsage

Enter the following commands to get CPU, memory and network usage information for the subordinate unit with reserved management IP address 10.11.101.102 using the OIDs:

snmpget -v2c -c Community 10.11.101.102 1.3.6.1.4.1.12356.101.13.2.1.1.3.1

snmpget -v2c -c Community 10.11.101.102 1.3.6.1.4.1.12356.101.13.2.1.1.4.1

snmpget -v2c -c Community 10.11.101.102 1.3.6.1.4.1.12356.101.13.2.1.1.5.1

Adding firewall local-in policies for the dedicated HA management interface

To add local-in polices for the dedicated management interface, enable ha-mgmt-inft-only and set intf to any. Enabling ha-mgmt-intf-only means the local-in policy applies only to the VDOM that contains the dedicated HA management interface. For example:

config firewall local-in-policy

edit 0

set ha-mgmt-intf-only enable

set intf any

set srcaddr internal-net

set dstaddr mgmt-int

set action accept

set service HTTPS

set schedule weekdays

end

NTP over dedicated HA management interfaces

If you set up dedicated management interfaces on each cluster unit, if NTP is enabled, the primary unit contacts an NTP server using the dedicated management interface. System time is then synchronized to the backup units through the HA heartbeat.

Example CLI:

config system interface

edit port5

set ip 172.16.79.46 255.255.255.0

end

config system ha

set group-name FGT-HA

set mode a-p

set ha-mgmt-status enable

config ha-mgmt-interfaces

edit 1

set interface port5

set gateway 172.16.79.1

end

set ha-direct enable

end

config system ntp

set ntpsync enable

set syncinterval 5

end