FortiOS 5.4 Online Help Link FortiOS 5.2 Online Help Link FortiOS 5.0 Online Help Link FortiOS 4.3 Online Help Link

Home > Online Help

> Chapter 13 - High Availability > FortiGate Session Life Support Protocol (FGSP)

FortiGate Session Life Support Protocol (FGSP)

In a network that already includes load balancing (either with load balancers or routers) for traffic redundancy, two identical FortiGate units can be integrated into the load balancing configuration using the FortiGate Session Life Support Protocol (FGSP). The external load balancers or routers can distribute sessions among the FortiGate units and the FGSP performs session synchronization of IPv4 and IPv6 TCP, UDP, ICMP, expectation, and NAT sessions to keep the session tables of both FortiGate units synchronized.

You can use the config system session-sync command to configure the FortiGate Session Life Support Protocol (FGSP) (previously called TCP session synchronization or standalone session synchronization) between two FortiGate units. The two FortiGate units must be the same model. The FGSP synchronizes both IPv4 and IPv6 TCP, UDP, ICMP, expectation, and NAT sessions. You can use this feature with external routers or load balancers configured to distribute or load balance sessions between two peer FortiGate units. If one of the peers fails, session failover occurs and active sessions fail over to the peer that is still operating. This failover occurs without any loss of data. As well, the external routers or load balancers will detect the failover and re-distribute all sessions to the peer that is still operating.

In previous versions of FortiOS the FGSP was called TCP session synchronization or standalone session synchronization. However, the FGSP has been expanded to include configuration synchronization and session synchronization of connectionless sessions, expectation sessions, and NAT sessions.
You cannot configure FGSP HA when FGCP HA is enabled. However FGSP HA is compatible with VRRP.
FGSP or standalone session synchronization is not supported if the FortiGate units are running different firmware versions.

The FGSP can be used instead of FGCP HA to provide session synchronization between two peer FortiGate units. If the external load balancers direct all sessions to one peer the affect is similar to active-passive FGCP HA. If external load balancers or routers load balance traffic to both peers, the effect is similar to active-active FGCP HA. The load balancers should be configured so that all of the packets for any given session are processed by the same peer. This includes return packets.

FGSP HA

By default, FGSP synchronizes all IPv4 and IPv6 TCP sessions and also synchronizes the configuration of the FortiGate units.

You can optionally enable session pickup to synchronize connectionless (UDP and ICMP) sessions, expectation sessions, and NAT sessions. If you do not enable session pickup, the FGSP does not share session tables for the particular session type and sessions do not resume after a failover. All sessions that are interrupted by the failover and must be re-established at the application level. Many protocols can successfully restart sessions with little, or no, loss of data. Others may not recover easily. Enable session pickup for sessions that may be difficult to reestablish. Since session pickup requires FortiGate resources, only enable this feature for sessions that you need to have synchronized.

You can also optionally add filters to control which sessions are synchronized. You can add filters to only synchronize packets from specified source and destination addresses, specified source and destination interfaces, and specified services.

Load balancing and session failover is done by external routers or load balancers instead of by the FGSP. The FortiGate units just perform session synchronization to support session failover.

Synchronizing the configuration

The FGSP also includes configuration synchronization, allowing you to make configuration changes once for both FortiGate units instead of requiring you to make duplicate configuration changes on each FortiGate unit. Settings that identify the FortiGate unit to the network, for example, interface IP addresses and BGP neighbor settings, are not synchronized so each FortiGate unit maintains its identity on the network.

 

By default configuration synchronization is disabled. You can use the following command to enable it.

config system ha

set standalone-config-sync enable

end

Synchronizing UDP and ICMP (connectionless) sessions

In many configurations, due to their non-stateful nature, UDP and ICMP sessions don't need to be synchronized to naturally failover. However, if its required you can configure the FGSP to synchronize UDP and ICMP sessions by entering the following command:

config system ha

set session-pickup enable

set session-pickup-connectionless enable

end

Synchronizing NAT sessions

By default, NAT session are not synchronized. However, the FGSP can synchronize NAT session if you enter the following command:

config system ha

set session-pickup enable

set session-pickup-nat enable

end

However, if you want NAT sessions to resume after a failover you should not configure NAT to use the destination interface IP address since the FGSP FortiGate units have different IP addresses. With this configuration, after a failover all sessions that include the IP addresses of interfaces on the failed FortiGate unit will have nowhere to go since the IP addresses of the failed FortiGate unit will no longer be on the network.

Instead, in an FGSP configuration, if you want NAT sessions to failover you should use IP pools with the type set to overload (which is the default IP pool type). For example:

config firewall ippool

edit FGSP-pool

set type overload

set startip 172.20.120.10

set endip 172.20.120.20

end

Then when you configure NAT firewall policies, turn on NAT and select to use dynamic IP pool and select the IP Pool that you added. Add the same IP pools and firewall policies to both FortiGate units.

Synchronizing asymmetric sessions

By default, asymmetric sessions are not synchronized. Normally, session synchronization cannot be asymmetric because it is stateful. So all of the packets of a given session must be processed on the same peer. This includes return packets.

However, if you have an asymmetric routing configuration, you can enter the following command to synchronize asymmetric sessions by dynamically detecting asymmetric sessions and disabling anti-reply for these sessions.

config system ha

set session-pickup enable

set session-pickup-expectation enable

end

The FGSP enforces firewall policies for asymmetric traffic, including cases where the TCP 3-way handshake is split between two FortiGates. For example, FGT-A receives the TCP-SYN, FGT-B receives the TCP-SYN-ACK, and FGT-A receives the TCP-ACK. Under normal conditions a firewall will drop this connection since the 3-way handshake was not seen by the same firewall. However two FortiGates with FGSP configured will be able to properly pass this traffic since the firewall sessions are synchronized.

This asymmetric function can also work with connectionless UDP and ICMP traffic. If traffic will be highly asymmetric, as described above, the following command must be enabled on both FortiGates.

config system ha

set session-pickup enable

set session-pickup-connectionless enable

end

Synchronizing asymmetric traffic can be very useful in situations where multiple Internet connections from different ISPs are spread across two FortiGates. Since it is typically not possible to guarantee Internet bound traffic leaving via an ISP will return using the exact same ISP, the FGSP provides critical firewall functions in this situation.

caution icon Asymmetric sessions may not be synchronized in low latency networks if the reply packet is received before the peer has received the session synchronization packet. This limitation usually only occurs in low latency networks.

The FGSP also has applications in virtualized computing environments where virtualized hosts move between data centers. The firewall session synchronization features of FGSP allow for more flexibility than in traditional firewalling functions.

Synchronizing expectation sessions

FortiOS session helpers keep track of the communication of Layer-7 protocols such as FTP and SIP that have control sessions and expectation sessions. Usually the control sessions establish the link between server and client and negotiate the ports and protocols that will be used for data communications. The session helpers then create expectation sessions through the FortiGate for the ports and protocols negotiated by the control session.

The expectation sessions are usually the sessions that actually communicate data. For FTP, the expectation sessions transmit files being uploaded or downloaded. For SIP, the expectation sessions transmit voice and video data. Expectation sessions usually have a timeout value of 30 seconds. If the communication from the server is not initiated within 30 seconds the expectation session times out and traffic will be denied.

By default the FGSP does not synchronize expectation sessions and if a failover occurs the sessions will have to be restarted.

If you want to synchronize expectation sessions so that they will continue after a failover you can enter the following command on both FortiGates to synchronize them:

config system ha

set session-pickup enable

set session-pickup-expectation enable

end

Security profile flow-based inspection and asymmetric traffic

Security profile inspection (flow or proxy based) for a session is not expected to work properly if the traffic in the session is balanced across more than one FortiGate in either direction. Flow-based inspection should be used in FGSP deployments.

For an environment where traffic is symmetric, security profile inspection can be used with the following limitations:

  • No session synchronization for the sessions inspected using proxy-based inspection. Sessions will drop and need to be reestablished after data path failover.
  • Sessions with flow-based inspection will failover; however, inspection of failed over sessions after the failover may not work.

A single FortiGate must see both the request and reply traffic for security profile inspection to function correctly. For environments where asymmetric traffic is expected, security profile inspection should not be used.

Notes and limitations

FGSP HA has the following limitations:

  • The FGSP is a global configuration option. As a result you can only add one service to a filter configuration. You cannot add custom services or service groups even if virtual domains are not enabled.
  • You can only add one filter configuration to a given FGSP configuration. However, you can add multiple filters by adding multiple identical FGSP configurations, each one with a different filter configuration.
  • Sessions accepted by security policies with security profiles configured are not synchronized.
  • FGSP HA is configured from the CLI.
  • FGSP HA is available for FortiGate units or virtual domains operating in NAT/Route or Transparent mode. NAT sessions are not synchronized in either mode (unless NAT synchronization is enabled as described in Synchronizing NAT sessions). In NAT/Route mode, only sessions for route mode security policies are synchronized. In Transparent mode, only sessions for normal Transparent mode policies are synchronized.
  • FGSP HA is supported for traffic on physical interfaces, VLAN interfaces, zones, aggregate interfaces, and NPx (NP4, NP6 etc.) accelerated interfaces. The FGSP has not been tested for inter-vdom links, between HA clusters, and for redundant interfaces.
  • The names of the matching interfaces, including VLAN interfaces, aggregate interfaces and so on, must be the same on both peers.

Configuring FGSP HA

You configure FGSP HA separately for each virtual domain to be synchronized. If virtual domain configuration is not enabled, you configure FGSP HA for the root virtual domain. When virtual domain configuration is enabled and you have added virtual domains you configure FGSP HA for each virtual domain to be synchronized. You don’t have to synchronize all virtual domains.

You must configure FGSP HA and network settings on both peers. Once you establish the initial configuration, the configurations of both FortiGate units are synchronized so when you change the configuration of one, the changes are synchronized to the other.

On each FortiGate unit, configuring FGSP HA consists of selecting the virtual domains to be synchronized using the syncvd field, selecting the virtual domain on the other peer that receives the synchronization packets using the peervd field, and setting the IP address of the interface in the peer unit that receives the synchronization packets using the peerip field. The interface with the peerip must be in the peervd virtual domain.

The syncvd and peervd settings must be the same on both peers. However, the peerip settings will be different because the peerip setting on the first peer includes the IP address of an interface on the second peer. And the peerip setting on the second peer includes the IP address of an interface on the first peer.

For FGSP HA to work properly all synchronized virtual domains must be added to both peers. The names of the matching interfaces in each virtual domain must also be the same; this includes the names of matching VLAN interfaces. Note that the index numbers of the matching interfaces and VLAN interfaces can be different. Also the VLAN IDs of the matching VLAN interfaces can be different.

Configuring the session synchronization link

When FGSP HA is operating, the peers share session information over an Ethernet link between the peers similar to an HA heartbeat link. Usually you would use the same interface on each peer for session synchronization. You should connect the session synchronization interfaces directly without using a switch or other networking equipment. For FortiGate-5000 systems you can use a backplane interface as the session synchronization link.

You can use different interfaces on each peer for session synchronization links. Also, if you have multiple sessions synchronization configurations, you can have multiple links between the peers. In fact if you are synchronizing a lot of sessions, you may want to configure and connect multiple session synchronization links to distribute session synchronization traffic to these multiple links.

You cannot configure backup session synchronization links. Each configuration only includes one session synchronization link.

The session synchronization link should always be maintained. If session synchronization communication is interrupted and a failure occurs, sessions will not failover and data could be lost.

Session synchronization traffic can use a considerable amount of network bandwidth. If possible, session synchronization link interfaces should only be used for session synchronization traffic and not for data traffic.

Basic example configuration

The following configuration example shows how to configure basic FGSP HA for the two peer FortiGate units shown below. The host names of peers are peer_1 and peer_2. Both peers are configured with two virtual domains: root and vdom_1. All sessions processed by vdom_1 are synchronized. The synchronization link interface is port3 which is in the root virtual domain. The IP address of port3 on peer_1 is 10.10.10.1. The IP address of port3 on peer_2 is 10.10.10.2.

Also on both peers, port1 and port2 are added to vdom_1. On peer_1 the IP address of port1 is set to 192.168.20.1 and the IP address of port2 is set to 172.110.20.1. On peer_2 the IP address of port1 is set to 192.168.20.2 and the IP address of port2 is set to 172.110.20.2.

Example FGSP HA network configuration

To configure FGSP HA
  1. Configure the load balancer or router to send all sessions to peer_1.
  2. Configure the load balancer or router to send all traffic to peer_2 if peer_1 fails.
  3. Use normal FortiGate configuration steps on peer_1:
  • Enable virtual domain configuration.
  • Add the vdom_1 virtual domain.
  • Add port1 and port2 to the vdom_1 virtual domain and configure these interfaces.
  • Set the IP address of port1 to 192.168.20.1.
  • Set the IP address of port2 to 172.110.20.1.
  • Set the IP address of port3 to 10.10.10.1.
  • Add route mode security policies between port1 and port2 to vdom_1.
  1. Enter the following commands to configure session synchronization for peer_1

config system session-sync

edit 1

set peerip 10.10.10.2

set peervd root

set syncvd vdom_1

end

 

  1. Use normal FortiGate configuration steps on peer_2:
  • Enable virtual domain configuration.
  • Add the vdom_1 virtual domain.
  • Add port1 and port2 to the vdom_1 virtual domain and configure these interfaces.
  • Set the IP address of port1 to 192.168.20.2.
  • Set the IP address of port2 to 172.110.20.2.
  • Set the IP address of port3 to 10.10.10.1.
  • Add route mode security policies between port1 and port2 to vdom_1.
  1. Enter the following command to configure session synchronization for peer_1

config system session-sync

edit 1

set peerip 10.10.10.1

set peervd root

set syncvd vdom_1

end

Now that the FortiGate units are connected and configured their configurations are synchronized, so when you make a configuration change on one FortiGate unit it is synchronized to the other one.

To add filters

You can add a filter to this basic configuration if you only want to synchronize some TCP sessions. For example you can enter the following command to add a filter so that only HTTP sessions are synchronized:

config system session-sync

edit 1

config filter

set service HTTP

end

end

You can also add a filter to control the source and destination addresses of the IPv4 packets that are synchronized. For example you can enter the following command to add a filter so that only sessions with source addresses in the range 10.10.10.100 to 10.10.10.200 are synchronized.

config system session-sync

edit 1

config filter

set srcaddr 10.10.10.100 10.10.10.200

end

end

You can also add a filter to control the source and destination addresses of the IPv6 packets that are synchronized. For example you can enter the following command to add a filter so that only sessions with destination addresses in the range 2001:db8:0:2::/64 are synchronized.

config system session-sync

edit 1

config filter

set dstaddr6 2001:db8:0:2::/64

end

end

To synchronize UDP and ICMP sessions

You enter the following command to add synchronization of UDP and ICMP sessions to this configuration:

config system ha

set session-pickup enable

set session-pickup-connectionless enable

end

To synchronize the configuration

Enter the following command to enable configuration synchronization.

config system ha

set standalone-config-sync enable

end

Verifying FGSP configuration and synchronization

You can use the following diagnose commands to verify that the FGSP and its synchronization functions are operating correctly.

FGSP configuration summary and status

Enter the following command to display a summary of the FGSP configuration and synchronization status:

diagnose sys session sync

sync_ctx: sync_started=1, sync_tcp=1, sync_others=1,

sync_expectation=1, sync_redir=0, sync_nat=1.

sync: create=12:0, update=0, delete=0:0, query=14

recv: create=14:0, update=0, delete=0:0, query=12

ses pkts: send=0, alloc_fail=0, recv=0, recv_err=0 sz_err=0

nCfg_sess_sync_num=5, mtu=16000

sync_filter:

1: vd=0, szone=0, dzone=0, saddr=0.0.0.0:0.0.0.0, daddr=0.0.0.0:0.0.0.0,

 

sync_started=1 shows that synchronization is working. If this is set to 0 then something is not correct with session synchronization and synchronization has not been able to start because of it.

sync_tcp=1, sync_others=1, sync_expectation=1, and sync_nat=1 show that the FGSP has been configured to synchronize TCP, connectionless, asymmetric, and NAT sessions.

sync: create=12:0 and recv: create=14:0 show that this FortiGate has synchronized 12 sessions to its peer and has received 14 sessions from its peer.

sync_filter shows the configured FGSP filter. In this case no filter has been created so all sessions are synchronized.

vd=0 indicates that root VDOM sessions are synchronized.

Verifying that sessions are synchronized

Enter the command diagnose sys session list to display information about the sessions being processed by the FortiGate. In the command output look for sessions that should be synchronized and make sure they contain output lines that include synced (for example, state=log may_dirty ndr synced) to confirm that they are being synchronized by the FGSP.

diagnose sys session list

session info: proto=6 proto_state=05 duration=469 expire=0 timeout=3600

flags=00000000 sockflag=00000000 sockport=21 av_idx=0 use=4

origin-shaper=

reply-shaper=

per_ip_shaper=

ha_id=0 policy_dir=0 tunnel=/

state=log may_dirty ndr synced

statistic(bytes/packets/allow_err): org=544/9/1 reply=621/7/0 tuples=2

orgin->sink: org pre->post, reply pre->post dev=46->45/45->46

gwy=10.2.2.1/10.1.1.1

hook=pre dir=org act=noop 192.168.1.50:45327->172.16.1.100:21(0.0.0.0:0)

hook=post dir=reply act=noop 172.16.1.100:21->192.168.1.50:45327(0.0.0.0:0)

pos/(before,after) 0/(0,0), 0/(0,0)

misc=0 policy_id=1 id_policy_id=0 auth_info=0 chk_client_info=0 vd=0

serial=00002deb tos=ff/ff ips_view=1 app_list=2000 app=16427

dd_type=0 dd_mode=0

per_ip_bandwidth meter: addr=192.168.1.50, bps=633