Checking routing
The ping and traceroute utilities are useful for investigating issues with network connectivity and routing.
Since you typically use these tools to troubleshoot, you can allow ICMP, the protocol used by these tools, in firewall policies and on interfaces only when you need them. Otherwise, disable ICMP for improved security and performance.
By default, FortiADC appliances do not respond to ping and traceroute. However, if the appliance does not respond, and there are no firewall policies that block it, ICMP type 0 (ECHO_REPSPONSE) might be effectively disabled.
To enable ping and traceroute responses:
1. Go to Networking > Interface.
2. Select the row for the network interface and click the edit icon.
3. Under Allow Access, enable ping.
4. Save the update.
The appliance should now respond when another device such as your management computer sends a ping or traceroute to that network interface.
To verify routes between clients and your servers:
1. Attempt to connect through the FortiADC appliance, from a client to a backend server, via HTTP and/or HTTPS.
If the connectivity test fails, continue to the next step.
2. Use the ping command on both the client and the server to verify that a route exists between the two. Test traffic movement in both directions: from the client to the server, and the server to the client. Servers do not need to be able to initiate a connection, but must be able to send reply traffic along a return path.
If the routing test
succeeds, continue with step
4.
If the routing test fails, continue to the next step.
3. Use the tracert or traceroute command on both the client and the server (depending on their operating systems) to locate the point of failure along the route.
If the route is broken when it reaches the FortiADC appliance, first examine its network interfaces and routes. To display network interface addresses and subnets, enter the CLI command:
show system interface
To display all recently-used routes with their priorities, enter the CLI command:
diagnose netlink route list
You may need to verify that the physical cabling is reliable and not loose or broken, that there are no IP address or MAC address conflicts or blacklisting, misconfigured DNS records, and otherwise rule out problems at the physical, network, and transport layer.
If these tests succeed, a route exists, but you cannot connect using HTTP or HTTPS, an application-layer problem is preventing connectivity.
4. For application-layer problems, on the FortiADC, examine the:
• virtual server policy and all components it references
• certificates (if connecting via HTTPS)
• server service/daemon
On routers and firewalls between the host and the FortiADC appliance, verify that they permit HTTP and/or HTTPS connectivity between them.
Testing for connectivity with ping
The ping command sends a small data packet to the destination and waits for a response. The response has a timer that may expire, indicating that the destination is unreachable via ICMP.
ICMP is part of Layer 3 on the OSI Networking Model. ping sends Internet Control Message Protocol (ICMP) ECHO_REQUEST (“ping”) packets to the destination, and listens for ECHO_RESPONSE (“pong”) packets in reply.
Some networks block ICMP packets because they can be used in a ping flood or denial of service (DoS) attack if the network does not have anti-DoS capabilities, or because ping can be used by an attacker to find potential targets on the network.
Beyond basic existence of a possible route between the source and destination, ping tells you the amount of packet loss (if any), how long it takes the packet to make the round trip (latency), and the variation in that time from packet to packet (jitter).
If ping shows some packet loss, investigate:
• cabling to eliminate loose connections
• ECMP, split horizon, or network loops
• all equipment between the ICMP source and destination to minimize hops
If ping shows total packet loss, investigate:
• cabling to eliminate incorrect connections
• all firewalls, routers, and other devices between the two locations to verify correct IP addresses, routes, MAC lists, and policy configurations
If ping finds an outage between two points, use traceroute to locate exactly where the problem is.
To use ping:
1. Log into the CLI via either SSH, Telnet, or the CLI Console widget of the web UI.
2. If you want to adjust the behavior of execute ping, first use the execute ping‑options command.
3. Enter the command:
execute ping <destination_ipv4>
where <destination_ipv4> is the IP address of the device that you want to verify that the appliance can connect to, such as 192.168.1.1.
If the appliance can reach the host via ICMP, output similar to the following appears:
PING 192.168.1.1 (192.168.1.1): 56 data bytes
64 bytes from 192.168.1.1: icmp_seq=0 ttl=253 time=6.5 ms
64 bytes from 192.168.1.1: icmp_seq=1 ttl=253 time=7.4 ms
64 bytes from 192.168.1.1: icmp_seq=2 ttl=253 time=6.0 ms
64 bytes from 192.168.1.1: icmp_seq=3 ttl=253 time=5.5 ms
64 bytes from 192.168.1.1: icmp_seq=4 ttl=253 time=7.3 ms
--- 192.168.1.1 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 5.5/6.5/7.4 ms
If the appliance cannot reach the host via ICMP, output similar to the following appears:
PING 10.0.0.1 (10.0.0.1): 56 data bytes
Timeout ...
Timeout ...
Timeout ...
Timeout ...
Timeout ...
--- 10.0.0.1 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss
“100% packet loss” and “Timeout” indicates that the host is not reachable.
Testing routes & latency with traceroute
The traceroute utility sends ICMP packets to test each hop along the route. It sends three packets to the destination, and then increases the time to live (TTL) setting by one, and sends another three packets to the destination. As the TTL increases, packets go one hop farther along the route until they reach the destination.
Most traceroute commands display their maximum hop count—that is, the maximum number of steps it will take before declaring the destination unreachable—before they start tracing the route. The TTL setting may result in routers or firewalls along the route timing out due to high latency.
Where ping only tells you if the signal reached its destination and returned successfully, traceroute shows each step of its journey to its destination and how long each step takes. If you specify the destination using a domain name, the traceroute output can also indicate DNS problems, such as an inability to connect to a DNS server.
By default, the traceroute utility uses UDP with destination ports numbered from 33434 to 33534. The traceroute utility usually has an option to specify use of ICMP ECHO_REQUEST (type 8) instead, as used by the Windows tracert utility. If you have a firewall and you want traceroute to work from both machines (Unix-like systems and Windows) you will need to allow both protocols inbound through your firewall (UDP ports 33434 - 33534 and ICMP type 8).
To use traceroute:
1. Log into the CLI via either SSH, Telnet, or the CLI Console widget of the web UI.
2. Enter the command:
execute traceroute {<destination_ipv4> | <destination_fqdn>}
where {<destination_ipv4> | <destination_fqdn>} is a choice of either the device’s IP address or its fully qualified domain name (FQDN).
For example, you might enter:
execute traceroute www.example.com
If the appliance has a complete route to the destination, output similar to the following appears:
traceroute to www.fortinet.com (66.171.121.34), 32 hops max, 84 byte packets
1 172.16.1.2 0 ms 0 ms 0 ms
2 209.87.254.221 <static-209-87-254-221.storm.ca> 2 ms 2 ms 2 ms
3 209.87.239.129 <core-2-g0-1-1104.storm.ca> 2 ms 1 ms 2 ms
4 67.69.228.161 2 ms 2 ms 3 ms
5 64.230.164.17 <core2-ottawa23_POS13-1-0.net.bell.ca> 3 ms 3 ms 2 ms
6 64.230.132.234 <core2-ottawatc_POS5-0-0.net.bell.ca> 20 ms 20 ms 20 ms
7 64.230.132.58 <core4-toronto21_POS0-12-4-0.net.bell.ca> 24 ms 21 ms 24 ms
8 64.230.138.154 <bx4-toronto63_so-2-0-0-0.net.bell.ca> 8 ms 9 ms 8 ms
9 64.230.185.145 <bx2-ashburn_so2-0-0.net.bell.ca> 23 ms 23 ms 23 ms
10 12.89.71.9 23 ms 22 ms 22 ms
11 12.122.134.238 <cr2.wswdc.ip.att.net> 100 ms 12.123.10.130 <cr2.wswdc.ip.att.net> 101 ms 102 ms
12 12.122.18.21 <cr1.cgcil.ip.att.net> 101 ms 100 ms 99 ms
13 12.122.4.121 <cr1.sffca.ip.att.net> 100 ms 98 ms 100 ms
14 12.122.1.118 <cr81.sj2ca.ip.att.net> 98 ms 98 ms 100 ms
15 12.122.110.105 <gar2.sj2ca.ip.att.net> 96 ms 96 ms 96 ms
16 12.116.52.42 94 ms 94 ms 94 ms
17 203.78.181.10 88 ms 87 ms 87 ms
18 203.78.181.130 90 ms 89 ms 90 ms
19 66.171.121.34 <fortinet.com> 91 ms 89 ms 91 ms
20 66.171.121.34 <fortinet.com> 91 ms 91 ms 89 ms
Each line lists the routing hop number, the IP address and FQDN (if any) of that hop, and the 3 response times from that hop. Typically a value of <1ms indicates a local router.
If the appliance does not have a complete route to the destination, output similar to the following appears:
traceroute to 10.0.0.1 (10.0.0.1), 32 hops max, 84 byte packets
1 172.16.1.2 0 ms 0 ms 0 ms
2 172.16.1.10 0 ms 0 ms 0 ms
3 * * *
4 * * *
The asterisks ( * ) indicate no response from that hop in the network routing.