Key Concepts : Understanding FortiDDoS rate limiting thresholds
Understanding FortiDDoS rate limiting thresholds
This section includes the following information:
Granular monitoring and rate limiting
Increasingly, instead of simple bandwidth attacks, attackers try to avoid detection by creating attacks that mimic the behavior of a large number of clients. Evading an NBA system is easy if attackers do coarse-grained rate-based control. Because the content of the malicious requests does not differ from that of legitimate ones, the resulting attacks are hard to defend against using standard techniques.
In contrast, FortiDDoS uses a combination of Layer 3, 4, and 7 counters and continuously adapts expected inbound and outbound rates for each of these traffic parameters.
Granular analytics also enable targeted mitigation responses. For example, if a few TCP connections are exceeding bandwidth, the system blocks those connections rather than all connections. If a single destination is under attack, FortiDDoS drops packets to that destination while others continue. During fragmented flood attacks, all non-fragmented packets continue as usual. During a port flood to a non-service port, the packets to other ports continue.
Granularity helps to increase the goodput (the throughput of useful data) of the system.
Table 1 lists the counters that FortiDDoS uses to detect subtle changes in the behavior of network traffic.
Table 1: FortiDDoS counters
Layer | Type | Counters |
3 | 1. Protocol flood | 256 protocols per SPP per direction |
2. Fragment flood | 1 counter per SPP per direction |
3. IP source flood & source tracking | 1 million sources per TP2 |
4. IP destination flood | 1 million destinations per TP2 |
4 | 1. TCP port flood | 65k per SPP per direction |
2. UDP port flood | 65k per SPP per direction |
3. ICMP type/code flood | 65k per SPP per direction |
4. TCP connection flood | 1 million connections per TP2 |
5. Legitimate IP table (for SYN and zombie flood) | 2 million IP addresses per TP2 |
6. SYN flood | 1 SYN counter per SPP per direction |
7. SYN rate/source | 1 million sources per TP2 |
8. Concurrent connections/source | 1 million sources per TP2 |
9. Concurrent connections/destination | 1 million destinations per TP2 |
10. ACK/destination | 1 million destinations per TP2 |
11. RST/destination | 1 million destinations per TP2 |
12. FIN/destination | 1 million destinations per TP2 |
7 | 1. HTTP method | 1 counter per SPP per direction |
2. URL Flood | 32,767 URLs per SPP per direction |
3. Host | 512 headers per SPP per direction |
4. Referer | 512 headers per SPP per direction |
5. Cookie | 512 headers per SPP per direction |
6. User-Agent | 512 headers per SPP per direction |
Source tracking table
FortiDDoS TP2 traffic processors maintain massive connection tables and still perform with low latency. Each TP2 has a source tracking table with 1,000,000 rows.
The source tracking table enables FortiDDoS to correlate sources with attack events and apply a more stringent blocking period to the sources that exceeded maximum rate limits.
The source tracking table also enables the special “per-source” thresholds described in
Table 2.
Table 2: Per-source thresholds
Counter | Description |
most-active-source | This counter establishes a maximum packet rate for any IP packet from a single source. A rate that exceeds the adjusted baseline is anomalous and treated as a Source Flood attack event. In conjunction with the Source Multiplier, the most-active-source threshold is useful in tracking and blocking non-spoofed sources that are participating in an attack. See Figure 17. How is the threshold determined? When it establishes baseline traffic statistics, FortiDDoS records the highest packet rate from a single source during the observation period. In a one hour observation period, FortiDDoS collects a data point for twelve five minute windows. The data point is the highest rate observed in any one second during the five minute window. If the packet rate data points for most-active-source are 1000, 2000, 1000, 2000, 1000, 2000, 1000, 2000, 3000, 2000, 1000, and 2000, the generated statistic is the highest one: 3000. |
syn-per-src | This counter establishes a maximum packet rate for SYN packets from a single source. A rate that exceeds the adjusted baseline is anomalous and treated as a SYN Flood From Source attack event. |
concurrent-connections-per-source | This counter establishes a maximum packet rate for concurrent connections from a single source. A count that exceeds the adjusted baseline is anomalous and treated as an Excessive Concurrent Connections Per Source attack event. |
Destination tracking table
Each TP2 has a destination table with 1,000,000 rows. This table tracks the packet rate for every destination and is used for “per-destination” thresholds.
The destination tracking table enables FortiDDoS to prevent destination flood attacks and slow connection attacks that are targeted at individual destinations. The “per-destination” thresholds enable it to do so without affecting the rates for other destinations in the SPP.
Table 3 describes the per-destination thresholds.
Table 3: Per-destination thresholds
Counter | Description |
most-active-destination | This counter establishes a maximum packet rate to any one destination. A rate that exceeds the adjusted baseline is anomalous and treated as a Destination Flood attack event. How is the threshold determined? When it establishes baseline traffic statistics, FortiDDoS records the highest packet rate to any single destination during the observation period. In a one hour observation period, FortiDDoS collects a data point for twelve five minute windows. The data point is the highest rate observed in any one second during the five minute window. If the packet rate data points for most-active-destination are 100000, 200000, 100000, 200000, 100000, 200000, 100000, 200000, 300000, 200000, 100000, and 2000, the generated statistic is the highest one: 300000. |
syn-per-dst, ack-per-dst, fin-per-dst, rst-per-dst | These counters establish a maximum packet rate for particular TCP packets to a single destination. A rate that exceeds the adjusted baseline is anomalous and treated as a Excessive TCP Packets Per Destination flood attack event. When the syn-per-dst limits are exceeded for a particular destination, the SYN flood mitigation mode tests are applied to all new connection requests to that particular destination. Traffic to other destinations is not subject to the tests. |
estab-per-dst | This counter establishes a maximum rate of established connections for a single destination. A rate that exceeds the adjusted baseline is anomalous and treated as an attack. |
concurrent-connections-per-destination | This counter establishes a maximum packet rate for concurrent connections to a single destination. A count that exceeds the adjusted baseline is anomalous and treated as an Excessive Concurrent Connections Per Destination attack event. |
Continuous learning and adaptive thresholds
Most NBA systems use fixed value thresholds. Traffic, however, is never static. It shows trends and seasonality (a predictable or expected variation).
FortiDDoS uses adaptive thresholds. Adaptive thresholds take into account the traffic’s average, trend, and seasonality (expected or predictable variations).
Traffic prediction
Unlike other network behavior analysis (NBA) systems, FortiDDoS never stops learning. It continuously models inbound and outbound traffic patterns for key Layer 3, Layer 4, and Layer 7 parameters.
FortiDDoS uses the following information to model normal and abnormal traffic:
• The historical base, or weighted average, of recent traffic (more weight is given to recent traffic)
• The trend, or slope, of the traffic
• The seasonality of traffic over historical time periods
FortiDDoS uses these statistics to create a forecast for the next traffic period.
Traffic is nondeterministic; therefore, the forecast cannot be exact. The extent to which an observed traffic pattern is allowed to exceed its forecast is bounded by thresholds. Generally speaking, a threshold is a baseline rate that the system uses to compare observed traffic rates to determine whether a rate anomaly is occurring.
The FortiDDoS system maintains multiple thresholds for each key Layer 3, Layer 4, and Layer 7 parameter:
• Configured minimum threshold
• Estimated threshold
• Adaptive limit maximum threshold
• Adjustments for proxy IP addresses
• Packet count multipliers applied to traffic associated with an attack
Figure 6 illustrates how the system maintains multiple thresholds. The sections that follow explain the significance of each.
Configured minimum thresholds
The configured minimum threshold is a baseline of normal counts or rates. The baseline can be generated (based on statistics collected during the learning period) or stipulated (based on defaults or manually configured settings).
The configured minimum threshold is a factor in setting rate limits, but it is not itself the rate limit. Rate limits are set by the estimated threshold, a limit that is subject to heuristic adjustment based on average, trend, and seasonality.
Many of the graphs in the Monitor menu display the configured minimum threshold as a reference.
Table 4 summarizes the alternative methods for setting the configured minimum threshold.
Table 4: Setting the configured minimum threshold
Menu | Usage |
Protection Profile > Thresholds > System Recommendation | The recommended method for setting the configured minimum thresholds. The configured minimum thresholds are a product of the observed rates adjusted by a percentage that you specify. |
Protection Profile > Thresholds > Thresholds | The thresholds configuration is open. You can set user-defined thresholds and fine-tune them. You might be able to set reasonable values for port and protocol thresholds based on your knowledge of your network’s services and server capacity. Most likely, you must become a FortiDDoS expert before you will be able to set reasonable values for Scalar thresholds. |
Protection Profile > Thresholds > Emergency Setup | Use if you do not have time to use Detection Mode to establish a baseline. |
Protection Profile > Thresholds > Factory Default | Use to quickly restore the system to high values. The factory defaults are high to avoid possible traffic disruption when you first put the system inline. In general, you use these settings together with Detection Mode when you are setting an initial baseline or a new baseline. |
Protection Profile > Thresholds > Percent Adjust | Use when you expect a spike in legitimate traffic due to an event that impacts business, like a news announcement or holiday shopping season. |
Estimated thresholds
The estimated threshold is a calculated rate limit, based on heuristic adjustments.
The system models an adjusted normal baseline based on average, trend, and seasonality. It uses the heuristics to distinguish attack traffic from increases in traffic volume that is the result of legitimate users accessing protected resources.
The minimum value of an estimated threshold is the configured minimum threshold. In other words, if it is not predicting normal traffic becoming heavier than the baseline, it allows a rate at least as high as the configured minimum threshold.
The maximum value of an estimated threshold is the product of the configured minimum threshold and the adaptive limit. In other words, the system does enforce an absolute maximum rate limit.
Adaptive limit
The adaptive limit is a percentage of the configured minimum threshold.
An adaptive limit of 100% means no dynamic threshold estimation adjustment takes place once the configured minimum threshold is reached (that is, the threshold is a fixed value).
The product of the configured minimum threshold and adaptive limit is the absolute maximum rate limit. If the adaptive limit is 150% (the default), the system can increase the estimated threshold up to 150% of the value of the configured minimum threshold.
There are scenarios where FortiDDoS drops legitimate traffic because it cannot adapt quickly enough to a sudden change in traffic patterns. For example, when a news flash or other important announcement increases traffic to a company’s website. In these situations, you can use the Protection Profiles > Thresholds > Percent Adjust configuration page to increase all configured thresholds by a specific percentage.
Adjustments for proxy IP addresses
FortiDDoS can take account of the possibility that a source IP address might be a proxy IP address, and adjust the threshold triggers accordingly. If a source IP address is determined to be a proxy IP address, the system adjusts thresholds for a few key parameters by a factor you specify on the Global Settings > Proxy IP page.
Packet count multipliers applied to traffic associated with an attack
Packet count multipliers are adjustments to counters that are applied to traffic associated with an attack so that the thresholds that control drop and block responses are triggered sooner. You can configure multipliers for the following types of traffic:
• Source floods—Traffic from a source that the system has identified as the source of a flood.
• Layer 7 floods—Traffic for attacks detected based on a URL or Host, Referer, Cookie, or User-Agent header field.
You can use the Protection Profiles > Settings page to specify packet count multipliers.
When both Source flood and Layer 7 flood conditions are met, the packet count multipliers are compounded. For example, when there is a User Agent flood attack, a source is sending a User-Agent that is overloaded. If the Source multiplier is 4 and the Layer 7 multiplier is 64, the total multiplier that is applied to such traffic is 4 x 64 = 264. In effect, each time the source sends a Layer 7 packet with that particular User-Agent header, FortiDDoS considers each packet the equivalent of 256 packets.
Hierarchical nature of protocols and implication on thresholds
An HTTP packet has Layer 7, Layer 4 (TCP), and Layer 3 (IP) properties. See
Figure 7. A packet must be within the estimated thresholds of all these counters in order to pass through the FortiDDoS gateway. When it sets recommended thresholds, the system takes account of this complexity. If you set thresholds manually, you must also be sure that Layer 7 rates are consistent with Layer 4 and Layer 3 rates.
Figure 8 illustrates system processing for an HTTP packet.
This packet is an IPv4 packet and TCP packet as well.
An IPv4 packet has following properties:
• Protocol
• TOS
• Fragment or not a fragment
• A source IP address (the system can monitor the packet rate from that specific source)
A TCP packet has following properties:
• Destination port
• SYN or not a SYN packet
• TCP connection tuple (the system can monitor the packet rate within that connection)
An HTTP packet has following properties:
• Method (for example, GET)
• URL
Figure 9 illustrates system processing for a UDP packet.
This packet is an IPv4 packet and UDP packet as well.
An IPv4 packet has following properties:
• Protocol
• TOS
• Fragment or not a fragment
• IP option values
• A source IP address, and hence packet rate from that specific source
A UDP packet has following properties:
• Destination port
If a server supports TCP, UDP, and ICMP services, the rate of IP packets is an aggregate of rates for TCP, UDP, and ICMP packets. Similarly if the same server is a web server as well as an SMTP server, the TCP packet rate is the sum of packet rates for port 80 and port 25.
To summarize, because determining thresholds is a hierarchical process, avoid setting low thresholds on common conditions that can cause FortiDDoS to block legitimate traffic as well as attack traffic. The more specific you are about the type of traffic you want to allow as ‘normal’, the more effective the FortiDDoS is in blocking other traffic.
