Improving performance
When you configure your FortiWeb appliance and its features, there are many settings and practices that can yield better performance.
System performance
- Delete or disable unused policies. FortiWeb allocates memory with each server policy, regardless of whether it is actually in active use. Configuring extra policies unnecessarily consumes memory and decreases performance.
- To reduce latency associated with DNS queries, use a DNS server on your local network as your primary DNS. See Configuring DNS settings.
- If your network’s devices support them, you can create one or more VLAN interfaces. VLANs reduce the size of a broadcast domain and the amount of broadcast traffic received by network hosts, which improves network performance. See Adding VLAN subinterfaces.
- If you have enabled the server health check feature as part of a server pool and one of the pool members is down for an extended period, you can improve the performance of your FortiWeb appliance by disabling the physical server, rather than allowing the server health check to continue checking for the server's responsiveness. See Configuring server up/down checks.
- Use the least intensive, earliest possible scan to deflect attacks. See Sequence of scans.
- Use Period Block if possible as the Action setting for DoS protection rules. This setting allows FortiWeb to conserve scanning resources that are under heavy demand during a DoS or DDoS attack.
Antivirus performance
- Disable scanning of BZIP2 if it is not necessary.
- Reduce the scanning buffer to the minimum necessary.
- Reduce the number of redundant levels of compression that FortiWeb will scan. Normally, people will not put a ZIP file within a ZIP file, because it is inconvenient to open and does not offer significant compression ratio improvements. Nested compression is usually used by viruses to bypass antivirus scanners.
Regular expression performance tips
- Use a simple string instead if possible. Generally, regular expressions should only be used when defining all matching text requires a complex pattern. Regular expressions such as:
^.*/index\.html$
- are usually more computationally intensive than a literal string comparison such as:
/index.html
- Reduce evaluation complexity.
|
Short regular expressions can sometimes be more complex to compute. Don’t look at the number of characters in the regular expression. Instead, think of both the usual and worst possible case in the match string: the maximum number of characters that must be compared to the pattern before a match can be verified or not.
The usual case will tell you the average CPU and RAM load. The worst case will tell you if your regular expression could sometimes cause potential hang-like conditions, temporarily blocking traffic throughput until it finishes evaluating.
|
|
If the worst possible match string is short and not complex to match, the regular expression may not be worth your time to optimize. |
For example, when using auto-learning to discover if street addresses are a valid input, scanning for postal codes or state abbreviations instead may dramatically improve performance. A pattern to fully match all possible street addresses is significantly more complex, involving many more computations, and the most difficult addresses to verify might be complex enough to impact traffic throughput.
|
|
If missed matches are an acceptable performance trade-off (for example, if matching 99% of cases is efficient, but matching 100% of cases would require deep recursion), or if you do not need to match the whole text, remove the unnecessary part of the regular expression.
For example, if a phone number always resembles 555-5555, your regular expression would not have to accommodate cases where a space separates the numbers, or it is prefixed by a country code. This is less comprehensive, but also less CPU-intensive.
|
- Avoid backtracking (i.e. revisiting the match string after failing to match part of the pattern). Backtracking occurs when regular expression features use recursion (definite or indefinite). This can increase execution time exponentially. Examples include the following:
-
Avoid nested parentheses with indefinite repeats such as:
^((a+)b+)*
which can take a very long time to evaluate, especially if a long string does not match, but this cannot be determined until the very last character is evaluated.
In the above example, both the + and * indicate matches that repeat potentially infinitely, forcing the regular expression engine to continue until it finds the longest possible match (or runs out of RAM; see Killing system-intensive processes). Using both in a nested set of parentheses compounds the problem.
-
Minimize capture groups and back-references such as:
(/a)(/b)/(c)
$0$1\?user=$2
To use back-references, FortiWeb must keep the text that matched the capture groups in memory, which increases RAM consumption.
-
Order matters if using alternate match patterns (i.e. multiple patterns are concatenated with a pipe ( | )). Put rare patterns last. If you put less likely patterns first, most times FortiWeb will be evaluating the string multiple times — not once — before it finds a match. This significantly decreases performance.
When comparing single characters, use character classes such as:
[abc]
instead of alternative matches like
(a|b|c)
Match character by character, not word by word. If words begin with the same characters, it is not efficient to evaluate the beginning of the match string multiple times — once for each possible word.
For example, to match the words “the”, “then”, “this”, and “these”, this expression is easy to read, but inefficient because it evaluates the first two characters (“th”) up to 4 times:
\b(this|the|then|these)\b
While harder to read, this expression improves performance, evaluating “th” once, and will match the most common word in English (“the”) before considering less probable words:
\bth(e(n|se)|is)\b
-
Reduce nested quantifiers such as:
(abc)+
(abc){1,6}
Worst-case evaluations do not increase computation time linearly, but exponentially. When such an expression is compiled, it also consumes much more RAM. Use the smallest possible repetition, or an alternative expression.
- Avoid Unicode character properties such as
/p{Nd} if you can use a character class instead. Due to the huge numbers and complexity of potential matches in Unicode, these can be dramatically slower.
-
Avoid look-ahead match conditions such as:
?!abcdefge
?=abcdefge
To do this, FortiWeb must make additional computations — in the example above, 8 in the best case scenario, an immediate match. FortiWeb also must keep the originally consumed match string in memory while it does this, which increases RAM consumption.
Logging performance
- If you have a FortiAnalyzer, store FortiWeb’s logs on the FortiAnalyzer to avoid resource usage associated with writing logs to FortiWeb’s own hard disks. See Configuring log destinations.
- If you do not need a traffic log, disable it to reduce the use of system resources. See Logging.
- Reduce repetitive log messages. Use the alert email settings, as shown in the illustration Log&Report > Log Policy > Email Policy, to define the interval that emails are sent if the same condition persists following the initial occurrence. See Configuring email settings.
Log&Report > Log Policy > Email Policy
- Avoid recording log messages using low severity thresholds, such as information or notification, to the local hard disk for an extended period of time. Excessive logging frequency saps system resources and can cause undue wear on the hard disk and may cause premature failure. See Configuring log destinations.
Report performance
Generating reports can be resource intensive. To avoid performance impacts, consider scheduling report generation during times with low traffic volume, such as at night and on weekends. See the illustration Log&Report > Report > Report Config and Scheduling reports.
Log&Report > Report > Report Config
Keep in mind that most reports are based upon log messages. All caveats regarding log performance also apply.
Auto-learning performance
- Each URL in an auto-learning report includes the right-click menu option Auto-learning. If a URL is dynamic or hard to predict effectively and may generate inaccurate data, you can improve performance by pausing or stopping auto-learning for that URL. See Pausing auto-learning for a URL.
- Once you have collected enough auto-learning data for generating protection profiles, consider turning off the auto-learning function to save resources. To do so, deselect the auto-learning profile in applicable server policies. See How operation mode affects server policy behavior.
- Use less computationally intensive data types and suspicious URLs, and disable unneeded ones, where possible. See Regular expression performance tips.
- Reduce the list of predefined data type groups to include just those the FortiWeb appliance is likely to encounter when gathering data for an auto-learning report. By pruning the list, you reduce the resources used to recognize data types, freeing them to improve the throughput of the FortiWeb appliance. See Auto-learning.
Auto Learn > Predefined Pattern > Data Type Group
- When configuring a suspicious URL pattern, clear one or more web server type options if you do not operate all three web servers, as shown in the illustration Auto Learn > Predefined Pattern > Suspicious URL. By pruning the list, you reduce the resources used by the FortiWeb appliance when applying the rule. See Auto-learning.
Auto Learn > Predefined Pattern > Suspicious URL
Disabling unnecessary server information disclosure signatures in Web Protection > Known Attacks > Signatures > Signature Details
The Information Disclosure feature can potentially require the FortiWeb appliance to rewrite the header of every request from a server, resulting in reduced performance. Fortinet recommends enabling this feature only to help you identify information disclosure through logging, and until you can reconfigure the server to omit such sensitive information.
Omitting large files from the backup in Web Protection > Web Anti-Defacement > Web Site with Anti-Defacement
Unless you need to back up large files, reduce the setting for the Skip Files Larger Than option from the default of 10 240 KB.
Use the Skip Files With These Extensions option to exclude specific types of large files, such as compressed files and video clips.
Vulnerability scan performance
Vulnerability scan performance depends on the speed and reliability of your network. It also can be impacted by your configuration. See Delay Between Each Request.
Packet capture performance
Packet capture can be useful for troubleshooting but can be resource intensive. (See Packet capture.) To minimize the performance impact on your FortiWeb appliance, use packet capture only during periods of minimal traffic. Use a local console CLI connection rather than a Telnet or SSH CLI connection, and be sure to stop the command when you are finished.
TCP transmission performance tuning
FortiWeb allows you to tune TCP transmission performance by adjusting the buffer parameter of TCP connections through the CLI over high-bandwidth, high-latency networks. Large-size file transmissions (usually larger than 150MB) or serious traffic congestion between FortiWeb and backend servers is a common situation that might cause clients to experience poor TCP performance.
The tcp-buffer option in system network-option defines the TCP_mem variable to indicate to FortiWeb how the TCP stack should behave regarding memory usage. It consists of three values (the values are measured in memory pages):
- low: This value indicates the peformance value for a desired low memory usage threshold. Below this point, the TCP stack does not adjust the memory usage by interacting with TCP receive and send buffers for the sockets.
- pressure: This value tells FortiWeb the point at which it must start pressuring memory usage down. Memory pressure is continued until the memory usage enters the lowe threshold and it maintains the default behavior of the low threshold. This downward pressure is applied by adjusting the TCP receive and send buffers for the sockets until the low threshold performance can be maintained.
- high: This value indicates the maximum memory pages FortiWeb may use. If this value is reached, TCP streams and packets are dropped until FortiWeb begins using fewer memory pages again.
Setting the tcp-buffer option as default, high, or max from the CLI specifies the three values to FortiWeb as following:
while tcp-buffer=default, (low, pressure, high) = (16384, 32768, 65536)
while tcp-buffer=high, (low, pressure, high) = (16384, 87380, 629145)
while tcp-buffer=max, (low, pressure, high) = (16384, 174760, 1258290)
Note that although the tcp-buffer option can provide an increase in throughput on high bandwidth networks, it decreases the number of concurrent TCP connections established on FortiWeb.
Example
config system network-option
set tcp-buffer high
end
Open topic with navigation