Fortinet black logo

Administration Guide

Combination access control & rate limiting

Combination access control & rate limiting

What if you want to allow a web crawler, but only if it is not too demanding, and comes from a source IP that is known to be legitimate for that crawler? What if you want to allow only a client that is a senior manager’s IP, and only if it hasn’t been infected by malware whose access rate is contributing to a DoS?

Advanced access control rules provide a degree of flexibility for these types of complex conditions. You can combine any or all of these criteria:

  • Source IP
  • User
  • Rate limit (including rate limiting for specific types of content)
  • HTTP header or response code
  • URL
  • Transaction or packet interval timeout
  • Geo IP
  • Parameter
  • Time period

You use the rule's filters to specify all criteria that you require allowed traffic to match.

The filters apply to request traffic only, with the following exceptions:

  • HTTP Response Code and Content Type apply to responses.
  • Signature Violation applies to either requests or responses, depending on which signatures you enable.
  • Occurrence applies to either requests or responses.

FortiWeb includes predefined rules that defend against some popular attacks. You cannot edit these predefined rules, but you can view their settings or create duplicates of them that you can edit (that is, by cloning).

Advanced access control is available even if FortiWeb derives client source IP addresses from the X-header field. For details, see Defining your proxies, clients, & X-headers.

To configure an advanced access control rule
  1. Go to Web Protection > Advanced Protection > Custom Policy, and select the Custom Rule tab.
    To access this part of the web UI, your administrator’s account access profile must have Read and Write permission to items in the Web Protection Configuration category. For details, see Permissions.
  2. Do one of the following:
  • To create a new rule, click Create New.
  • To create a new rule based on a predefined rule, select the predefined rule to use, and then click Clone.
  • If you are cloning a predefined rule, enter a name for your new rule, and then click OK. To edit or review the rule settings, select the rule, and then click Edit.
  • Configure these settings:
  • Name Type a unique name that can be referenced in other parts of the configuration. The maximum length is 63 characters.
    Action

    Select which action the FortiWeb appliance will take when it detects a violation of the rule:

    • Alert—Accept the request and generate an alert email and/or log message.
    • Alert & Deny—Block the request (or reset the connection) and generate an alert email and/or log message.

      You can customize the web page that FortiWeb returns to the client with the HTTP status code. For details, see Customizing error and authentication pages (replacement messages).

    • Deny (no log)—Block the request (or reset the connection).

    The default value is Alert.

    Caution: This setting is ignored when Monitor Mode is enabled.

    Note: Logging and/or alert email will occur only if enabled and configured. For details, see Logging and Alert email.

    Block Period

    Type the number of seconds that you want to block subsequent requests from the client after the FortiWeb appliance detects that the client has violated the rule.

    This setting is available only if Action is set to Period Block. The valid range is from 1 to 3,600 seconds (1 hour). For details, see Monitoring currently blocked IPs.

    Severity

    When rule violations are recorded in the attack log, each log message contains a Severity Level (severity_level) field. Select which severity level the FortiWeb appliance will use when it logs a violation of the rule:

    • Informative
    • Low
    • Medium
    • High

    The default value is Medium.

    Trigger Action Select which trigger, if any, that the FortiWeb appliance will use when it logs and/or sends an alert email about a violation of the rule. For details, see Viewing log messages.
    Bot Confirmation

    Enable to confirm if the client is indeed a bot. The system sends RBE (Real Browser Enforcement) JavaScript or CAPTCHA to the client to double check if it's a bot.

    For Browser

    Verification Method

    • Disabled: Not to carry out the real browser verification.
    • Real Browser Enforcement—Specifies whether FortiWeb returns a JavaScript to the client to test whether it is a web browser or automated tool when it meets any of the specified conditions. If the client fails the test or does not return results before the Validation Timeout expires, FortiWeb applies the Action. If the client appears to be a web browser, FortiWeb allows the client to exceed the action.
    • CAPTCHA Enforcement—Requires the client to successfully fulfill a CAPTCHA request. If the client cannot successfully fulfill the request within the Max Attempt Times or doesn't fulfill the request within the Validation Timeout, FortiWeb applies the Action and sends the CAPTCHA block page. For details, see Customizing error and authentication pages (replacement messages).

    Validation Timeout

    Enter the maximum amount of time (in seconds) that FortiWeb waits for results from the client.

    Available only when the Verification Method is Real Browser Enforcement or CAPTCHA Enforcement.

    Max Attempt Times

    If CAPTCHA Enforcement is selected for Verification Method, enter the maximum number of attempts that a client may attempt to fulfill a CAPTCHA request.

    For Mobile Client App

    Available only when Mobile Application Identification is enabled in System > Config > Feature Visibility.

    Verification Method

    • Disabled: Not to carry out the mobile token verification.
    • Mobile Token Validation: Requires the client to use mobile token to verify whether the traffic is from mobile devices.
      To apply mobile token validation, you must enable Mobile App Identification in Web Protection Profile.
  • Click OK.
  • Click Create New to add an entry to the set.
  • From Filter Type, select one of the following conditions that a request must match in order to be allowed, then click OK.
  • The Filter Type value determines which settings are displayed in the next dialog box.

    • Source IPv4/IPv6/IP Range—Type the IP address of a client that is allowed. Depending on your configuration of how FortiWeb derives the client’s IP, this may be the IP address that is indicated in an HTTP header rather than the IP header. For details, see Defining your proxies, clients, & X-headers.

      To enter an address range, enter the first and last address in the range separated by a hyphen. For example, for an IPv4 address, enter 192.0.2.1 – 192.0.2.155. For an IPv6 address, enter 2001::1-2001::100.

      For Meet this condition if, select one of the following:

      • Source IP matches—The request will match the condition if it contains the Source IPv4/IPv6/IP Range value.
      • Source IP does not match—The request will match the condition if it doesn't contain the Source IPv4/IPv6/IP Range value.
    • User—Enter a user name to match, and then specify whether the condition matches if the request contains the specified user name or matches only for user names other than the specified one.

      Note: This type of filter requires you to select a user tracking policy in any protection profile that uses this advanced access policy. For details, see Tracking users.

    • URL—Enter a literal URL, such as /folder1/index.htm that the HTTP request must contain in order to match the rule, or use wildcards to match multiple URLs, such as /folder1/* or /folder1/*/index.htm. Or type a regular expression that matches one or more URLs, such as /index\.jsp. Do not include the host name.

    To accept requests that do not match the URL, do not precede the URL with an exclamation mark (!). Use the CLI to configure the reverse-match {no | yes} setting for this filter. For details, see the FortiWeb CLI Reference:

    https://docs.fortinet.com/document/fortiweb/

    • HTTP Header—Indicate a single HTTP Header Name such as Host:, and all or part of its value in Header Value. The request matches the condition if that header matches the exact name or value, or matches your regular expression (depending on whether you have selected Simple String or Regular Expression). Value matching is case sensitive and supports null value match.

      If you select Header Value Reverse Match, the request matches the condition if the header does not contain the exact value or regular expression.

      If you select Missing Header Name, the request matches the condition if it does not contain the specified header. Please note that this setting does not take effect for HTTP2 packets without the following headers:

      • :method
      • :scheme
      • :path
      • :authority
      • :status

      HTTP2 packets without the above headers will not go far to be scanned against the custom rule settings. It will be considered as illegitimate and be abandoned directly when it arrives at FortiWeb at the first place.

      Optionally, enable HTTP Method Check and configure a simple string or regular expression for the HTTP method that FortiWeb will search for in the header field. When you enable HTTP Method Check, you can also enable HTTP Method Reverse Match so that the request matches the condition if the header does not contain the HTTP method's exact value or regular expression.

      To

    To prevent accidental matches, specify as much of the header’s value as possible. Do not use an ambiguous substring.

    For example, entering the value 192.0.2.1 would also match the IPs 192.0.2.10-19 and 192.0.2.100-199. This result is probably unintended. The better solution would be to configure either:

    • a regular expression such as ^192.0.2.1$ or
    • a source IP condition instead of an HTTP header condition
    • Access Rate Limit—This is the number of requests per second per client IP. Depending on your configuration of how FortiWeb will derive the client’s IP, this may be the IP address that is indicated in an HTTP header rather than the IP header. For details, see Defining your proxies, clients, & X-headers.

      You can add only one Access Rate Limit filter to each rule.

    • Signature Violation—Matches if FortiWeb detects a selected category or list of attack signatures in the request or response. The following categories are available:
      • Cross Site Scripting
      • Cross Site Scripting (Extended)
      • SQL Injection
      • SQL Injection (Extended)
      • Generic Attacks
      • Generic Attacks (Extended)
      • Known Exploits
      • Trojans
      • Information Disclosure
      • Bad Robot
      • Custom Signature (group or individual rule)
      • A custom rule Vulnerability-Scanning is predefined, with some signature categories and lists customized.
    • Geo IP—Choose the countries to match. If you select Yes, FortiWeb matches the traffic from all countries except the ones you select. If you select No, FortiWeb matches the traffic from the countries you select.

    To use one of these categories in an advanced access control rule, enable the corresponding item in your signatures configuration. For details, see Blocking known attacks & data leaks.

    • Transaction Timeout—Matches if the lifetime of a HTTP transaction exceeds the transaction timeout you specify. Specify a timeout value of 1 to 3600 seconds.
    • HTTP Response Code—Matches if a HTTP response code matches a code or range of codes that you specify. For example, 404 or 500-503. To specify more than one response code or range, create additional HTTP Response Code filters.
    • Content Type—Matches an HTTP response for a file that matches one of the specified types. Use with Occurrence to detect and control web scraping (content scraping) activity.
    • Packet Interval Timeout—Matches if the time period between packets arriving from either the client or server (request or response packets) exceeds the value in seconds you specify for Packet Timeout Interval. Enter a value from 1 to 60.
    • Time Period—Matches if the time period of a request matches that you specify. You can set a daily period or fixed period.
    • Occurrence—Matches if a transaction matches other filter types in the current rule at a rate that exceeds a threshold you specify.
  • Click OK to exit the sub-dialog and return to the rule configuration.
  • Repeat the previous steps for each individual criteria that you want to add to the access rule.
    For example, you can require both a matching request URL, HTTP header, and client source IP in order to allow a request.
    You can add only one Access Rate Limit filter to each rule.
  • Click OK to save the rule.
  • Go to Web Protection > Advanced Protection > Custom Policy, and select the Custom Policy tab.
  • Click Create New. Group the advanced access rules into a policy.
    For example, to create a policy that allows rate-limited access by 3 client IPs, you would group the corresponding 3 advanced access rules for each of those IPs into the policy.
  • Type a name for the custom policy which can be referenced in other parts of the configuration.
  • For Threat Weight, drag the bar to set the threat weight for each custom policy.
  • To apply the advanced access policy, select it as the Custom Policy in a protection profile. For details, see Configuring a protection profile for inline topologies or Configuring a protection profile for an out-of-band topology or asynchronous mode of operation.

    Attack log messages contain Custom Access Violation when this feature detects an unauthorized access attempt.

  • See also

    Combination access control & rate limiting

    What if you want to allow a web crawler, but only if it is not too demanding, and comes from a source IP that is known to be legitimate for that crawler? What if you want to allow only a client that is a senior manager’s IP, and only if it hasn’t been infected by malware whose access rate is contributing to a DoS?

    Advanced access control rules provide a degree of flexibility for these types of complex conditions. You can combine any or all of these criteria:

    • Source IP
    • User
    • Rate limit (including rate limiting for specific types of content)
    • HTTP header or response code
    • URL
    • Transaction or packet interval timeout
    • Geo IP
    • Parameter
    • Time period

    You use the rule's filters to specify all criteria that you require allowed traffic to match.

    The filters apply to request traffic only, with the following exceptions:

    • HTTP Response Code and Content Type apply to responses.
    • Signature Violation applies to either requests or responses, depending on which signatures you enable.
    • Occurrence applies to either requests or responses.

    FortiWeb includes predefined rules that defend against some popular attacks. You cannot edit these predefined rules, but you can view their settings or create duplicates of them that you can edit (that is, by cloning).

    Advanced access control is available even if FortiWeb derives client source IP addresses from the X-header field. For details, see Defining your proxies, clients, & X-headers.

    To configure an advanced access control rule
    1. Go to Web Protection > Advanced Protection > Custom Policy, and select the Custom Rule tab.
      To access this part of the web UI, your administrator’s account access profile must have Read and Write permission to items in the Web Protection Configuration category. For details, see Permissions.
    2. Do one of the following:
    • To create a new rule, click Create New.
    • To create a new rule based on a predefined rule, select the predefined rule to use, and then click Clone.
  • If you are cloning a predefined rule, enter a name for your new rule, and then click OK. To edit or review the rule settings, select the rule, and then click Edit.
  • Configure these settings:
  • Name Type a unique name that can be referenced in other parts of the configuration. The maximum length is 63 characters.
    Action

    Select which action the FortiWeb appliance will take when it detects a violation of the rule:

    • Alert—Accept the request and generate an alert email and/or log message.
    • Alert & Deny—Block the request (or reset the connection) and generate an alert email and/or log message.

      You can customize the web page that FortiWeb returns to the client with the HTTP status code. For details, see Customizing error and authentication pages (replacement messages).

    • Deny (no log)—Block the request (or reset the connection).

    The default value is Alert.

    Caution: This setting is ignored when Monitor Mode is enabled.

    Note: Logging and/or alert email will occur only if enabled and configured. For details, see Logging and Alert email.

    Block Period

    Type the number of seconds that you want to block subsequent requests from the client after the FortiWeb appliance detects that the client has violated the rule.

    This setting is available only if Action is set to Period Block. The valid range is from 1 to 3,600 seconds (1 hour). For details, see Monitoring currently blocked IPs.

    Severity

    When rule violations are recorded in the attack log, each log message contains a Severity Level (severity_level) field. Select which severity level the FortiWeb appliance will use when it logs a violation of the rule:

    • Informative
    • Low
    • Medium
    • High

    The default value is Medium.

    Trigger Action Select which trigger, if any, that the FortiWeb appliance will use when it logs and/or sends an alert email about a violation of the rule. For details, see Viewing log messages.
    Bot Confirmation

    Enable to confirm if the client is indeed a bot. The system sends RBE (Real Browser Enforcement) JavaScript or CAPTCHA to the client to double check if it's a bot.

    For Browser

    Verification Method

    • Disabled: Not to carry out the real browser verification.
    • Real Browser Enforcement—Specifies whether FortiWeb returns a JavaScript to the client to test whether it is a web browser or automated tool when it meets any of the specified conditions. If the client fails the test or does not return results before the Validation Timeout expires, FortiWeb applies the Action. If the client appears to be a web browser, FortiWeb allows the client to exceed the action.
    • CAPTCHA Enforcement—Requires the client to successfully fulfill a CAPTCHA request. If the client cannot successfully fulfill the request within the Max Attempt Times or doesn't fulfill the request within the Validation Timeout, FortiWeb applies the Action and sends the CAPTCHA block page. For details, see Customizing error and authentication pages (replacement messages).

    Validation Timeout

    Enter the maximum amount of time (in seconds) that FortiWeb waits for results from the client.

    Available only when the Verification Method is Real Browser Enforcement or CAPTCHA Enforcement.

    Max Attempt Times

    If CAPTCHA Enforcement is selected for Verification Method, enter the maximum number of attempts that a client may attempt to fulfill a CAPTCHA request.

    For Mobile Client App

    Available only when Mobile Application Identification is enabled in System > Config > Feature Visibility.

    Verification Method

    • Disabled: Not to carry out the mobile token verification.
    • Mobile Token Validation: Requires the client to use mobile token to verify whether the traffic is from mobile devices.
      To apply mobile token validation, you must enable Mobile App Identification in Web Protection Profile.
  • Click OK.
  • Click Create New to add an entry to the set.
  • From Filter Type, select one of the following conditions that a request must match in order to be allowed, then click OK.
  • The Filter Type value determines which settings are displayed in the next dialog box.

    • Source IPv4/IPv6/IP Range—Type the IP address of a client that is allowed. Depending on your configuration of how FortiWeb derives the client’s IP, this may be the IP address that is indicated in an HTTP header rather than the IP header. For details, see Defining your proxies, clients, & X-headers.

      To enter an address range, enter the first and last address in the range separated by a hyphen. For example, for an IPv4 address, enter 192.0.2.1 – 192.0.2.155. For an IPv6 address, enter 2001::1-2001::100.

      For Meet this condition if, select one of the following:

      • Source IP matches—The request will match the condition if it contains the Source IPv4/IPv6/IP Range value.
      • Source IP does not match—The request will match the condition if it doesn't contain the Source IPv4/IPv6/IP Range value.
    • User—Enter a user name to match, and then specify whether the condition matches if the request contains the specified user name or matches only for user names other than the specified one.

      Note: This type of filter requires you to select a user tracking policy in any protection profile that uses this advanced access policy. For details, see Tracking users.

    • URL—Enter a literal URL, such as /folder1/index.htm that the HTTP request must contain in order to match the rule, or use wildcards to match multiple URLs, such as /folder1/* or /folder1/*/index.htm. Or type a regular expression that matches one or more URLs, such as /index\.jsp. Do not include the host name.

    To accept requests that do not match the URL, do not precede the URL with an exclamation mark (!). Use the CLI to configure the reverse-match {no | yes} setting for this filter. For details, see the FortiWeb CLI Reference:

    https://docs.fortinet.com/document/fortiweb/

    • HTTP Header—Indicate a single HTTP Header Name such as Host:, and all or part of its value in Header Value. The request matches the condition if that header matches the exact name or value, or matches your regular expression (depending on whether you have selected Simple String or Regular Expression). Value matching is case sensitive and supports null value match.

      If you select Header Value Reverse Match, the request matches the condition if the header does not contain the exact value or regular expression.

      If you select Missing Header Name, the request matches the condition if it does not contain the specified header. Please note that this setting does not take effect for HTTP2 packets without the following headers:

      • :method
      • :scheme
      • :path
      • :authority
      • :status

      HTTP2 packets without the above headers will not go far to be scanned against the custom rule settings. It will be considered as illegitimate and be abandoned directly when it arrives at FortiWeb at the first place.

      Optionally, enable HTTP Method Check and configure a simple string or regular expression for the HTTP method that FortiWeb will search for in the header field. When you enable HTTP Method Check, you can also enable HTTP Method Reverse Match so that the request matches the condition if the header does not contain the HTTP method's exact value or regular expression.

      To

    To prevent accidental matches, specify as much of the header’s value as possible. Do not use an ambiguous substring.

    For example, entering the value 192.0.2.1 would also match the IPs 192.0.2.10-19 and 192.0.2.100-199. This result is probably unintended. The better solution would be to configure either:

    • a regular expression such as ^192.0.2.1$ or
    • a source IP condition instead of an HTTP header condition
    • Access Rate Limit—This is the number of requests per second per client IP. Depending on your configuration of how FortiWeb will derive the client’s IP, this may be the IP address that is indicated in an HTTP header rather than the IP header. For details, see Defining your proxies, clients, & X-headers.

      You can add only one Access Rate Limit filter to each rule.

    • Signature Violation—Matches if FortiWeb detects a selected category or list of attack signatures in the request or response. The following categories are available:
      • Cross Site Scripting
      • Cross Site Scripting (Extended)
      • SQL Injection
      • SQL Injection (Extended)
      • Generic Attacks
      • Generic Attacks (Extended)
      • Known Exploits
      • Trojans
      • Information Disclosure
      • Bad Robot
      • Custom Signature (group or individual rule)
      • A custom rule Vulnerability-Scanning is predefined, with some signature categories and lists customized.
    • Geo IP—Choose the countries to match. If you select Yes, FortiWeb matches the traffic from all countries except the ones you select. If you select No, FortiWeb matches the traffic from the countries you select.

    To use one of these categories in an advanced access control rule, enable the corresponding item in your signatures configuration. For details, see Blocking known attacks & data leaks.

    • Transaction Timeout—Matches if the lifetime of a HTTP transaction exceeds the transaction timeout you specify. Specify a timeout value of 1 to 3600 seconds.
    • HTTP Response Code—Matches if a HTTP response code matches a code or range of codes that you specify. For example, 404 or 500-503. To specify more than one response code or range, create additional HTTP Response Code filters.
    • Content Type—Matches an HTTP response for a file that matches one of the specified types. Use with Occurrence to detect and control web scraping (content scraping) activity.
    • Packet Interval Timeout—Matches if the time period between packets arriving from either the client or server (request or response packets) exceeds the value in seconds you specify for Packet Timeout Interval. Enter a value from 1 to 60.
    • Time Period—Matches if the time period of a request matches that you specify. You can set a daily period or fixed period.
    • Occurrence—Matches if a transaction matches other filter types in the current rule at a rate that exceeds a threshold you specify.
  • Click OK to exit the sub-dialog and return to the rule configuration.
  • Repeat the previous steps for each individual criteria that you want to add to the access rule.
    For example, you can require both a matching request URL, HTTP header, and client source IP in order to allow a request.
    You can add only one Access Rate Limit filter to each rule.
  • Click OK to save the rule.
  • Go to Web Protection > Advanced Protection > Custom Policy, and select the Custom Policy tab.
  • Click Create New. Group the advanced access rules into a policy.
    For example, to create a policy that allows rate-limited access by 3 client IPs, you would group the corresponding 3 advanced access rules for each of those IPs into the policy.
  • Type a name for the custom policy which can be referenced in other parts of the configuration.
  • For Threat Weight, drag the bar to set the threat weight for each custom policy.
  • To apply the advanced access policy, select it as the Custom Policy in a protection profile. For details, see Configuring a protection profile for inline topologies or Configuring a protection profile for an out-of-band topology or asynchronous mode of operation.

    Attack log messages contain Custom Access Violation when this feature detects an unauthorized access attempt.

  • See also