Fortinet black logo

Administration Guide

Configuring threshold based detection

Configuring threshold based detection

You can configure threshold based detection rules to define occurrence, time period, severity, and trigger policy, etc of the following suspicious behaviors, and thus FortiWeb judges whether the request comes from a human or a bot.

  • Crawler
  • Vulnerability Scanning
  • Slow Attack
  • Content Scraping
  • Illegal User Scan

To configure a threshold based detection rule

  1. Go to Bot Mitigation > Threshold Based Detection.
  2. Click Create New.
  3. For Name, enter a name for the threshold based detection rule that can be referenced in bot mitigation policy.
  4. Configure these settings:

    Bot Detection Settings

    Crawler Detection

    Occurrence

    Define the frequency that FortiWeb detects 403 and 404 response codes returned by the web server. The default value is 100.

    Within (Seconds)

    Specify the time period, in seconds, during which FortiWeb detects the 403 and 404 response codes. The default value is 10.

    Action

    Select which action FortiWeb will take when it detects a crawler:

    • Alert—Accept the connection and generate an alert email and/or log message.

    • Alert & Deny—Block the request (or reset the connection) and generate an alert and/or log message.

    • Deny (no log)—Block the request (or reset the connection).

    • Period Block—Block subsequent requests from the client for a number of seconds. Also configure Period Block.

    The default value is Alert.

    Period Block

    Enter the number of seconds that you want to block subsequent requests from a client after FortiWeb detects a crawler. The valid range is 1–3,600 seconds (1 hour).

    This setting is available only if Action is set to Period Block.

    Severity

    When policy violations are recorded in the attack log, each log message contains a Severity Level (severity_level) field. Select which severity level FortiWeb will use when it logs a crawler:

    • Informative
    • Low
    • Medium
    • High

    The default value is Medium.

    Trigger Policy

    Select the trigger, if any, that FortiWeb will use when it logs and/or sends an alert email about a crawler. For details, see Viewing log messages.

    Vulnerability Scanning Detection

    Occurrence

    Define the frequency that FortiWeb detects attack signatures. The default value is 100.

    Within (Seconds)

    Specify the time period, in seconds, during which FortiWeb monitors the attack signatures. The default value is 10.

    Action

    Select which action FortiWeb will take when it detects vulnerability scanning:

    • Alert—Accept the connection and generate an alert email and/or log message.

    • Alert & Deny—Block the request (or reset the connection) and generate an alert and/or log message.

    • Deny (no log)—Block the request (or reset the connection).

    • Period Block—Block subsequent requests from the client for a number of seconds. Also configure Period Block.

    The default value is Alert.

    Period Block

    Enter the number of seconds that you want to block subsequent requests from a client after FortiWeb detects vulnerability scanning. The valid range is 1–3,600 seconds (1 hour).

    This setting is available only if Action is set to Period Block.

    Severity

    When policy violations are recorded in the attack log, each log message contains a Severity Level (severity_level) field. Select which severity level FortiWeb will use when it logs vulnerability scanning:

    • Informative
    • Low
    • Medium
    • High

    The default value is Medium.

    Trigger Policy

    Select the trigger, if any, that FortiWeb will use when it logs and/or sends an alert email about vulnerability scanning. For details, see Viewing log messages.

    Slow Attack Detection

    HTTP Transaction Timeout

    Specify a timeout value, in seconds, for the HTTP transaction. The default value is 60.

    Packet Interval Timeout

    Specify the timeout value, in seconds, for interval between packets arriving from either the client or server (request or response packets). The default value is 10.

    Occurrence

    Define the frequency that FortiWeb detects slow attack activities. The default value is 5.

    Within (Seconds)

    Specify the time period, in seconds, during which FortiWeb detects slow attack activities. The default value is 100.

    Action

    Select which action FortiWeb will take when it detects slow attack activities:

    • Alert—Accept the connection and generate an alert email and/or log message.

    • Alert & Deny—Block the request (or reset the connection) and generate an alert and/or log message.

    • Deny (no log)—Block the request (or reset the connection).

    • Period Block—Block subsequent requests from the client for a number of seconds. Also configure Period Block.

    The default value is Alert.

    Period Block

    Enter the number of seconds that you want to block subsequent requests from a client after FortiWeb detects slow attack activities. The valid range is 1–3,600 seconds (1 hour).

    This setting is available only if Action is set to Period Block.

    Severity

    When policy violations are recorded in the attack log, each log message contains a Severity Level (severity_level) field. Select which severity level FortiWeb will use when it logs slow attack activities:

    • Informative
    • Low
    • Medium
    • High

    The default value is Medium.

    Trigger Policy

    Select the trigger, if any, that FortiWeb will use when it logs and/or sends an alert email about slow attack activities. For details, see Viewing log messages.

    Content Scraping Detection

    The content types include text/html, text/plain, text/xml, application/xml, application/soap+xml, and application/json.

    Occurrence

    Define the frequency that FortiWeb detects content scraping activities. The default value is 100.

    Within (Seconds)

    Specify the time period, in seconds, during which FortiWeb detects content scraping activities. The default value is 30.

    Action

    Select which action FortiWeb will take when it detects content scraping activities:

    • Alert—Accept the connection and generate an alert email and/or log message.

    • Alert & Deny—Block the request (or reset the connection) and generate an alert and/or log message.

    • Deny (no log)—Block the request (or reset the connection).

    • Period Block—Block subsequent requests from the client for a number of seconds. Also configure Period Block.

    The default value is Alert.

    Period Block

    Enter the number of seconds that you want to block subsequent requests from a client after FortiWeb detects content scraping activities. The valid range is 3,600 seconds (1 hour).

    This setting is available only if Action is set to Period Block.

    Severity

    When policy violations are recorded in the attack log, each log message contains a Severity Level (severity_level) field. Select which severity level FortiWeb will use when it logs content scraping activities:

    • Informative
    • Low
    • Medium
    • High

    The default value is Medium.

    Trigger Policy

    Select the trigger, if any, that FortiWeb will use when it logs and/or sends an alert email about content scraping activities. For details, see Viewing log messages.

    Illegal User Scan: Available only when you enable User Tracking in Web Protection Profile.

    Request URL

    Specify the URL used to match requests so that security headers can be applied to responses of the matched requests.

    After filling in the field with a regular expression, it is possible to fine-tune the expression in a Regular Expression Validator by clicking the >> button on the side. For details, see Appendix D: Regular expressions .

    Occurrence

    Define the frequency that FortiWeb detects username in requests. The default value is 100.

    Within (Seconds)

    Enter the length of time, in seconds, which FortiWeb detects frequency of username in requests. The default value is 10.

    Action

    Select which action FortiWeb will take when it detects illegal user scan:

    • Alert—Accept the connection and generate an alert email and/or log message.

    • Alert & Deny—Block the request (or reset the connection) and generate an alert and/or log message.

    • Deny (no log)—Block the request (or reset the connection).

    • Period Block—Block subsequent requests from the client for a number of seconds. Also configure Period Block.

    The default value is Alert.

    Period Block

    Enter the number of seconds that you want to block subsequent requests from a client after FortiWeb detects illegal user scan. The valid range is 1–3,600 seconds (1 hour).

    This setting is available only if Action is set to Period Block.

    Severity

    When illegal user scan is recorded in the attack log, each log message contains a Severity Level (severity_level) field. Select which severity level FortiWeb will use when it logs illegal user scan:

    • Informative
    • Low
    • Medium
    • High

    The default value is Medium.

    Trigger Policy

    Select the trigger, if any, that FortiWeb will use when it logs and/or sends an alert email about illegal user scan. For details, see Viewing log messages.

    Bot Confirmation Settings

    Bot Confirmation

    For Browser

    Verification Method

    • Disabled: Not to carry out the real browser verification.
    • Real Browser Enforcement: Specifies whether FortiWeb returns a JavaScript to the client to test whether it is a web browser.
    • CAPTCHA Enforcement: Requires the client to successfully fulfill a CAPTCHA request.

    Validation Timeout

    Enter the maximum amount of time (in seconds) that FortiWeb waits for results from the client.

    Available only when the Verification Method is Real Browser Enforcement or CAPTCHA Enforcement.

    Max Attempt Times

    If CAPTCHA Enforcement is selected for Verification Method, enter the maximum number of attempts that a client may attempt to fulfill a CAPTCHA request.

    Available only when the Verification Method is CAPTCHA Enforcement.

    For Mobile Client App

    Available only when Mobile Application Identification is enabled in System > Config > Feature Visibility.

    Verification Method

    • Disabled: Not to carry out the mobile token verification.
    • Mobile Token Validation: Requires the client to use mobile token to verify whether the traffic is from mobile devices.
      To apply mobile token validation, you must enable Mobile App Identification in Web Protection Profile.
  5. Click OK.
  6. You can view the details of the created rule in the threshold based detection rule table.

To apply the threshold based detection rule in a bot mitigation policy, see Configuring bot mitigation policy.

Configuring threshold based detection

You can configure threshold based detection rules to define occurrence, time period, severity, and trigger policy, etc of the following suspicious behaviors, and thus FortiWeb judges whether the request comes from a human or a bot.

  • Crawler
  • Vulnerability Scanning
  • Slow Attack
  • Content Scraping
  • Illegal User Scan

To configure a threshold based detection rule

  1. Go to Bot Mitigation > Threshold Based Detection.
  2. Click Create New.
  3. For Name, enter a name for the threshold based detection rule that can be referenced in bot mitigation policy.
  4. Configure these settings:

    Bot Detection Settings

    Crawler Detection

    Occurrence

    Define the frequency that FortiWeb detects 403 and 404 response codes returned by the web server. The default value is 100.

    Within (Seconds)

    Specify the time period, in seconds, during which FortiWeb detects the 403 and 404 response codes. The default value is 10.

    Action

    Select which action FortiWeb will take when it detects a crawler:

    • Alert—Accept the connection and generate an alert email and/or log message.

    • Alert & Deny—Block the request (or reset the connection) and generate an alert and/or log message.

    • Deny (no log)—Block the request (or reset the connection).

    • Period Block—Block subsequent requests from the client for a number of seconds. Also configure Period Block.

    The default value is Alert.

    Period Block

    Enter the number of seconds that you want to block subsequent requests from a client after FortiWeb detects a crawler. The valid range is 1–3,600 seconds (1 hour).

    This setting is available only if Action is set to Period Block.

    Severity

    When policy violations are recorded in the attack log, each log message contains a Severity Level (severity_level) field. Select which severity level FortiWeb will use when it logs a crawler:

    • Informative
    • Low
    • Medium
    • High

    The default value is Medium.

    Trigger Policy

    Select the trigger, if any, that FortiWeb will use when it logs and/or sends an alert email about a crawler. For details, see Viewing log messages.

    Vulnerability Scanning Detection

    Occurrence

    Define the frequency that FortiWeb detects attack signatures. The default value is 100.

    Within (Seconds)

    Specify the time period, in seconds, during which FortiWeb monitors the attack signatures. The default value is 10.

    Action

    Select which action FortiWeb will take when it detects vulnerability scanning:

    • Alert—Accept the connection and generate an alert email and/or log message.

    • Alert & Deny—Block the request (or reset the connection) and generate an alert and/or log message.

    • Deny (no log)—Block the request (or reset the connection).

    • Period Block—Block subsequent requests from the client for a number of seconds. Also configure Period Block.

    The default value is Alert.

    Period Block

    Enter the number of seconds that you want to block subsequent requests from a client after FortiWeb detects vulnerability scanning. The valid range is 1–3,600 seconds (1 hour).

    This setting is available only if Action is set to Period Block.

    Severity

    When policy violations are recorded in the attack log, each log message contains a Severity Level (severity_level) field. Select which severity level FortiWeb will use when it logs vulnerability scanning:

    • Informative
    • Low
    • Medium
    • High

    The default value is Medium.

    Trigger Policy

    Select the trigger, if any, that FortiWeb will use when it logs and/or sends an alert email about vulnerability scanning. For details, see Viewing log messages.

    Slow Attack Detection

    HTTP Transaction Timeout

    Specify a timeout value, in seconds, for the HTTP transaction. The default value is 60.

    Packet Interval Timeout

    Specify the timeout value, in seconds, for interval between packets arriving from either the client or server (request or response packets). The default value is 10.

    Occurrence

    Define the frequency that FortiWeb detects slow attack activities. The default value is 5.

    Within (Seconds)

    Specify the time period, in seconds, during which FortiWeb detects slow attack activities. The default value is 100.

    Action

    Select which action FortiWeb will take when it detects slow attack activities:

    • Alert—Accept the connection and generate an alert email and/or log message.

    • Alert & Deny—Block the request (or reset the connection) and generate an alert and/or log message.

    • Deny (no log)—Block the request (or reset the connection).

    • Period Block—Block subsequent requests from the client for a number of seconds. Also configure Period Block.

    The default value is Alert.

    Period Block

    Enter the number of seconds that you want to block subsequent requests from a client after FortiWeb detects slow attack activities. The valid range is 1–3,600 seconds (1 hour).

    This setting is available only if Action is set to Period Block.

    Severity

    When policy violations are recorded in the attack log, each log message contains a Severity Level (severity_level) field. Select which severity level FortiWeb will use when it logs slow attack activities:

    • Informative
    • Low
    • Medium
    • High

    The default value is Medium.

    Trigger Policy

    Select the trigger, if any, that FortiWeb will use when it logs and/or sends an alert email about slow attack activities. For details, see Viewing log messages.

    Content Scraping Detection

    The content types include text/html, text/plain, text/xml, application/xml, application/soap+xml, and application/json.

    Occurrence

    Define the frequency that FortiWeb detects content scraping activities. The default value is 100.

    Within (Seconds)

    Specify the time period, in seconds, during which FortiWeb detects content scraping activities. The default value is 30.

    Action

    Select which action FortiWeb will take when it detects content scraping activities:

    • Alert—Accept the connection and generate an alert email and/or log message.

    • Alert & Deny—Block the request (or reset the connection) and generate an alert and/or log message.

    • Deny (no log)—Block the request (or reset the connection).

    • Period Block—Block subsequent requests from the client for a number of seconds. Also configure Period Block.

    The default value is Alert.

    Period Block

    Enter the number of seconds that you want to block subsequent requests from a client after FortiWeb detects content scraping activities. The valid range is 3,600 seconds (1 hour).

    This setting is available only if Action is set to Period Block.

    Severity

    When policy violations are recorded in the attack log, each log message contains a Severity Level (severity_level) field. Select which severity level FortiWeb will use when it logs content scraping activities:

    • Informative
    • Low
    • Medium
    • High

    The default value is Medium.

    Trigger Policy

    Select the trigger, if any, that FortiWeb will use when it logs and/or sends an alert email about content scraping activities. For details, see Viewing log messages.

    Illegal User Scan: Available only when you enable User Tracking in Web Protection Profile.

    Request URL

    Specify the URL used to match requests so that security headers can be applied to responses of the matched requests.

    After filling in the field with a regular expression, it is possible to fine-tune the expression in a Regular Expression Validator by clicking the >> button on the side. For details, see Appendix D: Regular expressions .

    Occurrence

    Define the frequency that FortiWeb detects username in requests. The default value is 100.

    Within (Seconds)

    Enter the length of time, in seconds, which FortiWeb detects frequency of username in requests. The default value is 10.

    Action

    Select which action FortiWeb will take when it detects illegal user scan:

    • Alert—Accept the connection and generate an alert email and/or log message.

    • Alert & Deny—Block the request (or reset the connection) and generate an alert and/or log message.

    • Deny (no log)—Block the request (or reset the connection).

    • Period Block—Block subsequent requests from the client for a number of seconds. Also configure Period Block.

    The default value is Alert.

    Period Block

    Enter the number of seconds that you want to block subsequent requests from a client after FortiWeb detects illegal user scan. The valid range is 1–3,600 seconds (1 hour).

    This setting is available only if Action is set to Period Block.

    Severity

    When illegal user scan is recorded in the attack log, each log message contains a Severity Level (severity_level) field. Select which severity level FortiWeb will use when it logs illegal user scan:

    • Informative
    • Low
    • Medium
    • High

    The default value is Medium.

    Trigger Policy

    Select the trigger, if any, that FortiWeb will use when it logs and/or sends an alert email about illegal user scan. For details, see Viewing log messages.

    Bot Confirmation Settings

    Bot Confirmation

    For Browser

    Verification Method

    • Disabled: Not to carry out the real browser verification.
    • Real Browser Enforcement: Specifies whether FortiWeb returns a JavaScript to the client to test whether it is a web browser.
    • CAPTCHA Enforcement: Requires the client to successfully fulfill a CAPTCHA request.

    Validation Timeout

    Enter the maximum amount of time (in seconds) that FortiWeb waits for results from the client.

    Available only when the Verification Method is Real Browser Enforcement or CAPTCHA Enforcement.

    Max Attempt Times

    If CAPTCHA Enforcement is selected for Verification Method, enter the maximum number of attempts that a client may attempt to fulfill a CAPTCHA request.

    Available only when the Verification Method is CAPTCHA Enforcement.

    For Mobile Client App

    Available only when Mobile Application Identification is enabled in System > Config > Feature Visibility.

    Verification Method

    • Disabled: Not to carry out the mobile token verification.
    • Mobile Token Validation: Requires the client to use mobile token to verify whether the traffic is from mobile devices.
      To apply mobile token validation, you must enable Mobile App Identification in Web Protection Profile.
  5. Click OK.
  6. You can view the details of the created rule in the threshold based detection rule table.

To apply the threshold based detection rule in a bot mitigation policy, see Configuring bot mitigation policy.