Fortinet black logo

Administration Guide

Configuring anomaly detection policy

Configuring anomaly detection policy

Anomaly detection policies are part of a server policy. They are created on the Policy > Sever Policy page. All anomaly detection policies that you create will show up on the Machine Learning > Anomaly Detection page, where you can configure or edit them to your preference.

To configure an anomaly detection policy:

  1. Click Machine Learning > Anomaly Detection .
  2. Double-click the server policy that contains the desired anomaly detection policy (or highlight it and then click the Edit button on top of the page) to open it. The Edit Anomaly Detection Configuration page opens, which breaks down anomaly detection policy into several sections, each of which has various parameters you can use to configure the policy.
  3. Follow the instructions in the following subsections to configure an anomaly detection policy.
  4. Click OK when done.
Sections & Parameters Function
Learning Cycle

Sample Collection mode

Sample Collection Period

When a sample is collected, the system generalized it into a pattern. For example, “abcd_123@abc.com” and “abcdefgecdf_12345678@efg.com” will both be generalized to the pattern “A_N@A.A”. The anomaly detection model is built based on the patterns, not the raw samples.

Extended: In Extended mode, it's required to also set the learning time. In extended mode at least 2500 samples will be collected and the sample collection period lasts for the specified weeks. For example, if you choose extended mode and set 1 week, the system stops collecting samples after 1 week if at least 2500 samples are collected by then, or continues collecting samples after 1 week until 2500 samples are collected.

Normal:In Normal mode, the system builds an initial model when the sample count reaches 400.

The system runs the initial model to detect anomalies, while it keeps collecting more samples to refine it.
Once the number of samples accumulates to 1200, the system will evaluate whether the patterns vary largely since the initial model is built.

  • If there are very few patterns generalized, it indicates the patterns are stable. The system will switch the initial model to a standard model.
  • If a lot of new patterns keeps coming in, the system will continue collecting more samples to cover as much patterns as possible. It won't switch to standard model until the patterns become stable.

The above mentioned numbers 400 and 1200 are configurable through CLI. Refer to config waf machine-learning-policy in FortiWeb CLI Reference.

Whether in extended or normal mode, the system keeps refining the model even after it's in running status.

Dynamically update when parameters change

Applications change frequently as new URLs are added and existing parameters provide new functions. This means the mathematical model of the same parameter might be different from what FortiWeb originally observed during the collection phase. In this case, FortiWeb needs to rebuild the mathematical model for the parameter.

Enable this option to automatically update the mathematical models of the parameters when they are changed.

HMM Parameter Model Update

Application Change Sensitivity

This option appears when you enable Dynamically update when parameters change.

The system uses boxplots to determine whether a parameter has changed. The boxplot displays the probability distribution of the parameter value. During sample collection period, the system generates 2 or 4 boxplots. After anomaly detection model is built, the system will keep on generating new boxplots to display the probability distribution of the new inputs. If the probability distribution area of the newly generated boxplot doesn't overlap with any one of the sample boxplots, the system determines this parameter has changed.

For more information on boxplots, see Probability Boxplots.

Depending on the Application Change Sensitivity level, the system triggers model update when it observes different extent of overlapping area.

  • Low—The system triggers model update only when the entire data distribution area (from the maximum value to the minimum value, that is, the entire area containing all the data) of the new boxplot doesn't have any overlapping part with that of the sample boxplots.
  • Medium—The system triggers model update if the notch area (the median rectangular area in the boxplot where most of the data is located) of the new boxplot doesn't have any overlapping part with the entire data distribution areas of the sample boxplots.
  • High—The system triggers model update as long as the notch area of the new boxplot doesn't have any overlapping part with that of the sample boxplots.
Update parameter model when number of boxplots do not overlap

This option appears when you enable Dynamically update when parameters change.

The default value is 2, which means if 2 newly generated boxplots don't overlap with any one of the sample boxplots, FortiWeb automatically updates the anomaly detection model.

You can set a value from 1 to 2.

Anomaly Detection Settings
Strictness Level for Anomaly

The value of the strictness level ranges from 1 to 10.

The system uses the following formula to calculate whether a sample is an anomaly:

The probability of the anomaly > μ + the strictness level * σ

If the probability of the sample is larger than the value of "μ + the strictness level * σ", this sample will be identified as anomaly.

μ and σ are calculated based on the probabilities of all the samples collected during the sample collection period, where μ is the average value of all the parameters' probabilities, σ is the standard deviation. They are fixed values. So, the value of "μ + the strictness level * σ" varies with the strictness level you set. The smaller the value of the strictness level is, the more strict the anomaly detection model will be.

This options set a global value for all the parameters. If you want to adjust the strictness level for a specific parameter, See Manage anomaly-detecting settings.

Threat Model
View Threat Models

The system scans anomalies to verify whether they are attacks. It provides a method to check whether an anomaly is a real attack by the trained Support Vector Machine Model.

Click the View Threat Models link to enable or disable threat models for different types of threats such as cross-site scripting, SQL injection and code injection. Currently, seven trained Support Vector Machine Model are provided for seven attack types.

Action Settings
Action

All requests are scanned first by HMM and then by Threat model.

Double click the cells in the Action Settings table to choose the action FortiWeb takes when attack is verified for each of the following situations:

  • Alert—Accepts the connection and generates an alert email and/or log message.
  • Alert & Deny—Blocks the request (or resets the connection) and generates an alert and/or log message.
  • Period Block—Blocks the request for a certain period of time.
Block Period

Enter the number of seconds that you want to block the requests. The valid range is 1–3,600 seconds (1 hour).

This option only takes effect when you choose Period Block in Action.

Severity

Select the severity level for this anomaly type. The severity level will be displayed in the alert email and/or log message.

Trigger Action

Select a trigger policy that you have set in Log&Report > Log Policy > Trigger Policy. If potential or definite anomaly or HTTP Method Violation is detected, it will trigger the system to send email and/or log messages according to the trigger policy.

URL Replacer Policy

Select the name of the URL Replacer Policy that you have created in Machine Learning Templates.

If web applications have dynamic URLs or unusual parameter styles, you must adapt URL Replacer Policy to recognize them.

If you have not created an URL Replacer Policy yet, you can leave this option empty for now, and then edit this policy later when the URL Replacer Policy is created. For more information on URL Replacer Policy, see Configure a URL replacer rule

Allow sample collection for domains

Add domains in this table so that the system will collect samples and generate anomaly detection models for these domains.

Here's what you can do:

  • Click a domain or click the (View Domain) button in the Action column to view anomaly detection reports for that specific domain. See Viewing domain data
  • Click the (Refresh) button in the Action column to refresh the corresponding domain. Note:Refreshing deletes all existing learning results.
  • Click the (Export) button in the Action column to export the anomaly detection data of this domain.
  • Click Create New to add more domains to let FortiWeb perform sample collection and intrusion detection on those domains. You can use wildcard * to represent multiple domains. Refer to Maximum number of ADOMs, policies, & server pools per appliance for the maximum domain number supported by the Machine Learning feature for your FortiWeb Model.
  • Click Delete to remove the selected domain(s). Note: This will remove all machine-learning results related to those domain(s) as well.
  • Click Import to import the anomaly detection data from your local directory to FortiWeb

IP List Type and Source IP list

Add IP ranges in the Source IP list, then select Trust or Black to allow or disallow collecting traffic data samples from these IP addresses.

  • Trust: The system will collect samples only from the IP ranges in the Source IP list.
  • Black: The system will collect sample from any IP addresses except the ones in the Source IP list.

Whether selecting Trust or Black, if you leave the Source IP list blank, the system will collect traffic data samples from any IP addresses. The maximum number of samples collected from each random IP address is 30. You can change the maximum value through CLI command waf machine-learning-policy.

If you select Trust, then add IP ranges in the Source IP list, the sample collection limit will not take effect, which means FortiWeb will collect traffic data samples only from the specified IP ranges and will not limit the number of samples.

Configuring anomaly detection policy

Anomaly detection policies are part of a server policy. They are created on the Policy > Sever Policy page. All anomaly detection policies that you create will show up on the Machine Learning > Anomaly Detection page, where you can configure or edit them to your preference.

To configure an anomaly detection policy:

  1. Click Machine Learning > Anomaly Detection .
  2. Double-click the server policy that contains the desired anomaly detection policy (or highlight it and then click the Edit button on top of the page) to open it. The Edit Anomaly Detection Configuration page opens, which breaks down anomaly detection policy into several sections, each of which has various parameters you can use to configure the policy.
  3. Follow the instructions in the following subsections to configure an anomaly detection policy.
  4. Click OK when done.
Sections & Parameters Function
Learning Cycle

Sample Collection mode

Sample Collection Period

When a sample is collected, the system generalized it into a pattern. For example, “abcd_123@abc.com” and “abcdefgecdf_12345678@efg.com” will both be generalized to the pattern “A_N@A.A”. The anomaly detection model is built based on the patterns, not the raw samples.

Extended: In Extended mode, it's required to also set the learning time. In extended mode at least 2500 samples will be collected and the sample collection period lasts for the specified weeks. For example, if you choose extended mode and set 1 week, the system stops collecting samples after 1 week if at least 2500 samples are collected by then, or continues collecting samples after 1 week until 2500 samples are collected.

Normal:In Normal mode, the system builds an initial model when the sample count reaches 400.

The system runs the initial model to detect anomalies, while it keeps collecting more samples to refine it.
Once the number of samples accumulates to 1200, the system will evaluate whether the patterns vary largely since the initial model is built.

  • If there are very few patterns generalized, it indicates the patterns are stable. The system will switch the initial model to a standard model.
  • If a lot of new patterns keeps coming in, the system will continue collecting more samples to cover as much patterns as possible. It won't switch to standard model until the patterns become stable.

The above mentioned numbers 400 and 1200 are configurable through CLI. Refer to config waf machine-learning-policy in FortiWeb CLI Reference.

Whether in extended or normal mode, the system keeps refining the model even after it's in running status.

Dynamically update when parameters change

Applications change frequently as new URLs are added and existing parameters provide new functions. This means the mathematical model of the same parameter might be different from what FortiWeb originally observed during the collection phase. In this case, FortiWeb needs to rebuild the mathematical model for the parameter.

Enable this option to automatically update the mathematical models of the parameters when they are changed.

HMM Parameter Model Update

Application Change Sensitivity

This option appears when you enable Dynamically update when parameters change.

The system uses boxplots to determine whether a parameter has changed. The boxplot displays the probability distribution of the parameter value. During sample collection period, the system generates 2 or 4 boxplots. After anomaly detection model is built, the system will keep on generating new boxplots to display the probability distribution of the new inputs. If the probability distribution area of the newly generated boxplot doesn't overlap with any one of the sample boxplots, the system determines this parameter has changed.

For more information on boxplots, see Probability Boxplots.

Depending on the Application Change Sensitivity level, the system triggers model update when it observes different extent of overlapping area.

  • Low—The system triggers model update only when the entire data distribution area (from the maximum value to the minimum value, that is, the entire area containing all the data) of the new boxplot doesn't have any overlapping part with that of the sample boxplots.
  • Medium—The system triggers model update if the notch area (the median rectangular area in the boxplot where most of the data is located) of the new boxplot doesn't have any overlapping part with the entire data distribution areas of the sample boxplots.
  • High—The system triggers model update as long as the notch area of the new boxplot doesn't have any overlapping part with that of the sample boxplots.
Update parameter model when number of boxplots do not overlap

This option appears when you enable Dynamically update when parameters change.

The default value is 2, which means if 2 newly generated boxplots don't overlap with any one of the sample boxplots, FortiWeb automatically updates the anomaly detection model.

You can set a value from 1 to 2.

Anomaly Detection Settings
Strictness Level for Anomaly

The value of the strictness level ranges from 1 to 10.

The system uses the following formula to calculate whether a sample is an anomaly:

The probability of the anomaly > μ + the strictness level * σ

If the probability of the sample is larger than the value of "μ + the strictness level * σ", this sample will be identified as anomaly.

μ and σ are calculated based on the probabilities of all the samples collected during the sample collection period, where μ is the average value of all the parameters' probabilities, σ is the standard deviation. They are fixed values. So, the value of "μ + the strictness level * σ" varies with the strictness level you set. The smaller the value of the strictness level is, the more strict the anomaly detection model will be.

This options set a global value for all the parameters. If you want to adjust the strictness level for a specific parameter, See Manage anomaly-detecting settings.

Threat Model
View Threat Models

The system scans anomalies to verify whether they are attacks. It provides a method to check whether an anomaly is a real attack by the trained Support Vector Machine Model.

Click the View Threat Models link to enable or disable threat models for different types of threats such as cross-site scripting, SQL injection and code injection. Currently, seven trained Support Vector Machine Model are provided for seven attack types.

Action Settings
Action

All requests are scanned first by HMM and then by Threat model.

Double click the cells in the Action Settings table to choose the action FortiWeb takes when attack is verified for each of the following situations:

  • Alert—Accepts the connection and generates an alert email and/or log message.
  • Alert & Deny—Blocks the request (or resets the connection) and generates an alert and/or log message.
  • Period Block—Blocks the request for a certain period of time.
Block Period

Enter the number of seconds that you want to block the requests. The valid range is 1–3,600 seconds (1 hour).

This option only takes effect when you choose Period Block in Action.

Severity

Select the severity level for this anomaly type. The severity level will be displayed in the alert email and/or log message.

Trigger Action

Select a trigger policy that you have set in Log&Report > Log Policy > Trigger Policy. If potential or definite anomaly or HTTP Method Violation is detected, it will trigger the system to send email and/or log messages according to the trigger policy.

URL Replacer Policy

Select the name of the URL Replacer Policy that you have created in Machine Learning Templates.

If web applications have dynamic URLs or unusual parameter styles, you must adapt URL Replacer Policy to recognize them.

If you have not created an URL Replacer Policy yet, you can leave this option empty for now, and then edit this policy later when the URL Replacer Policy is created. For more information on URL Replacer Policy, see Configure a URL replacer rule

Allow sample collection for domains

Add domains in this table so that the system will collect samples and generate anomaly detection models for these domains.

Here's what you can do:

  • Click a domain or click the (View Domain) button in the Action column to view anomaly detection reports for that specific domain. See Viewing domain data
  • Click the (Refresh) button in the Action column to refresh the corresponding domain. Note:Refreshing deletes all existing learning results.
  • Click the (Export) button in the Action column to export the anomaly detection data of this domain.
  • Click Create New to add more domains to let FortiWeb perform sample collection and intrusion detection on those domains. You can use wildcard * to represent multiple domains. Refer to Maximum number of ADOMs, policies, & server pools per appliance for the maximum domain number supported by the Machine Learning feature for your FortiWeb Model.
  • Click Delete to remove the selected domain(s). Note: This will remove all machine-learning results related to those domain(s) as well.
  • Click Import to import the anomaly detection data from your local directory to FortiWeb

IP List Type and Source IP list

Add IP ranges in the Source IP list, then select Trust or Black to allow or disallow collecting traffic data samples from these IP addresses.

  • Trust: The system will collect samples only from the IP ranges in the Source IP list.
  • Black: The system will collect sample from any IP addresses except the ones in the Source IP list.

Whether selecting Trust or Black, if you leave the Source IP list blank, the system will collect traffic data samples from any IP addresses. The maximum number of samples collected from each random IP address is 30. You can change the maximum value through CLI command waf machine-learning-policy.

If you select Trust, then add IP ranges in the Source IP list, the sample collection limit will not take effect, which means FortiWeb will collect traffic data samples only from the specified IP ranges and will not limit the number of samples.