Static URL filter

This topic gives examples of the following advanced filter features:

• Block invalid URLs

• URL filter

• Block malicious URLs discovered by FortiSandbox

• Web content filter

Block invalid URLs

Use this setting to block websites when their SSL certificate CN field does not contain a valid domain name.

This option also blocks URLs that contains spaces. If there is a space in the URL, it must be written as %20 in the URL path.

To block invalid URLs in the GUI:

1. Go to Security Profiles > Web Filter and click Create New, or edit an existing profile.

2. In the Static URL Filter section, enable Block invalid URLs .

   

3. Click OK.

To block invalid URLs in the CLI:

config webfilter profile
    edit "webfilter"
        set options block-invalid-url
    next
end

URL filter

The URL filter uses specific URLs with patterns containing text and regular expressions so the FortiGate can process the traffic based on the filter action (exempt, block, allow, monitor) and web pages that match the criteria. Once a URL filter is configured, it can be applied to a firewall policy.

The following filter types are available:

URL filter type	Description
Simple	The FortiGate tries to strictly match the full context. For example, if you enter www.facebook.com in the URL field, it only matches traffic with www.facebook.com. It won't match facebook.com or message.facebook.com. When the FortiGate finds a match, it performs the selected URL action.
Regular expression/ wildcard	The FortiGate tries to match the pattern based on the rules of regular expressions or wildcards. For example, if you enter *fa* in the URL field, it matches all the content that has fa such as www.facebook.com, message.facebook.com, fast.com, and so on. When the FortiGate finds a match, it performs the selected URL action.

For more information, see the URL Filter expressions technical tip in the Knowledge Base.

The following actions are available:

URL filter action	Description
Exempt	The traffic is allowed to bypass the remaining FortiGuard web filters, web content filters, web script filters, antivirus scanning, and DLP proxy operations.
Block	The FortiGate denies or blocks attempts to access any URL that matches the URL pattern. A replacement message is displayed.
Allow	The traffic is passed to the remaining FortiGuard web filters, web content filters, web script filters, antivirus proxy operations, and DLP proxy operations. If the URL does not appear in the URL list, the traffic is permitted.
Monitor	The traffic is processed the same way as the Allow action. For the Monitor action, a log message is generated each time a matching traffic pattern is established.

The exempt URL filter action can be configured to bypass all or certain scanning and filtering operations. This setting can only be configured in the CLI.

If the action is set to exempt, use set exempt to select the scanning and filtering operations that exempt URLs skip.

config webfilter urlfilter
    edit <id>
        config entries
            edit <id>
                set action exempt
                set exempt {av web-content activex-java-cookie dlp fortiguard range-block pass antiphish all}
            next
        end
    next
end

Option	Description
av	Antivirus scanning
web-content	Web filter content matching
activex-java-cookie	ActiveX, Java, and cookie filtering
dlp	DLP scanning
fortiguard	FortiGuard web filtering
range-block	Range block feature
pass	Pass single connection from all
antiphish	Antiphish credential checking
all	Exempt from all scanning and filtering operations listed above

These exempt options are not visible in the GUI. Setting the URL filter Action to Exempt will exempt URLs from all security profiles.

In the following example, a URL filter will be created to block the facebook.com URL using a wildcard.

To create a URL filter for Facebook in the GUI:

1. Go to Security Profiles > Web Filter and click Create New, or edit an existing profile.

2. In the Static URL Filter section, enable URL Filter.

3. Click Create New. The New URL Filter pane opens.

4. For URL, enter *facebook.com, for Type, select Wildcard, and for Action, select Block.

   

5. Click OK. The entry appears in the table.

   

6. Configure the other settings as needed.

7. Click OK.

To apply the web filter profile to a firewall policy in the GUI:

1. Go to Policy & Objects > Firewall Policy.

2. Edit a policy, or create a new one.

3. In the Security Profiles section, enable Web Filter and select the profile that you created.

4. Set SSL Inspection to certificate-inspection.

   

   The no-inspection profile does not perform SSL inspection, so it should not be selected with other UTM profiles.

5. Configure the other settings as needed.

6. Click OK.

To create a URL filter for Facebook in the CLI:

config webfilter urlfilter
    edit 1
        set name "webfilter"
        config entries
            edit 1
                set url "*facebook.com"
                set type wildcard
                set action block
            next
        end
    next
end

To apply the URL filter to a web filter profile in the CLI:

config webfilter profile
    edit "webfilter" 
        config web
            set urlfilter-table 1
        end
        config ftgd-wf
            ...
        end
    next
end

To apply the web filter profile to a firewall policy in the CLI:

config firewall policy
    edit 1
        set name "WF"
        set srcintf "wan2"
        set dstintf "wan1"
        set srcaddr "all"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
        set utm-status enable
        set inspection-mode proxy
        set logtraffic all
        set webfilter-profile "webfilter" 
        set ssl-ssh-profile "certificate-inspection"
        set nat enable
    next
end

Verify the URL filter results by going to a blocked website. For example, when you go to the Facebook website, the replacement message appears:

To customize the URL web page blocked message:

1. Go to System > Replacement Messages.

2. In the HTTP section, select URL Block Page and click Edit.

3. Edit the HTML to customize the message. See Replacement messages for more information.

To check web filter logs in the GUI:

1. Go to Log & Report > Security Events.

2. Click the Web Filter card name.

3. If there are a lot of log entries, click Add Filter and select Event Type > urlfilter to display logs generated by the URL filter.

   

To check web filter logs in the CLI:

# execute log filter category utm-webfilter			
# execute log display

2: date=2023-08-10 time=15:02:25 eventtime=1691704944982929658 tz="-0700" logid="0315012544" type="utm" subtype="webfilter" eventtype="urlfilter" level="warning" vd="root" urlfilteridx=1 urlfilterlist="webfilter" policyid=1 poluuid="4a4b9d00-e471-51ed-71ec-c1a3bc8f773c" policytype="policy" sessionid=4198 srcip=1.1.1.2 srcport=55044 srccountry="Australia" srcintf="internal7" srcintfrole="lan" srcuuid="45eec070-e471-51ed-4b1c-930f37c5d882" dstip=157.240.3.35 dstport=443 dstcountry="United States" dstintf="wan1" dstintfrole="wan" dstuuid="45eec070-e471-51ed-4b1c-930f37c5d882" proto=6 service="HTTPS" hostname="www.facebook.com" profile="webfilter" action="blocked" reqtype="direct" url="https://www.facebook.com/" sentbyte=812 rcvdbyte=0 direction="outgoing" urlsource="Local URLfilter Block" msg="URL was blocked because it is in the URL filter list" crscore=30 craction=8 crlevel="high"

Block malicious URLs discovered by FortiSandbox

This setting blocks malicious URLs that FortiSandbox finds. Your FortiGate must be connected to a registered FortiSandbox.

For information on configuring FortiSandbox, see Using FortiSandbox post-transfer scanning with antivirus and Using FortiSandbox inline scanning with antivirus.

To block malicious URLs discovered by FortiSandbox in the GUI:

1. Go to Security Profiles > Web Filter and click Create New, or edit an existing profile.

2. In the Static URL Filter section, enable Block malicious URLs discovered by FortiSandbox.

   

3. Click OK.

To block malicious URLs discovered by FortiSandbox in the CLI:

config webfilter profile
    edit "webfilter"
        config web
            set blocklist enable
        end
    next
end

Web content filter

You can control access to web content by blocking webpages containing specific words or patterns. This helps to prevent access to pages with questionable material. You can specify words, phrases, patterns, wildcards, and regular expressions to match content on webpages. You can use multiple web content filter lists and select the best one for each web filter profile.

The maximum number of web content patterns in a list depends on the model of the device. To find the maximum number of web content patterns allowed for a device, go to the Maximum Values Table (https://docs.fortinet.com/max-value-table). Select the software version and models, and click Go. Maximum values are displayed. In the Search box, enter webfilter.content:entries to find the maximum number.

When configuring a web content filter list, the following patterns are available:

Web content pattern type	Description
Wildcard	Use this setting to block or exempt one word or text strings of up to 80 characters. You can also use wildcard symbols such as ? or * to represent one or more characters. For example, a wildcard expression forti*.com matches fortinet.com and fortiguard.com. The * represents any character appearing any number of times.
Regular expression	Use this setting to block or exempt patterns of regular expressions that use some of the same symbols as wildcard expressions, but for different purposes. In regular expressions, * represents the character before the symbol. For example, forti*.com matches fortiii.com but not fortinet.com or fortiice.com. In this case, the symbol * represents i appearing any number of times.

The web content filter scans the content of every webpage that is accepted by a firewall policy. The system administrator can specify banned words and phrases and attach a numerical value (or score) to the importance of those words and phrases. When the web content filter scan detects banned content, it adds the scores of banned words and phrases found on that page. If the sum is higher than a threshold set in the web filter profile, the FortiGate blocks the page.

The default score for web content filter is 10 and the default threshold is 10. This means that by default, a webpage is blocked by a single match. These settings can only be configured in the CLI.

Banned words or phrases are evaluated according to the following rules:

• The score for each word or phrase is counted only once, even if that word or phrase appears many times in the webpage.
• The score for any word in a phrase without quotation marks is counted.
• The score for a phrase in quotation marks is counted only if it appears exactly as written.

The following table is an example of how rules are applied to the webpage contents . For example, a webpage contains only this sentence:

The score for each word or phrase is counted only once, even if that word or phrase appears many times in the webpage.

Banned pattern	Assigned score	Score added to the sum for the entire page	Threshold score	Comment
word	20	20	20	Appears twice but is only counted once. The webpage is blocked.
word phrase	20	40	20	Each word appears twice but is only counted once, giving a total score of 40. The webpage is blocked.
word sentence	20	20	20	word appears twice and sentence does not appear, but since any word in a phrase without quotation marks is counted, the score for this pattern is 20. The webpage is blocked.
"word sentence"	20	0	20	This phrase does not appear exactly as written. The webpage is allowed.
"word or phrase"	20	20	20	This phrase appears twice but is only counted once. The webpage is blocked.

To configure a web content filter in the GUI:

1. Go to Security Profiles > Web Filter and click Create New, or edit an existing profile.

2. In the Static URL Filter section, enable Content Filter.

3. In the table, click Create New. The New Web Content Filter pane opens.

4. Configure the following settings:

   Pattern Type	Regular Expression
   Pattern	fortinet
   Language	Western
   Action	Block
   Status	Enable

   

5. Click OK. The entry appears in the table.

   

6. Configure the other settings as needed.

7. Click OK.

To configure a web content filter in the CLI:

1. Create the content (banned word) table:

   config webfilter content
      edit 1
         set name "webfilter"
         config entries
            edit "fortinet"
              set pattern-type regexp
              set status enable
              set lang western
              set score 10
              set action block
            next
         end
      next
   end

2. Apply the content table to the web filter profile:

   config webfilter profile
      edit "webfilter"
         config web
            set bword-threshold 10
            set bword-table 1
         end
         config ftgd-wf
            unset options
         end
      next
   end

To verify the content filter:

1. Go to a website with the word fortinet, such as www.fortinet.com.

   The website is blocked and a replacement page displays: