Importing/Updating User-defined Malware Hash

System defined groups are updated by its own service:

• Threat Stream Malware Hash
• FortiSandbox Malware Hash

You can update the Malware Hash using the following options:

• Import from a CSV File
• Update via API

Prerequisites:

Before proceeding, gather the following information about a threat feed web site.

• Website URL.
• Credentials required to access the website (optional).
• If the website is not supported by FortiSIEM, you must understand the format of the data returned by the URL.
  • If the data is in the comma-separated value format (the separator need not be a comma but could be any separator, then a simple integration is possible.)
  • If the data is any other format, for example, XML, then some code must be written for integration using the framework provided by FortiSIEM.

Import from a CSV File

Custom Websites - CSV Data - One-time Manual Import

Instead of manually adding Malware Hashes to a group individually, you can upload a CSV file with multiple entries. This requires that the data to be imported is already in a file in a comma-separated value (CSV) format.

Requirements for Importing

1. The CSV file columns must be in the following order:
   

   Botnet Name, Algorithm, Hash Code, Controller IP, Malware Type, Confidence, Severity, Asn, Origin, Country, Description, Date Found(MM/DD/YYYY), Last Seen(MM/DD/YYYY)

   If the fields are not in this order, then the whole file will not be imported.

2. Botnet Name, Algorithm, and Hash Code are required fields. The 3 required fields are linked, so any changes to the fields within each row will produce a unique entry.
   

   Examples:

   newbotnet, md5, 4da0bb01a96e70ce43ece0147f8438d1
   

   newbotnet1,sha256,8e53e4d236c99ba4dcf7c4cbf1cea93d023391ea
   

1. Go to RESOURCES > Malware Hash.
2. Select the group from the left panel or create a new group by clicking the + icon above the list of RESOURCES groups.
3. Select More > Update.
4. Select Import from a CSV file and choose the file to import.
5. Click Import.

Update via API

This section describes how to import Malware Hash information into FortiSIEM from external threat feed websites. Malware Hashes are used by malware to hide their own identity.

• Updating System Defined Malware Hash Group
• Custom Threat Feed Websites - CSV Data - Programmatic Import
• Custom Threat Feed Websites - Non-CSV Data - Programmatic Import via Java
• Custom Threat Feed Websites - Non-CSV Data -STIX Formatted Data and TAXII Import via Java
• Custom Threat Feed Websites - Programmatic Import via Python

Updating System Defined Malware Hash Group

The following websites are supported:

• Threat Stream Open Proxy (https://api.threatstream.com)
• Threat Stream TOR Node (https://api.threatstream.com)

Complete these steps to import data from these websites:

1. Go to RESOURCES > Malware Hash.
2. Select the folder and find the website you want to import data from.
3. Click More > Update.
4. Select Update via API.
   The link will be displayed in the URL field or else manually enter the URL and details.
5. Enter a Schedule by clicking the + icon.
6. Enter the schedule parameters - Start Time and Recurrence Pattern. FortiSIEM recommends no more frequent than hourly.
7. Click Save.
   You can use the edit icon to modify or delete icon to remove a Schedule.

Custom Threat Feed Websites - CSV Data - Programmatic Import via Java

Requirements for Importing

1. The Web Site Data requires the following:
   1. A file in comma-separated value format (separator can be any special character such as space, tab, hash, dollar etc.).
   2. An individual entry is in one line.
2. The CSV file columns must be in the following order:
   

   Botnet Name, Algorithm, Hash Code, Controller IP, Malware Type, Confidence, Severity, Asn, Origin, Country, Description, Date Found (MM/DD/YYYY), Last Seen (MM/DD/YYYY)

   If the fields are not in this order, then the whole file will not be imported.
   

3. The Botnet Name field is required and must be unique.

1. Go to RESOURCES > Malware Hash.
2. Select the folder or click + to add a new group under Malware Hash folder.
3. Click More > Update.
4. Select Update via API. The link will be displayed in the URL field or else manually enter the URL and details.
5. Click the edit icon near URL.
6. Enter the following information:
   1. In the URL field, enter the URL of the website.
      Note: Ensure you have an "http://" or "https://" prefix.
      
   2. (Optional) In the User Name field, enter the username associated with the API.
   3. (Optional) In the Password field, enter the password associated with the username.
   4. For Plugin Type, select Java.
   5. For Plugin Class, the default class 'com.accelops.service.threatfeed.impl.ThreatFeedWithMappingPolicyService' is shown. Do not modify this in any case.
   6. Enter the correct Field Separator (by default it is a comma).
   7. Select CSV as the Data Format.
   8. Select Data Update as Full.
   9. Enter the Data Mapping by choosing the mapped field and the corresponding position in the website data. For example, if the Botnet Name is in third position, choose 3 in the Position column. Click + if you must add more rows.
   10. Click Save.
7. Select an import schedule by clicking + on the Schedule. Select when to start the import and how often to import new data from the website.
   The imported data will show on the right pane after some time.

Custom Threat Feed Websites - Non-CSV Data - Programmatic Import via Java

This is the most general case where the website data format does not satisfy the previous conditions. In this case, write a Java plugin class by modifying the default system provided one.

1. Go to RESOURCES > Malware Hash.
2. Select the folder or click + to add a new group under Malware Hash folder.
3. Click More > Update.
4. Select Update via API.
5. Click the edit icon near URL.
6. Enter the following information:
7. Enter the URL of the website.
   1. Enter User Name and Password (optional).
   2. For Plugin Type, select Java.
   3. For Plugin Class, the custom Java class in this case.
   4. Select Custom as the Data Format.
      • Enter the Data Mapping by choosing the mapped field and the corresponding position in the website data. For example, if the IP is in third position, choose 3 in the Position column. Click + if you must add more rows.
      • Select Full as the Data Update value. Existing data will be overwritten. Select Incremental to preserve the existing data.

      For STIX-TAXII:

      • Enter the name of the STIX-TAXII Collection.
      • Select Full as the Data Update value. Existing data will be overwritten.
   5. Click Save.
8. Select an import schedule by clicking + on the Schedule. Select when to start the import and how often to import new data from the website.
   The imported data will display in the table after some time.

Custom Threat Feed Websites - Non-CSV Data -STIX Formatted Data and TAXII Import via Java

In this case, the threat feed data is available formatted as STIX and follows the TAXII protocol.

1. Go to RESOURCES > Malware Hash.
2. Select the folder or click + to add a new group under Malware Hash folder.
3. Click More > Update.
4. Select Update via API.
5. Click the edit icon near URL.
6. Enter the following information:
   1. Enter the URL of the website.
   2. Enter User Name and Password (optional).
   3. For Plugin Type, select Java.
   4. For Plugin Class, the custom Java class in this case.
   5. Enter the name of the STIX-TAXII Collection.
   6. Select STIX-TAXII as the Data Format.
   7. Select Data Update as Full.
   8. Click Save.
7. Select an import schedule by clicking + on the Schedule. Select when to start the import and how often to import new data from the website.
   The imported data will display in the table after some time.

Custom Threat Feed Websites - Programmatic Import via Python

In this case, the threat feed data is available via Python integration.

1. Go to RESOURCES > Malware Hash.
2. Select the folder or click + to add a new group under Malware Hash folder.
3. Click More > Update.
4. Select Update via API.
5. Click the edit icon next to URL and provide the following information:
   1. In the URL field, enter the URL of the website.
      Note: Include the "http://" or "https://" prefix.
   2. (optional) In the User Name field, enter the username used by the API.
   3. (optional) In the Password field, enter the password related to the username.
   4. For Plugin Type, select Python.
   5. Check the SSL Verify checkbox to include a check to the authenticity and validity of an SSL certificate.
      
   6. From the Plugin Name drop-down list, select the python script to use. Python scripts located under
      /opt/phoenix/data-definition/threatfeedIntegrations/
      will be available.
      Note: For more information on creating/using a Python script, see Appendix: Python Threat Feed Framework.
      
   7. For Data Update, select Full (Completely replace all data) or Incremental (add on to existing data).
6. Click Save.
7. Select an import schedule by clicking + on the Schedule. Select when to start the import and how often to import to get new data from the website.
   The imported data will show on the right pane after some time.