Importing Malware URLs

This section describes how to import Malware URL information into FortiSIEM from external threat feed websites.

Prerequisites

Before proceeding, gather the following information about a threat feed web site:

  • Website URL.
  • Credentials required to access the website (optional).
  • If the website is not supported by FortiSIEM, you may need to understand the format of the data returned by the URL.
    • If the data is in comma-separated value (CSV) format, then a simple integration is possible. Note that the separator need not be a comma but could be any separator.
    • If the data is any other format, for example, XML, then some code must be written for integration using the framework provided by FortiSIEM.

Threat Feed Websites with Built-in Support

The following websites are supported:

To import data from these websites, follow these steps:

  1. In the RESOURCES > Malware URLs, find the website you must import data from.  
  2. Select the folder.
  3. Click More > Update.
  4. Select Update via API. The link will show in the edit box.
  5. Enter a Schedule by clicking the + icon.
  6. Enter the schedule parameters: when to start and how often to import. FortiSIEM recommends no more frequent than hourly.

Custom Threat Feed Websites - CSV Data - One-time Manual Import

This requires that the data to be imported is already in a file in comma-separated value (CSV) format.

Requirements for Importing

  1. The CSV file columns must be in the following order:
    URL, Malware Type, Confidence, Description, Last Seen (MM/DD/YYYY)
    If the fields are not in this order, then the whole file will not be imported.
  2. The URL field is required and must be unique.
    Example: www.0800thissite.ru/zone/freebee.php
  1. Select RESOURCES > Malware URLs.
  2. Click the + button on the left navigation tree to open the Create New Malware URL Group dialog.
  3. Enter Group and add Description. Click Save to create the folder under Malware URLs.
  4. Select the folder just created.
  5. Select Import from a CSV file.
  6. Click Choose File; enter the file name and click Upload.

    The imported data will show on the right pane.

Custom Threat Feed Websites - CSV Data - Programmatic Import via Java

Requirements for Importing

  1. The Web Site Data requires the following:
    1. A file in comma-separated value format (separator can be any special character such as space, tab, hash, dollar etc.).
    2. An individual entry is in one line.
  2. The URL field is required and must be unique.
    Example: www.0800thissite.ru/zone/freebee.php

Follow these steps:

  1. Select RESOURCES > Malware URLs.
  2. Click the + button on the left navigation tree to open the Create New Malware URL Group dialog box.
  3. Enter Group and add Description. Click Save to create the folder under Malware URLs.
  4. Select the folder just created.
  5. Select More > Update > Update via API.
  6. Click the edit icon next to URL and:
    1. Enter the URL of the website.
    2. Enter User Name and Password (optional).
    3. For Plugin Type, select Java.
    4. For Plugin Class, the default class com.accelops.service.threatfeed.impl.ThreatstreamMalwareUrlUpdateService is shown. Do not modify this value for this case.
    5. Enter the correct Field Separator (by default it is a comma).
    6. Set Data Format to CSV.
    7. Select Data Update as Full to overwrite the existing data or Incremental to retain the existing data.
    8. Enter the Data Mapping by choosing the mapped field and the corresponding position in the website data. For example if the URL is in third position, then choose 3 in the Position column.
    9. Click Save.
  7. Select an import schedule by clicking + on the Schedule. Select when to start the import and how often to import new data from the website.
    The imported data will show on the right pane.

Custom Threat Feed Websites - Non-CSV Data - Programmatic Import via Java

This is the most general case where the website data format is not CSV. In this case, write a Java plugin by modifying the default class provided by the system.

After the class has been written and fully tested for correctness, follow these steps:

  1. Select RESOURCES > Malware URLs.
  2. Click the + button on the left navigation tree to open the Create New Malware URL Group dialog.
  3. Enter the Group name and add a Description. Click Save to create the folder under Malware URLs.
  4. Select the folder just created.
  5. Select More > Update > Update via API.
  6. Click the edit icon and:
    1. Enter the URL of the website.
    2. Enter User Name and Password (optional).
    3. For Plugin Type, select Java.
    4. For Plugin Class, enter the name of the custom Java plugin class.
    5. Select Custom as the Data Format.
    6. Select Data Update as Full to overwrite the existing data or Incremental to retain the existing data.
    7. Click Save.
  7. Select an import schedule by clicking + on the Schedule. Select when to start the import and how often to import new data from the website.
    The imported data will show on the right pane.

Custom Threat Feed Websites - STIX Formatted Data and TAXII Import via Java

In this case, the threat feed data is available formatted as STIX and follows the TAXII protocol.

  1. Select RESOURCES > Malware URLs.
  2. Click the + button on the left navigation tree to open the Create New Malware URL Group dialog box.
  3. Enter Group and add Description. Click Save to create the folder under Malware URLs.
  4. Select the folder just created.
  5. Select More > Update > Update via API.
  6. Click the edit icon and:
    1. Enter the URL of the website.
    2. Enter User Name and Password (optional).
    3. Do not edit the name of the Plugin Class.
    4. For Plugin Type, select Java.
    5. Select STIX-TAXII as the Data Format.
    6. Enter the name of the STIX-TAXII Collection.
    7. Select Full as the Data Update value. Existing data will be overwritten.
    8. Click Save.
  7. Select an import schedule by clicking + on the Schedule. Select when to start the import and how often to import new data from the website.
    The imported data will display on the right pane.

Custom Threat Feed Websites - Programmatic Import via Python

In this case, the threat feed data is available through Python integration.

  1. Select RESOURCES > Malware URLs.
  2. Click the + button on the left navigation tree to open the Create New Malware URL Group dialog box.
  3. Enter Group and add Description. Click Save to create the folder under Malware URLs.
  4. Select the folder just created.
  5. Select More > Update > Update via API.
  6. Click the edit icon next to URL and provide the following information:
    1. In the URL field, enter the URL of the website.

      Note: Include the "http://" or "https://" prefix.
    2. (optional) In the User Name field, enter the username used by the API.
    3. (optional) In the Password field, enter the password related to the username.
    4. For Plugin Type, select Python.
    5. Check the SSL Verify checkbox to include a check to the authenticity and validity of an SSL certificate.
    6. From the Plugin Name drop-down list, select the python script to use. Python scripts located under
      /opt/phoenix/data-definition/threatfeedIntegrations/
      will be available.
      Note: For more information on creating/using a Python script, see Appendix: Python Threat Feed Framework.

    7. For Data Update, select Full (Completely replace all data) or Incremental (add on to existing data).
  7. Click Save.
  8. Select an import schedule by clicking + on the Schedule. Select when to start the import and how often to import to get new data from the website.
    The imported data will show on the right pane after some time.