Home

> Key Concepts

Key Concepts

This section describes several key concepts used in FortiSIEM.

Clustering Architecture

FortiSIEM scales seamlessly from small enterprises to large and geographically distributed enterprises and service providers.

  • For smaller deployments, FortiSIEM can be deployed a single all-in-one hardware or virtual appliance that contains full functionality of the product.
  • For larger environments that need greater event handling throughput, FortiSIEM can be deployed in a cluster of Supervisor and Worker Virtual Appliances.

There are three types of FortiSIEM nodes – Collector, Worker and Supervisor. Collectors are used to scale data collection from various geographically separated network environments potentially behind firewalls. Collectors communicate to the devices, collect, parse and compress the data and then send to the Worker nodes over a secure HTTP(S) channel in a load balanced manner. Supervisor and Worker nodes reside inside the data center and perform data analysis functions using distributed co-operative algorithms.

There are five primary data analysis tasks:

  1. Indexing the data and storing in an event database
  2. Searching the data
  3. Correlating the data in a streaming mode to trigger rules (behavioral anomalies)
  4. Creating a user identity and location database for adding context to data
  5. Creating baselines for anomaly detection

For scalability, each of these tasks is divided into a heavyweight Worker component executed by the Worker nodes and a lightweight Master component executed by the Supervisor node. The Supervisor nodes also the GUI using a self-contained three-tier model – GUI, Application Server containing the business logic and a relational database for holding the FortiSIEM application state.

For scalable event storage, FortiSIEM provides three options:

  • Local disk
  • FortiSIEM NoSQL event database with data residing on an NFS Server
  • Elasticsearch distributed database

Hardware appliance and All-in-one virtual appliance solutions use the Local disk option while the NoSQL or Elasticsearch options can be exploited by a FortiSIEM cluster of Supervisor and Workers.

The NoSQL event database option is a purpose built FortiSIEM proprietary solution. The Supervisor and Worker nodes create and maintain the database indices. To scale event insertion and search performance in this mode requires (a) a fast communication network between the Supervisor/Worker nodes and the NFS Server and (b) high NFS IOPS that can be achieved using fast RAID disk or tiered SSD and magnetic disks.

Elasticsearch provides a true distributed, redundant columnar database option for scale-out database performance at the expense of higher storage needs. In this option, FortiSIEM Worker nodes push the data in real time to Elasticsearch cluster, which maintains the event database. FortiSIEM has developed an intermediate adaptation layer, so that the same GUI can run seamlessly on both Elasticsearch and FortiSIEM NoSQL event database.

Licensing

FortiSIEM is licensed based on the following:

  • Number of devices FortiSIEM monitors or receives logs from
  • Number of Windows Agents
  • Aggregate Events per Second (EPS) it receives

Note that FortiSIEM licensing is not based on storage - you can store and query the data as needed for your compliance needs without any concern regarding license. The license parameters can be perpetual or subscription based. Maintenance and FortiGuard Threat Intelligence are subscription based.

You can have unlimited devices in CMDB. However, the total number of devices that send logs to FortiSIEM or is monitored by FortiSIEM cannot exceed the device license. The devices under license are called 'Managed' while the remaining devices are called 'Unmanaged'. If you do a discovery and the number of newly discovered devices combined with Managed CMDB devices exceed the license, then the extra devices are tagged in CMDB as 'Unmanaged'. You can either buy more device license or exchange an Unmanaged device with a Managed device.

FortiSIEM calculates Events per Second (EPS) over a 3-minute period as the total number of events received over a 3-minute period divided by 180. FortiSIEM is a distributed system where events can be received at any node - Collector, Worker or Supervisor. The EPS licensing is enforced as follows:

At the end of every 3-minute interval, Incoming EPS is calculated at each event entry node (Collector, Worker or Supervisor) and the value is sent to the central decision-making engine on the Supervisor node.

  1. The Supervisor node takes all the Incoming EPS values and based on the Licensed EPS, computes the Allocated EPS for the next 3-minute interval for every node and communicates those values to every node.
  2. For the next 3-minute interval, each node accepts up to (Allocated EPS * 180) events. It also reports Incoming EPS for the current interval to the Supervisor node.
  3. The continuous EPS reallocation process continues.

FortiSIEM includes some additional refinements to EPS enforcement as follows:

  • Each Collector has a Guaranteed EPS and Allocated EPS for this Collector is always greater than Allocated EPS.
  • FortiSIEM keeps track of Unused EPS as the sum of positive differences of Allocated EPS and Incoming EPS over all nodes. At the beginning of the day (12:00 am), Unused EPS is set to 50% of previous day’s Unused EPS and then Unused EPS accumulates throughout the day before maxing out at five times Licensed EPS. Unused EPS can be used for bursting during attacks or other event surge periods, beyond Licensed EPS.

Multi-tenancy and Organizations

Multi-tenancy enables you to manage multiple groups of devices (Organizations) within a single installation. FortiSIEM provides logical separation between Organizations at an application layer. The users of one Organization cannot see another Organization’s data including devices, users and logs.

You have to choose the Service Provider Installation type when you first install FortiSIEM. Organizations can be defined in two ways:

  • By adding a Collector to an Organization – all devices sending logs to a Collector or all devices monitored by a Collector are automatically assigned to the Organization. To which the Collector belongs. Device Names and IP Addresses can overlap between two Organizations. This situation can be used to model is Remote Managed Service Providers.
  • By assigning IP ranges to Organizations – there are no Collectors and devices will be discovered from Supervisor node and send logs to Supervisor or Worker nodes. If the IP addresses of ALL interfaces of a device are wholly included within the IP range for an Organization, then the device is assigned to that Organization. Else, the device is assigned to the Super/Local Organization (see below).

In addition to user-defined Organizations, FortiSIEM creates two Organizations for ease of management:

  • Super/Local Organization – this can be used to model Service Provider’s own network.
    • For Organizations with Collectors, if a device sends log directly to Supervisor or Worker nodes or is discovered from the Supervisor node, then it belongs to the Super/Local Organization
    • For Organizations without Collectors, if the all IP addresses of a device (being discovered or sending logs) are not wholly included within the IP range for any Organization, then that device is assigned to the Super/Local Organization.
  • Super/Global Organization – this is a virtual Organization that can 'see' all the other Organizations. This is useful for Service Provider administrative users.

FortiSIEM Multi-tenancy principles are as follows:

  1. Users belonging to Super/Global Organization can see other organizations and their data.
  2. Users belonging to Super/Local Organization and user-defined Organizations can only see its own Organization.
  3. Devices and events are automatically tagged by FortiSIEM with the Organization Id and Name.
  4. Rules can be written at a Global level or for a specific Organization. Incidents trigger when rule conditions are met and they trigger independently for each organization. Each Incident is labeled with Customer Id and Name.
  5. Searches/Reports can be executed from Super/Global Organization for any combinations of Organizations.
  6. From a specific user-defined Organization or Super/Local Organization, Searches/Reports can run on that Organization.
  7. Viewing Incidents is simply a specific Search and follows the same principles as specified in 5 and 6.

Role-based Access Control

After installation, FortiSIEM automatically creates an admin user with Full Admin rights for Super/Global and Super/Local Organization. When the user creates a new Organization, FortiSIEM creates an admin user for that Organization. These are accounts with Full Admin rights. FortiSIEM users with Full Admin rights can create Roles and then create users and assign them a role.

A FortiSIEM role is based on the following aspects:

  • What the user can see:
    • Restrict GUI visibility by hiding parts of the GUI
    • Restrict some Organizations for Service Provider installations
    • Restrict data by writing filters on device type, event type and any parsed event attribute
  • What the user can do:
    • Restrict or even hide Admin tab where most of the configuration takes place

    • Restrict any other GUI tab
    • Restrict write capability on certain parts of the GUI

FortiSIEM has a few built-in roles that the users can customize to meet their own needs.

Discovery and CMDB

Discovery is a key differentiator for FortiSIEM as it enables users to seamlessly discover their infrastructure (the 'truth') and auto-populate CMDB, which can then be used to facilitate analytics.

Discovery can be of two types:

  • Simple LOG discovery – FortiSIEM has mappings for device type to log parser for all its in-built log parsers. When it sees a log that matches a parser, it associates the corresponding device type to that device and creates a CMDB entry.
  • Detailed device discovery – LOG discovery is very basic since only the Vendor and Model can be guessed (for example: Cisco IOS, Fortinet FortiGate, Microsoft Windows, Generic Linux). It is not possible to deduce more details about the device, for example: Operating System version, hardware model, installed patches, installed software, running processes, network device configurations, interfaces, monitor-able performance metrics etc. In addition to discovering all of the above, FortiSIEM can also discover certain inter-device relationships, for example, Virtualization Guest to Host mappings, WLAN AP to Controller mappings, Multi-context device to physical device mappings, network topology etc. Devices in the AWS Cloud and MS Azure Cloud can be discovered as well.

Discovered information is to automatically populate a CMDB. As new devices get added or deleted from the infrastructure, scheduled re-discoveries can keep FortiSIEM CMDB up to date. User can also define some rules to map certain groups of devices to certain CMDB device groups.

The key advantages of FortiSIEM Discovery and CMDB are as follows:

  1. The customer has an accurate picture of the infrastructure and its relationships from a simple discovery. If a new rogue device is added to the network, FortiSIEM re-discovery learns immediately and could alert to this potential security issue. If an inadvertent configuration change to a key file is made, FortiSIEM re-discovery or configuration monitoring also detect and alert.
  2. Performance and availability monitoring is automated since FortiSIEM simply learns what can be monitored and starts monitoring immediately. This approach eliminates human errors.
  3. Certain key CMDB Objects such as Business Services can remain up to date against infrastructure changes as they can be auto-populated by discovery.
  4. CMDB Objects makes rules and reports easy to create. First, this makes rules and reports very simple to write without a long explicit list of IP addresses or host names. Second, the rules do not need to be rewritten as devices get added or deleted. Since CMDB Objects
  5. Discovery enables configuration change detection both day-to-day changes and changes to golden versions.

Windows and Linux Agents

Some logs and performance metrics can be collected remotely from Windows servers via WMI and by running Winexe command. Some key performance metrics and file monitoring on Linux servers can be done via SSH. However, the following limitations exist:

For Windows Servers:

  • Not all metrics can be collected from a FortiSIEM Linux platform via WMI (for example: Sysmon, Generic Event Logs in the Event Log navigation tree, Registry changes). WMI can be used to collect only Windows Event logs.
  • File Integrity Monitoring Data collected via Windows Security logs is very verbose (~8 logs per file operation) and creates unnecessary noise in FortiSIEM.
  • Remotely running some programs such Winexe starts services on the servers – this may trigger security alerts in certain environments.
  • A domain account is required to collect certain logs. The regular account does not provide all logs.
  • WMI Service often creates CPU load on the servers when a large number of logs are pulled via WMI.
  • Collecting logs via polling from thousands of servers is not efficient. If a server is not responsive or slow, you have to wait for the connection to timeout and this wastes resources.

Linux Servers send log via syslog. However, if you want to collect File Integrity Monitoring Data, then certain configuration is required to be done remotely.

Agents provide a clean and efficient way to collect exactly the data that we need. FortiSIEM Agents are very lightweight and do not consume more than 5% of system CPU and memory. FortiSIEM Windows Agents have the following functionality:

  • Collect any Windows Event log including Security, Application and Performance event logs, DHCP/DNS logs, Sysmon logs etc.
  • Collect Custom log files
  • Detect registry changes
  • Detect File read, write and edits (FIM) with added user context
  • Run any PowerShell command and send the output as logs – this allows users to capture any data at periodic intervals and send to FortiSIEM.
  • Detect removable media insertion, deletion, read and write

FortiSIEM Windows Agent Manager can manage a large number of FortiSIEM Windows Agents using configuration templates. The user needs to create a template and associate to many servers. Windows Agents can be configured to send logs to FortiSIEM collectors in a round robin fashion. If one collector is not available, then the Agent can send it to the next Collector in the list. This provides a robust and scalable way to collect logs from a large number of servers.

Linux Agents can be used to detect file reads, writes and edits (FIM functionality) with added user context.

Business Services

A Business Service provides a collection of devices and applications serving a common business purpose. You can define a Business Service in FortiSIEM either manually or by the Dynamic CMDB Group framework that adds it to the Business Service once a device matching certain criteria appears in CMDB.

The primary objective of a Business Service is to assist in incident triage. Once a Business Service is defined, every incident is tagged with the impacted Business Services. A Business Service dashboard provides a top-level Incident centric view of Business Services. The user can take care of incidents for critical Business services and ensure that they stay up.

Parsers and Monitors

The ability to parse any log to any depth is a key SIEM functionality. FortiSIEM comes inbuilt with over 2500 event attributes, 175,000 event types and 250 parsers for various device types and applications. In addition, it has a flexible GUI based framework for customers to enhance existing log parsers, and create completely new device types, event attributes, event types and log parsers. User can test parser changes on a live system and apply them to become effective immediately on all nodes – so changes take effect without any downtime. Parsers can also be exported out of one system and imported into another system. In Service Provider environments, a parser change can be created at a global level and deployed to all organizations.

FortiSIEM also comes in-built with a large number of performance monitors and configuration pulling scripts for device types and applications. Discovery automatically enables the applicable monitors and user can adjust some parameters such as polling intervals. Similar to log parsers, the user can create and test performance monitors on a live system and apply them to become effective immediately on all nodes – so changes take effect without any downtime. Performance Monitors can also be exported out of one system and imported into another system.

FortiSIEM keeps track of changes to installed software and network device configuration. If a new configuration file needs to be monitored and can be obtained via a script, then the user can add them to the system. FortiSIEM monitors changes from current to the previous version, deviation from a blessed file, changes between running config and startup config for certain devices.

Entity Risk Score

User Identity and Location

FortiSIEM creates an Audit trail of User Identity and Location data in real time by associating a network identity (for example: an IP address, or MAC address) to user identity (for example: a user name, computer name, or domain or Cloud logon) and tying that to a location (like a wired switch port, a wireless LAN controller, or VPN gateway or geo-location for VPN logins). The associations are generated by piecing together various pieces of information from Windows Active Directory events, DHCP events, WLAN and VPN logon events and various Cloud service logon events, with discovery results.

FortiSIEM Supervisor and Worker nodes collaborate in a distributed manner to create User Identity and Location records. The IdentityWorker module on Worker nodes keep a partial User Identity and Location in-memory database based on the events that they see. Whenever IdentityWorker module on specific Worker sees new information, for example: a new IP address to User association, then it updates the database and communicates to the IdentityMaster module on Supervisor node. The global User Identity and Location database is created by IdentityMaster module on Supervisor node by combining information from all IdentityWorker modules. Whenever the IdentityMaster module sees new information, it sends a signal to parser modules in all nodes, which then gets the latest updates from the Supervisor node. The parser module injects IP to User meta-data into events in real time so that this information can be searched without complicated database join operations.

Searches, Reports and Compliance

FortiSIEM provides a unified way to search the data it collects from various devices. All data whether it is system logs, performance metrics or configuration changes, is converted to an event with parsed event attributes – this makes it easy to search.

Search can be done for real-time data or historical data. In real time mode, the search occurs in a streaming node on incoming data without touching the event database. In historical mode, the user specifies a time period and data residing in event database is searched for that time period. Searches can be specified on raw logs or parsed attributes. A rich variety of grouping and aggregation constructs are supported to display data at various granularity.

FortiSIEM comes pre-built with a large number of reports that can be used as starting points. User can customize them and save as their own reports for later use. Reports can be scheduled to run at specified times and delivered in PDF or CSV format via email. FortiSIEM provides a large number of compliance reports, each with reference to specific compliance mandates. To run these reports, user simply needs to add devices to the specific compliance device group (Business Service) and then run the report.

All searches run in a distributed fashion in FortiSIEM. For deployments with FortiSIEM NoSQL database, Supervisor node distributes each search query to Worker nodes and summarizes the partial results sent back from Worker nodes. Assuming you have sufficient NFS IOPS, Searches can be made faster up by adding Worker nodes. Worker nodes can be added to a live system. Since event data is centrally stored in NFS, the newly added Worker can participate in queries.

For deployments with Elasticsearch, Supervisor node sends each search query to Elasticsearch Coordinating node, which then distributes each search query to Elasticsearch Data Node and summarizes the partial results sent back from Data Node to the Supervisor node. Adding Elasticsearch Data Nodes can make up searches faster. Since each Data Node has its own storage, it takes some time for data to be distributed to the newly added Data Node. However, since data is stored locally on each Data Node, this solution scales horizontally.

Rules and Incidents

Rules detect bad behavioral anomalies for machines and users in real time. FortiSIEM has developed SQL-like XML based rule specification language. The user creates a rule from GUI, tests it using real events and then deploys the rule. The XML language is quite powerful and uses CMDB Objects (e.g. Device, Network and Application Groups, Event Type Groups, Malware Objects, Country groups, Watch Lists) to keep the rules concise.

A Rule specification involves multiple sub-patterns of events connected by temporal operators (AND, OR, AND NOT, FOLLOWED BY, and NOT FOLLOWED BY). Each sub-pattern is like a SQL Query with filters, group by attributes and thresholds on aggregates. The thresholds can be static or dynamically specified based on statistics. A rule can be nested – meaning a rule can be set to trigger another rule. A rule can also create a watch list that, like a CMDB Object, can be used in another rule.

Rule computation happens in a streaming mode using distributed in-memory computation involving Super and Worker nodes. Latest Rule definitions are distributed to Super and Worker nodes. Worker nodes evaluate each Rule based on the events it sees and periodically sends partial Rule results to the Supervisor node. Supervisor node keeps the global rule state machine and creates an incident when the Rule conditions are met. When a rule involves a statistical attribute (for example: mean or standard deviation), a baseline report is created which computes the statistics and updates the rule conditions. The baseline report also runs in a streaming mode using in-line distributed computation. When a CMDB Object changes, App Server module on the Supervisor node sends a change signal to the Worker nodes, which then download the changes. This distributed in-memory computation enables FortiSIEM to scale to near real time performance with high EPS and large number of rules.

Since FortiSIEM analyzes all data including logs, performance and availability metrics, flows and configuration changes, the rule engine can detect suspicious behavior. This ability to cross correlate across different functional IT domains is what makes FortiSIEM rule framework powerful.

Incident Notification Policy

Once an incident trigger, the user may want to take an action, for example: send an email, create a ticket or initiate a remediation action. Rather than attaching an action to an incident, which does not scale, FortiSIEM takes a policy-based approach. You can write Incident Notification policies involving Time Of Day, Incident Severity, Affected Items and Affected Organization and attach actions to policies. This allows you to create corporate wide policies on who works on what and on which times of the day. Affected Items are specified using CMDB Groups and Assigned Users can be specified using CMDB Users – this makes incident notification policies easy to specify and maintain.

Remediation Library

You may want to remediate an Incident by running a script. In FortiSIEM, this amounts to creating an Incident Notification Policy and attaching the Remediation Script as an Action to the Notification Policy. The remediation script may run on the Supervisor node or on the Collectors since the enforced devices may be behind a firewall.

When an Incident triggers and a Remediation Action has to be taken, App Server sends a signal to the involved enforcement points (Supervisor and Collectors). The enforcement point first retrieves necessary information (such as enforced on device IP or Host name, enforced on device credentials and Incident details) from the Supervisor node and passes that information to the Remediation Script. After the script executes, the Remediation results are attached to the Incident.

FortiSIEM provides a wide collection of inbuilt Remediation Scripts. The user can create new Remediation Scripts in FortiSIEM.

External Ticketing System Integration

This feature allows you to manage a FortiSIEM Incident in an external ticketing system. Several API based built-in integrations are available – ServiceNow, Salesforce and ConnectWise. A Java based framework is available for user to create integrations to other ticketing systems.

There can be four types of integrations – Device or Incident and Inbound or Outbound.

  • Incident Outbound Integration is used to create a ticket in an external ticketing system.
  • Incident Inbound Integration is used to update the external ticket status in FortiSIEM of a ticket opened previously using Incident Outbound Integration. If a ticket is closed in external ticketing system, then the ticket status is also updated in FortiSIEM.
  • Device Outbound Integration is used to update CMDB in an external ticketing system from FortiSIEM CMDB. Every ticketing system needs a CMDB.
  • Device Inbound Integration is used to update FortiSIEM device attributes from an external CMDB.

To use built-in Incident Outbound and Device Outbound Integrations, define an appropriate Integration and attach it as an Action to an Incident Notification Policy. You can use extensive field mappings to customize how the ticket will appear in the external ticketing system. Incident Inbound and Device Inbound integrations have to be scheduled to run at periodic intervals.

Dashboards

FortiSIEM offers various types of dashboards for the user to understand the data it collects and the incidents that are triggering in the system:

Summary Dashboards

Summary dashboards show a near real time view of health, up-time, incidents and other key performance metrics of many devices in a single spreadsheet format – each row is a device and each column is a metric. Cells are color-coded (Red, Yellow, Green) to highlight the values when they cross certain customizable limits. The advantage of this type dashboard is that user can simultaneously compare many metrics of many devices from a single view and instantaneously spot issues. User can customize the groups of devices and the corresponding metrics. The user can build multiple Summary dashboards. FortiSIEM has developed an in-memory database that powers this dashboard – continuous querying event database does not scale.

Widget Dashboards

Widget dashboards offer the more traditional Top N dashboard view – one chart for one metric. Many chart types – Gauge, Table, Bar, Donut, Map, Line, Combo (Table and Line), Tree Map, and Heat Map. Any FortiSIEM Report – whether it is reported on Events or on CMDB – can be added to a Widget dashboard. FortiSIEM Widget Dashboards has three distinctive advantages.

  • Color Coding – Items in each widget can be color coding based on thresholds – this can quickly help the user to spot problems
  • Dynamic Search – The user can filter the entire dashboard by Host Name or IP Address and quickly
  • Streaming Computation – The reports in the widget dashboard are computed in a streaming mode without making repeated queries to the event database. This makes the dashboards fast to load.

Business Service Dashboards

Business Service Dashboards provide a top-down view of Business Service health. User can see the incidents related to each Business Service and then drill down on the impacted devices and incidents.

Identity and Location Dashboards

Identity and Location dashboards provide a tabular view of network identity to user identity mappings.

Incident Dashboards

FortiSIEM provides two Incident Dashboards – Overview and Risk View.

  • Overview dashboard shows a top-down view into Incidents By Category, Top Incidents and where they are triggering, Top Impacted Devices and what Incidents they are triggering.
  • Risk View dashboard organizes devices and users by Risk.