Setting Native Elasticsearch Retention Threshold (Online Settings)

FortiSIEM offers space based thresholds, or a combination of space based thresholds and index lifecycle management (ILM) thresholds based on time duration limits for Native Elasticsearch. When ILM is available, events are moved from Hot to Warm (Warm age phase) and from Warm to Cold (Cold age phase), based on which policy, space threshold or age threshold, occurs first. Ensure you review the latest What's New section for any Elasticsearch limitations. Configure your Elasticsearch retention threshold by following the appropriate instructions after configuring your Elasticsearch deployment.
Note: AWS Elasticsearch and Cloud Elasticsearch do not have the ability to allow control over Hot/Cold storage configuration. Online Settings only work for Native Elasticsearch.

Configuring Native Elasticsearch Retention Threshold

Complete these steps to configure Native Elasticsearch free space and age retention threshold:

  1. Go to ADMIN > Settings > Database > Online Settings.
  2. Select the low percentage threshold, high percentage threshold, and age under:
    1. Hot Node - Free Space Threshold - Events are moved to Warm nodes based on the first occurrence of one of the following:
      • When the Hot node cluster disk free space falls below Low value, then events are moved to Warm nodes until the Hot node cluster disk free space reaches High value.
      • If the time duration limit set under Hot Age (the Warm age phase) is met, all events under this limit are moved to Warm nodes.
        If Warm node policy is not defined, but Cold is defined, then events are moved to Cold.
    2. Warm Node - Free Space Threshold - Events are moved to Warm nodes based on the first occurrence of one of the following:
      • When the Warm node cluster disk free space falls below Low value, then events are moved to Cold nodes until the Warm node cluster disk free space reaches High value.
      • If the time duration limit set under Warm Age (the Cold age phase) is met, all events under this limit are moved to Cold nodes.
        Note: In the fsiem_ilm_policy, the cold age phase is reflected as a sum of the warm age phase and cold age phase UI values.
    3. Cold Node - Free Space Threshold - When the Cold node cluster disk free space reaches Low value, then:
      • If Archive is defined, then events are archived until Cold node cluster disk free space reaches High value.
      • If Archive is not defined, events are purged until the Cold node cluster disk free pace reaches High value.
    4. Archive Threshold - Events are archived. When Archive Mount Point disk free space reaches Low value, then events are purged until disk free space reaches High value.
      Note: Archive must be configured in order for Archive Threshold to appear as an option.
  1. Click Save.