AWS/EC2 Collector Deployments

This Case covers the situation where your Collectors are deployed on AWS/EC2.

To configure Collector HA on AWS/EC2, you will need to configure your collectors as normal. Each Collector will need to have a specifically configured IAM Role. To configure an IAM Role to be later associated with each instance, you will need an IAM Policy. An IAM Policy contains the permissions needed for FortiSIEM to create VIP to handle Collector HA group. After the IAM Policy has been created, you will create the IAM Role then associate it to all the Collectors. Next, you will locate an unused VIP, and finally configure Collector HA on FortiSIEM using that VIP. The Collector HA Cluster needs to be created with one Leader and one or more Followers and a Virtual IP (VIP) that is always owned by the Leader. High Availability (HA) is then enabled via Virtual Router Redundancy Protocol (VRRP).

During normal operations:

• Logs sent to the VIP are handled by the Leader Collector (which owns the VIP).
• FortiSIEM Supervisor node distributes event pulling and performance monitoring jobs among all Collectors in the Cluster.

If the Leader Collector goes down:

• The Follower node with highest priority will become the Leader and own the VIP.
• Logs previously sent to the (failed) Leader Collector will automatically reach the new Leader Collector.
• FortiSIEM Supervisor node will automatically re-distribute event pulling and performance monitoring jobs previously assigned to the failed Leader Collector, to other Collectors in the HA Cluster.

If a Follower Collector goes down:

• App Server will distribute event pulling and performance monitoring jobs assigned to the failed Collector to other Collectors in the HA Cluster.
• If a failed Collector comes back up, then it will stay a Follower, but the event pulling jobs will be re-distributed among all the working Collectors in the HA Cluster.

Failure and Recovery:

• If the Leader Collector goes down then the Follower with the highest priority takes over. If the Leader Collector comes back up, then it will resume the Leader role due to its higher priority.
• If a Follower Collector goes down, the Leader will continue to operate. If the Follower comes back up, then it will remain a Follower while the Leader is operational.

• Step 1 - Configure Collectors
• Step 2 - Create IAM Policy
• Step 3 - Create IAM Role
• Step 4 - Associate IAM Role to all Collectors
• Step 5 - Locate Unused VIP
• Step 6 - Configure Collector HA Group Via VRRP

Step 1 - Configure Collectors

Configure Collectors as normal.

Step 2 - Create IAM Policy

To create an IAM policy, take the following steps.

Note: For more information, see https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create-console.html#access_policies_create-json-editor

1. Sign in to https://console.aws.amazon.com/iam/
2. From the left pane, select Policies.
3. Click Create policy.
4. From the Policy editor section, select JSON.
5. In the Policy editor pane, paste the following information:
   
   

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "ec2:AssociateAddress",
            "Resource": "*"
        }
    ]
}

6. Click Next.
7. Under Policy name, enter a name for the policy, such as "AssociateAddress".
8. Click Create Policy.

Step 3 - Create IAM Role

To create an IAM role, take the following steps.

1. Sign in to https://console.aws.amazon.com/iam/.
2. From the left pane, select Roles.
3. Click Create role.
4. Under Trusted entity type, select AWS service.
5. From the Service or use case drop-down list, select EC2.
6. Under Use case, select EC2.
   
   

   

7. Click Next.
8. In the search field under Permissions policies, enter the name of the policy you created in Create IAM Policy.
9. Add a checkmark to the checkbox to attach that policy to the role we're creating.
10. Click Next.
11. Under the Role name field, enter a name for this role, such as "AssociateAddress".
12. (Optional) In the Role description field, enter an explanation of this role, such as "Allows EC2 instances to call AWS services on your behalf."
13. Click Create role.

Step 4 - Associate IAM Role to all Collectors

For each collector to be part of the Collector HA group, take the following steps.

1. Sign in to the EC2 Dashboard (https://console.aws.amazon.com/ec2/v2/home).
2. From the left pane, select Instances.
3. Select Collector.
4. Open the Actions drop-down list, and select Security > Modify IAM role.
   
   

   

5. Under IAM role, select the role you created in Create IAM Role, and click Update IAM role.
   
   

   
   
   

6. Repeat steps 3-5 for any remaining collectors you with to include as part of the Collector HA group.

Step 5 - Locate Unused VIP

A public IP address is required to configure a Collector HA group. To locate a public IP address to use, take the following steps.

1. Sign in to the EC2 Dashboard (https://console.aws.amazon.com/ec2/v2/home).
2. From the left pane, expand Network & Security, and select Elastic IPs.
   
   

   
   
   
   
   

3. Under Elastic IP addresses, look for an available IP address that is Type Public IP and its Associated instance ID, Private IP address, and Association ID are all empty.
4. Ping the IP address. If it is not pingable, then it can be used as a VIP, as it confirms that the VIP is not in use. Record this VIP address. You will need it later, and need to input this into the VIP field in Step 6 - Configure Collector HA group Via VRRP.

Step 6 - Configure Collector HA Group Via VRRP

To create a new Collector High Availability configuration, take the following steps:

Ensure that the VIP and Collectors in a Collector HA group are on the same network. This is required for High Availability to work.

1. Navigate to Admin > Settings > System > Cluster Config.
2. Under Collector High Availability, click New. The Create Collector HA Group window appears. Take the following steps.
   1. From the Organization drop-down list, select your Organization with your collectors.
   2. In the Group Name field, enter the name of your Collector group.
   3. From HA Via, select VRRP.
   4. From the Collectors drop-down list, select the Collectors to include in the group.
      
      Note: The available Collectors displayed are those from the same Organization selected earlier.
      
      
   5. In the VIP field, enter the Virtual Internet Protocol address.
      Note: This VIP can be internal.
      
   6. In the VIP Router ID field, enter the VIP Router ID number (1-255).
3. For each collector, click Edit, and take the following steps.
   1. In the Interface field, enter the name of the interface. For example, eth0.
   2. From the Role drop-down list, select the Collector’s role (Leader, Follower).
      Note: Only one collector from the HA group can be assigned as a Leader.
      
   3. In the Priority field, enter the numeric priority of the Collector, with the high number being the greater priority (1-254).
      
      Note: The Collector Leader Priority should be higher than all of its Followers.
      
      
   4. Click Save.
      
   5. Repeat step 3a-e for each Collector. When done, proceed to step 4.
4. Click Test. If the test succeeds, proceed to step 5. If it fails, check your earlier configurations.
5. Click Save.