Prioritizing DH Group Configuration

In FOS 5.2, the default Diffie-Hellman DH group has changed from 5 to 14, to provide sufficient protection for stronger cipher suites that include AES and SHA2. Because of this change, both IKEv1 and IKEv2 now allow up to 3 DH groups to be configured in the phase 1 and phase 2 settings, while preserving the ordering since the initiator always begins by using the first group in the list. The default DH group in the configuration has been updated to include group 14 and 5, in that order. You can add and remove other groups and the order they appear in the configuration is the order in which they are negotiated.

The IKEv1 protocol does not natively provide for DH group negotiation in Aggressive Mode and Quick Mode. As a result, when multiple DH groups are used with IKEv1 Aggressive Mode or Quick Mode, delays in tunnel establishment can occur and so it is recommended to continue to configure matching DH groups on both peers whenever possible.

1. IKEv1 Aggressive Mode Initiator: Since Aggressive Mode includes the KE payload in the first message, FortiOS must select a group to use to generate the DH public number. FortiOS starts with the first group in the list. If the negotiation fails due to timeout, it will try the second group, and finally the third. If the third also fails, FortiOS goes back to the first DH group again, and starts over. Once it finds the correct group and the tunnel is established, it will continue to use that group for re-keying as long as the VPN connection remains up.

2. IKEv1 Aggressive Mode Responder: If the group proposed by the initiator doesn't match the group proposed by the responder, the negotiation fails due to no proposal chosen. Since authentication has not been established the responder cannot send a notify message to the initiator. The initiator will try the next DH group in its configuration when the negotiation time out occurs, which takes 30 seconds by default. At that point, if the next DH group is a successful match, the tunnel comes up.

3. IKEv1 Main Mode Initiator: In Main Mode, the SA and KE payloads come in different messages. The SA parameters, including DH group, are negotiated first in MM1 and MM2, then the KE payloads are exchanged in the MM3 and MM4 messages. So no change was made for Main Mode.

4. IKEv1 Main Mode Responder: As above, no change for Main Mode.

5. IKEv1 Quick Mode Initiator: Quick Mode has the same problem as Aggressive Mode, in that the SA proposal and KE payloads arrive in the same message. So unlike Main Mode, the initiator does not know the negotiated DH group prior to constructing its KE payload. Like with AM, it will start with the first group in the configured DH group list. If the negotiation times out, or if we receive a No-Proposal-Chosen notify message from the responder, we will switch to the next group in the list and try again.

6. IKEv1 Quick Mode Responder: Similar to Aggressive Mode, if the initiator's first group doesn't match, the Quick Mode will fail with a no proposal chosen error. In Quick Mode, you have the benefit of an authenticated IKE SA by which you can immediately notify the peer of the error. Sending the No-Proposal-Chosen notify to the initiator allows the initiator to try the next group immediately without waiting for a timeout.

7. IKEv2 SA_INIT/CHILD_SA Initiator: Like Aggressive Mode, in IKEv2 both the SA_INIT and CHILD_SA exchanges have the SA proposal and KE payload in the same message. However, unlike IKEv1, the IKEv2 RFC specifies a mechanism to handle this. In IKEv2, if the negotiated DH group does not match the group specified in the KE payload, the INVALID_KE notify message is sent and the initiator retries the exchange using the DH group specified in the notify message. Initiator side support for handling the INVALID_KE message has been added. This code wasn't needed previously, as the IKEv2 CLI allowed only one DH group to be configured. Now that the CLI restriction has been removed and multiple DH groups can be configured for IKEv2 in the phase1 and phase2 settings, the initiator will handle receipt of INVALID_KE messages as per the IKEv2 RFC. As long as the VPN connection remains up, the initiator will subsequently re-key using the negotiated group.