Multi-Chassis IPSec Redundancy
This chapter provides information about multi-chassis IPSec redundancy configurations.
Topics in this chapter include:
Applicability
This initial version of this chapter was based on SR OS Release 10.0.R8, but the MD-CLI in the current edition corresponds to SR OS Release 22.10.R2.
Overview
Multi-Chassis IPSec redundancy (MC-IPSec) is a stateful inter-chassis IPSec failover mechanism. IPSec tunnel states are synchronized between the primary and standby chassis. A tunnel group failure on the primary chassis or a primary chassis failure could trigger MC-IPSec failover to the standby chassis.
The following are some highlights of this feature:
Internet Key Exchange version 2 (IKEv2) only
Multi-active tunnel group only
The granularity of failover is tunnel group, which means a specific tunnel group could failover to the standby chassis independent of other tunnel groups on the primary chassis
Both static and dynamic LAN-to-LAN tunnels are supported
This feature has the following building blocks:
Primary chassis election: MC-IPSec mastership protocol (MIMP) runs between the chassis to elect a primary chassis with independent MIMP runs for each tunnel group
Synchronization: multi-chassis synchronization (MCS) synchronizes the IPSec states between chassis
Routing:
MC-IPSec-aware routing attracts traffic to the primary chassis
Shunting support
MC-IPSec-aware virtual router redundancy protocol (VRRP)
The figure MC-IPSec architecture shows two redundant IPSec chassis in the middle: a primary chassis and a standby chassis.
The fundamentals of MC-IPSec are:
Only the primary chassis processes encapsulating security payload (ESP) and IKE traffic. If the standby chassis receives traffic, it shunts it to the primary chassis, if possible. The traffic is discarded if the standby chassis fails to shunt the traffic.
The same local gateway address must be provisioned on both chassis.
MC-IPSec does not synchronize configurations.
MC-IPSec-aware routing attracts traffic to the primary chassis for both public and private services, which is achieved by exporting the corresponding IPSec routes to the routing protocol using a route policy and setting a different routing metric according to the MC-IPSec state.
In case of a Layer 2 public network, MC-IPSec-aware VRRP can be used to trigger VRRP switchover upon MC-IPSec switchover.
MCS synchronizes IPSec states between chassis so that existing IPSec tunnels do not need to be re-established upon switchover.
MIMP elects mastership between two chassis, and it can also detect chassis failure and tunnel group failure; a central BFD session can be associated with MIMP to achieve fast chassis failure detection.
Configuration
The example topology is shown in the figure Example topology.
The example setup includes:
-
an IPSec tunnel initiated by CE-1 and terminated on the primary chassis of the two SeGWs.
-
a public IES service "IES-1" and a private VPRN service "VPRN-2" configured on CE-1, SeGW-3, and SeGW-4.
-
VPRN "VPRN-2" (also) configured on P-5.
-
a static LAN-to-LAN tunnel with pre-shared key.
-
a local VPLS service "VPLS-3" on S-2 to simulate a Layer 2 switch.
-
VRRP 10 between SeGW-3 and SeGW-4 to provide a backup address 192.168.1.254, which is the default next hop for CE-1.
-
VRRP policy 1 bound to VRRP 10 on the primary chassis SeGW-3 to change the in-use priority upon MC-IPSec switchover.
-
OSPF as IGP running in the base routing instance between SeGW-3, SeGW-4, and P-5.
-
MP-BGP running between SeGW-3, SeGW-4, and P-5 for the VPN-IPv4 address family.
A ping in VPRN "VPRN-2" between loopback interface address 192.168.1.1 on CE-1 and 192.168.1.5 on P-5 is used to verify the connectivity over the IPSec tunnel.
The MC-IPSec configuration commands are shown below.
configure
redundancy
multi-chassis
peer <ip-address>
sync
ipsec
tunnel-group <1..64>
sync-tag <string>
mc-ipsec
bfd-liveness <boolean>
discovery-interval
interval-secs <1..1800>
boot <1..1800>
hold-on-neighbor-failure <2..25>
keep-alive-interval <5..500> # deciseconds
tunnel-group <1..64>
admin-state <boolean>
peer-group <1..64>
priority <0..255>
configure
policy-options
policy-statement <string>
entry <1..4294967295>
from
state ipsec-master-with-peer|ipsec-non-master|ipsec-master-without-peer
protocol
name ipsec
configure
service
ies <string>
interface <string>
dynamic-tunnel-redundant-nexthop <unicast-ipv4-address>
static-tunnel-redundant-nexthop <unicast-ipv4-address>
configure
service
vprn <string>
interface <string>
dynamic-tunnel-redundant-nexthop <unicast-ipv4-address>
static-tunnel-redundant-nexthop <unicast-ipv4-address>
configure
isa
tunnel-group <1..64>
ipsec-responder-only <boolean>
configure
vrrp
policy <1..9999>
priority-event
mc-ipsec-non-forwarding <tunnel-grp-id>
hold-clear <1..86400 seconds>
hold-set <1..86400 seconds>
priority
priority-level <1..254>
event-type (delta|explicit)
The parameters are the following:
- in the configure redundancy multi-chassis
context:
-
peer <ip-address> — This command creates or enters a multi-chassis peer. The peer address is by default the system address. This can be changed on the peer using the configure redundancy multi-chassis peer source-address command.
-
sync — This command enters the sync configuration context.
-
ipsec <boolean> — This command enables MCS to synchronize IPSec states.
-
tunnel-group <tunnel-group-id> sync-tag <tag-name> — This command enables MCS to synchronize the IPSec states of the specified tunnel group. The sync-tag parameter is used to match the tunnel group of the peer. The tunnel group states with the same sync-tag on both chassis will be synchronized.
-
-
mc-ipsec — This command enters the multi-chassis IPSec configuration context.
-
bfd-liveness <boolean> — The command bfd-liveness true enables tracking a central BFD session; if the BFD session goes down, then the system considers the peer as down and changes the MC-IPSec status of the configured tunnel group accordingly.
The BFD session uses the source address of MCS as its source address and the MCS peer address as the destination address. Other BFD parameters are configured in the bfd context on the interface that the MCS source address resides on.
The configuration of BFD is optional for MC-IPSec.
-
discovery-interval interval-secs <interval-1> [boot <interval-2>] — This command specifies the time interval that the tunnel group stays in discovery state. Interval 1 is used as discovery interval when a new tunnel group is added to multi-chassis redundancy (mp-ipsec); interval 2 is used as discovery interval after system boot-up. Interval 2 is optional, and when it is not specified, the value for interval 1 is used. Both intervals have a default value of 300 seconds.
-
hold-on-neighbor-failure <2..25> — This command specifies the number of keep-alive failures before considering the peer to be down. The default value is 3.
-
keep-alive-interval <5..500> — This command specifies the time interval of the mastership election protocol keep-alive packets in deciseconds. The default value is 10 deciseconds (1 s).
-
tunnel-group <tunnel-group-id> — This command enables multi-chassis redundancy for the specified tunnel group, or enters an already configured tunnel group context. The configured tunnel groups can failover independently.
— peer-group <tunnel-group-id> — This command specifies the corresponding tunnel group ID on the peer node. The peer tunnel group ID is not necessarily equal to local tunnel group ID.
— priority <priority> — This command specifies the local priority of the tunnel group, this is used to elect a primary chassis, where the higher number prevails. If the priorities are the same, then the peer which has more active ISAs wins; if the priority and the number of active ISAs are same, then the peer with higher IP address wins. The range is from 0 to 255 and the default value is 100.
-
-
-
-
in a from statement of a route policy entry:
-
state ipsec-master-with-peer | ipsec-non-master | ipsec-master-without-peer — These commands specify the MC-IPSec state in a from statement of a route policy entry:
-
ipsec-master-with-peer: The tunnel group is the primary chassis with a peer reachable.
-
ipsec-master-without-peer: The tunnel group is the primary chassis with peer unreachable.
-
ipsec-non-master: The tunnel group is not the primary chassis.
-
-
protocol name ipsec — This command specifies IPSec as protocol in a from statement of a route policy entry. protocol name ipsec refers to the /32 local gateway routes (of both static and dynamic tunnels) and reverse route of dynamic tunnel.
-
-
on a public or private IPSec interface in an IES or VPRN service:
-
static-tunnel-redundant-nexthop <ip-address> and dynamic-tunnel-redundant-nexthop <ip-address> — These commands specify the redundant next hop address on a public or private IPSec interface (with public or private tunnel SAP) for a static and dynamic IPSec tunnel respectively. The specified next hop address is used by the standby chassis to shunt traffic to the primary chassis in case it receives any traffic. The next hop address is resolved in the routing table of the corresponding service.
Note:-
Shunting is supported over:
-
directly connected SAPs
-
spoke SDP terminated IP interfaces
-
-
Shunting over auto-bind tunnel is not supported.
-
Shunting does not work if the tunnel group is down.
-
-
-
in the isa tunnel-group <id> context:
-
ipsec-responder-only <boolean> — With the command ipsec-responder-only true , the system only acts as IKE responder except for the automatic CHILD_SA rekey upon MC-IPSec switchover.
This command is required for MC-IPSec support of static LAN-to-LAN tunnels.
-
-
in the vrrp policy <id> priority-event context:
-
mc-ipsec-non-forwarding <tunnel-grp-id> — This command creates a VRRP policy priority event: mc-ipsec-non-forwarding, which is triggered whenever the specified tunnel group enters the non-forwarding state.
-
hold-clear <seconds> — This command configures the hold time before clearing the event. The range is from 0 to 86400 seconds and the default value is 0 s.
-
hold-set <seconds> — This command configures the hold time before setting the event. The range is from 0 to 86400 seconds and the default value is 0 s.
-
priority <priority-level> explicit — This command sets the VRRP in-use priority to the configured value upon the event. The range is from 0 to 254 and the default value is 0.
-
-
The initial configuration must include the following:
-
The system time of SeGW-3 and SeGW-4 must be the same for the feature to work. Nokia recommends to use a time synchronization protocol such as NTP or SNTP.
-
SeGW-3 and SeGW-4 must be IP reachable in the base routing instance because both MCS and MIMP run in the base routing instance.
Configuration of MC-IPSec
In this section, the following steps are described:
- configure CE-1
- configure S-2
- configure P-5
- configure IPSec tunnel on SeGW-3
- enable MC-IPSec for tunnel group on SeGW-3
- configure MC-IPSec-aware routing on SeGW-3
- configure MC-IPSec-aware VRRP on SeGW-3
- configure SeGW-4
Configure CE-1
On CE-1, the following is configured:
-
a public IES service "IES-1" and a private VPRN service "VPRN-2".
-
a static default route pointing to the VRRP backup address 172.16.1.254.
-
a static IPSec tunnel "tunnel-1" with local address 10.10.10.1 and remote address 10.10.20.1.
-
a loopback interface in VPRN "VPRN-2" with address 192.168.1.1/32 to be used as source address for the ping command to verify the connectivity between CE-1 and P-5 over the IPSec tunnel.
The following base router configuration on CE-1 includes a static route with next hop 172.16.1.254, which is the VRRP backup address.
# on CE-1:
configure {
router "Base" {
interface "int-CE-1-S-2" {
port 1/1/1:1000
ipv4 {
primary {
address 172.16.1.100
prefix-length 24
}
}
}
interface "system" {
ipv4 {
primary {
address 172.31.2.1
prefix-length 32
}
}
}
static-routes {
route 0.0.0.0/0 route-type unicast {
next-hop "172.16.1.254" { # VRRP backup address
admin-state enable
}
}
}
IPSec is configured as follows:
configure {
ipsec {
ike-policy 1 {
ike-transform [1]
ike-version-2 {
}
dpd { # dead peer detection (on peer side; not on MC-IPSec chassis)
}
}
ike-transform 1 {
}
ipsec-transform 1 {
}
Tunnel group 1 is configured as follows:
configure {
isa {
tunnel-group 1 {
admin-state enable
isa-scale-mode tunnel-limit-2k
primary 1/2
}
The public IES service is configured as follows:
configure {
service {
ies "IES-1" {
admin-state enable
service-id 1
customer "1"
interface "int-IPsec-Public-1" {
sap tunnel-1.public:1 {
}
ipv4 {
primary {
address 10.10.10.254
prefix-length 24
}
}
}
}
The private VPRN service on CE-1 is configured as follows:
configure {
service {
vprn "VPRN-2" {
admin-state enable
service-id 2
customer "1"
ipsec {
security-policy 1 {
entry 10 {
local-ip {
address 192.168.1.1/32
}
remote-ip {
address 192.168.1.5/32
}
}
}
}
interface "int-IPsec-private-1" {
tunnel true
sap tunnel-1.private:1 {
ipsec-tunnel "tunnel-1" {
admin-state enable
key-exchange {
dynamic {
ike-policy 1
ipsec-transform [1]
pre-shared-key "pass"
}
}
tunnel-endpoint {
local-gateway-address 10.10.10.1
remote-ip-address 10.10.20.1
delivery-service "IES-1"
}
security-policy {
id 1
}
}
}
}
interface "int-loopback-1" {
loopback true
ipv4 {
primary {
address 192.168.1.1
prefix-length 32
}
}
}
static-routes {
route 192.168.1.5/32 route-type unicast {
ipsec-tunnel "tunnel-1" {
admin-state enable
}
}
}
Configure S-2
On S-2, a local VPLS service 3 simulates a Layer 2 switch between CE-1, SeGW-3, and SeGW-4:
# on S-2:
configure {
service {
vpls "VPLS-3" {
admin-state enable
service-id 3
customer "1"
sap 1/1/c1/1:1 {
description "to SAP in IES 1 on SeGW-3"
}
sap 1/1/c1/2:1000 {
description "to router interface in CE-1"
}
sap 1/1/c1/3:1 {
description "to SAP in IES 1 on SeGW-4"
}
}
Configure P-5
P-5 simulates the core network router, connecting to SeGW-3 and SeGW-4. The configuration on P-5 includes the following:
-
a loopback interface with address 192.168.1.5/32 in VPRN "VPRN-2", which is the destination address of the ping traffic from CE-1.
-
an MP-BGP session for the VPN-IPv4 address family between P-5, SeGW-3, and SeGW-4.
-
GRE spoke SDPs to connect to SeGW-3 and SeGW-4.
On P-5, the following router interfaces are configured in the base router. OSPF is used as IGP.
# on P-5:
configure {
router "Base" {
interface "int-P-5-SeGW-3" {
port 1/1/c1/2:1000
ipv4 {
primary {
address 192.168.35.2
prefix-length 30
}
}
}
interface "int-P-5-SeGW-4" {
port 1/1/c1/1:1000
ipv4 {
primary {
address 192.168.45.2
prefix-length 30
}
}
}
interface "system" {
ipv4 {
primary {
address 192.0.2.5
prefix-length 32
}
}
}
ospf 0 {
admin-state enable
area 0.0.0.0 {
interface "int-P-5-SeGW-3" {
}
interface "int-P-5-SeGW-4" {
}
interface "system" {
}
}
}
On P-5, the following GRE SDPs are configured toward SeGW-3 and SeGW-4:
configure {
service {
sdp 53 {
admin-state enable
description "GRE SDP toward SeGW-3"
signaling off
far-end {
ip-address 192.0.2.3
}
}
sdp 54 {
admin-state enable
description "GRE SDP toward SeGW-4"
signaling off
far-end {
ip-address 192.0.2.4
}
}
VPRN "VPRN-2" is configured on P-5, as follows:
configure {
service {
vprn "VPRN-2" {
admin-state enable
service-id 2
customer "1"
bgp-ipvpn {
mpls {
admin-state enable
route-distinguisher "64496:2"
vrf-target {
community "target:64496:2"
}
}
}
interface "int-loopback-1" {
loopback true
ipv4 {
primary {
address 192.168.1.5
prefix-length 32
}
}
}
spoke-sdp 53:2 {
}
spoke-sdp 54:2 {
}
}
The BGP configuration on P-5 is as follows:
configure {
router "Base" {
autonomous-system 64496
bgp {
group "MPBGP" {
type internal
family {
vpn-ipv4 true
}
}
neighbor "192.0.2.3" {
group "MPBGP"
}
neighbor "192.0.2.4" {
group "MPBGP"
}
}
Configure IPSec tunnel on SeGW-3
The configuration on SeGW-3 is described in four consecutive sections. In this first section, the following is configured:
-
the tunnel group, which must be in multi-active mode before MC-IPSec can be enabled.
-
an interface "int-Redundant-1", which is a spoke-SDP terminated interface used for shunting.
-
GRE SDP 34 toward SeGW-4 and GRE SDP 35 toward P-5.
-
IPSec tunnel "tunnel-1" is the tunnel to CE-1; both SeGW-3 and SeGW-4 use the same local gateway address: 10.10.20.1.
The following configures tunnel group 1 on SeGW-3:
# on SeGW-3
configure {
isa {
tunnel-group 1 {
admin-state enable
isa-scale-mode tunnel-limit-2k
ipsec-responder-only true
multi-active {
isa 1/2 { }
}
}On SeGW-3, the following router interfaces are configured in the base router. A static route is configured toward CE-1. OSPF is the IGP used between SeGW-3, SeGW-4, and P-5.
configure {
router "Base" {
interface "int-SeGW-3-P-5" {
port 1/1/1:1000
ipv4 {
primary {
address 192.168.35.1
prefix-length 30
}
}
}
interface "int-SeGW-3-SeGW-4" {
port 1/1/3:1000
ipv4 {
primary {
address 192.168.34.1
prefix-length 30
}
}
}
interface "system" {
ipv4 {
bfd {
admin-state enable
}
primary {
address 192.0.2.3
prefix-length 32
}
}
}
static-routes {
route 10.10.10.0/24 route-type unicast {
next-hop "172.16.1.100" {
admin-state enable
}
}
}
ospf 0 {
admin-state enable
area 0.0.0.0 {
interface "int-SeGW-3-P-5" {
}
interface "int-SeGW-3-SeGW-4" {
}
interface "system" {
}
}
}The IPSec settings are as follows:
configure {
ipsec {
ike-policy 1 {
ipsec-lifetime 7200
ike-transform [1]
ike-version-2 {
}
}
ike-transform 1 {
isakmp-lifetime 172800
}
ipsec-transform 1 {
}The GRE SDPs are configured as follows:
configure {
service {
sdp 34 {
admin-state enable
description "GRE SDP toward SeGW-4"
signaling off
far-end {
ip-address 192.0.2.4
}
}
sdp 35 {
admin-state enable
description "GRE SDP toward P-5"
signaling off
far-end {
ip-address 192.0.2.5
}
}The public IES service is configured as follows. In a later step, a VRRP policy will be configured and applied.
configure {
service {
ies "IES-1" {
admin-state enable
service-id 1
customer "1"
interface "int-IPsec-Public-1" {
static-tunnel-redundant-nexthop 192.168.34.2
sap tunnel-1.public:1 {
}
ipv4 {
primary {
address 10.10.20.254
prefix-length 24
}
}
}
interface "int-SeGW-3-S-2" {
sap 1/1/2:1 {
description "SAP to switch S-2"
}
ipv4 {
primary {
address 172.16.1.252
prefix-length 24
}
vrrp 10 {
backup [172.16.1.254]
priority 200
ping-reply true
}
}
}The private VPRN service "VPRN-2" is configured as follows:
configure {
service {
vprn "VPRN-2" {
admin-state enable
service-id 2
customer "1"
ipsec {
security-policy 1 {
entry 10 {
local-ip {
address 192.168.1.5/32
}
remote-ip {
address 192.168.1.1/32
}
}
}
}
bgp-ipvpn {
mpls {
admin-state enable
route-distinguisher "64496:2"
vrf-target {
community "target:64496:2"
}
}
}
interface "int-IPsec-Private-1" {
tunnel true
static-tunnel-redundant-nexthop 192.168.20.2
sap tunnel-1.private:1 {
ipsec-tunnel "tunnel-1" {
admin-state enable
key-exchange {
dynamic {
ike-policy 1
ipsec-transform [1]
pre-shared-key "pass"
}
}
tunnel-endpoint {
local-gateway-address 10.10.20.1
remote-ip-address 10.10.10.1
delivery-service "IES-1"
}
security-policy {
id 1
}
}
}
}
interface "int-Redundant-1" {
ipv4 {
primary {
address 192.168.20.1
prefix-length 30
}
}
spoke-sdp 34:20 {
ingress {
vc-label 2049
}
egress {
vc-label 2048
}
}
}
spoke-sdp 34:2 {
description "SDP to SeGW-4"
}
spoke-sdp 35:2 {
description "SDP to P-5"
}
static-routes {
route 192.168.1.1/32 route-type unicast {
ipsec-tunnel "tunnel-1" {
admin-state enable
}
}
}
Enable MC-IPSec for tunnel group 1 on SeGW-3
In this section, the following steps are described:
-
Create a multi-chassis peer using the system address of SeGW-4.
-
Enable MCS for IPSec and tunnel group 1.
-
Enable MC-IPSec for the tunnel group with a configured priority 200.
-
Bind a central BFD session to MC-IPSec from the system interface.
Multi-chassis peer 192.0.2.4 is configured and MCS and MC-IPSec are enabled for tunnel group 1:
# on SeGW-3:
configure {
redundancy {
multi-chassis {
peer 192.0.2.4 {
admin-state enable
sync {
admin-state enable
ipsec true
tunnel-group 1 {
sync-tag "tag-1"
}
}
mc-ipsec {
bfd-liveness true
tunnel-group 1 {
admin-state enable
peer-group 1
priority 200
}
}
}BFD is enabled for MC-IPSec in the preceding configuration. BFD is configured on the system interface 192.0.2.3:
configure {
router "Base" {
interface "system" {
ipv4 {
bfd {
admin-state enable
}
primary {
address 192.0.2.3
prefix-length 32
}
}
}Configure MC-IPSec-aware routing on SeGW-3
In this step, a route policy is defined and applied to VPRN "VPRN-2".
Route policy "IPsec-to-MPBGP" exports static route 192.168.1.1/32 in VPRN "VPRN-2" to P-5. This policy sets the local preference of the prefix 192.168.1.1/32 according to the MC-IPSec state:
-
for the ipsec-master-with-peer state: local preference 200
-
for the ipsec-non-master state: local preference 100
-
for the ipsec-master-without-peer state: local preference 200
The state ipsec-master-without-peer can be used to attract traffic to the designated primary chassis in case of "dual master" (meaning two chassis lose the MIMP connection in the base routing instance). In this example, SeGW-3 has local preference 200 and SeGW-4 has local preference 100 for ipsec-master-without-peer.
The route policy is configured as follows:
# on SeGW-3:
configure {
policy-options {
community "vprn2" {
member "target:64496:2" { }
}
prefix-list "CE-1-Internal" {
prefix 192.168.1.1/32 type exact {
}
}
policy-statement "IPsec-to-MPBGP" {
entry 10 {
from {
prefix-list ["CE-1-Internal"]
state ipsec-master-with-peer
}
action {
action-type accept
local-preference 200
community {
add ["vprn2"]
}
}
}
entry 20 {
from {
prefix-list ["CE-1-Internal"]
state ipsec-non-master
}
action {
action-type accept
local-preference 100
community {
add ["vprn2"]
}
}
}
entry 30 {
from {
prefix-list ["CE-1-Internal"]
state ipsec-master-without-peer
}
action {
action-type accept
local-preference 200
community {
add ["vprn2"]
}
}
}
default-action {
action-type accept
community {
add ["vprn2"]
}
}
}The BGP configuration on SeGW-3 is as follows:
configure {
router "Base" {
autonomous-system 64496
bgp {
group "MPBGP" {
type internal
family {
vpn-ipv4 true
}
}
neighbor "192.0.2.4" {
group "MPBGP"
}
neighbor "192.0.2.5" {
group "MPBGP"
}
The route policy is applied as vrf-export in VPRN "VPRN-2":
configure {
service {
vprn "VPRN-2" {
bgp-ipvpn {
mpls {
vrf-export {
policy ["IPsec-to-MPBGP"]
}Configure MC-IPSec-aware VRRP on SeGW-3
In this section, a VRRP policy is defined that uses the mc-ipsec-non-forwarding priority event to lower the in-use VRRP priority upon MC-IPSec switchover, which ensures VRRP and MC-IPSec have the same primary chassis. The VRRP instance needs to be in preempt mode.
This VRRP policy is only configured on the designated VRRP primary chassis SeGW-3, not on the standby chassis. The VRRP policy is applied to the interface "int-SeGW3-S-2" of IES "IES-1".
VRRP policy 1 is configured as follows:
# on SeGW-3:
configure {
vrrp {
policy 1 {
priority-event {
mc-ipsec-non-forwarding 1 {
priority {
priority-level 50
event-type explicit
}
}
}
}VRRP policy 1 is applied in VRRP instance 10 in the IES service:
configure {
service {
ies "IES-1" {
interface "int-SeGW-3-S-2" {
sap 1/1/2:1 {
description "SAP to switch S-2"
}
ipv4 {
primary {
address 172.16.1.252
prefix-length 24
}
vrrp 10 {
backup [172.16.1.254]
priority 200
ping-reply true
policy 1
}
}
}
---snip---
Configure SeGW-4
The configuration on the standby chassis SeGW-4 is similar, but with different priorities and without the VRRP policy.
The tunnel group is configured in multi-active mode:
# on SeGW-4:
configure {
isa {
tunnel-group 1 {
admin-state enable
isa-scale-mode tunnel-limit-2k
ipsec-responder-only true
multi-active {
isa 1/2 { }
}
}The MCS and MC-IPSec configuration is as follows:
configure {
redundancy {
multi-chassis {
peer 192.0.2.3 {
admin-state enable
sync {
admin-state enable
ipsec true
tunnel-group 1 {
sync-tag "tag-1"
}
}
mc-ipsec {
bfd-liveness true
tunnel-group 1 {
admin-state enable
peer-group 1
priority 150
}
}
}
}The base router configuration on SeGW-4 includes the following router interfaces and a static route to CE-1. OSPF is used as IGP between SeGW-3, SeGW-4, and P-5.
configure {
router "Base" {
interface "int-SeGW-4-P-5" {
port 1/1/2:1000
ipv4 {
primary {
address 192.168.45.1
prefix-length 30
}
}
}
interface "int-SeGW-4-SeGW-3" {
port 1/1/3:1000
ipv4 {
primary {
address 192.168.34.2
prefix-length 30
}
}
}
interface "system" {
ipv4 {
bfd {
admin-state enable
}
primary {
address 192.0.2.4
prefix-length 32
}
}
}
static-routes {
route 10.10.10.0/24 route-type unicast {
next-hop "172.16.1.100" {
admin-state enable
}
}
}
ospf 0 {
admin-state enable
area 0.0.0.0 {
interface "int-SeGW-4-P-5" {
}
interface "int-SeGW-4-SeGW-3" {
}
interface "system" {
}
}
}The IPSec configuration is as follows:
configure {
ipsec {
ike-policy 1 {
ipsec-lifetime 7200
ike-transform [1]
ike-version-2 {
}
}
ike-transform 1 {
isakmp-lifetime 172800
}
ipsec-transform 1 {
}
The following route policy is configured on SeGW-4, The local preference is lower for the ipsec-master-without-peer state.
configure {
policy-options {
community "vprn2" {
member "target:64496:2" { }
}
prefix-list "CE-1-Internal" {
prefix 192.168.1.1/32 type exact {
}
}
policy-statement "IPsec-to-MPBGP" {
entry 10 {
from {
prefix-list ["CE-1-Internal"]
state ipsec-master-with-peer
}
action {
action-type accept
local-preference 200
community {
add ["vprn2"]
}
}
}
entry 20 {
from {
prefix-list ["CE-1-Internal"]
state ipsec-non-master
}
action {
action-type accept
local-preference 100
community {
add ["vprn2"]
}
}
}
entry 30 {
from {
prefix-list ["CE-1-Internal"]
state ipsec-master-without-peer
}
action {
action-type accept
local-preference 100
community {
add ["vprn2"]
}
}
}
default-action {
action-type accept
community {
add ["vprn2"]
}
}
}The BGP configuration on SeGW-4 is as follows:
configure {
router "Base" {
autonomous-system 64496
bgp {
group "MPBGP" {
type internal
family {
vpn-ipv4 true
}
}
neighbor "192.0.2.3" {
group "MPBGP"
}
neighbor "192.0.2.5" {
group "MPBGP"
}
}The following GRE SDPs are configured:
configure {
service {
sdp 43 {
admin-state enable
description "GRE SDP toward SeGW-3"
signaling off
far-end {
ip-address 192.0.2.3
}
}
sdp 45 {
admin-state enable
description "GRE SDP toward P-5"
signaling off
far-end {
ip-address 192.0.2.5
}
}The public IES service is configured as follows:
configure {
service {
ies "IES-1" {
admin-state enable
service-id 1
customer "1"
interface "int-IPsec-Public-1" {
static-tunnel-redundant-nexthop 192.168.34.1
sap tunnel-1.public:1 {
}
ipv4 {
primary {
address 10.10.20.254
prefix-length 24
}
}
}
interface "int-SeGW-4-S-2" {
sap 1/1/1:1 {
}
ipv4 {
primary {
address 172.16.1.253
prefix-length 24
}
vrrp 10 {
backup [172.16.1.254]
ping-reply true
}
}
}
}The private VPRN service is configured as follows:
configure {
service {
vprn "VPRN-2" {
admin-state enable
service-id 2
customer "1"
ipsec {
security-policy 1 {
entry 10 {
local-ip {
address 192.168.1.5/32
}
remote-ip {
address 192.168.1.1/32
}
}
}
}
bgp-ipvpn {
mpls {
admin-state enable
route-distinguisher "64496:2"
vrf-target {
community "target:64496:2"
}
vrf-export {
policy ["IPsec-to-MPBGP"]
}
}
}
interface "int-IPsec-Private-1" {
tunnel true
static-tunnel-redundant-nexthop 192.168.20.1
sap tunnel-1.private:1 {
ipsec-tunnel "tunnel-1" {
admin-state enable
key-exchange {
dynamic {
ike-policy 1
ipsec-transform [1]
pre-shared-key "pass"
}
}
tunnel-endpoint {
local-gateway-address 10.10.20.1
remote-ip-address 10.10.10.1
delivery-service "IES-1"
}
security-policy {
id 1
}
}
}
}
interface "int-Redundant-1" {
ipv4 {
primary {
address 192.168.20.2
prefix-length 30
}
}
spoke-sdp 43:20 {
ingress {
vc-label 2048
}
egress {
vc-label 2049
}
}
}
spoke-sdp 43:2 {
description "SDP to SeGW-3"
}
spoke-sdp 45:2 {
description "SDP to P-5"
}
static-routes {
route 192.168.1.1/32 route-type unicast {
ipsec-tunnel "tunnel-1" {
admin-state enable
}
}
}
}Verification
The following will be verified in this section:
- the MC-IPSec status and VRRP status on SeGW-3 and SeGW-4
- the status of the IPSec tunnel on CE-1
- the status of the IPSec tunnel on the SeGWs
Verify the MC-IPSec status on SeGW-3 and SeGW-4
The following is verified:
-
SeGW-3 is the primary chassis (master) and SeGW-4 is the standby for tunnel group 1 because SeGW-3 has the higher priority 200.
-
SeGW-3 is the primary node for VRRP instance 10 and SeGW-4 is the backup.
SeGW-3 is the primary chassis in tunnel group 1 with priority 200:
[/]
A:admin@SeGW-3# show redundancy multi-chassis mc-ipsec peer ip-address 192.0.2.4
===============================================================================
Multi-Chassis MC-IPsec
===============================================================================
Peer Name : (Not Specified)
Peer Addr : 192.0.2.4
Keep Alive Intvl: 1.0 secs Hold on Nbr Fail : 3
Discovery Intvl : 300 secs Discovery Boot Intvl : 300 secs
BFD : Enable
Last update : 02/16/2023 10:09:10
======================================================================
Multi-Chassis IPsec Multi Active Tunnel-Group Table
======================================================================
ID Peer Group Priority Admin State Mastership
----------------------------------------------------------------------
1 1 200 Up master
----------------------------------------------------------------------
Multi Active Tunnel Group Entries found: 1
======================================================================
===============================================================================
SeGW-4 is the standby chassis in tunnel group 1 with priority 150:
[/]
A:admin@SeGW-4# show redundancy multi-chassis mc-ipsec peer ip-address 192.0.2.3
===============================================================================
Multi-Chassis MC-IPsec
===============================================================================
Peer Name : (Not Specified)
Peer Addr : 192.0.2.3
Keep Alive Intvl: 1.0 secs Hold on Nbr Fail : 3
Discovery Intvl : 300 secs Discovery Boot Intvl : 300 secs
BFD : Enable
Last update : 02/16/2023 10:10:22
======================================================================
Multi-Chassis IPsec Multi Active Tunnel-Group Table
======================================================================
ID Peer Group Priority Admin State Mastership
----------------------------------------------------------------------
1 1 150 Up standby
----------------------------------------------------------------------
Multi Active Tunnel Group Entries found: 1
======================================================================
===============================================================================
SeGW-3 is the primary node for VRRP instance 10:
[/]
A:admin@SeGW-3# show router vrrp instance
===============================================================================
VRRP Instances
===============================================================================
Interface Name VR Id Own Adm State Base Pri Msg Int
IP Opr Pol Id InUse Pri Inh Int
-------------------------------------------------------------------------------
int-SeGW-3-S-2 10 No Up Master 200 1
IPv4 Up 1 200 No
Backup Addr: 172.16.1.254
-------------------------------------------------------------------------------
Instances : 1
===============================================================================
SeGW-4 is backup for VRRP instance 10:
[/]
A:admin@SeGW-4# show router vrrp instance
===============================================================================
VRRP Instances
===============================================================================
Interface Name VR Id Own Adm State Base Pri Msg Int
IP Opr Pol Id InUse Pri Inh Int
-------------------------------------------------------------------------------
int-SeGW-4-S-2 10 No Up Backup 100 1
IPv4 Up n/a 100 No
Backup Addr: 172.16.1.254
-------------------------------------------------------------------------------
Instances : 1
===============================================================================
Verify the IPSec tunnel on CE-1
The following is verified in this section:
- the connectivity between CE-1 and P-5
- the IPSec tunnel information
A ping command is launched from the loopback interface in VPRN "VPRN-2" on CE-1 to the loopback interface in VPRN "VPRN-2" on P-5:
[/]
A:admin@CE-1# ping 192.168.1.5 router-instance "VPRN-2"
PING 192.168.1.5 56 data bytes
64 bytes from 192.168.1.5: icmp_seq=1 ttl=63 time=2.44ms.
64 bytes from 192.168.1.5: icmp_seq=2 ttl=63 time=2.38ms.
64 bytes from 192.168.1.5: icmp_seq=3 ttl=63 time=2.38ms.
64 bytes from 192.168.1.5: icmp_seq=4 ttl=63 time=2.51ms.
64 bytes from 192.168.1.5: icmp_seq=5 ttl=63 time=2.50ms.
---- 192.168.1.5 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 2.38ms, avg = 2.44ms, max = 2.51ms, stddev = 0.053ms
The following command shows the IPSec tunnel information.
[/]
A:admin@CE-1# show ipsec tunnel
===============================================================================
IPsec Tunnels
===============================================================================
TunnelName LocalAddress SvcId Admn Keying
SapId RemoteAddress DlvrySvcId Oper Sec
Plcy
-------------------------------------------------------------------------------
tunnel-1 10.10.10.1 2 Up Dynamic
tunnel-1.private:1 10.10.20.1 IES-1 Up 1
-------------------------------------------------------------------------------
IPsec Tunnels: 1
===============================================================================
Verify the IPSec tunnel on the SeGWs
In this section, the following is verified:
-
the MCS database is in-sync, so the tunnel status is up on both chassis.
-
P-5 receives two VPN-IPv4 routes for prefix 192.168.1.1/32: the route from SeGW-3 has local preference 200; the route from SeGW-4 has local preference 100.
On both SeGWs, the IPSec tunnel with local address 10.10.20.1 and remote address 10.10.10.1 is up:
[/]
A:admin@SeGW-3# show ipsec tunnel
===============================================================================
IPsec Tunnels
===============================================================================
TunnelName LocalAddress SvcId Admn Keying
SapId RemoteAddress DlvrySvcId Oper Sec
Plcy
-------------------------------------------------------------------------------
tunnel-1 10.10.20.1 2 Up Dynamic
tunnel-1.private:1 10.10.10.1 IES-1 Up 1
-------------------------------------------------------------------------------
IPsec Tunnels: 1
===============================================================================
[/]
A:admin@SeGW-4# show ipsec tunnel
===============================================================================
IPsec Tunnels
===============================================================================
TunnelName LocalAddress SvcId Admn Keying
SapId RemoteAddress DlvrySvcId Oper Sec
Plcy
-------------------------------------------------------------------------------
tunnel-1 10.10.20.1 2 Up Dynamic
tunnel-1.private:1 10.10.10.1 IES-1 Up 1
-------------------------------------------------------------------------------
IPsec Tunnels: 1
===============================================================================
MCS is in sync on both SeGWs:
[/]
A:admin@SeGW-3# show redundancy multi-chassis sync
===============================================================================
Multi-chassis Peer Table
===============================================================================
Peer
-------------------------------------------------------------------------------
Peer IP Address : 192.0.2.4
Description : (Not Specified)
Authentication : Disabled
Source IP Address : 192.0.2.3
Admin State : Enabled
Warm standby : No
Remote warm standby : No
Sub-mgmt options :
DHCP lease threshold : Inactive
Local / Remote : -- / --
-------------------------------------------------------------------------------
Sync-status
-------------------------------------------------------------------------------
Client Applications : IPsec
Sync Admin State : Up
Sync Oper State : Up
Sync Oper Flags :
DB Sync State : inSync
Num Entries : 2
Lcl Deleted Entries : 0
Alarm Entries : 0
OMCR Standby Entries : 0
OMCR Alarm Entries : 0
Rem Num Entries : 2
Rem Lcl Deleted Entries : 0
Rem Alarm Entries : 0
Rem OMCR Standby Entries: 0
Rem OMCR Alarm Entries : 0
===============================================================================
===============================================================================
[/]
A:admin@SeGW-4# show redundancy multi-chassis sync
===============================================================================
Multi-chassis Peer Table
===============================================================================
Peer
-------------------------------------------------------------------------------
Peer IP Address : 192.0.2.3
Description : (Not Specified)
Authentication : Disabled
Source IP Address : 192.0.2.4
Admin State : Enabled
Warm standby : No
Remote warm standby : No
Sub-mgmt options :
DHCP lease threshold : Inactive
Local / Remote : -- / --
-------------------------------------------------------------------------------
Sync-status
-------------------------------------------------------------------------------
Client Applications : IPsec
Sync Admin State : Up
Sync Oper State : Up
Sync Oper Flags :
DB Sync State : inSync
Num Entries : 2
Lcl Deleted Entries : 0
Alarm Entries : 0
OMCR Standby Entries : 0
OMCR Alarm Entries : 0
Rem Num Entries : 2
Rem Lcl Deleted Entries : 0
Rem Alarm Entries : 0
Rem OMCR Standby Entries: 0
Rem OMCR Alarm Entries : 0
===============================================================================
===============================================================================
The following command shows that P-5 received two VPN-IPv4 routes for prefix 192.168.1.1/32: one from SeGW-3 with local preference 200 and one from SeGW-4 with local preference 100:
[/]
A:admin@P-5# show router bgp routes vpn-ipv4
===============================================================================
BGP Router ID:192.0.2.5 AS:64496 Local AS:64496
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP VPN-IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id IGP Cost
As-Path Label
-------------------------------------------------------------------------------
u*>i 64496:2:192.168.1.1/32 200 None
192.0.2.3 None 10
No As-Path 524286
*i 64496:2:192.168.1.1/32 100 None
192.0.2.4 None 10
No As-Path 524286
u*>i 64496:2:192.168.20.0/30 100 None
192.0.2.3 None 10
No As-Path 524286
*>i 64496:2:192.168.20.0/30 100 None
192.0.2.4 None 10
No As-Path 524286
u*>i 64496:2:192.168.20.1/32 100 0
192.0.2.3 None 10
No As-Path 524286
u*>i 64496:2:192.168.20.2/32 100 0
192.0.2.4 None 10
No As-Path 524286
-------------------------------------------------------------------------------
Routes : 6
===============================================================================
MC-IPSec failover scenarios
Two MC-IPSec failover scenarios are described in this section:
- MC-IPSec failover when MS-ISA is disabled
- MC-IPSec failover when the primary chassis SeGW-3 reboots
Failover when MS-ISA is disabled
Initially, MS-ISA is enabled, so SeGW-3 is the primary chassis and SeGW-4 is the standby:
[/]
A:admin@SeGW-3# show redundancy multi-chassis mc-ipsec peer ip-address 192.0.2.4
===============================================================================
Multi-Chassis MC-IPsec
===============================================================================
Peer Name : (Not Specified)
Peer Addr : 192.0.2.4
Keep Alive Intvl: 1.0 secs Hold on Nbr Fail : 3
Discovery Intvl : 300 secs Discovery Boot Intvl : 300 secs
BFD : Enable
Last update : 02/16/2023 10:09:10
======================================================================
Multi-Chassis IPsec Multi Active Tunnel-Group Table
======================================================================
ID Peer Group Priority Admin State Mastership
----------------------------------------------------------------------
1 1 200 Up master
----------------------------------------------------------------------
Multi Active Tunnel Group Entries found: 1
======================================================================
===============================================================================
[/]
A:admin@SeGW-3# show router vrrp instance
===============================================================================
VRRP Instances
===============================================================================
Interface Name VR Id Own Adm State Base Pri Msg Int
IP Opr Pol Id InUse Pri Inh Int
-------------------------------------------------------------------------------
int-SeGW-3-S-2 10 No Up Master 200 1
IPv4 Up 1 200 No
Backup Addr: 172.16.1.254
-------------------------------------------------------------------------------
Instances : 1
===============================================================================
[/]
A:admin@SeGW-4# show redundancy multi-chassis mc-ipsec peer ip-address 192.0.2.3
===============================================================================
Multi-Chassis MC-IPsec
===============================================================================
Peer Name : (Not Specified)
Peer Addr : 192.0.2.3
Keep Alive Intvl: 1.0 secs Hold on Nbr Fail : 3
Discovery Intvl : 300 secs Discovery Boot Intvl : 300 secs
BFD : Enable
Last update : 02/16/2023 10:10:22
======================================================================
Multi-Chassis IPsec Multi Active Tunnel-Group Table
======================================================================
ID Peer Group Priority Admin State Mastership
----------------------------------------------------------------------
1 1 150 Up standby
----------------------------------------------------------------------
Multi Active Tunnel Group Entries found: 1
======================================================================
===============================================================================
[/]
A:admin@SeGW-4# show router vrrp instance
===============================================================================
VRRP Instances
===============================================================================
Interface Name VR Id Own Adm State Base Pri Msg Int
IP Opr Pol Id InUse Pri Inh Int
-------------------------------------------------------------------------------
int-SeGW-4-S-2 10 No Up Backup 100 1
IPv4 Up n/a 100 No
Backup Addr: 172.16.1.254
-------------------------------------------------------------------------------
Instances : 1
===============================================================================
The following command disables the MS-ISA on the primary chassis SeGW-3, which will trigger an MC-IPSec failover.
configure {
card 1 {
mda 2 {
admin-state disable
With MS-ISA disabled, the MC-IPSec state of tunnel group 1 on SeGW-3 becomes notEligible, which means that the tunnel group is down, see the 7450 ESS, 7750 SR, and VSR Multiservice ISA and ESA Guide for details description of MIMP states.:
[/]
A:admin@SeGW-3# show redundancy multi-chassis mc-ipsec peer ip-address 192.0.2.4
===============================================================================
Multi-Chassis MC-IPsec
===============================================================================
Peer Name : (Not Specified)
Peer Addr : 192.0.2.4
Keep Alive Intvl: 1.0 secs Hold on Nbr Fail : 3
Discovery Intvl : 300 secs Discovery Boot Intvl : 300 secs
BFD : Enable
Last update : 02/16/2023 10:09:10
======================================================================
Multi-Chassis IPsec Multi Active Tunnel-Group Table
======================================================================
ID Peer Group Priority Admin State Mastership
----------------------------------------------------------------------
1 1 200 Up notEligible
----------------------------------------------------------------------
Multi Active Tunnel Group Entries found: 1
======================================================================
===============================================================================
SeGW-3 is backup for VRRP instance 10 with in-use priority 50, as per the VRRP policy 1:
[/]
A:admin@SeGW-3# show router vrrp instance
===============================================================================
VRRP Instances
===============================================================================
Interface Name VR Id Own Adm State Base Pri Msg Int
IP Opr Pol Id InUse Pri Inh Int
-------------------------------------------------------------------------------
int-SeGW-3-S-2 10 No Up Backup 200 1
IPv4 Up 1 50 No
Backup Addr: 172.16.1.254
-------------------------------------------------------------------------------
Instances : 1
===============================================================================
SeGW-4 is now the primary chassis in tunnel group 1. This is triggered by MC-IPSec failover, as per the mc-ipsec-non-forwarding event in VRRP policy 1.
[/]
A:admin@SeGW-4# show redundancy multi-chassis mc-ipsec peer ip-address 192.0.2.3
===============================================================================
Multi-Chassis MC-IPsec
===============================================================================
Peer Name : (Not Specified)
Peer Addr : 192.0.2.3
Keep Alive Intvl: 1.0 secs Hold on Nbr Fail : 3
Discovery Intvl : 300 secs Discovery Boot Intvl : 300 secs
BFD : Enable
Last update : 02/16/2023 10:10:22
======================================================================
Multi-Chassis IPsec Multi Active Tunnel-Group Table
======================================================================
ID Peer Group Priority Admin State Mastership
----------------------------------------------------------------------
1 1 150 Up master
----------------------------------------------------------------------
Multi Active Tunnel Group Entries found: 1
======================================================================
===============================================================================
SeGW-4 is primary for VRRP instance 10;
[/]
A:admin@SeGW-4# show router vrrp instance
===============================================================================
VRRP Instances
===============================================================================
Interface Name VR Id Own Adm State Base Pri Msg Int
IP Opr Pol Id InUse Pri Inh Int
-------------------------------------------------------------------------------
int-SeGW-4-S-2 10 No Up Master 100 1
IPv4 Up n/a 100 No
Backup Addr: 172.16.1.254
-------------------------------------------------------------------------------
Instances : 1
===============================================================================
The situation is restored by enabling MS-ISA on SeGW-3:
configure {
card 1 {
mda 2 {
admin-state enable
MC-IPSec failover when primary chassis reboots
The following tools command on SeGW-3 triggers an MC-IPSec switchover:
tools perform redundancy multi-chassis mc-ipsec force-switchover tunnel-group 1
[/]
A:admin@SeGW-3# tools perform redundancy multi-chassis mc-ipsec force-switchover tunnel-group 1
WARNING! Forcing a mastership switchover may significantly impact traffic. Are you sure (y/n)? y
Before the failure condition takes place, SeGW-3 is the primary chassis for tunnel group 1:
[/]
A:admin@SeGW-3# show redundancy multi-chassis mc-ipsec peer ip-address 192.0.2.4
===============================================================================
Multi-Chassis MC-IPsec
===============================================================================
Peer Name : (Not Specified)
Peer Addr : 192.0.2.4
Keep Alive Intvl: 1.0 secs Hold on Nbr Fail : 3
Discovery Intvl : 300 secs Discovery Boot Intvl : 300 secs
BFD : Enable
Last update : 02/16/2023 10:09:10
======================================================================
Multi-Chassis IPsec Multi Active Tunnel-Group Table
======================================================================
ID Peer Group Priority Admin State Mastership
----------------------------------------------------------------------
1 1 200 Up master
----------------------------------------------------------------------
Multi Active Tunnel Group Entries found: 1
======================================================================
===============================================================================
SeGW-3 is primary for VRRP instance 10:
[/]
A:admin@SeGW-3# show router vrrp instance
===============================================================================
VRRP Instances
===============================================================================
Interface Name VR Id Own Adm State Base Pri Msg Int
IP Opr Pol Id InUse Pri Inh Int
-------------------------------------------------------------------------------
int-SeGW-3-S-2 10 No Up Master 200 1
IPv4 Up 1 200 No
Backup Addr: 172.16.1.254
-------------------------------------------------------------------------------
Instances : 1
===============================================================================
SeGW-4 is the standby chassis for tunnel group 1:
[/]
A:admin@SeGW-4# show redundancy multi-chassis mc-ipsec peer ip-address 192.0.2.3
===============================================================================
Multi-Chassis MC-IPsec
===============================================================================
Peer Name : (Not Specified)
Peer Addr : 192.0.2.3
Keep Alive Intvl: 1.0 secs Hold on Nbr Fail : 3
Discovery Intvl : 300 secs Discovery Boot Intvl : 300 secs
BFD : Enable
Last update : 02/16/2023 10:10:22
======================================================================
Multi-Chassis IPsec Multi Active Tunnel-Group Table
======================================================================
ID Peer Group Priority Admin State Mastership
----------------------------------------------------------------------
1 1 150 Up standby
----------------------------------------------------------------------
Multi Active Tunnel Group Entries found: 1
======================================================================
===============================================================================
The VRRP state on SeGW-4 is backup:
[/]
A:admin@SeGW-4# show router vrrp instance
===============================================================================
VRRP Instances
===============================================================================
Interface Name VR Id Own Adm State Base Pri Msg Int
IP Opr Pol Id InUse Pri Inh Int
-------------------------------------------------------------------------------
int-SeGW-4-S-2 10 No Up Backup 100 1
IPv4 Up n/a 100 No
Backup Addr: 172.16.1.254
-------------------------------------------------------------------------------
Instances : 1
===============================================================================
The following command reboots the primary chassis SeGW-3:
[/]
A:admin@SeGW-3# admin reboot card active now
While SeGW-3 reboots, the IPSec state of SeGW-4 becomes eligible:
[/]
A:admin@SeGW-4# show redundancy multi-chassis mc-ipsec peer ip-address 192.0.2.3
===============================================================================
Multi-Chassis MC-IPsec
===============================================================================
Peer Name : (Not Specified)
Peer Addr : 192.0.2.3
Keep Alive Intvl: 1.0 secs Hold on Nbr Fail : 3
Discovery Intvl : 300 secs Discovery Boot Intvl : 300 secs
BFD : Enable
Last update : 02/16/2023 10:10:22
======================================================================
Multi-Chassis IPsec Multi Active Tunnel-Group Table
======================================================================
ID Peer Group Priority Admin State Mastership
----------------------------------------------------------------------
1 1 150 Up eligible
----------------------------------------------------------------------
Multi Active Tunnel Group Entries found: 1
======================================================================
===============================================================================
The VRRP state on SeGW-4 is primary (master):
[/]
A:admin@SeGW-4# show router vrrp instance
===============================================================================
VRRP Instances
===============================================================================
Interface Name VR Id Own Adm State Base Pri Msg Int
IP Opr Pol Id InUse Pri Inh Int
-------------------------------------------------------------------------------
int-SeGW-4-S-2 10 No Up Master 100 1
IPv4 Up n/a 100 No
Backup Addr: 172.16.1.254
-------------------------------------------------------------------------------
Instances : 1
===============================================================================
When SeGW-3 comes up, the IPSec state of tunnel group 1 is discovery, which means that the system has not established the MIMP session with its peer yet.
[/]
A:admin@SeGW-3# show redundancy multi-chassis mc-ipsec peer ip-address 192.0.2.4
===============================================================================
Multi-Chassis MC-IPsec
===============================================================================
Peer Name : (Not Specified)
Peer Addr : 192.0.2.4
Keep Alive Intvl: 1.0 secs Hold on Nbr Fail : 3
Discovery Intvl : 300 secs Discovery Boot Intvl : 300 secs
BFD : Enable
Last update : 02/16/2023 10:24:41
======================================================================
Multi-Chassis IPsec Multi Active Tunnel-Group Table
======================================================================
ID Peer Group Priority Admin State Mastership
----------------------------------------------------------------------
1 1 200 Up discovery
----------------------------------------------------------------------
Multi Active Tunnel Group Entries found: 1
======================================================================
===============================================================================
After a while, the preceding show command is repeated and the IPSec state for tunnel 1 on SeGW-3 is standby:
[/]
A:admin@SeGW-3# show redundancy multi-chassis mc-ipsec peer ip-address 192.0.2.4
===============================================================================
Multi-Chassis MC-IPsec
===============================================================================
Peer Name : (Not Specified)
Peer Addr : 192.0.2.4
Keep Alive Intvl: 1.0 secs Hold on Nbr Fail : 3
Discovery Intvl : 300 secs Discovery Boot Intvl : 300 secs
BFD : Enable
Last update : 02/16/2023 10:24:41
======================================================================
Multi-Chassis IPsec Multi Active Tunnel-Group Table
======================================================================
ID Peer Group Priority Admin State Mastership
----------------------------------------------------------------------
1 1 200 Up standby
----------------------------------------------------------------------
Multi Active Tunnel Group Entries found: 1
======================================================================
===============================================================================
The VRRP state on SeGW-3 is backup:
[/]
A:admin@SeGW-3# show router vrrp instance
===============================================================================
VRRP Instances
===============================================================================
Interface Name VR Id Own Adm State Base Pri Msg Int
IP Opr Pol Id InUse Pri Inh Int
-------------------------------------------------------------------------------
int-SeGW-3-S-2 10 No Up Backup 200 1
IPv4 Up 1 50 No
Backup Addr: 172.16.1.254
-------------------------------------------------------------------------------
Instances : 1
===============================================================================
SeGW-4 is the primary chassis in MC-IPSec tunnel group 1:
[/]
A:admin@SeGW-4# show redundancy multi-chassis mc-ipsec peer ip-address 192.0.2.3
===============================================================================
Multi-Chassis MC-IPsec
===============================================================================
Peer Name : (Not Specified)
Peer Addr : 192.0.2.3
Keep Alive Intvl: 1.0 secs Hold on Nbr Fail : 3
Discovery Intvl : 300 secs Discovery Boot Intvl : 300 secs
BFD : Enable
Last update : 02/16/2023 10:10:22
======================================================================
Multi-Chassis IPsec Multi Active Tunnel-Group Table
======================================================================
ID Peer Group Priority Admin State Mastership
----------------------------------------------------------------------
1 1 150 Up master
----------------------------------------------------------------------
Multi Active Tunnel Group Entries found: 1
======================================================================
===============================================================================
SeGW-4 is the primary node for VRRP instance 10:
[/]
A:admin@SeGW-4# show router vrrp instance
===============================================================================
VRRP Instances
===============================================================================
Interface Name VR Id Own Adm State Base Pri Msg Int
IP Opr Pol Id InUse Pri Inh Int
-------------------------------------------------------------------------------
int-SeGW-4-S-2 10 No Up Master 100 1
IPv4 Up n/a 100 No
Backup Addr: 172.16.1.254
-------------------------------------------------------------------------------
Instances : 1
===============================================================================
Configuration guidelines
The following is a list of guidelines for configuring MC-IPSec:
-
To avoid high CPU load and issues in some complex cases, the following are suggestions for configuring the IKEv2 lifetime:
-
Both IKE_SA and CHILD_SA lifetime on MC-IPSec chassis (SeGW-3 and SeGW-4) should be around three times larger than on the IPSec peer CE-1.
-
With the first rule, the lifetime of the side with smaller lifetime (IPSec peer CE-1) should not be too small (these being the default values):
-
IKE_SA: >= 86400 seconds
-
CHILD_SA: >= 3600 seconds
-
-
With the first rule, on the side with smaller lifetime (IPSec peer CE-1), the IKE_SA lifetime must be at least 3 times larger than CHILD_SA lifetime.
-
-
The IKE protocol is the control plane of IPSec, so IKE packets must be treated as high QoS priority in the end-to-end path of the public service. On the public interface, a SAP ingress QoS policy must be configured to ensure that IKE packets get high QoS priority.
-
Configure ipsec-responder-only true under tunnel-group for static LAN-to-LAN tunnels.
-
Enable dead peer detection (DPD) on the IPSec peer side (CE-1); disable DPD (default) on the MC-IPSec chassis side.
-
The direct and redundant physical link between MC-IPSec chassis must be configured with sufficient bandwidth for MCS and shunting traffic, and proper QoS configuration to make sure the MIMP and MCS packets are treated as high priority traffic.
-
The system time must be same on both MC-IPSec chassis.
-
Make sure the protection status is nominal on both chassis before provoking a controlled switchover. The protection status can be displayed with the show redundancy multi-chassis mc-ipsec peer ip-address <addr> command.
-
Wait at least five minutes between two consecutive switchovers if possible, to prevent a second switchover happening before the standby is ready to become the primary chassis.
Conclusion
MC-IPSec provides a stateful multi-chassis IPSec redundancy solution. This is very important in a carrier grade network, especially in applications such as mobile backhaul where high value mobile services run over IPSec tunnels.