Mobility for EVPN Hosts Within an R-VPLS
This chapter provides information about Mobility for EVPN Hosts Within an R-VPLS.
Topics in this chapter include:
Applicability
The information and configuration in this chapter are based on SR OS Release 21.10.R2. Efficient EVPN host mobility without tromboning or hairpinning in an R-VPLS is supported for IPv4 in SR OS Release 19.10.R3 and later and is supported for IPv6 in SR OS Release 20.5.R1 and later.
Overview
SR OS can populate a VPRN route table with host routes learned from the IPv4 Address Resolution Protocol (ARP) messages or IPv6 Neighbor Discovery (ND) protocol messages. The host routes can be advertised in the VPRN context as IP-VPN or EVPN route type 5 (RT5), to be used by an IP-VPN or EVPN core network for inter-subnet forwarding. SR OS supports draft-ietf-bess-evpn-inter-subnet-forwarding for a dynamic and efficient routing between remote hosts, avoiding hairpinning.
In SR OS releases earlier than Release 19.10.R3, inefficient hairpinning situations may occur when the VPRN is configured to advertise IPv4 host routes as IP-VPN or EVPN RT5 routes. Hairpinning in a broadcast domain after switchover for SR OS releases earlier than Release 19.10.R3 shows hairpinning in an EVPN broadcast domain with PE-1, PE-2, and PE-3.
When host-100 comes up, it sends a Gratuitous Address Resolution Protocol (GARP) message that is then learned on PE-2 and PE-3. PE-2 and PE-3 are configured to advertise host routes, so they generate an RT5 host route for prefix 10.0.0.100/32 of host-100. PE-3 selects its best RT5 for 10.0.0.100/32 and traffic from host-191 to host-100 uses the path via PE-1, PE-2, PE-4, and PE-6. However, when the active path between PE-4 and PE-6 fails, the standby path between PE-5 and PE-6 takes over and hairpinning occurs when PE-1 continues selecting PE-2 as the next hop, while a more efficient path is possible via next-hop PE-3. Traffic from host-191 to host-100 uses the path via PE-1, PE-2, PE-3, PE-5, and PE-6.
In SR OS Release 19.10.R3 and later, the more efficient path from host-191 via PE-1, PE-3, PE-5, and PE-6 to host-100 is used, as shown in Forwarding in a broadcast domain after switchover for SR OS Release 19.10.R3 and later.
In SR OS Release 19.10.R3 and later, EVPN host mobility is supported for IPv4 as described in section "Symmetric and Asymmetric IRB" of draft-ietf-bess-evpn-inter-subnet-forwarding. When a host moves from a source PE to a target PE in the same broadcast domain, the behavior for IPv4 hosts is one of the following.
-
The host initiates an ARP request or GARP.
-
The host sends a data packet without first initiating an ARP request or GARP.
-
The host does not send any traffic and the source PE generates an ARP request when the MAC address of the host expires and the EVPN-MAC is withdrawn.
All three scenarios are described in more detail later, where the move of host-100 from source PE-2 to target PE-3 is simulated.
For the first of these scenarios, the VPRN configuration on PE-2 is as follows:
# on PE-2:
configure {
service {
vprn "ip-vrf-16" {
admin-state enable
service-id 16
customer "1"
interface "evi-15" {
mac 00:00:00:00:00:02
vpls "sbd-15" {
evpn-tunnel {
}
}
}
interface "evi-17" {
mac 00:00:00:00:2f:17
ipv4 {
primary {
address 10.0.0.2
prefix-length 24
}
neighbor-discovery {
timeout 300
learn-unsolicited true
proactive-refresh true
local-proxy-arp true
host-route {
populate dynamic {
}
}
}
vrrp 1 {
backup [10.0.0.254]
passive true
ping-reply true
traceroute-reply true
}
}
vpls "evi-17" {
evpn {
arp {
learn-dynamic false
flood-garp-and-unknown-req false
advertise dynamic {
}
}
}
}
}
}
For IPv4, the behavior is controlled by the following commands.
-
ipv4>neighbor-discovery>host-route>populate [dynamic | evpn | static] configures PE-2 to advertise host routes. The type of ARP entry that can create a host route can be dynamic, EVPN, static, or a combination of these.
-
ipv4>neighbor-discovery>learn-unsolicited triggers the learning of an ARP entry upon receiving an ARP or GARP message that was not requested by the router.
-
ipv4>neighbor-discovery>proactive-refresh triggers the refresh of the ARP entry 30 seconds before aging out.
-
ipv4>neighbor-discovery>local-proxy-arp ensures that PE-2 replies to any received ARP request on behalf of the other hosts in the R-VPLS broadcast domain.
-
vpls>evpn>arp>learn-dynamic controls whether data path ARP messages received on EVPN connections can populate the ARP tables.
-
vpls>evpn>arp>flood-garp-and-unknown-req controls the flooding of Control Processing Module (CPM)-generated ARP requests to EVPN destinations.
-
vpls>evpn>arp>advertise [dynamic | static] enables PE-2 to advertise MAC and IP in EVPN-MAC routes for ARP entries of the dynamic or static type.
For IPv6, the corresponding commands are as follows:
-
ipv6>neighbor-discovery>host-route>populate [dynamic | evpn | static]
-
ipv6>neighbor-discovery>learn-unsolicited [global | link-local | both]
-
ipv6>neighbor-discovery>proactive-refresh [global | link-local | both] triggers the refresh of the ND entry upon aging out.
-
ipv6>neighbor-discovery>local-proxy-nd
-
vpls>evpn>nd>learn-dynamic
-
vpls>evpn>nd>advertise [dynamic | static]
For IPv6, CPM-generated Neighbor Solicitation (NS) messages are always flooded to EVPN destinations. This is not configurable in the vpls>evpn>nd context of the VPRN service, in contrast to the flood-garp-and-unknown-req command in the vpls>evpn>arp context for IPv4.
The behavior for IPv6 hosts when moving from a source PE to a target PE is one of the following.
-
The host initiates an unsolicited Neighbor Advertisement (NA).
-
The host sends traffic, without first initiating NA or NS messages.
-
The host does not send any traffic, and the source PE generates an NS message when the MAC address of the host expires and the EVPN-MAC is withdrawn.
All three scenarios are described in more detail later, where the move of host-66 from source PE-2 to target PE-3 is simulated.
Configuration
Example topology with system IP addresses shows the example topology with PE-1, PE-2, and PE-3 in an EVPN-MPLS network and PE-4, PE-5, and PE-6 in an MPLS network.
The initial configuration includes:
-
Cards, MDAs, ports
-
Router interfaces
-
IS-IS between PE-1, PE-2, PE-3 and between PE-4, PE-5, and PE-6
-
LDP between PE-1, PE-2, PE-3 and between PE-4, PE-5, and PE-6
-
BGP configured for the EVPN address family on PE-1, PE-2, and PE-3
On PE-1, BGP is configured as follows:
# on PE-1:
configure {
router "Base" {
autonomous-system 64500
bgp {
vpn-apply-export true
vpn-apply-import true
rapid-withdrawal true
family {
ipv4 false
evpn true
}
rapid-update {
evpn true
}
group "dc" {
type internal
}
neighbor "192.0.2.2" {
group "dc"
}
neighbor "192.0.2.3" {
group "dc"
}
}
The BGP configuration is similar on PE-2 and PE-3.
IPv4 host mobility
The following use cases for IPv4 host mobility are described:
-
Host initiates ARP request or GARP after moving
-
Host initiates non-ARP traffic after moving
-
Host does not send any traffic after moving
IPv4 host mobility case 1: host initiates ARP request or GARP after moving
The service configuration on PE-1 is as follows:
# on PE-1:
configure {
service {
vpls "sbd-15" {
admin-state enable
description "R-VPLS 15"
service-id 15
customer "1"
routed-vpls {
}
bgp 1 {
route-distinguisher "192.0.2.1:15"
}
bgp-evpn {
evi 15
routes {
ip-prefix {
advertise true
}
}
mpls 1 {
admin-state enable
auto-bind-tunnel {
resolution any
}
}
}
}
vprn "ip-vrf-16" {
admin-state enable
service-id 16
customer "1"
interface "evi-15" {
mac 00:00:00:00:00:01
vpls "sbd-15" {
evpn-tunnel {
}
}
}
interface "evi-20" {
mac 00:00:00:00:1e:20
ipv4 {
primary {
address 10.0.20.1
prefix-length 24
}
}
vpls "evi-20" {
}
}
}
vpls "evi-20" {
admin-state enable
description "R-VPLS 20"
service-id 20
customer "1"
routed-vpls {
}
sap pxc-10.a:20 {
}
}
VPRN "ip-vrf-16" has two interfaces: interface "evi-15" toward R-VPLS "sbd-15" and interface "evi-20" toward R-VPLS "evi-20". Host-191 is connected to interface "evi-20" of R-VPLS "evi-20".
PE-2 and PE-3 are configured with an anycast gateway, that is, a VRRP passive instance with the same backup IP address 10.0.0.254 on interface "evi-17" in VPRN "ip-vrf-16". The MAC address under VRRP is by default derived from the Virtual Router ID (VRID), so both PE-2 and PE-3 get MAC address 00:00:5E:00:01:01. The service configuration on PE-2 and PE-3 is similar.
# on PE-2:
configure {
service {
vpls "evi-17" {
admin-state enable
description "R-VPLS 17"
service-id 17
customer "1"
routed-vpls {
}
bgp 1 {
route-distinguisher "192.0.2.2:17" # on PE-3: 192.0.2.3:17
}
bgp-evpn {
evi 17
mpls 1 {
admin-state enable
auto-bind-tunnel {
resolution any
}
}
}
sap 1/1/1:17 { # on PE-3: sap 1/1/2:17
}
}
vpls "sbd-15" {
admin-state enable
description "R-VPLS 15"
service-id 15
customer "1"
routed-vpls {
}
bgp 1 {
route-distinguisher "192.0.2.2:15" # on PE-3: 192.0.2.3:15
}
bgp-evpn {
evi 15
routes {
ip-prefix {
advertise true
}
}
mpls 1 {
admin-state enable
auto-bind-tunnel {
resolution any
}
}
}
}
vprn "ip-vrf-16" {
admin-state enable
service-id 16
customer "1"
interface "evi-15" {
mac 00:00:00:00:00:02 # on PE-3: 00:00:00:00:00:03
vpls "sbd-15" {
evpn-tunnel {
}
}
}
interface "evi-17" {
mac 00:00:00:00:2f:17 # on PE-3: 00:00:00:00:3f:17
ipv4 {
primary {
address 10.0.0.2 # on PE-3: 10.0.0.3
prefix-length 24
}
neighbor-discovery {
timeout 300
learn-unsolicited true
proactive-refresh true
local-proxy-arp true
host-route {
populate dynamic {
}
}
}
vrrp 1 {
backup [10.0.0.254] # anycast IP address on PE-2, PE-3
passive true
ping-reply true
traceroute-reply true
}
}
vpls "evi-17" {
evpn {
arp {
learn-dynamic false
flood-garp-and-unknown-req false
advertise dynamic {
}
}
}
}
}
}
The ipv4>neighbor-discovery>host-route>populate dynamic ensures that route-table ARP-ND host routes are created for dynamic entries, not for static or EVPN entries. The learn-dynamic false command prevents PE-2 and PE-3 from learning ARP entries from ARP messages received on an EVPN destination. The flood-garp-and-unknown-req false command suppresses CPM-generated ARP to reduce unnecessary ARP flooding.
In this sample topology, an Epipe is used where a failover from the primary to the secondary path simulates a move of host-100 from PE-2 to PE-3. SAP 1/1/1:17 in R-VPLS "evi-17" on PE-2 is connected to a SAP of Epipe 17 on PE-4; SAP 1/1/2:17 in R-VPLS "evi-17" on PE-3 to a SAP of Epipe 17 on PE-5. The service configuration on PE-4 is as follows. The configuration on PE-5 is similar.
# on PE-4:
configure {
service {
oper-group "op-grp-1" {
}
epipe "Epipe 17" {
admin-state enable
service-id 17
customer "1"
spoke-sdp 46:17 {
oper-group "op-grp-1"
}
sap 1/1/2:17 {
description "SAP connected to SAP 1/1/1:17 on PE-2"
monitor-oper-group "op-grp-1"
}
}
sdp 46 {
admin-state enable
delivery-type mpls
ldp true
far-end {
ip-address 192.0.2.6
}
}
On PE-6, the service configuration is as follows:
# on PE-6:
configure {
service {
epipe "Epipe 17" {
admin-state enable
service-id 17
customer "1"
endpoint "EP17" {
}
spoke-sdp 64:17 {
endpoint {
name "EP17"
precedence primary
}
}
spoke-sdp 65:17 {
endpoint {
name "EP17"
}
}
sap 1/2/1:17 {
}
}
sdp 64 {
admin-state enable
delivery-type mpls
ldp true
far-end {
ip-address 192.0.2.4
}
}
sdp 65 {
admin-state enable
delivery-type mpls
ldp true
far-end {
ip-address 192.0.2.5
}
}
Host-100 is connected to SAP 1/2/1:17 in Epipe 17.
On PE-2 and PE-3, debugging is enabled (in classic CLI):
# on PE-2, PE-3:
debug
router "Base"
bgp
update
exit
exit
router service-name "ip-vrf-16"
ip
arp
route-table
exit
exit
exit
Initial phase
Initial situation with forwarding path via PE-2 shows that traffic from host-191 to host-100 is forwarded via PE-1, PE-2, PE-4, and PE-6.
Host-191 sends a traceroute to host-100 via PE-2 (10.0.0.2):
[/]
A:admin@PE-1# traceroute 10.0.0.100 router-instance "H-191" source-address 10.0.20.191
traceroute to 10.0.0.100 from 10.0.20.191, 30 hops max, 40 byte packets
1 10.0.20.1 (10.0.20.1) 3.96 ms 2.27 ms 2.07 ms
2 10.0.0.2 (10.0.0.2) 2.96 ms 2.95 ms 2.82 ms
3 10.0.0.100 (10.0.0.100) 10.9 ms 5.54 ms 5.07 ms
The ARP table for VPRN "ip-vrf-16" on PE-2 shows that IP address 10.0.0.100 corresponds to MAC address 00:00:00:00:01:00 and is learned dynamically:
[/]
A:admin@PE-2# show router 16 arp 10.0.0.100
===============================================================================
ARP Table (Service: 16)
===============================================================================
IP Address MAC Address Expiry Type Interface
-------------------------------------------------------------------------------
10.0.0.100 00:00:00:00:01:00 00h04m49s Dyn[I] evi-17
===============================================================================
The ARP table for VPRN "ip-vrf-16" on PE-3 shows that IP address 10.0.0.100 is advertised through EVPN:
[/]
A:admin@PE-3# show router 16 arp 10.0.0.100
===============================================================================
ARP Table (Service: 16)
===============================================================================
IP Address MAC Address Expiry Type Interface
-------------------------------------------------------------------------------
10.0.0.100 00:00:00:00:01:00 00h00m00s Evp[I] evi-17
===============================================================================
On PE-2, MAC address 00:00:00:00:01:00 is learned on SAP 1/1/1:17 in R-VPLS "evi-17":
[/]
A:admin@PE-2# show service id 17 fdb detail
===============================================================================
Forwarding Database, Service 17
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
17 00:00:00:00:01:00 sap:1/1/1:17 L/30 01/28/22 12:45:29
17 00:00:00:00:2e:17 cpm Intf 01/28/22 12:43:20
17 00:00:00:00:3f:17 mpls-1: EvpnS:P 01/28/22 12:43:29
192.0.2.3:524283
ldp:65538
17 00:00:5e:00:01:01 cpm Intf 01/28/22 12:43:20
-------------------------------------------------------------------------------
No. of MAC Entries: 4
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
On PE-3, the FDB for R-VPLS "evi-17" shows that MAC address 00:00:00:00:01:00 is advertised through EVPN:
[/]
A:admin@PE-3# show service id 17 fdb mac 00:00:00:00:01:00
===============================================================================
Forwarding Database, Service 17
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
17 00:00:00:00:01:00 mpls-1: Evpn 01/28/22 12:45:29
192.0.2.2:524283
ldp:65537
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
The route table for VPRN "ip-vrf-16" on PE-2 shows an ARP-ND host route with preference 1 for prefix 10.0.0.100/32:
[/]
A:admin@PE-2# show router 16 route-table 10.0.0.100
===============================================================================
Route Table (Service: 16)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
10.0.0.100/32 Remote ARP-ND 00h02m44s 1
10.0.0.100 0
-------------------------------------------------------------------------------
No. of Routes: 1
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
The route table for VPRN "ip-vrf-16" on PE-3 shows an EVPN Interface-ful (EVPN-IFF) host route for prefix 10.0.0.100/32 with preference 169:
[/]
A:admin@PE-3# show router 16 route-table 10.0.0.100
===============================================================================
Route Table (Service: 16)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
10.0.0.100/32 Remote EVPN-IFF 00h02m43s 169
evi-15 (ET-00:00:00:00:00:02) 0
-------------------------------------------------------------------------------
No. of Routes: 1
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
PE-3 receives the IP and MAC addresses of host-100 as EVPN type. PE-3 must not learn these IP and MAC addresses as dynamic because PE-3 must be prevented from advertising an RT5 route. If PE-3 advertised prefix 10.0.0.100, then PE-1 could select PE-3 as next hop to reach host-100, causing an undesired hairpinning forwarding behavior.
Host-100 sends an ARP request or GARP after moving
Host-100 sends an ARP request or GARP after switchover shows a switchover from the active to the standby path where host-100 sends an ARP request or GARP and its IP and MAC addresses are learned on PE-3 instead of PE-2. The failure is simulated by disabling the SDP from PE-4 to PE-6.
Due to the SDP failure on PE-4, the initial path can no longer be used. Host-100 sends an ARP request or GARP with its IP and MAC addresses. In the following example, PE-3 receives the following ARP request and replies to it:
1 2022/01/28 12:49:45.684 UTC MINOR: DEBUG #2001 vprn16 PIP
"PIP: ARP
instance 2 (16), interface index 6 (evi-17),
ARP ingressing on evi-17
Who has 10.0.0.254 ? Tell 10.0.0.100
"
2 2022/01/28 12:49:45.684 UTC MINOR: DEBUG #2001 vprn16 PIP
"PIP: ARP
instance 2 (16), interface index 6 (evi-17),
ARP egressing on evi-17
10.0.0.254 is at 00:00:5e:00:01:01
"
The Route Table Manager (RTM) for prefix 10.0.0.100 in VPRN "ip-vrf-16" is modified with preference 1 and owner ARP-ND. This behavior is due to the ipv4>neighbor-discovery>host-route>populate dynamic command.
3 2022/01/28 12:49:45.684 UTC MINOR: DEBUG #2001 vprn16 PIP
"PIP: ROUTE
instance 2 (16), RTM MODIFY event
New Route Info
prefix: 10.0.0.100/32 (0x119549018) preference: 1 metric: 0
backup metric: 0 owner: ARP-ND ownerId: 0
1 ecmp hops 0 backup hops:
hop 0: 10.0.0.100 @ if 6, weight 0
"
PE-3 sends an RT5 for prefix 10.0.0.100/32 to PE-1 and PE-2:
4 2022/01/28 12:49:45.685 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.2
"Peer 1: 192.0.2.2: UPDATE
Peer 1: 192.0.2.2 - Send BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 90
Flag: 0x90 Type: 14 Len: 45 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.3
Type: EVPN-IP-PREFIX Len: 34 RD: 192.0.2.3:15, tag: 0,
ip_prefix: 10.0.0.100/32 gw_ip 0.0.0.0
Label: 8388544 (Raw Label: 0x7fffc0)
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:15
mac-nh:00:00:00:00:00:03
bgp-tunnel-encap:MPLS
"
PE-3 sends EVPN-MAC routes for MAC 00:00:00:00:01:00 with an increased sequence number for MAC mobility: one EVPN-MAC route with MAC address 00:00:00:00:01:00 and IP address 10.0.0.100 and another EVPN-MAC route with MAC address 00:00:00:00:01:00 only and a null IP address.
5 2022/01/28 12:49:45.685 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.2
"Peer 1: 192.0.2.2: UPDATE
Peer 1: 192.0.2.2 - Send BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 128
Flag: 0x90 Type: 14 Len: 83 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.3
Type: EVPN-MAC Len: 37 RD: 192.0.2.3:17 ESI: ESI-0, tag: 0, mac len: 48
mac: 00:00:00:00:01:00, IP len: 4, IP: 10.0.0.100, label1: 8388528
Type: EVPN-MAC Len: 33 RD: 192.0.2.3:17 ESI: ESI-0, tag: 0, mac len: 48
mac: 00:00:00:00:01:00, IP len: 0, IP: NULL, label1: 8388528
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:17
bgp-tunnel-encap:MPLS
mac-mobility:Seq:1
"
The FDB for R-VPLS "evi-17" shows that MAC address 00:00:00:00:01:00 is dynamically learned on SAP 1/1/2:17 on PE-3:
[/]
A:admin@PE-3# show service id 17 fdb mac 00:00:00:00:01:00
===============================================================================
Forwarding Database, Service 17
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
17 00:00:00:00:01:00 sap:1/1/2:17 L/14 01/28/22 12:49:46
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
On PE-2, the FDB for R-VPLS "evi-17" is updated and PE-2 withdraws its EVPN-MAC route based on the higher sequence number of the received EVPN-MAC route for MAC address 00:00:00:00:01:00 with next hop 192.0.2.3:
[/]
A:admin@PE-2# show service id 17 fdb mac 00:00:00:00:01:00
===============================================================================
Forwarding Database, Service 17
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
17 00:00:00:00:01:00 mpls-1: Evpn 01/28/22 12:49:46
192.0.2.3:524283
ldp:65538
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
On PE-3, the ARP table for VPRN "ip-vrf-16" shows that IP address 10.0.0.100 is learned dynamically on interface "evi-17":
[/]
A:admin@PE-3# show router 16 arp 10.0.0.100
===============================================================================
ARP Table (Service: 16)
===============================================================================
IP Address MAC Address Expiry Type Interface
-------------------------------------------------------------------------------
10.0.0.100 00:00:00:00:01:00 00h04m40s Dyn[I] evi-17
===============================================================================
On PE-2, the ARP table for VPRN "ip-vrf-16" shows that the entry for IP address 10.0.0.100 is updated from dynamic to type EVPN:
[/]
A:admin@PE-2# show router 16 arp 10.0.0.100
===============================================================================
ARP Table (Service: 16)
===============================================================================
IP Address MAC Address Expiry Type Interface
-------------------------------------------------------------------------------
10.0.0.100 00:00:00:00:01:00 00h00m00s Evp[I] evi-17
===============================================================================
An ARP entry's change from dynamic to EVPN triggers a CPM-generated ARP request from PE-2, but the configured flood-garp-and-unknown-req false command prevents PE-2 from flooding the ARP request to EVPN destinations such as PE-3.
On PE-3, the route table for VPRN "ip-vrf-16" shows an ARP-ND host route for prefix 10.0.0.100:
[/]
A:admin@PE-3# show router 16 route-table 10.0.0.100
===============================================================================
Route Table (Service: 16)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
10.0.0.100/32 Remote ARP-ND 00h01m32s 1
10.0.0.100 0
-------------------------------------------------------------------------------
No. of Routes: 1
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
The route table for VPRN "ip-vrf-16" for prefix 10.0.0.100 shows that PE-2 removed its ARP-ND host route and the received EVPN route from PE-3 is used instead:
[/]
A:admin@PE-2# show router 16 route-table 10.0.0.100
===============================================================================
Route Table (Service: 16)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
10.0.0.100/32 Remote EVPN-IFF 00h01m31s 169
evi-15 (ET-00:00:00:00:00:03) 0
-------------------------------------------------------------------------------
No. of Routes: 1
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
IPv4 host mobility case 2: host sends traffic without first initiating an ARP request or GARP after moving
In use cases 2 and 3, the configuration of VPRN "ip-vrf-16" is modified on PE-2 and PE-3. The only difference from case 1 is that the flood-garp-and-unknown-req is configured, which is the default setting. The VPRN configuration on PE-2 is as follows:
# on PE-2:
configure {
service {
vprn "ip-vrf-16" {
admin-state enable
service-id 16
customer "1"
interface "evi-15" {
mac 00:00:00:00:00:02
vpls "sbd-15" {
evpn-tunnel {
}
}
}
interface "evi-17" {
mac 00:00:00:00:2f:17
ipv4 {
primary {
address 10.0.0.2
prefix-length 24
}
neighbor-discovery {
timeout 300
learn-unsolicited true
proactive-refresh true
local-proxy-arp true
host-route {
populate dynamic {
}
}
}
vrrp 1 {
backup [10.0.0.254]
passive true
ping-reply true
traceroute-reply true
}
}
vpls "evi-17" {
evpn {
arp {
learn-dynamic false
flood-garp-and-unknown-req true # default behavior
advertise dynamic {
}
}
}
}
}
Initial forwarding path
The initial forwarding path via PE-2 is restored by enabling the SDP from PE-4 to PE-6. The route table for VPRN "ip-vrf-16" on PE-2 shows the following ARP-ND host route for prefix 10.0.0.100/32:
[/]
A:admin@PE-2# show router 16 route-table 10.0.0.100
===============================================================================
Route Table (Service: 16)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
10.0.0.100/32 Remote ARP-ND 00h03m46s 1
10.0.0.100 0
-------------------------------------------------------------------------------
No. of Routes: 1
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
Host-100 generates non-ARP traffic after moving
On PE-4, the SDP from PE-4 to PE-6 is disabled, causing a switchover to the standby path. Host sends non-ARP frame after switchover shows the path after switchover. Host-100 generates non-ARP traffic after moving.
Host-100 sends a non-ARP frame with MAC source address 00:00:00:00:01:00 to host-191. The following steps occur:
-
PE-3 receives this frame with MAC 00:00:00:00:01:00 and updates its FDB.
-
PE-3 advertises an EVPN-MAC route for MAC 00:00:00:00:01:00 (with a null IP address) with a higher sequence number.
-
PE-2 receives this EVPN MAC route, updates its FDB and withdraws its EVPN-MAC routes for MAC 00:00:00:00:01:00.
-
The FDB update for MAC 00:00:00:00:01:00 triggers PE-2 to send an ARP request for MAC 00:00:00:00:01:00.
-
PE-2 is configured with flood-garp-and-unknown-req, so the ARP request is flooded to the EVPN destinations PE-1 and PE-3. PE-3 floods this ARP request to its SAPs and SDP-bindings; in this case, to SAP 1/1/2:17.
-
When the ARP request reaches host-100, it sends an ARP reply to the anycast IP address 10.0.0.254. This ARP reply is received by PE-3.
-
When PE-3 receives the ARP reply, it updates the ARP entry for 10.0.0.100 to type dynamic instead of type EVPN.
-
PE-3 is configured with populate dynamic, so it advertises an RT5 for prefix 10.0.0.100/32. Also, MAC 00:00:00:00:01:00 is now learned in ARP as local, so PE-3 sends an EVPN-MAC route with MAC 00:00:00:00:01:00 and IP prefix 10.0.0.100.
-
PE-2 receives the EVPN routes and updates the ARP entry for prefix 10.0.0.100 from type dynamic to type EVPN. PE-2 also removes its ARP-ND host route from the route table and withdraws its RT5 for prefix 10.0.0.100/32.
On PE-3, the route for prefix 10.0.0.100/32 is an ARP-ND host route:
[/]
A:admin@PE-3# show router 16 route-table 10.0.0.100
===============================================================================
Route Table (Service: 16)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
10.0.0.100/32 Remote ARP-ND 00h01m05s 1
10.0.0.100 0
-------------------------------------------------------------------------------
No. of Routes: 1
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
IPv4 host mobility case 3: host does not send any traffic after moving
The service configuration on PE-2 and PE-3 remains the same as in use case 2.
The forwarding path is restored by enabling the SDP from PE-4 to PE-6, so the initial situation is the same as in the preceding cases. PE-2 learns MAC address 00:00:00:00:01:00 on its local SAP 1/1/1:17, as follows:
[/]
A:admin@PE-2# show service id 17 fdb detail
===============================================================================
Forwarding Database, Service 17
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
17 00:00:00:00:01:00 sap:1/1/1:17 L/0 01/28/22 13:10:34
17 00:00:00:00:2f:17 cpm Intf 01/28/22 12:53:11
17 00:00:00:00:3f:17 mpls-1: EvpnS:P 01/28/22 12:43:29
192.0.2.3:524283
ldp:65538
17 00:00:5e:00:01:01 cpm Intf 01/28/22 12:43:20
-------------------------------------------------------------------------------
No. of MAC Entries: 4
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
The SDP from PE-4 to PE-6 is disabled and host-100 does not send any traffic, as shown in Host does not send any traffic after switchover.
The following steps occur:
-
When MAC 00:00:00:00:01:00 ages out in the FDB of R-VPLS 17 on PE-2, PE-2 withdraws the EVPN-MAC routes for MAC 00:00:00:00:01:00. The update for MAC 00:00:00:00:01:00 triggers PE-2 to send an ARP request for 10.0.0.100.
# on PE-2: 99 2022/01/28 13:12:04.028 UTC MINOR: DEBUG #2001 vprn16 PIP "PIP: ARP instance 2 (16), interface index 6 (evi-17), ARP egressing on evi-17 Who has 10.0.0.100 ? Tell 10.0.0.254 "101 2022/01/28 13:12:04.028 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.3 "Peer 1: 192.0.2.3: UPDATE Peer 1: 192.0.2.3 - Send BGP UPDATE: Withdrawn Length = 0 Total Path Attr Length = 42 Flag: 0x90 Type: 15 Len: 38 Multiprotocol Unreachable NLRI: Address Family EVPN Type: EVPN-MAC Len: 33 RD: 192.0.2.2:17 ESI: ESI-0, tag: 0, mac len: 48 mac: 00:00:00:00:01:00, IP len: 0, IP: NULL, label1: 0 " -
PE-2 is configured with flood-garp-and-unknown-req true. PE-2 floods the CPM-generated ARP request to PE-3. PE-3 forwards the ARP request to host-100.
-
Host-100 sends an ARP reply that is received by PE-3. PE-3 updates its FDB and ARP tables.
-
The FDB update on PE-3 makes PE-3 advertise an EVPN-MAC route for MAC 00:00:00:00:01:00 (with a null IP address). The ARP update makes PE-3 advertise an EVPN-MAC route with MAC 00:00:00:00:01:00 and IP prefix 10.0.0.100. PE-2 receives two EVPN-MAC routes from PE-3:
103 2022/01/28 13:12:04.033 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.3 "Peer 1: 192.0.2.3: UPDATE Peer 1: 192.0.2.3 - Received BGP UPDATE: Withdrawn Length = 0 Total Path Attr Length = 128 Flag: 0x90 Type: 14 Len: 83 Multiprotocol Reachable NLRI: Address Family EVPN NextHop len 4 NextHop 192.0.2.3 Type: EVPN-MAC Len: 37 RD: 192.0.2.3:17 ESI: ESI-0, tag: 0, mac len: 48 mac: 00:00:00:00:01:00, IP len: 4, IP: 10.0.0.100, label1: 8388528 Type: EVPN-MAC Len: 33 RD: 192.0.2.3:17 ESI: ESI-0, tag: 0, mac len: 48 mac: 00:00:00:00:01:00, IP len: 0, IP: NULL, label1: 8388528 Flag: 0x40 Type: 1 Len: 1 Origin: 0 Flag: 0x40 Type: 2 Len: 0 AS Path: Flag: 0x40 Type: 5 Len: 4 Local Preference: 100 Flag: 0xc0 Type: 16 Len: 24 Extended Community: target:64500:17 bgp-tunnel-encap:MPLS mac-mobility:Seq:5 " -
PE-3 is configured with populate dynamic, so it advertises an RT5 for prefix 10.0.0.100/32. In the route table for VPRN "ip-vrf-16", the route for IP prefix 10.0.0.100/32 is ARP-ND host route. PE-2 receives the following RT5 route from PE-3:
102 2022/01/28 13:12:04.033 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.3 "Peer 1: 192.0.2.3: UPDATE Peer 1: 192.0.2.3 - Received BGP UPDATE: Withdrawn Length = 0 Total Path Attr Length = 90 Flag: 0x90 Type: 14 Len: 45 Multiprotocol Reachable NLRI: Address Family EVPN NextHop len 4 NextHop 192.0.2.3 Type: EVPN-IP-PREFIX Len: 34 RD: 192.0.2.3:15, tag: 0, ip_prefix: 10.0.0.100/32 gw_ip 0.0.0.0 Label: 8388544 (Raw Label: 0x7fffc0) Flag: 0x40 Type: 1 Len: 1 Origin: 0 Flag: 0x40 Type: 2 Len: 0 AS Path: Flag: 0x40 Type: 5 Len: 4 Local Preference: 100 Flag: 0xc0 Type: 16 Len: 24 Extended Community: target:64500:15 mac-nh:00:00:00:00:00:03 bgp-tunnel-encap:MPLS " -
PE-2 receives the EVPN routes and updates its FDB and ARP tables. When the ARP entry changes its type from dynamic to EVPN, PE-2 withdraws its RT5 route.
On PE-2, the FDB for R-VPLS 17 shows an EVPN route for MAC 00:00:00:00:01:00:
[/]
A:admin@PE-2# show service id 17 fdb detail
===============================================================================
Forwarding Database, Service 17
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
17 00:00:00:00:01:00 mpls-1: Evpn 01/28/22 13:12:04
192.0.2.3:524283
ldp:65538
17 00:00:00:00:2f:17 cpm Intf 01/28/22 12:53:11
17 00:00:00:00:3f:17 mpls-1: EvpnS:P 01/28/22 12:43:29
192.0.2.3:524283
ldp:65538
17 00:00:5e:00:01:01 cpm Intf 01/28/22 12:43:20
-------------------------------------------------------------------------------
No. of MAC Entries: 4
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
IPv6 host mobility
The following use cases for IPv6 host mobility are described:
-
Host initiates an unsolicited NA message after moving
-
Host sends non-ND traffic after moving
-
Host does not send any traffic after moving
The configuration is identical in these use cases.
Example topology for initial forwarding path via PE-2 with IPv6 addresses shows the topology with IPv6 addresses for host-61 and host-66.
The services are the following:
-
R-VPLS "sbd-5" on PE-1, PE-2, and PE-3
-
VPRN "ip-vrf-6" on PE-1, PE-2, and PE-3
-
R-VPLS "evi-10" on PE-1; R-VPLS "evi-7" on PE-2 and PE-3
-
Epipe "Epipe 7" on PE-4, PE-5, and PE-6
-
Host-61 is connected to R-VPLS "evi-10" on PE-1
-
Host-66 is connected to Epipe "Epipe 7" on PE-6
The service configuration on PE-1 is as follows:
# on PE-1:
configure {
service {
vpls "evi-10" {
admin-state enable
description "R-VPLS 10"
service-id 10
customer "1"
routed-vpls {
}
sap pxc-10.a:10 {
}
}
vpls "sbd-5" {
admin-state enable
description "R-VPLS 5"
service-id 5
customer "1"
routed-vpls {
}
bgp 1 {
route-distinguisher "192.0.2.1:5"
}
bgp-evpn {
evi 5
routes {
ip-prefix {
advertise true
}
}
mpls 1 {
admin-state enable
auto-bind-tunnel {
resolution any
}
}
}
}
vprn "ip-vrf-6" {
admin-state enable
service-id 6
customer "1"
interface "evi-10" {
mac 00:00:00:06:1e:20
vpls "evi-10" {
}
ipv6 {
address 2001:db8::20:1 {
prefix-length 120
}
}
}
interface "evi-5" {
mac 00:00:00:00:06:01
vpls "sbd-5" {
evpn-tunnel {
}
}
ipv6 {
}
}
}
The service configuration on PE-2 is as follows. The service configuration on PE-3 is similar.
# on PE-2:
configure {
service {
vpls "sbd-5" {
admin-state enable
description "R-VPLS 5"
service-id 5
customer "1"
routed-vpls {
}
bgp 1 {
route-distinguisher "192.0.2.2:5" # on PE-3: 192.0.2.3:5
}
bgp-evpn {
evi 5
routes {
ip-prefix {
advertise true
}
}
mpls 1 {
admin-state enable
auto-bind-tunnel {
resolution any
}
}
}
}
vprn "ip-vrf-6" {
admin-state enable
service-id 6
customer "1"
interface "evi-5" {
mac 00:00:00:00:06:02 # on PE-3: 00:00:00:00:06:03
vpls "sbd-5" {
evpn-tunnel {
}
}
ipv6 {
}
}
interface "evi-7" {
mac 00:00:00:00:2f:07 # on PE-3: 00:00:00:00:3f:07
vpls "evi-7" {
evpn {
nd {
learn-dynamic false
advertise dynamic {
}
}
}
}
ipv6 {
link-local-address {
address fe80::10:2 # on PE-3: fe80::10:3
duplicate-address-detection false
}
address 2001:db8::10:2 { # on PE-3: 2001:db8::10:3
prefix-length 120
}
neighbor-discovery {
learn-unsolicited both
proactive-refresh both
local-proxy-nd true
host-route {
populate dynamic {
}
}
}
vrrp 1 {
backup [fe80::10:fe]
passive true
ping-reply true
traceroute-reply true
}
}
}
}
vpls "evi-7" {
admin-state enable
description "R-VPLS 7"
service-id 7
customer "1"
routed-vpls {
}
bgp 1 {
route-distinguisher "192.0.2.2:7" # on PE-3: 192.0.2.3:7
}
bgp-evpn {
evi 7
mpls 1 {
admin-state enable
auto-bind-tunnel {
resolution any
}
}
}
sap 1/1/1:7 { # on PE-3: sap 1/1/2:7
}
}
Debugging is enabled on PE-2 and PE-3 (in classic CLI):
# on PE-2, PE-3:
debug
router "Base"
bgp
update
exit
exit
router service-name "ip-vrf-6"
ip
route-table
icmp6 "evi-7"
neighbor "evi-7"
exit
exit
exit
Initially, the traceroute from host-66 to host-61 is via PE-2 (2001:db8::10:2):
[/]
A:admin@PE-6# traceroute 2001:db8::20:61 router-instance "H-66"
traceroute to 2001:db8::20:61, 30 hops max, 60 byte packets
1 2001:db8::10:2 (2001:db8::10:2) 1034 ms 2.80 ms 4.17 ms
2 :: * * *
3 2001:db8::20:61 (2001:db8::20:61) 11.1 ms 5.59 ms 4.95 ms
The following route table on PE-2 shows an ARP-ND host route for prefix 2001:db8::10:66/128:
[/]
A:admin@PE-2# show router 6 route-table 2001:db8::10:66
===============================================================================
IPv6 Route Table (Service: 6)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
2001:db8::10:66/128 Remote ARP-ND 00h00m33s 1
2001:db8::10:66 0
-------------------------------------------------------------------------------
No. of Routes: 1
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
IPv6 host mobility case 1: host initiates an unsolicited NA message after moving
On PE-2 and PE-3, the learn-unsolicited command is configured on interface "evi-7" in VPRN "ip-vrf-6". When an unsolicited NA message is received, a stale neighbor is created. If host-route>populate dynamic is enabled, a confirmation message is sent for all the neighbor entries created as stale, and if confirmed, the corresponding ARP-ND routes are added to the route table.
Disabling SDP 46 on PE-4 causes a failover from the primary path via PE-4 to the secondary path via PE-5, simulating host-66 moving from PE-2 to PE-3. To trigger an unsolicited NA message from host-66, its MAC address 00:00:00:00:66:06 is replaced by MAC address 00:00:00:00:06:06. Host-66 sends unsolicited NA message after switchover shows that host-66 sends an unsolicited NA message.
Host-66 advertises its new MAC address in unsolicited NA messages. PE-3 receives the following NA messages from host-66. PE-2 also receives the NA messages, but it rejects NA messages received on interface "evi-7" when learn-dynamic false is configured:
# on PE-3:
3 2022/01/28 13:19:55.747 UTC MINOR: DEBUG #2001 vprn6 TIP
"TIP: ICMP6_PKT
ICMP6 ingressing on evi-7 (vprn6):
fe80::10:6 -> ff02::1
Type: Neighbor Advertisement (136)
Code: No Code (0)
Tgt Addr: 2001:db8::10:66
Flags : Router Override
Option : Tgt Link Layer Addr 00:00:00:00:06:06
"
1 2022/01/28 13:19:55.747 UTC MINOR: DEBUG #2001 vprn6 TIP
"TIP: ICMP6_PKT
ICMP6 ingressing on evi-7 (vprn6):
fe80::10:6 -> ff02::1
Type: Neighbor Advertisement (136)
Code: No Code (0)
Tgt Addr: fe80::10:6
Flags : Router Override
Option : Tgt Link Layer Addr 00:00:00:00:06:06
"
PE-3 learns the MAC address dynamically:
[/]
A:admin@PE-3# show service id 7 fdb mac 00:00:00:00:06:06
===============================================================================
Forwarding Database, Service 7
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
7 00:00:00:00:06:06 sap:1/1/2:7 L/90 01/28/22 13:19:56
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
PE-3 sends CPM-generated NS messages that are also flooded to the EVPN destinations. The learn-dynamic false command prevents PE-2 from learning MAC addresses dynamically on an EVPN connection.
PE-3 sends an EVPN-MAC update to PE-2 and MAC address 00:00:00:00:06:06 appears in the FDB on PE-2 as an EVPN entry:
[/]
A:admin@PE-2# show service id 7 fdb mac 00:00:00:00:06:06
===============================================================================
Forwarding Database, Service 7
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
7 00:00:00:00:06:06 mpls-1: Evpn 01/28/22 13:19:56
192.0.2.3:524281
ldp:65538
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
The route table for VPRN "ip-vrf-6" on PE-3 shows an ARP-ND entry for destination prefix 2001:db8::10:66/128, as follows:
[/]
A:admin@PE-3# show router 6 route-table 2001:db8::10:66
===============================================================================
IPv6 Route Table (Service: 6)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
2001:db8::10:66/128 Remote ARP-ND 00h02m37s 1
2001:db8::10:66 0
-------------------------------------------------------------------------------
No. of Routes: 1
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
On PE-2, the route table for VPRN "ip-vrf-6" shows an EVPN entry for prefix 2001:db8::10:66/128:
[/]
A:admin@PE-2# show router 6 route-table 2001:db8::10:66
===============================================================================
IPv6 Route Table (Service: 6)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
2001:db8::10:66/128 Remote EVPN-IFF 00h02m39s 169
fe80::7:b0d1:3fa3:2f60-"evi-5" 0
-------------------------------------------------------------------------------
No. of Routes: 1
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
IPv6 host mobility case 2: host sends non-ND traffic after moving,
The service configuration is the same as in the use case 1. The only difference from use case 1 is the type of message that is sent by host-66 after moving.
Initially, the traceroute from host-66 to host-61 is via PE-2 (2001:db8::10:2):
[/]
A:admin@PE-6# traceroute 2001:db8::20:61 router-instance "H-66"
traceroute to 2001:db8::20:61, 30 hops max, 60 byte packets
1 2001:db8::10:2 (2001:db8::10:2) 7.88 ms 3.85 ms 3.67 ms
2 :: * * *
3 2001:db8::20:61 (2001:db8::20:61) 11.0 ms 5.19 ms 5.30 ms
A switchover from the primary path to the secondary path takes place, so host-66 moves from PE-2 to PE-3. Host generates non-ND traffic after switchover shows that host-66 sends non-ND traffic after moving.
The traceroute from host-66 to host-61 is via PE-3 (2001:db8::10:3) instead of PE-2, as follows:
[/]
A:admin@PE-6# traceroute 2001:db8::20:61 router-instance "H-66"
traceroute to 2001:db8::20:61 from 2001:db8::10:66, 30 hops max, 60 byte packets
1 2001:db8::10:3 (2001:db8::10:3) 7.68 ms 3.84 ms 3.65 ms
2 :: * * *
3 2001:db8::20:61 (2001:db8::20:61) 5.54 ms 5.55 ms 5.64 ms
On PE-3, MAC address 00:00:00:00:06:06 from host-66 is learned on the local SAP 1/1/2:17:
[/]
A:admin@PE-3# show service id 7 fdb mac 00:00:00:00:06:06
===============================================================================
Forwarding Database, Service 7
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
7 00:00:00:00:06:06 sap:1/1/2:7 L/0 01/28/22 13:29:59
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
PE-3 advertises MAC address 00:00:00:00:06:06 from host-66 in three EVPN-MAC routes: one with the global IP address 2001:db8::10:66, one with the link local IP address fe80::200:ff:fe00:606, and one with a null IP address. PE-2 receives the following EVPN-MAC routes from PE-3:
# on PE-2:
59 2022/01/28 13:29:59.437 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.3
"Peer 1: 192.0.2.3: UPDATE
Peer 1: 192.0.2.3 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 191
Flag: 0x90 Type: 14 Len: 146 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.3
Type: EVPN-MAC Len: 49 RD: 192.0.2.3:7 ESI: ESI-0, tag: 0, mac len: 48
mac: 00:00:00:00:06:06, IP len: 16, IP: fe80::10:6, label1: 8388496
Type: EVPN-MAC Len: 49 RD: 192.0.2.3:7 ESI: ESI-0, tag: 0, mac len: 48
mac: 00:00:00:00:06:06, IP len: 16, IP: 2001:db8::10:66, label1: 8388496
Type: EVPN-MAC Len: 33 RD: 192.0.2.3:7 ESI: ESI-0, tag: 0, mac len: 48
mac: 00:00:00:00:06:06, IP len: 0, IP: NULL, label1: 8388496
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:7
bgp-tunnel-encap:MPLS
mac-mobility:Seq:2
"
On PE-2, the following EVPN entry for MAC 00:00:00:00:06:06 is added to the FDB:
[/]
A:admin@PE-2# show service id 7 fdb mac 00:00:00:00:06:06
===============================================================================
Forwarding Database, Service 7
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
7 00:00:00:00:06:06 mpls-1: Evpn 01/28/22 13:29:59
192.0.2.3:524281
ldp:65538
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
The route table on PE-3 shows an ARP-ND host route for prefix 2001:db8::10:66/128:
[/]
A:admin@PE-3# show router 6 route-table 2001:db8::10:66
===============================================================================
IPv6 Route Table (Service: 6)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
2001:db8::10:66/128 Remote ARP-ND 00h01m38s 1
2001:db8::10:66 0
-------------------------------------------------------------------------------
No. of Routes: 1
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
PE-2 receives the following RT5 route from PE-3 for prefix 2001:db8::10:66/128:
95 2022/01/28 13:30:00.438 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.3
"Peer 1: 192.0.2.3: UPDATE
Peer 1: 192.0.2.3 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 106
Flag: 0x90 Type: 14 Len: 69 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.3
Type: EVPN-IP-PREFIX Len: 58 RD: 192.0.2.3:5, tag: 0,
ip_prefix: 2001:db8::10:66/128 gw_ip fe80::7:b0d1:3fa3:2f60
Label: 8388512 (Raw Label: 0x7fffa0)
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 16 Extended Community:
target:64500:5
bgp-tunnel-encap:MPLS
"
In the route table on PE-2, the route for prefix 2001:db8::10:66/128 is an EVPN route:
[/]
A:admin@PE-2# show router 6 route-table 2001:db8::10:66
===============================================================================
IPv6 Route Table (Service: 6)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
2001:db8::10:66/128 Remote EVPN-IFF 00h01m37s 169
fe80::7:b0d1:3fa3:2f60-"evi-5" 0
-------------------------------------------------------------------------------
No. of Routes: 1
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
IPv6 host mobility case 3: host does not send any traffic after moving
The service configuration is the same as use cases 1 and 2. SDP 46 is enabled on PE-4, so host-66 moves back to PE-2. The following traceroute shows that the forwarding path from host-66 to host-61 is via PE-2:
[/]
A:admin@PE-6# traceroute router 8 2001:db8::20:61 source 2001:db8::10:66
traceroute to 2001:db8::20:61 from 2001:db8::10:66, 30 hops max, 60 byte packets
1 2001:db8::10:2 (2001:db8::10:2) 3.76 ms 4.23 ms 4.00 ms
2 :: * * *
3 2001:db8::20:61 (2001:db8::20:61) 5.89 ms 5.40 ms 5.61 ms
The FDB on PE-2 shows that MAC address 00:00:00:00:06:06 is learned on the local SAP 1/1/1:7, as follows:
[/]
A:admin@PE-2# show service id 7 fdb detail
===============================================================================
Forwarding Database, Service 7
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
7 00:00:00:00:06:06 sap:1/1/1:7 L/30 01/28/22 13:36:03
7 00:00:00:00:2f:07 cpm Intf 01/28/22 13:17:40
7 00:00:00:00:3f:07 mpls-1: EvpnS:P 01/28/22 13:17:47
192.0.2.3:524281
ldp:65538
-------------------------------------------------------------------------------
No. of MAC Entries: 3
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
A failure is simulated, causing a failover from the primary path via PE-4 to the secondary path via PE-5. Host-66 does not send any traffic after switchover. Host does not send any traffic after switchover shows that PE-2 sends an NS message when the FDB entry for host-66 ages out.
On PE-2, MAC address 00:00:00:00:06:06 expires in the FDB for R-VPLS "evi-7", which triggers PE-2 to send an NS message for 2001:db8::10:66. This CPM-generated NS message is flooded to the EVPN destinations PE-1 and PE-3.
# on PE-2:
202 2022/01/28 13:37:34.634 UTC MINOR: DEBUG #2001 vprn6 TIP
"TIP: NBR
Sending NS for nbr addr 2001:db8::10:66 nbr type dynamic"
The NS message reaches host-66, which replies with an NA message. PE-3 receives the NA message and updates its FDB and ND tables. PE-2 also receives the NA message, but it rejects NA messages received on interface "evi-7" when no learn-dynamic is configured:
225 2022/01/28 13:37:34.638 UTC MINOR: DEBUG #2001 vprn6 TIP
"TIP: NBR
Ignore NA for target address 2001:db8::10:66 on evpn endpoint evi-7 because learn-dynamic is disabled."
PE-2 receives the following EVPN-MAC routes from PE-3:
237 2022/01/28 13:43:03.001 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.3
"Peer 1: 192.0.2.3: UPDATE
Peer 1: 192.0.2.3 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 191
Flag: 0x90 Type: 14 Len: 146 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.3
Type: EVPN-MAC Len: 49 RD: 192.0.2.3:7 ESI: ESI-0, tag: 0, mac len: 48
mac: 00:00:00:00:06:06, IP len: 16, IP: 2001:db8::10:66, label1: 8388512
Type: EVPN-MAC Len: 49 RD: 192.0.2.3:7 ESI: ESI-0, tag: 0, mac len: 48
mac: 00:00:00:00:06:06, IP len: 16, IP: fe80::10:6, label1: 8388512
Type: EVPN-MAC Len: 33 RD: 192.0.2.3:7 ESI: ESI-0, tag: 0, mac len: 48
mac: 00:00:00:00:06:06, IP len: 0, IP: NULL, label1: 8388512
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:7
bgp-tunnel-encap:MPLS
mac-mobility:Seq:3
"
PE-2 receives the following RT5 route from PE-3:
228 2022/01/28 13:37:35.638 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.3
"Peer 1: 192.0.2.3: UPDATE
Peer 1: 192.0.2.3 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 106
Flag: 0x90 Type: 14 Len: 69 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.3
Type: EVPN-IP-PREFIX Len: 58 RD: 192.0.2.3:5, tag: 0,
ip_prefix: 2001:db8::10:66/128 gw_ip fe80::7:b0d1:3fa3:2f60
Label: 8388512 (Raw Label: 0x7fffa0)
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 16 Extended Community:
target:64500:5
bgp-tunnel-encap:MPLS
"
Upon receiving the routes, PE-2 updates its FDB and ARP tables. After the switchover, MAC address 00:00:00:00:06:06 is no longer learned on a local SAP on PE-2, but is learned via an EVPN-MAC route from PE-3, as follows:
[/]
A:admin@PE-2# show service id 7 fdb detail
===============================================================================
Forwarding Database, Service 7
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
7 00:00:00:00:06:06 mpls-1: Evpn 01/28/22 13:37:35
192.0.2.3:524281
ldp:65538
7 00:00:00:00:2f:07 cpm Intf 01/28/22 13:17:40
7 00:00:00:00:3f:07 mpls-1: EvpnS:P 01/28/22 13:17:47
192.0.2.3:524281
ldp:65538
-------------------------------------------------------------------------------
No. of MAC Entries: 3
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
Conclusion
EVPN host mobility is supported in SR OS as described in draft-ietf-bess-evpn-inter-subnet-forwarding. This chapter describes several cases when a host moves from a source PE to a target PE within the same broadcast domain.