EVPN for MPLS Tunnels in Routed VPLS
This chapter provides information about EVPN for MPLS tunnels in routed VPLS.
Topics in this chapter include:
Applicability
This chapter was initially written for SR OS Release 15.0.R4, but the MD-CLI in the current edition is based on SR OS Release 21.10.R3. EVPN-MPLS and IP-prefix advertisement in routed VPLS (R-VPLS) without Multi-homing (MH) is supported in SR OS Release 14.0.R1, and later. EVPN-MPLS and IP-prefix advertisement in R-VPLS with all-active and single-active MH is supported in SR OS Release 14.0.R4, and later. Virtual Router Redundancy Protocol (VRRP) in passive mode is also supported in SR OS Release 14.0.R4, and later.
Chapter EVPN for VXLAN Tunnels (Layer 3) is prerequisite reading.
Overview
The EVPN-MPLS in R-VPLS feature matches the EVPN-VXLAN in R-VPLS feature, which is described in chapter EVPN for VXLAN Tunnels (Layer 3) The following capabilities are supported in an R-VPLS service where bgp-evpn mpls is enabled:
-
R-VPLS with Virtual Router Redundancy Protocol (VRRP) support on the VPRN interfaces
-
R-VPLS support including IP route advertisement (IP prefix routes — BGP-EVPN route type 5) with regular interfaces
-
R-VPLS support including IP route advertisement with evpn-tunnel interfaces
-
R-VPLS with IPv6 support on the VPRN IP interface
All-active and single-active MH Ethernet segments (ESs) are supported in R-VPLS. When Ethernet Segments (ESs) are used along with R-VPLS services in two or more PEs, passive VRRP provides an "anycast default gateway" that optimizes inter-subnet forwarding for hosts in the R-VPLS. Passive VRRP is described in the following section.
Passive VRRP
VRRP can be configured in passive mode, which suppresses the transmission and reception of keepalive messages. Passive VRRP can be configured in the base router, in an IES, or in a VPRN, using the following commands:
[/]
A:admin@PE-2# tree flat detail | match vrrp | match passive
configure groups group <string> router <string> interface <string> ipv4 vrrp <string
| number> passive <boolean>
configure groups group <string> router <string> interface <string> ipv6 vrrp <string | number> passive <boolean>
configure groups group <string> service ies <string> interface <string> ipv4 vrrp <string | number> passive <boolean>
configure groups group <string> service ies <string> interface <string> ipv6 vrrp <string | number> passive <boolean>
configure groups group <string> service vprn <string> interface <string> ipv4 vrrp <string | number> passive <boolean>
configure groups group <string> service vprn <string> interface <string> ipv6 vrrp <string | number> passive <boolean>
configure router <string> interface <string> ipv4 vrrp <number> passive <boolean>
configure router <string> interface <string> ipv6 vrrp <number> passive <boolean>
configure service ies <string> interface <string> ipv4 vrrp <number> passive <boolean>
configure service ies <string> interface <string> ipv6 vrrp <number> passive <boolean>
configure service vprn <string> interface <string> ipv4 vrrp <number> passive <boolean>
configure service vprn <string> interface <string> ipv6 vrrp <number> passive <boolean>
All PEs configured with passive VRRP become VRRP master and take ownership of the virtual IP and MAC addresses. Passive VRRP - vMAC/vIP advertised by GARP shows the use of passive VRRP where the VRID and default gateway (GW) are identical for all nodes, and therefore, the vMAC/vIP are identical. Each PE sends Gratuitous Address Resolution Protocol (GARP) messages with the same vMAC/vIP.
Ethernet VPN instance (EVI) 202 is configured on all PEs as an R-VPLS with passive VRRP. Each individual R-VPLS interface has a unique MAC/IP, but they all have the same vMAC/vIP because they share the same VRID and backup IP address. The vMAC address is auto-derived out of 00:00:5e:00:00:<VRID>, as per RFC 3768.
The behavior is as follows:
-
PEs advertise their real MAC/IP and their vMAC/vIP in EVPN for EVI 202.
-
All hosts in EVI 202 have a unique configured default GW.
-
When a CE sends upstream traffic to a remote subnet, the packets are routed by the closest PE because the vMAC address is local on each PE.
-
In case of ES failure, or in case of single-active MH if the traffic arrives at the non-Designated Forwarder (NDF) PE, the traffic will not be discarded at the peer ES PE. Virtual MAC addresses bypass the R-VPLS interface protection, so traffic can be forwarded between the PEs without being dropped. Note that if passive VRRP was not used in this case and the same regular interface anycast MAC/IP was used instead, the peer PE would discard the traffic due to the MAC Source Address (SA).
Passive VRRP provides an efficient anycast default gateway solution, with the following advantages compared to regular VRRP:
-
No need for multiple VRRP instances to achieve default GW load-balancing. Only one VRRP instance is in the R-VPLS, so only one default GW is needed for all hosts.
-
Fast convergence because all the nodes in the VRID are master.
-
Better scalability because there is no need for keepalive messages or BFD to detect failures.
Passive VRRP provides the following advantages compared to using the same anycast MAC/IP in all the Integrated Routing Bridging (IRB) interfaces:
-
VRRP vMAC SA bypasses the protection in the receiving R-VPLS service; therefore, frames with MAC SA matching the local vMAC address are not discarded, and VRRP vMAC SAs can be used in combination with EVPN multi-homing.
-
PEs will not show traps claiming duplicate IP addresses.
-
vMAC addresses are auto-derived from the VRID, so no need to configure the same MAC address in all the IRB interfaces.
-
PEs can still use their real (unique) IRB IP addresses when sending ICMP packets for troubleshooting purposes.
Configuration
In this section, the following use cases are described:
-
EVPN-MPLS R-VPLS without multi-homing
-
EVPN-MPLS R-VPLS with all-active multi-homing ES
-
EVPN-MPLS R-VPLS with single-active multi-homing ES
EVPN-MPLS R-VPLS without multi-homing
The first scenario describes R-VPLS support including IP route advertisement (BGP-EVPN route type 5) with EVPN tunnel interfaces, without multi-homing. VPLS 101 does not have any connected host, but the linked VPRN has SAP 1/2/1:10. R-VPLS with EVPN tunnel, without multi-homing shows the example topology used for R-VPLS with EVPN tunnel but without multi-homing. IP prefixes are advertised.
The initial configuration includes the following:
-
Cards, MDAs, ports
-
Router interface between PE-2 and PE-3
-
IS-IS (or OSPF)
-
LDP enabled on the router interface between PE-2 and PE-3
BGP is configured for the EVPN address family on PE-2 and PE-3. The BGP configuration on PE-2 is as follows. The BGP configuration on PE-3 is similar.
# on PE-2:
configure {
router "Base" {
autonomous-system 64500
bgp {
vpn-apply-export true
vpn-apply-import true
rapid-withdrawal true
peer-ip-tracking true
family {
ipv4 false
evpn true
}
rapid-update {
evpn true
}
group "internal" {
peer-as 64500
}
neighbor "192.0.2.3" {
group "internal"
}
}
The CEs are connected to SAP 1/2/1:10 in VPRN 10. R-VPLS 101 is bound to VPRN 10 and VPRN 10 has a dedicated interface "int-evi-101" for the EVPN tunnel. In general, if only one route-target (RT) is used for import and export in the EVPN-VPLS, it is good to add the EVI and have the route distinguisher (RD) and RT auto-derived from the EVI. It is simpler and avoids configuration mistakes. The service configuration on PE-2 is as follows:
# on PE-2:
configure {
service {
vpls "evi-101" {
admin-state enable
service-id 101
customer "1"
routed-vpls {
}
bgp 1 { # RD and RT are not manually configured in BGP context
}
bgp-evpn {
evi 101 # RD and RT will be auto-derived from the EVI
routes {
ip-prefix {
advertise true
}
}
mpls 1 {
admin-state enable
auto-bind-tunnel {
resolution any
}
}
}
}
vprn "VPRN 10" {
admin-state enable
service-id 10
customer "1"
interface "int-PE-2-CE-20" {
ipv4 {
primary {
address 172.16.2.1
prefix-length 24
}
}
sap 1/2/1:10 {
}
}
interface "int-evi-101" {
vpls "evi-101" {
evpn-tunnel {
}
}
}
}
-
The routed-vpls command is required so that R-VPLS "evi-101" can be bound to VPRN 10.
-
The service name "evi-101" must match the name in the VPRN 10 VPLS interface.
-
The VPRN 10 VPLS interface is configured with the keyword evpn-tunnel. This configuration has the advantage of not having to allocate IP addresses to the R-VPLS interfaces, however, it cannot be used when the R-VPLS has local SAPs.
The configuration is similar on PE-3. It is important that the RD is different on PE-2 and PE-3, but it is automatically the case when the RD is auto-derived from the configured EVI, as in the example. The RD on PE-2 is 192.0.2.2:101; on PE-3, the RD is 192.0.2.3:101.
PE-3 receives the following BGP-EVPN IP prefix route for prefix 172.16.2.0/24 from PE-2:
2 2022/02/24 11:00:28.145 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.2
"Peer 1: 192.0.2.2: UPDATE
Peer 1: 192.0.2.2 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 90
Flag: 0x90 Type: 14 Len: 45 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.2
Type: EVPN-IP-PREFIX Len: 34 RD: 192.0.2.2:101, tag: 0,
ip_prefix: 172.16.2.0/24 gw_ip 0.0.0.0 Label: 8388496 (Raw Label: 0x7fff90)
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:101
mac-nh:02:13:ff:ff:ff:a2
bgp-tunnel-encap:MPLS
"
GW IP 0.0.0.0 is an indication that an EVPN tunnel is in use. With EVPN tunnels, no IRB IP address needs to be configured in the VPRN. EVPN tunnels make provisioning easier to automate and save IP addresses from the tenant IP space.
The BGP tunnel encapsulation is MPLS, but the MPLS label in the debug message is not the same as in the service, because the router will strip the extra four lowest bits to get the 20-bit MPLS label. In the debug message, the label is 8388496. This is because the debug message is shown before the router can parse the label field and see if it corresponds to an MPLS label (20 bits) or a VXLAN VNI (24 bits). The MPLS label is calculated by dividing the label value by 24 (16), as follows: 8388496/16 = 524281.
The MAC next-hop extended community 02:13:ff:ff:ff:a2 is the MAC address of the interface "int-evi-101" in VPRN 10 on PE-2, as follows:
[/]
A:admin@PE-2# show router 10 interface "int-evi-101" detail | match "MAC Address"
MAC Address : 02:13:ff:ff:ff:a2 Mac Accounting : Disabled
The routing table for VPRN 10 on PE-3 contains the route for prefix 172.16.2.0/24 as the EVPN-IFF (IFF stands for Interface-ful) route with next-hop "int-evi-101" and interface name "ET-02:13:ff:ff:ff:a2" (ET stands for EVPN Tunnel), as follows:
[/]
A:admin@PE-3# show router 10 route-table
===============================================================================
Route Table (Service: 10)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
172.16.2.0/24 Remote EVPN-IFF 01h43m58s 169
int-evi-101 (ET-02:13:ff:ff:ff:a2) 0
172.16.3.0/24 Local Local 01h43m59s 0
int-PE-3-CE-30 0
-------------------------------------------------------------------------------
No. of Routes: 2
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
The forwarding database (FDB) for VPLS 101 on PE-3 shows an entry for MAC address 02:13:ff:ff:ff:a2 that is learned via EVPN. The MAC address is static (S) and protected (P). The MPLS label is 524281.
[/]
A:admin@PE-3# show service id 101 fdb detail
===============================================================================
Forwarding Database, Service 101
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
101 02:13:ff:ff:ff:a2 mpls-1: EvpnS:P 02/24/22 11:00:35
192.0.2.2:524281
ldp:65538
101 02:17:ff:ff:ff:a2 cpm Intf 02/24/22 11:00:34
-------------------------------------------------------------------------------
No. of MAC Entries: 2
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
When the CEs have IPv6 addresses, the VPRN configuration is similar on the PEs, but the ipv6 context must be enabled in the EVPN tunnel interface, so that the router can advertise and process BGP-EVPN routes type 5 with IPv6 prefixes. The configuration of the VPLS is identical for IPv4 and IPv6.
# on PE-2:
configure {
service {
vpls "evi-106" {
admin-state enable
service-id 106
customer "1"
routed-vpls {
}
bgp 1 {
}
bgp-evpn {
evi 106
routes {
ip-prefix {
advertise true
}
}
mpls 1 {
admin-state enable
auto-bind-tunnel {
resolution any
}
}
}
}
vprn "VPRN 16" {
admin-state enable
service-id 16
customer "1"
interface "int-PE-2-CE-26" {
sap 1/2/1:16 {
}
ipv6 {
address 2001:db8:16::2:1 {
prefix-length 120
}
}
}
interface "int-evi-106" {
vpls "evi-106" {
evpn-tunnel {
}
}
ipv6 {
}
}
}
When advertising IPv6 prefixes, the GW IP field in the route type 5 is always populated with the IPv6 address of the R-VPLS interface. In this example, because no specific IPv6 global address is configured, the GW IP will be populated with the auto-created link local address. The following BGP update is received by PE-3 for IPv6 prefix 2001:db8:16::2:0/120:
# on PE-3:
9 2022/02/24 11:00:35.338 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.2
"Peer 1: 192.0.2.2: UPDATE
Peer 1: 192.0.2.2 - Received BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 106
Flag: 0x90 Type: 14 Len: 69 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.2
Type: EVPN-IP-PREFIX Len: 58 RD: 192.0.2.2:106, tag: 0,
ip_prefix: 2001:db8:16::2:0/120 gw_ip fe80::14:1ff:fe02:1
Label: 8388480 (Raw Label: 0x7fff80)
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 16 Extended Community:
target:64500:106
bgp-tunnel-encap:MPLS
"
The IPv6 route-table on PE-3 is as follows:
[/]
A:admin@PE-3# show router 16 route-table ipv6
===============================================================================
IPv6 Route Table (Service: 16)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
2001:db8:16::2:0/120 Remote EVPN-IFF 01h50m01s 169
fe80::14:1ff:fe02:1-"int-evi-106" 0
2001:db8:16::3:0/120 Local Local 01h50m01s 0
int-PE-3-CE-36 0
-------------------------------------------------------------------------------
No. of Routes: 2
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
EVPN-MPLS R-VPLS with all-active MH
EVPN-MPLS R-VPLS with all-active MH ES shows the example topology with all-active multi-homing ES "AA-ESI-23".
BGP is configured between PE-2, PE-3, and PE-4 for address family EVPN. The configuration on PE-2 is as follows:
# on PE-2:
configure {
router "Base" {
autonomous-system 64500
bgp {
vpn-apply-export true
vpn-apply-import true
rapid-withdrawal true
peer-ip-tracking true
family {
ipv4 false
evpn true
}
rapid-update {
evpn true
}
group "internal" {
peer-as 64500
}
neighbor "192.0.2.3" {
group "internal"
}
neighbor "192.0.2.4" {
group "internal"
}
All-active multi-homing Ethernet segment "AA-ESI-23" is configured on PE-2 and PE-3, as follows:
# on PE-2, PE-3:
configure {
service {
system {
bgp {
evpn {
ethernet-segment "AA-ESI-23" {
admin-state enable
esi 01:00:00:00:00:23:00:00:00:01
multi-homing-mode all-active
df-election {
es-activation-timer 3
}
association {
lag "lag-1" {
}
}
}
}
}
}
The following services are configured on the PEs:
-
VPRN 20 has interfaces bound to VPLS 200 and VPLS 202. On PE-4, VPRN 20 also has an interface bound to VPLS 203.
-
VPLS 200 is configured as an EVPN tunnel that connects the PEs.
-
VPLS 202 and VPLS 203 have attachment circuits to CEs.
The services are configured on PE-2 as follows. The configuration on PE-3 and PE-4 is similar.
# on PE-2:
configure {
service {
vpls "evi-200" {
admin-state enable
service-id 200
customer "1"
routed-vpls {
}
bgp 1 {
}
bgp-evpn {
evi 200
routes {
ip-prefix {
advertise true
}
}
mpls 1 {
admin-state enable
auto-bind-tunnel {
resolution any
}
}
}
}
vpls "evi-202" {
admin-state enable
service-id 202
customer "1"
routed-vpls {
}
bgp 1 {
}
bgp-evpn {
evi 202
mpls 1 {
admin-state enable
auto-bind-tunnel {
resolution any
}
}
}
sap lag-1:20 {
}
}
vprn "VPRN 20" {
admin-state enable
service-id 20
customer "1"
interface "int-evi-200" {
vpls "evi-200" {
evpn-tunnel {
}
}
ipv6 {
}
}
interface "int-evi-202" {
mac 00:ca:fe:00:02:02
ipv4 {
primary {
address 172.16.20.2
prefix-length 24
}
vrrp 1 {
backup [172.16.20.254]
passive true
ping-reply true
traceroute-reply true
}
}
vpls "evi-202" {
}
ipv6 {
link-local-address {
address fe80::16:20:2
duplicate-address-detection false
}
address 2001:db8:16::20:2 {
prefix-length 120
}
vrrp 1 {
backup [fe80::16:20:fe]
passive true
ping-reply true
traceroute-reply true
}
}
}
ipv6 {
router-advertisement {
interface "int-evi-202" {
admin-state enable
use-virtual-mac true
}
}
}
}
The IPv6 VRRP backup address is in the same subnet as the link local address of the interface "int-evi-202". The option duplicate-address-detection false is configured on the link local address to disable Duplicate Address Detection (DAD) and set the IPv6 address as preferred. Also for IPv6, router advertisement must be enabled and configured to use the virtual MAC address.
Passive VRRP
EVI 202 is configured as an R-VPLS with passive VRRP. A passive-VRRP VRID instance suppresses the transmission and reception of keepalive messages. All PEs configured with passive VRRP become VRRP master and take ownership of the virtual IP and MAC address.
Each individual R-VPLS interface has a different MAC/IP on each PE. The MAC/IPs for "int-evi-202" on PE-2 are MAC 00:ca:fe:00:02:02 and IP 172.16.20.2/24 for IPv4 and the same MAC address with IPv6 2001:db8:16::20:2 and fe80::16:20:2. However, the R-VPLS interfaces on all PEs share the same VRID 1 and backup IP address 172.16.20.254, so the same vMAC/vIP 00:00:5e:00:01:01/172.16.20.254 and vMAC/vIP 00:00:5e:00:02:01/ fe80::16:20:fe are advertised by all PEs. PE-2 advertises the following EVPN MAC routes:
83 2022/02/24 15:09:15.841 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.4
"Peer 1: 192.0.2.4: UPDATE
Peer 1: 192.0.2.4 - Send BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 285
Flag: 0x90 Type: 14 Len: 240 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.2
Type: EVPN-MAC Len: 49 RD: 192.0.2.2:202 ESI: ESI-0, tag: 0, mac len: 48
mac: 00:00:5e:00:02:01, IP len: 16, IP: fe80::16:20:fe, label1: 8388416
Type: EVPN-MAC Len: 37 RD: 192.0.2.2:202 ESI: ESI-0, tag: 0, mac len: 48
mac: 00:00:5e:00:01:01, IP len: 4, IP: 172.16.20.254, label1: 8388416
Type: EVPN-MAC Len: 49 RD: 192.0.2.2:202 ESI: ESI-0, tag: 0, mac len: 48
mac: 00:ca:fe:00:02:02, IP len: 16, IP: fe80::16:20:2, label1: 8388416
Type: EVPN-MAC Len: 49 RD: 192.0.2.2:202 ESI: ESI-0, tag: 0, mac len: 48
mac: 00:ca:fe:00:02:02, IP len: 16, IP: 2001:db8:16::20:2, label1: 8388416
Type: EVPN-MAC Len: 37 RD: 192.0.2.2:202 ESI: ESI-0, tag: 0, mac len: 48
mac: 00:ca:fe:00:02:02, IP len: 4, IP: 172.16.20.2, label1: 8388416
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:202
bgp-tunnel-encap:MPLS
mac-mobility:Seq:0/Static
"
The three PEs advertise the same (anycast) vMAC/vIP in EVI 202 as protected, but each PE keeps its own MAC entry in the FDB. The following FDB shows that the source identifier for vMAC 00:00:5e:00:01:01 and vMAC 00:00:5e:00:02:01 is the CPM. These two vMAC entries with source identifier CPM are seen on all PEs.
[/]
A:admin@PE-2# show service id 202 fdb detail
===============================================================================
Forwarding Database, Service 202
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
202 00:00:01:00:00:11 sap:lag-1:20 L/0 02/24/22 15:09:21
202 00:00:01:00:00:16 sap:lag-1:20 L/0 02/24/22 15:09:22
202 00:00:04:00:00:41 mpls-1: Evpn 02/24/22 15:09:14
192.0.2.4:524281
ldp:65539
202 00:00:5e:00:01:01 cpm Intf 02/24/22 15:08:50
202 00:00:5e:00:02:01 cpm Intf 02/24/22 15:08:50
202 00:ca:fe:00:02:02 cpm Intf 02/24/22 15:08:50
202 00:ca:fe:00:02:03 mpls-1: EvpnS:P 02/24/22 15:09:03
192.0.2.3:524276
ldp:65538
202 00:ca:fe:00:02:04 mpls-1: EvpnS:P 02/24/22 15:09:14
192.0.2.4:524281
ldp:65539
-------------------------------------------------------------------------------
No. of MAC Entries: 8
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
The interface MAC 00:ca:fe:00:02:02 is local, so it also has the CPM as source identifier. MAC 00:ca:fe:00:02:03 is the PE-3's R-VPLS interface MAC and it is learned via EVPN-MPLS (mpls-1) as static (S) and protected (P). MAC address 00:ca:fe:00:02:04 on PE-4 is also static and protected.
PE-4 sends the following IP prefix route (BGP-EVPN route type 5) for prefix 172.16.23.0/24 to the other PEs:
37 2022/02/24 15:09:13.665 UTC MINOR: DEBUG #2001 Base Peer 1: 192.0.2.3
"Peer 1: 192.0.2.3: UPDATE
Peer 1: 192.0.2.3 - Send BGP UPDATE:
Withdrawn Length = 0
Total Path Attr Length = 90
Flag: 0x90 Type: 14 Len: 45 Multiprotocol Reachable NLRI:
Address Family EVPN
NextHop len 4 NextHop 192.0.2.4
Type: EVPN-IP-PREFIX Len: 34 RD: 192.0.2.4:200, tag: 0,
ip_prefix: 172.16.23.0/24 gw_ip 0.0.0.0
Label: 8388512 (Raw Label: 0x7fffa0)
Flag: 0x40 Type: 1 Len: 1 Origin: 0
Flag: 0x40 Type: 2 Len: 0 AS Path:
Flag: 0x40 Type: 5 Len: 4 Local Preference: 100
Flag: 0xc0 Type: 16 Len: 24 Extended Community:
target:64500:200
mac-nh:02:1b:ff:00:00:05
bgp-tunnel-encap:MPLS
"
The IP prefixes are advertised with next-hop equal to the EVPN-tunnel GW MAC "int-evi-200", as follows:
[/]
A:admin@PE-4# show router 20 interface "int-evi-200" detail | match "MAC Address"
MAC Address : 02:1b:ff:00:00:05 Mac Accounting : Disabled
The routing table for VPRN 20 on PE-2 contains IP-prefix 172.16.23.0/24 with next-hop 02:1b:ff:00:00:05, as follows:
[/]
A:admin@PE-2# show router 20 route-table
===============================================================================
Route Table (Service: 20)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
172.16.20.0/24 Local Local 00h17m12s 0
int-evi-202 0
172.16.23.0/24 Remote EVPN-IFF 00h16m48s 169
int-evi-200 (ET-02:1b:ff:00:00:05) 0
-------------------------------------------------------------------------------
No. of Routes: 2
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
The following IPv6 routing table for VPRN 20 on PE-2 contains prefix 2001:db8:16::23:0/120, which has also been advertised by PE-4. The next-hop is again "int-evi-200", only this time the link local IPv6 address is displayed (GW IP) instead of the MAC address. The next-hop is the GW IP value in the route type 5, as long as it is non-zero. When the GW IP address is zero, the route type 5 is expected to contain a mac-nh extended community. The MAC encoded in the extended community is used as next-hop in that case.
[/]
A:admin@PE-2# show router 20 route-table ipv6
===============================================================================
IPv6 Route Table (Service: 20)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
2001:db8:16::20:0/120 Local Local 00h17m10s 0
int-evi-202 0
2001:db8:16::23:0/120 Remote EVPN-IFF 00h16m46s 169
fe80::a5:9124:c1ed:83ce-"int-evi-200" 0
-------------------------------------------------------------------------------
No. of Routes: 2
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
The EVPN tunnel service VPLS 200 has all the MAC addresses of the EVPN interfaces within VPRN 20 as static (S) and protected (P), as follows:
[/]
A:admin@PE-2# show service id "evi-200" fdb detail
===============================================================================
Forwarding Database, Service 200
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
200 02:13:ff:00:00:05 cpm Intf 02/24/22 15:08:50
200 02:17:ff:00:00:05 mpls-1: EvpnS:P 02/24/22 15:09:03
192.0.2.3:524277
ldp:65538
200 02:1b:ff:00:00:05 mpls-1: EvpnS:P 02/24/22 15:09:14
192.0.2.4:524282
ldp:65539
-------------------------------------------------------------------------------
No. of MAC Entries: 3
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
The VRRP instance in each PE is master, as follows:
[/]
A:admin@PE-2# show router 20 vrrp instance
===============================================================================
VRRP Instances
===============================================================================
Interface Name VR Id Own Adm State Base Pri Msg Int
IP Opr Pol Id InUse Pri Inh Int
-------------------------------------------------------------------------------
int-evi-202 1 No Up Master 100 1
IPv4 Up n/a 100 No
Backup Addr: 172.16.20.254
int-evi-202 1 No Up Master 100 1
IPv6 Up n/a 100 Yes
Backup Addr: fe80::16:20:fe
-------------------------------------------------------------------------------
Instances : 2
===============================================================================
[/]
A:admin@PE-3# show router 20 vrrp instance
===============================================================================
VRRP Instances
===============================================================================
Interface Name VR Id Own Adm State Base Pri Msg Int
IP Opr Pol Id InUse Pri Inh Int
-------------------------------------------------------------------------------
int-evi-202 1 No Up Master 100 1
IPv4 Up n/a 100 No
Backup Addr: 172.16.20.254
int-evi-202 1 No Up Master 100 1
IPv6 Up n/a 100 Yes
Backup Addr: fe80::16:20:fe
-------------------------------------------------------------------------------
Instances : 2
===============================================================================
[/]
A:admin@PE-4# show router 20 vrrp instance
===============================================================================
VRRP Instances
===============================================================================
Interface Name VR Id Own Adm State Base Pri Msg Int
IP Opr Pol Id InUse Pri Inh Int
-------------------------------------------------------------------------------
int-evi-202 1 No Up Master 100 1
IPv4 Up n/a 100 No
Backup Addr: 172.16.20.254
int-evi-203 2 No Up Master 100 1
IPv4 Up n/a 100 No
Backup Addr: 172.16.23.254
int-evi-202 1 No Up Master 100 1
IPv6 Up n/a 100 Yes
Backup Addr: fe80::16:20:fe
int-evi-203 2 No Up Master 100 1
IPv6 Up n/a 100 Yes
Backup Addr: fe80::16:23:fe
-------------------------------------------------------------------------------
Instances : 4
===============================================================================
Operation
On PE-4, VPRN 20 has one interface bound to VPLS 202 and another interface bound to VPLS 203. CE-41 is attached to VPLS 202, whereas CE-43 is attached to VPLS 203. When ping messages are sent from CE-41 to CE-43, or vice versa, the messages go via VPRN 20, which has routes to both CEs, as follows:
[/]
A:admin@PE-4# show router 20 route-table
===============================================================================
Route Table (Service: 20)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
172.16.20.0/24 Local Local 00h19m37s 0
int-evi-202 0
172.16.23.0/24 Local Local 00h19m37s 0
int-evi-203 0
-------------------------------------------------------------------------------
No. of Routes: 2
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
[/]
A:admin@PE-4# show router 20 route-table ipv6
===============================================================================
IPv6 Route Table (Service: 20)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
2001:db8:16::20:0/120 Local Local 00h19m36s 0
int-evi-202 0
2001:db8:16::23:0/120 Local Local 00h19m36s 0
int-evi-203 0
-------------------------------------------------------------------------------
No. of Routes: 2
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
When traffic is sent between CE-11 and CE-41, which are both associated with VPLS 202, the forwarding is done by the VPLS and not via the VPRN. The FDB for VPLS 202 on PE-3 is as follows:
[/]
A:admin@PE-3# show service id 202 fdb detail
===============================================================================
Forwarding Database, Service 202
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
202 00:00:01:00:00:11 sap:lag-1:20 L/0 02/24/22 15:28:41
202 00:00:01:00:00:16 sap:lag-1:20 L/0 02/24/22 15:28:45
202 00:00:04:00:00:41 mpls-1: Evpn 02/24/22 15:28:40
192.0.2.4:524281
ldp:65539
202 00:00:5e:00:01:01 cpm Intf 02/24/22 15:09:03
202 00:00:5e:00:02:01 cpm Intf 02/24/22 15:09:03
202 00:ca:fe:00:02:02 mpls-1: EvpnS:P 02/24/22 15:09:04
192.0.2.2:524276
ldp:65538
202 00:ca:fe:00:02:03 cpm Intf 02/24/22 15:09:03
202 00:ca:fe:00:02:04 mpls-1: EvpnS:P 02/24/22 15:09:14
192.0.2.4:524281
ldp:65539
-------------------------------------------------------------------------------
No. of MAC Entries: 8
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
MAC 00:00:01:00:00:11 corresponds to CE-11 and is learned on SAP lag-1:20 on PE-3 and advertised via an EVPN MAC route to the BGP peers. MAC 00:00:04:00:00:41 corresponds to CE-41 and was advertised via an EVPN MAC route from PE-4, where the MAC was learned on SAP 1/2/1:41 of VPLS 202, as shown in the following FDB:
[/]
A:admin@PE-4# show service id 202 fdb detail
===============================================================================
Forwarding Database, Service 202
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
202 00:00:01:00:00:11 eES: Evpn 02/24/22 15:28:41
01:00:00:00:00:23:00:00:00:01
202 00:00:01:00:00:16 eES: Evpn 02/24/22 15:28:45
01:00:00:00:00:23:00:00:00:01
202 00:00:04:00:00:41 sap:1/2/1:41 L/90 02/24/22 15:28:40
202 00:00:5e:00:01:01 cpm Intf 02/24/22 15:09:14
202 00:00:5e:00:02:01 cpm Intf 02/24/22 15:09:14
202 00:ca:fe:00:02:02 mpls-1: EvpnS:P 02/24/22 15:09:16
192.0.2.2:524276
ldp:65538
202 00:ca:fe:00:02:03 mpls-1: EvpnS:P 02/24/22 15:09:16
192.0.2.3:524276
ldp:65539
202 00:ca:fe:00:02:04 cpm Intf 02/24/22 15:09:14
-------------------------------------------------------------------------------
No. of MAC Entries: 8
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
CE-43's MAC address is not present in VPLS 202's FDB. VPLS 203's FDB shows the CE-43's MAC address, but not CE-41's. Traffic between these two VPLS services goes via the VPRN and cannot use Layer 2 forwarding.
[/]
A:admin@PE-4# show service id 203 fdb detail
===============================================================================
Forwarding Database, Service 203
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
203 00:00:04:00:00:43 sap:1/2/1:43 L/90 02/24/22 15:28:40
203 00:00:5e:00:01:02 cpm Intf 02/24/22 15:09:14
203 00:00:5e:00:02:02 cpm Intf 02/24/22 15:09:14
203 00:ca:fe:00:23:04 cpm Intf 02/24/22 15:09:14
-------------------------------------------------------------------------------
No. of MAC Entries: 4
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
EVPN-MPLS R-VPLS with single-active MH
EVPN-MPLS R-VPLS with single-active multi-homing shows the example topology with single-active multi-homing ES "SA-ESI-23". The difference is that the ES is single-active and SDPs are used instead of a LAG.
The configuration is modified as follows:
-
LAG 1 is removed from MTU-1, PE-2, and PE-3.
-
Network interfaces are configured between MTU-1 and PE-2/PE-3 with IS-IS and LDP enabled.
-
SDPs are configured.
-
Ethernet segment "SA-ESI-23" is configured as single-active multi-homing. The SDP is associated with this ES.
-
VPLS 202 on PE-2 and PE-3 no longer has a SAP, but a spoke-SDP instead.
-
No changes are required on VPRN 20 or VPLS 200.
The service configuration on PE-2 is as follows. The configuration on PE-3 is similar. No changes are required on PE-4.
# on PE-2:
configure {
service {
system {
bgp {
evpn {
ethernet-segment "SA-ESI-23" {
admin-state enable
esi 01:00:00:00:00:23:00:00:00:02
multi-homing-mode single-active
df-election {
es-activation-timer 3
}
association {
sdp 21 {
}
}
}
}
}
}
---snip---
sdp 21 {
admin-state enable
delivery-type mpls
ldp true
far-end {
ip-address 192.0.2.1
}
}
---snip---
vprn "VPRN 20" {
admin-state enable
service-id 20
customer "1"
interface "int-evi-200" {
vpls "evi-200" {
evpn-tunnel {
}
}
ipv6 {
}
}
interface "int-evi-202" {
mac 00:ca:fe:00:02:02
ipv4 {
primary {
address 172.16.20.2
prefix-length 24
}
vrrp 1 {
backup [172.16.20.254]
passive true
ping-reply true
traceroute-reply true
}
}
vpls "evi-202" {
}
ipv6 {
link-local-address {
address fe80::16:20:2
duplicate-address-detection false
}
address 2001:db8:16::20:2 {
prefix-length 120
}
vrrp 1 {
backup [fe80::16:20:fe]
passive true
ping-reply true
traceroute-reply true
}
}
}
ipv6 {
router-advertisement {
interface "int-evi-202" {
admin-state enable
use-virtual-mac true
}
}
}
}
vpls "evi-200" {
admin-state enable
service-id 200
customer "1"
routed-vpls {
}
bgp 1 {
}
bgp-evpn {
evi 200
routes {
ip-prefix {
advertise true
}
}
mpls 1 {
admin-state enable
auto-bind-tunnel {
resolution any
}
}
}
}
vpls "evi-202" {
admin-state enable
service-id 202
customer "1"
routed-vpls {
}
bgp 1 {
}
bgp-evpn {
evi 202
mpls 1 {
admin-state enable
auto-bind-tunnel {
resolution any
}
}
}
spoke-sdp 21:20 {
}
}
PE-2 is the Designated Forwarder (DF) in the single-active ES, as shown in the following output:
[/]
A:admin@PE-2# show service id 202 ethernet-segment
No sap entries
===============================================================================
SDP Ethernet-Segment Information
===============================================================================
SDP Eth-Seg Status
-------------------------------------------------------------------------------
21:20 SA-ESI-23 DF
===============================================================================
No vxlan instance entries
[/]
A:admin@PE-3# show service id 202 ethernet-segment
No sap entries
===============================================================================
SDP Ethernet-Segment Information
===============================================================================
SDP Eth-Seg Status
-------------------------------------------------------------------------------
31:20 SA-ESI-23 NDF
===============================================================================
No vxlan instance entries
When traffic has been sent between CE-11 and CE-41, the FDB on PE-2 is as follows. MAC address 00:00:01:00:00:11 corresponds to CE-11 and has been learned on spoke-SDP 21:20; MAC address 00:00:04:00:00:41 corresponds to CE-41 and has been advertised by PE-4 in an EVPN-MAC route.
[/]
A:admin@PE-2# show service id 202 fdb detail
===============================================================================
Forwarding Database, Service 202
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
202 00:00:01:00:00:11 sdp:21:20 L/30 02/24/22 15:36:52
202 00:00:01:00:00:16 sdp:21:20 L/30 02/24/22 15:37:00
202 00:00:04:00:00:41 mpls-1: Evpn 02/24/22 15:36:56
192.0.2.4:524281
ldp:65539
202 00:00:5e:00:01:01 cpm Intf 02/24/22 15:08:50
202 00:00:5e:00:02:01 cpm Intf 02/24/22 15:08:50
202 00:ca:fe:00:02:02 cpm Intf 02/24/22 15:08:50
202 00:ca:fe:00:02:03 mpls-1: EvpnS:P 02/24/22 15:09:03
192.0.2.3:524276
ldp:65538
202 00:ca:fe:00:02:04 mpls-1: EvpnS:P 02/24/22 15:09:14
192.0.2.4:524281
ldp:65539
-------------------------------------------------------------------------------
No. of MAC Entries: 8
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
When the SDP between MTU-1 and DF PE-2 goes down, traffic from CE-41 to CE-11 is forwarded by PE-4 to DF PE-2. PE-2 cannot forward the packets to CE-11 directly, and will forward the packets to its ES peer PE-3. PE-3 will forward to CE-11 even if the MAC SA matches its own vMAC. Virtual MACs bypass the R-VPLS interface protection, so traffic can be forwarded between the PEs without being dropped.
Conclusion
EVPN can be used as the unified control plane VPN technology, not only for providing Layer 2 connectivity, but also Layer 3 (inter-subnet forwarding). EVPN for MPLS tunnels, along with multi-homing and passive VRRP, provides efficient layer-2/layer-3 connectivity to distributed hosts and routers.