EVPN R-VPLS Attached to IES
This chapter provides information about EVPN R-VPLS attached to IES.
Topics in this chapter include:
Applicability
This chapter was initially written based on SR OS Release 16.0.R3, but the MD-CLI configuration in the current edition corresponds to SR OS Release 23.10.R1.
Overview
R-VPLS services are often terminated on VPRN services. However, in some cases, R-VPLS services need to be terminated on IES services so that the traffic can be routed via the GRT. This is also supported for EVPN R-VPLS services.
Configuration
In this section, the following examples are configured:
-
EVPN-VXLAN R-VPLS attached to IES without multi-homing
-
EVPN-MPLS R-VPLS attached to IES with all-active and single-active multi-homing
EVPN-VXLAN R-VPLS attached to IES
EVPN-VXLAN R-VPLS attached to IES shows the example topology with EVPN-VXLAN configured on PE-2 and PE-4 and EVPN-VXLAN R-VPLSs 1 and 2 attached to IES 12 on PE-4.
CE-1 is in Autonomous System (AS) 64501 and the other nodes are in AS 64500.
The initial configuration includes the following:
-
Cards, MDAs, ports
-
Router interfaces
-
IS-IS between PE-2 and PE-4
Configuration on PE-2
On PE-2, BGP is configured for the EVPN address family, as follows:
# on PE-2:
configure {
router "Base" {
autonomous-system 64500
bgp {
rapid-withdrawal true
split-horizon true
rapid-update {
evpn true
}
group "internal-evpn" {
type internal
peer-as 64500
family {
evpn true
}
}
neighbor "192.0.2.4" {
group "internal-evpn"
}
}
EVPN-VXLAN VPLS 1 is an ordinary VPLS on PE-2, not an R-VPLS, and configured as follows. CE-1 is attached to SAP 1/1/c2/1:1 on PE-2.
# on PE-2:
configure {
service {
vpls "VPLS-1" {
admin-state enable
service-id 1
customer "1"
vxlan {
instance 1 {
vni 1
}
}
bgp 1 {
}
bgp-evpn {
evi 1
vxlan 1 {
admin-state enable
vxlan-instance 1
}
}
sap 1/1/c2/1:1 {
}
}
Configuration on PE-4
On PE-4, R-VPLS "evi-1" is configured as follows. CE-41 is attached to the SAP. The configuration of R-VPLS "evi-2" is similar.
# on PE-4:
configure {
service {
vpls "evi-1" {
admin-state enable
description "EVPN-VXLAN R-VPLS evi-1"
service-id 1
customer "1"
vxlan {
instance 1 {
vni 1
}
}
routed-vpls {
}
bgp 1 {
}
bgp-evpn {
evi 1
vxlan 1 {
admin-state enable
vxlan-instance 1
}
}
sap pxc-1.a:1 {
}
}
Both R-VPLSs are attached to IES 12, which is configured as follows. Interface "int-evi-1" gets IP address 10.0.1.4/24 and interface "int-evi-2" gets IP address 10.0.2.4/24; these addresses are used as next-hop in default static routes on CE-1, CE-41, and CE-42.
# on PE-4:
configure {
service {
ies "IES-12" {
admin-state enable
service-id 12
customer "1"
interface "int-evi-1" {
mac 00:00:00:00:01:04
vpls "evi-1" {
}
ipv4 {
primary {
address 10.0.1.4
prefix-length 24
}
}
}
interface "int-evi-2" {
mac 00:00:00:00:02:04
vpls "evi-2" {
}
ipv4 {
primary {
address 10.0.2.4
prefix-length 24
}
}
}
}
The BGP configuration on PE-4 includes an internal EVPN session with PE-2 (neighbor 192.0.2.2), an internal IPv4 session with CE-42 (neighbor 10.0.2.42), and an external IPv4 session with CE-1 (neighbor 10.0.1.1), as follows:
# on PE-4:
configure {
router "Base" {
bgp {
rapid-withdrawal true
peer-ip-tracking true
split-horizon true
rapid-update {
evpn true
}
group "external-ipv4" {
type external
peer-as 64501
family {
ipv4 true
}
local-as {
as-number 64500
}
}
group "internal-evpn" {
type internal
family {
evpn true
}
}
group "internal-ipv4" {
type internal
family {
ipv4 true
}
}
neighbor "10.0.1.1" {
group "external-ipv4"
ebgp-default-reject-policy {
import false
export false
}
}
neighbor "10.0.2.42" {
group "internal-ipv4"
}
neighbor "192.0.2.2" {
group "internal-evpn"
}
}
In this example, CE-41 is emulated as VPRN "CE-41" on PE-4. CE-41 is attached via port cross-connect (PXC) to R-VPLS "evi-1". The default static route has next-hop 10.0.1.4 on interface "int-evi-1" in IES 12. CE-41 has an EBGP-IPv4 session configured with neighbor CE-1 (10.0.1.1); CE-41 exports prefix 172.16.41.0/24 to CE-1. The configuration of VPRN "CE-41" on PE-4 is as follows:
# on PE-4:
configure {
service {
vprn "CE-41" {
admin-state enable
description "CE-41 attached to R-VPLS evi-1 on PE-4"
service-id 41
customer "1"
autonomous-system 64500
bgp {
router-id 10.0.1.41
rapid-withdrawal true
peer-ip-tracking true
split-horizon true
group "external" {
type external
peer-as 64501
family {
ipv4 true
}
local-as {
as-number 64500
}
export {
policy ["export-bgp-ipv4-41"]
}
}
neighbor "10.0.1.1" {
group "external"
ebgp-default-reject-policy {
import false
}
}
}
interface "int-1_41" {
mac 00:00:00:00:01:41
ipv4 {
primary {
address 10.0.1.41
prefix-length 24
}
}
sap pxc-1.b:1 {
}
}
interface "lo1" {
loopback true
mac 00:00:00:04:41:41
ipv4 {
primary {
address 172.16.41.41
prefix-length 24
}
}
}
static-routes {
route 0.0.0.0/0 route-type unicast {
next-hop "10.0.1.4" {
admin-state enable
}
}
}
}
CE-42 is emulated as VPRN "CE-42" on PE-4. CE-42 is attached via PXC to R-VPLS "evi-2". The default static route has next-hop equal to 10.0.2.4 on interface "int-evi-2" in IES 12. An IBGP-IPv4 session is configured to this IES interface (neighbor 10.0.2.4). CE-42 exports prefix 172.16.42.0/24 to this IES interface on PE-4. The configuration of VPRN "CE-42" on PE-4 is as follows:
# on PE-4:
configure {
service {
vprn "CE-42" {
admin-state enable
description "CE-42 attached to R-VPLS evi-2 on PE-4"
service-id 42
customer "1"
autonomous-system 64500
bgp {
router-id 10.0.2.42
rapid-withdrawal true
peer-ip-tracking true
split-horizon true
group "internal-ipv4" {
type internal
family {
ipv4 true
}
export {
policy ["export-bgp-ipv4-42"]
}
}
neighbor "10.0.2.4" {
group "internal-ipv4"
}
}
interface "int-2_42" {
mac 00:00:00:00:02:42
ipv4 {
primary {
address 10.0.2.42
prefix-length 24
}
}
sap pxc-1.b:2 {
}
}
interface "int-test42" {
mac 00:00:00:04:42:42
ipv4 {
primary {
address 172.16.42.42
prefix-length 24
}
}
sap pxc-1.b:42 {
}
}
static-routes {
route 0.0.0.0/0 route-type unicast {
next-hop "10.0.2.4" {
admin-state enable
}
}
}
}
The export policies are configured as follows:
# on PE-4:
configure {
policy-options {
prefix-list "172.16.41.x" {
prefix 172.16.41.0/24 type exact {
}
}
prefix-list "172.16.42.x" {
prefix 172.16.42.0/24 type exact {
}
}
policy-statement "export-bgp-ipv4-41" {
entry 10 {
from {
prefix-list ["172.16.41.x"]
}
action {
action-type accept
}
}
}
policy-statement "export-bgp-ipv4-42" {
entry 10 {
from {
prefix-list ["172.16.42.x"]
}
action {
action-type accept
}
}
}
}
Configuration on CE-1
On CE-1, the following static route is configured with next-hop 10.0.1. 4, which is the address on the interface "int-evi-1" in IES 12 on PE-4:
# on CE-1:
configure {
router "Base" {
static-routes {
route 0.0.0.0/0 route-type unicast {
next-hop "10.0.1.4" {
admin-state enable
}
}
}
The following loopback address is configured on CE-1 for test purposes:
# on CE-1:
configure
router "Base" {
interface "lo1" {
loopback
ipv4 {
primary {
address 172.16.1.1
prefix-length 24
}
}
}
On CE-1, EBGP-IPv4 sessions are configured to the IES interface "int-evi-1" on PE-4 (neighbor 10.0.1.4) and to CE-41 (neighbor 10.0.1.41) for the IPv4 address family. CE-1 exports prefix 172.16.1.0/24 to its peers. The BGP configuration is as follows:
# on CE-1:
configure {
policy-options {
prefix-list "172.16.1.x" {
prefix 172.16.1.0/24 type exact {
}
}
policy-statement "export-bgp-ipv4" {
entry 10 {
from {
prefix-list ["172.16.1.x"]
}
action {
action-type accept
}
}
}
}
router "Base" {
bgp {
rapid-withdrawal true
peer-ip-tracking true
split-horizon true
group "external" {
type external
peer-as 64500
family {
ipv4 true
}
ebgp-default-reject-policy {
import false
}
local-as {
as-number 64501
}
export {
policy ["export-bgp-ipv4"]
}
}
neighbor "10.0.1.4" {
group "external"
}
neighbor "10.0.1.41" {
group "external"
}
}
Verification
On PE-4, the following shows that five BGP sessions are established:
-
EBGP-IPv4 session with neighbor 10.0.1.1 (CE-1) from the base router
-
IBGP-IPv4 session with neighbor 10.0.2.42 (CE-42) from the base router
-
IBGP-EVPN session with neighbor 192.0.2.2 (PE-2) from the base router
-
EBGP-IPv4 session with neighbor 10.0.1.1 (CE-1) from VPRN "CE-41"
-
IBGP-IPv4 session to IES interface "int-evi-2" (10.0.2.4) from VPRN "CE-42"
Routes have been exchanged between the peers. The EBGP-IPv4 sessions are established using R-VPLS "evi-1".
[/]
A:admin@PE-4# show router bgp summary all
===============================================================================
BGP Summary
===============================================================================
Legend : D - Dynamic Neighbor
===============================================================================
Neighbor
Description
ServiceId AS PktRcvd InQ Up/Down State|Rcv/Act/Sent (Addr Family)
PktSent OutQ
-------------------------------------------------------------------------------
10.0.1.1
Def. Inst 64501 8 0 00h01m57s 1/1/1 (IPv4)
9 0
10.0.2.42
Def. Inst 64500 9 0 00h02m02s 1/1/1 (IPv4)
10 0
192.0.2.2
Def. Inst 64500 12 0 00h02m56s 2/2/7 (Evpn)
16 0
10.0.1.1
41 64501 9 0 00h02m02s 1/1/1 (IPv4)
9 0
10.0.2.4
42 64500 9 0 00h02m02s 1/1/1 (IPv4)
9 0
-------------------------------------------------------------------------------
On PE-4, the following route table includes the prefixes 10.0.1.0/24 of interface "int-evi-1" and 10.0.2.0/24 of "int-evi-2" in IES 12. Also, it includes the remote prefixes 172.16.1.0/24 and 172.16.42.0, which are received as BGP IPv4 routes from CE-1 and CE-42.
[/]
A:admin@PE-4# show router route-table
===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
10.0.1.0/24 Local Local 00h08m21s 0
int-evi-1 0
10.0.2.0/24 Local Local 00h08m21s 0
int-evi-2 0
172.16.1.0/24 Remote BGP 00h07m28s 170
10.0.1.1 0
172.16.42.0/24 Remote BGP 00h04m57s 170
10.0.2.42 0
192.0.2.2/32 Remote ISIS 00h14m38s 18
192.168.24.1 10
192.0.2.4/32 Local Local 00h14m52s 0
system 0
192.168.24.0/30 Local Local 00h14m52s 0
int-PE-4-PE-2 0
-------------------------------------------------------------------------------
No. of Routes: 7
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
The following route table for CE-41 includes the remote prefix 172.16.1.0/24 received as BGP IPv4 route with next-hop 10.0.1.1. CE-1 and CE-41 are both in subnet 10.0.1.0/24.
[/]
A:admin@PE-4# show router service-name "CE-41" route-table
===============================================================================
Route Table (Service: 41)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
0.0.0.0/0 Remote Static 00h13m37s 5
10.0.1.4 1
10.0.1.0/24 Local Local 00h13m37s 0
int-1_41 0
172.16.1.0/24 Remote BGP 00h12m34s 170
10.0.1.1 0
172.16.41.0/24 Local Local 00h13m37s 0
lo1 0
-------------------------------------------------------------------------------
No. of Routes: 4
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
Likewise, the following route table for CE-42 includes the remote prefix 172.16.1.0/24 received as BGP IPv4 route, but the next-hop is 10.0.2.4 instead of 10.0.1.1, because CE-42 is in subnet 10.0.2.0/24 whereas CE-1 is in subnet 10.0.1.0/24. Routing between the subnets 10.0.2.0/24 and 10.0.1.0/24 needs to be done in IES 12 on PE-4.
[/]
A:admin@PE-4# show router service-name "CE-42" route-table
===============================================================================
Route Table (Service: 42)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
0.0.0.0/0 Remote Static 00h13m37s 5
10.0.2.4 1
10.0.2.0/24 Local Local 00h13m37s 0
int-2_42 0
172.16.1.0/24 Remote BGP 00h07m17s 170
10.0.2.4 1
172.16.42.0/24 Local Local 00h05m19s 0
int-test42 0
-------------------------------------------------------------------------------
No. of Routes: 4
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
The following traceroute from CE-41 (172.16.41.41) to CE-1 (172.16.1.1) shows that no intermediate hops are required:
[/]
A:admin@PE-4# traceroute 172.16.1.1 router-instance "CE-41" source-address 172.16.41.41
traceroute to 172.16.1.1 from 172.16.41.41, 30 hops max, 40 byte packets
1 172.16.1.1 (172.16.1.1) 4.58 ms 4.88 ms 4.80 ms
The following traceroute from CE-42 (172.16.42.42) to CE-1 (172.16.1.1) shows the IP address 10.0.2.4 on the interface "int-evi-2" in IES 12 as an intermediate hop:
[/]
A:admin@PE-4# traceroute 172.16.1.1 router-instance "CE-42" source-address 172.16.42.42
traceroute to 172.16.1.1 from 172.16.42.42, 30 hops max, 40 byte packets
1 10.0.2.4 (10.0.2.4) 1.62 ms 2.45 ms 2.46 ms
2 172.16.1.1 (172.16.1.1) 4.89 ms 4.44 ms 4.83 ms
The following ARP table on PE-4 includes entries for IP addresses in subnets 10.0.1.0/24 on interface "int-evi-1" and 10.0.2.0/24 on interface "int-evi-2":
[/]
A:admin@PE-4# show router arp
===============================================================================
ARP Table (Router: Base)
===============================================================================
IP Address MAC Address Expiry Type Interface
-------------------------------------------------------------------------------
192.0.2.4 00:04:fe:00:00:00 00h00m00s Oth system
192.168.24.1 02:0e:01:01:00:01 03h42m44s Dyn[I] int-PE-4-PE-2
192.168.24.2 02:1a:01:01:00:0b 00h00m00s Oth[I] int-PE-4-PE-2
10.0.1.1 00:00:00:00:01:01 03h49m14s Dyn[I] int-evi-1
10.0.1.4 00:00:00:00:01:04 00h00m00s Oth[I] int-evi-1
10.0.1.41 00:00:00:00:01:41 03h59m17s Dyn[I] int-evi-1
10.0.2.4 00:00:00:00:02:04 00h00m00s Oth[I] int-evi-2
10.0.2.42 00:00:00:00:02:42 03h49m10s Dyn[I] int-evi-2
-------------------------------------------------------------------------------
No. of ARP Entries: 8
===============================================================================
The forwarding database (FDB) for R-VPLS 1 on PE-4 includes the MAC addresses corresponding to IP addresses 10.0.1.1, 10.0.1.4, and 10.0.1.41:
[/]
A:admin@PE-4# show service id "evi-1" fdb detail
===============================================================================
Forwarding Database, Service 1
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
1 00:00:00:00:01:01 vxlan-1: Evpn 11/09/23 08:33:42
192.0.2.2:1
1 00:00:00:00:01:04 cpm Intf 11/09/23 08:33:42
1 00:00:00:00:01:41 sap:pxc-1.a:1 LT/0 11/09/23 08:28:26
-------------------------------------------------------------------------------
No. of MAC Entries: 3
-------------------------------------------------------------------------------
Legend:L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf T=Trusted
===============================================================================
MAC address 00:00:00:00:01:01, which corresponds to IP address 10.0.1.1 on CE-1, is advertised in an EVPN MAC route by PE-2:
[/]
A:admin@PE-4# show router bgp routes evpn mac
===============================================================================
BGP Router ID:192.0.2.4 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP EVPN MAC Routes
===============================================================================
Flag Route Dist. MacAddr ESI
Tag Mac Mobility Label1
Ip Address
NextHop
-------------------------------------------------------------------------------
u*>i 192.0.2.2:1 00:00:00:00:01:01 ESI-0
0 Seq:0 VNI 1
n/a
192.0.2.2
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
EVPN-MPLS R-VPLS attached to IES
Example topology for EVPN-MPLS R-VPLS attached to IES shows the example topology for EVPN-MPLS R-VPLS attached to IES. All-active multi-homing (AA MH) is configured on PE-2 and PE-3, while single-active (SA) MH is configured on PE-4 and PE-5. R-VPLS "evi-3" is configured on all PEs. IES 30 is configured on PE-2 and PE-3, whereas IES 34 is configured on PE-4 and PE-5. On MTU-6, "VPLS-3" and "VPLS-4" are regular VPLSs, not routed.
The initial configuration on the nodes includes:
-
Cards, MDAs, ports
-
LAG "lag-1" on CE-1, PE-2, PE-3
-
Router interfaces between the PEs and toward MTU-6
-
IS-IS on these interfaces (alternatively, OSPF can be configured)
-
LDP on these interfaces
-
BGP configured for the EVPN address family on the PEs. PE-2 is the RR and has the following BGP configuration:
# on PE-2:
configure {
router "Base" {
bgp {
rapid-withdrawal true
peer-ip-tracking true
split-horizon true
rapid-update {
evpn true
}
group "internal-evpn" {
peer-as 64500
family {
evpn true
}
cluster {
cluster-id 192.0.2.2
}
}
neighbor "192.0.2.3" {
group "internal-evpn"
}
neighbor "192.0.2.4" {
group "internal-evpn"
}
neighbor "192.0.2.5" {
group "internal-evpn"
}
}
Configuration on PE-2 and PE-3
The service configuration on PE-2 and PE-3 is almost identical; only the IP address on the IES interface "int-evi-3" is different. The AA MH ES "ESI-23_3" is configured as follows, with LAG 1 and dot1q tag 3, so it is only applicable to VPLS "evi-3".
# on PE-2, PE-3:
configure {
service {
system {
bgp {
evpn {
ethernet-segment "ESI-23_3" {
admin-state enable
type virtual
esi 0x01000000002300030301
multi-homing-mode all-active
df-election {
es-activation-timer 3
}
association {
lag "lag-1" {
virtual-ranges {
dot1q {
q-tag 3 {
end 3
}
}
}
}
}
}
}
}
}
R-VPLS "evi-3" has EVPN-MPLS enabled and is configured on PE-2 and PE-3, as follows. SAP lag-1:3 matches the configured LAG and the q-tag range for ESI-23_3.
# on PE-2, PE-3:
configure {
service {
vpls "evi-3" {
admin-state enable
service-id 3
customer "1"
routed-vpls {
}
bgp 1 {
}
bgp-evpn {
evi 3
mpls 1 {
admin-state enable
ecmp 2
auto-bind-tunnel {
resolution any
}
}
}
sap lag-1:3 {
}
}
The following is the IES configuration on PE-2. In this example, IES 30 is only configured to demonstrate EVPN all-active multi-homing on R-VPLS with IES. If it were removed, everything still works and the connectivity between the CEs remains.
# on PE-2:
configure {
service {
ies "IES-30" {
admin-state enable
service-id 30
customer "1"
interface "int-evi-3" {
mac 00:00:00:00:03:02
vpls "evi-3" {
}
ipv4 {
primary {
address 10.0.3.2
prefix-length 24
}
}
}
}
The IES configuration on PE-3 is similar, only using IP address 10.0.3.3/24.
Configuration on PE-4 and PE-5
On PE-4, SDP 46 is configured toward MTU-6. An SA MH ES "ESI-45" is configured using this SDP, as follows:
# on PE-4:
configure {
service {
system {
bgp {
evpn {
ethernet-segment "ESI-45" {
admin-state enable
esi 0x01000000004500000001
multi-homing-mode single-active
df-election {
es-activation-timer 3
}
association {
sdp 46 {
}
}
}
}
}
}
sdp 46 {
admin-state enable
delivery-type mpls
ldp true
far-end {
ip-address 192.0.2.6
}
}
The configuration is similar on PE-5. SDP 56 is configured toward MTU-6 and ES "ESI-45" is configured with SDP 56 instead.
On PE-4, R-VPLSs "evi-3" and "evi-4" are configured with EVPN-MPLS, as follows:
# on PE-4:
configure {
service {
vpls "evi-3" {
admin-state enable
description "EVPN-MPLS R-VPLS 3"
service-id 3
customer "1"
routed-vpls {
}
bgp 1 {
}
bgp-evpn {
evi 3
mpls 1 {
admin-state enable
ecmp 2
auto-bind-tunnel {
resolution any
}
}
}
spoke-sdp 46:3 {
}
}
vpls "evi-4" {
admin-state enable
description "EVPN-MPLS R-VPLS 4"
service-id 4
customer "1"
routed-vpls {
}
bgp 1 {
}
bgp-evpn {
evi 4
mpls 1 {
admin-state enable
ecmp 2
auto-bind-tunnel {
resolution any
}
}
}
spoke-sdp 46:4 {
}
}
The configuration is similar on PE-5; only the spoke-SDPs are different (spoke-SDP 56:3 and 56:4).
On PE-4, IES 34 is configured with interfaces "int-evi-3" and "int-evi-4", as follows. Passive VRRP is configured on both interfaces. With passive VRRP configured on both PE-4 and PE-5, both PEs behave as primary.
# on PE-4:
configure {
service {
ies "IES-34" {
admin-state enable
service-id 34
customer "1"
interface "int-evi-3" {
mac 00:00:00:00:03:04
vpls "evi-3" {
}
ipv4 {
primary {
address 10.0.3.4
prefix-length 24
}
vrrp 1 {
backup [10.0.3.254]
passive true
ping-reply true
traceroute-reply true
}
}
}
interface "int-evi-4" {
mac 00:00:00:00:04:04
vpls "evi-4" {
}
ipv4 {
primary {
address 10.0.4.4
prefix-length 24
}
vrrp 1 {
backup [10.0.4.254]
passive true
ping-reply true
traceroute-reply true
}
}
}
}
The configuration of IES 34 is similar on PE-5, but the interface IP addresses are different: 10.0.3.5/24 and 10.0.4.5/24. The MAC addresses are also different.
To enable routing between CE-1 and CE-64 in a different subnet, BGP sessions are established with CE-1 (neighbor 10.0.3.1 in AS 64501) and CE-64 (neighbor 10.0.4.64 in AS 64500) for the IPv4 address family. The CEs export prefixes, but no export policy needs to be configured on PE-4 and PE-5. The BGP configuration on PE-4 is as follows:
# on PE-4:
configure {
router "Base" {
bgp {
rapid-withdrawal true
peer-ip-tracking true
split-horizon true
rapid-update {
evpn true
}
group "external" {
type external
peer-as 64501
family {
ipv4 true
}
local-as {
as-number 64500
}
}
group "internal-evpn" {
type internal
family {
evpn true
}
}
group "internal-ipv4" {
peer-as 64500
local-address 10.0.3.4
family {
ipv4 true
}
}
neighbor "10.0.3.1" {
group "external"
ebgp-default-reject-policy {
import false
export false
}
}
neighbor "10.0.4.64" {
group "internal-ipv4"
}
neighbor "192.0.2.2" {
group "internal-evpn"
}
}
The BGP configuration on PE-5 is almost identical; the local address is 10.0.3.5 instead.
Configuration on CE-1
The configuration on CE-1 includes the following:
-
Router interface to VPLS "evi-3" (ESI-23_3) with IP address 10.0.3.1/24 and LAG-1:3 assigned to it
-
Loopback interface with IP address 172.16.1.1/24 for test purposes
-
Static default route with next-hop 10.0.3.254, which is the VRRP backup address for IES interface "int-evi-3" on PE-4 and PE-5
-
Export policy to export prefix 172.16.1.0/24
-
BGP sessions for the IPv4 address family toward PE-4 (10.0.3.4), PE-5 (10.0.3.5), and CE-63 (10.0.3.63)
The router configuration on CE-1 is as follows:
# on CE-1:
configure {
policy-options {
prefix-list "172.16.1.x" {
prefix 172.16.1.0/24 type exact {
}
}
policy-statement "export-bgp-ipv4" {
entry 10 {
from {
prefix-list ["172.16.1.x"]
}
action {
action-type accept
}
}
}
}
router "Base" {
autonomous-system 64501
interface "int-CE-1-evi-3_ES-23" {
port lag-1:3
ipv4 {
primary {
address 10.0.3.1
prefix-length 24
}
}
}
interface "lo1" {
loopback
ipv4 {
primary {
address 172.16.1.1
prefix-length 24
}
}
}
interface "system" {
ipv4 {
primary {
address 192.0.2.1
prefix-length 32
}
}
}
bgp {
router-id 10.0.3.1
rapid-withdrawal true
peer-ip-tracking true
split-horizon true
group "external" {
type external
peer-as 64500
family {
ipv4 true
}
ebgp-default-reject-policy {
import false
}
local-as {
as-number 64501
}
export {
policy ["export-bgp-ipv4"]
}
}
neighbor "10.0.3.4" {
group "external"
}
neighbor "10.0.3.5" {
group "external"
}
neighbor "10.0.3.63" {
group "external"
}
}
static-routes {
route 0.0.0.0/0 route-type unicast {
next-hop "10.0.3.254" {
admin-state enable
}
}
}
Configuration on MTU-6
The configuration on MTU-6 includes the following:
-
Router interfaces
-
IS-IS
-
LDP
-
One policy to export prefix 172.16.63.0/24 and another policy to export prefix 172.16.64.0/24
-
BGP is not configured in the base router
The following service configuration on MTU-6 includes the SDP configuration and the VPLSs "VPLS-3" and "VPLS-4", which are not routed:
# on MTU-6:
configure {
service {
sdp 64 {
admin-state enable
delivery-type mpls
ldp true
far-end {
ip-address 192.0.2.4
}
}
sdp 65 {
admin-state enable
delivery-type mpls
ldp true
far-end {
ip-address 192.0.2.5
}
}
vpls "VPLS-3" {
admin-state enable
service-id 3
customer "1"
endpoint "CORE" {
}
spoke-sdp 64:3 {
endpoint {
name "CORE"
}
stp {
admin-state disable
}
}
spoke-sdp 65:3 {
endpoint {
name "CORE"
}
stp {
admin-state disable
}
}
sap pxc-1.a:3 {
}
}
vpls "VPLS-4" {
admin-state enable
service-id 4
customer "1"
endpoint "CORE" {
}
spoke-sdp 64:4 {
endpoint {
name "CORE"
}
stp {
admin-state disable
}
}
spoke-sdp 65:4 {
endpoint {
name "CORE"
}
stp {
admin-state disable
}
}
sap pxc-1.a:4 {
}
sap pxc-1.a:64 {
}
}
In this example, CE-63 and CE-64 are simulated by VPRNs "CE-63" and "CE-64". The default static route has next-hop 10.0.3.254, which is the VRRP backup address on interface "int-evi-3" in IES 34 on both PE-4 and PE-5. BGP is configured within CE-63 and CE-64. The prefix 172.16.63.0/24 is exported by BGP in CE-63 and prefix 172.16.64.0/24 is exported by BGP in CE-64. The configuration of CE-63 and CE-64 is as follows:
# on MTU-6:
configure {
service {
vprn "CE-63" {
admin-state enable
service-id 63
customer "1"
autonomous-system 64500
bgp {
router-id 10.0.3.63
rapid-withdrawal true
peer-ip-tracking true
split-horizon true
group "external" {
type external
peer-as 64501
family {
ipv4 true
}
ebgp-default-reject-policy {
import false
}
local-as {
as-number 64500
}
export {
policy ["export-bgp-ipv4-63"]
}
}
neighbor "10.0.3.1" {
group "external"
}
}
interface "int-1_63" {
mac 00:00:00:00:03:63
ipv4 {
primary {
address 10.0.3.63
prefix-length 24
}
}
sap pxc-1.b:3 {
}
}
interface "lo1" {
loopback true
ipv4 {
primary {
address 172.16.63.63
prefix-length 24
}
}
}
static-routes {
route 0.0.0.0/0 route-type unicast {
next-hop "10.0.3.254" {
admin-state enable
}
}
}
}
vprn "CE-64" {
admin-state enable
service-id 64
customer "1"
autonomous-system 64500
bgp {
router-id 10.0.4.64
rapid-withdrawal true
peer-ip-tracking true
split-horizon true
group "internal-ipv4" {
type internal
family {
ipv4 true
}
export {
policy ["export-bgp-ipv4-64"]
}
}
neighbor "10.0.3.4" {
group "internal-ipv4"
}
neighbor "10.0.3.5" {
group "internal-ipv4"
}
}
interface "int-2_64" {
mac 00:00:00:00:04:64
ipv4 {
primary {
address 10.0.4.64
prefix-length 24
}
}
sap pxc-1.b:4 {
}
}
interface "int-test" {
mac 00:00:00:06:64:64
ipv4 {
primary {
address 172.16.64.64
prefix-length 24
}
}
sap pxc-1.b:64 {
}
}
static-routes {
route 0.0.0.0/0 route-type unicast {
next-hop "10.0.4.254" {
admin-state enable
}
}
}
}
Verification
In the AA MH ES "ESI-23_3", PE-3 is the designated forwarder (DF) for R-VPLS "evi-3" and PE-2 is NDF, as follows:
[/]
A:admin@PE-2# show service id "evi-3" ethernet-segment
===============================================================================
SAP Ethernet-Segment Information
===============================================================================
SAP Eth-Seg Status
-------------------------------------------------------------------------------
lag-1:3 ESI-23_3 NDF
===============================================================================
No sdp entries
No vxlan instance entries
[/]
A:admin@PE-3# show service id "evi-3" ethernet-segment
===============================================================================
SAP Ethernet-Segment Information
===============================================================================
SAP Eth-Seg Status
-------------------------------------------------------------------------------
lag-1:3 ESI-23_3 DF
===============================================================================
No sdp entries
No vxlan instance entries
In the SA MH ES "ESI-45", PE-4 is NDF for R-VPLS "evi-3" and DF for R-VPLS "evi-4", as follows:
[/]
A:admin@PE-4# show service id "evi-3" ethernet-segment
No sap entries
===============================================================================
SDP Ethernet-Segment Information
===============================================================================
SDP Eth-Seg Status
-------------------------------------------------------------------------------
46:3 ESI-45 NDF
===============================================================================
No vxlan instance entries
[/]
A:admin@PE-4# show service id "evi-4" ethernet-segment
No sap entries
===============================================================================
SDP Ethernet-Segment Information
===============================================================================
SDP Eth-Seg Status
-------------------------------------------------------------------------------
46:4 ESI-45 DF
===============================================================================
No vxlan instance entries
The reverse is true for PE-5, which is DF for R-VPLS "evi-3" and NDF for R-VPLS "evi-4", as follows:
[/]
A:admin@PE-5# show service id "evi-3" ethernet-segment
No sap entries
===============================================================================
SDP Ethernet-Segment Information
===============================================================================
SDP Eth-Seg Status
-------------------------------------------------------------------------------
56:3 ESI-45 DF
===============================================================================
No vxlan instance entries
[/]
A:admin@PE-5# show service id "evi-4" ethernet-segment
No sap entries
===============================================================================
SDP Ethernet-Segment Information
===============================================================================
SDP Eth-Seg Status
-------------------------------------------------------------------------------
56:4 ESI-45 NDF
===============================================================================
No vxlan instance entries
CE-63 (VPRN 63 on MTU-6) has an external BGP IPv4 session with CE-1, whereas CE-64 (VPRN 64 on MTU-6) has internal BGP IPv4 sessions with IES interface "int-evi-3" on PE-4 and PE-5, as follows:
[/]
A:admin@MTU-6# show router service-name "CE-64" bgp summary all
===============================================================================
BGP Summary
===============================================================================
Legend : D - Dynamic Neighbor
===============================================================================
Neighbor
Description
ServiceId AS PktRcvd InQ Up/Down State|Rcv/Act/Sent (Addr Family)
PktSent OutQ
-------------------------------------------------------------------------------
10.0.3.1
63 64501 295 0 02h25m06s 1/1/1 (IPv4)
295 0
10.0.3.4
64 64500 295 0 02h25m09s 1/1/1 (IPv4)
295 0
10.0.3.5
64 64500 295 0 02h25m20s 1/0/1 (IPv4)
295 0
-------------------------------------------------------------------------------
The difference is that CE-63 (with IP address 10.0.3.63) is in the same subnet as CE-1 (10.0.3.1), whereas CE-64 is not (10.0.4.64). Routing between these subnets can be done in IES 34 on PE-4 and PE-5. CE-63 exports prefix 172.16.63.0/24 directly to CE-1, whereas CE-64 exports prefix 172.16.64.0/24 to PE-4 and PE-5 instead, which will advertise prefix 172.16.64.0/24 to their BGP peer CE-1. The following route table on CE-1 shows BGP route 172.16.63.0/63 with next-hop 10.0.3.63 (CE-63) and BGP route 172.16.64.0/64 with next-hop 10.0.3.4 (interface "int-evi-3" on PE-4):
[/]
A:admin@CE-1# show router route-table
===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
0.0.0.0/0 Remote Static 02h31m31s 5
10.0.3.254 1
10.0.3.0/24 Local Local 02h31m31s 0
int-CE-1-evi-3_ES-23 0
172.16.1.0/24 Local Local 02h51m39s 0
lo1 0
172.16.63.0/24 Remote BGP 02h25m13s 170
10.0.3.63 0
172.16.64.0/24 Remote BGP 02h24m55s 170
10.0.3.4 0
192.0.2.1/32 Local Local 02h51m39s 0
system 0
-------------------------------------------------------------------------------
No. of Routes: 6
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
In IES 34 on PE-4 (and PE-5), routing can be done between subnet 10.0.3.0/24 and 10.0.4.0/24. The following route table on PE-4 shows BGP route 172.16.1.0/24 with next-hop CE-1 (10.0.3.1) and BGP route 172.16.64.0/24 with next-hop CE-64 (10.0.4.64). The same entries occur in the route table on PE-5.
[/]
A:admin@PE-4# show router route-table
===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
10.0.3.0/24 Local Local 02h26m22s 0
int-evi-3 0
10.0.4.0/24 Local Local 02h26m22s 0
int-evi-4 0
172.16.1.0/24 Remote BGP 02h25m20s 170
10.0.3.1 0
172.16.64.0/24 Remote BGP 02h25m12s 170
10.0.4.64 0
---snip---
The route table of CE-63 (VPRN 63 on MTU-6) shows a BGP route for prefix 172.16.1.0/24 with next-hop 10.0.3.1 (CE-1), as follows:
[/]
A:admin@MTU-6# show router service-name "CE-63" route-table protocol bgp
===============================================================================
Route Table (Service: 63)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
172.16.1.0/24 Remote BGP 00h43m37s 170
10.0.3.1 0
-------------------------------------------------------------------------------
No. of Routes: 1
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
The route table of CE-64 (VPRN 64 on MTU-6) shows a BGP route for prefix 172.16.1.0/24 with next-hop 10.0.4.254 (VRRP backup address for IES interface "int-evi-4" on PE-4 and PE-5), as follows:
[/]
A:admin@MTU-6# show router service-name "CE-64" route-table
===============================================================================
Route Table (Service: 64)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
0.0.0.0/0 Remote Static 02h27m04s 5
10.0.4.254 1
10.0.4.0/24 Local Local 02h27m04s 0
int-2_64 0
172.16.1.0/24 Remote BGP 02h26m05s 170
10.0.4.254 1
172.16.64.0/24 Local Local 02h27m04s 0
int-test 0
-------------------------------------------------------------------------------
No. of Routes: 4
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
The connectivity between CE-1 and CE-63 is verified as follows:
[/]
A:admin@CE-1# ping 172.16.63.63 source-address 172.16.1.1
PING 172.16.63.63 56 data bytes
64 bytes from 172.16.63.63: icmp_seq=1 ttl=64 time=5.75ms.
64 bytes from 172.16.63.63: icmp_seq=2 ttl=64 time=5.80ms.
---snip---
The following traceroute command verifies the connectivity between CE-1 and CE-64. The intermediate hop is 10.0.3.4, the IP address of the IES interface "int-evi-3" on PE-4:
[/]
A:admin@CE-1# traceroute 172.16.64.64 source-address 172.16.1.1
traceroute to 172.16.64.64 from 172.16.1.1, 30 hops max, 40 byte packets
1 10.0.3.4 (10.0.3.4) 3.48 ms 3.87 ms 4.24 ms
2 172.16.64.64 (172.16.64.64) 6.28 ms 6.28 ms 6.13 ms
When the traceroute is launched from CE-64, the intermediate hop is 10.0.4.4, the IP address of the IES interface "int-evi-4" on PE-4:
[/]
A:admin@MTU-6# traceroute 172.16.1.1 router-instance "CE-64"
traceroute to 172.16.1.1, 30 hops max, 40 byte packets
1 10.0.4.4 (10.0.4.4) 2.91 ms 3.65 ms 3.98 ms
2 172.16.1.1 (172.16.1.1) 5.77 ms 6.44 ms 5.80 ms
The following ARP table on CE-1 contains entries for different nodes in the 10.0.3.0/24 subnet:
[/]
A:admin@CE-1# show router arp
===============================================================================
ARP Table (Router: Base)
===============================================================================
IP Address MAC Address Expiry Type Interface
-------------------------------------------------------------------------------
192.0.2.1 00:01:fe:00:00:00 00h00m00s Oth system
172.16.1.1 00:01:fe:00:00:00 00h00m00s Oth lo1
10.0.3.1 00:01:fe:00:01:41 00h00m00s Oth[I] int-CE-1-evi-3_ES-23
10.0.3.4 00:00:00:00:03:04 01h30m30s Dyn[I] int-CE-1-evi-3_ES-23
10.0.3.5 00:00:00:00:03:05 01h30m14s Dyn[I] int-CE-1-evi-3_ES-23
10.0.3.63 00:00:00:00:03:63 03h18m33s Dyn[I] int-CE-1-evi-3_ES-23
10.0.3.254 00:00:5e:00:01:01 02h10m51s Dyn[I] int-CE-1-evi-3_ES-23
-------------------------------------------------------------------------------
No. of ARP Entries: 7
===============================================================================
The ARP table on PE-4 contains entries for different nodes in subnets 10.0.3.0/24 and 10.0.4.0/24:
[/]
A:admin@PE-4# show router arp
===============================================================================
ARP Table (Router: Base)
===============================================================================
IP Address MAC Address Expiry Type Interface
-------------------------------------------------------------------------------
---snip---
10.0.3.1 00:01:fe:00:01:41 02h10m52s Dyn[I] int-evi-3
10.0.3.2 00:00:00:00:03:02 00h00m00s Evp[I] int-evi-3
10.0.3.3 00:00:00:00:03:03 00h00m00s Evp[I] int-evi-3
10.0.3.4 00:00:00:00:03:04 00h00m00s Oth[I] int-evi-3
10.0.3.5 00:00:00:00:03:05 00h00m00s Evp[I] int-evi-3
10.0.3.63 00:00:00:00:03:63 03h18m35s Dyn[I] int-evi-3
10.0.3.254 00:00:5e:00:01:01 00h00m00s Oth[I] int-evi-3
10.0.4.4 00:00:00:00:04:04 00h00m00s Oth[I] int-evi-4
10.0.4.5 00:00:00:00:04:05 00h00m00s Evp[I] int-evi-4
10.0.4.64 00:00:00:00:04:64 03h18m35s Dyn[I] int-evi-4
10.0.4.254 00:00:5e:00:01:01 00h00m00s Oth[I] int-evi-4
---snip---
-------------------------------------------------------------------------------
The FDB on PE-4 shows that MAC address 00:00:00:00:04:64-corresponding to 10.0.4.64 on CE-64-is learned on SDP 46:6, as follows.
[/]
A:admin@PE-4# show service id "evi-4" fdb detail
===============================================================================
Forwarding Database, Service 4
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
4 00:00:00:00:04:04 cpm Intf 11/09/23 08:51:50
4 00:00:00:00:04:05 mpls-1: EvpnS:P 11/09/23 08:51:56
192.0.2.5:524280
ldp:65539
4 00:00:00:00:04:64 sdp:46:4 LT/0 11/09/23 08:52:19
4 00:00:5e:00:01:01 cpm Intf 11/09/23 08:51:50
-------------------------------------------------------------------------------
No. of MAC Entries: 4
-------------------------------------------------------------------------------
Legend:L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf T=Trusted
===============================================================================
The FDB on PE-5 shows that MAC address 00:00:00:00:04:64 -corresponding to 10.0.4.64 on CE-64-is advertised as an EVPN MAC route with ESI "ESI-45", as follows:
[/]
A:admin@PE-5# show service id "evi-4" fdb detail
===============================================================================
Forwarding Database, Service 4
===============================================================================
ServId MAC Source-Identifier Type Last Change
Transport:Tnl-Id Age
-------------------------------------------------------------------------------
4 00:00:00:00:04:04 mpls-1: EvpnS:P 11/09/23 08:51:58
192.0.2.4:524280
ldp:65539
4 00:00:00:00:04:05 cpm Intf 11/09/23 08:51:56
4 00:00:00:00:04:64 eES: Evpn 11/09/23 08:52:19
01:00:00:00:00:45:00:00:00:01
4 00:00:5e:00:01:01 cpm Intf 11/09/23 08:51:56
-------------------------------------------------------------------------------
No. of MAC Entries: 4
-------------------------------------------------------------------------------
Legend:L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf T=Trusted
===============================================================================
Conclusion
With EVPN R-VPLS attached to IES services, EVPN services are connected to the base router, so the traffic can be routed in the global routing table (GRT).