EVPN R-VPLS Attached to IES

This chapter provides information about EVPN R-VPLS Attached to IES.

Topics in this chapter include:

Applicability

The information and configuration in this chapter are based on SR OS Release 16.0.R3. Routed VPLS (R-VPLS) services using EVPN-MPLS or EVPN-VXLAN can be attached to Internet Enhanced Services (IESs) in SR OS Release 16.0.R1, and later. R-VPLS using EVPN multi-homing is supported for EVPN-MPLS in SR OS Release 16.0.R1, and later. R-VPLS using multi-homing for EVPN-VXLAN will be supported in a later release; see the Release Notes.

Overview

R-VPLS services are often terminated on VPRN services. However, in some cases, R-VPLS services need to be terminated on IES services so that the traffic can be routed via the GRT. This is also supported for EVPN R-VPLS services.

In SR OS Release 16.0.R1, the following features are not supported for EVPN R-VPLSs attached to IESs:

  • Dynamic IGPs (such as IS-IS, OSPF, RIP) on the R-VPLS interface

  • EVPN tunnel on the IES interface

  • IP route advertisement on R-VPLS (for the IP/IPv6 prefix BGP-EVPN RT-5 routes)

Configuration

In this section, the following examples are configured:

  • EVPN-VXLAN R-VPLS attached to IES without multi-homing

  • EVPN-MPLS R-VPLS attached to IES with all-active and single-active multi-homing

EVPN-VXLAN R-VPLS attached to IES

EVPN-VXLAN R-VPLS attached to IES shows the example topology with EVPN-VXLAN configured on PE-2 and PE-4 and EVPN-VXLAN R-VPLSs 1 and 2 attached to IES 12 on PE-4.

Figure 1. EVPN-VXLAN R-VPLS attached to IES

CE-1 is in Autonomous System (AS) 64501 and the other nodes are in AS 64500.

The initial configuration includes the following:

  • Cards, MDAs, ports

  • Router interfaces

  • IS-IS between PE-2 and PE-4

Configuration on PE-2

On PE-2, BGP is configured for the EVPN address family, as follows:

*A:PE-2#
configure
    router
        bgp
            enable-peer-tracking
            rapid-withdrawal
            split-horizon
            rapid-update evpn
            group "internal"
                family evpn
                type internal
                peer-as 64500
                neighbor 192.0.2.4
                exit
            exit
            no shutdown
        exit

EVPN-VXLAN VPLS 1 is an ordinary VPLS on PE-2, not an R-VPLS, and configured as follows. CE-1 is attached to SAP 1/1/2:1 on PE-2.

*A:PE-2#
configure
    service
        vpls 1 name "VPLS-1" customer 1 create
            vxlan instance 1 vni 1 create
            exit
            bgp
            exit
            bgp-evpn
                evi 1
                vxlan bgp 1 vxlan-instance 1
                    no shutdown
                exit
            exit
            sap 1/1/2:1 create
            exit
            no shutdown
        exit

Configuration on PE-4

On PE-4, R-VPLS 1 is configured with service name "evi-1", as follows. CE-41 is attached to the SAP. The configuration of R-VPLS 2 is similar.

*A:PE-4#
configure
    service
        vpls 1 name "evi-1" customer 1 create
            description "EVPN-VXLAN R-VPLS 1"
            allow-ip-int-bind
            exit
            vxlan instance 1 vni 1 create
            exit
            bgp
            exit
            bgp-evpn
                evi 1
                vxlan bgp 1 vxlan-instance 1
                    no shutdown
                exit
            exit
            sap pxc-1.a:1 create
                no shutdown
            exit
            no shutdown
        exit

Both R-VPLSs are attached to IES 12, which is configured as follows. Interface "evi-1" gets IP address 10.0.0.4/24 and interface "evi-2" gets IP address 20.0.0.4/24; these addresses are used as next-hop in default static routes on CE-1, CE-41, and CE-42.

*A:PE-4#
configure
    service
        ies 12 name "IES-12" customer 1 create
            interface "evi-1" create
                address 10.0.0.4/24
                mac 00:00:00:10:00:04
                vpls "evi-1"
                exit
            exit
            interface "evi-2" create
                address 20.0.0.4/24   
                mac 00:00:00:20:00:04
                vpls "evi-2"
                exit
            exit
            no shutdown
        exit

The BGP configuration on PE-4 includes an EVPN session with PE-2 (neighbor 192.0.2.2), an internal IPv4 session with CE-42 (neighbor 20.0.0. 42), and an external IPv4 session with CE-1 (neighbor 10.0.0.1), as follows:

*A:PE-4#
configure
    router
        bgp
            enable-peer-tracking
            rapid-withdrawal
            split-horizon
            rapid-update evpn
            group "external-ipv4"
                family ipv4
                type external
                local-as 64500
                peer-as 64501
                neighbor 10.0.0.1
                exit
            exit
            group "internal-evpn"
                family evpn
                type internal
                neighbor 192.0.2.2
                exit
            exit
            group "internal-ipv4"
                family ipv4
                type internal
                neighbor 20.0.0.42
                exit
            exit
            no shutdown
        exit

In this example, CE-41 is emulated as VPRN 41 on PE-4. CE-41 is attached via port cross-connect (PXC) to R-VPLS 1. The default static route has next-hop 10.0.0.4 on interface "evi-1" in IES 12. CE-41 has an eBGP-IPv4 session configured with neighbor CE-1 (10.0.0.1); CE-41 exports prefix 172.16.41.0/24 to CE-1. The configuration of VPRN 41 on PE-4 is as follows:

*A:PE-4#
configure
    service
        vprn 41 name "CE-41" customer 1 create
            description "CE-41 attached to R-VPLS-1 on PE-4"
            autonomous-system 64500
            route-distinguisher 64500:41
            interface "int-1_41" create
                address 10.0.0.41/24
                mac 00:00:00:10:00:41
                sap pxc-1.b:1 create
                exit
            exit
            interface "lo1" create
                address 172.16.41.41/24
                mac 00:00:00:04:41:41
                loopback
            exit
            static-route-entry 0.0.0.0/0
                next-hop 10.0.0.4
                    no shutdown
                exit
            exit
            bgp
                router-id 10.0.0.41
                enable-peer-tracking
                rapid-withdrawal
                split-horizon
                group "external"
                    family ipv4
                    type external
                    export "export-bgp-ipv4-41" 
                    local-as 64500
                    peer-as 64501
                    neighbor 10.0.0.1
                    exit
                exit
            exit
            no shutdown
        exit

CE-42 is emulated as VPRN 42 on PE-4. CE-42 is attached via PXC to R-VPLS 2. The default static route has next-hop equal to 20.0.0.4 on interface "evi-2" in IES 12. An iBGP-IPv4 session is configured to this IES interface (neighbor 20.0.0.4). CE-42 exports prefix 172.16.42.0/24 to this IES interface on PE-4. The configuration of VPRN 42 on PE-4 is as follows:

*A:PE-4#
configure
    service
         vprn 42 name "CE-42" customer 1 create
            description "CE-42 attached to R-VPLS-2 on PE-4"
            autonomous-system 64500
            route-distinguisher 64500:42
            interface "int-1_42" create
                address 20.0.0.42/24
                mac 00:00:00:20:00:42
                sap pxc-1.b:2 create
                exit
            exit
            interface "test42" create
                address 172.16.42.42/24
                mac 00:00:00:04:42:42
                sap pxc-1.b:42 create
            exit
            static-route-entry 0.0.0.0/0
                next-hop 20.0.0.4
                    no shutdown
                exit
            exit
            bgp
                router-id 20.0.0.42
                enable-peer-tracking
                rapid-withdrawal
                split-horizon
                group "internal-ipv4"
                    family ipv4
                    type internal
                    export "export-bgp-ipv4-42" 
                    neighbor 20.0.0.4
                    exit
                exit
            exit
            no shutdown
        exit

The export policies are configured as follows:

*A:PE-4#
configure
    router
        policy-options
            begin
            prefix-list "172.16.41.x"
                prefix 172.16.41.0/24 exact
            exit
            prefix-list "172.16.42.x"
                prefix 172.16.42.0/24 exact
            exit
            policy-statement "export-bgp-ipv4-41"
                entry 10
                    from
                        prefix-list "172.16.41.x"
                    exit
                    action accept
                    exit
                exit
            exit
            policy-statement "export-bgp-ipv4-42"
                entry 10
                    from
                        prefix-list "172.16.42.x"
                    exit
                    action accept
                    exit
                exit
            exit
            commit
        exit

Configuration on CE-1

On CE-1, the following static route is configured with next-hop 10.0.0. 4, which is the address on the interface "evi-1" in IES 12 on PE-4:

*A:CE-1#
configure
    router
        static-route-entry 0.0.0.0/0
            next-hop 10.0.0.4
                no shutdown
            exit
        exit

The following loopback address is configured on CE-1 for test purposes:

*A:CE-1#
configure
    router
        interface "lo1"
            address 172.16.1.1/24
            loopback
            no shutdown
        exit

On CE-1, eBGP-IPv4 sessions are configured to the IES interface "evi-1" on PE-4 (neighbor 10.0.0.4) and to CE-41 (neighbor 10.0.0.41) for the IPv4 address family. CE-1 exports prefix 172.16.1.0/24 to its peers. The BGP configuration is as follows:

*A:CE-1#
configure
    router
        policy-options
            begin
            prefix-list "172.16.1.x"
                prefix 172.16.1.0/24 exact
            exit
            policy-statement "export-bgp-ipv4"
                entry 10
                    from
                        prefix-list "172.16.1.x"
                    exit
                    action accept
                    exit
                exit
            exit
            commit
        exit
        bgp
            enable-peer-tracking      
            rapid-withdrawal
            split-horizon
            rapid-update evpn
            group "external"
                family ipv4
                type external
                export "export-bgp-ipv4"
                local-as 64501
                peer-as 64500
                neighbor 10.0.0.4
                exit
                neighbor 10.0.0.41
                exit
            exit
            no shutdown
        exit

Verification

On PE-4, the following shows that five BGP sessions are established:

  • eBGP-IPv4 session with neighbor 10.0.0.1 (CE-1) from the base router

  • iBGP-IPv4 session with neighbor 20.0.0.42 (CE-42) from the base router

  • iBGP-EVPN session with neighbor 192.0.2.2 (PE-2) from the base router

  • eBGP-IPv4 session with neighbor 10.0.0.1 (CE-1) from VPRN 41 (CE-41)

  • iBGP-IPv4 session to IES interface "evi-2" (20.0.0.4) from VPRN 42 (CE-42)

Routes have been exchanged between the peers. The eBGP-IPv4 sessions are established using R-VPLS 1.

*A:PE-4# show router bgp summary all 

===============================================================================
BGP Summary
===============================================================================
Legend : D - Dynamic Neighbor
===============================================================================
Neighbor
Description
ServiceId          AS PktRcvd InQ  Up/Down   State|Rcv/Act/Sent (Addr Family)
                      PktSent OutQ
-------------------------------------------------------------------------------
10.0.0.1
Def. Instance  64501       81    0 00h38m21s 2/1/1 (IPv4)
                           80    0           
20.0.0.42
Def. Instance  64500       81    0 00h38m58s 1/1/1 (IPv4)
                           82    0           
192.0.2.2
Def. Instance  64500       87    0 00h40m06s 2/2/6 (Evpn)
                           91    0           

10.0.0.1
Svc: 41        64501       83    0 00h39m09s 2/1/1 (IPv4)
                           82    0           
20.0.0.4
Svc: 42        64500       81    0 00h38m58s 1/1/1 (IPv4)
                           81    0                    

-------------------------------------------------------------------------------
*A:PE-4#

On PE-4, the following route table includes the prefixes 10.0.0.0/24 of interface "evi-1" and 20.0.0.0/24 of "evi-2" in IES 12. Also, it includes the remote prefixes 172.16.1.0/24 and 172.16.42.0, which are received as BGP IPv4 routes from CE-1 and CE-42 (VPRN 42).

*A:PE-4# show router route-table 

===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags]                            Type    Proto     Age        Pref
      Next Hop[Interface Name]                                    Metric   
-------------------------------------------------------------------------------
10.0.0.0/24                                   Local   Local     01h48m54s  0
       evi-1                                                        0
20.0.0.0/24                                   Local   Local     01h48m54s  0
       evi-2                                                        0
172.16.1.0/24                                 Remote  BGP       00h35m59s  170
       10.0.0.1                                                     0
172.16.42.0/24                                Remote  BGP       00h53m44s  170
       20.0.0.42                                                    0
192.0.2.2/32                                  Remote  ISIS      01h50m40s  18
       192.168.24.1                                                 10
192.0.2.4/32                                  Local   Local     01h50m41s  0
       system                                                       0
192.168.24.0/30                               Local   Local     01h50m41s  0
       int-PE-4-PE-2                                                0
-------------------------------------------------------------------------------
No. of Routes: 7

The following route table for CE-41 includes the remote prefix 172.16.1.0/24 received as BGP IPv4 route with next-hop 10.0.0.1. CE-1 and CE-41 are both in subnet 10.0.0.0/24.

*A:PE-4# show router 41 route-table 

===============================================================================
Route Table (Service: 41)
===============================================================================
Dest Prefix[Flags]                            Type    Proto     Age        Pref
      Next Hop[Interface Name]                                    Metric   
-------------------------------------------------------------------------------
0.0.0.0/0                                     Remote  Static    01h05m00s  5
       10.0.0.4                                                     1
10.0.0.0/24                                   Local   Local     01h05m00s  0
       int-1_41                                                     0
172.16.1.0/24                                 Remote  BGP       00h43m09s  170
       10.0.0.1                                                     0
172.16.41.0/24                                Local   Local     01h05m11s  0
       lo1                                                          0
-------------------------------------------------------------------------------
No. of Routes: 4

Likewise, the following route table for CE-42 includes the remote prefix 172.16.1.0/24 received as BGP IPv4 route, but the next-hop is 20.0.0.4 instead of 10.0.0.1, because CE-42 is in subnet 20.0.0.0/24 whereas CE-1 is in subnet 10.0.0.0/24. Routing between the subnets 20.0.0.0/24 and 10.0.0.0/24 needs to be done in IES 12 on PE-4.

*A:PE-4# show router 42 route-table 

===============================================================================
Route Table (Service: 42)
===============================================================================
Dest Prefix[Flags]                            Type    Proto     Age        Pref
      Next Hop[Interface Name]                                    Metric   
-------------------------------------------------------------------------------
0.0.0.0/0                                     Remote  Static    01h08m20s  5
       20.0.0.4                                                     1
20.0.0.0/24                                   Local   Local     01h08m20s  0
       int-2_42                                                     0
172.16.1.0/24                                 Remote  BGP       00h46m09s  170
       20.0.0.4                                                     0
172.16.42.0/24                                Local   Local     01h08m20s  0
       test42                                                       0
-------------------------------------------------------------------------------
No. of Routes: 4

The following traceroute from CE-41 (172.16.41.41) to CE-1 (172.16.1.1) shows that no intermediate hops are required:

*A:PE-4# traceroute router 41 172.16.1.1 source 172.16.41.41 
traceroute to 172.16.1.1 from 172.16.41.41, 30 hops max, 40 byte packets
  1  172.16.1.1 (172.16.1.1)    3.93 ms  3.87 ms  4.01 ms
*A:PE-4#

The following traceroute from CE-42 (172.16.42.42) to CE-1 (172.16.1.1) shows the IP address 20.0.0.4 on the interface "evi-2" in IES 12 as an intermediate hop:

*A:PE-4# traceroute router 42 172.16.1.1 source 172.16.42.42 
traceroute to 172.16.1.1 from 172.16.42.42, 30 hops max, 40 byte packets
  1  20.0.0.4 (20.0.0.4)    1.91 ms  2.19 ms  2.25 ms
  2  172.16.1.1 (172.16.1.1)    4.15 ms  4.03 ms  4.00 ms
*A:PE-4#

The following ARP table on PE-4 includes entries for IP addresses in subnets 10.0.0.0/24 on interface "evi-1" and 20.0.0.0/24 on interface "evi-2":

*A:PE-4# show router arp 

===============================================================================
ARP Table (Router: Base)
===============================================================================
IP Address      MAC Address       Expiry    Type   Interface
-------------------------------------------------------------------------------
192.0.2.4       04:1b:ff:00:00:00 00h00m00s Oth    system
192.168.24.1    04:14:01:01:00:01 00h57m01s Dyn[I] int-PE-4-PE-2
192.168.24.2    04:1c:01:01:00:02 00h00m00s Oth[I] int-PE-4-PE-2
10.0.0.1        00:00:00:10:00:01 00h59m23s Dyn[I] evi-1
10.0.0.4        00:00:00:10:00:04 00h00m00s Oth[I] evi-1
10.0.0.41       00:00:00:10:00:41 00h00m00s Dyn[I] evi-1
20.0.0.4        00:00:00:20:00:04 00h00m00s Oth[I] evi-2
20.0.0.42       00:00:00:20:00:42 00h59m24s Dyn[I] evi-2
-------------------------------------------------------------------------------
No. of ARP Entries: 7

The forwarding database (FDB) for R-VPLS 1 on PE-4 includes the MAC addresses corresponding to IP addresses 10.0.0.1, 10.0.0.4, and 10.0.0.41:

*A:PE-4# show service id 1 fdb detail 

===============================================================================
Forwarding Database, Service 1
===============================================================================
ServId     MAC               Source-Identifier       Type     Last Change
                                                     Age      
-------------------------------------------------------------------------------
1          00:00:00:10:00:01 vxlan-1:                Evpn     10/23/18 11:07:03
                             192.0.2.2:1
1          00:00:00:10:00:04 cpm                     Intf     10/23/18 11:05:39
1          00:00:00:10:00:41 sap:pxc-1.a:1           L/0      10/23/18 11:06:31
-------------------------------------------------------------------------------
No. of MAC Entries: 3

MAC address 00:00:00:10:00:01, which corresponds to IP address 10.0.0.1 on CE-1, is advertised in an EVPN MAC route by PE-2:

A:PE-4# show router bgp routes evpn mac 
===============================================================================
 BGP Router ID:192.0.2.4        AS:64500       Local AS:64500      
===============================================================================
 Legend -
 Status codes  : u - used, s - suppressed, h - history, d - decayed, * - valid
                 l - leaked, x - stale, > - best, b - backup, p - purge
 Origin codes  : i - IGP, e - EGP, ? - incomplete

===============================================================================
BGP EVPN MAC Routes
===============================================================================
Flag  Route Dist.         MacAddr           ESI
      Tag                 Mac Mobility      Label1
                          Ip Address        
                          NextHop           
-------------------------------------------------------------------------------
u*>i  192.0.2.2:1         00:00:00:10:00:01 ESI-0
      0                   Seq:0             VNI 1
                          n/a
                          192.0.2.2

-------------------------------------------------------------------------------
Routes : 1

EVPN-MPLS R-VPLS attached to IES

Example Topology for EVPN-MPLS R-VPLS attached to IES shows the example topology for EVPN-MPLS R-VPLS attached to IES. All-active multi-homing (AA MH) is configured on PE-2 and PE-3, while single-active (SA) MH is configured on PE-4 and PE-5. R-VPLS 3 is configured on all PEs. IES 30 is configured on PE-2 and PE-3, whereas IES 34 is configured on PE-4 and PE-5. On MTU-6, VPLS 3 and 4 are regular VPLSs, not routed.

Figure 2. Example Topology for EVPN-MPLS R-VPLS attached to IES

The initial configuration on the nodes includes:

  • Cards, MDAs, ports

  • LAG 1 on CE-1, PE-2, PE-3

  • Router interfaces between the PEs and toward MTU-6

  • IS-IS on these interfaces (alternatively, OSPF can be configured)

  • LDP on these interfaces

  • BGP configured for the EVPN address family on the PEs. PE-2 is the RR and has the following BGP configuration:

*A:PE-2#
configure
    router
        bgp
            enable-peer-tracking
            rapid-withdrawal
            split-horizon
            rapid-update evpn
            group "internal-evpn"
                family evpn
                cluster 192.0.2.2
                peer-as 64500
                neighbor 192.0.2.3
                exit
                neighbor 192.0.2.4
                exit
                neighbor 192.0.2.5
                exit
            exit
            no shutdown

Configuration on PE-2 and PE-3

The service configuration on PE-2 and PE-3 is almost identical; only the IP address on the IES interface "evi-3" is different. The AA MH ES "ESI-23_3" is configured as follows, with LAG 1 and dot1q tag 3, so it is only applicable to VPLS 3.

configure
    service
        system
            bgp-evpn
                ethernet-segment "ESI-23_3" virtual create
                    esi 01:00:00:00:00:23:00:03:03:01
                    es-activation-timer 3
                    service-carving
                        mode auto
                    exit
                    multi-homing all-active
                    lag 1
                    dot1q
                        q-tag-range 3
                    exit
                    no shutdown
                exit

R-VPLS 3 has EVPN-MPLS enabled and is configured on PE-2 and PE-3, as follows. SAP lag-1:3 matches the configured LAG and the q-tag range for ESI-23_3.

configure
    service
        vpls 3 name "evi-3" customer 1 create
            allow-ip-int-bind
            exit
            bgp
            exit
            bgp-evpn
                evi 3
                mpls bgp 1
                    ecmp 2
                    auto-bind-tunnel
                        resolution any
                    exit
                    no shutdown
                exit
            exit
            sap lag-1:3 create
                no shutdown
            exit
            no shutdown
        exit

The following is the IES configuration on PE-2. In this example, IES 30 is only configured to demonstrate EVPN all-active multi-homing on R-VPLS with IES. If it were removed, everything still works and the connectivity between the CEs remains.

*A:PE-2#
configure
    service
        ies 30 name "IES-30" customer 1 create
            interface "evi-3" create
                address 30.0.0.2/24
                mac 00:00:00:30:00:02
                vpls "evi-3"
                exit
            exit
            no shutdown
        exit

The IES configuration on PE-3 is similar, only using IP address 30.0.0.3/24.

Configuration on PE-4 and PE-5

On PE-4, SDP 46 is configured toward MTU-6. An SA MH ES "ESI-45" is configured using this SDP, as follows:

*A:PE-4#
configure
    service
        sdp 46 mpls create
            far-end 192.0.2.6
            ldp
            no shutdown
        exit
        system
            bgp-evpn
                ethernet-segment "ESI-45" create
                    esi 01:00:00:00:00:45:00:00:00:01
                    es-activation-timer 3
                    service-carving
                        mode auto
                    exit
                    multi-homing single-active
                    sdp 46
                    no shutdown
                exit

The configuration is similar on PE-5. SDP 56 is configured toward MTU-6 and ES "ESI-45" is configured with SDP 56 instead.

On PE-4, R-VPLSs 3 and 4 are configured with EVPN-MPLS, as follows:

*A:PE-4#
configure
    service
        vpls 3 name "evi-3" customer 1 create
            description "EVPN-MPLS R-VPLS 3"
            allow-ip-int-bind
            exit
            bgp
            exit
            bgp-evpn
                evi 3
                mpls bgp 1
                    ecmp 2
                    auto-bind-tunnel
                        resolution any
                    exit
                    no shutdown
                exit
            exit
            spoke-sdp 46:3 create
                no shutdown
            exit
            no shutdown
        exit
        vpls 4 name "evi-4" customer 1 create
            description "EVPN-MPLS R-VPLS 4"
            allow-ip-int-bind
            exit
            bgp
            exit
            bgp-evpn
                evi 4
                mpls bgp 1
                    ecmp 2
                    auto-bind-tunnel  
                        resolution any
                    exit
                    no shutdown
                exit
            exit
            spoke-sdp 46:4 create
                no shutdown
            exit
            no shutdown
        exit

The configuration is similar on PE-5; only the spoke-SDPs are different (spoke-SDP 56:3 and 56:4).

On PE-4, IES 34 is configured with interfaces "evi-3" and "evi-4", as follows. Passive VRRP is configured on both interfaces. With passive VRRP configured on both PE-4 and PE-5, both PEs behave as master.

*A:PE-4#
configure
    service
        ies 34 name "IES-34" customer 1 create
            interface "evi-3" create
                address 30.0.0.4/24
                mac 00:00:00:30:00:04
                vrrp 1 passive
                    backup 30.0.0.254
                    ping-reply
                    traceroute-reply
                exit
                vpls "evi-3"
                exit
            exit
            interface "evi-4" create
                address 40.0.0.4/24
                mac 00:00:00:40:00:04
                vrrp 1 passive
                    backup 40.0.0.254
                    ping-reply
                    traceroute-reply
                exit
                vpls "evi-4"
                exit
            exit
            no shutdown
        exit

The configuration of IES 34 is similar on PE-5, but the interface IP addresses are different: 30.0.0.5/24 and 40.0.0.5/24. The MAC addresses are also different.

To enable routing between CE-1 and CE-64 in a different subnet, BGP sessions are established with CE-1 (neighbor 30.0.0.1 in AS 64501) and CE-64 (neighbor 40.0.0.64 in AS 64500) for the IPv4 address family. The CEs export prefixes, but no export policy needs to be configured on PE-4 and PE-5. The BGP configuration on PE-4 is as follows:

*A:PE-4#
configure
    router
        bgp
            enable-peer-tracking
            rapid-withdrawal
            split-horizon
            rapid-update evpn
            group "external"
                family ipv4
                type external
                local-as 64500
                peer-as 64501
                neighbor 30.0.0.1
                exit
            exit
            group "internal-evpn"
                family evpn
                type internal
                neighbor 192.0.2.2
                exit
            exit
            group "internal-ipv4"
                family ipv4
                peer-as 64500
                local-address 30.0.0.4
                neighbor 40.0.0.64
                exit
            exit
            no shutdown

The BGP configuration on PE-5 is almost identical; the local address is 30.0.0.5 instead.

Configuration on CE-1

The configuration on CE-1 includes the following:

  • Router interface to VPLS 3 (ESI-23_3) with IP address 30.0.0.1/24 and LAG-1:3 assigned to it

  • Loopback interface with IP address 172.16.1.1/24 for test purposes

  • Static default route with next-hop 30.0.0.254, which is the VRRP backup address for IES interface "evi-3" on PE-4 and PE-5

  • Export policy to export prefix 172.16.1.0/24

  • BGP sessions for the IPv4 address family toward PE-4 (30.0.0.4), PE-5 (30.0.0.5), and CE-63 (30.0.0.63)

The router configuration on CE-1 is as follows:

*A:CE-1>config>router# info 
----------------------------------------------
#--------------------------------------------------
echo "IP Configuration"
#--------------------------------------------------
        interface "int-CE-1-VPLS1_ES-23"
            address 30.0.0.1/24
            port lag-1:3
            no shutdown
        exit
        interface "lo1"
            address 172.16.1.1/24
            loopback
            no shutdown
        exit
        interface "system"
            address 192.0.2.1/32
            no shutdown
        exit
        autonomous-system 64501
#--------------------------------------------------
echo "Static Route Configuration"
#--------------------------------------------------
        static-route-entry 0.0.0.0/0
            next-hop 30.0.0.254
                no shutdown
            exit
        exit
#--------------------------------------------------
echo "Policy Configuration"
#--------------------------------------------------
        policy-options
            begin
            prefix-list "172.16.1.x"
                prefix 172.16.1.0/24 exact
            exit
            policy-statement "export-bgp-ipv4"
                entry 10
                    from
                        prefix-list "172.16.1.x"
                    exit
                    action accept
                    exit
                exit
            exit
            commit
        exit
#--------------------------------------------------
echo "BGP Configuration"
#--------------------------------------------------
        bgp
            router-id 30.0.0.1
            enable-peer-tracking
            rapid-withdrawal
            split-horizon
            rapid-update evpn
            group "external"
                family ipv4
                type external
                export "export-bgp-ipv4"
                local-as 64501
                peer-as 64500
                neighbor 30.0.0.4
                exit
                neighbor 30.0.0.5
                exit
                neighbor 30.0.0.63
                exit
            exit
            no shutdown
        exit
----------------------------------------------

Configuration on MTU-6

The configuration on MTU-6 includes the following:

  • Router interfaces

  • IS-IS

  • LDP

  • One policy to export prefix 172.16.63.0/24 and another policy to export prefix 172.16.64.0/24

  • BGP is not configured in the base router

The following service configuration on MTU-6 includes the SDP configuration and the VPLSs 3 and 4, which are not routed:

*A:MTU-6#
configure
    service
        sdp 64 mpls create
            far-end 192.0.2.4
            ldp
            no shutdown
        exit
        sdp 65 mpls create
            far-end 192.0.2.5
            ldp
            no shutdown
        exit
        vpls 3 name "VPLS3" customer 1 create
            endpoint "CORE" create
            exit
            sap pxc-1.a:3 create
            exit
            spoke-sdp 64:3 endpoint "CORE" create
            exit
            spoke-sdp 65:3 endpoint "CORE" create
            exit
           no shutdown
        exit
        vpls 4 name "VPLS4" customer 1 create
            endpoint "CORE" create
            exit
            sap pxc-1.a:4 create
            exit
            sap pxc-1.a:64 create
            exit
            spoke-sdp 64:4 endpoint "CORE" create
            exit
            spoke-sdp 65:4 endpoint "CORE" create
            exit
           no shutdown
        exit

In this example, CE-63 and CE-64 are simulated by VPRN 63 and VPRN 64. The default static route has next-hop 30.0.0.254, which is the VRRP backup address on interface "evi-3" in IES 34 on both PE-4 and PE-5. BGP is configured within VPRN 63 and 64. The prefix 172.16.63.0/24 is exported by BGP in VPRN 63 (CE-63) and prefix 172.16.64.0/24 is exported by BGP in VPRN 64 (CE-64). The configuration of VPRN 63 and VPRN 64 is as follows:

*A:MTU-6#
configure
    service
        vprn 63 name "CE-63" customer 1 create
            autonomous-system 64500
            route-distinguisher 65400:63
            interface "int-1_63" create
                address 30.0.0.63/24
                mac 00:00:00:30:00:63
                sap pxc-1.b:3 create
                exit
            exit
            interface "lo1" create
                address 172.16.63.63/24
                loopback
                no shutdown
            exit
            static-route-entry 0.0.0.0/0
                next-hop 30.0.0.254
                    no shutdown
                exit
            exit
            bgp
                router-id 30.0.0.63
                enable-peer-tracking
                rapid-withdrawal
                split-horizon
                group "external"
                    family ipv4
                    type external
                    export "export-bgp-ipv4-63"
                    local-as 64500
                    peer-as 64501
                    neighbor 30.0.0.1
                    exit
                exit
                no shutdown
            exit
            no shutdown
        exit
        vprn 64 name "CE-64" customer 1 create
            autonomous-system 64500
            route-distinguisher 65400:64
            interface "int-2_64" create
                address 40.0.0.64/24
                mac 00:00:00:40:00:64
                sap pxc-1.b:4 create
                exit
            exit
            interface "test" create
                address 172.16.64.64/24
                mac 00:00:00:06:64:64
                sap pxc-1.b:64 create
                exit
            exit
            static-route-entry 0.0.0.0/0
                next-hop 40.0.0.254
                    no shutdown
                exit
            exit
            bgp
                router-id 40.0.0.64
                enable-peer-tracking
                rapid-withdrawal
                split-horizon
                group "internal-ipv4"
                    family ipv4
                    type internal
                    export "export-bgp-ipv4-64"
                    neighbor 30.0.0.4
                    exit
                    neighbor 30.0.0.5
                    exit
                exit
                no shutdown
            exit
            no shutdown
        exit

Verification

In the AA MH ES "ESI-23_3", PE-3 is the designated forwarder (DF) for R-VPLS 3 and PE-2 is NDF, as follows:

*A:PE-2# show service id 3 ethernet-segment 

===============================================================================
SAP Ethernet-Segment Information
===============================================================================
SAP                   Eth-Seg                          Status
-------------------------------------------------------------------------------
lag-1:3               ESI-23_3                         NDF
===============================================================================
No sdp entries
No vxlan instance entries
*A:PE-2#

*A:PE-3# show service id 3 ethernet-segment 

===============================================================================
SAP Ethernet-Segment Information
===============================================================================
SAP                   Eth-Seg                          Status
-------------------------------------------------------------------------------
lag-1:3               ESI-23_3                         DF
===============================================================================
No sdp entries
No vxlan instance entries
*A:PE-3#

In the SA MH ES "ESI-45", PE-4 is NDF for R-VPLS 3 and DF for R-VPLS4, as follows:

*A:PE-4# show service id 3 ethernet-segment 
No sap entries

===============================================================================
SDP Ethernet-Segment Information
===============================================================================
SDP                   Eth-Seg                          Status
-------------------------------------------------------------------------------
46:3                  ESI-45                           NDF
===============================================================================
No vxlan instance entries
*A:PE-4# show service id 4 ethernet-segment 
No sap entries

===============================================================================
SDP Ethernet-Segment Information
===============================================================================
SDP                   Eth-Seg                          Status
-------------------------------------------------------------------------------
46:4                  ESI-45                           DF
===============================================================================
No vxlan instance entries
*A:PE-4#

The reverse is true for PE-5, which is DF for R-VPLS 3 and NDF for R-VPLS 4, as follows:

*A:PE-5# show service id 3 ethernet-segment 
No sap entries

===============================================================================
SDP Ethernet-Segment Information
===============================================================================
SDP                   Eth-Seg                          Status
-------------------------------------------------------------------------------
56:3                  ESI-45                           DF
===============================================================================
No vxlan instance entries
*A:PE-5# show service id 4 ethernet-segment 
No sap entries

===============================================================================
SDP Ethernet-Segment Information
===============================================================================
SDP                   Eth-Seg                          Status
-------------------------------------------------------------------------------
56:4                  ESI-45                           NDF
===============================================================================
No vxlan instance entries
*A:PE-5#

CE-63 (VPRN 63 on MTU-6) has an external BGP IPv4 session with CE-1, whereas CE-64 (VPRN 64 on MTU-6) has internal BGP IPv4 sessions with IES interface "evi-3" on PE-4 and PE-5, as follows:

*A:MTU-6# show router 64 bgp summary all 

===============================================================================
BGP Summary
===============================================================================
Legend : D - Dynamic Neighbor
===============================================================================
Neighbor
Description
ServiceId          AS PktRcvd InQ  Up/Down   State|Rcv/Act/Sent (Addr Family)
                      PktSent OutQ
-------------------------------------------------------------------------------
30.0.0.1
Svc: 63        64501       24    0 00h02m43s 2/1/1 (IPv4)
                           12    0           
30.0.0.4
Svc: 64        64500       21    0 00h07m34s 1/1/1 (IPv4)
                           19    0           
30.0.0.5
Svc: 64        64500       23    0 00h08m50s 1/0/1 (IPv4)
                           21    0           
-------------------------------------------------------------------------------
*A:MTU-6#

The difference is that CE-63 (with IP address 30.0.0.63) is in the same subnet as CE-1 (30.0.0.1), whereas CE-64 is not (40.0.0.64). Routing between these subnets can be done in IES 34 on PE-4 and PE-5. CE-63 exports prefix 172.16.63.0/24 directly to CE-1, whereas CE-64 exports prefix 172.16.64.0/24 to PE-4 and PE-5 instead, which will advertise prefix 172.16.64.0/24 to their BGP peer CE-1. The following route table on CE-1 shows BGP route 172.16.63.0/63 with next-hop 30.0.0.63 (CE-63) and BGP route 172.16.64.0/64 with next-hop 30.0.0.4 (interface "evi-3" on PE-4):

*A:CE-1# show router route-table 

===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags]                            Type    Proto     Age        Pref
      Next Hop[Interface Name]                                    Metric   
-------------------------------------------------------------------------------
0.0.0.0/0                                     Remote  Static    00h12m37s  5
       30.0.0.254                                                   1
30.0.0.0/24                                   Local   Local     00h12m37s  0
       int-CE-1-VPLS3_ES-23                                         0
172.16.1.0/24                                 Local   Local     00h18m37s  0
       lo1                                                          0
172.16.63.0/24                                Remote  BGP       00h11m49s  170
       30.0.0.63                                                    0
172.16.64.0/24                                Remote  BGP       00h12m10s  170
       30.0.0.4                                                     0
192.0.2.1/32                                  Local   Local     00h18m37s  0
       system                                                       0
-------------------------------------------------------------------------------
No. of Routes: 6

In IES 34 on PE-4 (and PE-5), routing can be done between subnet 30.0.0.0/24 and 40.0.0.0/24. The following route table on PE-4 shows BGP route 172.16.1.0/24 with next-hop CE-1 (30.0.0.1) and BGP route 172.16.64.0/24 with next-hop CE-64 (40.0.0.64). The same entries occur in the route table on PE-5.

*A:PE-4# show router route-table

===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags]                            Type    Proto     Age        Pref
      Next Hop[Interface Name]                                    Metric   
-------------------------------------------------------------------------------
30.0.0.0/24                                   Local   Local     00h32m24s  0
       evi-3                                                        0
40.0.0.0/24                                   Local   Local     00h32m24s  0
       evi-4                                                        0
172.16.1.0/24                                 Remote  BGP       00h25m47s  170
       30.0.0.1                                                     0
172.16.64.0/24                                Remote  BGP       00h30m23s  170
       40.0.0.64                                                    0
---snip---

The route table of CE-63 (VPRN 63 on MTU-6) shows a BGP route for prefix 172.16.1.0/24 with next-hop 30.0.0.1 (CE-1), as follows:

*A:MTU-6# show router 63 route-table protocol bgp 

===============================================================================
Route Table (Service: 63)
===============================================================================
Dest Prefix[Flags]                            Type    Proto     Age        Pref
      Next Hop[Interface Name]                                    Metric   
-------------------------------------------------------------------------------
172.16.1.0/24                                 Remote  BGP       00h33m39s  170
       30.0.0.1                                                     0
-------------------------------------------------------------------------------
No. of Routes: 1

The route table of CE-64 (VPRN 64 on MTU-6) shows a BGP route for prefix 172.16.1.0/24 with next-hop 40.0.0.254 (VRRP backup address for IES interface "evi-4" on PE-4 and PE-5), as follows:

*A:MTU-6# show router 64 route-table 

===============================================================================
Route Table (Service: 64)
===============================================================================
Dest Prefix[Flags]                            Type    Proto     Age        Pref
      Next Hop[Interface Name]                                    Metric   
-------------------------------------------------------------------------------
0.0.0.0/0                                     Remote  Static    00h40m35s  5
       40.0.0.254                                                   1
40.0.0.0/24                                   Local   Local     00h40m35s  0
       int-2_64                                                     0
172.16.1.0/24                                 Remote  BGP       00h33m32s  170
       40.0.0.254                                                   0
172.16.64.0/24                                Local   Local     00h40m35s  0
       test                                                         0
-------------------------------------------------------------------------------
No. of Routes: 4

The connectivity between CE-1 and CE-63 is verified as follows:

*A:CE-1# ping 172.16.63.63 source 172.16.1.1 
PING 172.16.63.63 56 data bytes
64 bytes from 172.16.63.63: icmp_seq=1 ttl=64 time=4.67ms.
64 bytes from 172.16.63.63: icmp_seq=2 ttl=64 time=4.97ms.
---snip---

The following traceroute command verifies the connectivity between CE-1 and CE-64. The intermediate hop is 30.0.0.4, the IP address of the IES interface "evi-3" on PE-4:

*A:CE-1# traceroute 172.16.64.64 source 172.16.1.1 
traceroute to 172.16.64.64 from 172.16.1.1, 30 hops max, 40 byte packets
  1  30.0.0.4 (30.0.0.4)    3.81 ms  3.79 ms  3.13 ms
  2  172.16.64.64 (172.16.64.64)    4.55 ms  5.29 ms  5.08 ms

When the traceroute is launched from CE-64, the intermediate hop is 40.0.0.4, the IP address of the IES interface "evi-4" on PE-4:

*A:MTU-6# traceroute router 64 172.16.1.1
traceroute to 172.16.1.1, 30 hops max, 40 byte packets
  1  40.0.0.4 (40.0.0.4)    3.08 ms  2.97 ms  2.93 ms
  2  172.16.1.1 (172.16.1.1)    4.61 ms  4.37 ms  4.91 ms

The following ARP table on CE-1 contains entries for different nodes in the 30.0.0.0/24 subnet:

*A:CE-1# show router arp 

===============================================================================
ARP Table (Router: Base)
===============================================================================
IP Address      MAC Address       Expiry    Type   Interface
-------------------------------------------------------------------------------
192.0.2.1       04:0f:ff:00:00:00 00h00m00s Oth    system
30.0.0.1        04:0f:ff:00:01:41 00h00m00s Oth[I] int-CE-1-VPLS3_ES-23
30.0.0.4        00:00:00:30:00:04 03h12m38s Dyn[I] int-CE-1-VPLS3_ES-23
30.0.0.5        00:00:00:30:00:05 03h12m41s Dyn[I] int-CE-1-VPLS3_ES-23
30.0.0.63       00:00:00:30:00:63 03h59m53s Dyn[I] int-CE-1-VPLS3_ES-23
30.0.0.254      00:00:5e:00:01:01 03h54m13s Dyn[I] int-CE-1-VPLS3_ES-23
172.16.1.1      04:0f:ff:00:00:00 00h00m00s Oth    lo1
-------------------------------------------------------------------------------
No. of ARP Entries: 7

The ARP table on PE-4 contains entries for different nodes in subnets 30.0.0.0/24 and 40.0.0.0/24:

*A:PE-4# show router arp 

===============================================================================
ARP Table (Router: Base)
===============================================================================
IP Address      MAC Address       Expiry    Type   Interface
-------------------------------------------------------------------------------
---snip---
30.0.0.1        04:0f:ff:00:01:41 00h51m51s Dyn[I] evi-3
30.0.0.2        00:00:00:30:00:02 00h00m00s Evp[I] evi-3
30.0.0.3        00:00:00:30:00:03 00h00m00s Evp[I] evi-3
30.0.0.4        00:00:00:30:00:04 00h00m00s Oth[I] evi-3
30.0.0.5        00:00:00:30:00:05 00h00m00s Evp[I] evi-3
30.0.0.63       00:00:00:30:00:63 00h57m32s Dyn[I] evi-3
30.0.0.254      00:00:5e:00:01:01 00h00m00s Oth[I] evi-3
40.0.0.4        00:00:00:40:00:04 00h00m00s Oth[I] evi-4
40.0.0.5        00:00:00:40:00:05 00h00m00s Evp[I] evi-4
40.0.0.64       00:00:00:40:00:64 00h57m33s Dyn[I] evi-4
40.0.0.254      00:00:5e:00:01:01 00h00m00s Oth[I] evi-4
---snip---
-------------------------------------------------------------------------------

The FDB on PE-4 shows that MAC address 00:00:00:40:00:64-corresponding to 40.0.0.64 on CE-64-is learned on SDP 46:6, as follows.

*A:PE-4# show service id 4 fdb detail 

===============================================================================
Forwarding Database, Service 4
===============================================================================
ServId     MAC               Source-Identifier       Type     Last Change
                                                     Age      
-------------------------------------------------------------------------------
4          00:00:00:40:00:04 cpm                     Intf     10/25/18 07:09:11
4          00:00:00:40:00:05 eMpls:                  EvpnS    10/25/18 07:10:09
                                                     P        
                             192.0.2.5:524280
4          00:00:00:40:00:64 sdp:46:4                L/0      10/25/18 07:15:26
4          00:00:5e:00:01:01 cpm                     Intf     10/25/18 07:09:11
-------------------------------------------------------------------------------
No. of MAC Entries: 4
-------------------------------------------------------------------------------
Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static Lf=Leaf
===============================================================================
*A:PE-4#

The FDB on PE-5 shows that MAC address 00:00:00:40:00:64 -corresponding to 40.0.0.64 on CE-64-is advertised as an EVPN MAC route with ESI "ESI-45", as follows:

*A:PE-5# show service id 4 fdb detail 

===============================================================================
Forwarding Database, Service 4
===============================================================================
ServId     MAC               Source-Identifier       Type     Last Change
                                                     Age      
-------------------------------------------------------------------------------
4          00:00:00:40:00:04 eMpls:                  EvpnS    10/25/18 07:10:09
                                                     P        
                             192.0.2.4:524284
4          00:00:00:40:00:05 cpm                     Intf     10/25/18 06:50:29
4          00:00:00:40:00:64 eES:                    Evpn     10/25/18 07:15:26
                             01:00:00:00:00:45:00:00:00:01
4          00:00:5e:00:01:01 cpm                     Intf     10/25/18 06:50:29
-------------------------------------------------------------------------------
No. of MAC Entries: 4

Conclusion

With EVPN R-VPLS attached to IES services, EVPN services are connected to the base router, so the traffic can be routed in the global routing table (GRT).