GRE Tunnel Origination and Termination Using Non-system IP Addresses

This chapter provides information about GRE tunnel origination and termination using non-system IP addresses.

Topics in this chapter include:

Applicability

This chapter was initially written based on SR OS Release 16.0.R5, but the CLI in the current edition corresponds to SR OS Release 23.3.R2. GRE SDPs and auto-bind GRE tunnels can originate and terminate on a non-system IP address in SR OS Release 16.0.R4 or later.

Overview

For scaling purposes, service providers typically deploy seamless MPLS or inter-AS scenarios. In many cases, the system IP address cannot be leaked between domains and a separate loopback address is used to terminate tunnels. GRE termination on a non-system IP address is supported in the following services:

VPLS with manually configured GRE spoke-SDPs
VPLS with BGP-AD using provisioned GRE SDPs (use-provisioned-sdp or prefer-provisioned-sdp CLI commands)
BGP-VPLS using provisioned GRE SDPs
Epipe with manually configured GRE spoke-SDPs
Epipe with BGP-VPWS using provisioned GRE SDPs
VPRN with manually configured GRE spoke-SDPs
VPRN with auto-bind GRE tunnel
IES with manually configured GRE spoke-SDPs

This chapter focuses on MPLS-over-GRE termination, but IP-over-GRE termination is also supported.

MPLS-over-GRE termination

GRE termination applies to GRE SDPs and auto-bind GRE tunnels concurrently on a system interface and on non-system interfaces with a subnet that is up to and including /16. In the following example, the non-system loopback address 10.0.1.1 with a subnet of /24 is configured as GRE termination on PE-1:

# on PE-1: 
configure
    router Base
        interface "lo1"
            address 10.0.1.1/24
            loopback
            gre-termination
            no shutdown
        exit

Only one interface can be configured as GRE termination. The following error is raised when attempting to configure a second loopback interface "lo2" as GRE termination on PE-1:

*A:PE-1>config>router>if$ gre-termination
MINOR: CLI Could not set gre-termination for interface "lo2".
MINOR: PIP #2078 Cannot config GRE termination - already set on interface "lo1"

Although the preceding examples are for loopback interfaces, GRE termination can also be configured on other router interfaces, but only one per node. The following shows an attempt to configure interface "int-PE-1-PE-2" on PE-1 as GRE termination. The same error message is raised. However, if it were the first interface on the node to be configured as GRE termination, the configuration would be accepted.

*A:PE-1>config>router>if# gre-termination
MINOR: CLI Could not set gre-termination for interface "int-PE-1-PE-2".
MINOR: PIP #2078 Cannot config GRE termination - already set on interface "lo1"

The maximum size of the GRE termination subnet is /16.

GRE termination cannot be applied on the following interface types:

Unnumbered network IP interfaces
IES interfaces
VPRN interfaces
CSC VPRN interfaces

MPLS-over-GRE origination

GRE SDPs and auto-bind GRE tunnels can originate and terminate on a non-system IP address. Manually configured SDPs can be configured with a non-system IP address as the far-end address. Optionally, a non-system local-end address can be configured for generating GRE from an interface other than the system interface. In the following example on PE-1, GRE SDP 120 uses loopback address 10.0.1.1 as the local-end address and 10.0.2.1 on PE-2 as the far-end address.

# on PE-1: 
configure 
    service
        sdp 120 create
            far-end 10.0.2.1
            local-end 10.0.1.1
            no shutdown
        exit

The local-end IP address can only be configured for GRE SDPs; the following error message is raised when attempting to configure an MPLS SDP with a local-end address:

*A:PE-1>config>service# sdp 122 mpls create
*A:PE-1>config>service>sdp$ local-end 10.0.1.1
MINOR: SVCMGR #7825 Invalid local-end address - local-end not supported for this sdp type

The local-end parameter value complies with the following rules:

A maximum of 15 distinct address values can be configured for all GRE SDPs in the configure service sdp local-end context, and all L2oGRE SDPs under the configure service system gre-eth-bridged tunnel-termination context.
The same source address cannot be used in both contexts because an address configured for an L2oGRE SDP matches an internally created interface that is not available to other applications.
The local-end address of a GRE SDP, when different from the system address, need not match the primary address of an interface that has the MPLS-over-GRE termination subnet configured, unless a GRE SDP or tunnel from the far-end router terminates on this address.

The primary IPv4 address of any local network IP interface, loopback or not, may be used. The following shows that IP address 192.168.12.1, as the IP address of the previously mentioned interface "int-PE-1-PE-2" toward PE-2, can be used as the local-end address:

# on PE-1: 
configure 
    service
        sdp 123 create
            far-end 10.0.2.1
            local-end 192.168.12.1
            no shutdown
        exit

The following shows that an error message is raised when attempting to configure an invalid local-end IP address, that is, an IP address that is not primary on a local router interface. In this case, local-end IP address 10.99.1.1 does not exist on PE-1.

*A:PE-1>config>service# sdp 120 create
*A:PE-1>config>service>sdp$ local-end 10.99.1.1
MINOR: SVCMGR #7827 Cannot configure local-end IP address - Local router interface with 
address does not exist, or address is not primary

For services that support auto-binding to a GRE tunnel, the following command configures a single alternate source address (in this case, 10.0.1.1) per system:

# on PE-1: 
configure 
    service
        system
            vpn-gre-source-ip 10.0.1.1
        exit

The default value of the single source address is the primary IPv4 address of the system interface. The value of the vpn-gre-source-ip parameter can be changed at any time. After a new value is configured, the system address will not be used in services that bind to the GRE tunnel.

The vpn-gre-source-ip parameter value complies with the following rules:

This single source address counts toward the maximum of 15 distinct address values per system used by all GRE SDPs under the configure service sdp local-end context and all L2oGRE SDPs under the configure service system gre-eth-bridged tunnel-termination context.
The same source address can be used in both vpn-gre-source-ip and configure service sdp local-end contexts.
The same source address cannot be used in both vpn-gre-source-ip and configure service system gre-eth-bridged tunnel-termination contexts because an address configured for an L2oGRE SDP matches an internally created interface that is not available to other applications.
The vpn-gre-source-ip address, when different from the system IP address, need not match the primary address of an interface that has the MPLS-over-GRE termination subnet configured, unless a GRE SDP or tunnel from the far-end router terminates on this address.

Configuration

Example topology shows the example topology with three SR OS nodes in AS 64500. Services will be configured on PE-1 and PE-2, while RR-3 is a route reflector (RR).

The initial configuration on the three PEs includes:

cards, MDAs, ports
router interfaces. The IP addresses shown on the figure are the system IP addresses 192.0.2.x/32.
IS-IS as IGP (alternatively, OSPF can be used)

GRE SDP termination on non-system IP addresses will be configured in the following use cases:

VPLS with manually configured T-LDP signaled SDP
Epipe with manually configured T-LDP signaled SDP
BGP-VPLS using a provisioned BGP-signaled SDP
BGP-AD in VPLS using a provisioned T-LDP signaled SDP
BGP-VPWS using a provisioned BGP-signaled SDP
VPRN with manually configured T-LDP signaled SDP
VPRN with auto-bind to GRE tunnel
IES with manually configured T-LDP signaled SDP

MPLS-over-GRE termination

On PE-1, PE-2, and RR-3, loopback interface "lo1" is configured as GRE termination with IPv4 address 10.0.x.1/24 for PE-x. The configuration on PE-1 is as follows:

# on PE-1: 
configure
    router Base
        interface "lo1"
            address 10.0.1.1/24
            loopback
            gre-termination
            no shutdown
        exit

This loopback interface will be used in the SDP configuration. With a /24 subnet, the SDP origination can be any address in the subnet. This is useful for providing entropy in the outer IPv4 header for load-balancing over the IP network.

MPLS-over-GRE origination: SDP local end

The local-end address must be reachable from the far-end router that terminates the GRE SDP. Therefore, the interface for this address can be added to IGP or BGP. Alternatively, a static route can be configured on the far-end router. In this example, IS-IS is enabled on the loopback interface with GRE termination, as follows:

# on PE-1, PE-2, RR-3:
configure
    router Base
        isis 0
            interface "lo1"
            exit

On PE-1, the following SDPs are configured with far-end 10.0.2.1 on PE-2 and local-end 10.0.1.1: SDP 120 with T-LDP signaling (default) and SDP 121 with BGP signaling.

# on PE-1: 
configure 
    service
        sdp 120 create
            signaling tldp       # default
            far-end 10.0.2.1
            local-end 10.0.1.1
            no shutdown
        exit
        sdp 121 create
            signaling bgp
            far-end 10.0.2.1
            local-end 10.0.1.1
            no shutdown
        exit

T-LDP signaled GRE SDPs

When T-LDP signaled SDPs, such as SDP 120 in the preceding example, are configured, T-LDP sessions are auto-created toward the far end of the SDPs. By default, LDP uses the system IP address as source address. However, if the source address for the T-LDP session does not match the destination transport address set by the remote PE, the T-LDP session will not come up and the GRE SDP will remain down.Mismatched T-LDP transport addresses shows an example where SDP auto-created T-LDP sessions use the local system addresses 192.0.2.x and far-end addresses 10.0.0.x, so the GRE SDPs will not come up.

Figure 2. Mismatched T-LDP transport addresses

Therefore, the local transport address of the T-LDP session must match the local-end address of the GRE SDP in the PE. These T-LDP sessions can be manually provisioned or auto-created via peer templates. The following configures T-LDP sessions between the non-system IP addresses on PE-1 and PE-2.

# on PE-1: 
configure
    router Base
        ldp
            targeted-session
                peer 10.0.2.1
                    local-lsr-id "lo1"
                exit

# on PE-2: 
configure
    router Base
        ldp
            targeted-session
                peer 10.0.1.1
                    local-lsr-id "lo1"
                exit

Matching T-LDP transport addresses shows the GRE T-LDP signaled SDPs with matching addresses for the T-LDP sessions.

Figure 3. Matching T-LDP transport addresses

BGP configuration

In this example, the L2 and L3 services are configured on PE-1 and PE-2, while RR-3 acts as the RR. On PE-1, BGP is configured with neighbor 10.0.3.1 and local address 10.0.1.1, as follows. Address family L2-VPN is required for L2 services using BGP-VPLS, BGP-AD, and BGP-VPWS; address family VPN-IPv4 is used for VPRN services.

# on PE-1:
configure
    router Base
        bgp
            rapid-withdrawal
            split-horizon
            group "internal"
                family vpn-ipv4 l2-vpn
                type internal
                local-address 10.0.1.1
                neighbor 10.0.3.1
                exit
            exit
            no shutdown

On RR-3, the BGP configuration is as follows.

# on RR-3: 
configure
    router Base
        bgp
            rapid-withdrawal
            split-horizon
            group "internal"
                family vpn-ipv4 l2-vpn
                type internal
                cluster 10.0.3.1
                local-address 10.0.3.1
                neighbor 10.0.1.1
                exit
                neighbor 10.0.2.1
                exit
            exit
            no shutdown
        exit

The loopback addresses 10.0.x.1 are configured for the local and neighbor addresses.

Note:

When the local address 10.0.x.1 is not configured, the system address 192.0.2.x will be used instead. However, in that case, no BGP sessions will be established and, therefore, no BGP routes will be exchanged between 192.0.2.x and 10.0.y.1, and no spoke-SDPs will be auto-created in L2 services using BGP-VPLS, BGP-AD, or BGP-VWPS. Likewise, no BGP-VPN routes will be exchanged between VPRNs on PE-1 and PE-2.

L2 services

L2 services on PE-1 and PE-2 shows the example topology with the following L2 services configured on PE-1 and PE-2:

VPLS 1 with manually configured spoke-SDP 120:1
Epipe 2 with manually configured spoke-SDP 120:2
BGP-VPLS 3 using PW template 1 (BGP-signaled SDP 121 is used)
LDP VPLS 4 with BGP-AD using PW template 1 (T-LDP signaled SDP 120 is used)
BGP-VPWS Epipe 5 using PW template 1 (BGP-signaled SDP 121 is used)

The CEs are VPRNs configured on the PEs and connected to the VPLSs via port cross-connect (PXC).

For a description of the BGP-VPLS parameters, see the "BGP VPLS" chapter in 7450 ESS, 7750 SR, and 7950 XRS Layer 2 Services and EVPN Advanced Configuration Guide for Classic CLI; for BGP-AD, see the "LDP VPLS Using BGP Auto-Discovery" chapter in 7450 ESS, 7750 SR, and 7950 XRS Layer 2 Services and EVPN Advanced Configuration Guide for Classic CLI; for BGP-VPWS, see the "BGP Virtual Private Wire Services" chapter in 7450 ESS, 7750 SR, and 7950 XRS Layer 2 Services and EVPN Advanced Configuration Guide for Classic CLI. For BGP-VPLS, BGP-AD, and BGP-VPWS, PW template 1 is configured with the use-provisioned-sdp command. The service configuration on PE-1 is as follows; the service configuration on PE-2 is similar.

# on PE-1: 
configure 
    service
        sdp 120 create
            far-end 10.0.2.1
            local-end 10.0.1.1
            keep-alive
                shutdown
            exit
            no shutdown
        exit
        sdp 121 create
            signaling bgp
            far-end 10.0.2.1
            local-end 10.0.1.1
            keep-alive
                shutdown
            exit
            no shutdown
        exit
        pw-template 1 name "PW1-use-prov-SDP" use-provisioned-sdp create
        exit
        vpls 1 name "VPLS-1" customer 1 create
            description "VPLS 1 with manually configured spoke-SDP"
            stp
                shutdown
            exit
            sap pxc-10.a:1 create
                no shutdown
            exit
            spoke-sdp 120:1 create
                no shutdown
            exit
            no shutdown
        exit
        epipe 2 name "Epipe-2" customer 1 create
            description "Epipe 2 with manually configured spoke-SDP"
            sap pxc-10.a:2 create
                no shutdown
            exit
            spoke-sdp 120:2 create
                no shutdown
            exit
            no shutdown
        exit
        vpls 3 name "BGP-VPLS-3" customer 1 create
            description "BGP-VPLS with use provisioned SDP"
            bgp
                route-distinguisher 64500:3
                route-target export target:64500:3 import target:64500:3
                pw-template-binding 1
                exit
            exit
            bgp-vpls
                max-ve-id 100
                ve-name "PE-1"
                    ve-id 1
                exit
                no shutdown
            exit
            stp
                shutdown
            exit
            sap pxc-10.a:3 create
                no shutdown
            exit
            no shutdown
        exit
        vpls 4 name "BGP-AD VPLS-4" customer 1 create
            description "BGP-AD for LDP VPLS with use provisioned SDP"
            bgp
                route-distinguisher 64500:4
                route-target export target:64500:4 import target:64500:4
                pw-template-binding 1
                exit
            exit
            bgp-ad
                vpls-id 64500:4
                no shutdown
            exit
            stp
                shutdown
            exit
            sap pxc-10.a:4 create
                no shutdown
            exit
            no shutdown
        exit
        epipe 5 name "BGP-VPWS-5" customer 1 create
            description "BGP-VPWS with use provisioned SDP"
            bgp
                route-distinguisher 64500:5
                route-target export target:64500:5 import target:64500:5
                pw-template-binding 1
                exit
            exit
            bgp-vpws
                ve-name "PE-1"
                    ve-id 1
                exit
                remote-ve-name "PE-2"
                    ve-id 2
                exit
                no shutdown
            exit
            sap pxc-10.a:5 create
                no shutdown
            exit
            no shutdown
        exit

The following BGP sessions are established between PE-1 and RR-3 for the VPN-IPv4 and L2VPN address families:

*A:PE-1# show router bgp summary all

===============================================================================
BGP Summary
===============================================================================
Legend : D - Dynamic Neighbor
===============================================================================
Neighbor
Description
ServiceId          AS PktRcvd InQ  Up/Down   State|Rcv/Act/Sent (Addr Family)
                      PktSent OutQ
-------------------------------------------------------------------------------
10.0.3.1
Def. Inst       64500      13    0 00h02m48s 0/0/0 (VpnIPv4)
                           15    0           3/3/3 (L2VPN)

-------------------------------------------------------------------------------

On PE-1, the following T-LDP session is established to 10.0.2.1 on PE-2:

*A:PE-1# show router ldp session ipv4

==============================================================================
LDP IPv4 Sessions
==============================================================================
Peer LDP Id         Adj Type  State         Msg Sent  Msg Recv  Up Time
------------------------------------------------------------------------------
10.0.2.1:0          Targeted  Established   52        53        0d 00:03:39
------------------------------------------------------------------------------
No. of IPv4 Sessions: 1
==============================================================================

On PE-1, the following SDPs are created with far end 10.0.2.1 and GRE delivery. For SDP 120, T-LDP signaling is used; BGP signaling is used for SDP 121.

*A:PE-1# show service sdp

============================================================================
Services: Service Destination Points
============================================================================
SdpId  AdmMTU  OprMTU  Far End          Adm  Opr         Del     LSP   Sig
----------------------------------------------------------------------------
120    0       8954    10.0.2.1         Up   Up          GRE     n/a   TLDP
121    0       8954    10.0.2.1         Up   Up          GRE     n/a   BGP
----------------------------------------------------------------------------
Number of SDPs : 2
----------------------------------------------------------------------------
Legend: R = RSVP, L = LDP, B = BGP, M = MPLS-TP, n/a = Not Applicable
        I = SR-ISIS, O = SR-OSPF, T = SR-TE, F = FPE
============================================================================

On PE-1, the following SDP-bindings are used:

*A:PE-1# show service sdp-using

===============================================================================
SDP Using
===============================================================================
SvcId      SdpId              Type   Far End              Opr   I.Label E.Label
                                                          State
-------------------------------------------------------------------------------
1          120:1              Spok   10.0.2.1             Up    524286  524286
2          120:2              Spok   10.0.2.1             Up    524285  524285
3          121:4294967295     BgpVp* 10.0.2.1             Up    524278  524277
4          120:4294967294     BgpAd  10.0.2.1             Up    524275  524275
5          121:4294967293     BgpVp* 10.0.2.1             Up    524276  524276
-------------------------------------------------------------------------------
Number of SDPs : 5
-------------------------------------------------------------------------------
===============================================================================
* indicates that the corresponding row element may have been truncated.

When the loopback interface "lo1" is configured as GRE termination on PE-1 and PE-2, the CEs can send traffic to each other. The following ping messages verify the connectivity between CE-11 and CE-21, CE-12 and CE-22, and so on:

*A:PE-1# ping router 11 10.0.11.21 rapid
PING 10.0.11.21 56 data bytes
!!!!!
---- 10.0.11.21 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 3.58ms, avg = 5.11ms, max = 10.3ms, stddev = 2.59ms
*A:PE-1# ping router 12 10.0.12.22 rapid
PING 10.0.12.22 56 data bytes
!!!!!
---- 10.0.12.22 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 3.37ms, avg = 4.54ms, max = 8.83ms, stddev = 2.15ms
*A:PE-1# ping router 13 10.0.13.23 rapid
PING 10.0.13.23 56 data bytes
!!!!!
---- 10.0.13.23 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 3.24ms, avg = 4.32ms, max = 8.02ms, stddev = 1.85ms
*A:PE-1# ping router 14 10.0.14.24 rapid
PING 10.0.14.24 56 data bytes
!!!!!
---- 10.0.14.24 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 3.31ms, avg = 4.45ms, max = 8.72ms, stddev = 2.14ms
*A:PE-1# ping router 15 10.0.15.25 rapid
PING 10.0.15.25 56 data bytes
!!!!!
---- 10.0.15.25 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 3.34ms, avg = 4.93ms, max = 8.62ms, stddev = 1.98ms

L3 services

L3 services on PE-1 and PE-2 shows the example topology with the following three L3 services configured on PE-1 and PE-2:

VPRN 6 with manually configured spoke-SDP 120:6
VPRN 7 with auto-bind to GRE tunnel
IES 8 with manually configured spoke-SDP 120:8

VPRN 6 is configured with a loopback interface and a GRE spoke-SDP, as follows:

# on PE-1: 
configure
    service
        system
            bgp-auto-rd-range 10.0.1.1 comm-val 60000 to 65000
        exit
        vprn 6 name "VPRN-6 with GRE spoke-SDP" customer 1 create
            interface "lo6" create
                address 172.16.16.1/32
                loopback
            exit
            bgp-ipvpn
                mpls
                    route-distinguisher auto-rd
                    vrf-target target:64500:6
                    no shutdown
                exit
            exit
            spoke-sdp 120:6 create
            exit
            no shutdown
        exit

The following forwarding information base (FIB) for VPRN 6 shows that the remote prefix is reachable via a transport tunnel using SDP 120:

*A:PE-1# show router 6 fib 1

===============================================================================
FIB Display
===============================================================================
Prefix [Flags]                                              Protocol
  NextHop
-------------------------------------------------------------------------------
172.16.16.1/32                                              LOCAL
  172.16.16.1 (lo6)
172.16.26.2/32                                              BGP_VPN
  10.0.2.1 (VPRN Label:524274 Transport:SDP:120)
-------------------------------------------------------------------------------
Total Entries : 2
-------------------------------------------------------------------------------
===============================================================================

VPRN 7 is configured with auto-bind-tunnel and the tunnel needs to be resolved using GRE. For services that support auto-binding to a GRE tunnel, the vpn-gre-source-ip parameter defines a single alternate source address for all VPRNs on the system. On PE-1, the configuration is as follows:

# on PE-1: 
configure
    service
        system
            vpn-gre-source-ip 10.0.1.1 
        exit
        vprn 7 name "VPRN-7 with auto-bind GRE" customer 1 create
            interface "lo7" create
                address 172.16.17.1/24
                loopback
            exit
            bgp-ipvpn
                mpls
                    auto-bind-tunnel
                        resolution-filter
                            gre
                        exit
                        resolution filter
                    exit
                    route-distinguisher auto-rd
                    vrf-target target:64500:7
                    no shutdown
                exit
            exit
            no shutdown
        exit

The following FIB for VPRN 7 shows that the remote prefix is reachable via a GRE transport tunnel:

*A:PE-1# show router 7 fib 1

===============================================================================
FIB Display
===============================================================================
Prefix [Flags]                                              Protocol
  NextHop
-------------------------------------------------------------------------------
172.16.17.0/24                                              LOCAL
  172.16.17.0 (lo7)
172.16.27.0/24                                              BGP_VPN
  10.0.2.1 (VPRN Label:524273 Transport:GRE)
-------------------------------------------------------------------------------
Total Entries : 2
-------------------------------------------------------------------------------
===============================================================================

IES 8 has an interface with a manually configured GRE spoke-SDP, as follows:

# on PE-1: 
configure 
    service
        ies 8 name "IES-8" customer 1 create
            interface "lo8" create
                address 172.16.18.1/24
                loopback
            exit
            interface "int-IES8-PE-1-PE-2" create
                address 172.16.128.1/30
                spoke-sdp 120:8 create
                    no shutdown
                exit
            exit
            no shutdown
        exit

On PE-1, the connectivity over the GRE spoke-SDP is verified as follows:

*A:PE-1# ping 172.16.128.2 rapid
PING 172.16.128.2 56 data bytes
!!!!!
---- 172.16.128.2 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 2.44ms, avg = 2.54ms, max = 2.69ms, stddev = 0.081ms

Conclusion

By default, GRE SDPs and auto-bind GRE tunnels are originated and terminated on the system IP address, but it is possible to use non-system IP addresses. This is useful in cases where the system IP address cannot be leaked between domains and a separate loopback address must be used to terminate tunnels.