GRE Tunnel Origination and Termination Using Non-system IP Addresses
This chapter provides information about GRE tunnel origination and termination using non-system IP addresses.
Topics in this chapter include:
Applicability
This chapter was initially written based on SR OS Release 16.0.R5, but the CLI in the current edition corresponds to SR OS Release 23.3.R2. GRE SDPs and auto-bind GRE tunnels can originate and terminate on a non-system IP address in SR OS Release 16.0.R4 or later.
Overview
For scaling purposes, service providers typically deploy seamless MPLS or inter-AS scenarios. In many cases, the system IP address cannot be leaked between domains and a separate loopback address is used to terminate tunnels. GRE termination on a non-system IP address is supported in the following services:
VPLS with manually configured GRE spoke-SDPs
VPLS with BGP-AD using provisioned GRE SDPs (use-provisioned-sdp or prefer-provisioned-sdp CLI commands)
BGP-VPLS using provisioned GRE SDPs
Epipe with manually configured GRE spoke-SDPs
Epipe with BGP-VPWS using provisioned GRE SDPs
VPRN with manually configured GRE spoke-SDPs
VPRN with auto-bind GRE tunnel
IES with manually configured GRE spoke-SDPs
This chapter focuses on MPLS-over-GRE termination, but IP-over-GRE termination is also supported.
MPLS-over-GRE termination
GRE termination applies to GRE SDPs and auto-bind GRE tunnels concurrently on a system interface and on non-system interfaces with a subnet that is up to and including /16. In the following example, the non-system loopback address 10.0.1.1 with a subnet of /24 is configured as GRE termination on PE-1:
# on PE-1:
configure
router Base
interface "lo1"
address 10.0.1.1/24
loopback
gre-termination
no shutdown
exit
Only one interface can be configured as GRE termination. The following error is raised when attempting to configure a second loopback interface "lo2" as GRE termination on PE-1:
*A:PE-1>config>router>if$ gre-termination
MINOR: CLI Could not set gre-termination for interface "lo2".
MINOR: PIP #2078 Cannot config GRE termination - already set on interface "lo1"
Although the preceding examples are for loopback interfaces, GRE termination can also be configured on other router interfaces, but only one per node. The following shows an attempt to configure interface "int-PE-1-PE-2" on PE-1 as GRE termination. The same error message is raised. However, if it were the first interface on the node to be configured as GRE termination, the configuration would be accepted.
*A:PE-1>config>router>if# gre-termination
MINOR: CLI Could not set gre-termination for interface "int-PE-1-PE-2".
MINOR: PIP #2078 Cannot config GRE termination - already set on interface "lo1"
The maximum size of the GRE termination subnet is /16.
GRE termination cannot be applied on the following interface types:
Unnumbered network IP interfaces
IES interfaces
VPRN interfaces
CSC VPRN interfaces
MPLS-over-GRE origination
GRE SDPs and auto-bind GRE tunnels can originate and terminate on a non-system IP address. Manually configured SDPs can be configured with a non-system IP address as the far-end address. Optionally, a non-system local-end address can be configured for generating GRE from an interface other than the system interface. In the following example on PE-1, GRE SDP 120 uses loopback address 10.0.1.1 as the local-end address and 10.0.2.1 on PE-2 as the far-end address.
# on PE-1:
configure
service
sdp 120 create
far-end 10.0.2.1
local-end 10.0.1.1
no shutdown
exit
The local-end IP address can only be configured for GRE SDPs; the following error message is raised when attempting to configure an MPLS SDP with a local-end address:
*A:PE-1>config>service# sdp 122 mpls create
*A:PE-1>config>service>sdp$ local-end 10.0.1.1
MINOR: SVCMGR #7825 Invalid local-end address - local-end not supported for this sdp type
The local-end parameter value complies with the following rules:
A maximum of 15 distinct address values can be configured for all GRE SDPs in the configure service sdp local-end context, and all L2oGRE SDPs under the configure service system gre-eth-bridged tunnel-termination context.
The same source address cannot be used in both contexts because an address configured for an L2oGRE SDP matches an internally created interface that is not available to other applications.
The local-end address of a GRE SDP, when different from the system address, need not match the primary address of an interface that has the MPLS-over-GRE termination subnet configured, unless a GRE SDP or tunnel from the far-end router terminates on this address.
The primary IPv4 address of any local network IP interface, loopback or not, may be used. The following shows that IP address 192.168.12.1, as the IP address of the previously mentioned interface "int-PE-1-PE-2" toward PE-2, can be used as the local-end address:
# on PE-1:
configure
service
sdp 123 create
far-end 10.0.2.1
local-end 192.168.12.1
no shutdown
exit
The following shows that an error message is raised when attempting to configure an invalid local-end IP address, that is, an IP address that is not primary on a local router interface. In this case, local-end IP address 10.99.1.1 does not exist on PE-1.
*A:PE-1>config>service# sdp 120 create
*A:PE-1>config>service>sdp$ local-end 10.99.1.1
MINOR: SVCMGR #7827 Cannot configure local-end IP address - Local router interface with
address does not exist, or address is not primary
For services that support auto-binding to a GRE tunnel, the following command configures a single alternate source address (in this case, 10.0.1.1) per system:
# on PE-1:
configure
service
system
vpn-gre-source-ip 10.0.1.1
exit
The default value of the single source address is the primary IPv4 address of the system interface. The value of the vpn-gre-source-ip parameter can be changed at any time. After a new value is configured, the system address will not be used in services that bind to the GRE tunnel.
The vpn-gre-source-ip parameter value complies with the following rules:
This single source address counts toward the maximum of 15 distinct address values per system used by all GRE SDPs under the configure service sdp local-end context and all L2oGRE SDPs under the configure service system gre-eth-bridged tunnel-termination context.
The same source address can be used in both vpn-gre-source-ip and configure service sdp local-end contexts.
The same source address cannot be used in both vpn-gre-source-ip and configure service system gre-eth-bridged tunnel-termination contexts because an address configured for an L2oGRE SDP matches an internally created interface that is not available to other applications.
The vpn-gre-source-ip address, when different from the system IP address, need not match the primary address of an interface that has the MPLS-over-GRE termination subnet configured, unless a GRE SDP or tunnel from the far-end router terminates on this address.
Configuration
Example topology shows the example topology with three SR OS nodes in AS 64500. Services will be configured on PE-1 and PE-2, while RR-3 is a route reflector (RR).
The initial configuration on the three PEs includes:
cards, MDAs, ports
router interfaces. The IP addresses shown on the figure are the system IP addresses 192.0.2.x/32.
IS-IS as IGP (alternatively, OSPF can be used)
GRE SDP termination on non-system IP addresses will be configured in the following use cases:
VPLS with manually configured T-LDP signaled SDP
Epipe with manually configured T-LDP signaled SDP
BGP-VPLS using a provisioned BGP-signaled SDP
BGP-AD in VPLS using a provisioned T-LDP signaled SDP
BGP-VPWS using a provisioned BGP-signaled SDP
VPRN with manually configured T-LDP signaled SDP
VPRN with auto-bind to GRE tunnel
IES with manually configured T-LDP signaled SDP
MPLS-over-GRE termination
On PE-1, PE-2, and RR-3, loopback interface "lo1" is configured as GRE termination with IPv4 address 10.0.x.1/24 for PE-x. The configuration on PE-1 is as follows:
# on PE-1:
configure
router Base
interface "lo1"
address 10.0.1.1/24
loopback
gre-termination
no shutdown
exit
This loopback interface will be used in the SDP configuration. With a /24 subnet, the SDP origination can be any address in the subnet. This is useful for providing entropy in the outer IPv4 header for load-balancing over the IP network.
MPLS-over-GRE origination: SDP local end
The local-end address must be reachable from the far-end router that terminates the GRE SDP. Therefore, the interface for this address can be added to IGP or BGP. Alternatively, a static route can be configured on the far-end router. In this example, IS-IS is enabled on the loopback interface with GRE termination, as follows:
# on PE-1, PE-2, RR-3:
configure
router Base
isis 0
interface "lo1"
exit
On PE-1, the following SDPs are configured with far-end 10.0.2.1 on PE-2 and local-end 10.0.1.1: SDP 120 with T-LDP signaling (default) and SDP 121 with BGP signaling.
# on PE-1:
configure
service
sdp 120 create
signaling tldp # default
far-end 10.0.2.1
local-end 10.0.1.1
no shutdown
exit
sdp 121 create
signaling bgp
far-end 10.0.2.1
local-end 10.0.1.1
no shutdown
exit
T-LDP signaled GRE SDPs
When T-LDP signaled SDPs, such as SDP 120 in the preceding example, are configured, T-LDP sessions are auto-created toward the far end of the SDPs. By default, LDP uses the system IP address as source address. However, if the source address for the T-LDP session does not match the destination transport address set by the remote PE, the T-LDP session will not come up and the GRE SDP will remain down.Mismatched T-LDP transport addresses shows an example where SDP auto-created T-LDP sessions use the local system addresses 192.0.2.x and far-end addresses 10.0.0.x, so the GRE SDPs will not come up.
Therefore, the local transport address of the T-LDP session must match the local-end address of the GRE SDP in the PE. These T-LDP sessions can be manually provisioned or auto-created via peer templates. The following configures T-LDP sessions between the non-system IP addresses on PE-1 and PE-2.
# on PE-1:
configure
router Base
ldp
targeted-session
peer 10.0.2.1
local-lsr-id "lo1"
exit
# on PE-2:
configure
router Base
ldp
targeted-session
peer 10.0.1.1
local-lsr-id "lo1"
exit
Matching T-LDP transport addresses shows the GRE T-LDP signaled SDPs with matching addresses for the T-LDP sessions.
BGP configuration
In this example, the L2 and L3 services are configured on PE-1 and PE-2, while RR-3 acts as the RR. On PE-1, BGP is configured with neighbor 10.0.3.1 and local address 10.0.1.1, as follows. Address family L2-VPN is required for L2 services using BGP-VPLS, BGP-AD, and BGP-VPWS; address family VPN-IPv4 is used for VPRN services.
# on PE-1:
configure
router Base
bgp
rapid-withdrawal
split-horizon
group "internal"
family vpn-ipv4 l2-vpn
type internal
local-address 10.0.1.1
neighbor 10.0.3.1
exit
exit
no shutdown
On RR-3, the BGP configuration is as follows.
# on RR-3:
configure
router Base
bgp
rapid-withdrawal
split-horizon
group "internal"
family vpn-ipv4 l2-vpn
type internal
cluster 10.0.3.1
local-address 10.0.3.1
neighbor 10.0.1.1
exit
neighbor 10.0.2.1
exit
exit
no shutdown
exit
The loopback addresses 10.0.x.1 are configured for the local and neighbor addresses.
When the local address 10.0.x.1 is not configured, the system address 192.0.2.x will be used instead. However, in that case, no BGP sessions will be established and, therefore, no BGP routes will be exchanged between 192.0.2.x and 10.0.y.1, and no spoke-SDPs will be auto-created in L2 services using BGP-VPLS, BGP-AD, or BGP-VWPS. Likewise, no BGP-VPN routes will be exchanged between VPRNs on PE-1 and PE-2.
L2 services
L2 services on PE-1 and PE-2 shows the example topology with the following L2 services configured on PE-1 and PE-2:
VPLS 1 with manually configured spoke-SDP 120:1
Epipe 2 with manually configured spoke-SDP 120:2
BGP-VPLS 3 using PW template 1 (BGP-signaled SDP 121 is used)
LDP VPLS 4 with BGP-AD using PW template 1 (T-LDP signaled SDP 120 is used)
BGP-VPWS Epipe 5 using PW template 1 (BGP-signaled SDP 121 is used)
The CEs are VPRNs configured on the PEs and connected to the VPLSs via port cross-connect (PXC).
For a description of the BGP-VPLS parameters, see the BGP VPLS chapter; for BGP-AD, see the LDP VPLS Using BGP Auto-Discovery chapter; for BGP-VPWS, see the BGP Virtual Private Wire Services chapter. For BGP-VPLS, BGP-AD, and BGP-VPWS, PW template 1 is configured with the use-provisioned-sdp command. The service configuration on PE-1 is as follows; the service configuration on PE-2 is similar.
# on PE-1:
configure
service
sdp 120 create
far-end 10.0.2.1
local-end 10.0.1.1
keep-alive
shutdown
exit
no shutdown
exit
sdp 121 create
signaling bgp
far-end 10.0.2.1
local-end 10.0.1.1
keep-alive
shutdown
exit
no shutdown
exit
pw-template 1 name "PW1-use-prov-SDP" use-provisioned-sdp create
exit
vpls 1 name "VPLS-1" customer 1 create
description "VPLS 1 with manually configured spoke-SDP"
stp
shutdown
exit
sap pxc-10.a:1 create
no shutdown
exit
spoke-sdp 120:1 create
no shutdown
exit
no shutdown
exit
epipe 2 name "Epipe-2" customer 1 create
description "Epipe 2 with manually configured spoke-SDP"
sap pxc-10.a:2 create
no shutdown
exit
spoke-sdp 120:2 create
no shutdown
exit
no shutdown
exit
vpls 3 name "BGP-VPLS-3" customer 1 create
description "BGP-VPLS with use provisioned SDP"
bgp
route-distinguisher 64500:3
route-target export target:64500:3 import target:64500:3
pw-template-binding 1
exit
exit
bgp-vpls
max-ve-id 100
ve-name "PE-1"
ve-id 1
exit
no shutdown
exit
stp
shutdown
exit
sap pxc-10.a:3 create
no shutdown
exit
no shutdown
exit
vpls 4 name "BGP-AD VPLS-4" customer 1 create
description "BGP-AD for LDP VPLS with use provisioned SDP"
bgp
route-distinguisher 64500:4
route-target export target:64500:4 import target:64500:4
pw-template-binding 1
exit
exit
bgp-ad
vpls-id 64500:4
no shutdown
exit
stp
shutdown
exit
sap pxc-10.a:4 create
no shutdown
exit
no shutdown
exit
epipe 5 name "BGP-VPWS-5" customer 1 create
description "BGP-VPWS with use provisioned SDP"
bgp
route-distinguisher 64500:5
route-target export target:64500:5 import target:64500:5
pw-template-binding 1
exit
exit
bgp-vpws
ve-name "PE-1"
ve-id 1
exit
remote-ve-name "PE-2"
ve-id 2
exit
no shutdown
exit
sap pxc-10.a:5 create
no shutdown
exit
no shutdown
exit
The following BGP sessions are established between PE-1 and RR-3 for the VPN-IPv4 and L2VPN address families:
*A:PE-1# show router bgp summary all
===============================================================================
BGP Summary
===============================================================================
Legend : D - Dynamic Neighbor
===============================================================================
Neighbor
Description
ServiceId AS PktRcvd InQ Up/Down State|Rcv/Act/Sent (Addr Family)
PktSent OutQ
-------------------------------------------------------------------------------
10.0.3.1
Def. Inst 64500 13 0 00h02m48s 0/0/0 (VpnIPv4)
15 0 3/3/3 (L2VPN)
-------------------------------------------------------------------------------
On PE-1, the following T-LDP session is established to 10.0.2.1 on PE-2:
*A:PE-1# show router ldp session ipv4
==============================================================================
LDP IPv4 Sessions
==============================================================================
Peer LDP Id Adj Type State Msg Sent Msg Recv Up Time
------------------------------------------------------------------------------
10.0.2.1:0 Targeted Established 52 53 0d 00:03:39
------------------------------------------------------------------------------
No. of IPv4 Sessions: 1
==============================================================================
On PE-1, the following SDPs are created with far end 10.0.2.1 and GRE delivery. For SDP 120, T-LDP signaling is used; BGP signaling is used for SDP 121.
*A:PE-1# show service sdp
============================================================================
Services: Service Destination Points
============================================================================
SdpId AdmMTU OprMTU Far End Adm Opr Del LSP Sig
----------------------------------------------------------------------------
120 0 8954 10.0.2.1 Up Up GRE n/a TLDP
121 0 8954 10.0.2.1 Up Up GRE n/a BGP
----------------------------------------------------------------------------
Number of SDPs : 2
----------------------------------------------------------------------------
Legend: R = RSVP, L = LDP, B = BGP, M = MPLS-TP, n/a = Not Applicable
I = SR-ISIS, O = SR-OSPF, T = SR-TE, F = FPE
============================================================================
On PE-1, the following SDP-bindings are used:
*A:PE-1# show service sdp-using
===============================================================================
SDP Using
===============================================================================
SvcId SdpId Type Far End Opr I.Label E.Label
State
-------------------------------------------------------------------------------
1 120:1 Spok 10.0.2.1 Up 524286 524286
2 120:2 Spok 10.0.2.1 Up 524285 524285
3 121:4294967295 BgpVp* 10.0.2.1 Up 524278 524277
4 120:4294967294 BgpAd 10.0.2.1 Up 524275 524275
5 121:4294967293 BgpVp* 10.0.2.1 Up 524276 524276
-------------------------------------------------------------------------------
Number of SDPs : 5
-------------------------------------------------------------------------------
===============================================================================
* indicates that the corresponding row element may have been truncated.
When the loopback interface "lo1" is configured as GRE termination on PE-1 and PE-2, the CEs can send traffic to each other. The following ping messages verify the connectivity between CE-11 and CE-21, CE-12 and CE-22, and so on:
*A:PE-1# ping router 11 10.0.11.21 rapid
PING 10.0.11.21 56 data bytes
!!!!!
---- 10.0.11.21 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 3.58ms, avg = 5.11ms, max = 10.3ms, stddev = 2.59ms
*A:PE-1# ping router 12 10.0.12.22 rapid
PING 10.0.12.22 56 data bytes
!!!!!
---- 10.0.12.22 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 3.37ms, avg = 4.54ms, max = 8.83ms, stddev = 2.15ms
*A:PE-1# ping router 13 10.0.13.23 rapid
PING 10.0.13.23 56 data bytes
!!!!!
---- 10.0.13.23 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 3.24ms, avg = 4.32ms, max = 8.02ms, stddev = 1.85ms
*A:PE-1# ping router 14 10.0.14.24 rapid
PING 10.0.14.24 56 data bytes
!!!!!
---- 10.0.14.24 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 3.31ms, avg = 4.45ms, max = 8.72ms, stddev = 2.14ms
*A:PE-1# ping router 15 10.0.15.25 rapid
PING 10.0.15.25 56 data bytes
!!!!!
---- 10.0.15.25 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 3.34ms, avg = 4.93ms, max = 8.62ms, stddev = 1.98ms
L3 services
L3 services on PE-1 and PE-2 shows the example topology with the following three L3 services configured on PE-1 and PE-2:
VPRN 6 with manually configured spoke-SDP 120:6
VPRN 7 with auto-bind to GRE tunnel
IES 8 with manually configured spoke-SDP 120:8
VPRN 6 is configured with a loopback interface and a GRE spoke-SDP, as follows:
# on PE-1:
configure
service
system
bgp-auto-rd-range 10.0.1.1 comm-val 60000 to 65000
exit
vprn 6 name "VPRN-6 with GRE spoke-SDP" customer 1 create
interface "lo6" create
address 172.16.16.1/32
loopback
exit
bgp-ipvpn
mpls
route-distinguisher auto-rd
vrf-target target:64500:6
no shutdown
exit
exit
spoke-sdp 120:6 create
exit
no shutdown
exit
The following forwarding information base (FIB) for VPRN 6 shows that the remote prefix is reachable via a transport tunnel using SDP 120:
*A:PE-1# show router 6 fib 1
===============================================================================
FIB Display
===============================================================================
Prefix [Flags] Protocol
NextHop
-------------------------------------------------------------------------------
172.16.16.1/32 LOCAL
172.16.16.1 (lo6)
172.16.26.2/32 BGP_VPN
10.0.2.1 (VPRN Label:524274 Transport:SDP:120)
-------------------------------------------------------------------------------
Total Entries : 2
-------------------------------------------------------------------------------
===============================================================================
VPRN 7 is configured with auto-bind-tunnel and the tunnel needs to be resolved using GRE. For services that support auto-binding to a GRE tunnel, the vpn-gre-source-ip parameter defines a single alternate source address for all VPRNs on the system. On PE-1, the configuration is as follows:
# on PE-1:
configure
service
system
vpn-gre-source-ip 10.0.1.1
exit
vprn 7 name "VPRN-7 with auto-bind GRE" customer 1 create
interface "lo7" create
address 172.16.17.1/24
loopback
exit
bgp-ipvpn
mpls
auto-bind-tunnel
resolution-filter
gre
exit
resolution filter
exit
route-distinguisher auto-rd
vrf-target target:64500:7
no shutdown
exit
exit
no shutdown
exit
The following FIB for VPRN 7 shows that the remote prefix is reachable via a GRE transport tunnel:
*A:PE-1# show router 7 fib 1
===============================================================================
FIB Display
===============================================================================
Prefix [Flags] Protocol
NextHop
-------------------------------------------------------------------------------
172.16.17.0/24 LOCAL
172.16.17.0 (lo7)
172.16.27.0/24 BGP_VPN
10.0.2.1 (VPRN Label:524273 Transport:GRE)
-------------------------------------------------------------------------------
Total Entries : 2
-------------------------------------------------------------------------------
===============================================================================
IES 8 has an interface with a manually configured GRE spoke-SDP, as follows:
# on PE-1:
configure
service
ies 8 name "IES-8" customer 1 create
interface "lo8" create
address 172.16.18.1/24
loopback
exit
interface "int-IES8-PE-1-PE-2" create
address 172.16.128.1/30
spoke-sdp 120:8 create
no shutdown
exit
exit
no shutdown
exit
On PE-1, the connectivity over the GRE spoke-SDP is verified as follows:
*A:PE-1# ping 172.16.128.2 rapid
PING 172.16.128.2 56 data bytes
!!!!!
---- 172.16.128.2 PING Statistics ----
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 2.44ms, avg = 2.54ms, max = 2.69ms, stddev = 0.081msConclusion
By default, GRE SDPs and auto-bind GRE tunnels are originated and terminated on the system IP address, but it is possible to use non-system IP addresses. This is useful in cases where the system IP address cannot be leaked between domains and a separate loopback address must be used to terminate tunnels.