Carrier Supporting Carrier IP VPNs
This chapter provides information about carrier supporting carrier IP VPN configurations.
Topics in this chapter include:
Applicability
This chapter was initially written for on SR OS Release 11.0.R1. The MD-CLI in the current edition corresponds to SR OS Release 22.2.R1. Carrier Supporting Carrier is supported on the 7750 SR and 7950 XRS.
Overview
Carrier Supporting Carrier (CSC) is a solution that allows one service provider (the Customer Carrier) to use the IP VPN service of another service provider (the Super Carrier) for some or all of its backbone transport. RFC 4364 defines a Carrier Supporting Carrier solution for BGP/MPLS IP VPNs that uses MPLS at the interconnection points between the two service providers to provide a scalable and secure solution.
A simplified CSC network topology is shown in CSC network topology. A CSC deployment involves the following types of devices:
-
CE — Customer premises equipment dedicated to one enterprise.
-
PE — Edge router managed and operated by the Customer Carrier that connects to CEs to provide business VPN or Internet services.
-
CSC-CE — Peering router managed and operated by the Customer Carrier that is connected to CSC-PEs for purposes of using the associated CSC IP VPN services for backbone transport. The CSC-CE may attach directly to CEs if it is also configured to be a PE for business VPN services.
-
CSC-PE — A PE router managed and operated by the Super Carrier that supports one or more CSC IP VPN services possibly in addition to other traditional PE services.
In the CSC solution, the CSC-CE and CSC-PE are directly connected by a link that supports MPLS. The CSC-CE distributes an MPLS label for every /32 IPv4 prefix it and any downstream PE uses as a BGP next-hop in routes associated with services offered by the Customer Carrier. BGP must be used as the label distribution protocol between CSC-CE and CSC-PE if the latter device is an SR OS node. Typically, the Customer Carrier and Super Carrier operate as two different Autonomous Systems (ASs) and therefore BGP, more specifically EBGP, is the best label distribution protocol, even if other options are available. The BGP session between CSC-CE and CSC-PE must be single-hop EBGP (or IBGP) if either device is an SR OS node.
In an SR OS CSC-PE, the interface to a CSC-CE is a special type of IP/MPLS interface that belongs to a VPRN configured for CSC mode. This special type of interface is called a CSC VPRN interface throughout the remainder of this chapter. The CSC VPRN interface has many of the same characteristics as a network interface of the base router but its association with a Virtual Routing and Forwarding (VRF) ensures that the traffic and control plane routes of the Customer Carrier are kept separate from other services.
When an SR OS CSC-PE receives a labeled-IPv4 route (with label L1, next-hop N1) from a CSC-CE BGP peer, the following actions take place in the CSC-PE:
-
The BGP route is installed into the routing table of the CSC VPRN (assuming the BGP route is the best route to the destination).
-
If the BGP route matches the VRF export policy, it is advertised to the core Multi-Protocol Border Gateway Protocol (MP-BGP) peers as a VPN-IPv4 route. The advertised label value is changed to label value L2.
-
BGP programs the line cards with an MPLS forwarding entry that swaps label value L2 for L1 and sends the MPLS packet over the CSC VPRN interface associated with next-hop N1.
When an SR OS CSC-PE receives a VPN-IPv4 route (with label L2, next-hop N2) the following actions take place in the CSC-PE:
-
If the VPN-IPv4 route matches the VRF import policy of a CSC VPRN, it is installed into the routing table of that CSC VPRN.
-
If the imported BGP-VPN route matches the BGP export policy associated with a CSC-CE BGP peer, it is advertised to that peer as a labeled-IPv4 route. The advertised label value is changed to label value L3.
-
BGP programs the line cards with an MPLS forwarding entry that swaps label value L3 for L2 and sends the packet inside the MPLS tunnel to next-hop N2.
Once a CSC-CE has learned a labeled-IPv4 route for a remote CSC-CE and vice versa, the two CSC-CEs can set up a BGP session between themselves and exchange VPN routes over this session if they are both PEs with services. Typically, this BGP session will be an IBGP session because the local and remote CSC-CEs belong to the same AS. The Layer 2 VPN and Layer 3 VPN routes exchanged by the CSC-CEs are resolved by the labeled-IPv4 routes they have for each other’s /32 IPv4 address.
Configuration
This section will walk through the steps to configure the CSC solution shown in CSC network topology. The IPv4 addresses in CSC network topology are the system IP addresses of the routers. The steps are the following:
-
Configure CSC-CE-1
-
Configure CSC service on CSC-PE-2
-
Verify exchange of labeled IPv4 routes between CSC-CE-1 and CSC-PE-2
-
Configure core connectivity for CSC-PE-2
-
Configure core connectivity for CSC-PE-3
-
Configure CSC service on CSC-PE-3
-
Verify exchange of VPN-IPv4 routes between CSC-PE-2 and CSC-PE-3
-
Configure CSC-CE-4
-
Verify exchange of labeled IPv4 routes between CSC-PE-3 and CSC-CE-4
-
Configure BGP session between CSC-CE-1 and CSC-CE-4
-
Verify exchange of VPN-IPv4 routes between CSC-CE-1 and CSC-CE-4
Step 1. Configure CSC-CE-1
This example assumes that CSC-CE-1 is a PE router with Layer 2 and Layer 3 VPN services that must extend across the CSC VPN service; assume that there are no further downstream PEs in AS 64496. The configuration of one such Layer 3 VPN service in CSC-CE-1 is as follows:
# on CSC-CE-1:
configure {
service {
vprn "VPRN1" {
admin-state enable
service-id 1
customer "1"
bgp-ipvpn {
mpls {
admin-state enable
route-distinguisher "64496:11"
vrf-target {
community "target:64496:1"
}
auto-bind-tunnel {
resolution any
}
}
}
interface "loopback-1" {
loopback true
ipv4 {
primary {
address 10.11.30.2
prefix-length 24
}
}
}
}
For brevity, the preceding configuration sample omits commands related to SAP IP interfaces, spoke-SDP IP interfaces, PE-CE routing protocols, QoS, IP filters, and so on. The loopback interface is used to test whether this prefix is learned at the remote CSC-CE-4.
The base routing instance of the CSC-CE is configured with the appropriate router ID and autonomous system number and the system interface is configured with an IPv4 address (usually the same as the router ID). If the router ID is not configured, by default, the system IP address is used as the router ID. The interface to CSC-PE-2 is created and configured. The base router configuration of CSC-CE-1 is as follows:
# on CSC-CE-1:
configure {
router "Base" {
autonomous-system 64496
interface "int-CSC-CE-1-CSC-PE-2" {
port 1/1/1:1 # connected to VPRN1 network interface on CSC-PE-2
ipv4 {
primary {
address 192.168.12.1
prefix-length 30
}
}
}
interface "system" {
ipv4 {
primary {
address 192.0.2.1
prefix-length 32
}
}
}
On CSC-CE-1, BGP is configured as the control plane protocol running on the interface to CSC-PE-2. The export policy exports the system IP address of CSC-CE-1 as a labeled-IPv4 route to CSC-PE-2; the import policy imports other system IP addresses, in this case, the system IP address of CSC-CE-4.
# on CSC-CE-1:
configure {
policy-options {
prefix-list "system-IP" {
prefix 192.0.2.0/29 type longer {
}
}
policy-statement "export-systemIP" {
entry 10 {
from {
prefix-list ["system-IP"]
protocol {
name [direct]
}
}
action {
action-type accept
}
}
default-action {
action-type reject
}
}
policy-statement "import-systemIP" {
entry 10 {
from {
prefix-list ["system-IP"]
}
action {
action-type accept
}
}
default-action {
action-type reject
}
}
}
router "Base" {
autonomous-system 64496
bgp {
group "CSC-PE" {
peer-as 64500
}
neighbor "192.168.12.2" {
split-horizon true
group "CSC-PE"
family {
label-ipv4 true
}
import {
policy ["import-systemIP"]
}
export {
policy ["export-systemIP"]
}
}
The peer type is EBGP (peer-as is different from the locally configured autonomous-system)
The address family for the EBGP session is label-ipv4 (the neighbor address is an IPv4 address). Family label-IPv4 causes MP-BGP negotiation of the address family for AFI=1 and SAFI=4 (IPv4 NLRI with MPLS labels), as can be observed from the following debug message (in this example, debugging is enabled on CSC-CE-1 for BGP OPEN messages using the command debug router bgp open).This BGP OPEN message can obviously only be seen when the BGP peer is up. The configuration for CSC-PE-2 will be shown later, but in order to have the debug message, it must be configured already.
# on CSC-CE-1:
1 2022/04/06 15:54:53.648 CEST MINOR: DEBUG #2001 Base BGP
"BGP: OPEN
Peer 1: 192.168.12.2 - Send (Passive) BGP OPEN: Version 4
AS Num 64496: Holdtime 90: BGP_ID 192.0.2.1: Opt Length 20 (ExtOpt F)
Opt Para: Type CAPABILITY: Length = 18: Data:
Cap_Code GRACEFUL-RESTART: Length 2
Bytes: 0x0 0x78
Cap_Code MP-BGP: Length 4
Bytes: 0x0 0x1 0x0 0x4
Cap_Code ROUTE-REFRESH: Length 0
Cap_Code 4-OCTET-ASN: Length 4
Bytes: 0x0 0x0 0xfb 0xf0
"
The split-horizon true command is optional. It prevents a best BGP route from the CSC-PE peer from being re-advertised back to that peer.
Step 2. Configure CSC service on SCS-PE-2
CSC-PE-2 must be configured with a VPRN in carrier-carrier-vpn mode to provide CSC service to CSC-CE-1. VPRN 1 is configured on CSC-PE-2, as follows:
# on CSC-PE-2:
configure {
service {
vprn "VPRN1" {
admin-state enable
service-id 1
customer "1"
autonomous-system 64500
router-id 192.0.2.2
carrier-carrier-vpn true
bgp-ipvpn {
mpls {
admin-state enable
route-distinguisher "64500:12"
vrf-target {
community "target:64500:1"
}
auto-bind-tunnel {
resolution any
}
}
}
bgp {
group "CSC-CE" {
as-override true
peer-as 64496
ebgp-default-reject-policy {
import false
}
export {
policy ["BGP-VPN-routes"]
}
}
neighbor "192.168.12.1" {
split-horizon true
group "CSC-CE"
family {
label-ipv4 true
}
}
}
network-interface "int-CSC-PE-2-CSC-CE-1" {
port 1/1/2:1
ipv4 {
primary {
address 192.168.12.2
prefix-length 30
}
}
}
}
The carrier-carrier-vpn true command is mandatory. It cannot be configured if the VPRN has any SAP or spoke-SDP access interfaces configured; they must first be deleted.
*[ex:/configure service vprn "VPRN1"]
A:admin@CSC-PE-2# carrier-carrier-vpn true
*[ex:/configure service vprn "VPRN1"]
A:admin@CSC-PE-2# commit
MINOR: COMMON #238: configure service vprn "VPRN1" carrier-carrier-vpn - Configuration change failed validation - combination of carrier-carrier-vpn and service interfaces present. - configure service vprn "VPRN1"
The auto-bind-tunnel command must be set appropriately for the type of transport desired to other CSC-PEs, but note that GRE is not supported.
*[ex:/configure service vprn "VPRN1" bgp-ipvpn mpls auto-bind-tunnel resolution-filter]
A:admin@CSC-PE-2# gre true
*[ex:/configure service vprn "VPRN1" bgp-ipvpn mpls auto-bind-tunnel resolution-filter]
A:admin@CSC-PE-2# commit
INFO: PIP #1195: configure service vprn "VPRN1" carrier-carrier-vpn - Cannot toggle carrier-carrier-vpn - GRE auto-bind enabled - configure service vprn "VPRN1" bgp-ipvpn mpls auto-bind-tunnel resolution-filter gre
The interface to CSC-CE-1 must be a network interface. A network interface can be associated with an entire Ethernet port, a VLAN sub-interface of an Ethernet port, an entire LAG or a VLAN sub-interface of a LAG. In all cases, the associated Ethernet ports must be configured in network or hybrid mode.
The peer type is EBGP (peer-as is different from the locally configured autonomous-system).
The address family for the EBGP session is label-ipv4 (the neighbor address is an IPv4 address). Address family label-ipv4 causes MP-BGP negotiation of the address family for AFI=1 and SAFI=4 (IPv4 NLRI with MPLS labels).
The split-horizon true command is optional. It prevents a best BGP route from the CSC-CE peer from being re-advertised back to that peer.
The as-override command replaces CSC-CE-1’s AS number 64496 with CSC-PE-2’s AS number 64500 in the AS_PATH attribute of routes advertised to CSC-CE-1. Without this configuration, CSC-CE-1 may reject routes originated by CSC-CE-4 as invalid due to an AS-path loop.
The export command applies a BGP export policy to the session. The configuration of the policy is as follows:
# on CSC-PE-2:
configure {
policy-options {
policy-statement "BGP-VPN-routes" {
entry 10 {
from {
protocol {
name [bgp-vpn]
}
}
action {
action-type accept
}
}
default-action {
action-type reject
}
}
The effect of the BGP export policy is to re-advertise VPN-IPv4 routes imported into the CSC VPRN (and used for forwarding) to CSC-CE-4.
Step 3. Verify exchange of labeled IPv4 routes
When steps 1 and 2 have been completed, CSC-CE-1 advertises the labeled-IPv4 route for its system IP address 192.0.2.1/32 to CSC-PE-2. This can be checked in the RIB Out of CSC-CE-1, as follows:
[/]
A:admin@CSC-CE-1# show router bgp routes 192.0.2.1/32 label-ipv4 hunt
===============================================================================
BGP Router ID:192.0.2.1 AS:64496 Local AS:64496
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP LABEL-IPV4 Routes
===============================================================================
-------------------------------------------------------------------------------
RIB In Entries
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
RIB Out Entries
-------------------------------------------------------------------------------
Network : 192.0.2.1/32
Nexthop : 192.168.12.1
Path Id : None
To : 192.168.12.2
Res. Nexthop : n/a
Local Pref. : n/a Interface Name : NotAvailable
Aggregator AS : None Aggregator : None
Atomic Aggr. : Not Atomic MED : None
AIGP Metric : None IGP Cost : n/a
Connector : None
Community : No Community Members
Cluster : No Cluster Members
Originator Id : None Peer Router Id : 192.0.2.2
IPv4 Label : 524287 Label Type : POP
Lbl Allocation : NEXT-HOP
Origin : IGP
AS-Path : 64496
Route Tag : 0
Neighbor-AS : 64496
Orig Validation: NotFound
Source Class : 0 Dest Class : 0
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
CSC-CE-1 has advertised a label value of 524287 with the prefix.
The following output shows the received label-IPv4 route in the RIB In for VPRN "VPRN1" on CSC-PE-2:
[/]
A:admin@CSC-PE-2# show router 1 bgp routes 192.0.2.1/32 label-ipv4 hunt
===============================================================================
BGP Router ID:192.0.2.2 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP LABEL-IPV4 Routes
===============================================================================
-------------------------------------------------------------------------------
RIB In Entries
-------------------------------------------------------------------------------
Network : 192.0.2.1/32
Nexthop : 192.168.12.1
Path Id : None
From : 192.168.12.1
Res. Nexthop : 192.168.12.1
Local Pref. : None Interface Name : int-CSC-PE-2-CSC-CE-1
Aggregator AS : None Aggregator : None
Atomic Aggr. : Not Atomic MED : None
AIGP Metric : None IGP Cost : 0
Connector : None
Community : No Community Members
Cluster : No Cluster Members
Originator Id : None Peer Router Id : 192.0.2.1
Fwd Class : None Priority : None
IPv4 Label : 524287
Flags : Used Valid Best IGP In-TTM In-RTM
Route Source : External
AS-Path : 64496
Route Tag : 0
Neighbor-AS : 64496
Orig Validation: NotFound
Source Class : 0 Dest Class : 0
Add Paths Send : Default
RIB Priority : Normal
Last Modified : 00h01m30s
-------------------------------------------------------------------------------
RIB Out Entries
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
Step 4. Configure core connectivity for CSC-PE-2
The next step is to configure the base router instance of CSC-PE-2 so that it can exchange VPN-IPv4 routes with CSC-PE-3 (and potentially other CSC-PEs). This requires:
-
Router ID and autonomous system configuration.
-
Network interface creation and configuration, including assignment of an IPv4 address to the system interface.
-
Configuration of the IGP protocol; in this example, IS-IS is used.
-
Configuration of the LDP protocol (optional).
-
Configuration of RSVP LSPs used to reach remote CSC-PE devices (optional).
-
Configuration of the BGP protocol.
The base router configuration of CSC-PE-2 is as follows:
# on CSC-PE-2:
configure {
router "Base" {
autonomous-system 64500
interface "int-CSC-PE-2-CSC-PE-3" {
port 1/1/1:1000
ipv4 {
primary {
address 192.168.23.1
prefix-length 30
}
}
}
interface "system" {
ipv4 {
primary {
address 192.0.2.2
prefix-length 32
}
}
}
bgp {
group "core" {
type internal
}
neighbor "192.0.2.3" {
group "core"
family {
vpn-ipv4 true
}
}
}
isis 0 {
admin-state enable
level-capability 2
area-address [49.01]
interface "int-CSC-PE-2-CSC-PE-3" {
interface-type point-to-point
}
interface "system" {
passive true
}
level 2 {
wide-metrics-only true
}
}
ldp {
interface-parameters {
interface "int-CSC-PE-2-CSC-PE-3" {
ipv4 {
}
}
}
}
The peer type is IBGP (type internal. It is also possible to configure this in a similar way as for EBGP, with the same value for peer-as as the locally configured autonomous-system).
The transport for the IBGP session is IPv4 (the neighbor address is an IPv4 address).
The family vpn-ipv4 command causes MP-BGP negotiation of the address family for AFI=1 and SAFI=128 (=0x80), as can be observed from the following debug trace of the BGP OPEN message from CSC-PE-2 to CSC-PE-3.
1 2022/04/06 16:01:45.985 CEST MINOR: DEBUG #2001 Base BGP
"BGP: OPEN
Peer 1: 192.0.2.3 - Send (Active) BGP OPEN: Version 4
AS Num 64500: Holdtime 90: BGP_ID 192.0.2.2: Opt Length 20 (ExtOpt F)
Opt Para: Type CAPABILITY: Length = 18: Data:
Cap_Code GRACEFUL-RESTART: Length 2
Bytes: 0x0 0x78
Cap_Code MP-BGP: Length 4
Bytes: 0x0 0x1 0x0 0x80
Cap_Code ROUTE-REFRESH: Length 0
Cap_Code 4-OCTET-ASN: Length 4
Bytes: 0x0 0x0 0xfb 0xf4
"
Step 5. Configure core connectivity for CSC-PE-3
The next step is to configure the base router instance of CSC-PE-3 so that it can exchange VPN-IPv4 routes with CSC-PE-2 and potentially other CSC-PEs. This requires:
-
Router ID and AS configuration.
-
Network interface creation and configuration, including assignment of an IPv4 address to the system interface.
-
Configuration of the IGP protocol; in this example IS-IS is used.
-
Configuration of the LDP protocol (optional).
-
Configuration of RSVP LSPs used to reach remote CSC-PE devices (optional).
-
Configuration of the BGP protocol.
The base router configuration of CSC-PE-3 is as follows:
# on CSC-PE-3:
configure {
router "Base" {
autonomous-system 64500
interface "int-CSC-PE-3-CSC-PE-2" {
port 1/1/2:1000
ipv4 {
primary {
address 192.168.23.2
prefix-length 30
}
}
}
interface "system" {
ipv4 {
primary {
address 192.0.2.3
prefix-length 32
}
}
}
bgp {
group "core" {
type internal
cluster {
cluster-id 192.0.2.3
}
}
neighbor "192.0.2.2" {
split-horizon true
group "core"
family {
vpn-ipv4 true
}
}
}
isis 0 {
admin-state enable
level-capability 2
area-address [49.01]
interface "int-CSC-PE-3-CSC-PE-2" {
interface-type point-to-point
}
interface "system" {
passive true
}
level 2 {
wide-metrics-only true
}
}
ldp {
interface-parameters {
interface "int-CSC-PE-3-CSC-PE-2" {
ipv4 {
}
}
}
}
The peer type is IBGP (type internal. Can also be configured with peer-as equal to the locally configured autonomous-system).
The transport for the IBGP session is IPv4 (the neighbor address is an IPv4 address).
The family vpn-ipv4 command causes MP-BGP negotiation of the address family for AFI=1 and SAFI=128.
The cluster command configures CSC-PE-2 as a route reflector for clients in the BGP group "core". This is not required and in a more typical deployment, the route reflector would be a separate router from any CSC-PE.
Step 6. Configure CSC service on CSC-PE-3
CSC-PE-3 must be configured with a VPRN in carrier-carrier-vpn mode to provide CSC service to CSC-CE-4. The configuration of the VPRN is as follows:
# on CSC-PE-3:
configure {
service {
vprn "VPRN1" {
admin-state enable
service-id 1
customer "1"
autonomous-system 64500
router-id 192.0.2.3
carrier-carrier-vpn true
bgp-ipvpn {
mpls {
admin-state enable
route-distinguisher "64500:13"
vrf-target {
community "target:64500:1"
}
auto-bind-tunnel {
resolution any
}
}
}
bgp {
group "CSC-CE" {
as-override true
peer-as 64496
ebgp-default-reject-policy {
import false
}
export {
policy ["BGP-VPN-routes"]
}
}
neighbor "192.168.34.2" {
split-horizon true
group "CSC-CE"
family {
label-ipv4 true
}
}
}
network-interface "int-CSC-PE-3-CSC-CE-4" {
port 1/1/1:1
ipv4 {
primary {
address 192.168.34.1
prefix-length 30
}
}
}
}
The carrier-carrier-vpn true command is mandatory. It cannot be configured if the VPRN has any SAP or spoke-SDP access interfaces configured; they must first be removed.
The auto-bind-tunnel command must be set appropriately for the type of transport desired to other CSC-PEs, but GRE is not supported.
The interface to CSC-CE-4 must be a network interface. A network interface can be associated with an entire Ethernet port, a VLAN sub-interface of an Ethernet port, an entire LAG or a VLAN sub-interface of a LAG. In all cases, the associated Ethernet ports must be configured in network or hybrid mode.
The peer type is EBGP (peer-as is different from the locally configured autonomous-system).
The address family for the EBGP session is label-ipv4 (the neighbor address is an IPv4 address). Address family label-ipv4 causes MP-BGP negotiation of the address family for AFI=1 and SAFI=4 (IPv4 NLRI with MPLS labels).
The split-horizon true command is optional. It prevents a best BGP route from the CSC-CE peer from being re-advertised back to that peer.
The as-override command replaces CSC-CE-4’s AS number 64496 with CSC-PE-3’s AS number 64500 in the AS_PATH attribute of routes advertised to CSC-CE-4. Without this configuration, CSC-CE-4 may reject routes originated by CSC-CE-1 as invalid due to an AS-path loop.
The export command applies a BGP export policy to the session. The configuration of the policy is as follows:
# on CSC-PE-3:
configure {
policy-options {
policy-statement "BGP-VPN-routes" {
entry 10 {
from {
protocol {
name [bgp-vpn]
}
}
action {
action-type accept
}
}
default-action {
action-type reject
}
}
The effect of the BGP export policy is to re-advertise VPN-IPv4 routes imported into the CSC VPRN (and used for forwarding) to CSC-CE-4.
Step 7. Verify exchange of VPN-IPv4 routes between CSC-PE-2 and CSC-PE-3.
When the preceding steps have been completed, CSC-PE-2 advertises the labeled-IPv4 route for 192.0.2.1/32 (the system IP address of CSC-CE-1) to CSC-PE-3. This can be checked in the RIB Out of CSC-PE-2, as follows:
[/]
A:admin@CSC-PE-2# show router bgp routes 192.0.2.1/32 vpn-ipv4 hunt
===============================================================================
BGP Router ID:192.0.2.2 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP VPN-IPv4 Routes
===============================================================================
-------------------------------------------------------------------------------
RIB In Entries
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
RIB Out Entries
-------------------------------------------------------------------------------
Network : 192.0.2.1/32
Nexthop : 192.0.2.2
Route Dist. : 64500:12 VPN Label : 524285
Path Id : None
To : 192.0.2.3
Res. Nexthop : n/a
Local Pref. : 100 Interface Name : NotAvailable
Aggregator AS : None Aggregator : None
Atomic Aggr. : Not Atomic MED : None
AIGP Metric : None IGP Cost : n/a
Connector : None
Community : target:64500:1
Cluster : No Cluster Members
Originator Id : None Peer Router Id : 192.0.2.3
Origin : IGP
AS-Path : 64496
Route Tag : 0
Neighbor-AS : 64496
Orig Validation: N/A
Source Class : 0 Dest Class : 0
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
CSC-PE-2 has advertised a VPN label value of 524285 with the prefix.
The following output shows the received route in the RIB In of CSC-PE-3:
[/]
A:admin@CSC-PE-3# show router bgp routes 192.0.2.1/32 vpn-ipv4 hunt
===============================================================================
BGP Router ID:192.0.2.3 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP VPN-IPv4 Routes
===============================================================================
-------------------------------------------------------------------------------
RIB In Entries
-------------------------------------------------------------------------------
Network : 192.0.2.1/32
Nexthop : 192.0.2.2
Route Dist. : 64500:12 VPN Label : 524285
Path Id : None
From : 192.0.2.2
Res. Nexthop : n/a
Local Pref. : 100 Interface Name : int-CSC-PE-3-CSC-PE-2
Aggregator AS : None Aggregator : None
Atomic Aggr. : Not Atomic MED : None
AIGP Metric : None IGP Cost : 10
Connector : None
Community : target:64500:1
Cluster : No Cluster Members
Originator Id : None Peer Router Id : 192.0.2.2
Fwd Class : None Priority : None
Flags : Used Valid Best IGP
Route Source : Internal
AS-Path : 64496
Route Tag : 0
Neighbor-AS : 64496
Orig Validation: N/A
Source Class : 0 Dest Class : 0
Add Paths Send : Default
Last Modified : 00h00m25s
VPRN Imported : 1
-------------------------------------------------------------------------------
RIB Out Entries
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
The label swap entries that BGP programmed in the line cards of CSC-PE-2 based on the received labeled-IPv4 route from CSC-CE-1 (Label Origin = ExtCarCarVpn) and the advertised VPN-IPv4 route to CSC-PE-3, as follows:
[/]
A:admin@CSC-PE-2# show router bgp inter-as-label
===============================================================================
BGP Inter-AS labels
Flags: B - entry has backup, P - entry is promoted
===============================================================================
NextHop Received Advertised Label
Label Label Origin
-------------------------------------------------------------------------------
192.168.12.1 524287 524285 ExtCarCarVpn
-------------------------------------------------------------------------------
Total Labels allocated: 1
===============================================================================
Step 8. Configure CSC-CE-4
In this example, CSC-CE-4 is a PE router with Layer 2 and Layer 3 VPN services that must extend across the CSC VPN service. The configuration of one such Layer 3 VPN service in CSC-CE-4 is as follows:
# on CSC-CE-4:
configure {
service {
vprn "VPRN1" {
admin-state enable
service-id 1
customer "1"
bgp-ipvpn {
mpls {
admin-state enable
route-distinguisher "64496:14"
vrf-target {
community "target:64496:1"
}
auto-bind-tunnel {
resolution any
}
}
}
interface "loopback-1" {
loopback true
ipv4 {
primary {
address 10.14.30.2
prefix-length 24
}
}
}
}
For brevity, the preceding configuration sample omits commands related to SAP IP interfaces, spoke-SDP IP interfaces, PE-CE routing protocols, QoS, IP filters, and so on.
The base routing instance of CSC-CE-4 is configured with the appropriate router ID and AS number and the system interface has an IPv4 address (usually the same as the router ID). The interface to CSC-PE-3 is configured. The base router configuration of CSC-CE-4 is as follows:
# on CSC-CE-4:
configure {
router "Base" {
interface "int-CSC-CE-4-CSC-PE-3" {
port 1/1/2:1 # connected to VPRN1 network interface on CSC-PE-3
ipv4 {
primary {
address 192.168.34.2
prefix-length 30
}
}
}
interface "system" {
ipv4 {
primary {
address 192.0.2.4
prefix-length 32
}
}
}
BGP is configured as the control plane protocol running on the interface to CSC-PE-3, as follows:
# on CSC-CE-4:
configure {
router "Base" {
autonomous-system 64496
bgp {
group "CSC-PE" {
peer-as 64500
}
neighbor "192.168.34.1" {
split-horizon true
group "CSC-PE"
family {
label-ipv4 true
}
import {
policy ["import-systemIP"]
}
export {
policy ["export-systemIP"]
}
}
The peer type is EBGP (peer-as is different from the locally configured autonomous-system).
The address family for the EBGP session is label-ipv4 (the neighbor address is an IPv4 address). Address family label-ipv4 causes MP-BGP negotiation of the address family for AFI=1 and SAFI=4 (IPv4 NLRI with MPLS labels).
The split-horizon true command is optional. It prevents a best BGP route from the CSC-PE peer from being re-advertised back to that peer.
The export and import commands apply BGP export and import policies to the session. The configuration of the policies is as follows:
# on CSC-CE-4:
configure {
policy-options {
prefix-list "system-IP" {
prefix 192.0.2.0/29 type longer {
}
}
policy-statement "export-systemIP" {
entry 10 {
from {
prefix-list ["system-IP"]
protocol {
name [direct]
}
}
action {
action-type accept
}
}
default-action {
action-type reject
}
}
policy-statement "import-systemIP" {
entry 10 {
from {
prefix-list ["system-IP"]
}
action {
action-type accept
}
}
default-action {
action-type reject
}
}
The purpose of the BGP export policy is to advertise the system IP address of CSC-CE-4 as a labeled-IPv4 BGP route toward CSC-PE-3. The import policy imports the system IP address of CSC-CE-1.
Step 9. Verify exchange of labeled IPv4 routes between CSC-PE-3 and CSC-CE-4
When the preceding steps are completed, CSC-PE-3 advertises the labeled-IPv4 route for 192.0.2.1/32 to CSC-CE-4. This can be checked in the RIB Out for CSC VPRN 1 on CSC-PE-3, as follows:
[/]
A:admin@CSC-PE-3# show router 1 bgp routes 192.0.2.1/32 label-ipv4 hunt
===============================================================================
BGP Router ID:192.0.2.3 AS:64500 Local AS:64500
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP LABEL-IPV4 Routes
===============================================================================
-------------------------------------------------------------------------------
RIB In Entries
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
RIB Out Entries
-------------------------------------------------------------------------------
Network : 192.0.2.1/32
Nexthop : 192.168.34.1
Path Id : None
To : 192.168.34.2
Res. Nexthop : n/a
Local Pref. : n/a Interface Name : NotAvailable
Aggregator AS : None Aggregator : None
Atomic Aggr. : Not Atomic MED : None
AIGP Metric : None IGP Cost : n/a
Connector : None
Community : No Community Members
Cluster : No Cluster Members
Originator Id : None Peer Router Id : 192.0.2.4
IPv4 Label : 524285 Label Type : SWAP
Lbl Allocation : NEXT-HOP
Origin : IGP
AS-Path : 64500 64500
Route Tag : 0
Neighbor-AS : 64500
Orig Validation: NotFound
Source Class : 0 Dest Class : 0
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
CSC-PE-3 has advertised a label value of 524285 with the prefix.
The following output shows the received route in the RIB In of CSC-CE-4:
[/]
A:admin@CSC-CE-4# show router bgp routes 192.0.2.1/32 label-ipv4 hunt
===============================================================================
BGP Router ID:192.0.2.4 AS:64496 Local AS:64496
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP LABEL-IPV4 Routes
===============================================================================
-------------------------------------------------------------------------------
RIB In Entries
-------------------------------------------------------------------------------
Network : 192.0.2.1/32
Nexthop : 192.168.34.1
Path Id : None
From : 192.168.34.1
Res. Nexthop : 192.168.34.1
Local Pref. : None Interface Name : int-CSC-CE-4-CSC-PE-3
Aggregator AS : None Aggregator : None
Atomic Aggr. : Not Atomic MED : None
AIGP Metric : None IGP Cost : 0
Connector : None
Community : No Community Members
Cluster : No Cluster Members
Originator Id : None Peer Router Id : 192.0.2.3
Fwd Class : None Priority : None
IPv4 Label : 524285
Flags : Used Valid Best IGP In-TTM In-RTM
Route Source : External
AS-Path : 64500 64500
Route Tag : 0
Neighbor-AS : 64500
Orig Validation: NotFound
Source Class : 0 Dest Class : 0
Add Paths Send : Default
RIB Priority : Normal
Last Modified : 00h01m12s
-------------------------------------------------------------------------------
RIB Out Entries
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
The BGP distributed labels are programmed in the line cards of CSC-PE-3 based on the received VPN-IPv4 routes from CSC-PE-2 (Label Origin = Internal) and the advertised labeled-IPv4 routes to CSC-CE-4:
[/]
A:admin@CSC-PE-3# show router 1 bgp inter-as-label
===============================================================================
BGP Inter-AS labels
Flags: B - entry has backup, P - entry is promoted
===============================================================================
NextHop Received Advertised Label
Label Label Origin
-------------------------------------------------------------------------------
192.0.2.2 524284 524283 Internal
192.0.2.2 524285 524285 Internal
-------------------------------------------------------------------------------
Total Labels allocated: 2
===============================================================================
In the preceding output, the second entry for NextHop 192.0.2.2 corresponds to the prefix 192.0.2.1/32; recall from Step 7 that CSC-PE-3 received the VPN-IPv4 route with label value 524285 and it can be seen from this step that it re-advertised the route to CSC-CE-4 with the same label value 524285.
Step 10. Configure BGP session between CSC-CE-1 and CSC-CE-4
The final step in the setup of the CSC solution shown in CSC network topology is the creation of a BGP session between CSC-CE-1 and CSC-CE-4 so that they can exchange routes belonging to VPN services they support. The configuration of this BGP session on CSC-CE-1 is as follows:
# on CSC-CE-1:
configure {
router "Base" {
bgp {
group "CSC-CE" {
type internal
}
neighbor "192.0.2.4" {
group "CSC-CE"
family {
vpn-ipv4 true
}
}
The configuration of the BGP session on CSC-CE-4 is similar, as follows:
# on CSC-CE-4:
configure {
router "Base" {
bgp {
group "CSC-CE" {
type internal
}
neighbor "192.0.2.1" {
group "CSC-CE"
family {
vpn-ipv4 true
}
}
The configuration of the BGP session between CSC-CE-1 and CSC-CE-4 has the following properties:
-
The peer type is IBGP (type internal. Alternatively, peer-as can be configured with the same value as the locally configured autonomous-system).
-
The transport for the IBGP session is IPv4 (the neighbor address is an IPv4 address).
-
The family vpn-ipv4 command causes MP-BGP negotiation of the address family for AFI=1 and SAFI=128.
Step 11. Verify exchange of VPN-IPv4 routes
When the preceding steps have been completed, CSC-PE-3 can advertise a VPN-IPv4 route for some IP prefix (for example, 10.11.30.0/24) to CSC-CE-4. This can be checked in the RIB In of CSC-CE-4 as follows:
[/]
A:admin@CSC-CE-4# show router bgp routes 10.11.30.0/24 vpn-ipv4 hunt
===============================================================================
BGP Router ID:192.0.2.4 AS:64496 Local AS:64496
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP VPN-IPv4 Routes
===============================================================================
-------------------------------------------------------------------------------
RIB In Entries
-------------------------------------------------------------------------------
Network : 10.11.30.0/24
Nexthop : 192.0.2.1
Route Dist. : 64496:11 VPN Label : 524286
Path Id : None
From : 192.0.2.1
Res. Nexthop : n/a
Local Pref. : 100 Interface Name : NotAvailable
Aggregator AS : None Aggregator : None
Atomic Aggr. : Not Atomic MED : None
AIGP Metric : None IGP Cost : 0
Connector : None
Community : target:64496:1
Cluster : No Cluster Members
Originator Id : None Peer Router Id : 192.0.2.1
Fwd Class : None Priority : None
Flags : Used Valid Best IGP
Route Source : Internal
AS-Path : No As-Path
Route Tag : 0
Neighbor-AS : n/a
Orig Validation: N/A
Source Class : 0 Dest Class : 0
Add Paths Send : Default
Last Modified : 00h02m52s
VPRN Imported : 1
-------------------------------------------------------------------------------
RIB Out Entries
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
The following command can be used to check that CSC-CE-4 has properly installed the preceding VPN-IPv4 route into the routing table of the importing VPRN service:
[/]
A:admin@CSC-CE-4# show router 1 route-table
===============================================================================
Route Table (Service: 1)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
10.11.30.0/24 Remote BGP VPN 00h03m33s 170
192.0.2.1 (tunneled:BGP) 1000
10.14.30.0/24 Local Local 00h07m09s 0
loopback-1 0
-------------------------------------------------------------------------------
No. of Routes: 2
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
Conclusion
Carrier Supporting Carrier is a scalable and secure solution for using an infrastructure IP VPN to transport traffic between dispersed CSC-CE devices belonging to an ISP or other service provider. Many different topology models are supported by SR OS. This chapter has explored one simplified configuration that can serve as the basis for more complicated setups.