Inter-domain VPN services
This chapter describes the different inter-domain VPN services on SR Linux.
Multi-instance IP-VRF
Multi-instance IP-VRF refers to an IP-VRF network instance that supports multiple instances of BGP-VPN (a generalization of BGP families that are used for VPN services, EVPN, or IP-VPN). Multi-instance IP-VRF is configured using the command network-instance protocols bgp-vpn bgp-instance.
Multi-instance IP-VRFs are used on service gateway PE routers that stitch multiple domains together, in accordance with draft-ietf-bess-evpn-ipvpn-interworking. These domains can use different BGP Layer 3 families or different tunnels in the data path.
The following figure illustrates PE1 and PE2 as service gateways that connect an EVPN-IFL domain to an IP-VPN domain. PE1 and PE2 use a single IP-VRF with two BGP-VPN instances.
--[ ]--
A:pe1# info with-context network-instance IP-VRF1
network-instance IP-VRF1 {
type ip-vrf
interface lo0.1 {
}
protocols {
bgp-evpn {
bgp-instance 2 {
dpath-domain-id 64500:2
encapsulation-type mpls
evi 10
mpls {
next-hop-resolution {
allowed-tunnel-types [ bgp te-policy-sr-policy-mpls-uncolored ]
}
}
}
}
bgp-ipvpn {
bgp-instance 1 {
dpath-domain-id 64500:2
mpls {
next-hop-resolution {
allowed-tunnel-types [ ldp sr-isis ]
}
}
}
}
bgp-vpn {
bgp-instance 1 {
route-distinguisher { rd 1:1 }
route-target {
export-rt target:64500:10
import-rt target:64500:10
}
}
bgp-instance 2 {}
}
}
}--[ ]--
A:pe2# info with-context network-instance IP-VRF1
network-instance IP-VRF1 {
type ip-vrf
interface lo0.1 {}
protocols {
bgp-evpn {
bgp-instance 2 {
dpath-domain-id 64500:2
encapsulation-type mpls
evi 10
mpls {
next-hop-resolution {
allowed-tunnel-types [ bgp te-policy-sr-policy-mpls-uncolored ]
}
}
}
}
bgp-ipvpn {
bgp-instance 1 {
dpath-domain-id 64500:2
mpls {
next-hop-resolution {
allowed-tunnel-types [ ldp sr-isis ]
}
}
}
}
bgp-vpn {
bgp-instance 1 {
route-distinguisher { rd 2:2 }
route-target {
export-rt target:64500:10
import-rt target:64500:10
}
}
bgp-instance 2 {}
}
}
}Up to two BGP instances are supported in the same network instance of type ip-vrf. One BGP instance is used for EVPN-IFL and the other BGP instance is used for IP-VPN.
If the same route is received on the same IP-VRF via different BGP owners, such as IP-VPN, EVPN-IFL, or BGP PE-CE, the selection and combined ECMP set is completed based on the best path selection for EVPN IFL routes. For more information, see Best path selection for EVPN IFL routes.
D-PATH is configured on each BGP instance to prevent control plane loops. For more information, see Domain path (D-PATH) attribute for inter-domain connectivity.
Multi-instance MAC-VRF
Multi-instance MAC-VRF refers to a MAC-VRF network instance that supports multiple instances of BGP-EVPN. Multi-instance MAC-VRF is configured using the network-instance protocols bgp-evpn bgp-instance command. Multi-instance MAC-VRFs are deployed on service gateways that interconnect broadcast domains across two different EVPN domains, as specified in RFC 9014. Each domain can operate its own IGP instance, BGP Autonomous System, or transport tunnel type.
Only a single BGP-EVPN encapsulation is supported per network instance. Only the combination of BGP-EVPN VXLAN and BGP-EVPN MPLS in the same MAC-VRF is allowed. When two BGP-EVPN instances are enabled in a MAC-VRF, no subinterfaces can be configured on the same MAC-VRF.
The following figure illustrates the multi-instance MAC-VRF structure in two service gateways - GW1 and GW2 - that interconnect an EVPN-VXLAN domain in the data center to an EVPN-MPLS domain in the WAN.
- EVPN A-D per ES and per EVI routes (type 1 routes) – consumed but not propagated to the other instance
- EVPN MAC/IP Advertisement routes (type 2 routes) – consumed and - if programmed in the MAC table - propagated to the other instance
- EVPN Inclusive Multicast Ethernet Tag routes (type 3 routes) – consumed but not propagated to the other instance
- EVPN Ethernet Segment routes (type 4 routes) – not imported because the MAC-VRF does not support local Ethernet Segments
- Only the best MAC/IP route is redistributed for a specified MAC and IP.
- In the case that multiple MAC/IP routes are received for the same MAC but with different IP addresses, all routes are redistributed to the other instance.
- EVPN MAC mobility and ARP/ND extended communities received on MAC/IP advertisement routes are propagated unchanged to the other instance.
The following considerations apply to multi-instance MAC-VRFs:
- As per RFC 9014, the configuration of the vlan-aware-bundle-eth-tag ethernet-tag-id command needs to be the same across the two instances.
- Multi-instance MAC-VRFs do not support proxy-ARP/ND or multicast snooping protocols.
- When two instances are configured in a MAC-VRF, one is configured with the bgp-evpn bgp-instance multi-homing-mode access command and the other instance with the bgp-evpn bgp-instance multi-homing-mode network (default value) command.
The following example shows the configuration of the MAC-VRF network instance BD2.
--{ * candidate shared default }--[ ]--
A:admin@gw1# info with-context
network-instance BD2 {
type mac-vrf
vxlan-interface vxlan0.1 {
}
protocols {
bgp-evpn {
bgp-instance 1 {
admin-state enable
vxlan-interface vxlan0.1
evi 1
ecmp 2
multi-homing-mode access
internal-tags {
set-tag-set [
dc
]
}
routes {
bridge-table {
inclusive-mcast {
originating-ip 11.12.11.12
}
}
}
}
bgp-instance 2 {
encapsulation-type mpls
evi 2
ecmp 2
mpls {
next-hop-resolution {
allowed-tunnel-types [
ldp
sr-isis
]
}
}
routes {
bridge-table {
inclusive-mcast {
originating-ip 11.12.11.12
}
}
}
}
}
bgp-vpn {
bgp-instance 1 {
route-distinguisher {
rd 100.11.12.1:1
}
route-target {
export-rt target:64500:1
import-rt target:64500:1
}
}
bgp-instance 2 {
route-distinguisher {
rd 100.11.12.2:1
}
route-target {
export-rt target:64500:1
import-rt target:64500:1
}
}
}
}
}Multi-instance gateway redundancy
- The EVPN-VXLAN instance must use the same route-distinguisher and originating-ip on all redundant gateways.
- The EVPN-MPLS instance must use the same route-distinguisher and originating-ip on all the gateways.
Remote leaf nodes receive equivalent IMET and MAC/IP routes from all redundant gateways, and BGP selects routes from only one of the gateways.
In addition, redundant gateways receive each other's IMET and MAC/IP advertisement routes. Gateway IMET routes are received on a gateway with an originating-ip and route-distinguisher matching the local originating-ip and route-distinguisher values, so EVPN multicast destinations are not created among the redundant gateways. BGP policies are needed to avoid control plane loops for MAC/IP Advertisement routes. These policies prevent gateways from redistributing MAC/IP Advertisement routes that are received from another gateway.
// dc1gw1 configuration
--{ candidate shared default }--[ network-instance BD1 ]--
A:admin@dc1gw1# info with-context
network-instance BD1 {
type mac-vrf
vxlan-interface vxlan0.1 {
}
protocols {
bgp-evpn {
bgp-instance 1 {
admin-state enable
vxlan-interface vxlan0.1
evi 1
ecmp 2
multi-homing-mode access
internal-tags {
set-tag-set [
dc
]
}
routes {
bridge-table {
inclusive-mcast {
originating-ip 11.12.11.12
}
}
}
}
bgp-instance 2 {
encapsulation-type mpls
evi 2
ecmp 2
mpls {
next-hop-resolution {
allowed-tunnel-types [
ldp
sr-isis
]
}
}
routes {
bridge-table {
inclusive-mcast {
originating-ip 11.12.11.12
}
}
}
}
}
bgp-vpn {
bgp-instance 1 {
route-distinguisher {
rd 100.11.12.1:1
}
route-target {
export-rt target:64500:1
import-rt target:64500:1
}
}
bgp-instance 2 {
route-distinguisher {
rd 100.11.12.2:1
}
route-target {
export-rt target:64500:1
import-rt target:64500:1
}
}
}
}
}
// dc1gw2 configuration
--{ candidate shared default }--[ network-instance BD1 ]--
A:admin@dc1gw2# info with-context
network-instance BD1 {
type mac-vrf
admin-state enable
vxlan-interface vxlan0.1 {
}
protocols {
bgp-evpn {
bgp-instance 1 {
admin-state enable
vxlan-interface vxlan0.1
evi 1
ecmp 2
multi-homing-mode access
internal-tags {
set-tag-set [
dc
]
}
routes {
bridge-table {
inclusive-mcast {
originating-ip 11.12.11.12
}
}
}
}
bgp-instance 2 {
encapsulation-type mpls
evi 2
ecmp 2
mpls {
next-hop-resolution {
allowed-tunnel-types [
ldp
sr-isis
]
}
}
routes {
bridge-table {
inclusive-mcast {
originating-ip 11.12.11.12
}
}
}
}
}
bgp-vpn {
bgp-instance 1 {
route-distinguisher {
rd 100.11.12.1:1
}
route-target {
export-rt target:64500:1
import-rt target:64500:1
}
}
bgp-instance 2 {
route-distinguisher {
rd 100.11.12.2:1
}
route-target {
export-rt target:64500:1
import-rt target:64500:1
}
}
}
}
}
// Routing policies to prevent loops (same configuration on both gateways)
--{ candidate shared default }--[ network-instance default protocols bgp neighbor * ]--
A:admin@dc1gw2# info
neighbor 100.1.0.1 { // peer in EVPN-VXLAN domain
peer-as 64500
peer-group overlay
export-policy [
allow-only-vxlan-and-add-SOO-DC1
]
import-policy [
drop-SOO-DC1
]
}
neighbor 100.1.0.11 { // peer in EVPN-MPLS domain
peer-as 64500
peer-group overlay
export-policy [
allow-only-mpls-and-add-SOO-DC1
]
import-policy [
drop-SOO-DC1
]
}
--{ candidate shared default }--[ routing-policy ]--
A:admin@dc1gw2# info with-context
routing-policy {
extended-community-set SOO-DC1 {
member [
origin:1:1
]
}
extended-community-set bgp-tunnel-encap:MPLS {
member [
bgp-tunnel-encap:MPLS
]
}
extended-community-set bgp-tunnel-encap:VXLAN {
member [
bgp-tunnel-encap:VXLAN
]
}
tag-set dc {
tag-value [
1
]
}
tag-set wan {
tag-value [
2
]
}
policy allow-only-mpls-and-add-SOO-DC1 {
statement 10 {
match {
family [
evpn
]
internal-tags {
tag-set [
dc
]
}
bgp {
extended-community {
extended-community-set bgp-tunnel-encap:VXLAN
}
}
}
action {
policy-result reject
}
}
statement 20 {
match {
family [
evpn
]
}
action {
policy-result accept
bgp {
extended-community {
operation add
referenced-sets [
SOO-DC1
]
}
}
}
}
}
policy allow-only-vxlan-and-add-SOO-DC1 {
statement 10 {
match {
family [
evpn
]
bgp {
extended-community {
extended-community-set bgp-tunnel-encap:MPLS
}
}
}
action {
policy-result reject
}
}
statement 20 {
match {
family [
evpn
]
}
action {
policy-result accept
bgp {
extended-community {
operation add
referenced-sets [
SOO-DC1
]
}
}
}
}
}
policy drop-SOO-DC1 {
statement 10 {
match {
family [
evpn
]
bgp {
extended-community {
extended-community-set SOO-DC1
}
}
}
action {
policy-result reject
}
}
}
}Source VTEP security in anycast redundant gateway scenarios
The VXLAN implementation on Broadcom chipsets requires the source VTEP IP address to be matched in ingress tables before a VXLAN packet is terminated. As a result, a node must know the source VTEP IP address to successfully terminate incoming VXLAN traffic.
In a single-instance EVPN VXLAN network, this requirement is normally satisfied because all VXLAN VTEPs are learned through IMET routes, which are exchanged among all nodes before traffic is forwarded. However, in a multi-instance EVPN VXLAN deployment with multiple redundant gateways, a leaf node may never learn the VTEP IP address of one of the anycast gateways, even though that gateway may still send traffic to it. This is a result of configuring the same route-distinguisher and originating-ip in all the redundant gateways for a specific MAC-VRF.
In gateway redundancy designs that include Layer 3 services, this issue does not typically arise, because Layer 3 services do not rely on configuring the same route-distinguisher and originating-ip in the same way, and leaf nodes learn all source VTEPs.
However, if the gateways provide only Layer 2 services, leaf nodes may learn the VTEP of only one of the gateways in the redundancy group. In this case, VXLAN packets sourced from the other gateway are not terminated.
- uses different RDs on each gateway
- does not carry real traffic
- exists to ensure that IMET routes are exchanged, allowing leaf nodes to learn all source VTEPs of the anycast gateways
When the above configuration is implemented in the redundant service gateways, all leaf nodes learn the service gateway VTEPs, and traffic is forwarded normally.
Inter-domain EVPN IFL and IP-VPN in a single IP-VRF BGP instance
This inter-domain EVPN IFL/IP-VPN solution is applicable to IP-VRF network instances that use a single BGP instance (enabled through either BGP-EVPN or BGP IP-VPN). This solution can be used to interconnect two domains seamlessly through a gateway.
Gateways provide IGP isolation between domains so that nodes in each domain do not need end-to-end visibility of all loopbacks in the network; they only need tunnels to the gateways. In the overlay layer, gateways provide prefix aggregation or advertise a single default route to the access domain to contain the number of routes that the access nodes need to program in their forwarding information bases (FIBs).
The following figure illustrates an example of how this inter-domain solution for Layer 3 services can be applied. In this scenario, there are two domains: an EVPN-VXLAN access domain and an EVPN-VXLAN core domain. Each domain runs its own IGP instance and loopback IP addresses are not leaked between them. This minimizes the number of VXLAN tunnels required on the access routers. Gateways GW1 and GW2 are configured with an IP-VRF that has EVPN-VXLAN IFL enabled, allowing them to redistribute EVPN IP Prefix routes between the two domains.
--{ * candidate shared default }--[ network-instance IP-VRF ]--
A:root@srl1# info with-context
network-instance IP-VRF {
type ip-vrf
vxlan-interface vxlan0.1 {
}
protocols {
bgp-evpn {
bgp-instance 1 {
vxlan-interface vxlan0.1
}
}
bgp-vpn {
allow-export {
}
bgp-instance 1 {
}
}
}
}
The configuration of allow-export is only supported in IP-VRF network instances and provides the following functionalities:
-
Received EVPN IFL or IP-VPN routes are re-exported to the configured BGP peers. In IP-VRF network instances that contain both BGP-EVPN and BGP-IPVPN instances, EVPN IFL or IP-VPN routes are automatically re-exported to the BGP-IPVPN instance without the allow-export command.
-
Received EVPN IFL routes are imported and re-advertised using the local RD/RT, next-hop, or mac-nh.
-
Local label or VNI is advertised with the routes, which preserves the VNI in VXLAN routes end-to-end.
-
Non-EVPN BGP attributes are propagated.
-
In EVPN IFL, the preceding functionality is supported regardless of the encapsulation in the network instance (VXLAN or MPLS).
Domain path (D-PATH) attribute for inter-domain connectivity
The BGP domain path (D-PATH) attribute is required in networks that expand multiple BGP, IP-VPN, or EVPN domains, as defined in draft-ietf-bess-evpn-ipvpn-interworking.
Similar to AS_PATH, D-PATH is composed of a sequence of domain segments. The following figure graphically represents each domain segment.
- Each domain segment is comprised of the domain_segment_length, domain_segment_value, where the domain segment value is a sequence of one or more domains.
- Each domain is represented by DOMAIN-ID:ISF_SAFI_TYPE, where the newly added domain, which is added by a gateway (GW), is always prepended to the left of the last existing last.
- The supported ISF_SAFI_TYPE values are:
- 0 = Local ISF route
- 70 = evpn
- 128 = safi 128 (IP-VPN domains)
- Labeled unicast IP or BGP PE-CE routes do not support D-PATH.
- The D-PATH attribute is only modified by a GW and not by an ABR/ASBR or RR. A GW is defined as a PE where an IP-VRF is instantiated, and that IP-VRF advertises or receives routes from multiple BGP owners (for example, EVPN IFL and BGP IP-VPN), multiple instances of the same owner (for example, IP-VRF with two BGP IP-VPN instances), or local routes.
+----------+------------+
|Seg Len=1 | 65000:1:128|
+----------+------------++----------+----------+-----------+
|Seg Len=2 |65000:2:70|65000:1:128|
+----------+----------+-----------++----------+----------+-----------+
|Seg Len=2 |65000:3:70|65000:1:128|
+----------+----------+-----------++----------+------------+
|Seg Len=1 | 65000:1:128|
+----------+------------+D-PATH configuration
In order for a router to add a D-PATH attribute or a new segment to the existing D-PATH, a domain-id needs to be configured. The D-PATH is modified on transmission or processed on reception based on the local network instance configuration. The domain-id is configured per BGP instance and the ISF_SAFI_TYPE automatically derived from the instance type that imported the original route. The domain-id is configured at the service BGP instance level as a 6 byte value that includes a global admin value and a local admin value, for example 65000:1.
-
IP-VRF BGP-EVPN MPLS and VXLAN instances (EVPN-IFL):
network-instance protocols bgp-evpn bgp-instance dpath-domain-id global-field:local-field
-
IP-VRF BGP IP-VPN MPLS:
network-instance protocols bgp-ipvpn bgp-instance dpath-domain-id global-field:local-field
-
IP-VRF network instance level for local routes (direct, static, IGP routes):
network-instance protocols bgp-vpn local-routes-dpath-domain-id global-field:local-field
EVPN IFF routes are considered local routes for the purpose of adding D-PATH when readvertising as an IP-VPN/EVPN-IFL route. The domain-id for local routes is configured under BGP-VPN because it only applies to VPN routes.
-
The value of a domain-id cannot be 0:0.
-
If local-routes-dpath-domain-id is not configured, the local routes are advertised into the BGP instances with no D-PATH.
-
If a network instance BGP instance is not configured with a domain-id:
-
Routes imported in that instance are re-advertised in a different instance without modifying the D-PATH.
-
Routes exported in that instance are advertised with the D-PATH modified to include the domain-id of the instance that imported the route in the first place.
-
-
There is no check for the uniqueness of the domain-id within the network instance or across network instances.
-
Modifying the domain-id list initiates a route-refresh for all the address families associated with the network instance.
D-PATH and loop detection
When an EVPN IFL or IP-VPN route with a D-PATH containing a locally-configured
domain-id on the IP-VRF is received where the route is
imported, the route becomes inactive. Additionally, the rib-in-out rib
in-post/local state of the route shows invalid-reason
domain-path-loop true. The local RIB, rib-in-pre,
and rib-in-post shows a list of network instances that detected a loop
caused by the presence of a local domain-id in the imported
route.
It is possible that a route has a looped domain for one IP-VRF and not for another. BGP installs the route in the IP-VRF that does not have loop and does not install the route in the IP-VRF that has the loop.
The invalid-reason domain-path-loop true output is exposed for routes that are not imported in any IP-VRF caused by matching the local domain-ids in the importing IP-VRFs.
D-PATH and best path selection
D-PATH is considered for BGP best path selection, as per draft-ietf-bess-evpn-ipvpn-interworking. In SR Linux, the best path selection process includes D-PATH in the tie-breaker sequence, as shown in the following list:
- Valid route preferred over invalid route
- Lowest route preference
- Highest Local Pref
- Shortest D-PATH
- Shortest AS Path
- Lowest Origin
- Lowest MED
- EBGP preferred over IBGP
- Lowest next-hop cost (break off for ECMP paths)
- Lowest next-hop type (TTM resolution wins over route table resolution)
- Route owner comparison (BGP preferred over BGP-label preferred over BGP-VPN)
- Lowest peer router ID
- Lowest peer IP address
- Route owner comparison (BGP EVPN IFL host preferred over BGP EVPN preferred over BGP IP-VPN)
As D-PATH is introduced in networks, not all the PEs will support D-PATH for BGP path selection. To guarantee compatibility in brown field scenarios, the command network instance(default) protocols bgp best-path-selection ignore-domain-path-length <boolean> is introduced, where:
- The command is set to false by default and the shorter D-PATH length is considered a tie breaker rule in the best path selection criteria for received routes with the same or different route distinguisher. The D-PATH length tiebreaker rule is compared immediately after the Local Preference comparison.
- When the command is set to true, the D-PATH is ignored for the purpose of best path selection.
- A tie-break reason in the BGP RIB model (domain-path-length) is displayed when the route is not active because of the D-PATH length.
Next-hop-self route reflector and inter-AS option B
Next-hop-self route reflectors (NHS-RR) are used in BGP networks to reduce the need for fully meshed iBGP connections within a single autonomous system (AS). In a fully meshed AS, iBGP routers do not advertise routes to their neighbors, while NHS-RR allows a route reflector (RR) to advertise learned iBGP routes to iBGP neighbors with its own address as the next hop. This feature connects different domains within the same AS and improves the overall scale.
Inter-AS option B is a method for interconnecting VPN sites located in different ASes. Using this method, autonomous system border routers (ASBRs) are directly connected and routes are exchanged on a single interface.
EVPN IFL and VPN-IPv4/IPv6 routes both support NHS-RR capability and inter-AS option B.
The preceding figure shows two different ASes that are connected via inter-AS option B ASBRs br4, br5, and br6. This method allows for the extension of EVPN and IP-VPN services across different MPLS or segement routing MPLS domains without providing services on border routers.
Egress PEs advertise EVPN and IP-VPN routes to adjacent border routers.
- Import and redistribute routes to the remote border routers or local PEs where the addresses of the remote border routers are used as the next hops and their own service MPLS labels are used.
- Program a label swap operation so that the ingress traffic service label is looked up and packets are forwarded with a new service label.
--{ + candidate shared default }--[ network-instance default ]--
A:pe1# info
type default
interface ethernet-1/10.0 {
}
interface system0.0 {
}
protocols {
bgp {
admin-state enable
autonomous-system 65001
router-id 10.0.0.1
bgp-label {
bgp-ipvpn {
next-hop-resolution {
ipv4-next-hops {
tunnel-resolution {
allowed-tunnel-types [
ldp
sr-isis
]
}
}
}
}
}
isis {
dynamic-label-block range-3-srgb
instance i14 {
admin-state enable
instance-id 1
level-capability L2
iid-tlv true
net [
49.0001.0000.0000.0001.00
]
trace-options {
trace [
adjacencies
interfaces
packets-all
]
}
segment-routing {
mpls {
dynamic-adjacency-sids {
all-interfaces true
}
}
}
interface ethernet-1/10.0 {
circuit-type point-to-point
ipv4-unicast {
admin-state enable
}
}
interface system0.0 {
passive true
ipv4-unicast {
admin-state enable
}
}
}
}
}
segment-routing {
mpls {
global-block {
label-range range-2-srgb
}
local-prefix-sid 1 {
interface system0.0
ipv4-label-index 1
}
}
}
}--{ + candidate shared default }--[ network-instance default protocols bgp ]--
A:pe2# info
admin-state enable
autonomous-system 65023
router-id 10.0.0.2
bgp-label {
bgp-ipvpn {
next-hop-resolution {
ipv4-next-hops {
tunnel-resolution {
allowed-tunnel-types [
ldp
sr-isis
]
}
}
ebgp-default-policy {
import-reject-all false
export-reject-all false
}
afi-safi evpn {
admin-state enable
evpn {
keep-all-routes true
rapid-update true
}
}
afi-safi l3vpn-ipv4-unicast {
admin-state enable
l3vpn-ipv4-unicast {
keep-all-routes true
rapid-update true
}
}
group overlay-ibgp {
peer-as 65023
afi-safi evpn {
}
afi-safi l3vpn-ipv4-unicast {
}
route-reflector {
client true
cluster-id 2.2.2.2
}
timers {
connect-retry 1
minimum-advertisement-interval 1
}
trace-options {
flag update {
modifier detail
}
}
}
neighbor 10.0.0.3 {
peer-group overlay-ibgp
}
neighbor 10.0.0.5 {
peer-group overlay-ibgp
}
neighbor 10.0.0.6 {
peer-group overlay-ibgp
}
}
}
}- inter-as-vpn
- next-hop-self-route-reflector
These commands affect all EVPN routes and trigger bgp_mgr to swap the
service label for all EVPN MPLS routes. These commands also change the next hop to self
in all routes with MPLS encapsulation.
For border router configuration, IGP is not enabled in interfaces to other border routers.
The inter-as-vpn true command allows received EVPN/IP-VPN routes to be retained in the BGP RIB and propagated to any eBGP or iBGP peer. To ensure label allocation, a dynamic label block must be configured for border routers. Label allocation re-advertises a received route into the adjacent AS with a local network instance MPLS label and ensures an MPLS label swap operation is completed.
--{ * candidate shared default }--[ ]--
A:srl1# info with-context network-instance default
network-instance default {
type default
protocols {
bgp {
bgp-label {
bgp-vpn {
dynamic-label-block range-6-bgp-lu
}
}The label block is shared by the EVPN inter-AS model B and EVPN NHS-RR features.
The inter-as-vpn true command has the same function as the keep-all-routes command for keeping the routes in the RIB.
--{ + candidate shared default }--[ ]--
A:br4# info with-context network-instance default
network-instance default {
type default
interface ethernet-1/10.0 {
}
interface ethernet-1/11.0 {
}
interface ethernet-1/12.0 {
}
interface system0.0 {
}
protocols {
bgp {
autonomous-system 65001
router-id 10.0.0.4
bgp-label {
bgp-vpn {
dynamic-label-block range-6-bgp-lu
}
bgp-ipvpn {
next-hop-resolution {
ipv4-next-hops {
tunnel-resolution {
allowed-tunnel-types [
ldp
sr-isis
]
}
}
}
}
}
ebgp-default-policy {
import-reject-all false
export-reject-all false
}
afi-safi evpn {
admin-state enable
evpn {
inter-as-vpn true
rapid-update true
default-received-encapsulation mpls
next-hop-resolution {
ipv4-next-hops {
tunnel-resolution {
allowed-tunnel-types [
bgp
ldp
sr-isis
]
}
}
}
}
}
afi-safi l3vpn-ipv4-unicast {
admin-state enable
l3vpn-ipv4-unicast {
inter-as-vpn true
rapid-update true
}
}
group overlay-ebgp {
peer-as 65023
afi-safi evpn {
}
timers {
connect-retry 1
minimum-advertisement-interval 1
}
trace-options {
flag update {
modifier detail
}
}
}
group overlay-ibgp {
peer-as 65001
afi-safi evpn {
}
timers {
connect-retry 1
minimum-advertisement-interval 1
}
trace-options {
flag update {
modifier detail
}
}
}
neighbor 10.4.5.2 {
peer-group overlay-ebgp
}
neighbor 10.4.6.2 {
peer-group overlay-ebgp
}
neighbor 10.0.0.1 {
peer-group overlay-ibgp
}
}
ldp {
admin-state enable
dynamic-label-block range-1-ldp
discovery {
interfaces {
interface ethernet-1/10.0 {
ipv4 {
admin-state enable
}
}
interface ethernet-1/11.0 {
ipv4 {
admin-state enable
}
}
interface ethernet-1/12.0 {
ipv4 {
admin-state enable
}
}
}
}
}
isis {
dynamic-label-block range-3-srgb
instance i14 {
admin-state enable
instance-id 1
level-capability L2
iid-tlv true
net [
49.0001.0000.0000.0004.00
]
segment-routing {
mpls {
dynamic-adjacency-sids {
all-interfaces true
}
}
}
interface ethernet-1/10.0 {
circuit-type point-to-point
ipv4-unicast {
admin-state enable
}
}
interface system0.0 {
passive true
ipv4-unicast {
admin-state enable
}
}
}
}
}
segment-routing {
mpls {
global-block {
label-range range-2-srgb
}
local-prefix-sid 1 {
interface system0.0
ipv4-label-index 4
}
}
}
}--{ + candidate shared default }--[ ]--
A:br4# info with-context system mpls
system {
mpls {
label-ranges {
static range-2-srgb {
shared true
start-label 100001
end-label 120000
}
static range-5-static-services {
shared false
start-label 3000
end-label 4000
}
dynamic range-1-ldp {
start-label 100
end-label 200
}
dynamic range-3-srgb {
start-label 120001
end-label 120999
}
dynamic range-4-evpn {
start-label 500
end-label 699
}
dynamic range-5-services {
start-label 1000
end-label 2000
}
dynamic range-6-bgp-lu {
start-label 122001
end-label 122201
}
}
services {
evpn {
dynamic-label-block range-4-evpn
}
network-instance {
dynamic-label-block range-5-services
}
}
}
}The next-hop-self-route-reflector command requires the configuration of a border router as a RR. The behavior of this command is equivalent to the inter-as-vpn command, with one difference: the use of next-hop-self-route-reflector allows the border router to receive and readvertise routes to RR clients within the same AS.
BGP next-hop resolution for EVPN/IP-VPN routes
- inter-as-vpn
- nhsrr-evpn
- resolution of tunnels through a fallback to FIB lookup
- resolution of local and static routes to the next hop in the absence of resolving tunnels
- For
IP-VPN:
--{ * candidate shared default }--[ ]-- A:srl1# info with-context network-instance default network-instance default { protocols { bgp { bgp-label { bgp-ipvpn { next-hop-resolution { } } } } } - For
EVPN:
--{ * candidate shared default }--[ ]-- A:srl3# info with-context network-instance default network-instance default { protocols { bgp { afi-safi evpn { next-hop-resolution { } } } } }
The next-hop-resolution configuration on the ASBR on the default network instance affects EVPN-MPLS routes but not EVPN-VXLAN routes. EVPN-VXLAN routes also ignore everything under the next-hop-resolution context.
The next-hop-resolution tunnel-resolution allowed-tunnel-types leaf restricts VXLAN and only allows MPLS tunnels.
- If the EVPN route type is an ES route (route type 4) then the route is resolved to any route in the default network instance route table, regardless of encapsulation type or node type.
- If the EVPN route type is different from an ES route, the following logic is
followed:
- If the route has an encapsulation type of VXLAN or if there is no encapsulation type found and default-received-encapsulation is set to VXLAN then the route is resolved over any RTM route.
- If the route has an encapsulation type of MPLS or default-received-encapsulation is set to MPLS then the resolution is selected based on the default network instance ASBR or service configuration.
Displaying next-hop-self route reflector and inter-AS option B information
You can display the service Incoming Label Mapping (ILM) information and the next hop resolution using the info from state command.
Swapped service labels
The following example displays the swapped labels using the info from state command. The BGP-RIB provides the state information for each imported and exported route.
// example for EVPN IFL route on a model B ASBR:
--{ candidate shared default }--[ ]--
A:br4# info from state with-context network-instance default bgp-rib evpn rib-in-out rib-in-post ip-prefix-routes 10.0.0.2:3 ethernet-tag-id 0 ip-prefix-length 24 ip-prefix 10.20.20.
0/24 neighbor 10.4.5.2
network-instance default {
bgp-rib {
evpn {
rib-in-out {
rib-in-post {
ip-prefix-routes 10.0.0.2:3 ethernet-tag-id 0 ip-prefix-length 24 ip-prefix 10.20.20.0/24 neighbor 10.4.5.2 {
esi 00:00:00:00:00:00:00:00:00:00
gateway-ip 0.0.0.0
attr-id 125
last-modified "2024-04-10T12:46:50.200Z (2 hours ago)"
used-route false
valid-route true
best-route true
stale-route false
pending-delete false
tie-break-reason none
label {
value 122010 // received service label
value-type mpls-label
}
invalid-reason {
rejected-route false
as-loop false
next-hop-unresolved false
cluster-loop false
label-allocation-failed false
fib-programming-failed false
}
}
}
}
}
}
}
--{ candidate shared default }--[ ]--
A:br4# info from state with-context network-instance default bgp-rib evpn rib-in-out rib-out-post ip-prefix-routes 10.0.0.2:3 ethernet-tag-id 0 ip-prefix-length 24 ip-prefix 10.20.20
.0/24 neighbor 10.0.0.1
network-instance default {
bgp-rib {
evpn {
rib-in-out {
rib-out-post {
ip-prefix-routes 10.0.0.2:3 ethernet-tag-id 0 ip-prefix-length 24 ip-prefix 10.20.20.0/24 neighbor 10.0.0.1 {
esi 00:00:00:00:00:00:00:00:00:00
gateway-ip 0.0.0.0
attr-id 147
next-hop 10.0.0.4
label {
value 122007 // advertised label, 122010 is swapped to 122007
value-type mpls-label
}
}
}
}
}
}
}Next hop state in VRF for Inter-AS Option B
The swapped label value is used to find the next hop resolution.
// The service label entry 122007 provides the NHG and next hop
--{ candidate shared default }--[ ]--
A:br4# info from state with-context network-instance default route-table mpls label-entry 122007
network-instance default {
route-table {
mpls {
label-entry 122007 {
operation swap
entry-type bgp
last-app-update "2024-04-10T12:46:50.185Z (2 hours ago)"
next-hop-group 373030816119
}
}
}
}
--{ candidate shared default }--[ ]--
A:br4# info from state with-context network-instance default route-table next-hop-group 373030816119
network-instance default {
route-table {
next-hop-group 373030816119 {
backup-next-hop-group 0
fib-programming {
last-successful-operation-type add
last-successful-operation-timestamp 2024-04-10T11:57:16.328Z
pending-operation-type none
last-failed-operation-type none
}
next-hop 0 {
next-hop 373030816115
resolved true
}
}
}
}
// In this example, the next hop is resolved to a local route 10.4.5.0 (ASBRs using single hop eBGP session)
--{ candidate shared default }--[ ]--
A:br4# info from state with-context network-instance default route-table next-hop 373030816115
network-instance default {
route-table {
next-hop 373030816115 {
type indirect
ip-address 10.4.5.2
resolving-route {
ip-prefix 10.4.5.0/30
route-type local
route-owner net_inst_mgr
}
mpls {
pushed-mpls-label-stack [
122010
]
}
}
}
}MPLS route table
The following example displays the the MPLS route table which summarizes the swap operation, incoming label, outgoing label, and associated next hop.
--{ candidate shared default }--[ ]--
A:br4# show network-instance default route-table mpls
+--------+-----------+---------+-----------+----------------------+-----------------+---------------+
| Label | Operation | Type | Next | Next-hop IP | Next-hop | Next-hop MPLS |
| | | | Net-Inst | (Type) | Subinterface | labels |
+========+===========+=========+===========+======================+=================+===============+
| 100 | POP | ldp | default | | | |
| 100002 | SWAP | sr-mpls | N/A | 10.1.4.1 (mpls) | ethernet-1/10.0 | 100002 |
| 100005 | POP | sr-mpls | default | | | |
| 120002 | SWAP | sr-mpls | N/A | 10.1.4.1 (mpls) | ethernet-1/10.0 | IMPLICIT_NULL |
| 122001 | SWAP | bgp | N/A | 10.0.0.1 (indirect) | | 1002 |
| 122002 | SWAP | bgp | N/A | 10.0.0.1 (indirect | | 1001 |
| 122003 | SWAP | bgp | N/A | 10.0.0.1 (indirect) | | 1003 |
| 122004 | SWAP | bgp | N/A | 10.4.5.2 (indirect) | | 122008 |
| 122005 | SWAP | bgp | N/A | 10.4.5.2 (indirect) | | 122011 |
| 122006 | SWAP | bgp | N/A | 10.4.5.2 (indirect) | | 122007 |
| 122007 | SWAP | bgp | N/A | 10.4.5.2 (indirect) | | 122010 |
| 122008 | SWAP | bgp | N/A | 10.4.5.2 (indirect) | | 122014 |
| 122009 | SWAP | bgp | N/A | 10.4.5.2 (indirect) | | 122009 |
| 122010 | SWAP | bgp | N/A | 10.4.5.2 (indirect) | | 122012 |
| 122011 | SWAP | bgp | N/A | 10.4.5.2 (indirect) | | 122013 |
+--------+-----------+---------+-----------+----------------------+-----------------+---------------+
// 122007 swapped with 122010 with next hop 10.4.5.2