BGP Monitoring Protocol Basics

This chapter provides information about BGP Monitoring Protocol Basics.

Topics in this chapter include:

Applicability

The information and configuration in this chapter are based on SR OS Release 16.0.R2. BGP Monitoring Protocol (BMP) support was introduced in SR OS Release 16.0.R1 for unicast IPv4/IPv6, VPN IPv4/IPv6, and labeled IPv4/IPv6. SR OS Release 16.0.R4 provides an additional six address families: EVPN, L2VPN, multicast IPv4/IPv6, multicast VPN IPv4/IPv6.

Overview

The BGP Monitoring Protocol (BMP) is a unidirectional protocol for providers to monitor the behavior of BGP on their routers. A router communicates information about one or more BGP sessions to a BMP station, also known as a BMP collector. A router sends information in BMP messages to a BMP station. A BMP station never sends any messages to a router. BMP is described in detail in RFC 7854. BMP Operational Overview shows an operational overview of BMP.

Figure 1. BMP Operational Overview

BMP Message Types lists the BMP message types that are defined in RFC 7854.

Table 1. BMP Message Types

BMP Message Type

Description

0

Route monitoring

1

Statistics report

2

Peer down notification

3

Peer up notification

4

Initiation message

5

Termination message

6

Route mirroring message

A BMP station (or BMP collector) typically is a dedicated server running network management or network controller software. Current examples of free and open-source BMP station software are OpenBMP and Open Daylight. Nokia has commercial BMP station support available through the Network Services Platform (NSP) controller. The simple operations and packet format of BMP resulted in many providers having created their own proprietary BMP-collector software.

BMP allows a router to report different types of information. A router can:

  • send BMP messages with notifications when neighbors go into or out of Established state (for example, the peer goes "up" or "down"). These notifications are called BMP peer-up and peer-down messages.

  • periodically send statistical information about one or more neighbors. This information consists of several counters; for example, how many routes are received from a specific neighbor, or how many of those routes were rejected or accepted because of an ingress policy. Other counters report how many errors were encountered; for example, AS-path loops, duplicate prefixes, withdrawals received, and so on.

  • report the exact routes that were received from a neighbor. This action is called route monitoring. To do this, a router first re-encapsulates a BGP route into its original BGP update message, then encapsulates that BGP update message within a BMP route monitoring message to send it to the BMP station.

Note:

BMP on an SR OS router will only report information about routes that were received from a neighbor, which is the standard BMP behavior documented in RFC 7854. BMP will also report upon routes leaked or redistributed into the BGP RIB. A limitation of RFC 7854 is that BMP does not monitor routes sent toward a specific BGP neighbor. Nokia supports RFC 7854, so does not support monitoring of routes that were sent toward a BGP neighbor.

Configuration

Basic configuration of BMP

There are two main steps to enable BMP monitoring on an SR OS router:

  1. Configure a BMP station. This configuration identifies the target to which BMP information will be sent.

  2. Configure one or more BGP neighbors. These are the BGP peering sessions that will be monitored by BMP and the configured BMP station.

Configuring a BMP station

BMP stations and associated parameters are configured in global configuration mode. This allows the BMP station to reside either within the base router instance, or in a VPRN routing instance. The Nokia BMP implementation can monitor BGP peers in a base Internet service or in a VPRN service instance.

BMP will initiate a separate TCP session for each VPRN BGP instance monitored. The BMP router will use a different source TCP port number toward each configured TCP destination port number of the BMP station. For example, if there are four VPRN services configured in addition to the base router instance, the BMP router will instantiate five TCP sessions between the BMP router and the BMP station (one TCP session to monitor the base router instance, and four TCP sessions to monitor the VPRN services).

SR OS supports the configuration of up to eight BMP stations. To configure a BMP station, use the following command syntax:

*A:Dut-C# configure bmp station Antwerp create

This configuration example creates a BMP station with the name "Antwerp". This name must be used when configuring BGP peers to be monitored by this station. The name can also be used in show router bmp commands.

The next step is to configure (at a minimum) the IP address and the TCP destination port the BMP station is listening to. These parameters inform the BMP router where to reach the BMP station. BMP does not use a well-known port number; a provider can select any TCP port number. BMP sessions from an SR OS router can run over either TCP IPv4 or TCP IPv6.

The following configures the IP address 100.1.1.10 and port number 5000 of the BMP station:

configure
    bmp
        station "Antwerp" create
            connection
                station-address 100.1.1.10 port 5000
            exit
        exit
    exit

This configuration example creates a BMP station that can be used to monitor one or more BGP peers. Next, configure the BGP peers to be monitored by this station.

Assigning the BGP peers to be monitored

To configure one or more BGP neighbors to be monitored by the BMP station, first configure the monitor command in the bgp context or one of its subcontexts. This command can be configured at the BGP instance level, at the BGP group level, or at the neighbor level.

In the following example, monitoring is enabled (no shutdown) for all BGP peers defined in the bgp context, where the BMP reporting goes to BMP station Antwerp.

configure
    router
        bgp
            monitor
                station Antwerp
                no shutdown
            exit
            group internal-1
              ---snip---
            Exit
            group internal-2
              ---snip---
            exit
        exit

By default, BMP, including each individually configured station, is in the administrative shutdown state. To allow BMP to start the BMP sessions, administratively enable the BMP station:

configure
    bmp
        no shutdown
        station Antwerp
            no shutdown
        exit
    exit

All peers in the BGP instance of the base router are now monitored by station "Antwerp". At this stage, the router will only send BMP peer-up and peer-down messages to the BMP station. To send additional information (such as periodic statistics messages, or to report incoming BGP routes) requires explicit configuration.

Configuring periodic statistics messages

Enabling periodic statistics messages is done under the configure bmp station command. The command to enable periodic statistics is stats-report-interval <seconds>:

configure
    bmp
        station Antwerp
            stats-report-interval 600
        exit
    exit

This configuration example will cause the router to send statistics messages for each monitored peer to the BMP station every 10 minutes (600 seconds).

Verifying that the BMP session between router and BMP station works

To display the state of a BMP session to a BMP station, use the show router bmp station <station-name> command:

show router bmp station Antwerp

The output of the show command for BMP station Antwerp is as follows:

*A:Dut-C# show router bmp station "Antwerp"

===============================================================================
BMP Station "Antwerp" (monitoring router "Base")
===============================================================================
Admin State      : enabled              Global BMP State : enabled
Station Address  : 100.1.1.10           Station Port     : 5000
Via Router       : Base
Stats Report     : 30 seconds
Connect Interval : 5 seconds            Local Routes     : not reporting
Reported families: ipv4

Session State    : ESTABLISHED          Last State Change: 08/29/2018 13:23:19
Reason Last Down : admin shutdown       Last Msg Sent    : 08/29/2018 13:23:19
Local Address    : 100.1.1.3            Local Port       : 51446
Routes Timer     : 2 seconds left       Stats Timer      : 3 seconds left
Connect Timer    : not running          Monitored Peers  : 0 of 1
Initiation Msgs  : 1                    Goodbye Msgs     : 0
Peer Up Msgs     : 0                    Peer Down Msgs   : 0
Route Report Msgs: 0                    Stat Report Msgs : 0
Bytes Sent       : 276                  Output Queue     : 0/5
===============================================================================
*A:Dut-C#

The output consists of two blocks of information.

  • The first block shows configuration information about this specific BMP station.

  • The second block shows dynamic information about the current BMP session from the router instance to the BMP station.

Verify that the Session State is "ESTABLISHED".

Configuring BMP route monitoring

Configuring BMP route monitoring requires explicit configuration under the monitor command in the BGP instance context.

It is possible to configure BMP to report pre-policy routes, or post-policy routes, or both. Pre-policy routes are incoming routes as they were before applying any ingress policy. Post-policy routes are resulting routes in the Adj-RIB-In and reflect the routes after applying any BGP ingress policy.

Configuring BMP to report both pre- and post-policy routes will result in the doubling of BMP messages to the BMP station. This is because the router will send a route-monitor message for each pre-policy route, and for each post-policy route. This doubles the amount of resources consumed by BMP (such as bandwidth consumed on the link between the router and the BMP station, and CPU usage). The impact of enabling BMP route monitoring on the router CPU is similar to adding a BGP neighbor.

To configure route monitoring, use the route-monitoring [pre-policy] [post-policy] command in the monitor configuration mode in the BGP configuration context:

configure
    router
        bgp
            monitor
                station Antwerp
                route-monitoring pre-policy
                no shutdown
            exit
        exit

The BMP route monitoring is enabled within the context where the monitor station command is configured: in the general bgp context, the group context, or the neighbor context. With this configuration, the BMP router will start sending route monitoring messages for every route received from every neighbor in the base router BGP instance. This can be verified via the show router bmp station <station-name> command, which displays the counter for "Route Report Msgs:".

Advanced BMP configuration options

The BMP configuration can be fully customized. The following sections describe some additional configuration options.

Configuring route monitoring for different address families

When route monitoring is enabled, by default the BMP router will only report received IPv4 routes to the BMP station. This aligns with the default BGP behavior, where only unicast IPv4 is enabled when configuring a neighbor under BGP. To enable route monitoring for additional BGP address families, additional explicit configuration is required. The additional address families are available and can be configured under the configure bmp station command context, as follows:

configure
    bmp
        station Antwerp
            family 
        exit
    exit

In SR OS Release 16.0.R1, a Nokia BMP router supports route monitoring of six address families:

  • unicast IPv4, unicast IPv6

  • VPN-IPv4, VPN-IPv6

  • label-IPv4, label-IPv6

SR OS Release 16.0.R4 provides an additional six address families:

  • EVPN

  • L2VPN

  • mcast-IPv4, mcast-IPv6

  • mcast-VPN-IPv4, mcast-VPN-IPv6

Configuring monitoring of locally generated routes

RFC 7854 BMP reports only the routes in the Adj-RIB-In that were received from monitored neighbors. However, the BGP-RIB can hold more routes than those routes BGP has learned from neighbors. These locally generated routes are called imported or leaked routes.

Imported routes are learned via redistributing routes into BGP from external sources, like static, connected, IS-IS, or OSPF. Leaked routes are BGP routes from other BGP service instances that are leaked into the base router BGP.

To configure the Nokia BMP router to extend route reporting and report these imported and leaked routes to a configured BMP station, configure the report-local-routes command under the BMP station:

configure 
    bmp
        station Antwerp
            report-local-routes
        exit
    exit

Configuring the frequency of router statistics reports

When periodic statistics are enabled, the router will send all the statistics as described in RFC 7854, section 4.8, except for statistic number 13 (number of duplicate update messages received).

The Nokia BMP router-supported statistics are:

  • 0 - number of prefixes rejected by inbound policy

  • 1 - number of duplicate prefix advertisements received

  • 2 - number of duplicate withdraws received

  • 3 - number of received updates invalidated due to cluster-list loop

  • 4 - number of received updates invalidated due to AS-path loop

  • 5 - number of received updates invalidated due to originator-id

  • 6 - number of received updates invalidated due to as-confed loop

  • 7 - total number of routes in Adj-RIB-In (all families)

  • 8 - total number of routes in loc-RIB (all families)

  • 9 - number of routes per address family in Adj-RIB-In (see Note)

  • 10 - number of routes per address family in loc-RIB (see Note)

  • 11 - number of updates subjected to treat-as-withdraw

  • 12 - number of prefixes subjected to treat-as-withdraw

  • 13 - not supported/reported by SR OS (number of duplicate update messages received)

Note:

These two statistics are per address family. The address family is specified as a BGP AFI/SAFI pair. Regardless of what families are configured or supported for route monitoring, a router will report the statistics of all address families that were negotiated with the neighbor.

The values shown in the preceding counters are the same values that are shown by the show router <vrid> bgp neighbor <ip-addr> [detail] command.

Customizing the TCP connection to the BMP station

BMP uses TCP sessions to send BMP messages to the BMP station. It is possible to customize the TCP-session settings using several configuration options. These options are under the configure bmp station <name> connection command context.

Setting the local address of the TCP session

For increased operational security, BMP collectors might restrict accepting BMP sessions from unknown routers. It is important to have a configuration option to force a BMP router to accept specific IP addresses. To enforce the source address of a BMP session, the provider can configure the "local-address <ip-address>".

A Nokia router BMP session can be over an IPv4 or IPv6 TCP session. The source IP address used by the BMP router can be configured using the local-address command. The local address can be an IPv4 or an IPv6 address. The address family (IPv4 or IPv6) must match the address family of the IP address configured in the station-address <ip-address> port <portnr> command:

configure
    bmp
        station "Antwerp"
            connection
                station-address 100:200:300::1 port 5000
                local-address 100:200:300::2
            exit
        exit
    exit
Setting the routing context of the BMP session

A Nokia router allows a provider to configure multiple virtual router instances.

The base router is such a virtual router. Each VRPN instance is also a virtual router.

A Nokia BMP router allows a provider to monitor a BGP VPRN session while the TCP connection of the BMP session is configured in another VPRN instance.

This functionality allows the provider to let a single BMP station connection, within a specific VPRN instance, monitor BGP sessions and instances resident in other virtual routers.

The TCP connection of a BMP session is by default active in the base router. This can be changed by adding additional vprn context configuration when configuring a BMP station, as follows:

configure
    bmp
        station "Antwerp"
            connection
                router service-name vprn-22
            exit
        exit
Connect-retry command

When a router initiates a BMP session, it will try to establish the TCP connection to the BMP station. If this attempt fails, the router will wait a short while, then retry to bring up the connection. The time between two such attempts increases over time. The first attempt waits 3 seconds. After each failed attempt, the waiting time doubles (exponential increase). The maximum time to wait between two attempts is by default 2 minutes (120 seconds). This maximum waiting time is configurable, as follows:

configure
    bmp
        station "Antwerp"
            connection
                connect-retry 600
            exit
        exit

This configuration example will set the maximum waiting time between two connection attempts to 10 minutes (600 seconds).

TCP keepalives

BMP does not have any mechanism to detect the liveness of a BMP station. As the protocol is unidirectional, a router will not detect that a BMP station is down or unreachable, until it tries to send data to the station. During normal operation, the TCP layer will inform the BMP layer of an error when BMP tries to send a message to a BMP station that is down or unreachable. After discovering the TCP error, BMP will close the BMP session and try to re-establish a new session. However, when the BMP router has nothing to send to the unreachable BMP station, the station is not detected that easily.

Providers might need to detect a BMP failure even quicker. To do that, providers have the option to configure "TCP keepalives" on the BMP session. TCP keepalives are a feature of the TCP protocol. TCP keepalives are used to ensure the liveness of a TCP connection, even when no data is sent.

BMP on a Nokia router can use TCP keepalives. No special support is needed on the BMP station or host operating system because this functionality is a basic operation of the TCP session.

TCP keepalives are disabled by default. To enable a BMP session with TCP keepalives, configure:

configure
    bmp
        station "Antwerp"
            connection
                tcp-keepalive
                    no shutdown
                exit
            exit
        exit
    exit

The default operational values of TCP keepalives on a BMP session are:

  • keep-idle (sometimes called keep-wait) 600 seconds

  • keep-interval 15 seconds

  • keep-count 4 times

A provider can change these values. Configuring more aggressive values-tuning values for faster convergence-will have a slight impact on CPU and bandwidth usage. Configuring less aggressive values lowers the risk of false positives. For normal BMP operation, the default values are a good starting point. The following is an example if a provider wants to use non-default TCP keepalive values.

configure
    bmp
        station "Antwerp"
            connection
                tcp-keepalive
                    keep-count 5
                    keep-idle 300
                    keep-interval 10
                    no shutdown
                exit
            exit
        exit
    exit

Conclusion

In this chapter, the basic operation of Nokia BMP technology is described. The BMP implementation on a Nokia router is fully dual-stack IPv4/IPv6 aware and supports the monitoring of active BGP neighbor state (up or down), the BGP pre- and post-policy routes received, and a set of associated statistics for the BGP Adj-RIB-In and RIB-IN.

Usually, the impact upon the router performance for each configured BMP station is similar to adding a BGP neighbor. The Nokia BMP implementation supports the monitoring of twelve address families (unicast IPv4/IPv6, VPN IPv4/IPv6, label IPv4/IPv6, EVPN, L2VPN, mcast-IPv4/IPv6, mcast-VPN-IPv4/IPv6) in SR OS Release 16.0.R4, and later.

The Nokia BMP implementation can use TCP timers to detect unreachable BMP collectors. There is support for monitoring BGP neighbors in the base router or in a VPRN instance and support for BMP collectors located in the GRT or in any other VPRN service instance.