gRPC-based RIB API

Each router stores information about how to forward IP and MPLS packets in a set of RIB (routing information base) and FIB (forwarding information base) tables. These tables are conventionally populated by the management plane of the router (static entries) and by control plane protocols such as BGP, OSPF, ISIS, RSVP, LDP or segment routing.

In some SDN (Software Defined Networking) use cases, it may be useful to augment the RIB/FIB state held by a router to also include forwarding entries programmed by an external controller. SR OS supports a powerful and flexible gRPC-based RIB API service for this purpose. Using gRPC between a controller (gRPC client) and a router (gRPC server) has many benefits:

• gRPC is open source, with broad industry support and a rich ecosystem of tools and applications.

• gRPC is fast and efficient. The combination of HTTP/2 and protocol buffers ensures that a minimum number of bytes are sent across the wire as part of an RPC.

• gRPC is supported by many languages and platforms, including C, C++, C#, Go, Java, Node.js, Python, and Ruby.

To build a gRPC client that implements the RIB API service you must obtain, from Nokia, the protobuf definition file for the Nokia SR OS RIB API service. This protobuf file defines a RIB API service and its supported RPCs. The RIB API service supports one bidirectional streaming RPC called Modify and one unary RPC called GetVersion. The Modify RPC is used to add, delete or replace entries in any of the following RIB/FIB tables:

• IPv4 route table of the base router

• IPv6 route table of the base router

• IPv4 tunnel table

• IPv6 tunnel table

• MPLS label forwarding table

The GetVersion RPC allows the client to request the overall RIB API version and the individual RIB table versions supported by the router.

For maximum programming flexibility and speed, the entries added by the RIB-API service are not processed or stored as configuration data; they are provided directly to the control plane and modeled as though learned from a pseudo-protocol. RIB-API entries have the same persistence characteristics as protocol routes: if a router (gRPC server) detects that a gRPC client has disconnected or terminated its RPC, or if the router reboots, dependent RIB API entries are removed and must be re-programmed if persistence is required.

A gRPC client cannot delete entries it does not own, including routes from other protocols, but it can supersede routes from other sources through appropriate programming of preference values.

A gRPC client can read RIB/FIB entries programmed using the RIB API service (by any client) and obtain other state information that it needs using the gNMI management interface. gNMI is another gRPC-based service supported by the router and it supports RPCs for configuration, one-time state retrieval and telemetry state subscriptions. The same client can have active gNMI and RIB API RPCs with the same target router and at the same time using the same TCP connection.

The Modify RPC allows a gRPC client to add, modify or delete RIB/FIB entries. To accomplish this, the client sends a stream of ModifyRequest messages to the server (router) and receives, in return, a stream of ModifyResponse messages. Each ModifyRequest message can include multiple Request messages. Each Request message has a 64-bit ID (used to pair it with a Response message) and conveys one of the following instructions:

• A request to add an entry to one of the five supported RIB/FIB tables. The add operation requires the client to specify values for all parameters of the route, tunnel or label entry being programmed. If the add operation is successful, the RIB/FIB entry is considered owned by the client that carried out the transaction.

• A request to replace an entry in one of the five supported RIB/FIB tables. This operation completely replaces a RIB/FIB entry that was previously programmed by the same client. All the parameters of the new route, tunnel or label entry must be specified, even values that did not change from the previous entry.

• A request to delete an entry in one of the five supported RIB/FIB tables. This operation deletes a RIB/FIB entry that was previously programmed by the same client. The delete operation requires only the key values of the entry that should be deleted.

• An end-of-rib marker for one of the five supported RIB/FIB tables. This operation is used to accelerate the removal of stale entries associated with a RIB/FIB table, instead of waiting for purge timers to expire. Additional details are discussed in this chapter.

• A next-hop-switch instruction. The client sends this request to manually activate the primary or backup next hop associated with a specific next-hop-group of a specific tunnel or label entry. This may be done to facilitate a maintenance action or to manually revert traffic back to a primary next hop after it recovers from a failure that diverted traffic to a backup next hop.

The following general points should be noted:

• The router does not support multiple RIB API RPC sessions with the same client IP. If a client machine has multiple independent controllers, they need to interact with the router using different IP addresses.

• It is up to the client to choose a unique 64-bit identifier for each Request transaction, but Request IDs must increase throughout the lifetime of the RPC session.

• If a gRPC client omits any parameter that is considered mandatory by the server side, the router assumes that the intended value for the parameter is zero (0). This may cause an error if the zero value is invalid or unavailable.

• A status code of OK sent by the router to a client (in a ModifyResponse message) only indicates that the request was valid. In the case of an add/delete/modify operation, it does not mean that the FIB was actually modified and in the case of a next-hop-switchover it does not mean that the switchover has actually occurred.

• A RIB/FIB entry programmed by a gRPC client may be unusable because none of its next hops are resolvable or the requested label resources are not available. The entry is still accepted by the router and acknowledged with an OK status code response to the client. If at a later time the entry becomes usable, it is activated by the router automatically.

The RIB API service proto definition requires the client to provide values for all of the parameters listed in IPv4 route table programming when performing an add or replace of an IPv4 route. When performing a delete operation, only the bolded parameters (the lookup keys) are required. IPv4 route table programming describes the meaning of each parameter and its valid range.

The router's RIB API database can hold up to eight different gPRC-programmed entries per IPv4 prefix. Typically N entries would be associated with N different gRPC clients although the same client can program multiple entries for the same prefix as long as the preference values are unique.

When an IPv4 route entry is successfully added or modified in the RIB API database, the router assesses whether the entry is valid or invalid and constantly re-evaluates this status. The entry is invalid if its next hop cannot be resolved to a gRPC-programmed IPv4 tunnel that is up.

If the entry is valid, the router compares it to all other valid API-programmed entries for the same IPv4 prefix. The router chooses any non-stale entry over a stale entry, then the entry with the lowest preference value, and then if there is a tie, the lowest metric, and then if there is a still a tie, the entry from the client with the lowest 128-bit IP address (an IPv4 address is encoded in the lower 32 bits).

If the entry is valid and the best relative to other RIB API entries then it is submitted to the route table manager. This software task compares the API route to all other non-API routes it has for the same IPv4 prefix. The router chooses the entry with the lowest RTM preference value, and then if there is a tie, the lowest metric, and then if there is still a tie, the entry submitted by the protocol with the lowest default preference.

If the route table manager selects the API route as the best route it is sent to the FIB manager for programming into the datapath.

The RIB API service proto definition requires the client to provide values for all of the parameters listed in IPv6 route table programming when performing an add or replace of an IPv6 route. When performing a delete operation, only the bolded parameters (the lookup keys) are required. IPv6 route table programming describes the meaning of each parameter and its valid range.

The router's RIB API database can hold up to eight different gPRC-programmed entries per IPv6 prefix. Typically N entries would be associated with N different gRPC clients although the same client can program multiple entries for the same prefix as long as the preference values are unique.

When an IPv6 route entry is successfully added or modified in the RIB API database, the router assesses whether the entry is valid or invalid and constantly re-evaluates this status. The entry is invalid if its next hop cannot be resolved to a gRPC-programmed IPv6 tunnel that is up.

If the entry is valid, the router compares it to all other valid API-programmed entries for the same IPv6 prefix. The router chooses any non-stale entry over a stale entry, then the entry with the lowest preference value, and then if there is a tie, the lowest metric, and then if there is a still a tie, the entry from the client with the lowest 128-bit IP address (an IPv4 address is encoded in the lower 32 bits).

If the entry is valid and the best relative to other RIB API entries then it is submitted to the route table manager. This software task compares the API route to all other non-API routes it has for the same IPv6 prefix. The router chooses the entry with the lowest RTM preference value, and then if there is a tie, the lowest metric, and then if there is still a tie, the entry submitted by the protocol with the lowest default preference.

If the route table manager selects the API route as the best route it is sent to the FIB manager for programming into the datapath.

The RIB API service proto definition requires the client to provide values for all of the parameters listed in IPv4 tunnel table programming when performing an add or replace of an IPv4 MPLS tunnel. When performing a delete operation, only the bolded parameters (the lookup keys) are required. IPv4 tunnel table programming describes the meaning of each parameter and its valid range.

The router's RIB API database can hold up to eight different gPRC-programmed entries per IPv4 tunnel endpoint. Typically N entries would be associated with N different gRPC clients although the same client can program multiple entries for the same tunnel endpoint as long as the preference values are unique

When an IPv4 tunnel endpoint entry is successfully added or modified in the RIB API database, the router assesses whether the entry is valid or invalid and constantly re-evaluates this status. The tunnel is invalid if none of its primary next hops can be resolved to an interface that is up or if MPLS programming using the RIB API is currently administratively disabled.

If the IPv4 tunnel entry is valid the router compares it to all other valid API-programmed entries for the same IPv4 endpoint address. The router chooses any non-stale entry over a stale entry, then the entry with the lowest preference value, and then if there is a tie, the lowest metric, and then if there is a still a tie, the entry from the client with the lowest 128-bit IP address (an IPv4 address is encoded in the lower 32 bits).

If the entry is valid and the best relative to other RIB API entries then it is programmed into the FIB and added to the base router IPv4 tunnel table. The tunnel entry is now active and can be used to resolve the next hops of other routes. For more information, see the 7450 ESS, 7750 SR, 7950 XRS, and VSR Unicast Routing Protocols Guide.

The RIB API service proto definition requires the client to provide values for all of the parameters listed in IPv6 tunnel table programming when performing an add or replace of an IPv6 MPLS tunnel. When performing a delete operation, only the bolded parameters (the lookup keys) are required. IPv6 tunnel table programming describes the meaning of each parameter and its valid range.

The router's RIB API database can hold up to eight different gPRC-programmed entries per IPv6 tunnel endpoint. Typically N entries would be associated with N different gRPC clients although the same client can program multiple entries for the same tunnel endpoint as long as the preference values are unique.

When an IPv6 tunnel endpoint entry is successfully added or modified in the RIB API database, the router assesses whether the entry is valid or invalid and constantly re-evaluates this status. The tunnel is invalid if none of its primary next hops can be resolved to an interface that is up or if MPLS programming using the RIB API is currently administratively disabled.

If the IPv6 tunnel entry is valid the router compares it to all other valid API-programmed entries for the same IPv6 endpoint address. The router chooses any non-stale entry over a stale entry, then the entry with the lowest preference value, and then if there is a tie, the lowest metric, and then if there is a still a tie, the entry from the client with the lowest 128-bit IP address (an IPv4 address is encoded in the lower 32 bits).

If the entry is valid and the best relative to other RIB API entries then it is programmed into the FIB and added to the base router IPv6 tunnel table. The tunnel entry is now active and can be used to resolve the next hops of other routes. For more information, see the 7450 ESS, 7750 SR, 7950 XRS, and VSR Unicast Routing Protocols Guide.

The RIB API service proto definition requires the client to provide values for all of the parameters listed in MPLS LFIB programming when performing an add or replace of an MPLS LFIB entry. When performing a delete operation, only the bolded parameters (the lookup keys) are required. MPLS LFIB programming describes the meaning of each parameter and its valid range.

The router's RIB API database can hold up to eight different gPRC-programmed entries per MPLS label value. Typically N entries would be associated with N different gRPC clients although the same client can program multiple entries for the same label value as long as the preference values are unique.

When an MPLS label entry is successfully added or modified in the RIB API database, the router assesses whether the entry is valid or invalid and constantly re-evaluates this status. The label entry is invalid if it is a SWAP operation and none of the primary next hops can be resolved to an interface that is up or if MPLS programming using the RIB API is administratively disabled or if the requested incoming label has already been allocated to another owner sharing the same reserved label block or if the requested incoming label is outside the reserved label block range.

If the label entry is valid the router compares it to all other valid API-programmed entries for the same label value. The router chooses any non-stale entry over a stale entry, then the entry with the lowest preference value, and then if there is a tie, the entry from the client with the lowest 128-bit IP address (an IPv4 address is encoded in the lower 32 bits).

If the label entry is valid and best relative to other RIB API entries then it is programmed into the forwarding plane.

The RIB API service proto definition allows each MPLS tunnel and each MPLS label entry to have multiple next-hop-groups, each with a primary next hop and optionally one backup next hop. When a tunnel or label entry has more than one next-hop-group, this instructs the router to spray matching traffic across the next-hop-groups based on an ECMP or weighted-ECMP algorithm.

At any time, traffic hashed to a particular next-hop-group uses only the primary or backup next hop for forwarding. The selection of the active next hop within each next-hop-group is influenced by failures and by next-hop-switch Request messages made by the owner gRPC client. The specific rules are:

• If the primary next hop is resolved to an up interface when the next-hop-group is initially activated then it immediately becomes the active next hop.

• If the primary next hop is unresolved when the next-hop-group is initially activated then no next hop is immediately activated (even if the backup next hop is up) and a fixed wait-timer is started (three seconds). If the primary next hop comes up during that timer window then it is immediately activated. If the timer runs out and the primary has not yet come up the backup next hop is activated and stays active even if the primary comes up a short while later, after the timer expired.

• If the currently active next hop fails, the system automatically activates the other next hop.

• If the system receives a next-hop-switch Request targeting this specific entry and next-hop-group then the next hop indicated in the Request message is immediately activated, as long as it is up. If the requested next hop is down the message is ignored.

  Note: The router returns a status of OK in response to a next-hop-switch Request as long as the key values identify a next-hop-group that exists for a tunnel or label entry owned by the gRPC client, even if the needed next hop is not activated.

A gRPC client can use the gRPC gNMI service (Get RPC, Subscribe RPC) to retrieve state information from the router that can help it make better programming decisions. All states maintained by the router (and exposed to model-driven management interfaces) are available to the gRPC client.

RIB/FIB API also introduces additional YANG state models that are complementary to the programming actions. This new state is available through the following YANG paths:

• state/router/route-fib

• state/router/tunnel-fib

• state/router/label-fib

• state/router/rib-api/route

• state/router/rib-api/tunnel

• state/router/rib-api/label

The corresponding show commands are also provided for reference.

• show router fib-telemetry route
• show router fib-telemetry tunnel
• show router fib-telemetry label
• show router rib-api route
• show router rib-api tunnel
• show router rib-api label

The state information represented by the state/router/route-fib and state/router/tunnel-fib paths, and the show router fib-telemetry route and show router fib-telemetry tunnel show commands list are not collected by default, because it requires additional processing. In order for this state to be collected you must configure the configure router fib-telemetry command. If this command is not configured then these states are not collected at all, and telemetry subscriptions are not supported for any of the following paths:

• /state/router/route-fib
• /state/router/tunnel-fib
• /state/router/label-fib

It is not possible for a single telemetry subscription to include any of these three paths in addition to other state paths outside of this tree. This is because of the potential volume of information in the tables described in this chapter.

For gNMI telemetry subscriptions, the following restrictions should be noted:

• If a route, tunnel or MPLS label entry is modified, and it covered by an ON-CHANGE subscription to a state path enabled by config>router>fib-telemetry, the update replays the current values of the entire entry (except for statistics), including values did not change from the last update. It is up to the client to compare the update to the previous one received if it needs to know the exact properties that changed.

• Subscriptions to list keys of state paths enabled by fib-telemetry are not supported.

A gRPC client can make a request for traffic statistics to be collected. Both ingress and egress statistics are available but not all types of entries support both.

Traffic statistics are expressed in number of packets and in octets and are provided without forwarding-class or QoS profile distinction.

The system provides capabilities to display or show, clear, or monitor statistics.