Segment routing with IPv6 data plane (SRv6)

Segment routing steers packets by encoding the packet-processing instructions for each intermediate and destination router directly in the packet header.

The datapath pushes a list of instructions, in the form of Segment Identifiers (SIDs), onto the packet to forward the payload directly to the destination using the shortest path, or using source routing using one or more transit routers. Each router that terminates a SID in the segment list executes the instructions associated with that SID.

SRv6 is an IPv6-based segment routing approach that introduces the Segment Routing Header (SRH) as a new IPv6 extension header.

The following table highlights the differences between SR-MPLS and SRv6:

Table 1. Differences between SR-MPLS and SRv6
Feature SR-MPLS SRv6
SID encoding 32-bit MPLS label, see Datapath programming by SID type 128-bit IPv6 address
Header MPLS label stack (no new IPv6 extension header) Segment Routing Header (SRH) – a new IPv6 extension header
Forwarding plane support No changes needed for IPv4 or IPv6 forwarding planes (uses existing MPLS forwarding). See Datapath programming by SID type Requires forwarding plane support to parse and act on the SRH
Tunnel destination Works for both IPv4 and IPv6 destinations Works only for IPv6 destinations
Typical use case Legacy MPLS networks, mixed IPv4/IPv6 environments Pure IPv6 networks requiring segment routing capabilities

Supported platforms

SRv6 is supported on the following platforms:

  • 7220 IXR-H5
  • 7220 IXR-H6
Note: The 7220 IXR-H5/H6 platforms provide basic SRv6 support and are the only platforms used for AI backend deployments. They primarily function as pure transit nodes for uSID data traffic.

SRv6 deployment for an AI backend use case

Note: The 7220 IXR-H5/H6 platforms provide SRv6 in the transit router role, typically used for AI backend deployments, which is in place for LLM training. They primarily function as pure transit nodes for uSID data traffic.

In AI backend networks, training traffic is often a sequence of bursts (for example, during large language model (LLM) training). Each flow requires low latency, high bandwidth, and precise congestion control. Ensuring traffic reaches its destination and maintaining predictability are key to non-blocking LLM training.

Traditional ECMP load balancing has low entropy for this environment. Because ECMP distributes traffic using a flow hash, multiple synchronized training flows may map to the same hash bucket and share a single path. If that path becomes congested, the resulting bottleneck can delay the whole training.

In AI backend, the NIC encodes an ordered segment list in the packet header to steer AI workload traffic. The NIC initiates the SRv6 transport via adding an outer IPv6 header that carries the segment sequence for the precomputed fabric path.

This removes probabilistic hashing via transit routers and gives deterministic traffic engineering imposed through the source node which is the NIC in AI backend networks.

Use cases for deterministic path placement and SRv6 in AI backends are discussed in the IETF document SRv6 for Deterministic Path Placement in AI Backends.

For a step-by-step example of how traffic is carried in an AI backend using 7220 IXR-H5/H6 systems as uSID transit routers, see 7220 IXR-H5/H6 with SRv6 for an AI backend deployment.

The following list summarizes the basic SRv6 capabilities supported by 7220 IXR-H5/H6 systems for an AI backend use case:

  • SRv6 datapath forwarding behavior
  • Local node SID processing
  • SRH handling
  • Packet extraction to CPM
  • ICMP generation and error handling

The unsupported capabilities include:

  • Adjacency SID processing
  • Service initiation or termination
  • SRv6 over dot1q/Q-n-Q sub-interfaces
  • Multi-SRH processing

7220 IXR-H5/H6 with SRv6 for an AI backend deployment

A GPU NIC applies SRv6 micro-SID (uSID) encapsulation. The 7220 IXR-H5/H6 systems act as uSID transit routers in the AI backend fabric.

For deployment context and supported SRv6 capabilities in this environment, see SRv6 deployment for an AI backend use case.

Figure 1. 7220 IXR-H5/H6 with SRv6 for an AI backend deployment

In AI backend networks, the GPU Network Interface Cards (NICs) perform the initial SRv6 encapsulation (H.Encaps.Red). The NICs take the original data (for example, RoCEv2 packets) and add an outer IPv6 header. The header includes the SRv6 SID for routing.

The 7220 IXR-H5/H6 nodes at leaf and spine layers operate as SRv6 transit endpoint nodes.

When a GPU NIC hands over an SRv6 packet to a 7220 IXR-H5/H6 node, the node performs an End function, shifts the SID, and performs a lookup to forward the packet to the downstream node.

This example traces a RoCEv2 packet from GPU1 to GPU3 through a defined SRv6 path: Leaf1, Spine5, and Leaf3.

When an AI job needs synchronization, NIC1 creates a RoCEv2 packet destined for NIC3.

  1. Originating NIC (NIC1) NIC1 creates a RoCEv2 packet that must be sent to NIC3. NIC1 encapsulates the RoCEv2 packet with an outer IPv6 header (H.Encaps.Red behavior).
    • IPv6 DA: 5f00:0:0100:0500:0300::
    • The packet has no SRH.
  2. First transit node (Leaf1)
    • Packet in: (IPv6.DA=5f00:0:0100:0500:0300::)(RoCEv2)
    • Leaf1 has the SID 5f00:0:0100::/48 instantiated with the End with NEXT-CSID, PSP, and USD behavior. As a result, it shifts, looks up, and forwards the packet.
    • Packet out: (IPv6. DA=5f00:0:0500:0300::)(RoCEv2)
  3. Intermediate transit node (Spine5)
    • Packet in: (IPv6. DA=5f00:0:0500:0300::)(RoCEv2)
    • Spine5 has the SID 5f00:0:0500::/48 instantiated with the End with NEXT-CSID, PSP, and USD behavior. As a result, it shifts, looks up, and forwards the packet.
    • Packet out: (IPv6. DA=5f00:0:0300::)(RoCEv2)
  4. Egress transit node (Leaf3)
    • Packet in: (IPv6. DA=5f00:0:0300::)(RoCEv2)
    • Leaf3 has the SID 5f00:0:0300::/48 instantiated with the End with NEXT-CSID, PSP, and USD behavior. As a result, it removes the outer IPv6 header and forwards the inner packet.
    • Packet out: (RoCEv2)
  5. Receiving NIC (NIC3) receives the RDMA over Converged Ethernet (RoCE) v2 packet, processes it, and passes data to GPU3.

SRv6 uSID filters

On 7220 IXR-H5/H6 platforms, SRv6 uSID filters determine how ACLs apply to packets based on their role in the SRv6 path.

Transit role

When the device is acting as a transit router (with or without an SRH), ACLs are performed on the outer IPv6 header.

  • Ingress ACLs are applied to the received packet before any uSID shift operations.
  • Egress ACLs are applied to the packet to be transmitted after a uSID shift, which may apply to a different outerIPv6 Destination Address (DA) than the one at ingress

USD (uSID decapsulation) scenario

When the device performs a USD operation, ACLs are applied to the inner IPv4/IPv6 header after the outer IPv6 header has been decapsulated.

Both ingress and egress ACLs in this scenario operate on the inner payload.

Note:

While system ACLs can operate on the outer header, currently this is not supported in SR Linux.

SRv6 forwarding behavior

The following tables describe how the 7220 IXR-H5/H6 platforms process SRv6 micro-SID (uSID) packets. They list forwarding outcomes based on the Destination Address (DA), including SID matching, decapsulation, and SRH processing.

Table 2. No matching SID
Scenario Outcome
Destination Address (DA) is a remote locator with no matching SID Forward based using route lookup
Table 3. Local Node SID
Scenario Outcome
DA = local locator + remote SID Shift SID, then route lookup
DA = local locator + adjacency SID Adjacency SID processing is not supported, Packet is extracted to CPM for error handling
Table 4. USD
Scenario Outcome
DA = local node SID + inner IPv4/IPv6 Decapsulate outer header; route inner packet
Table 5. Unsupported cases
Scenario Outcome
Service SID (uDT/uSRV) Not supported
Table 6. Next header processing
Scenario Outcome
IPv4/IPv6 Decapsulates the outer IPv6 header and forwards based on the inner IPv4/IPv6 header
ICMP Extracts the packet for ICMP processing
TCP/UDP Extracts the packet for local IP processing
SRH Extracts the packet as an SRv6 header error for ICMP generation
Other next headers Extracts the packet as an SRv6 next header error for ICMP generation
Table 7. Forwarding, no SRH
Scenario Outcome
DA = Block + uN (not EOC) Shift SID + route
DA = Block + uN + EOC, NH=IPv4/IPv6 Decap + route
DA = Block + uN + EOC, NH=ICMP/TCP Extract to CPM for further processing
DA = Block + uN + EOC, NH=Other Extract to CPM as an erroneous packet
Table 8. SRH processing (SL > 1)
Scenario Outcome
SL > 1 Decrement SL, update DA, forward
SL up to 7 Supported (max 8 SL)
Table 9. SRH processing (SL=1)
Scenario Outcome
SL = 1 PSP applied, SRH removed
Table 10. Single SRH processing (SL=0)
Scenario Outcome
NH=IPv4/IPv6 Decapsulate + route
NH=ICMP/TCP Extract to CPM for further processing
NH=Other Extract to CPM as an erroneous packet
Table 11. Multiple SRH headers
Scenario Outcome
Multiple SRH headers Packet dropped; discard stat increment