For feedback and comments:
documentation.feedback@alcatel-lucent.com

Table of Contents Previous Next Index PDF


G.8032 Ethernet Ring Protection Multiple Ring Topology
In This Chapter
This section provides information about G.8032 Ethernet ring protection multiple ring topologies.
Topics in this section include:
Applicability
This example is applicable to the 7950 XRS (as of 10.0.R4), the 7750 SR-7/12 and 7450 ESS-7/12 (as of 9.0.R1), and the 7450 ESS-6/6v with IOM3-XP or IMM and 7750 SR-c4/12 (as of 11.0.R1). It is not supported on a 7750 SR-1, 7450 ESS-1, 7710 SR, or using an IOM-2 or lower.
The configuration was tested on release 12.0.R5 and covers both a single ring and multiple ring topologies.
Overview
G.8032 Ethernet ring protection is supported for data service SAPs within a regular VPLS service, a PBB VPLS (I/B-component) or a routed VPLS (R-VPLS). G.8032 is one of the fastest protection schemes for Ethernet networks. This example covers the advanced topic of Multiple Ring Control, sometimes referred to as multi-chassis protection, with access rings being the most common form of multiple ring topologies. Single Rings are covered in G.8032 Ethernet Ring Protection Single Ring Topology .This example will use a VPLS service to illustrate the configuration of G.8032. For very large ring topologies, Provider Backbone Bridging (PBB) can also be used but is not configured in this example.
ITU-T G.8032v2 specifies protection switching mechanisms and a protocol for Ethernet layer network (ETH) Ethernet rings. Ethernet rings can provide wide-area multipoint connectivity more economically due to their reduced number of links. The mechanisms and protocol defined in ITU-T G.8032v2 are highly reliable with stable protection and never form loops, which would negatively affect network operation and service availability. Each ring node is connected to adjacent nodes participating in the same ring using two independent paths, which use ring links (configured on ports or LAGs). A ring link is bounded by two adjacent nodes and a port for a ring link is called a ring port. The minimum number of nodes on a ring is two.
The fundamentals of this ring protection switching architecture are:
Loop avoidance in the ring is achieved by guaranteeing that, at any time, traffic may flow on all but one of the ring links. This particular link is called the Ring Protection Link (RPL) and under normal conditions this link is blocked, so it is not used for traffic. One designated node, the RPL Owner, is responsible to block traffic over the one designated RPL. Under a ring failure condition, the RPL Owner is responsible for unblocking the RPL, allowing the RPL to be used for traffic. The protocol ensures that even without an RPL owner defined, one link will be blocked and it operates as a break before make protocol, specifically the protocol guarantees that no link is restored until a different link in the ring is blocked. The other side of the RPL is configured as an RPL neighbor. An RPL neighbor blocks traffic on the link.
The event of a ring link or ring node failure results in protection switching of the traffic. This is achieved under the control of the ETH_FF functions on all ring nodes. A Ring Automatic Protection Switching (R-APS) protocol is used to coordinate the protection actions over the ring. The protection switching mechanisms and protocol supports a multi-ring/ladder network that consists of connected Ethernet rings.
 
 
Ring Protection Mechanism
The Ring Protection protocol is based on the following building blocks:
 
When sub-rings are used they can either connect to a major ring (which is configured in the exact same way as a single ring) or another sub-ring, or to a VPLS service. When connected to a major/sub ring, there is the option to extend the sub-ring control service through the major ring or not. This gives the following three options for sub-ring connectivity:
1.
Sub-ring to a major/sub ring with a virtual channel — In this case, a data service on the major/sub ring is created which is used to forward the R-APS messages for the sub-ring over the major/sub ring, between the interconnection points of the sub-ring to the major/sub ring. This allows the sub-ring to operate as a fully connected ring and is mandatory if the sub-ring connects two major/sub rings since the virtual channel is the only mechanism that the sub-rings can use to exchange control messages. It also could improve failover times if the sub-ring was large as it provides two paths on the sub-ring interconnection nodes to propagate the fault indication around the sub-ring, whereas without a virtual channel the fault indication may need to traverse the entire sub-ring. Each sub-ring requires its own data service on the major/sub ring for the virtual channel.
2.
Sub-ring to a major/sub ring without a virtual channel — In this case the sub-ring is not fully connected and does not require any resources on the major/sub ring. This option requires that the R-APS messages are not blocked on the sub-ring over its RPL.
3.
Sub-ring to a VPLS service — This is similar to (2) above but uses a VPLS service instead of a major ring. In this option, sub-ring failures can initiate the sending of an LDP MAC flush message into the VPLS service when spoke or MPLS mesh SDPs are used in the VPLS service.
Eth-Ring Terminologies
The implementation of Ethernet Ring (eth-ring) on an SR/ESS uses a VPLS as the construct for a ring flow function (one for ETH_FF (solely for control) and one for each service_FF) and SAPs (on ports or LAGs) as ring links. The control VPLS must be a regular VPLS but the data VPLS can be a regular VPLS, a PBB (B/I-) VPLS or a routed VPLS. The state of the data service SAPs is inherited from the state of the control service SAPs. Table 18 displays a comparison between the ITU-T and SR/ESS terminologies.
 
 
There are various ways that multiple rings can be interconnected and the possible topologies may be large. Customers typically have two forms of networks; access ring edge networks or larger multiple ring networks. Both topologies require ring interconnection.
 
Figure 293: G.8032 Major Ring and Sub-Ring
Figure 293 shows a ring of six nodes, with a major ring (regular Ethernet ring) on the top four nodes and a sub-ring on the bottom. A major ring is a fully connected ring. A sub-ring is a partial ring that depends on a major ring or a VPLS topology for part of the ring interconnect. Two major rings can be connected by a single sub-ring or a sub-ring can support other sub-rings.
In the major ring (on nodes A, B, C and D), one path of the RPL owner is designated to be the RPL and the respective SAPs will be blocked in order to prevent a loop. The choice of where to put the RPL is up to the network administrator and can be different for different control instances of the ring allowing an RPL to be used for some other ring's traffic. In the sub-ring, one path is designated as the RPL and will be blocked. Both the major ring and the sub-ring have their own RPL. The sub-ring interconnects to the major ring on nodes C and D and has a virtual channel on the major ring. The SR/ESS supports both virtual channel and non-virtual channel rings. Schematics of the physical and logical topologies are also shown in Figure 293.
Note: G.8032 has defined a Ring-ID value (1-255) in the G.8032 protocol. The SR/ESS implementation only uses a Ring-ID value of 1, which complies with G.8032v2. The configuration on a node uses a ring instance with a number but all rings use a Ring-ID of 1. This ring instance number is purely local and does not have to match on other ring nodes. Only the VLAN ID must match between SR/ESS ring nodes. For consistency in this example, VPLS instances and Ethernet ring instances are shown as matching for the same ring.
 
An RPL owner and RPL neighbor are configured for both the major ring and sub-ring. The path and associated link will be the RPL when the ring is fully operational and will be blocked by the RPL owner whenever there is no fault on other ring links. Each ring RPL is independent. If a different ring link fails then the RPL will be unblocked by the RPL owner. The link shared between a sub-ring and the major ring is completely controlled by the major ring as if the sub-ring were not there. Each ring can completely protect one fault within its ring. When the failed link recovers, it will initially be blocked by one of its adjacent nodes. The adjacent node sends an R-APS message across the ring to indicate the error is cleared and after a configurable time, if reversion is enabled, the RPL will revert to being blocked with all other links unblocked. This ensures that the ring topology when fully operational is predictable.
If a specific RPL owner is not configured (not recommended by G.8032 specification), then the last link to become active will be blocked and the ring will remain in this state until another link fails. This operation makes the selection of the blocked link non-deterministic and is not recommended.
The protection protocol uses a specific control VLAN, with the associated data VLANs taking their forwarding state from the control VLAN. The control VLAN cannot carry data.
 
Load Balancing with Multiple Ring Instances
Each control ring is independent of the other control rings on the same topology. Therefore since the RPL is not used by one control ring it is often desirable to set up a second control ring that uses a different link as RPL. This spreads out traffic in the topology but if there is a link failure in the ring all traffic will be on the remaining links. In the examples below only a single control ring instance is configured. Other control and data rings could be configured if desired.
 
Provider Backbone (PBB) Support
PBB services also support G.8032 as data services (the services used for the control VPLS must be a regular VPLS). B/I-VPLS rings support both major rings and sub-rings. B-VPLS rings support MC-LAG as a dual homing option when aggregating I-VPLS traffic onto a B-VPLS ring. In other words, I-VPLS rings should not be dual-homed into two BEB (Backbone Edge Bridge) nodes where the B-VPLS uses G.8032 to get connected to the rest of the B-VPLS network as the only mechanism which can propagate MAC flushes between an I-VPLS and B-VPLS is an LDP MAC flush.
 
SR/ESS Implementation
G.8032 is built from VPLS components and each ring consists of the configuration components illustrated in Figure 294.
Figure 294: G.8032 Ring Components
These components consist of:
Figure 295 depicts the major ring and sub-ring interconnection components:
Figure 295: G.8032 Sub-Ring Interconnection Components
For a sub-ring, the configuration is the same as a single ring except at the junction of the major ring and the sub-ring. The interconnection of a sub-ring and a major ring links the control VPLS of the sub-ring to a data VPLS of the major ring when a virtual link is used. Similarly the data VPLS of the sub-ring is linked to a data VPLS of the major ring. G.8032 Sub-Ring Interconnection Components illustrates the relationship of a sub-ring and a major ring. Since this sub-ring has a virtual channel, the data VPLS 2 has both data SAPs from the sub-ring and data SAPs from the major ring. The virtual channel is also optional and in non-virtual-link cases, no VPLS instance is required (see non-virtual-link in the section Configuration of a Sub-Ring to a VPLS Service (with a Non-Virtual Link) ).
In Figure 295, the inner tag values are kept the same for clarity but in fact any encapsulation that is consistent with the next ring link will work. In other words, ring SAPs can perform VLAN ID translation and even when connecting a sub-ring to a major ring. This also means that other ports may reuse the same tags when connecting independent services.
The R-APS tags (ring automatic protection switching tags) and SAPs on the rings can either be dot1q or QinQ encapsulated. It is also possible to have the control VPLS using single tagged frames with the data VPLSs using double tagged framed; this requires the system to be configured with the new-qinq-untagged-sap parameter (configure system ethernet new-qinq-untagged-sap), with the ring path raps-tags and control VPLS SAPs configured as qtag.0, and the data VPLSs configured as QinQ SAPs. Note that STP cannot be enabled on SAPs connected to eth-rings.
R-APS messages received from other nodes are normally blocked on the RPL interface but the sub-ring case with non-virtual channel recommends that R-APS messages be propagated over the RPL. Configuring sub-ring non-virtual-link on all nodes on the sub-ring propagation of R-APS messages is mandatory in order to achieve this.
R-APS messages are forwarded out of the egress using forwarding class NC, this should be prioritized accordingly in the SAP egress QoS policy to ensure that congestion does not cause R-APS messages to be dropped which could cause the ring to switch to another path.
 
Configuration
This section describes the configuration of multiple rings. The eth-ring configuration commands are shown below.
configure eth-ring <ring-index>
              ccm-hold-time { [down <down-timeout>] [up <up-timeout>] }
              compatible-version {1|2}
              description <description-string>
              guard-time <time>
              node-id <xx:xx:xx:xx:xx:xx or xx-xx-xx-xx-xx-xx>
              path {a|b} [ {<port-id>|<lag-id>]} raps-tag <qtag[.<qtag>] ]
                    description <description-string>
                    eth-cfm
                          mep <mep-id> domain <md-index> association <ma-index>
                          ...
                    rpl-end
                    shutdown
              revert-time <time>
              rpl-node {owner|nbr}
              shutdown
              sub-ring {virtual-link|non-virtual-link}
                  interconnect [ring-id <ring-index>|vpls]
                      propagate-topology-change
 
Parameters:
down — This command specifies the timer which controls the delay between detecting that ring path is down and reporting it to the G.8032 protection module. If a non-zero value is configured, the system will wait for the time specified in the value parameter before reporting it to the G.8032 protection module. Note that this parameter applies only to ring path CCM. It does not apply to the ring port link state. To dampen ring port link state transitions, use the hold-time parameter from the physical member port. This is useful if the underlying path between two nodes is going across an optical system which implements its own protection.
up — This command specifies the timer which controls the delay between detecting that ring path is up and reporting it to the G.8032 protection module. If a non-zero value is configured, the system will wait for the time specified in the value parameter before reporting it to the G.8032 protection module. Note that this parameter applies only to ring path CCM. It does not apply to the member port link state. To dampen member port link state transitions, use the hold-time parameter from the physical member port.
Values:
<down-timeout> : [0..5000] in centiseconds - Default: 0
<up-timeout> : [0..5000] in deciseconds - Default: 20
1 centisecond = 10ms
1 decisecond = 100ms
compatible-version — This command configures eth-ring compatibility version for the G.8032 state machine and messages. The default is version 2 (ITU G.8032v2) and all 7x50 systems use version 2. If there is a need to interwork with third party devices that only support version 1, this can be set to version 1 allowing the reception of version 1 PDUs. Note that version 2 is encoded as 1 in the R-APS messages. Compatibility allows the reception of version 1 (encoded as 0) R-APS PDUs but, as per the G.8032 specification, higher versions are ignored on reception. For the SR/ESS, messages are always originated with version 2. Therefore if a third party switch supported version 3 (encoded as 2) or higher interworking is also supported provided the other switch is compatible with version 2.
description <description-string> — This configures a text string, up to 80 characters, which can be used to describe the use of the eth-ring.
guard-time <time> — The forwarding method, in which R-APS messages are copied and forwarded at every Ethernet ring node, can result in a message corresponding to an old request, that is no longer relevant, being received by Ethernet ring nodes. Reception of an old R-APS message may result in erroneous ring state interpretation by some Ethernet ring nodes. The guard timer is used to prevent Ethernet ring nodes from acting upon outdated R-APS messages and prevents the possibility of forming a closed loop. Messages are not forwarded when the guard-timer is running.
Values: [1..20] in deciseconds - Default: 5
1 decisecond = 100ms
node-id <xx:xx:xx:xx:xx:xx> or <xx-xx-xx-xx-xx-xx> — This allows the node identifier to be explicitly configured. By default the chassis MAC is used. Not required in typical configurations.
path {a|b} [ {<port-id>|lag-id} raps-tag <qtag[.<qtag>] ] — This parameter defines the paths around the ring, of which there are two in different directions on the ring: an “a” path and a “b” path, except on the interconnection node where a sub-ring connects to another major/sub ring in which case there is one path (either a or b) configured together with the sub-ring command. The paths are configured on a dot1q or QinQ encapsulated access or hybrid port or a LAG with the encapsulation used for the R-APS messages on the ring. These can be either single or double tagged.
description <description-string> — This configures a text string, up to 80 characters, which can be used to describe the use of the path.
eth-cfm — Configures the associated Ethernet CFM parameters.
mep <mep-id> domain <md-index> association <ma-index> — The MEP defined under the path is used for the G.8032 protocol messages, which are based on IEEE 802.1ag/Y.1731 CFM frames.
rpl-end — When configured, this path is expected to be one end of the RPL. This parameter must be configured in conjunction with the rpl-node.
shutdown — This command shuts down the path.
revert-time <time> — This command configures the revert time for an Eth-Ring. Revert time is the time that the RPL will wait before returning to the blocked state. Configuring no revert-time disables reversion, effectively setting the revert-time to zero.
Values: [60..720] in seconds - Default: 300
rpl-node {owner|nbr} — A node can be designated as either the owner of the RPL, in which case this node is responsible for the RPL, or the nbr, in which case this node is expected to be the neighbor to the RPL owner across the RPL. The nbr is optional and is included to be compliant with the specification. This parameter must be configured in conjunction with the rpl-end command. On a sub-ring without virtual channel it is mandatory to configure sub-ring non-virtual-link on all nodes on the sub-ring to ensure propagation of the R-APS messages around the sub-ring.
shutdown — This command shuts down the ring.
sub-ring {virtual-link|non-virtual-link} — This command is configured on the interconnection node between the sub-ring and its major/sub ring to indicate that this ring is a sub-ring. The parameter specifies whether it uses a virtual link through the major/sub ring for the R-APS messages or not. A ring configured as a sub-ring can only be configured with a single path.
interconnect [ring-id <ring-index>|vpls] — A sub-ring connects to either another ring or a VPLS service. If it connects to another ring (either a major ring or another sub-ring), the ring identifier must be specified and the ring to which it connects must be configured with both a path “a” and a path “b”, meaning that it is not possible to connect a sub-ring to another sub-ring on an interconnection node.
Alternatively, the vpls parameter is used to indicate the sub-ring connects to a VPLS service. Interconnection using a VPLS service requires the sub-ring to be configured with non-virtual-link.
propagate-topology-change — If a topology change event happens in the sub-ring, it can be optionally propagated with the use of this parameter to either the major/sub-ring it is connected to, using R-APS messages, or to the LDP VPLS SDP peers using an LDP “flush-all-from-me” message if the sub-ring is connected to a VPLS service.
The operation and configuration of G.8032 with multiple rings is described below based on the topology shown in Figure 296.
Figure 296: Ethernet Test Topology
The configuration is divided into the following sections:
 
 
 
Configuration of a Sub-Ring to a Major Ring with a Virtual Link
To configure an Ethernet ring using R-APS, there will be at least 2 VPLS services required for one Eth-Ring instance, one for the control channel and the other (or more) for data channel(s). The control channel is used for R-APS signaling while the data channel is for user data traffic. The state of the data channels is inherited from the state of the control channel.
Step 1.
Eth-Ring needs an R-APS tag to send/receive G.8032 signaling messages. To configure a control channel, an access SAP configuration is required on each path a/b port. The SAP configuration follows that of the port and must be either dot1q or QinQ, consequently the control and data packets are either single tagged or double tagged. Note that single tagged control frames are supported on a QinQ port by configuring the system with the new-qinq-untagged-sap parameter (configure system ethernet new-qinq-untagged-sap), and the ring path raps-tags and control VPLS SAPs configured as qtag.0.
In this example QinQ tags are used. The commands for the major and sub rings ring, on PE-1 for example, are:
*A:PE-1# configure port 1/1/1 ethernet mode access 
*A:PE-1# configure port 1/1/2 ethernet mode access
*A:PE-1# configure port 1/1/3 ethernet mode access
*A:PE-1# configure port 1/1/1 ethernet encap-type qinq 
*A:PE-1# configure port 1/1/2 ethernet encap-type qinq
*A:PE-1# configure port 1/1/3 ethernet encap-type qinq
 
Step 2.
Configuring ETH-CFM domain, association and MEP is required before configuring Ethernet ring. The standard domain format is none and the association name should be icc-based (Y.1731), however, the SR/ESS implementation is flexible in that it supports both IEEE and ICC formats. The eth-ring MEP requires sub-second CCM interval (10ms or 100ms) to be configured (or 1 second from 11.0.R.1 or later).
Note that the MEPs used for R-APS control normally will have CCM configured on the control channel path MEPs for failure detection. Alternatively, detecting a failure of the ring may be achieved by running Ethernet in the First Mile (EFM) at the port level if CCM is not possible at 100ms or 10ms (or 1 second as of release 11.0.R1). Also rings can be run without CFM although the ETH-CFM association must be configured for R-APS messages to be exchanged. To omit the failure detecting CCMs, it would be necessary to remove the ccm-enable from under the path MEPs and to remove the remote-mepid on the corresponding ETH CFM configuration.
Loss-of-signal, in conjunction with other OAM mechanisms, is applicable only when the nodes are directly connected.
 
Figure 297 shows the details of the MEPs and their associations configured when both the major and sub rings are used. Note the associations only need to be pair wise unique but for clarity five unique associations are used. Also, any name format can be used but it must be consistent on both adjacent nodes.
Figure 297: ETH-CFM MEP Associations
The configuration of ETH-CFM for the major and sub rings on each node is as follows. Note that the CCMs for failure detection are configured for 1 second intervals.
Ring node PE-1: Association 1 and 3 are used for the major ring and association 4 is used for the sub-ring.
*A:PE-1>config>eth-cfm# info
----------------------------------------------
        domain 1 format none level 2
            association 1 format icc-based name "1000000000001"
                ccm-interval 1
                remote-mepid 2
            exit
            association 3 format icc-based name "1000000000003"
                ccm-interval 1
                remote-mepid 6
            exit
            association 4 format icc-based name "1000000000004"
                ccm-interval 1
                remote-mepid 8
            exit
        exit
Ring node PE-2: Association 1 and 2 are used for the major ring.
*A:PE-2>config>eth-cfm# info
----------------------------------------------
        domain 1 format none level 2
            association 1 format icc-based name "1000000000001"
                ccm-interval 1
                remote-mepid 1
            exit
            association 2 format icc-based name "1000000000002"
                ccm-interval 1
                remote-mepid 4
            exit
        exit
Ring node PE-3: Association 2 and 3 are used for the major ring and association 5 is used for the sub-ring.
*A:PE-3>config>eth-cfm# info
----------------------------------------------
domain 1 format none level 2
association 2 format icc-based name "1000000000002"
ccm-interval 1
remote-mepid 3
exit
association 3 format icc-based name "1000000000003"
ccm-interval 1
remote-mepid 5
exit
association 5 format icc-based name "1000000000005"
ccm-interval 1
remote-mepid 10
exit
        exit
Ring node PE-4: Association 4 and 5 are used for the sub-ring.
*A:PE-4>config>eth-cfm# info
----------------------------------------------
        domain 1 format none level 2
            association 4 format icc-based name "1000000000004"
                ccm-interval 1
                remote-mepid 7
            exit
            association 5 format icc-based name "1000000000005"
                ccm-interval 1
                remote-mepid 9
            exit
        exit
 
Step 3.
Two paths must be configured to form a ring. In this example, VLAN tag 1.1 is used as control channel for R-APS signaling for the major ring (ring 1) on the ports shown in Figure 296 using the ETH CFM information shown in Figure 297. The revert-time is set to its minimum value and CCM messages are enabled on the MEP. The control-mep parameter is required to indicate that this MEP is used for ring R-APS messages.
Ring node PE-1:
*A:PE-1>config# eth-ring 1
*A:PE-1>config>eth-ring# info
----------------------------------------------
        description "Ethernet Ring 1"
        revert-time 60
        path a 1/1/1 raps-tag 1.1
            description "Ethernet Ring : 1 Path : pathA"
            eth-cfm
                mep 1 domain 1 association 1
                    ccm-enable
                    control-mep
                    no shutdown
                exit
            exit
            no shutdown
        exit
        path b 1/1/2 raps-tag 1.1
            description "Ethernet Ring : 1 Path : pathB"
            eth-cfm
                mep 5 domain 1 association 3
                    ccm-enable
                    control-mep
                    no shutdown
                exit
            exit
            no shutdown
        exit
        no shutdown
 
It is mandatory to configure a MEP under the path otherwise this error will be displayed.
*A:PE-1>config>eth-ring>path# no shutdown
INFO: ERMGR #1001 Not permitted - must configure eth-cfm MEP first
 
Note that while MEPs are mandatory, enabling CCMs on the MEPs under the paths as a failure detection mechanism is optional as explained earlier.
Ring node PE-2: This is configured as the RPL owner with the RPL being on path “b” as indicated by the rpl-end parameter.
*A:PE-2# configure eth-ring 1
*A:PE-2>config>eth-ring# info
----------------------------------------------
        description "Ethernet Ring 1"
        revert-time 60
        rpl-node owner
        path a 1/1/1 raps-tag 1.1
            description "Ethernet Ring : 1 Path : pathA"
            eth-cfm
                mep 2 domain 1 association 1
                    ccm-enable
                    control-mep
                    no shutdown
                exit
            exit
            no shutdown
        exit
        path b 1/1/2 raps-tag 1.1
            description "Ethernet Ring : 1 Path : pathB"
            rpl-end
            eth-cfm
                mep 3 domain 1 association 2
                    ccm-enable
                    control-mep
                    no shutdown
                exit
            exit
            no shutdown
        exit
        no shutdown
It is not permitted to configure a path as an RPL end without having configured the node on this ring to be either the RPL owner or nbr otherwise the following error message is reported.
*A:PE-2>config>eth-ring>path# rpl-end
INFO: ERMGR #1001 Not permitted - path-type rpl-end is not consistent with eth-ring 'rpl-node' type
 
Ring node PE-3: This is configured as the RPL neighbor with the RPL being on path “b” as indicated by the rpl-end parameter.
*A:PE-3>config>eth-ring# info
----------------------------------------------
        description "Ethernet Ring 1"
        revert-time 60
        rpl-node nbr
        path a 1/1/1 raps-tag 1.1
            description "Ethernet Ring : 1 Path : pathA"
            eth-cfm
                mep 6 domain 1 association 3
                    ccm-enable
                    control-mep
                    no shutdown
                exit
            exit
            no shutdown
        exit
        path b 1/1/2 raps-tag 1.1
            description "Ethernet Ring : 1 Path : pathB"
            rpl-end
            eth-cfm
                mep 4 domain 1 association 2
                    ccm-enable
                    control-mep
                    no shutdown
                exit
            exit
            no shutdown
        exit
        no shutdown
 
The link between PE-2 and PE-3 will be the RPL with PE-2 and PE-3 blocking that link when the ring is fully operational. In this example the RPL is using the same path, namely path “b”, on both PE-2 and PE-3 but this is not mandatory.
Step 4.
Ring nodes PE-1, PE-3 and PE-4 have a sub-ring. The sub-ring attaches to the major ring (ring 1). The sub-ring in this case will use a virtual-link. The interconnection ring instance identifier
(
ring-id) is specified and propagate-topology-change indicates that sub-ring flushing will be propagated to the major ring. Only one path is specified since the other path is not required at an interconnection node. Sub-rings are almost identical to major rings in operation except that sub-rings send MAC flushes towards their connected ring (either a major or sub ring). Major or sub rings never send MAC flushes to their sub-rings. Therefore a couple of sub-rings connected to a major ring can cause MACs to flush on the major ring but the major ring will not propagate a sub-ring MAC flush to other sub-rings.
Ring node PE-1: This node provides an interconnection between the major ring (1) and the sub-ring (2). Ring 2 is configured to be a sub-ring which interconnects to ring 1. It will use a virtual link on ring 1 to send R-APS messages to the other interconnection node and topology changes will be propagated from the sub-ring 2 to the major ring 1.
*A:PE-1>config# eth-ring 2
*A:PE-1>config>eth-ring# info
----------------------------------------------
        description "Ethernet Sub-ring 2 on Ring 1"
        revert-time 60
        sub-ring virtual-link
            interconnect ring-id 1
                propagate-topology-change
            exit
        exit
        path a 1/1/3 raps-tag 2.1
            description "Ethernet Ring : 2 Path : pathA"
            eth-cfm
                mep 7 domain 1 association 4
                    ccm-enable
                    control-mep
                    no shutdown
                exit
            exit
            no shutdown
        exit
        no shutdown
Ring node PE-3: The configuration of PE-3 is similar to PE-1 but PE-3 is the RPL neighbor, with the RPL end on path “a”, for the RPL between PE-3 and PE-4.
*A:PE-3>config# eth-ring 2
*A:PE-3>config>eth-ring# info
----------------------------------------------
        description "Ethernet Sub-ring 2 on Ring 1"
        revert-time 60
        rpl-node nbr
        sub-ring virtual-link
            interconnect ring-id 1
                propagate-topology-change
            exit
        exit
        path a 1/1/3 raps-tag 2.1
            description "Ethernet Ring : 2 Path : pathA"
            rpl-end
            eth-cfm
                mep 9 domain 1 association 5
                    ccm-enable
                    control-mep
                    no shutdown
                exit
            exit
            no shutdown
        exit
        no shutdown
 
Ring node PE-4: This node only has configuration for the sub-ring, ring 2. It is also the RPL owner, with path “b” being the RPL end, for the RPL between PE-3 and PE-4.
*A:PE-4>config# eth-ring 2
*A:PE-4>config>eth-ring# info
----------------------------------------------
        description "Ethernet Ring : 2"
        revert-time 60
        rpl-node owner
        path a 1/1/1 raps-tag 2.1
            description "Ethernet Ring : 2 : pathA"
            eth-cfm
                mep 8 domain 1 association 4
                    ccm-enable
                    control-mep
                    no shutdown
                exit
            exit
            no shutdown
        exit
        path b 1/1/3 raps-tag 2.1
            description "Ethernet Ring : 2 : pathB"
            rpl-end
            eth-cfm
                mep 10 domain 1 association 5
                    ccm-enable
                    control-mep
                    no shutdown
                exit
            exit
            no shutdown
        exit
        no shutdown
 
Until the Ethernet Ring instance is attached to a VPLS service, the ring operational status is down and the forwarding status of each port is blocked. This prevents the operator from creating a loop by mis-configuration. This state can be seen on ring node PE-1 as follows:
*A:PE-1# show eth-ring 1
===============================================================================
Ethernet Ring 1 Information
===============================================================================
Description        : Ethernet Ring 1
Admin State        : Up                 Oper State       : Down
Node ID            : d8:30:ff:00:00:00
Guard Time         :    5 deciseconds   RPL Node         : rplNone
Max Revert Time    :   60 seconds       Time to Revert   : N/A
CCM Hold Down Time :    0 centiseconds  CCM Hold Up Time :   20 deciseconds
Compatible Version : 2
APS Tx PDU         : Request State: 0xB
                     Sub-Code     : 0x0
                     Status       : 0x20  ( BPR )
                     Node ID      : d8:30:ff:00:00:00
Defect Status      :
 
Sub-Ring Type      : none
-------------------------------------------------------------------------------
Ethernet Ring Path Summary
-------------------------------------------------------------------------------
Path Port     Raps-Tag     Admin/Oper      Type            Fwd State
-------------------------------------------------------------------------------
  a  1/1/1    1.1             Up/Down      normal          blocked
  b  1/1/2    1.1             Up/Down      normal          blocked
===============================================================================
 
 
 
Step 5.
Path “a” and “b” configured in the eth-ring must be added as SAPs into a VPLS service (standard VPLS) using the eth-ring parameter. The SAP encapsulation values must match the values of the raps-tag configured for the associated path.
G.8032 uses the same raps-tag value on all nodes on the ring, as configured in this example. However, the SR/ESS implementation relaxes this constraint by requiring the tag to match only on adjacent nodes.
A VPLS service (identifier 1) is configured on PE-1, PE-2 and PE-3 for the control channel for the major ring (ring1), and another VPLS service (identifier 2) is used on PE-1, PE-3 and PE-4 for the sub-ring (ring 2).
Ring node PE-1: Control service for the major ring.
*A:PE-1>config>service# vpls 1
*A:PE-1>config>service>vpls# info
----------------------------------------------
            description "Control VID 1.1 for Ring 1 Major Ring"
            stp
                shutdown
            exit
            sap 1/1/1:1.1 eth-ring 1 create
                stp
                    shutdown
                exit
            exit
            sap 1/1/2:1.1 eth-ring 1 create
                stp
                    shutdown
                exit
            exit
            no shutdown
 
Ring node PE-2: Control service for the major ring.
*A:PE-2>config>service# vpls 1
*A:PE-2>config>service>vpls# info
----------------------------------------------
            description "Control VID 1.1 for Ring 1 Major Ring"
            stp
                shutdown
            exit
            sap 1/1/1:1.1 eth-ring 1 create
                stp
                    shutdown
                exit
            exit
            sap 1/1/2:1.1 eth-ring 1 create
                stp
                    shutdown
                exit
            exit
            no shutdown
 
Ring node PE-3: Control service for the major ring.
*A:PE-3>config>service# vpls 1
*A:PE-3>config>service>vpls# info
----------------------------------------------
            description "Control VID 1.1 for Ring 1 Major Ring"
            stp
                shutdown
            exit
            sap 1/1/1:1.1 eth-ring 1 create
                stp
                    shutdown
                exit
            exit
            sap 1/1/2:1.1 eth-ring 1 create
                stp
                    shutdown
                exit
            exit
            no shutdown
 
Note that you cannot add a normal SAP or SDP in a control channel VPLS, only SAPs with an eth-ring parameter can be added. Trying to add a SAP without this parameter into a control channel VPLS will result in the message below being displayed.
*A:PE-1>config>service>vpls# sap 1/1/7:1.1 create
MINOR: SVCMGR #1321 Service contains an Ethernet ring control SAP
 
For the sub-ring, the configuration of a split horizon group for the virtual channel on the major ring on the interconnection nodes is recommended. This avoids the looping of control R-APS messages in the case there is a mis-configuration in the major ring.
Ring node PE-1: Control service for the sub-ring. Notice that the first two SAPs connect to the major ring (ring1), these being for the virtual channel, and the third SAP connects to the sub-ring (ring 2).
*A:PE-1>config>service# vpls 2
*A:PE-1>config>service>vpls# info
----------------------------------------------
            description "Control/Virtual Channel VID 2.1 for Ring 2"
            split-horizon-group "shg-ring2" create
            exit
            stp
                shutdown
            exit
            sap 1/1/1:2.1 split-horizon-group "shg-ring2" eth-ring 1 create
                description "Ring 2 Interconnection using Ring 1"
                stp
                    shutdown
                exit
            exit
            sap 1/1/2:2.1 split-horizon-group "shg-ring2" eth-ring 1 create
                description "Ring 2 Interconnection using Ring 1"
                stp
                    shutdown
                exit
            exit
            sap 1/1/3:2.1 eth-ring 2 create
                stp
                    shutdown
                exit
            exit
            no shutdown
 
Ring node PE-2: Control service for the sub-ring. Sub-ring 2 is not present on PE-2, however, its virtual channel on major ring 1 needs to exist throughout ring 1.
*A:PE-2>config>service>vpls# info
----------------------------------------------
            description "Virtual Channel VID 2.1 for Ring 2"
            stp
                shutdown
            exit
            sap 1/1/1:2.1 eth-ring 1 create
                stp
                    shutdown
                exit
            exit
            sap 1/1/2:2.1 eth-ring 1 create
                stp
                    shutdown
                exit
            exit
            no shutdown
 
Note: If multiple virtual channels are used (due to the aggregation of multiple sub-rings into the same major ring), their configuration could be simplified on non-interconnection nodes on the major ring. To achieve this on a ring node such as PE-2, a default SAP could be used rather than configuring a VPLS per virtual channel. If QinQ SAPs are used then a default SAP of
1/1/[1,2]:qtag.* could be used but requires all control channels for sub-rings to be using qtag as the outer VLAN ID, or 1/1/[1,2]:* if dot1q SAPs were used. This is because the SAPs match explicit SAPs definitions first and the default SAP will handle any other traffic.
Ring node PE-3: Control service for the sub-ring. This is similar to the configuration of PE-1.
*A:PE-3>config>service# vpls 2
*A:PE-3>config>service>vpls# info
----------------------------------------------
            description "Control/Virtual Channel VID 2.1 for Ring 2"
            split-horizon-group "shg-ring2" create
            exit
            stp
                shutdown
            exit
            sap 1/1/1:2.1 split-horizon-group "shg-ring2" eth-ring 1 create
                stp
                    shutdown
                exit
            exit
            sap 1/1/2:2.1 split-horizon-group "shg-ring2" eth-ring 1 create
                stp
                    shutdown
                exit
            exit
            sap 1/1/3:2.1 eth-ring 2 create
                stp
                    shutdown
                exit
            exit
            no shutdown
 
Ring node PE-4: Control service for the sub-ring. Both SAPs are configured on the sub-ring
(ring 2).
*A:PE-4>config>service# vpls 2
*A:PE-4>config>service>vpls# info
----------------------------------------------
            description "Control VID 2.1 for Ring 2 Sub-ring"
            stp
                shutdown
            exit
            sap 1/1/1:2.1 eth-ring 2 create
                stp
                    shutdown
                exit
            exit
            sap 1/1/3:2.1 eth-ring 2 create
                stp
                    shutdown
                exit
            exit
            no shutdown
 
At this point, the Eth-Ring is operationally up and the RPL is blocking successfully on ring node PE-2 port 1/1/2, as expected for the RPL owner/end configuration and on port 1/1/3 on PE-3 as the RPL neighbor.
Show Output
An overview of all of the ring(s) can be shown using the following commands, in this case on
PE-1.
First, the ETH ring status is shown.
*A:PE-1# show eth-ring status
===============================================================================
Ethernet Ring (Status information)
===============================================================================
Ring   Admin  Oper       Path Information                 MEP Information
ID     State  State  Path         Tag       State     Ctrl-MEP CC-Intvl Defects
-------------------------------------------------------------------------------
1      Up     Up     a - 1/1/1    1.1       Up        Yes      1         -----
                     b - 1/1/2    1.1       Up        Yes      1         -----
2      Up     Up     a - 1/1/3    2.1       Up        Yes      1         -----
                     b - N/A                 -        -        -         -----
===============================================================================
Ethernet Tunnel MEP Defect Legend:
R = Rdi, M = MacStatus, C = RemoteCCM, E = ErrorCCM, X = XconCCM
*A:PE-1#
 
It is expected that the state is “up”, even on ring paths which are blocked. The “Defects” column refers to the CFM defects of the MEPs. If there is a problem, these will be flagged.
This output shows the ring and path forwarding states.
*A:PE-1# show eth-ring
===============================================================================
Ethernet Rings (summary)
===============================================================================
Ring Int  Admin Oper            Paths Summary                       Path States
ID   ID   State State                                               a     b
-------------------------------------------------------------------------------
1    -    Up    Up    a - 1/1/1    1.1       b - 1/1/2    1.1       U     U
2    1    Up    Up    a - 1/1/3    2.1       b - Not configured     U     -
===============================================================================
Ethernet Ring Summary Legend:   B - Blocked    U - Unblocked
*A:PE-1#
 
The specific ring information can be shown as follows.
Ring node PE-1:
*A:PE-1# show eth-ring 1
===============================================================================
Ethernet Ring 1 Information
===============================================================================
Description        : Ethernet Ring 1
Admin State        : Up                 Oper State       : Up
Node ID            : d8:30:ff:00:00:00
Guard Time         :    5 deciseconds   RPL Node         : rplNone
Max Revert Time    :   60 seconds       Time to Revert   : N/A
CCM Hold Down Time :    0 centiseconds  CCM Hold Up Time :   20 deciseconds
Compatible Version : 2
APS Tx PDU         : N/A
Defect Status      :
 
Sub-Ring Type      : none
-------------------------------------------------------------------------------
Ethernet Ring Path Summary
-------------------------------------------------------------------------------
Path Port     Raps-Tag     Admin/Oper      Type            Fwd State
-------------------------------------------------------------------------------
  a  1/1/1    1.1             Up/Up        normal          unblocked
  b  1/1/2    1.1             Up/Up        normal          unblocked
===============================================================================
*A:PE-1#
 
The status around the major ring can also be checked.
Ring node PE-2: Major ring.
*A:PE-2# show eth-ring 1
===============================================================================
Ethernet Ring 1 Information
===============================================================================
Description        : Ethernet Ring 1
Admin State        : Up                 Oper State       : Up
Node ID            : d8:31:ff:00:00:00
Guard Time         :    5 deciseconds   RPL Node         : rplOwner
Max Revert Time    :   60 seconds       Time to Revert   : N/A
CCM Hold Down Time :    0 centiseconds  CCM Hold Up Time :   20 deciseconds
Compatible Version : 2
APS Tx PDU         : Request State: 0x0
                     Sub-Code     : 0x0
                     Status       : 0xA0  ( RB BPR )
                     Node ID      : d8:31:ff:00:00:00
Defect Status      :
 
Sub-Ring Type      : none
-------------------------------------------------------------------------------
Ethernet Ring Path Summary
-------------------------------------------------------------------------------
Path Port     Raps-Tag     Admin/Oper      Type            Fwd State
-------------------------------------------------------------------------------
  a  1/1/1    1.1             Up/Up        normal          unblocked
  b  1/1/2    1.1             Up/Up        rplEnd          blocked
===============================================================================
*A:PE-2#
 
Note that PE-2 is the RPL owner with port 1/1/2 as an RPL end, which is blocked as expected. The revert-time is also shown to be the configured value. Detailed information is shown relating to the R-APS PDUs being transmitted on this ring as this node is the RPL owner.
When a revert is pending, the “Time to Revert” will show the number of seconds remaining before the revert occurs.
Ring node PE-3: Major ring.
*A:PE-3# show eth-ring 1
===============================================================================
Ethernet Ring 1 Information
===============================================================================
Description        : Ethernet Ring 1
Admin State        : Up                 Oper State       : Up
Node ID            : d8:32:ff:00:00:00
Guard Time         :    5 deciseconds   RPL Node         : rplNeighbor
Max Revert Time    :   60 seconds       Time to Revert   : N/A
CCM Hold Down Time :    0 centiseconds  CCM Hold Up Time :   20 deciseconds
Compatible Version : 2
APS Tx PDU         : N/A
Defect Status      :
Sub-Ring Type      : none
-------------------------------------------------------------------------------
Ethernet Ring Path Summary
-------------------------------------------------------------------------------
Path Port     Raps-Tag     Admin/Oper      Type            Fwd State
-------------------------------------------------------------------------------
  a  1/1/1    1.1             Up/Up        normal          unblocked
  b  1/1/2    1.1             Up/Up        rplEnd          blocked
===============================================================================
*A:PE-3#
 
PE-3 is the RPL neighbor with port 1/1/2 as an RPL end which is blocked as expected.
The information for the sub-ring can also be shown using the same command.
Ring node PE-1: Sub-ring.
*A:PE-1# show eth-ring 2
===============================================================================
Ethernet Ring 2 Information
===============================================================================
Description        : Ethernet Sub-ring 2 on Ring 1
Admin State        : Up                 Oper State       : Up
Node ID            : d8:30:ff:00:00:00
Guard Time         :    5 deciseconds   RPL Node         : rplNone
Max Revert Time    :   60 seconds       Time to Revert   : N/A
CCM Hold Down Time :    0 centiseconds  CCM Hold Up Time :   20 deciseconds
Compatible Version : 2
APS Tx PDU         : N/A
Defect Status      :
 
Sub-Ring Type      : virtualLink        Interconnect-ID  : 1
Topology Change    : Propagate
-------------------------------------------------------------------------------
Ethernet Ring Path Summary
-------------------------------------------------------------------------------
Path Port     Raps-Tag     Admin/Oper      Type            Fwd State
-------------------------------------------------------------------------------
  a  1/1/3    2.1             Up/Up        normal          unblocked
  b  -        -                 -/-         -               -
===============================================================================
*A:PE-1#
 
Note that only path “a” is active and unblocked. The second path, path “b” is not configured as only one path is required on an interconnection node. The “Sub-Ring Type” is shown to be a virtual link interconnecting to ring 1, with topology propagation enabled.
Ring node PE-3: Sub-ring.
*A:PE-3# show eth-ring 2
===============================================================================
Ethernet Ring 2 Information
===============================================================================
Description        : Ethernet Sub-ring 2 on Ring 1
Admin State        : Up                 Oper State       : Up
Node ID            : d8:32:ff:00:00:00
Guard Time         :    5 deciseconds   RPL Node         : rplNeighbor
Max Revert Time    :   60 seconds       Time to Revert   : N/A
CCM Hold Down Time :    0 centiseconds  CCM Hold Up Time :   20 deciseconds
Compatible Version : 2
APS Tx PDU         : N/A
Defect Status      :
 
Sub-Ring Type      : virtualLink        Interconnect-ID  : 1
Topology Change    : Propagate
-------------------------------------------------------------------------------
Ethernet Ring Path Summary
-------------------------------------------------------------------------------
Path Port     Raps-Tag     Admin/Oper      Type            Fwd State
-------------------------------------------------------------------------------
  a  1/1/3    2.1             Up/Up        rplEnd          blocked
  b  -        -                 -/-         -               -
===============================================================================
*A:PE-3#
 
PE-3 is the RPL neighbor with port 1/1/3 as an RPL end, which is blocked as expected.
Ring Node PE-4: Sub-ring.
*A:PE-4# show eth-ring 2
===============================================================================
Ethernet Ring 2 Information
===============================================================================
Description        : Ethernet Ring : 2
Admin State        : Up                 Oper State       : Up
Node ID            : d8:33:ff:00:00:00
Guard Time         :    5 deciseconds   RPL Node         : rplOwner
Max Revert Time    :   60 seconds       Time to Revert   : N/A
CCM Hold Down Time :    0 centiseconds  CCM Hold Up Time :   20 deciseconds
Compatible Version : 2
APS Tx PDU         : Request State: 0x0
                     Sub-Code     : 0x0
                     Status       : 0xA0  ( RB BPR )
                     Node ID      : d8:33:ff:00:00:00
Defect Status      :
 
Sub-Ring Type      : none
-------------------------------------------------------------------------------
Ethernet Ring Path Summary
-------------------------------------------------------------------------------
Path Port     Raps-Tag     Admin/Oper      Type            Fwd State
-------------------------------------------------------------------------------
  a  1/1/1    2.1             Up/Up        normal          unblocked
  b  1/1/3    2.1             Up/Up        rplEnd          blocked
===============================================================================
*A:PE-4#
 
PE-4 is the RPL owner with port 1/1/3 as an RPL end, which is blocked as expected.
The details of an individual path can be shown.
*A:PE-1# show eth-ring 1 path a
===============================================================================
Ethernet Ring 1 Path Information
===============================================================================
Description        : Ethernet Ring : 1 Path : pathA
Port               : 1/1/1              Raps-Tag         : 1.1
Admin State        : Up                 Oper State       : Up
Path Type          : normal             Fwd State        : unblocked
                                        Fwd State Change : 09/19/2014 17:45:19
Last Switch Command: noCmd
APS Rx PDU         : Request State: 0x0
                     Sub-Code     : 0x0
                     Status       : 0xA0  ( RB BPR )
                     Node ID      : d8:31:ff:00:00:00
===============================================================================
*A:PE-1#
 
The ring hierarchy created can be shown, either for all rings, or as below for a specific ring.
*A:PE-1# show eth-ring 1 hierarchy
===============================================================================
Ethernet Ring 1 (hierarchy)
===============================================================================
Ring Int  Admin Oper            Paths Summary                       Path States
ID   ID   State State                                               a     b
-------------------------------------------------------------------------------
1    -    Up    Up    a - 1/1/1    1.1       b - 1/1/2    1.1       U     U
2    1    Up    Up    a - 1/1/3    2.1       b - Not configured     U     -
===============================================================================
Ethernet Ring Summary Legend:   B - Blocked    U - Unblocked
*A:PE-1#
 
Step 6.
The user data channels are created on a separate VPLS, VPLS 11 in this example, using VLAN tag 1.11. The ring data channels must be on the same ports as the corresponding control channels configured above. The access into the data services can use normal SAPs and/or SDPs, for example the SAP on port 1/1/9 below. Customer data traverses the ring on a data SAP. Multiple parallel data SAPs in different data services can be controlled by one control ring instance (eth-ring 1 in the example).
Ring node PE-1: The first two data SAPs correspond to the major ring 1, while the third SAP is the data SAP on the sub-ring 2.
*A:PE-1>config>service# vpls 11
*A:PE-1>config>service>vpls# info
----------------------------------------------
            description "Data VPLS"
            stp
                shutdown
            exit
            sap 1/1/1:1.11 eth-ring 1 create
                stp
                    shutdown
                exit
            exit
            sap 1/1/2:1.11 eth-ring 1 create
                stp
                    shutdown
                exit
            exit
            sap 1/1/3:1.11 eth-ring 2 create
                stp
                    shutdown
                exit
            exit
            sap 1/1/9:1 create
                description "Sample Customer Service SAP"
            exit
            no shutdown
 
Ring node PE-3 (not shown) would be similar to ring node 1.
Ring node PE-2 is also similar with a single ring data service using VPLS 11 and tag 1.11.
*A:PE-2# configure service vpls 11
*A:PE-2>config>service>vpls# info
----------------------------------------------
            description "Data VPLS"
            stp
                shutdown
            exit
            sap 1/1/1:1.11 eth-ring 1 create
                stp
                    shutdown
                exit
            exit
            sap 1/1/2:1.11 eth-ring 1 create
                stp
                    shutdown
                exit
            exit
            sap 1/1/9:1 create
                description "Sample Customer Service SAP"
            exit
            no shutdown
 
Ring node PE- 4: On ring node PE-4 the data VLAN ID is configured as a normal ring data VPLS on ring 2.
*A:PE-4# configure service vpls 11
*A:PE-4>config>service>vpls# info
----------------------------------------------
            description "Data VPLS"
            stp
                shutdown
            exit
            sap 1/1/1:1.11 eth-ring 2 create
                stp
                    shutdown
                exit
            exit
            sap 1/1/3:1.11 eth-ring 2 create
                stp
                    shutdown
                exit
            exit
            sap 1/1/9:1 create
                description "Sample Customer Service SAP"
            exit
            no shutdown
 
All of the SAPs which are configured to use Ethernet rings can be shown. The output below is taken from PE-1, where there are:
*A:PE-1# show service sap-using eth-ring
===============================================================================
Service Access Points (Ethernet Ring)
===============================================================================
SapId              SvcId             Eth-Ring Path Admin Oper  Blocked Control/
                                                   State State         Data
-------------------------------------------------------------------------------
1/1/1:1.1          1                 1        a    Up    Up    No      Ctrl
1/1/2:1.1          1                 1        b    Up    Up    No      Ctrl
1/1/1:2.1          2                 1        a    Up    Up    No      Ctrl
1/1/2:2.1          2                 1        b    Up    Up    No      Ctrl
1/1/3:2.1          2                 2        a    Up    Up    No      Ctrl
1/1/1:1.11         11                1        a    Up    Up    No      Data
1/1/2:1.11         11                1        b    Up    Up    No      Data
1/1/3:2.11         11                2        a    Up    Up    No      Data
-------------------------------------------------------------------------------
Number of SAPs : 8
===============================================================================
*A:PE-1#
 
Statistics are available showing both the CCM and R-APS messages sent and received on a node. An associated clear command is available.
*A:PE-1# show eth-cfm statistics
===============================================================================
ETH-CFM System Statistics
===============================================================================
Rx Count           : 1201               Tx Count           : 1066
Dropped Congestion : 0                  Discarded Error    : 0
AIS Currently Act  : 0                  AIS Currently Fail : 0
===============================================================================
=================================
ETH-CFM System Op-code Statistics
=================================
Op-code      Rx Count   Tx Count
---------------------------------
ccm              1018       1018
...
raps              183         48
...
---------------------------------
Total            1201       1066
=================================
*A:PE-1#
 
To see an example of the console messages on a ring failure, when the unblocked port (1/1/1) on PE-2 is shutdown the following messages are displayed.
*A:PE-2# configure port 1/1/1 shutdown
2 2014/09/19 18:03:01.00 PDT WARNING: SNMP #2004 Base 1/1/1
"Interface 1/1/1 is not operational"
 
3 2014/09/19 18:03:01.00 PDT MINOR: ERING #2001 Base eth-ring-1
"Eth-Ring 1 path 0 changed fwd state to blocked"
 
4 2014/09/19 18:03:01.00 PDT MINOR: ERING #2001 Base eth-ring-1
"Eth-Ring 1 path 1 changed fwd state to unblocked"
*A:PE-2#
5 2014/09/19 18:03:01.01 PDT MAJOR: SVCMGR #2210 Base
"Processing of an access port state change event is finished and the status of a
ll affected SAPs on port 1/1/1 has been updated."
 
6 2014/09/19 18:03:04.33 PDT MINOR: ETH_CFM #2001 Base
"MEP 1/1/2 highest defect is now defRemoteCCM"
*A:PE-2#
 
For troubleshooting, the tools dump eth-ring <ring-index> command displays path information, the internal state of the control protocol, related statistics information and up to the last 16 protocol events (including messages sent and received, and the expiration of timers). An associated clear parameter exists, which clears the event information in this output when the command is entered. The following is an example of the output on PE-1.
*A:PE-1# tools dump eth-ring 1
ringId 1 (Up/Up): numPaths 2 nodeId d8:30:ff:00:00:00
 SubRing: none (interconnect ring 0, propagateTc  No), Cnt 1
  path-a, port 1/1/1 (Up), tag 1.1(Dn) status (Up/Dn/Blk)
      cc (Dn/Up): Cnt 8/7 tm 001 00:27:12.220/001 00:09:23.340
      state: Cnt 9 B/F 001 00:27:12.220/001 00:09:25.410, flag: 0x0
  path-b, port 1/1/2 (Up), tag 1.1(Up) status (Up/Up/Fwd)
      cc (Dn/Up): Cnt 4/4 tm 001 00:07:09.200/001 00:09:23.920
      state: Cnt 8 B/F 001 00:07:09.200/001 00:09:26.410, flag: 0x0
  FsmState=  PROT, Rpl = None, revert = 60 s, guard = 5 ds
    Defects =
    Running Timers = PduReTx
    lastTxPdu = 0xb000 Sf
    path-a Normal, RxId(I)= d8:31:ff:00:00:00, rx(F)= v1-0x00a0 Nr, cmd= None
    path-b Normal, RxId(I)= d8:31:ff:00:00:00, rx= v1-0xb000 Sf, cmd= None
  DebugInfo:  aPathSts 10, bPathSts 7, pm (set/clr) 0/0, txFlush 2
    RxRaps: ok 41 nok 0 self 4, TmrExp - wtr 0(0), grd 5, wtb 0
    Flush: cnt 22 (8/14/0) tm 001 00:27:12.220-001 00:27:12.220 Out/Ack 0/1
    RxRawRaps: aPath 17167 bPath 147 vPath 0
    Now: 001 00:28:16.500 , softReset: No - noTx 0
 
  Seq Event  RxInfo(Path: NodeId-Bytes)
             state:TxInfo (Bytes)            Dir  pA  pB        Time
  === ===== ==============================  ===== === === ================
  015   aDn
              PROT  : 0xb000  Sf            TxF-> Blk Fwd 001 00:07:09.190
  016   bDn
              PROT  : 0xb020  Sf            TxF-> Blk Blk 001 00:07:09.200
  017   pdu B: d8:32:ff:00:00:00-0xb000 Sf
              PROT  : 0xb020  Sf            RxF<- Blk Blk 001 00:09:22.980
  018   pdu A: d8:31:ff:00:00:00-0xb000 Sf
              PROT  : 0xb020  Sf            RxF<- Blk Blk 001 00:09:24.410
  019   pdu B: d8:32:ff:00:00:00-0x0000 Nr
              PROT  : 0xb020  Sf            Rx<-- Blk Blk 001 00:09:25.080
  000   aUp
              PROT  : 0xb060  Sf(DNF)       Tx--> Fwd Blk 001 00:09:25.410
  001   bUp
              PEND-G: 0x0020  Nr            Tx--> Fwd Blk 001 00:09:25.910
  002   pdu A: d8:31:ff:00:00:00-0x0000 Nr
              PEND-G: 0x0020  Nr            Rx<-- Fwd Blk 001 00:09:26.210
  003   pdu B: d8:31:ff:00:00:00-0x0000 Nr
              PEND-G: 0x0020  Nr            Rx<-- Fwd Blk 001 00:09:26.210
  004   pdu A: d8:31:ff:00:00:00-0x0000 Nr
              PEND-G: 0x0020  Nr            Rx<-- Fwd Blk 001 00:09:26.310
  005   pdu B: d8:31:ff:00:00:00-0x0000 Nr
              PEND-G: 0x0020  Nr            Rx<-- Fwd Blk 001 00:09:26.310
  006   pdu A: d8:31:ff:00:00:00-0x0000 Nr
              PEND  : 0x0020  Nr            Rx<-- Fwd Blk 001 00:09:26.410
  007   pdu
              PEND  :                       ----- Fwd Fwd 001 00:09:26.410
  008   pdu B: d8:31:ff:00:00:00-0x0000 Nr
              PEND  :                       Rx<-- Fwd Fwd 001 00:09:26.410
  009   pdu A: d8:31:ff:00:00:00-0x00a0 Nr(RB )
              PEND  :                       RxF<- Fwd Fwd 001 00:10:41.410
  010   pdu
              IDLE  :                       ----- Fwd Fwd 001 00:10:41.410
  011   pdu B: d8:31:ff:00:00:00-0x00a0 Nr(RB )
              IDLE  :                       Rx<-- Fwd Fwd 001 00:10:41.410
  012   pdu B: d8:31:ff:00:00:00-0xb000 Sf
              IDLE  :                       RxF<- Fwd Fwd 001 00:27:09.120
  013   pdu
              PROT  :                       ----- Fwd Fwd 001 00:27:09.120
  014   aDn
              PROT  : 0xb000  Sf            TxF-> Blk Fwd 001 00:27:12.220
 
*A:PE-1#
 
Configuration of a Sub-Ring to a Major Ring with a Non-Virtual Link
The differences from the above virtual link configuration with a non-virtual link for the sub-ring are:
The new configuration of sub-ring 2 on PE-1 is shown below, the configuration on PE-3 is similar.
*A:PE-1# configure eth-ring 2
*A:PE-1>config>eth-ring# info
----------------------------------------------
        description "Ethernet Sub-ring 2 on Ring 1"
        revert-time 60
        sub-ring non-virtual-link
            interconnect ring-id 1
                propagate-topology-change
            exit
        exit
        path a 1/1/3 raps-tag 2.1
            description "Ethernet Ring : 2 Path : pathA"
            eth-cfm
                mep 7 domain 1 association 4
                    ccm-enable
                    control-mep
                    no shutdown
                exit
            exit
            no shutdown
        exit
        no shutdown
 
The configuration of sub-ring 2 on PE-4 is shown below, note the configuration of the sub-ring non-virtual-link.
*A:PE-4# configure eth-ring 2
*A:PE-4>config>eth-ring# info
----------------------------------------------
        description "Ethernet Ring : 2"
        revert-time 60
        rpl-node owner
        sub-ring non-virtual-link
        exit
        path a 1/1/1 raps-tag 2.1
            description "Ethernet Ring : 2 : pathA"
            eth-cfm
                mep 8 domain 1 association 4
                    ccm-enable
                    control-mep
                    no shutdown
                exit
            exit
            no shutdown
        exit
        path b 1/1/3 raps-tag 2.1
            description "Ethernet Ring : 2 : pathB"
            rpl-end
            eth-cfm
                mep 10 domain 1 association 5
                    ccm-enable
                    control-mep
                    no shutdown
                exit
            exit
            no shutdown
        exit
        no shutdown
 
The SAP usage on PE-1 can be seen below with only the control and data SAPs to PE-4 now using sub-ring 2.
*A:PE-1# show service sap-using eth-ring
===============================================================================
Service Access Points (Ethernet Ring)
===============================================================================
SapId              SvcId             Eth-Ring Path Admin Oper  Blocked Control/
                                                   State State         Data
-------------------------------------------------------------------------------
1/1/1:1.1          1                 1        a    Up    Up    No      Ctrl
1/1/2:1.1          1                 1        b    Up    Up    No      Ctrl
1/1/3:2.1          2                 2        a    Up    Up    No      Ctrl
1/1/1:1.11         11                1        a    Up    Up    No      Data
1/1/2:1.11         11                1        b    Up    Up    No      Data
1/1/3:1.11         11                2        a    Up    Up    No      Data
-------------------------------------------------------------------------------
Number of SAPs : 6
===============================================================================
*A:PE-1#
The information relating to sub-ring 2 is shown below and it can be seen that this is now not using a virtual link, but that sub-ring 2 is still connected to major ring 1 and propagation is still enabled from the sub-ring to the major ring. The single ring path (a) is unblocked as the RPL is configured between PE-3 and PE-4.
*A:PE-1# show eth-ring 2
===============================================================================
Ethernet Ring 2 Information
===============================================================================
Description        : Ethernet Sub-ring 2 on Ring 1
Admin State        : Up                 Oper State       : Up
Node ID            : d8:30:ff:00:00:00
Guard Time         :    5 deciseconds   RPL Node         : rplNone
Max Revert Time    :   60 seconds       Time to Revert   : N/A
CCM Hold Down Time :    0 centiseconds  CCM Hold Up Time :   20 deciseconds
Compatible Version : 2
APS Tx PDU         : N/A
Defect Status      :
 
Sub-Ring Type      : nonVirtualLink     Interconnect-ID  : 1
Topology Change    : Propagate
-------------------------------------------------------------------------------
Ethernet Ring Path Summary
-------------------------------------------------------------------------------
Path Port     Raps-Tag     Admin/Oper      Type            Fwd State
-------------------------------------------------------------------------------
  a  1/1/3    2.1             Up/Up        normal          unblocked
  b  -        -                 -/-         -               -
===============================================================================
*A:PE-1#
 
Configuration of a Sub-Ring to a VPLS Service (with a Non-Virtual Link)
Sub-rings can be connected to VPLS services, in which case a virtual link is not used and is not configurable. While similar to the ring interconnect, there are a few differences.
Flush propagation is from the sub-ring to the VPLS, in the same way as it was for the sub-ring to the major ring. The same configuration parameter is used to propagate topology changes, note that in this case LDP flush messages (flush-all-from-me) are sent into the LDP portion of the network to account for ring changes without the need to configure anything in the VPLS service.
As with other rings, until an Ethernet ring instance is attached to the VPLS service, the ring operational status is down and the forwarding status of each port is blocked. This prevents operator from creating a loop by mis-configuration.
The topology for this case is shown in Figure 298. The configuration is very similar to the sub-ring with a non-virtual link described above, but ring 1 is replaced by a VPLS service using LDP signaled mesh SDPs between PE-1, PE-2 and PE-3 to create a fully meshed VPLS service. Note that either spoke or mesh SDPs using LDP could be used for the VPLS, however, only mesh SDPs have been used in this example.
Figure 298: Sub-Ring to VPLS Topology
The differences for the VPLS service connection to the configuration when the sub-ring is connected to a major ring without a virtual link are:
The configuration on PE-1 of the sub-ring 2 is as follows with the interconnect indicating a VPLS service. The configuration on PE-3 is similar.
*A:PE-1>config# eth-ring 2
*A:PE-1>config>eth-ring# info
----------------------------------------------
        description "Ethernet Sub-ring 2 on Ring 1"
        revert-time 60
        sub-ring non-virtual-link
            interconnect vpls
                propagate-topology-change
            exit
        exit
        path a 1/1/3 raps-tag 2.1
            description "Ethernet Ring : 2 Path : pathA"
            eth-cfm
                mep 7 domain 1 association 4
                    ccm-enable
                    control-mep
                    no shutdown
                exit
            exit
            no shutdown
        exit
        no shutdown
 
The configuration of sub-ring 2 on PE-4 is shown below, note the configuration of the sub-ring non-virtual-link.
*A:PE-4# configure eth-ring 2
*A:PE-4>config>eth-ring# info
----------------------------------------------
        description "Ethernet Ring : 2"
        revert-time 60
        rpl-node owner
        sub-ring non-virtual-link
        exit
        path a 1/1/1 raps-tag 2.1
            description "Ethernet Ring : 2 : pathA"
            eth-cfm
                mep 8 domain 1 association 4
                    ccm-enable
                    control-mep
                    no shutdown
                exit
            exit
            no shutdown
        exit
        path b 1/1/3 raps-tag 2.1
            description "Ethernet Ring : 2 : pathB"
            rpl-end
            eth-cfm
                mep 10 domain 1 association 5
                    ccm-enable
                    control-mep
                    no shutdown
                exit
            exit
            no shutdown
        exit
        no shutdown
 
The data service on PE-1 is shown below. The configuration on PE-3 is similar.
*A:PE-1>config>service# vpls 11
*A:PE-1>config>service>vpls# info
----------------------------------------------
            description "Data VPLS"
            stp
                shutdown
            exit
            sap 1/1/3:1.11 eth-ring 2 create
                stp
                    shutdown
                exit
            exit
            sap 1/1/9:1 create
                description "Sample Customer Service SAP"
            exit
            mesh-sdp 2:11 create
                no shutdown
            exit
            mesh-sdp 3:11 create
                no shutdown
            exit
            no shutdown
 
The state of the sub-ring can be seen below and shows the sub-ring is not using a virtual link, is connected to a VPLS service and has propagation of topology change events enabled. As earlier, the single ring path (a) is unblocked as the RPL is configured between PE-3 and PE-4.
 
*A:PE-1# show eth-ring 2
===============================================================================
Ethernet Ring 2 Information
===============================================================================
Description        : Ethernet Sub-ring 2 on Ring 1
Admin State        : Up                 Oper State       : Up
Node ID            : d8:30:ff:00:00:00
Guard Time         :    5 deciseconds   RPL Node         : rplNone
Max Revert Time    :   60 seconds       Time to Revert   : N/A
CCM Hold Down Time :    0 centiseconds  CCM Hold Up Time :   20 deciseconds
Compatible Version : 2
APS Tx PDU         : N/A
Defect Status      :
 
Sub-Ring Type      : nonVirtualLink     Interconnect-ID  : VPLS
Topology Change    : Propagate
-------------------------------------------------------------------------------
Ethernet Ring Path Summary
-------------------------------------------------------------------------------
Path Port     Raps-Tag     Admin/Oper      Type            Fwd State
-------------------------------------------------------------------------------
  a  1/1/3    2.1             Up/Up        normal          unblocked
  b  -        -                 -/-         -               -
===============================================================================
*A:PE-1#
 
In this case, if a topology change event occurs in the sub-ring, an LDP flush all-from-me message is sent by PE-1 and PE-3 to their LDP peers. This can be seen by enabling the following debugging for PE-1, where packets 1 and 2 are the flush messages.
*A:PE-1# debug router ldp peer 192.0.2.2 packet init
*A:PE-1# debug router ldp peer 192.0.2.3 packet init
*A:PE-1#
*A:PE-1# show debug
debug
    router "Base"
        ldp
            peer 192.0.2.2
                event
                exit
                packet
                    init
                exit
            exit
            peer 192.0.2.3
                event
                exit
                packet
                    init
                exit
            exit
        exit
    exit
exit
*A:PE-1#
*A:PE-1# configure port 1/1/3 shutdown
 
2 2014/09/20 11:28:36.01 PDT WARNING: SNMP #2004 Base 1/1/3
"Interface 1/1/3 is not operational"
 
3 2014/09/20 11:28:36.01 PDT MINOR: ERING #2001 Base eth-ring-2
"Eth-Ring 2 path 0 changed fwd state to blocked"
 
1 2014/09/20 11:28:36.01 PDT MINOR: DEBUG #2001 Base LDP
"LDP: LDP
Send Address Withdraw packet (msgId 6799) to 192.0.2.2:0
MAC Flush (All MACs learned from me)
Service FEC PWE3: ENET(5)/11 Group ID = 0 cBit = 0
"
*A:PE-1#
2 2014/09/20 11:28:36.01 PDT MINOR: DEBUG #2001 Base LDP
"LDP: LDP
Send Address Withdraw packet (msgId 183) to 192.0.2.3:0
MAC Flush (All MACs learned from me)
Service FEC PWE3: ENET(5)/11 Group ID = 0 cBit = 0
"
 
4 2014/09/20 11:28:36.03 PDT MAJOR: SVCMGR #2210 Base
"Processing of an access port state change event is finished and the status of a
ll affected SAPs on port 1/1/3 has been updated."
 
5 2014/09/20 11:28:38.99 PDT MINOR: ETH_CFM #2001 Base
"MEP 1/4/7 highest defect is now defRemoteCCM"
 
3 2014/09/20 11:28:39.27 PDT MINOR: DEBUG #2001 Base LDP
"LDP: LDP
Recv Address Withdraw packet (msgId 184) from 192.0.2.3:0
"
 
*A:PE-1# 
 
 
Operational Procedures
Operators may wish to configure rings with or without control over reversion. Reversion can be controlled by timers or the ring can be run without reversion allowing the operator to choose when the ring reverts. To change a ring topology, the manual or force switch command may be used to block a specified ring path. A ring will still address failures when run without reversion but will not automatically revert to the RPL when resources are restored. A clear command can be used to clear the manual or force state of a ring.
The following tools commands are available to control the state of paths on a ring.
tools perform eth-ring clear <ring-index>
tools perform eth-ring force <ring-index> path {a|b}
tools perform eth-ring manual <ring-index> path {a|b}
 
In the output below, path “b” of eth-ring 1 is manually blocked then cleared.
*A:PE-1# show eth-ring 1
===============================================================================
Ethernet Ring 1 Information
===============================================================================
Description        : Ethernet Ring 1
Admin State        : Up                 Oper State       : Up
Node ID            : d8:30:ff:00:00:00
Guard Time         :    5 deciseconds   RPL Node         : rplNone
Max Revert Time    :   60 seconds       Time to Revert   : N/A
CCM Hold Down Time :    0 centiseconds  CCM Hold Up Time :   20 deciseconds
Compatible Version : 2
APS Tx PDU         : N/A
Defect Status      :
 
Sub-Ring Type      : none
-------------------------------------------------------------------------------
Ethernet Ring Path Summary
-------------------------------------------------------------------------------
Path Port     Raps-Tag     Admin/Oper      Type            Fwd State
-------------------------------------------------------------------------------
  a  1/1/1    1.1             Up/Up        normal          unblocked
  b  1/1/2    1.1             Up/Up        normal          unblocked
===============================================================================
*A:PE-1#
*A:PE-1# tools perform eth-ring manual 1 path b
*A:PE-1# show eth-ring 1
===============================================================================
Ethernet Ring 1 Information
===============================================================================
Description        : Ethernet Ring 1
Admin State        : Up                 Oper State       : Up
Node ID            : d8:30:ff:00:00:00
Guard Time         :    5 deciseconds   RPL Node         : rplNone
Max Revert Time    :   60 seconds       Time to Revert   : N/A
CCM Hold Down Time :    0 centiseconds  CCM Hold Up Time :   20 deciseconds
Compatible Version : 2
APS Tx PDU         : Request State: 0x7
                     Sub-Code     : 0x0
                     Status       : 0x20  ( BPR )
                     Node ID      : d8:30:ff:00:00:00
Defect Status      :
 
Sub-Ring Type      : none
-------------------------------------------------------------------------------
Ethernet Ring Path Summary
-------------------------------------------------------------------------------
Path Port     Raps-Tag     Admin/Oper      Type            Fwd State
-------------------------------------------------------------------------------
  a  1/1/1    1.1             Up/Up        normal          unblocked
  b  1/1/2    1.1             Up/Up        normal          blocked
===============================================================================
*A:PE-1#
*A:PE-1# tools perform eth-ring clear 1
*A:PE-1# show eth-ring 1
===============================================================================
Ethernet Ring 1 Information
===============================================================================
Description        : Ethernet Ring 1
Admin State        : Up                 Oper State       : Up
Node ID            : d8:30:ff:00:00:00
Guard Time         :    5 deciseconds   RPL Node         : rplNone
Max Revert Time    :   60 seconds       Time to Revert   : N/A
CCM Hold Down Time :    0 centiseconds  CCM Hold Up Time :   20 deciseconds
Compatible Version : 2
APS Tx PDU         : N/A
Defect Status      :
 
Sub-Ring Type      : none
-------------------------------------------------------------------------------
Ethernet Ring Path Summary
-------------------------------------------------------------------------------
Path Port     Raps-Tag     Admin/Oper      Type            Fwd State
-------------------------------------------------------------------------------
  a  1/1/1    1.1             Up/Up        normal          unblocked
  b  1/1/2    1.1             Up/Up        normal          unblocked
===============================================================================
*A:PE-1#
 
Both the manual and force command block the path specified, however, the manual command fails if there is an existing forced switch or signal fail event in the ring, as seen below. The force command will block the port regardless of any existing ring state and there can be multiple force states simultaneously on a ring on different nodes.
*A:PE-1# tools perform eth-ring manual 1 path b
INFO: ERMGR #1001 Not permitted - The switch command is not compatible to the current state (MS), effective priority (MS) or rpl-node type (None)
 
Conclusion
Ethernet Ring APS provides an optimal solution for designing native Ethernet services with ring topology. With sub-rings, both multiple rings and access rings increase the versatility of G.8032. G.8032 has been expanded to more of the SR/ESS platforms by allowing R-APS with slower MEPs (including CCMs intervals of 1 second). This protocol provides simple configuration, operation and guaranteed fast protection time. The implementation also has a flexible encapsulation that allows dot1q, QinQ or PBB for the ring traffic. It could be utilized on various services such as mobile backhaul, business VPN access, aggregation and core.