Did you know? You can monitor MetroE links with CA Spectrum’s network monitoring application.
Metro Ethernet (“MetroE”) is a networking technology often used to connect customer networks. Because it is based on the Ethernet standard, MetroE offers lower cost and simpler implementation than other networking technologies of equivalent bandwidth.
As a network engineer with experience with several network monitoring applications, I’ve found that many of my colleagues and customers in network management are less familiar with MetroE than other technologies. Cisco has done a fabulous job of providing more information that I have leveraged below (and referenced at the end). I also include a brief section on how CA Spectrum can provide visibility into this network service option.
Figure 1: A simplified view of a point-to-point MetroE service
The two customer sites above are connected by means of a Metro Ethernet Network (MEN) maintained by a service provider. A CPE (customer premise equipment) device in each customer site is defined as the customer-side interface to the MEN. A UNI (user network interface) is defined as the dividing line between each customer site and the MEN. Each UNI is dedicated to a single customer site. An interface on a PE (provider edge) device is designated as the UNI.
Within the MEN, an EVC (Ethernet virtual circuit) provides connectivity between the UNIs. The EVC ensures that the UNIs communicate only with each other and with no other devices in the MEN.
MetroE Management Options for Network Monitoring Applications
1. Ethernet Local Management Interface (E-LMI): (Customer Focused) Similar to its counterpart in Frame Relay, this protocol was developed by the Metro Ethernet Forum. It operates on the link between the CE device and the PE device. E-LMI automates provisioning of the CE device. On-going fault notification (as detected by 802.1ag) to the CE device is most important to the enterprise customer. See Ethernet LMI below for an example of an Ethernet sub-interface state change to UP/DOWN by E-LMI. As with traditional Frame Relay WANs, the Layer 3 routing protocol also detects and routes around the failure. SNMP traps on the branch CE router, a link up/down SNMP trap, and syslog message are available to the campus network monitoring software tool.
2. IEEE 802.1ag Connectivity Fault Management (CFM): (Provider Focused) Provides “service” management. The customer purchases end-to-end connectivity (via EVC) through the service provider network, and CFM identifies and notifies the service provider of failed connections. At the user-facing PE, the CFM and E-LMI functions interoperate (communicate) to provide a true end-to-end circuit validation. The enterprise customer needs to be aware only that IEEE 802.1ag CFM is an available feature to the service provider because the customer does not directly interact or require any CFM configuration in the PE device.
3. Link Layer OAM (IEEE 802.3ah OAM): (Provider Focused) Provides link-level Ethernet OAM (operations, administration and management) and operates on a link-by-link basis. This protocol addresses discovery, link monitoring, remote fault detection, and remote loopback. Link Layer OAM interworks and is relayed to CFM on the same device. CFM can then notify remote devices of the localized fault, as previously described. As with CFM, no customer CE configuration is necessary.
4. IP SLA for Metro Ethernet: (Provider Focused) The IP SLAs for MetroE feature provides the capability to gather statistical measurements by sending and receiving Ethernet data frames between Ethernet CFM maintenance endpoints (MEPs). The performance metrics for IP SLAs Ethernet operations are measured between a source MEP and a destination MEP. Unlike existing IP SLAs operations that provide performance metrics for the IP layer, the IP SLAs Ethernet operation provides performance metrics for Layer 2.
You can manually configure individual Ethernet ping or Ethernet jitter operations by specifying the destination MEP identification number, name of the maintenance domain, and EVC or VLAN identifier or port level option. You also have the option to configure an IP SLAs auto Ethernet operation (ping or jitter) that will query the Ethernet CFM database for all maintenance endpoints in a given maintenance domain and EVC or VLAN.
When an IP SLAs auto Ethernet operation is configured, individual Ethernet ping or Ethernet jitter operations are automatically created based on the MEPs that were discovered. A notification mechanism exists between the IP SLAs and Ethernet CFM subsystems to facilitate the automatic creation of Ethernet ping or Ethernet jitter operations for applicable MEPs that are added to a given maintenance domain and EVC or VLAN while an auto Ethernet operation is running. The IP SLAs for Metro-Ethernet feature supports multi-operation scheduling of IP SLAs operations and proactive threshold violation monitoring through SNMP trap notifications and syslog messages.
Introduction to Ethernet Local Management Interface
Ethernet LMI is an Ethernet layer OAM protocol between a CE device and the PE in large Ethernet MAN and WANs. It provides information that enables service providers to auto configure CE devices with service parameters and parameter changes from a user provider edge (UPE) device. The E-LMI protocol also provides the means for notification of the status of an EVC.
In particular, the E-LMI protocol includes the following procedures:
1. Notification to the CE device of the addition of an EVC: An example use case of this is if a new branch office is connected to headquarters. With the use of E-LMI at the UNIs, the respective CPEs are informed of the availability of a new EVC once the service provider turns on the service. In particular, the service endpoints are notified of the corresponding VLAN ID to be used by a given service (a.k.a. C-VLAN to EVC map attribute).
2. Notification to the CE device of the deletion of an EVC: This is very similar to the previous examples, except the EVC is being removed.
3. Notification to the CE device of the availability (active/partially active) or unavailability (inactive) state of a configured EVC: The primary benefit is that the CE device can take some corrective action, such as rerouting traffic to a different EVC or other WAN service, when informed that an EVC has become inactive.
4. Notification to the CE device of the availability of the Remote UNI: As in the previous case, the CE device can take some corrective action, such as rerouting traffic to a different EVC or other WAN service, when informed that the remote UNI is down.
5. Communication of UNI and EVC attributes to the CE device:
– EVC identification – The network informs the CE device as to which VLAN ID is used to identify each EVC (C-VLAN to EVC map). This removes the possibility of a VLAN mismatch between the SP and customer’s equipment.
– Remote UNI identification – The network informs the CE device of the names of the remote UNIs associated to a given service. This can be used to confirm that the right endpoints have been connected by an EVC.
– Bandwidth profiles – The advantage of this is that if the enterprise has subscribed to a 50-Mbps service, then the CE device can automatically configure itself to shape the egress traffic to 50 Mbps on the WAN interface. By shaping to 50 Mbps rather than having the service provider police a 100-Mbps stream down to 50 Mbps, enterprise customers will reduce the number of dropped packets and increase the throughput they receive.
This example shows the only command necessary to configure E-LMI on the CUSTOMER CE device. The command globally enables E-LMI, but you can also enable it only on a specific interface.
Switch# config t Switch(config)# ethernet lmi global Switch(config)# exit
Common Use Cases for MetroE:
Provider Use Case 1
Today, a service provider may use SNMP to periodically poll devices for statistics and receive SNMP traps when faults occur. However, when a fault occurs, the service provider has no idea which customers are impacted. By implementing 802.1ag CFM, the continuity-check mechanism will determine which EVCs are impacted so the service provider knows exactly which customer services are down. The operator can verify the loss of connectivity using CFM loopback (ping), and localize the connection failure using CFM link trace. The problem can then be further diagnosed and remedied. Finally, CFM loopback may be used to verify that the remedial action has succeeded and that the service has been re-established.
The use of a network monitoring application can automate this process, allowing the operator to automatically:
1. Detect the problem by receiving the CFM traps and correlating the traps from each endpoint into one alarm for the connectivity problem
2. Localize and diagnose the problem
3. Repair any configuration issues, or reroute around the problem
4. Verify that service has been re-established
Obviously the primary advantage of this automation is the faster time to repair and the reduced need for human intervention. It is important to note, however, that 802.1ag is not a resiliency mechanism like Rapid Spanning Tree Protocol (IEEE 802.1w), which provides sub-second failover. Rather, it is a protocol that provides consistent monitoring of EVCs and the troubleshooting tools to isolate faults and verify connectivity.
Provider/Customer Use Case 2
A service provider today is likely deploying Carrier Ethernet services without support of a protocol that could automatically convey EVC attribute information to the CE. The most basic problem like a VLAN ID mismatch between customer and service provider may be very involved and require troubleshooting from both entities. With support of E-LMI across devices in a given UNI, EVC information can be propagatde to the customer such that the kind of problems mentioned earlier could be completely detected and corrected by the customer without operations involvement providing an immediate OPEX benefit to the service provider.
Provider/Customer Use Case 3
One of today’s challenges faced in the deployment of Carrier Ethernet services is the potential issue that occurs when a failure of an EVC or a remote UNI does not translate into a link status event at the CE. One of the options to handle this scenario is to communicate to the CE the status of an EVC and/or remote UNI with E-LMI between the customer and the service provider network. E-LMI acts as a messenger but does not control or determine the state of a connection. However, E-LMI can learn this information from the knowledge collected by other OAM protocols like IEEE 802.1ag. As such, IEEE 802.1ag CFM to E-LMI OAM Inter-working provides a viable alternative to achieve end-to-end CE fault notification. Upon notification by the network that a given EVC is inactive, the CE can re-route traffic onto another EVC that is acting as the backup path.
Monitoring MetroE with CA Spectrum Network Monitoring Application
From Cisco, there are two MIBs that apply to MetroE monitoring. Those are the CISCO-ETHER-CFM-MIB : Cisco Ethernet Connectivity Fault Management (CFM) MIB and the CISCO-EVC-MIB : Cisco Ethernet Virtual Connection MIB. The CISCO-ETHER-CFM-MIB is provider focused and the CISCO-EVC-MIB is customer focused.
CA Spectrum includes the CISCO-ETHER-CFM-MIB out of the box but does not have any alerts configured for the traps that maybe received as seen by the following screenshot. Events/Alerts may need to be configured for some/all of the traps listed below.
Figure 2: CA Spectrum trap support for Cisco MIBs
In the following screenshot you can see the many additional attributes that are available for polling and informational display. You may also want to set a CA Spectrum SpectroWatch for some of the attributes like UNI Status.
Figure 3: Additional Cisco metrics that can be monitored by CA Spectrum
CA Spectrum does NOT include the CISCO-EVC-MIB support out of the box. However, in the following screenshots you can see the information that is available both from traps and polled/informational display when the MIB is imported.
Events/Alerts will need to be configured for some/all of the traps listed below. You may also want to set a CA Spectrum SpectroWatch for some of the attributes available.
Figure 4: Information that is available both from traps and polled/informational display when the MIB is imported into CA Spectrum
Informational (does not display all of the attributes available):
Figure 5: Information that is available both from traps and polled/informational display when the MIB is imported into CA Spectrum
MetroE is a common connectivity solution for organizations that have campus-like settings – universities, hospitals, manufacturers, etc. CA Spectrum is an ideal network management solution for monitoring these connections and providing visibility into the performance and availability of the network service.
- Cisco, Ethernet Access for Next Generation Metro and Wide Area Networks, Cisco Validated Design – July 17, 2008
- Cisco, Catalyst 3750 Metro Switch Software Configuration Guide – Chapter 36 Configuring Ethernet CFM and E-LMI
- Cisco, Enabling Ethernet Local Management Interface
- Cisco, Cisco 3900 Series, Cisco 2900 Series, and Cisco 1900 Series Integrated Services Routers Generation 2 Software Configuration Guide, Configuring Ethernet CFM and Y.1731 Performance Monitoring on Layer 3 Interfaces
- Cisco, Configuring Cisco IOS IP SLAs for Metro-Ethernet