Layer 3 Protocols over ATM and LAN Emulation

In a typical backbone implementation of ATM, the ATM network must carry traffic that is connectionless and in a network layer protocol format, such as IP. IP data, for example, is formatted in packets, not cells; IP is typically carried over a broadcast medium such as Ethernet or Token Ring and uses IP rather than ATM addresses. The requirements for transport of IP, or other layer 3 protocols, are therefore fundamentally different from ATM.

Two main problems must be solved in carrying network layer protocol traffic across the ATM network:

• Packet encapsulation---how the network layer protocol packets are packaged into ATM cells

• Address resolution---how an ATM network device finds the location of an IP address and connects to it

• "Classical IP and ARP over ATM" (RFC 1577)---defines an application of classical IP in an ATM network environment using SVCs and PVCs and specifies mechanisms for address resolution and discovery.

• "Multiprotocol Encapsulation over ATM Adaptation Layer 5" (RFC 1483)---defines how various types of PDUs are encapsulated for transport over ATM.

RFC 1577 specifies that address resolution be accomplished by the ATMARP server, a centralized server that maintains a table of IP addresses to ATM addresses. The ARP server maintains this table for a single IP subnet, and any client that needs to communicate with another client can query the ARP server to get that device's ATM address and directly set up a connection to it.

How It Works

The following steps describe the process whereby a classical IP-over-ATM connection is set up between ATM switch router client A and client B.

Step 1 The initial IP packet sent by client A triggers a request to the ARP server to look up the IP address and the corresponding ATM address of client B in the ARP table.

For each packet with an unknown IP address, the client sends an ATMARP request to the ARP server. Until that address is resolved, any IP packet routed to the ATM interface causes the client to send another ATMARP request.

Step 2 The ARP server sends back a response to client A with the matching ATM address.

Step 3 Client A uses the ATM address it just obtained from the ARP server to set up an SVC directly to client B.

Step 4 When client B replies with an IP packet to client A, it also triggers a query to the ARP server.

When client B receives the ATM address for client A, it usually discovers it already has a call set up to client A's ATM address and does not set up another call.

Step 5 Once the connection is known to both clients, they communicate directly over the SVC.

In Cisco's implementation, the ATMARP client tries to maintain a connection to the ATMARP server. The ATMARP server can tear down the connection, but the client attempts once each minute to bring the connection back up. No error message is generated for a failed connection, but the client will not route packets until the ATMARP server is connected and translates IP network addresses.

The ATM switch router can be configured as an ATMARP client to work with any ATMARP server conforming to RFC 1577. Alternatively, one of the ATM switch routers in an LIS can be configured to act as the ATMARP server itself. In that case, it automatically acts as a client as well.

• Logical Link Control (LLC)/Subnetwork Access Protocol (SNAP) encapsulation---in this method, multiple protocol types can be carried across a single connection with the type of encapsulated packet identified by a standard LLC/SNAP header.

• Virtual connection multiplexing---in this method, only a single protocol is carried across an ATM connection, with the type of protocol implicitly identified at connection setup.

LLC encapsulation is provided to support routed and bridged protocols. In this encapsulation format, PDUs from multiple protocols can be carried over the same virtual connection. The type of protocol is indicated in the packet's SNAP header. By contrast, the virtual connection multiplexing method allows for transport of just one protocol per virtual connection.

The following steps occur when a connection between switch A and switch B needs to be set up to forward an IP packet:

Step 1 An IP packet with destination 123.233.45.3 arrives at switch A.

Step 2 Switch A finds the destination IP address in its map list.

Step 3 Using a statically configured map list, switch A identifies the ATM address corresponding to the next-hop IP address (for SVCs) or the VPI/VCI value corresponding to the next-hop IP address (for PVCs).

Step 4 If SVCs are used, signaling sets up a virtual connection to the destination ATM address. If PVCs are used, the connection follows the statically configured path of the virtual connection.

Step 5 The encapsulated packet is forwarded over the ATM virtual connection.

Advantages

Potential advantages of PVCs with InATMARP include the following:

• Many vendors support the InATMARP functions, making it widely interoperable.

• For interconnecting IP subnets, it is less complex than LANE.

Limitations

Potential limitations of PVCs with InATMARP include the following:

• No multicast capability is provided.

• Only the IP network layer protocol is supported.

• This implementation cannot take advantage of the router's traffic shaping capability; therefore, this implementation might not be a good choice for sending traffic through a public network.

• Because address resolution is limited to a single hop and inter-LIS traffic must traverse a router, multiple hops are required for communication between different LISs. This can increase the load on the network resources, particularly as it results in repeated SARs.

• Because PVCs are manually configured, the complexity of configuration and the possibility of error increase with the number of devices you have to configure.

Advantages

Potential advantages of PVCs with static mapping include the following:

• This implementation can support different Layer 3 protocols.

• Configuration is simple for a few nodes.

• Encapsulations is supported by many vendors.

Limitations

Potential limitations of this implementation include the following:

• Scalability is a problem, because the number of PVCs to configure grows exponentially with the addition of nodes.

• It is strongly recommended not to use OSPF as routing protocol if the network is a meshed one that requires multicasting.

• With PVCs, there is no dynamic reaction to ATM failures; a possible workaround is to use soft PVCs.

• Bridged and routed 1483 cannot exist on the same PVC, unless you enable Integrated Routing and Bridging (IRB) on the router.

Advantages

Potential advantages of SVCs with static mapping include the following:

• The implementation can support different Layer 3 protocols.

• Encapsulation is supported by many vendors.

• Multicasting and broadcasting are supported on static map subinterfaces that are of type multipoint. Multipoint routing in the form of PIM is also supported in this environment.

Limitations

Potential limitations of SVCs with static mapping include the following:

• It is strongly recommended not to use OSPF as a routing protocol if the network is a meshed one that requires multicasting.

• The signaling complexity inherent in using SVCs can be a troubleshooting liability.

In the WAN, this implementation might have the following additional limitations:

• There can be problems with signaling of SVCs when connecting to a service provider's address space. A possible alternative is to use SVCs inside a VP tunnel.

• There is no point-to-multipoint support.

Some risk occurs in using the ATM network itself to provide network management connectivity. However, this liability can be mitigated if you have redundant links and multiple paths. For this reason, implementations with SVCs might be preferable.

The following types of devices can be used to support LANE services:

• Directly attached ATM hosts with ATM NICs

• Layer 2 devices, such as switches with ATM interfaces or the ATM switch routers

• Layer 3 devices, such as routers with ATM interfaces

The LANE protocol defines mechanisms for emulating either an IEEE 802.3 Ethernet or an 802.5 Token Ring LAN. Specifically, LAN broadcasts are emulated as ATM unicasts. The current LANE protocol does not define a separate encapsulation for Fiber Distributed Data Interface (FDDI). (FDDI packets must be mapped into either Ethernet or Token Ring emulated LANs by using existing translational bridging techniques.) Fast Ethernet (100BaseT) and IEEE 802.12 (100VG-AnyLAN) both can be mapped unchanged because they use the same packet formats.

LANE defines a service interface for network layer protocols that is identical to existing MAC layers. No changes are required to existing upper layer protocols and applications. However, LANE does not emulate every particular physical or data-link characteristic. For example, it does not support carrier sense multiple access collision detect (CSMA/CD) for either Ethernet or Token Ring. LANE clients on an ATM switch router only support the IP protocol.

ATM NICs implement the LANE protocol and interface to the ATM network while presenting the current LAN service interface to the higher-level protocol drivers within the end system. The network-layer protocols on the end system continue to communicate as if they were on a known LAN, by using known procedures. However, they are able to take advantage of most of the advanced services of the ATM network.

---

Note In Cisco's LANE implementation, the LES and BUS are combined.

---

A VLAN is identified by a number, which is only significant to the Catalyst family of switches. On an ATM network, an emulated LAN is designated by a name. Therefore, the VLAN number must be mapped to the emulated LAN on the Catalyst switch. To create a VLAN that spans multiple Catalyst switches on an ATM network, you must assign the VLAN on each Catalyst switch to the same emulated LAN. Members of two or more different emulated LANs can communicate only through a router, whether they are on the same or different Catalyst switches.

Control direct VCC---The LEC, as part of its initialization, sets up a bidirectional point-to-point VCC to the LES for sending or receiving control traffic. The LEC is required to accept control traffic from the LES through this VCC and must maintain the VCC while participating as a member of the emulated LAN.

Control distribute VCC---The LES can optionally set up a unidirectional VCC back to the LEC for distributing control traffic. Whenever an LES cannot resolve an LE_ARP request from a LEC, it forwards the request out the control distribute VCC to all of the clients in the emulated LAN. The control distribute VCC enables information from the LES to be received whenever a new MAC address joins the LAN or whenever the LES cannot resolve an LE_ARP request.

Data direct VCC---Once an ATM address has been resolved by a LEC, this bidirectional point-to-point VCC is set up between clients that want to exchange unicast data traffic. Most client traffic travels via these VCCs.

Multicast send VCC---The LEC sets up a unidirectional point-to-point VCC to the BUS. This VCC is used by the LEC to send multicast traffic to the BUS for forwarding out the multicast forward VCC. The LEC also sends unicast data on this VCC until it resolves the ATM address of a destination.

Multicast forward VCC---The BUS sets up a unidirectional VCC to the LECs for distributing data from the BUS. This can either be a unidirectional point-to-point or unidirectional point-to-multipoint VCC. Data sent by a LEC over the multicast send VCC is forwarded to all LECs over the multicast forward VCC.

Configure direct VCC---This is a transient VCC set up by the LEC to the LECS for the purpose of obtaining the ATM address of the LES that controls the particular LAN the LEC wishes to join.

When an LEC first joins an emulated LAN, its LE_ARP table has no dynamic entries, and the LEC has no information about destinations on or behind its emulated LAN. To learn about a destination when a packet is to be sent, the LEC begins the following process to find the ATM address corresponding to the known MAC address:

To build a LANE connection from a PC to an ATM attached LEC, the LANE components perform the following steps:

Step 1 PC---Before starting the file transfer the PC must locate the file server on the network. To find the file server's MAC address, the PC broadcasts an ARP request with the file server's IP address.

Step 2 LEC on Catalyst 5000 switch---Receives ARP requests and forwards to the BUS configured on the ATM switch router.

Step 3 BUS on ATM switch router---Broadcasts the ARP request to all members of the emulated LAN using a point-to-multipoint VCC.

Step 4 LEC on file server---Receives the ARP request, recognizes its own IP address and responds with an ARP reply back to the BUS in the ATM switch router.

Step 5 BUS on ATM switch router---Forwards the ARP reply to the Catalyst 5000 switch.

Step 6 LEC on Catalyst 5000 switch---Forwards the ARP reply to the originating PC.

Step 7 PC---Starts sending the packets of the file transfer using the multicast send VCC connection from the Catalyst 5000 to the BUS on the ATM switch router, which forwards the packets over the multicast forward VCC to the file server. This gets the data moving in the interim until the data direct VCC is set up.

Step 8 LEC on file server---Starts to set up the direct VCC to the Catalyst 5000 switch using an LE_ARP request to the LES. This request asks for the ATM address that corresponds to the PC's MAC address. (The PC's MAC address was obtained from the original ARP request in Step 4.)

Step 9 LES on ATM switch router---Looks up the PC's MAC address in its look-up table and multicasts the LE_ARP request to all LECs.

Step 10 LEC on Catalyst 5000 switch---Receives the LE_ARP request and finds the PC's MAC address in its look-up table. (It learned the PC's MAC address in Step 2.)

Step 11 LEC on Catalyst 5000 switch---Adds its own ATM address into the LE_ARP request and returns it to the LES in the ATM switch router.

Step 12 LES on ATM switch router---Multicasts the LE_ARP reply to all members of the emulated LAN, including the file server.

Step 13 LEC on File Server---Receives the LE_ARP as part of the emulated LAN and signals for a data direct VCC to the Catalyst 5000 using the ATM address.

Step 14 ATM switch router---Sets up a data direct VCC between the Catalyst 5000 and the file server.

Step 15 PC---The file transfers directly from the PC using the direct data VCC from the Catalyst 5000 to the ATM-attached file server.

• The prefix field is the same for all LANE components on the ATM switch router; the prefix indicates the identity of the directly attached ATM switch router. The prefix value must be configured, either manually or by autoconfiguration, on the ATM switch router. In most cases, the autoconfigured prefix is used.

• The ESI field value assigned to every LEC on the interface is the first in the pool of MAC addresses assigned to the interface.

• The ESI field value assigned to every LES on the interface is the second in the pool of MAC addresses.

• The ESI field value assigned to the BUS on the interface is the third in the pool of MAC addresses.

• The ESI field value assigned to the LECS is the fourth in the pool of MAC addresses.

• The selector field value is set to the subinterface number of the LANE component---except for the LECS, which has a selector field value of 0.

The following example shows the autoconfigured ATM addresses for LANE components. The prefix is the default ILMI prefix:

Switch> show lane default
interface ATM2/0/0:
LANE Client:        47.00918100000000E04FACB401.00400B0A2A82.**
LANE Server:        47.00918100000000E04FACB401.00400B0A2A83.**
LANE Bus:           47.00918100000000E04FACB401.00400B0A2A84.**
LANE Config Server: 47.00918100000000E04FACB401.00400B0A2A85.00
note: ** is the subinterface number byte in hex
 

The syntax of address templates, the use of address templates, and the use of wildcard characters within an address template for LANE are very similar to the address templates of International Organization for Standardization of Connectionless Network Service (ISO CLNS). Refer to the ATM Switch Router Software Configuration Guide for details on using ATM address templates.

• The LECS always runs on the major interface.

The assignment of any other component to the major interface is identical to assigning that component to the 0 subinterface.

• The LES and the LEC of the same emulated LAN can be configured on the same subinterface.

• Clients of two different emulated LANs cannot be configured on the same subinterface.

• Servers of two different emulated LANs cannot be configured on the same subinterface.

At least one ATM switch router is required to run LANE. For example, you cannot run LANE on routers connected back-to-back.

• Supports both Ethernet and Token Ring legacy LANs without modification to upper layer protocols or applications.

• Provides multicast and broadcast for support of LAN applications that require this capability.

• Design allows for relatively easy scaling.

• Compared to RFC 1577 and RFC 1483 protocols, LANE is relatively complex to configure.

• Redundancy is problematic across vendors. Even with redundancy, there is a reaction time to switchover.

• Can be difficult to troubleshoot.

• Provides no QoS support.

• The load on LANE services, such as the number of nodes in an emulated LAN and the total number of emulated LANS, should be monitored. Extremely heavy demand can degrade network performance.

You must enter the ATM address of the LECS into the ATM switch routers (and other LANE client devices in the LANE cloud) and save it permanently, so that the value is not lost when the device is reset or powered off. The LECS address can be specified for the entire ATM switch router, or per port.

Step 4 Set up the LECS database.

After you have determined all LESs, BUSs, and LECs on all ATM subinterfaces on all routers and switches that will participate in LANE, and have displayed their ATM addresses, you can use the information to populate the LECS database.

You can set up a default emulated LAN, whether or not you set up any other emulated LANs. You can also set up some emulated LANs with restricted membership and others with unrestricted membership.

---

Note For fault tolerance, multiple LANE services and servers can be assigned to the emulated LAN. This requires the use of Cisco ATM switch routers and ATM edge devices end-to-end.

---

Step 5 Enable the LECS.

After you create the database entries appropriate to the type and to the membership conditions of the emulated LANs, you enable the configuration server on the selected ATM interface, router, or switch, and specify that the LECS ATM address is to be computed automatically.

---

Note Every LANE cloud (one or multiple emulated LANs) must have at least one LECS.

---

Step 6 Set up the LES/BUS.

For one default emulated LAN, you must set up one set of servers: one as a primary server and the rest as backup servers for the same emulated LAN. For multiple emulated LANs, you can set up servers for another emulated LAN on a different subinterface on the same interface of this router or switch, or you can place the servers on a different device.

---

Note When you set up an LES/BUS pair, you can combine them with a client on the same subinterface, a client on a different subinterface, or no client at all on the device.

---

Each emulated LAN is a separate subnetwork. Make sure that the clients of the same emulated LAN are assigned protocol addresses on the same subnetwork, and that clients of different emulated LANs are assigned protocol addresses on different subnetworks.

Where you put the clients is important, because any router with clients for multiple emulated LANs can route frames between those emulated LANs.

On any given router or switch, you can set up one client for one emulated LAN or multiple clients for multiple emulated LANs. You can set up a client for a given emulated LAN on any routers you select to participate in that emulated LAN. Any router with clients for multiple emulated LANs can route packets among those emulated LANs.

• The component and interface where the LECS will be located.

• The component, interface, and subinterface where the LES/BUS for each emulated LAN will be located. Each emulated LAN can have multiple servers for fault-tolerant operation.

• The component, interfaces, and subinterfaces where the clients for each emulated LAN will be located.

• The component and database name of the default database.

• The name of the default emulated LAN (optional).

• The names of the emulated LANs that have unrestricted membership.

• The names of the emulated LANs that have restricted membership.

The last three items in this list are very important; they determine how you set up each emulated LAN in the LECS database.

Example LANE Plan and Worksheet

• LECS redundancy---uses a master-backup scheme for a given set of emulated LANS.

  • There is one master LECS; there can be multiple backup LECS.

  • The databases of all LECS must be identical; that is, they must include the same LES addresses and corresponding ELAN names. The LECS turns on server redundancy by adjusting its database to accommodate multiple LES addresses for a particular emulated LAN. The additional servers provide backup for that emulated LAN.

  • LECs maintain multiple LECS addresses via ILMI; if the master LECS fails, a backup responds. When a LECS switches over, no previously joined clients are affected.

• LES/BUS redundancy---uses a master-backup scheme for a given emulated LAN.

  • The LECS always keeps an open VCC with each LES/BUS. In the case of an LES/BUS failure, the LECS establishes a connection with the next LES/BUS serving that emulated LAN.

  • When a LES/BUS switches over, momentary loss of clients occurs if any of the control VCCs go down. They then reinitialize and are all transferred to the new LES/BUS.

Configuration Overview

Step 1 Configure LES/BUS pairs on the switches and routers where you want to place these servers. There is no limit on the number of LES/BUS pairs you can configure per emulated LAN.

• Up to 16 LECS addresses can be handled by the LANE subsystem.

• There is no limit to the number of LESs that can be defined per emulated LAN.

• When a LECS switches over, no previously joined clients are affected.

• When a LES/BUS switches over, clients are momentarily lost until they are transferred to the new LES/BUS.

• LECSs come up automatically as masters until a higher-level LECS tells them otherwise.

• By default, when a higher-priority LES comes online, it does not preempt the current LES on the same emulated LAN. However, a higher-priority LES configured as preemptable does bump the current LES on the same emulated LAN when the LES comes online. In that case, there might be some changing of clients from one LES to another after a powerup, depending on the order of the LESs coming up. Changing should settle after the last highest priority LES comes up.

• If none of the specified LESs is up or connected to the master LECS, and more than one LES is defined for an emulated LAN, a configuration request for that specific emulated LAN is rejected by the LECS.

• Changes made to the list of LECS addresses on ATM switch routers can take up to a minute to propagate through the network. Changes made to the configuration database regarding LES addresses take effect almost immediately.

• Overriding any of the LECS addresses can cause SSRP to become nonoperational. To avoid affecting the fault-tolerant operation, do not override any LECS, LES, or BUS addresses.

• If an underlying ATM network failure occurs, there might be multiple master LECSs and multiple active LESs for the same emulated LAN. This situation creates a "partitioned" network. The clients continue to operate normally, but transmission between different partitions of the network is not possible. When the network break is repaired, the system recovers. This, however, is not a problem particular to LANE but would occur whenever a breakage occurs in the ATM network.

• Server redundancy guards against the failure of the hardware on which server components are running. This includes all the ATM interface cards in our routers and Catalyst switches. Fault tolerance is not effective for ATM network as a whole or for other switch or LAN failures.

Multiprotocol over ATM (MPOA) relieves the router bottleneck for inter-ELAN traffic by adding "cut-through" routing to existing LANE capability. (Intra-ELAN traffic continues to be serviced by LANE alone.) With cut-through routing, based on the Next Hop Resolution Protocol (NHRP), inter-ELAN traffic with significant flow (described later in this section) can avoid going through the router, a normal requirement of LANE, and can be switched via a direct connection through the ATM network.

In addition to the performance enhancement MPOA provides, there is the additional benefit of QoS support for features such as packetized video. IP's Resource Reservation Protocol (RSVP) parameters can be mapped to ATM's QoS parameters to take advantage of ATM's traffic contract.

An MPOA-enabled network uses the following components:

• Routers---run their conventional routing and discovery protocols, while also providing multicast forwarding between VLANs and forwarding on behalf of LANE-only clients. Routers can also forward short-lived "flows," such as DNS or SMTP queries from MPOA clients, also called MPCs.

• Edge devices---forward packets between an ATM backbone and LANs. Edge devices can serve as an MPOA client, as can a LEC. ATM-attached hosts or servers can also contain an MPOA client. The edge device is usually a LAN switch with a LANE interface.

• MPOA server (MPS)---is responsible for responding to queries from the MPOA client to resolve IP-to-ATM addresses.

The following steps describe the stages of an MPOA connection between ELAN 1 and ELAN 4:

Step 1 The first time traffic needs to be forwarded from the ingress MPOA client to the egress MPOA client, it is forwarded over the routers. This method ensures that both classical bridging and inter-VLAN routing operations are preserved and are always available.

Step 2 The MPOA client determines where there is a "significant flow." Significant flow means that a certain number of packets (ATM Forum default is 10) are sent to the same destination in a given time (ATM Forum default is 1 second).

Step 3 If a significant flow is detected, an MPOA query is initiated. To set up a direct "cut-through" connection, the edge devices (or MPOA clients) must obtain the ATM address of the exit point that corresponds to the respective Layer 3 destination address. To obtain this information, the MPOA client sends an MPOA query to the MPOA server at each hop. Meanwhile, the MPOA client continues sending data traffic to the default forwarder (the router) while it waits for a reply. Query between the MPOA servers is NHRP-based.

Step 4 Before the MPOA server at the egress router replies, it performs a cache imposition information exchange with the edge device where the destination is attached. A cache imposition helps to ensure reliable operation, validates forwarding information, and, optimally, provides information used to increase forwarding performance in the MPOA clients.

Step 5 The MPOA server can then respond to the MPOA query with the ATM address of the exit point or ATM-attached host used to reach the destination Layer 3 address.

Step 6 When the reply arrives at the source MPOA client, it sets up a direct inter-ELAN cut-through ATM connection.

• Like LANE, on which it is based, requires no modification to upper layer applications.

• Reduces latency caused by multiple router hops for inter-ELAN traffic.

• Provides for QoS support via RSVP.

• Can use Cisco SSRP for LANE redundancy with added redundancy at the router level using the Hot Standby Router Protocol (HSRP).

• Can be implemented incrementally, adding MPOA in areas where it is needed. The entire network does not have to be upgraded at the same time.

• Like LANE, is appropriate only for LAN, not WAN.

• Supports only IP unicast.