Table of Contents
Network Clock Synchronization
The term clocking when used in reference to network devices has several possible meanings. One is simply the time of day, which is provided on the ATM switch router by the network time protocol (NTP). A second meaning is the clocking that is used for the internal logic of the system processor, or CPU; this is called system clocking. Finally, clocking can refer to the timing signal used by the physical interfaces in putting data on the transmission media. This type of clocking, often called network clocking, is important in the transmission of CBR and VBR-RT data and is discussed in this chapter.
Note The information in this chapter is applicable to the Catalyst 8540 MSR, Catalyst 8510 MSR, and LightStream 1010 ATM switch router. For detailed configuration information, refer to the ATM Switch Router Software Configuration Guide and the ATM Switch Router Command Reference publication.
This chapter includes the following sections:
Clocking at the physical interface is used to control the speed with which data is transmitted on the physical connection. This is important in delay-sensitive data types, such as voice and video, because these types of data must be received and transmitted at the same rate at every step, or hop, in a connection. To accomplish this, all the interfaces involved must be synchronized so that within a given time window the same amount of data is transmitted or forwarded at every point in the connection. If synchronization is not present, data can be lost due to buffer overflow or underflow at some point along the way. Real-time, delay-sensitive data is intolerant of such loss.
Note Properly configured network clocking is necessary for the CBR and VBR-RT traffic categories when used to send delay-sensitive data types. If you are not using your ATM network for these data types, then you do not have to be concerned with clocking.
The ATM switch router can use one of its internal clock sources, or it can extract clocking from an external signal. Internal sources include:
- Oscillator on the processor (CPU)---present on all ATM switch router models and used as the default clock source if no network clock module is installed.
- Oscillator on the network clock module---an option available only on ATM switch router models that support the network clock module; if present, the network clock module is used as the default clock source.
Note Support for the network clock module is hardware dependent.
- Oscillator on a port adapter or interface module.
Note For a list of the specific port adapters and interface modules that can be configured as clock sources, refer to the ATM Switch Router Software Configuration Guide.
An external source is one derived from a signal coming into the ATM switch router. These can include:
- Another ATM switch
- A PBX which, in turn, can extract its clocking from a public telephone network
- A Building Integrated Timing Supply (BITS) source supplied to the network clock module using a T1 or E1 connection
Clock sources are rated by quality, or stratum level, where 1 represents the highest possible quality of clocking. The oscillator on the processor is a stratum 4 source, whereas the oscillator on the network clock module is a stratum 3 source (if two network clock modules are present) or stratum 3ND ("non duplicated," when only one module is present). Other sources vary widely in quality. In general, public telephone networks provide a high quality source.
In many cases, using a clocking signal from a telephone company is the simplest and best solution for a stable and reliable clocking signal, especially in those instances where you are already connecting to telephone equipment using circuit emulation services (CES).
For example, to meet its own need for internal consistency, a telephone company typically distributes a timing signal to govern its own networking operations. Therefore, the telephone company has already addressed timing requirements similar to those that an ATM switch router user must address in relation to their own CES operations. Consequently, a private branch exchange (PBX) can serve as a ready means for providing a timing signal to any user CBR device.
A major telephone carrier is often the timing signal of choice, because such signals are known to be highly stable, reliable, and accurate.
For more information on clocking configuration for CES, see the "Network Clocking for CES and CBR Traffic" section of the chapter "Circuit Emulation Services and Voice over ATM."
Clocking for all interfaces on the ATM switch router can be specified in a single global configuration that selects one or more sources to use for transmit clocking and assigns priorities to the sources. Additionally clocking used by a particular physical interface can be configured in three different modes:
- Network derived---Transmit clocking is derived from the source provided by the ATM switch router's internal clock distribution mechanism. The source can be external, for example, provided by a signal received on another interface, or it can be internal, that is, the oscillator on the system processor or network clock module. Network derived mode is the default for all interfaces on the ATM switch router.
- Loop-timed---Transmit clocking is derived from the clock source received on the interface.
- Free-running---Transmit clocking is derived from the port adapter's local oscillator, if present. If the port adapter does not have its own oscillator, the oscillator on the system processor or network clock module is used as the transmit clocking source. Unlike loop-timed, in free-running mode the interface is not synchronized with the incoming signal.
Clock Source and Distribution Example
Figure 7-1 illustrates the clocking sources and distribution configured for a switch. The clocking source configured as priority one is extracted from the data received at interface 0/0/0 and is distributed as the transmit clock to the rest of the switch through the backplane. Interface 3/0/0 is configured to use network-derived transmit clocking, received across the backplane from interface 0/0/0.
Figure 7-1: Transmit Clock Distribution
Since the port providing the network clock source could fail, Cisco IOS software provides the ability to configure a backup interface as a clock source with priority 2. If neither priority 1 or 2 is configured, the default (system clock) is used as the derived clock. However, you can also configure the system clock to priority 1 or 2.
Note If you are using ports on port adapters inserted into a SuperCAM to derive clocking, the primary and secondary (priority1 and 2) clock sources must be on different port adapters. No such restriction applies, however, when a full-width interface module is used.
When a backup clock source is configured as priority 2, that source becomes the supplier of transmit clocking to the system if the priority 1 interface or source should fail. The example clocking configuration shown in Figure 7-2 demonstrates the following:
- ATM switch router number two is configured to receive transmit clocking from an external reference clock source through interface 0/0/0.
- Interface 3/0/0 uses network-derived transmit clocking.
- The priority 1 clock source, interface 0/0/0, fails.
- The priority 2 clock source, interface 0/0/3, immediately starts providing the transmit clocking to the backplane and interface 3/0/0.
- If the network clock is configured as revertive, when the priority 1 interface, 0/0/0, has been functioning correctly for the required length of time (see note following), the interfaces using network-derived transmit clocking start to receive their clocking again from interface 0/0/0.
Note On the LightStream 1010 ATM switch router and Catalyst 8510 MSR platforms, if the NCDP is configured to be revertive, a failed clocking source node after a switchover is restored to use after it has been functioning correctly for at least one minute. On the Catalyst 8540 MSR the failed source is restored after about 25 seconds. The network clock is, by default, configured as nonrevertive.
Figure 7-2: Transmit Clocking Priority Configuration Example
Note If no functioning network clock source port exists at a given time, the default clock source is the system clock on the processor or on the network clock module, if present.
An ATM switch router with the network clock module offers several advantages over a system without the module, including greater resilience, a superior quality oscillator, and the ability to extract a clocking signal from a Building Integrated Timing Supply (BITS) source.
Note Consult your ATM switch router hardware documentation if you are unsure whether your model supports the network clock module.
If the ATM switch router equipped with the network clock module is extracting clocking from a line that fails, the network clock module can enter holdover mode. Because the network clock module has stored information about the incoming clock signal, it can faithfully reproduce the lost signal while in holdover mode until a switchover to another clock source occurs. This capability helps smooth the transition from one clocking source to another in the event of failure or the transition from one clocking source to another with a different line speed. The network clock module also significantly reduces shock and jitter in the clocking signal.
When equipped with two route processors and two network clock modules, network clocking is fully redundant. In the event of failure of a route processor, the network clock module on the secondary takes over.
Both the processor and the network clock module on the ATM switch router are equipped with a 19.44 Mhz oscillator. However, the network clock module oscillator provides stratum 3 clocking (when two are present) or stratum 3ND clocking (when only one is present), while the processor oscillator provides stratum 4 clocking.
The network clock module provides two ports for extracting clocking from a BITS source. The BITS ports are configured as either T1 or E1; the line type applies to both ports. In addition, each port can be configured as priority 1 or 2.
The Network Clock Distribution Protocol (NCDP) provides a means by which a network can be synchronized automatically to a primary reference source (PRS). PRS refers to one of the following:
- A device or location of a source that provides reference clocking to a network or networks.
- An entity, such as a PTSN, that provides reference clocking.
To achieve automatic network synchronization, the NCDP constructs and maintains a spanning network clock distribution tree. This tree structure is superimposed on the network nodes by the software, resulting in an efficient, synchronized network suitable for transport of traffic with inherent synchronization requirements, such as voice and video.
Note The NCDP is intended for use on ATM switch routers equipped with the FC-PFQ or with the network clock module.
Figure 7-3 shows a hypothetical network that is synchronized to an external PRS. This network has the following configuration for clocking sources:
- One port on node C is configured with priority 1 to receive reference clocking from a stratum 2 PRS.
- A second port on node C is configured with priority 1 to receive reference clocking from a stratum 3 PRS.
- A port on node J is configured with priority 2 to receive reference clocking from a stratum 2 PRS.
- A port on node D is configured with priority 2 to receive reference clocking from a stratum 3 PRS.
- Node E is configured with priority 2 to receive clocking from its system clock.
Figure 7-3: Network Synchronized to an External Clocking Source using NCDP
NCDP selects the root to be used for the clocking distribution tree by evaluating a vector comprised of the priority, stratum level, and PRS ID. These three elements can have the following values:
- priority: 1 (primary), 2 (secondary)
- stratum: 1, 2, 2e, 3, 3e, 4, 4e
- PRS: 0 (external source), 255 (internal source)
The first of these elements, priority, is specified in the manual configuration. The second element, stratum, is specified explicitly or, if the source is "system," it is determined by the software based on the stratum of the system's processor or network clock module, if present. The third element, external/internal, is determined by the software.
The clocking sources in Figure 7-3 have the following vectors:
- Node C, first port:1, 2, 0
- Node C, second port: 1, 3, 0
- Node F: 2, 2, 0
- Node F: 2, 4, 255
The vectors are evaluated first using the priority element; the vector with the highest priority wins. If there is a tie, a comparison of the stratum level is done, and the vector with the highest stratum level wins. If there is still a tie, then the source with the external clock source wins. If there is a tie among these three elements, the software checks the stratum of the oscillator on the switch (processor or network clock module). If there is still a tie, the ATM address associated with the vector becomes the tie breaker, with the vector having the lower ATM address declared the victor.
Evaluating the configuration vectors in Figure 7-3 results in the following:
1. The first port on node C is declared the root clocking source node. With node C as the root, the software constructs a spanning network clocking distribution tree using well-known VCs. The arrows in Figure 7-3 show the construction of the tree.
2. If the link on the first port of node C fails, or the reference clock provided on this link degrades to the point where it is unusable, node C uses the local oscillator (if FC-PFQ is present) or runs in holdover mode (if the network clock module is present) until it can switch over to the second port.
Note Configuring a second port on the primary node with the same priority provides a backup in the event of failure of a link without the need to switch over to the second node and reconstruct the distribution tree.
3. If the second link on node C fails, the distribution tree is reconstructed so that it is rooted at the port located on node F.
4. If the link on node F fails, node F uses its local oscillator (processor or network clock module).
5. Should the system clock source on node F fail, the local oscillator on the node with the highest stratum clock becomes the clocking source. In the event of a tie in stratum, the node with the lowest ATM address becomes the clocking source.
The location of the primary and secondary clock source nodes is important. Locating the primary and secondary clock source nodes as close to each other as possible minimizes the number of disruptions seen by end systems attached to the network as the clocking root moves from primary to secondary. The primary and secondary clock source nodes should also be located as close as possible to the center of the network to minimize the height of the spanning network clock distribution tree. This ensures that the algorithm will converge as quickly as possible, is more reliable because disruptions are contained within a limited portion of the tree, and minimizes the possibility of cumulative wander that could be introduced at each clocking stage.
An example of a configuration that takes these considerations into account in shown in Figure 7-4.
Figure 7-4: Network Configuration Optimized for NCDP
The network in Figure 7-4 is constructed so that the primary and secondary clock source nodes are physically adjacent and close to the center of the network. Further, to contain switchovers to a minimum number of nodes in the event of a change in root clock source node, every node that is adjacent to the primary clock source node is also adjacent to the secondary clock source node.
A further consideration in planning an NCDP implementation is the clock stratum. A node should extract clocking only from a source of equal or better stratum. When a network of switches participating in NDCP is comprised of devices of different stratum levels, a node at a higher stratum level (a lower numerical stratum value) never chooses to extract its clock from a link attaching it to a lower stratum level device (a higher numerical stratum level). Doing so can result in a partition of the network clock distribution tree into multiple trees.
The example network in Figure 7-5 is comprised of stratum 3 devices, such as the Catalyst 8540 MSR with the network clock module, and stratum 4 devices, such as the Catalyst 8510 MSR. Because the stratum 3 devices cannot extract clocking from the stratum 4 devices, the network becomes partitioned into two clocking domains.
The general rule is that interfaces with higher clocking priority should not be located on devices having a stratum lower than other devices in the network that depend upon it for their transmit clocking.
Figure 7-5: Partitioned Network Due to Misconfiguration
This section provides an overview of configuring network clocking using both manual configuration and the NCDP. NCDP is the recommended method, as it simplifies the configuration and automatically prevents timing loops from occurring in the network.
These overviews address configuration of clocking sources, priorities, and global mode.
Configuring network clocking with NCDP requires the following steps:
Step 1 From global configuration mode, enable the NCDP.
When you enable NCDP, the software selects the best clock source, as described in the "How it Works" section. You must enable NCDP on each node that participates in the protocol.
Step 2 Configure the diameter (optional).
You can optionally constrain the diameter of the spanning tree by specifying the maximum number of hops between any two nodes that participate in the protocol. Each node must be configured with the same maximum network diameter value for NCDP to operate correctly. For example, in Figure 7-4, if Node A has a maximum network diameter value of 11, Nodes B through F must have the same value.
Step 3 From interface configuration mode, configure network clock sources, priorities, stratum, and revertive behavior.
The priorities you assign to clock source for NCDP are system-wide. You must also specify the stratum level of each source, since this is used in the calculation of the clocking distribution.
See the "Considerations When Using NCDP" section for a discussion of clock stratum and placement of primary and secondary clock sources.
Manually configuring network clocking requires the following steps:
Step 1 From global configuration mode, configure the clocking sources and priorities for the system.
Select one of the following source types:
- System---specifies the oscillator on the system processor or the oscillator on the network clock module, if present.
- Interface---specifies a particular interface, which extracts the clock from its input signal.
- BITS---specifies the BITS ports on the network clock module. These ports extract clocking from a received signal on their E1 or T1 interfaces.
Use an external source, from an interface or from a BITS source, when you want to be synchronous with a network timing source.
We recommend that you configure priority 1 and priority 2 sources. You can then choose to specify revertive behavior for failed sources.
When you have completed the configuration in this step, all the interfaces on the ATM switch router will be in network-derived mode and take their transmit clocking from the specified priority 1 source (until such time as there is a failure of that source). You do not need to do Step 2 unless you want one or more interfaces to have a different clocking configuration.
Step 2 From interface configuration mode, configure the clocking mode for specific interfaces (optional).
Select one of the following clocking modes:
- Network derived---this is the preferred clocking mode for CBR traffic. The option is provided here to allow you to revert to network derived clocking on an interface that has previously been configured to use another mode.
- Loop timed---this method can be used when the interface is connected to another device with a very accurate clock (better than stratum 4). This method is required when the interface must be synchronous with the clock provided by the attached system. Only one of the interfaces to a link should be configured as loop timed. If both ends of the connection are loop timed, the interfaces can intermittently go up and down, or "flap."
- Free running--- this method uses the local oscillator on the port adapter, if present, and otherwise the oscillator on the processor; therefore, it does not provide synchronous clocking. This method is sometimes used to isolate an interface from the rest of the system for purposes of troubleshooting.
The CES modules on the ATM switch router offer additional clocking configuration options for use in circumstances where you must accommodate more than one clock source in the network or where no PRS is configured. These options are described in the "Network Clocking for CES and CBR Traffic" section in the chapter "Circuit Emulation Services and Voice over ATM."
Posted: Mon Aug 16 14:04:49 PDT 1999
Copyright 1989-1999©Cisco Systems Inc.
