Network Clock Synchronization

Oscillator on the processor (CPU)---present on all ATM switch router models and used as the default clock source if no network clock module is installed.
Oscillator on the network clock module---an option available only on ATM switch router models that support the network clock module; if present, the network clock module is used as the default clock source.

Note

Oscillator on a port adapter or interface module.

Note

ATM Switch Router Software Configuration Guide

An external source is one derived from a signal coming into the ATM switch router. These can include:

Another ATM switch
A PBX which, in turn, can extract its clocking from a public telephone network
A Building Integrated Timing Supply (BITS) source supplied to the network clock module using a T1 or E1 connection

Clock sources are rated by quality, or stratum level, where 1 represents the highest possible quality of clocking. The oscillator on the processor is a stratum 4 source, whereas the oscillator on the network clock module is a stratum 3 source (if two network clock modules are present) or stratum 3ND ("non duplicated," when only one module is present). Other sources vary widely in quality. In general, public telephone networks provide a high quality source.

Network Clock Sources for Circuit Emulation Services

In many cases, using a clocking signal from a telephone company is the simplest and best solution for a stable and reliable clocking signal, especially in those instances where you are already connecting to telephone equipment using circuit emulation services (CES).

For example, to meet its own need for internal consistency, a telephone company typically distributes a timing signal to govern its own networking operations. Therefore, the telephone company has already addressed timing requirements similar to those that an ATM switch router user must address in relation to their own CES operations. Consequently, a private branch exchange (PBX) can serve as a ready means for providing a timing signal to any user CBR device.

A major telephone carrier is often the timing signal of choice, because such signals are known to be highly stable, reliable, and accurate.

For more information on clocking configuration for CES, see the "Network Clocking for CES and CBR Traffic" section of the chapter "Circuit Emulation Services and Voice over ATM."

Clock Distribution Modes

Clocking for all interfaces on the ATM switch router can be specified in a single global configuration that selects one or more sources to use for transmit clocking and assigns priorities to the sources. Additionally clocking used by a particular physical interface can be configured in three different modes:

Network derived---Transmit clocking is derived from the source provided by the ATM switch router's internal clock distribution mechanism. The source can be external, for example, provided by a signal received on another interface, or it can be internal, that is, the oscillator on the system processor or network clock module. Network derived mode is the default for all interfaces on the ATM switch router.
Loop-timed---Transmit clocking is derived from the clock source received on the interface.
Free-running---Transmit clocking is derived from the port adapter's local oscillator, if present. If the port adapter does not have its own oscillator, the oscillator on the system processor or network clock module is used as the transmit clocking source. Unlike loop-timed, in free-running mode the interface is not synchronized with the incoming signal.

Clock Source and Distribution Example

Figure 8-1 illustrates the clocking sources and distribution configured for a switch. The clocking source configured as priority one is extracted from the data received at interface 0/0/0 and is distributed as the transmit clock to the rest of the switch through the backplane. Interface 3/0/0 is configured to use network-derived transmit clocking, received across the backplane from interface 0/0/0.

Figure 8-1: Transmit Clock Distribution

Since the port providing the network clock source could fail, Cisco IOS software provides the ability to configure a backup interface as a clock source with priority 2. If neither priority 1 or 2 is configured, the default (system clock) is used as the derived clock. However, you can also configure the system clock to priority 1 or 2.

Note On the Catalyst 8540 MSR, if you are using ports on port adapters inserted into a carrier module to derive clocking, the primary and secondary (priority1 and 2) clock sources must be on different port adapters. No such restriction applies, however, when a full-width interface module is used.

Clock Source Failure and Revertive Behavior

When a backup clock source is configured as priority 2, that source becomes the supplier of transmit clocking to the system if the priority 1 interface or source should fail. The example clocking configuration shown in Figure 8-2 demonstrates the following:

ATM switch router number two is configured to receive transmit clocking from an external reference clock source through interface 0/0/0.
Interface 3/0/0 uses network-derived transmit clocking.
The priority 1 clock source, interface 0/0/0, fails.
The priority 2 clock source, interface 0/0/3, immediately starts providing the transmit clocking to the backplane and interface 3/0/0.
If the network clock is configured as revertive, when the priority 1 interface, 0/0/0, has been functioning correctly for the required length of time (see note following), the interfaces using network-derived transmit clocking start to receive their clocking again from interface 0/0/0.

Note

Figure 8-2: Transmit Clocking Priority Configuration Example

Note If no functioning network clock source port exists at a given time, the default clock source is the system clock on the processor or on the network clock module, if present.

About the Network Clock Module

An ATM switch router with the network clock module offers several advantages over a system without the module, including greater resilience, a superior quality oscillator, and the ability to extract a clocking signal from a Building Integrated Timing Supply (BITS) source.

Note Consult your ATM switch router hardware documentation if you are unsure whether your model supports the network clock module.

Resilience

If the ATM switch router equipped with the network clock module is extracting clocking from a line that fails, the network clock module can enter holdover mode. Because the network clock module has stored information about the incoming clock signal, it can faithfully reproduce the lost signal while in holdover mode until a switchover to another clock source occurs. This capability helps smooth the transition from one clocking source to another in the event of failure or the transition from one clocking source to another with a different line speed. The network clock module also significantly reduces shock and jitter in the clocking signal.

When equipped with two route processors and two network clock modules, network clocking is fully redundant. In the event of failure of a route processor, the network clock module on the secondary takes over.

Oscillator Quality

Both the processor and the network clock module on the ATM switch router are equipped with a 19.44 Mhz oscillator. However, the network clock module oscillator provides stratum 3 clocking (when two are present) or stratum 3ND clocking (when only one is present), while the processor oscillator provides stratum 4 clocking.

BITS Derived Clocking

The network clock module provides two ports for extracting clocking from a BITS source. The BITS ports are configured as either T1 or E1; the line type applies to both ports. In addition, each port can be configured as priority 1 or 2.

The Network Clock Distribution Protocol

The Network Clock Distribution Protocol (NCDP) provides a means by which a network can be synchronized automatically to a primary reference source (PRS). PRS refers to one of the following:

A device or location of a source that provides reference clocking to a network or networks.
An entity, such as a PTSN, that provides reference clocking.

To achieve automatic network synchronization, the NCDP constructs and maintains a spanning network clock distribution tree. This tree structure is superimposed on the network nodes by the software, resulting in an efficient, synchronized network suitable for transport of traffic with inherent synchronization requirements, such as voice and video.

Note The NCDP is intended for use on ATM switch routers equipped with the FC-PFQ or with the network clock module.

How it Works

Figure 8-3 shows a hypothetical network that is synchronized to an external PRS. This network has the following configuration for clocking sources:

One port on node C is configured with priority 1 to receive reference clocking from a stratum 2 PRS.
A second port on node C is configured with priority 1 to receive reference clocking from a stratum 3 PRS.
A port on node F is configured with priority 2 to receive reference clocking from a stratum 2 PRS.
Node E is configured with priority 2 to receive clocking from its system clock.

Figure 8-3: Network Synchronized to an External Clocking Source using NCDP

NCDP selects the root to be used for the clocking distribution tree by evaluating a vector comprised of the priority, stratum level, and PRS ID. These three elements can have the following values:

priority: 1 (primary), 2 (secondary)
stratum: 1, 2, 2e, 3, 3e, 4, 4e
PRS: 0 (external source), 255 (internal source)

The first of these elements, priority, is specified in the manual configuration. The second element, stratum, is specified explicitly or, if the source is "system," it is determined by the software based on the stratum of the system's processor or network clock module, if present. The third element, external/internal, is determined by the software.

The clocking sources in Figure 8-3 have the following vectors:

Node C, first port:1, 2, 0
Node C, second port: 1, 3, 0
Node F: 2, 2, 0
Node E: 2, 4, 255

The vectors are evaluated first using the priority element; the vector with the highest priority wins. If there is a tie, a comparison of the stratum level is done, and the vector with the highest stratum level wins. If there is still a tie, then the source with the external clock source wins. If there is a tie among these three elements, the software checks the stratum of the oscillator on the switch (processor or network clock module). If there is still a tie, the ATM address associated with the vector becomes the tie breaker, with the vector having the lower ATM address declared the victor.

Evaluating the configuration vectors in Figure 8-3 results in the following:

1. The first port on node C is declared the root clocking source node. With node C as the root, the software constructs a spanning network clocking distribution tree using well-known virtual connections. The arrows in Figure 8-3 show the construction of the tree.

2. If the link on the first port of node C fails, or the reference clock provided on this link degrades to the point where it is unusable, node C uses the local oscillator (if FC-PFQ is present) or runs in holdover mode (if the network clock module is present) until it can switch over to the second port.

Note

3. If the second link on node C fails, the distribution tree is reconstructed so that it is rooted at the port located on node F.

4. If the link on node F fails, node F uses its local oscillator (processor or network clock module).

5. Should the system clock source on node F fail, the local oscillator on the node with the highest stratum clock becomes the clocking source. In the event of a tie in stratum, the node with the lowest ATM address becomes the clocking source.

Considerations When Using NCDP

The location of the primary and secondary clock source nodes is important. Locating the primary and secondary clock source nodes as close to each other as possible minimizes the number of disruptions seen by end systems attached to the network as the clocking root moves from primary to secondary. The primary and secondary clock source nodes should also be located as close as possible to the center of the network to minimize the height of the spanning network clock distribution tree. This ensures that the algorithm will converge as quickly as possible, is more reliable because disruptions are contained within a limited portion of the tree, and minimizes the possibility of cumulative wander that could be introduced at each clocking stage.

An example of a configuration that takes these considerations into account in shown in Figure 8-4.

Figure 8-4: Network Configuration Optimized for NCDP

The network in Figure 8-4 is constructed so that the primary and secondary clock source nodes are physically adjacent and close to the center of the network. Further, to contain switchovers to a minimum number of nodes in the event of a change in root clock source node, every node that is adjacent to the primary clock source node is also adjacent to the secondary clock source node.

A further consideration in planning an NCDP implementation is the clock stratum. A node should extract clocking only from a source of equal or better stratum. When a network of switches participating in NDCP is comprised of devices of different stratum levels, a node at a higher stratum level (a lower numerical stratum value) never chooses to extract its clock from a link attaching it to a lower stratum level device (a higher numerical stratum level). Doing so can result in a partition of the network clock distribution tree into multiple trees.

The example network in Figure 8-5 is comprised of stratum 3 devices, such as the Catalyst 8540 MSR with the network clock module, and stratum 4 devices, such as the Catalyst 8510 MSR. Because the stratum 3 devices cannot extract clocking from the stratum 4 devices, the network becomes partitioned into two clocking domains.

The general rule is that interfaces with higher clocking priority should not be located on devices having a stratum lower than other devices in the network that depend upon it for their transmit clocking.

Figure 8-5: Partitioned Network Due to Misconfiguration

Finally, the only nodes that you would configure clock sources on would normally be those nodes where a good clock source is available that you want to distribute to the other nodes in the network---often from a link attaching the node to a service provider network. For example, if you have a network of 20 switches, you should only configure sources on the 2 switches that have lines to a cloud that you are using for the clock source; you should not configure any sources on the other 18 switches.

Configuration Overview

This section provides an overview of configuring network clocking using both manual configuration and the NCDP. NCDP is the recommended method, as it simplifies the configuration and automatically prevents timing loops from occurring in the network.

Network Clocking Configuration with NCDP

Configuring network clocking with NCDP requires the following steps:

Step 1 From global configuration mode, enable the NCDP.

When you enable NCDP, the software selects the best clock source, as described in the "How it Works" section. You must enable NCDP on each node that participates in the protocol.

Step 2 Configure the clock sources, their priorities, and stratums.

You must specify the clocking sources, their priorities, and associated stratums used by NCDP in constructing the clock distribution tree. The priorities you assign to clock source for NCDP are system-wide. You must also specify the stratum level of each source, since it is used in calculating the clock distribution.

If you do not configure a clock source, NCDP code advertises its default source of network clock (its local oscillator); if no nodes in the network have a clock source configured, the tree is built so that it is rooted at the switch having the highest stratum oscillator and lowest ATM address.

See the "Considerations When Using NCDP" section for a discussion of clock stratum and placement of primary and secondary clock sources.

Step 3 Configure the optional global parameters.

Optional NCDP parameters you can configure at the global level include the maximum number of hops between any two nodes, the revertive behavior, and the values of the NCDP hello and hold timers.

You can constrain the diameter of the spanning tree by specifying the maximum number of hops between any two nodes that participate in the protocol. Each node must be configured with the same maximum network diameter value for NCDP to operate correctly. For example, in Figure 8-4, if Node A has a maximum network diameter value of 11, Nodes B through F must have the same value.

For an explanation of revertive behavior, see the "Clock Source Failure and Revertive Behavior" section.

Step 4 From interface configuration mode, configure the optional per-interface parameters.

On a per-interface basis, you can enable or disable NCDP, specify the cost metric associated with the port, and change the control virtual circuit used to transport protocol messages between adjacent protocol entities.

Manual Network Clocking Configuration

Manually configuring network clocking requires the following steps:

Step 1 From global configuration mode, configure the clocking sources and priorities for the system.

Select one of the following source types:

System---specifies the oscillator on the system processor or the oscillator on the network clock module, if present.
Interface---specifies a particular interface, which extracts the clock from its input signal.
BITS---specifies the BITS ports on the network clock module. These ports extract clocking from a received signal on their E1 or T1 interfaces.

Use an external source, from an interface or from a BITS source, when you want to be synchronous with a network timing source.

We recommend that you configure priority 1 and priority 2 sources. You can then choose to specify revertive behavior for failed sources.

When you have completed the configuration in this step, all the interfaces on the ATM switch router will be in network-derived mode and take their transmit clocking from the specified priority 1 source (until such time as there is a failure of that source). You do not need to do Step 2 unless you want one or more interfaces to have a different clocking configuration.

Step 2 From interface configuration mode, configure the clocking mode for specific interfaces (optional).

Select one of the following clocking modes:

Network derived---this is the preferred and default clocking mode for CBR traffic. The option is provided here to allow you to revert to network derived clocking on an interface that has previously been configured to use another mode.
Loop timed---this method can be used when the interface is connected to another device with a very accurate clock (better than stratum 4). This method is required when the interface must be synchronous with the clock provided by the attached system. Only one of the interfaces to a link should be configured as loop timed. If both ends of the connection are loop timed, the interfaces can intermittently go up and down, or "flap."
Free running---this method uses the local oscillator on the port adapter, if present, and otherwise the oscillator on the processor; therefore, it does not provide synchronous clocking. This method is sometimes used to isolate an interface from the rest of the system for purposes of troubleshooting.

Network Clocking Configuration for Circuit Emulation Services

The CES modules on the ATM switch router offer additional clocking configuration options for use in circumstances where you must accommodate more than one clock source in the network or where no PRS is configured. These options are described in the "Network Clocking for CES and CBR Traffic" section in the chapter "Circuit Emulation Services and Voice over ATM."

Table of Contents

Network Clock Synchronization

Clock Source and Distribution Example