|
|
This chapter describes basic tasks that you can perform to troubleshoot your router and network. For detailed troubleshooting procedures and a variety of scenarios, see the Internetwork Troubleshooting Guide. For complete details on all debug commands, see the Debug Command Reference.
For a complete description of the troubleshooting commands in this chapter, refer to the "Troubleshooting Commands" chapter of the Configuration Fundamentals Command Reference. To locate documentation of other commands that appear in this chapter, use the command reference index or search online.
To perform general fault management, complete the tasks in the following sections:
In addition, some chapters in the Cisco IOS software configuration guides include fault management tasks in a monitoring and maintaining section.
To provide information about system processes, the Cisco IOS software includes an extensive list of EXEC commands that begin with the word show, which, when executed, display detailed tables of system information. Following is a list of the more common system management show commands. Perform these tasks in EXEC mode to display the information described:
| Task | Command |
|---|---|
| Display information about the CPU and midplane for the Cisco 7200 series routers. | show c7200 |
| Display information stored in NVRAM when the router crashes. This command is only useful to your technical support representative. This command is supported on the Cisco 7000 family routers. | show context |
| Display a message indicating whether an environmental warning condition currently exists, the temperature and voltage information, the last measured value from each of the six test points stored in nonvolatile memory, or the environmental specifications. This command is supported on the Cisco 7000 family routers. | show environment [all | last | table] |
| Display all GT64010 internal registers and interrupt status on the Cisco 7200 series routers. | show gt64010 |
| Display memory pool statistics including summary information about the activities of the system memory allocator and a block-by-block listing of memory use. | show memory [type] [free] [summary] |
| Display information about the peripheral component interconnect (PCI) hardware registers or bridge registers for the Cisco 7200 series routers. | show pci {hardware | bridge [register]} |
| Display information about all active processes. | show processes [cpu] |
| Display information about memory usage. | show processes memory |
| Display the configured protocols. | show protocols |
| Display stack usage of processes and interrupt routines, including the reason for the last system reboot. This command is only useful to your technical support representative. | show stacks |
| Display subsystem information. | show subsys [class class | name name] |
| Display the status of TCP connections. | show tcp [line-number] |
| Display a concise description of TCP connection endpoints. | show tcp brief [all] |
| Display a snapshot of the time-division multiplexing (TDM) bus connection or data memory in a Cisco AS5200 access server. | show tdm {connections | data} [motherboard | slot number] |
| Display general information about the router or VIP card when reporting a problem. | show tech-support [page] [password]
show controllers vip slot-number tech-support |
Look for specific show commands in the tables of configuration tasks found throughout the chapters in Cisco IOS software configuration guides. See the Cisco IOS software command references for detailed descriptions of the commands.
Some routers have an environmental monitor which monitors the physical condition of the router. If a measurement exceeds acceptable margins, a warning message is printed to the system console. The system software collects measurements once every 60 seconds, but warnings for a given test point are printed at most once every four hours. If the temperature measurements are out of specification more than the shutdown margin, the software shuts the router down but the fan will stay on. The router has to be manually turned off and on after such a shutdown. You can query the environmental monitor using the show environment command at any time to determine whether a measurement is out of tolerance. Refer to the System Error Messages publication for a description of environmental monitor warning messages.
On routers with an environmental monitor, if the software detects that any of its temperature test points have exceeded maximum margins, it performs the following steps in this order:
1 ) Saves the last measured values from each of the six test points to internal nonvolatile memory.
2 ) Interrupts the system software and causes a shutdown message to be printed on the system console.
3 ) Shuts off the power supplies after a few milliseconds of delay.
The following is the message the system displays if temperatures exceed maximum margins, along with a message indicating the reason for the shutdown:
Router# %ENVM-1-SHUTDOWN: Environmental Monitor initiated shutdown %ENVM-2-TEMP: Inlet temperature has reached SHUTDOWN level at 64(C)
Refer to the hardware installation and maintenance publication for your router for more information about environmental specifications.
Complete the tasks in the following sections to test basic network connectivity:
The TCP keepalive capability allows a router to detect when the host with which it is communicating experiences a system failure, even if data stops being transmitted (in either direction). This is most useful on incoming connections. For example, if a host failure occurs while talking to a printer, the router might never notice, because the printer does not generate any traffic in the opposite direction. If keepalives are enabled, they are sent once every minute on otherwise idle connections. If five minutes pass and no keepalives are detected, the connection is closed. The connection is also closed if the host replies to a keepalive packet with a reset packet. This will happen if the host crashes and comes back up again.
To set up the TCP keepalive packet service, perform the following task in global configuration mode:
| Task | Command |
|---|---|
| Generate TCP keepalive packets on idle network connections, either incoming connections initiated by a remote host, or outgoing connections initiated by a user. | service {tcp-keepalives-in | tcp-keepalives-out} |
As an aid to diagnosing basic network connectivity, many network protocols support an echo protocol. The protocol involves sending a special datagram to the destination host, then waiting for a reply datagram from that host. Results from this echo protocol can help in evaluating the path-to-host reliability, delays over the path, and whether the host can be reached or is functioning.
To use the echo protocol, perform the following task in either user or privileged EXEC mode:
| Task | Command |
|---|---|
| Invoke a diagnostic tool for testing connectivity. | ping [protocol] {host | address} |
Look for specific ping commands in the tables of configuration tasks found throughout the chapters in Cisco IOS software configuration guides. See the Cisco IOS software command references for detailed descriptions of the command.
| Task | Command |
|---|---|
| Trace packet routes through the network (privileged level). | trace [protocol] [destination] |
You can test the status of the following items:
 | Caution We do not recommend using these test commands; they are intended to aid manufacturing personnel in checking system functionality. |
To test the status of Flash memory, perform the following task in privileged EXEC mode:
| Task | Command |
|---|---|
| Test Flash memory on MCI and envm Flash EPROM interfaces. | test flash |
To test the status of system memory, perform the following task in privileged EXEC mode:
| Task | Command |
|---|---|
| Diagnose Multibus memory, including nonvolatile memory. | test memory |
| Caution Do not use this test to diagnose problems with an operational server. |
To test the status of the interfaces, perform the following task on a nonoperational server in privileged EXEC mode:
| Task | Command |
|---|---|
| Check network interfaces. | test interfaces |
By default, routers send the output from the debug EXEC command and system error messages to a logging process. The logging process controls the distribution of logging messages to the various destinations, such as the logging buffer, terminal lines, or a UNIX syslog server, depending on your configuration. The process also sends messages to the console. When the logging process is on, the messages are displayed on the console after the process that generated them has finished.
When the logging process is disabled, messages are sent only to the console. The messages are sent as they are generated, so error and debug output will be interspersed with prompts or output from the command.
You can set the severity level of the messages to control the type of messages displayed for the console and each of the destinations. You can timestamp log messages or set the syslog source address to enhance real-time debugging and management.
Refer to the System Error Messages publication for information on possible error messages.
Message logging is enabled by default. It must be enabled in order to send messages to any destination other than the console.
To disable message logging, use the no logging on command. Disabling the logging process can slow down the router because a process must wait until the messages are written to the console before continuing.
To re-enable message logging after it has been disabled, perform the following task in global configuration mode:
| Task | Command |
|---|---|
| Enable message logging. | logging on |
To enable slave Versatile Interface Processor (VIP) cards to log important messages to the console, perform the following task in global configuration mode:
| Task | Command |
|---|---|
| Enable slave message logging. | service slave-log |
If message logging is enabled, you can send messages to specified locations, in addition to the console.
To specify the locations that receive messages, perform one or more of the following tasks in global configuration mode:
| Task | Command |
|---|---|
| Log messages to an internal buffer. | logging buffered [size] |
| Log messages to a nonconsole terminal. | terminal monitor |
| Log messages to a UNIX syslog server host. | logging host |
The logging buffered command copies logging messages to an internal buffer. The buffer is circular, so newer messages overwrite older messages after the buffer is full. To display the messages that are logged in the buffer, use the show logging EXEC command. The first message displayed is the oldest message in the buffer. To clear the current contents of the buffer, use the clear logging privileged EXEC command.
The EXEC command terminal monitor locally accomplishes the task of displaying the system error messages to a nonconsole terminal.
The logging command identifies a syslog server host to receive logging messages. The argument host is the name or Internet address of the host. By issuing this command more than once, you build a list of syslog servers that receive logging messages. The no logging command deletes the syslog server with the specified address from the list of syslogs.
You can configure the system to synchronize unsolicited messages and debug command output with solicited device output and prompts for a specific line. You can identify the types of messages to be output asynchronously based on the level of severity. You can also determine the maximum number of buffers for storing asynchronous messages for the terminal after which messages are dropped.
When synchronous logging of unsolicited messages and debug command output is turned on, unsolicited device output is displayed on the console or printed after solicited device output is displayed or printed. Unsolicited messages and debug command output is displayed on the console after the prompt for user input is returned. Therefore, unsolicited messages and debug command output are not interspersed with solicited device output and prompts. After the unsolicited messages are displayed, the console displays the user prompt again.
To configure for synchronous logging of unsolicited messages and debug command output with solicited device output and prompts, perform the following tasks, beginning in global configuration mode:
| Task | Command |
|---|---|
| Step 1 Specify the line to be configured for synchronous logging of messages. | line [aux | console | vty] line-number [ending-line-number] |
| Step 2 Enable synchronous logging of messages. | logging synchronous [level severity-level | all] [limit number-of-buffers] |
By default, log messages are not timestamped. You can enable timestamping of log messages by performing the following task in global configuration mode:
| Task | Command |
|---|---|
| Enable log timestamps. | service timestamps log uptime
or service timestamps log datetime [msec] [localtime] [show-timezone] |
You can limit messages displayed to the selected device by specifying the severity level of the error message. To do so, perform one of the following tasks in global configuration mode:
| Task | Command |
|---|---|
| Limit messages logged to the console. | logging console level |
| Limit messages logged to the terminal lines. | logging monitor level |
| Limit messages logged to the syslog servers. | logging trap level |
If you have enabled syslog messages traps to be sent to an SNMP network management station with the snmp-server enable trap command, you can also change the level of messages sent and stored in a history table on the router. You can also change the number of messages that get stored in the history table.
Messages are stored in the history table because SNMP traps are not guaranteed to reach their destination. By default, one message of the level warning and above (see Table 14) is stored in the history table even if syslog traps are not enabled.
To change the level and table size defaults, perform the following tasks in global configuration mode:
| Task | Command |
|---|---|
| Change the default level of syslog messages stored in the history file and sent to the SNMP server. | logging history level |
| Change the number of syslog messages that can be stored in the history table. | logging history size number |
The logging console command limits the logging messages displayed on the console terminal to messages with a level number at or below the specified severity level, which is specified by the level argument. Table 14 lists the error message level keywords and corresponding UNIX syslog definitions in order from the most severe level to the least severe level.
| Level Keyword | Level | Description | Syslog Definition |
|---|---|---|---|
| emergencies | 0 | System unusable | LOG_EMERG |
| alerts | 1 | Immediate action needed | LOG_ALERT |
| critical | 2 | Critical conditions | LOG_CRIT |
| errors | 3 | Error conditions | LOG_ERR |
| warnings | 4 | Warning conditions | LOG_WARNING |
| notifications | 5 | Normal but significant condition | LOG_NOTICE |
| informational | 6 | Informational messages only | LOG_INFO |
| debugging | 7 | Debugging messages | LOG_DEBUG |
The no logging console command disables logging to the console terminal.
The default is to log messages to the console at the debugging level and those level numbers that are lower, which means all levels. The logging monitor command defaults to debugging also. The logging trap command defaults to informational.
To display logging messages on a terminal, use the terminal monitor EXEC command.
Current software generates four categories of error messages:
You can also configure the syslog facility in which error messages are sent by performing the following task in global configuration mode:
| Task | Command |
|---|---|
| Configure system log facilities. | logging facility facility-type |
Table 15 lists the logging facility type keywords and their descriptions.
| Facility Type Keyword | Description |
|---|---|
| auth | Indicates the authorization system. |
| cron | Indicates the cron facility. |
| daemon | Indicates the system daemon. |
| kern | Indicates the Kernel. |
| local0-7 | Reserved for locally defined messages. |
| lpr | Indicates line printer system. |
| Indicates mail system. | |
| news | Indicates USENET news. |
| sys9 | Indicates system use. |
| sys10 | Indicates system use. |
| sys11 | Indicates system use. |
| sys12 | Indicates system use. |
| sys13 | Indicates system use. |
| sys14 | Indicates system use. |
| syslog | Indicates the system log. |
| user | Indicates user process. |
| uucp | Indicates UNIX-to-UNIX copy system. |
Refer also to your syslog manual pages.
To display logging information, perform the following task in EXEC mode:
To set up the syslog daemon on a 4.3 BSD UNIX system, include a line such as the following in the /etc/syslog.conf file:
local7.debugging /usr/adm/logs/cisco.log
The debugging keyword specifies the syslog level; see Table 14 for a general description of other keywords. The local7 keyword specifies the logging facility to be used; see Table 15 for a general description of other keywords.
The syslog daemon sends messages at this level or at a more severe level to the file specified in the next field. The file must already exist, and the syslog daemon must have permission to write to it.
By default, a syslog message contains the IP address of the interface it uses to leave the router. To require that all syslog messages contain the same IP address, regardless of which interface they use, perform the following task in global configuration mode:
| Task | Command |
|---|---|
| Set the syslog source address. | logging source-interface type number |
| Task | Command |
|---|---|
| Display the state of each debugging option. | show debugging |
| Display a list and brief description of all the debug command options. | debug ? |
| Begin message logging for the specified debug command. | debug command |
| Turn message logging off for the specified debug command. | no debug command |
| Caution The system gives high priority to debugging output. For this reason, debugging commands should be turned on only for troubleshooting specific problems or during troubleshooting sessions with technical support personnel. Excessive debugging output can render the system inoperable. |
| Task | Command |
|---|---|
| Enable timestamping of system debug messages. | service timestamps debug uptime
or service timestamps debug datetime [msec] [localtime] [show-timezone] |
Normally, the messages are displayed only on the console terminal. See the section "Set the Error Message Display Device" earlier in this chapter to change the output device.
|
|