Table of Contents

Error Message Format
Process Status Features

Process Failures
Error Messages

Troubleshooting a Process Problem

Starting a Process

Desktop Interface
Command-line Interface

Stopping a Process

Desktop Interface
Command-line Interface

Troubleshooting Suggestions

Troubleshooting

Essentials provides you with methods for evaluating and troubleshooting problems.

The following sections are presented in this chapter:

• Error Message Format

• Process Status Features

• Troubleshooting a Process Problem

For detailed information on troubleshooting, refer to the online help.

Error Message Format

Essentials displays two types of error messages:

• Interface error messages

• Back-end error messages stored in the syslog

Interface error messages are displayed in dialog boxes with descriptions of the probable causes and recommended corrective actions, if any. (See Figure 11-1.)

Three types of interface error messages are available:

Some interface error messages include a Details button. (See Figure 11-2.)

Click Details for additional information and recommended corrective action. An informational dialog box appears. (See Figure 11-3.)

Back-end error messages result from problems that occur in processes running on the Essentials server. Back-end error messages are stored in the syslog.

Figure 11-4 shows the syslog error message format.

Process Status Features

Two interfaces are available for viewing and troubleshooting process problems:

• Essentials desktop (navigation tree)

• Command-line interface (CLI)

Table 11-1 shows the process features available in the desktop interface and their command-line equivalents.

Process Failures

Several events can cause a process to fail. For example, the database engine might fail for one of the following reasons:

• On UNIX systems, the database uses a small amount of space in the /tmp file system. If this space fills up, the database can no longer accept connections from client code and fails.

• The file system containing the database file is full.

Error Messages

Refer to the appropriate Essentials installation guide for the location of the error message logs.

If you cannot solve a process problem, call the Technical Assistance Center (TAC). Refer to the error log files to furnish the information they need to help you solve the problem.

Troubleshooting a Process Problem

The Process Failures table gives you information about potential process failures. It tells you that the process has failed, or an administrator has shut down the process.

Scenario: A process behaves in an unexpected way and you want to know the possible cause.

To troubleshoot a process, perform the following steps:

Step 1 Select Admin > Troubleshooting > Process Failures.

Step 2 Click on the process name to display the Process Details table.

Click Back to return to the Process Failures table.

Step 3 Click any process state to display the System Log.

Click Back to return to the Process Failures table.

Step 4 Click Update at any time to refresh the fields.

Starting a Process

Scenario: You check process status and notice that the DbServer process in the Process Status table shows a state of "Failed to run." You attempt to restart the process.

Desktop Interface

To attempt a process restart using the navigation tree, perform the following steps:

Step 1 Select System Admin > Start Process.

The Start Process dialog box appears. (See Figure 11-5.)

Command-line Interface

To restart a process using the CLI, perform the following steps:

Step 1 Enter pdexec DbServer.

Step 2 Enter pdshow DbServer again to see whether the process is operating properly.

If the process is operating properly, the following message appears:

Process= DbServer
State  = Running normally
Pid    = 21473
RC     = 0
Signo  = 0
Start  = 19:16:15 06/02/1997
Stop   = Not applicable
Core   = Not applicable
Info   = Data server (dbeng50) invoked
 

If the process is still not running normally, check the error log. If you cannot solve the problem, call the TAC and provide them with the error log information.

Stopping a Process

Scenario: You want to back up all data you have stored on your hard drive, so you shut down the DbServer process.

Desktop Interface

To stop the DbServer process using the navigation tree, perform the following steps:

Step 1 Select Admin > System Admin > Stop Process.

The Stop Process dialog box appears. (See Figure 11-6.)

Command-line Interface

To stop the DbServer process using the CLI, perform the following steps:

Step 1 Enter pdterm DbServer.

Step 2 Enter pdshow DbServer.

The following status message appears, showing that the process has been stopped.

Process= DbServer
State  = Administrator has shut down this server
Pid    = 0
RC     = 0
Signo  = 0
Start  = 19:16:15 06/02/97
Stop   = 11:27:05 06/03/97
Core   = Not applicable
Info   = Not applicable
 

Troubleshooting Suggestions

Table 11-2 lists troubleshooting suggestions. If the action items suggested do not resolve the error, check the release notes supporting your platform for possible workarounds, or contact the Cisco TAC or your customer support.

Table 11-2: Troubleshooting Suggestions

Error	Possible Reasons	Actions
Admin: unable to log on to Essentials. (Windows NT only.)	Incompatible Microsoft Internet Information Server (IIS) installed.	Check installation instructions, unload old MicroSoft IIS, then load correct MicroSoft IIS, if necessary.
"Authorization required. Please log in with your username and password".	Incompatible browser causing cookie failure (unable to retrieve cookie).	Refer to the installation documentation for supported Internet Explorer/Navigator software.
Database: inaccessible. This can appear by Process status showing one of the following: ICServer not running DIServer not running AvLoader not running DbMonitor not running DbServer not running EssentialsOSG not running Alternatively, the error message "failed to get complete list of domains" could appear on an Add Device operation.	Server is unable to connect to the database, which is corrupt or inaccessible.	1 . Log in to Essentials as admin. 2 . Select Admin > Troubleshooting > Process Failures to get a list of Essentials backend processes that have failed. 3 . Select Admin > Troubleshooting > Self Test. Click Create to create a report. Click Display to display the report. 4 . Select Admin > Troubleshooting > Collect Server Info. 5 . Click on Product Database Status link to get detailed database status. 6 . Contact the Cisco TAC or your customer support.
Database: ODBC error with Essentials. (Windows NT only.)	ODBC resource .dll and ODBC driver manager are different versions.	Install ODBC from Windows NT CD (selecting SQL server).
Device Configuration: archive cannot retrieve the configuration module for Catalyst devices.	Incorrect password given when adding or importing the device.	Enter the correct Telnet and enable passwords for the Catalyst devices in the Essentials database. The configuration archive uses Telnet to gather module configurations for Catalyst devices. In order for the configuration archive to successfully gather the ATM and RSM module configurations, these modules must have the same Telnet passwords as that for the supervisors of the Catalyst 5000 Family of devices. See Essentials online help for more information on entering passwords.
Device Configuration: archive cannot retrieve the running configuration for a device.	Incorrect Read and Write community strings given when adding or importing the device.	Enter the correct Read and Write community strings in the Essentials database. Change the order of the protocols used to retrieve the configuration. (The configuration archive downloads configurations from devices using three different transport protocols - TFTP, Telnet, and RCP - normally in that order.) See Essentials online help for more information on setting the transport protocol order used for gathering configurations.
Device Configuration: archive cannot retrieve the startup configuration for a device.	Incorrect password given when adding or importing the device.	Enter the correct Telnet and enable passwords for the device in the Essentials database. If the device is configured for TACACS authentication, add the TACACS username and password (not the Telnet password) in the Essentials database when you import the device.
		If the device is configured for local user authentication, add the local username and password in the Essentials database.
		If the device is configured for Telnet authentication, ensure that you use the Telnet password, enable password, enable secret (if configured), local user name and password (if configured) - do not enter either local or TACACS username and password.
		See Essentials online help for more information on entering passwords and TACACS, local, and RCP information.
Device Configuration: "DNS hostname mismatch. ip_address unknown to DNS."	Device does not have the DNS server set up to resolve the host name.	Ensure that the DNS server recognizes the device hostname. Alternatively, specify the IP address instead of the hostname.
Device Configuration: server runtime error when running Tasks > Device Configuration or Admin > Device Configuration tasks.	CMLogger is not running.	1 . Log in to Essentials as admin. 2 . Select Admin > System Admin > Start Process. 3 . Start the System. If the configuration tasks still fail, select Admin > Troubleshooting > Process Failures to get a list of Essentials backend processes that have failed. 4 . Select Admin > Troubleshooting > Self Test. Click Create to create a report. Click Display to display the report. 5 . Select Admin > Troubleshooting > Collect Server Info. 6 . Click on Product Database Status link to get detailed database status. 7 . Contact the Cisco TAC or your customer support.
Device Configuration: SNMP timeout prevents TFTP from retrieving the running configuration for a device.	SNMP did not allow sufficient time for the operation.	Increase the SNMP timeout by configuring SNMP retries and time-outs. See Essentials online help for more information on configuring system-wide SNMP time-outs and retries.
Display: applet cannot start: class browserServer not found. (Solaris only)	The server name is not in the httpd.conf file.	Add the server name in the httpd.conf file.
Display: only right side of Essentials Window displayed.	Incompatible browser software.	Refer to the installation documentation for supported Internet Explorer/Navigator software.
	Desktop is not registered in DNS.	Register desktop in DNS.
Inventory: device import from local database fails. (Solaris only.)	User bin is not a member of the CiscoWorks group.	Add group membership before starting Essentials.
	Name resolution is incorrect.	Correct the name resolution. If that is not possible, then remote import rules will be applied; add .rhosts to the bin home directory.
Inventory: device import fails from remote NMS.	Essentials and the remote NMS reside in different DNS domains.	Set up Essentials and remote NMS stations in the same DNS domains.
Inventory: device serial or router chassis numbers differ from those shown on outside labels.	Hardware reports get data from the user-defined optional serial number field when the device or router is added to the Essentials (or via Change Device Attributes) and not from the SNMP variable chassis serial number. The manually changed serial number takes precedence over the external number.	No action required, although you can manually change the serial number back to match the external number.
Inventory: device stays in a pending state.	Corrupt database.	Stop Essentials. Install a backup database if available; otherwise, install the basic database, px.db, over the corrupt database.
	DIServer not running	Check the process status. If DIServer not running, restart it.
	A broadcast address has been imported and is being used for an SNMP write.	Suspend the device. Run the address validation tool on the device (Tools > Connectivity Tools > Validate Device Addresses) to ensure that a broadcast or network address is not being used.
Inventory: devices not importing.	Access list applied to the SNMP-server community configuration.	Add permit statements to the access lists on all routers.
	SNMP timeout.	Increase SNMP slow timeout and retry values.
	Reverse DNS lookup failure.	Add device entry to local host file.
	Device name not configured in DNS or localhost file.	Add device entry to DNS or localhost file.
Inventory: cannot add device to database.	HP OpenView/SNMP has old version of wsnmp.dll files.	Remove or rename HP OpenView version wsnmp.dll files.
Performance: Essentials running slow when importing device. (Windows NT only.)	Incompatible software.	Refer to the installation documentation. Verify that correct SP and Microsoft IIS have been loaded. If necessary, unload and reload Essentials.
Printing: cannot print graphs in Essentials.	Browser print restriction.	To print an Essentials window, use the Print Screen or Capture Screen function on your machine.
Reports: browser hangs when running reports.	Incompatible browser software.	Refer to installation documentation for supported browser software.
Software Management: approver cannot change scheduled time for Distribute Images jobs using Software Management.	Maker-checker is enforced on Distribute Images jobs.	When Distribute Images jobs require approval, Software Management doesn't allow the schedule time for the job to be changed from Browse Job Status dialog boxs. Create a new job and submit for approval.
Software Management: cannot Undo an Upgrade operation on Microcom and Catalyst devices.	Undo software upgrade not supported on device.	Check Supported Device Matrix in online help for supported devices and software releases and Software Management features.
Software Management: Distribute Images and Image Import jobs fail on a device.	Defective software running on the device.	Go to CCO and examine the software image. If it is deferred, then contact Cisco TAC or your customer support. If the software image is not deferred, enable debugging and check the Enable Debugging check box (Admin > Software Management > Edit Preferences). Rerun the job and then use the Mail or Copy Log file option to extract Software Management debugging information. Send the information to Cisco TAC or your customer support with a complete description of the problem.
Software Management: job remains in "pending" state after scheduled time.	Essentials server is not functioning, or has been powered off or rebooted before job schedule time arrives.	Software Management moves job to error state 1 hour after job schedule time. Do not alter the job while in "pending" state; the system will take care of it. If necessary, create another job.
Software Management: job remains in "running" state, and the Job Details report shows no progress on job.	Essentials server is not functioning or has been powered off or rebooted while job is running, causing job to be abnormally terminated.	Software Management moves job to error state one hour after job schedule time. Do not alter the job while in "pending" state; the system will take care of it. If necessary, create another job.
Software Management: while adding modem images or CIP microcode images into the software management library, the image type is displayed as "Unknown." Software management cannot retrieve attributes from images.	Images for Cisco 3640 digital modems are not imported as AS5300 format files. Microcom firmware image is not the combined firmware/DSP code. CIP microcode version is older than 22.0.	Download a supported version of software/firmware from CCO. Check Supported Device Matrix in online help for supported devices and software releases.
Software Management: cannot schedule Distribute Images and Image Import jobs.	at service is not running or is incorrectly configured.	If Essentials is running on Windows NT, use Control Panel > Services to check that the at service is running. If it is not, start it manually. If Essentials is running on Solaris, check that /usr/bin/at command is present. Check that the at.deny file in /usr/lib/cron directory does not contain bin user name.
Software Management: Essentials cannot upload images from device.	Essentials needs Read-Write SNMP access to device.	Configure the Read-Write community string on the device.
Software Management: Mail or Copy Log File function does not mail log files. (Windows NT only.)	Incorrect e-mail address.	Correct e-mail address in the Mail or Copy Log File submenu.
Software Management: MICA/Microcom/CIP card: Software Management does not recognize the MICA/Microcom/CIP cards on AS5x00 or 7x00 device.	Devices running an unsupported version of Cisco IOS System Software.	See the Supported Device Matrix in online help for supported devices and software releases.
Software Management: RCP is not being used to transfer software images between Essentials and devices.	Device does not support RCP protocol (only IOS devices support RCP). RCP is not properly configured on Essentials server.	Check whether your device is Cisco IOS based. Make sure that RCP is defined as the preferred protocol. Make sure that the configuration (Admin > System Admin > System Configuration) is properly configured with an RCP username. If Essentials is running on a Windows NT machine, verify that CRMrsh service is running correctly on the Essentials server. (Verify this using the Control Panel > Services.) If the service is "stopped," run it manually. Check the Event Viewer to make sure service has started properly. (Access Event Viewer from Administrative Tools group, then view the application log by selecting Log > Application.) If Essentials is running on Solaris, make sure that the home directory for RCP user account contains the .rhosts file and can be written to by bin user.
Software Management: "Internal Error: Can't resolve address for proxy" message when invoking functions "Browse Bugs by Device" or "Locate Devices by Bugs".	Proxy or DNS incorrectly set up.	Check the proxy URL in Admin > System Admin > System Configuration. If you configure proxy using a host name, check the DNS configuration on the Essentials server. Check that proxy configuration is not set for login each time. If these actions do not solve the problem, run your browser on the server where Essentials is installed, configure proxy in the browser, and check whether www.cisco.com is accessible. Contact the Cisco TAC or your customer support.
Software Management: Schedule Synchronization Job report is not mailed. (Windows NT only.)	Incorrect e-mail address.	Correct e-mail address in the Schedule Synchronization Job submenu.
	SMTP server not configured.	Configure SMTP using Admin > System Admin > System Configuration.
Software Management: unable to download Cisco IOS (error 4151).	/var/tmp file has insufficient space to accommodate IOS image.	Increase /var/tmp space.
Software Management: CCO Upgrade Analysis screen and Recommend Image Upgrade (from Distribute Images) screen time-out.	Connection to CCO from Essentials server is slow. CCO server machine is down. CCO filters are not set with the right criteria.	These operations require access to CCO for image information. Make sure that the CCO server is up and running. Select the right filtering criteria, and retry the operation. Select fewer number of devices and retry the operation. If these actions do not solve the problem, check the proxy configuration, as described in Software Management.
Software Management: upgrade failure.	Software Management does not allow a direct upgrade from version 4.0 software to version 4.2 X.25 software on Cisco 700 series.	Upgrade the device to version 4.1 first (any feature set) and then upgrade to version 4.2 X.25 software image.
Syslog Analyzer: filters not taking effect immediately after changing.	It takes about 5 minutes for filters to take effect.	If you need the filters to take effect immediately, you must restart the Syslog Analyzer collector.
Syslog Analyzer: no messages on any generated syslog report.	Network devices are not sending messages to Essentials server.	Examine Syslog Analyzer Collector Status (Admin > Syslog Analysis > Collector Status). If numbers are all zero, then verify that network devices are sending messages to the Essentials server. Refer to online help for information on setting up Cisco IOS/Catalyst device.
Syslog Analyzer: message source is given as "???" in /var/syslog_info (Solaris only.)	Solaris bug - syslogd cannot resolve the source address of the network device sending the message.	Add the name resolution for this device to DNS, /etc/host, or your naming service. Alternatively, install the Solaris patch 103291-02. This will change the "???" to an octal IP address in brackets ([171.69.219.72.1.2]). This allows the format to be parsed by Syslog Analyzer.
Syslog Analyzer: syslog messages get appended to /var/log/syslog_info (Solaris) or to c:\Program Files\CSCOpx\log\syslog.log (Windows NT), but all Syslog Reports are empty.	Processes not running properly.	Check whether Syslog Analyzer is running properly (Admin > System Admin > Process Status); if not, restart it. Check that CMLogger, RmeOrb, and DBServer are running; if not, restart system.
	Timestamp problem.	If "Messages Processed" counter is not zero, check timestamps for message in syslog file. If there are two timestamps, and the second timestamp is current, then Syslog Analyzer uses the second timestamp. If the second timestamp is older than 7 days, the reports do not display it. If the "Message Processed" counter is zero, and the "Messages Filtered" counter is not zero, then change the filters. If the "Messages Processed" and "Messages Filtered" counters are zero, but the "Invalid Messages" counter is not zero, contact Cisco TAC or your customer support.
Syslog Analyzer: remote collector not running properly when installed and started on non-Essentials machine.	Incorrect configuration parameters.	Check remote collector table for name and status of remote collector: Admin > Syslog Analysis > Collector Status. View SAEnvProperties.ini and check that parameter BGSERVER is set to the hostname of the Essentials server. Perform a ping command using this host name to ensure that it is resolvable and reachable from this machine. On Windows NT, also ensure that the PORT parameter is set to 514.
Syslog Analyzer: remote collector - messages in syslog file, but not in reports.	Running incorrect version of Java.	Install Java 1.1.5 or higher.
	Remote collector stopped.	On Solaris, check using the command: /usr/bin/ps -ef | grep java Restart using: sh /opt/CSCOsac/lib/ sacStart.sh
	Remote collector not installed correctly.	On Windows NT, use Control Panel > Services to verify. If a Syslog_Collector is not listed, then re-install using SacNTService/install. If the collector is installed but not running, start the remote collector from Control Panel > Services. (You must specify the properties file using the -pr option.).
Syslog Analyzer: logging is enabled in the IOS/Catalyst device to send messages to Essentials, but is not working.	Messages sent to the Essentials server by network devices are logged by a process independent of the Syslog Analyzer. For Solaris, the logging process is syslogd. For Windows NT, it is the Essentials syslog service. The problem may be that the syslog daemons are not running properly.	Connect to the network device and generate a syslog message as follows: 1 . Telnet to the device and log in. The prompt changes to host >. 2 . Type enable and the enable password. The prompt changes to host #. 3 . Type configure terminal. You are now in configuration mode, and the prompt changes to host (config)#. 4 . Ensure that logging is enabled by entering logging on. 5 . Specify the IP address of the Essentials server to receive the router syslog messages; for example, type logging 123.45.255.90. 6 . Type End. For Solaris, view the file pointed to by the line "local7.info" in the file /etc/syslog.conf (default /var/log/syslog_info). If this file does not exist, create one, and make sure it is accessible by syslogd. For Windows NT, view the file in C:\Program Files \CSCOpx\log\syslog.log Send an HUP signal to syslogd (kill -HUP `cat/etc/syslog.pid').
Syslog Analyzer: logging is enabled in the IOS/Catalyst device to send messages to Essentials, but is not working.		7 . If the syslog message from the device is there, check the device configuration. Refer to the following "Possible Reasons" section. If the syslog message is not there, check that the syslog daemons are running properly. For Solaris, type: /usr/ucb/ps -aux | grep syslogd For Windows NT, go to Control Panel and ensure that Essentials syslog service is in "Started" state.
	The device might be configured incorrectly.	Ensure that the device is logging to the correct Essentials server. (Refer to the device system documentation for details on enabling syslog.)
Syslog Analyzer: the following error message is sent to the Windows NT Event Viewer when using remote NT Windows collector: "Could not start the Syslog Collector service on the server_name ERROR 0002: The system cannot find the file "specified" new messages not appearing in reports after changing syslog message file (defined using: Admin > Syslog Analysis > Change Storage Options).	Installation failure.	When installing on a remote Windows NT collector, use the command: SacNTService/install (not SacNTService.exe/install).
Syslog Analyzer: the following error messages are sent to the Windows NT Event Viewer when using remote Windows NT collector: "Could not start the Syslog Collector service on the server_name ERROR 1067: The process terminated unexpectedly" and "SacNTService: The service cannot be started without the properties file specified, please specify the properties file you want to use".	Configuration failure.	After the SacNTService is installed on a remote Windows NT collector, it must be configured using Control Panel > Services. Ensure that the Start up Parameters field contains the location of the SaenvProperties.ini file (for example, -pr c:\\temp\\SaenvProperties.ini) (Remember to use \\ to separate the directory paths.)
Syslog Analyzer: new messages not appearing in reports after changing syslog message file (defined using: Admin > Syslog Analysis > Change Storage Options).	New filename needs defining in configuration information.	On Windows NT, run the registry editor Regedit. Then, go to: HKEY_LOCAL_SYSTEM > System > CurrentControlSet > Services > crmlog Set Parameters to the name of the file for logging syslog messages. On Solaris, modify the /etc/syslog.conf file. (Refer to the Solaris man pages for more information.)
Syslog Analyzer: server runtime error when running Tasks > Syslog Analyzer tasks.	CMLogger is not running.	1 . Log in to Essentials as admin. 2 . Select Admin > System Admin > Start Process. 3 . Start the system. If the task still fails, select Admin > Troubleshooting > Process Failures to get a list of Essentials back end processes that have failed. 4 . Select Admin > Troubleshooting > Self Test. Click Create to create a report. Click Display to display the report. 5 . Select Admin > Troubleshooting > Collect Server Info. 6 . Click on Product Database Status link to get detailed database status. 7 . Contact the Cisco TAC or your customer support.
Syslog Analyzer: Unexpected Device Report (Tasks > Syslog Analysis > Unexpected Device Report) contains syslog messages that should not be in the Standard Report (Tasks > Syslog Analysis > Standard Report).	The Syslog Standard Device Report displays messages for all selected devices and dates. These devices must be managed by Essentials inventory. This error could arise if the device is not managed by Essentials inventory.	Add device to inventory by selecting Admin > Inventory > Add Device. After adding the device, messages formerly in the Unexpected Device Report will not be transferred to the Standard Report, but new messages for the device will be displayed in the Standard Report.
	If the device is managed by Essentials inventory, it could be a name resolution problem. (Syslog analyzer uses all IP addresses associated with the device name to try to map it to a device managed by Essentials Inventory.)	Verify the device name to IP address mapping as follows: For Windows NT, view the syslog.log file in C:\Program Files\CSCOpx\log. For Solaris, view the syslog_info file in /var/log. Note the source of the syslog messages from the device (the source is the host name appearing immediately after the timestamp). Obtain a list of IP addresses (perform nslookup on the device name at the command prompt). Generate a detail device report (Tasks > Inventory > Detailed Device Report) for the particular device. Check the Network Address column and verify that the IP addresses returned from nslookup appear on the list.
		If any IP addresses are not on the list, then the device name to IP address mapping is incorrect. Update the naming services (DNS, /etc/hosts, etc.) with missing IP addresses.
System Configuration: server runtime error when running Admin > System Admin > Device Configuration task.	CMLogger is not running.	1 . Log in to Essentials as admin. 2 . Select Admin > System Admin > Start Process. 3 . Start the System. If Admin > System Admin > Device Configuration option still fails, select Admin > Troubleshooting > Process Failures to get a list of Essentials backend processes that have failed. 4 . Select Admin > Troubleshooting > Self Test. Click Create to create a report. Click Display to display the report. 5 . Select Admin > Troubleshooting > Collect Server Info. 6 . Click on Product Database Status link to get detailed database status. 7 . Contact the Cisco TAC or your customer support.
"TFTP server on the device timed out" error appears during operation on Cisco 700 Series devices.	Cannot access TFTP client on Essentials server	Check whether the TFTP client is available on the Essentials server. Solaris - /usr/bin/tftp Windows NT - nnt\system32\tftp.exe

Posted: Wed Oct 6 12:06:07 PDT 1999
Copyright 1989-1999©Cisco Systems Inc.