|
|
This chapter describes the Automation system in Cisco Info Center, how to set it up, and how to use it.
This chapter includes the following sections:
Automation can be used to manage the large numbers of events received from the Cisco Info Server. For example, Automation can delete all events from the system that have a severity of clear and that have been in the system for longer than a specific period of time.
You can also use Automation to change event data or run an external command when certain events occur in the Cisco Info Server. For example, when a critical event occurs in a network, you can use Automation to call a person after a specific period of time.
Two sets of information exist inside an Cisco Info Server configured to do Automation:
The Automation Builder tool allows you to create triggers and actions using Cisco Info Server SQL.
Within Cisco Info Center, SQL is used to control the various Automation mechanisms. SQL is a language for manipulating and extracting information from databases. The Cisco Info Server SQL is a sub-set of the full SQL.
For more information and a full command reference, see Section, "Cisco Info Server SQL", in Appendix A.
Within the Cisco Cisco Info Server, a database called alerts exists. This database contains the status table. This table contains the current event information in the Cisco Info Server. The fields contained within this database are what you view in the Event List.
A trigger is a mechanism for detecting when an Automation action should be activated. Triggers use a filter to interrogate the alerts.status table. When events match that filter, the associated Automation action is activated.
When any event matches the filter, the trigger is said to be primed. See Figure 5-1.
Each trigger can have links to two types of actions:
Four types of triggers exist. The trigger type is defined by the way it calls the action. The following list shows the four types of triggers:
These trigger types are described in detail in Section, "Types of Automation Triggers", in Appendix C.
Actions can consist of a data effect and/or an external effect. Actions can be called by a trigger once or many times, depending upon the trigger's execution mode.
When an action is called, it has available to it all the fields from the event being used to call it. They may be substituted into the effects by referring to them as @fieldname where fieldname is the name of a field in an alert record.
The available fields are:
Additional fields may be available when the Cisco Info Server has had fields added to it.
Data effects consist of a Cisco Info Server SQL statement or statements, which manipulate the date held in the Cisco Info Server tables.
External effects consist of UNIX commands or an external application.
The Automation Builder is used to create triggers and the links to actions.
The recommended method of starting the Automation Builder, is to click on the Automation button on the Conductor. It can also be run from the command line as:
host% $OMNIHOME/bin/nco_auto
The Automation window comprises a menu bar, a special scrolled window, and a Status line at the bottom, as shown in Figure 5-2.
Automation Window
The scrollable area has two vertical scrollbars. There are two columns of icons: one set representing triggers and the other representing actions.
The left hand scrollbar scrolls the triggers and the right hand scrollbar scrolls the actions. This scrolling takes place independently of the other column of icons.
When a trigger icon is shown grayed out, the trigger is currently inactive.
Figure 5-3 shows the Automation Trigger window. The Condition tab is displayed by default.
1. Specify the name of the trigger in the Name field. The name appears under its icon in the Automation window.
2. Specify an SQL statement in the Condition field or drag and drop a filter from the Filter Builder. The following example shows how an SQL statement begins:
select * from alerts.status where ...
It must end with a semicolon (;). For details on creating Cisco Info Server SQL statements, see Section, "Cisco Info Server SQL", in Appendix A.
3. Specify a value in the Sample Rate field to determine how often the trigger should be checked. The value is rounded up to the nearest five seconds when in use. With a timed sample rate and a delayed trigger, the threshold is the number of sample intervals evaluated. Therefore, when the sample rate is 20 and the threshold is 5, the trigger is evaluated every 20 seconds, and start the trigger after 100 seconds. When the sample rate is 0, the trigger is a real- time trigger. In this case, the trigger is evaluated on every update to the Cisco Info Server. The updates may come in batches or one at a time, and so there is no relation between time and threshold.
To add settings to the trigger, click on the Settings tab in the Automation Trigger window. Figure 5-4 shows the trigger settings
1. Select the trigger type:
(a) Edge
This trigger calls its ascent action only when the number of rows matched increases from zero to a defined number. The trigger does not then call another action, except when you specify a descent action. A descent action is called when the number of matched rows falls below zero.
(b) Level
This trigger calls its ascent action at a given sample rate as long as the number of rows matched is at least one.
(c) Delayed Edge
This trigger works in the same way as the edge trigger except that when the number of rows matched increases from zero to a defined number, the trigger waits a given delay period. When the condition still matches the action, it is called, and the trigger continues as an edge trigger.
(d) Delayed Level
This trigger works in the same way as the level trigger except that when the number of rows matched increases from zero to a defined number, the trigger waits a given delay period. When the condition still matches after the delay, the action is called, and the trigger continues as an level trigger.
2. Select the execution mode:
(a) Set For Each Matched Row to set the trigger to execute its actions for every event that matched its condition. This mode allows for a many-to-many mapping between particular events and actions taken.
(b) Set Once Only to set trigger to only execute its actions with the data from the first match of the condition. This allows a one-to-one mapping between overall state and actions taken.
3. In the Ascent Action field, specify the name of an action to be executed on the upward edge of an edge trigger or while the condition exists on a level trigger.
The action at this point does not have to exist. When you wish to create the action or edit an existing action, click on the Edit button. When the action does not exist, the application prompts to see when you want to create a new action. When you do, or the action already exists, the Automation Actions window is displayed. See the next section for details on editing that window and its associated action.
4. In the Descent Action field, specify the name of the action to be executed on the downward edge of an edge trigger. A descent action is never called for a level trigger. Either or both action fields may be left empty.
To add comments to the trigger, click on the Comments tab in the Automation Trigger window. Figure 5-5 shows the comments field.
When the trigger is applied or tested, the SQL syntax is checked automatically for errors. When any errors appear, an error dialog is displayed detailing the nature of the error. When the error is in the syntax, a Parse Error message is displayed. You may then return and correct the error.
1. Click on the Active toggle option. When it is not enabled, this trigger is not used when Automation is evaluating triggers.
2. Select Apply to write all the changes to the trigger into the system. The SQL in the condition is also checked. When the SQL is not correct, an error message appears. You must correct the errors before you can proceed.
3. Select Close to remove the window from the display.
Note, you can also activate the trigger from the Automation window. See Figure 5-5. Click on the right mouse button on the trigger, then select the Active option.
To revert the contents to how they were before the window was opened or last applied, select the Undo option.
Figure 5-6 shows the Automation Action window.
1. Specify a unique name for the action in the Name field.
2. Specify the Cisco Info Server SQL statements in the Data Effect field. When the SQL for the action is executed, the commands update the entire Cisco Info Server database, not just the records selected in the trigger.
When you want to use only the events returned by the trigger, you should use the @ field names. For example, the following statement sets the Severity to 4 (Major) on only those events returned by the trigger:
update alerts.status set Severity=4 where Serial=@Serial;
To add an external effect to the action, click on the External Effect tab in the Automation Action window. Figure 5-7 shows the external effects.
1. Specify the full path name of the executable to be run in the Executable field, for example: (/usr/application/bin/<program_name>). The command can have environment variables referenced in its definition, for example:
$NETSOFT/bin/<program_name>
2. Specify the arguments passed to this executable program when it is run in the Arguments field. The arguments can refer to the @ variables as in the data effect, for example:
-node @Node -error @Summary -severity @Severity
3. Specify the name of the machine on which the executable should be run in the Host field. The host must have a process control agent running on it for this to work. The process control agent need not be managing Cisco Info Center components and can be placed on a system specifically so Automation's external effects can be carried out on that system.
4. Specify the UNIX user ID with which to run this executable in the Effective User field.
5. Click on the Apply button to write all the changes to the action into the system. The Data Effect is checked for syntax and semantic errors and when any are present, an error window appears. Alternatively, click on the Close button to remove the window from the display. When you click on the Undo button, the contents reverts to how they were when the window was opened or last applied.
To add comments to the action, click on the Comments tab on the Automation Action window.
1. Click on the Enabled toggle option. When it is not enabled, this action is not used.
2. Click on the Apply button to write all the changes to the action into the system. The SQL in the condition is also checked. When the SQL is not correct, an error message appears. You must correct the errors before you can proceed.
3. Click on the Close button to remove the window from the display.
To revert the contents to how they were before the window was opened or last applied, click on the Undo button.
To display which triggers use an action, click on an Action icon. Yellow arrows point to the triggers that use the action as their ascent action. Blue arrows point to triggers which use the action as their descent action.
To display the trigger definition or action definition window as appropriate, double-click on either a Trigger or Action icon. The window is displayed with all the settings for the trigger or action already filled in. You can edit these in the same way as when you create a trigger or an action.
As well as being able to specify the name of the ascent and descent actions in the trigger editing window, you can graphically associate triggers and actions. To associate a trigger to an action:
1. Press the Shift key.
2. Click on the mouse button on a Trigger icon.
3. Drag the mouse pointer over an Action icon. A dashed arrow line appears across the display.
4. Release the mouse button to set that action as the ascent action of the trigger.
To set the action as a descent action, press the Control key and click on the mouse button over a trigger icon, then drag the pointer over an action icon. Release the mouse button to set that action as the descent action of the trigger.
This also works for action to trigger. Press the Shift key and click on the mouse button on an action, then drag the mouse pointer over a trigger. Release the mouse button to set that trigger to use the action as its ascent action. The Control key is used in the same way to set the descent action.
Popup menus are available when you click on the right mouse button over a Trigger icon, an Action icon or the scrolled window background.
When you click on the right mouse button over a trigger or action, the popup menus give you the option to delete or edit the trigger or action. The edit option is functionally the same as when you double-click on the icon.
To edit the Automation trigger, select the Edit option from the popup menu.
To clear the ascent action for the trigger, select the Clear Ascent Action option from the popup menu.
To delete the trigger, select the Delete option from the popup menu.
To edit the action, select the Edit option from the popup menu.
To delete the action, select the Delete option from the popup menu.
You can also click on the right mouse button on the window background to display three options.
Changes made to the Automation triggers and actions are automatically stored in the Cisco Info Server. To allow for the recording and reuse of triggers and actions, Import and Export options are available.
Select the File menu's Export option to display the Automation Export window.
The window contains the standard elements of a file selection window with the addition of two elements: the trigger/action display selector and the Export item selector.
The trigger/action display selector controls what is shown in the Export item selector. It is comprised of two check boxes: one selects the display of triggers, the other selects the display of actions.
The Export item selector allows you to select which triggers and actions are to be exported to the file. This is a multiple selection list; click on an item to select that item. Press the Control key and click on an item to add the item to the current selection. Shift-click on an item to select all the entries between the first selected item and the currently selected item.
The items are the names of the triggers and actions, each followed by either (T) for a trigger or (A) for an action.
Click on the OK button to export the currently selected items on the list to the file named in the Selection text field.
Select the File menu's Import option to display the Automation Import window.
Select the file to be imported. Note, when the imported file contains triggers or actions whose names match currently existing triggers or actions, the imported trigger or action is discarded. To import same named triggers, first delete the conflicting triggers and actions.
This section describes the Automations provided with Cisco Info Center.
delete from alerts.details where Identifier not in ((select Identifier from alerts.status));
None
Active
600 seconds
delete from alerts.jounral where Identifier not in ((select Serial from alerts.status));
None
Active
600 seconds
select * from alerts.status where Severity <> 0 and ObjectStatus = 4
update alerts.status via @Identifier set ObjectStatus = @OldObjectStatus;
InActive
7 seconds
select * from alerts.status where ObjectType like 'endpoint' and Summary like 'Deleted';
update alerts.status set Severity = 0, ObjectStatus = 4, class = 1070 where Serial <> @Serial and ObjectStatus <> 4 and LastOccurrence <= @LastOccurrence and NEName = '@NEName' and PSAP = '@PSAP' and DSAP = '@DSAP' and Slot = '@Slot' and ((ObjectType = '@ObjectType') or (ObjectType = 'connection'))
InActive
6 seconds
select * from alerts.status where ((ObjectType = 'line.access') or (ObjectType = `line.trunk))' and Summary like 'Deactivated';
update alerts.status set Severity = 0, ObjectStatus = 4, class = 1070 where Serial <> @Serial and ObjectStatus <> 4 and LastOccurrence <= @LastOccurrence and NEName = '@NEName' and PSAP = '@PSAP' and Slot = '@Slot' and (ObjectType = '@ObjectType');
InActive
6 seconds
select * from alerts.status where ObjectType = 'card' and Summary like '[rR]emoved';
update alerts.status set Severity = 0, ObjectStatus = 4, class = 1070 where ObjectStatus <> 4 and Serial <> @Serial and LastOccurrence <= @LastOccurrence and NEName = '@NEName' and ObjectType = 'card' and Slot = '@Slot';
InActive
7 seconds
select * from alerts.status where ObjectType = 'NE' and Summary like 'removed from network';
update alerts.status set Severity = 0, ObjectStatus = 4, class = 1000 where Serial <> @Serial and ObjectStatus <> 4 and Class <> 1000 and LastOccurrence <= @LastOccurrence and NEName = '@NEName' and ObjectType = 'NE';
InActive
7 seconds
select * from alerts.status where ObjectType like 'port' and (Summary like 'Deactivated') or (Summary like 'deleted';
update alerts.status set Severity = 0, ObjectStatus = 4, class = 1070 where ObjectStatus <> 4 and Serial <> @Serial and LastOccurrence <= @LastOccurrence and NEName = '@NEName' and PSAP = `@PSAP' and Slot = '@Slot' and ObjectType = `@ObjectType';
InActive
8 seconds
delete from alerts.status where (Manager = `ConnectionMonitor') and (Summary like `A probe process running .* has connected');
update alerts.status set Severity = 0 where Manager = `@Manager' and (Summary like `A probe process running .* has disconnected');
InActive
15 seconds
delete from alerts.status where (StateChange < getdate - 86400);
None
InActive
920 seconds
The external action re-generates the look-up file for the Cisco SV+ Mediators and also updates the alerts.status databases for the events returned by the trigger with the updated NEAddress and NEType.
select * from alerts.status '(Manager in ('SV+','NNM')) and ((NEName not like 'NETWORK') and (((NEAddress = '') and (NEType = 999)) or (((NEAddress <> '') and (NEType = 999)) or (((NEAddress = '') or (NEAddress = '16.0.0.0')) and (NEType in (1,2,3,4,5,6,11,12,0))))))';
/opt/Omnibus/utils/generate_lookups/generate_lookups.auto
<Cisco_SV+_hostname>
0
Active
20 seconds
delete from alerts.status where (Severity = 0) and (StateChange < getdate - 1800);
None
InActive
241 seconds
This Automation sets the Event List to flash for uncleared events that have not been acknowledged.
update alerts.status set Flash = 1 where (Acknowledged = 0) and ((Severity > 1) and (Flash = 0))
None
InActive
4 seconds
update alerts.status set Flash = 0 where (Acknowledged = 1) and ((Severity > 1) and (Flash = 1))
None
InActive
3 seconds
Trigger Condition | Select * from alerts.status where Summary like `Link Up'; |
Action Data Effect | Update alerts.status set Severity = 0 where Summary like `Link Down' and Node = `@Node' and AlertKey = `@AlertKey' and LastOccurrence <= @LastOccurrence; |
This Automation detects a Link Up event and clears any Link Down event which came from the same node, when it was matched on the AlertKey field, and was older. There should be a similar Automation detecting Node Up and clearing Node Down. This situation can be applied to many different events, resulting in many different Automations to handle them. All of these Automations require processing resources from the Cisco Info Server to find events which require action, execute Automations, and update clients.
An alternative method of resolving this problem would be to create one Automation to recognize many different types of association. This is possible as the only component that differs between Automations is often the text or fault code that is being searched for. This requires a generic way of looking at certain faults.
By taking the majority of standard up/down Automations from different management systems, it is possible to create a generic way of treating them. The first step in this approach is to analyze an event record and extract the following information:
This information could be constructed and used in a model as follows:
Use the standard field Type. From this we could set:
1 = Problem. These events are cleared later by another event.
2 = Resolution. These events clear matching Type=1 events.
To do this we need to find a corresponding and opposite event. Use the following two standard fields:
Use the standard field AlertGroup. This should be set so the values match for other events which clear or will be cleared by this event. For example, Link Up, Link Down, and Link Protocol Fail might all have an AlertGroup of Link Warnings.
When the fault related information has been gathered, a small number of Cisco Info Server fields are used within the Automations:
| Actual Event Type | Alert Group Value | Type Value |
|---|---|---|
ASMTempAboveNormal | ASMTempAboveNormal | 1 |
ASMTemperature Normal | ASMTempAboveNormal | 2 |
AXISUserLogin | AXISUserLoginLogout | 1 |
AXISUserLogout | AXISUserLoginLogout | 2 |
shelfLossLmiAlarmTrap | shelfLossLmiAlarmClear | 1 |
shelfLossLmiAlarmClearTrap | shelfLossLmiAlarmClear | 2 |
@NEName, @AlertKey
These two fields are used to uniquely identify an Enterprise element.
@LastOccurrence
This standard time stamp field is used to ensure that a problem event is not cleared by a resolution event which occurred before the problem event itself.
With the fields set, Automations can now correlate the problems and resolutions independently of the source of the events. The Automation can also expire events based on an expiration time in the Type field. The following example shows the set of Automations used to achieve this functionality.
| Trigger | Action | Purpose |
|---|---|---|
GenericClear | GenericClear | Detects and deletes events of @Type = 2 (resolutions), then clears (sets severity to 0) any associated problems |
The generic clear Automation executes in real time. As this Automation substantially reduces the overall numbers of Automations required, real time processing overhead can be incurred.
select * from alerts.status where Type = 2
update alerts.status set Severity = 0, BitMapField ='@BitMapField', ObjectStatus = 4 where Type = 1 and NEName = `@NEName' and AlertGroup = `@AlertGroup' and AlertKey = `@AlertKey' and LastOccurrence <= @LastOccurrence;
delete from alerts.status via `@Identifier';
InActive
3 seconds
The generic clear Automation above is applicable to events that have a resolution and fault state. For events that have a pair of associated events, but not a case of fault and resolutions, the Type 3/Type 4 Automations are used.
These Automations are applicable to events that indicate the latest state of the card.
For example:
In the above sequence, the final state is the Card Removal. This event should clear the Card Insert event, indicating the latest status of the card action.
Now, when the card is inserted, the current state of the Card is Inserted and this new event should clear the Card Removed event.
In the Generic Clear, the Resolution clears the Fault event, while in the above example neither of the events are Resolution or Fault Cases. These events indicate the current state of the Object and the Automation provides a toggle effect.
The Automation uses the same fields as the Generic Clears.
| Actual Event Type | Alert Group Value | Type Value |
|---|---|---|
FunctionalModuleRemove | FuncModInsertRemove | 3 |
FunctionalModuleInsert | FuncModInsertRemove | 4 |
Switch Card Removed | CardInserted | 3 |
Switch Card Inserted | CardInserted | 4 |
Endpoint Added | endpointdeleted | 3 |
Endpoint Deleted | endpointdeleted | 4 |
This Automation selects all events that have the Type = 3 and Grade <> 5, and deletes the corresponding Type 4 event. The action also updates the Grade = 5 for the Type 3 event selected in the trigger. This has been done so the Automation does not keep executing the trigger once this condition has been satisfied.
select * from alerts.status where Type = 3 and Grade <> 5
delete from alerts.status where Type = 4 and LastOccurrence <= @LastOccurrence and AlertGroup = `@AlertGroup' and NEName = `@NEName' and AlertKey = `@AlertKey';
update alerts.status via `@Identifier'set Grade = 5;
InActive
5 seconds
This Automation selects all events that have the Type = 4 and Grade <> 5, and deletes the corresponding Type 3 event. The action also updates the Grade = 5 for the Type 4 event selected in the trigger. This has been done so the Automation does not keep executing the trigger once this condition has been satisfied.
select * from alerts.status where Type = 4 and Grade <> 5
delete from alerts.status where Type = 3 and LastOccurrence <= @LastOccurrence and AlertGroup = `@AlertGroup' and NEName = `@NEName' and AlertKey = `@AlertKey';
update alerts.status via `@Identifier'set Grade = 5;
InActive
5 seconds
Posted: Thu Apr 1 10:59:45 PST 1999
Copyright 1989-1999©Cisco Systems Inc.