|
|
July 1998
This release note describes the features, enhancements, and caveats for Release 2.0 of NetFlow FlowCollector. The following information is included:
FlowCollector 2.0 is used with the NetFlow Services and NetFlow data export features on Cisco 7500, 7200, and 7000 series routers and the Multilayer Switching (MLS) and NetFlow data export features on Catalyst 5000 series switches.
FlowCollector 2.0 supports NetFlow export datagram format Versions, 1, 5, and 7 (Versions 2 through 4 and Version 6 were either not released or are not supported by FlowCollector).
The following new features and enhancements are available in FlowCollector 2.0.
FlowCollector 2.0 now supports the following new aggregation schemes:
The output of this aggregation scheme consists of one record for each unique combination of source IP address, source port, destination port, and protocol present in the flow data received by FlowCollector during the current collection period. Each output record contains the following fields:
| Key fields: | srcaddr, srcport, destport, protocol |
| Value fields: | packet count, byte count, and flow count |
The output of this aggregation scheme consists of one record for each unique combination of source IP address, destination IP address, source port, destination port, protocol, input interface, output interface, source autonomous system number, and destination autonomous system number present in the flow data received by FlowCollector during the current collection period. Each output record contains the following fields:
| Key fields: | srcaddr, dstaddr, srcport, dstport, protocol, input interface, output interface, src_as, dst_as |
| Value fields: | packet count, byte count, and flow count |
The output of this aggregation scheme consists of one record for each unique combination of input interface, output interface, masked source IP address, masked destination IP address, source mask, and destination mask present in the flow data received by FlowCollector during the current collection period. Each output record contains the following fields:
| Key fields: | input interface, output interface, masked srcaddr, masked dstaddr, src_mask, dst_mask |
| Value fields: | packet count, byte count, and flow count |
FlowCollector 2.0 has moved from a process-based architecture to a thread-based architecture.
Because of the move from a process-based architecture to a thread-based architecture, and because file I/O performance has been improved through block I/O, FlowCollector 2.0 is capable of handling approximately five times the load as compared to the 1.0 FCS version of FlowCollector under certain identical test conditions.
For FlowCollector 2.0, the header of a data file created by FlowCollector includes three additional fields and some changed fields.
The added fields are:
The FLOWS and MISSED fields provide data about received NetFlow export traffic during a collection period. The RECORDS field provides applications consuming FlowCollector data files with a convenient count of the number of records present in the data file.
The changed fields are:
The following is an example of the new data file header:
SOURCE 171.71.34.37|FORMAT A|AGGREGATION ASMatrix|PERIOD 15|STARTTIME 881972378| ENDTIME 881973278|FLOWS 39757|MISSED 0|RECORDS 5
The I/O mechanism used in FlowCollector 2.0 ensures the integrity and completeness of each data file FlowCollector generates. If data I/O should fail for any reason, FlowCollector logs error messages through syslog and nfc.log. FlowCollector does not generate an incomplete data file.
FlowCollector now supports NetFlow export datagram format Version 7 (from Catalyst 5000 series switches) for all the existing FlowCollector aggregation schemes.
If your network includes switching devices that support NetFlow export datagram format Version 7, you can use the USE_SHORT_CUT_ADDRESS_AS_SOURCE_IP configuration parameter (refer to the section "Retaining Router IP Addresses for Switched Export Packets," later in this publication) in the nf.resources file to configure FlowCollector to retain the IP address of the short-cut router as the source of data switched through a Cisco Catalyst 5000 series switch. In such a case, the switch represents a short-cut path around the router.
The ROUTER_GROUPNAME configuration parameter in the nf.resources file allows you to substitute a user-specified IP address or label for a set of IP addresses from which FlowCollector will receive NetFlow export datagrams.
The syntax of the parameter is:
ROUTER_GROUPNAME label {
a.b.c.d
.
.
.
w.x.y.z
}
where label is either an IP address or an ASCII word. Each of the IP addresses in the body of the ROUTER_GROUPNAME block must be on a separate line. An example of a ROUTER_GROUPNAME definition follows:
ROUTER_GROUPNAME blab-gateway {
172.69.1.172
172.69.1.173
192.71.1.25
}
If applicable, the mapped ROUTER_GROUPNAME will be used with all aggregation schemes, but FlowCollector uses the real IP address to report errors involving receipt of an invalid or unsolicited NetFlow export packet.
FlowCollector now permits you to specify the source addresses from which it will receive NetFlow export packets. You can use the ACCEPT_PACKETS_FROM parameter in the nf.resources file to prevent FlowCollector from accepting unsolicited packets.
The syntax of the ACCEPT_PACKETS_FROM parameter is
ACCEPT_PACKETS_FROM {
a.b.c.d
.
.
.
w.x.y.z
}
where each of the IP addresses (or ROUTER_GROUPNAME labels) defined in the body of the ACCEPT_PACKETS_FROM block must be on a separate line. An example of a ACCEPT_PACKETS_FROM definition follows:
ACCEPT_PACKETS_FROM {
131.108.2.1
131.108.2.2
131.108.2.3
blab_gateway
}
If your network includes switching devices that support Version 7 NetFlow export datagrams, you can configure FlowCollector to retain the IP address of the short-cut router as the source of data switched through a Cisco Catalyst 5000 series switch. To do this, you must edit the USE_SHORT_CUT_ADDRESS_AS_SOURCE_IP parameter in the nf.resources file. The syntax of the parameter is
USE_SHORT_CUT_ADDRESS_AS_SOURCE_IP value
where value is either yes or no. The default setting is no. If you change the setting to yes, FlowCollector uses the IP address of the bypassed router as the source of the corresponding flow.
You can now specify the location of a script file that FlowCollector will execute after it has written a new data file. This capability makes it easier for your client applications to process a new data file without having to poll for it. FlowCollector invokes the script with the absolute path name of the newly written FlowCollector data file. FlowCollector expects the location of your user-supplied script to be defined by the USER_SCRIPT_LOCATION parameter in the nf.resources file. This parameter is read only at startup.
For example, if the path name for your script is /opt/CSCOnfc/my_script.sh, the revised USER_SCRIPT_LOCATION parameter in the nf.resources file would read:
USER_SCRIPT_LOCATION /opt/CSCOnfc/my_script.sh
FlowCollector now periodically appends the absolute path names of data files that it has generated to a list in a log file named filesready.YYYY_MM_DD, where YYYY_MM_DD represent the year, month, and day timestamp used to identify the file. The filesready file is located with the other log files in the $NFC_DIR/logs directory. There is one such file per DataSetPath setting per day.
Typically, a client application would read this file every n minutes, process it to determine the names of any newly added data files, and then retrieve those new data files.
After it finishes writing a new data file, FlowCollector appends the absolute path name of the new data file onto the list in the filesready file. If FlowCollector deletes some data files as instructed by the FileRetain setting in its thread definitions, it updates the corresponding filesready file. The following example shows the partial contents and organization of a typical filesready file.
/opt/CSCOnfc/Data/1998_02_11/171.71.34.79/Protocol/171.71.34.79.2135 /opt/CSCOnfc/Data/1998_02_11/171.71.34.79/DetailASMatrix/171.71.34.79.2136 /opt/CSCOnfc/Data/1998_02_11/171.71.34.79/Protocol/171.71.34.79.2136 /opt/CSCOnfc/Data/1998_02_11/171.71.34.79/DetailASMatrix/171.71.34.79.2137 /opt/CSCOnfc/Data/1998_02_11/171.71.34.79/Protocol/171.71.34.79.2137 /opt/CSCOnfc/Data/1998_02_11/171.71.34.79/DetailASMatrix/171.71.34.79.2138 /opt/CSCOnfc/Data/1998_02_11/171.71.34.79/Protocol/171.71.34.79.2138 /opt/CSCOnfc/Data/1998_02_11/171.71.34.79/DetailASMatrix/171.71.34.79.2139 /opt/CSCOnfc/Data/1998_02_11/171.71.34.79/Protocol/171.71.34.79.2139
FlowCollector Version 2.0 includes a utility that provides an easy way to generate all the debugging information necessary for support and troubleshooting purposes. The utility is called show-tech, and is invoked by entering the following command line at the UNIX prompt:
$ $NFC_DIR/bin/nfcollector show-tech
When invoked, the show-tech command creates a log file named show-tech.log in the directory $NFC_DIR/logs, and saves the following information in it:
FlowCollector 2.0 accepts data in NetFlow export record format Version 1, Version 5, and Version 7.
You no longer need to specify which version of the export record format FlowCollector should expect on a port. FlowCollector now automatically detects the version number in a received NetFlow export packet.
The NetFlow FlowCollector user interface (NFUI) now contains embedded help menus to assist you in navigating through the user interface and understanding menu operations. The help menus explain all the options available for retrieving, configuring, and reviewing the FlowCollector startup configuration parameters and statistics. To display the help menu, enter h from within any menu.
There are two new configuration files in FlowCollector 2.0:
For the purpose of aggregation, you may choose to specify your own labels to be used in place of autonomous system numbers. You can also choose to aggregate one or more blocks of autonomous system numbers and identify them with a label. Any undefined autonomous system numbers are aggregated together and appear as "Others" in the data file.
The following lines show examples of the syntax used in these configuration files:
1:Your_Network 2 10, 15 20, 30:My_Network 35, 40:My_Network
In this example, the entry:
| 1:Your_Network | Indicates that a flow with an autonomous system number of 1 is aggregated under the label Your_Network in the data file. |
| 2 | Indicates that a flow with an autonomous system number of 2 is aggregated under the label 2 in the data file. |
| 10, 15 | Indicates that a flow with an autonomous system number in the range from 10 to 15 is aggregated under a label that is the same as its autonomous system number. For example, if a flow has the autonomous system number 13, the label of the flow in the data file is 13. Flows within the range, but with different autonomous system numbers, are aggregated individually in the data file. |
| 20, 30:My_Network 35, 40:My_Network | Indicate that the autonomous system number is replaced by the label My_Network, and that flows within the specified ranges are aggregated together in the data file under the label My_Network in the data file. |
FlowCollector 2.0 allows filtering of NetFlow export records by the following new fields:
For the purpose of aggregation, you may choose to specify a range of source or destination ports. A range of ports is defined by using a comma to separate two numbers (an optional space can be added for legibility). A range can span any set of ports up to the maximum number of ports available on the system (currently 65,535). The following example shows a range of ports:
50, 100
You can also define a range of source or destination ports to be treated as one logical port, and assign a label to represent that range of ports. The following example shows a range of ports to be treated as the logical port named 10K_19K_Port_Range.
10000, 19999: 10K_19K_Port_Range
In this case, traffic is aggregated and reported for the logical port 10K_19K_Port_Range, rather than for each of the individual port numbers in the range.
The following lines show examples of the syntax used in defining ports:
21:ftp 88 50, 100 10000, 19999: 10K_19K_Port_Range 20000, 29999: My_Range 40000, 49999: My_Range
In the example above, the entry:
| 21:ftp | Indicates that a flow with port number of 21 is aggregated under the label ftp in the data file. |
| 88 | Indicates that a flow with a port number of 88 is aggregated under the label 88 in the data file. |
| 50, 100 | Indicates that a flow with a port number in the range from 50 to 100 is aggregated under a label that is the same as its port number. For example, if a flow has the port number 75, the label of the flow in the data file is 75. Flows within the range, but with different port numbers, are aggregated individually in the data file. |
| 10000, 19999: 10K_19K_Port_Range | Indicates that the port number of any port in the range is replaced by the label 10K_19K_Port_Range, and that flows within the range are aggregated together under the label 10K_19K_Port_Range in the data file. |
| 20000, 29999:My_Range 40000, 49999:My_Range | Indicate that the port number of any port in the two ranges is replaced by the label My_Range, and that flows within the specified ranges are aggregated together under the label My_Range in the data file. |
FlowCollector 2.0 is capable of detecting a previously installed version of FlowCollector and will preserve the existing configuration files.
During the installation process, the installation script prompts you to choose between one of two installation methods:
The default is to install the new configuration files and save the existing configuration files with .old extensions.
On Cisco 7500 series routers with a Route Switch Processor (RSP) and Versatile Interface Processors (VIP), the VIP can be configured to switch packets it receives with no per-packet intervention on the part of the RSP. The dCEF feature allows the VIP to capture NetFlow statistics and perform NetFlow data export directly from the VIP in a manner similar to NetFlow data export from the RSP when running CEF on the RSP. Each VIP maintains its own independent flow cache and can generate its own export packets containing statistics on expired flows.
To allow FlowCollector to distinguish between export record sequence numbers sent by different VIPs, two new fields have been added to the header of the Version 5 export datagrams:
These two fields are used to distinguish between controllers (engines). NetFlow datagrams exported by an RSP have an engine type of 0 (zero) and an engine ID that matches the previous 0 value exported by earlier Cisco IOS software. NetFlow datagrams exported by a VIP controller have an export engine type of 1 and an engine ID that corresponds to the slot number in which the VIP resides.
Read this section before installing FlowCollector 2.0. It contains information related to platform support, product packaging and documentation, and preinstallation preparation before installing FlowCollector.
FlowCollector is available for the following platforms:
FlowCollector requires at least 1 MB of disk space for its binary and configuration files.
FlowCollector generates output files containing aggregated data. These files require additional disk space; the exact amount depends on the flow arrival rate, collection interval, number of aggregation schemes specified, and data file retention policies.
FlowCollector 2.0 is a full (non-incremental), packaged product that replaces earlier versions of FlowCollector.
The FlowCollector 2.0 product suite includes the following files:
The NetFlow FlowCollector Installation and User Guide provides task-based information for installing and using FlowCollector. The guide is available in hard copy and as an electronic document on the World Wide Web and on the Documentation CD-ROM.
Table 1 contains a list of patches to HP-UX Version 11.0. These patches are produced and distributed by Hewlett-Packard. You should acquire these patches from Hewlett-Packard and install them on your workstation before attempting to run FlowCollector 2.0.
| Name | Version Number | Description |
|---|---|---|
| PHCO_14704 | 1.0 | libc cumulative patch |
| PHCO_14859 | 1.0 | Cumulative 10.20 libc compatibility support |
| PHKL_14750 | 1.0 | Fixes pthread_cond_timedwait(3T) error return |
| PHKL_14762 | 1.0 | POSIX semaphore deadlock/hang |
| PHNE_15047 | 1.0 | Cumulative ARPA Transport patch |
| PHSS_14577 | 1.0 | HP aC++ runtime library components (A.03.10) |
| PHSS_14582 | 1.0 | milli.a patch |
| PHSS_14583 | 1.0 | LIBCL patch |
FlowCollector 2.0 requires that the value of the data size (maxdsiz) system tunable parameter in HP-UX Version 11.0 be set to at least 524288 KB for satisfactory FlowCollector 2.0 operation. The HP-UX Version 11.0 default for maxdsiz is 65536 KB. Use the UNIX ulimit -a command to determine the current value of this parameter. If the value of this parameter falls below the recommended minimum value, you must change the value to 524288 KB and then rebuild your kernel.
If you attempt to install FlowCollector 2.0 on a workstation whose maxdsiz value is below 524288 KB, the FlowCollector 2.0 installation script examines the current value, detects that it doesn't match the recommended minimum value, displays the following message, and puts a similar message in the installation log file named nfc_install.log, which is located at /opt/CSCOnfc/logs when the installation process is complete
ERROR: Existing datasize is "65536" The required datasize is at least "524288" System parameters validation failed. Please consult your system administrator or your system vendor technical support for information on changing the system parameters and rebuilding the kernel before running NetFlow FlowCollector
This section lists the notes and caveats for Release 2.0.
If you plan to use NetFlow FlowAnalyzer with FlowCollector 2.0, verify that you have NetFlow FlowAnalyzer 2.0. FlowCollector 2.0 supports only NetFlow FlowAnalyzer 2.0.
Table 2 lists the caveats that are still open in FlowCollector 2.0.
| Number | Description |
|---|---|
| CSCdk12553 | All NFC suite processes should run with the same priorities.
Because the NetFlow FlowCollector daemon (NFCD) and the NetFlow FlowCollector user interface (NFUI) have lower process priorities than NFCollector, NFUI requests might wait longer when the incoming packet rate is high. |
| CSCdk15726 | Overflow of sysUpTime or time stamps confuses FlowCollector.
If the sysUpTime, firstTimeStamp, or lastTimeStamp counters overflow and begin counting again, FlowCollector will not detect the overflow and may report inaccurate totalActiveTime, firstTimeStamp, or lastTimeStamp values. |
| CSCdk19588 | NFCD does not start using regular user ID.
If you login using a UNIX user ID and attempt to start FlowCollector, it will not start properly. To be sure that FlowCollector starts properly, log in as root before attempting to start FlowCollector. |
This section provides updates and corrections for the NetFlow FlowCollector Installation and User Guide. Read this section before you install and configure FlowCollector 2.0.
This section contains information on changes to the following chapters of the NetFlow FlowCollector Installation and User Guide:
On page 2-2, under the heading "NetFlow Services Support," replace the second bullet with the following updated version:
On page 3-2 under the heading "Installing FlowCollector," after the paragraph that begins:
Whether you are installing on a Solaris platform or on an HP-UX platform, ...
Add the following new paragraphs:
For example, FlowCollector 2.0 requires that the value of the data size (maxdsiz) system tunable parameter in HP-UX Version 11.0 be set to at least 524288 KB for satisfactory FlowCollector 2.0 operation. If you attempt to install FlowCollector 2.0 on a workstation whose maxdsiz value is below 524288 KB, the FlowCollector 2.0 installation script checks the current value, detects that it doesn't match the recommended minimum value, displays the following message, and puts a similar message in the installation log file named nfc_install.log, which is located at /opt/CSCOnfc/logs when the installation process is complete.
ERROR: Existing datasize is "65536" The required datasize is at least "524288" System parameters validation failed. Please consult your system administrator or your system vendor technical support for information on changing the system parameters and rebuilding the kernel before running NetFlow FlowCollector
The HP-UX Version 11.0 default for maxdsiz is 65536. If the value of this parameter falls below the recommended minimum value of 524288 KB, you must change the value to 524288 KB and then rebuild your kernel.
On page 3-2 under the heading "Installing FlowCollector," replace the paragraph at the bottom of the page that begins:
For example, the installation script searches for files from a previously installed version...
with the following updated version of the paragraph:
The installation script also searches for files from a previously installed version of FlowCollector. If it detects a previously installed version, it preserves existing data and configuration files. (Preserving the configuration files retains any additions or changes to the FlowCollector resource definitions or parameter settings that you may have made while using the previously installed version of FlowCollector.) Later in the installation process, the installation script allows you to specify whether you want to use the existing configuration files, or use the new configuration files. Depending on your choice, the unused files are given an alternate file name suffix and saved in case you need them later.
On page 5-4, in Table 5-2, for the SOURCE keyword, replace the last sentence in the description with the following updated sentence:
If you are using the new ROUTER_GROUPNAME feature, the label will be the group name specified in the ROUTER_GROUPNAME configuration parameter.
On page 5-4, in Table 5-2, for the FORMAT keyword, replace the last sentence in the description with the following updated sentence:
Currently, the format tag is "A."
On page 5-4, in Table 5-2, for the STARTTIME keyword, delete the last sentence in the description:
The count of UTC seconds is from the moment FlowCollector was started in the current session.
On page 5-4, in Table 5-2, for the FLOWS keyword, replace the description with the following updated sentence:
The total number of NetFlow export records that are aggregated in this data file.
On page 5-5, in Table 5-2, for the MISSED keyword, replace the first sentence in the description with the following updated sentence:
The number of flow records that FlowCollector should have received, but did not.
On page 6-10, in Table 6-3, the key fields labeled "input" and "output" should be "input interface" and "output interface," respectively.
On page 6-11, in Table 6-4, the key fields labeled "input" and "output" should be "input interface" and "output interface," respectively.
On page 6-15, under the heading "NetMatrix," replace the paragraph, table and note with the following updated paragraph, table, and note.
The output of this aggregation scheme consists of one record for each unique combination of input interface, output interface, masked source IP address, masked destination IP address, source mask, and destination mask present in the flow data received by FlowCollector during the current collection period. Each output record contains the following fields:
| Key fields: | input interface, output interface, masked srcaddr, masked dstaddr, src_mask, dst_mask |
| Value fields: | packet count, byte count, and flow count |
On page 6-15, under the heading "DetailASMatrix," replace the paragraph, table and note with the following updated paragraph, table, and note.
The output of this aggregation scheme consists of one record for each unique combination of source IP address, destination IP address, source port, destination port, protocol, input interface, output interface, source autonomous system number, and destination autonomous system number present in the flow data received by FlowCollector during the current collection period. Each output record contains the following fields:
| Key fields: | srcaddr, dstaddr, srcport, dstport, protocol, input interface, output interface, src_as, dst_as |
| Value fields: | packet count, byte count, and flow count |
On page A-3, under the heading "Using show-tech to Capture Troubleshooting Information," add the following new bullet
to the list after the bullet
For information on Cisco IOS software features related to NetFlow services on Cisco 7500, 7200, and 7000 series routers, refer to the Cisco IOS Release 11.1(2) or later configuration guides and command references.
For information on software features related to Multilayer Switching (MLS) and NetFlow data export on Catalyst 5000 series switches, refer to the Multilayer Switching User Guide.
For additional information about using the NetFlow FlowAnalyzer application to display FlowCollector traffic statistics in a graphical form, refer to the NetFlow FlowAnalyzer Installation and User Guide.
Cisco Connection Online (CCO) is Cisco Systems' primary, real-time support channel. Maintenance customers and partners can self-register on CCO to obtain additional information and services.
Available 24 hours a day, 7 days a week, CCO provides a wealth of standard and value-added services to Cisco's customers and business partners. CCO services include product information, product documentation, software updates, release notes, technical tips, the Bug Navigator, configuration notes, brochures, descriptions of service offerings, and download access to public and authorized files.
CCO serves a wide variety of users through two interfaces that are updated and enhanced simultaneously: a character-based version and a multimedia version that resides on the World Wide Web (WWW). The character-based CCO supports Zmodem, Kermit, Xmodem, FTP, and Internet e-mail, and it is excellent for quick access to information over lower bandwidths. The WWW version of CCO provides richly formatted documents with photographs, figures, graphics, and video, as well as hyperlinks to related information.
You can access CCO in the following ways:
For a copy of CCO's Frequently Asked Questions (FAQ), contact cco-help@cisco.com. For additional information, contact cco-team@cisco.com.
Cisco documentation and additional literature are available in a CD-ROM package. The Documentation CD-ROM, a member of the Cisco Connection Family, is updated monthly. Therefore, it might be more current than printed documentation. To order copies of the Documentation CD-ROM, contact your local sales representative or call customer service. The CD-ROM package is available as a single package or as an annual subscription. You can also access Cisco documentation on the World Wide Web at http://www.cisco.com, http://www-china.cisco.com, or http://www-europe.cisco.com.
If you are reading Cisco product documentation on the World Wide Web, you can submit comments electronically. Click Feedback in the toolbar and select Documentation. After you complete the form, click Submit to send it to Cisco. We appreciate your comments.
|
|