[rancid] Script help / pointers

[Alan McKinnon]

On 03/11/2015 18:37, AJ Schroeder wrote:
> Hello list,
> 
>  
> 
> A while ago I hacked together a couple of scripts to get some Nortel and
> Avaya devices backing up. I basically copied some of the existing
> scripts to make things work, however the issue is that they “just work”.
> We are going to be supporting these devices for the foreseeable future
> so I’d like to improve the scripts from their current form.
> 
>  
> 
> Specifically, I am having a hard time processing the logouts properly
> because I am not sure what the scripts are looking for to say “I’m done
> with show run, continue on”. I have gotten things to work by just adding
> a logout command to the run_commands sub in the nortellogin, but I feel
> this is not the most graceful way of exiting. I believe not being able
> to process things is causing backups to take longer than it should.
> 
>  
> 
> The scripts are located here:
> 
>  
> 
> https://github.com/ajschroeder/rancid/blob/master/nortelrancid
> 
> https://github.com/ajschroeder/rancid/blob/master/nortellogin
> 
>  
> 
> If anyone has time to help or critique it would be greatly appreciated.


I haven't looked at your scripts yet (might do so later), but most
rancid parser scripts work basically the same way. Most devices with a
sane Unix-like cli behave much like a telnet session - there's a login
process and when that completes there's a prompt. These two are in that
order, and the prompt follows a pattern - text followed by a ">" or "#".
This can be quite reliably found with a regex so the script knows what
it looks like. Then there is a sequence of commands run by the script
itself which are echoed to the terminal along with the command output.
The script sees a prompt, followed by a command (that it defined and
knows) so it passes all text received until the next prompt to the sub
defined for that command. The commands are all in @commandtable.

The main loop looks like this:

TOP: while(<INPUT>) {

    if (/[>#]\s?exit$/) {
        last;
    }
    while (/[>#]\s*($cmds_regexp)\s*$/) {
        $cmd = $1;
        if (!defined($prompt)) {
            $prompt = ($_ =~ /^([^#>]+[#>])/)[0];
            $prompt =~ s/([][}{)(\\])/\\$1/g;
        }
        if (! defined($commands{$cmd})) {
            last TOP;
        }
        $rval = &{$commands{$cmd}};		<==== magic goes here
        delete($commands{$cmd});
        if ($rval == -1) {
            last TOP;
        }
    }
}

It's all driven by the line marked "magic goes here" and @commandtable
is an array of hashes. Each hash contains the router command as the name
and the sub that processes the output as the value. The call
"&{$commands{$cmd}}" calls the sub, which runs until it finds a prompt,
whereupon the main loop above continues. The whole thing keeps going
round and round till the main loop finds the command "exit" issued -
this ends the loop and the script terminate. rancid-run then carries on
with the next item in router.db. It's all very simple, logical classic
Perl, and TheSimplestThingThatCouldPossiblyWork which probably explains
why rancid is still here after so many years, why every ISP I know of
uses it, and why every competitor seems to fall by the wayside.

To set things up for a new device type, you need to research the device
first. Define exactly what it does with prompts and echoing of command
and output, and how you log out.

The one important bit that isn't in the rancid parser script is how to
logout/exit - that is in clogin and can differ a lot between device
types. This is why so often you'll find a new type called say xyzrancid
with a corresponding modified xyzlogin. Modifying the parser script
often involves ripping out how chunks of each sub that simply do not
apply to your device. Knowledge of IOS helps here so you can tell what
the Cisco parser is doing and adapt it to what your device does.

That's it really, truthfully it's lots of common sense and donkey work.
To see how fast your parser runs, call it directly in debug mode and
copy the clogin command echoed at the beginning. Run that command
directly and observe what happens. You should be familiar with your
devices enough to know if it's moving along at the right speed.

The last thing is if you are worried about processing time, don't be
afraid to crank PAR_COUNT waaaaaaaay up. This is 2015 and 1995 called -
they want their Pentium ones back. Modern hosts, even small VMs, cope
very nicely with 100 telnet sessions with ease (they are idle most of
the time anyway). By example, I found a teeny little VM with 1 cpu and
512 MB on an over-committed ESXi host levelled out at about 80 parallel
processes. The size of command output does not affect rancid at all.
Whether it's a 100 line show run or a huge beast from a 9K, rancid still
chugs along processing one line at a time till it finds a logout.


-- 
Alan McKinnon
alan.mckinnon at gmail.com