From nicolas.strina at smart-telecom.ch Thu Oct 3 13:47:45 2002 From: nicolas.strina at smart-telecom.ch (Nicolas Strina) Date: Thu, 3 Oct 2002 15:47:45 +0200 Subject: Debug Message-ID: <002901c26ae3$74325010$dea8a8c0@onyx> Hello, Is there any way to debug with Rancid ? I am actually trying to understand why it don't work. But when i use clogin (On cisco's) it's working perfectly. Regards, Nicolas ---------------------------------------- STRINA Nicolas IP Network Manager VTX Services - Internet Service Provider Av. de Lavaux 101 CH-1009 Pully nicolas.strina at smart-telecom.ch http://www.vtx.ch ---------------------------------------- From nicolas.strina at smart-telecom.ch Thu Oct 3 14:19:21 2002 From: nicolas.strina at smart-telecom.ch (Nicolas Strina) Date: Thu, 3 Oct 2002 16:19:21 +0200 Subject: Debug Again Message-ID: <002a01c26ae7$ddcd8ae0$dea8a8c0@onyx> Hello Again, If i do a tail on the log file i have this : $ tail -f backbone.20021003.161605 !ROM Bootstrap: Version 11.1(20)AA2, EARLY DEPLOYMENT RELEASE SOFTWARE (fc1) ir00.pul.lsn.fe0-0.ch.vtxnet.net: missed cmd(s): write term ir00.pul.lsn.fe0-0.ch.vtxnet.net: End of run not found ===================================== If i run directly from clogin i am able to see the result of wr term. Someone have an idea ? Regards, Nicolas ---------------------------------------- STRINA Nicolas IP Network Manager VTX Services - Internet Service Provider Av. de Lavaux 101 CH-1009 Pully nicolas.strina at smart-telecom.ch http://www.vtx.ch ---------------------------------------- From JRizzo at ea.com Thu Oct 3 14:39:22 2002 From: JRizzo at ea.com (Rizzo, Joe) Date: Thu, 3 Oct 2002 07:39:22 -0700 Subject: Debug Again Message-ID: <194487E5B6AC1E4E82FCBD56E5799791BC6581@eahq-mb3.rws.ad.ea.com> I usually set NOPIPE=foo, then run rancid with "-d". IE rancid -d router_to_debug A file will be created in the current directory named (router name).raw Maybe reviewing the .raw file will help you determine what is going on. Hope that helps, Joe -----Original Message----- From: Nicolas Strina [mailto:nicolas.strina at smart-telecom.ch] Sent: Thursday, October 03, 2002 9:19 AM To: rancid-discuss at shrubbery.net Subject: Debug Again Hello Again, If i do a tail on the log file i have this : $ tail -f backbone.20021003.161605 !ROM Bootstrap: Version 11.1(20)AA2, EARLY DEPLOYMENT RELEASE SOFTWARE (fc1) ir00.pul.lsn.fe0-0.ch.vtxnet.net: missed cmd(s): write term ir00.pul.lsn.fe0-0.ch.vtxnet.net: End of run not found ===================================== If i run directly from clogin i am able to see the result of wr term. Someone have an idea ? Regards, Nicolas ---------------------------------------- STRINA Nicolas IP Network Manager VTX Services - Internet Service Provider Av. de Lavaux 101 CH-1009 Pully nicolas.strina at smart-telecom.ch http://www.vtx.ch ---------------------------------------- From nicolas.strina at smart-telecom.ch Thu Oct 3 15:21:02 2002 From: nicolas.strina at smart-telecom.ch (Nicolas Strina) Date: Thu, 3 Oct 2002 17:21:02 +0200 Subject: Debug Again. Message-ID: <002e01c26af0$7bed4dc0$dea8a8c0@onyx> Hello, Some news. I have that: $ ./rancid -d ir00.pul.lsn.fe0-0.ch.vtxnet.net executing clogin -t 90 -c"show version;write term" ir00.pul.lsn.fe0-0.ch.vtxnet.net HIT COMMAND:VTX-PUL-1(3620)#show version In ShowVersion: VTX-PUL-1(3620)#show version TYPE = 3600 ir00.pul.lsn.fe0-0.ch.vtxnet.net: missed cmd(s): write term ir00.pul.lsn.fe0-0.ch.vtxnet.net: missed cmd(s): write term ir00.pul.lsn.fe0-0.ch.vtxnet.net: End of run not found ir00.pul.lsn.fe0-0.ch.vtxnet.net: End of run not found !ROM Bootstrap: Version 11.1(20)AA2, EARLY DEPLOYMENT RELEASE SOFTWARE (fc1) $ If i run the .clogin directly it work. Regards, Nicolas ---------------------------------------- STRINA Nicolas IP Network Manager VTX Services - Internet Service Provider Av. de Lavaux 101 CH-1009 Pully nicolas.strina at smart-telecom.ch http://www.vtx.ch ---------------------------------------- From heas at shrubbery.net Thu Oct 3 16:16:46 2002 From: heas at shrubbery.net (john heasley) Date: Thu, 3 Oct 2002 09:16:46 -0700 Subject: Debug Again. In-Reply-To: <002e01c26af0$7bed4dc0$dea8a8c0@onyx>; from nicolas.strina@smart-telecom.ch on Thu, Oct 03, 2002 at 05:21:02PM +0200 References: <002e01c26af0$7bed4dc0$dea8a8c0@onyx> Message-ID: <20021003091646.D6799@shrubbery.net> Thu, Oct 03, 2002 at 05:21:02PM +0200, Nicolas Strina: > Hello, > > Some news. I have that: > > $ ./rancid -d ir00.pul.lsn.fe0-0.ch.vtxnet.net > executing clogin -t 90 -c"show version;write term" > ir00.pul.lsn.fe0-0.ch.vtxnet.net > HIT COMMAND:VTX-PUL-1(3620)#show version > In ShowVersion: VTX-PUL-1(3620)#show version > TYPE = 3600 > ir00.pul.lsn.fe0-0.ch.vtxnet.net: missed cmd(s): write term > ir00.pul.lsn.fe0-0.ch.vtxnet.net: missed cmd(s): write term > ir00.pul.lsn.fe0-0.ch.vtxnet.net: End of run not found > ir00.pul.lsn.fe0-0.ch.vtxnet.net: End of run not found > !ROM Bootstrap: Version 11.1(20)AA2, EARLY DEPLOYMENT RELEASE SOFTWARE > (fc1) > $ > > If i run the .clogin directly it work. rancid is apparently missing something; most likely the end of the previous command. if you send the .raw file, i can tell you what is being missed. From christophe.belmont at c-s.fr Wed Oct 9 12:30:21 2002 From: christophe.belmont at c-s.fr (BELMONT Christophe) Date: Wed, 9 Oct 2002 14:30:21 +0200 Subject: Rancid & Sendmail Message-ID: <004801c26f8f$a26bdea0$a53702c2@CLEOPATRE> Hi all, I have a server running Solaris in production. I would like to install rancid but I would prefer not to install sendmail on this machine (sendmail is huge and not really trivial to administrate) since rancid should be able to send an email using normal "mail" command. I would like to know if it's possible to do that and how? Thanks in advance, Christophe. --- Christophe Belmont Network Engineer NOC G?ANT Tel : +33 6 81 73 29 56 From sean.mcnamara at villanova.edu Wed Oct 9 14:51:12 2002 From: sean.mcnamara at villanova.edu (Sean McNamara) Date: Wed, 09 Oct 2002 10:51:12 -0400 Subject: Rancid & Sendmail References: <004801c26f8f$a26bdea0$a53702c2@CLEOPATRE> Message-ID: <3DA44260.5020802@villanova.edu> Christophe, This shouldn't be a problem at all. It's not necessary to have sendmail running as a daemon (or configured beyond the default Sun package) in order to send mail from a machine. Sendmail should still be installed; however, you can prevent it from daemonizing at startup by renaming the file /etc/rc2.d/S88sendmail to /etc/rc2.d/.S88sendmail (files starting with a dot will be ignored at boot). Basically, to sum up: sendmail _is_ required to be installed in order to send messages with mail, but it does not need to run at startup, or have any non-default configuration performed. Good luck.. ..Sean. BELMONT Christophe wrote: >Hi all, > >I have a server running Solaris in production. I would like to install >rancid but I would prefer not to install sendmail on this machine >(sendmail is huge and not really trivial to administrate) since rancid >should be able to send an email using normal "mail" command. I would >like to know if it's possible to do that and how? > >Thanks in advance, >Christophe. > >--- >Christophe Belmont >Network Engineer >NOC G?ANT >Tel : +33 6 81 73 29 56 > > > > > > From asr at latency.net Wed Oct 9 17:32:28 2002 From: asr at latency.net (Adam Rothschild) Date: Wed, 9 Oct 2002 13:32:28 -0400 Subject: Rancid & Sendmail In-Reply-To: <004801c26f8f$a26bdea0$a53702c2@CLEOPATRE>; from christophe.belmont@c-s.fr on Wed, Oct 09, 2002 at 02:30:21PM +0200 References: <004801c26f8f$a26bdea0$a53702c2@CLEOPATRE> Message-ID: <20021009133228.B35193@latency.net> On 2002-10-09-08:30:21, BELMONT Christophe wrote: > I have a server running Solaris in production. I would like to > install rancid but I would prefer not to install sendmail on this > machine (sendmail is huge and not really trivial to administrate) > since rancid should be able to send an email using normal "mail" > command. I would like to know if it's possible to do that and how? No need for Sendmail. So long as your MTA's wrappers and 'mail' command know how to behave (I'm running rancid on boxen w/ qmail and postfix), you're fine. -a From heas at shrubbery.net Wed Oct 9 18:08:30 2002 From: heas at shrubbery.net (john heasley) Date: Wed, 9 Oct 2002 11:08:30 -0700 Subject: Rancid & Sendmail In-Reply-To: <3DA44260.5020802@villanova.edu>; from sean.mcnamara@villanova.edu on Wed, Oct 09, 2002 at 10:51:12AM -0400 References: <004801c26f8f$a26bdea0$a53702c2@CLEOPATRE> <3DA44260.5020802@villanova.edu> Message-ID: <20021009110830.A14365@shrubbery.net> Wed, Oct 09, 2002 at 10:51:12AM -0400, Sean McNamara: > Christophe, > > This shouldn't be a problem at all. It's not necessary to have > sendmail running as a daemon (or configured beyond the default Sun > package) in order to send mail from a machine. Sendmail should still > be installed; however, you can prevent it from daemonizing at startup by > renaming the file /etc/rc2.d/S88sendmail to /etc/rc2.d/.S88sendmail > (files starting with a dot will be ignored at boot). > btw, if you do this, you will want to have a cron job that periodically runs the queue (sendmail -q). so, if a message is temporarily undeliverable, it will be retried. From christophe.belmont at c-s.fr Thu Oct 10 08:09:02 2002 From: christophe.belmont at c-s.fr (BELMONT Christophe) Date: Thu, 10 Oct 2002 10:09:02 +0200 Subject: Rancid & Sendmail In-Reply-To: <3DA44260.5020802@villanova.edu> Message-ID: <004301c27034$4bdc1500$a53702c2@CLEOPATRE> Thank you all, I have it running now. Btw, thanks to the developers and contributors, that's a very nice and useful tool. Christophe. --- Christophe Belmont Network Engineer NOC G?ANT Tel : +33 6 81 73 29 56 -----Message d'origine----- De?: Sean McNamara [mailto:sean.mcnamara at villanova.edu] Envoy??: 09 October 2002 16:51 ??: BELMONT Christophe Cc?: rancid-discuss at shrubbery.net Objet?: Re: Rancid & Sendmail Christophe, This shouldn't be a problem at all. It's not necessary to have sendmail running as a daemon (or configured beyond the default Sun package) in order to send mail from a machine. Sendmail should still be installed; however, you can prevent it from daemonizing at startup by renaming the file /etc/rc2.d/S88sendmail to /etc/rc2.d/.S88sendmail (files starting with a dot will be ignored at boot). Basically, to sum up: sendmail _is_ required to be installed in order to send messages with mail, but it does not need to run at startup, or have any non-default configuration performed. Good luck.. ..Sean. BELMONT Christophe wrote: >Hi all, > >I have a server running Solaris in production. I would like to install >rancid but I would prefer not to install sendmail on this machine >(sendmail is huge and not really trivial to administrate) since rancid >should be able to send an email using normal "mail" command. I would >like to know if it's possible to do that and how? > >Thanks in advance, >Christophe. > >--- >Christophe Belmont >Network Engineer >NOC G?ANT >Tel : +33 6 81 73 29 56 > > > > > > From jlewis at lewis.org Mon Oct 14 21:19:27 2002 From: jlewis at lewis.org (jlewis at lewis.org) Date: Mon, 14 Oct 2002 17:19:27 -0400 (EDT) Subject: telnet hanging Message-ID: I've installed rancid-2.2.2 on a Red Hat 7.2 system and have noticed a problem where telnet (spawned from clogin) hangs on one particular router (the first in router.db). When I check the router, the telnet session has successfully logged in and is just sitting idle. I've seen it hung for as much as an hour before I've killed the telnet process. I did a quick search of the archive and didn't see anyone reporting something similar. Anyone seen/solved this before? ---------------------------------------------------------------------- Jon Lewis *jlewis at lewis.org*| I route System Administrator | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ From mpd at indiana.edu Mon Oct 14 21:47:27 2002 From: mpd at indiana.edu (Matthew Davy) Date: Mon, 14 Oct 2002 16:47:27 -0500 Subject: telnet hanging In-Reply-To: ; from jlewis@lewis.org on Mon, Oct 14, 2002 at 05:19:27PM -0400 References: Message-ID: <20021014164726.R8743@indiana.edu> I've had problems with telnet and ssh hanging on certain routers for a while now. Seems to get better or worse depending on what version of IOS we're running. (Incidentally, I haven't seen any problems with Junipers). Last week it started hanging on the same router about every other time I ran a diff. I saw a note about a possible expect bug on Linux, but I couldn't get the older version which was posted on the rancid page to build and I upgraded to the newest versions of expect and tcl and it didn't fix it either. I installed the same versions of rancid (2.2.2), expect (5.38) and tcl (8.3) on a FreeBSD box last Friday and have been running diffs every hour since without a problem. - Matt On Mon, Oct 14, 2002 at 05:19:27PM -0400, jlewis at lewis.org wrote: > I've installed rancid-2.2.2 on a Red Hat 7.2 system and have noticed a > problem where telnet (spawned from clogin) hangs on one particular router > (the first in router.db). When I check the router, the telnet session has > successfully logged in and is just sitting idle. I've seen it hung for as > much as an hour before I've killed the telnet process. > > I did a quick search of the archive and didn't see anyone reporting > something similar. Anyone seen/solved this before? > > ---------------------------------------------------------------------- > Jon Lewis *jlewis at lewis.org*| I route > System Administrator | therefore you are > Atlantic Net | > _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ From jlewis at lewis.org Mon Oct 14 22:31:55 2002 From: jlewis at lewis.org (jlewis at lewis.org) Date: Mon, 14 Oct 2002 18:31:55 -0400 (EDT) Subject: telnet hanging In-Reply-To: <00d501c273c9$ab184370$0200a8c0@dipsy> Message-ID: [attribution removed since he didn't appear to cc: the list] > I had the same problem with the version of expect that came stock with > RH7.2. The following is mentioned on the RANCID site: > > Linux users with expect 5.32.* (or > 5.25.*); there appears to be a bug in > tcl 8.3 that causes collections to hang often and likely all the time. Below Doh! I should have scrolled down that page...it's actually been open in one of my Opera tabs since I installed RANCID. The Red Hat 7.3 expect rpm installed without any complaints, so I'll see if that solves it. Sorry for wasting everyone's time. ---------------------------------------------------------------------- Jon Lewis *jlewis at lewis.org*| I route System Administrator | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ From heas at shrubbery.net Mon Oct 14 22:52:41 2002 From: heas at shrubbery.net (john heasley) Date: Mon, 14 Oct 2002 15:52:41 -0700 Subject: telnet hanging In-Reply-To: ; from jlewis@lewis.org on Mon, Oct 14, 2002 at 06:31:55PM -0400 References: <00d501c273c9$ab184370$0200a8c0@dipsy> Message-ID: <20021014155241.R24879@shrubbery.net> Mon, Oct 14, 2002 at 06:31:55PM -0400, jlewis at lewis.org: > [attribution removed since he didn't appear to cc: the list] > > I had the same problem with the version of expect that came stock with > > RH7.2. The following is mentioned on the RANCID site: > > > > Linux users with expect 5.32.* (or > 5.25.*); there appears to be a bug in > > tcl 8.3 that causes collections to hang often and likely all the time. Below > > Doh! I should have scrolled down that page...it's actually been open in > one of my Opera tabs since I installed RANCID. > > The Red Hat 7.3 expect rpm installed without any complaints, so I'll see > if that solves it. if you refresh that page, you'll also see: Update: we're told that kernel 2.4.11 + expect-5.32.2-62 + tcl-8.3.3-65 are a winning combination. We have not tested this ourselves. seems odd that its just that one box. anyway, i'm sure we'll hear from folks if these do not solve it. From jlewis at lewis.org Wed Oct 16 02:25:30 2002 From: jlewis at lewis.org (jlewis at lewis.org) Date: Tue, 15 Oct 2002 22:25:30 -0400 (EDT) Subject: telnet hanging In-Reply-To: Message-ID: On Mon, 14 Oct 2002 jlewis at lewis.org wrote: > The Red Hat 7.3 expect rpm installed without any complaints, so I'll see > if that solves it. Just upgrading expect and tcl to Red Hat 7.3's rpms didn't solve the problem. I'm currently running expect-5.32.2-67 tcl-8.3.3-67 kernel-2.4.9-13 Tomorrow, I'll try upgrading to kernel-2.4.18-10 and see if it makes a difference. If not, I guess I'll try rebuilding an expect rpm with the suggested patch. ---------------------------------------------------------------------- Jon Lewis *jlewis at lewis.org*| I route System Administrator | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ From jlewis at lewis.org Fri Oct 18 14:33:23 2002 From: jlewis at lewis.org (jlewis at lewis.org) Date: Fri, 18 Oct 2002 10:33:23 -0400 (EDT) Subject: telnet hanging In-Reply-To: Message-ID: On Tue, 15 Oct 2002 jlewis at lewis.org wrote: > Just upgrading expect and tcl to Red Hat 7.3's rpms didn't solve the > problem. I'm currently running > > expect-5.32.2-67 > tcl-8.3.3-67 > kernel-2.4.9-13 > > Tomorrow, I'll try upgrading to kernel-2.4.18-10 and see if it makes a > difference. If not, I guess I'll try rebuilding an expect rpm with the > suggested patch. The kernel upgrade didn't help either. I ended up rebuilding the tcltk src.rpm for Red Hat 7.2 with the suggested one line patch, and that seems to have solved the telnet hanging problem. Now I have another odd one that I hope isn't a FAQ. I've added a bunch of cisco access-servers (5200's, 5300's, and a 5800) to my rancid setup, and for some reason, these ciscos seem to vary the number of !'s separating sections of the config from one 'write term' to the next even though nobody has logged in and made changes to the config. So I'm constantly being emailed diffs like: ip classless ip route 0.0.0.0 0.0.0.0 209.208.16.1 no ip http server ! + ! access-list 20 permit ... access-list 20 permit ... access-list 20 permit ... access-list 20 deny any followed by ip classless ip route 0.0.0.0 0.0.0.0 209.208.16.1 no ip http server ! - ! access-list 20 permit ... access-list 20 permit ... access-list 20 permit ... access-list 20 deny any Obviously, this would appear to be more of a cisco problem than a rancid one, but I suspect someone else must have run into this and wonder if anyone's come up with a fix or knows how to get the cisco's to stop doing this. ---------------------------------------------------------------------- Jon Lewis *jlewis at lewis.org*| I route System Administrator | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ From heas at shrubbery.net Fri Oct 18 19:20:42 2002 From: heas at shrubbery.net (john heasley) Date: Fri, 18 Oct 2002 12:20:42 -0700 Subject: telnet hanging In-Reply-To: ; from jlewis@lewis.org on Fri, Oct 18, 2002 at 10:33:23AM -0400 References: Message-ID: <20021018122042.J16223@shrubbery.net> Fri, Oct 18, 2002 at 10:33:23AM -0400, jlewis at lewis.org: > On Tue, 15 Oct 2002 jlewis at lewis.org wrote: > > > Just upgrading expect and tcl to Red Hat 7.3's rpms didn't solve the > > problem. I'm currently running > > > > expect-5.32.2-67 > > tcl-8.3.3-67 > > kernel-2.4.9-13 > > > > Tomorrow, I'll try upgrading to kernel-2.4.18-10 and see if it makes a > > difference. If not, I guess I'll try rebuilding an expect rpm with the > > suggested patch. > > The kernel upgrade didn't help either. I ended up rebuilding the tcltk > src.rpm for Red Hat 7.2 with the suggested one line patch, and that seems > to have solved the telnet hanging problem. with the same versions of expect and tcl listed above? > Now I have another odd one that I hope isn't a FAQ. I've added a bunch of > cisco access-servers (5200's, 5300's, and a 5800) to my rancid setup, and > for some reason, these ciscos seem to vary the number of !'s separating > sections of the config from one 'write term' to the next even though > nobody has logged in and made changes to the config. never seen this, but the attached patch ought to do it. > So I'm constantly being emailed diffs like: > > ip classless > ip route 0.0.0.0 0.0.0.0 209.208.16.1 > no ip http server > ! > + ! > access-list 20 permit ... > access-list 20 permit ... > access-list 20 permit ... > access-list 20 deny any > > followed by > > ip classless > ip route 0.0.0.0 0.0.0.0 209.208.16.1 > no ip http server > ! > - ! > access-list 20 permit ... > access-list 20 permit ... > access-list 20 permit ... > access-list 20 deny any > > Obviously, this would appear to be more of a cisco problem than a rancid > one, but I suspect someone else must have run into this and wonder if > anyone's come up with a fix or knows how to get the cisco's to stop doing > this. > > ---------------------------------------------------------------------- > Jon Lewis *jlewis at lewis.org*| I route > System Administrator | therefore you are > Atlantic Net | > _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ -------------- next part -------------- Index: rancid.in =================================================================== RCS file: /home/rancid/.CVS/rancid/bin/rancid.in,v retrieving revision 1.119 diff -d -u -r1.119 rancid.in --- rancid.in 2002/09/29 23:00:41 1.119 +++ rancid.in 2002/10/18 19:16:05 @@ -869,7 +869,7 @@ # This routine processes a "write term" sub WriteTerm { print STDERR " In WriteTerm: $_" if ($debug); - my($lineauto) = 0; + my($lineauto,$comment) = (0,0); while () { tr/\015//d; @@ -897,6 +897,16 @@ # some versions have other crap mixed in with the bits in the # block above /^! (Last configuration|NVRAM config last)/ && next; + + # skip multiple comment lines to avoid toggling extra comment on some + # access servers. grrr. + if (/^!/) { + next if ($comment); + ProcessHistory("","","",$_); + $comment++; + next; + } + $comment = 0; # Dog gone Cool matches to process the rest of the config /^tftp-server flash / && next; # kill any tftp remains From jlewis at lewis.org Fri Oct 18 19:51:54 2002 From: jlewis at lewis.org (jlewis at lewis.org) Date: Fri, 18 Oct 2002 15:51:54 -0400 (EDT) Subject: telnet hanging In-Reply-To: <20021018122042.J16223@shrubbery.net> Message-ID: On Fri, 18 Oct 2002, john heasley wrote: > > > expect-5.32.2-67 > > > tcl-8.3.3-67 > > > kernel-2.4.9-13 > > > > > The kernel upgrade didn't help either. I ended up rebuilding the tcltk > > src.rpm for Red Hat 7.2 with the suggested one line patch, and that seems > > to have solved the telnet hanging problem. > > with the same versions of expect and tcl listed above? tcl-8.3.3-65rancid expect-5.32.2-65rancid These were built from the source rpm for tcltk that came with Red Hat 7.2. > never seen this, but the attached patch ought to do it. Cool. I'll give it a try. ---------------------------------------------------------------------- Jon Lewis *jlewis at lewis.org*| I route System Administrator | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ From randy at psg.com Tue Oct 22 12:01:43 2002 From: randy at psg.com (Randy Bush) Date: Tue, 22 Oct 2002 05:01:43 -0700 Subject: juniper mode problem Message-ID: i have a juniper problem which appears to be related to the class of the user. when the user had superuser privs, i got the failure as follows: psg1.psg.com jlogin error: Error: TIMEOUT reached psg1.psg.com: missed cmd(s): show chassis hardware detail,show \ chassis environment,show version,show chassis ssb,show chassis \ scb,show chassis sfm detail,show configuration,show chassis \ routing-engine,show chassis fpc detail,show chassis \ firmware,show chassis feb,show chassis clocks,show system \ boot-messages psg1.psg.com: End of run not found so i made the class of the rancid user 'operator' and now i get: Index: configs/psg1.psg.com =================================================================== retrieving revision 1.2 diff -u -4 -r1.2 psg1.psg.com @@ -173,15 +173,9 @@ # # psg1> show configuration groups { statics { - routing-options { - rib { - static { - route 205.238.48.0/24 discard; - } - } - } + routing-options { /* ACCESS-DENIED */ }; } } etc randy From avram at juniper.net Tue Oct 22 13:22:52 2002 From: avram at juniper.net (Avram Dorfman) Date: Tue, 22 Oct 2002 09:22:52 -0400 Subject: juniper mode problem In-Reply-To: Message-ID: <5E1E50A6-E5C1-11D6-9F95-0003938CDB0C@juniper.net> Getting a timeout while logging in has nothing to do with user class. Have you traced the login script to see what it's timing out on? One thing to lookout for is that if you're logging into the same router multiple times in rapid succession, you might get rate limited - no obvious signs of that here, but it does appear as a login timeout b/c the RE rate limiters start to drop SYNs. -Avram On Tuesday, October 22, 2002, at 08:01 AM, Randy Bush wrote: > i have a juniper problem which appears to be related to the class of > the > user. > > when the user had superuser privs, i got the failure as follows: > > psg1.psg.com jlogin error: Error: TIMEOUT reached > psg1.psg.com: missed cmd(s): show chassis hardware detail,show \ > chassis environment,show version,show chassis ssb,show chassis \ > scb,show chassis sfm detail,show configuration,show chassis \ > routing-engine,show chassis fpc detail,show chassis \ > firmware,show chassis feb,show chassis clocks,show system \ > boot-messages > psg1.psg.com: End of run not found > > so i made the class of the rancid user 'operator' and now i get: > > Index: configs/psg1.psg.com > =================================================================== > retrieving revision 1.2 > diff -u -4 -r1.2 psg1.psg.com > @@ -173,15 +173,9 @@ > # > # psg1> show configuration > groups { > statics { > - routing-options { > - rib { > - static { > - route 205.238.48.0/24 discard; > - } > - } > - } > + routing-options { /* ACCESS-DENIED */ }; > } > } > > etc > > randy > > From randy at psg.com Tue Oct 22 18:42:21 2002 From: randy at psg.com (Randy Bush) Date: Tue, 22 Oct 2002 11:42:21 -0700 Subject: breaking juniper ssh session Message-ID: i suspect that rancid is being rude, well different, in some way when it terminates a juniper ssh session Oct 22 18:25:03 psg1 sshd[4921]: Connection closed by 147.28.0.39 Oct 22 18:25:03 psg1 inetd[577]: /usr/sbin/sshd[4921]: exit status 0xff00 anyone have a clue? randy From asp at partan.com Tue Oct 22 19:45:06 2002 From: asp at partan.com (Andrew Partan) Date: Tue, 22 Oct 2002 15:45:06 -0400 Subject: breaking juniper ssh session In-Reply-To: References: Message-ID: <20021022194506.GA67570@partan.com> On Tue, Oct 22, 2002 at 11:42:21AM -0700, Randy Bush wrote: > i suspect that rancid is being rude, well different, in some way > when it terminates a juniper ssh session Its just more verbose than others. Newer version of junos are less noisy. --asp From randy at psg.com Tue Oct 22 22:34:31 2002 From: randy at psg.com (Randy Bush) Date: Tue, 22 Oct 2002 15:34:31 -0700 Subject: breaking juniper ssh session References: <20021022194506.GA67570@partan.com> Message-ID: >> i suspect that rancid is being rude, well different, in some way >> when it terminates a juniper ssh session > Its just more verbose than others. Newer version of junos are less > noisy. it is a rather new version of junos, and it does not make such noises when i manually ssh in and quit out. randy From jlewis at lewis.org Tue Oct 22 22:59:22 2002 From: jlewis at lewis.org (jlewis at lewis.org) Date: Tue, 22 Oct 2002 18:59:22 -0400 (EDT) Subject: breaking juniper ssh session In-Reply-To: Message-ID: On Tue, 22 Oct 2002, Randy Bush wrote: > >> i suspect that rancid is being rude, well different, in some way > >> when it terminates a juniper ssh session > > Its just more verbose than others. Newer version of junos are less > > noisy. > > it is a rather new version of junos, and it does not make such > noises when i manually ssh in and quit out. I wonder what would happen if you just add the appropriate quit command to the end of @commands in jrancid? ---------------------------------------------------------------------- Jon Lewis *jlewis at lewis.org*| I route System Administrator | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ From heas at shrubbery.net Tue Oct 22 23:07:46 2002 From: heas at shrubbery.net (john heasley) Date: Tue, 22 Oct 2002 16:07:46 -0700 Subject: breaking juniper ssh session In-Reply-To: ; from jlewis@lewis.org on Tue, Oct 22, 2002 at 06:59:22PM -0400 References: Message-ID: <20021022160746.O11429@shrubbery.net> Tue, Oct 22, 2002 at 06:59:22PM -0400, jlewis at lewis.org: > On Tue, 22 Oct 2002, Randy Bush wrote: > > > >> i suspect that rancid is being rude, well different, in some way > > >> when it terminates a juniper ssh session > > > Its just more verbose than others. Newer version of junos are less > > > noisy. > > > > it is a rather new version of junos, and it does not make such > > noises when i manually ssh in and quit out. > > I wonder what would happen if you just add the appropriate quit command to > the end of @commands in jrancid? > jlogin (-c blah) does this for you. even waits for EOF. this problem has been there for ages and last i recall asking juniper about it, they recognized it as a bug. From randy at psg.com Wed Oct 23 04:32:13 2002 From: randy at psg.com (Randy Bush) Date: Tue, 22 Oct 2002 21:32:13 -0700 Subject: juniper being silly Message-ID: manually i can login just fine % rancid/bin/jlogin -c 'show conf;show conf' psg1.psg.com psg1.psg.com spawn ssh -c 3des -x -l rancid psg1.psg.com RGnet / PSGnet / NSRC Bainbridge Island, WA USA psg1.psg.com 2002.10.22-1 +1 206 780-0431 / 206 356 8341 Last login: Wed Oct 23 04:26:59 2002 from rip.psg.com^M --- JUNOS 5.4R2.4 built 2002-09-11 01:27:32 UTC rancid at psg1> rancid at psg1> set cli complete-on-space off Disabling complete-on-space rancid at psg1> set cli screen-length 0 Screen length set to 0 rancid at psg1> show conf groups { statics { routing-options { rib { static { route 205.238.48.0/24 discard; } } } } } /* ------------------------------------------------------------------------ system configuration ------------------------------------------------------------------------ */ system { host-name psg1; domain-name psg.com; no-redirects; authentication-order password; location { country-code US; postal-code 98110; npa-nxx 206780; latitude 47.6107864; longitude -122.5752106; altitude 20; } root-authentication { ... but from the logs, i get psg1.psg.com jlogin error: Error: TIMEOUT reached psg1.psg.com: missed cmd(s): show chassis hardware detail,show chassis environment,show version,show chassis ssb,show chassis scb,show chassis sfm detail,show configuration,show chassis routing-engine,show chassis fpc detail,show chassis firmware,show chassis feb,show chassis clocks,show system boot-messages psg1.psg.com: End of run not found ===================================== and then sometimes it works. randy From asp at partan.com Wed Oct 23 05:20:29 2002 From: asp at partan.com (Andrew Partan) Date: Wed, 23 Oct 2002 01:20:29 -0400 Subject: juniper being silly In-Reply-To: References: Message-ID: <20021023052028.GA3288@partan.com> On Tue, Oct 22, 2002 at 09:32:13PM -0700, Randy Bush wrote: > but from the logs, i get > psg1.psg.com jlogin error: Error: TIMEOUT reached > and then sometimes it works. I only get these sorts of occaisional failures from old ciscos with no longer adequate CPUs. Or if routing is flapping. Or if name->IP mapping is hitting more than one router. Or expands to multiple IP addrs some of which are down. Or you have duplicate IP addrs. --asp From heas at shrubbery.net Wed Oct 23 05:32:26 2002 From: heas at shrubbery.net (john heasley) Date: Wed, 23 Oct 2002 05:32:26 +0000 Subject: juniper being silly In-Reply-To: <20021023052028.GA3288@partan.com>; from asp@partan.com on Wed, Oct 23, 2002 at 01:20:29AM -0400 References: <20021023052028.GA3288@partan.com> Message-ID: <20021023053226.A29490@shrubbery.net> Wed, Oct 23, 2002 at 01:20:29AM -0400, Andrew Partan: > On Tue, Oct 22, 2002 at 09:32:13PM -0700, Randy Bush wrote: > > but from the logs, i get > > psg1.psg.com jlogin error: Error: TIMEOUT reached > > and then sometimes it works. > > I only get these sorts of occaisional failures from old ciscos with > no longer adequate CPUs. > > Or if routing is flapping. Or if name->IP mapping is hitting more > than one router. Or expands to multiple IP addrs some of which are > down. Or you have duplicate IP addrs. > --asp try (or sh morale equiv) setenv NOPIPE YES jrancid -d psg1.psg.com if you manage to get an instance where it fails, send the .raw file to me. from the error log, its getting as far as as "show chassis fpc detail" and getting buggered somewhere in there, assuming the error point is consistent. From randy at psg.com Wed Oct 23 05:52:47 2002 From: randy at psg.com (Randy Bush) Date: Tue, 22 Oct 2002 22:52:47 -0700 Subject: juniper being silly References: <20021023052028.GA3288@partan.com> <20021023053226.A29490@shrubbery.net> Message-ID: yes, always at the same place. see appended and i succeed many many many times with the test you recommended % env | grep PIPE NOPIPE=YES % jrancid -d psg1.psg.com executing echo jlogin -c"show chassis clocks;show chassis environment;show chassis firmware;show chassis fpc detail;show chassis hardware detail;show chassis routing-engine;show chassis scb;show chassis sfm detail;show chassis ssb;show chassis feb;show system boot-messages;show version;show configuration" psg1.psg.com executing echo jlogin -c"show chassis clocks;show chassis environment;show chassis firmware;show chassis fpc detail;show chassis hardware detail;show chassis routing-engine;show chassis scb;show chassis sfm detail;show chassis ssb;show chassis feb;show system boot-messages;show version;show configuration" psg1.psg.com HIT COMMAND:rancid at psg1> show chassis clocks In ShowChassisClocks: rancid at psg1> show chassis clocks HIT COMMAND:rancid at psg1> show chassis environment In ShowChassisEnvironment: rancid at psg1> show chassis environment HIT COMMAND:rancid at psg1> show chassis firmware In ShowChassisFirmware: rancid at psg1> show chassis firmware HIT COMMAND:rancid at psg1> show chassis fpc detail In ShowChassisFpcDetail: rancid at psg1> show chassis fpc detail HIT COMMAND:rancid at psg1> show chassis hardware detail In ShowChassisHardware: rancid at psg1> show chassis hardware detail HIT COMMAND:rancid at psg1> show chassis routing-engine In ShowChassisRoutingEngine: rancid at psg1> show chassis routing-engine HIT COMMAND:rancid at psg1> show chassis scb In ShowChassisSCB: rancid at psg1> show chassis scb HIT COMMAND:rancid at psg1> show chassis sfm detail In ShowChassisSCB: rancid at psg1> show chassis sfm detail HIT COMMAND:rancid at psg1> show chassis ssb In ShowChassisSCB: rancid at psg1> show chassis ssb HIT COMMAND:rancid at psg1> show chassis feb In ShowChassisSCB: rancid at psg1> show chassis feb HIT COMMAND:rancid at psg1> show system boot-messages In ShowSystemBootMessages: rancid at psg1> show system boot-messages HIT COMMAND:rancid at psg1> show version In ShowVersion: rancid at psg1> show version HIT COMMAND:rancid at psg1> show configuration In ShowConfiguration: rancid at psg1> show configuration % but it is six hours since i managed to snatch a copy of psg1's config :-( randy --- starting: Tue Oct 22 22:15:00 PDT 2002 Trying to get all of the configs. # psg1.psg.com jlogin error: Error: TIMEOUT reached psg1.psg.com: missed cmd(s): show chassis hardware detail,show chassis environment,show version,show chassis ssb,show chassis scb,show chassis sfm detail,show configuration,show chassis routing-engine,show chassis fpc detail,show chassis firmware,show chassis feb,show chassis clocks,show system boot-messages psg1.psg.com: End of run not found ===================================== Getting missed routers: round 1. # psg1.psg.com jlogin error: Error: TIMEOUT reached psg1.psg.com: missed cmd(s): show chassis hardware detail,show chassis environment,show version,show chassis ssb,show chassis scb,show chassis sfm detail,show configuration,show chassis routing-engine,show chassis fpc detail,show chassis firmware,show chassis feb,show chassis clocks,show system boot-messages psg1.psg.com: End of run not found ===================================== Getting missed routers: round 2. # psg1.psg.com jlogin error: Error: TIMEOUT reached psg1.psg.com: missed cmd(s): show chassis hardware detail,show chassis environment,show version,show chassis ssb,show chassis scb,show chassis sfm detail,show configuration,show chassis routing-engine,show chassis fpc detail,show chassis firmware,show chassis feb,show chassis clocks,show system boot-messages psg1.psg.com: End of run not found ===================================== Getting missed routers: round 3. # psg1.psg.com jlogin error: Error: TIMEOUT reached psg1.psg.com: missed cmd(s): show chassis hardware detail,show chassis environment,show version,show chassis ssb,show chassis scb,show chassis sfm detail,show configuration,show chassis routing-engine,show chassis fpc detail,show chassis firmware,show chassis feb,show chassis clocks,show system boot-messages psg1.psg.com: End of run not found ===================================== Getting missed routers: round 4. # psg1.psg.com jlogin error: Error: TIMEOUT reached psg1.psg.com: missed cmd(s): show chassis hardware detail,show chassis environment,show version,show chassis ssb,show chassis scb,show chassis sfm detail,show configuration,show chassis routing-engine,show chassis fpc detail,show chassis firmware,show chassis feb,show chassis clocks,show system boot-messages psg1.psg.com: End of run not found cvs diff: Diffing . cvs diff: Diffing configs cvs commit: Examining . cvs commit: Examining configs ending: Tue Oct 22 22:25:03 PDT 2002 From heas at shrubbery.net Wed Oct 23 06:10:17 2002 From: heas at shrubbery.net (john heasley) Date: Wed, 23 Oct 2002 06:10:17 +0000 Subject: juniper being silly In-Reply-To: ; from randy@psg.com on Tue, Oct 22, 2002 at 10:52:47PM -0700 References: <20021023052028.GA3288@partan.com> <20021023053226.A29490@shrubbery.net> Message-ID: <20021023061017.B29490@shrubbery.net> hmm. now i suspect tcl or expect. try setting NOPIPE in bin/env and letting the collection run. Tue, Oct 22, 2002 at 10:52:47PM -0700, Randy Bush: > yes, always at the same place. see appended > > and i succeed many many many times with the test you recommended > > % env | grep PIPE > NOPIPE=YES > % jrancid -d psg1.psg.com > executing echo jlogin -c"show chassis clocks;show chassis environment;show chassis firmware;show chassis fpc detail;show chassis hardware detail;show chassis routing-engine;show chassis scb;show chassis sfm detail;show chassis ssb;show chassis feb;show system boot-messages;show version;show configuration" psg1.psg.com > executing echo jlogin -c"show chassis clocks;show chassis environment;show chassis firmware;show chassis fpc detail;show chassis hardware detail;show chassis routing-engine;show chassis scb;show chassis sfm detail;show chassis ssb;show chassis feb;show system boot-messages;show version;show configuration" psg1.psg.com > HIT COMMAND:rancid at psg1> show chassis clocks > In ShowChassisClocks: rancid at psg1> show chassis clocks > HIT COMMAND:rancid at psg1> show chassis environment > In ShowChassisEnvironment: rancid at psg1> show chassis environment > HIT COMMAND:rancid at psg1> show chassis firmware > In ShowChassisFirmware: rancid at psg1> show chassis firmware > HIT COMMAND:rancid at psg1> show chassis fpc detail > In ShowChassisFpcDetail: rancid at psg1> show chassis fpc detail > HIT COMMAND:rancid at psg1> show chassis hardware detail > In ShowChassisHardware: rancid at psg1> show chassis hardware detail > HIT COMMAND:rancid at psg1> show chassis routing-engine > In ShowChassisRoutingEngine: rancid at psg1> show chassis routing-engine > HIT COMMAND:rancid at psg1> show chassis scb > In ShowChassisSCB: rancid at psg1> show chassis scb > HIT COMMAND:rancid at psg1> show chassis sfm detail > In ShowChassisSCB: rancid at psg1> show chassis sfm detail > HIT COMMAND:rancid at psg1> show chassis ssb > In ShowChassisSCB: rancid at psg1> show chassis ssb > HIT COMMAND:rancid at psg1> show chassis feb > In ShowChassisSCB: rancid at psg1> show chassis feb > HIT COMMAND:rancid at psg1> show system boot-messages > In ShowSystemBootMessages: rancid at psg1> show system boot-messages > HIT COMMAND:rancid at psg1> show version > In ShowVersion: rancid at psg1> show version > HIT COMMAND:rancid at psg1> show configuration > In ShowConfiguration: rancid at psg1> show configuration > % > > but it is six hours since i managed to snatch a copy of psg1's config :-( > > randy > > --- > > starting: Tue Oct 22 22:15:00 PDT 2002 > > > Trying to get all of the configs. > # > psg1.psg.com jlogin error: Error: TIMEOUT reached > psg1.psg.com: missed cmd(s): show chassis hardware detail,show chassis environment,show version,show chassis ssb,show chassis scb,show chassis sfm detail,show configuration,show chassis routing-engine,show chassis fpc detail,show chassis firmware,show chassis feb,show chassis clocks,show system boot-messages > psg1.psg.com: End of run not found > ===================================== > Getting missed routers: round 1. > # > psg1.psg.com jlogin error: Error: TIMEOUT reached > psg1.psg.com: missed cmd(s): show chassis hardware detail,show chassis environment,show version,show chassis ssb,show chassis scb,show chassis sfm detail,show configuration,show chassis routing-engine,show chassis fpc detail,show chassis firmware,show chassis feb,show chassis clocks,show system boot-messages > psg1.psg.com: End of run not found > ===================================== > Getting missed routers: round 2. > # > psg1.psg.com jlogin error: Error: TIMEOUT reached > psg1.psg.com: missed cmd(s): show chassis hardware detail,show chassis environment,show version,show chassis ssb,show chassis scb,show chassis sfm detail,show configuration,show chassis routing-engine,show chassis fpc detail,show chassis firmware,show chassis feb,show chassis clocks,show system boot-messages > psg1.psg.com: End of run not found > ===================================== > Getting missed routers: round 3. > # > psg1.psg.com jlogin error: Error: TIMEOUT reached > psg1.psg.com: missed cmd(s): show chassis hardware detail,show chassis environment,show version,show chassis ssb,show chassis scb,show chassis sfm detail,show configuration,show chassis routing-engine,show chassis fpc detail,show chassis firmware,show chassis feb,show chassis clocks,show system boot-messages > psg1.psg.com: End of run not found > ===================================== > Getting missed routers: round 4. > # > psg1.psg.com jlogin error: Error: TIMEOUT reached > psg1.psg.com: missed cmd(s): show chassis hardware detail,show chassis environment,show version,show chassis ssb,show chassis scb,show chassis sfm detail,show configuration,show chassis routing-engine,show chassis fpc detail,show chassis firmware,show chassis feb,show chassis clocks,show system boot-messages > psg1.psg.com: End of run not found > > cvs diff: Diffing . > cvs diff: Diffing configs > cvs commit: Examining . > cvs commit: Examining configs > > ending: Tue Oct 22 22:25:03 PDT 2002 From randy at psg.com Wed Oct 23 15:01:15 2002 From: randy at psg.com (Randy Bush) Date: Wed, 23 Oct 2002 08:01:15 -0700 Subject: juniper being silly References: <20021023052028.GA3288@partan.com> <20021023053226.A29490@shrubbery.net> <20021023061017.B29490@shrubbery.net> Message-ID: > hmm. now i suspect tcl or expect. try setting NOPIPE in bin/env and > letting the collection run. no change randy From heas at shrubbery.net Wed Oct 23 21:34:15 2002 From: heas at shrubbery.net (john heasley) Date: Wed, 23 Oct 2002 14:34:15 -0700 Subject: juniper being silly In-Reply-To: ; from randy@psg.com on Wed, Oct 23, 2002 at 08:01:15AM -0700 References: <20021023052028.GA3288@partan.com> <20021023053226.A29490@shrubbery.net> <20021023061017.B29490@shrubbery.net> Message-ID: <20021023143415.P8823@shrubbery.net> Wed, Oct 23, 2002 at 08:01:15AM -0700, Randy Bush: > > hmm. now i suspect tcl or expect. try setting NOPIPE in bin/env and > > letting the collection run. > > no change > > randy stumped. possible solutions 1) set NOPIPE to YES in bin/eng and $debug to 1 in bin/jrancid and send the .raw from the next failure. 2) give me access to the router sorry; this one seems tricky. From randy at psg.com Wed Oct 23 23:21:28 2002 From: randy at psg.com (Randy Bush) Date: Wed, 23 Oct 2002 16:21:28 -0700 Subject: juniper being silly References: <20021023052028.GA3288@partan.com> <20021023053226.A29490@shrubbery.net> <20021023061017.B29490@shrubbery.net> <20021023143415.P8823@shrubbery.net> Message-ID: got it! psg1.psg.com spawn ssh -c 3des -x -l rancid psg1.psg.com RGnet / PSGnet / NSRC Bainbridge Island, WA USA psg1.psg.com 2002.10.22-1 +1 206 780-0431 / 206 356 8341 Enter passphrase for key '/usr/home/randy/.ssh/id_dsa': Error: TIMEOUT reached the conf file says the user is rancid, but it is running from my cron job, and i have a public key over there. any way out of this one? i ain't putting my ssh passphrase in an unencrypted text file anywhere. create a separate rancid user i guess. pita. randy From heas at shrubbery.net Wed Oct 23 23:33:10 2002 From: heas at shrubbery.net (john heasley) Date: Wed, 23 Oct 2002 16:33:10 -0700 Subject: juniper being silly In-Reply-To: ; from randy@psg.com on Wed, Oct 23, 2002 at 04:21:28PM -0700 References: <20021023052028.GA3288@partan.com> <20021023053226.A29490@shrubbery.net> <20021023061017.B29490@shrubbery.net> <20021023143415.P8823@shrubbery.net> Message-ID: <20021023163310.F8823@shrubbery.net> Wed, Oct 23, 2002 at 04:21:28PM -0700, Randy Bush: > got it! > > psg1.psg.com > spawn ssh -c 3des -x -l rancid psg1.psg.com > RGnet / PSGnet / NSRC Bainbridge Island, WA USA > psg1.psg.com 2002.10.22-1 +1 206 780-0431 / 206 356 8341 > > Enter passphrase for key '/usr/home/randy/.ssh/id_dsa': > Error: TIMEOUT reached > > the conf file says the user is rancid, but it is running from my > cron job, and i have a public key over there. > > any way out of this one? i ain't putting my ssh passphrase in > an unencrypted text file anywhere. create a separate rancid user > i guess. pita. > > randy cloginrc(5) cloginrc(5) ... add identity {} May be used to specify an alternate identity file for use with ssh(1). See ssh's -i option for details. Default: your default identity file. see ssh(1).