From fusionfoto at gmail.com Mon Jul 1 20:01:18 2019 From: fusionfoto at gmail.com (FF) Date: Mon, 1 Jul 2019 16:01:18 -0400 Subject: [rancid] [SPAM?] Cisco NX "chatty" with Power info In-Reply-To: References: <50cc4a0483d54568a4d3e2547471fece@aquafin.be> <20190311204733.GI30597@shrubbery.net> <2cdcf9197356879b1dfa1d0486010ad5d3f363fa.camel@skno.by> Message-ID: Hi, So I haven't messed with FILTER_OSC yet.. I would be glad to drop the power monitoring dialogue, except it seems to be the only one I've identified that tells me if a power supply has failed or if we are running in redundant power supply mode. That's the real challenge. On other platforms, Cisco has an option where you can just get an "OK" instead of all the hoopla. I need to see if I can embed a grep or the "| exclude" in the actual command sent to the N9K to resolve this. The erroneous chatter is a shame. Maybe there is a way to add it to the definition for FILTER_OSC. Thanks! On Fri, Jun 28, 2019 at 4:57 PM Piegorsch, Weylin William wrote: > Have you tied playing with FILTER_OSC? I haven't, I've just lived with > this output, but I remember similar conversations on this topic in the past. > > > > And you?re right ? NX-OS a while ago (2012-ish) didn?t have this issue; it > was a software update somewhere around 18months ago (30 months ago?) that I > started noticing it. > > > > weylin > > > > > > *From:* Vacheslav Zouhairy > *Sent:* Friday, June 28, 2019 1:13 AM > *To:* rancid-discuss at shrubbery.net > *Subject:* Re: [rancid] [SPAM?] Cisco NX "chatty" with Power info > > > > hmm i have almost the same hardware but your software is a bit newer, but > i do not get that in the output. it seems you will have to see what command > produces the offending output and tweak the nxos template not to run that > command > > > > On Thu, 2019-06-27 at 11:14 -0400, FF wrote: > > Hi, > > > > Thanks for responding.. > > > > This is: > > > > Software > BIOS: version 07.59 > NXOS: version 7.0(3)I7(3) > BIOS compile time: 08/26/2016 > NXOS image file is: bootflash:///nxos.7.0.3.I7.3.bin > NXOS compile time: 2/12/2018 13:00:00 [02/12/2018 19:13:48] > > > Hardware > cisco Nexus9000 C93120TX Chassis > Intel(R) Core(TM) i3- CPU @ 2.50GHz with 16400992 kB of memory. > > > > Thanks for your help! > > > > > > > > On Thu, Jun 27, 2019 at 5:01 AM Vacheslav Zouhairy > wrote: > > what model of nexus is this? > > > > On Thu, 2019-06-27 at 00:07 -0400, FF wrote: > > > > ok, > > > > So upgraded to 3.9 with the same behavior. > > > > Built a cisco-nxp personality that had an | exclude OK line on it to > remove the wattage line... but suddenly rancid couldn't log in to those > units. > > Even though clogin would work fine. Went and modified the cisco-nx > definition... but the exclude is ignored. > > > > So now 2.3 and 3.9 behave identically in this situation... > > > > This is the whole output for the section in question: > > > > Index: configs/10.10.10.114 > > =================================================================== > > retrieving revision 1.1157 > > diff -u -4 -r1.1157 10.10.10.114 > > @@ -80,10 +80,10 @@ > > !Env: Power Actual Total > > !Env: Supply Model Input Capacity Status > > !Env: (Watts ) (Watts ) > > !Env: ------- ---------- --------------- ------ ---------- > -------------------- > > - !Env: 1 N9K-PAC-1200W-B 192 W 1200 > W Ok > > - !Env: 2 N9K-PAC-1200W-B 174 W 1200 > W Ok > > + !Env: 1 N9K-PAC-1200W-B 190 W 1200 > W Ok > > + !Env: 2 N9K-PAC-1200W-B 176 W 1200 > W Ok > > !Env: Power Usage Summary: > > !Env: -------------------- > > !Env: Power Supply redundancy mode (configured) > PS-Redundant > > !Env: Power Supply redundancy mode (operational) > PS-Redundant > > > > Any suggestions? > > > > Thanks in advance! > > > > > > > > On Mon, Mar 11, 2019 at 5:02 PM FF wrote: > > Apparently I'm running a massively old version (2.3.8). I'll try upgrading > to 3.9 first. > > > > thanks in advance > > > > On Mon, Mar 11, 2019 at 4:47 PM heasley wrote: > > Fri, Mar 08, 2019 at 07:06:45PM +0000, Nick Nauwelaerts: > > what version of rancid & nx-os are you running? > > good question. > > if the answer is 3.9, please show us the complete output of show > environment power > > > i notice you only have 1 column less as me, you seem to miss "actual > output". > > > > > > // nick > > > > > > > > > > From: Rancid-discuss [mailto:rancid-discuss-bounces at shrubbery.net] On > Behalf Of FF > > Sent: Friday, March 8, 2019 18:05 > > To: rancid-discuss at shrubbery.net > > Subject: [SPAM?] [rancid] Cisco NX "chatty" with Power info > > > > > > !Env: Power Actual Total > > > > !Env: Supply Model Input Capacity > Status > > > > !Env: (Watts ) (Watts ) > > > > !Env: ------- ---------- --------------- ------ ---------- > -------------------- > > > > - !Env: 1 N9K-PAC-1200W-B 180 W 1200 W > Ok > > > > + !Env: 1 N9K-PAC-1200W-B 182 W 1200 W > Ok > > > > !Env: 2 N9K-PAC-1200W-B 162 W 1200 W > Ok > > > > !Env: Power Usage Summary: > > > > Every time Rancid runs, we get erroneous reports because the power usage > fluctuates by 1-2 watts per run. Any suggestions on how to keep the good > information (availability, etc) without getting this level of detail? > > > > thanks in advance! > > > > > > -- > > FF > > > > ________________________________ > > > > Volg Aquafin op Facebook | Twitter< > https://twitter.com/aquafinnv> | YouTube< > http://www.youtube.com/channel/UCk_4P5BJ-MtEEDCkCsR_KqQ?feature=mhee> | > LinkedIN | Instagram< > https://www.instagram.com/aquafin_nv/> > > > > In het kader van de uitoefening van onze taken verzamelen we bij Aquafin > persoonsgegevens. Hoe we omgaan met deze gegevens en wat de rechten van de > betrokkenen zijn, kan je nalezen in onze privacy policy< > https://www.aquafin.be/nl-be/privacy-policy>. > > > > P Denk aan het milieu. Druk deze mail niet onnodig af. > > > _______________________________________________ > > Rancid-discuss mailing list > > Rancid-discuss at shrubbery.net > > http://www.shrubbery.net/mailman/listinfo/rancid-discuss > > > > -- > > FF > > > _______________________________________________ > > Rancid-discuss mailing list > > > > *Rancid-discuss at shrubbery.net * > > > > > *http://www.shrubbery.net/mailman/listinfo/rancid-discuss * > > > > > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > http://www.shrubbery.net/mailman/listinfo/rancid-discuss > -- FF -------------- next part -------------- An HTML attachment was scrubbed... URL: From fusionfoto at gmail.com Mon Jul 1 20:43:30 2019 From: fusionfoto at gmail.com (FF) Date: Mon, 1 Jul 2019 16:43:30 -0400 Subject: [rancid] [SPAM?] Cisco NX "chatty" with Power info In-Reply-To: References: <50cc4a0483d54568a4d3e2547471fece@aquafin.be> <20190311204733.GI30597@shrubbery.net> <2cdcf9197356879b1dfa1d0486010ad5d3f363fa.camel@skno.by> Message-ID: For whomever may be helped by it... I just piggybacked on the filter passwords concept in /usr/local/libexec/rancid Here is what the diff looks like. It could be refined to just knock out the oscillating chatter on that line, or in case there are other PSUs... work for that. But my goal was to create as little collateral damage as possible. This seemed to be the easiest way to do it. Thanks for everyone who offered advice and assistance. @@ -1684,6 +1684,11 @@ sub WriteTerm { $lineauto = 1 if /^ modem auto/; /^ speed / && $lineauto && next; # kill speed on serial lines /^ clockrate / && next; # kill clockrate on serial interfaces + + if (/N9K-PAC-1200W-B/) { + ProcessHistory ("Power Supply Chatter removed\n"); + next; + } if (/^(enable )?(password|passwd)( level \d+)? / && $filter_pwds >= 1) { ProcessHistory("ENABLE","","","!$1$2$3 \n"); next; On Mon, Jul 1, 2019 at 4:01 PM FF wrote: > Hi, > > So I haven't messed with FILTER_OSC yet.. > > I would be glad to drop the power monitoring dialogue, except it seems to > be the only one I've identified that tells me if a power supply has failed > or if we are running in redundant power supply mode. > > That's the real challenge. On other platforms, Cisco has an option where > you can just get an "OK" instead of all the hoopla. > > I need to see if I can embed a grep or the "| exclude" in the actual > command sent to the N9K to resolve this. The erroneous chatter is a shame. > Maybe there is a way to add it to the definition for FILTER_OSC. > > Thanks! > > On Fri, Jun 28, 2019 at 4:57 PM Piegorsch, Weylin William > wrote: > >> Have you tied playing with FILTER_OSC? I haven't, I've just lived with >> this output, but I remember similar conversations on this topic in the past. >> >> >> >> And you?re right ? NX-OS a while ago (2012-ish) didn?t have this issue; >> it was a software update somewhere around 18months ago (30 months ago?) >> that I started noticing it. >> >> >> >> weylin >> >> >> >> >> >> *From:* Vacheslav Zouhairy >> *Sent:* Friday, June 28, 2019 1:13 AM >> *To:* rancid-discuss at shrubbery.net >> *Subject:* Re: [rancid] [SPAM?] Cisco NX "chatty" with Power info >> >> >> >> hmm i have almost the same hardware but your software is a bit newer, but >> i do not get that in the output. it seems you will have to see what command >> produces the offending output and tweak the nxos template not to run that >> command >> >> >> >> On Thu, 2019-06-27 at 11:14 -0400, FF wrote: >> >> Hi, >> >> >> >> Thanks for responding.. >> >> >> >> This is: >> >> >> >> Software >> BIOS: version 07.59 >> NXOS: version 7.0(3)I7(3) >> BIOS compile time: 08/26/2016 >> NXOS image file is: bootflash:///nxos.7.0.3.I7.3.bin >> NXOS compile time: 2/12/2018 13:00:00 [02/12/2018 19:13:48] >> >> >> Hardware >> cisco Nexus9000 C93120TX Chassis >> Intel(R) Core(TM) i3- CPU @ 2.50GHz with 16400992 kB of memory. >> >> >> >> Thanks for your help! >> >> >> >> >> >> >> >> On Thu, Jun 27, 2019 at 5:01 AM Vacheslav Zouhairy >> wrote: >> >> what model of nexus is this? >> >> >> >> On Thu, 2019-06-27 at 00:07 -0400, FF wrote: >> >> >> >> ok, >> >> >> >> So upgraded to 3.9 with the same behavior. >> >> >> >> Built a cisco-nxp personality that had an | exclude OK line on it to >> remove the wattage line... but suddenly rancid couldn't log in to those >> units. >> >> Even though clogin would work fine. Went and modified the cisco-nx >> definition... but the exclude is ignored. >> >> >> >> So now 2.3 and 3.9 behave identically in this situation... >> >> >> >> This is the whole output for the section in question: >> >> >> >> Index: configs/10.10.10.114 >> >> =================================================================== >> >> retrieving revision 1.1157 >> >> diff -u -4 -r1.1157 10.10.10.114 >> >> @@ -80,10 +80,10 @@ >> >> !Env: Power Actual Total >> >> !Env: Supply Model Input Capacity >> Status >> >> !Env: (Watts ) (Watts ) >> >> !Env: ------- ---------- --------------- ------ ---------- >> -------------------- >> >> - !Env: 1 N9K-PAC-1200W-B 192 W 1200 >> W Ok >> >> - !Env: 2 N9K-PAC-1200W-B 174 W 1200 >> W Ok >> >> + !Env: 1 N9K-PAC-1200W-B 190 W 1200 >> W Ok >> >> + !Env: 2 N9K-PAC-1200W-B 176 W 1200 >> W Ok >> >> !Env: Power Usage Summary: >> >> !Env: -------------------- >> >> !Env: Power Supply redundancy mode (configured) >> PS-Redundant >> >> !Env: Power Supply redundancy mode (operational) >> PS-Redundant >> >> >> >> Any suggestions? >> >> >> >> Thanks in advance! >> >> >> >> >> >> >> >> On Mon, Mar 11, 2019 at 5:02 PM FF wrote: >> >> Apparently I'm running a massively old version (2.3.8). I'll try >> upgrading to 3.9 first. >> >> >> >> thanks in advance >> >> >> >> On Mon, Mar 11, 2019 at 4:47 PM heasley wrote: >> >> Fri, Mar 08, 2019 at 07:06:45PM +0000, Nick Nauwelaerts: >> > what version of rancid & nx-os are you running? >> >> good question. >> >> if the answer is 3.9, please show us the complete output of show >> environment power >> >> > i notice you only have 1 column less as me, you seem to miss "actual >> output". >> > >> > >> > // nick >> > >> > >> > >> > >> > From: Rancid-discuss [mailto:rancid-discuss-bounces at shrubbery.net] On >> Behalf Of FF >> > Sent: Friday, March 8, 2019 18:05 >> > To: rancid-discuss at shrubbery.net >> > Subject: [SPAM?] [rancid] Cisco NX "chatty" with Power info >> > >> > >> > !Env: Power Actual Total >> > >> > !Env: Supply Model Input Capacity >> Status >> > >> > !Env: (Watts ) (Watts ) >> > >> > !Env: ------- ---------- --------------- ------ ---------- >> -------------------- >> > >> > - !Env: 1 N9K-PAC-1200W-B 180 W 1200 W >> Ok >> > >> > + !Env: 1 N9K-PAC-1200W-B 182 W 1200 W >> Ok >> > >> > !Env: 2 N9K-PAC-1200W-B 162 W 1200 W >> Ok >> > >> > !Env: Power Usage Summary: >> > >> > Every time Rancid runs, we get erroneous reports because the power >> usage fluctuates by 1-2 watts per run. Any suggestions on how to keep the >> good information (availability, etc) without getting this level of detail? >> > >> > thanks in advance! >> > >> > >> > -- >> > FF >> > >> > ________________________________ >> > >> > Volg Aquafin op Facebook | Twitter< >> https://twitter.com/aquafinnv> | YouTube< >> http://www.youtube.com/channel/UCk_4P5BJ-MtEEDCkCsR_KqQ?feature=mhee> | >> LinkedIN | Instagram< >> https://www.instagram.com/aquafin_nv/> >> > >> > In het kader van de uitoefening van onze taken verzamelen we bij >> Aquafin persoonsgegevens. Hoe we omgaan met deze gegevens en wat de rechten >> van de betrokkenen zijn, kan je nalezen in onze privacy policy< >> https://www.aquafin.be/nl-be/privacy-policy>. >> > >> > P Denk aan het milieu. Druk deze mail niet onnodig af. >> >> > _______________________________________________ >> > Rancid-discuss mailing list >> > Rancid-discuss at shrubbery.net >> > http://www.shrubbery.net/mailman/listinfo/rancid-discuss >> >> >> >> -- >> >> FF >> >> >> _______________________________________________ >> >> Rancid-discuss mailing list >> >> >> >> *Rancid-discuss at shrubbery.net * >> >> >> >> >> *http://www.shrubbery.net/mailman/listinfo/rancid-discuss * >> >> >> >> >> _______________________________________________ >> Rancid-discuss mailing list >> Rancid-discuss at shrubbery.net >> http://www.shrubbery.net/mailman/listinfo/rancid-discuss >> > > > -- > FF > -- FF -------------- next part -------------- An HTML attachment was scrubbed... URL: From heas at shrubbery.net Tue Jul 2 23:48:13 2019 From: heas at shrubbery.net (john heasley) Date: Tue, 2 Jul 2019 23:48:13 +0000 Subject: [rancid] Watchguard xml file In-Reply-To: References: Message-ID: <20190702234813.GA21015@shrubbery.net> Sat, Jun 29, 2019 at 11:46:23AM +0000, Wayne Eisenberg: > Hi, > > OK, so I can get into the firewall and pull the config with "export config to console". However, the config file is a very large xml file, this one is about 2MB in size. However, it seems like it only recorded the first 388KB of data. Is there a size limit on what rancid can process, or maybe there was a character in the xml that rancid didn't like and it just aborted processing it? How would I go about troubleshooting this? > there is no such limit. I would suspect a PAGER is involved, causing the output to cease. From kevin.moralez at gmail.com Wed Jul 3 13:48:09 2019 From: kevin.moralez at gmail.com (Kevin Morales) Date: Wed, 3 Jul 2019 07:48:09 -0600 Subject: [rancid] Unable to figure out "end of run not found" In-Reply-To: <1d2fd209f746444a8175c9d70ff35fa7@aquafin.be> References: <45f85a90802ea6391caee3dc8f7dcd734655e43c.camel@skno.by> <43D42FB0-DD40-4447-8E0D-D1E61B277756@comscore.com> <1d2fd209f746444a8175c9d70ff35fa7@aquafin.be> Message-ID: Hello!, How I can fix the problem when I run rancid for ZTE Router?. I get this error: *End of run not found* the two file 172.17.1.6.new and 172.17.1.6.raw don't show any error!. NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 loadtype: device type cisco loadtype: found device type cisco in /usr/local/rancid/etc/rancid.types.base executing clogin -t 90 -c"show version;show install active;show vlan;show running-config" 172.17.1.6 PROMPT MATCH: RT-ZTE# HIT COMMAND: RT-ZTE #show version In ShowVersion: RT-ZTE #show version HIT COMMAND: RT-ZTE #show install active In ShowInstallActive: RT-ZTE #show install active HIT COMMAND: RT-ZTE #show vlan In ShowVLAN: RT-ZTE #show vlan HIT COMMAND: RT-ZTE #show running-config In WriteTerm: RT-ZTE #show running-config *172.17.1.6 : End of run not found* 172.17.1.6: clean_run is false 172.17.1.6: found_end is false ! Thanks! On Wed, Jun 19, 2019 at 3:02 PM Nick Nauwelaerts < nick.nauwelaerts at aquafin.be> wrote: > iirc: > > ; is to seperate commands and wil execute the whole command string without > checking the return value of the previous command > > && will do the same, but if previous command returns not null (you can > check the return code of the previous command with "echo $?") it will end > the command list. > > > > the syntax is question "NOPIPE=yes" will set the environment variable > NOPIPE but only for the context of the command that's executed. > > appending ; between NOPIPE=yes & the command will _NOT_ do what you > expect; it will execute and empty cmd with nopipe env set to yes, then > execute your command with default envvars. > > > > > > you can compare: > > > > AHHA=aha env | grep -i aha > > output -> AHHA=aha > > AHHA=aha; env | grep -i aha > > output -> nothing > > > > export with set the envvar for your current session: > > env | grep -i nop > > -> output nothing > > export NOPIPE=yes > > env | grep -i nop > > output ->NOPIPE=yes > > > > all examples done in bash instead of my preferred zsh. if you use csh/tcsh > or fish i would guess you know what you're doing and can adapt the config > to work. > > > > bottom line, this is correct for most bourne shell derivatives: > > NOPIPE=yes rancid -d -t > > > > > > here's the catch, if you run rancid from cron you will either need to > uncomment the NOPIPE line in yr rancid.conf or add them to your crontab > entry. > > > > > > side note: > > the next rancid version will most likely change this behaviour: > > > https://github.com/haussli/rancid/commit/94318333c8f0d746abdd22cf4430636a394def8f > > > > > > > > // nick > > > > > > > > *From:* Rancid-discuss [mailto:rancid-discuss-bounces at shrubbery.net] *On > Behalf Of *Gauthier, Chris > *Sent:* Tuesday, June 18, 2019 00:10 > *To:* Piegorsch, Weylin William ; Michael Newton < > mnewton at pofp.com>; Vacheslav Zouhairy > *Cc:* rancid-discuss at shrubbery.net > *Subject:* Re: [rancid] Unable to figure out "end of run not found" > > > > Interesting. I thought it would get mixed up into the value of the > variable?. I?m not an expert programmer at all, but thought I needed to use > the ; to separate the commands appropriately. But, my expertise on shell > variables is a tad (understatement, really) limited. So, I shall defer! :) > > > > Cheers, > > Chris > > > > > > *Chris**?* > > *Gauthier* > > Senior Network Engineer > > | > > Comscore > > t +1 *(503) 331-2704* <(503)%20331-2704> > > | > > *cgauthier at comscore.com* > > *comscore.com* > > ???This e-mail (including any attachments) may contain information that is > private, confidential, or protected by attorney-client or other privilege. > If you received this e-mail in error, please delete it from your system and > notify sender. > > *From: *"Piegorsch, Weylin William" > *Date: *Monday, June 17, 2019 at 2:05 PM > *To: *"Gauthier, Chris" , Michael Newton < > mnewton at pofp.com>, Vacheslav Zouhairy > *Cc: *"rancid-discuss at shrubbery.net" > *Subject: *Re: [rancid] Unable to figure out "end of run not found" > > > > I actually don't use the semicolon. Not sure if this is bash specific, sh > specific, or posix general, but without the semicolon it sets the global > environment variable only for the duration of that command following the > variable definition, and unset it upon returning control to the cli. See > also your "export" comment, which has correlating implications regarding > environment vs namespace vs scope. > > But I'm not an experienced programmer, and don't pretend to grok the > various nuances, benefits, and pitfalls known by those who actually know > what they're doing, so if using the semicolon is better I all ears. > > weylin > > Sent from Outlook on my 'Droid > > > ------------------------------ > > *From:* Gauthier, Chris > *Sent:* Monday, June 17, 2019 4:22:02 PM > *To:* Piegorsch, Weylin William; Michael Newton; Vacheslav Zouhairy > *Cc:* rancid-discuss at shrubbery.net > *Subject:* Re: [rancid] Unable to figure out "end of run not found" > > > > Don?t forget the ; between the NOPIPE=yes and the rest of the command! :-) > > > > Some flavors of linux also want you to use the export command.. > > > > *Chris**?* > > *Gauthier* > > Senior Network Engineer > > | > > Comscore > > t +1 *(503) 331-2704* <(503)%20331-2704> > > | > > *cgauthier at comscore.com* > > *comscore.com* > > ???This e-mail (including any attachments) may contain information that is > private, confidential, or protected by attorney-client or other privilege. > If you received this e-mail in error, please delete it from your system and > notify sender. > > *From: *Rancid-discuss on behalf > of "Piegorsch, Weylin William" > *Date: *Saturday, June 15, 2019 at 7:52 AM > *To: *Michael Newton , Vacheslav Zouhairy < > m_zouhairy at skno.by> > *Cc: *"rancid-discuss at shrubbery.net" > *Subject: *Re: [rancid] Unable to figure out "end of run not found" > > > > > So this got me looking for how to do debug output per-host > > > > If you?re using Linux, the command is: > > > > NOPIPE=yes rancid -d -t > > > > This will generate two files: > > .new > > .raw > > > > .new is the parsed output > > .raw is pure log of the session (ie, it even captures non-printing > characters). I believe it?s only dumped if you set both NOPIPE and -d. > > > > I?ve used the .raw output on many occasions. At the moment I?m even > troubleshooting a device CPU issue that rancid tripped on that I wouldn?t > have found but for this; ?clogin -c ? was even working > fine. > > > > weylin > > > > *From: *Michael Newton > *Date: *Monday, June 10, 2019 at 11:25 AM > *To: *Vacheslav Zouhairy > *Cc: * > *Subject: *Re: [rancid] Unable to figure out "end of run not found" > > > > No, there is not. But I guess you're thinking maybe the login doesn't work > because of heavy traffic. > > > > So this got me looking for how to do debug output per-host. There doesn't > seem to be, but that got me to this posting: > https://www.shrubbery.net/pipermail/rancid-discuss/2015-October/008742.html > > And that made me notice that my router.db entry was corrupt (wrong device > type.) > > > > So, typical user error. Thanks for (indirectly) getting me in the right > direction though! > > > > Mike > > > > > > On Mon, 10 Jun 2019 at 00:22, Vacheslav Zouhairy > wrote: > > Is there any bandwidth hog on those switches by any chance? > > On Fri, 2019-06-07 at 18:16 -0600, Michael Newton wrote: > > Hi all, we manage about 200-300 Brocade ICX switches across a number > > of locations. All but two are being successfully polled. The two in > > question (same firmware and a similar config to the others) > > consistently show "end of run not found" when run as part of the cron > > job. But, when run manually for troubleshooting, everything seems to > > work. > > > > `sudo -u rancid flogin switchname` works fine and logs me into the > > switch. > > > > `sudo -u rancid flogin -c 'show version;show clock' switchname` > > likewise works fine. Logs in, runs the commands, and logs out. > > > > `sudo -u rancid rancid -t foundry -d switchname` runs through > > everything perfectly. The switchname.new > > file has the expected > > output. > > > > Yet the log for the automated process consistently shows this: > > > > switchname: End of run not found > > end > > > > Is there anything else I can do to troubleshoot this? Thanks in > > advance! > > > > Mike > > _______________________________________________ > > Rancid-discuss mailing list > > Rancid-discuss at shrubbery.net > > http://www.shrubbery.net/mailman/listinfo/rancid-discuss > > > > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > http://www.shrubbery.net/mailman/listinfo/rancid-discuss > > > > ------------------------------ > > *Volg Aquafin op Facebook | Twitter > | YouTube > | > LinkedIN | Instagram > * > > In het kader van de uitoefening van onze taken verzamelen we bij Aquafin > persoonsgegevens. Hoe we omgaan met deze gegevens en wat de rechten van de > betrokkenen zijn, kan je nalezen in onze privacy policy > . > > P Denk aan het milieu. Druk deze mail niet onnodig af. > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > http://www.shrubbery.net/mailman/listinfo/rancid-discuss > -- *Kevin Morales* -------------- next part -------------- An HTML attachment was scrubbed... URL: From Wayne.Eisenberg at CarolinasIT.com Wed Jul 3 16:18:25 2019 From: Wayne.Eisenberg at CarolinasIT.com (Wayne Eisenberg) Date: Wed, 3 Jul 2019 16:18:25 +0000 Subject: [rancid] Watchguard xml file In-Reply-To: <20190702234813.GA21015@shrubbery.net> References: <20190702234813.GA21015@shrubbery.net> Message-ID: If I run the export command manually, it just dumps the whole thing to the screen without any breaks or requests to 'hit space to continue' or things like that, so I don't *think* it's a page length type setting? Actually, I just did another review and I'm thinking that it has something to do with the prompt definition. Just so we're looking at the same thing, the files are here: https://github.com/hillscott/rancid-watchguard. Forked from https://bitbucket.org/aquerubin/rancid-vyatta. In the xtmlogin file, it sets the prompt (line 436) to something I don't see. In this original state, xtmlogin never recognized it finished the login. When I changed that line to set prompt ">>|#" then xtmlogin completes successfully. (The prompt for this watchguard firewall is "WG#") ----------- foreach router [lrange $argv $i end] { set router [string tolower $router] send_user "$router\n" # device timeout set timeout [find timeout $router] if { [llength $timeout] == 0 } { set timeout $timeoutdflt } set prompt ">>" # Figure out username if {[info exists username]} { ----------- However, in the xtm.pm module, line 102 defines it again. ----------- while (/\s*($cmds_regexp)\s*$/) { $cmd = $1; $prompt = ">>"; if (!defined($prompt)) { $prompt = ($_ =~ /^([^>]+>)/)[0]; $prompt =~ s/([][}{)(\\])/\\$1/g; print STDERR ("PROMPT MATCH: $prompt\n") if ($debug); } print STDERR ("HIT COMMAND:$_") if ($debug); if (! defined($commands{$cmd})) { print STDERR "$host: found unexpected command - \"$cmd\"\n"; $clean_run = 0; last TOP; } $rval = &{$commands{$cmd}}($INPUT, $OUTPUT, $cmd); delete($commands{$cmd}); if ($rval == -1) { $clean_run = 0; last TOP; } } ----------- Once you get to the sub ShowConfiguration section, on line 199 if it sees the prompt, end. Guess what? The "#" character is inside the config (there is some html code in one of the xml sections) and that is where the config ends. ----------- sub ShowConfiguration { my($INPUT, $OUTPUT, $cmd) = @_; my($lines) = 0; my($snmp) = 0; print STDERR " In ShowConfiguration: $_" if ($debug); # We don't care about password filtering as passwords are hashed # So don't use this if you need it (or develop the functionality). if ($filter_pwds >= 1){ print STDERR "WARNING: Password filtering isn't implemented yet!\n"; print STDERR "Either disable password filtering in rancid.conf"; print STDERR " or don't use this plugin.\n"; } s/^[a-z]+@//; ProcessHistory("","","","# $_"); while (<$INPUT>) { tr/\015//d; next if (/^\s*$/); # end of config - hopefully. # end-of-config tag. appears to end with "\nPROMPT:~$". if (/$prompt/) { $found_end++; last; } ----------- So I'm thinking if I can figure out a different way to define the prompt to be more than just the # sign (at least in the xtm.pm), that should do the trick? Can you do something like $prompt = "#$" ? Wayne -----Original Message----- From: john heasley Sent: Tuesday, July 02, 2019 7:48 PM To: Wayne Eisenberg Cc: 'rancid-discuss at shrubbery.net' Subject: Re: [rancid] Watchguard xml file Sat, Jun 29, 2019 at 11:46:23AM +0000, Wayne Eisenberg: > Hi, > > OK, so I can get into the firewall and pull the config with "export config to console". However, the config file is a very large xml file, this one is about 2MB in size. However, it seems like it only recorded the first 388KB of data. Is there a size limit on what rancid can process, or maybe there was a character in the xml that rancid didn't like and it just aborted processing it? How would I go about troubleshooting this? > there is no such limit. I would suspect a PAGER is involved, causing the output to cease. From heas at shrubbery.net Wed Jul 3 17:12:10 2019 From: heas at shrubbery.net (john heasley) Date: Wed, 3 Jul 2019 17:12:10 +0000 Subject: [rancid] Unable to figure out "end of run not found" In-Reply-To: References: <45f85a90802ea6391caee3dc8f7dcd734655e43c.camel@skno.by> <43D42FB0-DD40-4447-8E0D-D1E61B277756@comscore.com> <1d2fd209f746444a8175c9d70ff35fa7@aquafin.be> Message-ID: <20190703171210.GC19867@shrubbery.net> Wed, Jul 03, 2019 at 07:48:09AM -0600, Kevin Morales: > Hello!, > > How I can fix the problem when I run rancid for ZTE Router?. I get this > error: *End of run not found* > > the two file 172.17.1.6.new and 172.17.1.6.raw don't show any error!. > > NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 > > loadtype: device type cisco > loadtype: found device type cisco in /usr/local/rancid/etc/rancid.types.base > executing clogin -t 90 -c"show version;show install active;show vlan;show > running-config" 172.17.1.6 > PROMPT MATCH: RT-ZTE# > HIT COMMAND: RT-ZTE #show version > In ShowVersion: RT-ZTE #show version > HIT COMMAND: RT-ZTE #show install active > In ShowInstallActive: RT-ZTE #show install active > HIT COMMAND: RT-ZTE #show vlan > In ShowVLAN: RT-ZTE #show vlan > HIT COMMAND: RT-ZTE #show running-config > In WriteTerm: RT-ZTE #show running-config > *172.17.1.6 : End of run not found* > 172.17.1.6: clean_run is false > 172.17.1.6: found_end is false > ! found end means that it found the end of the config; for type cisco, that means "^end". clean run means that it found the cli logout; for type cisco, that means "prompt[>#] exit$" From kevin.moralez at gmail.com Wed Jul 3 17:22:26 2019 From: kevin.moralez at gmail.com (Kevin Morales) Date: Wed, 3 Jul 2019 11:22:26 -0600 Subject: [rancid] Unable to figure out "end of run not found" In-Reply-To: <20190703171210.GC19867@shrubbery.net> References: <45f85a90802ea6391caee3dc8f7dcd734655e43c.camel@skno.by> <43D42FB0-DD40-4447-8E0D-D1E61B277756@comscore.com> <1d2fd209f746444a8175c9d70ff35fa7@aquafin.be> <20190703171210.GC19867@shrubbery.net> Message-ID: ok, but rancid do not save the output of command., How I can fix this problem? On Wed, Jul 3, 2019 at 11:12 AM john heasley wrote: > Wed, Jul 03, 2019 at 07:48:09AM -0600, Kevin Morales: > > Hello!, > > > > How I can fix the problem when I run rancid for ZTE Router?. I get this > > error: *End of run not found* > > > > the two file 172.17.1.6.new and 172.17.1.6.raw don't show any error!. > > > > NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 > > > > loadtype: device type cisco > > loadtype: found device type cisco in > /usr/local/rancid/etc/rancid.types.base > > executing clogin -t 90 -c"show version;show install active;show vlan;show > > running-config" 172.17.1.6 > > PROMPT MATCH: RT-ZTE# > > HIT COMMAND: RT-ZTE #show version > > In ShowVersion: RT-ZTE #show version > > HIT COMMAND: RT-ZTE #show install active > > In ShowInstallActive: RT-ZTE #show install active > > HIT COMMAND: RT-ZTE #show vlan > > In ShowVLAN: RT-ZTE #show vlan > > HIT COMMAND: RT-ZTE #show running-config > > In WriteTerm: RT-ZTE #show running-config > > *172.17.1.6 : End of run not found* > > 172.17.1.6: clean_run is false > > 172.17.1.6: found_end is false > > ! > > found end means that it found the end of the config; for type cisco, > that means "^end". > > clean run means that it found the cli logout; for type cisco, that > means "prompt[>#] exit$" > > -- *Kevin Morales* -------------- next part -------------- An HTML attachment was scrubbed... URL: From weylin at bu.edu Wed Jul 3 17:29:15 2019 From: weylin at bu.edu (Piegorsch, Weylin William) Date: Wed, 3 Jul 2019 17:29:15 +0000 Subject: [rancid] Unable to figure out "end of run not found" In-Reply-To: References: <45f85a90802ea6391caee3dc8f7dcd734655e43c.camel@skno.by> <43D42FB0-DD40-4447-8E0D-D1E61B277756@comscore.com> <1d2fd209f746444a8175c9d70ff35fa7@aquafin.be> <20190703171210.GC19867@shrubbery.net> Message-ID: <5820A16C-B7E4-4914-98AE-4ABACD5415AD@bu.edu> If you do the ?NOPIPE? thing and run rancid with the -d flag, that will create two files: .new, and .raw. Examine the .raw file, which is the byte-for-byte capture of the CLI session. weylin From: Kevin Morales Date: Wednesday, July 3, 2019 at 1:24 PM To: john heasley Cc: Nick Nauwelaerts , Weylin Piegorsch , "rancid-discuss at shrubbery.net" Subject: Re: [rancid] Unable to figure out "end of run not found" ok, but rancid do not save the output of command., How I can fix this problem? On Wed, Jul 3, 2019 at 11:12 AM john heasley > wrote: Wed, Jul 03, 2019 at 07:48:09AM -0600, Kevin Morales: > Hello!, > > How I can fix the problem when I run rancid for ZTE Router?. I get this > error: *End of run not found* > > the two file 172.17.1.6.new and 172.17.1.6.raw don't show any error!. > > NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 > > loadtype: device type cisco > loadtype: found device type cisco in /usr/local/rancid/etc/rancid.types.base > executing clogin -t 90 -c"show version;show install active;show vlan;show > running-config" 172.17.1.6 > PROMPT MATCH: RT-ZTE# > HIT COMMAND: RT-ZTE #show version > In ShowVersion: RT-ZTE #show version > HIT COMMAND: RT-ZTE #show install active > In ShowInstallActive: RT-ZTE #show install active > HIT COMMAND: RT-ZTE #show vlan > In ShowVLAN: RT-ZTE #show vlan > HIT COMMAND: RT-ZTE #show running-config > In WriteTerm: RT-ZTE #show running-config > *172.17.1.6 : End of run not found* > 172.17.1.6: clean_run is false > 172.17.1.6: found_end is false > ! found end means that it found the end of the config; for type cisco, that means "^end". clean run means that it found the cli logout; for type cisco, that means "prompt[>#] exit$" -- Kevin Morales -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin.moralez at gmail.com Wed Jul 3 17:33:08 2019 From: kevin.moralez at gmail.com (Kevin Morales) Date: Wed, 3 Jul 2019 11:33:08 -0600 Subject: [rancid] Unable to figure out "end of run not found" In-Reply-To: <5820A16C-B7E4-4914-98AE-4ABACD5415AD@bu.edu> References: <45f85a90802ea6391caee3dc8f7dcd734655e43c.camel@skno.by> <43D42FB0-DD40-4447-8E0D-D1E61B277756@comscore.com> <1d2fd209f746444a8175c9d70ff35fa7@aquafin.be> <20190703171210.GC19867@shrubbery.net> <5820A16C-B7E4-4914-98AE-4ABACD5415AD@bu.edu> Message-ID: Thanks Piegorsh, I did it.. NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 but in the two file 172.17.1.6.new and 172.17.1.6.raw don't see anything about this error. both show the correct command output. On Wed, Jul 3, 2019 at 11:29 AM Piegorsch, Weylin William wrote: > If you do the ?NOPIPE? thing and run rancid with the -d flag, that will > create two files: .new, and .raw. Examine the .raw file, which is the > byte-for-byte capture of the CLI session. > > weylin > > > > *From: *Kevin Morales > *Date: *Wednesday, July 3, 2019 at 1:24 PM > *To: *john heasley > *Cc: *Nick Nauwelaerts , Weylin Piegorsch < > weylin at bu.edu>, "rancid-discuss at shrubbery.net" < > rancid-discuss at shrubbery.net> > *Subject: *Re: [rancid] Unable to figure out "end of run not found" > > > > ok, but rancid do not save the output of command., > > > > How I can fix this problem? > > > > On Wed, Jul 3, 2019 at 11:12 AM john heasley wrote: > > Wed, Jul 03, 2019 at 07:48:09AM -0600, Kevin Morales: > > Hello!, > > > > How I can fix the problem when I run rancid for ZTE Router?. I get this > > error: *End of run not found* > > > > the two file 172.17.1.6.new and 172.17.1.6.raw don't show any error!. > > > > NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 > > > > loadtype: device type cisco > > loadtype: found device type cisco in > /usr/local/rancid/etc/rancid.types.base > > executing clogin -t 90 -c"show version;show install active;show vlan;show > > running-config" 172.17.1.6 > > PROMPT MATCH: RT-ZTE# > > HIT COMMAND: RT-ZTE #show version > > In ShowVersion: RT-ZTE #show version > > HIT COMMAND: RT-ZTE #show install active > > In ShowInstallActive: RT-ZTE #show install active > > HIT COMMAND: RT-ZTE #show vlan > > In ShowVLAN: RT-ZTE #show vlan > > HIT COMMAND: RT-ZTE #show running-config > > In WriteTerm: RT-ZTE #show running-config > > *172.17.1.6 : End of run not found* > > 172.17.1.6: clean_run is false > > 172.17.1.6: found_end is false > > ! > > found end means that it found the end of the config; for type cisco, > that means "^end". > > clean run means that it found the cli logout; for type cisco, that > means "prompt[>#] exit$" > > > > > -- > > *Kevin Morales* > -- *Kevin Morales* -------------- next part -------------- An HTML attachment was scrubbed... URL: From heas at shrubbery.net Wed Jul 3 17:40:48 2019 From: heas at shrubbery.net ('john heasley') Date: Wed, 3 Jul 2019 17:40:48 +0000 Subject: [rancid] Watchguard xml file In-Reply-To: References: <20190702234813.GA21015@shrubbery.net> Message-ID: <20190703174048.GD19867@shrubbery.net> Wed, Jul 03, 2019 at 04:18:25PM +0000, Wayne Eisenberg: > If I run the export command manually, it just dumps the whole thing to the screen without any breaks or requests to 'hit space to continue' or things like that, so I don't *think* it's a page length type setting? > > Actually, I just did another review and I'm thinking that it has something to do with the prompt definition. Just so we're looking at the same thing, the files are here: https://github.com/hillscott/rancid-watchguard. Forked from https://bitbucket.org/aquerubin/rancid-vyatta. > > In the xtmlogin file, it sets the prompt (line 436) to something I don't see. In this original state, xtmlogin never recognized it finished the login. When I changed that line to > set prompt ">>|#" > then xtmlogin completes successfully. (The prompt for this watchguard firewall is "WG#") > > However, in the xtm.pm module, line 102 defines it again. i'm not familiar with this device, but redefining (or refining) the prompt is normal. the filter functions and login scripts begin with something loose, and once it sees the prompt, it can be refined to be more precise, and may later further refine it (eg: in run_commands) to match the prompt when/if it changes in config or other modes that are platform dependent. > ----------- > while (/\s*($cmds_regexp)\s*$/) { > $cmd = $1; > $prompt = ">>"; ^^^^^^^^^^^^ this is probably a mistake; should be part of the while() regex. I suspect it might be here because the author could not make the regex below match correctly. > if (!defined($prompt)) { > $prompt = ($_ =~ /^([^>]+>)/)[0]; > $prompt =~ s/([][}{)(\\])/\\$1/g; > print STDERR ("PROMPT MATCH: $prompt\n") if ($debug); > } > print STDERR ("HIT COMMAND:$_") if ($debug); > if (! defined($commands{$cmd})) { > print STDERR "$host: found unexpected command - \"$cmd\"\n"; > $clean_run = 0; > last TOP; > } > $rval = &{$commands{$cmd}}($INPUT, $OUTPUT, $cmd); > delete($commands{$cmd}); > if ($rval == -1) { > $clean_run = 0; > last TOP; > } > } > ----------- > Once you get to the sub ShowConfiguration section, on line 199 if it sees the prompt, end. Guess what? The "#" character is inside the config (there is some html code in one of the xml sections) and that is where the config ends. seems that the prompt is ">>". > ----------- > sub ShowConfiguration { > my($INPUT, $OUTPUT, $cmd) = @_; > my($lines) = 0; > my($snmp) = 0; > print STDERR " In ShowConfiguration: $_" if ($debug); > # We don't care about password filtering as passwords are hashed > # So don't use this if you need it (or develop the functionality). > if ($filter_pwds >= 1){ > print STDERR "WARNING: Password filtering isn't implemented yet!\n"; > print STDERR "Either disable password filtering in rancid.conf"; > print STDERR " or don't use this plugin.\n"; > } > s/^[a-z]+@//; > ProcessHistory("","","","# $_"); > while (<$INPUT>) { > tr/\015//d; > next if (/^\s*$/); > # end of config - hopefully. > # end-of-config tag. appears to end with "\nPROMPT:~$". > if (/$prompt/) { > $found_end++; > last; > } > ----------- > > So I'm thinking if I can figure out a different way to define the prompt to be more than just the # sign (at least in the xtm.pm), that should do the trick? Can you do something like $prompt = "#$" ? its better to anchor it and have it be as complete as reasonable. eg: not # not hostname# but ^hostname# look at ios.pm. . > Wayne > > > > -----Original Message----- > From: john heasley > Sent: Tuesday, July 02, 2019 7:48 PM > To: Wayne Eisenberg > Cc: 'rancid-discuss at shrubbery.net' > Subject: Re: [rancid] Watchguard xml file > > Sat, Jun 29, 2019 at 11:46:23AM +0000, Wayne Eisenberg: > > Hi, > > > > OK, so I can get into the firewall and pull the config with "export config to console". However, the config file is a very large xml file, this one is about 2MB in size. However, it seems like it only recorded the first 388KB of data. Is there a size limit on what rancid can process, or maybe there was a character in the xml that rancid didn't like and it just aborted processing it? How would I go about troubleshooting this? > > > > there is no such limit. I would suspect a PAGER is involved, causing the output to cease. > From heas at shrubbery.net Wed Jul 3 18:43:27 2019 From: heas at shrubbery.net (john heasley) Date: Wed, 3 Jul 2019 18:43:27 +0000 Subject: [rancid] Unable to figure out "end of run not found" In-Reply-To: References: <43D42FB0-DD40-4447-8E0D-D1E61B277756@comscore.com> <1d2fd209f746444a8175c9d70ff35fa7@aquafin.be> <20190703171210.GC19867@shrubbery.net> <5820A16C-B7E4-4914-98AE-4ABACD5415AD@bu.edu> Message-ID: <20190703184327.GC11812@shrubbery.net> Wed, Jul 03, 2019 at 11:33:08AM -0600, Kevin Morales: > Thanks Piegorsh, > > I did it.. > > NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 > > but in the two file 172.17.1.6.new and 172.17.1.6.raw don't see anything > about this error. both show the correct command output. correct command output and matching the criteria that i described below for type cisco are not necessarily the same thing. read it again. > On Wed, Jul 3, 2019 at 11:29 AM Piegorsch, Weylin William > wrote: > > > *172.17.1.6 : End of run not found* > > > 172.17.1.6: clean_run is false > > > 172.17.1.6: found_end is false > > > ! > > > > found end means that it found the end of the config; for type cisco, > > that means "^end". > > > > clean run means that it found the cli logout; for type cisco, that > > means "prompt[>#] exit$" From Wayne.Eisenberg at CarolinasIT.com Wed Jul 3 18:49:20 2019 From: Wayne.Eisenberg at CarolinasIT.com (Wayne Eisenberg) Date: Wed, 3 Jul 2019 18:49:20 +0000 Subject: [rancid] Watchguard xml file In-Reply-To: <20190703174048.GD19867@shrubbery.net> References: <20190702234813.GA21015@shrubbery.net> <20190703174048.GD19867@shrubbery.net> Message-ID: -----Original Message----- From: 'john heasley' Sent: Wednesday, July 03, 2019 1:41 PM To: Wayne Eisenberg Cc: 'john heasley' ; 'rancid-discuss at shrubbery.net' Subject: Re: [rancid] Watchguard xml file >> However, in the xtm.pm module, line 102 defines it again. >i'm not familiar with this device, but redefining (or refining) the prompt is normal. the filter functions and login scripts begin with something loose, and once it sees the prompt, it can be refined to be more precise, and >may later further refine it (eg: in run_commands) to match the prompt when/if it changes in config or other modes that are platform dependent. Ah, if I only had that skill. >> ----------- >> while (/\s*($cmds_regexp)\s*$/) { >> $cmd = $1; >> $prompt = ">>"; ^^^^^^^^^^^^ this is probably a mistake; should be part of the while() regex. I suspect it might be here because the author could not make the regex below match correctly. >> if (!defined($prompt)) { >> $prompt = ($_ =~ /^([^>]+>)/)[0]; >> $prompt =~ s/([][}{)(\\])/\\$1/g; >> print STDERR ("PROMPT MATCH: $prompt\n") if ($debug); >> } >> ----------- >> Once you get to the sub ShowConfiguration section, on line 199 if it sees the prompt, end. Guess what? The "#" character is inside the config (there is some html code in one of the xml sections) and that is where the config ends. >seems that the prompt is ">>". Yes, in this example. I wanted to show the original file, not something that I modded. In my current version, the line is $prompt = ">>|#" which works, but causes the problem of the config getting truncated because it sees "#" as the prompt. The $prompt should either be the entire thing or some string that ends in #. >> ----------- >> sub ShowConfiguration { >> my($INPUT, $OUTPUT, $cmd) = @_; >> my($lines) = 0; >> my($snmp) = 0; >> print STDERR " In ShowConfiguration: $_" if ($debug); >> # We don't care about password filtering as passwords are hashed >> # So don't use this if you need it (or develop the functionality). >> if ($filter_pwds >= 1){ >> print STDERR "WARNING: Password filtering isn't implemented yet!\n"; >> print STDERR "Either disable password filtering in rancid.conf"; >> print STDERR " or don't use this plugin.\n"; >> } >> s/^[a-z]+@//; >> ProcessHistory("","","","# $_"); >> while (<$INPUT>) { >> tr/\015//d; >> next if (/^\s*$/); >> # end of config - hopefully. >> # end-of-config tag. appears to end with "\nPROMPT:~$". >> if (/$prompt/) { >> $found_end++; >> last; >> } >> ----------- >> >> So I'm thinking if I can figure out a different way to define the prompt to be more than just the # sign (at least in the xtm.pm), that should do the trick? Can you do something like $prompt = "#$" ? >its better to anchor it and have it be as complete as reasonable. eg: >not # >not hostname# >but ^hostname# >look at ios.pm. Looking, but I don't see anywhere that it defines the prompt. It uses it a lot, but doesn't define it. Thanks, Wayne From kevin.moralez at gmail.com Wed Jul 3 19:18:58 2019 From: kevin.moralez at gmail.com (Kevin Morales) Date: Wed, 3 Jul 2019 13:18:58 -0600 Subject: [rancid] Unable to figure out "end of run not found" In-Reply-To: <20190703184327.GC11812@shrubbery.net> References: <43D42FB0-DD40-4447-8E0D-D1E61B277756@comscore.com> <1d2fd209f746444a8175c9d70ff35fa7@aquafin.be> <20190703171210.GC19867@shrubbery.net> <5820A16C-B7E4-4914-98AE-4ABACD5415AD@bu.edu> <20190703184327.GC11812@shrubbery.net> Message-ID: I am sorry, I dont get you, What do you want I do? on my Rancid Server I execute: [rancid at localhost bin]$ NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 On Wed, Jul 3, 2019 at 12:43 PM john heasley wrote: > Wed, Jul 03, 2019 at 11:33:08AM -0600, Kevin Morales: > > Thanks Piegorsh, > > > > I did it.. > > > > NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 > > > > but in the two file 172.17.1.6.new and 172.17.1.6.raw don't see anything > > about this error. both show the correct command output. > > correct command output and matching the criteria that i described below > for type cisco are not necessarily the same thing. read it again. > > > On Wed, Jul 3, 2019 at 11:29 AM Piegorsch, Weylin William > > > wrote: > > > > *172.17.1.6 : End of run not found* > > > > 172.17.1.6: clean_run is false > > > > 172.17.1.6: found_end is false > > > > ! > > > > > > found end means that it found the end of the config; for type cisco, > > > that means "^end". > > > > > > clean run means that it found the cli logout; for type cisco, that > > > means "prompt[>#] exit$" > -- *Kevin Morales* -------------- next part -------------- An HTML attachment was scrubbed... URL: From KyleSheeter at XRITE.com Wed Jul 3 20:17:46 2019 From: KyleSheeter at XRITE.com (Sheeter, Kyle) Date: Wed, 3 Jul 2019 20:17:46 +0000 Subject: [rancid] Rancid.Conf Disappeared on Ubuntu Update Message-ID: Hey all, I was doing some Ubuntu upgrades on my server, and just noticed that RANCID stop sending me updates. Ran the rancid-run command and then found out that my rancid.conf file disappeared. Anyone know the best way to recreate the conf file? All of my other information is still there it seems, and the DB is still populated with my old network data. Thanks! Kyle James Sheeter Please be advised that this email may contain confidential information. If you are not the intended recipient, please notify us by email by replying to the sender and delete this message. The sender disclaims that the content of this email constitutes an offer to enter into, or the acceptance of, any agreement; provided that the foregoing does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment. -------------- next part -------------- An HTML attachment was scrubbed... URL: From weylin at bu.edu Wed Jul 3 20:52:30 2019 From: weylin at bu.edu (Piegorsch, Weylin William) Date: Wed, 3 Jul 2019 20:52:30 +0000 Subject: [rancid] Unable to figure out "end of run not found" In-Reply-To: References: <43D42FB0-DD40-4447-8E0D-D1E61B277756@comscore.com> <1d2fd209f746444a8175c9d70ff35fa7@aquafin.be> <20190703171210.GC19867@shrubbery.net> <5820A16C-B7E4-4914-98AE-4ABACD5415AD@bu.edu> <20190703184327.GC11812@shrubbery.net> Message-ID: <3E6063D0-2EE0-4A69-A886-0DC54DF88A0D@bu.edu> Hi Kevin, I think you said this is a ZTE device, but that you?re using -t cisco. is ZTE a cisco device? weylin From: Kevin Morales Date: Wednesday, July 3, 2019 at 3:18 PM To: john heasley Cc: Weylin Piegorsch , Nick Nauwelaerts , "rancid-discuss at shrubbery.net" Subject: Re: [rancid] Unable to figure out "end of run not found" I am sorry, I dont get you, What do you want I do? on my Rancid Server I execute: [rancid at localhost bin]$ NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 On Wed, Jul 3, 2019 at 12:43 PM john heasley > wrote: Wed, Jul 03, 2019 at 11:33:08AM -0600, Kevin Morales: > Thanks Piegorsh, > > I did it.. > > NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 > > but in the two file 172.17.1.6.new and 172.17.1.6.raw don't see anything > about this error. both show the correct command output. correct command output and matching the criteria that i described below for type cisco are not necessarily the same thing. read it again. > On Wed, Jul 3, 2019 at 11:29 AM Piegorsch, Weylin William > > wrote: > > > *172.17.1.6 : End of run not found* > > > 172.17.1.6: clean_run is false > > > 172.17.1.6: found_end is false > > > ! > > > > found end means that it found the end of the config; for type cisco, > > that means "^end". > > > > clean run means that it found the cli logout; for type cisco, that > > means "prompt[>#] exit$" -- Kevin Morales -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin.moralez at gmail.com Wed Jul 3 20:53:14 2019 From: kevin.moralez at gmail.com (Kevin Morales) Date: Wed, 3 Jul 2019 14:53:14 -0600 Subject: [rancid] Unable to figure out "end of run not found" In-Reply-To: <3E6063D0-2EE0-4A69-A886-0DC54DF88A0D@bu.edu> References: <43D42FB0-DD40-4447-8E0D-D1E61B277756@comscore.com> <1d2fd209f746444a8175c9d70ff35fa7@aquafin.be> <20190703171210.GC19867@shrubbery.net> <5820A16C-B7E4-4914-98AE-4ABACD5415AD@bu.edu> <20190703184327.GC11812@shrubbery.net> <3E6063D0-2EE0-4A69-A886-0DC54DF88A0D@bu.edu> Message-ID: Yes, my Router is ZTE and I am using CISCO type, because the command is the same to see the configuration..show running-config On Wed, Jul 3, 2019 at 2:52 PM Piegorsch, Weylin William wrote: > Hi Kevin, > > I think you said this is a ZTE device, but that you?re using -t cisco. is > ZTE a cisco device? > > weylin > > > > *From: *Kevin Morales > *Date: *Wednesday, July 3, 2019 at 3:18 PM > *To: *john heasley > *Cc: *Weylin Piegorsch , Nick Nauwelaerts < > nick.nauwelaerts at aquafin.be>, "rancid-discuss at shrubbery.net" < > rancid-discuss at shrubbery.net> > *Subject: *Re: [rancid] Unable to figure out "end of run not found" > > > > I am sorry, I dont get you, What do you want I do? > > > > on my Rancid Server I execute: > > [rancid at localhost bin]$ NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 > > > > On Wed, Jul 3, 2019 at 12:43 PM john heasley wrote: > > Wed, Jul 03, 2019 at 11:33:08AM -0600, Kevin Morales: > > Thanks Piegorsh, > > > > I did it.. > > > > NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 > > > > but in the two file 172.17.1.6.new and 172.17.1.6.raw don't see anything > > about this error. both show the correct command output. > > correct command output and matching the criteria that i described below > for type cisco are not necessarily the same thing. read it again. > > > On Wed, Jul 3, 2019 at 11:29 AM Piegorsch, Weylin William > > > wrote: > > > > *172.17.1.6 : End of run not found* > > > > 172.17.1.6: clean_run is false > > > > 172.17.1.6: found_end is false > > > > ! > > > > > > found end means that it found the end of the config; for type cisco, > > > that means "^end". > > > > > > clean run means that it found the cli logout; for type cisco, that > > > means "prompt[>#] exit$" > > > > > -- > > *Kevin Morales* > -- *Kevin Morales* -------------- next part -------------- An HTML attachment was scrubbed... URL: From heas at shrubbery.net Thu Jul 4 00:10:36 2019 From: heas at shrubbery.net (john heasley) Date: Thu, 4 Jul 2019 00:10:36 +0000 Subject: [rancid] Unable to figure out "end of run not found" In-Reply-To: References: <1d2fd209f746444a8175c9d70ff35fa7@aquafin.be> <20190703171210.GC19867@shrubbery.net> <5820A16C-B7E4-4914-98AE-4ABACD5415AD@bu.edu> <20190703184327.GC11812@shrubbery.net> <3E6063D0-2EE0-4A69-A886-0DC54DF88A0D@bu.edu> Message-ID: <20190704001036.GB84573@shrubbery.net> Wed, Jul 03, 2019 at 02:53:14PM -0600, Kevin Morales: > Yes, my Router is ZTE and I am using CISCO type, because the command is the > same to see the configuration..show running-config I have no idea what ZTE is; does it behave *exactly* the same as IOS? It seems not. > > > > found end means that it found the end of the config; for type cisco, > > > > that means "^end". Does it's config end with: " end "? > > > > clean run means that it found the cli logout; for type cisco, that > > > > means "prompt[>#] exit$" in your .raw file, does the last prompt where clogin exited the cli, match the regex "prompt[>#] exit$" ? clearly these sanity checks are not working with your ZTE device. You need to figure-out why and correct it, likely by creating your own rancid module for ZTE with a customized inloop() function. you can probably use the parsing functions from the ios module, like the 'ciscoshtech' example that comes with rancid uses 2 modules. > On Wed, Jul 3, 2019 at 2:52 PM Piegorsch, Weylin William > wrote: > > > Hi Kevin, > > > > I think you said this is a ZTE device, but that you?re using -t cisco. is > > ZTE a cisco device? > > > > weylin > > > > > > > > *From: *Kevin Morales > > *Date: *Wednesday, July 3, 2019 at 3:18 PM > > *To: *john heasley > > *Cc: *Weylin Piegorsch , Nick Nauwelaerts < > > nick.nauwelaerts at aquafin.be>, "rancid-discuss at shrubbery.net" < > > rancid-discuss at shrubbery.net> > > *Subject: *Re: [rancid] Unable to figure out "end of run not found" > > > > > > > > I am sorry, I dont get you, What do you want I do? > > > > > > > > on my Rancid Server I execute: > > > > [rancid at localhost bin]$ NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 > > > > > > > > On Wed, Jul 3, 2019 at 12:43 PM john heasley wrote: > > > > Wed, Jul 03, 2019 at 11:33:08AM -0600, Kevin Morales: > > > Thanks Piegorsh, > > > > > > I did it.. > > > > > > NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 > > > > > > but in the two file 172.17.1.6.new and 172.17.1.6.raw don't see anything > > > about this error. both show the correct command output. > > > > correct command output and matching the criteria that i described below > > for type cisco are not necessarily the same thing. read it again. > > > > > On Wed, Jul 3, 2019 at 11:29 AM Piegorsch, Weylin William > > > > > wrote: > > > > > *172.17.1.6 : End of run not found* > > > > > 172.17.1.6: clean_run is false > > > > > 172.17.1.6: found_end is false > > > > > ! > > > > > > > > found end means that it found the end of the config; for type cisco, > > > > that means "^end". > > > > > > > > clean run means that it found the cli logout; for type cisco, that > > > > means "prompt[>#] exit$" From kevin.moralez at gmail.com Fri Jul 5 13:16:35 2019 From: kevin.moralez at gmail.com (Kevin Morales) Date: Fri, 5 Jul 2019 07:16:35 -0600 Subject: [rancid] Unable to figure out "end of run not found" In-Reply-To: <20190704001036.GB84573@shrubbery.net> References: <1d2fd209f746444a8175c9d70ff35fa7@aquafin.be> <20190703171210.GC19867@shrubbery.net> <5820A16C-B7E4-4914-98AE-4ABACD5415AD@bu.edu> <20190703184327.GC11812@shrubbery.net> <3E6063D0-2EE0-4A69-A886-0DC54DF88A0D@bu.edu> <20190704001036.GB84573@shrubbery.net> Message-ID: Thanks John! The configuration finish in some case with: ! ZXR10-01# $ ! ZXR10-02# $ ! ZXR10-03# and sorry, I don't have experience with programation.., Thanks! On Wed, Jul 3, 2019 at 6:10 PM john heasley wrote: > Wed, Jul 03, 2019 at 02:53:14PM -0600, Kevin Morales: > > Yes, my Router is ZTE and I am using CISCO type, because the command is > the > > same to see the configuration..show running-config > > I have no idea what ZTE is; does it behave *exactly* the same as IOS? > It seems not. > > > > > > found end means that it found the end of the config; for type > cisco, > > > > > that means "^end". > > Does it's config end with: > > " > end > "? > > > > > > clean run means that it found the cli logout; for type cisco, that > > > > > means "prompt[>#] exit$" > > in your .raw file, does the last prompt where clogin exited the cli, match > the regex > > "prompt[>#] exit$" > ? > > clearly these sanity checks are not working with your ZTE device. You > need to figure-out why and correct it, likely by creating your own > rancid module for ZTE with a customized inloop() function. you can > probably use the parsing functions from the ios module, like the > 'ciscoshtech' example that comes with rancid uses 2 modules. > > > On Wed, Jul 3, 2019 at 2:52 PM Piegorsch, Weylin William > > wrote: > > > > > Hi Kevin, > > > > > > I think you said this is a ZTE device, but that you?re using -t cisco. > is > > > ZTE a cisco device? > > > > > > weylin > > > > > > > > > > > > *From: *Kevin Morales > > > *Date: *Wednesday, July 3, 2019 at 3:18 PM > > > *To: *john heasley > > > *Cc: *Weylin Piegorsch , Nick Nauwelaerts < > > > nick.nauwelaerts at aquafin.be>, "rancid-discuss at shrubbery.net" < > > > rancid-discuss at shrubbery.net> > > > *Subject: *Re: [rancid] Unable to figure out "end of run not found" > > > > > > > > > > > > I am sorry, I dont get you, What do you want I do? > > > > > > > > > > > > on my Rancid Server I execute: > > > > > > [rancid at localhost bin]$ NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 > > > > > > > > > > > > On Wed, Jul 3, 2019 at 12:43 PM john heasley > wrote: > > > > > > Wed, Jul 03, 2019 at 11:33:08AM -0600, Kevin Morales: > > > > Thanks Piegorsh, > > > > > > > > I did it.. > > > > > > > > NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 > > > > > > > > but in the two file 172.17.1.6.new and 172.17.1.6.raw don't see > anything > > > > about this error. both show the correct command output. > > > > > > correct command output and matching the criteria that i described below > > > for type cisco are not necessarily the same thing. read it again. > > > > > > > On Wed, Jul 3, 2019 at 11:29 AM Piegorsch, Weylin William < > weylin at bu.edu > > > > > > > > wrote: > > > > > > *172.17.1.6 : End of run not found* > > > > > > 172.17.1.6: clean_run is false > > > > > > 172.17.1.6: found_end is false > > > > > > ! > > > > > > > > > > found end means that it found the end of the config; for type > cisco, > > > > > that means "^end". > > > > > > > > > > clean run means that it found the cli logout; for type cisco, that > > > > > means "prompt[>#] exit$" > -- *Kevin Morales* -------------- next part -------------- An HTML attachment was scrubbed... URL: From STUART.WALTON at QVC.COM Thu Jul 4 08:23:51 2019 From: STUART.WALTON at QVC.COM (STUART WALTON) Date: Thu, 4 Jul 2019 08:23:51 +0000 Subject: [rancid] Restore a Palo Alto Firewall from a Rancid bacup Message-ID: Hi Has anyone used a backup from Rancid to restore a Palo Alto Firewall? If so how have you done it? (I have the backup but it does not appear to be in the correct format) I have searched the discussion but cannot seem to find the answer. Any help would be appreciated. Regards Stu ---------------------------- This message (including any attachments) contains confidential information intended for a specific individual and purpose, and is protected by law. If you are not the intended recipient of this e-mail (even if the e-mail address above is yours), (i) you may not use, copy or retransmit it, (ii) please delete this message and (iii) please notify the sender immediately. Any disclosure, copying, or distribution of this message or the taking of any action based on it, is strictly prohibited. ---------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From heas at shrubbery.net Fri Jul 5 17:42:51 2019 From: heas at shrubbery.net (john heasley) Date: Fri, 5 Jul 2019 17:42:51 +0000 Subject: [rancid] Restore a Palo Alto Firewall from a Rancid bacup In-Reply-To: References: Message-ID: <20190705174251.GG55957@shrubbery.net> Thu, Jul 04, 2019 at 08:23:51AM +0000, STUART WALTON: > Hi > > Has anyone used a backup from Rancid to restore a Palo Alto Firewall? > > If so how have you done it? (I have the backup but it does not appear to be in the correct format) > > I have searched the discussion but cannot seem to find the answer. Any help would be appreciated. I do not know much of anything about PAN devices. However, be aware that, depending upon your rancid configuration, passwords may be removed. Also, see the FAQ S1 Q5 for another caveat that may apply to PAN. Also, include the error you received when attempting to load the config. It might provide clue to someone with more experience with PAN. From heas at shrubbery.net Mon Jul 8 19:55:11 2019 From: heas at shrubbery.net (john heasley) Date: Mon, 8 Jul 2019 19:55:11 +0000 Subject: [rancid] Unable to figure out "end of run not found" In-Reply-To: References: <20190703171210.GC19867@shrubbery.net> <5820A16C-B7E4-4914-98AE-4ABACD5415AD@bu.edu> <20190703184327.GC11812@shrubbery.net> <3E6063D0-2EE0-4A69-A886-0DC54DF88A0D@bu.edu> <20190704001036.GB84573@shrubbery.net> Message-ID: <20190708195511.GE60909@shrubbery.net> Fri, Jul 05, 2019 at 07:16:35AM -0600, Kevin Morales: > Thanks John! > > The configuration finish in some case with: > > ! > ZXR10-01# > > $ > ! > ZXR10-02# > > $ > ! > ZXR10-03# > > and sorry, I don't have experience with programation.., it would need to handle the check like exos.pm; by counting valid output. Maybe try just using that module with a private device type like: zte;script;rancid -t zte zte;login;xlogin zte;module;exos zte;inloop;exos::inloop zte;command;exos::ShowVersion;show version zte;command;exos::WriteTerm;show configuration > Thanks! > > On Wed, Jul 3, 2019 at 6:10 PM john heasley wrote: > > > Wed, Jul 03, 2019 at 02:53:14PM -0600, Kevin Morales: > > > Yes, my Router is ZTE and I am using CISCO type, because the command is > > the > > > same to see the configuration..show running-config > > > > I have no idea what ZTE is; does it behave *exactly* the same as IOS? > > It seems not. > > > > > > > > found end means that it found the end of the config; for type > > cisco, > > > > > > that means "^end". > > > > Does it's config end with: > > > > " > > end > > "? > > > > > > > > clean run means that it found the cli logout; for type cisco, that > > > > > > means "prompt[>#] exit$" > > > > in your .raw file, does the last prompt where clogin exited the cli, match > > the regex > > > > "prompt[>#] exit$" > > ? > > > > clearly these sanity checks are not working with your ZTE device. You > > need to figure-out why and correct it, likely by creating your own > > rancid module for ZTE with a customized inloop() function. you can > > probably use the parsing functions from the ios module, like the > > 'ciscoshtech' example that comes with rancid uses 2 modules. > > > > > On Wed, Jul 3, 2019 at 2:52 PM Piegorsch, Weylin William > > > wrote: > > > > > > > Hi Kevin, > > > > > > > > I think you said this is a ZTE device, but that you?re using -t cisco. > > is > > > > ZTE a cisco device? > > > > > > > > weylin > > > > > > > > > > > > > > > > *From: *Kevin Morales > > > > *Date: *Wednesday, July 3, 2019 at 3:18 PM > > > > *To: *john heasley > > > > *Cc: *Weylin Piegorsch , Nick Nauwelaerts < > > > > nick.nauwelaerts at aquafin.be>, "rancid-discuss at shrubbery.net" < > > > > rancid-discuss at shrubbery.net> > > > > *Subject: *Re: [rancid] Unable to figure out "end of run not found" > > > > > > > > > > > > > > > > I am sorry, I dont get you, What do you want I do? > > > > > > > > > > > > > > > > on my Rancid Server I execute: > > > > > > > > [rancid at localhost bin]$ NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 > > > > > > > > > > > > > > > > On Wed, Jul 3, 2019 at 12:43 PM john heasley > > wrote: > > > > > > > > Wed, Jul 03, 2019 at 11:33:08AM -0600, Kevin Morales: > > > > > Thanks Piegorsh, > > > > > > > > > > I did it.. > > > > > > > > > > NOPIPE=yes ./rancid -d -t cisco 172.17.1.6 > > > > > > > > > > but in the two file 172.17.1.6.new and 172.17.1.6.raw don't see > > anything > > > > > about this error. both show the correct command output. > > > > > > > > correct command output and matching the criteria that i described below > > > > for type cisco are not necessarily the same thing. read it again. > > > > > > > > > On Wed, Jul 3, 2019 at 11:29 AM Piegorsch, Weylin William < > > weylin at bu.edu > > > > > > > > > > wrote: > > > > > > > *172.17.1.6 : End of run not found* > > > > > > > 172.17.1.6: clean_run is false > > > > > > > 172.17.1.6: found_end is false > > > > > > > ! > > > > > > > > > > > > found end means that it found the end of the config; for type > > cisco, > > > > > > that means "^end". > > > > > > > > > > > > clean run means that it found the cli logout; for type cisco, that > > > > > > means "prompt[>#] exit$" > > > > > -- > *Kevin Morales* From heas at shrubbery.net Mon Jul 8 20:12:11 2019 From: heas at shrubbery.net ('john heasley') Date: Mon, 8 Jul 2019 20:12:11 +0000 Subject: [rancid] Watchguard xml file In-Reply-To: References: <20190702234813.GA21015@shrubbery.net> <20190703174048.GD19867@shrubbery.net> Message-ID: <20190708201211.GF60909@shrubbery.net> Wed, Jul 03, 2019 at 06:49:20PM +0000, Wayne Eisenberg: > -----Original Message----- > From: 'john heasley' > Sent: Wednesday, July 03, 2019 1:41 PM > To: Wayne Eisenberg > Cc: 'john heasley' ; 'rancid-discuss at shrubbery.net' > Subject: Re: [rancid] Watchguard xml file > > > >> However, in the xtm.pm module, line 102 defines it again. > > >i'm not familiar with this device, but redefining (or refining) the prompt is normal. the filter functions and login scripts begin with something loose, and once it sees the prompt, it can be refined to be more precise, and >may later further refine it (eg: in run_commands) to match the prompt when/if it changes in config or other modes that are platform dependent. > > Ah, if I only had that skill. > > >> ----------- > >> while (/\s*($cmds_regexp)\s*$/) { > >> $cmd = $1; > >> $prompt = ">>"; > ^^^^^^^^^^^^ this is probably a mistake; should be part of the while() regex. I suspect it might be here because the author could not make the regex below match correctly. > > >> if (!defined($prompt)) { > >> $prompt = ($_ =~ /^([^>]+>)/)[0]; > >> $prompt =~ s/([][}{)(\\])/\\$1/g; > >> print STDERR ("PROMPT MATCH: $prompt\n") if ($debug); > >> } > >> ----------- > >> Once you get to the sub ShowConfiguration section, on line 199 if it sees the prompt, end. Guess what? The "#" character is inside the config (there is some html code in one of the xml sections) and that is where the config ends. > > >seems that the prompt is ">>". > > Yes, in this example. I wanted to show the original file, not something that I modded. In my current version, the line is > $prompt = ">>|#" > which works, but causes the problem of the config getting truncated because it sees "#" as the prompt. The $prompt should either be the entire thing or some string that ends in #. yes, this is why it refines the prompt match to be the complete thing, but it has to see one before it can extract it. and your inloop set is at the top of the loop, so it never refines it to be the whole prompt. > >> ----------- > >> sub ShowConfiguration { > >> my($INPUT, $OUTPUT, $cmd) = @_; > >> my($lines) = 0; > >> my($snmp) = 0; > >> print STDERR " In ShowConfiguration: $_" if ($debug); > >> # We don't care about password filtering as passwords are hashed > >> # So don't use this if you need it (or develop the functionality). > >> if ($filter_pwds >= 1){ > >> print STDERR "WARNING: Password filtering isn't implemented yet!\n"; > >> print STDERR "Either disable password filtering in rancid.conf"; > >> print STDERR " or don't use this plugin.\n"; > >> } > >> s/^[a-z]+@//; > >> ProcessHistory("","","","# $_"); > >> while (<$INPUT>) { > >> tr/\015//d; > >> next if (/^\s*$/); > >> # end of config - hopefully. > >> # end-of-config tag. appears to end with "\nPROMPT:~$". > >> if (/$prompt/) { > >> $found_end++; > >> last; > >> } > >> ----------- > >> > >> So I'm thinking if I can figure out a different way to define the prompt to be more than just the # sign (at least in the xtm.pm), that should do the trick? Can you do something like $prompt = "#$" ? it has to be as a set (regex or glob), like; [#$]. but that is a single atom; if your prompt is or may be ">>", then you likely need to use a group atom, like (>>|#). > >its better to anchor it and have it be as complete as reasonable. eg: > >not # > >not hostname# > >but ^hostname# > > >look at ios.pm. > > Looking, but I don't see anywhere that it defines the prompt. It uses it a lot, but doesn't define it. its starts with [>#] in the while() (and exit match); then refines it to be a match the entire prompt with regex atoms escaped in the if(!defined($prompt)). after that, it anchors the prompt match when appropiate; /^$prompt/. you should do similarly for this watchguard device. I suspect that you can just steal the ios.pm inloop() and modify the initial prompt matching. It could be kinkier, but it is a good starting point. i think i;ve answered everything. From Chris.Davis at principia.edu Tue Jul 9 21:55:56 2019 From: Chris.Davis at principia.edu (Chris Davis) Date: Tue, 9 Jul 2019 21:55:56 +0000 Subject: [rancid] Extreme switch policy backup. Message-ID: We've just gotten a few Extreme switches (model X440-G2) and I've gotten them set up in Rancid. But while I get the configs, I have a few policies as well. They're kept as .pol files on the switch. Is there a way to include the policy files in the backup that Rancid takes? It would be particularly helpful. I've done some searching, and seen folks ask about it. But no real answers. Lots of modifications to commands from 4 years ago but nothing current. There's a command that will print it all out, just not sure how to add it into the mix. Don't like to modify something like Rancid if there's already a way within the system to make it happen. Thanks. Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From lsy.annie at gmail.com Tue Jul 9 22:02:18 2019 From: lsy.annie at gmail.com (annie lee) Date: Wed, 10 Jul 2019 08:02:18 +1000 Subject: [rancid] v3.9 ignoring timestamp and minor filesize in IOS/XE Message-ID: I know this question has been asked a couple of times, and i found a thread providing a workaround to modify the code in ios.pm but not in details. Anyone can share what the line should be before and after ? Before : if (/(\d+) bytes (available|total) \((\d+) bytes used\)/) { After : if NN [GMK]B free ?? Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From lsy.annie at gmail.com Wed Jul 10 01:53:42 2019 From: lsy.annie at gmail.com (annie lee) Date: Wed, 10 Jul 2019 11:53:42 +1000 Subject: [rancid] Palo Alto (Panorama) configuration Message-ID: Hi All, Another question, just added a new PaloAlto to rancid (3.9) but not much configurations being backup (not even interfaces addresses) Anything need to be changed/added to backup the entire configuration ? 1.1.1.1;palo-alto;up Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From danm at prime.gushi.org Wed Jul 10 08:39:34 2019 From: danm at prime.gushi.org (Dan Mahoney (Gushi)) Date: Wed, 10 Jul 2019 01:39:34 -0700 (PDT) Subject: [rancid] Getting a lot of noise related to ce_switch.log and ce_switch.log.bak In-Reply-To: <20180911163631.GB2325@shrubbery.net> References: <20180911163631.GB2325@shrubbery.net> Message-ID: On Tue, 11 Sep 2018, heasley wrote: > Mon, Sep 10, 2018 at 01:45:42AM -0700, Dan Mahoney (Gushi): >> Hey all, >> >> I'm running Rancid built from freebsd packages, rancid3-3.7 >> >> Periodically, my ASR9K's log something like this: >> >> !Flash: harddisk: 24753 -rwx 800470016 Wed Sep 10 20:00:00 2014 >> VM-ASR9K-px-4.3.4.tar >> - !Flash: harddisk: 24623 -rw- >> ce_switch.log >> + !Flash: harddisk: 24781 -rw- 8192017 Mon Sep 10 05:10:03 2018 >> ce_switch.log.bak >> !Flash: harddisk: 24688 -rw- 1048576 Thu Sep 11 02:08:46 2014 >> kd.bin_0_RSP0_CPU0 >> !Flash: harddisk: 24625 drwx 4096 Thu Sep 11 01:38:55 2014 >> idiags >> !Flash: harddisk: 24626 -rw- 0 Thu Sep 11 01:40:24 2014 >> ahci.log >> !Flash: harddisk: 24627 drwx 4096 Thu Sep 11 02:20:32 2014 >> np >> - !Flash: harddisk: 24783 -rw- 8192017 Fri Sep 7 08:18:57 2018 >> ce_switch.log.bak >> + !Flash: harddisk: 24628 -rw- >> ce_switch.log >> !Flash: harddisk: 6442434560 bytes total (4 GB free) >> >> I thought I saw something on the mailing lists that this was fixed in a >> prior version, but I guess not. How would I go about tweaking rancid so >> these bits are ignored? > > add a filter to DirSlotN(). i see that your device is renaming files, > causing the fileno to change. I'll add that filter for 3.9. Sorry to revive an old thread. I've upgraded to 3.9, but this doesn't seem to have been fixed: @@ -382,14 +382,14 @@ !Flash: harddisk: 25131 drwx 4096 Mon Jun 25 21:00:51 2012 showtech-temp !Flash: harddisk: 25248 -rwx 420587520 Thu Sep 11 15:23:40 2014 custom-smus-jtl-2014-sept.tar !Flash: harddisk: 25249 -rwx 452402176 Thu Sep 11 15:28:01 2014 ASR9K-px-4.3.4.tar !Flash: harddisk: 25250 -rwx 800470016 Thu Sep 11 15:36:25 2014 VM-ASR9K-px-4.3.4.tar - !Flash: harddisk: 25268 -rw- 8192017 Sat Jul 6 08:07:41 2019 ce_switch.log.bak + !Flash: harddisk: 25135 -rw- ce_switch.log !Flash: harddisk: 25252 -rw- 1048576 Tue Mar 5 00:26:44 2019 kd.bin_0_RSP0_CPU0 !Flash: harddisk: 25137 drwx 4096 Fri Sep 12 01:33:47 2014 idiags !Flash: harddisk: 25138 -rw- 0 Fri Sep 12 01:35:16 2014 ahci.log !Flash: harddisk: 25139 drwx 4096 Thu Feb 28 20:12:29 2019 np - !Flash: harddisk: 25140 -rw- ce_switch.log + !Flash: harddisk: 25269 -rw- 8192017 Tue Jul 9 04:57:56 2019 ce_switch.log.bak !Flash: harddisk: 25141 -rwx 63 Sun Feb 24 22:37:37 2019 chkfs_repair.log !Flash: harddisk: 25266 -rw- 40 Mon Feb 25 05:39:35 2019 uptime_static_data !Flash: harddisk: 25143 -rw- -Dan -- --------Dan Mahoney-------- Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC FB: fb.com/DanielMahoneyIV LI: linkedin.com/in/gushi Site: http://www.gushi.org --------------------------- From m_zouhairy at skno.by Wed Jul 10 08:59:26 2019 From: m_zouhairy at skno.by (Vacheslav Zouhairy) Date: Wed, 10 Jul 2019 11:59:26 +0300 Subject: [rancid] Getting a lot of noise related to ce_switch.log and ce_switch.log.bak In-Reply-To: References: <20180911163631.GB2325@shrubbery.net> Message-ID: <6918a10adcb579634312f4a933823f3c0f7b8325.camel@skno.by> and this device is not being restarted daily? On Wed, 2019-07-10 at 01:39 -0700, Dan Mahoney (Gushi) wrote: > On Tue, 11 Sep 2018, heasley wrote: > > > Mon, Sep 10, 2018 at 01:45:42AM -0700, Dan Mahoney (Gushi): > > > Hey all, > > > > > > I'm running Rancid built from freebsd packages, rancid3-3.7 > > > > > > Periodically, my ASR9K's log something like this: > > > > > > !Flash: harddisk: 24753 -rwx 800470016 Wed Sep 10 > > > 20:00:00 2014 > > > VM-ASR9K-px-4.3.4.tar > > > - !Flash: harddisk: 24623 -rw- > > > ce_switch.log > > > + !Flash: harddisk: 24781 -rw- 8192017 Mon Sep 10 > > > 05:10:03 2018 > > > ce_switch.log.bak > > > !Flash: harddisk: 24688 -rw- 1048576 Thu Sep 11 > > > 02:08:46 2014 > > > kd.bin_0_RSP0_CPU0 > > > !Flash: harddisk: 24625 drwx 4096 Thu Sep 11 > > > 01:38:55 2014 > > > idiags > > > !Flash: harddisk: 24626 -rw- 0 Thu Sep 11 > > > 01:40:24 2014 > > > ahci.log > > > !Flash: harddisk: 24627 drwx 4096 Thu Sep 11 > > > 02:20:32 2014 > > > np > > > - !Flash: harddisk: 24783 -rw- 8192017 Fri Sep 7 > > > 08:18:57 2018 > > > ce_switch.log.bak > > > + !Flash: harddisk: 24628 -rw- > > > ce_switch.log > > > !Flash: harddisk: 6442434560 bytes total (4 GB free) > > > > > > I thought I saw something on the mailing lists that this was > > > fixed in a > > > prior version, but I guess not. How would I go about tweaking > > > rancid so > > > these bits are ignored? > > > > add a filter to DirSlotN(). i see that your device is renaming > > files, > > causing the fileno to change. I'll add that filter for 3.9. > > Sorry to revive an old thread. > > I've upgraded to 3.9, but this doesn't seem to have been fixed: > > @@ -382,14 +382,14 @@ > !Flash: harddisk: 25131 drwx 4096 Mon Jun 25 21:00:51 > 2012 > showtech-temp > !Flash: harddisk: 25248 -rwx 420587520 Thu Sep 11 > 15:23:40 2014 > custom-smus-jtl-2014-sept.tar > !Flash: harddisk: 25249 -rwx 452402176 Thu Sep 11 15:28:01 > 2014 > ASR9K-px-4.3.4.tar > !Flash: harddisk: 25250 -rwx 800470016 Thu Sep 11 > 15:36:25 2014 > VM-ASR9K-px-4.3.4.tar > - !Flash: harddisk: 25268 -rw- 8192017 Sat Jul 6 08:07:41 > 2019 > ce_switch.log.bak > + !Flash: harddisk: 25135 -rw- > ce_switch.log > !Flash: harddisk: 25252 -rw- 1048576 Tue Mar 5 00:26:44 > 2019 > kd.bin_0_RSP0_CPU0 > !Flash: harddisk: 25137 drwx 4096 Fri Sep 12 > 01:33:47 2014 > idiags > !Flash: harddisk: 25138 -rw- 0 Fri Sep 12 01:35:16 > 2014 > ahci.log > !Flash: harddisk: 25139 drwx 4096 Thu Feb 28 > 20:12:29 2019 > np > - !Flash: harddisk: 25140 -rw- > ce_switch.log > + !Flash: harddisk: 25269 -rw- 8192017 Tue Jul 9 04:57:56 > 2019 > ce_switch.log.bak > !Flash: harddisk: 25141 -rwx 63 Sun Feb 24 > 22:37:37 2019 > chkfs_repair.log > !Flash: harddisk: 25266 -rw- 40 Mon Feb 25 > 05:39:35 2019 > uptime_static_data > !Flash: harddisk: 25143 -rw- > > -Dan > From heas at shrubbery.net Wed Jul 10 21:42:09 2019 From: heas at shrubbery.net (john heasley) Date: Wed, 10 Jul 2019 21:42:09 +0000 Subject: [rancid] Palo Alto (Panorama) configuration In-Reply-To: References: Message-ID: <20190710214209.GD36475@shrubbery.net> Wed, Jul 10, 2019 at 11:53:42AM +1000, annie lee: > Hi All, > > Another question, just added a new PaloAlto to rancid (3.9) but not much > configurations being backup (not even interfaces addresses) > Anything need to be changed/added to backup the entire configuration ? > > 1.1.1.1;palo-alto;up Please use the built-in type for PAN: paloalto. if that is still lacking, please be more specific about what commands are missing. it collects show system info;show chassis inventory;show config running From lsy.annie at gmail.com Wed Jul 10 23:23:20 2019 From: lsy.annie at gmail.com (annie lee) Date: Thu, 11 Jul 2019 09:23:20 +1000 Subject: [rancid] Palo Alto (Panorama) configuration In-Reply-To: <20190710214209.GD36475@shrubbery.net> References: <20190710214209.GD36475@shrubbery.net> Message-ID: Hi John, Thanks for your reply and apology for the typo on the paloalto type. (1.1.1.1;paloalto;up) Below are the sample config for one of the firewall configs (removed all the ip addresses). Basically there are heaps more configs (routing, policy, NAT, virtual router and etc...) i can see from the Panorama. Not sure its similar to F5 tweak that we need to add the partition to grab the full configs. Rgds On Thu, Jul 11, 2019 at 7:42 AM john heasley wrote: > Wed, Jul 10, 2019 at 11:53:42AM +1000, annie lee: > > Hi All, > > > > Another question, just added a new PaloAlto to rancid (3.9) but not much > > configurations being backup (not even interfaces addresses) > > Anything need to be changed/added to backup the entire configuration ? > > > > 1.1.1.1;palo-alto;up > > Please use the built-in type for PAN: paloalto. if that is still lacking, > please be more specific about what commands are missing. it collects > > show system info;show chassis inventory;show config running > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- !RANCID-CONTENT-TYPE: paloalto ! # #hostname: palo-fw01 #ip-address: 1.1.1.1 #public-ip-address: unknown #netmask: 255.255.255.0 #default-gateway: 1.1.1.254 #ip-assignment: static #ipv6-address: unknown #ipv6-link-local-address: #ipv6-default-gateway: #mac-address: #family: 3000 #model: PA-3055 #serial: #cloud-mode: non-cloud #sw-version: 8.1.6 #global-protect-client-package-version: 5.0.1 #url-db: paloaltonetworks #global-protect-clientless-vpn-version: 0 #global-protect-clientless-vpn-release-date: #logdb-version: 8.1.8 #platform-family: 3000 #vpn-disable-mode: off #multi-vsys: off #operational-mode: normal # # # config { mgt-config { users; } shared { application; application-group; service; service-group; botnet { configuration { http { dynamic-dns { enabled yes; threshold 5; } malware-sites { enabled yes; threshold 5; } recent-domains { enabled yes; threshold 5; } ip-domains { enabled yes; threshold 10; } executables-from-unknown-sites { enabled yes; threshold 5; } } other-applications { irc yes; } unknown-applications { unknown-tcp { destinations-per-hour 10; sessions-per-hour 10; session-length { maximum-bytes 100; minimum-bytes 50; } } unknown-udp { destinations-per-hour 10; sessions-per-hour 10; session-length { maximum-bytes 100; minimum-bytes 50; } } } } report { topn 100; scheduled yes; } } authentication-profile; local-user-database { user; } server-profile { ldap; } authentication-sequence; content-preview { application-type { technology; category; } application; } } devices { localhost.localdomain { network { interface { ethernet; loopback { units; } vlan { units; } tunnel { units; } } vlan; virtual-wire; profiles { monitor-profile { default { interval 3; threshold 5; action wait-recover; } } } ike { crypto-profiles { ike-crypto-profiles { Suite-B-GCM-256 { encryption aes-256-cbc; hash sha384; dh-group group20; lifetime { hours 8; } } } ipsec-crypto-profiles { Suite-B-GCM-128 { esp { encryption aes-128-gcm; authentication none; } dh-group group19; lifetime { hours 1; } } Suite-B-GCM-256 { esp { encryption aes-256-gcm; authentication none; } dh-group group20; lifetime { hours 1; } } } global-protect-app-crypto-profiles { default { encryption aes-128-cbc; authentication sha1; } } } gateway; } qos { profile { default { class { class1 { priority real-time; } class2 { priority high; } class3 { priority high; } class4 { priority medium; } class5 { priority medium; } class6 { priority low; } class7 { priority low; } class8 { priority low; } } } } } virtual-router { default { protocol { bgp { enable no; dampening-profile { default { cutoff 1.25; reuse 0.5; max-hold-time 900; decay-half-life-reachable 300; decay-half-life-unreachable 900; enable yes; } } } } } } tunnel { ipsec; global-protect-gateway; global-protect-site-to-site; } } deviceconfig { system { ip-address 172.1.0.9; netmask 255.255.255.0; update-server updates.paloaltonetworks.com; service { disable-telnet yes; disable-http yes; } default-gateway 172.1.0.1; panorama-server pan.fw.int; hostname m1-edge-pa01; } setting { config { rematch yes; } management { hostname-type-in-syslog FQDN; } } high-availability { interface { ha1 { ip-address 192.168.0.7; netmask 255.255.255.252; link-speed auto; link-duplex auto; } ha2 { link-speed auto; link-duplex auto; } ha1-backup { port ethernet1/11; ip-address 192.168.0.3; netmask 255.255.255.252; } ha2-backup { port ethernet1/12; ip-address 192.168.1.3; netmask 255.255.255.252; } } group { group-id 5; description palo-fw; peer-ip 1.1.1.88; monitoring { path-monitoring { enabled no; } link-monitoring { enabled no; } } configuration-synchronization { enabled yes; } mode { active-passive { passive-link-state auto; } } peer-ip-backup 1.1.1.8; election-option { heartbeat-backup yes; timers { recommended; } } } enabled yes; } } vsys { vsys1 { application; application-group; zone { trust { network { virtual-wire; } } untrust { network { virtual-wire; } } } service; service-group; schedule; rulebase { security { rules; } } group-mapping; import { network { interface; } } global-protect { global-protect-portal; } } } } } From lsy.annie at gmail.com Thu Jul 11 01:01:42 2019 From: lsy.annie at gmail.com (annie lee) Date: Thu, 11 Jul 2019 11:01:42 +1000 Subject: [rancid] Palo Alto (Panorama) configuration In-Reply-To: References: <20190710214209.GD36475@shrubbery.net> Message-ID: i tried to grab the configs from the panorama and it's what i wanted :-) apology, im pretty new to the paloalto and panorama device/setup. thanks and glad i can backup the palo/panorama configs without any tweaking. On Thu, Jul 11, 2019 at 9:23 AM annie lee wrote: > Hi John, > > Thanks for your reply and apology for the typo on the paloalto type. > (1.1.1.1;paloalto;up) > Below are the sample config for one of the firewall configs (removed all > the ip addresses). > Basically there are heaps more configs (routing, policy, NAT, virtual > router and etc...) i can see from the Panorama. > Not sure its similar to F5 tweak that we need to add the partition to grab > the full configs. > > Rgds > > On Thu, Jul 11, 2019 at 7:42 AM john heasley wrote: > >> Wed, Jul 10, 2019 at 11:53:42AM +1000, annie lee: >> > Hi All, >> > >> > Another question, just added a new PaloAlto to rancid (3.9) but not much >> > configurations being backup (not even interfaces addresses) >> > Anything need to be changed/added to backup the entire configuration ? >> > >> > 1.1.1.1;palo-alto;up >> >> Please use the built-in type for PAN: paloalto. if that is still lacking, >> please be more specific about what commands are missing. it collects >> >> show system info;show chassis inventory;show config running >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgauthier at comscore.com Thu Jul 11 14:19:00 2019 From: cgauthier at comscore.com (Gauthier, Chris) Date: Thu, 11 Jul 2019 14:19:00 +0000 Subject: [rancid] Palo Alto (Panorama) configuration In-Reply-To: References: <20190710214209.GD36475@shrubbery.net> Message-ID: I have run into the issues seen below, as we migrated to a fully-managed Panorama ecosystem in recent months. The output of the ?show configuration running? (or whatever it is) is more limited on the managed device because (I believe) what is being shown is only the locally-managed configuration. I haven?t looked yet to see if there is a workaround. --Chris Chris Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. From: Rancid-discuss on behalf of annie lee Date: Wednesday, July 10, 2019 at 6:02 PM To: john heasley Cc: "rancid-discuss at shrubbery.net" Subject: Re: [rancid] Palo Alto (Panorama) configuration i tried to grab the configs from the panorama and it's what i wanted :-) apology, im pretty new to the paloalto and panorama device/setup. thanks and glad i can backup the palo/panorama configs without any tweaking. On Thu, Jul 11, 2019 at 9:23 AM annie lee > wrote: Hi John, Thanks for your reply and apology for the typo on the paloalto type. (1.1.1.1;paloalto;up) Below are the sample config for one of the firewall configs (removed all the ip addresses). Basically there are heaps more configs (routing, policy, NAT, virtual router and etc...) i can see from the Panorama. Not sure its similar to F5 tweak that we need to add the partition to grab the full configs. Rgds On Thu, Jul 11, 2019 at 7:42 AM john heasley > wrote: Wed, Jul 10, 2019 at 11:53:42AM +1000, annie lee: > Hi All, > > Another question, just added a new PaloAlto to rancid (3.9) but not much > configurations being backup (not even interfaces addresses) > Anything need to be changed/added to backup the entire configuration ? > > 1.1.1.1;palo-alto;up Please use the built-in type for PAN: paloalto. if that is still lacking, please be more specific about what commands are missing. it collects show system info;show chassis inventory;show config running -------------- next part -------------- An HTML attachment was scrubbed... URL: From cra at wpi.edu Thu Jul 11 14:37:51 2019 From: cra at wpi.edu (Anderson, Charles R) Date: Thu, 11 Jul 2019 14:37:51 +0000 Subject: [rancid] Palo Alto (Panorama) configuration In-Reply-To: References: <20190710214209.GD36475@shrubbery.net> Message-ID: <20190711143748.urpalsoan7pyq6nx@angus.ind.wpi.edu> You can use "show config merged" to see the local device's config merged with the templates from Panorama. On Thu, Jul 11, 2019 at 02:19:00PM +0000, Gauthier, Chris wrote: > I have run into the issues seen below, as we migrated to a fully-managed Panorama ecosystem in recent months. The output of the ?show configuration running? (or whatever it is) is more limited on the managed device because (I believe) what is being shown is only the locally-managed configuration. I haven?t looked yet to see if there is a workaround. > > --Chris > > > Chris Gauthier Senior Network Engineer | Comscore > t +1 (503) 331-2704 | > cgauthier at comscore.com > comscore.com > ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. > From: Rancid-discuss on behalf of annie lee > Date: Wednesday, July 10, 2019 at 6:02 PM > To: john heasley > Cc: "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Palo Alto (Panorama) configuration > > i tried to grab the configs from the panorama and it's what i wanted :-) > apology, im pretty new to the paloalto and panorama device/setup. > > thanks and glad i can backup the palo/panorama configs without any tweaking. > > On Thu, Jul 11, 2019 at 9:23 AM annie lee > wrote: > Hi John, > > Thanks for your reply and apology for the typo on the paloalto type. (1.1.1.1;paloalto;up) > Below are the sample config for one of the firewall configs (removed all the ip addresses). > Basically there are heaps more configs (routing, policy, NAT, virtual router and etc...) i can see from the Panorama. > Not sure its similar to F5 tweak that we need to add the partition to grab the full configs. > > Rgds > > On Thu, Jul 11, 2019 at 7:42 AM john heasley > wrote: > Wed, Jul 10, 2019 at 11:53:42AM +1000, annie lee: > > Hi All, > > > > Another question, just added a new PaloAlto to rancid (3.9) but not much > > configurations being backup (not even interfaces addresses) > > Anything need to be changed/added to backup the entire configuration ? > > > > 1.1.1.1;palo-alto;up > > Please use the built-in type for PAN: paloalto. if that is still lacking, > please be more specific about what commands are missing. it collects > > show system info;show chassis inventory;show config running From heas at shrubbery.net Thu Jul 11 15:15:12 2019 From: heas at shrubbery.net (john heasley) Date: Thu, 11 Jul 2019 15:15:12 +0000 Subject: [rancid] Palo Alto (Panorama) configuration In-Reply-To: References: <20190710214209.GD36475@shrubbery.net> Message-ID: <20190711151512.GA4422@shrubbery.net> Thu, Jul 11, 2019 at 02:19:00PM +0000, Gauthier, Chris: > I have run into the issues seen below, as we migrated to a fully-managed Panorama ecosystem in recent months. The output of the ?show configuration running? (or whatever it is) is more limited on the managed device because (I believe) what is being shown is only the locally-managed configuration. I haven?t looked yet to see if there is a workaround. > > --Chris I have no experience with these. If more commands are necessary, lmk. > Chris Gauthier Senior Network Engineer | Comscore > t +1 (503) 331-2704 | > cgauthier at comscore.com > comscore.com > ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. > From: Rancid-discuss on behalf of annie lee > Date: Wednesday, July 10, 2019 at 6:02 PM > To: john heasley > Cc: "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Palo Alto (Panorama) configuration > > i tried to grab the configs from the panorama and it's what i wanted :-) > apology, im pretty new to the paloalto and panorama device/setup. > > thanks and glad i can backup the palo/panorama configs without any tweaking. > > On Thu, Jul 11, 2019 at 9:23 AM annie lee > wrote: > Hi John, > > Thanks for your reply and apology for the typo on the paloalto type. (1.1.1.1;paloalto;up) > Below are the sample config for one of the firewall configs (removed all the ip addresses). > Basically there are heaps more configs (routing, policy, NAT, virtual router and etc...) i can see from the Panorama. > Not sure its similar to F5 tweak that we need to add the partition to grab the full configs. > > Rgds > > On Thu, Jul 11, 2019 at 7:42 AM john heasley > wrote: > Wed, Jul 10, 2019 at 11:53:42AM +1000, annie lee: > > Hi All, > > > > Another question, just added a new PaloAlto to rancid (3.9) but not much > > configurations being backup (not even interfaces addresses) > > Anything need to be changed/added to backup the entire configuration ? > > > > 1.1.1.1;palo-alto;up > > Please use the built-in type for PAN: paloalto. if that is still lacking, > please be more specific about what commands are missing. it collects > > show system info;show chassis inventory;show config running From heas at shrubbery.net Thu Jul 11 15:16:51 2019 From: heas at shrubbery.net (john heasley) Date: Thu, 11 Jul 2019 15:16:51 +0000 Subject: [rancid] Palo Alto (Panorama) configuration In-Reply-To: <20190711143748.urpalsoan7pyq6nx@angus.ind.wpi.edu> References: <20190710214209.GD36475@shrubbery.net> <20190711143748.urpalsoan7pyq6nx@angus.ind.wpi.edu> Message-ID: <20190711151651.GB4422@shrubbery.net> Thu, Jul 11, 2019 at 02:37:51PM +0000, Anderson, Charles R: > You can use "show config merged" to see the local device's config merged with the templates from Panorama. Does this work with "non-managed" (better term?) configs? And, was this command introduced recently? From cgauthier at comscore.com Thu Jul 11 18:16:10 2019 From: cgauthier at comscore.com (Gauthier, Chris) Date: Thu, 11 Jul 2019 18:16:10 +0000 Subject: [rancid] Palo Alto (Panorama) configuration In-Reply-To: <20190711151651.GB4422@shrubbery.net> References: <20190710214209.GD36475@shrubbery.net> <20190711143748.urpalsoan7pyq6nx@angus.ind.wpi.edu> <20190711151651.GB4422@shrubbery.net> Message-ID: Yes, the command "show config merged" gives the locally-managed config output AND the configuration that is pushed out by Panorama. I'll make a custom device type and see how this works in my environment. If it works, I'll post the results here. I will also test with a non-Panorama-managed system. --Chris ? Chris Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. -----Original Message----- From: Rancid-discuss on behalf of john heasley Date: Thursday, July 11, 2019 at 8:17 AM To: "Anderson, Charles R" Cc: "rancid-discuss at shrubbery.net" Subject: Re: [rancid] Palo Alto (Panorama) configuration Thu, Jul 11, 2019 at 02:37:51PM +0000, Anderson, Charles R: > You can use "show config merged" to see the local device's config merged with the templates from Panorama. Does this work with "non-managed" (better term?) configs? And, was this command introduced recently? _______________________________________________ Rancid-discuss mailing list Rancid-discuss at shrubbery.net https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fwww.shrubbery.net%2fmailman%2flistinfo%2francid-discuss&c=E,1,ZBO_SpPdPN9F0GTa50thF3JK2iNVO_jcwwSZwho1q8BVBoP9LydezSjLupULi9-PCcBbEWhWi1x-kRvg-KGqTG6CANfUm1cA6XPL5VPANHGtvC7Gc3N4Pg4SarAO&typo=1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgauthier at comscore.com Thu Jul 11 18:19:45 2019 From: cgauthier at comscore.com (Gauthier, Chris) Date: Thu, 11 Jul 2019 18:19:45 +0000 Subject: [rancid] Palo Alto (Panorama) configuration In-Reply-To: References: <20190710214209.GD36475@shrubbery.net> <20190711143748.urpalsoan7pyq6nx@angus.ind.wpi.edu> <20190711151651.GB4422@shrubbery.net> Message-ID: <19611A45-0DE9-4E51-903F-7DFAE7040C40@comscore.com> Just validated the ?show config merged? command works with any PA firewall, managed by Panorama or not. Chris Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. From: Rancid-discuss on behalf of "Gauthier, Chris" Date: Thursday, July 11, 2019 at 11:16 AM To: john heasley , "Anderson, Charles R" Cc: "rancid-discuss at shrubbery.net" Subject: Re: [rancid] Palo Alto (Panorama) configuration Yes, the command "show config merged" gives the locally-managed config output AND the configuration that is pushed out by Panorama. I'll make a custom device type and see how this works in my environment. If it works, I'll post the results here. I will also test with a non-Panorama-managed system. --Chris Chris? Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. -----Original Message----- From: Rancid-discuss on behalf of john heasley Date: Thursday, July 11, 2019 at 8:17 AM To: "Anderson, Charles R" Cc: "rancid-discuss at shrubbery.net" Subject: Re: [rancid] Palo Alto (Panorama) configuration Thu, Jul 11, 2019 at 02:37:51PM +0000, Anderson, Charles R: > You can use "show config merged" to see the local device's config merged with the templates from Panorama. Does this work with "non-managed" (better term?) configs? And, was this command introduced recently? _______________________________________________ Rancid-discuss mailing list Rancid-discuss at shrubbery.net https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fwww.shrubbery.net%2fmailman%2flistinfo%2francid-discuss&c=E,1,ZBO_SpPdPN9F0GTa50thF3JK2iNVO_jcwwSZwho1q8BVBoP9LydezSjLupULi9-PCcBbEWhWi1x-kRvg-KGqTG6CANfUm1cA6XPL5VPANHGtvC7Gc3N4Pg4SarAO&typo=1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From lsy.annie at gmail.com Thu Jul 11 21:43:05 2019 From: lsy.annie at gmail.com (annie lee) Date: Fri, 12 Jul 2019 07:43:05 +1000 Subject: [rancid] Palo Alto (Panorama) configuration In-Reply-To: <19611A45-0DE9-4E51-903F-7DFAE7040C40@comscore.com> References: <20190710214209.GD36475@shrubbery.net> <20190711143748.urpalsoan7pyq6nx@angus.ind.wpi.edu> <20190711151651.GB4422@shrubbery.net> <19611A45-0DE9-4E51-903F-7DFAE7040C40@comscore.com> Message-ID: Thats good to know on the new cli (show config merged will grab everything from the firewall and panorama). How do we add the cli and diff to rancid ?? On Fri, Jul 12, 2019 at 4:20 AM Gauthier, Chris wrote: > Just validated the ?show config merged? command works with any PA > firewall, managed by Panorama or not. > > > Chris? Gauthier Senior Network Engineer | Comscore > t +1 *(503) 331-2704* <(503)%20331-2704> | > *cgauthier at comscore.com* > *comscore.com* > ???This e-mail (including any attachments) may contain information that is > private, confidential, or protected by attorney-client or other privilege. > If you received this e-mail in error, please delete it from your system and > notify sender. > > *From: *Rancid-discuss on behalf > of "Gauthier, Chris" > *Date: *Thursday, July 11, 2019 at 11:16 AM > *To: *john heasley , "Anderson, Charles R" < > cra at wpi.edu> > *Cc: *"rancid-discuss at shrubbery.net" > *Subject: *Re: [rancid] Palo Alto (Panorama) configuration > > > > Yes, the command "show config merged" gives the locally-managed config > output AND the configuration that is pushed out by Panorama. I'll make a > custom device type and see how this works in my environment. If it works, > I'll post the results here. I will also test with a non-Panorama-managed > system. > > --Chris > > *Chris**?* > > *Gauthier* > > Senior Network Engineer > > | > > Comscore > > t +1 *(503) 331-2704* <(503)%20331-2704> > > | > > *cgauthier at comscore.com* > > *comscore.com* > > ???This e-mail (including any attachments) may contain information that is > private, confidential, or protected by attorney-client or other privilege. > If you received this e-mail in error, please delete it from your system and > notify sender. > > -----Original Message----- > From: Rancid-discuss on behalf of > john heasley > Date: Thursday, July 11, 2019 at 8:17 AM > To: "Anderson, Charles R" > Cc: "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Palo Alto (Panorama) configuration > > Thu, Jul 11, 2019 at 02:37:51PM +0000, Anderson, Charles R: > > You can use "show config merged" to see the local device's config merged > with the templates from Panorama. > > Does this work with "non-managed" (better term?) configs? And, was this > command introduced recently? > > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > > https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fwww.shrubbery.net%2fmailman%2flistinfo%2francid-discuss&c=E,1,ZBO_SpPdPN9F0GTa50thF3JK2iNVO_jcwwSZwho1q8BVBoP9LydezSjLupULi9-PCcBbEWhWi1x-kRvg-KGqTG6CANfUm1cA6XPL5VPANHGtvC7Gc3N4Pg4SarAO&typo=1 > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > http://www.shrubbery.net/mailman/listinfo/rancid-discuss > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lsy.annie at gmail.com Thu Jul 11 21:53:06 2019 From: lsy.annie at gmail.com (annie lee) Date: Fri, 12 Jul 2019 07:53:06 +1000 Subject: [rancid] v3.9 ignoring timestamp and minor filesize in IOS/XE In-Reply-To: References: Message-ID: Some sample of diff that im receiving .. NX : (time/date difference only) - !Flash: volatile: 60 Jul 11 16:30:01 2019 .nginx/ + !Flash: volatile: 60 Jul 11 17:30:02 2019 .nginx/ - !Flash: logflash: 4096 Jan 09 02:02:06 2019 vdc_1/ + !Flash: logflash: 4096 Jul 10 23:30:01 2019 vdc_1/ IOS : (% difference) - !Flash: debug: 5 MB total (93% free) + !Flash: debug: 5 MB total (92% free) On Wed, Jul 10, 2019 at 8:02 AM annie lee wrote: > I know this question has been asked a couple of times, and i found a > thread providing a workaround to modify the code in ios.pm but not in > details. > Anyone can share what the line should be before and after ? > > Before : if (/(\d+) bytes (available|total) \((\d+) bytes used\)/) { > After : if NN [GMK]B free ?? > > Thanks in advance > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgauthier at comscore.com Thu Jul 11 22:51:33 2019 From: cgauthier at comscore.com (Gauthier, Chris) Date: Thu, 11 Jul 2019 22:51:33 +0000 Subject: [rancid] Palo Alto (Panorama) configuration In-Reply-To: References: <20190710214209.GD36475@shrubbery.net> <20190711143748.urpalsoan7pyq6nx@angus.ind.wpi.edu> <20190711151651.GB4422@shrubbery.net> <19611A45-0DE9-4E51-903F-7DFAE7040C40@comscore.com> Message-ID: I?m working through that right now. Chris Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. From: annie lee Date: Thursday, July 11, 2019 at 2:43 PM To: "Gauthier, Chris" Cc: john heasley , "Anderson, Charles R" , "rancid-discuss at shrubbery.net" Subject: Re: [rancid] Palo Alto (Panorama) configuration Thats good to know on the new cli (show config merged will grab everything from the firewall and panorama). How do we add the cli and diff to rancid ?? On Fri, Jul 12, 2019 at 4:20 AM Gauthier, Chris > wrote: Just validated the ?show config merged? command works with any PA firewall, managed by Panorama or not. Chris? Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. From: Rancid-discuss > on behalf of "Gauthier, Chris" > Date: Thursday, July 11, 2019 at 11:16 AM To: john heasley >, "Anderson, Charles R" > Cc: "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Palo Alto (Panorama) configuration Yes, the command "show config merged" gives the locally-managed config output AND the configuration that is pushed out by Panorama. I'll make a custom device type and see how this works in my environment. If it works, I'll post the results here. I will also test with a non-Panorama-managed system. --Chris Chris? Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. -----Original Message----- From: Rancid-discuss > on behalf of john heasley > Date: Thursday, July 11, 2019 at 8:17 AM To: "Anderson, Charles R" > Cc: "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Palo Alto (Panorama) configuration Thu, Jul 11, 2019 at 02:37:51PM +0000, Anderson, Charles R: > You can use "show config merged" to see the local device's config merged with the templates from Panorama. Does this work with "non-managed" (better term?) configs? And, was this command introduced recently? _______________________________________________ Rancid-discuss mailing list Rancid-discuss at shrubbery.net https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fwww.shrubbery.net%2fmailman%2flistinfo%2francid-discuss&c=E,1,ZBO_SpPdPN9F0GTa50thF3JK2iNVO_jcwwSZwho1q8BVBoP9LydezSjLupULi9-PCcBbEWhWi1x-kRvg-KGqTG6CANfUm1cA6XPL5VPANHGtvC7Gc3N4Pg4SarAO&typo=1 _______________________________________________ Rancid-discuss mailing list Rancid-discuss at shrubbery.net http://www.shrubbery.net/mailman/listinfo/rancid-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From lsy.annie at gmail.com Thu Jul 11 23:00:01 2019 From: lsy.annie at gmail.com (annie lee) Date: Fri, 12 Jul 2019 09:00:01 +1000 Subject: [rancid] Palo Alto (Panorama) configuration In-Reply-To: References: <20190710214209.GD36475@shrubbery.net> <20190711143748.urpalsoan7pyq6nx@angus.ind.wpi.edu> <20190711151651.GB4422@shrubbery.net> <19611A45-0DE9-4E51-903F-7DFAE7040C40@comscore.com> Message-ID: Hi Chris, Thats very kind of you to spend time doing that and thanks for that. Rgds On Fri, Jul 12, 2019 at 8:51 AM Gauthier, Chris wrote: > I?m working through that right now. > > > Chris? Gauthier Senior Network Engineer | Comscore > t +1 *(503) 331-2704* <(503)%20331-2704> | > *cgauthier at comscore.com* > *comscore.com* > ???This e-mail (including any attachments) may contain information that is > private, confidential, or protected by attorney-client or other privilege. > If you received this e-mail in error, please delete it from your system and > notify sender. > > *From: *annie lee > *Date: *Thursday, July 11, 2019 at 2:43 PM > *To: *"Gauthier, Chris" > *Cc: *john heasley , "Anderson, Charles R" < > cra at wpi.edu>, "rancid-discuss at shrubbery.net" > > *Subject: *Re: [rancid] Palo Alto (Panorama) configuration > > > > Thats good to know on the new cli (show config merged will grab everything > from the firewall and panorama). > > How do we add the cli and diff to rancid ?? > > > > On Fri, Jul 12, 2019 at 4:20 AM Gauthier, Chris > wrote: > > Just validated the ?show config merged? command works with any PA > firewall, managed by Panorama or not. > > > > *Chris**?* > > *Gauthier* > > Senior Network Engineer > > | > > Comscore > > t +1 *(503) 331-2704* <(503)%20331-2704> > > | > > *cgauthier at comscore.com* > > *comscore.com* > > ???This e-mail (including any attachments) may contain information that is > private, confidential, or protected by attorney-client or other privilege. > If you received this e-mail in error, please delete it from your system and > notify sender. > > *From: *Rancid-discuss on behalf > of "Gauthier, Chris" > *Date: *Thursday, July 11, 2019 at 11:16 AM > *To: *john heasley , "Anderson, Charles R" < > cra at wpi.edu> > *Cc: *"rancid-discuss at shrubbery.net" > *Subject: *Re: [rancid] Palo Alto (Panorama) configuration > > > > Yes, the command "show config merged" gives the locally-managed config > output AND the configuration that is pushed out by Panorama. I'll make a > custom device type and see how this works in my environment. If it works, > I'll post the results here. I will also test with a non-Panorama-managed > system. > > --Chris > > *Chris**?* > > *Gauthier* > > Senior Network Engineer > > | > > Comscore > > t +1 *(503) 331-2704* <(503)%20331-2704> > > | > > *cgauthier at comscore.com* > > *comscore.com* > > ???This e-mail (including any attachments) may contain information that is > private, confidential, or protected by attorney-client or other privilege. > If you received this e-mail in error, please delete it from your system and > notify sender. > > -----Original Message----- > From: Rancid-discuss on behalf of > john heasley > Date: Thursday, July 11, 2019 at 8:17 AM > To: "Anderson, Charles R" > Cc: "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Palo Alto (Panorama) configuration > > Thu, Jul 11, 2019 at 02:37:51PM +0000, Anderson, Charles R: > > You can use "show config merged" to see the local device's config merged > with the templates from Panorama. > > Does this work with "non-managed" (better term?) configs? And, was this > command introduced recently? > > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > > https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fwww.shrubbery.net%2fmailman%2flistinfo%2francid-discuss&c=E,1,ZBO_SpPdPN9F0GTa50thF3JK2iNVO_jcwwSZwho1q8BVBoP9LydezSjLupULi9-PCcBbEWhWi1x-kRvg-KGqTG6CANfUm1cA6XPL5VPANHGtvC7Gc3N4Pg4SarAO&typo=1 > > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > http://www.shrubbery.net/mailman/listinfo/rancid-discuss > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgauthier at comscore.com Fri Jul 12 16:23:33 2019 From: cgauthier at comscore.com (Gauthier, Chris) Date: Fri, 12 Jul 2019 16:23:33 +0000 Subject: [rancid] Palo Alto (Panorama) configuration In-Reply-To: References: <20190710214209.GD36475@shrubbery.net> <20190711143748.urpalsoan7pyq6nx@angus.ind.wpi.edu> <20190711151651.GB4422@shrubbery.net> <19611A45-0DE9-4E51-903F-7DFAE7040C40@comscore.com> Message-ID: I?m getting some interesting results in my testing. Rancid Version: 3.7 I have a pair of PA-5050?s managed by Panorama that have been only getting the ?show config running? output (the limited output). I made a new device type in etc/rancid.types.conf: panw;script;rancid -t paloalto panw;login;panlogin panw;module;panos panw;inloop;panos::inloop panw;command;rancid::RunCommand;set cli scripting-mode on panw;command;rancid::RunCommand;set cli pager off panw;command;panos::ShowInfo;show system info panw;command;panos::ShowConfig;show config merged This works well for my test unit (PA-220, unmanaged), but I am having problems with the PA-5050?s. For reference: Here is the device type of ?paloalto? in etc/rancid.types.base: paloalto;script;rancid -t paloalto paloalto;login;panlogin paloalto;module;panos paloalto;inloop;panos::inloop paloalto;command;rancid::RunCommand;set cli scripting-mode on paloalto;command;rancid::RunCommand;set cli pager off paloalto;command;panos::ShowInfo;show system info paloalto;command;panos::ShowConfig;show config running With the PA-5050?s, started with the following lines in router.db: pa-1.example.com;paloalto;up;PA-5050 ha pair pa-2.example.com;paloalto;up;PA-5050 ha pair They?ve been getting the limited output because of the show config running command and that they?re managed by Panorama. I altered the router.db file to: pa-1.example.com;panw;up;PA-5050 ha pair pa-2.example.com;panw;up;PA-5050 ha pair I got the email that said the original devices were deleted and the new devices were added. - pa-1.example.com;paloalto;up;PA-5050 - pa-2.example.com;panw;paloalto;up;PA-5050 + pa-1.example.com;panw;up;PA-5050 + pa-2.example.com;panw;panw;up;PA-5050 I checked the config files after running rancid again a couple times and the config was unchanged. The output captured doesn?t seem to have changed. Next, I troubleshot it by doing ?NOPIPE=yes rancid -d -t panw pa-1.example.com? and reviewing the output. It captured everything cleanly, as far as I can tell. No errors. It?s like the diff is not catching the difference in output? What might I try next? --Chris Chris Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. From: annie lee Date: Thursday, July 11, 2019 at 4:00 PM To: "Gauthier, Chris" Cc: john heasley , "Anderson, Charles R" , "rancid-discuss at shrubbery.net" Subject: Re: [rancid] Palo Alto (Panorama) configuration Hi Chris, Thats very kind of you to spend time doing that and thanks for that. Rgds On Fri, Jul 12, 2019 at 8:51 AM Gauthier, Chris > wrote: I?m working through that right now. Chris? Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. From: annie lee > Date: Thursday, July 11, 2019 at 2:43 PM To: "Gauthier, Chris" > Cc: john heasley >, "Anderson, Charles R" >, "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Palo Alto (Panorama) configuration Thats good to know on the new cli (show config merged will grab everything from the firewall and panorama). How do we add the cli and diff to rancid ?? On Fri, Jul 12, 2019 at 4:20 AM Gauthier, Chris > wrote: Just validated the ?show config merged? command works with any PA firewall, managed by Panorama or not. Chris? Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. From: Rancid-discuss > on behalf of "Gauthier, Chris" > Date: Thursday, July 11, 2019 at 11:16 AM To: john heasley >, "Anderson, Charles R" > Cc: "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Palo Alto (Panorama) configuration Yes, the command "show config merged" gives the locally-managed config output AND the configuration that is pushed out by Panorama. I'll make a custom device type and see how this works in my environment. If it works, I'll post the results here. I will also test with a non-Panorama-managed system. --Chris Chris? Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. -----Original Message----- From: Rancid-discuss > on behalf of john heasley > Date: Thursday, July 11, 2019 at 8:17 AM To: "Anderson, Charles R" > Cc: "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Palo Alto (Panorama) configuration Thu, Jul 11, 2019 at 02:37:51PM +0000, Anderson, Charles R: > You can use "show config merged" to see the local device's config merged with the templates from Panorama. Does this work with "non-managed" (better term?) configs? And, was this command introduced recently? _______________________________________________ Rancid-discuss mailing list Rancid-discuss at shrubbery.net https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fwww.shrubbery.net%2fmailman%2flistinfo%2francid-discuss&c=E,1,ZBO_SpPdPN9F0GTa50thF3JK2iNVO_jcwwSZwho1q8BVBoP9LydezSjLupULi9-PCcBbEWhWi1x-kRvg-KGqTG6CANfUm1cA6XPL5VPANHGtvC7Gc3N4Pg4SarAO&typo=1 _______________________________________________ Rancid-discuss mailing list Rancid-discuss at shrubbery.net http://www.shrubbery.net/mailman/listinfo/rancid-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgauthier at comscore.com Fri Jul 12 17:50:59 2019 From: cgauthier at comscore.com (Gauthier, Chris) Date: Fri, 12 Jul 2019 17:50:59 +0000 Subject: [rancid] Rancid.Conf Disappeared on Ubuntu Update Message-ID: I have to admit, I wish the etc/ directory was part of a Git repo. I could do it locally, but would be a nice feature enhancement. Chris Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. From: Rancid-discuss on behalf of "Sheeter, Kyle" Date: Wednesday, July 3, 2019 at 1:18 PM To: "rancid-discuss at shrubbery.net" Subject: [rancid] Rancid.Conf Disappeared on Ubuntu Update Hey all, I was doing some Ubuntu upgrades on my server, and just noticed that RANCID stop sending me updates. Ran the rancid-run command and then found out that my rancid.conf file disappeared. Anyone know the best way to recreate the conf file? All of my other information is still there it seems, and the DB is still populated with my old network data. Thanks! Kyle James Sheeter Please be advised that this email may contain confidential information. If you are not the intended recipient, please notify us by email by replying to the sender and delete this message. The sender disclaims that the content of this email constitutes an offer to enter into, or the acceptance of, any agreement; provided that the foregoing does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgauthier at comscore.com Fri Jul 12 17:58:32 2019 From: cgauthier at comscore.com (Gauthier, Chris) Date: Fri, 12 Jul 2019 17:58:32 +0000 Subject: [rancid] Palo Alto (Panorama) configuration In-Reply-To: References: <20190710214209.GD36475@shrubbery.net> <20190711143748.urpalsoan7pyq6nx@angus.ind.wpi.edu> <20190711151651.GB4422@shrubbery.net> <19611A45-0DE9-4E51-903F-7DFAE7040C40@comscore.com> Message-ID: <9995977F-1CCE-40CD-8E0B-EBE5B780EEB3@comscore.com> So, if you look at my posting below, I made a rather dumb copy/paste error in my ?panw? definition. The first line should read: panw;script;rancid -t paloalto not: panw;script;rancid -t paloalto Thanks to Heasley for pointing that out! I would have not seen that for a while. Having changed the line as shown above, the ?show config merged? now works great on Panorama-managed and non-managed PA devices. --Chris Chris Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. From: Rancid-discuss on behalf of "Gauthier, Chris" Date: Friday, July 12, 2019 at 9:24 AM To: annie lee Cc: "rancid-discuss at shrubbery.net" Subject: Re: [rancid] Palo Alto (Panorama) configuration I?m getting some interesting results in my testing. Rancid Version: 3.7 I have a pair of PA-5050?s managed by Panorama that have been only getting the ?show config running? output (the limited output). I made a new device type in etc/rancid.types.conf: panw;script;rancid -t paloalto panw;login;panlogin panw;module;panos panw;inloop;panos::inloop panw;command;rancid::RunCommand;set cli scripting-mode on panw;command;rancid::RunCommand;set cli pager off panw;command;panos::ShowInfo;show system info panw;command;panos::ShowConfig;show config merged This works well for my test unit (PA-220, unmanaged), but I am having problems with the PA-5050?s. For reference: Here is the device type of ?paloalto? in etc/rancid.types.base: paloalto;script;rancid -t paloalto paloalto;login;panlogin paloalto;module;panos paloalto;inloop;panos::inloop paloalto;command;rancid::RunCommand;set cli scripting-mode on paloalto;command;rancid::RunCommand;set cli pager off paloalto;command;panos::ShowInfo;show system info paloalto;command;panos::ShowConfig;show config running With the PA-5050?s, started with the following lines in router.db: pa-1.example.com;paloalto;up;PA-5050 ha pair pa-2.example.com;paloalto;up;PA-5050 ha pair They?ve been getting the limited output because of the show config running command and that they?re managed by Panorama. I altered the router.db file to: pa-1.example.com;panw;up;PA-5050 ha pair pa-2.example.com;panw;up;PA-5050 ha pair I got the email that said the original devices were deleted and the new devices were added. - pa-1.example.com;paloalto;up;PA-5050 - pa-2.example.com;panw;paloalto;up;PA-5050 + pa-1.example.com;panw;up;PA-5050 + pa-2.example.com;panw;panw;up;PA-5050 I checked the config files after running rancid again a couple times and the config was unchanged. The output captured doesn?t seem to have changed. Next, I troubleshot it by doing ?NOPIPE=yes rancid -d -t panw pa-1.example.com? and reviewing the output. It captured everything cleanly, as far as I can tell. No errors. It?s like the diff is not catching the difference in output? What might I try next? --Chris Chris? Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. From: annie lee Date: Thursday, July 11, 2019 at 4:00 PM To: "Gauthier, Chris" Cc: john heasley , "Anderson, Charles R" , "rancid-discuss at shrubbery.net" Subject: Re: [rancid] Palo Alto (Panorama) configuration Hi Chris, Thats very kind of you to spend time doing that and thanks for that. Rgds On Fri, Jul 12, 2019 at 8:51 AM Gauthier, Chris > wrote: I?m working through that right now. Chris? Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. From: annie lee > Date: Thursday, July 11, 2019 at 2:43 PM To: "Gauthier, Chris" > Cc: john heasley >, "Anderson, Charles R" >, "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Palo Alto (Panorama) configuration Thats good to know on the new cli (show config merged will grab everything from the firewall and panorama). How do we add the cli and diff to rancid ?? On Fri, Jul 12, 2019 at 4:20 AM Gauthier, Chris > wrote: Just validated the ?show config merged? command works with any PA firewall, managed by Panorama or not. Chris? Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. From: Rancid-discuss > on behalf of "Gauthier, Chris" > Date: Thursday, July 11, 2019 at 11:16 AM To: john heasley >, "Anderson, Charles R" > Cc: "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Palo Alto (Panorama) configuration Yes, the command "show config merged" gives the locally-managed config output AND the configuration that is pushed out by Panorama. I'll make a custom device type and see how this works in my environment. If it works, I'll post the results here. I will also test with a non-Panorama-managed system. --Chris Chris? Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. -----Original Message----- From: Rancid-discuss > on behalf of john heasley > Date: Thursday, July 11, 2019 at 8:17 AM To: "Anderson, Charles R" > Cc: "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Palo Alto (Panorama) configuration Thu, Jul 11, 2019 at 02:37:51PM +0000, Anderson, Charles R: > You can use "show config merged" to see the local device's config merged with the templates from Panorama. Does this work with "non-managed" (better term?) configs? And, was this command introduced recently? _______________________________________________ Rancid-discuss mailing list Rancid-discuss at shrubbery.net https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fwww.shrubbery.net%2fmailman%2flistinfo%2francid-discuss&c=E,1,ZBO_SpPdPN9F0GTa50thF3JK2iNVO_jcwwSZwho1q8BVBoP9LydezSjLupULi9-PCcBbEWhWi1x-kRvg-KGqTG6CANfUm1cA6XPL5VPANHGtvC7Gc3N4Pg4SarAO&typo=1 _______________________________________________ Rancid-discuss mailing list Rancid-discuss at shrubbery.net http://www.shrubbery.net/mailman/listinfo/rancid-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgauthier at comscore.com Fri Jul 12 18:15:39 2019 From: cgauthier at comscore.com (Gauthier, Chris) Date: Fri, 12 Jul 2019 18:15:39 +0000 Subject: [rancid] Restore a Palo Alto Firewall from a Rancid bacup In-Reply-To: <20190705174251.GG55957@shrubbery.net> References: <20190705174251.GG55957@shrubbery.net> Message-ID: <6863732B-B571-47F7-BE51-747924E4F76E@comscore.com> Rancid configs for PAN can NOT be used to restore the config, unless you cut and paste the configuration. This is because the native config files are stored in XML format and that is the format the Palo Alto utilities expect when performing restorations. --Chris ? Chris Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. -----Original Message----- From: Rancid-discuss on behalf of john heasley Date: Friday, July 5, 2019 at 10:43 AM To: STUART WALTON Cc: "rancid-discuss at shrubbery.net" Subject: Re: [rancid] Restore a Palo Alto Firewall from a Rancid bacup Thu, Jul 04, 2019 at 08:23:51AM +0000, STUART WALTON: > Hi > > Has anyone used a backup from Rancid to restore a Palo Alto Firewall? > > If so how have you done it? (I have the backup but it does not appear to be in the correct format) > > I have searched the discussion but cannot seem to find the answer. Any help would be appreciated. I do not know much of anything about PAN devices. However, be aware that, depending upon your rancid configuration, passwords may be removed. Also, see the FAQ S1 Q5 for another caveat that may apply to PAN. Also, include the error you received when attempting to load the config. It might provide clue to someone with more experience with PAN. _______________________________________________ Rancid-discuss mailing list Rancid-discuss at shrubbery.net https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fwww.shrubbery.net%2fmailman%2flistinfo%2francid-discuss&c=E,1,qrWANWlQYaUeaaoEGf6I-WmqahOFpLboIOsZz7b3yKfSUzpY5cUajZzVEWvA4kobgPxxfRU1MaUB91_9kWsr_BYI8TlZE-d1DrWcD7WIFEmJsZMiU0LMHAkW&typo=1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From heas at shrubbery.net Fri Jul 12 18:20:57 2019 From: heas at shrubbery.net (john heasley) Date: Fri, 12 Jul 2019 18:20:57 +0000 Subject: [rancid] Restore a Palo Alto Firewall from a Rancid bacup In-Reply-To: <6863732B-B571-47F7-BE51-747924E4F76E@comscore.com> References: <20190705174251.GG55957@shrubbery.net> <6863732B-B571-47F7-BE51-747924E4F76E@comscore.com> Message-ID: <20190712182057.GA16982@shrubbery.net> Fri, Jul 12, 2019 at 06:15:39PM +0000, Gauthier, Chris: > Rancid configs for PAN can NOT be used to restore the config, unless you cut and paste the configuration. This is because the native config files are stored in XML format and that is the format the Palo Alto utilities expect when performing restorations. > so, store both in rancid. what is the cmd to retrieve the xml format? From scott.granados at gmail.com Fri Jul 12 18:44:22 2019 From: scott.granados at gmail.com (Scott Granados) Date: Fri, 12 Jul 2019 14:44:22 -0400 Subject: [rancid] Restore a Palo Alto Firewall from a Rancid bacup In-Reply-To: <20190712182057.GA16982@shrubbery.net> References: <20190705174251.GG55957@shrubbery.net> <6863732B-B571-47F7-BE51-747924E4F76E@comscore.com> <20190712182057.GA16982@shrubbery.net> Message-ID: <3EABF3D7-A4F7-44A8-B5D2-96B78FB98087@gmail.com> It?s not XML, it?s JSUN if I understand where you?re going with this. From exec mode Set cli config-output-format default Also other variables here can be set for set form andother formats which you can select and display with a ? In the config-output-format parameter field. Thanks > On Jul 12, 2019, at 2:20 PM, john heasley wrote: > > Fri, Jul 12, 2019 at 06:15:39PM +0000, Gauthier, Chris: >> Rancid configs for PAN can NOT be used to restore the config, unless you cut and paste the configuration. This is because the native config files are stored in XML format and that is the format the Palo Alto utilities expect when performing restorations. >> > > so, store both in rancid. what is the cmd to retrieve the xml format? > > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > http://www.shrubbery.net/mailman/listinfo/rancid-discuss From cgauthier at comscore.com Fri Jul 12 18:56:35 2019 From: cgauthier at comscore.com (Gauthier, Chris) Date: Fri, 12 Jul 2019 18:56:35 +0000 Subject: [rancid] Restore a Palo Alto Firewall from a Rancid bacup In-Reply-To: <3EABF3D7-A4F7-44A8-B5D2-96B78FB98087@gmail.com> References: <20190705174251.GG55957@shrubbery.net> <6863732B-B571-47F7-BE51-747924E4F76E@comscore.com> <20190712182057.GA16982@shrubbery.net> <3EABF3D7-A4F7-44A8-B5D2-96B78FB98087@gmail.com> Message-ID: Exported config files are in XML format. Here is a link to the documentation. Nowhere in their documentation does it reference using JSON as the format for import/export. Also, Palo Alto has a "scheduled export" facility, especially if you are using Panorama. We use RANCiD to track the changes more than anything, but use the utility to auto-export configs. https://docs.paloaltonetworks.com/pan-os/8-1/pan-os-admin/firewall-administration/manage-configuration-backups/save-and-export-firewall-configurations.html --Chris ? Chris Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. -----Original Message----- From: Scott Granados Date: Friday, July 12, 2019 at 11:44 AM To: john heasley Cc: "Gauthier, Chris" , "rancid-discuss at shrubbery.net" Subject: Re: [rancid] Restore a Palo Alto Firewall from a Rancid bacup It?s not XML, it?s JSUN if I understand where you?re going with this. >From exec mode Set cli config-output-format default Also other variables here can be set for set form andother formats which you can select and display with a ? In the config-output-format parameter field. Thanks > On Jul 12, 2019, at 2:20 PM, john heasley wrote: > > Fri, Jul 12, 2019 at 06:15:39PM +0000, Gauthier, Chris: >> Rancid configs for PAN can NOT be used to restore the config, unless you cut and paste the configuration. This is because the native config files are stored in XML format and that is the format the Palo Alto utilities expect when performing restorations. >> > > so, store both in rancid. what is the cmd to retrieve the xml format? > > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fwww.shrubbery.net%2fmailman%2flistinfo%2francid-discuss&c=E,1,sOD-u4Fb7FVnpwIC-I0Noqe21OYAOvq8QodxcvUVO6-_RwELL2hG9BvQdat-eHRfzF59pW8ydxDEwG45J8a3oI9ghdsNO9UKZn3Kwl9xyPeaQm2MlpRKXQLW2A,,&typo=1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From heas at shrubbery.net Fri Jul 12 19:15:07 2019 From: heas at shrubbery.net (john heasley) Date: Fri, 12 Jul 2019 19:15:07 +0000 Subject: [rancid] Extreme switch policy backup. In-Reply-To: References: Message-ID: <20190712191507.GC16982@shrubbery.net> Tue, Jul 09, 2019 at 09:55:56PM +0000, Chris Davis: > We've just gotten a few Extreme switches (model X440-G2) and I've gotten them set up in Rancid. But while I get the configs, I have a few policies as well. They're kept as .pol files on the switch. Is there a way to include the policy files in the backup that Rancid takes? It would be particularly helpful. I've done some searching, and seen folks ask about it. But no real answers. Lots of modifications to commands from 4 years ago but nothing current. There's a command that will print it all out, just not sure how to add it into the mix. Don't like to modify something like Rancid if there's already a way within the system to make it happen. what is the command to display the policy? can you provide an example of the command and output, from prompt to the next prompt? is the output format and order stable? i see an incomplete example here; http://www.shrubbery.net/pipermail/rancid-discuss/2014-May/007659.html From erikm at buh.org Fri Jul 12 19:18:34 2019 From: erikm at buh.org (Erik Muller) Date: Fri, 12 Jul 2019 21:18:34 +0200 Subject: [rancid] Restore a Palo Alto Firewall from a Rancid bacup In-Reply-To: <6863732B-B571-47F7-BE51-747924E4F76E@comscore.com> References: <20190705174251.GG55957@shrubbery.net> <6863732B-B571-47F7-BE51-747924E4F76E@comscore.com> Message-ID: On 7/12/19 14:15 , Gauthier, Chris wrote: > Rancid configs for PAN can NOT be used to restore the config, unless you > cut and paste the configuration. This is because the native config files > are stored in XML format and that is the format the Palo Alto utilities > expect when performing restorations. Having recently needed to deal with a bunch of PAs, I ran into that same issue and ended up writing a tool (https://github.com/ermuller/bracematch) to simplify the process. RE the other question about Panorama vs device configs, if you're backing up your Panorama configuration (which has been fine via Rancid in my experience) as well as the base config on the device, you don't need to backup the merged configuration. And you probably shouldn't pull the merged config, for restore purposes, as anything other than the local device configuration will come from the Panorama templates once the device is replaced. Of course, the merged config might still be convenient to save to easily see the complete policy set active on a given box. -e From scott.granados at gmail.com Fri Jul 12 19:23:30 2019 From: scott.granados at gmail.com (Scott Granados) Date: Fri, 12 Jul 2019 15:23:30 -0400 Subject: [rancid] Restore a Palo Alto Firewall from a Rancid bacup In-Reply-To: References: <20190705174251.GG55957@shrubbery.net> <6863732B-B571-47F7-BE51-747924E4F76E@comscore.com> <20190712182057.GA16982@shrubbery.net> <3EABF3D7-A4F7-44A8-B5D2-96B78FB98087@gmail.com> Message-ID: <9768DF85-6672-4BA0-BB17-E1C83A034D8F@gmail.com> We haven?t bothered with Panorama much because unlike the firewalls themselves the Panorama interface is very poor with screen readers and other accessibility technologies used. In AWS we do a lot of exporting of configs and use S3 to bootstrap the virtual appliances so there may be a difference in what I?m working with. We can edit the configs in S3 and they an be automatically imported or grabbed on boot. On the hardware though I thought it was selectable. I?ll review the link you sent, thank you. Just queried my PA and the choices I have to export or import configs are JSUN, XML, SET or Default which looks like JSUN to me so not sure why that?s duplicated. I am just setting the CLI variable I assume you?re using a different mechanism that?s different. Thanks If you?re connecting via SSH and pulling the config I don?t see why you couldn?t set it to what ever format you wanted and then push with the correct flag set at the head of the request. > On Jul 12, 2019, at 2:56 PM, Gauthier, Chris wrote: > > Exported config files are in XML format. Here is a link to the documentation. Nowhere in their documentation does it reference using JSON as the format for import/export. > > Also, Palo Alto has a "scheduled export" facility, especially if you are using Panorama. We use RANCiD to track the changes more than anything, but use the utility to auto-export configs. > > https://docs.paloaltonetworks.com/pan-os/8-1/pan-os-admin/firewall-administration/manage-configuration-backups/save-and-export-firewall-configurations.html > > --Chris > > > > ? > Chris? Gauthier Senior Network Engineer | Comscore > t +1 (503)?331-2704 | > cgauthier at comscore.com > comscore.com > ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. > -----Original Message----- > From: Scott Granados > Date: Friday, July 12, 2019 at 11:44 AM > To: john heasley > Cc: "Gauthier, Chris" , "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Restore a Palo Alto Firewall from a Rancid bacup > > It?s not XML, it?s JSUN if I understand where you?re going with this. > > From exec mode > Set cli config-output-format default > > Also other variables here can be set for set form andother formats which you can select and display with a ? In the config-output-format parameter field. > > Thanks > > > > On Jul 12, 2019, at 2:20 PM, john heasley wrote: > > > > Fri, Jul 12, 2019 at 06:15:39PM +0000, Gauthier, Chris: > >> Rancid configs for PAN can NOT be used to restore the config, unless you cut and paste the configuration. This is because the native config files are stored in XML format and that is the format the Palo Alto utilities expect when performing restorations. > >> > > > > so, store both in rancid. what is the cmd to retrieve the xml format? > > > > _______________________________________________ > > Rancid-discuss mailing list > > Rancid-discuss at shrubbery.net > > https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fwww.shrubbery.net%2fmailman%2flistinfo%2francid-discuss&c=E,1,sOD-u4Fb7FVnpwIC-I0Noqe21OYAOvq8QodxcvUVO6-_RwELL2hG9BvQdat-eHRfzF59pW8ydxDEwG45J8a3oI9ghdsNO9UKZn3Kwl9xyPeaQm2MlpRKXQLW2A,,&typo=1 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgauthier at comscore.com Fri Jul 12 19:28:47 2019 From: cgauthier at comscore.com (Gauthier, Chris) Date: Fri, 12 Jul 2019 19:28:47 +0000 Subject: [rancid] Restore a Palo Alto Firewall from a Rancid bacup In-Reply-To: <9768DF85-6672-4BA0-BB17-E1C83A034D8F@gmail.com> References: <20190705174251.GG55957@shrubbery.net> <6863732B-B571-47F7-BE51-747924E4F76E@comscore.com> <20190712182057.GA16982@shrubbery.net> <3EABF3D7-A4F7-44A8-B5D2-96B78FB98087@gmail.com> <9768DF85-6672-4BA0-BB17-E1C83A034D8F@gmail.com> Message-ID: Yes, you can export the different formats, but the restore expects XML, in my experience. Also, for those using Panorama, Erik?s advice to rely on Panorama is sound. Been there, done that, don?t want to restore again, but it worked! --Chris Chris Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. From: Scott Granados Date: Friday, July 12, 2019 at 12:23 PM To: "Gauthier, Chris" Cc: john heasley , "rancid-discuss at shrubbery.net" Subject: Re: [rancid] Restore a Palo Alto Firewall from a Rancid bacup We haven?t bothered with Panorama much because unlike the firewalls themselves the Panorama interface is very poor with screen readers and other accessibility technologies used. In AWS we do a lot of exporting of configs and use S3 to bootstrap the virtual appliances so there may be a difference in what I?m working with. We can edit the configs in S3 and they an be automatically imported or grabbed on boot. On the hardware though I thought it was selectable. I?ll review the link you sent, thank you. Just queried my PA and the choices I have to export or import configs are JSUN, XML, SET or Default which looks like JSUN to me so not sure why that?s duplicated. I am just setting the CLI variable I assume you?re using a different mechanism that?s different. Thanks If you?re connecting via SSH and pulling the config I don?t see why you couldn?t set it to what ever format you wanted and then push with the correct flag set at the head of the request. On Jul 12, 2019, at 2:56 PM, Gauthier, Chris > wrote: Exported config files are in XML format. Here is a link to the documentation. Nowhere in their documentation does it reference using JSON as the format for import/export. Also, Palo Alto has a "scheduled export" facility, especially if you are using Panorama. We use RANCiD to track the changes more than anything, but use the utility to auto-export configs. https://docs.paloaltonetworks.com/pan-os/8-1/pan-os-admin/firewall-administration/manage-configuration-backups/save-and-export-firewall-configurations.html --Chris Chris? Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. -----Original Message----- From: Scott Granados Date: Friday, July 12, 2019 at 11:44 AM To: john heasley Cc: "Gauthier, Chris" , "rancid-discuss at shrubbery.net" Subject: Re: [rancid] Restore a Palo Alto Firewall from a Rancid bacup It?s not XML, it?s JSUN if I understand where you?re going with this. >From exec mode Set cli config-output-format default Also other variables here can be set for set form andother formats which you can select and display with a ? In the config-output-format parameter field. Thanks > On Jul 12, 2019, at 2:20 PM, john heasley wrote: > > Fri, Jul 12, 2019 at 06:15:39PM +0000, Gauthier, Chris: >> Rancid configs for PAN can NOT be used to restore the config, unless you cut and paste the configuration. This is because the native config files are stored in XML format and that is the format the Palo Alto utilities expect when performing restorations. >> > > so, store both in rancid. what is the cmd to retrieve the xml format? > > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fwww.shrubbery.net%2fmailman%2flistinfo%2francid-discuss&c=E,1,sOD-u4Fb7FVnpwIC-I0Noqe21OYAOvq8QodxcvUVO6-_RwELL2hG9BvQdat-eHRfzF59pW8ydxDEwG45J8a3oI9ghdsNO9UKZn3Kwl9xyPeaQm2MlpRKXQLW2A,,&typo=1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul at prt.org Fri Jul 12 19:30:28 2019 From: paul at prt.org (Paul Thornton) Date: Fri, 12 Jul 2019 20:30:28 +0100 Subject: [rancid] Extreme switch policy backup. In-Reply-To: <20190712191507.GC16982@shrubbery.net> References: <20190712191507.GC16982@shrubbery.net> Message-ID: <94eafe33-dde6-6c05-4727-59892963124b@prt.org> Hi We had a patch to 2.3's xrancid which we were running at some stage in the past N years that did this already - but can't I find it, and we aren't running it on our current rancid system either. Thanks to Chris' E-mail at least I've been reminded of that. It wasn't a hard thing to add. On 12/07/2019 20:15, john heasley wrote: > Tue, Jul 09, 2019 at 09:55:56PM +0000, Chris Davis: >> We've just gotten a few Extreme switches (model X440-G2) and I've gotten them set up in Rancid. But while I get the configs, I have a few policies as well. They're kept as .pol files on the switch. Is there a way to include the policy files in the backup that Rancid takes? It would be particularly helpful. I've done some searching, and seen folks ask about it. But no real answers. Lots of modifications to commands from 4 years ago but nothing current. There's a command that will print it all out, just not sure how to add it into the mix. Don't like to modify something like Rancid if there's already a way within the system to make it happen. > > what is the command to display the policy? can you provide an example of > the command and output, from prompt to the next prompt? is the output > format and order stable? > > i see an incomplete example here; > http://www.shrubbery.net/pipermail/rancid-discuss/2014-May/007659.html The format isn't great. The switch basically outputs Policies at Policy Server: Policy: Number of clients bound to policy: Client: My hunch would be not to try and parse this lot at all, but just execute the 'show policy detail' and wait for the prompt to come back. I'm pretty sure that's all we did; I remember it just diffed everything and you saw quickly if a policy was added/removed just as easily. It is theoretically possible for someone to have a prompt matching string in the policy file as a comment, but lets ignore that madness for now. This example shows three policies as an example: * ag1.hbr.2 # dis clip * ag1.hbr.3 # show policy detail Policies at Policy Server: Policy: as65001-in-v4 entry term10 { if match all { nlri 185.0.0.0/23 exact ; nlri 185.0.2.0/24 exact ; nlri 185.0.3.0/24 exact ; } then { local-preference 500 ; community add "65301:200" ; permit ; } } entry term999 { if match all { } then { deny ; } } Number of clients bound to policy: 1 Client: bgp bound once Policy: as65001-in-v6 entry term10 { if match all { nlri 2001:db8:0::/45 ; } then { local-preference 500 ; community add "65301:200" ; permit ; } } entry term999 { if match all { } then { deny ; } } Number of clients bound to policy: 1 Client: bgp bound once Policy: as65001-out-v4 entry term10 { if match all { nlri 0.0.0.0/0 exact ; } then { permit ; } } entry term999 { if match all { } then { deny ; } } Number of clients bound to policy: 1 Client: bgp bound once * ag1.hbr.3 # From heas at shrubbery.net Fri Jul 12 20:05:30 2019 From: heas at shrubbery.net (john heasley) Date: Fri, 12 Jul 2019 20:05:30 +0000 Subject: [rancid] Extreme switch policy backup. In-Reply-To: <20190712200409.B946A17541@sea.shrubbery.net> <94eafe33-dde6-6c05-4727-59892963124b@prt.org> Message-ID: <20190712200530.GH16982@shrubbery.net> Fri, Jul 12, 2019 at 08:30:28PM +0100, Paul Thornton: > Hi > > We had a patch to 2.3's xrancid which we were running at some stage in > the past N years that did this already - but can't I find it, and we > aren't running it on our current rancid system either. Thanks to Chris' > E-mail at least I've been reminded of that. > > It wasn't a hard thing to add. > > On 12/07/2019 20:15, john heasley wrote: > > Tue, Jul 09, 2019 at 09:55:56PM +0000, Chris Davis: > >> We've just gotten a few Extreme switches (model X440-G2) and I've gotten them set up in Rancid. But while I get the configs, I have a few policies as well. They're kept as .pol files on the switch. Is there a way to include the policy files in the backup that Rancid takes? It would be particularly helpful. I've done some searching, and seen folks ask about it. But no real answers. Lots of modifications to commands from 4 years ago but nothing current. There's a command that will print it all out, just not sure how to add it into the mix. Don't like to modify something like Rancid if there's already a way within the system to make it happen. > > > > what is the command to display the policy? can you provide an example of > > the command and output, from prompt to the next prompt? is the output > > format and order stable? > > > > i see an incomplete example here; > > http://www.shrubbery.net/pipermail/rancid-discuss/2014-May/007659.html > > The format isn't great. The switch basically outputs > Policies at Policy Server: > Policy: > > Number of clients bound to policy: > Client: > > My hunch would be not to try and parse this lot at all, but just execute > the 'show policy detail' and wait for the prompt to come back. I'm > pretty sure that's all we did; I remember it just diffed everything and > you saw quickly if a policy was added/removed just as easily. > It is theoretically possible for someone to have a prompt matching > string in the policy file as a comment, but lets ignore that madness for > now. > > This example shows three policies as an example: > > * ag1.hbr.2 # dis clip > * ag1.hbr.3 # show policy detail > Policies at Policy Server: > Policy: as65001-in-v4 > entry term10 { Cool. Could you test this? diff --git a/etc/rancid.types.base b/etc/rancid.types.base index 18139479..6c3a80aa 100644 --- a/etc/rancid.types.base +++ b/etc/rancid.types.base @@ -381,6 +381,7 @@ extreme;command;exos::ShowMemory;show memory extreme;command;exos::ShowDiag;show diag extreme;command;exos::ShowSwitch;show switch extreme;command;exos::ShowSlot;show slot +extreme;command;exos::ShowPolicy;show policy detail extreme;command;exos::WriteTerm;show configuration detail extreme;command;exos::WriteTerm;show configuration # diff --git a/lib/exos.pm.in b/lib/exos.pm.in index fd7d1482..710a5c0f 100644 --- a/lib/exos.pm.in +++ b/lib/exos.pm.in @@ -1,7 +1,5 @@ package exos; ## -## $Id$ -## ## @PACKAGE@ @VERSION@ @copyright@ # @@ -161,6 +159,21 @@ sub ShowDiag { return(0); } +# This routine parses "show policy detail" +sub ShowPolicy { + my($INPUT, $OUTPUT, $cmd) = @_; + print STDERR " In ShowPolicy: $_" if ($debug); + + while (<$INPUT>) { + tr/\015//d; + last if (/^$prompt/); + next if (/^(\s*|\s*$cmd\s*)$/); + + ProcessHistory("POLICY","","","# $_"); + } + return(0); +} + # This routine parses "show slot" sub ShowSlot { my($INPUT, $OUTPUT, $cmd) = @_; From lsy.annie at gmail.com Fri Jul 12 22:35:25 2019 From: lsy.annie at gmail.com (annie lee) Date: Sat, 13 Jul 2019 08:35:25 +1000 Subject: [rancid] Palo Alto (Panorama) configuration In-Reply-To: <9995977F-1CCE-40CD-8E0B-EBE5B780EEB3@comscore.com> References: <20190710214209.GD36475@shrubbery.net> <20190711143748.urpalsoan7pyq6nx@angus.ind.wpi.edu> <20190711151651.GB4422@shrubbery.net> <19611A45-0DE9-4E51-903F-7DFAE7040C40@comscore.com> <9995977F-1CCE-40CD-8E0B-EBE5B780EEB3@comscore.com> Message-ID: Hi Chris, I've made similar chnages on v3.9 but not getting the new 'merged' config based on yours. Below are the panw code i added : panw;script;rancid -t paloalto panw;login;panlogin panw;module;panos panw;inloop;panos::inloop panw;command;panos::ShowInfo;show system info panw;command;panos::ShowInventory;show chassis inventory panw;command;panos::ShowConfig;show config merged Unfortunately still didnt captured the panorama configs. On Sat, Jul 13, 2019 at 3:58 AM Gauthier, Chris wrote: > So, if you look at my posting below, I made a rather dumb copy/paste error > in my ?panw? definition. The first line should read: > > > > panw;script;rancid -t paloalto > > > > not: > > panw;script;rancid -t paloalto > > > > > > Thanks to Heasley for pointing that out! I would have not seen that for a > while. Having changed the line as shown above, the ?show config merged? > now works great on Panorama-managed and non-managed PA devices. > > > > --Chris > Chris? Gauthier Senior Network Engineer | Comscore > t +1 *(503) 331-2704* <(503)%20331-2704> | > *cgauthier at comscore.com* > *comscore.com* > ???This e-mail (including any attachments) may contain information that is > private, confidential, or protected by attorney-client or other privilege. > If you received this e-mail in error, please delete it from your system and > notify sender. > > *From: *Rancid-discuss on behalf > of "Gauthier, Chris" > *Date: *Friday, July 12, 2019 at 9:24 AM > *To: *annie lee > *Cc: *"rancid-discuss at shrubbery.net" > *Subject: *Re: [rancid] Palo Alto (Panorama) configuration > > > > I?m getting some interesting results in my testing. > > > > Rancid Version: 3.7 > > > > I have a pair of PA-5050?s managed by Panorama that have been only getting > the ?show config running? output (the limited output). I made a new device > type in etc/rancid.types.conf: > > > > panw;script;rancid -t paloalto > > panw;login;panlogin > > panw;module;panos > > panw;inloop;panos::inloop > > panw;command;rancid::RunCommand;set cli scripting-mode on > > panw;command;rancid::RunCommand;set cli pager off > > panw;command;panos::ShowInfo;show system info > > panw;command;panos::ShowConfig;show config merged > > > > This works well for my test unit (PA-220, unmanaged), but I am having > problems with the PA-5050?s. > > > > For reference: Here is the device type of ?paloalto? in > etc/rancid.types.base: > > paloalto;script;rancid -t paloalto > > paloalto;login;panlogin > > paloalto;module;panos > > paloalto;inloop;panos::inloop > > paloalto;command;rancid::RunCommand;set cli scripting-mode on > > paloalto;command;rancid::RunCommand;set cli pager off > > paloalto;command;panos::ShowInfo;show system info > > paloalto;command;panos::ShowConfig;show config running > > > > With the PA-5050?s, started with the following lines in router.db: > > pa-1.example.com;paloalto;up;PA-5050 ha pair > > pa-2.example.com;paloalto;up;PA-5050 ha pair > > > > They?ve been getting the limited output because of the show config running > command and that they?re managed by Panorama. I altered the router.db file > to: > > pa-1.example.com;panw;up;PA-5050 ha pair > > pa-2.example.com;panw;up;PA-5050 ha pair > > > > I got the email that said the original devices were deleted and the new > devices were added. > > > > - pa-1.example.com;paloalto;up;PA-5050 > > - pa-2.example.com;panw;paloalto;up;PA-5050 > > + pa-1.example.com;panw;up;PA-5050 > > + pa-2.example.com;panw;panw;up;PA-5050 > > > > I checked the config files after running rancid again a couple times and > the config was unchanged. The output captured doesn?t seem to have > changed. Next, I troubleshot it by doing ?NOPIPE=yes rancid -d -t panw > pa-1.example.com? and reviewing the output. It captured everything > cleanly, as far as I can tell. No errors. It?s like the diff is not > catching the difference in output? > > > > What might I try next? > > > > --Chris > > > > > > *Chris**?* > > *Gauthier* > > Senior Network Engineer > > | > > Comscore > > t +1 *(503) 331-2704* <(503)%20331-2704> > > | > > *cgauthier at comscore.com* > > *comscore.com* > > ???This e-mail (including any attachments) may contain information that is > private, confidential, or protected by attorney-client or other privilege. > If you received this e-mail in error, please delete it from your system and > notify sender. > > *From: *annie lee > *Date: *Thursday, July 11, 2019 at 4:00 PM > *To: *"Gauthier, Chris" > *Cc: *john heasley , "Anderson, Charles R" < > cra at wpi.edu>, "rancid-discuss at shrubbery.net" > > *Subject: *Re: [rancid] Palo Alto (Panorama) configuration > > > > Hi Chris, > > > > Thats very kind of you to spend time doing that and thanks for that. > > > > Rgds > > > > On Fri, Jul 12, 2019 at 8:51 AM Gauthier, Chris > wrote: > > I?m working through that right now. > > > > *Chris**?* > > *Gauthier* > > Senior Network Engineer > > | > > Comscore > > t +1 *(503) 331-2704* <(503)%20331-2704> > > | > > *cgauthier at comscore.com* > > *comscore.com* > > ???This e-mail (including any attachments) may contain information that is > private, confidential, or protected by attorney-client or other privilege. > If you received this e-mail in error, please delete it from your system and > notify sender. > > *From: *annie lee > *Date: *Thursday, July 11, 2019 at 2:43 PM > *To: *"Gauthier, Chris" > *Cc: *john heasley , "Anderson, Charles R" < > cra at wpi.edu>, "rancid-discuss at shrubbery.net" > > *Subject: *Re: [rancid] Palo Alto (Panorama) configuration > > > > Thats good to know on the new cli (show config merged will grab everything > from the firewall and panorama). > > How do we add the cli and diff to rancid ?? > > > > On Fri, Jul 12, 2019 at 4:20 AM Gauthier, Chris > wrote: > > Just validated the ?show config merged? command works with any PA > firewall, managed by Panorama or not. > > > > *Chris**?* > > *Gauthier* > > Senior Network Engineer > > | > > Comscore > > t +1 *(503) 331-2704* <(503)%20331-2704> > > | > > *cgauthier at comscore.com* > > *comscore.com* > > ???This e-mail (including any attachments) may contain information that is > private, confidential, or protected by attorney-client or other privilege. > If you received this e-mail in error, please delete it from your system and > notify sender. > > *From: *Rancid-discuss on behalf > of "Gauthier, Chris" > *Date: *Thursday, July 11, 2019 at 11:16 AM > *To: *john heasley , "Anderson, Charles R" < > cra at wpi.edu> > *Cc: *"rancid-discuss at shrubbery.net" > *Subject: *Re: [rancid] Palo Alto (Panorama) configuration > > > > Yes, the command "show config merged" gives the locally-managed config > output AND the configuration that is pushed out by Panorama. I'll make a > custom device type and see how this works in my environment. If it works, > I'll post the results here. I will also test with a non-Panorama-managed > system. > > --Chris > > *Chris**?* > > *Gauthier* > > Senior Network Engineer > > | > > Comscore > > t +1 *(503) 331-2704* <(503)%20331-2704> > > | > > *cgauthier at comscore.com* > > *comscore.com* > > ???This e-mail (including any attachments) may contain information that is > private, confidential, or protected by attorney-client or other privilege. > If you received this e-mail in error, please delete it from your system and > notify sender. > > -----Original Message----- > From: Rancid-discuss on behalf of > john heasley > Date: Thursday, July 11, 2019 at 8:17 AM > To: "Anderson, Charles R" > Cc: "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Palo Alto (Panorama) configuration > > Thu, Jul 11, 2019 at 02:37:51PM +0000, Anderson, Charles R: > > You can use "show config merged" to see the local device's config merged > with the templates from Panorama. > > Does this work with "non-managed" (better term?) configs? And, was this > command introduced recently? > > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > > https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fwww.shrubbery.net%2fmailman%2flistinfo%2francid-discuss&c=E,1,ZBO_SpPdPN9F0GTa50thF3JK2iNVO_jcwwSZwho1q8BVBoP9LydezSjLupULi9-PCcBbEWhWi1x-kRvg-KGqTG6CANfUm1cA6XPL5VPANHGtvC7Gc3N4Pg4SarAO&typo=1 > > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > http://www.shrubbery.net/mailman/listinfo/rancid-discuss > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgauthier at comscore.com Mon Jul 15 17:55:14 2019 From: cgauthier at comscore.com (Gauthier, Chris) Date: Mon, 15 Jul 2019 17:55:14 +0000 Subject: [rancid] Palo Alto (Panorama) configuration In-Reply-To: References: <20190710214209.GD36475@shrubbery.net> <20190711143748.urpalsoan7pyq6nx@angus.ind.wpi.edu> <20190711151651.GB4422@shrubbery.net> <19611A45-0DE9-4E51-903F-7DFAE7040C40@comscore.com> <9995977F-1CCE-40CD-8E0B-EBE5B780EEB3@comscore.com> Message-ID: So, once again, cut and paste bit me?. My sincere apologies. Change the first line to read: panw;script;rancid -t panw Chris Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. From: annie lee Date: Friday, July 12, 2019 at 3:35 PM To: "Gauthier, Chris" Cc: "rancid-discuss at shrubbery.net" Subject: Re: [rancid] Palo Alto (Panorama) configuration Hi Chris, I've made similar chnages on v3.9 but not getting the new 'merged' config based on yours. Below are the panw code i added : panw;script;rancid -t paloalto panw;login;panlogin panw;module;panos panw;inloop;panos::inloop panw;command;panos::ShowInfo;show system info panw;command;panos::ShowInventory;show chassis inventory panw;command;panos::ShowConfig;show config merged Unfortunately still didnt captured the panorama configs. On Sat, Jul 13, 2019 at 3:58 AM Gauthier, Chris > wrote: So, if you look at my posting below, I made a rather dumb copy/paste error in my ?panw? definition. The first line should read: panw;script;rancid -t paloalto not: panw;script;rancid -t paloalto Thanks to Heasley for pointing that out! I would have not seen that for a while. Having changed the line as shown above, the ?show config merged? now works great on Panorama-managed and non-managed PA devices. --Chris Chris? Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. From: Rancid-discuss > on behalf of "Gauthier, Chris" > Date: Friday, July 12, 2019 at 9:24 AM To: annie lee > Cc: "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Palo Alto (Panorama) configuration I?m getting some interesting results in my testing. Rancid Version: 3.7 I have a pair of PA-5050?s managed by Panorama that have been only getting the ?show config running? output (the limited output). I made a new device type in etc/rancid.types.conf: panw;script;rancid -t paloalto panw;login;panlogin panw;module;panos panw;inloop;panos::inloop panw;command;rancid::RunCommand;set cli scripting-mode on panw;command;rancid::RunCommand;set cli pager off panw;command;panos::ShowInfo;show system info panw;command;panos::ShowConfig;show config merged This works well for my test unit (PA-220, unmanaged), but I am having problems with the PA-5050?s. For reference: Here is the device type of ?paloalto? in etc/rancid.types.base: paloalto;script;rancid -t paloalto paloalto;login;panlogin paloalto;module;panos paloalto;inloop;panos::inloop paloalto;command;rancid::RunCommand;set cli scripting-mode on paloalto;command;rancid::RunCommand;set cli pager off paloalto;command;panos::ShowInfo;show system info paloalto;command;panos::ShowConfig;show config running With the PA-5050?s, started with the following lines in router.db: pa-1.example.com;paloalto;up;PA-5050 ha pair pa-2.example.com;paloalto;up;PA-5050 ha pair They?ve been getting the limited output because of the show config running command and that they?re managed by Panorama. I altered the router.db file to: pa-1.example.com;panw;up;PA-5050 ha pair pa-2.example.com;panw;up;PA-5050 ha pair I got the email that said the original devices were deleted and the new devices were added. - pa-1.example.com;paloalto;up;PA-5050 - pa-2.example.com;panw;paloalto;up;PA-5050 + pa-1.example.com;panw;up;PA-5050 + pa-2.example.com;panw;panw;up;PA-5050 I checked the config files after running rancid again a couple times and the config was unchanged. The output captured doesn?t seem to have changed. Next, I troubleshot it by doing ?NOPIPE=yes rancid -d -t panw pa-1.example.com? and reviewing the output. It captured everything cleanly, as far as I can tell. No errors. It?s like the diff is not catching the difference in output? What might I try next? --Chris Chris? Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. From: annie lee > Date: Thursday, July 11, 2019 at 4:00 PM To: "Gauthier, Chris" > Cc: john heasley >, "Anderson, Charles R" >, "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Palo Alto (Panorama) configuration Hi Chris, Thats very kind of you to spend time doing that and thanks for that. Rgds On Fri, Jul 12, 2019 at 8:51 AM Gauthier, Chris > wrote: I?m working through that right now. Chris? Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. From: annie lee > Date: Thursday, July 11, 2019 at 2:43 PM To: "Gauthier, Chris" > Cc: john heasley >, "Anderson, Charles R" >, "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Palo Alto (Panorama) configuration Thats good to know on the new cli (show config merged will grab everything from the firewall and panorama). How do we add the cli and diff to rancid ?? On Fri, Jul 12, 2019 at 4:20 AM Gauthier, Chris > wrote: Just validated the ?show config merged? command works with any PA firewall, managed by Panorama or not. Chris? Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. From: Rancid-discuss > on behalf of "Gauthier, Chris" > Date: Thursday, July 11, 2019 at 11:16 AM To: john heasley >, "Anderson, Charles R" > Cc: "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Palo Alto (Panorama) configuration Yes, the command "show config merged" gives the locally-managed config output AND the configuration that is pushed out by Panorama. I'll make a custom device type and see how this works in my environment. If it works, I'll post the results here. I will also test with a non-Panorama-managed system. --Chris Chris? Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. -----Original Message----- From: Rancid-discuss > on behalf of john heasley > Date: Thursday, July 11, 2019 at 8:17 AM To: "Anderson, Charles R" > Cc: "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Palo Alto (Panorama) configuration Thu, Jul 11, 2019 at 02:37:51PM +0000, Anderson, Charles R: > You can use "show config merged" to see the local device's config merged with the templates from Panorama. Does this work with "non-managed" (better term?) configs? And, was this command introduced recently? _______________________________________________ Rancid-discuss mailing list Rancid-discuss at shrubbery.net https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fwww.shrubbery.net%2fmailman%2flistinfo%2francid-discuss&c=E,1,ZBO_SpPdPN9F0GTa50thF3JK2iNVO_jcwwSZwho1q8BVBoP9LydezSjLupULi9-PCcBbEWhWi1x-kRvg-KGqTG6CANfUm1cA6XPL5VPANHGtvC7Gc3N4Pg4SarAO&typo=1 _______________________________________________ Rancid-discuss mailing list Rancid-discuss at shrubbery.net http://www.shrubbery.net/mailman/listinfo/rancid-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From heas at shrubbery.net Mon Jul 15 22:00:30 2019 From: heas at shrubbery.net (john heasley) Date: Mon, 15 Jul 2019 22:00:30 +0000 Subject: [rancid] Restore a Palo Alto Firewall from a Rancid bacup In-Reply-To: References: <20190705174251.GG55957@shrubbery.net> <6863732B-B571-47F7-BE51-747924E4F76E@comscore.com> Message-ID: <20190715220030.GJ3992@shrubbery.net> Fri, Jul 12, 2019 at 09:18:34PM +0200, Erik Muller: > On 7/12/19 14:15 , Gauthier, Chris wrote: > > Rancid configs for PAN can NOT be used to restore the config, unless you > > cut and paste the configuration. This is because the native config files > > are stored in XML format and that is the format the Palo Alto utilities > > expect when performing restorations. > > Having recently needed to deal with a bunch of PAs, I ran into that same > issue and ended up writing a tool (https://github.com/ermuller/bracematch) > to simplify the process. > > RE the other question about Panorama vs device configs, if you're backing > up your Panorama configuration (which has been fine via Rancid in my How are you backing the Panorama configuration? is that just another rancid 'paloalto' target? > experience) as well as the base config on the device, you don't need to > backup the merged configuration. And you probably shouldn't pull the > merged config, for restore purposes, as anything other than the local > device configuration will come from the Panorama templates once the device > is replaced. Of course, the merged config might still be convenient to > save to easily see the complete policy set active on a given box. > > -e > > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > http://www.shrubbery.net/mailman/listinfo/rancid-discuss From cgauthier at comscore.com Mon Jul 15 22:30:42 2019 From: cgauthier at comscore.com (Gauthier, Chris) Date: Mon, 15 Jul 2019 22:30:42 +0000 Subject: [rancid] Restore a Palo Alto Firewall from a Rancid bacup In-Reply-To: <20190715220030.GJ3992@shrubbery.net> References: <20190705174251.GG55957@shrubbery.net> <6863732B-B571-47F7-BE51-747924E4F76E@comscore.com> <20190715220030.GJ3992@shrubbery.net> Message-ID: <893F6AFF-178A-48D3-AE90-4978698CC011@comscore.com> The only way in CLI to do a "show run" type of output in XML format is to execute the following commands. This holds true for both Panorama and Pan-OS (not managed by Panorama): User at Palo-Alto-FW> set cli config-output-format xml User at Palo-Alto-FW> configure Entering configuration mode [edit] User at Palo-Alto-FW# show ****Truncated to hide my config**** --Chris ? Chris Gauthier Senior Network Engineer | Comscore t +1 (503) 331-2704 | cgauthier at comscore.com comscore.com ???This e-mail (including any attachments) may contain information that is private, confidential, or protected by attorney-client or other privilege. If you received this e-mail in error, please delete it from your system and notify sender. -----Original Message----- From: Rancid-discuss on behalf of john heasley Date: Monday, July 15, 2019 at 3:00 PM To: Erik Muller Cc: "rancid-discuss at shrubbery.net" Subject: Re: [rancid] Restore a Palo Alto Firewall from a Rancid bacup Fri, Jul 12, 2019 at 09:18:34PM +0200, Erik Muller: > On 7/12/19 14:15 , Gauthier, Chris wrote: > > Rancid configs for PAN can NOT be used to restore the config, unless you > > cut and paste the configuration. This is because the native config files > > are stored in XML format and that is the format the Palo Alto utilities > > expect when performing restorations. > > Having recently needed to deal with a bunch of PAs, I ran into that same > issue and ended up writing a tool (https://github.com/ermuller/bracematch) > to simplify the process. > > RE the other question about Panorama vs device configs, if you're backing > up your Panorama configuration (which has been fine via Rancid in my How are you backing the Panorama configuration? is that just another rancid 'paloalto' target? > experience) as well as the base config on the device, you don't need to > backup the merged configuration. And you probably shouldn't pull the > merged config, for restore purposes, as anything other than the local > device configuration will come from the Panorama templates once the device > is replaced. Of course, the merged config might still be convenient to > save to easily see the complete policy set active on a given box. > > -e > > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fwww.shrubbery.net%2fmailman%2flistinfo%2francid-discuss&c=E,1,hdku7bLUQv7d0MAZOo8JrRXyca7FQEKjBwWLzlp0SJrUL-sb15koHXRbLiFA-stZLGQTyAvtcN8gShdbJ7Kpb47cHU_aXg5ZJBdwGDVSJSgIWDsF&typo=1 _______________________________________________ Rancid-discuss mailing list Rancid-discuss at shrubbery.net https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fwww.shrubbery.net%2fmailman%2flistinfo%2francid-discuss&c=E,1,bcAQYO-5xrzHw_0wfIv6Q3dm9-YAo8bMXWeVwZUulp3epd9ZkICII1QaJ_OJNdOV1XBK8gk0mx4wElmLp_3tZbcNWaLh8Q-9CLt0HJWGahly9knQqA,,&typo=1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From nineoften at hotmail.com Tue Jul 16 20:47:33 2019 From: nineoften at hotmail.com (Ni Ne) Date: Tue, 16 Jul 2019 20:47:33 +0000 Subject: [rancid] Anyone using RANCID against VMware vCenter/ESX servers to get network details? Message-ID: Curious if anyone has written modules for RANCID against VMware infrastructure. Would be pretty helpful to include them in the RANCID repos to show at network details of all hosts in your network. vmNICs in use, vSwitches, etc. -------------- next part -------------- An HTML attachment was scrubbed... URL: From heas at shrubbery.net Wed Jul 17 00:36:04 2019 From: heas at shrubbery.net (heasley) Date: Wed, 17 Jul 2019 00:36:04 +0000 Subject: [rancid] Getting a lot of noise related to ce_switch.log and ce_switch.log.bak In-Reply-To: References: <20180911163631.GB2325@shrubbery.net> Message-ID: <20190717003604.GG58685@shrubbery.net> Wed, Jul 10, 2019 at 01:39:34AM -0700, Dan Mahoney (Gushi): > On Tue, 11 Sep 2018, heasley wrote: > > > Mon, Sep 10, 2018 at 01:45:42AM -0700, Dan Mahoney (Gushi): > >> Hey all, > >> > >> I'm running Rancid built from freebsd packages, rancid3-3.7 > >> > >> Periodically, my ASR9K's log something like this: > >> > >> !Flash: harddisk: 24753 -rwx 800470016 Wed Sep 10 20:00:00 2014 > >> VM-ASR9K-px-4.3.4.tar > >> - !Flash: harddisk: 24623 -rw- > >> ce_switch.log > >> + !Flash: harddisk: 24781 -rw- 8192017 Mon Sep 10 05:10:03 2018 > >> ce_switch.log.bak > >> !Flash: harddisk: 24688 -rw- 1048576 Thu Sep 11 02:08:46 2014 > >> kd.bin_0_RSP0_CPU0 > >> !Flash: harddisk: 24625 drwx 4096 Thu Sep 11 01:38:55 2014 > >> idiags > >> !Flash: harddisk: 24626 -rw- 0 Thu Sep 11 01:40:24 2014 > >> ahci.log > >> !Flash: harddisk: 24627 drwx 4096 Thu Sep 11 02:20:32 2014 > >> np > >> - !Flash: harddisk: 24783 -rw- 8192017 Fri Sep 7 08:18:57 2018 > >> ce_switch.log.bak > >> + !Flash: harddisk: 24628 -rw- > >> ce_switch.log > >> !Flash: harddisk: 6442434560 bytes total (4 GB free) > >> > >> I thought I saw something on the mailing lists that this was fixed in a > >> prior version, but I guess not. How would I go about tweaking rancid so > >> these bits are ignored? > > > > add a filter to DirSlotN(). i see that your device is renaming files, > > causing the fileno to change. I'll add that filter for 3.9. > > Sorry to revive an old thread. > > I've upgraded to 3.9, but this doesn't seem to have been fixed: My mistake; I made this change to ios.pm, but did not also change iosxr.pm. I'll work on that change. From bjorn at basis-consulting.com Thu Jul 18 10:25:30 2019 From: bjorn at basis-consulting.com (=?UTF-8?Q?Bj=C3=B8rn_Skobba?=) Date: Thu, 18 Jul 2019 12:25:30 +0200 Subject: [rancid] Dell EMC S5200-ON series switches running OS10 Message-ID: Hi, first of all, I'm new to rancid and the list, so please bear with me :) I have a question regarding devices (in this case a S5296F-ON switch) running OS10 Network Operating System. We have quite a few Force10 S-series switches running FTOS which rancid happily pulls config from. The new S5200-series switches support only OS10 (and some 3rd party OS'es), and I have been struggling with getting rancid to pull config. I have tried different device types like dell, force10 and smc. Before digging deeper into the fine details; has anyone successfully gotten rancid to work with OS10 and can point me in the right direction? Many thanks Bj?rn -------------- next part -------------- An HTML attachment was scrubbed... URL: From heas at shrubbery.net Thu Jul 18 14:47:25 2019 From: heas at shrubbery.net (john heasley) Date: Thu, 18 Jul 2019 14:47:25 +0000 Subject: [rancid] Dell EMC S5200-ON series switches running OS10 In-Reply-To: References: Message-ID: <20190718144725.GB59336@shrubbery.net> Thu, Jul 18, 2019 at 12:25:30PM +0200, Bj?rn Skobba: > Hi, > first of all, I'm new to rancid and the list, so please bear with me :) > > I have a question regarding devices (in this case a S5296F-ON switch) > running OS10 Network Operating System. > > We have quite a few Force10 S-series switches running FTOS which rancid > happily pulls config from. The new S5200-series switches support only OS10 > (and some 3rd party OS'es), and I have been struggling with getting rancid > to pull config. > > I have tried different device types like dell, force10 and smc. > > Before digging deeper into the fine details; has anyone successfully gotten > rancid to work with OS10 and can point me in the right direction? I haven't seen one myself; but from the limited info I find on dell.com, it looks similar to the Fujitsu, with a different vocabulary. Perhaps try that, else contact me off list and I'll try to help. From heas at shrubbery.net Fri Jul 19 01:34:00 2019 From: heas at shrubbery.net (john heasley) Date: Fri, 19 Jul 2019 01:34:00 +0000 Subject: [rancid] Getting a lot of noise related to ce_switch.log and ce_switch.log.bak In-Reply-To: <20190718211104.3450B248F8B@sea.shrubbery.net> <20190717003604.GG58685@shrubbery.net> Message-ID: <20190719013400.GA63701@shrubbery.net> Wed, Jul 17, 2019 at 12:36:04AM +0000, heasley: > Wed, Jul 10, 2019 at 01:39:34AM -0700, Dan Mahoney (Gushi): > > On Tue, 11 Sep 2018, heasley wrote: > > > > > Mon, Sep 10, 2018 at 01:45:42AM -0700, Dan Mahoney (Gushi): > > >> Hey all, > > >> > > >> I'm running Rancid built from freebsd packages, rancid3-3.7 > > >> > > >> Periodically, my ASR9K's log something like this: > > >> > > >> !Flash: harddisk: 24753 -rwx 800470016 Wed Sep 10 20:00:00 2014 > > >> VM-ASR9K-px-4.3.4.tar > > >> - !Flash: harddisk: 24623 -rw- > > >> ce_switch.log > > >> + !Flash: harddisk: 24781 -rw- 8192017 Mon Sep 10 05:10:03 2018 > > >> ce_switch.log.bak > > >> !Flash: harddisk: 24688 -rw- 1048576 Thu Sep 11 02:08:46 2014 > > >> kd.bin_0_RSP0_CPU0 > > >> !Flash: harddisk: 24625 drwx 4096 Thu Sep 11 01:38:55 2014 > > >> idiags > > >> !Flash: harddisk: 24626 -rw- 0 Thu Sep 11 01:40:24 2014 > > >> ahci.log > > >> !Flash: harddisk: 24627 drwx 4096 Thu Sep 11 02:20:32 2014 > > >> np > > >> - !Flash: harddisk: 24783 -rw- 8192017 Fri Sep 7 08:18:57 2018 > > >> ce_switch.log.bak > > >> + !Flash: harddisk: 24628 -rw- > > >> ce_switch.log > > >> !Flash: harddisk: 6442434560 bytes total (4 GB free) > > >> > > >> I thought I saw something on the mailing lists that this was fixed in a > > >> prior version, but I guess not. How would I go about tweaking rancid so > > >> these bits are ignored? > > > > > > add a filter to DirSlotN(). i see that your device is renaming files, > > > causing the fileno to change. I'll add that filter for 3.9. > > > > Sorry to revive an old thread. > > > > I've upgraded to 3.9, but this doesn't seem to have been fixed: > > My mistake; I made this change to ios.pm, but did not also change iosxr.pm. > I'll work on that change. ftp://ftp.shrubbery.net/pub/rancid/alpha/rancid-3.9.99.tar.gz or diff --git a/CHANGES b/CHANGES index fbf20763..4139a17a 100644 --- a/CHANGES +++ b/CHANGES @@ -1,6 +1,4 @@ 3.9.99 - iosxr.pm: DirSlotN(): drop the file number from all files. - Missing Arista documentation - github.com/inphobia GC "procket" from manpages & README diff --git a/lib/iosxr.pm.in b/lib/iosxr.pm.in index 5c2e7008..1af4fd8e 100644 --- a/lib/iosxr.pm.in +++ b/lib/iosxr.pm.in @@ -555,7 +555,7 @@ sub DirSlotN { } # filter frequently changing files from IOX bootflash, hardiska, # and nvram - if ($dev =~ /(bootflash|disk[012]|harddisk|nvram)/) { + if ($dev =~ /(bootflash|disk0|harddisk|nvram)/) { if (/\s(\.python-history|aaa|\.bash_history)\s*$/ || /\s(ce_switch.log\S*|cisco_support|errmsg_cont)\s*$/ || /\s(genstr_cont|temp_cont|temp_cont|temp_static_data)\s*$/ || @@ -564,47 +564,50 @@ sub DirSlotN { # 57 -rw- 23100 volt_cont # 614788 drwx 4096 Fri Aug 20 12:06:25 2010 temp_cont # to - # -rw- volt_cont - # drwx temp_cont - if (/\s*\d+\s+(\S+\s+)(\d+)(\s+)()(\s+)/) { + # 57 -rw- volt_cont + # 614788 drwx temp_cont + if (/(\s*\d+\s+\S+\s+)(\d+)(\s+)()(\s+)/) { my($a, $sz, $c, $dt, $d, $rem) = ($1, $2, $3, $4, $5, $'); my($szl) = length($sz); my($fmt) = "%s%-". $szl ."s%s%s%s%s"; - $_ = sprintf($fmt, $c, $dt, $d, $rem); + $_ = sprintf($fmt, $a, "", $c, $dt, $d, $rem); ProcessHistory("FLASH","keysort",$rem,"!Flash: $dev: $_"); next; - } elsif (/\s*\d+\s+(\S+\s+\d+\s+)(\d+\s+\w+\s+\d+\s+\d+:\d+)/) { + } elsif (/(\s*\d+)(\s+\S+\s+\d+\s+)(\d+\s+\w+\s+\d+\s+\d+:\d+)/) { # XR >= 6.3; dir disk0:, but harddisk: is diff format. wtf # drop fileno size, & date. # " 8002 drwxr-xr-x 2 4096 Jan 17 15:27 np" - my($perm, $dt, $rem) = ($1, $2, $'); - my($dtl) = length($dt); - my($fmt) = "%s%-". $dtl ."s%s"; - $_ = sprintf($fmt, $perm, "", $rem); + my($fn, $perm, $dt, $rem) = ($1, $2, $3, $'); + my($fnl, $dtl) = (length($fn), length($dt)); + my($fmt) = "%-". $fnl ."s%s%-". $dtl ."s%s"; + $_ = sprintf($fmt, "", $perm, "", $rem); ProcessHistory("FLASH","keysort",$rem,"!Flash: $dev: $_"); next; - } elsif (/\s*\d+\s+(\S+\s+)(\d+)(\s+)(\w+ \w+\s+\d+ \d+:\d+:\d+ \d+)/) { - my($b, $sz, $c, $dt, $rem) = ($1, $2, $3, $4, $'); - my($szl, $dtl) = (length($sz), length($dt)); - my($fmt) = "%s%-". $szl ."s%s%-". $dtl ."s%s"; - $_ = sprintf($fmt, $b, "", $c, "", $rem); + } elsif (/(\s*)(\d+)(\s+\S+\s+)(\d+)(\s+)(\w+ \w+\s+\d+ \d+:\d+:\d+ \d+)/) { + my($a, $fn, $b, $sz, $c, $dt, $rem) = ($1, $2, $3, $4, $5, + $6, $'); + my($fnl, $szl, $dtl) = (length($fn), length($sz), + length($dt)); + my($fmt) = "%s%-". $fnl ."s%s%-". $szl ."s%s%-". $dtl ."s%s"; + $_ = sprintf($fmt, $a, "", $b, "", $c, "", $rem); ProcessHistory("FLASH","keysort",$rem,"!Flash: $dev: $_"); next; } } else { - if (/\s*\d+\s+(\S+\s+)(\d+)(\s+)(\s+)/) { - my($sz, $c, $dt, $d, $rem) = ($1, $2, $3, $4, $'); - ProcessHistory("FLASH","keysort",$rem,"!Flash: $dev: $sz$c$dt$d$rem"); + if (/(\s*\d+\s+\S+\s+)(\d+)(\s+)()(\s+)/) { + my($a, $sz, $c, $dt, $d, $rem) = ($1, $2, $3, $4, $5, $'); + ProcessHistory("FLASH","keysort",$rem,"!Flash: $dev: $_"); next; - } elsif (/\s*\d+\s+(\S+\s+\d+\s+)(\d+\s+\w+\s+\d+\s+\d+:\d+)/) { + } elsif (/(\s*\d+)(\s+\S+\s+\d+\s+)(\d+\s+\w+\s+\d+\s+\d+:\d+)/) { # XR >= 6.3; dir disk0:, but harddisk: is diff format. wtf - my($perm, $dt, $rem) = ($1, $2, $'); - ProcessHistory("FLASH","keysort",$rem,"!Flash: $dev: $perm$dt$rem"); + my($fn, $perm, $dt, $rem) = ($1, $2, $3, $'); + ProcessHistory("FLASH","keysort",$rem,"!Flash: $dev: $_"); next; - } elsif (/\s*\d+\s+(\S+\s+)(\d+)(\s+)(\w+ \w+\s+\d+ \d+:\d+:\d+ \d+)/) { + } elsif (/(\s*)(\d+)(\s+\S+\s+)(\d+)(\s+)(\w+ \w+\s+\d+ \d+:\d+:\d+ \d+)/) { # XR < 6.3 & etc. - my($b, $sz, $c, $dt, $rem) = ($1, $2, $3, $4, $'); - ProcessHistory("FLASH","keysort",$rem,"!Flash: $dev: $b$sz$c$dt$rem"); + my($a, $fn, $b, $sz, $c, $dt, $rem) = ($1, $2, $3, $4, $5, + $6, $'); + ProcessHistory("FLASH","keysort",$rem,"!Flash: $dev: $_"); next; } } From heas at shrubbery.net Fri Jul 19 20:32:40 2019 From: heas at shrubbery.net (john heasley) Date: Fri, 19 Jul 2019 20:32:40 +0000 Subject: [rancid] Restore a Palo Alto Firewall from a Rancid bacup In-Reply-To: <893F6AFF-178A-48D3-AE90-4978698CC011@comscore.com> References: <20190705174251.GG55957@shrubbery.net> <6863732B-B571-47F7-BE51-747924E4F76E@comscore.com> <20190715220030.GJ3992@shrubbery.net> <893F6AFF-178A-48D3-AE90-4978698CC011@comscore.com> Message-ID: <20190719203240.GA88216@shrubbery.net> Mon, Jul 15, 2019 at 10:30:42PM +0000, Gauthier, Chris: > The only way in CLI to do a "show run" type of output in XML format is to execute the following commands. This holds true for both Panorama and Pan-OS (not managed by Panorama): > > User at Palo-Alto-FW> set cli config-output-format xml > User at Palo-Alto-FW> configure > Entering configuration mode > [edit] > User at Palo-Alto-FW# show > > > > ****Truncated to hide my config**** > > --Chris I am confused; please help me understand so that we wrap-up this issue. There are two configs, the normal one in show config run, and one that comes from panorama config (if in use) that is visible on the "panorama clients" (my term) with show config merged. the panorama (master) offers a cli, just like a panorama client, where the panorama configuration can be viewed with 'show config run'. these configs can be dumped as xml or text. only xml can be loaded. Do i have all of this correct? I did not glean much useful info from the palo alto website. thanks > -----Original Message----- > From: Rancid-discuss on behalf of john heasley > Date: Monday, July 15, 2019 at 3:00 PM > To: Erik Muller > Cc: "rancid-discuss at shrubbery.net" > Subject: Re: [rancid] Restore a Palo Alto Firewall from a Rancid bacup > > Fri, Jul 12, 2019 at 09:18:34PM +0200, Erik Muller: > > On 7/12/19 14:15 , Gauthier, Chris wrote: > > > Rancid configs for PAN can NOT be used to restore the config, unless you > > > cut and paste the configuration. This is because the native config files > > > are stored in XML format and that is the format the Palo Alto utilities > > > expect when performing restorations. > > > > Having recently needed to deal with a bunch of PAs, I ran into that same > > issue and ended up writing a tool (https://github.com/ermuller/bracematch) > > to simplify the process. > > > > RE the other question about Panorama vs device configs, if you're backing > > up your Panorama configuration (which has been fine via Rancid in my > > How are you backing the Panorama configuration? is that just another > rancid 'paloalto' target? > > > experience) as well as the base config on the device, you don't need to > > backup the merged configuration. And you probably shouldn't pull the > > merged config, for restore purposes, as anything other than the local > > device configuration will come from the Panorama templates once the device > > is replaced. Of course, the merged config might still be convenient to > > save to easily see the complete policy set active on a given box. > > > > -e From erikm at buh.org Fri Jul 19 21:47:14 2019 From: erikm at buh.org (Erik Muller) Date: Fri, 19 Jul 2019 23:47:14 +0200 Subject: [rancid] Restore a Palo Alto Firewall from a Rancid bacup In-Reply-To: <20190715220030.GJ3992@shrubbery.net> References: <20190705174251.GG55957@shrubbery.net> <6863732B-B571-47F7-BE51-747924E4F76E@comscore.com> <20190715220030.GJ3992@shrubbery.net> Message-ID: On 7/16/19 0:00 , john heasley wrote: > Fri, Jul 12, 2019 at 09:18:34PM +0200, Erik Muller: >> On 7/12/19 14:15 , Gauthier, Chris wrote: >>> Rancid configs for PAN can NOT be used to restore the config, unless you >>> cut and paste the configuration. This is because the native config files >>> are stored in XML format and that is the format the Palo Alto utilities >>> expect when performing restorations. >> >> Having recently needed to deal with a bunch of PAs, I ran into that same >> issue and ended up writing a tool (https://github.com/ermuller/bracematch) >> to simplify the process. >> >> RE the other question about Panorama vs device configs, if you're backing >> up your Panorama configuration (which has been fine via Rancid in my > > How are you backing the Panorama configuration? is that just another > rancid 'paloalto' target? Exactly, the Panorama instance just looks like another PANOS device, with the same basic CLI. All the configuration rules and templates that are deployed to the managed devices are stored as just a normal part of the Panorama box's standard config, so from a rancid perspective it's just another normal paloalto box, and Just Works (AFAICT - I've not checked it closely, but it appears to be complete). -e >> experience) as well as the base config on the device, you don't need to >> backup the merged configuration. And you probably shouldn't pull the >> merged config, for restore purposes, as anything other than the local >> device configuration will come from the Panorama templates once the device >> is replaced. Of course, the merged config might still be convenient to >> save to easily see the complete policy set active on a given box. >> >> -e >> >> From erikm at buh.org Fri Jul 19 22:29:19 2019 From: erikm at buh.org (Erik Muller) Date: Sat, 20 Jul 2019 00:29:19 +0200 Subject: [rancid] Restore a Palo Alto Firewall from a Rancid bacup In-Reply-To: <20190719203240.GA88216@shrubbery.net> References: <20190705174251.GG55957@shrubbery.net> <6863732B-B571-47F7-BE51-747924E4F76E@comscore.com> <20190715220030.GJ3992@shrubbery.net> <893F6AFF-178A-48D3-AE90-4978698CC011@comscore.com> <20190719203240.GA88216@shrubbery.net> Message-ID: <7fb64eca-06e3-3e16-838f-72c5fbf066db@buh.org> On 7/19/19 22:32 , john heasley wrote: > Mon, Jul 15, 2019 at 10:30:42PM +0000, Gauthier, Chris: >> The only way in CLI to do a "show run" type of output in XML format is to execute the following commands. This holds true for both Panorama and Pan-OS (not managed by Panorama): >> >> User at Palo-Alto-FW> set cli config-output-format xml >> User at Palo-Alto-FW> configure >> Entering configuration mode >> [edit] >> User at Palo-Alto-FW# show >> >> >> >> ****Truncated to hide my config**** >> >> --Chris > > I am confused; please help me understand so that we wrap-up this issue. > > There are two configs, the normal one in show config run, and one that > comes from panorama config (if in use) that is visible on the "panorama > clients" (my term) with show config merged. Correct. Each PANOS device that's managed via Panorama has a local persistent configuration that includes device-specific things like local management address, HA-pair, user accounts... Panorama stores in it's config a bunch of rulesets and templates that can be applied to the managed devices; when it pushes those to a managed device they're merged at runtime into that device's live config, but not part of that box's actual local config. > the panorama (master) offers a cli, just like a panorama client, where > the panorama configuration can be viewed with 'show config run'. > > these configs can be dumped as xml or text. only xml can be loaded. > > Do i have all of this correct? I did not glean much useful info from the > palo alto website. all correct, TTBOMK. -e > thanks > >> -----Original Message----- >> From: Rancid-discuss on behalf of john heasley >> Date: Monday, July 15, 2019 at 3:00 PM >> To: Erik Muller >> Cc: "rancid-discuss at shrubbery.net" >> Subject: Re: [rancid] Restore a Palo Alto Firewall from a Rancid bacup >> >> Fri, Jul 12, 2019 at 09:18:34PM +0200, Erik Muller: >>> On 7/12/19 14:15 , Gauthier, Chris wrote: >>>> Rancid configs for PAN can NOT be used to restore the config, unless you >>>> cut and paste the configuration. This is because the native config files >>>> are stored in XML format and that is the format the Palo Alto utilities >>>> expect when performing restorations. >>> >>> Having recently needed to deal with a bunch of PAs, I ran into that same >>> issue and ended up writing a tool (https://github.com/ermuller/bracematch) >>> to simplify the process. >>> >>> RE the other question about Panorama vs device configs, if you're backing >>> up your Panorama configuration (which has been fine via Rancid in my >> >> How are you backing the Panorama configuration? is that just another >> rancid 'paloalto' target? >> >>> experience) as well as the base config on the device, you don't need to >>> backup the merged configuration. And you probably shouldn't pull the >>> merged config, for restore purposes, as anything other than the local >>> device configuration will come from the Panorama templates once the device >>> is replaced. Of course, the merged config might still be convenient to >>> save to easily see the complete policy set active on a given box. >>> >>> -e > From heas at shrubbery.net Sat Jul 20 15:09:55 2019 From: heas at shrubbery.net (john heasley) Date: Sat, 20 Jul 2019 15:09:55 +0000 Subject: [rancid] Restore a Palo Alto Firewall from a Rancid bacup In-Reply-To: <7fb64eca-06e3-3e16-838f-72c5fbf066db@buh.org> References: <20190705174251.GG55957@shrubbery.net> <6863732B-B571-47F7-BE51-747924E4F76E@comscore.com> <20190715220030.GJ3992@shrubbery.net> <893F6AFF-178A-48D3-AE90-4978698CC011@comscore.com> <20190719203240.GA88216@shrubbery.net> <7fb64eca-06e3-3e16-838f-72c5fbf066db@buh.org> Message-ID: <20190720150955.GD47274@shrubbery.net> Sat, Jul 20, 2019 at 12:29:19AM +0200, Erik Muller: > On 7/19/19 22:32 , john heasley wrote: > > Mon, Jul 15, 2019 at 10:30:42PM +0000, Gauthier, Chris: > >> The only way in CLI to do a "show run" type of output in XML format is to execute the following commands. This holds true for both Panorama and Pan-OS (not managed by Panorama): > >> > >> User at Palo-Alto-FW> set cli config-output-format xml > >> User at Palo-Alto-FW> configure > >> Entering configuration mode > >> [edit] > >> User at Palo-Alto-FW# show > >> > >> > >> > >> ****Truncated to hide my config**** > >> > >> --Chris > > > > I am confused; please help me understand so that we wrap-up this issue. > > > > There are two configs, the normal one in show config run, and one that > > comes from panorama config (if in use) that is visible on the "panorama > > clients" (my term) with show config merged. > > Correct. Each PANOS device that's managed via Panorama has a local > persistent configuration that includes device-specific things like local > management address, HA-pair, user accounts... > Panorama stores in it's config a bunch of rulesets and templates that can > be applied to the managed devices; when it pushes those to a managed device > they're merged at runtime into that device's live config, but not part of > that box's actual local config. > > > the panorama (master) offers a cli, just like a panorama client, where > > the panorama configuration can be viewed with 'show config run'. > > > > these configs can be dumped as xml or text. only xml can be loaded. > > > > Do i have all of this correct? I did not glean much useful info from the > > palo alto website. > > all correct, TTBOMK. > -e > Super; thanks. Is it sensible to collect all three? ie: the xml of the base, the base, and the merged. > > > >> -----Original Message----- > >> From: Rancid-discuss on behalf of john heasley > >> Date: Monday, July 15, 2019 at 3:00 PM > >> To: Erik Muller > >> Cc: "rancid-discuss at shrubbery.net" > >> Subject: Re: [rancid] Restore a Palo Alto Firewall from a Rancid bacup > >> > >> Fri, Jul 12, 2019 at 09:18:34PM +0200, Erik Muller: > >>> On 7/12/19 14:15 , Gauthier, Chris wrote: > >>>> Rancid configs for PAN can NOT be used to restore the config, unless you > >>>> cut and paste the configuration. This is because the native config files > >>>> are stored in XML format and that is the format the Palo Alto utilities > >>>> expect when performing restorations. > >>> > >>> Having recently needed to deal with a bunch of PAs, I ran into that same > >>> issue and ended up writing a tool (https://github.com/ermuller/bracematch) > >>> to simplify the process. > >>> > >>> RE the other question about Panorama vs device configs, if you're backing > >>> up your Panorama configuration (which has been fine via Rancid in my > >> > >> How are you backing the Panorama configuration? is that just another > >> rancid 'paloalto' target? > >> > >>> experience) as well as the base config on the device, you don't need to > >>> backup the merged configuration. And you probably shouldn't pull the > >>> merged config, for restore purposes, as anything other than the local > >>> device configuration will come from the Panorama templates once the device > >>> is replaced. Of course, the merged config might still be convenient to > >>> save to easily see the complete policy set active on a given box. > >>> > >>> -e > > From vv.corto at gmail.com Mon Jul 22 15:51:09 2019 From: vv.corto at gmail.com (Florin Vlad Olariu) Date: Mon, 22 Jul 2019 08:51:09 -0700 Subject: [rancid] Rancid and the Cisco 5000 Nexus Platform Message-ID: Hello, I have some cisco Nexus 5k and I'm having some trouble grabbing the "show run" through rancid. In my setup I commented out most commands in the "rancid.types.base" file except for the "show run" section. The problem is that with the file commented, rancid can't manage to grab the output because, according to the logs, "End of run not found". Is this message based on finding the word "end" in the configuration? Because if that's the requirement, then even when manually doing "show run" it's not there. The curious thing is that if I un-comment all the other show commands, then rancid does manage to grab the router config, although of course that is not ideal. Below [1] you can find the "rancid.types.base" config. Connectivity to the device is NOT a problem. Version used is both 7.x and 5.x trains and both have problems. Thank you! [1] cisco-nx;script;rancid -t cisco-nx cisco-nx;login;clogin cisco-nx;module;nxos cisco-nx;inloop;nxos::inloop #cisco-nx;command;rancid::RunCommand;term no monitor-force #cisco-nx;command;nxos::ShowVersion;show version #cisco-nx;command;nxos::ShowVersionBuild;show version build-info all #cisco-nx;command;nxos::ShowLicense;show license #cisco-nx;command;nxos::ShowLicense;show license usage #cisco-nx;command;nxos::ShowLicense;show license host-id #cisco-nx;command;nxos::ShowRedundancy;show system redundancy status #cisco-nx;command;nxos::ShowEnv;show environment clock #cisco-nx;command;nxos::ShowEnv;show environment fan #cisco-nx;command;nxos::ShowEnv;show environment fex all fan #cisco-nx;command;nxos::ShowEnvTemp;show environment temperature #cisco-nx;command;nxos::ShowEnvPower;show environment power #cisco-nx;command;nxos::ShowBoot;show boot #cisco-nx;command;nxos::DirSlotN;dir bootflash: #cisco-nx;command;nxos::DirSlotN;dir debug: #cisco-nx;command;nxos::DirSlotN;dir logflash: #cisco-nx;command;nxos::DirSlotN;dir slot0: #cisco-nx;command;nxos::DirSlotN;dir usb1: #cisco-nx;command;nxos::DirSlotN;dir usb2: #cisco-nx;command;nxos::DirSlotN;dir volatile: #cisco-nx;command;nxos::ShowModule;show module #cisco-nx;command;nxos::ShowModule;show module xbar ##cisco-nx;command;nxos::ShowModule;show module X;add, but wait for show all ##cisco-nx;command;nxos::ShowModule;show module X epld;add, but wait for show all #cisco-nx;command;nxos::ShowInventory;show inventory #cisco-nx;command;nxos::ShowIntTransceiver;show interface transceiver #cisco-nx;command;nxos::ShowVTP;show vtp status;drop? #cisco-nx;command;nxos::ShowVLAN;show vlan #cisco-nx;command;nxos::ShowDebug;show debug #cisco-nx;command;nxos::ShowCores;show cores vdc-all #cisco-nx;command;nxos::ShowProcLog;show processes log vdc-all #cisco-nx;command;nxos::ShowFex;show module fex #cisco-nx;command;nxos::ShowFex;show fex cisco-nx;command;nxos::WriteTerm;show running-config -------------- next part -------------- An HTML attachment was scrubbed... URL: From heas at shrubbery.net Mon Jul 22 18:59:06 2019 From: heas at shrubbery.net (john heasley) Date: Mon, 22 Jul 2019 18:59:06 +0000 Subject: [rancid] Rancid and the Cisco 5000 Nexus Platform In-Reply-To: References: Message-ID: <20190722185906.GF58503@shrubbery.net> Mon, Jul 22, 2019 at 08:51:09AM -0700, Florin Vlad Olariu: > Hello, > > I have some cisco Nexus 5k and I'm having some trouble grabbing the "show > run" through rancid. In my setup I commented out most commands in the > "rancid.types.base" file except for the "show run" section. The problem is > that with the file commented, rancid can't manage to grab the output > because, according to the logs, "End of run not found". Is this message > based on finding the word "end" in the configuration? Because if that's the > requirement, then even when manually doing "show run" it's not there. please show us the error from the log file and tell us what version of rancid. also, please follow the test in the FAQ S3 Q2. Also, for some devices show version is required; as the device type can affect other parsing. I doubt that is the problem for nxos, but you also commented this: > #cisco-nx;command;rancid::RunCommand;term no monitor-force which i suspect is the problem, having now seen the errors. > The curious thing is that if I un-comment all the other show commands, then > rancid does manage to grab the router config, although of course that is > not ideal. Below [1] you can find the "rancid.types.base" config. > > cisco-nx;script;rancid -t cisco-nx please read the warning at the top of etc/rancid.types.base From dennis.jasch at anticlockwise.com.au Tue Jul 23 03:53:39 2019 From: dennis.jasch at anticlockwise.com.au (Dennis Jasch) Date: Tue, 23 Jul 2019 03:53:39 +0000 Subject: [rancid] Rancid with Dell PowerConnect M8024-k Message-ID: Hi, I have been Googleing a lot on how to get this to work, but had no luck yet. Observium version: 19.7.9977 Rancid version: 3.9 Device: Dell PowerConnect M8024-k The Observium PHP script to generate the rancid router.db classifies the switch as "dell" - is this correct? I'm lead to believe it may have to be "smc". I have tried both, but neither seems to successfully pull the config. The process seems to just hang indefinitely. Testing using: /opt/rancid/bin/clogin -c"show version;" 10.x.x.x seems to work correctly. Logs seem to suggest: 10.x.x.x: End of run not found Any suggestions would be greatly appreciated. Regards, Dennis. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vv.corto at gmail.com Tue Jul 23 08:29:01 2019 From: vv.corto at gmail.com (Florin Vlad Olariu) Date: Tue, 23 Jul 2019 01:29:01 -0700 Subject: [rancid] Rancid and the Cisco 5000 Nexus Platform In-Reply-To: <20190722185906.GF58503@shrubbery.net> References: <20190722185906.GF58503@shrubbery.net> Message-ID: Hi John, thanks for the reply. I am running version 3.9 [2] and the logs looks like in [1]. I tried un-commenting the line that states "#cisco-nx;command;rancid::RunCommand;term no monitor-force" but it doesn't work anyway. Reading about your comment on "show version" made me try and un-comment that line... (and only that line) and after that it worked!. But why do I need to have show version in there at all for this to properly work? An alternative solution I had was to put variables "$clean_run" and "$found_end" to 1 in the /usr/local/rancid/bin/rancid file, but of course this is not ideal as it applies to all types of routers. Any idea how can I gather config _without_ needing "show version" also? [1] [rancid at tvvsmtarist001 logs]$ cat rancidconf.20190722.143428 starting: Mon Jul 22 14:34:28 UTC 2019 Trying to get all of the configs. tlcxx-mgmt-001a.mlp.com: End of run not found ===================================== Getting missed routers: round 1. tlcxx-mgmt-001a.mlp.com: End of run not found ===================================== Getting missed routers: round 2. tlcxx-mgmt-001a.mlp.com: End of run not found ===================================== Getting missed routers: round 3. tlcxx-mgmt-001a.mlp.com: End of run not found ===================================== Getting missed routers: round 4. tlcxx-mgmt-001a.mlp.com: End of run not found [2] [rancid at tvvsmtarist001 logs]$ /usr/local/rancid/bin/rancid-run -V rancid 3.9 On 22 July 2019 at 19:59:07, john heasley (heas at shrubbery.net) wrote: Mon, Jul 22, 2019 at 08:51:09AM -0700, Florin Vlad Olariu: > Hello, > > I have some cisco Nexus 5k and I'm having some trouble grabbing the "show > run" through rancid. In my setup I commented out most commands in the > "rancid.types.base" file except for the "show run" section. The problem is > that with the file commented, rancid can't manage to grab the output > because, according to the logs, "End of run not found". Is this message > based on finding the word "end" in the configuration? Because if that's the > requirement, then even when manually doing "show run" it's not there. please show us the error from the log file and tell us what version of rancid. also, please follow the test in the FAQ S3 Q2. Also, for some devices show version is required; as the device type can affect other parsing. I doubt that is the problem for nxos, but you also commented this: > #cisco-nx;command;rancid::RunCommand;term no monitor-force which i suspect is the problem, having now seen the errors. > The curious thing is that if I un-comment all the other show commands, then > rancid does manage to grab the router config, although of course that is > not ideal. Below [1] you can find the "rancid.types.base" config. > > cisco-nx;script;rancid -t cisco-nx please read the warning at the top of etc/rancid.types.base -------------- next part -------------- An HTML attachment was scrubbed... URL: From heas at shrubbery.net Tue Jul 23 13:36:34 2019 From: heas at shrubbery.net (john heasley) Date: Tue, 23 Jul 2019 13:36:34 +0000 Subject: [rancid] Rancid with Dell PowerConnect M8024-k In-Reply-To: References: Message-ID: <20190723133634.GA32130@shrubbery.net> Tue, Jul 23, 2019 at 03:53:39AM +0000, Dennis Jasch: > Hi, > > I have been Googleing a lot on how to get this to work, but had no luck yet. > > Observium version: 19.7.9977 > Rancid version: 3.9 > Device: Dell PowerConnect M8024-k > > The Observium PHP script to generate the rancid router.db classifies the switch as "dell" - is this correct? I'm lead to believe it may have to be "smc". I can not say, I do not know this device and dell OEMs all of their switch h/w, except perhaps white box h/w. If the cli and config look like another device type in rancid, then that type will likely work. Else, perhaps show an example of the cli and config to the list. smc would be my guess as well. so, maybe show us the errors and try the debug procedure from the FAQ S3 Q2. If you discover one, please lmk and I will document it in rancid.types.base along with the others. > I have tried both, but neither seems to successfully pull the config. The process seems to just hang indefinitely. > > Testing using: /opt/rancid/bin/clogin -c"show version;" 10.x.x.x seems to work correctly. > > Logs seem to suggest: > 10.x.x.x: End of run not found > > Any suggestions would be greatly appreciated. > > Regards, > Dennis. > > > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > http://www.shrubbery.net/mailman/listinfo/rancid-discuss From heas at shrubbery.net Tue Jul 23 17:42:20 2019 From: heas at shrubbery.net (john heasley) Date: Tue, 23 Jul 2019 17:42:20 +0000 Subject: [rancid] Rancid and the Cisco 5000 Nexus Platform In-Reply-To: References: <20190722185906.GF58503@shrubbery.net> Message-ID: <20190723174220.GB28688@shrubbery.net> Tue, Jul 23, 2019 at 01:29:01AM -0700, Florin Vlad Olariu: > I am running version 3.9 [2] and the logs looks like in [1]. I tried > un-commenting the line that states > "#cisco-nx;command;rancid::RunCommand;term no monitor-force" but it doesn't > work anyway. keep that; it prevents logs/etc from mangling prompts and commands that rancid wants to match. > Reading about your comment on "show version" made me try and un-comment > that line... (and only that line) and after that it worked!. But why do I > need to have show version in there at all for this to properly work? > > An alternative solution I had was to put variables "$clean_run" and > "$found_end" to 1 in the /usr/local/rancid/bin/rancid file, but of course > this is not ideal as it applies to all types of routers. > > Any idea how can I gather config _without_ needing "show version" also? As I mentioned, the model sometimes affects the handling of the config. I do not remember off the top why this is so in nxos. i'll try to look later. its not that much extra data and it should all be commented. From ryan.g at atwgpc.net Wed Jul 24 00:09:29 2019 From: ryan.g at atwgpc.net (Ryan Gelobter) Date: Tue, 23 Jul 2019 19:09:29 -0500 Subject: [rancid] Rancid with Dell PowerConnect M8024-k In-Reply-To: <20190723133634.GA32130@shrubbery.net> References: <20190723133634.GA32130@shrubbery.net> Message-ID: We have around 100 Dell M8024-k switches and use device type smc with no issues. Regards, Ryan On Tue, Jul 23, 2019 at 8:36 AM john heasley wrote: > Tue, Jul 23, 2019 at 03:53:39AM +0000, Dennis Jasch: > > Hi, > > > > I have been Googleing a lot on how to get this to work, but had no luck > yet. > > > > Observium version: 19.7.9977 > > Rancid version: 3.9 > > Device: Dell PowerConnect M8024-k > > > > The Observium PHP script to generate the rancid router.db classifies the > switch as "dell" - is this correct? I'm lead to believe it may have to be > "smc". > > I can not say, I do not know this device and dell OEMs all of their > switch h/w, except perhaps white box h/w. If the cli and config look > like another device type in rancid, then that type will likely work. > Else, perhaps show an example of the cli and config to the list. > > smc would be my guess as well. so, maybe show us the errors and try > the debug procedure from the FAQ S3 Q2. > > If you discover one, please lmk and I will document it in rancid.types.base > along with the others. > > > I have tried both, but neither seems to successfully pull the config. > The process seems to just hang indefinitely. > > > > Testing using: /opt/rancid/bin/clogin -c"show version;" 10.x.x.x seems > to work correctly. > > > > Logs seem to suggest: > > 10.x.x.x: End of run not found > > > > Any suggestions would be greatly appreciated. > > > > Regards, > > Dennis. > > > > > > > _______________________________________________ > > Rancid-discuss mailing list > > Rancid-discuss at shrubbery.net > > http://www.shrubbery.net/mailman/listinfo/rancid-discuss > > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > http://www.shrubbery.net/mailman/listinfo/rancid-discuss > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dennis.jasch at anticlockwise.com.au Wed Jul 24 00:51:34 2019 From: dennis.jasch at anticlockwise.com.au (Dennis Jasch) Date: Wed, 24 Jul 2019 00:51:34 +0000 Subject: [rancid] Rancid with Dell PowerConnect M8024-k In-Reply-To: References: <20190723133634.GA32130@shrubbery.net>, Message-ID: Hi, may I ask, if that is via telnet or SSH? And which version of rancid you are using? Autoenable is set o 0, I presume? Regards, Dennis. ________________________________ From: Ryan Gelobter Sent: Wednesday, 24 July 2019 10:09 AM To: john heasley Cc: Dennis Jasch ; rancid-discuss at shrubbery.net Subject: Re: [rancid] Rancid with Dell PowerConnect M8024-k EXTERNAL: This email originated from outside Anticlockwise. We have around 100 Dell M8024-k switches and use device type smc with no issues. Regards, Ryan On Tue, Jul 23, 2019 at 8:36 AM john heasley > wrote: Tue, Jul 23, 2019 at 03:53:39AM +0000, Dennis Jasch: > Hi, > > I have been Googleing a lot on how to get this to work, but had no luck yet. > > Observium version: 19.7.9977 > Rancid version: 3.9 > Device: Dell PowerConnect M8024-k > > The Observium PHP script to generate the rancid router.db classifies the switch as "dell" - is this correct? I'm lead to believe it may have to be "smc". I can not say, I do not know this device and dell OEMs all of their switch h/w, except perhaps white box h/w. If the cli and config look like another device type in rancid, then that type will likely work. Else, perhaps show an example of the cli and config to the list. smc would be my guess as well. so, maybe show us the errors and try the debug procedure from the FAQ S3 Q2. If you discover one, please lmk and I will document it in rancid.types.base along with the others. > I have tried both, but neither seems to successfully pull the config. The process seems to just hang indefinitely. > > Testing using: /opt/rancid/bin/clogin -c"show version;" 10.x.x.x seems to work correctly. > > Logs seem to suggest: > 10.x.x.x: End of run not found > > Any suggestions would be greatly appreciated. > > Regards, > Dennis. > > > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > http://www.shrubbery.net/mailman/listinfo/rancid-discuss _______________________________________________ Rancid-discuss mailing list Rancid-discuss at shrubbery.net http://www.shrubbery.net/mailman/listinfo/rancid-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From ryan.g at atwgpc.net Wed Jul 24 11:43:53 2019 From: ryan.g at atwgpc.net (Ryan Gelobter) Date: Wed, 24 Jul 2019 06:43:53 -0500 Subject: [rancid] Rancid with Dell PowerConnect M8024-k In-Reply-To: References: <20190723133634.GA32130@shrubbery.net> Message-ID: Its through SSH. We use RADIUS so upon logging into the switches you're already at the enable prompt. So I do have autoenable set to 1. I'm using rancid 3.7 and 3.4. I just setup 3.9 on a different machine and copied my routers.db file over along with the .cloginrc and it worked without issue. For giggles I set autoenable to 0 and changed my ssh configuration on the switch to remove autoenable and specified an enable password in .cloginrc. RANCID ran and backed up without issue. Whats in your rancid logs? What version of the switch are you running? The reason you should use smc over dell is because the show run command the dell device type uses doesn't work on the m8024k. But also because in rancid.types.base it says to use it for the m8024k :P On Tue, Jul 23, 2019 at 7:51 PM Dennis Jasch < dennis.jasch at anticlockwise.com.au> wrote: > Hi, > > may I ask, if that is via telnet or SSH? And which version of rancid you > are using? Autoenable is set o 0, I presume? > > Regards, > Dennis. > > ------------------------------ > *From:* Ryan Gelobter > *Sent:* Wednesday, 24 July 2019 10:09 AM > *To:* john heasley > *Cc:* Dennis Jasch ; > rancid-discuss at shrubbery.net > *Subject:* Re: [rancid] Rancid with Dell PowerConnect M8024-k > > > *EXTERNAL:* This email originated from outside Anticlockwise. > We have around 100 Dell M8024-k switches and use device type smc with no > issues. > > Regards, > Ryan > > On Tue, Jul 23, 2019 at 8:36 AM john heasley wrote: > > Tue, Jul 23, 2019 at 03:53:39AM +0000, Dennis Jasch: > > Hi, > > > > I have been Googleing a lot on how to get this to work, but had no luck > yet. > > > > Observium version: 19.7.9977 > > Rancid version: 3.9 > > Device: Dell PowerConnect M8024-k > > > > The Observium PHP script to generate the rancid router.db classifies the > switch as "dell" - is this correct? I'm lead to believe it may have to be > "smc". > > I can not say, I do not know this device and dell OEMs all of their > switch h/w, except perhaps white box h/w. If the cli and config look > like another device type in rancid, then that type will likely work. > Else, perhaps show an example of the cli and config to the list. > > smc would be my guess as well. so, maybe show us the errors and try > the debug procedure from the FAQ S3 Q2. > > If you discover one, please lmk and I will document it in rancid.types.base > along with the others. > > > I have tried both, but neither seems to successfully pull the config. > The process seems to just hang indefinitely. > > > > Testing using: /opt/rancid/bin/clogin -c"show version;" 10.x.x.x seems > to work correctly. > > > > Logs seem to suggest: > > 10.x.x.x: End of run not found > > > > Any suggestions would be greatly appreciated. > > > > Regards, > > Dennis. > > > > > > > _______________________________________________ > > Rancid-discuss mailing list > > Rancid-discuss at shrubbery.net > > http://www.shrubbery.net/mailman/listinfo/rancid-discuss > > > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > http://www.shrubbery.net/mailman/listinfo/rancid-discuss > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dennis.jasch at anticlockwise.com.au Wed Jul 24 11:57:04 2019 From: dennis.jasch at anticlockwise.com.au (Dennis Jasch) Date: Wed, 24 Jul 2019 11:57:04 +0000 Subject: [rancid] Fw: Rancid with Dell PowerConnect M8024-k In-Reply-To: References: <20190723133634.GA32130@shrubbery.net> , <20190723175640.GD28688@shrubbery.net>, Message-ID: Hi, responding to Ryan Gelobter ? Yes, it seems SSH works. Guess the main source of confusion is the Observium PHP script generating them as Dell, rather than SMC. Had sure seen the comment in the source as well, but previously the switches were accessed via telnet, which seems problematic (at least for me) D ________________________________ From: Dennis Jasch Sent: Wednesday, 24 July 2019 11:40 AM To: john heasley Subject: Re: [rancid] Rancid with Dell PowerConnect M8024-k Hi, I have tested dllogin, hlogin, and clogin, each with telnet and SSH * telnet * dllogin does not authenticate * hlogin does not authenticate * clogin works as expected * SSH * dllogin does not authenticate * hlogin works as expected * clogin works as expected I attached test output. I was able to get it to work all the way through via SSH and smc. I guess the Observium PHP script to generate router.db might need an adjustment to output this device as smc and not dell. Regards, Dennis. ________________________________ From: john heasley Sent: Wednesday, 24 July 2019 3:56 AM To: Dennis Jasch Subject: Re: [rancid] Rancid with Dell PowerConnect M8024-k EXTERNAL: This email originated from outside Anticlockwise. Tue, Jul 23, 2019 at 02:26:34PM +0000, Dennis Jasch: > Hi, > > thanks for the response! > > It seems the process is getting stuck at dllogin: > > rancid at observium:/opt/rancid/bin$ ./clogin -t 300 -c "show switch;" 10.253.249.55 > 10.253.249.55 > spawn ssh -x -l acadmin 10.253.249.55 > acadmin at 10.253.249.55's password: > > sy2-blade-chassis-01-sw1>enable > Password:********* > > sy2-blade-chassis-01-sw1# > sy2-blade-chassis-01-sw1#terminal length 0 > > sy2-blade-chassis-01-sw1#terminal width 132 > ^ > % Invalid input detected at '^' marker. > > sy2-blade-chassis-01-sw1#show switch > > Management Standby Preconfig Plugged-in Switch Code > SW Status Status Model ID Model ID Status Version > --- ---------- --------- ------------- ------------- ------------- ----------- > 1 Stack Mbr Oper Stby PCM8024-k PCM8024-k OK 5.1.0.1 > 2 Mgmt Sw PCM8024-k PCM8024-k OK 5.1.0.1 > > sy2-blade-chassis-01-sw1# > sy2-blade-chassis-01-sw1#exit > > sy2-blade-chassis-01-sw1>exitConnection to 10.253.249.55 closed. > rancid at observium:/opt/rancid/bin$ > rancid at observium:/opt/rancid/bin$ > rancid at observium:/opt/rancid/bin$ ./dllogin -t 300 -c "show switch;" 10.253.249.55 > 10.253.249.55 > spawn ssh -x -l acadmin 10.253.249.55 > acadmin at 10.253.249.55's password: > > sy2-blade-chassis-01-sw1> > > >From here on it just times out eventually. > > clogin seems to work just fine. That device is a switch card in a blade chassis, and is fairly "Cisco-esk". So, I would assume dllogin is not quite compatible. that sure sounds like smc. Can you test with hlogin, which is what smc uses? > I had been trying to create a new type in rancid.types.conf, and tell it to use regular clogin - but have not managed to do so yet - any pointers would be highly appreciated. this depends upon the module; the older monolithic modules like smc have the login program hard-coded, but the new ones that have 'command' lines in etc/rancid.types.* utilize the login definition therein. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: test cases.txt URL: From vv.corto at gmail.com Thu Jul 25 10:57:11 2019 From: vv.corto at gmail.com (Florin Vlad Olariu) Date: Thu, 25 Jul 2019 03:57:11 -0700 Subject: [rancid] Rancid and the Cisco 5000 Nexus Platform In-Reply-To: <20190723174220.GB28688@shrubbery.net> References: <20190722185906.GF58503@shrubbery.net> <20190723174220.GB28688@shrubbery.net> Message-ID: Hey John, So I poked around various rancid files. I discovered that the "/usr/local/rancid/lib/rancid/nxos.pm" file/module sets a "type" variable in the "ShowVersion" function which is then checked at the end (see [1]) to test if we're on a NXOS platform. This module acknowledges the fact that NXOS does not have an "end" marker, but I'm not sure why a "nxos.pm" module would need to test if this is a nexus platform? Or at least, why bundle it in the "ShowVersion" function without the option to include that output in the config collected or not? I'm sure there's some historical reason for it, just don't have the context. For my particular case, I commented out line 1021 and left only the $linecnt check in. This way I can get rid of "show version" output in my config file and it affects only cisco-nx. This is clearly a hack that will make upgrading more difficult, but my knowledge of perl is basically 0 so can't really propose a more sane thing here :(. Thanks for your input. Hope somebody finds this hack useful. [1] 1018 # The ContentEngine lacks a definitive "end of config" marker. If we 1019 # know that it is NXOS and we have seen at least 5 lines 1020 # of write term output, we can be reasonably sure that we got the config. 1021 # if (($type eq "NXOS") && $linecnt > 5) { 1022 if ($linecnt > 5) { On 23 July 2019 at 18:42:21, john heasley (heas at shrubbery.net) wrote: Tue, Jul 23, 2019 at 01:29:01AM -0700, Florin Vlad Olariu: > I am running version 3.9 [2] and the logs looks like in [1]. I tried > un-commenting the line that states > "#cisco-nx;command;rancid::RunCommand;term no monitor-force" but it doesn't > work anyway. keep that; it prevents logs/etc from mangling prompts and commands that rancid wants to match. > Reading about your comment on "show version" made me try and un-comment > that line... (and only that line) and after that it worked!. But why do I > need to have show version in there at all for this to properly work? > > An alternative solution I had was to put variables "$clean_run" and > "$found_end" to 1 in the /usr/local/rancid/bin/rancid file, but of course > this is not ideal as it applies to all types of routers. > > Any idea how can I gather config _without_ needing "show version" also? As I mentioned, the model sometimes affects the handling of the config. I do not remember off the top why this is so in nxos. i'll try to look later. its not that much extra data and it should all be commented. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vv.corto at gmail.com Thu Jul 25 12:29:37 2019 From: vv.corto at gmail.com (Florin Vlad Olariu) Date: Thu, 25 Jul 2019 14:29:37 +0200 Subject: [rancid] Improving Rancid's processing speed when having 1k+ devices Message-ID: Well, as per title, is there any way to improve rancid's speed with so many devices? At the moment I set PAR_COUNT to 300, so it will connect in parallel to 300 devices at a time, but the reality is that most time does not seem to be taken by connecting and retrieving config but by what happens next in the file processing and git-comitting. To give you some stats, with current settings it takes around 9 minutes to do 1200 devices. I have only 1 group with all devices under the same group. Any trick you might have, please let me know! Thanks, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From emille at abccommunications.com Thu Jul 25 15:14:28 2019 From: emille at abccommunications.com (Emille Blanc) Date: Thu, 25 Jul 2019 08:14:28 -0700 Subject: [rancid] Improving Rancid's processing speed when having 1k+ devices In-Reply-To: References: Message-ID: <4FBAFC2ECF5D6244BA4A26C1C94A1E271528FFD3F3@exchange> I've seen/heard stories of people pre-empting rancid with an snmp-get of the config-last-changed / last committed OID, to generate a list of devices to run against. Have always wanted to set that up for our instance as we are approaching the 500 device mark, but it's not become a big enough problem for us... yet. From: Rancid-discuss [mailto:rancid-discuss-bounces at shrubbery.net] On Behalf Of Florin Vlad Olariu Sent: Thursday, July 25, 2019 5:30 AM To: rancid-discuss at shrubbery.net Subject: [rancid] Improving Rancid's processing speed when having 1k+ devices Well, as per title, is there any way to improve rancid's speed with so many devices? At the moment I set PAR_COUNT to 300, so it will connect in parallel to 300 devices at a time, but the reality is that most time does not seem to be taken by connecting and retrieving config but by what happens next in the file processing and git-comitting. To give you some stats, with current settings it takes around 9 minutes to do 1200 devices. I have only 1 group with all devices under the same group. Any trick you might have, please let me know! Thanks, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From heas at shrubbery.net Thu Jul 25 16:38:52 2019 From: heas at shrubbery.net (john heasley) Date: Thu, 25 Jul 2019 16:38:52 +0000 Subject: [rancid] Improving Rancid's processing speed when having 1k+ devices In-Reply-To: <4FBAFC2ECF5D6244BA4A26C1C94A1E271528FFD3F3@exchange> References: <4FBAFC2ECF5D6244BA4A26C1C94A1E271528FFD3F3@exchange> Message-ID: <20190725163852.GE47419@shrubbery.net> Thu, Jul 25, 2019 at 08:14:28AM -0700, Emille Blanc: > I've seen/heard stories of people pre-empting rancid with an snmp-get of the config-last-changed / last committed OID, to generate a list of devices to run against. a building block for that is in the FAQ S3 Q10; using syslog .... From heas at shrubbery.net Thu Jul 25 16:55:31 2019 From: heas at shrubbery.net (john heasley) Date: Thu, 25 Jul 2019 16:55:31 +0000 Subject: [rancid] Improving Rancid's processing speed when having 1k+ devices In-Reply-To: References: Message-ID: <20190725165531.GF47419@shrubbery.net> Thu, Jul 25, 2019 at 02:29:37PM +0200, Florin Vlad Olariu: > Well, as per title, is there any way to improve rancid's speed with so many > devices? At the moment I set PAR_COUNT to 300, so it will connect in > parallel to 300 devices at a time, but the reality is that most time does > not seem to be taken by connecting and retrieving config but by what > happens next in the file processing and git-comitting. > > To give you some stats, with current settings it takes around 9 minutes to > do 1200 devices. I have only 1 group with all devices under the same group. > > Any trick you might have, please let me know! Typically, the network and, more so, the devices are the slow part. Some devices are much slower than others. more parallelism helps a lot - your high PAR_COUNT. other thoughts: - cvs is slow. use svn or git. svn is probably faster; but I have not benchmarked the two for the functions that rancid uses. - make sure that the rancid user is not process rlimited to less than ~605 processes; or PAR_COUNT * 2 + 5 or so. - perl is a meory pig. if the host/vm has memory pressure, this would be something to address. - retrieving device output does not require much cpu, but process does use some - dont starve it - use rancid.conf:NOPIPE=YES; i think this is faster because perl is a pig. - if you only need configs, then reduce what is collected to just show version and show running. or have one hourly group that collects that, and a daily group that collects everything. less processing, and esp many fewer regexes. multiple groups might help, at least for the SCM part. split your one large group into a few. make sure to use a separate cron for each so that they run in parallel. I havent attempted to benchmark or optimize any parts for a while. There was a complaint about the start-up time for control_rancid, which seems to me to be inconsequential, but I do not know what the users were attempting to do with rancid that made this matter. There are other benefits to this, so I've started to re-write it; this is not ready yet. 9 minutes for 1200 devices seems reasonable to me. :) From scott.granados at gmail.com Thu Jul 25 17:16:43 2019 From: scott.granados at gmail.com (Scott Granados) Date: Thu, 25 Jul 2019 13:16:43 -0400 Subject: [rancid] Improving Rancid's processing speed when having 1k+ devices In-Reply-To: <20190725165531.GF47419@shrubbery.net> References: <20190725165531.GF47419@shrubbery.net> Message-ID: <49A1A81E-E346-42E6-8CD7-A67208BBD2DD@gmail.com> I would also recommend running multiple rancid servers maybe scatter them geographically so it?s not a single machine pulling all the weight. Break the work loads up among them. > On Jul 25, 2019, at 12:55 PM, john heasley wrote: > > Thu, Jul 25, 2019 at 02:29:37PM +0200, Florin Vlad Olariu: >> Well, as per title, is there any way to improve rancid's speed with so many >> devices? At the moment I set PAR_COUNT to 300, so it will connect in >> parallel to 300 devices at a time, but the reality is that most time does >> not seem to be taken by connecting and retrieving config but by what >> happens next in the file processing and git-comitting. >> >> To give you some stats, with current settings it takes around 9 minutes to >> do 1200 devices. I have only 1 group with all devices under the same group. >> >> Any trick you might have, please let me know! > > Typically, the network and, more so, the devices are the slow part. Some > devices are much slower than others. more parallelism helps a lot - your > high PAR_COUNT. other thoughts: > > - cvs is slow. use svn or git. svn is probably faster; but I have not > benchmarked the two for the functions that rancid uses. > - make sure that the rancid user is not process rlimited to less than ~605 > processes; or PAR_COUNT * 2 + 5 or so. > - perl is a meory pig. if the host/vm has memory pressure, this would be > something to address. > - retrieving device output does not require much cpu, but process does use > some - dont starve it > - use rancid.conf:NOPIPE=YES; i think this is faster because perl is a pig. > - if you only need configs, then reduce what is collected to just show version > and show running. or have one hourly group that collects that, and a daily > group that collects everything. less processing, and esp many fewer regexes. > > multiple groups might help, at least for the SCM part. split your one large > group into a few. make sure to use a separate cron for each so that they run > in parallel. > > I havent attempted to benchmark or optimize any parts for a while. There was > a complaint about the start-up time for control_rancid, which seems to me to > be inconsequential, but I do not know what the users were attempting to do > with rancid that made this matter. There are other benefits to this, so I've > started to re-write it; this is not ready yet. > > 9 minutes for 1200 devices seems reasonable to me. :) > > _______________________________________________ > Rancid-discuss mailing list > Rancid-discuss at shrubbery.net > http://www.shrubbery.net/mailman/listinfo/rancid-discuss From ugob at lubik.ca Thu Jul 25 18:52:42 2019 From: ugob at lubik.ca (Ugo Bellavance) Date: Thu, 25 Jul 2019 14:52:42 -0400 Subject: [rancid] Fortinet private key problem Message-ID: Hi, I'm trying to get rancid to work with my Fortinet device. It seems to work OK, except for the fact that it doesn't collect the whole config. It looks like it's stuck in the removal of the private key. It stops like this: # set private-key "-----BEGIN ENCRYPTED PRIVATE KEY----- # Connection to server.xxx.xxx closed. I checked the code for filter cycling RSA private keys, but I don't know where would be the problem. Any help or suggestion would be appreciated. Thanks, -- Ugo Bellavance (ugob at lubik.ca) -------------- next part -------------- An HTML attachment was scrubbed... URL: From heas at shrubbery.net Thu Jul 25 19:17:43 2019 From: heas at shrubbery.net (john heasley) Date: Thu, 25 Jul 2019 19:17:43 +0000 Subject: [rancid] Fortinet private key problem In-Reply-To: References: Message-ID: <20190725191743.GU47419@shrubbery.net> Thu, Jul 25, 2019 at 02:52:42PM -0400, Ugo Bellavance: > Hi, > > I'm trying to get rancid to work with my Fortinet device. It seems to work > OK, except for the fact that it doesn't collect the whole config. It looks > like it's stuck in the removal of the private key. It stops like this: > > # set private-key "-----BEGIN ENCRYPTED PRIVATE KEY----- > # > Connection to server.xxx.xxx closed. > > I checked the code for filter cycling RSA private keys, but I don't know > where would be the problem. > > Any help or suggestion would be appreciated. what version of rancid? show us example input. test that you can run the command with the login script and receive the full output. From vv.corto at gmail.com Fri Jul 26 08:55:24 2019 From: vv.corto at gmail.com (Florin Vlad Olariu) Date: Fri, 26 Jul 2019 01:55:24 -0700 Subject: [rancid] Dell Force10 module change Message-ID: The Force10s that my company uses seem to produce the following lines when running "show run": ! Version 9.7(0.0) ! Last configuration change at Wed Oct 10 10:13:32 2018 by ! Startup-config last updated at Fri Jul 26 03:00:20 2019 by The change I made simply filters these lines: [rancid at tvvsmtarist001 ~]$ diff -u ran-bin/usr/libexec/rancid/f10rancid /usr/local/rancid/bin/f10rancid --- ran-bin/usr/libexec/rancid/f10rancid 2019-07-26 08:45:09.956957727 +0000 +++ /usr/local/rancid/bin/f10rancid 2019-07-26 08:33:03.536463469 +0000 @@ -443,6 +443,9 @@ # skip crap /^Current Configuration/ && next; + # CUSTOM: killing more variable stuff + /^! (Version|Last configuration|Startup-config last)/ && next; + It would be nice if a variant of this change could end up in the main force10 module. Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From vv.corto at gmail.com Fri Jul 26 09:34:49 2019 From: vv.corto at gmail.com (Florin Vlad Olariu) Date: Fri, 26 Jul 2019 02:34:49 -0700 Subject: [rancid] Improving Rancid's processing speed when having 1k+ devices In-Reply-To: <49A1A81E-E346-42E6-8CD7-A67208BBD2DD@gmail.com> References: <20190725165531.GF47419@shrubbery.net> <49A1A81E-E346-42E6-8CD7-A67208BBD2DD@gmail.com> Message-ID: On 25 July 2019 at 18:16:48, Scott Granados (scott.granados at gmail.com(mailto:scott.granados at gmail.com)) wrote: >?I would also recommend running multiple rancid servers maybe scatter them geographically so it?s not a single machine pulling all the weight. Break the work loads up among them. Great advice which didn't cross my mind. Might have to resort to this if I want ~ 1m poll times. On 25 July 2019 at 17:55:31, john heasley (heas at shrubbery.net) wrote: > - cvs is slow. use svn or git. svn is probably faster; but I have not benchmarked the two for the functions that rancid uses. I do use git already. Not sure git itself is to blame for the slowdown though. > - make sure that the rancid user is not process rlimited to less than ~605 processes; or PAR_COUNT * 2 + 5 or so. My `ulimit -u` gives "4096". I don't this this is a factor? > - perl is a meory pig. if the host/vm has memory pressure, this would be something to address. > - retrieving device output does not require much cpu, but process does use some - dont starve it I have a Xeon 8-core box, and when running it with PAR_COUNT=400 it runs to 50+ load, but only for a short period (the time it takes to connect to devices) after ~ 2 minutes it goes back to normal, so I don't think CPU is really the problem. Furthermore, I have 32G of ram, and running `watch free -h` it does not look like rancid uses *that* much memory, maybe ~ 5 G. >?- use rancid.conf:NOPIPE=YES; i think this is faster because perl is a pig. Tried this, but no difference in time :( > - if you only need configs, then reduce what is collected to just show version and show running. or have one hourly group that collects that, and a daily group that collects everything. less processing, and esp many fewer regexes. I only need configs and the way rancid is configured already only pools "show run" (or equivalent). Seems only real solution might be to break down the amount of hosts between different machines. Thanks John. From KyleSheeter at XRITE.com Fri Jul 26 09:37:36 2019 From: KyleSheeter at XRITE.com (Sheeter, Kyle) Date: Fri, 26 Jul 2019 09:37:36 +0000 Subject: [rancid] Adjust Rancid-Run Default Run Location Message-ID: Hey guys, I have been trying to figure out what happened to my RANCID install after a linux upgrade, and it looks like it adjusted some parameters that my predecessor setup when he built the machine. He used a subdirectory (/home/rancid/rancid/) to store all of our RANCID files, but when I did the ubuntu upgrade now rancid-run just runs from the default directory. I looked over the man page but didn't see anything on how to change that. Anyone have some good documentation on how to change that? Appreciate the help. Cheers, [Description: cid:3427271570_9070894] Kyle James Sheeter Manager - Global IT Operations and Network Infrastructure, Information Technology 4300 44th Street SE Grand Rapids, MI 49512 KyleSheeter at Xrite.com P: 616-803-2222 | M: 810-488-6160 www.xrite.com www.pantone.com Please be advised that this email may contain confidential information. If you are not the intended recipient, please notify us by email by replying to the sender and delete this message. The sender disclaims that the content of this email constitutes an offer to enter into, or the acceptance of, any agreement; provided that the foregoing does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 16854 bytes Desc: image001.png URL: From jandrewartha at ccgs.wa.edu.au Fri Jul 26 10:45:11 2019 From: jandrewartha at ccgs.wa.edu.au (James Andrewartha) Date: Fri, 26 Jul 2019 18:45:11 +0800 Subject: [rancid] ArubaOS 8 and rancid Message-ID: Hi, I'm trying to use https://github.com/miken32/rancid-aruba against an Aruba 8 virtual wireless controller, but it's barfing becuase clogin is sending "terminal length 0": rancid at propus:~$ clogin -c "show version" aruba-mc-poc. aruba-mc-poc spawn ssh -c aes128-cbc,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm at openssh.com,aes256-gcm at openssh.com,chacha20-poly1305 at openssh.com -x -l admin aruba-mc-poc admin at aruba-mc-poc.network's password: NOTICE NOTICE -- This server has active licenses that will expire in 27 days NOTICE NOTICE -- See 'show license' for details. NOTICE (ccgs-aruba-mc-poc) ^[mynode] # (ccgs-aruba-mc-poc) ^[mynode] #terminal length 0 ^ Invalid input detected at '^' marker. (ccgs-aruba-mc-poc) ^[mynode] # Error: TIMEOUT reached The github site says you can use the built-in clogin, but I don't think that's true any more? I'm running rancid 3.9-1~bpo9+1. Thanks, -- James Andrewartha Network & Projects Engineer Christ Church Grammar School Claremont, Western Australia Ph. (08) 9442 1757 Mob. 0424 160 877 From weylin at bu.edu Fri Jul 26 11:29:51 2019 From: weylin at bu.edu (Piegorsch, Weylin William) Date: Fri, 26 Jul 2019 11:29:51 +0000 Subject: [rancid] Improving Rancid's processing speed when having 1k+ devices In-Reply-To: <20190725165531.GF47419@shrubbery.net> References: <20190725165531.GF47419@shrubbery.net> Message-ID: > 9 minutes for 1200 devices seems reasonable to me. :) Heh - I've got around 3,000. I'm having an issue with PAR that I haven't fully addressed, so I'm still only doing 5 at a time and getting 4- to 5-hour run times. We made a choice at one point to put all "do-diff" groups on one line in cron, that didn?t help at all but haven't yet backed that down. If we were to break that up appropriately, we'd have around 1200 in the largest group, several hundred in a few, and a number of group (about 15 altogether) with <10. We could break things up further, but at some point you have to ust accept large router.db files because there's managerial overhead trying to manage a large number of rancid groups and keeping it synchronized against CDP and LLDP discoveries and CMDB database in a dynamic environment. Our old server we stood up in 2002 using rancid 1.2 was set to PAR=100 and getting about 45min for the entire suite. We never actually hit 100 simultaneous connections, we maxed out at around 60-70 because by the time the 71st connection was opened the 1st was completing. Of course, that was for a server stood-up in 2002, so take that for whatever it's worth. Is 9 min too long? weylin ?On 7/25/19, 12:55 PM, "john heasley" wrote: Thu, Jul 25, 2019 at 02:29:37PM +0200, Florin Vlad Olariu: > Well, as per title, is there any way to improve rancid's speed with so many > devices? At the moment I set PAR_COUNT to 300, so it will connect in > parallel to 300 devices at a time, but the reality is that most time does > not seem to be taken by connecting and retrieving config but by what > happens next in the file processing and git-comitting. > > To give you some stats, with current settings it takes around 9 minutes to > do 1200 devices. I have only 1 group with all devices under the same group. > > Any trick you might have, please let me know! Typically, the network and, more so, the devices are the slow part. Some devices are much slower than others. more parallelism helps a lot - your high PAR_COUNT. other thoughts: - cvs is slow. use svn or git. svn is probably faster; but I have not benchmarked the two for the functions that rancid uses. - make sure that the rancid user is not process rlimited to less than ~605 processes; or PAR_COUNT * 2 + 5 or so. - perl is a meory pig. if the host/vm has memory pressure, this would be something to address. - retrieving device output does not require much cpu, but process does use some - dont starve it - use rancid.conf:NOPIPE=YES; i think this is faster because perl is a pig. - if you only need configs, then reduce what is collected to just show version and show running. or have one hourly group that collects that, and a daily group that collects everything. less processing, and esp many fewer regexes. multiple groups might help, at least for the SCM part. split your one large group into a few. make sure to use a separate cron for each so that they run in parallel. I havent attempted to benchmark or optimize any parts for a while. There was a complaint about the start-up time for control_rancid, which seems to me to be inconsequential, but I do not know what the users were attempting to do with rancid that made this matter. There are other benefits to this, so I've started to re-write it; this is not ready yet. 9 minutes for 1200 devices seems reasonable to me. :) From ugob at lubik.ca Fri Jul 26 12:24:35 2019 From: ugob at lubik.ca (Ugo Bellavance) Date: Fri, 26 Jul 2019 08:24:35 -0400 Subject: [rancid] Possible bug Message-ID: Hi, I think that there might be a problem with the fnlogin script. It may because I'm attempting to execute it on a Fortiweb system (not Fortigate), but there is one last ' "send "end\r" ' that shouldn't be there. Sample of ssh session with the unit, doing the same thing as the fnolgin script: [rancid at server bin]$ ssh -l ranciduser fortiweb.example.com ranciduser at fortiweb.example.com's password: fortiweb $ config system console fortiweb (console) $ set output standard fortiweb (console) $ end fortiweb $ end Command fail. CLI parsing error. I'm using the fnlogin script "3915 2018-10-29 21:05:01Z" I don't have a Fortigate unit to test, so I do not know if it's OS-related or not. Thanks, -- Ugo Bellavance (ugob at lubik.ca) -------------- next part -------------- An HTML attachment was scrubbed... URL: From jandrewartha at ccgs.wa.edu.au Fri Jul 26 13:13:05 2019 From: jandrewartha at ccgs.wa.edu.au (James Andrewartha) Date: Fri, 26 Jul 2019 21:13:05 +0800 Subject: [rancid] ArubaOS 8 and rancid In-Reply-To: References: Message-ID: On 26/07/19 18:45, James Andrewartha wrote: > I'm trying to use https://github.com/miken32/rancid-aruba against an > Aruba 8 virtual wireless controller, but it's barfing becuase clogin is > sending "terminal length 0": Never mind, when it ran from cron it was fine. -- James Andrewartha Network & Projects Engineer Christ Church Grammar School Claremont, Western Australia Ph. (08) 9442 1757 Mob. 0424 160 877 From ugob at lubik.ca Fri Jul 26 13:31:41 2019 From: ugob at lubik.ca (Ugo Bellavance) Date: Fri, 26 Jul 2019 09:31:41 -0400 Subject: [rancid] Fortinet private key problem In-Reply-To: <20190725191743.GU47419@shrubbery.net> References: <20190725191743.GU47419@shrubbery.net> Message-ID: I found a fix: In fnrancid (version 3724 2017-08-01 17:58:06Z), I had to change the regex a bit because on my system (Fortiweb, not Fortigate), the double quote is on another line. I haven't been able to make it work with the new line (tried \n and \r\n) and it didn't work so I just removed the double quote in the regex. # if (/^\s*-----END (RSA|ENCRYPTED) PRIVATE KEY-----"/) { if (/^\s*-----END (RSA|ENCRYPTED) PRIVATE KEY-----/) { I'm not sure how future-proof this is but tips are welcome to make the regex work with this input: OSnA0DuUpx2/FvoFbJM9jmx= -----END ENCRYPTED PRIVATE KEY----- " unset passwd Thanks, On Thu, Jul 25, 2019 at 3:17 PM john heasley wrote: > Thu, Jul 25, 2019 at 02:52:42PM -0400, Ugo Bellavance: > > Hi, > > > > I'm trying to get rancid to work with my Fortinet device. It seems to > work > > OK, except for the fact that it doesn't collect the whole config. It > looks > > like it's stuck in the removal of the private key. It stops like this: > > > > # set private-key "-----BEGIN ENCRYPTED PRIVATE KEY----- > > # > > Connection to server.xxx.xxx closed. > > > > I checked the code for filter cycling RSA private keys, but I don't know > > where would be the problem. > > > > Any help or suggestion would be appreciated. > > what version of rancid? show us example input. test that you can run the > command with the login script and receive the full output. > -- Ugo Bellavance (ugob at lubik.ca) -------------- next part -------------- An HTML attachment was scrubbed... URL: From heas at shrubbery.net Mon Jul 29 17:48:05 2019 From: heas at shrubbery.net (john heasley) Date: Mon, 29 Jul 2019 17:48:05 +0000 Subject: [rancid] Possible bug In-Reply-To: References: Message-ID: <20190729174805.GG28054@shrubbery.net> Fri, Jul 26, 2019 at 08:24:35AM -0400, Ugo Bellavance: > Hi, > > I think that there might be a problem with the fnlogin script. It may > because I'm attempting to execute it on a Fortiweb system (not Fortigate), > but there is one last ' "send "end\r" ' that shouldn't be there. > > Sample of ssh session with the unit, doing the same thing as the fnolgin > script: > > [rancid at server bin]$ ssh -l ranciduser fortiweb.example.com > ranciduser at fortiweb.example.com's password: > fortiweb $ config system console > > fortiweb (console) $ set output standard > > fortiweb (console) $ end > > fortiweb $ end > Command fail. CLI parsing error. it should be sending 'config global' first. Have you altered the script? > I'm using the fnlogin script "3915 2018-10-29 21:05:01Z" This part of the script has not changed since then. > I don't have a Fortigate unit to test, so I do not know if it's OS-related > or not. From heas at shrubbery.net Mon Jul 29 17:53:26 2019 From: heas at shrubbery.net (john heasley) Date: Mon, 29 Jul 2019 17:53:26 +0000 Subject: [rancid] Adjust Rancid-Run Default Run Location In-Reply-To: References: Message-ID: <20190729175326.GH28054@shrubbery.net> Fri, Jul 26, 2019 at 09:37:36AM +0000, Sheeter, Kyle: > Hey guys, > > I have been trying to figure out what happened to my RANCID install after a linux upgrade, and it looks like it adjusted some parameters that my predecessor setup when he built the machine. He used a subdirectory (/home/rancid/rancid/) to store all of our RANCID files, but when I did the ubuntu upgrade now rancid-run just runs from the default directory. > > I looked over the man page but didn't see anything on how to change that. Anyone have some good documentation on how to change that? etc/rancid.conf:BASEDIR see rancid.conf(5); presumably the upgrade saved a copy of the old file as etc/rancid.conf.. From heas at shrubbery.net Mon Jul 29 18:06:44 2019 From: heas at shrubbery.net (john heasley) Date: Mon, 29 Jul 2019 18:06:44 +0000 Subject: [rancid] Improving Rancid's processing speed when having 1k+ devices In-Reply-To: References: <20190725165531.GF47419@shrubbery.net> <49A1A81E-E346-42E6-8CD7-A67208BBD2DD@gmail.com> Message-ID: <20190729180644.GJ28054@shrubbery.net> Fri, Jul 26, 2019 at 02:34:49AM -0700, Florin Vlad Olariu: > On 25 July 2019 at 18:16:48, Scott Granados > (scott.granados at gmail.com(mailto:scott.granados at gmail.com)) wrote: > > >?I would also recommend running multiple rancid servers maybe scatter them geographically so it?s not a single machine pulling all the weight. Break the work loads up among them. > > Great advice which didn't cross my mind. Might have to resort to this > if I want ~ 1m poll times. topologically close servers can help, but I would just run more processes instead. less mgmt overhead. > > - make sure that the rancid user is not process rlimited to less than ~605 > processes; or PAR_COUNT * 2 + 5 or so. > > My `ulimit -u` gives "4096". I don't this this is a factor? unlikely. make sure its not others; -n -d. you'd see processes being killed in the logs ... Are your configs very large? I have one group of 252 devices that are scattered around the global totaling 1.2G of on-disk rancid output which takes about 28m to collect with 16 processes. From weylin at bu.edu Mon Jul 29 21:01:56 2019 From: weylin at bu.edu (Piegorsch, Weylin William) Date: Mon, 29 Jul 2019 21:01:56 +0000 Subject: [rancid] Improving Rancid's processing speed when having 1k+ devices In-Reply-To: <20190729180644.GJ28054@shrubbery.net> References: <20190725165531.GF47419@shrubbery.net> <49A1A81E-E346-42E6-8CD7-A67208BBD2DD@gmail.com> <20190729180644.GJ28054@shrubbery.net> Message-ID: > topologically close servers can help, but I would just run more processes instead. Agree in 99% of cases. Though, there are rare niche scenarios where having geographically co-located servers can help. Slow WAN connections ("dial-up"); high latency or high packet loss connections (satellite); unreliable WAN links (ship at sea); and so forth. weylin ?On 7/29/19, 2:06 PM, "john heasley" wrote: Fri, Jul 26, 2019 at 02:34:49AM -0700, Florin Vlad Olariu: > On 25 July 2019 at 18:16:48, Scott Granados > (scott.granados at gmail.com(mailto:scott.granados at gmail.com)) wrote: > > > I would also recommend running multiple rancid servers maybe scatter them geographically so it?s not a single machine pulling all the weight. Break the work loads up among them. > > Great advice which didn't cross my mind. Might have to resort to this > if I want ~ 1m poll times. topologically close servers can help, but I would just run more processes instead. less mgmt overhead. > > - make sure that the rancid user is not process rlimited to less than ~605 > processes; or PAR_COUNT * 2 + 5 or so. > > My `ulimit -u` gives "4096". I don't this this is a factor? unlikely. make sure its not others; -n -d. you'd see processes being killed in the logs ... Are your configs very large? I have one group of 252 devices that are scattered around the global totaling 1.2G of on-disk rancid output which takes about 28m to collect with 16 processes.