f 1 - VCS Hardware Troubleshooting & Linux OS (1)
-
Upload
derrick-darbey -
Category
Documents
-
view
224 -
download
0
Transcript of f 1 - VCS Hardware Troubleshooting & Linux OS (1)
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
1/65
VCS Hardware Troubleshooting &Boot Process
6/17/201
Vernon !e"ee
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
2/65
#b$ecti%es
'(ter this lesson) learners will be able to*
+ecall the ,ost co,,on (or,s o( hardware (ailure on the
Cisco VCS
!eter,ine i( one o( these hardware (ailures is ta-ing "lace
+ecall the i,"ortant ste"s o( the boot "rocess on the
Cisco VCS and use this -nowledge to troubleshoot issues
with VCS booting
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
3/65
Phase 1 . VCS Hardware
H!!
C
'S
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
4/65
VCS Harddri%es
sda. Pri,ar !ri%e3 ( this (ails) the VCS will not boot at
all3 This is ,ounted to the root and /tandberg
sdb 4 Secondar !ri%e3 ( this (ails) the VCS will act 5odd3
This is ,ounted to /,nt/harddis-3
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
5/65
S!' ailure*
The VCS will ne%er begin loading the #S3 So,ething
si,ilar to this ,a be (roen on the console*
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
6/65
S!B (ailure*
S!B can (ail in se%eral was*
138 9hen the sste, boots) sdb will ne%er be detected
238 sdb will go unreachable a(ter the sste, has booted
38 sdb ,a be detected during sste, boot) but bounce
bac- and (orth between o"erating and not o"erating
"ro"erl
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
7/65
Testing sdb* ls
:ood*
ls 4l /de%/sd;
brw4rw4444 1 root root *2> /de%/sda
brw4rw4444 1 root root *2> /de%/sda1
brw4rw4444 1 root root *2> /de%/sda2
brw4rw4444 1 root root *2> /de%/sda
brw4rw4444 1 root root 201241241= 1>*2> /de%/sda>
brw4rw4444 1 root root *2> /de%/sda6
brw4rw4444 1 root root *2> /de%/sda7
brw4rw4444 1 root root *2> /de%/sda /de%/sdb
brw4rw4444 1 root root *2> /de%/sdb1
brw4rw4444 1 root root *2> /de%/sdb2
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
8/65
Testing sdb* ls
Bad*
ls 4l /de%/sd;
brw4rw4444 1 root root *2> /de%/sda
brw4rw4444 1 root root *2> /de%/sda1
brw4rw4444 1 root root *2> /de%/sda2
brw4rw4444 1 root root *2> /de%/sda
brw4rw4444 1 root root 201241241= 1>*2> /de%/sda>
brw4rw4444 1 root root *2> /de%/sda6
brw4rw4444 1 root root *2> /de%/sda7
brw4rw4444 1 root root *2> /de%/sda3=0 2010410416 r1
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
10/65
Testing sdb* s,artctl
Bad*
? @ s,artctl 44all /de%/sdb
s,artctl >3=0 2010410416 r1
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
11/65
Testing sdb* s,artctl
Bad*
? @ s,artctl 44all /de%/sdb
s,artctl >3=0 2010410416 r1
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
12/65
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
13/65
Testing sdb* d(
Bad*
? @ d( M gre" sdb
? @
Sdb is not ,ounted
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
14/65
Testing Ithernet Ports
:ood*
? @ i(con(ig 4a M gre" eth
eth0 Lin- enca"*Ithernet H9addr 00*10**1A*'0*B=
eth1 Lin- enca"*Ithernet H9addr 00*10**1A*'0*B>
eth2 Lin- enca"*Ithernet H9addr 00*10**1A*'0*B6
eth Lin- enca"*Ithernet H9addr 00*10**1A*'0*B7
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
15/65
Testing Ithernet Ports
Bad*
? @ i(con(ig 4a M gre" eth
eth0 Lin- enca"*Ithernet H9addr 00*10**1A*'0*B=
eth1 Lin- enca"*Ithernet H9addr 00*10**1A*'0*B6
eth2 Lin- enca"*Ithernet H9addr 00*10**1A*'0*B7
'll = Ithernet "orts ,ust show
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
16/65
Serial u,ber +ange (or Potential
C ssues ( the serial nu,ber is in the range >2'1A2674>2'2>>10(or VCS and >>'000014>>'0001 (or Conductor) the VCS
is at ris- (or a (ailed C3
Iact Serial u,bers can be seen on the ecel doc athtt"*//172313107/"atches/SN20listN20o(
N20"otentialN20CN20issueD%er23ls
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
17/65
an ssues
' VCS can run (ine with 2 (ailed (ans
Process +' i( reOuested b custo,er e%en i( onl one
(an (ailure3 Custo,er will get a re(urbished VCS that ,a
ha%e ,ore se%ere "roble,s3 VCS running > and 6 will re"ort a (an alar, e%en i( the
(an is now o"erating at a "ro"er s"eed3 'lar,s can be
,anuall reset3
Current VCS code will onl raise (an alar, i( 2 or ,ore(ans ha%e (ailed3
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
18/65
VCS :enerations
>2'0 . Brage1
>2'1)>2'2 . Brage2
Brage1 is Ouite old and has reached the e"eced H!! li(e
(or sdb
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
19/65
Phase 2 . VCS Boot Process
:rub
nit scri"ts
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
20/65
VCS Boot Process 4 :rub
The VCS boots o(( o( grub on sda13 :rub will then ha%e
the VCS boot either o(( o( sda> Fi,age18 or sda6
Fi,age283
The acti%e i,age is ,ounted onto /3
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
21/65
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
22/65
VCS Boot Process . rc3ssinit
/etc/rc3ssinit calls all o( the startu" scri"ts in /etc/init3d/3
Scri"ts starting with I are called (irst) a(ter this scri"ts
starting with S are called3 Scri"ts are called in nu,berical
order3
Ia,"le* I00bootlogd is called (irst and SAA%,toolsd is
called last3
/etc/init3d/ scri"ts are called with an argu,ent o( either
start) sto") or restart3
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
23/65
VCS Boot Process . ounting o(
other Partitions /etc/init3d/I26,ount reads /etc/"artitions3con( to get thelocation o( the ro and rw "artitions3
The scri"t will then detect which "artition is ,ounted as
root and ,ount the a""ro"riate rw "artition on /tandberg3 +o "artition 1 is /de%/sda>
+w "artition 1 is /de%/sda7
+o "artition 2 is /de%/sda6 +w "artition 2 is /de%/sda
oa-Dcrashlogs F"re47328 / crashlogsF732[8
3tar3g (ile o( recent crashes ote an "rocess on the bo can crash . so,e serious) so,e not
VCS so(tware F5a""8 will auto,aticall restart) but call and registration statewill be lost
5net is har,less's are 5linustatus3" or 5,anage,ent(ra,ewor-3" F"rior to 7328
5winbindd could necessitate a restart
Loo- inside crash L du,"s to see i( the are re"etiti%e
Load into htt"s*//103>031>23110to see i( theUre -nown
Please chec- the out"ut ,a-es sense based on s,"to,s/logs( in doubt)please do check with us!
http://10.50.152.110/http://10.50.152.110/ -
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
36/65
Y 2010 Cisco and/or its a((iliates3 'll rights reser%ed3 Cisco Con(idential 6
clusterdb
Tetual du,"s o( the Cluster!B tables
Those with a serial nu,ber su((i* sharded F"eer4s"eci(ic8 records
i3e3 records under co,"lete control o( that "eer
on4sharded Fglobal8 tables
'n "eer can edit
Status is t"icall sharded
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
37/65
Y 2010 Cisco and/or its a((iliates3 'll rights reser%ed3 Cisco Con(idential 7
harddis-logs
'll logs in logs/ subdirector
Prior to 7* ou need to untar the logs3tar3g (ile too
Logs are on a single "artition on the VCS . not %ersion4s"eci(ic
u"grade3log
Logs u"grades on the bo3 'llows correlation o( u"grade trans(or,s) reboots)ser%ice and load changes) etc3
tandberg4webcon(ig3log
Logs all con(iguration changes3 Ver use(ul3
a"ache;log
Logs web ser%er accesses and errors) including TS "robes3
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
38/65
Y 2010 Cisco and/or its a((iliates3 'll rights reser%ed3 Cisco Con(idential #2#Q%c-/li/sFQ/l/nod!ex90Rusc01!cisco!comQ &otocolFQU3&Q LevelFQ1QUTCTimeFQ2012-0-20 09:49:4)E24S
essages include*
+egistration +eOuested M 'cce"ted M +e$ected E
Search 'tte,"ted M Co,"leted M Cancelled E
Source 'liases +ewritten
Call 'tte,"ted M Connected M +e$ected M !isconnected E
essage Sent M +ecei%ed E FH322> Signalling8
+eOuest Sent M +ecei%ed E FSP Signalling8
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
53/65
Y 2010 Cisco and/or its a((iliates3 'll rights reser%ed3 Cisco Con(idential >
harddis-logs . networ-Dlog
Logs actual networ- ,essages3 Ia,"le H322>*
2012-0-2)T1#:9:1K01:00 usc01-vcs1 tvcs: UTCTimeFQ2012-0-2)12:9:1E410Q ModuleFQnetwo8!h#2#Q LevelFQN57Q: 3st-ipFQ10!0!1"1!2#Q3st-potFQ120Q M 3et/ilFQ%endin@ >!22 %etup >!#2#v" (/ndwidth:")8ps3estli/s:h/ll/m2!ex90Rusc01!cisco!com 3estC%dd:V&v4VVTC&VV10!0!1"1!2#:120VW Jendo:TN3(.=+Q
Ia,"le SP3 Trace with Call4!*2012-0-2)T1":00:19K01:00 usc01-vcs1 tvcs: UTCTimeFQ2012-0-2)1:00:19E#Q ModuleFQnetwo8!sipQ LevelFQN57Q: %c-ipFQ10!0!1"!1Q%c-potFQ20#Q 3et/ilFQ=eceive =euest MethodFNJT.E =euest-U=Fsip:usc01-1-ed@/stonRcisco!comE C/ll-3Fddd1"#9ce)/1dR10!!)0!22Q
2012-0-2)T1":00:19K01:00 usc01-vcs1 tvcs: UTCTimeFQ2012-0-2)1:00:19E4Q ModuleFQnetwo8!sipQ LevelFQN57Q: 3st-ipFQ10!0!1"!1Q3st-potFQ20#Q 3et/ilFQ%endin@ =esponse CodeF100E MethodFNJT.EToFsip:usc01-1-ed@/stonRcisco!comE C/ll-3Fddd1"#9ce)/1dR10!!)0!22S
ote* uni(iedDlog G networ- [ ,essages [ de%elo"erDlog
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
54/65
Y 2010 Cisco and/or its a((iliates3 'll rights reser%ed3 Cisco Con(idential >=
!iagnostic Logging
Pro%ides a si,"le wa to get "rotocollogs and other "ertinent log ,essages(ro, the VCS
Select aintenance ] !iagnostics ]!iagnostics Logging
+e"laces 5netlog (ro, 730 onwards
Set 5networ- to !IBK: FeOui%alent to5netlog 28
'lso set 5interwor-ing to !IBK:
unless ouUre sure no interwor-ing isgoing on
9ill a((ect what is in sna"shot Flog ista-en (ro, uni(iedDlog8
Logging is started/sto""ed acrosswhole cluster) but ,ust be downloaded
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
55/65
Y 2010 Cisco and/or its a((iliates3 'll rights reser%ed3 Cisco Con(idential >>
ew Sna"shot eatures in 732
'lar,s (older logging acti%e alar,s
Sensors logs now logs 5a""stats . the +esourceKsage out"ut Fcallsand registrations in use at that "oint in ti,e8
Res!urce"sa#e $tem&1&'
(a))s $tem&1&'
Traversa) $tem&1&'
(urre*t $tem&1&'0/(urre*t'
+a, $tem&1&'-/+a,'
T!ta) $tem&1&'680/T!ta)'
/Traversa)'
.!*Traversa) $tem&1&'
(urre*t $tem&1&'0/(urre*t'
+a, $tem&1&'8/+a,'
T!ta) $tem&1&'1099/T!ta)'
/.!*Traversa)'
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
56/65
Y 2010 Cisco and/or its a((iliates3 'll rights reser%ed3 Cisco Con(idential >6
Ti"s) tric-s and co,,on issues^
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
57/65
Y 2010 Cisco and/or its a((iliates3 'll rights reser%ed3 Cisco Con(idential >7
9or-ing with !iagnostic Logs
Tracing SP call
Loo- (or the VTI
Then trace the Call4!
Beware o( (or-ing
!ebugging nterwor-ing
#"enLogicalChannel F#LC8 onl sent on ,edia
Kse the 5LCList out"ut (or ,onitoring logical channels
Con(iguration
5Status sna"shot is ,ore co,"lete than stat/con(
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
58/65
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
59/65
Y 2010 Cisco and/or its a((iliates3 'll rights reser%ed3 Cisco Con(idential >A
+'
Should onl be used (or hardware (ailure
See earlier diagnosis o( so,e issues
See +' :uide on Cisco website FVCS !ocs/Troubleshooting8*
htt"*//www3cisco3co,/en/KS/docs/tele"resence/in(rastructure/%cs/troubleshooting/+'DProceduresD(orDCiscoDVCSDandDTelePresenceDConductorD'""liances3"d(
actor reset scri"t Flog is as root) run 5(actor4reset8 reinstalls the last%alid i,ageR suitable (or ,an so(tware issues
http://www.cisco.com/en/US/docs/telepresence/infrastructure/vcs/troubleshooting/RMA_Procedures_for_Cisco_VCS_and_TelePresence_Conductor_Appliances.pdfhttp://www.cisco.com/en/US/docs/telepresence/infrastructure/vcs/troubleshooting/RMA_Procedures_for_Cisco_VCS_and_TelePresence_Conductor_Appliances.pdfhttp://www.cisco.com/en/US/docs/telepresence/infrastructure/vcs/troubleshooting/RMA_Procedures_for_Cisco_VCS_and_TelePresence_Conductor_Appliances.pdfhttp://www.cisco.com/en/US/docs/telepresence/infrastructure/vcs/troubleshooting/RMA_Procedures_for_Cisco_VCS_and_TelePresence_Conductor_Appliances.pdf -
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
60/65
Y 2010 Cisco and/or its a((iliates3 'll rights reser%ed3 Cisco Con(idential 60
Securit Vulnerabilities
5VCS ails scan (or reason
Chec- out grwi-i list (or (ies in later VCS %ersions*
htt"*//grwi-i/inde3"h"/VCS/SecuritDoti(ications
There are a nu,ber o( (alse "ositi%es Fe3g3 PHP C:) 9eb!'V8
( ou ha%e a CVI nu,ber (or a third "art a""lication Fe3g3 '"ache)o"enssl8) chec- out the re"ort e3g3*
htt"*//c%e3,itre3org/cgi4bin/c%ena,e3cgiXna,eGCVI420124001
This will t"icall sa which %ersions are %ulnerable
Then chec- against the %ersion o( so(tware running on the ,ost recentVCS %ersion
or certi(icate errors) (a,iliarise and "oint towards VCS Certi(icateCreation and Kse :uide FCisco3co,) VCS Con(iguration :uides8*
htt"*//www3cisco3co,/en/KS/docs/tele"resence/in(rastructure/%cs/con(igDgu
http://www.cisco.com/en/US/docs/telepresence/infrastructure/vcs/config_guide/Cisco_VCS_Certificate_Creation_and_Use_Deployment_Guide_X7-2.pdfhttp://www.cisco.com/en/US/docs/telepresence/infrastructure/vcs/config_guide/Cisco_VCS_Certificate_Creation_and_Use_Deployment_Guide_X7-2.pdfhttp://grwiki/index.php/VCS/Security_Notificationshttp://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2012-0031http://www.cisco.com/en/US/docs/telepresence/infrastructure/vcs/config_guide/Cisco_VCS_Certificate_Creation_and_Use_Deployment_Guide_X7-2.pdfhttp://www.cisco.com/en/US/docs/telepresence/infrastructure/vcs/config_guide/Cisco_VCS_Certificate_Creation_and_Use_Deployment_Guide_X7-2.pdfhttp://www.cisco.com/en/US/docs/telepresence/infrastructure/vcs/config_guide/Cisco_VCS_Certificate_Creation_and_Use_Deployment_Guide_X7-2.pdfhttp://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2012-0031http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2012-0031http://grwiki/index.php/VCS/Security_Notificationshttp://grwiki/index.php/VCS/Security_Notifications -
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
61/65
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
62/65
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
63/65
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
64/65
Y 2010 Cisco and/or its a((iliates3 'll rights reser%ed3 Cisco Con(idential 6=
#ther co,,on issues
Stuc- calls in status) "re4732
Call status snc between 5stat and what was seen on the web had issuesbe(ore 732 . rewritten in 732 and issues resol%ed FCSCtr2
-
8/11/2019 f 1 - VCS Hardware Troubleshooting & Linux OS (1)
65/65
WuestionsX