TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber...

56
TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 [email protected]

Transcript of TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber...

Page 1: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

TCP/IP Troubleshooting

Tips & Tools

Gord

on W

ebber

Will

iam

Data

Syste

ms

Aug

ust

2011

Gordon.W

ebber@

willd

ata

.com

Page 2: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

AG

EN

DA

. . . . . .

•K

now

Your N

etw

ork

•A

cti

on P

lans /

Proble

m D

ete

rm

inati

on

•Tools

–G

eneral U

sage

•U

ndersta

ndin

g t

he C

om

mon T

ools

.(pin

g, tr

aceroute

, nets

tat,

nslo

okup, …

)

•P

roble

m D

iagnosis

Tip

s

Page 3: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Know

Your

Netw

ork

! . . .

•In order to m

anage any netw

ork successfully, you m

ust be

aware of the topology.

•Before any successful, and tim

ely, problem resolution can be

attempted, a (current !)

netw

ork diagram is e

ssential.

•The diagram (and associated documentation) should indicate

all nodes and all possible paths, and detail the subnets,

addresses and software (especially versions) available at each

node.

•O

nly

then is it

possib

le t

o c

reate

an a

ppro

priate

action p

lan…

Page 4: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Action P

lans . . . . .

•W

here t

o S

tart?

-First, identify

the p

roble

m. This will

determ

ine the right tools to use, and the right place to start

testing from (

!“

Top-d

ow

n”

or “

Bott

om

-up”

!). Progressive

testing m

ay be needed to isolate the problem area.

•Netw

ork problems usually fall into two or three categories:-

•N

o c

onnecti

on c

an b

e m

ade.

•C

onnecti

ons c

an b

e m

ade, but

are u

nsta

ble

, O

R , not

all

functi

ons o

perate

.

•C

onnecti

ons a

re s

table

but

perfo

rm

ance is p

oor.

Mis

info

rm

ation A

necdote

Page 5: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Action P

lans . . . . .

Connecti

vit

yissues can be caused by:-

Application errors

Failed bind

Failed netw

ork connections

Power failures

Bad configuration/changes

Security restrictions

Hardware failures

Perfo

rm

anceissues can be caused by:-

Insufficient bandwidth

Congestion

Bottlenecks

Routing

Priorities

Fragmentation

Retries

Application errors

Broadcasts

Switch faults

Page 6: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Action P

lans . . . . .

1. Investi

gate

(ALL) e

rror m

essages –

these m

ay indicate the

nature and location of the failure [

e.g

.“ttl”expired,no

path available

,packet size too large (“nofragment”

is on)].

2. C

lassif

y t

he e

rror–ask what works and what doesn’t, and for

whom . . .

�Problems affecting one person m

ay be local and

physical (e.g. check the cables/switch/vlan

first)

�Problems affecting m

ore than one user are m

ore likely

to be the netw

ork or application

�Problems affecting m

ore than one person & m

ore than

one netw

ork path are m

ore likely to be the

application.

!! S

yslo

gd

!!

Page 7: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Action P

lans . . . . .

3. Test

connecti

vit

y(e

nd-t

o-e

nd) –using Ping/Traceroute.

Be careful to ensure that the packets take the same path

as the problem connection (i.e. ensure correct source

interface address –

you m

ay need to use an “extended”

PING).

�If PING fails, note the location and investigate there.

�If PING succeeds (note that this is ICMP, the

connection probably uses TCP, so this m

ay N

OT be

a conclusive test), try with a TCP PING if available

�If PING succeeds try again with larger packets, if

appropriate.

Page 8: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Action P

lans . . . . .

For E

xam

ple

:P

roble

m r

eporte

d a

s …

“end-u

ser c

annot

connect

to a

pplicati

on”

�Starting at the end-user system ensure local physical

connections are good, then check the next layer, such

as local switch ports, vlans, routers, and even firewalls.

�Then, test each “hop”by progressive steps across the

netw

ork.

�Then ensure that the system running the required

application is connected at the netw

ork level (“ping”

from that system outbound via the interface in question.

If all these results are good, then the issue is probably with

the application and not a netw

ork problem!

Page 9: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Tools

. . . . .

Dis

cla

imer:

The f

act

that

som

e t

ools

are m

enti

oned in t

his

presenta

tion w

hile o

ther t

ools

are n

ot,

in n

o w

ay im

plies

recom

mendati

on o

f th

e t

ools

menti

oned, nor

condem

nati

on o

f th

ose t

ools

not

menti

oned.

The p

urpose o

f th

is p

resenta

tion is s

imply

to m

ake

att

endees a

ware t

hat

such t

ools

exis

t, a

nd t

he a

ttendees

should

make u

p t

heir

ow

n m

ind a

s t

o t

he s

uit

abilit

y o

f any

tool used o

n t

heir

ow

n s

yste

m.

Page 10: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

“Com

mon”

Tools

. . . . .

“P

IN

G”

-proves that connectivity exists

“TR

ACER

TE”

-discovers the netw

ork path (also “tracert”)

“N

ETSTA

T”

-to locate connection inform

ation

ALL

ALL

ALL

ALL - ---All connections to a stack

All connections to a stack

All connections to a stack

All connections to a stack

ALLConn

ALLConn

ALLConn

ALLConn

- ---TCP/IP connections

TCP/IP connections

TCP/IP connections

TCP/IP connections

ARp

ARp

ARp

ARp

- ---Query ARP table or entry information

Query ARP table or entry information

Query ARP table or entry information

Query ARP table or entry information

CONFIG

CONFIG

CONFIG

CONFIG - ---Configuration data

Configuration data

Configuration data

Configuration data

COnn

COnn

COnn

COnn

- ---Active TCP/IP connections (Default)

Active TCP/IP connections (Default)

Active TCP/IP connections (Default)

Active TCP/IP connections (Default)

DEvlinks

DEvlinks

DEvlinks

DEvlinks

- ---Devices and links

Devices and links

Devices and links

Devices and links

Gate

Gate

Gate

Gate - ---Current known gateways

Current known gateways

Current known gateways

Current known gateways

HOme

HOme

HOme

HOme

- ---Home address list

Home address list

Home address list

Home address list

PORTList

PORTList

PORTList

PORTList

- ---Display port reservation list

Display port reservation list

Display port reservation list

Display port reservation list

ROUTe

ROUTe

ROUTe

ROUTe

- ---Display routing information

Display routing information

Display routing information

Display routing information

SOCKets

SOCKets

SOCKets

SOCKets

- ---Socket interface users and sockets

Socket interface users and sockets

Socket interface users and sockets

Socket interface users and sockets

STATS

STATS

STATS

STATS - ---TCP/IP statistics

TCP/IP statistics

TCP/IP statistics

TCP/IP statistics

TCP

TCP

TCP

TCP - ---Displays detailed info about the stack

Displays detailed info about the stack

Displays detailed info about the stack

Displays detailed info about the stack

TELnet

TELnet

TELnet

TELnet

- ---Telnet connection information

Telnet connection information

Telnet connection information

Telnet connection information

z/OS command format:

z/OS command format:

z/OS command format:

z/OS command format:

--------------------

--------------------

--------------------

--------------------

NETSTAT < Option | Command > <

NETSTAT < Option | Command > <

NETSTAT < Option | Command > <

NETSTAT < Option | Command > <

Target >

Target >

Target >

Target > < Output > < (Select >

< Output > < (Select >

< Output > < (Select >

< Output > < (Select >

E.g.:

E.g.:

E.g.:

E.g.:

TSO NETSTAT CONN (PORT 25

TSO NETSTAT CONN (PORT 25

TSO NETSTAT CONN (PORT 25

TSO NETSTAT CONN (PORT 25

TSO NETSTAT TCP TCPIP

TSO NETSTAT TCP TCPIP

TSO NETSTAT TCP TCPIP

TSO NETSTAT TCP TCPIP

No

te t

ha

t “NETSTAT …..(REPORT

”w

ill c

olle

ct

the

ou

tpu

t

to a

da

tase

t; f

or

ea

se

of

read

ing o

r in

pu

t to

a R

EX

X?

Page 11: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

“N

slo

okup”

-test domain name resolution (& “

DIG

”)

“Snm

p”

-where SNMP is supported, there are m

any

tools available to extract further inform

ation

(MIB data), once the problem area has been

located (e.g. Monitors, such as “

Im

ple

x”for

z/O

S ; “

iReasonin

g”elsewhere)

--

--

-

“TIV

OLI”

-IBM netw

ork tools (Monitor and trace facilities)

“C

trace”

-z/O

S trace tool

“EX

IG

EN

CE”-WDS trace “expert”system

(now

ZTS ! –

“ZEN

Trace &

Solv

e”)

Oth

er

Tools

. . . . .

Page 12: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

“TP

ing”

-(“TurboPing”) “PING”using TCP packets

“Tcpdum

p”

-(also W

indump& SSLdump) is a packet

sniffer found on m

any (most?) open platform

s.

“Eth

ereal”

-open system packet analyser (& “

Wir

eshark”)

“P

char”

-is a reim

plementation of Van Jacobson's

(“Mr Traceroute”) p

ath

charutility which

analyses the individual hops of a path.

“N

etc

at”

-Netcatis a utility which reads and writes data

across netw

ork connections. It is a netw

ork

debugging and exploration tool. (+ p

ort

-scanner

!)

“V

isualR

oute

”-path checker and graphical display

“N

eoTrace”

-(M

cAfee) Internet locator: enhanced traceroute

….e

tc

* N

ew

*

Ncat

from

Nm

ap

Oth

er

Tools

. . . . .

Page 13: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Tools

in D

eta

il . . . . .

“P

ing”

-“P

acket

INternetw

ork

Groper”, is usually

ICMP-based, which works if ICMP is allowed to

pass. If not perm

itted, then an application-based

ping can be used [e.g. “A

PIN

G”(UDP) or “TPing”

(TCP)].

Ping tests by sending out

IC

MP

Requestpackets, and receiving

IC

MP

Replies, therefore verifying up to (ISO)

layer 3

. . .

C:

C:C:

C:\ \\\>ping 66.249.85.99 (

>ping 66.249.85.99 (

>ping 66.249.85.99 (

>ping 66.249.85.99 ( www.google.co.uk

www.google.co.uk

www.google.co.uk

www.google.co.uk

----

-use I

P a

ddress o

r U

RL )

Pinging 66.249.85.99 with 32 bytes of data:

Pinging 66.249.85.99 with 32 bytes of data:

Pinging 66.249.85.99 with 32 bytes of data:

Pinging 66.249.85.99 with 32 bytes of data:

Reply from 66.249.85.99: bytes=32 time=22ms TTL=244

Reply from 66.249.85.99: bytes=32 time=22ms TTL=244

Reply from 66.249.85.99: bytes=32 time=22ms TTL=244

Reply from 66.249.85.99: bytes=32 time=22ms TTL=244

Reply from 66.249.85.99: bytes=32 time=22ms TTL=244

Reply from 66.249.85.99: bytes=32 time=22ms TTL=244

Reply from 66.249.85.99: bytes=32 time=22ms TTL=244

Reply from 66.249.85.99: bytes=32 time=22ms TTL=244

Reply from 66.249.85.99: bytes=32 time=42ms TTL=244

Reply from 66.249.85.99: bytes=32 time=42ms TTL=244

Reply from 66.249.85.99: bytes=32 time=42ms TTL=244

Reply from 66.249.85.99: bytes=32 time=42ms TTL=244

Reply from 66.249.85.99: bytes=32 time=22ms TTL=244

Reply from 66.249.85.99: bytes=32 time=22ms TTL=244

Reply from 66.249.85.99: bytes=32 time=22ms TTL=244

Reply from 66.249.85.99: bytes=32 time=22ms TTL=244

Ping statistics for 66.249.85.99: Packets: Sent=4,

Ping statistics for 66.249.85.99: Packets: Sent=4,

Ping statistics for 66.249.85.99: Packets: Sent=4,

Ping statistics for 66.249.85.99: Packets: Sent=4, Recvd

Recvd

Recvd

Recvd=4, Lost=0 (0% loss),

=4, Lost=0 (0% loss),

=4, Lost=0 (0% loss),

=4, Lost=0 (0% loss),

Approx. round trip times in milliseconds: Min=22ms, Max=42ms, Av

Approx. round trip times in milliseconds: Min=22ms, Max=42ms, Av

Approx. round trip times in milliseconds: Min=22ms, Max=42ms, Av

Approx. round trip times in milliseconds: Min=22ms, Max=42ms, Ave=27ms

e=27ms

e=27ms

e=27ms

Page 14: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Layers

. . . . . .

ISO

7-L

ayer N

etw

ork M

odel

Layer 1: Physical -

defines the real hardware.

Layer 2: Data Link -defines the form

at of data (frame/packet). (M

AC)

Layer 3: Netw

ork -

responsible for routing datagrams. (IP)

Layer 4: Transport -

manages data betw

een netw

ork and user. TCP/U

DP)

Layer 5: Session -defines the form

at of the data sent.

Layer 6: Presentation -

converts to/from local representation of data.

Layer 7: Application -

provides netw

ork services to the end-users.

TC

P/IP

4-L

ayer (

Unix

/D

oD

) N

etw

ork M

odel

Layer 1: Link -

defines the netw

ork hardware and device drivers.

Layer 2: Netw

ork -

addressing, routing, delivery. (IP / ICMP) (ARP)

Layer 3: Transport -

communication; end-to-end integrity. (TCP / UDP)

Layer 4: Application -

user applications.

(DNS, arp, telnet, smtp, http, ftp, traceroute….)

Page 15: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

ICM

P T

ypes/C

odes . . . . . .

ICMP Types:

ICMP Types:

ICMP Types:

ICMP Types:

0 Echo Reply

0 Echo Reply

0 Echo Reply

0 Echo Reply

3

3

3

3 Destination Unreachable

Destination Unreachable

Destination Unreachable

Destination Unreachable

4 Source Quench

5 Redirect

6 Alternate Host Address

8 Echo

8 Echo

8 Echo

8 Echo

9 Router Advertisement

10 Router Solicitation

11

11

11

11 Time Exceeded

Time Exceeded

Time Exceeded

Time Exceeded

12 Parameter Problem

13 Timestamp

14 Timestamp Reply

15 Information Request

16 Information Reply

17 Address Mask Request

18 Address Mask Reply

30

30

30

30 Traceroute

Traceroute

Traceroute

Traceroute

31 Datagram Conversion Error

32 Mobile Host Redirect

33 IPv6 Where-Are-You

34 IPv6 I-Am-Here

35 Mobile Registration Request

36 Mobile Registration Reply

37 Domain Name Request

38 Domain Name Reply

ICMP Codes:

ICMP Codes:

ICMP Codes:

ICMP Codes:

3

3

3

3 Destination Unreachable

Destination Unreachable

Destination Unreachable

Destination Unreachable

0 Net Unreachable

1 Host Unreachable

2 Protocol Unreachable

3 Port Unreachable

4 Fragmentation Needed and DF Set

5 Source Route Failed

6 Destination Network Unknown

7 Destination Host Unknown

8 Source Host Isolated

9 Communication with DestNetwork Prohibited

10 Communication with DestHost Prohibited

11 DestNetwork Unreachable for Type of Service

12 DestHost Unreachable for Type of Service

13 Communication Administratively Prohibited

14 Host Precedence Violation

15 Precedence cutoffin effect

11

11

11

11 Time Exceeded

Time Exceeded

Time Exceeded

Time Exceeded

0 Time to Live exceeded in Transit

1 Fragment Reassembly Time Exceeded

Ref: “www.iana.org/assignments/icmp-parameters”

Page 16: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Usage:

pin

g[-t

] [

-a] [

-n c

ount]

[-l siz

e] [

-f] [

-i T

TL] [

-v T

OS]

[-r

count]

[-s

count]

[[-j

host-

list]

| [

-k h

ost-

list]

][-w

tim

eout]

target_

nam

e

Opti

ons:

-tP

ing t

he s

pecif

ied h

ost

unti

l sto

pped.

To s

ee s

tati

sti

cs a

nd c

onti

nue -

type C

ontr

ol-

Break;

To s

top -

type C

ontr

ol-

C.

-aR

esolv

e a

ddresses t

o h

ostn

am

es.

-ncount

Num

ber o

f echo r

equests

to s

end.

-l

siz

eSend b

uff

er s

ize.

-fSet

Don't

Fragm

ent

flag in p

acket.

-ITTL

Tim

e T

o L

ive.

-vTO

SType O

f Servic

e.

-r

count

Record r

oute

for c

ount

hops.

-scount

Tim

esta

mp f

or c

ount

hops.

-jhost-

list

Loose s

ource r

oute

alo

ng h

ost-

list.

-khost-

list

Str

ict

source r

oute

alo

ng h

ost-

list.

-wti

meoutTim

eout

in m

illiseconds t

o w

ait

for e

ach r

eply

.

Tools

in D

eta

il . . . . .

PIN

G (Windows)

Page 17: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

C:

C:C:

C:\ \\\>ping 66.249.85.

>ping 66.249.85.

>ping 66.249.85.

>ping 66.249.85.55

55

55

55 � ���

non-existent addresses

Pinging 66.249.85.55 with 32 bytes of data:

Pinging 66.249.85.55 with 32 bytes of data:

Pinging 66.249.85.55 with 32 bytes of data:

Pinging 66.249.85.55 with 32 bytes of data:

Request timed out.

Request timed out.

Request timed out.

Request timed out.

Request timed out. (

Request timed out. (

Request timed out. (

Request timed out. (or

or

or

or “ “““Destination Unreachable ?

Destination Unreachable ?

Destination Unreachable ?

Destination Unreachable ?) )))

Request timed out. (

Request timed out. (

Request timed out. (

Request timed out. (if a return path is available

if a return path is available

if a return path is available

if a return path is available) )))

Request timed out.

Request timed out.

Request timed out.

Request timed out.

Ping statistics for 66.249.85.55: Packets: Sent=4,

Ping statistics for 66.249.85.55: Packets: Sent=4,

Ping statistics for 66.249.85.55: Packets: Sent=4,

Ping statistics for 66.249.85.55: Packets: Sent=4, Recvd

Recvd

Recvd

Recvd=0, Lost=4 (100% loss),

=0, Lost=4 (100% loss),

=0, Lost=4 (100% loss),

=0, Lost=4 (100% loss),

Draw

backs:

�Extra traffic on the netw

ork.

�“T

ime T

o L

ive”(T

TL) set to a high value to ensure penetration.

�Netw

ork devices m

ay n

ot

allow

Ping/ICMP and m

ay drop its priority.

�May not take the same path as user traffic; delay (latency) reported

may n

otbe representative for the application(s).

�Low feedback on fault and location.

Tools

in D

eta

il . . . . .

PIN

G

Page 18: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Usage:

tracert

[-d

] [

-h m

axim

um

_hops] [

-j h

ost-

list]

[-w

tim

eout]

target_

nam

e

Opti

ons:

-d

Do n

ot

resolv

e a

ddresses t

o h

ostn

am

es.

-h m

axim

um

_hops

Maxim

um

num

ber o

f hops t

o s

earch f

or t

arget.

-j h

ost-

list

Loose s

ource r

oute

alo

ng h

ost-

list.

-w t

imeout

Wait

tim

eout

milliseconds f

or e

ach r

eply

.

Tools

in D

eta

il . . . . .

TR

ACER

OU

TE (Windows)

�Also uses ICMP ! (although some platform

s use UDP)

�Good for spotting “loops”in the routing

�“T

ime T

o L

ive”(T

TL*) is incremented for each positive response.

�Each “hop”in the path is identified (Names m

ay be resolved!).

�“Per hop”round-trip delays can be identified.

�D

raw

backsare sim

ilar to those of “Ping”.

( *

= a

nti-lo

opin

g function o

f TCP/IP )

Page 19: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

C:

C:C:

C:\ \\\> >>>tracert

tracert

tracert

tracert

66.249.85.55 (

66.249.85.55 (

66.249.85.55 (

66.249.85.55 ( www.google.co.uk

www.google.co.uk

www.google.co.uk

www.google.co.uk

-----

-----

-----

-----

use

use

use

use IP address or URL

IP address or URL

IP address or URL

IP address or URL ) )))

Tracing route to 66.249.85.55 over a maximum of 30 hops

Tracing route to 66.249.85.55 over a maximum of 30 hops

Tracing route to 66.249.85.55 over a maximum of 30 hops

Tracing route to 66.249.85.55 over a maximum of 30 hops

1 1 ms 1 ms 1 ms 81.144.212.33

1 1 ms 1 ms 1 ms 81.144.212.33

1 1 ms 1 ms 1 ms 81.144.212.33

1 1 ms 1 ms 1 ms 81.144.212.33

2 7 ms 6 ms 6 ms 62.7.96.41

2 7 ms 6 ms 6 ms 62.7.96.41

2 7 ms 6 ms 6 ms 62.7.96.41

2 7 ms 6 ms 6 ms 62.7.96.41

3 6 ms 6 ms 6 ms core2

3 6 ms 6 ms 6 ms core2

3 6 ms 6 ms 6 ms core2

3 6 ms 6 ms 6 ms core2- ---gig2

gig2

gig2

gig2- ---1.kingston.ukcore.bt.net [194.72.3.2]

1.kingston.ukcore.bt.net [194.72.3.2]

1.kingston.ukcore.bt.net [194.72.3.2]

1.kingston.ukcore.bt.net [194.72.3.2]

4 7 ms 7 ms 7 ms core2

4 7 ms 7 ms 7 ms core2

4 7 ms 7 ms 7 ms core2

4 7 ms 7 ms 7 ms core2- ---pos7

pos7

pos7

pos7- ---3.ealing.ukcore.bt.net [62.6.201.42]

3.ealing.ukcore.bt.net [62.6.201.42]

3.ealing.ukcore.bt.net [62.6.201.42]

3.ealing.ukcore.bt.net [62.6.201.42]

5 7 ms 7 ms 7 ms core2

5 7 ms 7 ms 7 ms core2

5 7 ms 7 ms 7 ms core2

5 7 ms 7 ms 7 ms core2- ---pos10

pos10

pos10

pos10- ---0.redbus.ukcore.bt.net [194.74.65.202]

0.redbus.ukcore.bt.net [194.74.65.202]

0.redbus.ukcore.bt.net [194.74.65.202]

0.redbus.ukcore.bt.net [194.74.65.202]

6 8 ms 7 ms 8 ms 194.74.65.38

6 8 ms 7 ms 8 ms 194.74.65.38

6 8 ms 7 ms 8 ms 194.74.65.38

6 8 ms 7 ms 8 ms 194.74.65.38

7 7 ms 7 ms 7 ms 72.14.238.244

7 7 ms 7 ms 7 ms 72.14.238.244

7 7 ms 7 ms 7 ms 72.14.238.244

7 7 ms 7 ms 7 ms 72.14.238.244

8 16 ms 16 ms 16 ms 216.239.43.91

8 16 ms 16 ms 16 ms 216.239.43.91

8 16 ms 16 ms 16 ms 216.239.43.91

8 16 ms 16 ms 16 ms 216.239.43.91

9 22 ms 22 ms 22 ms 72.14.232.209

9 22 ms 22 ms 22 ms 72.14.232.209

9 22 ms 22 ms 22 ms 72.14.232.209

9 22 ms 22 ms 22 ms 72.14.232.209

10 * * * Request timed out.

10 * * * Request timed out.

10 * * * Request timed out.

10 * * * Request timed out.

11 * * * Request timed out.

11 * * * Request timed out.

11 * * * Request timed out.

11 * * * Request timed out.

12 *

12 *

12 *

12 * etc,etc

etc,etc

etc,etc

etc,etc

. . .

. . .

. . .

. . .

< <<<-----

-----

-----

-----

default maximum of 30

default maximum of 30

default maximum of 30

default maximum of 30

TR

ACER

OU

TE

Tools

in D

eta

il . . . . .

TRACEROUTE should be run in BOTH directions!!

Look for unsuitable (long) routes and high latency

Page 20: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Som

e p

latf

orm

s g

ive s

tatu

s indic

ato

rs…

!H -

Host

unreachable

. (D

esti

nati

on N

et

unreachable

) T

he r

oute

r h

as n

o

route

to t

he t

arget

syste

m.

!N -

Netw

ork u

nreachable

.

!P -

Proto

col unreachable

.

!S -

Source r

oute

failed. A

route

r is b

lockin

g s

ource-r

oute

d p

ackets

.

!F -

Fragm

enta

tion n

eeded. (

Check t

he M

TU

confi

gurati

on a

t th

e r

oute

r).

!X -

Com

munic

ati

on a

dm

inis

trati

vely

prohib

ited. Traceroute

blo

cked!

TR

ACER

OU

TE

Tools

in D

eta

il . . . . .

TRACEROUTE can be enhanced by visualization, as is often seen in

graphical traceroute

tools : such a

s . . .

Page 21: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Tra

ceR

oute

Tools

. . . . . .

Vis

ualR

oute

-1

Page 22: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Vis

ualR

oute

-2

Tra

ceR

oute

Tools

. . . . . .

Learn m

ore at:

http://w

ww.visualroute.com

Page 23: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Tra

ceR

oute

Tools

. . . . . .

Pin

gP

lott

er

Page 24: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Tools

in D

eta

il . . . . .

Where the target system is external to the local netw

ork, and especially where

routing is not available to/from the local netw

ork, there are several sites around

the W

orld that offer the ability to run “Ping”and “Traceroute”to be instigated by

remote control from their web site.

Basically, this is a “proxy”service ; the remote site issuing the test on your behalf.

This is suitable for determ

ining the general availability of thetarget system (i.e.

from anywhere on the Internet), but does not test specific routes.

“w

ww

.sam

spade.o

rg”used to be an excellent example of this type of service,

but is not currently available in its previous form

.

Further directions to such services can be found at :-

“ww

w.t

raceroute

.org”

TR

ACER

OU

TE –

Alt

ernati

ves

Page 25: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

NETSTA

T<

Opti

on |

Com

mand >

< T

arget

>

< O

utp

ut

> <

(Sele

ct

>

TSO

NETSTA

T C

ON

N

TSO

NETSTA

T S

OCK

TSO

NETSTA

T D

EV

TSO

NETSTA

T R

OU

TE

TSO

NETSTA

T T

CP

TCP

IP

Als

o “

onets

tat”

Can b

e issued f

rom

eit

her T

SO

or U

SS ;

the r

esult

s a

re t

he

sam

e.

Tools

in D

eta

il . . . . .

NETSTA

T(z/O

S)

NB. Netstatoptions will vary depending upon the platform

!

Note the following examples from z/O

S and W

indows. . .

Page 26: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Tools

in D

eta

il . . . . .

DevName

DevName

DevName

DevName: LCS1

: LCS1

: LCS1

: LCS1 DevType

DevType

DevType

DevType: LCS

: LCS

: LCS

: LCS DevNum

DevNum

DevNum

DevNum: 0E20

: 0E20

: 0E20

: 0E20

DevStatus

DevStatus

DevStatus

DevStatus: Ready

: Ready

: Ready

: Ready

LnkName

LnkName

LnkName

LnkName: ETH1

: ETH1

: ETH1

: ETH1 LnkType

LnkType

LnkType

LnkType: ETH

: ETH

: ETH

: ETH LnkStatus

LnkStatus

LnkStatus

LnkStatus: Ready

: Ready

: Ready

: Ready

NetNum

NetNum

NetNum

NetNum: 3

: 3

: 3

: 3 QueSize

QueSize

QueSize

QueSize: 0

: 0

: 0

: 0

IpBroadcastCapability

IpBroadcastCapability

IpBroadcastCapability

IpBroadcastCapability: Yes

: Yes

: Yes

: Yes

MacAddress

MacAddress

MacAddress

MacAddress: 000255305115

: 000255305115

: 000255305115

: 000255305115

ActMtu

ActMtu

ActMtu

ActMtu: 1500

: 1500

: 1500

: 1500

BSD Routing Parameters:

BSD Routing Parameters:

BSD Routing Parameters:

BSD Routing Parameters:

MTU Size: 00000 Metric: 00

MTU Size: 00000 Metric: 00

MTU Size: 00000 Metric: 00

MTU Size: 00000 Metric: 00

DestAddr

DestAddr

DestAddr

DestAddr: 0.0.0.0

: 0.0.0.0

: 0.0.0.0

: 0.0.0.0 SubnetMask

SubnetMask

SubnetMask

SubnetMask: 255.255.0.0

: 255.255.0.0

: 255.255.0.0

: 255.255.0.0

Packet Trace Setting:

Packet Trace Setting:

Packet Trace Setting:

Packet Trace Setting:

Protocol: 253

Protocol: 253

Protocol: 253

Protocol: 253 TrRecCnt

TrRecCnt

TrRecCnt

TrRecCnt: 00000000

: 00000000

: 00000000

: 00000000 PckLength

PckLength

PckLength

PckLength: FULL

: FULL

: FULL

: FULL

SrcPort

SrcPort

SrcPort

SrcPort: *

: *

: *

: * DestPort

DestPort

DestPort

DestPort: *

: *

: *

: *

IpAddr

IpAddr

IpAddr

IpAddr: *

: *

: *

: * SubNet

SubNet

SubNet

SubNet: *

: *

: *

: *

Multicast Specific:

Multicast Specific:

Multicast Specific:

Multicast Specific:

Multicast Capability: Yes

Multicast Capability: Yes

Multicast Capability: Yes

Multicast Capability: Yes

Group

Group

Group

Group RefCnt

RefCnt

RefCnt

RefCnt

-----

-----

-----

-----

------

------

------

------

224.0.0.1 0000000001

224.0.0.1 0000000001

224.0.0.1 0000000001

224.0.0.1 0000000001

Link Statistics:

Link Statistics:

Link Statistics:

Link Statistics:

BytesIn

BytesIn

BytesIn

BytesIn

= 420328206

= 420328206

= 420328206

= 420328206

Inbound Packets = 2865741

Inbound Packets = 2865741

Inbound Packets = 2865741

Inbound Packets = 2865741

Inbound Packets In Error = 1360

Inbound Packets In Error = 1360

Inbound Packets In Error = 1360

Inbound Packets In Error = 1360

Inbound Packets Discarded = 0

Inbound Packets Discarded = 0

Inbound Packets Discarded = 0

Inbound Packets Discarded = 0

Inbound Packets With No Protocol = 0

Inbound Packets With No Protocol = 0

Inbound Packets With No Protocol = 0

Inbound Packets With No Protocol = 0

NETSTA

T(z/O

S) –”D

EV”

MVS TCP/IP NETSTAT CS V1R5 TCPIP Name: TCPIP

MVS TCP/IP NETSTAT CS V1R5 TCPIP Name: TCPIP

MVS TCP/IP NETSTAT CS V1R5 TCPIP Name: TCPIP

MVS TCP/IP NETSTAT CS V1R5 TCPIP Name: TCPIP

Name: APIASHB Subtask: 007E1048

Name: APIASHB Subtask: 007E1048

Name: APIASHB Subtask: 007E1048

Name: APIASHB Subtask: 007E1048

Type:

Type:

Type:

Type: Dgram

Dgram

Dgram

Dgram

Status: UDP Conn: 00001A1A

Status: UDP Conn: 00001A1A

Status: UDP Conn: 00001A1A

Status: UDP Conn: 00001A1A

BoundTo

BoundTo

BoundTo

BoundTo: 192.168.1.156..12004

: 192.168.1.156..12004

: 192.168.1.156..12004

: 192.168.1.156..12004

ConnTo

ConnTo

ConnTo

ConnTo: *..*

: *..*

: *..*

: *..*

Type: Stream Status: Listen Conn: 00001A19

Type: Stream Status: Listen Conn: 00001A19

Type: Stream Status: Listen Conn: 00001A19

Type: Stream Status: Listen Conn: 00001A19

BoundTo

BoundTo

BoundTo

BoundTo: 192.168.1.156..12004

: 192.168.1.156..12004

: 192.168.1.156..12004

: 192.168.1.156..12004

ConnTo

ConnTo

ConnTo

ConnTo: 0.0.0.0..0

: 0.0.0.0..0

: 0.0.0.0..0

: 0.0.0.0..0

Name: APIASHB Subtask: 007E12D8

Name: APIASHB Subtask: 007E12D8

Name: APIASHB Subtask: 007E12D8

Name: APIASHB Subtask: 007E12D8

Type:

Type:

Type:

Type: Dgram

Dgram

Dgram

Dgram

Status: UDP Conn: 00001A18

Status: UDP Conn: 00001A18

Status: UDP Conn: 00001A18

Status: UDP Conn: 00001A18

BoundTo

BoundTo

BoundTo

BoundTo: 192.168.1.156..12000

: 192.168.1.156..12000

: 192.168.1.156..12000

: 192.168.1.156..12000

ConnTo

ConnTo

ConnTo

ConnTo: *..*

: *..*

: *..*

: *..*

Type: Stream Status: Listen Conn: 00001A17

Type: Stream Status: Listen Conn: 00001A17

Type: Stream Status: Listen Conn: 00001A17

Type: Stream Status: Listen Conn: 00001A17

BoundTo

BoundTo

BoundTo

BoundTo: 192.168.1.156..12000

: 192.168.1.156..12000

: 192.168.1.156..12000

: 192.168.1.156..12000

ConnTo

ConnTo

ConnTo

ConnTo: 0.0.0.0..0

: 0.0.0.0..0

: 0.0.0.0..0

: 0.0.0.0..0

. . .

. . .

. . .

. . .NETSTA

T(z/O

S) –”SOCK”

Page 27: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Usage:

nets

tat

[-a

] [

-b] [

-e] [

-n] [

-o] [

-p p

roto

] [

-r] [

-s] [

-v] [

inte

rval]

-a D

ispla

ys a

ll c

onnecti

ons a

nd lis

tenin

g p

orts

.-n

D

ispla

ys a

ddresses a

nd p

ort

num

bers in n

um

eric

al fo

rm

.-r

D

ispla

ys t

he r

outi

ng t

able

.. . .e

tc

C:

C:C:

C:\ \\\> >>>netstat

netstat

netstat

netstat

- ---a aaa

Active Connections

Active Connections

Active Connections

Active Connections

Proto Local Address Foreign Address State

Proto Local Address Foreign Address State

Proto Local Address Foreign Address State

Proto Local Address Foreign Address State

TCP

TCP

TCP

TCP wdsgdw:epmap

wdsgdw:epmap

wdsgdw:epmap

wdsgdw:epmap

0.0.0.0:0 LISTENING

0.0.0.0:0 LISTENING

0.0.0.0:0 LISTENING

0.0.0.0:0 LISTENING

TCP

TCP

TCP

TCP wdsgdw:microsoft

wdsgdw:microsoft

wdsgdw:microsoft

wdsgdw:microsoft- ---ds

dsds

ds

0.0.0.0:0 LISTENING

0.0.0.0:0 LISTENING

0.0.0.0:0 LISTENING

0.0.0.0:0 LISTENING

TCP wdsgdw:1028 0.0.0.0:0 LISTENING

TCP wdsgdw:1028 0.0.0.0:0 LISTENING

TCP wdsgdw:1028 0.0.0.0:0 LISTENING

TCP wdsgdw:1028 0.0.0.0:0 LISTENING

TCP wdsgdw:1241 0.0.0.0:0 LISTENING

TCP wdsgdw:1241 0.0.0.0:0 LISTENING

TCP wdsgdw:1241 0.0.0.0:0 LISTENING

TCP wdsgdw:1241 0.0.0.0:0 LISTENING

TCP wdsgdw:10110 0.0.0.0:0 LISTENING

TCP wdsgdw:10110 0.0.0.0:0 LISTENING

TCP wdsgdw:10110 0.0.0.0:0 LISTENING

TCP wdsgdw:10110 0.0.0.0:0 LISTENING

UDP

UDP

UDP

UDP wdsgdw:microsoft

wdsgdw:microsoft

wdsgdw:microsoft

wdsgdw:microsoft- ---ds

dsds

ds

*:*

*:*

*:*

*:*

UDP

UDP

UDP

UDP wdsgdw:isakmp

wdsgdw:isakmp

wdsgdw:isakmp

wdsgdw:isakmp

*:*

*:*

*:*

*:*

UDP wdsgdw:1033 *:*

UDP wdsgdw:1033 *:*

UDP wdsgdw:1033 *:*

UDP wdsgdw:1033 *:*

UDP wdsgdw:4500 *:*

UDP wdsgdw:4500 *:*

UDP wdsgdw:4500 *:*

UDP wdsgdw:4500 *:*

UDP

UDP

UDP

UDP wdsgdw:ntp

wdsgdw:ntp

wdsgdw:ntp

wdsgdw:ntp

*:*

*:*

*:*

*:*

UDP wdsgdw:1900 *:*

UDP wdsgdw:1900 *:*

UDP wdsgdw:1900 *:*

UDP wdsgdw:1900 *:*

Tools

in D

eta

il . . . . .

NETSTA

T (Windows)

Page 28: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

In g

eneral, it

is q

uit

e c

om

mon t

o s

eek a

n I

P t

arget

usin

g a

UR

L (which acts

rather like a PATH name).

This

enta

ils s

endin

g t

he U

RL t

o a

“D

om

ain

Nam

e S

erver”

(or “

Resolv

er”)

in z

/O

S t

erm

s) t

o h

ave t

he n

am

e t

ransla

ted (

i.e. a “

table

lookup”) into

an

IP

address (

this

may o

ccur

locally b

y u

se o

f th

e “

Hosts

”file

**).

The I

P a

ddress r

etu

rned is t

hen u

sed t

o a

ddress t

he t

arget.

--

--

--

--

--

--

--

--

--

--

--

--

--

-

This

pro

cess m

ay a

lso b

e p

erf

orm

ed in r

evers

e;

i.e. th

e D

NS s

erv

er

can t

ransla

te

an I

P a

ddre

ss into

a U

RL !

The u

se o

f a U

RL m

eans t

hat

rem

ote

servic

es c

an b

e f

ailed-o

ver, relo

cate

d

or r

ebuilt

wit

hout

the u

sers n

eedin

g t

o k

now

!

Tools

in D

eta

il . . . . .

DN

S . . .

**HOSTS file from W

indows :-

( C:\WINDOWS\system32\drivers\etc )

127.0.0.1 localhost

192.168.1.45 lizzie

192.168.1.45 wds.local

192.168.1.45 wds

192.168.1.43 wdsnfs

Page 29: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

The g

lobal D

om

ain

Nam

e S

yste

m is a

hie

rarchy o

f servers/servic

es

spread

across t

he I

nte

rnet.

At

its c

ore is a

set

of

servers t

hat

manage

the b

ase

dom

ain

s;

such a

s “

com

”, “edu”, “gov”

…etc

When a

nam

e is “

looked u

p”

it h

appens f

rom

rig

ht

to left

-recursiv

ely

.

Take w

ww

.google

.co.u

k…

. Fir

st

the s

erver is locate

d t

hat

contr

ols

the “

uk”

dom

ain

(there is

an implied “root”

service where all top-level servers are known).

. This

will in

dic

ate

the “

co.u

k”

server ;

whic

h in t

urn w

ill in

dic

ate

th

e “

google

.co.u

k”

server.

. The “

google

.co.u

k”

server w

ill have I

P a

ddresses (

an “

A”

record)

for w

eb (

“w

ww

”) a

nd m

ail s

ervic

es (note: “w

ww”is not the only

canonical form

used!)

Tools

in D

eta

il . . . . .

DN

S . . .

NA

MED

.CO

NF

-lists the “zones”(eg. “google.co.uk”)

ZO

NE F

ILES

-hold the IP addresses

NB. Zone inform

ation changed at the bottom of a “layer”

is

propagated upwards by “Zone Transfer”

at preset times.

Page 30: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Usage:

nslo

okup

NA

ME , or

, N

AM

E1 N

AM

E2 � ���

(cfz/O

S “Resolver”

)or

com

mand

set

option all

all

all

all

[ [[[no]debug

no]debug

no]debug

no]debug

[no]d2

[no]d2

[no]d2

[no]d2

[ [[[no]defname

no]defname

no]defname

no]defname

[ [[[no]recurse

no]recurse

no]recurse

no]recurse

[ [[[no]search

no]search

no]search

no]search

[ [[[no]vc

no]vc

no]vc

no]vc

domain=NAME

domain=NAME

domain=NAME

domain=NAME

srchlist

srchlist

srchlist

srchlist=N1[/N2/.../N6]

=N1[/N2/.../N6]

=N1[/N2/.../N6]

=N1[/N2/.../N6]

root=NAME

root=NAME

root=NAME

root=NAME

retry=x

retry=x

retry=x

retry=x

timeout=X

timeout=X

timeout=X

timeout=X

type=X

type=X

type=X

type=X

querytype

querytype

querytype

querytype=X

=X=X

=X

class=X

class=X

class=X

class=X

[ [[[no]msxfr

no]msxfr

no]msxfr

no]msxfr

ixfrver

ixfrver

ixfrver

ixfrver=X

=X

=X

=X

Server

NA

ME

Exit

Tools

in D

eta

il . . . . .

NSLO

OK

UP

(Windows)

“Lookup”failure will cause connectivity failure, and symptoms can be

mistaken for a routing problem!

----

z/O

S often acts as a relay, passing the requests on to a netw

ork

DNS

server.

Page 31: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

C:

C:C:

C:\ \\\> >>>nslookup

nslookup

nslookup

nslookup

>

> >

> set debug

set debug

set debug

set debug

>

> >

> www.google.co.uk

www.google.co.uk

www.google.co.uk

www.google.co.uk

Server:

Server:

Server:

Server: my.router

my.router

my.router

my.router

Address: 192.168.27.1

Address: 192.168.27.1

Address: 192.168.27.1

Address: 192.168.27.1

------------

------------

------------

------------

(debug information)

(debug information)

(debug information)

(debug information)

Got answer:

Got answer:

Got answer:

Got answer:

HEADER:

HEADER:

HEADER:

HEADER:

opcode

opcode

opcode

opcode

= QUERY, id = 3,

= QUERY, id = 3,

= QUERY, id = 3,

= QUERY, id = 3, rcode

rcode

rcode

rcode

= NOERROR

= NOERROR

= NOERROR

= NOERROR

header flags: response, want recursion, recursion avail

header flags: response, want recursion, recursion avail

header flags: response, want recursion, recursion avail

header flags: response, want recursion, recursion avail. ...

questions = 1, answers = 1, authority records = 0, ad

questions = 1, answers = 1, authority records = 0, ad

questions = 1, answers = 1, authority records = 0, ad

questions = 1, answers = 1, authority records = 0, additional = 0

ditional = 0

ditional = 0

ditional = 0

QUESTIONS:

QUESTIONS:

QUESTIONS:

QUESTIONS:

www.google.co.uk.uk.willdata.com

www.google.co.uk.uk.willdata.com

www.google.co.uk.uk.willdata.com

www.google.co.uk.uk.willdata.com, type = A, class = IN

, type = A, class = IN

, type = A, class = IN

, type = A, class = IN

ANSWERS:

ANSWERS:

ANSWERS:

ANSWERS:

- --->

>

>

> www.google.co.uk.uk.willdata.com

www.google.co.uk.uk.willdata.com

www.google.co.uk.uk.willdata.com

www.google.co.uk.uk.willdata.com

internet address = 212.69.199.183

internet address = 212.69.199.183

internet address = 212.69.199.183

internet address = 212.69.199.183

ttl

ttl

ttl

ttl

= 60 (1 min)

= 60 (1 min)

= 60 (1 min)

= 60 (1 min)

------------

------------

------------

------------

Non

Non

Non

Non- ---authoritative answer:

authoritative answer:

authoritative answer:

authoritative answer:

Name:

Name:

Name:

Name: www.google.co.uk.uk.willdata.com

www.google.co.uk.uk.willdata.com

www.google.co.uk.uk.willdata.com

www.google.co.uk.uk.willdata.com

Address: 212.69.199.183

Address: 212.69.199.183

Address: 212.69.199.183

Address: 212.69.199.183

Tools

in D

eta

il . . . . .

NSLO

OK

UP

(Windows)

� ���--

---(

Retr

ieved from

a c

ache! )

Page 32: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Dom

ain

Inte

rnet

Groper: A tool for system administrators; it issues DNS queries

and form

ats/interprets the answers…. Quite popular (a

llegedly

!) with hackers…

dig @

dig @

dig @

dig @lizzie

lizzie

lizzie

lizzie

www.google.co.uk

www.google.co.uk

www.google.co.uk

www.google.co.uk

any

any

any

any

; <<>>

; <<>>

; <<>>

; <<>> DiG

DiG

DiG

DiG

9.3.1 <<>> @

9.3.1 <<>> @

9.3.1 <<>> @

9.3.1 <<>> @lizzie

lizzie

lizzie

lizzie

www.google.co.uk

www.google.co.uk

www.google.co.uk

www.google.co.uk

any

any

any

any

; (1 server found) ; global options:

; (1 server found) ; global options:

; (1 server found) ; global options:

; (1 server found) ; global options: printcmd

printcmd

printcmd

printcmd

; Got answer:

; Got answer:

; Got answer:

; Got answer:

;;

;;

;;

;; - --->>HEADER<<

>>HEADER<<

>>HEADER<<

>>HEADER<<- ---

opcode

opcode

opcode

opcode: QUERY, status: NOERROR, id: 16774

: QUERY, status: NOERROR, id: 16774

: QUERY, status: NOERROR, id: 16774

: QUERY, status: NOERROR, id: 16774

;; flags:

;; flags:

;; flags:

;; flags: qr

qrqr

qr

rd

rd

rd

rd ra

rara

ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:

;; QUESTION SECTION:

;; QUESTION SECTION:

;; QUESTION SECTION:

; ;;;www.google.co.uk

www.google.co.uk

www.google.co.uk

www.google.co.uk. IN ANY

. IN ANY

. IN ANY

. IN ANY

;; ANSWER SECTION:

;; ANSWER SECTION:

;; ANSWER SECTION:

;; ANSWER SECTION:

www.google.co.uk

www.google.co.uk

www.google.co.uk

www.google.co.uk. 86399 IN CNAME

. 86399 IN CNAME

. 86399 IN CNAME

. 86399 IN CNAME www.google.com

www.google.com

www.google.com

www.google.com. ...

;; Query time: 63

;; Query time: 63

;; Query time: 63

;; Query time: 63 msec

msec

msec

msec

;; SERVER: 192.168.1.45#53(192.168.1.45)

;; SERVER: 192.168.1.45#53(192.168.1.45)

;; SERVER: 192.168.1.45#53(192.168.1.45)

;; SERVER: 192.168.1.45#53(192.168.1.45)

;; WHEN: Mon Feb 5 14:11:43 2007

;; WHEN: Mon Feb 5 14:11:43 2007

;; WHEN: Mon Feb 5 14:11:43 2007

;; WHEN: Mon Feb 5 14:11:43 2007

;; MSG SIZE rcvd: 62

;; MSG SIZE rcvd: 62

;; MSG SIZE rcvd: 62

;; MSG SIZE rcvd: 62

. . . . .>

. . . . .>

. . . . .>

. . . . .>

Tools

in D

eta

il . . . . .

DIG

Usage:

dig

[@

glo

bal-

server] [

dom

ain

] [

q-t

ype] [

q-c

lass] {

q-o

pt}

{glo

bal-

d-o

pt}

host

[@

local-

server] {

local-

d-o

pt}

[ h

ost

[@

local-

server] {

local-

d-o

pt}

[...]

]

Page 33: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

>. . . . .

>. . . . .

>. . . . .

>. . . . .

dig @

dig @

dig @

dig @lizzie

lizzie

lizzie

lizzie

www.google.com

www.google.com

www.google.com

www.google.com

any

any

any

any

; <<>>

; <<>>

; <<>>

; <<>> DiG

DiG

DiG

DiG

9.3.1 <<>> @

9.3.1 <<>> @

9.3.1 <<>> @

9.3.1 <<>> @lizzie

lizzie

lizzie

lizzie

www.google.com

www.google.com

www.google.com

www.google.com

any

any

any

any

; (1 server found) ; global options:

; (1 server found) ; global options:

; (1 server found) ; global options:

; (1 server found) ; global options: printcmd

printcmd

printcmd

printcmd

; Got answer:

; Got answer:

; Got answer:

; Got answer:

;;

;;

;;

;; - --->>HEADER<<

>>HEADER<<

>>HEADER<<

>>HEADER<<- ---

opcode

opcode

opcode

opcode: QUERY, status: NOERROR, id: 60773

: QUERY, status: NOERROR, id: 60773

: QUERY, status: NOERROR, id: 60773

: QUERY, status: NOERROR, id: 60773

;; flags:

;; flags:

;; flags:

;; flags: qr

qrqr

qr

rd

rd

rd

rd ra

rara

ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 3

; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 3

; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 3

; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 3

;; QUESTION SECTION:

;; QUESTION SECTION:

;; QUESTION SECTION:

;; QUESTION SECTION:

; ;;;www.google.com

www.google.com

www.google.com

www.google.com. IN ANY

. IN ANY

. IN ANY

. IN ANY

;; ANSWER SECTION:

;; ANSWER SECTION:

;; ANSWER SECTION:

;; ANSWER SECTION:

www.google.com

www.google.com

www.google.com

www.google.com. 86400 IN CNAME

. 86400 IN CNAME

. 86400 IN CNAME

. 86400 IN CNAME www.l.google.com

www.l.google.com

www.l.google.com

www.l.google.com. ...

;; ADDITIONAL SECTION:

;; ADDITIONAL SECTION:

;; ADDITIONAL SECTION:

;; ADDITIONAL SECTION:

www.l.google.com

www.l.google.com

www.l.google.com

www.l.google.com. 149 IN A 66.249.93.104

. 149 IN A 66.249.93.104

. 149 IN A 66.249.93.104

. 149 IN A 66.249.93.104

www.l.google.com

www.l.google.com

www.l.google.com

www.l.google.com. 149 IN A 66.249.93.99

. 149 IN A 66.249.93.99

. 149 IN A 66.249.93.99

. 149 IN A 66.249.93.99

www.l.google.com

www.l.google.com

www.l.google.com

www.l.google.com. 149 IN A 66.249.93.147

. 149 IN A 66.249.93.147

. 149 IN A 66.249.93.147

. 149 IN A 66.249.93.147

;; Query time: 56

;; Query time: 56

;; Query time: 56

;; Query time: 56 msec

msec

msec

msec

;; SERVER: 192.168.1.45#53(192.168.1.45)

;; SERVER: 192.168.1.45#53(192.168.1.45)

;; SERVER: 192.168.1.45#53(192.168.1.45)

;; SERVER: 192.168.1.45#53(192.168.1.45)

;; WHEN: Mon Feb 5 14:15:13 2007

;; WHEN: Mon Feb 5 14:15:13 2007

;; WHEN: Mon Feb 5 14:15:13 2007

;; WHEN: Mon Feb 5 14:15:13 2007

;; MSG SIZE rcvd: 100

;; MSG SIZE rcvd: 100

;; MSG SIZE rcvd: 100

;; MSG SIZE rcvd: 100

Tools

in D

eta

il . . . . .

DIG

Page 34: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Domain name:

Domain name:

Domain name:

Domain name:

google.co.uk

google.co.uk

google.co.uk

google.co.uk

Registrant:

Registrant:

Registrant:

Registrant:

Google Inc

Google Inc

Google Inc

Google Inc

Registrant type:

Registrant type:

Registrant type:

Registrant type:

Non

Non

Non

Non- ---UK Corporation

UK Corporation

UK Corporation

UK Corporation

Registrant's address:

Registrant's address:

Registrant's address:

Registrant's address:

1600 Amphitheatre Parkway

1600 Amphitheatre Parkway

1600 Amphitheatre Parkway

1600 Amphitheatre Parkway

Mountain View

Mountain View

Mountain View

Mountain View

CA

CACA

CA94043

94043

94043

94043

United States

United States

United States

United States

Registrant's agent:

Registrant's agent:

Registrant's agent:

Registrant's agent:

Markmonitor

Markmonitor

Markmonitor

Markmonitor

Inc. t/a

Inc. t/a

Inc. t/a

Inc. t/a Markmonitor

Markmonitor

Markmonitor

Markmonitor

[Tag = MARKMONITOR]

[Tag = MARKMONITOR]

[Tag = MARKMONITOR]

[Tag = MARKMONITOR]

URL: http://

URL: http://

URL: http://

URL: http://www.markmonitor.com

www.markmonitor.com

www.markmonitor.com

www.markmonitor.com

Tools

in D

eta

il . . . . .

WH

OIS

Relevant dates:

Relevant dates:

Relevant dates:

Relevant dates:

Registered on: 14

Registered on: 14

Registered on: 14

Registered on: 14- ---Feb

Feb

Feb

Feb- ---1999

1999

1999

1999

Renewal date: 14

Renewal date: 14

Renewal date: 14

Renewal date: 14- ---Feb

Feb

Feb

Feb- ---2009

2009

2009

2009

Last updated: 17

Last updated: 17

Last updated: 17

Last updated: 17- ---Jan

Jan

Jan

Jan- ---2007

2007

2007

2007

Registration status:

Registration status:

Registration status:

Registration status:

Renewal request being processed.

Renewal request being processed.

Renewal request being processed.

Renewal request being processed.

Name servers:

Name servers:

Name servers:

Name servers:

ns1.google.com

ns1.google.com

ns1.google.com

ns1.google.com

ns2.google.com

ns2.google.com

ns2.google.com

ns2.google.com

ns3.google.com

ns3.google.com

ns3.google.com

ns3.google.com

ns4.google.com

ns4.google.com

ns4.google.com

ns4.google.com

Page 35: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Esti

mate

s b

andw

idth

, la

tency a

nd p

acket

loss o

n n

etw

ork lin

ks.

This is a re-w

orking of the “pathchar”

utility, written by Van Jacobson and, like

traceroute, is based on repeated packet transmission and TTL variation (itcan

use ICMP or UDP).

It is available for most

“*nix”systems : It works for IPv4 & IPv6.

Traceroute

(UDP) knows when it has found its target by using a port number

beyond the “norm

al range”…

when ICMP “port unreachable”is returned it’s

there!

Pcharsends m

any packets, one hop at a tim

e, with varying the sizes, until the

target is reached or the path fails. It calculates the latency from the ICMP

message response tim

es, and the throughput per hop from the variance in

response speeds. Collectively, this also gives the overall round-trip delay for the

whole path.

It is not fool-proof ; it’s traffic m

ay n

otbe allowed ; it is not a “Holy Grail”; but

it does give a good indication!

Tools

in D

eta

il . . . . .

Pchar

Page 36: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

pchar

to www.l.google.com

(66.249.93.104) using UDP/IPv4

Using raw socket input

Packet size increments from 32 to 1500 by 32

46 test(s) per repetition : 32 repetition(s) per hop

Warning: target host did not respond to initial test.

Warning: target host did not respond to initial test.

Warning: target host did not respond to initial test.

Warning: target host did not respond to initial test.

0: 192.168.1.231 (dhcp-192-168-1-231.uk.willdata.com)

Partial loss: 0 / 1472 (0%)

Partial char: rtt

= 0.959029 ms, (b = 0.001150 ms/B), r2 = 0.999475

stddev

rtt= 0.003212, stddevb = 0.000004

Partial queueing: avg

= 0.000171 ms (148 bytes)

Hop char: rtt

= 0.959029 ms, bw

= 6954.330709 Kbps

Hop queueing: avg

= 0.000171 ms (148 bytes)

1: 81.144.212.33 (81.144.212.33)

Partial loss: 0 / 1472 (0%)

Partial char: rtt

= 5.784087 ms, (b = 0.005317 ms/B), r2 = 0.999798

stddev

rtt= 0.009218, stddevb = 0.000011

Partial queueing: avg

= 0.002336 ms (667 bytes)

Hop char: rtt

= 4.825058 ms, bw

= 1919.855256 Kbps

Hop queueing: avg

= 0.002165 ms (519 bytes)

2: 62.7.96.41 (62.7.96.41)

Partial loss: 0 / 1472 (0%)

Partial char: rtt

= 5.824306 ms, (b = 0.005317 ms/B), r2 = 0.999847

stddev

rtt= 0.008008, stddevb = 0.000010

Partial queueing: avg

= 0.001486 ms (667 bytes)

Hop char: rtt

= 0.040220 ms, bw

= --.---

Kbps

Hop queueing: avg

= -0.000850 ms (0 bytes)

3: 194.72.3.66 (core2-gig10-1.kingston.ukcore.bt.net)

???

???

???

??? - ---

process hangs at this point!

process hangs at this point!

process hangs at this point!

process hangs at this point!

Tools

in D

eta

il . . . . .

Pchar

-./pcharwww.google.co.uk

This example shows a

“pchar”

test across a path

where icmpresponses are

notallowed.

Page 37: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

pcharto 192.168.1.8 (192.168.1.8) using UDP/IPv4

Using raw socket input

Packet size increments from 32 to 1500 by 32

46 test(s) per repetition : 32 repetition(s) per hop

0: 192.168.1.231 (dhcp-192-168-1-231.uk.willdata.com)

Partial loss: 0 / 1472 (0%)

Partial char: rtt= 10.792415 ms, (b = 0.003369 ms/B), r2 = 0.157013

stddevrtt= 0.950840, stddevb = 0.001177

Partial queueing: avg= 0.015037 ms (4463 bytes)

Hop char: rtt= 10.792415 ms, bw= 2374.706954 Kbps

Hop queueing: avg= 0.015037 ms (4463 bytes)

1: 192.168.1.8 (zplex.uk.willdata.com)

Path length: 1 hops

Path length: 1 hops

Path length: 1 hops

Path length: 1 hops

Path char:

Path char:

Path char:

Path char: rtt

rtt

rtt

rtt= 10.792415 ms r2 = 0.157013

Path bottleneck

Path bottleneck

Path bottleneck

Path bottleneck: 2374.706954 Kbps

Path pipe

Path pipe

Path pipe

Path pipe: 3203 bytes

Path

Path

Path

Path queueing

queueing

queueing

queueing: average = 0.015037 ms (4463 bytes)

Start time: Thu Feb 1 09:07:32 2007

End time: Thu Feb 1 09:14:22 2007

Tools

in D

eta

il . . . . .

Pchar

-./pchar192.168.1.8 (a local address)

Partial loss = number of pkts/ percentage pktslost

Partial char = RTT, delay Byte, min delay pkt

Partial queueing= ave. queue of data incl. of this hop

Hop char

= RTT and b/width for the current hop

Hop queueing

= average queue of data this hop

Path bottleneck = “bottleneck”(achieved) bandwidth

Path pipe = Bandwidth-Delay Product = traffic

“on the wire”(cfRWIN buffer)

Page 38: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Rem

em

ber:

ICMP m

ay be restricted over the test path

Not all platform

s have the same controls or defaults

Think of the impact on the netw

ork of using these kind of tools!!

The figures produced are estimates (ref. pchar“m

an pages”of pcharand, as

already m

entioned for some previous tools, the results will probably not reflect

the exact behaviour of the applications using the same path.

Tools

in D

eta

il . . . . .

Pchar

Learn m

ore at:

http://w

ww.kitchenlab.org/w

ww/bmah/Software/pchar/

Page 39: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Netc

at-a read/w

rite utility for netw

orks (TCP or UDP).

It can be used on its own or be driven by user code.

It is also a very powerful netw

ork debugging and exploration

tool, which can create alm

ost any kind of connection:-

•Outbound or inbound, TCP or UDP, to or from any ports

•Full DNS forw

ard/reverse checking, with appropriate warnings

•Ability to use any local source port

•Ability to use any locally-configured netw

ork source address

•Built-in port-scanning capabilities, with randomizer

•Can read command line arguments from standard input

•Slow-send m

ode, one line every N seconds

•Hex dump of transmitted and received data

•Ability to let another program service established connections

•Telnet-options responder

Tools

in D

eta

il . . . . .

Netc

at

Good for testing applications and application paths, but does not

“test”

or measure the netw

ork itself.

Bew

are o

f m

isuse!

Page 40: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

connect to somewhere:

connect to somewhere:

connect to somewhere:

connect to somewhere: nc

ncnc

nc

[ [[[- ---options] hostname

options] hostname

options] hostname

options] hostname port[s

port[s

port[s

port[s] [ports] ...

] [ports] ...

] [ports] ...

] [ports] ...

listen for inbound:

listen for inbound:

listen for inbound:

listen for inbound: nc

ncnc

nc

- ---l

l l

l - ---p port [options] [hostname] [port]

p port [options] [hostname] [port]

p port [options] [hostname] [port]

p port [options] [hostname] [port]

options:

options:

options:

options:- ---d detach from console, background mode

d detach from console, background mode

d detach from console, background mode

d detach from console, background mode

- ---e

e e

e prog

prog

prog

prog

inbound program to exec [dangerous!!]

inbound program to exec [dangerous!!]

inbound program to exec [dangerous!!]

inbound program to exec [dangerous!!]

- ---g gateway source

g gateway source

g gateway source

g gateway source- ---routing hop

routing hop

routing hop

routing hop point[s

point[s

point[s

point[s], up to 8

], up to 8

], up to 8

], up to 8

- ---G num source

G num source

G num source

G num source- ---routing pointer: 4, 8, 12, ...

routing pointer: 4, 8, 12, ...

routing pointer: 4, 8, 12, ...

routing pointer: 4, 8, 12, ...

- ---h this help

h this help

h this help

h this help

- ---i

i i

i secs

secs

secs

secs

delay interval for lines sent, ports scanned

delay interval for lines sent, ports scanned

delay interval for lines sent, ports scanned

delay interval for lines sent, ports scanned

- ---l listen mode, for inbound connects

l listen mode, for inbound connects

l listen mode, for inbound connects

l listen mode, for inbound connects

- ---L listen harder, re

L listen harder, re

L listen harder, re

L listen harder, re- ---listen on socket close

listen on socket close

listen on socket close

listen on socket close

- ---n numeric

n numeric

n numeric

n numeric- ---only IP addresses, no DNS

only IP addresses, no DNS

only IP addresses, no DNS

only IP addresses, no DNS

- ---o file hex dump of traffic

o file hex dump of traffic

o file hex dump of traffic

o file hex dump of traffic

- ---p port local port number

p port local port number

p port local port number

p port local port number

- ---r randomize local and remote ports

r randomize local and remote ports

r randomize local and remote ports

r randomize local and remote ports

- ---s

s s

s addr

addr

addr

addr

local source address

local source address

local source address

local source address

- ---t answer TELNET negotiation

t answer TELNET negotiation

t answer TELNET negotiation

t answer TELNET negotiation

- ---u UDP mode

u UDP mode

u UDP mode

u UDP mode

- ---v verbose [use twice to be more verbose]

v verbose [use twice to be more verbose]

v verbose [use twice to be more verbose]

v verbose [use twice to be more verbose]

- ---w

w w

w secs

secs

secs

secs

timeout for connects and final net reads

timeout for connects and final net reads

timeout for connects and final net reads

timeout for connects and final net reads

- ---z zero

z zero

z zero

z zero- ---I/O mode [used for scanning]

I/O mode [used for scanning]

I/O mode [used for scanning]

I/O mode [used for scanning]

port numbers can be individual or ranges:

port numbers can be individual or ranges:

port numbers can be individual or ranges:

port numbers can be individual or ranges: m mmm- ---n nnn

[inclusive]

[inclusive]

[inclusive]

[inclusive]

Tools

in D

eta

il . . . . .

Netc

at

Learn m

ore at:

http://netcat.sourceforge.net/

http://nmap.org/ncat/

Page 41: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

C:

C:C:

C:\ \\\> >>>nc

ncnc

nc

- ---v

v v

v www.google.co.uk

www.google.co.uk

www.google.co.uk

www.google.co.uk

80

8080

80

www.l.google.com

www.l.google.com

www.l.google.com

www.l.google.com

[216.239.59.103] 80 (http) open

[216.239.59.103] 80 (http) open

[216.239.59.103] 80 (http) open

[216.239.59.103] 80 (http) open

GET / HTTP/1.0

GET / HTTP/1.0

GET / HTTP/1.0

GET / HTTP/1.0

HTTP/1.0 302 Found

HTTP/1.0 302 Found

HTTP/1.0 302 Found

HTTP/1.0 302 Found

Location: http://

Location: http://

Location: http://

Location: http://www.google.co.uk

www.google.co.uk

www.google.co.uk

www.google.co.uk/ ///

Cache

Cache

Cache

Cache- ---Control: private

Control: private

Control: private

Control: private

Set

Set

Set

Set- ---Cookie:

Cookie:

Cookie:

Cookie:

PREF=ID=bebf53d3e8c044c6:TM=1170500572:LM=1170500572:S=DBxO29wrW

PREF=ID=bebf53d3e8c044c6:TM=1170500572:LM=1170500572:S=DBxO29wrW

PREF=ID=bebf53d3e8c044c6:TM=1170500572:LM=1170500572:S=DBxO29wrW

PREF=ID=bebf53d3e8c044c6:TM=1170500572:LM=1170500572:S=DBxO29wrWXh5ex5E;

Xh5ex5E;

Xh5ex5E;

Xh5ex5E;

expires=Sun, 17

expires=Sun, 17

expires=Sun, 17

expires=Sun, 17- ---Jan

Jan

Jan

Jan- ---2038 19:14:07 G

2038 19:14:07 G

2038 19:14:07 G

2038 19:14:07 G

MT; path=/; domain=.

MT; path=/; domain=.

MT; path=/; domain=.

MT; path=/; domain=.google.com

google.com

google.com

google.com

Content

Content

Content

Content- ---Type: text/html

Type: text/html

Type: text/html

Type: text/html

Server: GWS/2.1

Server: GWS/2.1

Server: GWS/2.1

Server: GWS/2.1

Content

Content

Content

Content- ---Length: 221

Length: 221

Length: 221

Length: 221

Date: Sat, 03 Feb 2007 11:02:52 GMT

Date: Sat, 03 Feb 2007 11:02:52 GMT

Date: Sat, 03 Feb 2007 11:02:52 GMT

Date: Sat, 03 Feb 2007 11:02:52 GMT

Connection: Keep

Connection: Keep

Connection: Keep

Connection: Keep- ---Alive

Alive

Alive

Alive

<HTML><HEAD><meta http

<HTML><HEAD><meta http

<HTML><HEAD><meta http

<HTML><HEAD><meta http- ---equiv="content

equiv="content

equiv="content

equiv="content- ---type" content="text/

type" content="text/

type" content="text/

type" content="text/html;charset

html;charset

html;charset

html;charset=utf

=utf

=utf

=utf- ---8">

8">

8">

8">

<TITLE>302 Moved</TITLE></HEAD><BODY>

<TITLE>302 Moved</TITLE></HEAD><BODY>

<TITLE>302 Moved</TITLE></HEAD><BODY>

<TITLE>302 Moved</TITLE></HEAD><BODY>

<H1>302 Moved</H1>

<H1>302 Moved</H1>

<H1>302 Moved</H1>

<H1>302 Moved</H1>

The document has moved

The document has moved

The document has moved

The document has moved

<A HREF="http://

<A HREF="http://

<A HREF="http://

<A HREF="http://www.google.co.uk

www.google.co.uk

www.google.co.uk

www.google.co.uk/">here</A>.

/">here</A>.

/">here</A>.

/">here</A>.

</BODY></HTML>

</BODY></HTML>

</BODY></HTML>

</BODY></HTML>

Tools

in D

eta

il . . . . .

Netc

at

-Retrieve page from web server

Page 42: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

c:

c:c:

c:\ \\\> >>>nc

ncnc

nc

- ---l

l l

l - ---p 23

p 23

p 23

p 23 - ---t

t t

t - ---e

e e

e cmd.exe

cmd.exe

cmd.exe

cmd.exe

192.1

68.2

7.1

0

Tools

in D

eta

il . . . . .

Netc

at

-“N

C”to “NC”connection

C:

C:C:

C:\ \\\Documents and Settings

Documents and Settings

Documents and Settings

Documents and Settings\ \\\gdw

gdw

gdw

gdw> >>>netstat

netstat

netstat

netstat

- ---a aaa

Active Connections

Active Connections

Active Connections

Active Connections

Proto Local Address Foreign Address State

Proto Local Address Foreign Address State

Proto Local Address Foreign Address State

Proto Local Address Foreign Address State

TCP

TCP

TCP

TCP wds

wds

wds

wds- ---gdw:ftp

gdw:ftp

gdw:ftp

gdw:ftp

wds

wds

wds

wds- ---gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

TCP

TCP

TCP

TCP wds

wds

wds

wds- ---gdw:telnet

gdw:telnet

gdw:telnet

gdw:telnet

wds

wds

wds

wds- ---gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

TCP

TCP

TCP

TCP wds

wds

wds

wds- ---gdw:epmap

gdw:epmap

gdw:epmap

gdw:epmap

wds

wds

wds

wds- ---gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

TCP

TCP

TCP

TCP wds

wds

wds

wds- ---gdw:microsoft

gdw:microsoft

gdw:microsoft

gdw:microsoft- ---ds

dsds

ds

wds

wds

wds

wds- ---gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

TCP wds

TCP wds

TCP wds

TCP wds- ---gdw:1032 wds

gdw:1032 wds

gdw:1032 wds

gdw:1032 wds- ---gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

TCP wds

TCP wds

TCP wds

TCP wds- ---gdw:5354 wds

gdw:5354 wds

gdw:5354 wds

gdw:5354 wds- ---gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

TCP wds

TCP wds

TCP wds

TCP wds- ---gdw:10110 wds

gdw:10110 wds

gdw:10110 wds

gdw:10110 wds- ---gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

gdw.wds.local:0 LISTENING

. . . .

. . . .

. . . .

. . . .

192.1

68.2

7.1

0

C:

C:C:

C:\ \\\> >>>nc

ncnc

nc

192.168.27.10 23

192.168.27.10 23

192.168.27.10 23

192.168.27.10 23

Microsoft Windows XP [Version 5.1.2600] . . .

Microsoft Windows XP [Version 5.1.2600] . . .

Microsoft Windows XP [Version 5.1.2600] . . .

Microsoft Windows XP [Version 5.1.2600] . . .

C:

C:C:

C:\ \\\> >>>ipconfig

ipconfig

ipconfig

ipconfig

ipconfig

ipconfig

ipconfig

ipconfig

Windows IP Configuration

Windows IP Configuration

Windows IP Configuration

Windows IP Configuration

Ethernet adapter Local Area Connection:

Ethernet adapter Local Area Connection:

Ethernet adapter Local Area Connection:

Ethernet adapter Local Area Connection:

Connection

Connection

Connection

Connection- ---specific DNS Suffix . :

specific DNS Suffix . :

specific DNS Suffix . :

specific DNS Suffix . :

IP Address. . . . . . . . . . . . :

IP Address. . . . . . . . . . . . :

IP Address. . . . . . . . . . . . :

IP Address. . . . . . . . . . . . : 192.168.27.10

192.168.27.10

192.168.27.10

192.168.27.10

Subnet Mask . . . . . . . . . . . : 255.255.255.0

Subnet Mask . . . . . . . . . . . : 255.255.255.0

Subnet Mask . . . . . . . . . . . : 255.255.255.0

Subnet Mask . . . . . . . . . . . : 255.255.255.0

Default Gateway . . . . . . . . . : 192.168.27.1

Default Gateway . . . . . . . . . : 192.168.27.1

Default Gateway . . . . . . . . . : 192.168.27.1

Default Gateway . . . . . . . . . : 192.168.27.1

C:

C:C:

C:\ \\\>^C

>^C

>^C

>^C

C:

C:C:

C:\ \\\> >>>ipconfig

ipconfig

ipconfig

ipconfig

Windows IP Configuration

Windows IP Configuration

Windows IP Configuration

Windows IP Configuration

Ethernet adapter Local Area Connection:

Ethernet adapter Local Area Connection:

Ethernet adapter Local Area Connection:

Ethernet adapter Local Area Connection:

Connection

Connection

Connection

Connection- ---specific DNS Suffix . :

specific DNS Suffix . :

specific DNS Suffix . :

specific DNS Suffix . :

IP Address. . . . . . . . . . . . :

IP Address. . . . . . . . . . . . :

IP Address. . . . . . . . . . . . :

IP Address. . . . . . . . . . . . : 192.168.27.50

192.168.27.50

192.168.27.50

192.168.27.50

Subnet Mask . . . . . . . . . . . : 255.255.255.0

Subnet Mask . . . . . . . . . . . : 255.255.255.0

Subnet Mask . . . . . . . . . . . : 255.255.255.0

Subnet Mask . . . . . . . . . . . : 255.255.255.0

Default Gateway . . . . . . . . . : 192.168.27.1

Default Gateway . . . . . . . . . : 192.168.27.1

Default Gateway . . . . . . . . . : 192.168.27.1

Default Gateway . . . . . . . . . : 192.168.27.1

192.1

68.2

7.5

0

Page 43: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

SN

MP

-M

IBs

. . . . . .

Learn m

ore at:

http://w

ww.ireasoning.com/

iReasonin

g

Page 44: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

SN

MP

-M

IBs

. . . . . .

IM

PLEX

Page 45: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

•“O

riginal”capture routine -

TC

PD

UM

P+ L

IB

PC

AP(the Promiscuous Capture Libary) or

Win

Pcap.

Available on m

ost "open" platform

s.

•SSLD

UM

Pis TCPDUMP with SSL decryption capability.

•ETH

ER

EA

Lis a packet analyzer based on TCPDUMP.

•W

IR

ESH

AR

Kis the latest incarnation of ETHERAL

Shows actual packets on the netw

ork with “breakdown”.

Good for true analysis of the netw

ork a

ndfor establishing

"common use“baselines.

•EX

IG

EN

CEprovides sim

ilar functionality for z/O

S.

Tools

in D

eta

il . . . . .

Packet

Analy

sers –

“Snif

fers”

Page 46: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Tools

in D

eta

il . . . . .

“W

ireshark”

The three panes show the traffic

flow, the headers, and the data in

dump form

at.

Highlighting is reflected in the lower

panes.

This image shows the IP header . . .

Page 47: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Tools

in D

eta

il . . . . .

“W

ireshark”

This image shows the UDP header . . .

Page 48: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Tools

in D

eta

il . . . . .

“W

ireshark”

This image shows the DATA; in this case

a DNS Query.

( http://w

ww.w

ireshark.org/)

Page 49: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Tools

in D

eta

il . . . . .

This image shows the equivalent displays in

EXIG

ENCE; in this case for an FTP session.

( http://w

ww.w

illdata.com/)

“EX

IG

EN

CE”

Page 50: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Tools

in D

eta

il . . . . .

“ZEN

Trace a

nd S

olv

e”

Page 51: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Tools

in D

eta

il . . . . .

ZTS -Exigence in the ZEN Framework.

( http://w

ww.w

illdata.com/)

“ZEN

Trace a

nd S

olv

e”

Page 52: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

And, In

Passin

g . . . . .

Netw

ork &

Securit

y t

este

rs

“N

essus”

-(“

The T

enable

New

t”) a security

vulnerablility

scanner.

( www.nessus.org

)

“N

map”

-a netw

ork and security scanner

( insecure.org

& nmap.org)

Use r

esponsib

ly –

Use w

ith c

are !

Page 53: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

> >>>nmap

nmap

nmap

nmap

- ---v

v v

v - ---A 192.168.27.50

A 192.168.27.50

A 192.168.27.50

A 192.168.27.50

Starting

Starting

Starting

Starting Nmap

Nmap

Nmap

Nmap

4.20 ( http://

4.20 ( http://

4.20 ( http://

4.20 ( http://insecure.org

insecure.org

insecure.org

insecure.org

) at 2007

) at 2007

) at 2007

) at 2007- ---02

0202

02- ---03 11:40 GMT Standard Time

03 11:40 GMT Standard Time

03 11:40 GMT Standard Time

03 11:40 GMT Standard Time

Initiating ARP Ping Scan at 11:40

Initiating ARP Ping Scan at 11:40

Initiating ARP Ping Scan at 11:40

Initiating ARP Ping Scan at 11:40

Scanning

Scanning

Scanning

Scanning

192.168.27.50 [1 port]

192.168.27.50 [1 port]

192.168.27.50 [1 port]

192.168.27.50 [1 port]

Completed ARP Ping Scan at 11:40, 0.20s elapsed (1 total hosts)

Completed ARP Ping Scan at 11:40, 0.20s elapsed (1 total hosts)

Completed ARP Ping Scan at 11:40, 0.20s elapsed (1 total hosts)

Completed ARP Ping Scan at 11:40, 0.20s elapsed (1 total hosts)

Initiating Parallel DNS resolution of 1 host. at 11:40

Initiating Parallel DNS resolution of 1 host. at 11:40

Initiating Parallel DNS resolution of 1 host. at 11:40

Initiating Parallel DNS resolution of 1 host. at 11:40

Completed Parallel DNS resolution of 1 host. at 11:40, 0.03s ela

Completed Parallel DNS resolution of 1 host. at 11:40, 0.03s ela

Completed Parallel DNS resolution of 1 host. at 11:40, 0.03s ela

Completed Parallel DNS resolution of 1 host. at 11:40, 0.03s elapsed

psed

psed

psed

Initiating SYN Stealth Scan at 11:40 : Scanning 192.168.27.50 [1

Initiating SYN Stealth Scan at 11:40 : Scanning 192.168.27.50 [1

Initiating SYN Stealth Scan at 11:40 : Scanning 192.168.27.50 [1

Initiating SYN Stealth Scan at 11:40 : Scanning 192.168.27.50 [1697 ports]

697 ports]

697 ports]

697 ports]

Discovered open port 135/tcp on 192.168.27.50

Discovered open port 135/tcp on 192.168.27.50

Discovered open port 135/tcp on 192.168.27.50

Discovered open port 135/tcp on 192.168.27.50

Completed SYN Stealth Scan at 11:40, 39.05s elapsed (1697 total

Completed SYN Stealth Scan at 11:40, 39.05s elapsed (1697 total

Completed SYN Stealth Scan at 11:40, 39.05s elapsed (1697 total

Completed SYN Stealth Scan at 11:40, 39.05s elapsed (1697 total ports)

ports)

ports)

ports)

Initiating Service scan at 11:40 : Scanning 1 service on 192.168

Initiating Service scan at 11:40 : Scanning 1 service on 192.168

Initiating Service scan at 11:40 : Scanning 1 service on 192.168

Initiating Service scan at 11:40 : Scanning 1 service on 192.168.27.50

.27.50

.27.50

.27.50

Completed Service scan at 11:41, 11.63s elapsed (1 service on 1

Completed Service scan at 11:41, 11.63s elapsed (1 service on 1

Completed Service scan at 11:41, 11.63s elapsed (1 service on 1

Completed Service scan at 11:41, 11.63s elapsed (1 service on 1 host)

host)

host)

host)

Warning: OS detection for 192.168.27.50 will be MUCH less relia

Warning: OS detection for 192.168.27.50 will be MUCH less relia

Warning: OS detection for 192.168.27.50 will be MUCH less relia

Warning: OS detection for 192.168.27.50 will be MUCH less reliable because we did not

ble because we did not

ble because we did not

ble because we did not

find at least 1 open and 1 closed TCP port

find at least 1 open and 1 closed TCP port

find at least 1 open and 1 closed TCP port

find at least 1 open and 1 closed TCP port

. . .

. . .

. . .

. . .

Host 192.168.27.50 appears to be up ... good.

Host 192.168.27.50 appears to be up ... good.

Host 192.168.27.50 appears to be up ... good.

Host 192.168.27.50 appears to be up ... good.

Interesting ports on 192.168.27.50:

Interesting ports on 192.168.27.50:

Interesting ports on 192.168.27.50:

Interesting ports on 192.168.27.50:

Not shown: 1696 filtered ports

Not shown: 1696 filtered ports

Not shown: 1696 filtered ports

Not shown: 1696 filtered ports

PORT STATE SERVICE VERSION

PORT STATE SERVICE VERSION

PORT STATE SERVICE VERSION

PORT STATE SERVICE VERSION

135/tcp open

135/tcp open

135/tcp open

135/tcp open msrpc

msrpc

msrpc

msrpc

Microsoft Windows RPC

Microsoft Windows RPC

Microsoft Windows RPC

Microsoft Windows RPC

MAC Address:

MAC Address:

MAC Address:

MAC Address: xx:xx:xx:xx:xx:xx

xx:xx:xx:xx:xx:xx

xx:xx:xx:xx:xx:xx

xx:xx:xx:xx:xx:xx

(Dell ESG

(Dell ESG

(Dell ESG

(Dell ESG Pcba

Pcba

Pcba

Pcba

Test)

Test)

Test)

Test)

Running (JUST GUESSING) : Microsoft Windows 2000|XP (98%)

Running (JUST GUESSING) : Microsoft Windows 2000|XP (98%)

Running (JUST GUESSING) : Microsoft Windows 2000|XP (98%)

Running (JUST GUESSING) : Microsoft Windows 2000|XP (98%)

No exact OS matches for host (test conditions non

No exact OS matches for host (test conditions non

No exact OS matches for host (test conditions non

No exact OS matches for host (test conditions non- ---ideal).

ideal).

ideal).

ideal).

Network Distance: 1 hop : TCP Sequence Prediction: Difficulty=0

Network Distance: 1 hop : TCP Sequence Prediction: Difficulty=0

Network Distance: 1 hop : TCP Sequence Prediction: Difficulty=0

Network Distance: 1 hop : TCP Sequence Prediction: Difficulty=0 (Trivial joke)

(Trivial joke)

(Trivial joke)

(Trivial joke)

. . .

. . .

. . .

. . .

OS and Service detection performed.

OS and Service detection performed.

OS and Service detection performed.

OS and Service detection performed. Nmap

Nmap

Nmap

Nmap

finished: 1 IP address (1 host up) scanned in

finished: 1 IP address (1 host up) scanned in

finished: 1 IP address (1 host up) scanned in

finished: 1 IP address (1 host up) scanned in

67.000 seconds

67.000 seconds

67.000 seconds

67.000 seconds

Raw packets sent: 3517 (162.066KB) | Rcvd: 86 (47

Raw packets sent: 3517 (162.066KB) | Rcvd: 86 (47

Raw packets sent: 3517 (162.066KB) | Rcvd: 86 (47

Raw packets sent: 3517 (162.066KB) | Rcvd: 86 (4770B)

70B)

70B)

70B)

Tools

in D

eta

il . . . . .

Nm

ap

(edited)

(NB. This sample has

been edited to fit !)

Page 54: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Outl

ine S

teps:

•Check the stack –

“pin

g”local loopback

•“p

ing”the remote host/server name

•“p

ing”with IPaddress–the DNS m

ay be down

•If “ping”fails “

traceroute”-find where it stops

•Use “

nets

tat”

to check the interface

•Check routing (is it as expected?)

•If ping works, try “

teln

et”

(standard port 23)

•If “

teln

et”

works try t

eln

et

to t

he a

pplicati

on p

ort

•If that works try the application

•Use “

nets

tat”

to check the connection exists

•Check your syslogs(remember USS ! “syslogd”!)

•Do you s

tillhave a failure? …

trace it!

Pro

ble

m D

iagnosis

. . .

Page 55: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

•K

now

Your N

etw

ork !

•K

eep U

p-t

o-D

ate

Docum

enta

tions &

Dia

gram

s !

•K

now

the T

ools

(m

ost

tools

can b

e u

sed f

or p

racti

ce a

t any t

ime)

•P

lan Y

our A

pproach t

o A

ny P

roble

m

•Sto

p , L

ook , a

nd L

ISTEN

!!

Sum

mary

. . . . .

Page 56: TCP/IP Troubleshooting Tips & Tools - SHARE · TCP/IP Troubleshooting Tips & Tools Gordon Webber William Data Systems August 2011 ... (ping, traceroute, ... Network devices may not

Thank y

ou !