The following procedure will tell you how to identify the E1 condition experience in San Diego and how to recovery.

1 Identify the issue1. BSSM alarms.

a. You will see ACN reliability failures alarms from BTS to all CIUs.

# 1101393 Ack Clear "05 Sep 03 12:49:07" "O%:CBS1:Cells1:MC800BTS1326:MCBTSSubsystem1:Root1:BTSCallProcessing1" WARNING "Signaling failure // ACN Reliable Delivery Failure - Count = 4176 dnid = 0xf84801 dpid = 0x3010"

b. You will see connectivity alarms on audit failures for addresses 57, 58 and 59

# 520735 Ack Clear "05 Sep 03 12:41:03" "O%:CBS1:EnhancedBSC1:Passport610:11pMSWSubsystem57:BTSAggregate1:ConnectivityAudit1" MAJOR "Active to Active Connection Failure with Traffic Ranges: 1 sd1 VPI(4/228,5/229,6/230); 2 sd1 VPI(8/228,9/229,10/230); 3 sd1 VPI(12/228,13/229,14/230); 4 sd1 VPI(16/228,17/229,18/230); 5 sd2 VPI(20/228,21/229,2 TRUNCATED - TOO MANY ALARMS"

# 520737 Ack Clear "05 Sep 03 12:41:03" "O%:CBS1:EnhancedBSC1:Passport610:11pMSWSubsystem57:BTSAggregate1:ConnectivityAudit1" CRITICAL "Total Ping Failure "

c. You will see HeartBeat alarms from the BTSs connected to the Discos and Address 17.

# 1840746 Nak Active "05 Sep 03 13:02:15 "O%:CBS1:Cells1:MC1900BTS946:MCBTSSubsystem1" CRITICAL "Heartbeat Not Received From Subsystem"

2. On the passport do the following commands.

> d ciuif/* ***Note that txPkt and rxPkt will not increment.

CiuIf/* Use -noTabular to see the many hidden attributes.+=====+-----+----+-----+--------------------+--------------------|CiuIf|osiAd|osiO|osiUs| txPkt | rxPkt| | min |per | age | |+=====+-----+----+-----+--------------------+--------------------| 0|unlck|ena |activ| 34589349| 15612429

NORTEL NETWORKS CONFIDENTIAL - 1 -

| 1|unlck|ena |activ| 34604212| 15615612| 2|unlck|ena |activ| 34666373| 15643690| 3|unlck|ena |activ| 34666372| 15618346| 4|unlck|ena |activ| 21302300| 9448911| 5|unlck|ena |activ| 21302300| 9494876| 6|unlck|ena |activ| 34599280| 15608483| 7|unlck|ena |activ| 34599284| 15603403| 8|unlck|ena |activ| 34533693| 15562582| 9|unlck|ena |activ| 34533694| 15569074| 10|unlck|ena |activ| 21304176| 9466421| 11|unlck|ena |activ| 21304176| 9484977ok 2003-09-07 11:56:26.93

2 Lock the Health Monitor MO on the BSSM.a. Lock the health monitor mo by executing the following command from an xterm window.

a. cliappb. cd :cbs1:bsm1:hm1;c. action –noconfirm lock;d. ls –f (AdministrativeState);e. exit;

3 Recovery steps.1. Identify what cards are active.

a. d sh ca/* sps*NOTE* Card that are hot equal standby. Cards that are serv equal active.11p card pairs are as follows (2/3, 4/5, 8/9, 10/11)

Shelf Card/* SpServ+====+-----+----+-----+-------+-------+-------+-------+----+-----|Card|osiAd|osiO|osiUs|osiAvai|osiProc|osiCntr|osiAlar|osiS|osiUn| | min |per | age | l | | l | m |tby | knw+====+-----+----+-----+-------+-------+-------+-------+----+-----| 0|unlck|ena |activ| | | | |cold|false| 1|unlck|ena |activ| | | | |serv|false| 2|unlck|ena |activ| | | | |hot |false| 3|unlck|ena |activ| | | | |serv|false| 4|unlck|ena |activ| | | | |hot |false| 5|unlck|ena |activ| | | | |serv|false| 6|unlck|ena |activ| | | | |hot |false| 7|unlck|ena |activ| | | | |serv|false| 8|unlck|ena |activ| | | | |hot |false| 9|unlck|ena |activ| | | | |serv|false| 10|unlck|ena |activ| | | | |hot |false| 11|unlck|ena |activ| | | | |serv|false

NORTEL NETWORKS CONFIDENTIAL - 2 -

| 12|unlck|ena |activ| | | | |serv|false| 13|unlck|ena |activ| | | | |hot |false| 14|unlck|ena |activ| | | | |hot |false| 15|unlck|ena |activ| | | | |serv|falseok 2003-09-07 11:57:48.98

2. Identify what cards have CIUs. a. d ciuif/* interfacename

*NOTE* The interface name will show a dlep component with a number. Multiply the dlep # by 2 this will give you the number of the active card of the pair. For example if the interfacename is dlep/1 ds1/0. This means that dlep/1 ds1/0 is located on 11p card pair of 2/3.

CiuIf/*+=====+--------------------+----------------------------------------|CiuIf| interfaceName |Response+=====+--------------------+----------------------------------------| 0|Dlep/1 DS1/0 Chan/0 || 1|Dlep/1 DS1/4 Chan/0 || 2|Dlep/4 DS1/0 Chan/0 || 3|Dlep/4 DS1/4 Chan/0 || 4|Dlep/2 DS1/0 Chan/0 || 5|Dlep/2 DS1/4 Chan/0 || 6|Dlep/1 DS1/1 Chan/0 || 7|Dlep/1 DS1/5 Chan/0 || 8|Dlep/4 DS1/1 Chan/0 || 9|Dlep/4 DS1/5 Chan/0 || 10|Dlep/2 DS1/1 Chan/0 || 11|Dlep/2 DS1/5 Chan/0 |ok 2003-09-07 11:59:36.66

3. Perform a switch lp command on all the active card.

>switch lp/xwait 3 minutes before executing the next lp command..

where x = the cards identify using the criteria from steps 1 and step 2.Execute this command for all active 11p cards that have CIUs associated with them.

NORTEL NETWORKS CONFIDENTIAL - 3 -

4. Verify switch lp command were successful. a. d sh ca/* sps *NOTE* Verify that all the cards that were identify in step 1 and step 2 as being active with CIU connected are now hot and the mate is now serv.

4 Unlock the Health Monitor MO on the BSSM. 1. Unlock the health monitor mo by executing the following command. a. from an xterm window type cliapp

b. cd :cbs1:bsm1:hm1;c. action unlock;d. ls –f (AdministrativeState);e. exit;

5 Execute an MTX download.1. From a BSSM xterm do the following commad.

a. cliapp b. cd :cbs1:mtx1:mtxsubsystem1; c. action –noconfirm bsmagentreset; d. exit; e. bsmci

f. table mtxinv g. confirm off h. bottom i. init

6 Verify issue has been resolved.1. BSSM alarms. a. You will see ACN reliability failures from BTS to all CIUs alarms clear.

b. You will see connectivity alarms on audit failures for addresses 57, 58 and 59 clear.c. You will see HeartBeat alarms from the BTSs connected to the Discos and Address 17 clear as well

2. On the passport do the following commands.

d ciuif/*

***Note that Tx and Rx will increment.

NORTEL NETWORKS CONFIDENTIAL - 4 -