FOR0383 Software Quality Assurance
description
Transcript of FOR0383 Software Quality Assurance
04/21/23 Dr Andy Brooks 1
Lecture 3
London Ambulance System (LAS) 26 & 27 October and November 4 1992
FOR0383 Software Quality Assurance
04/21/23 Dr Andy Brooks 2
Facts about LAS ~ 1992
• LAS was the largest in the world, handling 1,300-1,600 ‘999 calls’ daily.
• £69.7 million budget• 2,700 staff• 305 A&E ambulances (Accident and Emergency)
• 445 PTS ambulances (Patient Transport Service)
• 1 helicopter• 0.5 million A&E journeys per year
04/21/23 Dr Andy Brooks 3
Deficiencies of manual system• Identification of exact location
– “About half-way along London Road…”
• Physical movement of paper• Maintaining proper vehicle status• Time consuming use of voice
– “Can you repeat that address please…”
• Identifying duplicated calls– “This sounds like a report of the same accident…”
• Dealing with call backs– “It has been 25 minutes, the ambulance is not here yet…”
• Identifying special incidents– “Could this be a tanker chemical spill?...”
04/21/23 Dr Andy Brooks 4
With Computer Aided Dispatch (CAD)
• a gazetteer provided telephone box identification
• no more paper
• timely updates of resource availability
• intelligent help to identify duplicates
• direct mobilisation of an ambulance on completion of an emergency call
04/21/23 Dr Andy Brooks 5
Some Inquiry Conclusions• software was not complete and not fully
tested and the backup was not tested• there were outstanding data transmission
problems to/from MDTs (Mobile Data Terminals)
• there was scepticism over AVLS accuracy (Automatic Vehicle Location System)
• the ambulance crews had no confidence in the system and had not been fully trained
“Train train and train again”Ian Tighe, 999 rescue, The Computer Bulletin October 1997
04/21/23 Dr Andy Brooks 6
• CAC (Central Ambulance Control) staff were placed in unfamiliar positions in the control room and could not easily work together to solve problems
• the system relied on near perfect information of vehicle location and crew/vehicle status
• no attempt was made to deal with the effects of inaccurate or incomplete data which led to an increase in the number of exception messages
Some Inquiry Conclusions
04/21/23 Dr Andy Brooks 7
Poor interface between crews, MDTs, and the system.
• the system failed to capture all of the data• ambulance crews sometimes failed to press
the correct status button due to the pressure of dealing with certain incidents
• there were radio black spots• ambulance crews sometimes failed to press
status buttons because of frustration• there were radio communication bottlenecks
during busy periods
04/21/23 Dr Andy Brooks 8
• missing or swapped callsigns• there were faults in handshaking routines:
MDTs and the system showed different status!• sometimes ambulance crews intentionally did
not press the correct status button or pressed buttons in an incorrect order (frustration)
• ambulance crews sometimes took a different vehicle to the one they had logged onto
• incorrect or missing vehicle locations• too few call takers
Poor interface between crews, MDTs, and the system.
04/21/23 Dr Andy Brooks 9
Unreliability, slowness and operator interface• the system fell over and screens locked
– staff were advise to re-boot
• there was a failure to identify all duplicate calls• exception messages were not prioritised• messages scrolled off the top of screens!• the software made resource allocation errors• there were slow response times for certain
screen based activities
04/21/23 Dr Andy Brooks 10
26 and 27 October 1992
• the system went live
• there were no paper records
• there was no manual back up
• the launch was across all of London
• resource allocators were separated from radio operators and exception rectifiers
• system proposed resource allocations were used
04/21/23 Dr Andy Brooks 11
• it was extremely difficult for staff to intervene and correct problems
• the system rapidly knew the correct location and status of fewer and fewer vehicles
• multiple vehicles were sent to the same incident or not the closest vehicle was sent
• the number of exception messages rapidly increased so that staff could not clear the queue
• each effect quickly reinforced the others– see the Cause/Effect diagram in the Inquiry Report
26 and 27 October 1992
04/21/23 Dr Andy Brooks 12
Ambulance crew frustration• crew frustration may have led crews not
to press the status buttons in the correct order
• crew frustration may have led crews taking a different vehicle to the one allocated to them
• crew frustration may have led to a different vehicle and crew responding to an incident
04/21/23 Dr Andy Brooks 13
Crash on November 4• after 27 October, it was decided to revert to a semi-manual
operation• computer-based call taking and the gazetteer were for the most
part reliable• radio voice channels were available to resolve any mobilisation
misunderstandings• additional call taking staff were allocated to each shift• but shortly after 2am on November 4th, the system slowed
significantly and then locked up• attempts to reboot the system failed• a decision was made to revert to a manual system with voice
mobilisations
04/21/23 Dr Andy Brooks 14
What went wrong on November 4?
• a minor programming error left a piece of code which used up a small amount of memory within the file server every time a vehicle was mobilised
• after 3 weeks all the available memory on the file server had been used!
• normal testing would probably not have caught this particular error
• the fallback to a second server was never implemented for the semi-manual operation!
Response times 26/27 October(The ORCON specified response time from taking the call and arriving at the scene is 14 minutes.)
04/21/23 Dr Andy Brooks 15
Calls and average ring times 26 October(Average ring times peak at 10 minutes!)
04/21/23 Dr Andy Brooks 16
Total A&E Patients Carried, October(October 26/27 were normal in terms of demands on LAS services. Excessive demands was not responsible for the poor response times.)
04/21/23 Dr Andy Brooks 17
Calls recorded by call logger(On October 26/27 the number of calls were within the range within which it can be predicted 95% of calls will fall.)
04/21/23 Dr Andy Brooks 18