Eﬀect of team training on aircraft maintenance technicians: … › ... › documents › media...

International Journal of Industrial Ergonomics 27 (2001) 141–157

Effect of team training on aircraft maintenance technicians:computer-based training versus instructor-based training

David C. Krausa, Anand K. Gramopadhyeb,*aGalaxy Scientific Corporation, Tucker, GA 30084, USA

bCollege of Engineering and Science, Department of Industrial Engineering, Clemson University, Clemson, SC 29634-0920, USA

Received 1 December 1999; received in revised form 20 September 2000; accepted 4 October 2000

Abstract

The backbone of a safe and reliable air transportation system is the aircraft maintenance technician. Theseindividuals work in a complex environment requiring above average teamwork skills as they coordinate, communicateand cooperate with other technicians and inspectors as well as with other departments within their organization. Newtechnologies, such as multimedia computers, may be able to provide the necessary training to enhance their teamwork

skills and improve their performance. To determine the effectiveness and applicability of computer-based multimediateam training for aircraft maintenance technicians, a controlled study was conducted to examine the transfer effects ofcomputer-based team training (CBT) on team performance in the aircraft maintenance environment. To facilitate the

research, a computer-based multimedia tool – the aircraft maintenance team training (AMTT) software – wasdeveloped. Results of this study indicate that computer-based team training was as effective as traditional instructorbased team training.

Relevance to industry

Though the advantages of teamwork in the airline industry are widely recognized, it is not readily supported by work

cultures. Providing teamwork skills will improve both performance and safety. Moreover, the importance of teamtraining has also been emphasized in the national plan for aviation in human factors. # 2001 Elsevier Science B.V.All rights reserved.

Keywords: Aviation maintenance technician; Teamwork; Training; Multimedia; Transfer effects; Computer-based training; Aviation

maintenance environment

1. Introduction

A sound aircraft maintenance system, one of thecritical elements in ensuring a safe and reliable

transportation system (FAA, 1991), is a complexone with many interrelated human and machinecomponents. The backbone of this system is theaircraft maintenance technician (AMT), whoattends to the needs and requirements of main-taining an aircraft for a safe and operationallyefficient flight. To ensure quality, federal aviationregulations (FARs), industry and federal policies,and approved corporate policies and procedures

*Corresponding author. Tel.: +1-864-656-5540; fax: +1-

864-656-0795.

E-mail address: [email protected] (A.K. Gramo-

padhye).

0169-8141/01/$ - see front matter # 2001 Elsevier Science B.V. All rights reserved.

PII: S 0 1 6 9 - 8 1 4 1 ( 0 0 ) 0 0 0 4 4 - 5

specifically control the work performed on anaircraft. Unfortunately, though, maintenanceand aircrew related aircraft accidents still occur(Johnson, 1997). Though 75% of aircraft accidentsare classified as either pilot or human error, arecent study concluded that 18% of all accidentsare maintenance related (Phillips, 1994). Recogniz-ing this fact, the FAA, under the auspices of thenational plan for aviation human factors, haspursued human factors research (FAA 1991,1993), focusing in the maintenance arena on theAMT (e.g., Drury et al., 1990; Shepherd, 1992;Shepherd et al., 1995) to develop interventions tomake the maintenance procedures, and hence themaintenance technicians, more reliable and/ormore error free.

One facet of this issue, the development of theincreased technical skills needed by modernAMTs, has been continually addressed by theFAA. For example, the Federal Governmentprovides regulations concerning training on tech-nical skills to assure that anyone working on anaircraft will meet certain minimum competencyrequirements. Specific regulations concerningtraining are given in Title 14 of the code of federalregulations (CFR). Part 147 of Title 14 specifiesthe curricula for the airframe and powerplant(A&P) schools, while Part 65 covers the certifica-tion requirements for mechanics and recurrenttraining. The FAA, through the Office of AviationMedicine, has also funded efforts for the develop-ment of advanced training tools to train the AMTsof the future (FAA, 1991, 1993, 1995). Theseadvanced tools include, for example, the Boeing767 environmental control system tutor (ECS) andthe multimedia system for training aviationregulations (STAR). It is anticipated that theapplication of these new training technologies willhelp reduce the gap between current AMT skillsand those needed for the maintenance of advancedsystems.

But as task analyses of aircraft inspection andmaintenance activities (Drury et al., 1990; Taylor,1990; FAA, 1991, 1993) have revealed, the com-plexity of the aircraft inspection and maintenancesystem is overwhelming, requiring above averagecoordination, communication and cooperationbetween inspectors, maintenance personnel, super-

visors and various other sub systems (planning,stores, and shop) to be effective and efficient. Alarge portion of inspector and maintenancetechnician work is accomplished through team-work. In a typical maintenance environment, theinspector first looks for defects and reports them.Then, the maintenance personnel repair thereported defects and work with the originalinspector or the buy-back inspector to ensure thatthe job meets pre-defined standards. During theentire process, the inspectors and maintenancetechnicians work with their colleagues from thesame shift and the next shift as well as personnelfrom planning, stores, and other areas as part of alarger team to ensure that the task gets completed(FAA, 1991). Thus, in a typical maintenanceenvironment, the technician is challenged to workautonomously but still be a team member, com-municating and coordinating the activities withother technicians and inspectors.

Though the advantages of teamwork are widelyrecognized in the airline industry (Hackman,1990), the work culture assigns responsibility forfaulty work on individual AMTs rather than onthe teams in which they work. The reasons for thiscould be the individual licensing process andpersonal liability, both of which often results inAMTs and their supervisors being less willing toshare their knowledge and work across shifts withless experienced or less skilled colleagues. Theproblem is further compounded since the moreexperienced inspectors and mechanics are retiringand being replaced by a much younger and lessexperienced workforce.

Furthermore, the AMTs are not trained to workas team members. While the curriculum of theaircraft maintenance technology schools (AMTS)provide the necessary technical skills for studentsto receive both their airframe and power plantcertificates (A&P License) as specified in the codeof federal regulations (FAR Part 147, AppendixB), it currently does not address any instructionin teamwork skills. In fact, the current technicalschool environment encourages students tocompete, and as a result, the AMTs are oftenill-prepared for cooperative work. To preparestudent AMTs for the workplace, new ways haveto be found to build students’ technological,

D.C. Kraus, A.K. Gramopadhye / International Journal of Industrial Ergonomics 27 (2001) 141–157142

interpersonal and socio-technical competence byincorporating team training and communicationskills into their curriculum. The importance ofteams has been emphasized in the National Planfor Aviation in Human Factors (FAA, 1991, 1993;Shepherd, 1992) where both the aircraft industryand government groups agreed that additionalresearch needs to be conducted to evaluate team-work in the aircraft maintenance/inspection en-vironment. Recent work has examined the effectsof team training when applied to students at anAircraft Maintenance Technology School. Using apreviously designed framework, a pilot studyconducted by Gramopadhye et al. (1996) found apositive correlation between team skills trainingand the improvement of team performance andoverall task performance in an aircraft mainte-nance situation. In addition, the study concludedthat student AMTs needed instruction in teamskills to prepare them for the tasks found in theaircraft maintenance environment. The study,however, did not address the issue of the appro-priate training delivery system, the method forpresenting these skills training.

With computer-based technology becomingcheaper, the future will bring an increasedapplication of advanced technology to training.Over the past decade, instructional technologistshave provided numerous technology-based train-ing devices with the promise of improved efficiencyand effectiveness. Example of such technologyincludes computer simulation, interactive videodiscs and other derivatives of computer basedapplications (Johnson, 1990) and the media richcomputer-based training software (system trainingfor aviation regulations – STAR) for teachingfederal aviation regulations (FARs) to A&Pstudents (Chandler, 1996). Furthermore, multi-media has assisted in teaching difficult andcomplex skills (Gordon, 1994). Layton (1992)stated that the domain of aircraft maintenance israpidly becoming the focus of computer-basedtraining (CBT) aids. With the use of desktopcomputers with multimedia packages, new main-tenance job aids have been developed to teachtechnical skills to maintenance technicians. AMTsmay learn a variety of skills from CBT that rangefrom scheduling preventive maintenance to apply-

ing expert systems for fault diagnosis and repair.Lufthansa Airlines believes so strongly in CBTthat it has instituted computer-based training withvideo overlays to update the technical skills ofmaintenance technicians (Reichow, 1994). Becauseof the advantages offered, CBT may play a role inteam training in the aircraft maintenance environ-ment. It is important, therefore, to examine theeffectiveness and applicability of computer-basedmultimedia team training for aircraft maintenancetechnicians. Since no one has yet examined thisarea, the general objective of this study was tounderstand the transfer effects of computer-basedteam training on team performance in the aircraftmaintenance environment. To facilitate the study,a computer-based multimedia training tool – theaircraft maintenance team training (AMTT) soft-ware – was developed. The software program,which has been previously described in great detail(FAA, 1995; Kraus and Gramopadhye, 1999), wasbased on the instructional systems development(ISD) model (Leshin et al., 1992). During thedevelopment of the AMTT software, Kraus (1996)reviewed a number of training strategies in orderto select a variety of training tactics that best fit themultimedia environment.

2. Background

In a controlled study, 36 subjects (aircraftmaintenance technicians – AMTs) were assignedto two groups. These groups were designated as (1)Group IBT (instructor-based training) whichreceived team training instruction through tradi-tional instructor-based training, and (2) GroupCBT (computer-based training) which receivedteam training instruction through multimediacomputer-based training in the form of the AMTTsoftware. Every effort was made to maintain aconstant curriculum and presentation sequence forboth groups so that the only difference in thetraining between the two groups was the type ofdelivery system.

The study was divided into two phases: the in-structional phase and the evaluation phase. In theinstructional phase, the subjects received trainingon the four specific team skills – communication,

D.C. Kraus, A.K. Gramopadhye / International Journal of Industrial Ergonomics 27 (2001) 141–157 143

decision-making, interpersonal relationships, andleadership. In the evaluation phase, the effective-ness of the training was evaluated using objectiveand subjective measures as the teams performedtwo aircraft maintenance tasks on the hangarfloor.

3. Methodology

Upon completion of the team training, thesubjects in each group were randomly assigned tosix three-member teams. After detailed discussionswith instructors, mechanics, and training person-nel at an A&P School (FAR, Part 147) and acertified domestic repair station (FAR, Part 145),each team was required to perform the followingtwo tasks which were representative of normalaircraft maintenance.

1. Task 1 – routine maintenance task: determin-ing the center of gravity of an aircraft

and

2. Task 2 – non-routine maintenance task:trouble shooting an electrical problem on anaircraft.

The order in which the tasks were performedwas counterbalanced within each group. Thefollowing section describes each task in detail.

3.1. Task 1 – routine maintenance (RM)

As part of the routine maintenance task, eachteam was to determine the center of gravity of aKing Air 90A aircraft, a normal routine main-tenance activity requiring a team effort to execute.To reflect a true maintenance environment, workcards providing general procedural instructionswere supplied to the teams. For evaluationpurposes, the routine maintenance task (Task 1)was subdivided into four major sub-tasks:

1. Sub-task 1.1 – towing,2. Sub-task 1.2 – setup,3. Sub-task 1.3 – weighing and calculating, and4. Sub-task 1.4 – roll out.

Since weighing an aircraft to determine thecenter of gravity requires that the aircraft belocated in a level and enclosed area, such as ahangar, with all doors and windows closed,the team’s first task (Sub-task 1.1) was to towthe aircraft into the hangar. This task requiredthat one person drive the towing tug while theother two-team members walked at the wing tipsto prevent accidental damage to the plane. Thistask was considered to start at the receipt ofthe work cards and was deemed finished when theaircraft was positioned and secured in the hangar.

The setup for weighing (Sub-task 1.2) whichstarted the moment Sub-task 1.1 ended, requiredthat the team first secure the platform scales fromthe storeroom, then position the scales in frontof the landing gears, and finally roll the aircraftonto the scales. This procedure required oneperson to drive the towing tug, another to ridethe brakes in the cockpit, and the third to monitorthe movement of the aircraft in order to preventaccidental damage to the aircraft. Positioning thechocks fore and aft of the wheels, as well as ridingthe brakes was critical for the safety of the aircraftand the maintenance personal during the setupprocedure. This sub-task was considered completewhen the brakes were set, the chocks were in place,and the tow bar was disconnected from theaircraft.

The weighing and calculating task (Sub-task 1.3)started at the conclusion of Sub-task 1.2. Once allthe steps listed in the work cards were accom-plished, the scales were read to obtain the weightof the aircraft. This task was considered completewhen the team submitted their calculations to theevaluators.

Roll out (Sub-task 1.4) was the final taskperformed by the team. This sub-task, whichstarted when the team initiated the reconnectionof the tow bar to the aircraft, was deemed finishedwhen the aircraft was moved completely off thescales and parked properly with the wheelschocked and the scales and miscellaneous equip-ment put away. As with the setup (Sub-task 1.2),this procedure required a team effort with oneperson driving the tug, a second person riding thebrakes and a third person monitoring the aircraft’smovement.


3.2. Task 2 – non-routine maintenance (NM)

The second task was a non-routine maintenancetask involving trouble shooting an electricalproblem. To assure consistency throughout theexperiment, each team read a narrative from asimulated pilot’s log describing a problem withthe nose landing gear warning light. According tothe pilot’s log, on final approach to the airport, thenose landing gear warning light indicated that thenose landing gear was not down and locked whenin fact it was. The team had to diagnose theproblem, then identify/find and rectify it within aone-hour time period. This was an open-endedproblem and, therefore, no guidance in the form ofwork cards was provided.

The overall non-routine maintenance task wassubdivided into three separate but overlappingproblems: (1) the circuit breaker for the landinggear lights had been placed in the ‘‘off’’ position,(2) a burned out bulb had been inserted into thelanding gear light socket, and (3) the wireconnecting the down and locked switch on thelanding gear to the landing gear warning light hadbeen disconnected. This third problem, which wasnot as obvious as the first two, necessitated the useof wiring diagrams located in the maintenancemanual.

4. Data collection

As the teams performed the routine mainte-nance and non-routine maintenance tasks, theirperformance was evaluated using the three mea-sures – self-evaluation, instructor’s evaluation, andtask performance evaluation – discussed in thefollowing section.

4.1. Self-evaluation

Upon the completion of the RM and NM tasks,all subjects completed a questionnaire rating theperformance of their teams on the application ofthe team skills of communication, decision-mak-ing, interpersonal relationships, and leadership.These were the team skills emphasized in an earlierFAA (1995) report on teamwork in the aircraft

maintenance environment. The self-evaluationconsisted of a questionnaire with seven questionson each of the team skills with the subjects ratingthe teams on their application of each skill using aseven-point Likert scale.

4.2. Instructor’s evaluation

Upon completion of both the routine and non-routine maintenance tasks, three independentevaluators completed a questionnaire identical tothe subject’s judging the teams on the applicationof the team skills. Fig. 1 gives sample questionsused on the self-evaluation and instructor evalua-tion.

4.3. Task performance evaluation

As the teams performed the routine mainte-nance (RM) and non-routine maintenance (NM)tasks, the three evaluators independently mon-itored them and evaluated their performance in thecategories of accuracy, safety, and speed. Table 1provides a description of each of the performancemeasures.

5. Results

5.1. Effects of training delivery system on teamskills perception

5.1.1. Self-evaluationAfter completing both the routine and non-

routine maintenance tasks, the subjects in groupsIBT and CBT, scored their team on the applicationof the four team skills. A comparison of the scoresfor the two task types are shown in Fig. 2. Withone exception, group CBT’s use of communicationskills, both groups scored the use of team skillshigher on the non-routine maintenance task. Sincethis task was a problem-solving activity, it wasunstructured, unlike the routine maintenance taskwhich had work cards outlining step-by-stepprocedures. Thus, team members were requiredto use their functional understanding of theaircraft’s electrical system as well as the technicaldata available in the maintenance manual. The


Fig. 1. Sample questions from evaluator’s/subject’s questionnaire – communication.


multi-layered problem in the NM task created asituation necessitating an iterative problem-solvingprocess consisting of:

1. analysis of the current system state (e.g., Is thenose landing gear light illuminated?),

2. application of reasoning to diagnose the pro-blem (e.g., there may be a lack power to thelight or the light bulb may be burned out),

3. decision making (e.g., we have a consensus tocheck the light bulb first), and

4. use of feedback to determine the effectiveness oftheir decision (e.g., A new bulb has beeninstalled, but it is still not illuminated).

All team members became involved in this processas the team diagnosed, located and repaired theproblems. The higher self-evaluation scores on thenon-routine tasks suggested that individual teammembers perceived an increase in the applicationof team skills during its performance. Thus, theapplication of team skills may be sensitive to tasktype. These results were similar to those reportedby Taylor (1990) in their study of the effects of crewresource management training in maintenance.

The subjects indicated increased problem-solvingabilities after receiving training on teamworkskills.

In summary, for self-evaluation, the scores oneach team skill category were similar for both thegroups for both the routine maintenance andthe non-routine maintenance tasks, indicating thatthe effect of training delivery system on the teams’perception of their application of team skills iscomparable. According to the team training modelfor the aircraft maintenance environment (Krauset al., 1996), the task type (e.g., procedural,diagnostic or combination) is an external factoraffecting team process over time. Comparable self-evaluation scores for the routine and non-routinemaintenance tasks indicate that the subjects inboth the groups were equally effective in applyingthe knowledge gained from team skills training tothe two different types of tasks.

5.1.2. Instructor’s evaluationAs each team completed the routine and non-

routine maintenance tasks, the evaluators scoredthe members on their use of the four team skills.

Table 1

Description of performance measures used for routine and non-routine maintenance tasks

Routine maintenance (RM) tasks

Accuracy 1. Number of errors or number of times the team’s procedure varied form the work card

2. Number of time an improper tool was used

3. Number of times that the equipment was handled incorrectly

Safety 1. Number of times the safety of the aircraft was in jeopardy

2. Number of times the safety of an individual was in jeopardy

Speed 1. Time to complete the sub-task (in min)

2. Percent of task completed within time constraint

Non-routine maintenance (NM) tasks

Accuracy 1. Was the problem diagnosed correctly?

2. Did the team locate the problem?

3. Did the team fix the problem?

Speed 1. Time taken to diagnose the problem

2. Time taken to locate the problem

3. Time taken to fix the problem

Safety 1. Number of times the safety of the aircraft was in jeopardy

2. Number of times the safety of the individual was in jeopardy


The results are provided in Fig. 3. The instructor’sevaluation did not reveal any significant differ-ences between the groups in each of the team skillscategories. Thus, according to the evaluators’scores, the effect of training delivery system onsubject’s use of team skills in the performance ofmaintenance tasks, both routine and non-routine,were comparable.

The increase in use of team skills by teams inGroup IBT during the non-routine task was alsonoted by the independent evaluators (Fig. 3).According to the evaluators’ observation, how-ever, the teams in group CBT had a decrease in theapplication of three team skills – decision-making,interpersonal relationships and leadership – duringthe non-routine task. This is in variance with whatwas reported by the team members in group CBTon the self-evaluation questionnaire.

Interestingly, for each skill category in theroutine and non-routine maintenance tasks, theinstructors were more conservative in their ratingsthan the subjects. A similar result was noted byGlickman et al. (1987) in a study examining thedevelopment of team skills over time. In theirstudy, the instructors’ ratings were consistentlylower than the rating given by the team leader.

5.2. Effects of team training delivery systemon task performance

As the teams performed the routine and non-routine maintenance tasks, the evaluators recordedthe performance of the teams on accuracy, speedand safety violations. The following sectionsdiscuss these results.

5.2.1. AccuracyThe measure of accuracy for the routine task

was the consolidation of (1) the number of errorsmade or the number of times the procedurefollowed by the team differed from the procedurespecified in the work card, (2) the number of timesan improper tool was used to perform a sub-task,and (3) the number of times equipment washandled incorrectly. Fig. 4 shows the number oferrors made by groups IBT and CBT in perform-ing the routine maintenance task. Although thetotal number of errors were the same for eachgroup (10 each), the sub-tasks in which theyoccurred differed. For group IBT, the majority oferrors occurred during Sub-tasks 1.1 and 1.2(towing the aircraft and rolling the aircraft ontothe scales), whereas for group CBT most of the

Fig. 2. Self-evaluation comparison of tasks (RM vs. NM) for groups IBT and CBT.


errors occurred for Sub-task 1.3 (weighing andcalculating). The procedures for all three sub-taskswere provided in the work cards presented to theteams prior to beginning the task. Most of theseincidents occurred because the team’s proceduresdiffered from those specified in the work cards.

Table 2 gives a breakdown of the typical errorsmade by the teams by sub-task. A review of thedata reveals that the majority of the incidents werecreated by three teams in each group: for groupIBT–T1, T3 and T5; group CBT–T8, T9 and T12.However, by averaging across the teams within agroup, the number of incidents per team was low.Hence, statistically, there were no significantdifferences between the groups in performing theroutine maintenance tasks.

It is interesting to note that the number ofaccuracy incidents for group IBT peaked duringSub-task 1.2 (roll up), whereas for group CBT thenumber of accuracy incidents peaked during Sub-task 1.3 (weighing and calculating) (Fig. 4). It canbe hypothesized that teams were experiencing oneor more of the four stages of team growth firstidentified by Tuckman (1965). In a review of 50studies on team development, Tuckman noted that

the groups experienced four stages of growth:forming (a period of uncertainty characterized bythe group members trying to determine their placein the group), Storming (a period of conflict wheregroup members resist the influence of the groupand rebel against the leader, the task or both),Norming (a period where in-group feelings andcohesiveness develop and new standards and rolesare evolved), and Performing (a period whereinissues have been resolved, and the group becomesproficient in achieving its goals). In this study,most of the subjects knew one another (those fromthe overhaul facility having worked in the samecompany and students having taken classestogether), but their assignment to the three-member teams was the first opportunity for themto work as a unit to accomplish specified tasks. Asa result, the time taken in the forming stage wasminimal. The peaks in accuracy incidents duringSub-tasks 1.2 and 1.3 were indicative of thestorming stage. During these times, the teammembers tended to ‘do their own thing’, notfollowing the procedures specified in the workcards, exemplifying resistance to the influence ofthe group and rebellion against the task. By the

Fig. 3. Instructor’s evaluation comparison of tasks (RM vs. NM) for groups IBT and CBT.


fourth sub-task, the teams in both the groupsseemed to be working well together. The numberof accuracy incidents had decreased, and the teamsseemed to have grown into the performing stagewhen group energy channeled towards the accom-plishment of the task.

Since half the teams in both groups performedthe routine maintenance task first and the other

half performed the non-routine maintenance taskfirst, it may be argued that half the teams, thoseperforming the non-routine maintenance first,should have grown past the storming stage priorto the routine maintenance task. There are twopossible explanations why this may not be true.First, the stages of team growth are not timedependent (Tuckman, 1965). Some teams will pass

Fig. 4. Number of accuracy incidents for groups IBT and CBT during the routine maintenance task.

Table 2

Typical accuracy errors that occurred during the routine maintenance task

Sub-tasks Typical errors that occurred

Sub-task 1.1 – towing * Failed to check oil and to drain toilet waste water systema

* Failed to close hangar doors

Sub-task 1.2 – set up * Failed to place all control surfaces in neutral position* Failed to close passenger door

Sub-task 1.3 – weighing and calculating * Failed to measure from the main wing spar* Made incorrect measurement from the wheel center line to reference datum

Sub-task 1.4 – roll out * Failed to properly place chocks* Failed to return scales to storage

aThere was no toilet waste in the aircraft, but the teams were required to check the waste water system.


through a growth stage faster than others. Also, asnoted by Wheelan (1994), some teams may getstuck in a stage or even regress to a previous stage.The teams in this study that started with the non-routine task may not have arrived at the stormingstage until the routine task. Or they may havestarted their storming stage during the non-routinetask and carried it forward to the routine task.Second, the team organization changed because anew team leader was typically selected. The newteam organization coupled with a new and totallydifferent task may have caused the team membersto regress to an earlier stage, moving from aperforming stage back to a storming stage, as theyfaced a new situation with new leadership.

Accuracy for the non-routine maintenance taskwas measured by whether or not the teamscorrectly diagnosed the problem (yes or no),located the problem (yes or no) and fixed theproblem (yes or no). All 12 teams diagnosed,located and fixed Problems 1 and 2. Only twoteams, T1 and T6, could not locate and fixProblem 3 within the allocated time. Statisticallythere were no significant differences between thegroups; thus, the type training delivery system hadno affect on the accuracy measures for the teamsperforming non-routine maintenance tasks.

The correlation analysis showed a positivecorrelation between the total average of post-training scores a subject received on all four teamskills and the time it took to complete the routinemaintenance task (r ¼ 0:4683, p50:004). Thiswould indicate that the subjects were applyingthe team skills knowledge they had acquired inorder to finish the task in a timely manner. Apositive correlation was observed between theaccuracy measure for routine maintenance andthe instructors’ evaluation of the teams on the useof communication skills during the routine main-tenance task (r ¼ 0:4322, p50:0085), instructor’sevaluation on the use of decision-making skillsduring the routine maintenance task (r ¼ 0:3411,p50:0418), and self-evaluation of interpersonalrelationships skills (r ¼ 0:4661, p50:0042). Thesefindings are consistent with other studies on theeffects of communication and decision-making onteam performance. In a study of the teamevolution and maturation model, Morgan, Salas

and Glickman (1994) found that as the team’sperformance improves, the perception of the teammembers concerning communication and coordi-nation increases. Also, in a study investigatingwhether teamwork process measures are asso-ciated with outcome measure, Brannick et al.(1995) found that team effectiveness was positivelyassociated with decision-making and communica-tion skills.

An obvious revelation after analyzing theaccuracy scores for the routine maintenance andnon-routine maintenance tasks was the overalllow number of errors for both groups. Thehigh-accuracy scores achieved by the teams canprobably be explained within the speed accuracytradeoff (SATO) context (Drury and Gramopad-hye, 1991). Training delivered at A&P schools andthrough various training departments focuses onaccuracy, emphasizing the need to minimize errorssince these can be catastrophic (e.g., the case of acontinental commuter airlines, the ContinentalExpress, that flew without the tail de-icing bootproperly attached, or the United DC-10 whoseengine failed, severing control hydraulic lines andcausing a crash at Sioux City). This perception ofaccuracy obviously seemed to transfer to theperformance of the teams for both the RM andNM tasks.

5.2.2. SafetyBoth the groups had almost the same number of

safety violations for the routine and non-routinemaintenance tasks. Table 3 provides a list oftypical safety violations that occurred during bothtasks. The safety violations were noted by theevaluators whenever they observed that either theaircraft’s or an individual’s safety was jeopardized.Fig. 5 indicates that the majority of the safetyproblems occurred in the routine maintenance taskfor Sub-task 1.2 when the plane was rolled up ontothe scales. This is a critical time in the routinemaintenance task. If the aircraft wheels are notchocked correctly and the parking brakes notproperly set, the aircraft could roll off the scalescausing serious injuries and damage. In fact, suchan incident did occur during the performance ofthe routine task for Team T12. The team failed toproperly set the parking brakes after rolling the


aircraft onto the platform scales, and during theweighing, the aircraft rolled off the scales towardsthe rear of the hanger. Fortunately, the chockswere properly placed stopping the aircraft and

preventing damage to the aircraft, the hanger, andthe personnel. Interestingly, the above incident istypical of anecdotal evidence that describes similarhorror stories wherein wide-body aircraft have

Table 3

Typical safety violations that occurred during the routine and non-routine maintenance tasks

Tasks Typical safety violations

Routine maintenance tasks:

Sub-task 1.1 (towing) * Removed wheel chocks prior to connecting towing tug* Failed to walk with the aircraft while towing it to the hangar

Sub-task 1.2 (roll up) * Failed to place chocks fore and aft of the scales* Failed to properly set parking brakes

Sub-task 1.3 (weighing and calculating) * No safety violations

Sub-task 1.4 (roll out) * No safety violations

Non-routine maintenance tasks:

Problem 1 (circuit breaker) * No safety violations

Problem 2 (burned out bulb) * No safety violations

Problem 3 (disconnected wire)* Caused damage to junction box* Replaced wire without first checking wiring diagram in the maintenance manual

Fig. 5. Number of safety incidents for groups IBT and CBT during the routine maintenance task.


rolled out of the aircraft hangar, causing severedamage, both to life and property. Correlationanalysis of safety scores and instructor’s evalua-tion of the teams revealed that teams whichdisplayed higher communication, decision-mak-ing, interpersonal relationship, and leadershipskills had fewer safety violations on the routinemaintenance task (r ¼ ÿ0:5702, p50:0003; r ¼ÿ0:8062, p50:0001; r ¼ ÿ0:5312, p50:0009;r ¼ ÿ0:4179, p50:0112, respectively).

Statistically, the measurements on safety viola-tions did not show a significant difference betweengroups for the non-routine maintenance task.Group IBT had one safety violation while GroupCBT had four. Although Group CBT had agreater number of safety problems, a review ofthe data shows that the majority of the incidentswere produced by one team, T11. In fact, theywere caused by one person of that team. Thisindividual, in his haste to locate the third problem,caused damage to the aircraft and injury to himselfin three separate events. Excluding this oneindividual, we can conclude that the team trainingdelivery system had no effect on the safetyperformance of the teams as they performed thenon-routine maintenance task.

The lack of safety violations during the non-routine maintenance task could possibly be anartifact of the task. The NM task was essentially aproblem solving one. Thus, it was more of acognitive task unlike the routine maintenance taskwhich was a highly procedural and manual, in-volving movement of the aircraft into the hangar,positioning the aircraft, and rolling the aircraftonto and off of the scales. Hence, the opportunitiesfor safety violations were much less for the NMtask than for the RM.

The overall number of safety violations for boththe RM and NM tasks was much lower than thosetypically reported in the ‘‘real world’’ aircraftmaintenance environment (Phillips, 1994; Rankinand Allen, 1996). The existing study was per-formed in a clean and quiet training hangarenvironment. There were no other AMTs workingon the aircraft at the same time, thereby minimiz-ing work interruptions and workflow. On anactual hangar floor, there are multiple skill groupssuch as avionics, hydraulics, maintenance, among

others, with multiple crews working on a singleaircraft. It is a highly complex and dynamicenvironment wherein an individual AMT mustnot only work with his own team members butmust also communicate and coordinate with othercrews, supervisors, and inspectors. Thus, the workof a team is not only dependent on the intra-teamfactors but also on inter-team factors. In addition,there are other factors present in the ‘‘real world’’but missing in the current study that couldpossibly contribute to the number of safetyviolations: environmental factors (e.g., noise,lighting); organizational factors (e.g., gate pres-sure, late night shift); subject factors (e.g., part-time workers, shift workers). Hence, we should beextremely careful before we draw any general-izations on the safety violations found in thecurrent study.

5.2.3. SpeedThe results of the statistical tests did not reveal

any significant differences between the two groupson overall task completion times for the routineand non-routine maintenance tasks. Analyzing theperformance of the groups on the routine main-tenance task clearly indicated that there was alarge variability in the performance of teams forthe task as a whole (for group IBT, SD=32.4; forgroup CBT, SD=24.5), and for each individualsub-task (Sub-task 1.1: for group IBT, SD=9.72;for group CBT, SD=18.3, Sub-task 1.2: for groupIBT, SD=27.6, for group CBT, SD=12.9, Sub-task 1.3: for group IBT, SD=30.2; for group CBT,SD=31.1, Sub-task 1.4: for group IBT, SD=3.2;for group CBT, SD=0.5). The teams expendedover half their allotted time in the performance ofSub-task 1.3 (weighing and calculating) (Fig. 6).This was due to the fact that only a rough outlineof the calculation procedures was provided in thework cards. More detailed step-by-step procedureswere located in the manuals for the aircraft. Acommon complaint heard from the Lockheedsubjects was, ‘‘We don’t have to do this calculationat work. We just put the weights into thecomputer, and it calculates it for us.’’ Leadershipplayed an important role during this sub-task asevidenced by the fact that most teams tended toflounder not understanding how to calculate the


center of gravity. The turning point came when thesubject who had assumed the leadership rolerealized that he needed to take control andrequested the manual containing the center ofgravity calculations. After this point, the leadercommunicated the procedures to the team, anddecisions were made as the team worked towardscompleting the task. Interestingly, a negativecorrelation was observed between the completiontime for routine maintenance, and the instructors’evaluation of the teams on the use of leadershipskills (r ¼ ÿ0:3305, p50:049) and self-evaluationof leadership skills (r ¼ ÿ0:7029, p50:0001).

Sub-task 1.4 (roll out) took the shortest amountof time. There are three possible explanations forthis: (1) the teams were performing well ascohesive units at this stage, (2) the teams hadlearned from previous mistakes made during therollup (Sub-task 1.2) and, thus, were familiar withthe aircraft and the procedures, and (3) it was arelatively short and easy procedure to accomplish.It is hypothesized that all three factors impactedthe completion time for Sub-task 1.4. Regardlessof the sub-task completion times, it is clear that thetraining delivery system did not demonstrate anydifferential effect on the task completion times forthe routine maintenance task.

Although no significant differences were ob-served between the groups on the overall comple-tion times for the non-routine maintenance task,analysis of the results revealed that group CBTrequired significantly less time to complete Pro-blem 2 (burned out bulb) (Fig. 7). All the teamsfollowed a similar procedure to identify the burnedout bulb in the nose landing gear indicator: (1)remove the suspected bulb, (2) insertion of thesuspect bulb into a functioning landing gearindicator, and (3) replacement if the bulb did notlight up. This procedure was confounded by an irisbuilt into the bulb holders. When the iris wastwisted, it would shut out all light from the bulb.In addition, there was a separate test circuit to allthe bulbs which permitted a maintenance techni-cian to test a bulb by pushing on the bulb holder.Some of the teams were unaware of the built-inbulb tester, and all the teams were unaware of theiris. Those teams that accidentally twisted the irisclosed while replacing the bulb became confusedand frustrated because a functioning bulb wasfailing to work in a functioning landing gearindicator. Because of the above situations, differ-ences in completion times on Problem 2 betweenthe groups could be attributed to some teamsaccidentally twisting the iris closed. Therefore, the

Fig. 6. Time taken to complete sub-tasks for groups IBT and CBT for the routine maintenance task.


difference in completion time for Problem 2 couldnot be attributed to the specific training deliverysystem.

6. Conclusions

The general goals of this research were toexamine the role of the advanced technology,specifically computer-based multimedia presenta-tions, in imparting team skills training to aircraftmaintenance technicians and to evaluate thetransfer effects of the computer-based trainingdelivery system to the operational environment. Aspart of the research, a multimedia, computer-based team training software – Aircraft Main-tenance Team Training (AMTT) software – wasused. The important conclusions drawn are:

1. Team training enhanced the knowledge ofindividuals on team skills. The type of trainingdelivery system, however, did not have asignificant effect on the individual’s ability toacquire team skills knowledge.

2. The self and instructors’ evaluations on theapplication of team skills in the operationalenvironment and the performance of theteams on the routine and non-routine main-

tenance tasks were comparable. The transferof knowledge from the classroom to thehangar floor was the same regardless of thetype of training delivery system. For the CBTand IBT teams, the results were unambiguous,indicating clearly that AMTT was as effectivein delivering team training instructions as IBT.

3. The results of this research have obviousramifications for the use of AMTT for teamtraining in the aircraft maintenance environ-ment. In addition to being as effective asexisting instructor-based team training meth-odologies, use of AMTT for team training hasthese other advantages:a. Standardization: AMTT provides a sys-

tematic and consistent curriculum. Air-craft maintenance instructors at variousfacilities use their own unique trainingstrategies (straight lectures, classroomdiscussions, video examples, etc.). Inaddition, some maintenance instructorswho are technically competent may nothave sufficient team skills knowledge totrain AMTs on teamwork. As a result,when certain teamwork skills are taught,there may be a failure to emphasizecertain important team concepts. TheAMTT software provides a standardized

Fig. 7. Time taken by groups IBT and CBT to complete the problems for the non-routine maintenance task.


and systematic team skills training pro-gram which aircraft maintenance instruc-tors (at certified repair stations, airlinecompanies, general aviation stations, andA&P schools) may use to provide teamskills training.

b. Adaptability: most maintenance trainingis accomplished via on-the-job training orclassroom training, both of which aremanpower intensive. Traditional trainingrequires careful scheduling of personnel orencumber others in the training process.CBT can be done at convenient timeswhen trainees are available and need onlyinvolve the person being trained. Inaddition, AMTT allows the trainees tolearn at their own pace.

c. Record keeping: The record keepingcapabilities of AMTT tracks the student’sprogress and provides the training super-visor with information concerning thelevel of understanding being achieved.Information concerning the trainee’sknowledge test and perception scores aremaintained in a database, and instructors(supervisors) are able to use this informa-tion to design remedial training as well aspotential future training programs.

d. Cost effectiveness: team training usingAMTT can be cost effective because: (1)it can be delivered on-site thus eliminatingtravel expenses for the trainer and thetrainee. (2) it can minimize down-time byproviding training at times that are con-venient to the trainee and the company’swork schedule. In larger organizations,AMTT can be delivered to many people atmultiple sites thus proving to be costeffective.

e. Use of advanced technology: many facil-ities, for example A&P Schools and fixedbased general aviation facilities, do nothave access to larger aircraft. The AMTTsoftware provides team skills trainingagainst the backdrop of maintaining aDC-9. Thus the trainees not only acquireknowledge and skills on teamwork butalso gained an understanding of the

importance of teamwork in the mainte-nance of wide-body aircraft.

Acknowledgements

This research was funded by a grant from theFederal Aviation Administration, Office of Avia-tion Medicine (Program Manager: Jean Watson)through the Galaxy Scientific Corporation to Dr.Anand Gramopadhye. We would like to thankLockheed Martin Aircraft Center and the AircraftMaintenance Technology Program at GreenvilleTechnical College for the use of its facilities inconducting this research. We also acknowledge thesupport of Doyle Arnold, Frank Webb, and RonKnauer of Greenville Technical College, and DonCope, Mike Mason, and Hy Small, at LockheedMartin Aircraft Center.

References

Chandler, T.N., 1996. System for training aviation regulations

(star): development and evaluation. Human Factors in

Aviation Maintenance – Phase Six: Progress Report. FAA,

Washington, DC.

Drury, C.G., Prabhu, P., Gramopadhye, A.K., 1990. Training

for visual inspection. Proceedings of the 3rd Federal Aviation

Administration Meeting on Human Factors in Aviation

Maintenance and Inspection: Training Issue. FAA, Atlantic

City, NJ.

Drury, C.G., Gramopadhye, A.K., 1991. Speed and accuracy in

aircraft inspection. Work performed under Subcontract No.

8901014-SC-03 to Galaxy Scientific Corporation. FAA,

Washington, DC.

FAA (Federal Aviation Administration) Office of Aviation

Medicine. 1991. Human Factors in Aviation Maintenance –

Phase One: Progress Report. DOT/FAA/AM-91/16. FAA,

Washington, DC.



Phase Two Progress Report. FAA, Washington, DC.



Phase IV: Progress Report. FAA,Washington, DC.

Glickman, A.S., Zimmer, S.R., Montero, C., Guerette, P.J.,

Campbell, W.J., Morgan, B.B. Jr., Salas, E., 1987. The

evolution of teamwork skills: an empirical assessment with

implications for training. Technical Report No. 87-016.

Office of Naval Research, Orlando, Fl.

Gordon, S.E., 1994. Systematic Training Program Design:

Maximizing Effectiveness and Minimizing Liability. Prentice-

Hall, Englewood Cliffs, NJ.


Gramopadhye, A.K., Kraus, D., Jebaraj, D., Rao, P., 1996.

Advanced Technology Applied to Team Training: Aircraft

Maintenance Team Training (AMTT) Software. Proceedings

of the Human Factors and Ergonomics Society 40th Annual

Meeting, 1072–1076.

Hackman, J.R., 1990. A normative model of work team

effectiveness. Technical Report No. 2. Yale University,

New Haven, CT.

Johnson, W.B., 1990. Advanced technology training for

aviation maintenance. Final Report of the 3rd FAA Meeting

on Human Factors Issues in Aircraft Maintenance and

Inspection. Atlantic City, FAA, NJ.

Johnson, W.B., 1997. Human factors training for aviation

maintenance personnel. Proceedings of the Human Factors

and Ergonomic Society 41st Annual Meeting. Albuquerque,

NM.

Kraus, D.C., 1996. Team Training, Teamwork and Team

Performance: Role of Advanced Technology for Team

Training in the aircraft Maintenance Environment. Clemson

University, Clemson, SC.

Kraus, D., Gramopadhye, A.K., 1999. Computer-based Team

Training: The Aircraft Maintenance Environment Example,

Computers and Industrial Engineering, Vol. 36, No. 9,

635–654.

Kraus, D.C., Gramopadhye, A.K., Blackmon, R.B., 1996.

Teams in the aircraft maintenance environment. Proceedings

of the 5th Industrial Engineering Research Conference.

Leshin, C.B., Pollock, J., Reigeluth, C.M., 1992. Instructional

Design Strategies and Tactics. Educational Technology

Publications, Englewood Cliffs, NJ.

Layton, C.F., 1992. Emerging technologies for maintenance job

aids. 7th Federal Aviation Administration Meeting on

Human Factors Issues in Aircraft Maintenance and Inspec-

tion. FAA, Atlanta, GA.

Phillips, E.H., 1994. Focus on accident prevention key to

future airlines safety. Aviation Week and Space Technology,

52–53.

Rankin, B., Allen, J., 1996. Boeing Introduces

MEDA: Maintenance Error Detection Aid. Airliner.

April–June.

Reichow, D., 1994. Application of computer-based training for

improved maintenance training efficiency. Eighth Federal

Aviation Administration Meeting on Human Factors Issues

in Aircraft Maintenance and Inspection. FAA, Alexandria,

VA.

Shepherd, W.T., 1992. Meeting objectives. Human factors

issues in aircraft maintenance and inspection: Final Report,

FAA, Washington, DC, pp. 14–38.

Shepherd, W., Layton, C.F., Gramopadhye, A.K., 1995.

Human factors in aviation maintenance: current FAA

research. Proceeding of the 8th International Symposium

on Aviation Psychology, FAA, Washington, DC, pp.

466–471.

Taylor, J., 1990. Facilitation of information exchange among

organizational units within industry. Second Federal Avia-

tion Administration Meeting on Human Factors Issues in

Aircraft Maintenance and Inspection, FAA, Washington,

DC, pp. 69–78.

Tuckman, B.W., 1965. Developmental sequence in small

groups. Psychological Bulletin 63 (6), 384–399.

Wheelan, S.A., 1994. Group Process: A Developmental

Perspective. Allyn & Bacon, Boston, MA.


Eﬀect of team training on aircraft maintenance technicians: … › ... › documents › media...

Documents

Transcript of Eﬀect of team training on aircraft maintenance technicians: … › ... › documents › media...