Katrina Bedell Avers1

William B. Johnson2

Joy Banks1

Darin Nei1

1Civil Aerospace Medical Institute Federal Aviation Administration Oklahoma City, OK 731252Aircraft Certification Service Federal Aviation Administration Washington, DC 20591

August 2011

Final Report

Prioritizing Maintenance Human Factors Challenges and Solutions: Workshop Proceedings

DOT/FAA/AM-11/11Office of Aerospace MedicineWashington, DC 20591

OK-11-0024-JAH

Federal AviationAdministration

NOTICE

This document is disseminated under the sponsorship of the U.S. Department of Transportation in the interest

of information exchange. The United States Government assumes no liability for the contents thereof.

___________

This publication and all Office of Aerospace Medicine technical reports are available in full-text from the Civil Aerospace Medical Institute’s publications Web site:

www.faa.gov/library/reports/medical/oamtechreports

i

Technical Report Documentation Page

1. Report No. 2. Government Accession No. 3. Recipient's Catalog No.

DOT/FAA/AM-11/11 4. Title and Subtitle 5. Report Date

August 2011 Prioritizing Maintenance Human Factors Challenges and Solutions: Workshop Proceedings 6. Performing Organization Code 7. Author(s) 8. Performing Organization Report No. Avers KB,1 Johnson WB,2 Banks JO,1 Nei D1 9. Performing Organization Name and Address 10. Work Unit No. (TRAIS)

11. Contract or Grant No.

1FAA Civil Aerospace Medical Institute P.O. Box 25082 Oklahoma City, OK 73125

Aircraft Certification Service Federal Aviation Administration Washington, DC 20591

12. Sponsoring Agency name and Address 13. Type of Report and Period Covered Office of Aerospace Medicine Federal Aviation Administration 800 Independence Ave., S.W. Washington, DC 20591

14. Sponsoring Agency Code

15. Supplemental Notes Work was accomplished under approved task AM-A-08-HHR-521 16. Abstract Thirty delegates, mostly from the Federal Aviation Administration’s Office of Aviation Safety, but also from the U.S. aviation industry, the National Transportation Safety Board, and Transport Canada, assembled for two days at the Civil Aerospace Medical Institute in Oklahoma City. These leaders in maintenance human factors research met to discuss challenges and prioritize solutions for human factors issues in aviation maintenance. The delegates reviewed international incidents and accidents and corporate operating procedures to identify systematic human factor precursors to maintenance errors. Each human factor issue was discussed and subsequently prioritized by each delegate. Analysis of the data revealed eight core issues, including: use of technical publications, fatigue/alertness, safety culture, event data collection, return on investment for human factors, prioritization of human factors, professionalism and generational issues, and attention to required inspection items. Solutions and action items for each of these issues were discussed and recommendations were developed. The prioritized list of problems and suggested industry actions are detailed in the following report. The recommendations stemming from this report can be used to inform future research and development for maintenance human factors.

17. Key Words 18. Distribution Statement

Maintenance, Human Factors, Fatigue, LOSA, Duty Time, Research, Safety Culture, Return On Investment, Professionalism

Document is available to the public through the Defense Technical Information Center, Ft. Belvoir, VA 22060; and the National Technical Information Service, Springfield, VA 22161

19. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price Unclassified Unclassified 23

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized

iii

ACkNOwlEdgmENTs

This report summarizes the proceedings of the first Office of Aviation Safety (AVS) Maintenance Human Factors Leaders Workshop held at the Civil Aerospace Medical Institute in Oklahoma City, Oklahoma, August 4 and 5, 2010.

Dr. Bill Johnson, Chief Scientific and Technical Advisor for Human Factors in Maintenance, facilitated the workshop for key AVS and selected industry and international personnel responsible for developing and delivering maintenance human factors (MX HF) information relevant to Title 14 CFR Parts 65, 91,121, 135, and 145. Work-shop delegates participated in guiding organizational human factors initiatives and curricula development based on their experience in human factors R&D, operations, or investigation.

We thank the workshop delegates for their wholehearted participation and responsiveness to coordination requests. Their contributions will advance the knowledge and understanding of Maintenance Human Factors in the industry.

Special thanks to Drs. Robert Johnson and Melchor Antuñano of the Civil Aerospace Medical Institute and to the Human Factors Research Division for the use of their training facility and for supporting this effort. Thanks to Tara Bergsten for transcribing the presentations, to Lena Dobbins and Erin McManus for administrative support, and to Janine King and Suzanne Thomas for administering the course evaluation, final proofing, and formatting of the report.

Finally, a special thank-you to Mike Wayda and Kathy Wade for tirelessly reviewing and editing the manuscript. We appreciate you all.

Delegates OrganizationVictoria Anderson AVPRichard (Rick) Anglemyer SCSIKatrina Avers AAMJoy Banks AAMMark Brock ANMBrian T. Capone ASWGreg Carroll ASO Jennifer Ciaccio AFSFred Etheridge GulfstreamKeith A. Frable ASOVictoria Frazier AIRKevin Gildea AAMJim R. Hein AWPJohn (Jay) Hiles AFSBill Huntley AAR Corp.Bill Johnson AIRCarl (Steve) Keesey ASWTerry Kleiser AGLKen Larcher AAMMartin Maurino Transport CanadaGuy Minor AWPDwayne Pittsenbarger ASOBill Rankin BoeingBobby Reed ACEMary Schooley ANMJohn Sims AALVickie A. Stahlberg AWPMichele Wallentine ANMKatherine Wilson NTSBNadine Yeager ACE

v

CONTENTs

Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Workshop Infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Delegates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Workshop Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Workshop Proceedings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Day One Presentations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Day Two Presentations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Workshop Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Pre-Workshop Activity and Concerns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Post-Workshop Prioritization of Concerns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Workshop Evaluation and Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Evaluations of Workshop Content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Evaluations of Overall Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Suggestions for Improvement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Workshop Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

vii

ExECuTIvE summAry

Thirty delegates, mostly from the Federal Aviation Administration’s Office of Aviation Safety, but also from the U.S. aviation industry, the National Transportation Safety Board, and Transport Canada, assembled for two days at the Civil Aerospace Medical Institute in Oklahoma City, Oklahoma. The workshop was hosted by the Chief Scientific Technical Advisor’s program. The meeting format combined key presentation topics (i.e., fatigue, maintenance event data reporting, maintenance accidents, calculating the return-on-investment in human factors, human factors training, and the Original Equipment Manufacturer/Maintenance Repair Overhaul industry perspective) with extensive discussion. The purpose of the workshop was to discuss and prioritize human factor chal-lenges and solutions in aviation maintenance operations.

The final rank order, based on substantial delibera-tion, of the top eight significant challenges and solutions included: 1. Use of Technical Publications, 2. Fatigue/Alertness, 3. Safety Culture, 4. Event Data (MEDA, LOSA, ASAP), 5. Return on Investment for Human Factors, 6. Prioritization of Human Factors, 7. Professionalism and Generational Issues, and 8. Attention to Required Inspection Items.

The workshop delegates felt that the FAA is addressing some of these challenges, but there is substantial opportu-nity to increase attention to each of these topics.

Overall, the consensus was to increase attention to the top eight challenges, both with research and development, as well as with operational funding. Challenges surrounding technical publications continue to be a significant con-tributing factor to maintenance events. This issue crosses many FAA airworthiness organizations, including Flight

Standards and Aircraft Certification. It is a major challenge that will require significant effort. More research needs to be directed toward the use of technical documentation and identifying why mechanics are failing to follow procedures.

The group believed that maintenance fatigue risk management issues should be addressed immediately, with the emerging flight crew regulations. The attention to maintenance fatigue rulemaking should be elevated while the maintenance fatigue risk management-applied research is delivering significant educational materials. The group’s consensus was that continued efforts should be supported by FAA research and operational funds. Moreover, the FAA should proceed with Hours of Service rules and Fatigue Risk Management guidelines that will improve safety across operations.

A well-established safety culture is a critical foundation that must be in place before many of the human factor challenges can be addressed effectively. The group members agreed that safety culture is a significant challenge in cur-rent operational environments and will require substantial effort. Ultimately, the group felt that it was difficult to separate safety culture from the other seven high-priority challenges, and attention to the seven challenges will ad-dress safety culture. That said, many delegates suggested that Aviation Safety Inspectors be taught how to promote cultural change.

Voluntary reporting systems, like the Aviation Safety Action Program (ASAP), Boeing’s Maintenance Error Decision Aid (MEDA), and the evolving maintenance and ramp line operations safety audits (LOSA), are critical for the proactive and predictive risk assessment necessary for Safety Management Systems (SMS). As an industry, we must increase attention to these programs and take corresponding actions.

We have detailed the prioritized challenges and solu-tions in the following report.

1

Prioritizing Maintenance HuMan Factors cHallenges and solutions:

WorksHoP Proceedings

BACkgrOuNd

The Office of Aviation Safety (AVS) has a long-standing research and development initiative related to human fac-tors in aviation maintenance. The program was initiated in 1988 from Washington, D.C., Headquarters of the Office of Aviation Medicine (now referred to as the Office of Aerospace Medicine). Since that time, the program has been administrated by the Aircraft Maintenance Division (AFS-300) of the Flight Standards Service (AFS).

AFS-300 is the primary initiator of requirements and commits an Aviation Safety Inspector (.50 FTE) working with AFS-330 to serve as the point of contact in Washing-ton Headquarters. The Chief Scientific Technical Advisor (CSTA) for Aircraft Maintenance Systems works closely with AFS-300, AFS-330, and other Federal Aviation Administration (FAA) entities (e.g., FAA Safety Team) to collaborate on the direction of maintenance human factors activities. Many of these activities are conducted by contractors and other FAA organizations, such as the Civil Aerospace Medical Institute Human Factors Re-search Division. Program funding is primarily provided by AVS research and development funding. Selected applied projects are funded through AFS operational funds. Recently, funds from the AVS Chief Scientific and Technical Advisor’s (CSTA’s) program have also been allocated to selected projects.

Since 1988, the FAA has taken a leadership role to conduct an annual maintenance human factors confer-ence. From 1997 to 2002, responsibility for these meetings were shared and rotated among FAA, Transport Canada, and the Civil Aviation Authority (CAA) of the United Kingdom. There was a short period, in 2003 and 2004, when the conferences were not held. Starting in 2005, the FAA teamed with the U.S. Air Transport Associa-tion to co-sponsor the meeting. In 2010, the meeting returned to the United Kingdom. The annual meetings have ranged in size from 30, in 1988, to more than 400 in the year 2000. Delegates typically represent the international industry and regulators. FAA participation usually represents 15% of the delegates.

The annual maintenance human factors conferences are always highly rated for the technical content but also the external and inter-FAA networking and communication that foster the success for all human factors programs.

However, the large size of the meeting has restricted the format to formal lectures, some Q&A, and minimal discussion. For some time, there has been a lingering notion that key FAA MX human factors personnel need to have a small action-oriented meeting made up of FAA personnel who are human factors leaders in their respective FAA organizations. After many years of discussing such a meeting, the Chief Scientific Technical Advisor office funded and created the Maintenance Human Factors Leaders Workshop (Figure 1).

wOrkshOp INfrAsTruCTurE

delegatesInvitations to attend the workshop were sent to in-

dividuals identified as leaders in aviation maintenance human factors both internationally and within AVS (Figure 2). Although the majority of invitees were from AFS, the meeting planners extended invitations to indus-try leaders, scientists, and international representatives. Thirty invitees participated in the workshop, and all of them brought considerable human factors knowledge and experience to the workshop. For example, several of the AFS delegates were members of the FAA Safety Team and had extensive experience delivering human factors presentations. Many of the delegates were involved in multiple maintenance related accident investigations while others were involved in maintenance human factors research or aviation safety inspector training. Clearly, this workshop was not a conventional human factors training event, but instead, a meeting of the AVS Maintenance Human Factors leaders.

workshop format The workshop was designed to foster discussion, analy-

sis, and recommendations regarding MX HF challenges and solutions. Prior to the workshop meeting, attendees were asked to consider MX HF challenges and to identify their “top 5” concerns. These concerns formed the basis for workshop introductions and discussions. Twelve formal presentations were delivered after the introductions; each presentation involved substantial question and answer period with discussion. This format fostered relevant, informal conversation and was the basis for many of the workshop conclusions.

2

Figure 1. The AVS MX HF leaders in action

Figure 2. A depiction of the attendee affiliation

3

wOrkshOp prOCEEdINgs

day One presentationsThis section will summarize each workshop presenta-

tion that occurred on Day One and provide the suggested action items for the FAA.

Opening Ceremony. Dr. Robert Johnson, CAMI’s deputy director, opened the meeting and welcomed the delegates to CAMI and the Mike Monroney Aeronautical Center. Dr. Johnson stated that CAMI and the Human Factors Research Division were delighted to host the first AVS Maintenance Human Factor Leader’s Workshop. CAMI feels particularly attached to this topic because of their continued participation in numerous MX HF studies. CAMI Human Factors researchers have been integrally involved in the human factors survey of inter-national and FAA maintenance inspectors, maintenance fatigue risk management research, and the evolving work with maintenance and ramp Line Operation Safety Audits (LOSA).

Introductions. The workshop began with an extensive introduction and discussion session. Each speaker came with a prepared and prioritized list of their perceived maintenance human factors challenges. The list and discussion are described in the workshop recommenda-tions section.

Canadian Maintenance Human Factor Requirements. Mr. Martin Maurino (Civil Aviation Program Manager, Standards Branch, Transport Canada) described the structure of the Canadian Aviation Regulations with specific attention to Part V, Subpart 73 – Approved Maintenance Organizations (CAR 573.06). Under Part V, Subpart 73, technical, regulatory and human factors training guidelines are described. It specifies that human factors training must be delivered to all staff with technical responsibilities. The human factors training must include initial training and update training, with special provisions for training on new procedures as they arise. The initial Canadian human factor training is mandatory and must be two days long. The training must be classroom-based and cannot include any computer-based training. The training must cover the so-called “Dirty Dozen” - the 12 factors known to influence human error in maintenance operations. In addition, Transport Canada requires Safety Management System training to be linked with the human factors training. Currently, they are working to implement a rule for fatigue risk management systems that will be integrated with the rule for safety management systems and human factors training.

Action Items. Transport Canada has had their human factors rules in place for nearly 10 years, and they have not had a negative financial affect on airlines or other maintenance organizations. Attention to human factors

and the Dirty Dozen have become culturally ingrained at most maintenance organizations in Canada. As a result, this human factors culture will have an inevitable impact on the ease of implementation of Safety Management System (SMS) and Fatigue Risk Management Systems. The experience, in Canada, appears to be directly trans-ferable to the U.S. Federal Aviation Administration. The rule is not complicated, but is explicit about what should be included in human factors training and how it must be delivered. The acceptance and overall quality of computer-based training has evolved since the Canadian rule was implemented. Thus, the FAA should not limit the application of computer-based delivery of human factors information.

International Perspective on Human Factors and Event Reporting Systems. Dr. William (Bill) Rankin (Technical Fellow and Lead of MX Human Factors Group, Boeing Airplane Aviation Services) is involved with an extensive number of domestic and international carriers as part of the Boeing customer support for the Maintenance Error Decision Aid (MEDA).

Dr. Rankin first reviewed the European Aviation Safety Agency (EASA) rules for human factors programs. There are 10 major human factors training requirements, many of which are listed as suggested HF training topics in the FAA’s advisory materials. EASA requires initial and then recurrent training on a biannual basis. Since there are about 1,200 U.S. repair stations with EASA Part 145 certificates, a very high percentage of U.S. maintenance workers are already receiving human factors training. Generally, the EASA rules are in harmonization with the Transport Canada requirements.

Dr. Rankin closed with a review of Boeing’s MEDA form, considered to be the “Gold Standard” for event reporting systems. Nearly 1,000 airlines have received training from Boeing. Most U.S. carriers in the Aviation

Figure 3. CAMI Deputy Director, Dr. Robert Johnson (left), welcomes workshop delegates

4

Safety Action Program use MEDA as the basis for all event investigation. MEDA has been successful for many reasons. The primary reasons are: simplicity of use, data remains in the hands of the airline or MRO, and Boeing has offered continuing product support throughout the life cycle of the tool.

Action Items. EASA is an international leader in main-tenance human factors regulations. In the U.S., the EASA rules are followed by 1,200 U.S. maintenance organiza-tions and are accepted without issue. Dr. Rankin and the workshop attendees suggested that the FAA should immediately accept the EASA rules and harmonize with Europe and Canada.

Human Fatigue in Domestic Maintenance Operations. Dr. Katrina Avers (Industrial/Organizational Research Psychologist and Principal Investigator for the FAA’s

research program on human fatigue in maintenance operations, Civil Aerospace Medical Institute – Human Factors Research Division) chairs the FAA’s multi-disci-plinary maintenance fatigue workgroup. The workgroup includes representatives from industry, academia, and government and is identifying and developing practical, science-based methods for fatigue risk management.

Dr. Avers first discussed the NTSB’s recommendation to investigate fatigue in aviation maintenance operations and the FAA’s subsequent responses over the past 20 years. Most recently, the FAA Flight Standards Maintenance Division commissioned a multi-disciplinary workgroup to examine potential fatigue risk management solutions in maintenance operations. The workgroup has addressed the issue with both short-term and long-term solutions. The short-term solutions are intended to have immediate effect and improve awareness of fatigue-related hazards and personal fatigue countermeasures. These solutions include fatigue awareness materials such as posters, a 2010 calendar, newsletters, mechanic and supervisor training on fatigue countermeasures, a fatigue symptom checklist, and fatigue assessment forms, among others. The long-term solutions provide the FAA, the company, and individuals with clearly defined fatigue risk manage-ment responsibilities. The solutions include an operational handbook on how to develop a fatigue risk management system (FRMS) at all levels within the organization. The handbook will outline the guidelines based on interna-tional best practices and will provide users with all of the tools necessary to implement an FRMS. The workgroup also intends to make recommendations to improve hours of service limitations based on scientific modeling tools and the practicalities of maintenance operations.

All of the tools created by the workgroup are avail-able on the FAA’s human factors Web site and accessible through mxfatigue.com. An automated fatigue assessment form will be made available in the near future.

Dr. Avers reported that the working group conducted an informal survey of the members asking whether there is a necessity for a maintenance fatigue regulation. Twenty-one of the 25 members responded to the survey, and the results indicated that 100% of the respondents felt that a regulation was the only way to ensure industry action on fatigue issues.

Action Items. While the FAA has made substantial prog-ress in developing short-term solutions to address fatigue in aviation maintenance operations, it is simply not enough. The industry needs better hours of service rules, and the FAA should proceed with guidelines that will improve safety across maintenance operations. The attention that is currently being given to pilot fatigue rules should also be given to maintenance operations. There is no reason to delay such rulemaking for maintenance personnel.

Figure 4. Dr. Bill Rankin discusses the EASA rules for HF programs and the MEDA reporting form

Figure 5. Dr. Katrina Avers identifies practical, science based tools for reducing maintenance fatigue.

5

Aviation Safety Inspector Training. Mr. Rick Anglemyer (Manager for FAA Inspec-tor HF Training Project, Southern California Safety Institute—SCSI) manages the contract at SCSI that has deliv-ered a three-day human factors training course to about 1,200 Flight Standards Airworthi-ness Inspectors. With this training, he and his colleagues have empowered the FAA workforce to understand and apply the fundamentals of human factors with the companies that they oversee. Students say the three-day course has been very valuable to them. Mr. Anglemyer outlined the course content and also stimulated discussion about the possible content of a second-generation/recurrent course. Many of the workshop delegates had previ-ously attended the SCSI course, were enthusiastic about the current course, and offered suggestions for recurrent training. Some suggestions included: Teach inspectors how to market MX HF programs to the industry, create ways for inspectors to assist industry with the return-on-investment (ROI) in HF programs, provide supplementary training on hazard analysis and applied risk assessment, and teach inspectors to communicate and facilitate a positive safety culture.

Action Items. Flight Standards should begin specification for a recurrent training for their aviation safety inspectors. This would require a new maintenance human factors course. Many delegates suggested that the new course should em-phasize teaching the ASI to promote cultural change, with respect to human factors, both for the industry and for their FAA colleagues.

Maintenance Repair and Overhaul Perspectives on Human Factors Challenges in Maintenance. The final presenters for Day One represented both the MRO industry and original equipment manufacturers (OEM). AAR Corporation has multiple repair stations worldwide. Gulfstream Aerospace is an OEM but also operates many repair stations around the world.

Mr. Bill Huntley (Cor-porate Director for Human Factors & Safety Manage-ment, AAR Corporation) began by expanding on his top 5 human factors concerns. He talked about the fact that production requirements are often the driving force, making it dif-ficult to make human factors

considerations an important priority. He believes that there should be increased effort in collecting and using data to make a strong business case for human factors (HF) initiatives. He believes that HF training techniques must continue to evolve. On a related issue, he said that it has been very difficult to hire qualified personnel as human factors leaders.

Mr. Huntley described the nature of the MRO work-force and its challenges, noting that the MRO labor pool is a “revolving door,” which makes it difficult to develop and maintain a corporate safety culture. He commented that having many international locations is another challenge to maintaining high standards and quality along with a safety culture. He was positive about the potential of quality human factors programs to reduce error, lower costs, and to ensure continuing safety. He also looks to the benefits of voluntary reporting systems and various SMS initiatives that integrate human factors initiatives.

Original Equipment Manufacturer (OEM) Perspectives on Human Factors Challenges in Maintenance. Mr. Fred Etheridge (Manager of Compliance and Technical Training, Gulfstream Aerospace) described the human factors programs at Gulfstream Aerospace. The company complies with the EASA Part 145 require-ments for repair stations. Therefore, about 98% of its employees have had initial HF training, and many groups are in the first stage of the two-year recurrent training. Their company training department is called “Gulfstream University.”

Gulfstream is a desirable place for employment and experiences a very low turnover rate. Their cur-rent workforce is made up of both a senior aged workforce and a new generation of young workers. Gulf-stream recognizes that the generational gap in their workforce presents some human factors and communication challenges.

The company has an active safety management system and plans to integrate their fatigue risk management into the SMS.

Action Items. Both companies report active HF programs because they operate under the Part 145 Repair Station Rules regarding HF programs. This is an indication that regulations do encourage/force compliance. That said, there is a lot of variance among repair stations, and any FAA regulatory activity should be aware that one size does not fit all. Future research could target specific types of operations to better inform regulatory activity.

Based on comments from the AAR and Gulfstream pre-senters, the SMS should be developed in close cooperation with all HF initiatives.

Figure 6. Mr. Rick Anglemyer describing current ASI HF training

Figure 7. Mr. Bill Huntley outlining AAR Corporation’s approach to HF

Figure 8. Mr. Fred Etheridge discusses Gulfstream’s success with the safety management system

6

day Two presentationsDay Two started with MX HF accident-related data

presented by the NTSB, the FAA, and Transport Canada (presenting International Air Transport Association data). The day ended with guidance from the FAA on the regu-latory process and the required procedures for initiating scientific research. The day closed with a prioritization of action items. This section summarizes each speaker’s remarks and lists the collective action items from the three presenters.

Investigative Reports on Maintenance-Related Accidents.

Dr. Katherine Wilson (Human Performance Investigator, U.S. National Transportation Safety Board) presented MX-related data from five NTSB accidents. She covered the facts of each accident and the maintenance and human factors-related issues that may have contributed to the accident. The accidents included: Air Wisconsin Flight 3919 in Philadelphia (a gear-up landing); Air Midwest Flight 5481 in Charlotte, NC (a flight rigging failure); Air Sunshine Flight 527 in the Bahamas (an engine failure and ditching); Chalk’s Ocean Airways Flight 101 (wing attachment/spar failure on aging aircraft); and Delta Airlines Flight 1288 in Tallahassee, Florida (an uncontained turbine failure). The maintenance-related shortcomings in these accidents were related to the fol-lowing issues: failure to follow procedures; unqualified technician without task specific training; improper over-sight of work; and inadequate inspection. Dr. Wilson also talked about worker fatigue and about the NTSB’s long-standing recommendations to the FAA regarding fatigue and maintenance personnel. She suggested that the FAA capitalize on some of the guidance and regulatory materials developed by the trucking industry. Specifically, she referred to Schneider Trucking’s attention to sleep apnea programs and the resulting positive return-on-investment for both cost and safety.

Ms. Victoria Anderson (Senior Accident Investigator, FAA, AVP-100) was involved with the FAA’s investigative team for the Alaska Airlines Flight 261 on January 31, 2000, off the coast of Southern California. Ms. Anderson

described the flight and subsequent investigation. This flight experienced a loss of pitch control re-sulting from failure of the horizontal stabilizer trim system jackscrew assembly. The failure was caused by insufficient lubrication of the jackscrew assembly. The FAA and NTSB identified a number of factors that con-tributed to the accident. The causal factors ranged from the difficultly of perform-ing the lubrication task to

the complexity of measuring the acceptable wear limits. There was also discussion about the corporate culture and procedures that would permit a marginally worn com-ponent to continue to fly without adequate lubrication.

Prior to joining Trans-port Canada, Mr. Martin Maurino (Civil Aviation Program Manager, Stan-dards Branch, Transport Canada) was responsible for compiling international accident data for the In-ternational Air Transport Association (IATA). This presentation was based on his work with IATA. He reported only on ac-cidents between 2005 and 2009. Examples included Tuninter Flight 1153, an ATR-72, which crashed due to fuel starvation caused by the incorrect installation of a fuel gauge from an ATR-42.• Helios Airways Flight 522, B-737-300, pressurization

failure and failure of the flight crew to respond to pressurization alarms caused by maintenance leaving the pressurization mode selector in manual position rather than automatic. The flight crew did not notice the error during pre-flight inspection.

• SAS Flights 1209/2748/2867 gear failures on DH Dash 8-Q400 aircraft. All gear failures were related to technical documentation and procedures associated with landing gear components.

• United Airlines Flight 267, an A-320, in which the auto brake system was cross-wired after landing gear replacement. Technical documentation was confusing and was not followed.

Figure 11. Mr. Martin Maurino describing international accident data from IATA

Figure 9. Dr. Katherine Wilson identifies human accidents

Figure 10. Ms. Victoria Anderson describes the FAA’s investigation of Alaska Airlines Flight 261

7

• Air India Flight 717, a new B777-200ER, had a nose gear collapse at the gate. Maintenance personnel had placed the gear selector switch in the “up” position while safety pins were not installed for the landing gear; thus, the correct procedure was not followed.

Mr. Maurino reported that IATA data from 2008 showed that 15% of the world airliner accidents were caused by a maintenance error. Twenty-eight percent of accidents involving aircraft malfunctions involved maintenance. In 57% of the maintenance accidents, a deficient maintenance organization was cited as a con-tributing factor.

Action Items. The presenters felt that aviation mainte-nance experts must develop their own approaches to reduce risk in the areas identified by the accidents. However, the accident reports can identify some organizational and regulatory approaches that would reduce such risk in the future.

The factors that seemed to appear across accidents included, but were not limited to: • use of technical documentation,• corporate culture and related organizational factors,• situational awareness regarding required inspection

items,• engineer/mechanic personal responsibility,• proper post maintenance inspections, and• design for maintainability.

It is no surprise that most of the contributing factors identified in these accidents were also on the delegates’ list of challenges for maintenance human factors. These challenges are described in more detail in the recom-mendations section.

Technical Community Requirements Group (TCRG) Process. Dr. Bill Johnson (Chief Scientific & Technical Ad-

visor for Maintenance and Ramp Human Fac-tors, FAA, AIR-100) described the TCRG process, the AVS pro-cedures for defining research, developing requirements, and as-signing priorities for research funding. Dr. Johnson characterized the TCRG as a very organized process for defining and prioritizing research and develop-ment (R&D). Even though the Aviation

Safety Act was created to ensure that resources be allocated to both flight and maintenance operations, not every project gets funded due to insufficient resources. The process requires substantial development and preparation and, for the most part, every organization receives some funding for these projects.

Dr. Johnson reported that the process typically plans projects three years in advance. During the summer of 2010, the TCRG planned the TCRG requirements for FY-2013. While there is a provision for requirements that may “pop up,” the emphasis is on good long-term planning. Once a project is approved, it can be moved forward in the schedule if necessary.

Dr. Johnson provided the delegates with a percent chart to show how the resources are projected for allocation for FY-2012. He showed some example R&D projects from numerous AVS entities, as well as details about current projects and deliverables from recent AFS maintenance human factors-funded projects. Some examples of recent MX HF projects included: • fatigue risk management,• maintenance and ramp line operations safety assurance, • extensive training for FAA Aviation Safety Inspectors,• recurrent training for Inspection Authorization Cer-

tificates,• HF Ops Manuals for Maintenance, Ramps, and

Airports,• the Maintenance Human Factors Presentation System,• support of Aviation Safety Action Program, and• looking to the future of aviation maintenance/

engineering.

Dr. Johnson also covered some examples of new or con-tinued projects proposed for FY-2013. Examples included:• fatigue risk management,• line operations safety audit,• future of maintenance/engineering (including Next-

Gen),• addressing technical documentation,• knowledge capture of senior personnel, and• cost-effectiveness of MX HF programs.

The session concluded with details of the Web-based TCRG process. Dr. Johnson created an interactive example and briefly answered the following questions from the list of TCRG requirements:• How to title the R&D requirement to attract proper

attention,• How to briefly describe the requirement,• When to identify the project as a NextGen requirement,• What are the steps necessary to do the R&D,• How to build the background for the project or a

related activity,

Figure 12. Dr. Bill Johnson demonstrating how the TCRG process works

8

• What to do if the project is related to an existing or upcoming regulation, and

• How to define specific deliverables for each year of the project.

Action Items. The TCRG process is generated and justified by the technical community. Accordingly, field ideas and accompanying management support are very important factors that influence the R&D management team and AVS as they select the projects for funding. Any maintenance human factors requirements should be submitted to the AFS-300 technical community requirement group representative and/or the CSTA for maintenance human factors.

Proactive Safety Management in Maintenance and Ramp. Dr. Kevin Gildea (Human Factors Research Psycholo-gist and Principal Investiga-tor for the FAA’s research program on line operation safety audits (LOSA) in maintenance and ramp operations, Civil Aerospace Medical Institute – Human Factors Research Division) presented the scientific and operational rationale for line operation safety audits. LOSA provides a unique op-portunity to identify threats and errors in the MX and ramp environments before they lead to incidents and accidents. This is accom-plished by observing nor-mal day-to-day operations in a non-intrusive, non-punitive, anonymous manner. With the rarity of aviation accidents, the aviation community can be lulled into a sense of security. Many threats and errors can remain unidenti-fied or unaddressed for years or decades before they cause damage, injury, or loss of life. When using rare incidents and accidents as the only metrics, relatively risky and inef-ficient operations can look statistically similar to relatively safe and efficient operations. Thus, threats and errors must be addressed in a proactive manner in normal operations.

This form of proactive intervention is new to the MX and ramp communities but has already provided benefits. In the first two years of LOSA observations, airlines have already realized safety benefits and financial savings in the millions of dollars.

The MX and ramp LOSA forms and procedures were recently approved by the ATA Human Factors Committee after two years of development, beta testing, and refine-ment. CAMI is creating electronic database tools to assist in the collection, analysis, and sharing of LOSA findings. The forms, procedures, and a standalone version of the software are currently available. CAMI will also provide computer-based training modules and other LOSA sup-port materials.

Development and beta testing will continue for Part 135 base MX and Part 121 MX and ramp in FY-2011. Further database development will extend the data collec-tion, analysis, and dissemination capabilities. The ability to compare performance worldwide with virtually unlimited quantities of data will be available. This will provide greater opportunities to identify threats, errors, and associated remedies before injuries, damage, or loss of life occurs.

The industry is very interested in the LOSA process. Interested parties include maintenance, repair, and overhaul (MRO) companies, original equipment manufacturers (OEM), and international carriers and service providers. Future efforts will seek to extend the benefits of LOSA to these companies.

Action Items. The delegates agreed that LOSA research is critical as a proactive safety measure. It is necessary to continue funding LOSA and to extend Maintenance and Ramp LOSA to MRO, OEM, and larger Part 91 opera-tors. As LOSA is further developed, it will be important to create and provide a Memorandum of Agreement (MOA) for LOSA teams and begin calculating an ROI for LOSA observations. Once finalized, it is critical to communicate the ROI methodology with air carriers and service providers.

Maintenance Human Factors Rulemaking Procedures. Mr. John (Jay) Hiles (Aviation Safety Inspector, National

Staff Specialist, Human Factors, FAA, AFS-330) reviewed the general rulemaking procedures in accordance with Title 14 CFR Part 11. He described the process as somewhat protected and did not distribute his slides. Mr. Hiles demonstrated how the Office of Rulemak-ing uses a Rulemaking Project Report (RPR) to track the process. For example, when an Office of Primary Responsibil-ity (OPR) first identifies the need for a rule, the

Figure 13. Dr. Kevin Gildea shows how LOSA for maintenance and ramp can improve aviation safety

Figure 14. Mr. Jay Hiles reviews the general rulemaking procedures for maintenance human factors

9

OPR will file with the rule-making counsel, composed of managers and directors and chaired by the Director of Rulemaking. The process also requires guidance from an assembled team of subject matter experts, an analysis from the office of rulemaking, aviation policy economists, and general counsel. Although the process is thorough, tedious, and requires a lot of patience, it is effective.

Mr. Hiles speculated on what human factors rules would look like if they became a component of part 121.375. He thought that any rulemaking on the part 121 training programs would require approval rather than acceptance. He believed that corresponding Advisory materials would require training curricula comparable to the requirements outlined by EASA and Transport Canada. Eventually, these rules would promulgate to parts 135, 145, 163, and others.

Action Items. Discussion after Mr. Hiles’ presenta-tion focused on the importance of a requirement for HF training in maintenance operations. A number of delegates voiced concern that rulemaking with respect to maintenance human factors and maintenance fatigue risk management takes unnecessarily long. Delegates recommended that these maintenance human factors rulemaking efforts should receive an elevated priority.

Safety or Financial Return-on-Investment From Hu-man Factor Programs. Dr. Bill Johnson (Chief Scientific

& Technical Advisor, FAA, AIR-100) be-gan the presentation with information from Chapter 6 of The Operator’s Manual for Human Factors in Maintenance (FAA, 2007). The chapter shows how to calculate a ROI for specific hu-man factors programs interventions. The chapter, written by Dr. Johnson, emphasized that HF programs must be justified by

multiple small successes rather than by trying to imply that the HF program prevented a large catastrophic event.

After Dr. Johnson demonstrated how financial calcula-tions for a human factors intervention basically works, he explained that it is much more difficult to have an ROI calculation for safety. “Safety” is intangible and not conducive to a simple calculation but requires a number of operational measures. The discussion identified potential safety measures such as number of reworks, gate-returns, warranty claims, or lost-time job injuries. While cost

can be applied to such measures, the ROI becomes one of money rather than of safety. Inspector Keith Frable suggested an alternative method of computing ROI. Essentially, he suggested calculating the cost of inaction or “not doing something” as a way to assess the ROI.

As a final example of the complexity associated with ROI on safety programs, Johnson led a discussion to calculate the safety return on the workshop financial investment. Positive results identified by the delegates included:• Recommendations for new programs and priorities,• Development of a shared list of critical challenges, and• Establishment/reinforcement of a network of FAA

maintenance HF leaders.

While delegates emphasized the benefit and importance of the networks developed at the workshop, most struggled to put a safety ROI on such critical communications.

Action Items. Discussions indicated that the industry and FAA must pay increasing attention to the financial and safety ROI in human factors initiatives. The delegates believe this topic is important and should be prioritized for continuing applied research and development.

Dr. Johnson challenged the delegates to apply the simple model from the Operator’s Guide to the interven-tions used in the organization they oversee. He added that findings from ASAP and other voluntary reports or event investigations are excellent ways to identify targets of opportunity for ROI calculations.

wOrkshOp rECOmmENdATIONs

pre-workshop Activity and ConcernsBefore the workshop, organizers asked each attendee

to prepare one slide as part of their formal introduction to the group. Each attendee was asked to list their name, organizational affiliation, and “top 5” concerns regard-ing maintenance human factors. This section begins by summarizing that information.

For starters, the group was quite experienced, with an average aviation industry experience of 25 years. The range was from three years to 46 years, with one attendee having both pilot and A&P mechanic credentials for 44 and 40 years, respectively.

A review of the “top 5” concerns and analysis of the frequency of each concern revealed that many concerns were repeated across delegates. The top 8 concerns coming into the workshop, and the number of times they were identified, are shown in Table 1.

The introduction session was in-depth and expended most of the first morning. The level of discussion indicated a high level of commitment to developing solutions for MX HF issues and proved to be an excellent way to set

Figure 15. Dr. Bill Johnson shows delegates how to calculate return-on-investment

10

the tone for the entire workshop. The group dynamic permitted time for all to speak and delegates shared their time equitably.

The list of pre-workshop and post-workshop concerns evidenced substantial overlap. This section will only dis-cuss the pre-workshop concerns that were not identified in the top 8 post-workshop concerns.

Technical Knowledge and Skill. Approximately one-third of workshop delegates identified technical knowledge and skill as a pre-workshop concern. The workshop discus-sion revealed a concern with the underlying capabilities of new hires, many of whom are recent graduates of 14 CFR part 147 mechanic training programs. Many be-lieved that the schools and the FAA certification process do not ensure that new employees are fit for immediate employment in today’s aviation maintenance workplace. Prior to the events of 9/11 and radical changes to the aviation maintenance industry, there was time to train/mentor new mechanics. In today’s industry, the training programs have been reduced, and the structured time for on-the-job training has also been reduced. This chal-lenge is particularly exacerbated in the MRO industry because there is no requirement for a mechanic to have mechanic credentials.

Although the concerns regarding technical knowledge and skills were multi-faceted, the workshop delegates fo-cused specifically on the training content of most aircraft mechanic schools. The delegates noted that use of techni-cal documentation and modern technology procedures is not being taught in most aircraft mechanic schools.

Solutions for Technical Knowledge and Skill Deficien-cies. As is often the proposed solution, more and better training is recommended. Although training may be over-prescribed, it does seem warranted for this particular issue. Workshop participants discussed upcoming revi-sions to CFR 147, while others believed the proposed revisions are insufficient.

An alternative to training regulation might include im-proved tracking of task errors and task-time overruns. The improved documentation of error costs will likely result

in an improved awareness of the cost savings associated with additional training. This suggestion is inherently linked to the ROI calculations that are discussed in the “top 8” post-workshop concerns.

An alternative to improved training is to improve/simplify procedures and, perhaps, increase inspection.

Workplace Pressure. Workplace pressure comes in many forms for aviation maintenance personnel. In the airline operator environment (e.g., gate, flight line), the goal is on-time performance. In the maintenance repair and overhaul organization, there is pressure to complete maintenance on a pre-planned schedule. In most cases, specific tasks have an expected performance time. Re-gardless of the type of operation, the cost and margin for the maintenance job is based on time, and maintenance personnel are pushed for on-time task performance. As a result, maintenance personnel experience explicit, as well as implicit, pressure.

Unfortunately, workplace pressure is a breeding ground for procedural non-compliance. Peer-accepted non-compliance transitions into an organizational norm (“everyone does it that way”). While such procedural non-compliance is not always a safety breach, it is a known hazard. It is also against the regulations and can lead to FAA action against maintenance personnel or their organization.

Overall, delegates felt that pressure is a negative as-pect of many maintenance organizations and should be addressed with structured mitigation practices. These practices can and should be developed by the industry with FAA support.

Solutions for Workplace Pressure. It is virtually impos-sible to eliminate workplace pressure to perform the job quickly, since that is the nature of the business. Airplanes are expensive, and time on the ground for maintenance does not generate revenue. While much maintenance is performed at night, when aircraft are generally on the ground, there are not enough hours in the night or people to do the job to lessen the pressure. Workplace pressure is a characteristic of aviation maintenance.

Table 1. Priority Rank of Pre-Workshop MX HF Concerns

Rank Pre-Workshop Concerns Frequency1 Fatigue/Alertness 15 2 Technical Knowledge and Skill Levels 9 3 Return-on-Investment (ROI) Issues 9 4 Technical Publication Complacency 7 5 Production Pressure 6 6 Shiftwork Issues 6 7 Safety Culture in Maintenance 5 8 General Work Environment 4

11

Delegates discussed safety culture and its role in miti-gating the effects of pressure. Some specific solutions that could begin to address workplace pressure are:• start an applied R&D program with the FAA to

mitigate its effects,• structure event investigations to identify if workplace

pressure was a contributing factor,• assess the impact of pressure on error,• identify ways to mitigate pressure, based on data, • give maintenance personnel a reasonable approach to

address real and/or perceived pressure, • give middle managers tools or avenues to address

pressure with senior management and the workers they supervise, and

• recognize that addressing pressure is a difficult matter associated with an overall corporate safety culture.

General Work Environment. This concern is very broad but was mentioned by four delegates at the outset of the meeting. This can include aspects such as corporate cul-ture, general house-keeping, corporate and interpersonal communications, personal occupational safety, lighting, and training. This term is very broad and is difficult to address without additional detailed explanation and discussion. The workshop did not dedicate additional time to this issue in the general forum.

post-workshop prioritization of Concerns This section of the report will discuss the eight most

significant challenges that workshop delegates identified at the conclusion of the workshop. The workshop closed with an open forum discussion that was directed toward generating a list of prioritized concerns and action items. Delegates generated a list of topics and recorded all sug-gestions on white boards and charts around the room. Approximately 25 topics were identified and briefly dis-cussed. There was some redundancy in the list of 25, so the list was collapsed to create a final list. Using the final list, each attendee ranked the 10 most important topics with a closed ballot. Table 2 shows the top 8 challenges that emerged from that list. We received topic rankings from 26 of the 30 delegates. The topics were put into

a spreadsheet with subsequent rankings from each at-tendee. A point value was assigned to each rank (topics ranked number 1 were given 10 points, topics ranked as number 2 were given 9 points, etc.). The topic with the highest overall points was then assigned a rank order priority of 1. This calculation method was applied to all of the concerns to provide a rank-ordered prioritization.

The following subsections outline the top 8 challenges and offer proposed actions that are based on discussion from the workshop. To the extent possible, the actions will be listed in a bulleted list for easier comprehension and action.

Use of Technical Publications. “Fail-ure to follow proce-dures” continues to be the number-one cause of mainte-nance related events. The accident presen-tations by the FAA, NTSB, and Trans-port Canada/IATA showed “failure to follow procedures” as a contributing fac-tor in most of the accidents. Therefore, it is no surprise that the use of technical publications is the top-rated concern among AVS maintenance HF Leaders.

Unfortunately, the root cause analysis of an accident often stops after the simple finding of “failure to follow procedures.” This failure goes far beyond a “lazy mechanic” who chooses to be non-compliant. In fact, a number of other contributors have often been identified as the rea-son behind “failure to follow procedures.” For example, it may be due to an organizational issue or corporate norm. Some procedures are known from memory, while others are simply too difficult to follow with instructions in multiple manuals and multiple media, both hard-copy and digital. Mechanics sometimes get lost in the warn-ings, linked-references, and other minutia and can miss the safety-critical important details.

Table 2. Priority Rank of Post-Workshop MX HF Concerns

Rank Post-Workshop Concerns Score1 Use of Technical Publications 207 2 Fatigue/Alertness 189 3 Safety Culture in Maintenance 153 4 Event Reporting (ASAP, MEDA, other VRP) 122 5 Return-on-Investment (ROI) in MX HF 117 6 Establish MX HF as a Priority 82 7 Professionalism and Generational Issues 65 8 Attention to Required Inspection Items 46

Figure 16. Dr. Bill Johnson requesting action items from the attendees

12

In the contract MRO industry, maintenance personnel must use customer manuals for repairs and maintenance. Unfortunately, there are often significant differences in the procedures to accomplish the same task on the same model aircraft, where one customer has a half-page of instructions and another has seven pages of instructions.

In the workshop discussions, “failure to follow proce-dures” and the use of technical documentation were linked to at least five broad causes: 1) a cultural norm that allows or encourages non-compliance, 2) problematic documen-tation of technical procedures, 3) conflicting guidelines for the same task, 4) difficulty executing procedures, and 5) miscellaneous personal or work environmental factors.

One thing is certain: The list of causes underlying “failure to follow procedures” could go on and on! Since the regulated aviation industry continues to rank technical documentation as a leading safety risk, we must begin to find solutions. The industry can do better! It must address this problem.

Actions to Address Use of Technical Publications Chal-lenges. Many industries have begun to make their pub-lications synchronized and available on visual displays with video attachments that can be accessed with a cell phone or personal digital assistant (PDA). The aviation industry should pursue similar technological avenues to reduce some of the safety risk associated with expired or inaccessible technical documentation.

Perhaps time and technology will help address the challenges. However, a more proactive approach than waiting for time and technology is preferred. The fol-lowing is a list of possible activities to help better define the problem and potential solutions:• Conduct a FAA R&D project to identify the multiple

issues underlying “failure to follow procedures” and develop mitigating strategies (a proposal was submit-ted to the AVS Technical Community Requirement Group during FY2010).

• Develop event and accident investigation tools that go beyond a finding of “Failure to follow procedures.” For example, what was the corporate norm, what was the lighting, when was the mechanic’s vision last tested, what was the availability of the documentation, what was the level of pressure to complete the task, what was the oversight by experience and trained person-nel, was the mechanic rested, and what was the time of day? Often the root cause may extend beyond the documentation.

Fatigue/Alertness. The maintenance workforce is tired. Few debate that assertion. The risk associated with a fa-tigued workforce is clearly documented but has not yet

been addressed by the international aviation maintenance community.

A significant FAA study (Johnson, Mason, Hall, & Watson, 2001) showed that the average mechanic sleeps about 5 hours per day. Since 2000, it is generally known that the mechanic workforce has decreased in size, and hourly wages have decreased by as much as 35%. The fatigue issue is exacerbated by the closing of many airline hubs. Many maintenance workers must commute great distances by air or automobile because their base location closed and they cannot afford to relocate. The result is an accelerated work week where workers can complete a 40-hour work week in about 2½ days. They sleep in the poor conditions of shared “crash pads” for the few days when they are at their away-from-home work location. The issue of fatigue seems to be rooted in operations that can benefit both the workforce and the employ-ers in some way. Although there are some personal or corporate benefits with today’s schedules, the safety risk cannot be ignored.

The FAA Flight Standards Service, with the assistance of the Civil Aerospace Medical Institute, has begun to address the challenge by improving fatigue awareness through educational materials. Although this is a necessary first step in fatigue risk management, it is not enough. The workshop delegates agreed that regulations are the only way to ultimately address this issue.

Actions to Address Fatigue/Alertness Challenges. There are many ways to address the fatigue issue. The FAA, through research and development, is currently promoting safety with a hybrid approach to fatigue risk management with hours of service limitations. The current R&D project has outlined the elements needed in a rule and is developing the necessary support materials. The report on current R&D progress led to a number of workshop recommen-dations that include: • FAA should immediately initiate rules on fatigue risk

management for maintenance organizations. This recommendation is non-negotiable and is supported by industry, labor, scientists, and FAA inspectors.

• FAA should continue to work with industry to support fatigue awareness in lieu of no regulation.

• FAA should continue to objectively document fatigue-related events, the corresponding costs, and potential ROI.

Safety Culture in Maintenance. It can be easily argued that the entire final list of prioritized maintenance HF challenges can be driven by an organization’s safety cul-ture. Safety culture was ultimately rated as the number 3 concern of workshop delegates.

13

Simply defined, safety culture is a shared belief in the value of safety wherein each individual can articulate and practice specific methods of assuring safety. While safety culture can be a bit abstract, there are many concrete indices of a healthy safety culture. For example, an orga-nization with a healthy safety culture will often have an active event-reporting system and a “just culture” policy.

There is no doubt that an organization’s culture is dif-ficult to change. Maintenance personnel typically have difficulty working with abstract concepts like safety cul-ture. However, safety culture can be made more concrete with organizational programs and procedures. Once the programs and procedures are in place and an organization begins to reward its employees for compliance with the safety culture programs and procedures, a healthy safety culture will follow.

Actions to Address Safety Culture in Maintenance. This entire report provides action recommendations that will improve or promote a healthy safety culture. The workshop delegates identified specific recommendations that will help an organization embrace a stronger safety culture. Some of the most basic actions include: • do not expect a regulation about safety culture,• communicate a safety culture from the top down,• use MX HF programs to help promote and ensure a

healthy safety culture,• use ROI calculations to justify promotional programs,• consider hiring consultants to help measure safety

culture and change,• be patient – culture change is slow, • nourish the current good cultural characteristics, and• respond to opportunities for improvement.

Event Reporting. Event reporting was ranked as the number 4 concern for MX HF. Event reporting is critical because it provides the necessary data to support evolv-ing Safety Management Systems (SMS) – without data, educated safety action is impossible. Event-reporting systems can be either reactive, proactive, or both.

Dr. Bill Rankin described the Boeing MEDA system and shared some of the challenges and successes he experienced. The successful application of MEDA is characterized by, but not limited to, proper training for investigators and all employees, corporate commitment to a just culture, and application and communication of the lessons learned from the MEDA data.

Dr. Kevin Gildea described a two-year-old FAA-ATA cooperative project on maintenance and ramp line opera-tions safety audits. This system, modeled after a similar flight deck initiative, enables peer-to-peer audits of nor-mal operations. The process identifies the strengths and weakness of on-going maintenance and ramp operations.

The shortcomings are identified and threats are managed accordingly. Workshop delegates rated maintenance and ramp LOSA as a very strong contributor and component of SMS.

The FAA’s voluntary reporting systems are absolutely critical to SMS. The FAA’s Aviation Safety Action Program (ASAP) was discussed extensively. Like MEDA and LOSA the program’s success must be based on education, trust, fairness, communication, and application of the lessons learned from the reports. Most delegates felt that ASAP is one of the best examples of voluntary reporting. The ASAP reporting system empowers workers, the company, and the government to learn from events.

Each of the aforementioned reporting tools is accom-panied by implementation challenges. Some of the most predominant issues across reporting systems seem to be education, trust, and corporate or government politics. For example, some see voluntary reporting as a “get-out-of-jail-free card,” while others fear punishment from observation. Regardless of the challenges, the workshop delegates agreed that FAA and Industry leadership can-not back down on the principles of just-culture or on the high value of voluntary reporting.

Actions to Address Event Reporting Challenges. Given the multi-faceted challenges of event reporting, specific actions were recommended. Some of the key actions needed to improve event reporting include: • provide extensive education to everyone involved in

event reporting (including investigators, workers, cor-porate and government senior management, congress, the press/public),

• guard the fundamental principles of data protection and “just culture,”

• ensure that companies, governments, and individuals learn from the data,

• monitor the corrective actions progress from the reported data, and

• use data and corrective actions as a means to calculate the safety and financial ROI for event reporting systems.

Return-on-Investment in Maintenance HF. Number five on the post-workshop list of critical challenges was “Return-on-Investment.” In his presentation, Dr. Johnson showed how to make ROI calculations. The majority of the presentation was about financial return, although safety return is also believed to be highly critical. As discussed previously, the calculation of safety return is evasive and can be more difficult to estimate. In a very safe system it is difficult to measure the incremental change that one program or intervention may have on the total system safety. Regardless of difficulty, the workshop delegates agreed the safety ROI should be pursued.

14

For the past several years (since 2007), AFS-300 has submitted a research requirement related to ROI. It has never made it out of the AVS technical community requirements group committee for consideration by the Research and Development Management Team. FAA senior management has begun to ask for ROI data for some human factors R&D. To answer these requests, the R&D community must strive to implement ROI practices.

Return-on-investment calculation procedures and the demonstrated return can influence organizational actions on other maintenance HF challenges. For example, an ROI calculation might affect an organization’s priorities and investment in a human factors program, safety culture, fatigue risk management system, or technical publications.

Actions to Address Return-on-Investment in Maintenance HF. The workshop delegates suggested actions that could be used to improve the use of ROI for maintenance HF. Specific activities that could affect increased use of ROI models include:• Fund the proposed TCRG requirement on ROI

research and development that has been submitted into the process,

• Promote the ROI model presented in the Operator’s Manual for Human Factors in Maintenance (FAA, 2007),

• Recommend that TCRG requirements and propos-als include a plan for calculating ROI for safety and finances, and

• Encourage the aviation maintenance industry to tell ROI success stories and not treat such information as proprietary and/or competitive source.

Establish MX HF as a Priority. The sixth challenge was to establish maintenance HF as a priority for both industry and government. The European Aviation Safety Authority, with assistance from the Joint Aviation Authori-ties, prioritized human factors by making maintenance human factors a requirement for training amongst all current aviation maintenance workers, managers, and support personnel.

Recognizably important, raising the priority of main-tenance human factors is a fundamental issue like the number three ranking, safety culture. In other words, the prioritization of human factors, in general, will be achieved when the other identified MX HF challenges are elevated in priority.

Actions to Establish Maintenance HF as a Priority. The FAA has supported R&D funding for MX HF since 1988. In part, the support is in response to the Aviation Safety Act’s specific language on attention to human factors in maintenance and flight deck issues. The workshop delegates agreed that this important funding should be maintained at a reasonable level. Some specific action

items to elevate the priority of maintenance human fac-tors include: • Enact FAA regulations for maintenance human fac-

tors training, and• Create methods and support industry in the calcu-

lation of safety and financial ROI in maintenance human factors.

Professionalism and Generational Issues. The seventh identified challenge is a combination of two topics that are loosely related, professionalism and generational issues. The similarity is that both issues are tied to individual behavior.

In his first year in office (2009), Administrator Randy Babbitt placed a high focus on individual responsibility and professionalism. He provided examples of individual acts of unprofessionalism that lead to catastrophic events. In instances in which individuals came to work unfit for duty, it was considered failed personal responsibility. Professionalism and personal responsibility issues are a bit more abstract than HF issues such as training for communication or the use of technical documentation. Regardless, personal responsibility is critical to the safety of aviation operations and must be sought by every avia-tion professional.

The airline industry woes of this past decade have decimated airline retirement funds at a time when the public stock market has also collapsed many retirement savings. As a result, the industry has as many as four gen-erations in the workforce. These generations have wide differences in value systems, work ethics, personal ethics, ways of communicating, ways of accessing information, and more. Despite these generational differences, the workforce must work together to perform maintenance. Generational diversity can be either a hazard to safety, if mismanaged, or a significant industry strength if man-aged properly.

Actions to Address Professionalism and Generational Issues. The issues of professionalism and generational differences could benefit from the academic rigors and expertise offered by a robust applied research and development program. While in-depth study would benefit industry and the public, it is not the kind of activity that is aligned with the capabilities of the aviation industry and correspond-ing funding priorities. Regardless, the delegates believe the government must take leadership on these projects. A number of actions can be pursued to address the issues of professionalism and generational differences and include: • research and develop various types of training to ad-

dress such things as cross-training, mentorship, and communication,

• create a promotional video on AMT professionalism,• create multi-media videos that appeal to different age

15

groups with instructions about dealing with genera-tional issues, and

• encourage the use of new technologies (e.g., PDA, cell phone).

Attention to Required Inspection Items. A significant number of the accidents discussed in the workshop, as well as many FAA actions against organizations and individuals, are related to improper attention to Re-quired Inspection Items (RII). This situation seems to be failure to follow procedures, combined with compla-cency toward required inspection items. This challenge, again, is a combination of such issues as safety culture, technical documentation problems, professionalism and responsibility, and elevating the status of maintenance human factors.

Actions to Address Attention to Required Inspection Items. Given the criticality of required inspection items, the workshop delegates generated a number of immediate solutions. Interventions could include:• start a campaign that improves attention to required

inspection items among mechanics and inspectors,• involve the FAA Safety Team in the creation and dis-

semination of RII educational materials,• produce an instructional/motivational video and

Maintenance Human Factor Presentation System seg-ment that addresses the RII topic,

• encourage the use of air carrier/operator-specific train-ing guides, and

• encourage operators to development an RII On-the-Job Training (OJT) program for the individuals approved to conduct inspections for required inspection items.

Other Actions Recommended by the AVS MX HF Leaders. Workshop delegates provided recommendations that were not directly linked to the top 8 challenges. The additional suggestions include: • Create extensive documentation and associated white-

paper of the first AVS MX HF leaders workshop.• Add additional content to the Maintenance Human

Factors Presentation System DVD.• The MHFPS system has been distributed to about

20,000 users worldwide. About 5,000 of the copies were sent based on individual E-mail requests from around the world. The system, created in 2008, has been very popular and is the basis for many human factors training programs. It covers about five fun-damental human factors topics, has 150 PowerPoint slides, 40 Flash animations, and 11 video snippets. In September 2010, another video on fatigue entitled, “Grounded” was released. It will be integrated with the MHFPS and provided as a stand-alone supplement

to fatigue countermeasure training. Currently, there is no planned funding for FY-2011 additions to the MHFPS. Additional segments could be completed when funding becomes available.

• Update Advisory Circular (AC) 120.72 Maintenance Resource Management Guidelines. This AC was prepared in 1999-2000 and is overdue for a major revision. AFS has not funded this task in FY-2011, but it could be completed with resource allocation. The workshop delegates strongly recommended that this action be completed.

• FAA Aviation Safety Inspectors find themselves in a position to audit existing MX HF programs. There is a requirement for materials to support such audits. AFS has not allocated funding for this task in FY-2011. This could be completed with resource allocation. The workshop delegates strongly recommend that this action be completed.

workshop Evaluation and CommentsAn invitation and hyperlink to an online course evalu-

ation was sent to all 30 delegates following the work-shop. The invitation to provide course feedback assured anonymity. Within a two-week time period, 27 of the delegates responded (90%) with feedback. The evaluation form consisted of 17 items and was designed to assess attendee perceptions of workshop content, participation benefits, and the overall quality of the workshop. Delegates were also asked to provide comments or suggestions for improvement. The following sections will outline the results of the evaluation form.

Evaluations of workshop ContentRespondents were asked to indicate their level of agree-

ment (strongly disagree, disagree, agree, or strongly agree) with eight statements regarding workshop content. The response from delegates was overwhelmingly positive with every respondent (100%) agreeing that the workshop was well organized, constructive, and covered useful mate-rial. The majority of respondents felt that the workshop information was practical for his/her needs and interests (96.3%), but all of the respondents (100%) thought the workshop contained the appropriate level of detail, was appropriately paced, encouraged active involvement, and provided useful visual aids and handouts. Overall, the responses indicated that the workshop content was delivered in a manner that met the objectives of workshop organizers and delegates.

Evaluation of Participant Benefits. To quantify the benefits of the workshop, delegates were asked to indi-cate their level of agreement (strongly disagree, disagree, agree, or strongly agree) with a series of eight statements

16

regarding the benefits of the workshop. The responses indicate that the workshop’s benefits are far-reaching. The majority of delegates agreed (96.3%) that the workshop materials were personally beneficial (e.g., helped focus personal thoughts on MX HF, provided new insights into MX HF, and provided new information to aid in MX HF presentations). All respondents (100%) agreed that they learned information that could help them do their jobs better and that the workshop recommenda-tions could benefit FAA senior management and U.S. domestic aviation maintenance operations. The majority (96.3%) believed the workshop recommendations could benefit MX HF research and development and FAA MX HF operations. Overall, the responses indicated that the workshop was personally beneficial and could have far-reaching implications for both the FAA and domestic maintenance operations.

Evaluations of Overall QualityEach respondent was asked one broad evaluation of the

workshop overall. Respondents were asked to evaluate the course as being either poor, fair, good, or excellent. Even though a few respondents disagreed with individual items regarding workshop content and participation benefits, all of the respondents thought the workshop training was either good (14.8%) or excellent (85.2%).

suggestions for ImprovementWorkshop delegates were asked two open-ended ques-

tions. The first question, “How could the workshop be improved?” was answered by 15 respondents. A review of the suggestions for improvement revealed two common themes—extension and recurrence. Six respondents sug-gested extending the meeting to allow for more in-depth discussion, while four respondents recommended making the meeting a recurrent or annual event to continue the discussion and momentum. The remaining five sug-gestions for improvement were beneficial but would be classified as miscellaneous.

The second question was very broad and simply asked, “Any other comments or suggestions?” Twelve delegates responded, and all of the responses were complimentary or constructive. The common, overall theme revealed a positive appreciation for the workshop and expectations for continued discussion of MX HF solutions.

workshop summaryOverall, the workshop provided new insights into

the human factor issues surrounding aviation mainte-nance. More importantly, it prioritized the issues and recommended solutions. This workshop utilized a multi-disciplinary group that involved mechanics, industry managers, scientists, inspectors, and investigators. The findings should provide a reasonable representation of the core issues and provide guidance for continued hu-man factors research. Feedback from attendees indicated that more such workshops should be conducted to utilize multiple perspectives as we work together to improve the future safety of aviation.

rEfErENCEs

Federal Aviation Administration (2007). The operator’s manual for human factors in maintenance. Washing-ton, DC: Federal Aviation Administration Office of Aviation Safety. Retrieved 7/6/2011 from http://www.atcvantage.com/docs/FAA_OPs_MANUAL_HF_in_AO.pdf

Johnson, W., Mason, F., Hall, S., & Watson, J. (2001). Evaluation of aviation maintenance working environments, fatigue, and human performance. Retrieved from https://hfskyway.faa.gov/HFTest/Bibliography%20of%20Publications%5CMX%20FAA(Former%20HFSkyway)%5CDocument%20List%5CEvaluation%20of%20aviation%20 M a i n t e n a n c e % 2 0 W o r k i n g % 2 0 Environments ,%20Fat igue ,%20and%20Human%20Performance.pdf