Baselining an Acceptable Level of Safety With Regards to ...

Tom Nesthus Engineering Research Psychologist, FAA Civil Aerospace Medical Institute

Baselining an Acceptable Level of Safety With Regards to Fatigue

Fatigue Management Approaches Symposium 5-6 April 2016, Montréal, Canada

2

Baselining an Acceptable Level of Safety With Regards to Fatigue

Thomas E Nesthus, Ph.D. FAA

Civil Aerospace Medical Institute Oklahoma City, OK

Federal Aviation Administration Baselining an Acceptable Level of

Safety With Regards to Fatigue

Thomas E Nesthus, Ph.D. FAA

Civil Aerospace Medical Institute Oklahoma City, OK

ICAO Symposium for Fatigue Management Approaches in Aviation April 5, 2016

Federal Aviation Administration 3

Overview I. Fatigue regulations

II. Daily fatigue management and adjustment

III. Confidence building with safe regulations

IV. Provisions for safely exceeding regulation limitations

V. Advancement of science and utilization within the industry



Safety and Fatigue Mitigation Begins with Good Regulations

I. Need “fatigue smart” prescriptive limitations/regulations

A. Should include policies/procedures for monitoring, enforcement, and adjustment

B. Must have confidence that these regulations promote safe (fatigue-minimizing) limitations and restrictions 1) Particularly for operations approaching limitations



Fatigue Management for Day-to-Day Operations II. Need Policy/Procedure for managing fatigue on a daily basis within the

structure of the prescriptive limitations/regulations A. Recommend components such as:

• Senior level management commitment • Current flight time and duty period limitations • Rest scheme consistent with limitations • Fatigue reporting policy and system • Fatigue education and awareness training program • Fatigue incident reporting process • Flightcrew fatigue monitoring system • Evaluation process

B. Must have a mandate for corrective action when evidence of fatigue risk

is found



Regulatory Confidence building III. Confidence in regulations comes with experience and monitoring

A. Monitoring daily operations and making adjustments when elevated fatigue risk is found, is crucial 1) Particularly with trip sequences, back-to-back operations, and those

near the limitations of the regulation

B. Confidence also gained by cooperative sharing of information with others in the industry 1) Voluntary sharing of examples of “discovered” risk and “offered”

solutions (i.e., please learn from our experience) 2) Sharing best safety practices



Safely Exceeding Regulation Limitations IV. Develop policy/procedures for comparisons with known safe operations as the baseline for extensions or exceedance proposals

A. AMOC must target specific limitation(s) and justify request for exceeding those limits

B. Must include enhanced fatigue mitigation strategies that provide an “alternative method of compliance” (AMOC) to the rule

1) If you are asking to fly longer routes you must propose more than just the minimum mitigations

C. Requires identification of appropriate operational safety indicators and data collection procedures



Safely Exceeding Regulation Limitations (cont) IV. Develop policy/procedures for comparisons with known safe operations as the baseline for extensions or exceedance proposals

D. AMOC must ultimately provide a demonstration of an equivalent level of safety

1) accomplished through required data collection and appropriate comparisons (i.e., with a safety standard operation; SSO)

E. Must involved appropriate analyses (e.g., equivalence testing)

F. Requires a thorough review of the results

G. Recommend publishing the results



Advance science and share with the industry V. Publish results for advancement of science and utilization within the industry A.Research is expensive, but it needs to be done

B.Expanding the body of research in this area, fills knowledge gaps

C.Published results on extended FDPs is particularly important for safety cased development



Moderator’s Question: • Please inform us with a little more discussion on the topic of

equivalence testing?



AMOC and Statistical Procedures • Traditional ANOVA or t-tests look primarily at variability and group

differences • However, we cannot conclude equivalence from a difference

test…Why? • Because non-significance can be the result of a number of things

including small samples with inadequate power to show differences • Altman & Bland (1995) stated: “Absence of evidence is not evidence

of absence” • This simply means that the data are inadequate to determine if

there is a difference • A difference might still be found with a larger sample size

Altman, DG and Bland, JM. Absence of evidence is not evidence of absence. BMJ 1995, Aug 19, 311(7003):485



Statistical Procedures (cont.) • But, there are methods to test group equivalency, and should be

implemented whenever this is the true goal of the research question • Since it is not possible to prove that two treatments are identical, such

as with generic vs original drug formulations*, the aim is to show that they’re bio-effects do not differ by more than a specified and immaterial amount

• By using this analysis approach, you must determine what a

meaningful/practical difference is, even before you start *FDA (1992), “Bioavailability and Bioequivalence Requirements,” in U. S. Code of Federal Regulations (vol. 21, chap. 320),Washington, DC: U. S. Government Printing Office



Regular t-test vs. Two One-Sided Tests (TOST) Approach

Wellek, S. (2010). Testing Statistical Hypotheses of Equivalence and Noninferiority.



Confidence Intervals and Practical Differences Approach



Summary



Equivalence Testing References Barker, Luman, McCauley, and Chu. Assessing Equivalence: An Alternative to the Use of Difference Tests for Measuring Disparities in Vaccination Coverage. Am J Epidemiol 2002;156:1056–1061. Barker, Rolka, Rolka, and Brown. Equivalence Testing for Binomial Random Variables: Which Test to Use? Am Stat 2001; 55(4): 279-87. Bland and Altman. Comparing methods of measurement: why plotting difference against standard method is misleading. Lancet 1995; 346:1085-87. Farrington and Manning. Test statistics and sample size formulae for comparing binomial trials with null hypothesis of non-zero difference or nonunity relative risk. Stat Med 1990; 9:1447–54. Jones B, Jarvis P, Lewis JA, et al. Trials to assess equivalence: the importance of rigorous methods. BMJ 1996;313:36–9.



Equivalence Testing References Richter S and Richter C. A Method for Determining Equivalence in Industrial Applications. Quality Engineering, 14(3), 375–380 (2002). Schuirmann, D.J. A comparison of the Two One-Sided Tests Procedure and the Power Approach for assessing the equivalence of average bioavailability, J. Pharmacokinetics and pharmacodynamics, 115: 1567, 1987. Wellek, S. Testing Statistical Hypotheses of Equivalence and Noninferiority. 2nd Ed, Chapman & Hall/CRC Press, 2010, ISBN: 978-1-4398-0818-4. An intuitive discussion is available from Dr. Harvey Motulsky and the authors of GraphPad Prism in an online Statistical Guide that outlines their approach to equivalence testing: http://www.graphpad.com/guides/prism/6/statistics/index.htm?stat_testing_for_equivalence_with_c.htm

Baselining an Acceptable Level of Safety With Regards to ...

Documents

Transcript of Baselining an Acceptable Level of Safety With Regards to ...