l-l

TACTICAL DECISION MAKING UNDER STRESS (TADMUS): MAPPING A PROGRAM OF RESEARCH TO A REAL WORLD INCIDENT-THE USS VINCENNES.

Joan Hall Johnston, Jams A. Cannon-Bowers, & Eduardo Salas Naval Air Warfare Center Training Systems Division Code 4961

12350 Research Parkway Orlando, FL 32826-3275, USA

ABSTRACT

Such events as the one involving the USS Vincennes, where the decision to initiate countermeasures was the incorrect one, have focused attention on the human factor in decision making. The objective of the TADMUS program has been to apply developments in decision theory, individual and team training, and information display to the problem of enhancing tactical decision quality under conditions of stress. Sponsored by the Offtce of Naval Research, TADMUS is in its 8th year as a cooperative program in human factors and training involving SPAWAR Systems Center, San Diego, NAWCTSD, as well as other Navy, industrial, and academic organizations. The technology is being demonstrated and evaluated in the context of surface ship air warfare scenarios. This address will describe how the TADMUS program was founded and how it has progressed on a variety of R&D issues having to do with advanced training and human factors in order to address real world problems.

INTRODUCTION

On JuIy 3, 1988 an AEGIS cruiser named the USS VINCENNES accidentally shot down a commercial aircraft, Flight 655, over the Arabian Gulf killing 290 people. A number of consequences resulted from this catastrophe that included the initiation in 1990 of an Office of Naval Research sponsored research and development program named Tactical Decision Making Under Stress (TADMUS). The major program goal has been to prescribe empirically-based principles and guidelines to enhance team tactical decision making performance in complex, knowledge rich environments. Therefore, program objectives have focused on applying developments in decision theory, individual and team training, and information display to the problem of enhancing tactical decision quality under conditions of stress (Ref 1).

TADMUS is in its eighth year as a cooperative program in human factors and training involving the Naval Air Warfare Center Training Systems Division, Space and Naval Warfare Systems Center, San Diego as well as other Navy, industrial, and academic organizations. To date, the written products generated from this program in its first six years number more than 200 publications in the form ofjoumal articles, technical reports, book chapters, and symposium proceedings. In addition, over 100 product transitions have been provided to the

fleet training communities in the form of lectures, workshops, demonstrations and implementations of training tools, methods, and strategies. Finally, three large-scale advanced embedded training research initiatives have resulted (Ref 2; Ref 13). This paper describes how the strategic planning for the TADMUS program enabled us to design a roadmap for achieving a program of research accomplishments for advancing current and future developments of Navy combat team training.

BACKGROUND AND APPROACH

Three TADMUS program thrusts were identified for the purpose of meeting its program goal: definition and measurement of critical decision tasks; examination of stress effects on decision making; and development of training and simulation principles (Ref 1). Program success would be assessed by the emphasis on empirical research, rapid transition of research products to the Navy training community; and research findings that would support the development of advanced team training technologies. Below is a description of program requirements that defined our strategy for meeting program goals.

Technical Advisors Board. A Technical Advisory Board populated by senior researchers and high level fleet representatives was appointed to review technical progress twice per year until program completion. Without a doubt, the frequent meetings ensured that researchers remained focused on areas of work that were going to bear fruit; overall, the Technical Advisory Board helped keep the research on track.

Subiect Matter Expertise. Inclusion of Navy subject matter expertise, training commands, and ship teams in the design and development of research experiments and products was an integral part of the TADMUS plan, To date, virtually hundreds of active duty officers and enlisted men and women have played a role in shaping the research products so they would have a realistic and substantive impact on current and future Navy training.

Defining and Designing a Realistic Team Task. The VINCENNES incident was due, in part, to errors resulting from the “Air Warfare” (AW) team interactions, therefore, the research domain was to examine the AW task in a ship’s Combat Information Center (CIC). The CIC is the central information processing and tactical decision making area for a

Paper presented at the RTO HFM Symposium on “Collaborative Crew Performance in Complex Operational Systems”, held in Edinburgh, United Kingdom, 20-22 April 1998, and published in RTO MP-4.

1-2

surface combatant (Ref 4). The main focus of research would be the Air Warfare team on an Aegis capable ship, which is composed of the Commanding Officer, Tactical Action Officer, Air Warfare Coordinator, Tactical Information Coordinator, Identification Supervisor, and the Electronics Warfare Supervisor. During air warfare, the AW team performs a series of tasks including detecting, tracking, identifying radar contacts, taking action on these contacts, and performing battle damage assessment (Ref 4).

Once the operational task was determined, then choices had to be made regarding a team-based research testbed. High costs and lack of access meant that utilizing a ship’s CIC or a shore- based high-fidelity team trainer was out of the question. Therefore, a five-person networked PC-based system called the Decision Making Evaluation Facility for Tactical Teams (DEFTT) was developed to support research experiments. Although, as a result of a research tradeoff, DEFTT was low in functional fidelity, it was determined that the system presented the tactical problem with enough cognitive fidelity that we could “simulate” the actual AW team activities taking place in the real combat system (Ref 5; Ref 6).

Without a doubt, conducting team experiments was Teams. the most challenging of all the research tasks. We determined that most of the experiments would include fve-person teams. Consequently, as with developing DEFTT, it was not realistic to expect our research participants to be highly experienced operators that had worked together as a team for an extended period of time (i.e., intact). Therefore, we developed a plan so that we could eventually gain access to such teams. We organized our research tasks to be conducted at three shorebased training commands where we could have access to novice, experienced, and very experienced trainees. In addition, we spent the first two years of the program at a high fidelity combat team training facility where we conducted numerous interviews and developed a database from which our research scenarios were developed. We were able to establish DEFTT at one of the school commands, and another command had already adopted it as a trainer for division officers, department heads, and prospective commanding officers. In addition, we established a low fidelity 3-person team trainer at a school command that allowed novice Navy recruits to participate in some of the basic research experiments. To date, over 280 teams of Navy trainees have participated in the research. Once we had established a reliable research protocol we were able to have intact and ad hoc experienced ship teams participate in the research (over IO teams to date), as well.

Event-Based Scenarios, Stressors. and Measurement. To ensure that the innovative training could be evaluated, we developed a strategy-the Event-Based Approach to Training, or EBAT-to support research scenario design, measurement

tool development, and operational stressor implementation (Ref 7; Ref 8). We structured two pairs of AW Arabian Gulf scenarios. Each pair was composed of one low and one high stress scenario, but both shared the same events. Stressors were defined as workload (e.g., added aircraft and ships) and information ambiguity (e.g., increased number of difficult problems to solve). Each scenario was developed with three significant “events” whereby individual and team behaviors could be specified by subject matter experts, and observed by trained raters (e.g., two hostile aircraft popup at close range to the team’s own ship). The EBAT strategy thus provided a way to ensure we could assess individual and team performance with measurement tools that were designed to capture performance processes and outcomes (Ref 7). Next is a brief description of the tools as they related to EBAT.

The Behavior Observation Booklet was designed for assessing individual task processes. At each scenario event, we identified task requirements at the individual operator level so that they had a performance score for each event. An outcome score measured by the Sequenced Actions and Latencies Index represents the ability to perform the tasks correctly and on time. The Air Warfare Team Observation Measure assesses team level performance for information exchange, initiative, supporting behaviors, and communication. The team outcome measurement tool (Air Warfare Team Performance Index) assesses timeliness and accuracy as a team on the detect-to-engage sequence. As a result of developing these tools, we were able to assess the stressfulness of research scenarios, and to guide assessment of the impact of the TADMUS training (Ref 9).

The Training Research Agenda. In the beginning of TADMUS, syntheses of the research literature on decision making, teams, and stress were conducted to identify and develop innovative training strategies (Ref 1). Consequently, a research agenda was designed so as to test the individual effects of such training on enhancing skills in critical thinking for decision making, teamwork and team self-correction, handling stress exposure, and leadership (Ref 10). In addition, training strategies and methods were tested to assess the impact of part task training, cross-training, and instructor training to enhance performance feedback strategies (Ref I I ; Ref 6; Ref 12). The data collection effort started in 1992 and continues through 1999 in order to establish the impact of an integrated training program with the decision support system developed by SPAWAR in San Diego. To date, empirical data has been collected from over 95 five-person teams (including experienced intact ship teams) and findings have shown training imposed significant improvements in performance, often at levels of 30-40 percent (Ref 10).

Product Transitions: Short-term and Long Term. Although an applied research program, emphasis was placed on ensuring the fleet would gain short and long term benefits from our

findings. In the short-term, numerous such activities as workshops and demonstrations have and continue to take place. As an example of building a strategy to transition training to the shipboard environment, we demonstrated empirical support for training teamwork skills (Team Dimensional Training) in the laboratory and then in the shipboard environment, whereby we eventually gained the support and endorsement of the afloat training group for incorporating and implementing Team Dimensional Training (Ref 12; Ref 13). For the long-term, we have initiated several advanced research programs to ensure that shipboard embedded training that includes our TADMUS training methods, tools, and strategies will be incorporated into new ship platforms of the 21” Century (Ref 2).

SUMMARY AND CONCLUSIONS

We have described the strategy we used to guide us on the roadmap for achieving the TADMUS objectives. The Technical Advisory Board, subject matter expertise, research task design and team participants, EBAT, the research agenda, and short and long-term transitions were crucial to ensuring the program’s ongoing success. In conclusion, the vital components in all of these tasks were: (1) ensuring that empirical results were based on a reasonably realistic task that included team participation, and (2) that fleet participation- the customer-had input throughout the program.

REFERENCES

1.

2.

3.

4.

5.

Cannon-Bowers, J.A., Salas, E., & Grossman, J.D., “Improving Tactical Decision Making Under Stress: Research Directions and Applied Implications”, in the “International Applied Military Psychology Symposium”, 1991.

Naval Air Warfare Center Training Systems Division. “Research and Development Project Summaries”. NAWCTSD, October 1997.

Zachary, W., Bilazarian, P., Bums, J., & Cannon-Bowers, J. A., “Advanced Embedded Training Concepts For Shipboard Systems”, in the “Proceedings of the 19’h annual Interservice/Industry Training, Simulation and Education Conference”, December 1997, pp. 670-679.

“CG 52 Class Combat System Doctrine”, Naval Surface Force. COMNAVSURFLANT Instruction C35 16.39. March 1989.

Holl, R. E. & Cooke, J. R., “A Recognition-primed decision (RPD) model of rapid decision making”, in the “Proceedings of the I lth annual Interservice/Industry Training Systems Conference”, 1989, pp. 337-342.

6.

7.

8.

9.

10.

11.

12.

13.

l-3

Kirlik, A., Rothrock, L., Walker, N., & Fisk, A. D., “Simple Strategies or Simple Tasks? Dynamic Decision Making in “Complex” Worlds”, in the “Proceedings of the Human Factors and Ergonomics Society 40th Annual Meeting”, Human Factors and Ergonomics Society, 1996, I, pp. 184-188.

Hall, J. K., Dwyer, D. J., Cannon-Bowers, J. A., Salas, E., & Volpe, C. E., “Toward Assessing Team Tactical Decision Making Under Stress: The Development of a Methodology for Structuring Team Training Scenarios.” in the “Proceedings of the 15th Annual InterserviceiIndustry Training Systems Conference”, December 1993, pp. 87-98.

Oser, R. L., Cannon-Bowers, J. A., Dwyer, D. J., & Salas, E., “Establishing a Learning Environment for JSIMS: Challenges and Considerations”, in the “Proceedings of the 1 9’h annual InterserviceiIndustry Training, Simulation and Education Conference,” December 1997, pp. 144- 153.

Johnston, J. A., Smith-Jentsch, K. A., & Cannon-Bowers, J. A., “Performance Measurement Tools for Enhancing Team Decision Making”, in M. T. Brannick, E. Salas, & C. Prince (Eds.), “Team Performance Assessment and Measurement: Theory, Methods, and Applications”, Mahwah, NJ: USA, LEA, 1997, pp. 31 l-327.

Salas, E., Cannon-Bowers, J. A., & Johnston, J. H., “How can you turn a team of experts into an expert team? Emerging training strategies”, in C. Zsambok & G. Klein (Eds.), “Naturalistic decision making “, Hillsdale, NJ, USA, LEA, 1997, pp. 359-370.

Duncan, P. C., Rouse, W. B., Johnston, J. H., Cannon- Bowers, J. A., Salas, E., & Bums, J. J., “Training Teams Working in Complex Systems: A Mental Model-Based Approach.” in W. B. Rouse (Ed.), “Human/technology interaction in complex systems”, 8, 1996, pp. 173-23 1.

Smith-Jentsch, K.A. Johnston, J.H., Cannon-Bowers, J.A., & Salas, E., “Team Dimensional Training: A Methodology for Enhanced Shipboard Training”, in the “Proceedings of the 1 gth annual meeting of the Interservice/Industry Training, Simulation and Education Conference”, December 1997.

Smith-Jentsh, K.A., Payne, S. C, & Johnston, J.H. “Guided Team Self-Correction: A Methodology for Enhancing Experiential Team Training”, in K.A., Smith- Jentsch (Chair), “When, How, and Why Does Practice Make Perfect?“, Paper presented at the 1 llh annual conference of the Society for Industrial and Organizational Psychology, 1996.