Development of Training Themes for Joint, Interagency,..(2010) Ross, K. US Army Research
-
Upload
lauren-cat -
Category
Documents
-
view
215 -
download
1
description
Transcript of Development of Training Themes for Joint, Interagency,..(2010) Ross, K. US Army Research
U.S. Army Research Institute For the Behavioral and Social Sciences
Research Note ####
Development of Training Themes for Joint, Interagency, Intergovernmental and Multinational (JIIM) Operations
Karol G. Ross Cognitive Performance Group
Anna Grome
Klein Associates Division, ARA
Meagan C. Arrastia Cognitive Performance Group
Brooke Schaab
US Army Research Institute
James Ong Stottler Henke Associates, Inc.
David Spangler
Global Innovation and Design, Inc.
May 2010
Approved for public release; distribution is unlimited.
U.S. Army Research Institute for the Behavioral and Social Sciences A Directorate of the Department of the Army Deputy Chief of Staff, G1 Authorized and approved for distribution:
MICHELLE SAMS, Ph.D.
Director
Technical review by
NOTICES
REPORT DOCUMENTATION PAGE Form Approved OMB No. 0704-0188
Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188), Washington, DC 20503.
1. AGENCY USE ONLY (Leave blank)
2. REPORT DATE
February 2010
3. REPORT TYPE AND DATES COVERED
October 15, 2006 through October 31, 2009
4. TITLE AND SUBTITLE
Development of Training Themes for Joint, Interagency, Intergovernmental and
Multinational (JIIM) Operations
5. FUNDING NUMBERS
W91WAW-07-C-0067
6. AUTHORS - Karol G. Ross, Anna Grome, Meagan C. Arrastia, Brooke Schaab, James Ong, David Spangler
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)
Stottler Henke Associates, Inc.
951 Mariner’s Island Blvd., Suite 360
San Mateo, CA 94404
8. PERFORMING ORGANIZATION REPORT NUMBER
Phase II Research Report No. 345
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)
ERDC Contracting
Champaign Office
PO Box 9005
Champaign, IL. 61826
10. SPONSORING/MONITORING AGENCY REPORT NUMBER
11. SUPPLEMENTARY NOTES
NONE
12a. DISTRIBUTION/AVAILABILITY STATEMENT
Distribution authorized to US Government only; Proprietary Information (DFARS – SBIR Data Rights). Other requests for this document shall be referred to the DoD Controlling Office or the DoD SBIR Program Office.
12b. DISTRIBUTION CODE
13. ABSTRACT (Maximum 200 words)
The contemporary operating environment requires a new emphasis on collaboration across services, across government agencies and with non-governmental agencies and host nations to create global stability in the interest of national security. This report documents an effort to capture the expertise of experienced military and non-military players, combined with operational lessons learned and previous research to insure that training and education will prepare our forces for Joint, Interagency, Intergovernmental and Multinational (JIIM) Operations. First, we reviewed documents that reflected lessons learned and insights into these operations. Second, we conducted interviews with experts in collaboration between the military and other organizations, as well as with host nation officials and citizens. Third, we analyzed interview transcripts to derive themes that expressed the high-level cognitive skills in these operations. We identified these themes: Understand the situation within its historical, regional, and cultural context; Understand the other participants; Shift perspective; Establish and maintain common ground; Build capability to affect the situation; Visualize the operation; Support information exchange; and Maintain flexibility. We describe our six-step thematic analysis method and provide analysis tables to clarify the process. Finally, we administered surveys and held focus groups to validate the themes with experienced Civil Affairs officers.
14. SUBJECT TERMS
Stability Operations, Joint Interagency Intergovernmental and Multinational Training, Training Themes, HASE, JIIM, Thematic Analysis
15. NUMBER OF PAGES
186
16. PRICE CODE
17. SECURITY CLASSIFICATION OF REPORT
UNCLASSIFIED
18. SECURITY CLASSIFICATION OF THIS PAGE
UNCLASSIFIED
19. SECURITY CLASSIFICATION OF ABSTRACT
UNCLASSIFIED
20. LIMITATION OF ABSTRACT
UNLIMITED
NSN 7540-01-280-5500 Computer Generated STANDARD FORM 298 (Rev 2-89)
ii
Research Note ####
Development of Training Themes for Joint, Interagency,
Intergovernmental and Multinational (JIIM) Operations
Karol G. Ross
Cognitive Performance Group
Anna Grome
Klein Associates Division, ARA
Meagan C. Arrastia
Cognitive Performance Group
Brooke Schaab
US Army Research Institute
James Ong
Stottler Henke Associates, Inc.
David Spangler
Global Innovation and Design, Inc.
Fort Leavenworth Research Unit
Stanley M. Halpin, Chief
U.S. Army Research Institute for the Behavioral and Social Sciences
2511 Jefferson Davis Highway, Arlington, Virginia 22202-3926
May 2010
iii
DEVELOPMENT OF TRAINING THEMES FOR JOINT, INTERAGENCY,
INTERGOVERMENTAL AND MULTINATIONAL (JIIM) OPERATIONS
ACKNOWLEDGEMENTS
The authors would like to thank Partners International Foundation who organized the
Health African Scenarios Exercise workshop and shared workshop data with us. We also
thank faculty at Joint Forces Staff College, members of the 91st Civil Affairs Battalion at
Ft. Bragg, the 350th
Civil Affairs Command and others who shared their expertise and
perspectives on Joint, Interagency, Intergovernmental, and Multinational stability and
support operations.
iv
DEVELOPMENT OF TRAINING THEMES FOR JOINT, INTERAGENCY,
INTERGOVERMENTAL AND MULTINATIONAL (JIIM) OPERATIONS
EXECUTIVE SUMMARY
Research Requirement:
The contemporary operating environment for U.S. military forces requires a new
emphasis on collaboration across services, across government agencies, with non-
governmental agencies, and with host nations in order to create global stability in the
interest of our national security. While operations are driven by U.S. national objectives,
our interests are increasingly dependent on defining common goals with others to support
the security and stability of a wide range of regions and nations in which we currently are
engaged or soon will find ourselves conducting operations. The skills to determine
relevant objectives and take effective actions in this collaborative environment have long
been a part of U.S. military capabilities, but these skills are now more in the foreground
of operations and are required of a wider range of personnel from the tactical to the
strategic level. New and emerging doctrine directs the nature of these interactions for the
U.S. military. While doctrine provides guidance and structure, it does not capture the
expertise required to successfully apply that doctrine. The purpose of this research effort
was to identify the high-level cognitive skills required for effective performance in Joint,
Interagency, Intergovernmental and Multinational (JIIM) environments to serve as
themes to support training development.
Procedure:
The approach to theme development included three steps. First, we identified and
analyzed relevant documents that reflected lessons learned or other insights into
operations in JIIM environments. Second, we conducted a series of interviews with
subject matter experts whom we had identified as experienced in operations that required
collaboration among the military and other organizations, as well as host nation officials
and citizens. Third, we analyzed the interview transcripts to derive themes that expressed
the high-level cognitive skills evident in the experiences. To represent our findings, we
combined interview findings with insights gained from the documents to produce a
matrix of themes.
The method for theme validation included a quantitative and qualitative approach.
First a survey instrument consisting of 54 items was developed from the previous
interviews with JIIM operators in order to explore the degree of consensus among a
different set of SMEs on the value of the eight themes identified. The items are
statements describing the skills that make up each theme with the answer choices on a 5-
point Likert scale ranging from very important to mission success to very unimportant to
mission success. In addition to the survey, a focus group was conducted to discuss the
themes with a subset of the individuals that have actually had to employ such skills in
v
JIIM environments. Feedback about the themes in their own words, as well as examples
of each theme, added richness to the data that allowed the themes to be further explored.
Findings:
A total of eight themes emerged from the data. The themes are listed below. Each
theme is presented as a cognitive act that the practitioner in the domain must perform. In
addition, the set of themes was integrated into a descriptive model to show their inter-
relationships and application over the course of an operation.
1. Understand the situation within its historical, regional, and cultural context
2. Understand the other participants
3. Shift perspective
4. Establish and maintain common ground
5. Build capability to affect the situation
6. Visualize the operation
7. Support information exchange
8. Maintain flexibility
The themes were validated as important to the vast majority of the SMEs surveyed.
They recommended few changes to the wording of the definitions and accompanying
cognitive challenges.
Utilization and Dissemination of Findings:
This research produced a set of high-level cognitive skills in the form of themes that
will be used to build a training product. The scenario-based tutorials in the product will
support the development of a common understanding of successful collaboration in JIIM
environments in a training audience that varies in levels of expertise. Use of the product
will prepare participants to engage in more difficult training and exercises based on the
understanding of the doctrine, processes and skills needed for successful collaboration
which they will acquire.
vi
DEVELOPMENT OF TRAINING THEMES FOR JOINT, INTERAGENCY,
INTERNATIONAL AND MULTINATIONAL (JIIM) OPERATIONS
CONTENTS
INTRODUCTION .............................................................................................................. 1
DEVELOPMENT OF TRAINING THEMES.................................................................... 2
Method ............................................................................................................................ 3
Research Question ...................................................................................................... 3
Participants .................................................................................................................. 3
Data Collection Procedure .......................................................................................... 4
Analysis Plan .............................................................................................................. 4
Results ............................................................................................................................. 5
VALIDATION OF THEMES............................................................................................. 8
Method ............................................................................................................................ 8
Research Question ...................................................................................................... 8
Participants .................................................................................................................. 8
Data Collection Procedure .......................................................................................... 8
Analysis Plan .............................................................................................................. 9
Results ............................................................................................................................. 9
Survey Results ............................................................................................................ 9
Focus Group Results ................................................................................................. 10
DISCUSSION ................................................................................................................... 10
REFERENCES ................................................................................................................. 12
APPENDIX A: LIST OF PARTICIPANTS AND DOCUMENTS ............................... A-1
APPENDIX B: EXAMPLES OF DATA ANALYSIS TABLES ................................... B-1
APPENDIX C: THEMES FOR OPERATIONS IN JIIM ENVIRONMENTS .............. C-1
APPENDIX D: THEME VALIDATION SURVEY ...................................................... D-1
APPENDIX E: THEME VALIDATION FOCUS GROUP GUIDE ............................... E-1
APPENDIX F: THEME VALIDATION SURVEY RESULTS ..................................... F-1
vii
LIST OF FIGURES
Figure 1. Model of High-Level Cognitive Skills Required for Joint, Interagency,
Intergovernmental and Multinational Operations ............................................................... 7
LIST OF TABLES
Table 1. Summary of Survey Findings for Validation of Themes .................................... 10
1
DEVELOPMENT OF TRAINING THEMES FOR JOINT, INTERAGENCY,
INTERGOVERMENTAL AND MULTINATIONAL (JIIM) OPERATIONS
INTRODUCTION
The contemporary operating environment for U.S. military forces requires a new
emphasis on collaboration across services, across government agencies, with non-
governmental agencies, and with host nations in order to create global stability in the
interest of our national security. While operations are driven by U.S. national objectives,
our interests are increasingly dependent on defining common goals with others to support
the security and stability of a wide range of regions and nations in which we currently are
engaged or soon will find ourselves conducting operations. The skills to determine
relevant objectives and take effective actions in this collaborative environment have long
been a part of U.S. military capabilities, but these skills are now more in the foreground
of operations and are required of a wider range of personnel from the tactical to the
strategic level.
New and emerging doctrine directs the nature of these interactions for the U.S.
military.1 While doctrine provides guidance and structure, it does not capture the
expertise required to successfully apply that doctrine. The purpose of this research effort
was to identify the high-level cognitive skills required for effective performance in Joint,
Interagency, Intergovernmental and Multinational (JIIM) environments.
We addressed this requirement by identifying the expertise of experienced members
of the military and non-military players in these settings. We combined that
understanding of expertise with operational lessons learned and previous research in
order to build an integrated set of themes that reflect the skills required for collaboration
in the JIIM environment. Expertise in operations in JIIM environments is concentrated
within certain elements of the military forces such as Civil Affairs, combat arms units
who have successfully engaged in counterinsurgency operations, members of Provincial
Reconstruction Teams, and those who have served in Joint or State Department positions.
Additionally, insights into expert performance are also found in non-government
organizations that are part of the collaborative efforts these operations require. Our
approach to understanding expert performance was to conduct interviews across this
range of experienced people using a critical incident approach and then to identify themes
that can inform the focus of training and education. The purpose of this report is to
document the results which will be used for the development of a training product
intended to build an understanding of this domain across a diverse audience and to
improve collaboration in the JIIM environment. The process by which we turned the
expertise into training themes and then validated the themes is described here in detail to
stimulate a consistent approach to developing themes as the basis of training development
in complex domains. We present the themes in both a matrix and integrated into a model
that indicates the application of the themes in operation.
1 See for example U.S. Army Field Manual 3-07 Stability Operations, October 2008; Joint Publication 3-
07.3, Peace Operations, 17 Oct 07; and DoD Directive 3000.05, Military Support for Stability, Security,
Transition, and Reconstruction (SSTR) Operations, 28 Nov 05.
2
DEVELOPMENT OF TRAINING THEMES
The concept of using themes to guide cognitive development through situated
learning was introduced in the research literature as part of the constructivist approach to
instruction. The goal of theme-based instruction is to go beyond superficial familiarity
with concepts and facts to mastering the conceptual complexity of an area. Thematic
instruction aids the learner in going over the same problem from different viewpoints.
This instructional strategy is said to promote cognitive flexibility in a domain of practice
and avoids counter productive training, i.e., training that inhibits transfer of knowledge to
field performance. Examples of counter-productive training include oversimplification of
concepts, linear presentation of material which ignores the inter-related nature of
concepts, using one exemplar leading to a student perception of one right answer for
complex problems, and using simple analogies for complex systems (Spiro et al, 1992).
Theme-based instruction when applied as a method to help the student explore situations
from different perspectives produces a more cognitively complex understanding of a
domain more quickly, as well as more flexible transfer of concepts to field performance.
Theme-based training of cognitive skills has since been successfully implemented
for the military. Spiro and Jehng (1990) provided a model of Cognitive Flexibility to
guide learning which also proposed the concept of a deeper level of cognitive insight
gained by theme-based exploration of situations or scenarios. They asserted that this
method of learning supported the ability to spontaneously restructure the knowledge
gained in adaptive response to changing situations. Spiro, Feltovich, Jacobson, &
Coulson, (1992) demonstrated this approach with adult learners in the area of tactical
thinking. They used one case study, the battle of Chancellorsville—a case study often
used in U.S. Army tactical education—to support theme-based learning. Their efforts
were focused on demonstrating that a deeper conceptual understanding of tactics could be
gained and generalized to new settings by using a theme-based structure to examine a
scenario or case in depth. During the development of the ―Think like a Commander‖
(TLAC) training environment, a number of high-level cognitive skills common across
expert tacticians were summarized to guide deliberate practice in this domain. The
approach of TLAC is to explore a situation from the multiple perspectives. In line with
the earlier examples of this type of instructional strategy cited above, the high-level
constructs were dubbed ―themes‖ (Ross, & Lussier, 1999). This successful theme-based
approach to training thinking skills (Lussier, Shadrick, & Prevou, 2003) has since been
used to produce training for military crisis management thinking skills in the Red Cape
tool (Shadrick, Schaefer, & Beaubien, 2007).
Theme-based training as an approach to situated or scenario-based training and
education is dependent on generating acceptable themes that reflect the high-level
cognitive skills in a domain. While a theme-based approach has been successful in
military training in limited applications, the approach for identifying relevant and useful
themes to guide such training has not been documented.
Thematic analysis is generally used to pull meaning from the data in order to
understand a phenomenon or specific aspects of a phenomenon that are psychological
(such as the experience of choosing cosmetic surgery or choosing treatment during
3
serious illness) or social in nature (such as patterns of drug abuse in a particular segment
of society). Our use of thematic analysis is phenomenological in nature as we seek to
understand particular situations from the point of view of subject matter experts, but is
aimed specifically at understanding their high-level cognition for the purpose of
supporting learning that will go beyond the procedural. In the qualitative research
literature, ―[t]hematic analysis is widely used, but there is no clear agreement about what
thematic analysis is and how you go about doing it‖ (Braun, & Clarke, 2006, p. 79).
There are a number of methods that can generally be thought of as thematic analysis
(such as conversation analysis, interpretative phenomenological analysis, discourse
analysis, and narrative analysis). Our approach is detailed below.
Method
Our approach included three steps. First, we identified and analyzed relevant
documents that reflected lessons learned or other insights into JIIM operations. Second,
we conducted a series of interviews with subject matter experts whom we had identified
as experienced in operations that required collaboration among the military and other
organizations, as well as host nation officials and citizens. Third, we analyzed interview
transcripts to derive themes that expressed the high-level cognitive skills evident in the
experiences. To represent our findings, we combined interview findings with insights
gained from the documents we had reviewed to produce a matrix of themes.
Research Question
The focus of this research effort is on the high-level cognitive skills that underlie
successful performance in JIIM operations. Our intention was to reflect the findings of
critical incident accounts and the perceptions of the individual as they pertained to
successful performance.
Participants
We recruited interview participants with experience in some aspect of JIIM
environments. Our goal was to compare multiple perspectives at tactical and operations
levels. Participants in the military or associated with military organizations were
volunteers. Participants from other organizations were compensated for their time as
subject matter experts (SMEs). We interviewed a total of 15 military or military-related
SMEs; three African nationals, one affiliated with a non-government organization and
two with a government health organization; and five SMEs with Department of State
experience for a total of 23 interviews. Data from one additional military interviewee was
not used.
Prior to the interviews we reviewed a number of relevant documents that capture the
skills required of participants in JIIM operations. (See Appendix A for a complete list.)
We reviewed 1) JIIM lessons learned (Agrait, & Loughran, 2007); 2) transcripts of three
days of meetings of the HASE (Healthy Africa Scenarios Exercise) Workshop in Ghana,
Africa held January 21-25, 2008; 3) a briefing on the Joint, Interagency, and
4
Multinational Planner‘s Course from the Joint Forces Staff College; 4) a report on
cognitive challenges in operations other than war (Miller et al., 2003); 5) a report that
reviewed the cognitive challenges of working across military organizations (O‘Dea et al.,
2006); 6) a report on the elements of cross-cultural competence required for military
operations (Ross, 2008); and 7) a report on modeling cross-cultural competence in the
U.S. Army (McCloskey, Grandjean, & Ross, in publication).
Data Collection Procedure
A semi-structured interview protocol was used that was based on the Critical
Decision Method (CDM; Crandall, Klein, & Hoffman, 2006; Hoffman, Crandall, &
Shadbolt, 1998). All interviews were digitally recorded, and all interviews were
transcribed for analysis.
Analysis Plan
The guiding principle in our analysis was to find the elements of performance that
are key to successful planning and operations in JIIM environments. Key aspects were
not determined by a simple prevalence count, i.e., how many times certain performance
elements were mentioned, but by the emphasis placed on how the operations are
accomplished or how they fail, i.e., expert strategies. Existing documents that had
previously examined operations in terms of expertise or lessons learned helped us
determine the emphasis to place on elements of performance.
Our approach was inductive, i.e., the themes identified are linked strongly to the data
gathered specifically for this project or to reports that are based on data about
performance in JIIM environments. We developed questions to help us indirectly assess
the elements of expertise by gathering rich examples of performance and insights and
strategies that drove performance. In this approach, the themes may bear little relation to
the actual questions asked in the interviews. One assumption of the research is that as
expertise develops, people are not generally able to articulate the how and why of their
cognitive performance. Therefore, the analysis is grounded in the specifics of the data.
The data has been collected so as to expose the cognitive challenges and strategies
inherent in situations without directly asking participants why they did things the way
they did. The inductive process does not try to fit the data into a pre-conceived
framework, but creates the framework from the data. The process also provides a rich
description of the cognitive challenges and strategies as opposed to a description of the
procedures involved in planning and operations. Analysis requires interpretation of
specific data to general themes. Specifics are coded into categories first within each
individual data item (interview or document) and then summarized across the data set.
The first step in the process was to read through the data set to immerse the analysis
team in the findings. The research question informs how reading proceeds. Both analysts
made notes of interesting ideas in the data and documented those that might be converted
into coding categories. Review of the selected documents was interspersed with reviews
of the interview transcripts. The selected documents were not theoretical background, but
5
were previous findings based on interviews that had been analyzed in a thematic manner
and lessons learned generated by experienced participants. Two analysts experienced in
thematic analysis for cognitive performance and experienced in the subject area
conducted the first step. The outcome was a set of preliminary themes from each analyst
and notes to connect each potential theme to the interviews and documents. During this
step more themes were generated than are retained during the final analysis.
The second step was to generate initial themes of expert high-level cognitive skills.
The two analysts independently reviewed each other‘s themes and rationale for each and
then discussed the overlap and wording for each theme. The outcome of this step was an
agreed upon set of codes that were to serve as preliminary themes.
The third step was to convert the preliminary themes into a representation to help
the team understand the nature of each theme. A three-column table was constructed to
present an initial name for each theme, a one-paragraph definition of the theme, and a list
of the cognitive challenges associated with the theme. An example of this can be seen in
Appendix B.
The fourth step was to review the themes more exhaustively against the data set to
identify areas needing refinement. In this step, both analysts re-read the data set. A third
analyst was added to the team and also reviewed each data item (report or interview).
Each item was coded according to the themes that had been developed, and an additional
field was created for ―other‖ themes or interesting elements that emerged during the
complete review. The number of instances of a theme found in the data set, or prevalence,
does not necessarily mean a theme is more crucial. Our inclusion of other documents in
the data set allowed us to examine the nature of performance in JIIM environments in
general across a wide range of data and previous analysis in order to make judgments
about the criticality of any one aspect of performance and judge whether to retain
inclusion of a theme in the representation.
The fifth step was to refine the name of each theme and the corresponding definition and
cognitive challenges for each. Our goal was to have a name for each theme that was easy
to remember and to create definitions and explanations of cognitive challenges that were
drawn from the data, i.e., using the words of the experienced interview participants or
findings in the documentation to the extent possible to define and explain the themes.
The sixth step was to create a model of expertise in JIIM planning and operations that
reflected the high-level cognitive skills of experienced practitioners as opposed to a
procedural representation. Our goal was to reflect the flow of cognitive performance and
inter-relationship among the themes.
Results
The interview participants were grouped into military (15), Africa Nationals (3), and
Department of State personnel (4). In the interviews, we attempted to concentrate on
experiences that were based in relief, development, or reconstruction operations. Some
experiences also contained aspects of security operations. Our primary focus was on
6
understanding military performance in JIIM operations which accounted for the majority
of interview participants.
Our analysis resulted in a total of eight themes. The themes are listed below. Each
theme is presented as a cognitive act that the practitioner in the domain must perform.
9. Understand the situation within its historical, regional, and cultural context
10. Understand the other participants
11. Shift perspective
12. Establish and maintain common ground
13. Build capability to affect the situation
14. Visualize the operation
15. Support information exchange
16. Maintain flexibility
The themes are presented in a table format including the cognitive challenges for
each at Appendix C.
The construction of our representation was heavily influenced by our previous work
with Klein‘s macrocognition framework for understanding high-level cognitive
performance (Klein et al. 2003). The elements of macrocognition were used to help
describe the cognitive challenges we discovered in this domain. In the cognitive
challenges column we entered macrocognitive labels for the overarching abilities we
believed were associated with each theme or we used elements found to be important to
cross-cultural competence (e.g., sensemaking, decision making or willingness to engage)
as documented in McCloskey et al, in publication. We constructed items for each
challenge that are specific manifestations of the macrocognitive or cultural elements in
terms of skills and challenges in this specific domain (e.g., ―Understand local/regional
and organizational power structures‖ or ―See the situation from another‘s point of view to
influence and predict behavior‖).
We indicated the high-level skills in the order in which they occur and labeled their
relationships in performance. The themes representation also indicates a progression in
complexity from initial understanding of the situation to maintaining flexibility during
operations.
We created a graphic descriptive model of the themes as a high-level depiction of the
cognitive skills that support effective performance in the JIIM environment. The model
captures how the high-level skills occur in progression as planning and implementation
proceed, but also are mutually supportive and require reiteration throughout operations.
In addition, some actions are precursors of others in that they set the stage for success in
applying the subsequent skill. See Figure 1 below.
7
Figure 1. Model of High-Level Cognitive Skills Required for Joint, Interagency, Intergovernmental and Multinational
Operations
8. Maintain flexibility across interactions to understand and adapt new
approaches and to be resilient when attempts to understand others or
implement plans are not immediately successful.
Provide knowledge to
3. Shift perspective –
Understand how others
see specific aspects of the
situation and see your
organization.
4. Establish and maintain
common ground – What
goals can you work on
together that are mutually
beneficial?
1. Understanding the
situation within its
historical, regional, and
cultural context – Learn
to read these types of
situations.
6. Visualize the
operation including 2d
and 3d order
consequences
7. Support of
information exchange
– understand
communication
preferences
5. Build the capacity
to affect the situation
by using all assets
and expertise across
organizations
2. Understanding the
other participants in the
specific situation –
Who are they? How do
they do business? What
has their role been in
this situation?
Is the basis for effective
Must be implemented as you Provide knowledge to
Provide
knowledge to
8
VALIDATION OF THEMES
Method
The method for this phase of the project had two steps, one quantitative and one
qualitative. The intent was to investigate the validity of the eight themes for successfully
operating in a JIIM environment. First a survey instrument consisting of 54 items was
developed from the previous interviews with participants in JIIM environments in order
to explore the degree of consensus among a different set of SMEs on the value of the
eight themes identified. See Appendix D for a complete list of the survey items. The
items are statements describing the skills that make up each theme with the answer
choices on a 5-point Likert scale ranging from very important to mission success to very
unimportant to mission success. In addition to the survey, a focus group was conducted to
discuss the themes with a subset of individuals that have actually had to employ such
skills in JIIM environments. Feedback about the themes in their own words, as well as
examples of each theme, added richness to the data that allowed the themes to be further
explored. The survey data was analyzed in order to find out whether or not the themes are
meaningful to experts who have worked in JIIM environments and if there was consensus
about the importance of the themes to operations. The focus group transcripts were
analyzed for the denial or affirmation of the themes‘ utility as descriptions of effective
performance in JIIM.
Research Question
Our research question was ―Do these themes have utility as descriptions of effective
performance in the JIIM environment?‖ This question was addressed with the survey
described above. The focus group was intended to provide us with incidents in which the
skills described were crucial to mission success.
Participants
We recruited Civil Affairs personnel with experience in many JIIM environments.
Eighteen reservist Civil Affairs officers and NCOs were surveyed. The mean age of the
sample was 47 years and the average number of years in service was 23. The sample
consisted of three Colonels, eight Lieutenant Colonels, two Majors, three Sergeants First
Class, one Master Sergeant, and one Staff Sergeant with 72% of the sample being
officers. Of the 18 participants, six were asked to volunteer to participate in a focus group
based on the extent of their experience working in JIIM environments.
Data Collection Procedure
All participants signed the informed consent form, which were collected before the
administration of the surveys, as well as the beginning of the focus groups. A semi-
structured interview protocol was used for the focus groups which consisted of first
describing the themes followed by probing questions to determine the importance of each
9
theme and the relevance of the statements used to describe theme. Critical incidents in
which the themes were crucial to the success of missions were also elicited. The focus
groups were digitally recorded and transcribed for analysis.
Analysis Plan
The survey data was analyzed to determine the degree of consensus among the
respondents on the value of the eight themes. This analysis consisted of five steps:
1. The data was entered into SPSS, data-analysis software.
2. The data was cleaned and checked for any errors during data entry.
3. The mean answer choice and standard deviation for each question was calculated.
A table was constructed listing the frequency of each answer choice, the mean,
and the standard deviation of each question.
4. The individual questions were combined into their thematic groups. Once this was
done the themes could be reported on instead of the individual questions.
5. The mean answer choice and standard deviation for each theme was calculated.
This indicated the general consensus of the importance of each theme to mission
success within a JIIM environment. See Table 2 below.
The six participants who subsequently formed the focus group were asked to identify
incidents that illustrated one or more themes. After the focus group recording was
transcribed:
1. The themes were read through to re-familiarize the researcher.
2. The transcripts of the focus groups were read through to familiarize the
researcher with the data.
3. The transcripts were analyzed for each theme, one at a time. Incidents mentioned
were coded and catalogued under each theme they described.
Results
Survey Results
Each of the eight themes was validated with the use of the 54 items in the survey.
Each item was rated on a scale from 1 to 6 with 1 being very important and 6 being very
unimportant. Each item was rated as important on average with means ranging from 1.1
to 2.1. See Appendix F for a full listing of means and standard deviations. The rate of
importance for each theme ranged from 91-100% with Shifting Perspective being
reported as important or very important by an average of 91% of the sample and
Maintaining Flexibility being reported as important or very important by 100% of the
sample. The percentage of participants that found each theme important or very important
is listed in Table 2 below. Although the sample size does not allow for generalization of
the results, the findings provide some confidence that these themes are applicable for
JIIM training.
10
Table 1. Summary of Survey Findings for Validation of Themes
Theme Related
Item
Numbers
Total
Items
% of Participants that
Found the Theme ―Very
Important‖ or ―Important‖
for JIIM Operations
Understand the situation within its
historical, regional, and cultural
context
1-9 9 95.1%
Understand the other participants 10-16 7 96.8%
Shift perspective 17-24 8 91.0%
Establish and maintain common
ground
25-31 7 99.2%
Build capacity to affect the situation 32-38 7 95.2%
Visualize the operation 39-44 6 94.4%
Support information exchange 45-49 5 91.1%
Maintain Flexibility 50-54 5 100%
Focus Group Results
In order to further investigate the validity of the JIIM themes and improve them we
went over the phrasing of the themes and other data in the matrix, as well as gathered
example. Our analysis of the focus groups‘ transcripts confirmed that the challenges and
training needs reflected in the themes were meaningful to experienced practitioners. We
also revised the wording, but generally the wording was accepted by the focus group
members. The eight themes were accepted by all the members of the Civil Affairs sample
and the participants of the focus groups expanded on each theme by relating an example
from their experience in theater. They agreed that in order to better work in JIIM
environments in the future, training needs to be targeted towards specific issues that fall
within these different themes.
DISCUSSION
The themes we derived from interviews with experienced personnel and documents
in the area of JIIM operations are appropriate for guiding training development relating to
the high-level cognitive skills needed for effective performance in a collaborative
environment. Themes are used in situated learning designs to increase the learner‘s
ability to see a situation from multiple perspectives. Secondly, they are used to reinforce
habits of thought that are consistent with expert performance to guide deliberate practice
of these thought processes to accelerate expertise. While theme-based situated learning
has been used successfully to teach advanced concepts, we believe that the potential for
accelerating the acquisition of entry-level knowledge is equally as important. Too often
entry-level knowledge is taught in a manner that is boring and decontextualized. These
methods result in a lack of retention and a lack in the student‘s ability to generalize
11
knowledge to new settings and to higher-level learning, essentially requiring the student
to re-learn introductory information later in the education or training process. The themes
will be used to support the development of scenario-based tutorials which will support
learning for a training audience of mixed levels of experts. Building scenarios around the
themes will help learners at different levels understand the basis for successful
implementation of doctrine and processes.
12
REFERENCES
Agrait, R. & Loughran, J. J. (2007). Lessons Learned from Joint, Interagency
International and Multinational (JIIM) Workshops and Papers. Vienna, VA:
ThoughtLink, Inc.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative
Research in Psychology, 3, 77-101.
Crandall, B., Klein, G., & Hoffman, R. R. (2006). Working minds: A practitioner’s guide
to cognitive task analysis. Cambridge, MA: The MIT Press.
Hoffman, R. R., Crandall, B. W., & Shadbolt, N. R. (1998). Use of the critical decision
method to elicit expert knowledge: A case study in cognitive task analysis
methodology. Human Factors, 40(2), 254-276.
Klein, G., Ross, K. G., Moon, B., Klein, D. E., Hoffman, R. R., Hollnagel, E. (May/June
2003). Macrocognition. IEEE Intelligent Systems, 81-85.
Lussier, J. W., Shadrick, S. B., & Prevou, M. I. (2003). Think Like a Commander
prototype: Instructor’s guide to adaptive thinking. (Research Product 2003-02).
Alexandria, VA: U. S. Army Research Institute for the Behavioral and Social
Sciences.
McCloskey, M., Grandjean, A., & Ross, K. (in publication). Assessing learning and
development in Army cross-cultural competence. (Phase 1 SBIR Report.)
Alexandria, VA: U. S. Army Research Institute for the Behavioral and Social Sciences.
Miller, T. E., Phillips, J. K., Battaglia, D. A., Wiggins, S. L., Baxter, H., Mills, J. A., &
Ross, K. G. (2003). Collaborative online training for operations other than war
(OOTW) (Final Technical Report for Contract #F33165-00-C-6003 Air Force
Research Laboratory, Mesa, AZ). Fairborn, OH: Klein Associates Inc.
O'Dea, A., Ross, K.G., McHugh, A, Phillips, J.K., Throne, M.H., & McCloskey, M. &
Mills, J.A (2006). Global teams: Enhancing the performance of multinational staffs
through collaborative online training (Research Report 1849, AD A469425).
.Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social
Sciences
Ross, K. G. (2008, May). Toward an operational definition of cross-cultural competence
from interview data. Patrick AFB, FL: Defense Equal Opportunity Management
Institute (DEOMI). (Available from the Cognitive Performance Group
13
Ross, K. G., & Lussier J. W. (1999). A Training Solution for Adaptive Battlefield
Performance. Proceedings of the Interservice/Industry Training, Simulation &
Education Conference (I/ITSEC), 1999. Orlando FL.
Shadrick, S. B., Schaefer, P. S., & Beaubien, J. (2007). Development and Content
Validation of Crisis Response Training Package Red Cape: Crisis Action Planning
and Execution (Research Report 1875). Alexandria, VA: U. S. Army Research
Institute for the Behavioral and Social Sciences.
Spiro, R. J., Feltovich, P. J., Jacobson, M. J., & Coulson, R. L. (1992). Cognitive
Flexibility, constructivism, and hypertext: Random access instruction for advanced
knowledge acquisition in ill-structured domains. In T. Duffy, & D. Jonassen (Eds.).
Constructivism and the Technology of Instruction: A Conversation (pp.57-75).
Hillsdale, NJ: Lawrence Erlbaum Associates.
Spiro, R.J., Feltovich, P.J., Coulson, R.L., Jacobson, M.J., Durgunoglu, A., Ravlne, S., &
Jehng, J. (1992). Knowledge acquisition for application: Cognitive flexibility and
transfer of training in ill-structured domains (ARI Research Note 92-21).
.Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social
Sciences.
A-2
Experience Base Organization Point of View
1 Iraq, Afghanistan Dutch
Marines
NATO
Multinat‘l Ops; Mentoring/Training of
Iraqi Ministry of Interior/Defense
Leadership; Gov‘t infrastructure building,
2 Mozambique US Army Working in Mozambique with the local
NGO and host nation to build HIV/AIDS
laboratories. Embassy Affairs; Security
Cooperation; NGO/IGO Coordination;
Reconstruction Project Mgmt.
3 Afghanistan US Army Studied police reform in Afghanistan
through interagency interaction.
4 Iraq US Army Worked as Chief of Transportation in Iraq.
5 Afghanistan US Army Worked with the UN and the host nation
to stabilize the area.
6 Iraq, Afghanistan US Marines
7 Ethiopia, Kenya US Army CAO working with the host nations in
Ethiopia and Kenya to provide
humanitarian assistance.
8 Kenya US Army Working in Kenya with the embassy and
host nation to fix water sanitation.
9 Djibouti, Ethiopia,
Yemen,
Philippines
US Army CAO worked with local tribes to provide
humanitarian assistance.
10 Afghanistan,
Niger, Kenya, Iraq
US Army COA working in Afghanistan with host
nation to provide humanitarian assistance.
11 Iraq, Chad US Army CAO working in Chad and Iraq with DOD
and the State Department.
12 Iraq, Afghanistan US Army CAO working in Iraq and Afghanistan
with NGOs.
13 Iraq, Kenya,
Afganistán,
Senegal, Cameron
US Army COA working in Afghanistan with host
nation to provide humanitarian assistance.
14 Afghanistan, HOA US Army In Afghanistan, he worked with a local
NGO to provide a medical clinic. In the
HOA, he worked with the State Dept. and
the Dept. of Defense.
15 Afghanistan, Iraq US Army Worked with a local NGO in Afghanistan
to provide for Pakistan refugee camps and
educate villagers on irrigation and
agriculture. Worked with another NGO in
Iraq to provide aid in Baghdad.
16 Honduras US Army
Corps of Eng.
Reconstruction and Stabilization Project
Mgmt.
A-3
17 Afghanistan Dutch
Marines, NL
MOD
Multinat‘l Strategic Ops and Intel; Int‘l
Political Advisement
18 2 African Nations,
Central and S.
America
State Dept. State Dept representative that met to
develop COA with military personnel.
19 Haiti, Iraq UN , State
Dept.
Representative
Worked with the UN on environmental
missions in Haiti and Iraq.
20 Central Asia State Dept.
Representative
Worked with the host nations on
environmental missions in Central Asia.
21 Iraq State Dept.
Representative
State Dept. Representative on a
multinational committee to exchange
information in the Green Zone of Iraq.
22 Lithuania US Embassy Worked with the host nation and the State
Department.
23 Rwanda International
Non-
Government
Organization
Worked in a political-military
organization with local NGOs to obtain
humanitarian assistance in Rwanda during
the genocide of 1994.
24 Ghana African
Health
Research
Center
African national working with NGOs and
US Army in health research laboratory.
25 Ghana International
Non-
Government
Organization
African national working with US Army
in health research laboratory.
Other Materials
1 ―Lessons Learned from Joint, International, and Multinational (JIIM) Workshops and
Papers‖– ThoughtLink, Inc. (Produced as part of this project)
2 Transcript of the Healthy Africa Scenarios Exercise Workshops January 21-25, 2008
3 JIMPC General Overview Briefing – Joint Forces Staff College
4 ―Global Teams‖ Project Report (interview-based themes)
5 ―Operations Other Than War (OOTW)‖ Project Report (interview-based themes)
6 Cross-Cultural Competence Operational Definition Report (interview-based competence
modeling)
7 Cross-Cultural Competence Assessment Report (interview-based competence modeling)
Interview with Military Personnel
B-2
Interview with Military Personnel
Experience in Iraq
Theme Title
1. Understand the
situation within its
historical, regional,
and cultural
context
2. Understand the
other participants When in Iraq… ―The first reaction you have as a civilian walking
into that is how do you find your way around? In my case, having not
been a diplomat and not having come out of the State Dept., not having
NOT been a part of an embassy country team before, and I had a steep
learning curve just on the diplomatic side, on the State Dept., country
team side to figure out how all those processes and people worked and
interacted, etc. On top of that, though, I had to look at MNFI, and MNCI,
and MNDB, and all of those other HQ that are layered there and figure
out how to effectively integrate my staff with that group of people.‖
(analyst note: this is about understanding the other players, your own
role, and how you fit into the mix)
―Some of these guys have very, very distinct impressions on whether
the civilians are of any use at all in a battle space. And so culture is a big
deal. We need to get these guys to seek out the civilians, find and know
what their capabilities are and exploit those capabilities in the battle space
so they don‘t have to do those things that the civilians can do. And then
collaborate…..So anything that we can do that gives the military guys a
sense of a positive contribution civilians can do that helps them
understand that they‘re a different culture, that it helps them understand
that we have different communications processes and capabilities, and
that our op tempos are different.‖
(analyst note: This overlaps with ―building capability: theme – e.g.,
identifying and leveraging expertise, using all available assets)
3. Shift Perspective
4. Establish and
maintain common
ground
―I found frequently, that the civilian side of the equation was
competing with the military side. There was lack of common objective
and understanding of the long-term desired end state. A specific example,
front page of the Washington Post here 6, 8, 10 months ago, DoD‘s
Current State, Undersecretary Brinkley is in Iraq, came in and briefed us
and said we want a stand-up, state-owned enterprise—state-owned
enterprise that‘s a failed legacy of Saddam‘s socialist sort of approach to
economics—lots of employees, very little output, very poor quality, and
the embassy and the State dept. had established as an end state, stamp out
Interview with Military Personnel
B-3
Theme Title
all state-owned enterprises, privatize and make it a competitive economy
that can merge into the international arena. So you have specific
assignment of civil affairs officers, under Gen. Carelli, working under
Brinkley with millions of dollars of funding, to stamp out state-owned
enterprise, ostensibly you get people off the streets, get them employed
because that reduces the number of potential candidates for insurgents.
And the other side you have a bunch of civilians beating themselves to
death, trying to stamp out state-owned enterprise and to create privatized
business to build the economy.‖
(analyst note: This is an example of lack of common ground between
civilian and military players)
Plane crash incident: ―…it was a civilian airplane, with six people on
board, German registered, one Iraqi, five Germans, the airplane
disappeared, it was overdue for fuel, couldn‘t find it, and the question
then is ‗well who is responsible for search and rescue?‘ And MNFI and
MNCI maintained that they were not. There was no good answer for then
who was…..there was no acceptance and no knowledge of who was
responsible for search and rescue in that country where we owned the
airspace.
My initial reaction was one of disbelief and frustration that MNFI, which
owned the airspace, and which had air assets available didn‘t
acknowledge any role or responsibility….Then sort of aghast that there
wasn‘t some contingency plan that would have prescribed the process and
steps that a person would take. So we created it on the fly….and it took
so long…And the process was not visible to me. So part of the challenge
is that the military, the MNFI processes, are not really visible to a large
population of the civilian leadership and planners who have to depend on
it. It was not collaborative; it was two separate stove pipes coming in and
trying to work together.
…finally the FRAGO got disseminated out to MNCI, which then resulted
in aircraft availability. Which in the end we didn‘t use because __ in Iraq
had already taken off and gone. We lost face with the government of Iraq.
It set us back in terms of our ability to work with them because they felt
we intentionally didn‘t respond, that we intentionally didn‘t give them the
information, that we wanted them to look bad, etc…‖
(analyst note: This is an example about lack of common ground, as far as
who is responsible for what. This overlaps with ―knowing the other
participants‖ and their roles and responsibilities. Following the crash,
there was no shared knowledge of who is responsible for search and
Interview with Military Personnel
B-4
Theme Title
rescue. No understanding of who‘s in charge and who owns that
role/responsibility).
―If I were to walk in that office again…I‘d ask what‘s our plan?
What are our capabilities? Whose responsibility is it? What are the assets
that are dedicated to that response? And what is the communication
network that is employed to make that work smoothly?‖
(analyst note: This is about building capability, common ground,
knowing participants and roles/responsibilities)
5. Build capability
to affect the
situation
―One of the things I fought for, was to get them to identify
transportation as a critical, or essential… I think they called it an
Essential Services—oil, water, electricity, they were all designated
essential and transportation was not. Well transportation infrastructure is
critical and you can‘t distribute oil in Iraq without transportation, you
can‘t create autonomy without transportation, you can‘t move guns…all
the other things for war, much less a civil economy without highways and
rail. And so it should have been that determination should have been
made early that transportation was critical. And then how would you
leave out the senior person in transportation from the campaign planning?
(analyst note: this is about making your own role and expertise clear to
others. Convincing other stakeholders of the importance of your input)
6. Visualize the
operation
―…there‘s no continuity from incumbent to incumbent, there are no
archives, and no records in the civilian side, and which weren‘t while I
was there. I inherited no records, I had no overlap from my predecessor,
very, very minimal plan existence so there‘s no continuity.‖
(analyst note: This is about a lack of continuity. No visioning of how
what you do (or don‘t do) as far as record-keeping and continuity impacts
the future of the operation)
7. Support
information
exchange
Re Gen. Casey‘s Campaign Plan…‖I hadn‘t seen it…. you would
think that if transportation infrastructure is critical to our success in Iraq,
you would have thought that there would have been an overt attempt to
get Transportation and Secretary of Transportation‘s fingerprints on that
campaign plan. There had been no effort that I‘m aware of to do any of
that. And so I got that plan and I sat down and I wrote a significant
number of comments specifically related to this footprint issue. And I
tried to run that up channel, they never got their copy of the plan back
either, when I left it was probably still in the safe. But, because I had not
seen it, and had not been given the opportunity to coordinate on it, to
provide input, and I am suspicious that very few people on the civilian
side had a chance to act on it.…This is a clear case of two stovepipes
Interview with Military Personnel
B-5
Theme Title
competing instead of collaborating. I didn‘t see the campaign plan until
very late. My ability to provide input to it was greatly constrained, and
I‘m not even sure today, how many of my comments got to anybody who
would‘ve read them and been able to do anything with them…But the
bigger issue is how do you coordinate serious documents, serious plans
across that spectrum of all those people, all the relative stakeholders.‖
(analyst note: This is an example of poor info exchange. He had no
involvement in Casey‘s campaign plan, no input.)
―We can‘t collaborate if everything is done on SIPR because the
civilians don‘t have access. So we need to look at ways to do… I mean,
first of all, how do you share information? And what are the tools for
sharing that information? And what are the impediments for sharing that
information? And I said SIPR is one of them. Over classification and then
not having a common IT or… the other side of that is that you have the
military guys who are very far advanced, IWS and all those other tools.
They don‘t interface with the civilians on that.‖
(analyst note: this is about recognizing impediments to information
exchange with certain players)
C-2
TITLE
DEFINITION
COGNITIVE CHALLENGES
1. Understand the
situation within
its historical,
regional, and
cultural context
Do your homework. Understand the
context and history of the situation
and the participants. Identify
drivers/causes of the current
situation. Recognize that chaos is
typical of relief situations and some
development situations. A clear
framework for seeing the situation
provides a stable basis for
assessment and decision making.
Learn how to read situations from
others who can see more in the
situation than you can.
Declarative knowledge,
development of a general mental
model of relief/development
situations, sensemaking
Understand and
differentiate long-term and
short-term issues (for
example, know and use the
Capital Analysis and
Performance Strategy
(CAPS) framework).
Know and apply the
Tactical Conflict
Assessment Framework
(TCAF) when conflict is a
part of the situation.
Understand the timeline up
to the current situation.
Identify and understand
critical events on the
timeline.
Identify and understand
regional and cultural factors
that have influenced the
current situation.
―Read‖ novel situations to
focus on priority indicators
(leading up to and during
operations).
2. Understand the
other participants
Know the types of organizations and
entities that can be involved and
identify those who are involved in
the specific situation. Be aware of
their historical roles and
relationships, as well as their
priorities and their willingness to
collaborate. Also recognize there
may be differences in how they do
business. There can be differences in
work pace, work hours; integration
into the community; maintenance of
clear cut roles and responsibilities
versus diffuse roles; communication
styles; metrics for progress; comfort
Declarative knowledge,
development and application of a
mental model of relief/development
operations, cultural competence,
willingness to engage others to
understand their organization
Consider both
organizational culture and
culture of host nation to
understand how others
conduct business.
Understand local/regional
and organizational power
structures.
Anticipate impacts of
C-3
TITLE
DEFINITION
COGNITIVE CHALLENGES
with and availability of technology;
authority structures; and whether
they concentrate on tasks or
relationships to do business.
Anticipate how these differences can
affect goal focus, information
sharing, planning, and decision
making in order to work more
effectively together. Understand
local power structures to anticipate
how decisions will be made,
approved and supported in the host
nation/local region.
cultural and organizational
values and differences on
decision making and action.
Understand the language
used in other organizations
to describe situations and
priorities.
Understand Interagency
Management System
designed for post
conflict/stabilization
situations.
3. Shift
Perspective
Shift your perspective to see the
situation from another person's point
of view. Shift Perspective to
understand, predict, and influence
behavior and foster communication.
Be aware of how you and your
organization appear to other
organizations and the host
nation/region. Be able to explain
your organization's position.
Problems are multi-dimensional and
aspects of the situation that are
salient to you may be background to
others and vice versa. Be aware of
the tendency to interpret others‘
decisions and actions based on your
own experience base, beliefs,
assumptions, and value system.
When seeking to understand others‘
rationale for decision-making,
consider their intent, their priorities,
patterns of living, and long-term
goals.
Cultural competence, prediction,
persuasion, negotiation,
collaboration, sensemaking, rapport
building, relationship building
See yourself as others see you.
Present yourself and your
organization in a way that is
meaningful to others.
See the situation from
another‘s point of view to
influence and predict
behavior.
Anticipate how your own
actions and decisions will
be interpreted by host
nation in order to mitigate
negative fallout.
4. Establish and
maintain common
ground
Establish common ground as a basis
for common goals and unity of
effort. There are usually many
diverse organizations and countries
with which we interact in non-
combat operations, each with their
own goals and cultures. Building in
the time necessary to find common
Collaboration, rapport building,
relationship building, problem
identification, solution
identification
Be willing to engage others
and build relationships.
Maintain relationships over the
course of the operation.
C-4
TITLE
DEFINITION
COGNITIVE CHALLENGES
ground, and actually discovering it
becomes a significant challenge.
Establishing common ground
requires a willingness to engage and
build relationships with people who
may be very different from you.
Building and maintaining these
relationships over the course of the
operation takes time. Self-regulation
to maintain control and openness
when dealing with others takes
practice. The resulting relationships
create the basis for establishing
common goals and coordinated
actions.
Engage in self-regulation.
Discover common interests in
the situation.
Negotiate goals and solutions.
5. Build
capability to
affect the
situation
Capability is gained by knowing and
using all assets to address common
goals. It is difficult to anticipate who
will be involved in a situation and
what assets will be available.
Different functional cells of the
military organization may have to
self-organize in response to the
situation in the early stages of
operation to leverage assets and
opportunities. The diverse set of
resources and expertise that may
exist in your own organization and
other participant organizations is
often not explicit. They can go
untapped if not deliberately sought
out. Explore who has expertise
around different issues or problems.
Recognize the military can bring
technical skill and disciplined
decision making to the situation and
how that can best be leveraged
without "shutting out" others who
have ownership in the situation and
specific expertise the military does
not. Understand own boundaries for
action and manage expectations.
Contribute to the creation of a
solution process to which multiple
players can contribute.
Declarative knowledge,
envisioning asset employment,
recognizing and leveraging options
Identify available expertise.
Know and use all assets;
understand how resources
are accessed and deployed.
Match assets to the ―right‖
problems, not just those
most salient to your
organization.
Develop workarounds where
resource constraints or
barriers to access exist.
Manage expectations.
C-5
TITLE
DEFINITION
COGNITIVE CHALLENGES
6. Visualize the
operation
Anticipate the need to transition to
the next phase including the
resources needed for different
players. Realize your phase of
operations may be different than
those of others. A well-developed
exit strategy up front is crucial. It
guides both large-scale and smaller,
incremental decision making.
Visualize the transition from relief to
development. Recognize the risks to
different players in the situation in
relation to a decision or action.
Visualize risks and mitigating
actions in a dynamic manner. Some
issues develop rapidly and some over
time. Crisis situations are likely to
contain both kinds of problems.
Some solutions can be affected
rapidly and some can only be
accomplished over the long-term.
Know the difference and don't let
short-term solutions overcome long-
term possibilities by ignoring second
and third order consequences.
Mental simulation,
synchronization, identification of
leverage points, prediction,
contingency thinking, decision
making, cultural competence
Understand how the dynamic
unfolding of the operations
may interact with the
region‘s historical and local
interests.
Anticipate resource needs for
the long-term; recognize
conflicts between short-
term and long-term issues.
Recognize different phases of
operation of different
players.
Develop a transition strategy
early to guide decisions and
actions.
Anticipate 2nd
and 3rd
order
effects of decisions and
actions to create lasting
solutions.
7. Support
information
exchange
Recognize different organizations‘
information needs, priorities, and
sense of urgency for information-
sharing, and be prepared to articulate
your own. Understand that the
methods and channels of
communication differ across
organizations such as method of
presentation; who shares
information; how authority for
information sharing is managed; how
complete information must be to
support decisions; central access to
information versus local access for
all involved. Recognize different
communication styles--flexible
versus standardized, embedded in
relationships versus "all business"
focus. Recognize adherence to a
Uncertainty management,
collaboration, coordination,
negotiation, persuasion, perspective
taking
Identify and respond to
different informational
needs of your collaborators.
Recognize that different
agencies may have different
priorities, and as a result,
different senses of urgency
for information.
Share your own need for
information and rationale
with collaborative partners.
Respect the different
communication channels
and methods for sharing
information.
C-6
TITLE
DEFINITION
COGNITIVE CHALLENGES
formal chain of command; degree of
autonomy allowed to individuals;
likelihood of junior individuals to
speak out. Opportunities and limits
to information exchange can vary.
Create access to information sharing
in terms of where and how you
conduct business. Understand own
planning process and coordination
requirements for the situation.
8. Maintain
flexibility
Recognize when your approach or
stance in a situation isn‘t working,
and be willing to adapt it. Recognize
and apply behavior that is most
likely to be successful in each
situation to successfully
communicate with others. Be aware
that there are viable approaches and
solutions to problems that might be
outside the realm of the way you‘ve
approached them in the past. Be
willing to adapt your tried and true
approaches. Recognize that the first
approach or attempt may not be
successful, and that is not the same
as failure.
Problem detection, resilience,
adaptation, re-planning,
observation, flexible problem
solving, situation assessment,
decision making
Be reflective and aware of
your own and your
organization‘s biases.
Refrain from force-fitting a
certain approach.
Put more ―mental energy‖ into
understanding the situation
than in choosing a solution.
Be aware of when it‘s time to
shift your approach and try
something different.
Recognize when your style is
not effectively
communicating and have
other methods to substitute.
D-2
Survey of Cognitive Training Themes for Joint, Interagency, Intergovernmental and
Multinational (JIIM) Operating Environments:
The purpose of the questionnaire is to learn what you believe are important skills and
abilities that Service Members need in order to adapt to cognitive challenges in the context of
Joint, Interagency, Intergovernmental, and Multinational Operations. Your responses will only
be used within the JTF-ITS project team.
This is a two-part questionnaire:
Part 1 Personal Information (4 items)
Part 2 Rating the Performance Abilities (55 items)
The entire survey will take approximately 60 minutes to complete. Please rate each item
according to the instructions provided for that section. If you do not understand these instructions
or the items to be rated, the facilitator will provide clarification.
Thank you for your participation. Your input is valuable for understanding how to
manage the cognitive challenges that can be found in Joint, Interagency, Intergovernmental and
Multi-national (JIIM) operating environments.
Joint Task Force, Intelligent Tutoring System (JTF-ITS), SBIR Ph. 2
Dr. Brooke Schaab, Research Sponsor, (Army Research Institute) 757-203-3306
Mr. Jim Ong, Principal Investigator, (Stottler Henke Assoc., Inc) 650-931-2700
Dr. Karol Ross (Cognitive Performance Group) 407-282-4433
D-3
Part 1 Personal Information ______ __________
Please tell us about your background and experiences by completing this section. Your responses
on this questionnaire will remain confidential, and all questionnaire results will be reported
without attribution to any of the respondents.
1. Identification: Respondent‘s ID Number _______
Age: ______ Gender: M F Rank: ___________ MOS/Branch ________
Years of Service: _______ Service: (Circle one) Army, Air Force, Navy, USMC
Current Duty Position: ___________________________________________________
Date and Location of Last Deployment: ______________________________________
Please circle one answer for the following questions.
a. Have you worked with Government Agencies or Departments as part of your civilian
occupation? YES NO Don‘t Know
b. Have you ever been assigned to a Joint Task Force staff? YES NO Don‘t Know
c. Have your military duties required you to work within a JIIM environment?
YES NO Don‘t Know
d. Have your military duties required you to work with an Allied or Host Nation military
unit? YES NO Don‘t Know
2. Most Recent Deployments: (in a Joint, Interagency, Intergovernmental, Multinational
Operation)
Location
(Country or Region)
Duty Position Date (Month/Year)
From… To…
3. Cultural Training and Education:
D-4
Please list any training or education that prepared you for a Joint, Interagency,
Intergovernmental or Multinational command or staff assignment:
Type of Training or Education Location Date (Month/Year)
4. Other Information:
Please provide other information regarding your background, operational experience, or
education/training that you believe to be relevant working with another culture.
D-5
Part 2 Rating Performance Requirements
Based on your experience working within a Joint Interagency, Intergovernmental, Multinational (JIIM) environment, rate the
importance of the following abilities.
Abilities
Very
Important
1
Important
2
Not
Sure
3
Unimportant
4
Very
Unimportant
5
1.
Doing homework to understand the historical context of
the situation and the participants before arrive
2. Identifying the drivers or causes of the current situation.
3. Having a clear framework to assess the situation.
4.
Having an assessing framework when the situation is
chaotic
5.
Learning to read a situation from others who understand
the situation better than I do
6.
Differentiating long-term and short-team issues when
assessing a situation
7. Establishing a timeline leading to the current situation.
8. Identifying and focusing on key indicators in a situation
9. Understanding of regional factors
10.
Knowing what types of organizations and entities could
be involved in the situation
11. Understanding the role of each agency before I arrive to a
situation
12. Understanding the willingness of different organizations
to collaborate
13.
Understanding how the agencies involved conduct
business, such as, work hours, formality of organizational
structure, pace of work, goals, and communication styles
14.
Understanding local power structures to understand how
decisions will be made, approved and supported
D-6
Based on your experience working within a Joint Interagency, Intergovernmental, Multinational (JIIM) environment, rate the
importance of the following abilities.
Abilities
Very
Important
1
Important
2
Not
Sure
3
Unimportant
4
Very
Unimportant
5
15. I need to understand the effect of culture on goals and
priorities.
16.
It is important to understand the Interagency Management
System, to conduct a successful mission.
17. Being able to understand the situation from another
person‘s point of view
18. Taking another person‘s perspective, to better predict and
influence their behavior
19. Taking another person‘s perspective so that I can
communicate with them better
20. Being aware of how the US Military appears to other
players in the situation
21. Presenting the role of the US Military in a way that is
meaningful to others
22.
Understanding that my experiences, values, and beliefs
are not the only way to interpret the situation
23. Seeing myself as others see me.
24.
Anticipating how my actions and the actions of the US
Military will be perceived to avoid negative consequences
25.
Unifying efforts by establishing common goals
26. Taking the time to find common ground with other
players in the situation
27. Building and maintain relationships
28. Maintaining emotional self control
D-7
Based on your experience working within a Joint Interagency, Intergovernmental, Multinational (JIIM) environment, rate the
importance of the following abilities.
Abilities
Very
Important
1
Important
2
Not
Sure
3
Unimportant
4
Very
Unimportant
5
29. Maintaining openness to others
30. Being willing to engage with others
31. Negotiating goals and solutions
32.
Increasing our capacity by sharing resources across
organizations.
33. Understanding and using all assets in the situation
34. Identifying who has expertise about different problems
across all players
35. Various players having ownership and contribute assets
36. Understanding the boundaries of US Military actions
37. Managing expectations about US Military actions
38. Recognizing that your phase of operation may be different
from the phase of operations of other players
39. Anticipating the resources needed for all the different
players to transition to the next phase of operations
40. A well-developed exist strategy
41.
Visualizing the transition from addressing short-term
emergency issues to long-term stability issues
42. Understanding when long-term and short-term solutions
conflict
D-8
Based on your experience working within a Joint Interagency, Intergovernmental, Multinational (JIIM) environment, rate the
importance of the following abilities.
Abilities
Very
Important
1
Important
2
Not
Sure
3
Unimportant
4
Very
Unimportant
5
43.
Creating or at least not interfering with long-term
solutions when responding to a crisis
44.
Visualizing an operation to avoid potential negative
consequences
45.
Recognizing that different organizations have different
needs, priorities, and urgencies for sharing information
46.
Understanding that different organizations have different
methods and lines of authority for sharing information
47.
Creating access for information sharing by deciding where
and how to conduct business
48.
Understanding the military‘s planning and process when
sharing information with other organizations
49.
Sharing information about the military planning and
coordination process with other players when trying to
share information with other organizations
50. Recognizing when my approach isn‘t working and
adapting it
51. Understanding the methods I have tried before may not
work a new situation
52.
Being resilient and trying new approaches when the first
attempt does not work
53. Understanding a situation in order to create flexible
solutions
54. Having more than one style of communicating
55. Understanding doctrine 3-07
D-9
Based on your training and experience, what other performance factors are important in a JIIM environment?
Thank you for your responses. This completes the JTF-ITS survey.
Please turn in this your responses to the Facilitator.
E- 2
Personal Information ______ __________
Please tell us about your background and experiences by completing this section. Your responses
on this questionnaire will remain confidential, and all questionnaire results will be reported
without attribution to any of the respondents.
5. Identification: Respondent‘s ID Number _______
Age: ______ Gender: M F Rank: ___________ MOS/Branch ________
Years of Service: _______ Service: (Circle one) Army, Air Force, Navy, USMC
Current Duty Position: ___________________________________________________
Date and Location of Last Deployment: ______________________________________
Please circle one answer for the following questions.
e. Have you worked with Government Agencies or Departments as part of your civilian
occupation? YES NO Don‘t Know
f. Have you ever been assigned to a Joint Task Force staff? YES NO Don‘t Know
g. Have your military duties required you to work within a JIIM environment?
YES NO Don‘t Know
h. Have your military duties required you to work with an Allied or Host Nation military
unit? YES NO Don‘t Know
6. Most Recent Deployments: (in a Joint, Interagency, Intergovernmental, Multinational
Operation)
Location
(Country or Region)
Duty Position Date (Month/Year)
From… To…
E- 3
7. Cultural Training and Education:
Please list any training or education that prepared you for a Joint, Interagency,
Intergovernmental or Multinational command or staff assignment:
Type of Training or Education Location Date (Month/Year)
8. Other Information:
Please provide other information regarding your background, operational experience, or
education/training that you believe to be relevant working with another culture.
The matrix of themes was attached for discussion to gather examples of each theme.
F-2
Item
1
Very
Important
2
Important
3
Not Sure
4
Unimportant
5
Very
Unimportant
Mean
SD
1. Doing my homework
to understand the
historical context of
the situation and the
participants before
arriving
88.9%
(16)
11.1%
(2) 0% 0% 0% 1.1 .3
2. Identifying the
drivers or causes of
the current situation
77.8%
(14)
16.7%
(3) 0% 0% 0% 1.2 .4
3. Having a clear
framework to assess
the situation
61.1%
(11)
33.3%
(6)
5.6%
(1) 0% 0% 1.4 .6
4. Having an assessing
framework when the
situation is chaotic
50%
(9)
38.9%
(7)
11.1%
(2) 0% 0% 1.6 .7
5. Learning to read a
situation from others
who understand the
situation better than I
do
50%
(9)
50%
(9) 0% 0% 0% 1.5 .5
6. Differentiating long-
term and short-team
issues when assessing
a situation
50%
(9)
50%
(9) 0% 0% 0% 1.5 .5
7. Establishing a
timeline leading to
the current situation
22.2%
(4)
66.7%
(12)
5.6%
(1)
5.6%
(1) 0% 1.9 .7
8. Identifying and
focusing on key
indicators in a
situation
50% (9) 38.9%
(7) 11.1 (2) 0% 0% 1.6 .7
9. Understanding of
regional factors
72.2%
(13)
27.8%
(5) 0% 0% 0% 1.3 .5
10. Knowing what types
of organizations and
entities could be
involved in the
situation
83.3%
(15)
16.7%
(3) 0% 0% 0% 1.2 .4
11. Understanding the
role of each agency
before I arrive to a
55.6 (10) 44.4%
(8) 0% 0% 0% 1.4 .5
F-3
Item
1
Very
Important
2
Important
3
Not Sure
4
Unimportant
5
Very
Unimportant
Mean
SD
situation
12. Understanding the
willingness of
different
organizations to
collaborate
38.9%
(7)
61.1%
(11) 0% 0% 0% 1.6 .5
13. Understanding how
the agencies involved
conduct business,
such as, work hours,
formality of
organizational
structure, pace of
work, goals, and
communication styles
38.9%
(7) 55.6 (10)
5.6%
(1) 0% 0% 1.7 .6
14. Understanding local
power structures to
understand how
decisions will be
made, approved and
supported
66.7%
(12)
33.3%
(6) 0% 0% 0% 1.3 .5
15. Understanding the
effect of culture on
goals and priorities
66.7%
(12)
33.3%
(6) 0% 0% 0% 1.3 .5
16. Understanding the
Interagency
Management System,
to conduct a
successful mission
22.2%
(4)
61.1%
(11)
11.1%
(2)
5.6%
(1) 0% 2 .8
17. Being able to
understand the
situation from
another person‘s
point of view
33.3%
(6)
61.1%
(11)
5.6%
(1) 0% 0% 1.7 .6
18. Taking another
person‘s perspective,
to better predict and
influence their
behavior
33.3%
(6)
50%
(9)
16.7%
(3) 0% 0% 1.8 .7
19. Taking another
person‘s perspective
so that I can
33.3%
(6)
55.6%
(10)
5.6%
(1)
5.6%
(1) 0% 1.8 .8
F-4
Item
1
Very
Important
2
Important
3
Not Sure
4
Unimportant
5
Very
Unimportant
Mean
SD
communicate with
them better
20. Being aware of how
the US Military
appears to other
players in the
situation
55.6%
(10)
38.9%
(7) O%
5.6%
(1) 0% 1.6 .8
21. Presenting the role of
the US Military in a
way that is
meaningful to others
72.2%
(13)
22.2%
(4)
5.6%
(1) 0% 0% 1.3 .6
22. Understanding that
my experiences,
values, and beliefs
are not the only way
to interpret the
situation
66.7%
(12)
33.3%
(6) 0% 0% 0% 1.3 .5
23. Seeing myself as
others see me
44.4%
(8)
33.3%
(6)
11.1%
(2)
11.1%
(2) 0% 1.9 1
24. Anticipating how my
actions and the
actions of the US
Military will be
perceived to avoid
negative
consequences
72.2%
(13)
22.2%
(4) 0%
5.6%
(1) 0% 1.4 .8
25. Unifying efforts by
establishing common
goals
44.4%
(8)
55.6%
(10) 0% 0% 0% 1.6 .5
26. Taking the time to
find common ground
with other players in
the situation
50%
(9)
50%
(9) 0% 0% 0% 1.5 .5
27. Building and
maintaining
relationships
77.8%
(14)
22.2%
(4) 0% 0% 0% 1.2 .4
28. Maintaining
emotional self control
61.1%
(11)
38.9%
(7) 0% 0% 0% 1.4 .5
29. Maintaining openness 38.9% 55.6% 5.6% 0% 0% 1.7 .6
F-5
Item
1
Very
Important
2
Important
3
Not Sure
4
Unimportant
5
Very
Unimportant
Mean
SD
to others (7) (10) (1)
30. Being willing to
engage with others
61.1%
(11)
38.9%
(7) 0% 0% 0% 1.4 .5
31. Negotiating goals and
solutions
61.1%
(11)
38.9%
(7) 0% 0% 0% 1.4 .5
32. Increasing our
capacity by sharing
resources across
organizations
16.7%
(3)
77.8%
(14)
5.6%
(1) 0% 0% 1.9 .5
33. Understanding and
using all assets in the
situation
55.6%
(10)
44.4%
(8) 0% 0% 0% 1.4 .5
34. Identifying who has
expertise about
different problems
across all players
61.1%
(11)
38.9%
(7) 0% 0% 0% 1.4 .5
35. Various players
having ownership
and contributing
assets
50%
(9)
38.9%
(7)
11.1%
(2) 0% 0% 1.6 .7
36. Understanding the
boundaries of the US
Military‘s actions
66.7%
(12)
27.8%
(5)
5.6%
(1) 0% 0% 1.4 .6
37. Managing
expectations about
the US Military‘s
actions
55.6%
(10)
33.3%
(6)
5.6%
(1) 5.6% (1) 0% 1.6 .8
38. Recognizing that
your phase of
operation may be
different from the
phase of operations
of other players
38.9%
(7)
61.1%
(11) 0% 0% 0% 1.6 .5
39. Anticipating the
resources needed for
all the different
players to transition
to the next phase of
operation
27.8%
(5) 66.7 (12)
5.6%
(1) 0% 0% 1.8 .5
40. A well developed
exist-strategy
66.7 (12) 27.8%
(5)
5.6%
(1) 0% 0% 1.4 .6
F-6
Item
1
Very
Important
2
Important
3
Not Sure
4
Unimportant
5
Very
Unimportant
Mean
SD
41. Visualizing the
transition from
addressing short-term
emergency issues to
long-term stability
issues
22.2%
(4)
72.2%
(13) 0% 0% 0% 1.8 .4
42. Understanding when
long-term and short-
term solutions
conflict
22.2%
(4)
61.1%
(11) 11.1 (2)
5.6%
(1) 0% 2 .8
43. Creating or at least
not interfering with
long-term solutions
when responding to a
crisis
27.8%
(5)
72.2%
(13) 0% 0% 0% 1.7 .4
44. Visualizing an
operation to avoid
potential negative
consequences
44.4%
(8)
55.6%
(10) 0% 0% 0% 1.6 .5
45. Recognizing that
different
organizations have
different needs,
priorities, and
urgencies for sharing
information
33.3%
(6)
66.7%
(12) 0% 0% 0% 1.7 .5
46. Understanding that
different
organizations have
different methods and
lines of authority for
sharing information
22.2%
(4)
77.8%
(14) 0% 0% 0% 1.8 .4
47. Creating access for
information sharing
by deciding where
and how to conduct
business
33.3%
(6)
50%
(9)
16.7%
(3) 0% 0% 1.8 .7
48. Understanding the
military‘s planning
and process when
sharing information
with other
organizations
44.4%
(8)
55.6%
(10) 0% 0% 0% 1.6 .5
F-7
Item
1
Very
Important
2
Important
3
Not Sure
4
Unimportant
5
Very
Unimportant
Mean
SD
49. Sharing information
about the military
planning and
coordination process
with other players
when trying to share
information
27.8%
(5)
44.4%
(8)
16.7%
(3)
11.1%
(2) 0% 2.1 1
50. Recognizing when
my approach isn‘t
working and adapting
it
66.7%
(12)
33.3%
(6) 0% 0% 0% 1.3 .5
51. Understanding the
methods I have tried
before may not work
in a new situation
44.4%
(8)
55.6%
(10) 0% 0% 0% 1.6 .5
52. Being resilient and
trying new
approaches when the
first attempt does not
work
61.1%
(11)
38.9%
(7) 0% 0% 0% 1.4 .5
53. Understanding a
situation in order to
create flexible
solutions
55.6%
(10)
44.4%
(8) 0% 0% 0% 1.4 .5
54. Having more than
one style of
communicating
55.6%
(10)
44.4%
(8) 0% 0% 0% 1.4 .5