Predicting trained task performance:

The interaction of taxonomy, data, and modeling

• Goal: Predict and optimize performance on trained tasks

• Four dimensions of analysis: 1. Task type2. Training methods3. Performance measures4. Training principles

The function of taxonomic analyses

• Develop analyses for dimensions that can:- Relate similar tasks- Cover task and training domains- Capture meaningful aspects of performance- Provide useful generalizations for optimization

constraints

Iterative development process

Analysis framework

Empirical training and performance data

Task analyses

Target phenomena

(including general principles)

Task models

Modeling performance using task features:An earlier approach

Roth, Thomas J. 1992. Reliability and validity assessment of a taxonomy for predicting relative stressor effects on human performance. Micro Analysis & Design Technical Report 5060-1.

• Roth described military tasks as weighted vectors of five features:

• Task features were found to be non-independent.

• Need task features that are (more) independent and provide more detail about cognition.

Attention

Perception

Psychomotor

Physical

Cognitive

A decomposition for cognitive tasksPerception/attentional processing

Cognitive/affective processing

Physical/communicative response

Vision, hearing, tactile sensation

Executive control/MonitoringMemory/RepresentationReasoning/Problem solvingMotivation/AffectConcept formationImagery

Synthesis Response planning

Language/SpeechManipulation/Fine motorAction/Gross motor

Speech planningMotor planning

Modeling the data entry task• Task of data entry is simple, but decomposable:

Read number

Encode number

Type number

Perceptual processing

Cognitive processing

Response

Plan motor output

• Performance is measured by speed and accuracy of typing

• Training consists of practice and repetition (1 pass/item)

Synthesis

Planning

Understanding componential performance

Consider effects of training practice and repetition on:

…as measured by speed and accuracy of data entry.

Reading

Planning

Encoding

Typing

Empirical phenomenaFixed processing of perception and response

Reading and typing numbers requires a fixed amount of time, which does not improve with practice.

Repetition priming of motor planning

Repeatedly planning the motor response for numbers leads to specific learning and speeding of responses.

Speed–accuracy trade-off

Increased response speed is associated, for most people, with a decrease in the accuracy of responses.

Encoding improvement of only some percepts

Repeated encoding of numerals does not improve with practice, but (non-usual) encoding numbers from words does.

Modeling implementation of phenomena

• Reading and typing speeds are constant, but depend on format.

tread(n) = cReadFormat ; ttype(n) = cTypeFormat

• Repetitious motor planning speeds responses (according to the power law of learning) and lowers accuracy.

tplanning(n) = ap + bp(N(n,t) + pp)-p

aplanning(n) = f(tplanning(n))

• Repetitious encoding only affects speed for less familiar percepts

tencoding(n) = ae + be(N(n,t) + pe)-e OR cEncoding

aencoding(n) = ?

The IMPRINT simulation

Practice = N

RT

= t

da

ta e

ntr

ym(n

)

Data Entry

Encoding

Typing

Reading

Repeated motor planning

Iteration: Decomposition Modeling Experimentation

• FeedbackFact: Accuracy can be improved with feedback. Update model to decouple speed and accuracy functions. Include monitoring function in task decomposition.

•FatigueFact: Speed of cognitive processing decreases without

repetition priming. Update model to reflect cognitive fatigue from practice.

•Error typesPrediction: Accuracy decline is due to motor planning errors. Examine effects on accuracy of encoding and planning. Representations and error types in task may be different.

Iteration: Enhancing the taxonomic analyses

• Individual differencesFact: Not everyone exhibits the speed–accuracy trade-off.Fact: Higher cognitive ability leads to faster skill acquisition. Add a dimension of individual variation.