Rohlings Interpretive Method: How Can a Flexible Battery Perform Like a Fixed Battery Martin L....

Rohling’s Interpretive Method: Rohling’s Interpretive Method: How Can a Flexible Battery How Can a Flexible Battery

Perform Like a Fixed BatteryPerform Like a Fixed Battery

Martin L. Rohling, Ph.D.Martin L. Rohling, Ph.D.

Associate ProfessorAssociate Professor

Department of PsychologyDepartment of Psychology

University of South AlabamaUniversity of South Alabama

January 13, 2007 Rohling - CCPN Orlando, FL2

Clinical vs. Mechanical DiagnosisClinical vs. Mechanical Diagnosis

Much research has been conducted since Meehl (1954) found clinical judgment to be less accurate than mechanical or “actuarial” judgement

e.g., Dawes, Faust, & Meehl (1989); Filskov (1981); Garb (1989); Garb (1994); Garb (1998); Grove et al. (2000); Sawyer (1966); and Wedding & Faust (1989)

Such results influential in causing NPs to turn to different versions of the HRB (Russell, 1998).

“Batteries” have been defined as the method by which one can avoid the clinical errors highlighted by Meehl an others, using “actuarial” rules for diagnosis (Russell, 1995; Russell et al., 2005).


Rohling’s Interpretive Method Rohling’s Interpretive Method (RIM): Development History(RIM): Development History

Conducted several meta-analysis with Dr. Laurence Binder at the Portland, OR – VA

The last of these focused on the residual cognitive effects of mild head injury. Binder, Rohling, & Larrabee (1997)

Binder et al. grouped effect sizes (ES) into domains of neuropsychological functioning based on factor analytic studies. e.g., Leonberger, Nicks, Larrabee, & Goldfader (1992)


RIM Generated fromRIM Generated fromMeta-Analytic ProceduresMeta-Analytic Procedures

Meta-analysis (MA) combines effect sizes (ES) across samples assuming that they all sample the population M for the particular effect of interest. Common method ES calculation is a standardized

mean difference score (e.g., Glass’ delta). delta = difference between con. & exp. group’s M’s

divided by con. group’s SD. delta analogous to Z score - linear equivalent of T

score used in clinical neuropsychology


RIM Generated fromRIM Generated fromMeta-Analytic ProceduresMeta-Analytic Procedures

Binder et al. (1997) combined ES’s generated from various tests into cognitive domains.

Why not similarly combine ES’s, or T scores, from a single patient into cognitive domains in the same way that it is accomplished in MA. Each test score is treated as a ES that reflects the

individual’s ability within a domain. ES can be combined based on homogeneity of

variance, so as to avoid combining apples and oranges.


Introduction to the RIM AnalysisIntroduction to the RIM Analysis

Flexible battery (multiple measure) use: Is the most frequently cited model of assessment

among neuropsychologists. Only 7% of neuropsychologists use a fixed

battery (Rabin et al, 2006, ACN). Regarding the suitability, practicality, and

usefulness of any fixed battery: “We know of no batteries that fully satisfy these

criteria.”(Lezak, Howieson & Loring 2004, Neuropsych. Assess., 4th

ed, p 648.)


Advantages of Flexible BatteryAdvantages of Flexible Battery

Dynamic & responsive to clinician’s needs Covers 1 or many domains “Flexible”, can be adapted for each patient Can “oversample” domains Well suited for hypothesis-driven approach


Potential Problems with aPotential Problems with aFlexible BatteryFlexible Battery

Inflated error rates Multicollinearity Weighting decision problems Unknown veracity/reliability of sets of tasks Human judgment errors


Human Judgment ErrorsHuman Judgment Errors(Wedding & Faust, 1989, ACN)(Wedding & Faust, 1989, ACN)

Hindsight bias Confirmatory bias Overreliance on salient data Under-utilization of base rates Failure to take into account co-variation


Potential Benefits withPotential Benefits withRohling’s Interpretive Method (RIM)Rohling’s Interpretive Method (RIM)

Judgment errors can threaten reliability & validity of multiple measure test batteries.

RIM was designed to reduce these effects. Based on meta-analytic techniques.

Uses a linear combination of scores placed on a common metric.


Potential Benefits of RIMPotential Benefits of RIM

A strategy that produces summary results analogous to those generated in a fixed-battery approach (e.g., HII, GNDS, AIR).

Takes advantage of psychometric properties of same metric data, e.g., T Scores.


Today’s Presentation - IntentToday’s Presentation - Intent

Present a set of procedures that allows for a quantitatively-based comparison of an overall battery of measures. Non-specific to battery measures

themselves. Can be used for any individual patient.

Demonstrate importance and practicality of use of established statistical indices. (e.g., alpha, beta, effect size).


Today’s Intent (cont’d)Today’s Intent (cont’d)

Present a data format for any set of measures to be inspected at: Global level (OTBM) Domain level (DTBM) Test measure level (ITBM)

Present a series of calculations to assist in the generation of these indices.

Present Steps in conjunction with clinical judgment from an informed position.


Common RIM Domains of Common RIM Domains of FunctioningFunctioning

Symptom Validity (SV) Tests Emotional / Personality (EP) Measures Meta-Cognition, Pain, or other self-ratings Estimated Premorbid General Ability (EPGA) Test Battery Means

Overall (OTBM), Domain (DTBM), & Instrument (ITBM)

Cognitive Domains: VC, PO, EF, AML, VML, AW, PS

Non-Cognitive Domains: PM, LA, SP


Sample RIM: Summary TableSample RIM: Summary TableRIM Summary Statistics: T-scores, variances, ES, & power.

Column # 1 2 3 4 5 6 7 8 9 10 11 12 R o w

Cognitive Domain M sd n Hetero. p value Classify % TI ES CI PreM

Nec. 1-sample

t test anova s & w

Power (1-b)

1 Symptom Validity (SV) 46.8 9.8 8 --- Average 25% -.32 + 5.8 52.6 --- --- .2045

2 Emotional Personality (EP) 30.6 17.2 10 .0033 Mld-Md 60% -1.38 + 9.0 39.6 .0060 --- ---

3 Meta-Cognition (MC) --- --- --- --- --- --- --- --- --- --- --- ---

4 Est. Pre. Gen. Ability (EPGA) 45.2 5.7 8 --- Average 13% --- + 3.3 41.9 --- --- ---

5 Overall TBM (OTBM) 36.6 12.4 69 .0063 Mild 57% -.89 + 2.5 39.1 <.00012 --- ---

6 Domain TBM (DTBM) 38.0 6.6 7 --- Mild 57% -1.17 + 3.1 41.1 .02721 .0068 ---

7 Instrument TBM (ITBM) 33.0 13.5 12 .0855 Mld-Md 67% -1.18 + 6.5 39.5 .00961 --- ---

8 Verbal Comprehension (VC) 38.6 7.1 6 --- Mild 67% -1.03 + 4.8 43.4 --- --- .6989

9 Perceptual Organization (PO) 33.9 10.2 6 --- Mld-Md 50% -1.37 + 7.0 40.9 .04321 --- ---

10 Executive Functioning (EF) 40.1 12.3 12 --- Blw Avg 50% -.53 + 5.9 46.0 --- --- -.5306

11 Auditory Mem. & Learn (AML) 32.0 11.5 20 --- Mld-Md 70% -1.45 + 4.2 36.2 <.00011 W ---

12 Visual Mem. & Learn (AML) 29.5 13.4 11 --- Moderate 82% -1.52 + 6.7 36.2 .00311 W ---

13 Attention/Work Memory (AW) 45.0 9.0 9 --- Average 22% -.03 + 5.0 50.0 --- --- .0591

14 Processing Speed (PS) 46.9 10.6 6 --- Average 17% .20 + 7.1 54.0 --- --- ---

15 Global/Miscellaneous (GM) --- --- --- --- --- --- --- --- --- --- --- ---

16 PsychoMotor (PM) 44.7 11.6 4 --- Blw Avg 25% -.49 + 7.5 52.2 --- --- .1919

17 Language/Aphasia (LA) 13.0 --- 1 --- Severe 100% --- --- --- --- --- ---

18 Sensory Perception (SP) --- --- --- --- --- --- --- --- --- --- --- ---


Sample RIM: Graphic DisplaySample RIM: Graphic Display

0

10

20

30

40

50

60

70

80

90

100

SV

EP

EP

GA

OT

BM

DT

BM

ITB

M VC

PO EF

AM

L

VM

L

AW PS

PM LA

SP


Brief of RIM Steps:Brief of RIM Steps:• There are 24 steps to the RIM process

17 calculation steps:Advice on design of the batteryCalculation of summary statisticsGeneration of graphic displays

7 interpretative steps.• Detail a systematic procedure for use of the

statistical summary table and graphic displays to: Assess and verify summary data. Identify strengths/limitations of current data. Obtain a reliable diagnosis. Develop tx plans based on sound judgments.

• We briefly review each step in just a moment.


Support for the RIM ProcessSupport for the RIM Process

Rational support/reasoning: Reduce clinical judgment errors. The RIM is a Process, not a program. Rather, the RIM is a way of formalizing

thinking & interpretation of individual case data.

This is operationalizing what many flexible battery clinicians are already doing in their head.


Support for the RIM Process:Support for the RIM Process:Specific AdvantagesSpecific Advantages

Psychometric properties at level with fixed, co-normed batteries, without their limitations.

Flexibility of test selection.

Flexibility of theoretical view of cognition (domain structure)


Support for the RIM Process:Support for the RIM Process:Specific AdvantagesSpecific Advantages

Quantitatively support your conclusions and interpretations Statistical evaluation Measure of confidence in findings Measure of limitations of findings Ability to present data at different

levels of interpretation Greater defensibility


The RIM has a Set of The RIM has a Set of Procedure or Specific StepsProcedure or Specific Steps


RIM Steps 1-4: Summary DataRIM Steps 1-4: Summary Data

1) Design & administer battery. Use well standardized recently normed tests.

2) Estimate premorbid general ability. Use Reading (WTAR), Regression (OPIE-III), &

academic records (rank, SAT, ACT).

3) Convert test scores to a common metric. We recommend T scores, but z or SS OK too.

4) Assign scores to domains. Factor analysis to support assignment

(Tulsky et al., 2003)



5) Calculate domain M, sd, & n.6) Calculate test battery means (TBM).

Overall TBM – All scores, large N & high power. Domain TBM – Avoids domain over weighting.

(e.g., attention & memory). Instrument TBM – One score per norm sample.

7) Calculate p for heterogeneity. Have you put “apples & oranges” together?

8) Determine categories of impairment. Recommend using of Heaton et al. (2003).



9) Determine % of test impaired. Analogous to Halstead Impairment Index

# scores below cutoff / total # of of scores

10) Calculate ES for all summary stats. Use Cohen’s d = (Me – Mc) / SD pooled

11) Calculate confidence interval for stats. 90% CI = 1.645 x SEM

12) Upper limit of performance for impair. Look for overlap between 90% CI of EPGA

(lower) & Summary Stats (upper)



13) Conduct one-sample t tests. Use EPGA as reference point

14) Conduct a between-subjects ANOVA. Looking for strengths & weaknesses

15) Conduct power analyses. Only needed for those NS differences

16) Sort scores for visual inspection.

17) Graphically display summary statistics.


RIM Steps 18-20: InterpretationRIM Steps 18-20: Interpretation

18) Assess battery validity. Examine the Symptom Validity scores. Caution in accepting low power results. Look at heterogeneity of summary stats.

1. Normative sample unrepresentative of patient.2. Scores assigned to wrong domain. 3. Inconsistent performance on construct measures.

19) Examine influence of psychopathology. Examine scores for heterogeneity.

20) Check OTBM, DTBM, & ITBM impaired.


RIM Steps 21-24: InterpretationRIM Steps 21-24: Interpretation

21) Examine strengths/weaknesses looking at:1. Confidence intervals overlap.2. Results from one-sample t tests.3. Results of ANOVA.4. %TI show differences otherwise not evident.

Determine if pattern existed premorbidly.

22) Examine non-cognitive domains. Psychomotor, Lang/Aphasia, Sensory Percept

23) Explore Type II errors –need more tests?24) Examine sorted T-scores

Look for patterns missed by summary stats.


RIM Sample Case 1: Obvious TBIRIM Sample Case 1: Obvious TBI Age: 37 Handed: Left Race: Euro-American Sex: Female Ed: 14 years Occup: Nursing Marital: Sep. 10 yrs Living: Camper in

parent’s backyard

Reason for Referral:TBI in head-on boat accident. Propeller hit pt in right parietal-occipital lobe (LOC = 7 days; GCS = 3). Eval. to determine capacity for medical & financial decisions, parenting skills, occupational prognosis, & disability status. Significant emotional, behavioral, occupational, and social problems pre-TBI.


RIM Sample Case 1: Obvious TBIRIM Sample Case 1: Obvious TBI

0

10

20

30

40

50

60

70

80

90

100S

V

EP

EP

GA

OT

BM

DT

BM

ITB

M

VC

PO

EF

AM

L

VM

L

AW PS

PM

LA

SP


RIM Sample Case 1: Obvious TBIRIM Sample Case 1: Obvious TBIRIM Summary Statistics: T-scores, variances, ES, & power.

Column # 1 2 3 4 5 6 7 8 9 10 11 12 R o w


Nec. 1-sample

t test anova s & w

Power (1-b)

1 Symptom Validity (SV) 46.8 9.8 8 --- Average 25% -.32 + 5.8 52.6 --- --- .2045

2 Emotional Personality (EP) 30.6 17.2 10 .0033 Mld-Md 60% -1.38 + 9.0 39.6 .0060 --- ---



5 Overall TBM (OTBM) 36.6 12.4 69 .0063 Mild 57% -.89 + 2.5 39.1 <.00012 --- ---

6 Domain TBM (DTBM) 38.0 6.6 7 --- Mild 57% -1.17 + 3.1 41.1 .02721 .0068 ---

7 Instrument TBM (ITBM) 33.0 13.5 12 .0855 Mld-Md 67% -1.18 + 6.5 39.5 .00961 --- ---

8 Verbal Comprehension (VC) 38.6 7.1 6 --- Mild 67% -1.03 + 4.8 43.4 --- --- .6989

9 Perceptual Organization (PO) 33.9 10.2 6 --- Mld-Md 50% -1.37 + 7.0 40.9 .04321 --- ---

10 Executive Functioning (EF) 40.1 12.3 12 --- Blw Avg 50% -.53 + 5.9 46.0 --- --- -.5306

11 Auditory Mem. & Learn (AML) 32.0 11.5 20 --- Mld-Md 70% -1.45 + 4.2 36.2 <.00011 W ---

12 Visual Mem. & Learn (AML) 29.5 13.4 11 --- Moderate 82% -1.52 + 6.7 36.2 .00311 W ---

13 Attention/Work Memory (AW) 45.0 9.0 9 --- Average 22% -.03 + 5.0 50.0 --- --- .0591

14 Processing Speed (PS) 46.9 10.6 6 --- Average 17% .20 + 7.1 54.0 --- --- ---


16 PsychoMotor (PM) 44.7 11.6 4 --- Blw Avg 25% -.49 + 7.5 52.2 --- --- .1919

17 Language/Aphasia (LA) 13.0 --- 1 --- Severe 100% --- --- --- --- --- ---



TBI Dose Response CurvesTBI Dose Response CurvesDikmen ES’sDikmen ES’s Meyers’ T ScoresMeyers’ T Scores

-3

-2.5

-2

-1.5

-1

-.5

0

hrs < 1 hrs 1-23 day 1-6 day 7-13 day 14-28 day > 28

10

15

20

25

30

35

40

45

50

55

60

< 1 hr 1-23 hrs 1-6 days 7-13 days 14-28 days > 28 days

Severity of TBI based on LOC

Ove

rall

Test

Bat

tery

Mea

n-T

Min y = -3.5x + 36.4

Max y = -2.1x + 56.2

90%ile y = -2.2x + 53.5

75%ile y = -2.3x + 50.4

50%ile y = -2.6x + 47.6

25%ile y = -3.1x + 44.6

10%ile y = -3.7x + 42.2


Combined Dikmen & Meyers Combined Dikmen & Meyers Estimates: ES, T, & DifferenceEstimates: ES, T, & Difference

LOCLOC nn ESES TT Mean Diff.Mean Diff.(EPGA – OTBM)(EPGA – OTBM)

Trauma ControlTrauma Control 121 .00 48.1 ---

G1: < 1hrG1: < 1hr 299 -.11 46.5 1.5

G2: 1-24 hrG2: 1-24 hr 152 -.22 44.2 3.8

G3: 1-6 dayG3: 1-6 day 99 -.33 42.7 5.6

G4: 7-13 dayG4: 7-13 day 56 -.68 38.1 10.2

G5: 14-28 dayG5: 14-28 day 46 -1.29 29.4 19.1

G6: > 28 dayG6: > 28 day 43 -1.49 27.7 20.6


Return to Work Study: OTBM’s for Return to Work Study: OTBM’s for 4 Groups of TBI Survivors4 Groups of TBI Survivors

GroupGroup nn MM SDSD ESES

DisabledDisabled 17 32.8 6.4 -2.29

UnemployedUnemployed 96 39.5 6.1 -1.01

Below PreviousBelow Previous 32 43.3 4.6 -.36

At PreviousAt Previous 137 45.1 5.2 -.45


RIM Sample Case 1: Obvious TBI RIM Sample Case 1: Obvious TBI Normal Distribution of T ScoresNormal Distribution of T Scores

0

5

10

15

20

25

Per

cen

t

0 10 20 30 40 50 60 70 80 90 100OTBM


RIM Sample Case 2: Subtle DiabetesRIM Sample Case 2: Subtle Diabetes

Reason for Referral: 2 yrs dangerous work habits. Eval to see if atrial fib & Type II diabetes impairs cognition. Hospitalized “TIA-like” Sx. Admitted to problems for 20 yrs, cardiac dysrhythmia & bradycardia, pacemaker, blood sugar difficult to manage, & family Hx of heart disease & diabetes.

Age: 55 Handed: Right Race: Euro-American Sex: Male Ed: 13 years Occup: Mechanic Marital: Married 20 yr Living: at home w/wife


RIM Sample Case 2: Subtle DiabetesRIM Sample Case 2: Subtle Diabetes

0

10

20

30

40

50

60

70

80

90

100S

V

EP

EP

GA

OT

BM

DT

BM

ITB

M

VC

PO EF

AM

L

VM

L

AW PS

PM LA

SP


RIM Sample Case 2: Subtle DiabetesRIM Sample Case 2: Subtle DiabetesRIM Summary Statistics: T-scores, variances, ES, & power.

Column # 1 2 3 4 5 6 7 8 9 10 11 12 R o w


Nec. 1-sample

t test anova s & w

Power (1-b)

1 Symptom Validity (SV) 45.3 2.2 7 --- Average 0% -.65 + 1.3 46.6 --- .0014 ---

2 Emotional Personality (EP) 37.4 14.7 8 .0814 Mild 50% -1.00 + 8.5 45.9 --- .0458 ---



5 Overall TBM (OTBM) 44.6 9.7 57 --- Blw Avg 25% -1.04 + 2.1 46.7 --- <.0001 ---

6 Domain TBM (DTBM) 44.4 4.9 7 --- Blw Avg 14% -1.88 + 3.0 47.4 .1978 .0058 ---

7 Instrument TBM (ITBM) 41.2 9.5 12 --- Blw Avg 58% -1.54 + 5.6 46.8 --- .0023 ---

8 Verbal Comprehension (VC) 49.6 9.0 5 --- Average 20% -.37 + 6.6 55.2 --- --- .1710

9 Perceptual Organization (PO) 45.7 9.3 6 --- Average 17% -.92 + 6.2 51.9 --- --- .6155

10 Executive Functioning (EF) 40.8 6.8 11 --- Blw Avg 36% -2.14 + 3.2 44.0 --- .0003 ---

11 Auditory Mem. & Learn (AML) 48.6 14.1 15 .0176 Average 20% -.34 + 6.0 54.6 --- --- .3476

12 Visual Mem. & Learn (AML) 47.4 5.2 8 --- Average 0% -1.10 + 2.9 50.3 --- .0377 ---

13 Attention/Work Memory (AW) 42.8 7.3 9 --- Blw Avg 33% -1.66 + 3.9 46.7 --- .0051 ---

14 Processing Speed (PS) 36.0 4.6 4 --- Mild 50% -4.10 + 3.7 39.7 W .0061 ---


16 PsychoMotor (PM) 19.0 2.8 2 --- Severe 100% -4.22 + 3.3 22.3 W .0410 ---

17 Language/Aphasia (LA) 36.8 4.5 2 --- Mild 50% -1.70 + 5.3 42.1 --- --- .2935

18 Sensory Perception (SP) 45.5 1.0 2 --- Average 0% -.63 + 1.1 46.6 --- --- .1237


RIM Sample Case 2: Subtle Diabetes RIM Sample Case 2: Subtle Diabetes Normal Distribution of T ScoresNormal Distribution of T Scores

0

5

10

15

20

25

Perc

en

t

0 10 20 30 40 50 60 70 80 90 100OTBM

Histogram


RIM Critiques: Concern 1RIM Critiques: Concern 1

The method of calculating the standard deviations (SDs) for summary statistics and domain scores is incorrect. Since many of the remaining steps of the

RIM depend on the use of these SDs, this error is magnified in the subsequent steps.

SDs statistically can not exceed 9.99 and are more likely to be around 6.4


Response 1: RIM Response 1: RIM MsMs 4 Datasets 4 Datasets

0

5

10

15

20

25

30

Perc

ent

20 25 30 35 40 45 50 55 60 65 70Mean OTBM WAIS-R

Mean OTBM for Dataset 1 (WAIS-R)

0

5

10

15

20

25

30

Perc

ent

20 25 30 35 40 45 50 55 60 65 70Mean OTBM Meyers

Mean OTBM for Dataset 3 (Meyers)

0

5

10

15

20

25

30

Pe

rce

nt

20 25 30 35 40 45 50 55 60 65 70Mean OTBM Green

Mean OTBM for Dataset 2 (Green)

0

5

10

15

20

25

30

Perc

ent

20 25 30 35 40 45 50 55 60 65 70Mean OTBM HRB

Mean OTBM for Dataset 4 (HRB)


Inter-Individual Inter-Individual Ms Ms && SDs SDs

N Mn SD

Dataset 1 Psych Pts

WAIS-R 457 43.2 7.2

WAIS 150 45.0 9.1

Dataset 2 (Green) 904 44.8 7.3

Dataset 3 (Meyers) 1,734 42.0 7.3

Dataset 4 (HRB) 114 42.8 6.8

Total 4 Samples 3,359 43.1 7.4


Response 1: RIM Response 1: RIM SDsSDs 4 Datasets 4 Datasets

0

5

10

15

20

25

Perc

en

t

0 2 4 6 8 10 12 14 16 18 20WAIS-R OTBM-SD

OTBM-sd for Dataset 1 (WAIS-R)

0

5

10

15

20

25

Pe

rce

nt

0 2 4 6 8 10 12 14 16 18 20Meyers OTBM-SD

OTBM-sd for Dataset 3 (Meyers)

0

5

10

15

20

25

Perc

en

t

0 2 4 6 8 10 12 14 16 18 20Green's OTBM-SD

OTBM-sd for Dataset 2 (Green)

0

5

10

15

20

25

Pe

rce

nt

0 2 4 6 8 10 12 14 16 18 20HRB OTBM-SD

OTBM-sd for Dataset 4 (HRB)


Intra-Individual Intra-Individual MsMs & & SDsSDs

N Mn SD % > 9.99

Dataset 1 Psych Pts

WAIS-R 457 6.8 2.0 7%

WAIS 150 7.4 2.2 10%

Dataset 2 (Green) 904 11.4 2.9 65%

Dataset 3 (Meyers) 1,734 11.9 2.9 56%

Dataset 4 (HRB) 114 10.6 2.4 61%

Total 4 Samples 3,359 10.8 2.8 50%



More false-positives then clinical judgment. Palmer et al. (2004) expressed concern

that We failed to distinguish “statistical” from

“clinical” significance. This failure is a critical error that precludes

the prudent clinician from using the RIM.


Response 2: RIM vs. Manual Response 2: RIM vs. Manual Detecting Differences – Overall %Detecting Differences – Overall %

% of Total S’s MANUAL METHOD

RIM t TEST VIQ-PIQ: NS VIQ-PIQ: Sig. Marginal M’s

VIQ-PIQ: NS 54% 23% 78%

VIQ-PIQ: Sig. 1% 21% 22%

Marginal M’s 55% 45% 100%


Response 2: RIM vs. Manual Response 2: RIM vs. Manual Detecting Differences – Detecting Differences – ESsESs

Means (SDs) MANUAL METHOD

RIM t TEST VIQ-PIQ: NS VIQ-PIQ: Sig.Marginal

M’s

VIQ-PIQ: NS .38 (.30) .80 (.41) .50 (.39)

VIQ-PIQ: Sig. 1.58 (.82) 1.70 (.86) 1.69 (.85)

Marginal M’s .40 (.37) 1.22 (.80) .90 (.71)


Response 2: RIM vs. ManualResponse 2: RIM vs. ManualDetecting Differences ScoresDetecting Differences Scores

Means (SDs) MANUAL METHOD

RIM t TEST VIQ-PIQ: NS VIQ-PIQ: Sig. Marginal M’s

VIQ-PIQ: NS 3.9 (2.5) 13.2 (3.7) 6.7 (5.2)

VIQ-PIQ: Sig. 6.7 (0.8) 19.0 (6.5) 16.9 (8.5)

Marginal M’s 4.0 (2.5) 15.9 (6.0) 9.3 (7.4)



Clinicians who use the RIM will: Idiosyncratically assign scores to cognitive

domains. This will result in low inter-rater reliability in

analysis & diagnosis.



Scores on domains are unit weighted, which introduces error.

Willson & Reynolds (2004) said scores load on multiple domains. Assignment to domains & weights depend on: Battery of tests administered. Patients whose test scores are being

examined.


Response 4: Cross-Valid. Unit WtsResponse 4: Cross-Valid. Unit Wts

Conducted 4 multiple reg. on 457 pts’ WAIS-R. Split sample in ½ - assess shrinkage.

Regressed patients’ verbal subtests onto PIQ. Generated ideal weights for the 1st ½ of sample.

Used wts to predict PIQs in the 2nd ½ of sample. Pre-PIQs regressed on actual PIQs 2nd ½ sample.

Also, generated weights for the 2nd ½ of sample. Use Pre-PIQ’s regress on actual PIQs 1st ½ sample.

Repeated, except performance subtests predict VIQ split sample ½ & generate same statistics as before.


Response 4: Cross-Valid. Unit WtsResponse 4: Cross-Valid. Unit Wts

Purpose of these procedures: How much variance in wts. is sample specific. Amount of shrinkage using cross-validated wts. Shrinkage error compared to error introduced by

using “unit wts” vs. “ideal wts.” Results: 98% of the variance accounted for with

unit wts. Compared to ideal weights. Support use of unit wts. Rather than ideal wts.

See, Dawes, R. M. (1979).



Multiple measures used to generate composite scores: Results in less accurate estimates of

the cognitive domains.


Response 5: Estimate FSIQ Using Response 5: Estimate FSIQ Using Scaled Score Means’sScaled Score Means’s

Diff. between 1 predictor and additional predictors

% Improved

M Absolute Value of

Diff

(SEM of Improved)

1 predictor (Voc) --- 4.99 (.47) 2 predictors (1 +OA) 51% 4.42 (.49) 3 predictors (2 + BD) 70% 3.58 (.45) 4 predictors (3 + Sim) 70% 3.01 (.36) 5 predictors (4 + DSp) 81% 2.42 (.25) 6 predictors (5 + Info) 79% 2.53 (.26) 7 predictors (6 + PA) 81% 2.11 (.21) 8 predictors (7 + Comp) 88% 2.06 (.18) 9 predictors (8 + DSy) 93% 1.31 (.14) 10 predictors (9 + Arith) 91% 1.36 (.12) 11 predictors (10 + PC) 93% 1.27 (.06)

0

1

2

3

4

5

6

Cell M

ean

Diff 1

Diff 2

Diff 3

Diff 4

Diff 5

Diff 6

Diff 7

Diff 8

Diff 9

Diff 1

0

Diff 1

1



A general ability factor is used to represent premorbid functioning for all domains.

This not supported by the literature. This results in inaccurate conclusions

regarding degree of impairment suffered by a patient in each cognitive domains assessed.


Domain Means CorrelationsDomain Means CorrelationsAll were Significant ( p < .001 )

1 2 3 4 5 6 7 8 9 10 11

1- Premorbid .76 .71 .62 .56 .79 .68 .53 .54 .30 .28

2- OTBM .76 .98 .81 .82 .84 .81 .77 .77 .54 .53

3- DTBM .71 .98 .77 .79 .78 .81 .78 .80 .62 .62

4- Attention/Working Mem .64 .81 .77 .64 .69 .54 .68 .53 .37 .31

5- Proc. Spd/Mental Flex .62 .82 .79 .64 .72 .64 .50 .55 .44 .44

6- Verbal Reasoning .79 .84 .78 .69 .72 .64 .54 .55 .36 .30

7- Visual Reasoning .68 .81 .81 .54 .64 .64 .51 .70 .41 .45

8- Verbal Memory .53 .77 .78 .68 .50 .54 .51 .62 .34 .32

9- Visual Memory .54 .77 .80 .53 .55 .55 .70 .62 .37 .40

10- Dom Motor/Sensory .30 .54 .62 .37 .44 .36 .41 .34 .37 .53

11- Non-Dom Motor/Sens .28 .53 .62 .31 .44 .30 .45 .32 .40 .53



Norms used come from samples that are of undocumented comparability.

Furthermore, even when norms used were generated from different but comparable samples, their format prohibits ready comparisons.


Response 7: Split-Half ReliabilityResponse 7: Split-Half Reliability

Analyze Dataset 2: OTBM’s from 42 DV’s Individuals’ data split into two sets

21 test variables for each OTBM (1 & 2) 2 independent OTBMs created for pt. Split DV’s intentionally-separated so no

normative sample included both OTBM’s



Results r = .81, 66% of variance accounted Slope of the regression line was .82 (SE = .027) Intercept 9.2 (SE = 1.20).

Mean OTBM-1 = 45.0 (sd = 7.3) Mean OTBM-2 = 43.6 (sd = 7.2) Results simulate worse case scenario.

used an entirely different set of norms. Est. test-retest r for OTBM 42 DV’s increased

r from .81 to .88 using the Spearman-Brown correction).



No overlap in normative samples. Worst-case condition, generally administer

instruments (e.g., WAIS-III) with OTBMs generated from “co-normed” variables. Meyers & Rohling test-retest reliability of .86.

When different norms used, often gave same instruments (e.g., AVLT or RCFT)

No instrument used OTBM-1 included OTBM-2 Heaton et al.’s (2001) - schizophrenic pts.

Obtained a test-retest reliability of .97. Comparing 2 identical batteries, not worst-case.



The RIM will result in an undue inflation of clinicians’ confidence. Such overconfidence results in more error in

a interpretation, not less.


RIM vs. Tulsky et al. (2003): Case 1RIM vs. Tulsky et al. (2003): Case 1WAIS & WMS Battery Full Battery

Index or Domain ScoresDisc.

Model RIM

Model # RIM Model #

S EPGA1 (WTAR) 103 102 1 106 5

U Overall TBM2 (FSIQ) (112) 104 18 96 I 70

M Domain TBM2 --- 105 6 98 7

M Inst TBM2 (FSIQ/GMI) --- 104 2 93 I 14

1 Verbal Comp (VCI) 120 * 118 * 3 110 6

2 Percept Organ (POI) 121 * 117 * 3 107 5

3 Attent/Work Mem (WMI) 105 105 2 95 I* 9

4 Process Speed (PSI) 93 ** 95 ** 2 86 I* 5

5 Aud Mem & Learn (AMI) 97 ** 97 ** 4 81I***** 18

6 Vis Mem & Learn (VMI) 94 95 4 98 13

7 Executive Function (EF) --- --- 0 100 I* 15

8 Psycho-Motor (PM) --- --- 0 111 6


RIM vs. Tulsky et al. (2003): Case 2RIM vs. Tulsky et al. (2003): Case 2WAIS & WMS Battery Full Battery

Index or Domain Scores Disc. Model RIM Model # RIM Model #

S EPGA1 (WTAR) 125 120 1 117 5

U Overall TBM2 (FSIQ) (119) 103 18 96 I 53

M Domain TBM2 --- 105 6 95 I 7

M Inst TBM2 (FSIQ/GMI) --- 106 2 93 I 11

1 Verbal Comp (VCI) 124 122 3 117 5

2 Percept Organ (POI) 95 97 3 97 3

3 Attent/Work Mem (WMI) 108 108 2 102 7

4 Process Speed (PSI) 98 92 2 83 5

5 Aud Mem & Learn (AMI) 111 110 4 99 I 9

6 Vis Mem & Learn (VMI) 104 103 4 89 I 10

7 Executive Function (EF) --- --- 0 94 I 14

8 Psycho-Motor (PM) --- --- 0 77 6


Summary of the Rohling Summary of the Rohling Interpretive Method of Statistical Interpretive Method of Statistical

Analysis of Individual Analysis of Individual Neuropsychological Test DataNeuropsychological Test Data


Summary of RIM StepsSummary of RIM Steps

24 total steps to the process 17 calculation steps

Battery Design Calculation of summary statistics Generation of graphic displays

7 interpretative steps Use of summary table and graphic displays to:

Assess and verify summary data Identify strengths/limitations of current data Obtain a reliable diagnosis Develop tx plans based on clinical judgments.


Summary of RIM AdvantagesSummary of RIM Advantages

Formulize thinking interpretation of data: Operationalize what you already do.

Reduce judgment errors thru RIM Process. Take advantage of psychometric properties

at level with fixed, co-normed batteries. Allows flexibility of test selection. Allows flexibility of theoretical view of

cognition (e.g., domain structure)


Summary of RIM Advantages cont’dSummary of RIM Advantages cont’d

Gives Quantitative support for your conclusions and interpretations Statistical evaluation Measure of confidence in findings Measure of limitations of findings Ability to present data at different levels of

interpretation

Equals greater defensibility


Our RIM Cautions/ConcernsOur RIM Cautions/Concerns

Does not “replace” clinical judgment, rather, informs clinical judgment. This still means CJ errors are possible.

Susceptibility T-Scores to distrib. deviance

Process, not program Pre-morbid ability estimates

Domain selection, test placement


RIM is Not Alone Out There!

Dawn Flanagan, Ph.D., at St. Johns University in New York independently developed a similar method The Cattell-Horn-Carroll (CHC) Cross Battery Approach.

Second edition of Essentials of Cross-Battery Assessment (Flanagan, Ortiz, & Alfonso, in press) is due out in March, which explains her method, along with co-authors Some of her work can also be found on the website by

Dumont-Willis.

Published Research Findings Published Research Findings Using the RIMUsing the RIM

1) RIM vs. HRB

2) Variance Accounted for by SVT

3) Effect of Depression on NP Results

4) Prediction of Employment after TBI


RIM of HRB: OTBM vs. HIIRIM of HRB: OTBM vs. HII

Heaton et al.’s (1991) HRB norms for OTBM T Score (M=50, sd=10)

OTBM r with HII = -.79 (p < .0001) 62% variance account.

Over predicts low Under predicts high

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

HII

20 25 30 35 40 45 50 55 60

OTBM


RIM of HRB: OTBM vs. GNDSRIM of HRB: OTBM vs. GNDS

OTBM r with GNDS = -.87 76% variance acc. OTBM neither under

nor over predicts across range of GNDS

Intercept impairment is T Score = 46.0 Reitan & Wolfson (1993)

(GNDS = 29)0

10

20

30

40

50

60

70

80

GNDS

20 25 30 35 40 45 50 55 60

OTBM


RIM of HRB: OTBM’s RIM of HRB: OTBM’s Relationship to Global IndicesRelationship to Global Indices

INDICES OF FUNCTIONINDICES OF FUNCTION Correlation CoefficientCorrelation Coefficient

Halstead Impairment IndexHalstead Impairment Index .79

Average Impairment RatingAverage Impairment Rating .90

Global Neuro. Deficit ScaleGlobal Neuro. Deficit Scale .87

RIM: Domain TBMRIM: Domain TBM .99

RIM: Instrument TBMRIM: Instrument TBM .95

RIM: % Tests ImpairedRIM: % Tests Impaired .96


RIM of HRB: Diagnostic RIM of HRB: Diagnostic Classification Using the HIIClassification Using the HII

BR 65%BR 65% Sens.Sens. Spec.Spec. PPVPPV NPVNPV % Corr.% Corr.

HIIHII .64 .66 .77 .51 65%

AIRAIR .58 .78 .82 .51 65%

GNDSGNDS .78 .63 .79 .62 73%

OTBMOTBM .90 .32 .70 .65 69%

ITBMITBM .86 .37 .71 .60 69%

%TI%TI .85 .56 .78 .68 74%


RIM of HRB: Cross-Validation of RIM of HRB: Cross-Validation of RIM using HRB in 2 SamplesRIM using HRB in 2 Samples

Regressed Dikmen & Meyers TBI data Generated a predicted HII for pts in OK dataset. Correlation actual & predicted HII = .95

Sen = .60, Spec = .77, PPV = .78, NPV = .59 Overall % Correct Classification = 71%

Predicted HII from MNB’s OTBM & got a more accurate indicator of impairment than actual HII


Factor Loadings of Domain ScoresFactor Loadings of Domain Scores

Genuine Normal Genuine Neuro Exag Normal

Obj Perf

Self-Report

Obj Perf

Self-Report

Obj Perf

Self-Report

NPT .57 -.03 .64 -.01 .89 .11

SVT .58 -.08 .63 -.02 .87 .12

MCI -.03 .56 .04 .55 .33 .81

PSX -.07 .58 -.06 .55 .03 .91

Eigen 1.30 1.83 1.22 1.67 2.02 1.14

% Var. 33% 46% 30% 42% 51% 29%


MM & & SDsSDs of Composite of Composite ZZ scores scores

Genuine Exaggerate

Normal Neurologic Normal Neurologic

Neuropsych Test Scores

.33(.62)

.19(.64)

-.60(.80)

-.79(.65)

Symptom Validity

.51(.38)

.50(.30)

-1.25(.94)

-.50(.52)

Memory Complaints

.14(.93)

.41(.92)

-.62(.93)

.49(.54)

Psychiatric Symptoms

.10(.95)

.46(.96)

-.39(.85)

.19(.99)


Mean Z score on Objective TestsMean Z score on Objective Tests Small diff. between

Gen. Normal & Gen. Neuro. on NPT

No diff. between Exag. Normal & Exag. Neuro on NPT

Deficits for Exag. Neuro were more modest than for Exag. Normals on SVT

Interaction between Validity & Neuro Status.

-1.5

-1.3

-1.1

-.9

-.7

-.5

-.3

-.1

.1

.3

.5

Cell M

ean

Neuropsych Ability Symptom Validity

Exag-Neuro

Exag-Normal

Gen-Neuro

Gen-Normal


Mean Mean ZZ score Self-Report score Self-Report No diff. between Gen.

Neuro. & Exag. Neuro on Memory Complaints

No diff. between Gen. & Exag. Neuro on Psych. symptoms

Deficits for Exag. Normal on Psych. symptoms & Memory Complaints, the latter is larger

Interaction between Validity & Neuro Status.

-1.5

-1.3

-1.1

-.9

-.7

-.5

-.3

-.1

.1

.3

.5

Cell M

ean

Memory Complaints Psychiatric Symptoms

Exag-Neuro

Exag-Normal

Gen-Neuro

Gen-Normal


Depression Study: ReferenceDepression Study: Reference

Rohling, M. L., Green, P., Allen, L. M., & Iverson, G. L. (2002). Depressive symptoms and neurocognitive test scores in patients passing symptom validity tests. Archives of Clinical Neuropsychology, 17, 205-222.


Mood Group AssignmentMood Group Assignment

Patients classified into 2 subgroups From entire sample, 420 passed all SVTs

Sample split based on BDI Low-Depressed 25%ile on BDI (< 10)

n = 178, M = 6 (3) High-Depressed 75%ile on BDI (> 25)

n = 187, M = 31 (6)


Depression Study ParticipantsDepression Study Participants

All 365 patients referred for evaluation for compensation-related purposes

All diagnostic groups included 53% Head injury referrals 22% Medical referrals 14% Psychiatric referrals 11% Other neurological

Age = 42 (11); Ed = 13 (3); Sex = 64% males; Non-English = 18%; Handedness = 9% Left


Results Mood & Validity StatusResults Mood & Validity Status

107

(30%)

68

(19%)

159

(44%)

27

(7%)

SVT Status

Mood BDI

NonDep 25%ile

Depress 75%ile

Genuine Exaggerating

175(48%)

186(52%)

266(74%)

95(26%)


Results: Sample Split by ValidityResults: Sample Split by Validity

-1.5-1.3-1.1-.9-.7-.5-.3-.1.1.3.5.7.91.11.3

High-Dep

Low-Dep

EPT MCI OTBMEPT MCI OTBM EPT MCI OTBMEPT MCI OTBM EPT MCI OTBMEPT MCI OTBM

Total SampleTotal Sample Gen PtsGen Pts Exag PtsExag Pts

Z-s

core

sZ

-sco

res

-1.5-1.3-1.1-.9-.7-.5-.3-.1.1.3.5.7.91.11.3

High-DepHigh-Dep

Low-Dep Low-Dep


Exaggerating patients accounted for 39% of High-Dep group 14% of Low-Dep group

Mood & Effort used as IVs and Cognition DV Effect for effort, no effect for mood

However, when Memory Complaints DV Effects for both effort and mood

Also, when other Emotion/Personality DV Effects for both effort and mood

Effect of Mood Depends on EffortEffect of Mood Depends on Effort


Effect of Mood Depends on EffortEffect of Mood Depends on Effort

When both mood groups were included in regression analysis, as predicted:

Memory ratings related to mood (r = .60; p < .0001)

Mood not correlated with cognition (r = .10; p > .10)

Memory ratings not related to cognition (r = .13, p = .06)


Mood ReplicationMood Replication

Gervais’ pain sample findings (n = 177) Exaggerating patients accounted for

55% of High-Dep; 33% of Low-Dep group Memory ratings related to mood (r = .55) Mood not correlated with cognition (r = .06) Memory ratings related to cognition (r = .15) Group means correlated with Green’s .94

all patient (High-D, Low-D, Gen, & Exag).


SVT Results Test Score

WMT CARB TOMM N Mean OTBM

Mean Pain

Pass Pass Pass 340 0.24 -0.30

Pass Fail Pass 14 -0.06 -0.09

Fail Pass Pass 74 -0.54 0.14

Fail Fail Pass 35 -0.74 0.57

Fail Pass Fail 17 -0.96 0.39

Fail Fail Fail 38 -1.39 0.67

Effect if Pain on OTBMEffect if Pain on OTBM


Effect if Pain on OTBMEffect if Pain on OTBM Pain Rating Status

Low Rating (n = 41)

High Rating (n = 80)

M (SD) M (SD) g (ES) p

Objective Performance

Mean SVT .43 (.08) .40 (.12) -.03 .1103

OTBM .38 (.54) .14 (.68) -.38 .0596

Demographic Comparisons

Pain Rating (0 – 5) .59 (.50) 4.31 (.47) -7.75 <.0001

Age in Years 36.8 (10.0) 40.0 (10.2) .32 .0317

Years of Education 11.9 (2.9) 11.4 (2.5) .19 .3466

WAIS-R/III or MAB VIQ* 101.3 (12.3) 93.0 (13.0) .65 .0029

WRAT-III Reading Subtest 44.4 (5.8) 41.8 (6.0) .44 .1537

English 1st Language (% total) 95.1 (---) 93.6 (---) --- .7320

Male (% total) 63.4 (---) 62.5 (---) --- .9215


Return to Work after InjuryReturn to Work after Injury

Three main hypotheses using MNB-RIM OTBM will predict return to work level Cognitive domain that will be most

predictive will be executive function Adding the Patient Competency Rating

Scale will improve work prediction PCRS is by Prigatano (1985)


Return to Work: ANOVA of OTBMReturn to Work: ANOVA of OTBM

GroupGroup nn MM SDSD ESES

DisabledDisabled 17 32.8 6.4 -2.29

UnemployedUnemployed 96 39.5 6.1 -1.01

Below PreviousBelow Previous 32 43.3 4.6 -.36

At PreviousAt Previous 137 45.1 5.2 -.45


Logistic Regression Using OTBMLogistic Regression Using OTBM

PredictedPredicted

ObservedObservedDisableDisable

Un-Un-employemploy

Below Below PrevPrev

AtAtPrevPrev % Corr% Corr

DisabledDisabled 2 12 0 3 12%

UnemployedUnemployed 1 48 0 47 50%

Below PreviousBelow Previous 0 9 0 23 0%

At PreviousAt Previous 0 25 0 112 82%


Return to Work: SummaryReturn to Work: Summary

OTBM differences between groups Disabled /Unemployed not able to

separate Below/At Previous not able to separate Collapsed groups result in 71% correct

above base rate of 52% correct


Return to Work: Domain AnalysisReturn to Work: Domain Analysis

Executive function not the most predictive Most of variance carried by Perceptual

Organization & Working Memory

Using Cognitive Domains OTBM increases % Correct from 71% to 74%

Incremental validity of PCRS very low. 7% of the variance


Return to Work: Domain AnalysisReturn to Work: Domain Analysis

By including premorbid variables, increases diagnostic accuracy; most helpful being: Premorbid IQ, level of occupation, & education

Including acute measures increases accuracy; most helpful being: LOC group Time since injury


Depression Study ConclusionsDepression Study Conclusions

Memory complaints not synonymous with impairment in compensation sample Findings replicated

Effort accounts for more variance in self-ratings of cognition & objective performance than mood Findings replicated


0

10

20

30

40

50

60

70

80

90

100

Cell

Mean

SV

EP

MC

EP

GA

OT

BM

DT

BM

ITB

M

VC

PO

EF

AM

L

VM

L

AW PS

PM

What’s wrong with this patient-1? (Key: RCPS)


RIM Summary Statistics: T-scores, variances, ES, & power.

Column # 1 2 3 4 5 6 7 8 9 10 11 12 R o w


Nec. 1-sample

t test ANOVA

s & w Power (1-b)

1 Symptom Validity (SV) 31.7 23.1 10 <.0001 Mld-Mod 50% -.90 + 12.0 43.7 .03381 --- ---

2 Emotional Personality (EP) 40.9 12.1 11 --- Low Avg 55% -.64 + 5.9 46.8 .03191 --- ---

3 Meta-Cognition (MC) 20.3 12.1 9 --- Mod-Sev 100% --- + 6.6 26.9 < .0001 --- ---


5 Overall TBM (OTBM) 42.1 10.3 50 --- Low Avg 42% -.85 + 2.5 44.6 <.00013 .0100 ---

6 Domain TBM (DTBM) 43.7 6.8 7 --- Low Avg 43% -1.13 + 4.2 47.9 .03943 --- ---

7 Instrument TBM (ITBM) 37.7 12.7 12 --- Mild 50% -1.20 + 6.1 43.8 .00513 --- ---

8 Verbal Comprehension (VC) 38.1 5.0 5 --- Mild 60% -2.45 + 3.6 41.7 .00503 W ---

9 Perceptual Organization (PO) 54.8 2.2 4 --- Average 0% .99 + 1.8 56.6 --- S ---

10 Executive Functioning (EF) 37.3 10.1 12 --- Mild 58% -1.52 + 4.8 42.1 .00093 W ---

11 Aud. Mem. & Learn (AML) 37.4 6.1 9 --- Mild 67% -2.30 + 3.3 40.7 .00013 W ---

12 Visual Mem. & Learn (VML) 45.1 2.6 5 --- Average 0% -1.26 + 2.0 47.1 .01023 --- ---

13 Attent/Work Mem. (AW) 45.2 12.9 10 --- Average 30% -.50 + 5.8 51.9 --- ---

14 Processing Speed (PS) 48.4 11.8 5 --- Average 40% -.25 + 8.7 57.1 --- ---


16 PsychoMotor (PM) 42.2 8.3 7 --- Low Avg 43% -.85 + 5.1 47.3 .03783 --- ---

17 Language/Aphasia (LA) --- --- --- --- --- --- --- --- --- --- --- ---

18 Pain Complaints (PC) 33.4 47.2 2 < .0001 Mld-Mod 50% -.49 + 54.9 88.3 --- --- ---

What’s wrong with this patient-1? (Key: RCPS)


0

10

20

30

40

50

60

70

80

90

100

Cell

Mean

SV

-T

EP-T

EPG

A-T

OTB

M-T

DTB

M-T

ITB

M-T

VC

-T

PO

-T

EF-T

ML-T

AW

-T

PS

-T

Cell Point ChartError Bars: 90% Confidence Interval

What’s wrong with this patient-2? (Key: JSVD)


Column # 1 2 3 4 5 6 7 8 9 10 11 12 R o w


Nec. 1-sample

t test anova s & w

Power (1-b)

1 Symptom Validity (SV) 49.1 8.2 4 --- Average 0% -.09 + 9.6 58.7 --- --- ---

2 Emotional Personality (EP) 46.5 9.5 9 --- Average 33% -.35 + 5.9 52.4 --- --- .42

3 Est. Pre. Gen. Ability (EPGA) 62.0 6.0 7 --- Excel 0% 1.20 + 4.5 57.5 .002 --- ---

4 Overall TBM (OTBM) 50.9 15.4 57 <.0001 Average 18% -1.10 + 3.4 54.3 <.0001 --- ---

5 Domain TBM (DTBM) 53.5 7.9 6 --- Average 0% -.85 + 6.5 60.0 .05 .02 ---

6 Instrument TBM (ITBM) 49.6 11.3 14 --- Average 14% -1.24 + 5.3 54.9 .001 --- ---

7 Verbal Comprehension (VC) 62.3 4.0 6 --- Excel 0% .03 + 3.3 65.6 --- S ---

8 Perceptual Organization (PO) 55.2 5.6 6 --- Ab Avg 0% -.68 + 4.6 59.8 .05 --- ---

9 Executive Functioning (EF) 45.5 7.4 14 --- Average 21% -.65 + 4.5 49.0 <.0001 W ---

10 Memory & Learning (ML) 44.7 22.6 15 <.0001 Blw Avg 33% -1.73 + 10.3 55.0 .01 W ---

11 Attention/Working Mem (AW) 63.1 17.3 7 .0132 Excel 14% .11 + 12.7 75.8 --- S ---

12 Processing Speed (PS) 49.6 7.5 9 --- Average 11% -1.24 + 4.7 54.3 .001 --- ---

13 Global/Miscellaneous (GM) --- --- 0 --- --- --- --- --- --- --- --- ---

14 Language/Aphasia (LA) --- --- 0 --- --- --- --- --- --- --- --- ---

15 Sensory Perception (SP) --- --- 0 --- --- --- --- --- --- --- --- ---

What’s wrong with this patient-2? (Key: JSVD)


0

10

20

30

40

50

60

70

80

90

100

Cell

Me

an

SV EP EPGA OTBM DTBM ITBM VC PO EF AML VML AW PS PM

What’s wrong with this patient-3? (Key: NPAD)



Column # 1 2 3 4 5 6 7 8 9 10 11 12 R o w


Nec. 1-sample

t test

ANOVA

s & w

Power (1-b)

1 Symptom Validity (SV) 13.3 18.6 8 .0021 Severe 88% -2.46 + 10.0 23.3 .0008 --- ---

2 Emotional Personality (EP) 20.8 19.5 8 .0008 Moderate 75% -1.88 + 10.5 31.3 .0038 --- ---


4 Est. Pre. Gen. Ability (EPGA) 41.5 3.1 5 --- Blw Avg 60% --- + 2.3 39.2 --- .0576 ---

5 Overall TBM (OTBM) 40.2 12.3 42 .0364 Blw Avg 55% -.11 + 3.1 43.3 --- --- .1727

6 Domain TBM (DTBM) 39.9 6.8 7 --- Mild 57% -.28 + 3.3 43.2 --- --- .1623

7 Instrument TBM (ITBM) 38.4 16.4 10 .0079 Mild 60% -.23 + 8.6 47.0 --- --- .1657

8 Verbal Comprehension (VC) 35.6 1.9 3 --- Mild 100% -2.14 + 1.9 37.5 .0332 --- ---

9 Perceptual Organization (PO) 47.8 13.9 3 --- Average 33% --- + 13.3 61.1 --- --- ---

10 Executive Functioning (EF) 46.9 13.4 12 --- Average 33% --- + 6.5 53.4 --- S ---

11 Auditory Mem. & Learn (AML) 34.8 10.6 9 --- Mld-Md 78% -.76 + 5.8 40.6 --- --- ---

12 Visual Mem. & Learn (VML) 39.2 3.2 4 --- Mild 50% -.73 + 2.6 41.8 --- --- .3074

13 Attention/Working Mem (AW) 45.0 9.0 5 --- Average 20% --- + 6.6 51.6 --- --- ---

14 Processing Speed (PS) 30.3 13.0 6 --- Mld-Md 83% -1.13 + 8.8 39.1 --- W ---


16 PsychoMotor (PM) 14.0 2.8 2 --- Severe 100% -4.90 + 3.4 17.4 .0353 --- ---

17 Language/Aphasia (LA) --- --- --- --- --- --- --- --- --- --- --- ---


What’s wrong with this patient-3? (Key: NPAD)


0

10

20

30

40

50

60

70

80

90

100

Ce

ll M

ea

n

SV

-T

EP

-T

EP

GA

-T

OT

BM

-T

DT

BM

-T

ITB

M-T

VC

-T

PO

-T

EF

-T

ML

-T

AW

-T

PS

-T

GM

-T

LA

-T

Ce ll Point Chart: Error Bars 90% Confide nce Inte rval

What’s wrong with this patient-4? (Key: SMAA)



Column # 1 2 3 4 5 6 7 8 9 10 11 12 R o w

Cognitive Domain M sd n Hetero. p value

Classify % TI ES 80% CI

PreM Nec.

1-sample t test

anova s & w

Power (1-b)

1 Symptom Validity (SV) 53.5 0.6 5 --- Abv Avg 0% .35 .5 54.0 --- --- --- 2 Emotional Personality (EP) 43.3 4.0 10 .0001 Low Avg % -.67 6.0 49.3 --- --- --- 3 Est. Pre. Gen. Ability (EPGA) 58.6 2.7 6 --- Abv Avg 0% .86 1.7 56.9 --- --- --- 4 Overall TBM (OTBM) 50.4 10.8 86 .0001 Average 14% -.80 1.8 52.2 .0001 --- --- 5 Domain TBM (DTBM) 47.9 5.3 7 --- Average 0% -.99 3.0 50.9 .0019 --- --- 6 Instrument TBM (ITBM) 46.6 8.9 20 --- Average 25% -1.11 3.0 49.9 .0009 --- --- 7 Verbal Comprehension (VC) 51.3 6.9 5 --- Average 0% -.68 4.7 56.6 .0785 --- --- 8 Perceptual Organization (PO) 44.6 8.6 8 --- Low Avg 13% -1.30 4.5 49.1 .0024 --- --- 9 Executive Function (EF) 46.8 3.8 10 --- Average 0% -1.09 1.8 48.6 .0001 --- ---

10 Memory & Learning (ML) 57.8 11.7 29 --- High Avg 7% -.07 3.3 61.1 --- S --- 11 Attention/Working Mem (AW) 48.2 9.3 16 --- Average 25% -.96 3.5 51.7 .0004 --- --- 12 Processing Speed (PS) 45.4 9.4 14 --- Average 36% -1.22 3.8 49.2 .0002 --- --- 13 Global/Miscellaneous (GM) 41.3 2.3 3 --- Low Avg 0% -1.60 2.0 43.3 .0059 --- --- 14 Language/Aphasia (LA) 38.5 --- 1 --- Mild 100% -1.86 --- --- --- --- --- 15 Sensory Perception (SP) --- --- --- --- --- --- --- --- --- --- --- ---

What’s wrong with this patient-4? (Key: SMAA)


Rohling’s Interpretive Method: Use Rohling’s Interpretive Method: Use of Meta-Analytic Procedures for of Meta-Analytic Procedures for

Single Case Data AnalysisSingle Case Data Analysis

Martin L. RohlingQuestions & Comments Welcome!Questions & Comments Welcome!


CT/MRI Data

Participant Demographic Information Variable Sample Sizes (N = 124) Gender

Male 82 Female 42

Ethnicity Caucasian 119 Other 5


CT/MRI

Diagnostic Groups Sample Size MVA/TBI 47 Blow to Head 32 LCVA 24 RCVA 21


CT/MRI

1. All were Right Handed.

2. All were followed by Dr. Meyers through hospitalization and rehabilitation.

3. None were involved in litigation.

4. All passed internal validity checks.


CT/MRI

CT/MRI Location Left Frontal 59 Left Parietal 37 Left Temporal 34 Left Occipital 6 Right Frontal 40 Right Parietal 42 Right Temporal 31 Right Occipital 3


CT/MRI

All were given MNB CT/MRI data coded for injury reported on

MRI/CT at the time of injury Present = 1 Absent = 0


CT/MRI

Independent Sample 1-tailed t-test on each lobe

On CT/MRI report Present = 1 Absent = 0


CT/MRI Data


Brain Regions Involved in the Performance of WAIS-III Arithmetic


Brain Regions Involved in the Performance of the Boston Naming Test


Brain Regions Involved in the Performance of the Rey-CFT Copy


Brain Regions Involved in the Performance of the AVLT Total Score


CT/MRI

NP tests generally behaved as expected A more “Systemic” or “Domain” like

approach better at explaining results Construct of “Executive Function” not

supported.


Domains used by the MNB

Attention/Working Memory:

Digit Span Forced Choice Animal Naming Sentence Rep AVLT 1

Processing Speed/Mental Flexibility:

Digit Symbol Dichotic Both Trails A Trails B



Verbal Reasoning Similarities Arithmetic Information COWA Dichotic Left Dichotic Right Boston Naming Token Test

Visual Reasoning Picture Completion Block Design JOL Category RCFT Copy



Verbal Memory AVLT Total AVLT Immediate AVLT Delayed AVLT Recognition

Visual Memory RCFT Immediate RCFT Delayed RCFT Recognition



Motor and Sensory Finger Tapping Dominant Hand Finger Tapping Non-Dominant Hand Finger Localization Dominant Hand Finger Localization Non-Dominant Hand


Commonality of Reduced O2


Domain Consistency

N = 936 Passed all validity checks No missing data Not involved in litigation

Calculated Domain M’s Regression used to predict Domain M’s

using all on other Domain M’s


Domain Means Correlations1 2 3 4 5 6

1 – Premorbid .76 .71 .62 .56 .79 2 - OTBM .76 .98 .81 .82 .84 3 - DTBM .71 .98 .77 .79 .78 4 - Attent/Work Mem .64 .81 .77 .64 .69 5 – Pro Spd/Mental Flex .62 .82 .79 .64 .72 6 - Verbal Reason .79 .84 .78 .69 .72 7 - Visual Reason .68 .81 .81 .54 .64 .64 8 - Verbal Memory .53 .77 .78 .68 .50 .54 9 - Visual Memory .54 .77 .80 .53 .55 .5510 - Dom Motor/Sensory .30 .54 .62 .37 .44 .3611 - Nond Motor/Sensory .28 .53 .62 .31 .44 .30

All were Significant p < .001


Domain M’s Correlations (cont.)

7 8 9 10 11 1 - Premorbid .68 .53 .54 .30 .28 2 - OTBM .81 .77 .77 .54 .53 3 - DTBM .81 .78 .80 .62 .62 4 - Attent/Work Mem .54 .68 .53 .37 .31 5 - ProcSpd/Ment Flex .64 .50 .55 .44 .44 6 - Verbal Reasoning .64 .54 .55 .36 .30 7 - Visual Reasoning .51 .70 .41 .45 8 - Verbal Memory .51 .62 .34 .32 9 - Visual Memory .70 .62 .37 .40 10 - Dom Motor/Sen.41 .34 .37 .53 11 - Nond Motor/Sen .45 .32 .40 .53

All were Significant p < .001


Domains Regression Equations

Attention & Working Memory (Verbal Reasoning) * .315 (Verbal Memory) * .273 (Processing Speed) * .193 Constant = 10.972



Processing Speed/ Mental Flexibility Verbal Reasoning * .401 Visual Reasoning * .284 Attention & Working Memory * .230 Constant = 2.434



Verbal Reasoning Processing Speed * .361 Attention & Working Memory * .354 Visual Reasoning * .243 Constant = 2.5



Visual Reasoning Visual Memory * .322 Processing Speed/Mental Flexibility * .213 Verbal Reasoning * .208 Constant = 11.813



Verbal Memory Attention & Working Memory * .738 Visual Memory * .388 Constant = -7.615



Visual Memory Visual Reasoning * .698 Verbal Memory * .311 Processing Speed * .0909 Constant = -5.517


Regression

Adjusted SER R2 of the Estimate

Attent/Working Memory .79 .63 4.88 Processing Speed .77 .60 5.31 Verbal Reasoning .80 .64 5.04 Visual Reasoning .78 .61 4.88 Verbal Memory .75 .56 7.96 Visual Memory .77 .59 7.11


Review

Took a battery of well known tests Developed Norms Identified Validity, Reliability, Sensitivity

and Specificity. Internal Validity Checks and Internal

Consistency

Rohlings Interpretive Method: How Can a Flexible Battery Perform Like a Fixed Battery Martin L....

Documents

Transcript of Rohlings Interpretive Method: How Can a Flexible Battery Perform Like a Fixed Battery Martin L....