Testing Dose-Response with Multivariate Ordinal Data

Bernhard KlingenbergAsst. Prof. of StatisticsWilliams College, MA

Paper available at www.williams.edu/~bklingen

In Collaboration with Aldo Solari, Luigi Salmaso and Fortunato Pesarin,

University of Padova

OutlineIntroduction

• Safety and Toxicity Data

• Notation and hypothesis of interest

• Stochastic Ordering

• Theorem (SMH IJD)

Testing SMH• Simple Test statistics

• Permutation Approach

• Step-down methods for indiv. endpoint significance

• Increase power

Example • Parallel, 5 dose group study with rats (8 rats per dose

group) • 25 Adverse Events from exposure to Perchlorethylene

Introduction: Safety and ToxicitySafety and Toxicity Data:

To capture large number of possible manifestations of a dose (exposure) effect on safety or toxicity: Multiple Endpoints

One such collection of endpoints to evaluate neurophysiological effects: Functional Observational Battery (FOB)

Others: Drug Safety, Disease progression

Introduction: FOBGoal: Evaluation of neurophysiological effects to a

toxin (Perchlorethylene)

Data1: 2 groups (No exposure vs. 1.5g/kg exposure) 8 rats in each group Each evaluated at 25 endpoints (various

effects), grouped into 6 domains Response ordinal, on a scale from 1 (no effect)

to 4 (most severe reaction)

1 Moser (1986) Journal of the American College of Toxicology

Introduction: FOB

Introduction: Notationk-dimensional response vectors:

Control Treatment

Random Sample

Control Treatment

Hypothesis of interest: “No dose effect”

),...,( 111 kYY1Y ),...,( 221 kYY2Y

11n11 YY ,,22n21 YY ,,

21 Y Y :H0 12 Y Y :H 1vs.d st

Introduction: No Toxicity“ ”: For all response sequences

Control Treatment

“ “: Stochastically larger 2

Control Treatment

Note: Rejection of H0 does not lead to H1

d ),,( 1 kjj

),,Pr( ),,Pr( 21211111 kkkk jYjYjYjY

st

),,Pr( ),,Pr( 21211111 kkkk jYjYjYjY

),,Pr( ),,Pr( 21211111 kkkk jYjYjYjY

2 Marschall & Olkin (1979) Inequalities: Theory of Majorization and Its Applications

SMHUsually only interested if k margins are equal or not.

I.e., for each adverse event ,

Def.: Simultaneous Marginal Homogeneity (SMH) 3:

hhhhhh jjYjYH all for )Pr( )Pr(: 210

k

hh

d

h

k

h

h YYH1

211

0

Vector of marginal probabilities are equal under the two exposures, for all adverse events

simultaneously

:,,1 kh h

3 Agresti and Klingenberg (2005) JRSS C, Klingenberg and Agresti (2006), Biometrics

SMHSMH with just two adverse events

Control Treatment

1 2 3 4

1

2

…

3

4

Lacrimation

Aro

usal

Lacrimation

Aro

usal

1 2 3 4

1

2

…

3

4

SMHTheorem:

Prior assumption plausible when dealing with adverse events data (increase in exposure shift towards higher outcome categories)

IJD SMH

Cumulative marginal inhomogeneity: )Pr( )Pr( 22 hhhh jYjY

Testing SMHConsequence of Theorem: If prior assumption

plausible, can use permutation approach to test hypothesis of SMH

Test for SMH: Modeling approach via cumulative logits (proportional odds form) 4

Estimation (ML, conditional ML, GEE,…) computationally impossible, Asymptotics invalid

3

4 Han, Catalano, Senchaudhuri, Metha (2004) Exact Analysis of Dose Response for Multiple Correlated Binary Outcomes, Biometrics.

Testing SMHLet

Simple test statistic: Standardized differences in marginal sample proportions

given by (from multinomial assumption):

Testing SMHTo take advantage of ordinal nature: Consider

scoring function

Let be score matrix

Look at difference in mean scores:

Estimate covariance matrix

4

under SMH assuming working independence

Testing SMHTest statistic for sparse data, ignoring correlation

among adverse events:

with

This gives global test of safety/toxicity

Permutation approach: 16!/(8!8!) = 12870 possible permutations, many leading to identical values of

Advantage of permutation approach: Incorporates dependence by resampling entire vectors; exact significance levels

Testing SMHExample: Arousal Endpoint

61.1

Computation with equally spaced scores:

292.82 ,2 ,3,1 33 ,2 ,1,0 t

03.0

01.005.0

01.001.004.0

01.003.002.006.0

Note:

Testing SMHPermutation Distribution:

observed

Perm. Distr.

Asympt. Distr.

Testing SMHIdentifying which individual adverse events are

significant leads to multiple hypotheses testing:

Use test statistic (standardized mean score difference) for individual tests

Multiplicity adjustments via step-down approach of Westfall &Young (1993), using distribution of maximum test statistic

khjjYjYH hhhhhh ,,1, all for )Pr( )Pr(: 210

Testing SMHPermutation Distribution:

Observed maximum

Perm. Distr.

Testing SMH

Testing SMHHow sensitive are results to assigned scores?

Consider the scores that maximize (obtainable via isotonic regression; data-driven)

Appropriate for safety/toxicity data; maximizes the contrast btw. the mean score differences

)( hhz

14.2)(

),1 ,64.0 ,0 ,0(max

max

hh

h

z

Equally spaced scores: 61.1hz

With

maxh

Testing SMH

Testing DomainsDomain effects?

Some endpoints may measure similar effects

Multiplicity adjustment at the endpoint level may be too conservative, leading to some false negatives

Adjusted P-value for domain less than or equal to smallest adjusted P-value within domain

“Proof”: Let endpoint h be in the first domain Dom1:

Testing DomainsImportant Consequence (Robustness Property):

Consonant domain test statistic:

Reject only (at domain level) if at least one endpoint within domain significant

If no significant endpoint, domain also not significant

For domain significance, it is irrelevant how many, potentially

non-significant endpoints are grouped into a domain! *

* Provided the same test statistic is used for all intersection hypotheses

Testing DomainsDissonant domain test statistic:

Accumulate effects over endpoints within domain

Even though no individual endpoint is significant, several marginally significant ones can result in significant domain P-value

SummaryTesting dose-response for multivariate ordinal data

Correlated ordinal responses (typical for toxicity or safety data) are often sparse and imbalanced use permutation approach

Instead of modeling dose-response, we focused on testing SMH vs. stochastic ordering

SMH IJD, but SMH IJD under

Test statistic: zh= (difference in mean scores) / s.e., for

each endpoint, assuming working independence s.e. derived from multinomial model and estimated under

SMH Multiplicity adjusted P-values for each endpoint from

Westfall and Young’s step-down procedure

12 Y Y

st

Conclusions for FOBNeuromuscular domain showed significant effect

Domain P-value 0.003 with dissonant test Domain P-value 0.025 with consonant test

Adverse events in neuromuscular domain that show increased toxicity at the 1.5 g/kg exposure level when compared to control

Sensorimotor domain also shows increased level of toxicity, although no individual adverse event is significant

Gait (p= 0.025)

Hindelimb (p=0.044)

Forelimb (p=0.096)

Did not showHow methods extend to several dose levelsEffect of discreteness (used mid P-values

throughout)Combining P-values instead of test statistics, with

functions other than the maximum

Thank you and Go Gators