Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1,...
Transcript of Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1,...
![Page 1: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/1.jpg)
Clinical trial design and crossClinical trial design and cross--cultural cultural issues for issues for PROsPROs
Linda AbetzMapi Values
![Page 2: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/2.jpg)
ObjectivesObjectives
• Trial design issues and potential solutions
• Cross-cultural issues and potential solutions
![Page 3: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/3.jpg)
List of AbbreviationsList of Abbreviations
QoL-Quality of LifeHRQL=Health-related quality of lifePRO=Patient reported outcomesQALY=Quality Adjusted Life YearsMD=Missing dataN=Sample SizeBL=BaselineW1,W2,W3, etc=Week 1, Week, 2, Week 3, etc
![Page 4: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/4.jpg)
Progress in Defining OutcomesProgress in Defining Outcomes
QoL
HRQL““Represents the patient’s Represents the patient’s evaluation of the impact of a evaluation of the impact of a health condition and its treatment health condition and its treatment on relevant aspects of life”on relevant aspects of life”“A multidimensional construct”“A multidimensional construct”
PRO “The patient’s report of “The patient’s report of a health condition and a health condition and its treatment”its treatment”
Implications?
© MAPI VALUES, April 2003
![Page 5: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/5.jpg)
Patient ReportedPatient Reported Outcomes: Outcomes: Concepts?Concepts?
Health-relatedQuality of Life
HRQL
Productivity
QALYs Activity of daily living
Symptoms
Discomfort
Bother
Cost
From Laurie Burke, Evidence Review Branch, DDMAC, FDA
Satisfaction
![Page 6: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/6.jpg)
Reasons for ConductingReasons for ConductingPRO StudiesPRO Studies
Improved PRO benefits over alternative treatment may assist regulatory strategiesDemonstrates value in patient outcomes for treatment expendituresProvides clinicians and patients with more complete product information
![Page 7: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/7.jpg)
Steps in the development of PRO Steps in the development of PRO strategystrategy
Hypothetical basis for PRO as outcome
Study design preliminary protocol
Patient interviews
Desk researchLiterature review
Discussion of HRQoL concepts with clinical team
Desktop evaluation of available instruments
Selection of instrument(sApproval by authors of instruments
Further validation work?Translations & Psychometric validation
Development of a new questionnaire
ValidationTranslations & Psychometric validation
Integration in the trial protocol
![Page 8: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/8.jpg)
An ‘Integrated’ approachAn ‘Integrated’ approachincorporating the design and validation of incorporating the design and validation of
PRO measures into overall clinical PRO measures into overall clinical development activitiesdevelopment activities
Phase IPhase III
Phase III PRO Program- sub-study- stand alone
Phase IPhase II
Instrument Validation- cross sectional study- longitudinal study- phase III feasibility (“Go/No-Go”) and
strategy- publication strategy
Instrument Development- literature review- patient interviews- expert opinions/inputs- clinical hypotheses
Phase IClinicalDevelopmentProgram
PROProgram
![Page 9: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/9.jpg)
Clinical Trial DesignClinical Trial Design
How it can impact results
![Page 10: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/10.jpg)
Things to considerThings to consider
PRO hypothesesChosen questionnairesModes of administrationTiming of assessmentsTraining of investigatorsIncorporation into protocolDevelopment of analysis plan
![Page 11: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/11.jpg)
PRO hypothesesPRO hypotheses
PRO domains impacted by diseasePRO domains impacted by producto positive/negative effectso equivalence
Keep in mind the inclusion/exclusion criteriaHypotheses will aid in the selection of an appropriate questionnaire
![Page 12: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/12.jpg)
Chosen patientChosen patient--reported reported questionnaires should bequestionnaires should be
Relevant to the patient trial population Clear and easy to understandReliable, valid and responsive Have a relevant recall period Available in local languages (or can be translated appropriately)Easy to interpret
![Page 13: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/13.jpg)
Developing or Adapting an Developing or Adapting an instrument: Prerequisitesinstrument: Prerequisites
Relevance of quality of life to clinical scenarioAbsence of existing valid and appropriate measuresFeasibility/favourable logistics of qualitative research
![Page 14: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/14.jpg)
Psychometric AnalysisPsychometric AnalysisAcceptability missing data
descriptive statisticsScaling assumption item-item correlations
item-scale correlationsfactorial/multitrait analysis
Reliability reproducibilityinternal consistency (Cronbach’s alpha)
Validity content validitycriterion/construct validityclinical validity
Sensitivity discriminative powerresponsiveness over time
![Page 15: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/15.jpg)
Modes of AdministrationModes of AdministrationRespondento Patiento Proxy (spouse, carer, parent, clinician, nurse)
Type of questionnaireo Self-administered
• Paper/pen (self-administered)• Palm-pilot and other computer administered
o Interview-administered• By phone• Face-to-face
![Page 16: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/16.jpg)
Timing of assessmentsTiming of assessmentsDon’t expect QoL improvement if there is no clinical improvement or if major AEIs trial long enough to see PRO improvement?Timing of when benefits and AE occur should be taken into accounto If AE occur in W4 and dissipate followed by benefits by W6,
when should you assess?Recall period in questionnaire vs timing of assessmentso Avoid overlap: BL, 2W, 4W - 4W recall…
![Page 17: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/17.jpg)
Training of investigatorsTraining of investigators
Essential to train investigators and CROs on: o importance of PRO data captureo how questionnaire was selected/developed and what
it containso how to administer the questionnaire and what it will
be used forWithout their buy-in, risk of missing data is high
![Page 18: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/18.jpg)
Collection and Management of Collection and Management of PRO in clinical trialsPRO in clinical trials
No queriesQuality checks on identification between questionnaires and CRFs only
![Page 19: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/19.jpg)
Implementing PRO into the Implementing PRO into the protocolprotocol
Introduction & justification of choice of the questionnaire(s)Objectives: Primary or secondary?Timing of assessments-CRUCIAL
• Based on clinical, symptom and side effect hypotheses: don’t expect improvements if no clinical or symptom improvements are expected or if major side effects are expected
• Is the trial long enough to detect PRO improvements?• At a minimum, baseline and endpoint, though one mid-point is
recommended
Analysis plan (usually very brief/provided as separate)Integrate into investigator/monitor meetingo 15-20 minutes of instructions on how to implement questionnaires
![Page 20: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/20.jpg)
Analysis of Analysis of PROsPROs
![Page 21: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/21.jpg)
Main Issues to considerMain Issues to consider
Power calculationsWrite analysis plan prior to analysisGet scoring documentation ASAPTreatment of missing datao item levelo questionnaire level
Multiple testingSize of differences in scores over timeo not just statistical significance
![Page 22: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/22.jpg)
What do I need to know to What do I need to know to analyse results?analyse results?
What are the PRO hypotheses? What type of questionnaires did we choose? o What are the scoring methods?o Are there reference scores available to help in interpretation?
Is the questionnaire psychometrically valid in our clinical trial population?What type of missing data do we have (by treatment)? o How will we handle missing data?
How are we controlling for multiple endpoints/tests?What were the clinical results?o Do your results make sense in line with clinical findings?
![Page 23: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/23.jpg)
Cultural adaptation of PRO Cultural adaptation of PRO QuestionnairesQuestionnaires
![Page 24: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/24.jpg)
CrossCross--cultural Adaptationcultural Adaptation
1st intermediary version in target language
Translation back to source language and
comparison with original
2nd intermediary version intarget language
Test by patients or lay people
3rd intermediary version
Originator
Sourcequestionnaire
Production of final versions in each
languageInternational harmonisation
Report of quality rating and rationale for choice of translation
Forward translation to target language
Forward translation totarget language
![Page 25: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/25.jpg)
Proposed check list for Proposed check list for psychometricspsychometrics
Mandatory for the original version
Construct Validity Internal consistency relia.Score distributionClinical validity(Concurrent validity)Test-retest reliability(Responsiveness over time)
For translated version Mandatory Recommended
+++
++
(+) +(+)
![Page 26: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/26.jpg)
Pooling DataPooling Data
![Page 27: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/27.jpg)
Patient ReportedPatient Reported Outcomes (PRO) Outcomes (PRO) questionnaires in international trials :questionnaires in international trials :
key questionskey questionsAre the different linguistic versions conceptually and linguistically equivalent and are they applicable to each target country?Can I analyse treatment effect on PRO using pooled data?How cultural differences impact analyses?How can one address the appearance of a strong “response signal” in one region and the absence of any signal in another region within the same multinational trial?What are Health authorities requirements?What is accepted?
![Page 28: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/28.jpg)
Available methods to assess Available methods to assess structural validitystructural validity
“Classical” psychometric methodso Principal Components Analysis (with factor rotation)o Multi-trait analyses (based on item-scale correlations)
Modern psychometric methodso Confirmatory Factor Analysis / Structural equation
modeling o Rasch model / Item Response Theory (probabilistic
approach)
![Page 29: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/29.jpg)
Indicators of satisfaction with medication Indicators of satisfaction with medication and intention to comply with treatmentand intention to comply with treatment
![Page 30: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/30.jpg)
Analysis of international CT Analysis of international CT PRO data: Recommendations PRO data: Recommendations o Describe acceptability (MD) and score distribution at
country levelo Test structural validity at country/language version level
using ad-hoc or baseline CT data (prior unblinding)o Address cross-cultural equivalence: Rasch and CFAo Assess criterion (clinical/known groups/concurrent/
predictive) validity and responsiveness to change over time at country level
• may be problematic due to small sample sizes
![Page 31: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/31.jpg)
Analysis of international CT Analysis of international CT PRO data: RecommendationsPRO data: Recommendations
o Analyse treatment effect on PRO on pooled data if at least structural validity in each language version is satisfactory.
• If not, analysis at country level should be considered.o The analysis plan (finalised prior unblinding) should
describe how data are going to be analysed and specify the decision rules to include (or not) a country in the pooled analysis
![Page 32: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/32.jpg)
Prerequisites for a successful Prerequisites for a successful international trial using a PRO international trial using a PRO
measuremeasure
• Clearly identifiable original instruments• Cross-culturally acceptable concepts/Cross-
culturally valid language versions• Psychometrically comparable data• Consistent understanding of PRO evaluation
by users (drug companies, regulatoryauthorities, clinicians)
![Page 33: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/33.jpg)
Criteria for successCriteria for success
Choose a specific instrumentSet up clear PRO hypothesesInvolve the clinical/marketing teams in the definition of PRO assessment strategyIntegrate clinical and QoL monitoringRelevant timing of assessments and relevant reporters/modesSet your objective and stick to it when publishing the results
![Page 34: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/34.jpg)
Concluding Remarks (1)Concluding Remarks (1)o Start to think about PRO early in the drug development
process in order to budget resources (time/money). o Go to the patients to find out if PRO is affected before
deciding to assess PRO. Go to clinicians to find out if patients’ PRO will sway treatment decisions.
o Assess product/competitor profiles and trial protocol when making decisions
o Assess available instruments for reliability, validity, responsiveness. If acceptable, try to use an available instrument to maximize resources.
o A generic/disease specific approach is often the best method to
![Page 35: Clinical trial design and cross-cultural issues for PROs · 2007. 1. 17. · W1,W2,W3, etc=Week 1, Week, 2, Week 3, etc. ... Validity content validity. ... o Assess available instruments](https://reader036.fdocuments.in/reader036/viewer/2022071016/5fcf4a19c53e3b4321021852/html5/thumbnails/35.jpg)
Concluding remarks (2)Concluding remarks (2)
o Ascertain comprehensiveness, responsiveness & interpretability.
o Keep interpretation in mind when designing the assessment strategy
o Remember audience needs when reporting results.