Post on 21-May-2020
Validity of Self-reported Periodontal disease:
a systematic review
Research protocol
July 2015
Hadeel M. Abbood1
Dr. Tatiana V. Macfarlane1
Dr. George Cherukara1
1 University of Aberdeen, Aberdeen Dental School and Hospital, AB25 2ZD
2
Table of contents
Introduction .............................................................................................................................. 3
Rationale: Why it is important ............................................................................................... 3
Aim ............................................................................................................................................ 4
Objectives.................................................................................................................................. 4
Method ...................................................................................................................................... 4
STUDY SELECTION ................................................................................................................................................... 4 SEARCH STRATEGY ................................................................................................................................................... 4 INCLUSION AND EXCLUSION CRITERIA .......................................................................................................................... 4 DATA EXTRACTION .................................................................................................................................................. 5 QUALITY ASSESSMENT .............................................................................................................................................. 5
Milestone ................................................................................................................................... 6
References:................................................................................................................................ 7
Table of appendixes
Appendix 1 Map of searching terms in MEDLINE ................................................................... 9 Appendix 2 Flow diagram of study selection process ............................................................. 10
Appendix3 Data extraction form ............................................................................................ 11 Appendix 4 Quality assessment tool (AMSTAR) ................................................................... 12
Appendix 5 Methodology Checklist 5: Studies of Diagnostic Accuracy ................................ 14
3
Introduction
Periodontal disease is a group of diseases of the periodontal tissues that lead to attachment
loss and alveolar bone destruction. The prognosis of periodontal disease in some but not all
patients may results in tooth loss, especially if there is no treatment (Löe et al. 1986).
Diagnosis depends on the findings from the history and clinical examination of the patient
(Clerehugh, Tugnait & Genco 2013), which can includes several separate indices. These
indices include Gingival Index (GI), Plaque Index (PLI), bleeding on probing (BOP), loss of
attachment (LOA), furcation involvement, and tooth mobility. The information gathered will
allow the clinician to decide on a logical basis whether additional special tests are required,
such as radiographic examination and tests to detect biomarkers in the saliva and the gingival
crevicular fluid. (Chapple, Gilbert 2002).
Self-reporting is easy to apply, cost-effective, and provides immediate results. It can be relied
upon for national public health surveys (Tourangeau, Yan 2007). Self-report has gained
popularity as an important method in screening and motivational interviewing (Lundahl,
Burke 2009). Self-rated health assessment and utilization of healthcare services are important
determinants of health, and have particular relevance for public health (Chakraborty et al.
2003). Self-report is a potent, skilled, and accepted means of assessing many diseases, such
as assessing cancer, cardiovascular disease (Newell et al. 1999), and juvenile rheumatoid
arthritis (Wright et al. 1994). It can also be used to assess risk factors for disease, such as diet
(Willett 1990, Rimm et al. 1992), physical activity (Wolf et al. 1994), high blood pressure
(Tormo et al. 2000), and general health (Sheridan, Mulhern & Martin 1998). An example for
self-reported questionnaires survey is the Queensland preventive health surveys that are
designed to make the respondent self-assess their own health (Queensland Government
2015).
The Scottish health survey (SHeS) is another example of self-reported health. It provides a
detailed picture of the health of the Scottish population in private households. It is designed
to make a major contribution to the monitoring of health in Scotland. SHeS also monitor oral
health, especially the periodontal health and the oral hygiene using a self-reported
questionnaire (the Scottish Government 2015).
Rationale: Why it is important
For public health interventions against periodontal disease to be developed, implemented, and
evaluated, an adequate monitoring of the disease is required. Clinical assessment of
periodontal disease is extremely need more resources and facilities, and cannot be used in
several state-based surveillance systems. The use of valid, economical, and low-resource self-
reported measures of periodontal disease would be of great benefit in a variety of ways.
Firstly, it would aid in studying periodontal disease epidemiologically at a larger scale than is
feasible with the present clinical measures, because using surveys can incorporate a larger
population than clinical examination. Secondly, questions regarding periodontal disease
could easily be added to ongoing studies to evaluate associations with other diseases and
conditions. The use of self-report would allow for an easier and low-cost method of obtaining
data for research and would support the evolution of oral health programs (Siegal, Martin &
Kuthy 1988, Kallio 1996). Finally, self-reported measures would allow for surveillance of the
periodontal condition of populations over time, in national, state, or regional surveillance
programs.
4
It has been over 10 years since the last systematic review of the validity of self-reported
periodontal disease was published (Blicher et al 2005). In this time, several studies using self-
reported periodontal disease have been reported.
It is the time now to conduct another systematic review, and further assess the validity of
self-reported periodontal disease. Such a review will update the previous systematic review
by Blicher et al.
Aim
To update the systematic review published in 2005 by Blicher et al. which reviewed literature
up to June (2nd
week) 2004.
Objectives
1. Summarise new findings and combine them with the previous review by Blicher et al.
2. Identify the most effective means of self-reporting of periodontitis.
Method
Study selection
The research aimed to analyse all the papers that assessed the validity of self-reported
gingival and periodontal diseases by comparing with a ‘clinical gold standard’.
Search strategy
MEDLINE search strategy was adopted from 1966-2004 review by Blicher et al. 2005 and
will be updated to include studies until 2015. Embase database will be added to the electronic
search.
The search strategy terms will be grouped into three categories. Group I terms will be related
to gingivitis and periodontal disease (gingivitis, gingival, gingival disease, periodontal,
periodontal disease, periodontitis, tooth mobility, loss of attachment, bleeding gum); group II
will include terms associated with self-reporting (questionnaire, self-assessment, self-report,
self-reported), and group III terms concerning process of validation (comparison, compared,
validity, validation) (see appendix 1). Intra-group terms will be combined with the Boolean
commands ‘OR’, and Inter-group terms will be combined with ‘AND’.
Two journals (Journal of Clinical Periodontology and Periodontology 2000) will be searched
manually from 2005-2015. Publications in languages other than English will not be included.
Inclusion and exclusion criteria
We will include studies in English language only; any other languages will be excluded. The
studies to be included should use the clinical gold standards for each questionnaire.
Studies that used self-report but did not validate these measures or validate them in overall or
in combination of more than one oral health measure will be excluded.
5
Data extraction
After completing the database search and manual search, two independent reviewers will
screen the title and the abstract of each study following the inclusion criteria. If disagreement
occurs between the two reviewers, a third reviewer will be asked consulted. Data extraction
will be done by two independent reviewers. They will extract the required data from each
study. These information include: study type, population description (sample size, mean age,
gender, method of recruitment), question used for self-report, clinical gold standard, and time
of data collection. A standardised data extraction form will be used for data extraction (see
appendix 3). The data extracted will be used to construct a 2*2 table to calculate the
sensitivity and specificity. Predictive values will also be used. If some information is missing
we will contact the authors of the studies to provide us with the required information.
The data from research studies will be used to create tables with group questions asked by
subject and contain information regarding specific phrasing, reference gold standard. Validity
of self-assessment will be quoted as reported by the authors: percentage of agreement,
sensitivity/specificity, p-value, predictive values, correlation/regression coefficient or
descriptive methods as reported by the authors, We will calculate an additional statistics
based on the data provided in the manuscript when needed and possible. Good validity level
will be arbitrarily defined as sum of either: positive and negative values, or specificity and
sensitivity, exceeding or equal to 120%. This value was arbitrary chosen by Blicher et al.
(2005). If a study compared a self-reported question against multiple clinical measures, only
the most relevant one will be chosen.
Quality assessment
The studies will be analysed according to the guidelines for systematic reviews of diagnostic
studies (Devillé et al. 2002). Each publication will be reviewed using specially designed data
collection sheets (see appendix 3) with respect to: population profile, selection criteria, type
of self-reporting, nature of questions, reference clinical gold standard, and results of
validation.
Quality assessment tool (QUADAS-2) will be used to evaluate the risk of bias and
applicability of primary diagnostic accuracy for each study. QUADAS-2 is designed to assess
the quality of primary diagnostic accuracy studies; the tool consists of four key domains
regarding patient selection, index test, reference standard, flow and timing. The tool is
completed in four phases: 1) state the review question; 2) develop review specific guidance;
3) review the published flow diagram for the primary study or construct a flow diagram if
none is reported; 4) judgement of bias and applicability. Each domain is assessed in terms of
the risk of bias and the first three are also assessed in terms of concerns regarding
applicability. To help reach a judgement on the risk of bias, signalling questions are included.
These flag aspects of study design related to the potential for bias and aim to help reviewers
make risk of bias judgments. Risk of bias is judged as “low”, “high”, or “unclear”. If all
signalling questions for a domain are answered “yes” then risk of bias can be judged “low”. If
any signalling question is answered “no” this flags the potential for bias. Review authors then
need to use the guidelines developed in phase 2 to judge risk of bias. The “unclear” category
should be used only when insufficient data are reported to permit a judgment. QUADAS-2 is
available on the link below:
http://www.bris.ac.uk/medialibrary/sites/quadas/migrated/documents/quadas2.pdf
QUADAS-2 is available on the link below:
http://www.bris.ac.uk/medialibrary/sites/quadas/migrated/documents/quadas2.pdf
6
Quality assessment tool for systematic review (AMSTAR) will be used to evaluate the
previous systematic review by Blicher et al. in yes/no fashion (Shea et al. 2007)(see appendix
4).
Study protocol will be registered with PROSPERO http://www.crd.york.ac.uk/PROSPERO/
We aim to publish this review in a peer reviewed journal.
We will follow Prisma guidelines for reporting of systematic reviews http://www.prisma-
statement.org/ .
We will register our study in systematic review data repository (SRDR)
http://www.srdr.ahrq.gov/home/index
Milestone
Protocol finalised - August 2015
Search strategy- August to September 2015
Study selection -September
Data extraction –September - October
Quality assessment-October – November
Summary of results –November
Writing final report -November
Circulation of report for comment –December
Submission for publication –January 2015
7
References:
Blicher, B., Joshipura, K. & Eke, P. 2005, "Validation of self-reported periodontal disease: a
systematic review", Journal of dental research, vol. 84, no. 10, pp. 881-890.
Chakraborty, N., Islam, M.A., Chowdhury, R.I., Bari, W. & Akhter, H.H. 2003,
"Determinants of the use of maternal health services in rural Bangladesh.", Health
Promot Int, vol. 18, no. 4, pp. 327-337.
Chapple, I.L.C. & Gilbert, A.D. 2002, "Understanding Periodontal Diseases: Assessment and
Diagnostic Procedures in Practice." in Quintessence Publishing Co., , pp. 111-128.
Clerehugh, V., Tugnait, A. & Genco, J.R. 2013, Periodontology at a glance, 4th edn, Wiley-
Blackwell.
Devillé, W.L., Buntinx, F., Bouter, L.M., Montori, V.M., de Vet, H.C., van der Windt, D.A.
& Bezemer, P.D. 2002, "Conducting systematic reviews of diagnostic studies: didactic
guidelines.", BMC Med Res Methodol, vol. 3, no. 2, pp. 9.
Kallio, P. 1996, "Self-assessed bleeding in monitoring gingival health among adolescents.",
Community Dent Oral Epidemiol, vol. 24, pp. 128-132.
Löe, H., Anerud, A., Boysen, H. & Morrison, E. 1986, "Natural history of periodontal disease
in man. Rapid, moderate and no loss of attachment in Sri Lankan laborers 14 to 46 years
of age", Journal of Clinical Periodontology, vol. 13, no. 5, pp. 431-445.
Lundahl, B. & Burke, B.L. 2009, "The effectiveness and applicability of motivational
interviewing: A practice-friendly review of four meta-analyses.", Journal of Clinical
Psychology, vol. 65, no. 11, pp. 1232-1245.
Newell, S.A., Girgis, A., Sanson-Fisher, R.W. & Savolainen, N.J. 1999, "The accuracy of
self-reported health behaviors and risk factors relating to cancer and cardiovascular
disease in the general population: a critical review.", American journal of preventive
medicine, vol. 17, pp. 211-229.
Queensland Government 2015, 12 May, 2015-last update, Queensland preventive health
surveys. Available: https://www.health.qld.gov.au/epidemiology/publications/phs-
qld.asp [2015, 07/22].
Rimm, E.B., Giovannucci, E.L., Stampfer, M.J., Colditz, G.A., Litin, L.B. & Willett, W.C.
1992, "Reproducibility and validity of an expanded self-administered semiquantitative
food frequency questionnaire among male health professionals.", American Journal of
Epidemiology, vol. 135, pp. 1114-1126.
Shea, B.J., Grimshaw, J.M., Wells, G.A., Boers, M., Andersson, N., Hamel, C., Porter, A.C.,
Tugwell, P., Moher, D. & Bouter, L.M. 2007, "Development of AMSTAR: a
measurement tool to assess the methodological quality of systematic reviews", BMC
Medical Research Methodology, vol. 7, no. 10.
8
Sheridan, C.L., Mulhern, M. & Martin, D. 1998, "Validation of a self-report measure of
somatic health.", Psychol Rep, , pp. 679-687.
Siegal, M.D., Martin, B. & Kuthy, R.A. 1988, "Usefulness of a local oral health survey in
program development.", J Public Health Dent, vol. 48, pp. 121-124.
the Scottish Government 2015, Wednesday, March 04, 2015-last update, Scottish Health
Survey. Available: http://www.gov.scot/Topics/Statistics/Browse/Health/scottish-health-
survey [2015, 07/22].
Tormo, M.J., Navarro, C., Chirlaque, M.D. & Barber, X. 2000, "Validation of self diagnosis
of high blood pressure in a sample of the Spanish EPIC cohort: overall agreement and
predictive values. EPIC Group of Spain.", J Epidemiol Community Health, vol. 54, pp.
221-226.
Tourangeau, R. & Yan, T. 2007, "Sensitive questions in surveys.", Psychological Bulletin,
vol. 133, no. 5, pp. 859.
Willett, W. 1990, Nutritional epidemiology. Oxford University Press, New York.
Wolf, A.M., Hunter, D.J., Coblitz, G.A., Manson, J.E., Stampfer, M.J., Corsano, K.A. & et al.
1994, "Reproducibility and validity of a self-administered physical activity
questionnaire.", International journal of epidemiology, vol. 23, pp. 991-999.
Wright, F.V., Law, M., Crombie, V., Goldsmith, C.H. & Dent, P. 1994, "Development of a
self-report functional status index for juvenile rheumatoid arthritis.", Journal of
Rheumatology, vol. 21, pp. 536-544.
9
Appendix 1 Map of searching terms in MEDLINE
AND
All the groups results will be combined with
AND
AND Only Group I and II will be combined with AND for more results (some studies do not use the validity
terms in their titles)
Group I terms regarding
periodontal disease
Group II terms regarding
self-report
Group III terms regarding
validity
10
Appendix 2 Flow diagram of study selection process
Records that will be identified through
database searching (n=?)
Scr
een
ing
Incl
ud
ed
Eli
gib
ilit
y
Iden
tifi
cati
on
Additional records that will be identified through hand-searching
journals (n = ?)
Records screening (n = ?)
Records after screening (n = ?)
Excluding records due to language or validity (n
= ?)
Full-text articles assessment for eligibility
(n = ?)
Full-text articles excluded (n = ?)
- No PD - No self-reported - Neither PD nor self-
reported - Not looking at
validity of self-reporting
Studies that will be included in qualitative synthesis
(n =?)
Removal of Duplication (n = ?)
11
Appendix3 Data extraction form
ID: First author’s sure name:
Title: Country of study:
Year of publication
Type of study
Participants Number: Male Female Age range:
Data extraction
Self-reported
Question
Clinical Gold
Standard
Validity test DOR
(Diagnistic odds retio)
12
Appendix 4 Quality assessment tool (AMSTAR)
1. Was an ‘a priori’ design provided?
The research question and inclusion criteria should be established before
the conduct of the review.
Yes
No
Can’t answer
Not applicable
2. Was there duplicate study selection and data extraction?
There should be at least two independent data extractors and a
consensus procedure for disagreements should be in place.
Yes
No
Can’t answer
Not applicable
3. Was a comprehensive literature search performed?
At least two electronic sources should be searched. The report must
include years and databases used (e.g. Central, EMBASE, and
MEDLINE). Key words and/or MESH terms must be stated and where
feasible the search strategy should be provided. All searches should be
supplemented by consulting current contents, reviews, textbooks,
specialized registers, or experts in the particular field of study, and by
reviewing the references in the studies found.
Yes
No
Can’t answer
Not applicable
4. Was the status of publication (i.e. grey literature) used as an
inclusion criterion?
The authors should state that they searched for reports regardless of
their publication type. The authors should state whether or not they
excluded any reports (from the systematic review), based on their
publication status, language etc.
Yes
No
Can’t answer
Not applicable
5. Was a list of studies (included and excluded) provided?
A list of included and excluded studies should be provided.
Yes
No
Can’t answer
Not applicable
6. Were the characteristics of the included studies provided?
In an aggregated form such as a table, data from the original studies
should be provided on the participants, interventions and outcomes. The
ranges of characteristics in all the studies analyzed e.g. age, race, sex,
relevant socioeconomic data, disease status, duration, severity, or other
diseases should be reported.
Yes
No
Can’t answer
Not applicable
7. Was the scientific quality of the included studies assessed and
documented?
‘A priori’ methods of assessment should be provided (e.g., for
effectiveness studies if the author(s) chose to include only randomized,
double-blind, placebo controlled studies, or allocation concealment as
inclusion criteria); for other types of studies alternative items will be
relevant.
Yes
No
Can’t answer
Not applicable
13
8. Was the scientific quality of the included studies used
appropriately in formulating conclusions?
The results of the methodological rigor and scientific quality should be
considered in the analysis and the conclusions of the review, and
explicitly stated in formulating recommendations.
Yes
No
Can’t answer
Not applicable
9. Were the methods used to combine the findings of studies
appropriate?
For the pooled results, a test should be done to ensure the studies were
combinable, to assess their homogeneity (i.e. Chi-squared test for
homogeneity, I²). If heterogeneity exists a random effects model should
be used and/or the clinical appropriateness of combining should be
taken into consideration (i.e. is it sensible to combine?).
Yes
No
Can’t answer
Not
applicable
10. Was the likelihood of publication bias assessed?
An assessment of publication bias should include a combination of
graphical aids (e.g., funnel plot, other available tests) and/or statistical
tests (e.g., Egger regression test).
Yes
No
Can’t answer
Not applicable
11. Was the conflict of interest stated?
Potential sources of support should be clearly acknowledged in both the
systematic review and the included studies.
Yes
No
Can’t answer
Not applicable
14
Appendix 5 Methodology Checklist 5: Studies of Diagnostic Accuracy
S I G N
Methodology Checklist 5: Studies of Diagnostic Accuracy
This checklist is based on the work of the QUADAS2 team at Bristol Univeristy (http://www.bris.ac.uk/quadas/).
Study identification (Include author, title, reference, year of publication)
Guideline topic: Key Question No:
Before completing this checklist, consider:
1. Is the paper really a study of diagnostic accuracy? It should be comparing a specific diagnostic test against another, and not a general paper or comment on diagnosis.
2. Is the paper relevant to key question? Analyse using PICO (Patient or Population Intervention Comparison Outcome). IF NO REJECT (give reason below). IF YES complete the checklist..
Reason for rejection: Reason for rejection: 1. Paper not relevant to key question □ 2. Other reason □ (please specify):
Checklist completed by:
All the questions in the following sections have associated footnotes providing short explanations behind each of the questions. Users who want more detailed explanations should consult the QUADAS-2: Background Document.
DOMAIN 1 – PATIENT SELECTION
Risk of bias
In a well conducted diagnostic study… Is that true in this study?
1.1 A consecutive sequence or random selection of patients is enrolled.
i
Yes
No
Can’t say
1.2 Case – control methods are not used.ii Yes
No
Can’t say
1.3 Inappropriate exclusions are avoided.iii Yes
No
Can’t say
Applicability
1.4 The included patients and settings match the key question.
iv
Yes
No
Can’t say
DOMAIN 2 – INDEX TEST
Risk of bias
In a well conducted diagnostic study… Is that true in this study?
2.1 The index test results interpreted without knowledge of the results of the reference standard.
v
Yes
No
Can’t say
2.2 If a threshold is used, it is pre-specified.vi Yes
No
Can’t say
15
Applicability
2.3 The index test, its conduct, and its interpretation is similar to that used in practice with the target population of the guideline.
vii
Yes
No
Can’t say
DOMAIN 3 – REFERENCE STANDARD
Risk of bias
In a well conducted diagnostic study… Is that true in this study?
3.1 The reference standard is likely to correctly identify the target condition.
viii
Yes
No
Can’t say
3.2 Reference standard results are interpreted without knowledge of the results of the index test.
ix
Yes
No
Can’t say
Applicability
3.3 The target condition as defined by the reference standard matches that found in the target population of the guideline.
x
Yes
No
Can’t say
DOMAIN 4 – FLOW AND TIMING
Risk of bias
In a well conducted diagnostic study… Is that true in this study?
4.1 There is an appropriate interval between the index test and reference standard.
xi
Yes
No
Can’t say
4.2 All patients receive the same reference standard.xii
Yes
No
Can’t say
4.3 All patients recruited into the study are included in the analysis.
xiii
Yes
No
Can’t say
SECTION 5: OVERALL ASSESSMENT OF THE STUDY
5.1 How well was the study done to minimise bias?
Code as follows:xiv
High quality (++)
Acceptable (+)
Low quality (-)□
Unacceptable – reject 0
5.2 What is your assessment of the applicability of this study to our target population?
Directly applicable
Some indirectness (Please explain in the following section for Notes)
5.2 Notes. Summarise the authors conclusions. Add any comments on your own assessment of the study, and the extent to which it answers your question.
16
i Studies should enrol either all eligible patients suspected of having the target condition during a specified period, or a random sample of those patients. The essential point is that investigators should have no freedom of choice as to which individual patients are or are not included. ii There is evidence that studies comparing patients with known disease with a control group without
the condition tend to exaggerate diagnostic accuracy. iii Inappropriate exclusions may result in either overestimates (eg by excluding ‘difficult to diagnose’
patients) or underestimates (eg by excluding patients with ‘red flags’ suggesting presence of disease) of the degree of diagnostic accuracy. iv Patients included in the study should match the target population of the guideline in terms of severity
of the target condition, demographic features, presence of differential diagnosis or co-morbidity, setting of the study and previous testing protocols. v This is similar to the question of ‘blinding’ in intervention studies. The index test should always been
done first, or by a separate investigator with no knowledge of the outcome of the reference test. vi Bias can be introduced if a threshold level is set after data has been collected. Any minimum
threshold should be specified at the start of the trial. vii
Variations in test technology, execution, or interpretation (eg use of a higher ultrasound transducer frequency) may affect estimates of diagnostic accuracy. viii
Estimates of test accuracy are based on the assumption that the reference standard is 100% sensitive (=accurately diagnoses the target condition). ix This is the similar to question 2.1, but in this case relates to making sure the reference standard is
applied without any prior knowledge of the outcome of previous tests. x The definition of the target condition used when testing the reference standard may differ from that
used by the NHS in Scotland. eg threshold levels used in laboratory cultures may differ. xi The index test and reference standard should be performed as close together in time as possible,
otherwise changes in the patients condition is likely to invalidate the results. xii
In some cases the choice of reference standard may be influenced by the outcome of the index test or the urgency of the need for diagnosis. Use of different reference standards is likely to lead to overestimates of both sensitivity and specificity. xiii
Not including all patients in the analysis may lead to bias as there may be some systematic difference between those lost to follow-up and those analysed. xiv
Rate the overall methodological quality of the study, using the following as a guide: High quality (++): Majority of criteria met. Little or no risk of bias. Results unlikely to be changed by further research. Acceptable (+): Most criteria met. Some flaws in the study with an associated risk of bias, Conclusions may change in the light of further studies. Low quality (-): Either most criteria not met, or significant flaws relating to key aspects of study design. Conclusions likely to change in the light of further studies.