EVIDENCE-BASED ORTHODONTICS

Fourth and forty?Greg Huang

Seattle, Wash

My colleague, Anne-Marie Bollen, has re-ceived a flurry of e-mails from disgruntledorthodontists regarding her recently pub-

lished systematic review,1 which reports that orthodon-tic treatment might be associated with minor worseningof the average periodontal status of orthodontic pa-tients. As a coauthor of the review, I was approached atthe recent American Association of Orthodontists meet-ing in Denver by some orthodontists whom I wouldconsider “friendly” to the evidence-based camp. Theywere not happy, either. “Greg, how could you beinvolved with a review like this? It is not helpful to ourprofession.” I tried to describe some of the challengeswe encountered as we conducted this review, such asthe relatively poor quality of available evidence, thedifficulty in finding studies with appropriate controlsubjects, and the multitude of outcome measures fromthe various studies. Although we based our review onthe best evidence available, we knew that the smalldifferences we found were probably not clinicallysignificant, even though they were statistically signifi-cant. I also described our sample as composed mainlyof adolescents, many undergoing premolar extractions,and stated that our data would not generalize well tosome current trends in orthodontic therapy, such as anincreasing adult population, and nonextraction treat-ment. With all those caveats, I defended the findings ofour review and stated that, after 10 months of hardwork, based on the best evidence available, we couldnot say that orthodontic treatment resulted in an aver-age improvement in periodontal health. My colleaguepromptly replied, “Don’t you think the authors werebiased?”

Although I am sure my jaw dropped to the ground,I quickly pulled it back up and explained that minimiz-ing bias was one of our primary goals. We had noreason to wish for the results we obtained and under-

Associate professor, Evidence-based Dentistry, Department of Orthodontics,University of Washington, Seattle.Reprint requests to: Greg Huang, HSC Box 357446, Health Sciences D569,Department of Orthodontics, University of Washington, Seattle, WA 98195;e-mail, ghuang@u.washington.edu.Submitted and accepted, June 2008.Am J Orthod Dentofacial Orthop 2008;134:174-50889-5406/$34.00Copyright © 2008 by the American Association of Orthodontists.

doi:10.1016/j.ajodo.2008.06.002

174

stood that our findings did not support the commonlyheld belief that orthodontics helps periodontal health.Anne-Marie and I are both practicing orthodontists,and, if we were biased, it would be in favor of supportingthe benefits of orthodontics. But our goal was to performa comprehensive and systematic assessment of the liter-ature; this is why we searched many databases, per-formed hand-searching, translated articles from foreignlanguages, used inclusion/exclusion criteria, and judgedthe quality of the studies. These are all commonlyaccepted guidelines intended to minimize bias in asystematic review. After trying to be as objective aspossible and laboring through all the steps in the reviewprocess, we had to report our findings honestly, whetherthey were beneficial, detrimental, or neutral to theorthodontic specialty. Although statistically significant,the literature was inconclusive on whether this worsen-ing was due to small, overall worsening, or perhaps toisolated, severe worsening. Because the average wors-ening was indeed small, we could expect that therewere would be some positive and negative scatteraround the mean. However, the final result we found,using accepted methods for pooling data, was for a verysmall average worsening. Our methods are described ingreat detail, and others are certainly welcome to repli-cate them.

I guess I should not be disappointed by my col-leagues’ questioning of the results of our review. Afterall, it indicates that they are keeping current with theevidence and are not shying away from being criticalreaders. And I have stated on several occasions that weshould not automatically believe the findings of asystematic review just because it is published, includ-ing any that I am associated with. I also believe that weneed to know the basic guidelines for conducting andassessing systematic reviews to be able to judge theirquality and validity. In fact, if my coauthors and I werenegligent in any of our review methods, I would bemost happy to have these deficiencies identified so thatwe could explain or correct the issues, as appropriate.However, if we did follow accepted practices in con-ducting our review, labeling it biased because one didnot agree with the findings is unfair. When I serve as ajournal reviewer for the systematic reviews submitted

for publication, I cannot say, “don’t publish this review

American Journal of Orthodontics and Dentofacial OrthopedicsVolume 134, Number 2

Huang 175

because I don’t like the results.” It is my obligation toidentify specific methodologic deficiencies, such as stepsthat were omitted or performed incorrectly, and describewhy these issues might have led to biased results.

So, as always, attending an orthodontic meetingoffered many educational experiences in addition tothose in the lecture halls. I learned that many evidence-based supporters might still need reinforcement on therules for systematic reviews. I learned that, in spite ofall the apparent interest in evidence-based orthodontics,it might still be a bitter pill to swallow. I learned that wemust continue to nudge orthodontists toward evidence-based practice, and that they need to push back to havetheir questions and concerns addressed. I can’t imaginethat any evidence-based supporters would wish to assem-ble a group of practitioners who could be led blindly,foregoing critical thinking. Evidence-based practicerequires just the opposite.

I admit, for a split second, when my colleague

asked whether we were biased, I felt like it was 4th and

40, and I was ready to punt. After all, I had just givena lecture on evidence-based principles the day before,and minimizing bias was a key topic that I covered. Onthe plane ride home, though, I reflected on this experi-ence and decided this was only a temporary setback.After all, I learned some valuable lessons from myfellow orthodontists because they spoke their mindsand asked tough questions. In retrospect, I want tothank them. They were probably equally frustrated withour review and perhaps ready to give up on evidence-based orthodontics. How can systematic reviews comeup with these unpleasant results? I hope after morethought, though, they also benefited from our conversa-tion.

REFERENCE

1. Bollen AM, Cunha-Cruz J, Bakko DW, Huang GJ, Hujoel PP.The effects of orthodontic therapy on periodontal health: asystematic review of controlled evidence. J Am Dent Assoc

2008;139:413-22.