Effectiveness of a Specially Designed Dental Model for ... Model for Training, Evaluation, and...

1430 Journal of Dental Education ■ Volume 80, Number 12

Effectiveness of a Specially Designed Dental Model for Training, Evaluation, and Standardization of Pocket ProbingMasayo Sunaga, RDH, BLA; Masato Minabe, DDS, PhD; Koji Inagaki, DDS, PhD; Atsuhiro Kinoshita, DDS, PhD

Abstract: The aim of this study was to evaluate the effectiveness of a dental model in training, evaluation, and standardization of examiners in pocket probing and to determine the appropriate thresholds of accuracy and measuring time when using this model for evaluation of probing skills without measuring patients’ pockets repeatedly. In 2011-12, a total of 66 dental professionals and 20 dental students in Japan measured the probing depths of 24 artificial teeth using the six-point method on a dental model. All examiners measured the probing depths of six tooth groups and then checked the correct depths in each group. Each examiner measured four groups in a group-by-group manner. For each group, the measuring time and examiner’s accuracy were recorded. Receiver operating characteristic (ROC) curves for various thresholds of measuring time were drawn for thresholds of accuracies to determine the passing mark as a skilled examiner. The accuracy significantly increased from the first to the fourth measure-ments, and the measuring time was significantly reduced for both the professionals and students. The total measuring time was significantly longer for the students than the professionals. The students’ accuracy was significantly lower than that of the profes-sionals in the first measurement group. The increasing rate of accuracy was significantly higher for the students than the profes-sionals. These results and ROC curves suggested that the dental model is effective for periodontal pocket probing training and for the evaluation and standardization of examiners’ probing skill at a preclinical level. An examiner having accuracy ≥80% within four minutes for six tooth measurements in this model could be considered a skilled examiner.

Ms. Sunaga is Assistant Professor, Department of Educational Media Development, Graduate School of Medical and Dental Sciences, Curricular Management Division, Institute of Education, Tokyo Medical and Dental University, Tokyo, Japan; Dr. Minabe is Professor, Division of Periodontology, Department of Oral Science, Graduate School of Dental Science, Kanagawa Dental University, Kanagawa, Japan; Dr. Inagaki is Professor, Department of Dental Hygiene, Aichi Gakuin Junior College, Aichi, Japan; and Dr. Kinoshita is Professor and Chair, Department of Educational Media Development, Graduate School of Medical and Dental Sciences; Curricular Management Division, Institute of Education, Tokyo Medical and Dental University, Tokyo, Japan. Direct correspondence to Dr. Atsuhiro Kinoshita, Department of Educational Media Development, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, 1-5-45 Yushima Bunkyo-ku, Tokyo 113-8510, Japan; +81-3-5803-4643; [email protected].

Keywords: dental education, periodontics, dental hygiene education, teaching materials, periodontal pocket probing, preclinical education

Submitted for publication 6/25/16; accepted 9/28/16

It is essential for dental professionals to measure periodontal pocket depths accurately during periodontal examinations. Dental and dental

hygiene students usually practice on typodonts before mock clinical practice. However, few dental models with structures simulating the bottom of periodontal pockets, on which students can practice probing pockets with various depths and patterns, are commercially available. Therefore, in general, most students and young dentists have no choice but to improve their probing skill through clinical practice with real patients.

Since 2008, we have been developing a train-ing model that students can use to learn periodontal probing. We introduced a new dental model (2009 model) that simulates the structure of the bottom of periodontal pockets for probing exercises, and we

conducted a study that found the model was fea-sible and effective for student practice.1 Based on the results of that study, we improved the material representing the gingiva by reducing its stickiness and modified the design of the bottom of the pocket in order to develop more appropriate models for the education of dental and dental hygiene students. The dental model (Nissin 500H-PRO; Nissin Dental Products, Inc., Kyoto, Japan) used in the present study was designed to have deep and narrow pockets in order to give an inaccurate measurement unless walking of the probe and circumferential probing are performed correctly (Figure 1). Furthermore, eliminating the sloped bottom of pockets at borders of sites in six-point periodontal pocket probing enables the examiner to read the measurement value clearly, even at the border of the sites.

December 2016 ■ Journal of Dental Education 1431

sites for the purpose of calibration, standardiza-tion, evaluation, or training of examiners as well as treatment planning. Buduneli et al. investigated the accuracy and reproducibility of pocket probing in simulated periodontal pockets made of plates and holes with unknown depth and used only for the experiment.2 However, few dental models have been introduced for practice, evaluation, and standardiza-tion of examiners in pocket probing. The aims of our study were to evaluate the effectiveness of our newly developed dental model in training, evaluation, and standardization of examiners in pocket probing and to determine appropriate thresholds of accuracy and

The results of previous research have suggested that periodontal probing depths are affected by many factors. Studies have investigated the accuracy and reproducibility of two different manual probes in simulated periodontal pockets,2 the effects of dif-ferences of shape of tip, tip diameter, and design of scale,3-5 inter- and intra-examiner variability,6-9 differences between constant force electronic probes or sensor probes such as Florida probes or Florida disk probes and manual probes,10-19 and the effects of probing pressures.4,20-22 In the studies conducted to evaluate probing accuracy,9,12,17,23,24 the participants had to undergo repeated pocket probing in the same

Figure 1. Dental model used in study

Note: Panels are as follows: a) front view of dental model for probing practice; b) full mouth views of inner structure without artificial gingival; model has furcation defects on some first molars to practice examining furcation involvements; c) schematic diagram in a buc-colingual sectional view of dental model with a periodontal probe inserted into the pocket.


patients in the second half of the fifth year and in the sixth year. At the time of this study, the students who were in their fourth year had only measured shallow gingival sulcus once in almost healthy peri-odontal tissue during mock clinical practice and had never measured periodontal pockets in periodontitis patients. Therefore, the method of probing pockets was explained to them using a video demonstration showing probing angulation and the walking method prior to the experiment.

Methods of MeasurementA total of 24 teeth were divided into four groups

of six teeth each, representing one tooth from the mandibular and maxillary incisors, premolars, and molars. These tooth groups were named groups A, B, C, and D. Periodontal pocket depths were measured at the following six sites per tooth: mesio-buccal, mid-buccal, disto-buccal, mesio-lingual, mid-lingual, and disto-lingual, based on the six-point-method.

A sample of the recording chart used in this study appears in Figure 2. As the permutation of the four groups is 4! = 4 x 3 x 2 x 1 = 24, a total of 24 variations of recording charts (Chart No. 1-24 in Figure 2) were prepared. An examiner who was as-signed recording Chart No. 1, for example, measured and confirmed the pocket depths in the order A-B-C-D. The recording charts with 24 variations were distributed randomly to examiners.

The actual procedure of measurement and confirmation using Chart No. 1 with the order A-B-C-D is shown in Figure 2. Each examiner measured all four groups of six teeth, in the order listed in the assigned recording chart. The examiners checked the setup depths in the group and verified them on the model immediately after the measurement of each group. After verification of the correct depths in the group, examiners were allowed to examine pocket depths in the next group. Similarly, all pocket depths in four groups of six teeth, totaling 24 teeth, were examined and verified by each examiner. The measuring time in seconds was recorded in each measurement of group. After measuring the four groups, each examiner answered a questionnaire to evaluate the effectiveness of the model.

Statistical AnalysisThe examiner’s accuracy in percentage and

measuring time in seconds of each tooth group were calculated. “Correct pocket depth” was defined as the setup depth ±1 mm. The setup depths were provided

measuring time to use this model for evaluation of probing skills without measuring the patient’s pock-ets repeatedly.

Materials and MethodsThe Ethics Committee of Tokyo Medical and

Dental University determined that this study need not be overseen by the committee in compliance with Ethical Guidelines for Clinical Studies and Ethical Guidelines for Epidemiological Studies (No. 718). Participants in the study were 66 dental profession-als working at 11 organizations: nine private dental clinics (12 dentists and 40 dental hygienists), one university dental hospital (one dentist and nine dental hygienists), and one private hospital (one dentist and three dental hygienists), as well as 20 dental students in their fourth year at the School of Dentistry, Tokyo Medical and Dental University.

In 2011-12, these participants measured the probing depths of 24 artificial teeth using the six-point method on a dental model for pocket probing (Nissin 500H-PRO). All examiners used a dental model with the same design of pocket depths and a manual periodontal probe (Hu-Friedy CP-15 University of North Carolina; Hu-Friedy Mfg. Co., LLC, Chicago, IL, USA). The dental professionals measured the probing depth of the model attached to a manikin (Nissin Simple Mannequin), and the stu-dents took measurements in a manikin attached with a simulator (Nissin CDS-100). Averages and standard deviations of the career years for dentists and dental hygienists were 20.4±12.3 years and 12.6±9.5 years, respectively. The dental professionals and the stu-dents had not used the model previously.

The undergraduate course of dentistry at the schools included in this study consists of six years. Periodontology was included in the curriculum from the first half of the fifth year onwards at the time of this study. The students learn to measure probing depth during mock clinical practice on each other as part of “Preventive Dentistry Practice” in the first half of their fourth year. During the mock clinical practice, instructors explain that the probing pressure should be no more than 20 g and that a practical test is to place the probe point under the thumbnail and press until blanching occurs. When students enter their fifth year, they practice measuring pocket depths on phantom heads in the “Phantom Practice of Peri-odontology” course in the first half of the year; they then perform clinical practice on real periodontitis


Figure 2. Example of actual procedure of measurement and confirmation using Chart No. 1 with an order A-B-C-D


group of six teeth within a threshold measuring time and at a higher accuracy than a threshold, the case was defined as positive for the threshold of measuring time and the threshold of accuracy. Conversely, when an examiner exceeded a threshold measuring time or measured at an accuracy lower than a threshold, then the case was considered to be negative. Then, sensitivities and false positive rates were calculated for various combinations of thresholds of measuring time and thresholds of accuracies. In the calculation of sensitivities and false positive rates, we defined the students as examiners who should be evaluated as negative and the professionals as examiners who should be evaluated positive.

ResultsChanges in the accuracies and measuring time

in seconds from the first to the fourth measure-ments by the students and professionals are shown in Figure 3. The accuracies significantly increased from the first to the fourth measurements, and the

as data of the product by Nissin Dental Products, Inc. Differences in accuracies and measuring time between the orders of groups were analyzed using the paired Student’s t-test. We defined the increasing rate of accuracy and the reduction rate of measuring time as follows: increasing rate of accuracy = (accuracy of 4th group – accuracy of 1st group)/accuracy of 1st group; and reduction rate of measuring time = (measuring time of 1st group – measuring time of 4th group)/measuring time of 1st group. Differences in the increasing rates of accuracy and reduction rates of measuring time between the students and profession-als were tested using the unpaired Student’s t-test.

In addition, in order to use this model for evalu-ation of pocket probing skills, we assumed various thresholds for measuring time, ranging from 120 to 420 seconds in increments of 30 seconds, and various thresholds for accuracy on a measurement group with six teeth, ranging from 50% to 100% in steps of 5%. Receiver operating characteristics (ROC) curves for the assumed thresholds were drawn according to the following definition. When an examiner measured a

Figure 3. Changes in accuracy (graph on left) and measuring time in seconds (graph on right) from first to fourth mea-surements by students (n=20) and professionals (n=66)

Accuracy rate: mean% ±SE. Measuring time: mean sec. ±SE. *p<0.05; **p<0.01; ***p<0.001, Student’s t-test for paired and unpaired samples


formed, the higher accuracies they achieved and the shorter time they spent. These results indicate that the model is effective for pocket probing training. The mean accuracy of the first measurement by the students who had very little experience with measur-ing deep periodontal pockets was significantly lower than that of the first measurement by the experienced professionals. This suggests that the accuracy with this model reflects the probing skill.

Since there was no time limit for measurement in this study, the students could take as long as they wanted. In addition, it is conceivable that the profes-sionals measured pockets at a greater speed taking into consideration the patient burden in clinical settings. In routine clinical examinations, it is not practical to take a long time to measure pocket depths repeatedly. Therefore, a person who could measure pockets correctly in this model within a particular time period is expected to possess adequate probing skills. This explains why the students’ accuracy in the fourth measurement was significantly higher than that of the professionals.

As for the threshold of accuracy to use this model for evaluation of pocket probing skills, 80% was considered to be the most appropriate because

measuring time in seconds was significantly reduced for both the dental professionals and the students. The mean total measuring time in seconds (SE) was significantly longer for the students (1238±64 sec.) than the professionals (995±40 sec.). However, no statistically significant differences were detected between the means of total accuracy for the students (90.9±1.0%) and the professionals (90.3±0.8%). The students’ accuracy was significantly lower than that of the professionals in the first measurement, and it was higher in the fourth measurement. The students’ measuring time was significantly longer than that of the professionals in the second to the fourth mea-surement groups. The increasing rate of accuracy was significantly higher for the students than the professionals (Figure 4). No statistical differences were found in the reduction rates of measuring time between the students and professionals.

Figure 5 shows ROC curves representing the relations of sensitivities and false positive rates. The ROC curves for various thresholds of measuring time from 120 to 420 seconds in steps of 30 seconds were drawn for thresholds of accuracies from 50% to 100% in steps of 5%. The ROC curves for 50% to 80% formed a group of curves that differed from the curves for 85% to 100%, and 80% was the highest accuracy threshold in the group. The point represent-ing 240 seconds of measuring time threshold on the ROC curve for 80% was nearest to the upper left corner of the graph. According to the results of the questionnaire to evaluate the effectiveness of the model, most of the participants considered the dental model suitable for pocket probing training and felt that the dental model would contribute to the evalu-ation of probing skills (Figure 6).

DiscussionThe significantly higher increasing rate of ac-

curacy among the students indicates that the model was more effective for beginners like students than for professionals. As professionals may not improve their probing skill in a short time as in this study, the increase of accuracy among the professionals is the result of being accustomed to the dental model rather than improvement of their probing skill. The increase of accuracy among the students can be attributed to both being accustomed to the model and improve-ment of skill. The differences in the increasing rates suggest that the model is effective for pocket probing training. The more measurements the students per-

Figure 4. Comparison of increasing rates of accuracy (graph on left) and reduction rates of measuring time (graph on right) for students (n=20) and professionals (n=66)

Mean% ±standard error. **p<0.01, unpaired Student’s t-test


As the students who participated in this study had only practiced probing once by measuring the pocket depths of a classmate in a mock clinical prac-tice, it was difficult to say whether they employed ap-propriate probing pressure. However, their accuracy improved from the first to the fourth measurements. Although this dental model cannot detect excessive probing pressure, it does not provide accurate depths when measured with pressure that is too low. If students recorded depths that were too shallow due to inadequate pressure, they would be able to train and develop the proper pressure to obtain the correct depths at each feedback just after the measurement. In addition, students would be able to obtain the correct depths with sufficient pressure in the next

there were slight differences between the ROC curve for 80% of accuracy threshold and those for less than 80%, whereas the ROC curves above 80% were conspicuously different from that of 80% (Figure 5). Regarding the threshold of measuring time, 240 seconds would be the most appropriate because the point representing 240 seconds on the ROC curve for 80% of accuracy threshold was nearest on the curve to the upper left corner representing 100% of sensitivity and none of the false positive rate. There-fore, an examiner who measures the pocket depths in a group of six teeth with an accuracy ≥80% within 240 seconds (four minutes) could be considered a skilled examiner, capable of serving as a calibrated or standardized examiner in periodontal clinical studies.

Figure 5. ROC curves representing relations of sensitivities and false positive rates


the probe and circumferential probing, probe inser-tion with an appropriate angle, and less oversight of deep narrow pockets.

Another limitation of this study was that the effectiveness of the dental model was demonstrated only at the preclinical level and that it was carried out on a dental model mounted on a manikin in the laboratory. Further studies are necessary to confirm whether the criteria for using the dental model in this study would be valid and feasible for evaluation and standardization of examiners’ probing skills in clinical settings with real patients. Comparing the

measurement, resulting in the acquisition of appropri-ate probing pressure and improvement in accuracy.

One of the limitations of this study was that the model could not detect excessive probing pressure from the measured probing depths. An improved ver-sion of the model that would provide incorrect depths when measured with excessive probing pressure is required. However, such a model would be too com-plicated and expensive to produce. In dental schools, this model could be used in an objective structured clinical examination (OSCE) to evaluate not only accuracy but also strict enforcement of walking of

Figure 6. Results of questionnaire completed by participating dental professionals (n=66) and students (n=20)


Nissin Dental Products, Inc., Kyoto, Japan. One of the coauthors, Atsuhiro Kinoshita, is listed as a joint inventor on the patent. The other authors declared no conflicts of interest directly related to the content of this article.

REFERENCES1. Sunaga M, Kondo K, Adachi T, et al. Development and

evaluation of a new dental model at Tokyo Medical and Dental University for the practice of periodontal pocket probing. J Dent Educ 2013;77(9):1185-92.

2. Buduneli E, Aksoy O, Kose T, Atilla G. Accuracy and reproducibility of two manual periodontal probes: an in vitro study. J Clin Periodontol 2004;31(10):815-9.

3. Atassi F, Newman HN, Bulman JS. Probe tine diameter and probing depth. J Clin Periodontol 1992;19(5):301-4.

4. Garnick JJ, Silverstein L. Periodontal probing: probe tip diameter. J Periodontol 2000;71(1):96-103.

5. Holtfreter B, Alte D, Schwahn C, et al. Effects of different manual periodontal probes on periodontal measurements. J Clin Periodontol 2012;39(11):1032-41.

6. Badersten A, Nilveus R, Egelberg J. Reproducibility of probing attachment level measurements. J Clin Periodon-tol 1984;11(7):475-85.

7. Janssen PT, Drayer A, Faber JA, Helderman WH. Accu-racy of repeated single versus averages of repeated dupli-cates of probing depth measurements. J Clin Periodontol 1988;15(9):569-74.

8. Araujo MW, Hovey KM, Benedek JR, et al. Reproducibil-ity of probing depth measurement using a constant-force electronic probe: analysis of inter- and intra-examiner variability. J Periodontol 2003;74(12):1736-40.

9. Andrade R, Espinoza M, Gomez EM, et al. Intra- and inter-examiner reproducibility of manual probing depth. Braz Oral Res 2012;26(1):57-63.

10. Gibbs CH, Hirschfeld JW, Lee JG, et al. Description and clinical evaluation of a new computerized peri-odontal probe: the Florida probe. J Clin Periodontol 1988;15(2):137-44.

11. Magnusson I, Clark WB, Marks RG, et al. Attachment level measurements with a constant force electronic probe. J Clin Periodontol 1988;15(3):185-8.

12. Walsh TF, Saxby MS. Inter- and intra-examiner variability using standard and constant force periodontal probes. J Clin Periodontol 1989;16(3):140-3.

13. Osborn J, Stoltenberg J, Huso B, et al. Comparison of measurement variability using a standard and constant force periodontal probe. J Periodontol 1990;61(8): 497-503.

14. Osborn JB, Stoltenberg JL, Huso BA, et al. Comparison of measurement variability in subjects with moderate periodontitis using a conventional and constant force periodontal probe. J Periodontol 1992;63(4):283-9.

15. Perry DA, Taggart EJ, Leung A, Newburn E. Compari-son of a conventional probe with electronic and manual pressure-regulated probes. J Periodontol 1994;65(10): 908-13.

16. Wang SF, Leknes KN, Zimmerman GJ, et al. Intra- and inter-examiner reproducibility in constant force probing. J Clin Periodontol 1995;22(12):918-22.

probing skill ability with real patients or by OSCEs of trained students using the model and those of untrained students would be required in the future.

We confirmed that the pocket depths did not differ from the setup depths at all sites after the study. However, in a few cases, when the same type of the model was used for practical training, part of the silicone elastomer serving as the bottom of the peri-odontal pocket detached from the hard resin, which was considered as artificial alveolar bone resulting in deeper pocket depths. Changes in pocket depth on multiple usage must be examined precisely in future. Also, the variation among the different models used in this study was not examined.

As 70% of the professionals surveyed re-sponded positively to the question “Was it easy to find the base of the periodontal pockets?” and 94% of the professionals responded positively to the question “Do you think the dental model is suitable for pocket probing training?,” the resistance of the artificial gingiva and the artificial base of the pocket may be considered suitable for training. However, further studies are necessary to confirm whether the resistance of the model was capable of mimicking that of the real periodontal attachment apparatus. We consider that differences in the observed resistance would not negatively influence the acquisition of probing skills in real patients.

ConclusionThe dental model evaluated in this study was

found to be effective for training periodontal pocket probing and for evaluation and standardization of examiners’ probing skills at a preclinical level. An examiner with an accuracy ≥80% within four minutes during six tooth measurements on this model could be considered a skilled examiner.

AcknowledgmentsThe authors express their sincere gratitude to

the dentists, dental hygienists, and dental students who participated in this study. The dental models evaluated were produced and provided by Nissin Dental Products, Inc., Kyoto, Japan based on our requirements.

DisclosureThe model was jointly patented by Tokyo

Medical and Dental University, Tokyo, Japan and


21. Larsen C, Barendregt DS, Slot DE, et al. Probing pressure, a highly undervalued unit of measure in periodontal prob-ing: a systematic review on its effect on probing pocket depth. J Clin Periodontol 2009;36(4):315-22.

22. Al Shayeb KN, Turner W, Gillam DG. Accuracy and re-producibility of probe forces during simulated periodontal pocket depth measurements. Saudi Dent J 2014;26(2): 50-5.

23. Hensel E, Gesch D, Biffar R, et al. Study of health in Pomerania (SHIP): a health survey in an East German region: objectives and design of the oral health section. Quintessence Int 2003;34(5):370-8.

24. Drucker SD, Prieto LE, Kao DW. Periodontal probing calibration in an academic setting. J Dent Educ 2012; 76(11):1466-73.

17. Grossi SG, Dunford RG, Ho A, et al. Sources of error for periodontal probing measurements. J Periodontal Res 1996;31(5):330-6.

18. Karpinia K, Magnusson I, Gibbs C, Yang MCK. Accuracy of probing attachment levels using a CEJ probe versus traditional probes. J Clin Periodontol 2004;31(3):173-6.

19. Ishihata K, Wakabayashi N, Wadachi J, et al. Reproduc-ibility of probing depth measurement by an experimental periodontal probe incorporating optical fiber sensor. J Periodontol 2012;83(2):222-7.

20. Lang NP, Nyman S, Senn C, Joss A. Bleeding on probing as it relates to probing pressure and gingival health. J Clin Periodontol 1991;18(4):257-61.

Effectiveness of a Specially Designed Dental Model for ... Model for Training, Evaluation, and...

Documents

Transcript of Effectiveness of a Specially Designed Dental Model for ... Model for Training, Evaluation, and...