Introduction

When a third party is responsible for the costs of

orthodontic treatment, funds are limited, which

correspondingly limits treatment eligibility. An

index is typically used to identify those patients

who are in greatest need of orthodontic treatment.

Occlusal indexes can identify patients in need of

orthodontic treatment and prioritize their treat-

ment requirements, both for quality assurance and

for use in research (1, 2). Occlusal indexes are

common in Northern Europe and are accepted in

clinical orthodontics (3). In the United States,

however, the use of occlusal indexes in every day

practice is extremely limited(4) due primarily to the

fact that the American Association of Orthodontics

(AAO) does not recognize any index rating classi-

fication or coding system as a scientifically valid

measure of the need for orthodontic treatment (5).

Nevertheless, public health planners in 15 states

have adopted several occlusal indexes with arbi-

trary cut-off scores to determine eligibility for

orthodontic care with state funds (6). The Handi-

capping Labiolingual Deviation (HLD) index is one

of the indexes used in the United States, developed

originally to identify those with handicapping

malocclusions (7). Some states have modified the

HLD index in order to determine and prioritize

eligibility for state-funded orthodontic treatment.

For example, Maryland’s index, HLD (Md), has

Community Dent Oral Epidemiol 2010; 38: 50–57All rights reserved

� 2009 John Wiley & Sons A/S

Assessment of the validity ofHLD (CalMod) in identifyingorthodontic treatment needCooke M, Gerbert B, Gansky S, Miller A, Nelson G, Orellana M. Assessment ofthe validity of HLD (CalMod) in identifying orthodontic treatment need.Community Dent Oral Epidemiol 2010; 38: 50–57. � 2009 John Wiley & Sons A ⁄ S

Abstract – Objective: The purpose of this study was to assess the validity of theHandicapped Labio-Lingual Deviation index with California modifications,HLD (CalMod), in identifying handicapping malocclusions. Methods: A set of153 study casts representing all types of malocclusion was utilized in this study.Models were randomly chosen the UCSF Division of Orthodontics clinic.Treatment need was determined by the HLD (CalMod) index and by a panel of13 orthodontists, conventionally established as the ‘gold standard’. SpearmanRank correlation analysis was used to evaluate the correlation between HLD(CalMod) and the gold standard. The Classification and Regression Tree(CART) modeling was used to determine the HLD (CalMod) cut-off point oforthodontic treatment need according to the gold standard. Results: ASpearman Rank correlation Coefficient of 0.71 demonstrated a moderatelyhigh correlation between HLD (CalMod) and the gold standard. The CARTmodeling determined a value of 18.5 as the cut-off point of HLD (CalMod) fororthodontic treatment need, considerably lower than the cut-off point of 26currently used by Medi-Cal. At a value of 26 points as the cut-off HLD(CalMod) displayed a low sensitivity (25.9%) and high specificity (96.8%).Witha cut-off point of 18.5, specificity decreased to 55.6% while sensitivity increaseddramatically to 92.9%. Conclusion: Our results show that the HLD (CalMod)with a cut-off point of 26 fails to indentify a considerable percentage ofhandicapping malocclusions. More studies should be done assessing theefficacy of the HLD (CalMod) in identifying handicapping malocclusion.

Mary Cooke1, Barbara Gerbert2, Stuart

Gansky2, Arthur Miller3, Gerald Nelson3

and Maria Orellana3

1Private Practice, Napa, CA, 2Department of

PRDS, School of Dentistry, University of

California San Francisco, San Francisco, CA,3Department of Orofacial Sciences, School of

Dentistry, University of California San

Francisco, San Francisco, CA

Key words: malocclusion; orthodontics;public health policy

Maria Orellana, Orofacial Sciences,University of California, San Francisco, CA,USA.Tel.: +1 415 476 4730Fax: +1 415 502 1013e-mail: Maria.orellana@ucsf.edu

Submitted 8 January 2009;accepted 26 August 2009

50 doi: 10.1111/j.1600-0528.2009.00506.x

raised the cut-off from 13 to 15 points and modified

the HLD’s original scoring formula for overjet and

overbite (8). The State of Washington modification

includes five qualifying conditions and a cut-off

point of 30 (9).

California uses the HLD index with modifica-

tion, hence the designation HLD (CalMod).

Twelve factors are weighted and summed to

provide a score: overjet, overbite, open bite, cleft

lip-palate, anterior crowding, mandibular protru-

sion, labiolingual spread, deep impinging overbite,

severe traumatic deviations, crossbite of individ-

ual anterior teeth, ectopic eruption of anterior

teeth, and posterior unilateral crossbite. California

modified the index as a result of settlements

stemming from two lawsuits. As a result of the

first lawsuit, two qualifying exceptions, recog-

nized as causing injury to the buccal tissue, were

added: deep impinging bites and crossbites of

individual teeth, with tissue damage. In addition,

unilateral posterior crossbite was inserted as a

weighted factor. As an outcome of the second

lawsuit, a reversed overjet greater than 3.5 mm

was added as a qualifying exception. At that time,

an overjet greater than 9 mm was also inserted as

a qualifying exception. These modifications

encompass the current HLD (CalMod) index with

a cutoff point of 26.

Many interest groups involved in government

policy contribute to a public treatment benefit,

especially when funds are limited (10). Three issues

are paramount, the index validity, the given cut-off

point and the qualifying exceptions. A valid index

must accurately measure what it intended to

measure; in the case of the HLD (CalMod) index,

to measure the level of a handicapping malocclu-

sion. Within a specific population, it is essential to

establish a relationship between the index values

and the providers’ opinions of treatment need. One

should specifically ask: Does this index accurately

reflect providers’ opinions?

An occlusal index is validated on evidence that

an occlusal anomaly will have a short or long term

effect on the dental ⁄ oral structures and by

comparing its scores with a conventional ‘gold

standard’, commonly the subjective consensus

opinion of a group of experienced specialists (11).

Yet, the locale in which an orthodontic specialist

practices has an effect on his or her evaluation of

the treatment need (12). In a large country such as

the United States, one can expect to find regional

differences that may influence decisions regarding

the need for orthodontic treatment.

There have been few studies utilizing large

panels of experts to assess the validity and reli-

ability of specific occlusal indexes (9, 13) and only

one involving the HLD (CalMod) (14). There has

been no study assessing the validity of the HLD

(CalMod) with a panel of orthodontic specialists

practicing in California.

The objective of this study was to determine the

validity of the HLD (CalMod) in identifying hand-

icapping malocclusions. Specifically, we wanted to

assess if the HLD (CalMod) reflects the providers’

opinion, the ‘gold standard’, of what constitutes

handicapping malocclusion.

Material and methods

One hundred and fifty-three sets of models were

randomly chosen from the UCSF Division of

Orthodontics clinic. The models represented a

spectrum of malocclusions and included pretreat-

ment models and posttreatment models. Models

were not included if any appliances were visible.

Models were trimmed to Centric Occlusion (CO)

and were evaluated for bubbles or imperfections

that could influence or interfere with evaluation of

the malocclusion. Each model was evaluated and

any information that could not be determined from

the model alone was listed on the ‘information

sheet’ that was kept with each model. This sheet

included information such as impacted teeth,

missing teeth, and CR-CO shifts. The models were

then assigned a random number from 1001 to 1153.

This study was approved by the University of

California San Francisco Committee on Human

Research (CHR#H2582-24865-01).

Gold standardOriginally, 15 orthodontists from the San Francisco

Bay area were recruited to participate in this study.

One subject never began the study and a second

completed the evaluation on only 23 models.

Hence, that participant’s observations were not

included in the study results (Table 1). Inclusion

criteria for participants were five or more years of

clinical orthodontic experience and being active in

clinical practice on a half-time basis or greater. All

participants had to be members of the AAO. Any

orthodontist that had had previous experience with

orthodontic indexes, with the exception of the

California Medi-Cal ⁄ California Children Services

(CCS) index, was excluded from the study. The

orthodontists in this study were selected on the

51

Assessment of the validity of HLD

basis of their availability and commitment to

this study and do not reflect the distribution of

orthodontists in the state of California.

The orthodontic professionals were each pro-

vided one on-site session for evaluating the

models. During the session, casts were displayed

in numerical order on countertops in a large room.

At the beginning of the session, the following

verbal and written instructions were given to each

orthodontist: ‘You are the orthodontic consultant for a

private corporation for which a limited fund has been

established to provide orthodontic treatment for person-

nel. You are to evaluate these study casts of personnel

and answer the following question: In your opinion, to

what extent does this occlusion need orthodontic treat-

ment?’

The orthodontists scored the 153 pairs of cast and

recorded the need for treatment of each pair as a

score of 1–12 on an adjectival scale where:

• 1–3 = No treatment needed

• 4–6 = Treatment optional or elective

• 7–9 = Treatment advisable

• 10–12 = Treatment essential

All models with a score of ‘7’ or above were

considered to be in the treatment category. In

models with a gold standard score of greater than

9.5, orthodontic treatment was deemed medically

necessary. This would correspond to a HLD score

of 26 or above indicating a handicapping maloc-

clusion with medical justification for orthodontic

treatment. For the orthodontic specialists’ evalua-

tions, interrater reliability was evaluated with

analysis of variance and Scheffe’s multiple

comparison tests. Linear regression was used to

compare the gold standard and HLD(CalMod);

since this showed a clear quadratic relationship, a

square root transformation of HLD(CalMod) was

used to meet linear regression assumptions.

HLD (CalMod) indexEach model was evaluated and scored with the

instructions given by Medi-Cal for evaluation with

the HLD (CalMod) index by a qualified orthodontist

(7, 15). It was first determined if it possessed one or

more automatically-qualifying-exception traits

such as cleft lip and palate deformity; a severe

traumatic deviation; deep impinging bites affecting

the soft tissue of the palate; crossbite on individual

anterior teeth with destruction of soft tissue; overjet

greater than 9 mm; and reverse overjet greater than

3.5 mm (16). In addition, each model was evalu-

ated by the scoring protocol, whether or not an

automatically qualifying exception existed. Some

assumptions had to be made given that photos and

radiographs were not available. For example, if an

overjet of greater than 9 mm was determined, lip

incompetence was assumed or if a negative overjet

of greater than 3.5 mm was determined, difficulty

masticating was assumed.

Intra-rater reliability for HLD (CalMod) was

evaluated using Lin’s concordance correlation,

which assesses equivalence of 2 measurements.

The reproducibility of measurements was assessed

by statistically analyzing the difference between

double measurements made 4 months apart on a

subset of 40 casts randomly chosen.

Validity of HLD (CalMod) was evaluated by two

methods: (i) Spearman Rank correlation was used

to evaluate the correlation between HLD (CalMod)

and the gold standard. (ii) the Classification and

Regression Tree (CART) modeling (Salford

Systems Inc, San Diego, CA, USA) was used to

determine the cut-off of orthodontic treatment need

(17, 18). Specifically, CART interactively evaluated

the predictor to determine the optimal cut-off: as

this is highly dependent, 10-fold cross-validation

was used to make the results robust and avoid

overfitting (19).

Results

Gold standard scoresEach model was given 13 scores (one from each

orthodontic rater). The mean of the 13 scores

from each model was compared to the mode and

the median. The average mean score for all casts

was 7.2, the average mode was 6.9 and the

average median was 7.1. The mean was chosen as

the gold standard score for each model. Table 2

shows the number of models that fell into the

four different orthodontic needs categories. The

Scores breakdown differed from the orthodontist

evaluation sheet because the gold standard score

was an average of 13 scores rounded to the

closest tenth. The cut-off between categories was

Table 1. Orthodontist subject characteristics

Mean age range 35–70 50Average years certified range 6–41 20.1Number of different programs 6Private practice 8Group practice 2University practice 3Male 10Female 3

52

Cooke et al.

determined by choosing a number that split the

difference of the highest number of the first cat-

egory and the lowest number of the next category.

Interrater reliability analysis showed rater 4 to be

significantly different from every other rater apart

from rater 5 while rater 5 was significantly

different from five of the other 12 raters. Rater

9 was significantly different from 3 other raters.

Apart from raters 4, 5 and 9, the raters had a

maximum of 2 significantly different findings

(Table 3). There was no relationship between

rater’s orthodontic experience and mean model

score and between rater’s age and mean model

score.

HLD (CalMod) scoresLin’s concordance correlation showed high levels

of intraexaminer reliability: 0.894. A Spearman

Rank correlation Coefficient of 0.71 demonstrated

a moderately high correlation between HLD (Cal-

Mod) and the gold standard (P < 0.01). Fig. 1

shows the quadratic relationship between the gold

standard and HLD (CalMod). The relationship

between the gold standard and the square root of

HLD (CalMod) is essentially linear.

Sensitivity and specificity of HLD (CalMod)

were determined using the HLD (CalMod) cut-off

of 26 points. The results displayed a low sensitivity

(25.9%) and high specificity (96.8%) for this mea-

sure as it is used currently (Table 4). When we

applied CART modeling to determine the cut-off of

HLD (CalMod) for orthodontic treatment need, we

arrived at a value of 18.5 points, considerably lower

than the cut-off of 26 currently used by Medi-Cal.

Table 2. Distribution of cases according to the goldstandard

Category

Goldstandardrange

Numberof cases

No treatment needed 1)3.5 21Treatment optional >3.5)6.5 28Treatment advisable >6.5)9.5 77Treatment essential >9.5 27

Table 3. Descriptive statistics of orthodontist rater scores

Mean SD SE Count Minimum Maximum # Missing

Rater 1 6.869 2.662 0.215 153 1 11 0Rater 3 6.647 2.575 0.210 150 1 12 3Rater 4 9.955 2.558 0.214 143 2 12 10Rater 5 8.348 2.734 0.255 115 1 12 38Rater 6 7.678 2.696 0.223 146 1 12 7Rater 7 6.553 2.681 0.217 152 1 11 1Rater 8 6.203 2.165 0.175 153 1 11 0Rater 9 8.054 2.962 0.243 148 1 12 5Rater 10 6.926 3.203 0.262 149 1 12 4Rater 12 6.270 3.333 0.274 148 1 12 5Rater 13 7.020 3.284 0.265 153 1 12 0Rater 14 6.894 3.529 0.287 151 1 12 2Rater 15 6.033 2.753 0.224 151 1 10 2

Table 4. Sensitivity and specificity of HLD (CalMod)with a cut-off of 26

HLD (CalMod)with cut off of 26

Gold Standard

Handicappingmalocclusion

Nohandicappingmalocclusion

Handicappingmalocclusion

7 4

NO handicappingmalocclusion

20 122

Sensitivity (%) 25.9Specificity (%) 96.8Prevalence (%) 17.7(+) Predictivevalue (%)

63.6

()) Predictivevalue (%)

85.9

0

1

2

3

4

5

6

7

0 2 4 6 8 10 12Gold standard

Sqr

root

of

HL

D(C

alM

od)

Fig. 1. Gold standard scores versus the square root of theHLD (CalMod) index scores.

53

Assessment of the validity of HLD

The sensitivity and specificity were then recalcu-

lated using the new cut-off of 18.5 for HLD

(CalMod). Specificity decreased to 55.6% while

sensitivity increased dramatically to 92.9%. Posi-

tive predictive value was still low at 62.5%

(Table 5).

Discussion

We found the HLD (CalMod) index to be fairly

easy to utilize in terms of the measurement tech-

niques employed. However, some of the instruc-

tions designated by the Medi-Cal statutes were

unclear, specifically section 531–11 (c) (2), which

states that only teeth that are visible in the

study model should be considered. This is later

contradicted in section 531–11(c) (6) where exam-

ples of ectopic eruption are described. Our obser-

vations are in agreement with those of Parker

(1998) (16) who recognized the confusion sur-

rounding the definition of ectopic. In his paper, he

provided some clarification for this definition as it

is applied to the HLD (CalMod). However, he

failed to address how ‘teeth in the maxillary sinus

and ascending ramus of the mandible’ (clearly

unerupted, ectopic teeth) will be visible in the

study model.

Our analysis showed that the HLD (CalMod)

was well correlated with the gold standard. How-

ever, the cut-off for HLD (CalMod) did not corre-

spond with the gold standard threshold level for

models that displayed handicapping malocclu-

sions. A good example of this discrepancy is

shown in Fig. 1. This specific case was given a

gold standard score of 11.0, which falls in the

treatment essential category. However, the HLD

(CalMod) index only gave this case a score of 12,

well below the cut-off of 26.

Our results are in agreement with a previous

study comparing the HLD (Md) with the HLD

(CalMod) index (4) that identified several

weaknesses in the latter. For example, it fails to

identify localized crowding and missing teeth or

spacing in the anterior segment (Fig. 2). It does not

weight open bites heavily enough and fails to

identify bilateral crossbites. Further, those maloc-

clusions with fairly aligned arches and large overjet

(Figs 3 and 4) that fall short of the 9 mm qualifying

exception would not qualify under the HLD (Cal-

Mod) index.

In the late 90s, approximately 1 700 000 people

were eligible for MediCal-mandated orthodontic

Table 5. Sensitivity and specificity of HLD (CalMod)with a cut-off of 18.5

HLD (CalMod)with cut off of 18.5

Gold standard

Handicappingmalocclusion

Nohandicappingmalocclusion

Handicappingmalocclusion

15 9

NO handicappingmalocclusion

12 117

Sensitivity (%) 55.6Specificity (%) 92.9Prevalence (%) 17.7(+) Predictivevalue (%)

62.5

()) Predictivevalue (%)

90.7

Fig. 2. This model had a gold standard score of 11.0 (out of a maximum of 12) and an HLD (CalMod) score of 12.

54

Cooke et al.

benefits in the state of California. Of this group,

160 745 potential patients were assessed for eligi-

bility by the HLD (CalMod) index. The orthodon-

tists involved determined that 58 193 (36%) of this

number qualified for treatment. Study models of

these patients were then measured and scored

again by a group of orthodontists at the fiscal

intermediary. Only 27 637 were approved for

treatment, less than half (47%) of the patients

already approved by the orthodontist providers

and only 17% of all the referred patients (20).

The present study showed that the expert opin-

ion of a panel of orthodontist considered two and

one-half times more patients to have handicapped

malocclusion (27) than the HLD (CalMod) index

(11). When the cut-off is reduced to 18.5, the HLD

(CalMod) index increases the number of patients

with handicapped malocclusions to 24.

The expert opinion of a group of orthodontists is

considered the ‘gold standard’. It is against this

‘gold standard’ that occlusal indexes, such as the

HLD, are validated. Why is it then possible to find

such a discrepancy between the HLD (CalMod)

index and the gold standard in terms of treatment

that is deemed medically necessary? Why is there a

difference between the opinions of the orthodon-

tists reviewing the cases as the fiscal intermediaries

and the orthodontists submitting the cases to be

approved for treatment? Are both sets of opinions

not considered to be the ‘gold standard’? As a

result of the differences in opinion between

these parties, 53% of the cases submitted by

orthodontists are refused Medi-Cal funded treat-

ment. It has been suggested that better monitoring

and better training in the use of the index are

needed for better ‘budget control’ (20). We believe,

Fig. 3. This model had a gold standard score of 10.8 (out of a maximum of 12) and an HLD (CalMod) score of 24.

Fig. 4. This model had a gold standard score of 10.6 (out of a maximum of 12) and an HLD (CalMod) score of 18.

55

Assessment of the validity of HLD

given our findings, that a comprehensive revision

of HLD (CalMod) use in the State of California

should be undertaken not solely for ‘budget con-

trol’ but to achieve what this index was designed to

accomplish: to indentify the patients with handi-

capping malocclusions.

Another issue that arises from our results is the

sensitivity and specificity of this index. Sensitivity

refers to the proportion of people with disease who

have a positive test result whereas specificity refers

to the proportion of people without disease who

have a negative test result. When an index has a high

sensitivity, a negative result rules out the diagnosis.

On the other hand, when an index has a high

specificity, a positive result rules in the diagnosis.

The HLD (CalMod) index has a low sensitivity

(25.9%), meaning that a score below the cut-off

score of 26 does not rule out the possibility that the

case has handicapping malocclusion and that

orthodontic treatment would be needed. However,

a score of 26 or higher strongly indicates a patient

with orthodontic treatment need. This conclusion is

corroborated by the large number of false negatives

we found (Table 4) and also by several examples

such as the one shown in Fig. 1.

When the sensitivity and specificity were deter-

mined using the new cut-off of 18.5 for HLD (Cal

Mod), specificity decreased (92.9%) and sensitivity

increased dramatically (55.6). Beglin et al. (14)

demonstrated that, with a more lenient cut-off of

12, sensitivity increased to (91.7%) and specificity

decreased to 82.3%.

Poor sensitivity may be a result of cut-off based

on the index’s author’s personal opinion, or a result

of a policy decision that was made without a

validation process to determine a cut-off that

reflects the professional experts’ opinion (gold

standard) (6). If the latter is true, it is then a

question of policy as to where the cut-off will be

placed.

The positive predictive value of a test is the

probability that the patient has the condition being

studied when the test for the condition gives a

positive result. The positive predictive value for

HLD (CalMod) was 63.6 meaning that 63.6% of the

patients with scores of 26 or greater actually have

orthodontic treatment needs or handicapping mal-

occlusions. Conversely, the negative predictive

value is the probability that the patient does not

have the condition being studied (in this case,

orthodontic treatment need) when the test for the

condition is negative. The negative predictive value

for HLD (CalMod) was 85.9% signifying that

almost 86% of the models that score 26 or below

do not need orthodontic treatment.

This is somehow a high value. However, unlike

specificity and sensitivity, predictive values are

heavily influenced by the prevalence and less

influenced by the cut-off level. For instance even

when the cut-off was lowered to 18.5, the positive

predictive value was still low at 62.5% and

the negative predictive values was still high at

90.7%.

In the State of California, the HLD (CalMod)

index is used to identify those with handicapping

malocclusions with the premise that funding

should be provided based on a physical handicap

caused by the malocclusion. Our results show that

the HLD (CalMod) with a cut-off of 26 fails to

indentify a considerable percentage of handicap-

ping malocclusions. We recommend further stud-

ies examining patient characteristics that might be

associated with misclassification.

References1. Carlos JP. Evaluation of indices of malocclusion. Int

Dent J 1970;20:606–17.2. Tang EL, Wei SH. Recording and measuring maloc-

clusion: a review of the literature. Am J OrthodDentofacial Orthop 1993;103:344–51.

3. Shaw WC, Richmond S, O’Brien KD. The use ofocclusal indices: a European perspective. Am JOrthod Dentofacial Orthop 1995;107:1–10.

4. Han H, Davidson WM. A useful insight into 2occlusal indexes: HLD(Md) and HLD(CalMod). Am JOrthod Dentofacial Orthop 2001;120:247–53.

5. American Association of Orthodontists. AmericanAssociation of Orthodontists Bulletin 2001(Fall 2001).

6. Younis JW, Vig KW, Rinchuse DJ, Weyant RJ. Avalidation study of three indexes of orthodontictreatment need in the United States. CommunityDent Oral Epidemiol 1997;25:358–62.

7. Draker HL. Handicapping labio-lingual deviations:A proposed index for public health purposes. Am JOrthod 1960;46:295–305.

8. Code of Maryland Regulations. Attachment No 1,Baltimore, MD: EPSDT Services Regulation; 1992.

9. DeGuzman L, Bahiraei D, Vig KW, Vig PS, WeyantRJ, O’Brien K. The validation of the Peer AssessmentRating index for malocclusion severity and treatmentdifficulty. Am J Orthod Dentofacial Orthop1995;107:172–6.

10. Parker WS. A study of 1000 malocclusions selectedby the HLD (CalMod) Index. Am J Orthod Dento-facial Orthop 1999;115:343–51.

11. Brook PH, Shaw WC. The development of an indexof orthodontic treatment priority. Eur J Orthod1989;11:309–20.

12. Richmond S, Daniels CP. International comparisonsof professional assessments in orthodontics: Part

56

Cooke et al.

2–treatment outcome. Am J Orthod DentofacialOrthop 1998;113:324–8.

13. Richmond S, Shaw WC, O’Brien KD, Buchanan IB,Jones R, Stephens CD et al. The developmentof the PAR Index (Peer Assessment Rating):reliability and validity. Eur J Orthod 1992;14:125–39.

14. Beglin FM, Firestone AR, Vig KW, Beck FM, KuthyRA, Wade D. A comparison of the reliability andvalidity of 3 occlusal indexes of orthodontic treat-ment need. Am J Orthod Dentofacial Orthop2001;120:240–6.

15. Parker WS. Useful data from application of the HLD(CalMod) Index. Am J Orthod Dentofacial Orthop2000;117:435–7.

16. Parker WS. The HLD (CalMod) index and the indexquestion. Am J Orthod Dentofacial Orthop1998;114:134–41.

17. Breiman L. Classification and regression trees. Bel-mont, Calif.: Wadsworth International Group; 1984.

18. Salford Systems; available at: http://salford-sys-tems.com/cart.php [last accessed 20 September2009], 2009.

19. Gansky SA. Dental data mining: potential pitfalls andpractical issues. Adv Dent Res 2003;17:109–14.

20. Theis JE, Huang GJ, King GJ, Omnell ML. Eligibilityfor publicly funded orthodontic treatment deter-mined by the handicapping labiolingual deviationindex. Am J Orthod Dentofacial Orthop 2005;128:708–15.

57

Assessment of the validity of HLD