Mapping

Mapping

Workshop on Universal Reporting Parameters for the Speech

of Individuals with Cleft Palate

Washington DC, April/May 2004

Tara Whitehill, Hong Kong

Proposed Three-Stage Plan

• Evaluating: The usual speech evaluation by the local speech specialist in cleft palate.

• Mapping: Conversion from the local speech evaluation to the universal reporting measures.

• Reporting: Universal parameters that describe the cardinal features of speech in individuals with cleft lip/palate and related craniofacial anomalies regardless of the individual’s primary language.

Already covered

• Evaluation (range of protocols and procedures; key issues)

• Parameters (proposal; key issues)

• Focus: mapping

Fanny LAW Sze Kuen

International Parameters for Cleft Palate Speech: A Pilot Study

Honours dissertation

Bachelor of Science (Speech & Hearing Science)

University of Hong Kong

May 2004

Supervisor: Dr. T. L. Whitehill

DRAFT - in progress

• 6 sets of client forms from 5 centres

• 5 clients each set = 30 client forms

• Original forms only (no recordings, no additional data such as transcriptions, unless provided on original form)

• Two groups of “mappers”– 5 “experts” - members of WG who supplied

forms

– 5 students - little/no previous experience cleft

• Used “current” version of parameters and guidelines

• Students given session to familiarize/

clarify terminology• Mapped from client forms onto parameters

form• Identifying information masked (client, examiner,

centre)

• Provided feedback on ease/difficulty of mapping each parameter– Either for client form or by set (centre)

• Weighted scores - not included in this study

“Outcome measures”

• Agreement– Within expert group– Within student group– Across all mappers

• Feedback (ease/difficulty)

Why use students?

• In future, unlikely mappers will be naïve

• However, “expert” group was restricted to individuals who had developed and debated the parameters

Agreement

• A big problem was “missing data”• OK if client form stated MD (or WNL or NAD),

but what if didn’t? - assume OK? tick MD?

• Calculated

(a) absolute agreement (Condition 1)

(b) disregarding “missing data” (Condition 2)

Trial Results amongexperts

Results amongstudents

Agreement and explanation

1 1 1 1 1 1 1 1 1 1 Agreed among experts, students and allmappers in both Condition 1 and 2.

2 1 1 1 1 2 2 2 2 2 Agreed among experts, and students in bothCondition 1 and 2

Not agreed among all mappers in bothconditions

3 1 MD 2 1 1 2 1 1 1 Not agreed among any groups in eitherconditions

4 1 MD 1 MD 1 1 1 1 1 Agreed among students in Condition 1 Agreed among experts in Condition 2 Agreed among all mappers in Condition 2

5 MD MD NA 2 MD 2 2 2 MD Trial for experts was deleted as there would be

no agreement when the MD and NA wereexcluded.

Agreed among students in Condition 2 Agreed among all mappers in Condition 2

Key: MD = missing data, NA = no response, 1 = WNL/Absent, 2 = Present

Table 1. Examples to illustrate how agreement was defined

Primary Parameters

Hypernasality

Agreement• Experts Condition 1 - 60%

Condition 2 - 63%• Students Condition 1 - 73%

Condition 2 - 73%• All Condition 1 - 57%

Condition 2 - 60%

Not much missing data

Primary Parameters

HypernasalityFeedback/Issues to consider

• Some difficulties when no. scale points differed (e.g., client form - 7 pt. scale; parameters form - 4 pt. scale)

• In such cases, mappers reported difficulty, and agreement decreased

• Implication: explicit instructions?• Another difficulty: “mild to moderate” or “moderate to

severe” (students selected less severe and experts more severe!)

• “Set B” - definition of “mild” = “evident but acceptable”. One expert rater mapped onto WNL.

Primary Parameters

Hyponasality




Condition 2 - 97%

Excellent, when missing data is accounted for

Primary Parameters

HyponasalityFeedback/Issues to consider

• Missing data

• Otherwise, excellent

• (binary scale)

Primary Parameters

Audible Nasal Emissionwith/without Nasal Turbulence

Level One (absent, present, MD)

• Experts Condition 1 - 63%

Condition 2 - 87%

• Students Condition 1 - 77%

Condition 2 - 80%

• All Condition 1 - 57%

Condition 2 - 70%

Level Two (subcategories)


Condition 2 - 60%


Condition 2 - 38%


Condition 2 - 27%

Primary Parameters

Audible Nasal Emissionwith/without Nasal Turbulence

Feedback/Issues to consider• Low agreement for subcategories• “phoneme specific” - e.g. “audible for plosive and /s/”• Confusion about pervasiveness e.g. “mild but

consistent”, “phoneme specific and frequent”• Results of mirror test (if not audible in speech)• (NE w/wout nasal turbulence - not raised here)

Primary Parameters

Weak Oral Pressures




Condition 2 - 46%

Primary Parameters

Weak Oral Pressures

Feedback/Issues to consider• All 9 mappers found this parameter difficult to map• Primarily because of lack of data on client forms - no

form had specific category for this• Mappers needed to derive the information from other

information; particularly difficult for students• Also, students unclear about definition

Primary Parameters

Substitution ErrorsLevel One (absent, present, MD)


Condition 2 - 73%


Condition 2 - 85%


Condition 2 - 70%

Level Two (subcategories)


Condition 2 - 30%


Condition 2 - 15%


Condition 2 - 20%

• Agreement not bad for level one, but very poor for subcategories; missing data was not the main problem

Primary Parameters

Substitution Errors

Feedback/Issues to consider• All forms included level one (present/absent/MD)• Subcategories: most forms had patterns, but few

included frequency information • Some patterns which expert listeners considered

‘related to cleft palate’ not included here (e.g. nasal substitution/realization’, ‘double articulation’)

• Confusion between this parameter and ‘Developmental Delay or Other Articulation/Phonological Errors’ (even for experts) e.g. omission, backing /t/ -> [k].

Secondary Parameters

Errors related to dentition, occlusion, palatal vault configuration




Condition 2 - 53%

Better when missing data taken into consideration, but still pretty poor


Errors related to dentition, occlusion, palatal vault configuration

Feedback/Issues to consider• No specific category for this on client forms;

information had to be extracted from other information

• Some forms (Sets B, E, F) included categories such as ‘lateralization’, ‘palatalization’, ‘dentalization’ - but still not clear whether these due to organic factors or “Other Articulation/Phonological Errors” - even for experts

• Errors related to hearing problem? fistula?


Developmental Delay or Other Articulation/Phonological Errors




Condition 2 - 43%

Again, better when missing data taken into account, but still pretty poor


Developmental Delay or Other Articulation/Phonological Errors

Feedback/Issues to consider• Mappers needed to make judgement regarding

whether errors were related to cleft or not • Confusion with both “Substitution Errors” and “Errors

related to dentition..”• Client forms rarely specified cause or explanation of

error patterns - mappers needed to judge• Special cases e.g. “no oral consonants”


Voice/Laryngeal Disorder




Condition 2 - 96%

Agreement was good, especially when missing data was taken into account


Voice/Laryngeal Disorder

Feedback/Issues to consider• Similar to hyponasality - mappers were familiar with

term• When data was provided on client form, it was clear• Binary choice; no subcategories

Global RatingSeverity/Intelligibility

• Not decided at time of this study

• Only 2 of the 6 sets included such data

Conclusions

• Reliability lower among students than experts

• Experts were those who developed form

• Who will be mapping in the future?

Mapping

Documents

Transcript of Mapping