A

OaDAapPAtRisgccCasocar©

K

f

0d

Physiotherapy 94 (2008) 204–211

On-call competence: developing a tool for self-assessment

Sandy Thomas a,∗, Suzanne Gough b, Mary-Ann Broad c, Jane Cross d,Beverley Harden e, Paul Ritson f, Matthew Quint g

a School of Allied Health Professions, Faculty of Health and Social Care, University of the West of England, Glenside Campus,Blackberry Hill, Stapleton, Bristol BS16 1DD, UK

b School of Health, Psychology and Social Care, Faculty of Health, Social Care and Education,Manchester Metropolitan University, Manchester M13 OJA, UK

c Cardiff and Vale NHS Trust, University Hospital of Wales, Cardiff CF14 4XW, UKd School of Allied Health Professions, University of East Anglia, Norwich NR4 7TJ, UK

e Winchester and Eastleigh Health Trust, Eastleigh SO50 5JF, UKf Royal Liverpool Childrens Hospital, Alder Hey Hospital, West Derby, Liverpool L12 2AP, UK

g Queen Alexandra Hospital, Cosham PO6 3LY, UK

bstract

bjective To develop and validate a self-evaluation tool (questionnaire) to identify perceived competence and confidence felt by physiother-pists undertaking respiratory on-call duty.esign A questionnaire to rate specific acute care respiratory competencies was developed from criteria in the Emergency Duty Guidelines.triangulation approach was used to evaluate content validity. The questionnaire was completed by a group of respiratory physiotherapists

nd a group of non-respiratory physiotherapists. Feedback from respiratory physiotherapists was through open (written) questions, and 15articipated in semi-structured telephone interviews.articipants A total of 263 participants: a purposive sample of 180 respiratory physiotherapists attending ‘On course for on-call’ courses inutumn 2005, and a convenience sample of 83 non-respiratory physiotherapists attending on-call training courses or the 2005 Congress of

he Chartered Society of Physiotherapy.esults Thirty-eight questionnaire items were compiled and tested (12 assessment and 10 treatment competencies, 10 range and six confidence

tems). There were 222 responses (86% overall response rate). Cronbach’s alpha found internal consistency for all items (α > 0.9). There was aignificant difference in competence scores between the respiratory group (median 118, interquartile range 111 to 121) and the non-respiratoryroup (median 76, interquartile range 64 to 87) (P < 0.001). Participants endorsed the content validity of competencies. The importance ofurrent, regular clinical experience in determining competence was highlighted. Self-concept, self-efficacy and attitude to on-call may affectompetence rating, and reliance on self-evaluation alone in the assessment of clinical competence was questioned.onclusions Respiratory physiotherapists involved in on-call training supported the development of a tool to assess self-perceived competence,

nd endorsed the content validity of the questionnaire items. The study findings support the validity of the tool as a competency measureince it is able to discriminate between those at different levels of seniority and experience. Participants varied in their approach to rating theirwn competence in unfamiliar situations; this seemed to depend on self-concept and self-efficacy. Since this tool is based on core respiratory

ompetencies, it may also be used as supportive continuous professional development evidence for the Knowledge and Skills Frameworknd professional development portfolios. It also has the potential adaptation for use with physiotherapy students and those working in acuteespiratory care.

2008 Chartered Society of Physiotherapy. Published by Elsevier Ltd. All rights reserved.

eywords: Self assessment; Self-evaluation programmes; Professional competence

I

∗ Corresponding author. Tel.: +44 1666 503821/117 3288626;ax: +44 117 3288408.

E-mail address: sandy.thomas@uwe.ac.uk (S. Thomas).

bm

031-9406/$ – see front matter © 2008 Chartered Society of Physiotherapy. Publisoi:10.1016/j.physio.2008.02.006

; Respiratory therapy

ntroduction

On-call respiratory physiotherapy may be a requirement ofoth National Health Service and private healthcare employ-ent. Physiotherapists have a duty to ensure that they are

hed by Elsevier Ltd. All rights reserved.

iothera

c[tlaiiuombbaiia

bpai‘smatrow

aaotobtUcfu[s[

O

iafataai

oipghptfiipSsit

A

aacmsppmtTttafpttmHesiTtn

rone

P

S. Thomas et al. / Phys

ompetent to practice in the context within which they work1,2]. Those on a respiratory on-call rota may be required toreat acute and critically ill patients with respiratory prob-ems in a range of different hospital wards, high dependencynd intensive care environments. Recently, there has beenncreasing national awareness of deficiencies in the med-cal management of acutely unwell patients, resulting innnecessary mortality and morbidity [3]. The Royal Collegef Physicians [4] recommended that multidisciplinary teamembers working with acutely ill patients should not only

e competent in recognising patients at risk of deterioration,ut should also be able to respond rapidly to such situationsnd undertake appropriate training. These expectations maket even more challenging for ‘non-respiratory’ on-call phys-otherapists to maintain their competence to practice treatingcutely and critically ill patients.

For the purpose of this article, key terms haveeen defined as follows: ‘non-respiratory physiothera-ists/respondents’ encompasses junior physiotherapists andny senior physiotherapist whose normal duties do notnvolve the management of respiratory patients. Hence,respiratory physiotherapists/respondents’ are defined asenior staff whose clinical caseload includes the manage-ent of patients with respiratory conditions or respiratory

nd co-existing conditions. ‘On-call (emergency) respira-ory physiotherapy’ has been defined as the provision ofespiratory/cardiorespiratory/cardiothoracic or combinationsf respiratory and orthopaedic physiotherapy, out of normalorking hours.A literature search using Medline, Cinahl, Embase, Amed

nd ERIC databases (1990 to November 2007) identified 17rticles containing the key words ‘competence’ and ‘on-call’r ‘emergency duty’ physiotherapy. Only five articles fea-ured physiotherapy on-call competence [5–9]. The use ofn-call/emergency duty competencies in New Zealand haseen reported by Reeve [9], and previous research identifiedhe existence of physiotherapy competencies [6,7,10] in theK. To date, no on-call self-evaluation questionnaires or self-

ompetency assessments have been studied or validated. Aurther nine relevant documents were identified using man-al searches relating to guidance for qualifying programmes11], competence and capability [1], professional conduct andtandards [2,11–15], and health professional self-assessment16].

n-call competency

In 2002, members of the Association of Chartered Phys-otherapists in Respiratory Care (ACPRC) raised concernsbout the lack of an accepted national minimum standardor on-call preparation [5,17]. Harden et al. [6] found thatlthough some trusts had apparently robust schemes for

raining on-call competence, there was considerable vari-bility and the assessment of competence was somewhatd hoc. Gough and Doherty [7] also identified variabil-ty in on-call preparation and education. Locally devised

act

py 94 (2008) 204–211 205

n-call competency frameworks are currently being usedn 88% of trusts, and 61% enforce completion prior tohysiotherapists undertaking on-call duties [7]. The Emer-ency Duty Guidelines [17] were developed, piloted andave been used nationally to support physiotherapists andhysiotherapy managers in developing a safe and compe-ent workforce. Development of the ACPRC’s ‘On courseor on-call’ training programme [18] focused on prepar-ng physiotherapists to achieve the competencies identifiedn the Guidelines [17]. However, there is an absence of arofession-specific, respiratory on-call self-evaluation tool.uch a tool would be useful in order to promote consistency oftandards [17], allow physiotherapists to identify their learn-ng needs, and potentially facilitate transferability betweenrusts.

ssessing competence

Competence may be viewed as a broad term that impliesn integration of personal attributes (knowledge, skills andttitudes) within the context of the roles (tasks, competen-ies) required in a job [19,20]. Additionally, the concept ofeta-competence incorporates professional qualities such as

elf-development, communication, creativity, analysis androblem solving [21]. Given the complexity of the com-etency concept, there has been considerable debate and aeasure of mistrust about the acceptability of lists of specific

asks or competence checklists as measures of assessment [1].he Chartered Society of Physiotherapy (CSP) recognises

he need for a holistic approach to competence in physio-herapy, and argues that a technical–rational approach thatttempts to break competence down into measurable per-ormances does not take the complexity of physiotherapyractice into account [1]. Hager and Gonczi [20] suggesthat these two approaches are not mutually exclusive, andhat a list of the competencies (tasks) required for a job role

ay be combined with attributes for an integrated approach.owever, achievement of a series of tasks does not nec-

ssarily imply competence. Since competence is contextpecific, the ability to perform these tasks in one contexts no guarantee that the competence/task is transferable.he CSP [1] acknowledges that competencies and compe-

ency frameworks may be of value when identifying learningeeds.

Key categories for measuring competence are frequentlyelated to a clinical reasoning model which identifies stagesf patient assessment, communication, analysis of patienteeds, treatment planning, interventions, implementation andvaluation of outcomes [8,12–14,17].

urpose of the study

The purpose of this study was to develop and validateself-evaluation tool (questionnaire) to identify perceived

ompetence and confidence felt by physiotherapists under-aking respiratory on-call duty.

2 iotherapy 94 (2008) 204–211

M

R

ift(

I

cdt1d[cataw

P

tr

P

awaitpih

oaciet

I

rTl

Table 1Questionnaire items

Category Item

Assessment 1 I can perform a simple chest assessment throughobservation, palpation and auscultation

2 I can take an appropriate history of an on-callcase

3 I can gain informed consent from patients in anon-call setting

4 I understand the need for confidentiality in anon-call setting

5 The information I collect is accurate andappropriate

6 I can communicate effectively in an on-callsetting

7 I can identify the patient’s main respiratoryproblems

8 I can select the appropriate objective markers9 I can interpret arterial blood gases

10 I can identify chest X-ray findings of relevanceto physiotherapy

11 My analysis of patient’s assessment findings isappropriate

12 I can identify patients who are becomingcritically ill and take appropriate action

Treatment 1 I can produce an effective treatment plan2 I can implement my treatment plan3 I can direct the future care for patients after my

intervention4 I can discuss management plans with patients

and carers5 I can discuss management plans with other

professionals6 I can use appropriate equipment safely and

effectively7 I can treat problems associated with volume loss8 I can treat problems associated with sputum

retention9 I can treat problems associated with increased

work of breathing10 I can manage patients who develop respiratory

failure

Range I feel able to perform a safe and effectiveassessment and treatment for:

1 Patients following abdominal surgery2 Patients following cardiothoracic surgery3 Patients on intensive care unit4 Patients with chronic respiratory disease5 Patients with multiple trauma6 Patients with unstable spine7 Patients with neurological deficits8 Patients who are unstable (e.g. cardiovascular

instability)9 Paediatric ward–babies and children with

respiratory problems10 Paediatric intensive care unit–babies and

children

Confidence 1 I feel worried about being on-call2 I feel well prepared to be on-call3 I feel supported by my senior staff when on-call

06 S. Thomas et al. / Phys

ethodology

esearch process

The research reported in this article relates to the stepsnvolved in the development of a questionnaire to be usedor a national survey planned for 2008. The final version ofhe questionnaire is now available on the ACPRC websitewww.acprc.org).

tem construction

A comprehensive checklist of assessment and treatmentompetencies (items) specific to acute respiratory care waserived from the Emergency Duty Guidelines and compe-ency literature. Twenty-eight items were initially identified,2 for patient assessment and decision making (CSP Stan-ards 2–7 [12]), 10 for patient treatment (CSP Standards 8–1812]) and six to measure the construct of confidence in on-all work. Categorical response questions for each item usedconsistent set of Likert scales with five options graded 4

o 0 (strongly agree, agree, neither agree nor disagree, dis-gree and strongly disagree, respectively). The initial draftas piloted with 48 physiotherapists across five trusts.

ilot feedback

Feedback led to some amendments including the addi-ion of 10 range items for context-specific competence. Theevised questionnaire was used for the main study (Table 1).

articipants

One hundred and eighty respiratory physiotherapiststtending ‘On course for on-call’ courses in Autumn 2005ere invited to participate by completing a questionnaire

nd giving feedback through open questions. This numberncludes 14 questionnaires posted to therapists nominated byhe course attendees. This purposive sample of physiothera-ists with a specific interest in on-call training and expertisen respiratory physiotherapy represented 72% of UK acuteospital trusts.

Eighty-three non-respiratory physiotherapists attendingn-call training courses and those on an on-call rota whottended the 2005 Congress of the CSP were invited toomplete the questionnaire. This convenience sample wasntended to include physiotherapists at different levels ofxperience (from the Congress) as well as juniors undertakingheir first on-call training programmes.

nterviews

Follow-up interviews were conducted with 15/138 (11%)espiratory participants who left contact details at the courses.he semi-structured, tape-recorded telephone interviews

asted for 15–20 minutes, and explored the participants’ rat-

4 I feel I have something to offer my patients5 I feel able to judge how appropriate a call is6 I feel confident working alone when on-call

iothera

ia

D

alutdarucaifost

R

Oialrc

mtctT

Q

w

R

sit‘

N

(dobric

TC

C

E

R

C

S. Thomas et al. / Phys

ng decisions and their views on content validity, clarity,mbiguity and comprehensiveness of items in the tool.

ata analysis

Quantitative data were analysed using Statistical Pack-ge for the Social Sciences Version 13 [22]. A significanceevel of P = 0.05 was utilised. A Mann–Whitney test wassed to identify differences in competence scores betweenhe two groups. A principal components analysis was con-ucted on 36 questionnaire items to explore factor structure,nd item congruity was tested using Cronbach’s alpha. Aeceiver operating characteristic (ROC) curve analysis wassed to identify a ‘cut-off’ point in the competence score thatould approximate a ‘pass mark’. Interviews were transcribednd content analysed by two independent researchers whodentified key themes and fed these back to the intervieweesor comments and agreement. Key themes from responses topen written questions were also identified until the point ofaturation, and these were then integrated with the interviewhemes.

esults

An overall response rate of 86% (222/263) was achieved.ne questionnaire was excluded due to several missing

tems, leaving 221 questionnaires for analysis. Scores for

ssessment, treatment, range and confidence were calcu-ated for each respondent based on totalling individualatings (0–4) for each scaled item. ‘Paediatric intensiveare unit’ (R10) and ‘I feel I have something to offer

faw

able 2linical experience

Non-respiratory phys

linical gradeStudent physiotherapist 15 (18)Junior physiotherapist 50 (60)Senior 2 physiotherapist 4 (5)Senior 1 physiotherapist 11 (13)Clinical specialist/Educator/Manager 3 (4)

xperienceLess than 2 years qualified 65 (78)3–5 years qualified 2 (2)Over 6 years qualified 16 (19)

espiratory caseloadRespiratory = all of caseload 2 (2)Respiratory = most of caseload 4 (5)Respiratory = some of caseload 16 (19)Rarely see respiratory patients 6 (7)Never see respiratory patients 55 (66)

linical settingInpatient only 32 (39)Outpatient only 10 (12)Community only 6 (7)Combination 16 (19)Other/student 19 (23)

py 94 (2008) 204–211 207

y patients’ (C6) were excluded from this analysis ashey were missing from some questionnaires. The clini-al grade, years of experience and respiratory caseload ofhe respiratory and non-respiratory groups are shown inable 2.

uestionnaire data analysis

Most items had a 100% response rate. Response ratingsere not normally distributed for any individual item.

espiratory respondentsOver 90% of respiratory respondents either agreed or

trongly agreed with all of the assessment and treatmenttems. Only two items (range statements) had a mean of lesshan 3 (agree) for the respiratory group: ‘Paediatric’ (R9) andPatients with unstable spine’ (R7).

on-respiratory respondentsThe only items that scored a mean mark higher than 3

agree) were ‘Auscultation and palpation’ (A1) and ‘Confi-entiality’ (A4). Table 3 gives median scores for each sectionf the questionnaire. A significant difference was foundetween the median total competence score for the respi-atory and non-respiratory groups (P < 0.001). This is shownn Fig. 1. Fig. 2 illustrates a trend in the relationship betweenlinical grade and competency score.

Principal components analysis on 36 questionnaire itemsound only one factor accounting for 70% of the vari-nce. This suggests a close association between items thatas supported by a Cronbach’s alpha coefficient of 0.984.

iotherapists n (%) Respiratory physiotherapists n (%)

0 (0)0 (0)

14 (10)75 (54)49 (36)

5 (4)28 (20)

105 (76)

48 (35)62 (45)26 (19)

1 (1)1 (1)

80 (58)0 (0)1 (1)

56 (41)1 (1)

208 S. Thomas et al. / Physiotherapy 94 (2008) 204–211

Table 3Median and interquartile ranges (IQR) for each section of the questionnaire

Non-respiratoryphysiotherapists

Respiratoryphysiotherapists

Median (IQR) Median (IQR)

Assessment score 31 (27 to 37) 48 (46 to 48)*

Treatment score 26 (22 to 30) 40 (38 to 40)*

Range score 18 (15 to 22) 31 (27 to 33)*

Total competence score 76 (64 to 87) 118 (111 to 121)*

Confidence score 9 (6 to 12) 19 (16 to 20)*

* Significant difference between the two groups (P < 0.001).

F

Tbat

s2ig

F

T

d

••••

P

hu

‘t

‘i

‘

TF

ATRC

TC

C

ig. 1. Box plot of competence scores for each group.

ables 4 and 5 highlight significant correlations (P < 0.001)etween the section scores for assessment, treatment, rangend confidence, suggesting that these are too strongly relatedo be considered as separate factors.

An ROC curve analysis identified that a cut-off point (pos-

ible ‘pass mark’) of 75% would give 92% sensitivity and1% specificity (the possibility of false-positives occurring)n distinguishing between the respiratory and non-respiratoryroups.

o7

f

able 4actor analysis and alpha (section scores)

Number of items Number of responses

ssessment score 12 221reatment score 10 221ange score 9 221onfidence score 5 221

able 5orrelation matrix (section scores)

Treatment score

orrelation Assessment score 0.959*

Treatment scoreRange score

* Significant correlation (P < 0.001).

ig. 2. Total competence score for each clinical grade.

hemes from interviews and open questions

Four main themes emerged from the integrated qualitativeata, relating to:

perceived value of the tool;specific recommendations for changes and additions;competence rating decisions; andconcerns about self-assessment.

erceived value of the toolMost participants welcomed the tool and felt that it could

elp to standardise respiratory competence nationally and beseful for transferability between trusts.

I just think this sort of thing is absolutely necessary’ (Par-icipant 3)

It can only bring more unity of standards for physios workingn respiratory’ (Participant 2)

If it’s standardised by the CSP or the ACPRC, anybody that’sutside the trust, people might take it a bit better’ (Participant

)

Participants identified that the main use of the tool wasor individuals to recognise their learning needs, particularly

Variance explained by each factor (%) Cronbach’s alpha

91.6 0.9644.41 0.9752.71 0.9211.25 0.920

Range score Confidence score

0.867* 0.867*

0.864* 0.858*

0.799*

iothera

ps

‘r

‘t8

S

miei

Cs

fw

‘n(

‘dt

‘ys8

ussmemrito

‘m(

‘b

‘ti(

D

npoiasotsr

cyowiolacitarais

rppfelpbuufiediEbdwa

S. Thomas et al. / Phys

rior to training sessions, which could then be geared morepecifically to ‘problem’ areas.

Such a tool as you are developing would not allow people toemain unconsciously incompetent’ (Participant 2)

I think it makes a more structured discussion. . ..to make sureime spent with them is so much more useful’ (Participant)

pecific recommendations for changes and additionsThe specificity and relevance of treatment and assess-

ent items was well supported. Although participants feltt was important to keep the tool as succinct as possible, oth-rs felt that a little more detail was needed, particularly fornexperienced staff.

ompetence rating decisions and concerns ofelf-assessment

Recent and regular experience was the main reason givenor strongly agreeing with items; however, where experienceas not regular/recent, ratings were more variable.

I disagreed with R2 (Cardiothoracic surgery) because I haveever done it, therefore would not be safe and effective’Participant 13)

I put neither agree nor disagree with the statement on pae-iatrics as I don’t see them. I have the skills but would needraining’ (Participant 1)

I haven’t basically treated a cardiothoracic patient for about 7ears. . .but I put agree because I still felt I had the appropriatekills to do a thorough assessment and treatment’ (Participant)

Participants in this study perceived that individuals shouldltimately be responsible for their own competence; however,ome had significant reservations about the willingness ofome to accept this responsibility, and the ability of others toake an accurate self-evaluation. Participants also acknowl-

dged that they did not have time to ensure that all staffembers were competent, yet felt uneasy about leaving the

esponsibility up to the individual. Concern was raised thatnexperienced staff may not be able to self-evaluate effec-ively, and may over-rate or under-rate their skills dependingn their self-concept.

It’s not about me going round to everyone individually andaking them competent. . .it’s a physically impossible task’

Participant 8)

I think it’s just difficult to get individuals to take responsi-

ility for their own competency’ (Participant 11)

I know it’s their responsibility to make themselves compe-ent to do on-call, but I still have this little worry that if I leavet completely up to them, and then something did happen. . ..’Participant 6)

watsa

py 94 (2008) 204–211 209

iscussion

This study generated an on-call self-assessment question-aire to identify perceived competence and confidence felt byhysiotherapists undertaking respiratory on-call duty. Resultsf this study indicated strong internal consistency betweentems in each category including assessment, treatment, rangend confidence. The high response rate for each item demon-trates acceptability of this tool to physiotherapists. Resultsf the factor analysis identified that competencies within theool were core to acute/on-call respiratory physiotherapy, andupported the inclusion of each item within the construct ofespiratory competence.

The tool highlighted a trend in the relationship betweenlinical grade and competency score. The ROC curve anal-sis identified that a cut-off point (possible ‘pass mark’)f 75% would identify true-positives on 92% of occasions,ith the possibility of false-positives (non-respiratory phys-

otherapists achieving a score of over 75%) occurring 21%f the time (specificity). Specificity may have been evenower but was adversely affected by a number of physiother-pists in the non-respiratory group (26%) whose caseloadurrently included some respiratory patients. These find-ngs support the ability of the tool to distinguish betweenhe respiratory and non-respiratory groups, and its potentials a measure of competency. A pass mark of 75% wouldepresent a minimum score for a clinician who is able togree with all of the competence items, and would, accord-ng to this preliminary study, have acceptable sensitivity andpecificity.

Competency concerns were raised relating to recent andegular experience. This was evident in the range of com-etence section, and seemed to depend on the individual’serspective of how transferable their skills were to dif-erent types of patient. The importance attached to recentxperience appears to depend on the individual’s under-ying self-concept and self-efficacy. Therapists may haveotentially transferable assessment and treatment skills,ut while some have the self-efficacy to apply them innfamiliar situations, others have an inherent fear of thenknown and need regular, recent experience to feel con-dent. These findings are comparable with those of Stewartt al. [23], who found that negative expressions of confi-ence were more likely to be related to anxiety than perceivedncompetence, and that anxiety was lessened by experience.va and Regehr [16] argue that self-efficacy is not sta-le and may vary in response to trivial factors. This castsoubt on both validity and reliability of the rating schemehen used purely as a self-assessment tool for summative

ssessment.Under-rating due to lack of confidence can be addressed

ith support and training; however, those who over-rate may

ppear over-confident because they are not fully aware ofhe competencies required, or may be reluctant to admit anyhortcomings. Kruger and Dunning [24] found that those whore least competent are also those who are least capable of

2 iothera

jbbmiwa

tmmtcfbnahtataatoisavd

idampao

L

cmauiSucsKesia

ipvtSa

C

scttttabmnaiqdaimbuddfa

A

bsfsDWM

E

F

C

10 S. Thomas et al. / Phys

udging their own competence. This concern was also raisedy participants in the current study. Similar findings have alsoeen reported with undergraduate psychology students [24],edical students [25] and qualified doctors [26]. Orest [27]

nterviewed physiotherapists who felt that self-assessmentas not an easy process, and required a degree of experience

nd self-awareness.Participants perceived that the tool may be useful in iden-

ifying/evaluating competencies, but there seems to be ainority who, either due to lack of awareness of require-ents, inappropriate levels of confidence or a reluctance

o work on-call, cannot be relied upon to evaluate theirompetence effectively. Regehr [28] argues that health pro-essionals may not only fail to identify gaps in their learning,ut may also lack the motivation to address their learningeeds. This self-evaluation tool may help these individu-ls by raising their awareness of competencies required;owever, further intervention and support is also likelyo be needed. Eva and Regehr [16] suggest that for self-ssessment to be effective, it is important for individualso seek feedback from peers, teachers and experts as wells from self-administered tests. Although some trusts havelready developed their own self-evaluation/competencyools [6,7], a nationally accepted tool would be useful inrder to promote consistency of standards and transferabil-ty between trusts. Most participants in this study felt thatelf-evaluation should only constitute part of competencessessment, but agreed that this national tool could make aaluable contribution to the process and supported its futureevelopment.

This self-evaluation tool may have an important rolen supporting individuals in their continuing professionalevelopment. Importantly, the self-assessment approachcknowledges that responsibility for competent practice ulti-ately lies with the individual. Whilst the tool may assist

hysiotherapists to identify specific learning needs, it couldlso facilitate on-call trainers to plan and evaluate their trust’sn-call training programme.

imitations of this study

This on-call self-evaluation tool focused on the specificompetencies required for applying the clinical reasoningodel in respiratory on-call situations. Other important skills

nd attributes, such as learning behaviour, attitudes and val-es, professional behaviour and record keeping, have not beenncluded, thus it cannot capture competence in its entirety.uch generic attributes are, however, included in undergrad-ate physiotherapy programmes in accordance with CSPurriculum guidelines [11] and Quality Assurance Agencyubject benchmarking statements [15], and feature in thenowledge and Skills Framework [13]; hence there is cov-

rage at both under- and postgraduate levels. The purposiveample may not represent the views of all respiratory phys-otherapists involved in on-call training, only those whottended the ‘On-course for on call’ courses. External valid-

R

py 94 (2008) 204–211

ty of the findings is limited, as the sample of non-respiratoryhysiotherapists was not selected at random. A national sur-ey using the final questionnaire is now planned to addresshis issue and test repeatability and responsiveness to change.ome ambiguity led to missing data that limited the use ofll the range and confidence items for the final scoring.

onclusions

Respiratory physiotherapists involved in on-call trainingupported the development of a tool to assess self-perceivedompetence, and endorsed the content validity of the ques-ionnaire items. Participants varied in their approach to ratingheir own competence in unfamiliar situations; this seemedo depend on self-concept and self-efficacy. Due to concernshat self-evaluation alone may be unreliable in some cases, theccompaniment of additional measures of competence maye appropriate in clinical practice. This self-evaluation toolay assist on-call physiotherapists by raising their aware-

ess of competencies required; however, additional supportnd education are also likely to be needed. Further researchs required to assess repeatability and responsiveness of theuestionnaire to change. However, the tool does appear toemonstrate an ability to distinguish between respiratorynd non-respiratory therapists, and between different clin-cal grades. This supports its potential use as an outcome

easure for future experimental research. Since this tool isased on core respiratory competencies [17], it may also besed as supportive continuous professional development evi-ence for Knowledge and Skills Framework and professionalevelopment portfolios. It also has the potential adaptationor use with physiotherapy students and those working incute respiratory care.

cknowledgements

The authors would like to thank the participants, the mem-ers of the ACPRC for their considerable contribution to thistudy, and colleagues at the School of Allied Health Pro-essions at the University of the West of England for theirupport during this research. The authors also wish to thankr Paul White and Dr Shea Palmer at the University of theest of England, and Dr Abebaw Yohannes at Manchesteretropolitan University.

thical approval: Southmead Research Ethics Committee.

unding: University of the West of England.

onflict of interest: None.

eferences

[1] Chartered Society of Physiotherapy. Competence and capabilityresource pack. London: Chartered Society of Physiotherapy; 2005.

iothera

[

[

[

[

[

[

[

[

[

[[[

[

[

[

[

[

S. Thomas et al. / Phys

[2] Health Professions Council. Standards of conduct performance andethics. London: Health Professions Council; 2003.

[3] National Confidential Enquiry into Patient Outcome and Death. Anacute problem. London: National Confidential Enquiry into PatientOutcome and Death; 2005. Available at: http://213.198.120.192/2005report/NCEPOD Report 2005.pdf.

[4] Royal College of Physicians. Acute medicine making it work forpatients–a blueprint for organisation and training. London: RoyalCollege of Physicians; 2004. Available at: http://www.rcplondon.ac.uk/pubs/books/AcuteMedicine/.

[5] Cross J, Harden B, Thomas S, ten Hove R. Competence in on-callphysiotherapy part two: developing an assessment tool. Int J Ther Rehab2003;10:364–7.

[6] Harden B, Cross J, Thomas S. An exploration of emergency respiratoryon-call service provision within the United Kingdom. J Assoc ChartPhysiother Respir Care 2005;37:17–21.

[7] Gough S, Doherty J. Emergency on-call duty preparation and educationfor newly qualified physiotherapists: a national survey. Physiotherapy2007;93:37–44.

[8] Thomas S, Cross J, Harden B, ten Hove R. Competence in on callphysiotherapy part one: designing a framework. Int J Ther Rehab2003;10:321–7.

[9] Reeve J. A survey of physiotherapy on-call and emergency duty servicesin New Zealand. NZ J Physiother 2003;31:75–83.

10] Cross V. Approaching consensus in clinical competence assessment.Physiotherapy 2001;87:341–50.

11] Chartered Society of Physiotherapy. Curriculum framework for qual-ifying programmes in physiotherapy. London: Chartered Society ofPhysiotherapy; 2002.

12] Chartered Society of Physiotherapy. Core standards of physiotherapy

practice. London: Chartered Society of Physiotherapy; 2005.

13] National Health Service. Knowledge and skills framework. London:Department of Health; 2004.

14] Health Professions Council. Standards of proficiency. London: HealthProfessions Council; 2005.

[

[

Available online at www.s

py 94 (2008) 204–211 211

15] Quality Assurance Agency for Higher Education. Subject bench-marking statements. London: Quality Assurance Agency for HigherEducation; 2001.

16] Eva K, Regehr G. Self-assessment in the health professions: a reformu-lation and research agenda. Acad Med 2005;80:S46–54.

17] Chartered Society of Physiotherapy. Emergency respiratory, on callworking: guidance for physiotherapists. Information paper no. PA53.London: Chartered Society of Physiotherapy; 2002.

18] Quint M, Broad M, Harden B, editors. On-course for on-call CD-ROM.Association of Chartered Society of Physiotherapists in RespiratoryCare, London; 2005.

19] Eraut M. Concepts of competence. J Interprof Care 1998;12: 127–39.20] Hager P, Gonczi A. What is competence? Med Teach 1996;18: 15–8.21] Cheetham G, Chivers G. Towards a holistic model of professional

competence. J Eur Ind Train 1996;20:20–30.22] SPSS. Statistical package for the social sciences version 13.0. Chicago:

SPSS Inc.; 2004.23] Stewart J, O’Halloran C, Barton JR, Singleton SJ, Harrigan, Spencer

J. Clarifying the concepts of confidence and competence to pro-duce appropriate self-evaluation measurement scales. Med Educ2000;34:903–9.

24] Kruger J, Dunning D. Unskilled and unaware of it: how difficulties inrecognizing one’s own incompetence lead to inflated self-assessments.J Pers Soc Psychol 1999;77:1121–34.

25] Wooliscroft JO, TenHaken J, Smith J, Calhoun JG. Medical students’clinical self-assessments: comparisons with external measures of per-formance and the students’ self assessments of overall performance andeffort. Acad Med 1993;68:285–94.

26] Hodges B, Regehr G, Martin D. Difficulties in recognizing one’s ownincompetence: novice physicians who are unskilled and unaware of it.

Acad Med 2001;76:S87–9.

27] Orest M. Clinicians perceptions of self-assessment in clinical practice.Phys Ther 1995;75:825–9.

28] Regehr G, Eva K. Self-assessment, self-direction and the self-regulatingprofessional. Clin Orthop 2006;449:34–8.

ciencedirect.com