Auditing and self-assessment model for youth work

Meiju Hovi 2013(Kirsi Autio modified from training material)

Assessment model—what is it?

• A tool for assessing the quality of youth work and developing it

• Centring on learning from one's work and developing work through the learning

• The assessment criteria exist for five activity forms1. Open youth evenings (youth centres)2. Small groups with goals3. Camping activities4. Online-based youth work5. Online game activities

Assessment model—what is it?

• The logic of the criteria for the assessment work model comes from the auditing and assessment model in use in the UK (www.ofstedt.gov.uk)

• In Finland, the assessment model of youth work emphasises self-learning in the activities and at the sites and peer auditing in developing the work instead of external audits

• The assessment model has been developed in collaboration youth leaders

• The criteria are updated based on the experiences obtained

• Used in various parts of Finland

Assessment model—why?

• Youth work requires a qualitative assessment tool quantitative indicators are an important but insufficient way

of assessing youth work

• Assessment is needed as the foundation of the development work

• Helps identify the strengths and areas of development of the activities

• Youth workers (and why not the youths!) find the criteria an easy way to understand what types of components good work is made up of

• With the criteria and levels, a shared meaning and understanding is created for the content of high-quality activities

Assessment model—how does it work?• Peer auditing

– Youthworkers´colleagues perform the auditing and give feedback on the evening/audit event in question based on their observations

– The auditing is always done in pairs so that the assessment is always based on the observations of the same event by at least two people.

– The auditor pairs are formed so that auditors come from different youth centres or from a neighbouring municipality, for instance. For example, a regional auditing structure has been developed in south-eastern Finland as collaboration between several municipalities.

• Self-assessments– The work community/employee assesses the activities for a

specific period of time, selects the areas of improvement and prepares an action plan for the development with his/her supervisor

Using the criteria of the auditing model in audits

The target of the audit is solely the activity form agreed upon (open youth work , small groups with goals, camp activities etc.)

– On the one hand, it is important for the auditor to restrict the target of the audit on the one hand and, on the other, visualise the big picture in which the activities being audited belong

• The activities and actions, i.e., what can be seen happening, are audited

– The goals are not audited– The employees’ personal skills or tendencies are not

assessed. For example, when assessing criterion 5 (activeness of the interaction), the instructors are not assessed as individuals; instead, the focus is on how present they are in the activities of the evening and with the youths. Each instructor naturally implements their presence through their personal style, some for example being more talkative than others.

Work community activities Activities of the youths Resources

1. Division of responsibilities of the instructors2. Utilising the expertise of the instructors3. Space control4. Situation management5. Activeness of the interaction6. Meeting the youths as individuals7. Utilising customer feedback in planning the activities8. Utilising indicators in planning the activities9. Knowledge of the area10. Collaboration and partnership11. Partnership networks12. Planning of activities13. Gender sensitivity14. Opposing racism15. Ethnic equality16. Youth work methods and tools17. Environmental education18. Media education19. Promoting a healthy lifestyle20. Collaboration with the homes21. Operating models22. Rules23. Safety24. Interaction between the instructors25. Internal flow of information and communication between the instructors26. Behaviour and speech culture27. The youths’ learning

28. Scope of involvement29. Decision-making structure30. The youths’ participationin communication31. Peer activities32. Division of duties between the

youths, workload33. Interaction between the youths

34. Instruction resources35. Utilising the spaces36. Furnishing of the spaces37. Activity equipment38. Computers39. Cleanliness and general appearance of the premises40. Sustainable development—sorting and recycling41. Sustainable procurements42. Energy consumption43. Information material targeted at the youth44. Communicating about the activities

ASSESSMENT CRITERIA: Open youth evenings at youth centres

Criteria level descriptionsEach criterion has four assessment levels• Excellent (4), good (3), satisfactory (2), poor/insufficient (1)

Logic of the criteria and levels • The level descriptions are explanations of what is required to reach that level• Some of the levels are normative, such as safety• Good activities or policies mentioned for a lower level are included in the

higher level but not separately mentioned• At the excellent level, the actions are common to the entire work community

or pair; an individual employee who works excellently cannot reach the excellent level

• The cannot assess column• Used if the observer notices when making the common assessment that the

observations are not sufficient to assess the criteria and the instructors were not asked about it in more detail

• NOT a so-called 0 level If the activities are assessed so that they do not reach level 1, level 1 is

still used and the issue is mentioned in the auditing report in the notes column or as an area of development

Example of a criterion

ASSESSMENT CRITERION

4 EXCELLENT 3 GOOD 2 SATISFACTORY 1 INSUFFICIENT/POOR

10. Gender sensitivity

Conscious actions of the instructors exceed the traditional borders of gender roles.

The actions of the instructors enable all youths to have physical and mental space to work equally at the centre.

The activities have been planned so that they are attractive to girls and boys.

Gender sensitivity has not been taken into account in the planning and implementation of the activities.

 

Auditing process

• (Selecting and informing the sites to be audited)

• Preparing for the audit (peer auditors have undergone auditing training)

• Auditing situation and the auditor’s role• Assessment• Auditing report• Feedback meeting

Preparing for the audit

• Contact the site to be audited and agree on practical measures

– Times of the audit and feedback meeting

• Getting to know the youth facility’s website

• Reviewing the criteria

• Division of duties with the audit pair– Both will make observations but the criteria that must be asked

about can be divided– Agree what topics to discuss with the youths, what division of

work

• Division of work in writing the audit report

Auditor’s role

• Inspects issues professionally and ethically• Bases the observations on criteria• Remembers that the assessment is confidential• In the audit situation, is sensitive to what he/she listens,

sees and observes, and asks questions if needed• Does not compare the activity being audited with that of

his/her own site is open to new issues and views• Can and wants to work interactively• Works as the “critical friend” in the feedback event

– Does not give instructions or advice or order areas of development– Is appropriate and fair– Can give constructive, honest and encouraging feedback– Believes in openness

The audit• Observation = listening, watching, feeling the atmosphere,

“expresses curiosity”• Presenting questions

to the instructors to the youths

• Writing down observations to the criteria set or audit form• The criteria guide the observation and presenting of questions• Duration 1.5-2h• Check with the auditing pair/group during the audit:

• what criteria need more information, what has been discussed

with the youths and instructors • Write down observations during the audit the report

will be written faster and you remember the justifications for the assessment better

Assessment

• The assessment is done in stages. First, as individual work, an assessment of the evening using the criteria set and level descriptions.

– At this stage, you can use half points, i.e., note that something is between good and satisfactory

• preparing the shared assessment with the work pair/audit group

– Come up with a shared view and assessment– If everyone agrees on the assessment, no discussion is needed,

but the justifications of the assessment should be stated, particularly for the criteria where the level is satisfactory or insufficient/poor

– The cannot assess column is used if the observations and discussions did not provide sufficient information for the audit note in the additional information column

– Select the strengths and areas of development and justify them

Auditing report• The task of the report is to document the audit and benefit the work

community, instructors and organisation, helping them develop the audited activity

• Write as soon after the auditing as possible—agree on who writes the final form of the report

• A successful auditing report is clear and highlights a suitable number of central strengths and development areas: 1–3 of both

– a strong list of strengths may cause “inflation”– a long list of criteria that need development may make it more

difficult to choose the areas that will be developed and decrease the desire to develop the work

• Not everyone who uses the report may have been present in the audit and/or the feedback meeting make the text as descriptive as possible so that a person who was not present gets an idea of the content and a feeling of the evening and the assessment of the auditors

• The strengths and areas of development are written as whole sentences. Also their justifications.

Areas of development• The areas of development should preferably be selected

directly from the criteria• When describing the areas of development, you can use the

expression “on the one hand… on the other…”as follows:– “A structure has been created for the involvement activities (centre

meetings) but the instructor still dominates the related activities. As an area of development, we see the better use of the existing structures, “blowing life” into the structures. The instructors’ work attitude conveyed an interest in listening to the youths, so the premise for the involvement activities is excellent.”

• Do not use expressions such as “must,” “has to be,” “is necessary” or other imperatives, as the auditor’s role is not to rule on what criteria should be developed or how

• You should not, however, be shy about giving critical and constructive feedback, as the work community/instructor usually is aware of the areas that need development. The way in which the matters are expressed is extremely important.

Recording strengths• The strengths are usually found in the criteria that have been

marked excellent or good• A strength can be a thought, practice, policy or idea that

contains the aspects of more than one criterion, e.g.,– “The premises are really clean, good and efficiently used. The

community exudes a warm spirit and the mutual interaction works in all directions.”

• A criterion that as a whole is not yet excellent or good can also be listed as a strength, if there are related issues for which you want to give encouraging feedback, e.g.,

– “Although the collaboration with the networks does not appear to work particularly well, the instructors’ active style, persistence and search for creative solutions for activating networks is worth noting.”

• In the strengths, you can describe the observed atmosphere, e.g.,

– “The atmosphere at the centre was genial and safe during the evening.”

Feedback meeting• The audit report is sent in advance to the audited work community and its

supervisor• The feedback meeting is arranged shortly after the audit so that the justifications

for the assessment are still fresh in everyone’s memory. The auditors participate in the meeting, along with the entire work community and the supervisor.

• The duration of the meeting is approximately 1h; one of the auditors is the chairperson

• Idea of the audit—the colleagues provide their “observations for use by the work community” to support the development of the work

• Not all criteria need to be reviewed but auditors must be able to justify the level of the assessments made if necessary

– the work community can rectify misunderstandingsthe criterion assessment level can be changed, if justified

• The selected strengths and areas of development are discussed• The feedback is given with respect to the criteria• The feedback targets actions and issues, not people• The feedback is a shared assessment of the audit pair/group of the activities

– in the feedback meeting, it is not fair to say, for example, “Well, in my opinion your activity was very well planned but…”

• During the feedback meeting, avoid discussing youth work in general and concentrate on the content of the audit

On utilising the assessment modelAuditing and self-assessment model for youth work is one tool:1. Make the work visible

• Describing the activities and, consequently, youth work helps justify the significance and impact of youth work

2. Outline and focus the discussion on the content of the work; create a mutual understanding and an agreement on the significance of high-quality activities

3. Provide orientation to new workers4. Plan work content and select areas of focus5. Develop assessment expertise and skills for providing

feedback6. Learn from the work of colleagues

• Good ideas and practices (and learning from the not-so-good ones...)