EVALUATION OF THE SKILLS FOR LIFE QUALITY INITIATIVE 2005-06 An evaluation of the Skills for Life Quality Initiative 2005-06 and a review of its predecessor initiatives FINAL REPORT MAY 2006 Commissioned by LSC through CfBT from the National Research and Development Centre for Adult Literacy and Numeracy (NRDC): the Institute of Education, University of London with the University of Sheffield and Lancaster University

CONTENTS

EXECUTIVE SUMMARY .................................................................................5

Introduction....................................................................................................5

Conclusion .....................................................................................................7

Key findings ...................................................................................................8

Recommendations.........................................................................................9

1. REVIEW OF DOCUMENTATION..............................................................12

1.1 Origins of the quality improvement initiatives ....................................12

1.2 The Basic Skills Quality Initiative .........................................................14

1.3 Skills for Life Quality Initiative..............................................................15

1.4 The wider context for quality improvement in the learning and skills sector ............................................................................................................19

2 CONSORTIUM ACHIEVEMENT AGAINST TARGETS .............................21

2.1 Overall achievement of the consortium's targets ...............................21

2.2 Accredited training ................................................................................22

2.3 Achievement of Skills for Life Quality Initiative priorities for 2005-06.......................................................................................................................22

2.4 Activities undertaken by the Consortium in 2005-06..........................23

2.4.1 Skills for Life Whole Organisational Approach ................................23

2.4.2 Embedding Skills for Life Quality Initiative ......................................23

2.4.3 Leadership and Management Skills ..................................................24

2.4.4 Facilitator Network..............................................................................24

2.4.5 E- Learning ..........................................................................................24

2.4.6 Key Skills .............................................................................................24 2.4.7 Continuing professional development/information, advice and guidance……………………………………………………………………………25

2.4.8 Dissemination of good practice.........................................................25

2.5 The Consortium’s progress on training activities ..............................25

- 2 -

2.6 The Consortium's progress under the 'Facilitator Network Strand' ..25

2.7 Conclusions ...........................................................................................25

3. PERCEPTIONS OF THE SKILLS FOR LIFE QUALITY INITIATIVE ........28

3.1 IMPACT UPON RESOURCES ................................................................29

3.1.1 Quantity and quality of training and materials .................................29

3.1.2 Facilitation ...........................................................................................30

3.1.3 Achievement of targets relating to training and facilitation............32

3.2 IMPACT OF POLICY...............................................................................34

3.2.1 Stop/start nature of the initiative .......................................................34

3.2.2 Regional delivery ................................................................................35

3.2.3 Readiness of the workforce ...............................................................37

3.2.4 Range of initiatives .............................................................................38

3.3 IMPACT ON SYSTEMS...........................................................................38

3.3.1 Organisational change .......................................................................38

3.3.2 Examples of impact ............................................................................40

3.3.3 Impact on other sub-systems ............................................................42 Table 1 Impact of the Skills for Life Quality Initiative upon systems...42

3.4 IMPACT ON PEOPLE .............................................................................43

3.4.1 Benefits for practitioners ...................................................................43

3.4.2 Maintaining progress..........................................................................46

5. CONCLUSIONS AND RECOMMENDATIONS .........................................49

5.1 Conclusions and recommendations summarised under the aims of the evaluation...............................................................................................49

5.2 Overall summary of the Achievements of the Skills for Life Quality Initiative ........................................................................................................51

5.3 The two-way implementation staircase................................................53 ACKNOWLEDGEMENTS………………………………………………………...55

- 3 -

ANNEXES .....................................................................................................56

ANNEX 1 .......................................................................................................56 References ................................................................................................56

ANNEX 2 .......................................................................................................58 Research methods ...................................................................................58 1.1 Influences and design issues............................................................58 1.2 Evidence and rigour ...........................................................................58 1.3 Self-evaluation ....................................................................................59 1.4 The research process ........................................................................59 1.4.1 Desk research ..................................................................................59 1.4.2 Survey ..............................................................................................59 1.4.3 Interviews.........................................................................................60 1.4.4 Focus groups...................................................................................60 1.4.5 Use of other NRDC data..................................................................60 1.4.6 Expert seminar ................................................................................61 Table 2: Overall research design ............................................................62

ANNEX 3 .......................................................................................................64 Skills for Life Quality Initiative online survey ........................................64

ANNEX 4 .......................................................................................................72 CfBT led consortium’s progress towards targets set under the Skills for Life Quality Initiative contract: A report to the NRDC .....................72

- 4 -

EXECUTIVE SUMMARY Introduction NRDC, at Lancaster University, the University of Sheffield and the Institute of Education, University of London, has conducted an evaluation of the impact of the Skills for Life Quality Initiative. The purpose of the evaluation was:

• To assess what has been achieved through the Skills for Life Quality Initiative's operation since 2000; such achievements to be analysed in terms of impact upon practitioners, policy, systems and resources

• To analyse the main strengths and weaknesses of the planning and delivery of the Skills for Life Quality Initiative to date to inform the future development of the programme

• To assess the extent to which the targets and goals of the Skills for Life Quality Initiative have been met.

• To consider the consequences for providers of the delivery of the Skills for Life Quality Initiative through the LSC regional infrastructure

• To assess what still needs to be done through Skills for Life Quality Initiative activity in order to support the implementation of the Skills for Life strategy to 2010

• To generate proposals for action for post March 2006. Whilst the main focus of this evaluation is the impact of the Skills for Life Quality Initiative in 2005-06, in order to view the achievements of the Skills for Life Quality Initiative in context, the report includes a review of all Skills for Life Quality Initiatives since 2000. In addition, to compare achievement against targets for 2005-06 and the previous year, section 2 of the report provides a review of achievements for 2004-05 and 2005-06. In addition to the specific purposes noted above, the evaluation also sought to assess achievement of the two overall goals of the Skills for Life Quality Initiative:

• Professionalise the Skills for Life workforce • Develop a whole organisation approach to Skills for Life

The LSC became responsible for funding and co-ordinating the implementation of the DfES Skills for Life strategy in 2003. As part of this strategy, the Skills for Life Quality Initiative was introduced in 2003-04 with a budget of £9 million. Between 2003 and 2004, the management and delivery of the Skills for Life Quality Initiative was divided between two consortia. The main activities in 2003-04 were training and support for leaders and management, development of a facilitation network and professional development for teachers, trainers and support staff. The contractor responsible for the leadership and management programme and the development of the facilitation network was CfBT, in partnership with the University of Wolverhampton. The contractor leading the programme of professional development was the Basic Skills Agency (BSA) and partners.

- 5 -

The contract to lead and develop the Skills for Life Quality Initiative in 2004-05 was awarded to a CfBT consortium. The two key aims for the initiative in this year were to develop further a professional, Skills for Life workforce and a ‘whole organisation’ approach to Skills for Life through an increased emphasis on regional development. In 2005-06 the consortium led by CfBT continued to manage and deliver the Skills for Life Quality Initiative. This consortium included NIACE, LLU+ at London South Bank University, CTAD, LSDA, University of Wolverhampton; and NRDC. A significant change for the 2005-06 phase of the programme was that regions, building on the successes of 2004-05, were to take a leading role on planning and delivery. The evaluation methodology comprised five stages; a review of relevant documentation concerning the development of Skills for Life Quality Initiatives since 2000, a web-site survey, interviews in five LSC regions with LSC regional leads, provider managers and practitioners, five focus groups and an expert seminar. The review of documentation stage comprised:

• An analysis of quantitative data on progress towards targets set under the Skills for Life Quality Initiative contract, drawing on CfBT data.

• A review of documents relating to the history and development of the various Skills for Life Quality Initiatives since 2000 and their evaluation.

The web-site survey involved a short on-line quantitative survey of participants conducted between February and mid-March. The survey was published on the NRDC web-site and the Skills for Life Quality Initiative web-site. Links to the survey were sent round via NRDC contact lists and networks in order to stimulate participants to complete it. Thirty individual key informant interviews were conducted. Respondents were drawn from five regions and from a range of organisations, concentrating on representatives from managers and teachers in provider organisations and the LSC Regional Leads. Interviews were conducted either face to face or by telephone. The regions chosen were the North- West, Yorkshire and Humberside, the East Midlands, the East of England and London. Six focus groups were held to elicit perceptions of Skills for Life Quality Initiative policy development, design and delivery from:

• 2 meetings of CfBT Skills for Life Quality Initiative groups (i) Senior Partners Expert Group (ii) Consortium Group

• CfBT Regional Skills for Life Quality Initiative Managers. • Practitioners including managers, teacher-trainers and teachers (2

groups) • A semi-structured focus group held during a residential training event

for participants on the NRDC Practitioner-led Research Initiative.

- 6 -

Use was also made of other NRDC data from the study of the Impact of the Skills for Life learning infrastructure on learners (the Learners Study) and from the Effective Practice studies. NRDC has recently completed five studies under this title, one each concerned with reading, writing, ESOL, numeracy and ICT. In order to address what still needed to be done to support the implementation of the Skills for Life strategy, an expert seminar was conducted. Seminar members consisted of 16 expert commentators drawn from the Skills for Life Quality Initiative management, LSC regions, CfBT Regional Quality Improvement Managers, provider managers, teachers and teacher-educators. This group met together for half a day on 22nd March to hear the emerging findings from the evaluation, to critically analyse what had been achieved, and to make proposals for what more still needed to be done. The Skills for Life Quality Initiative ran over financial years, consequently any references to years in this report, such as 2004-05 and 2005-06, refer to events and achievements that took place in the financial year rather than the academic year. Conclusion This evaluation has demonstrated that the Skills for Life Quality Initiative has been an influential intervention into the quality of Skills for Life provision. Large scale reform of the provision of Skills for Life involving changing the culture of individual provider organisations and the practice of individual teachers is a long term process but significant progress has been made and evidence of organisational change is beginning to emerge. The consensus of all respondents was that a comprehensive range of high quality materials and a framework of support had been developed. Whilst respondents acknowledged the challenges facing facilitators working with a wide range of organisations within the different settings and different parts of the sector, overall this work was highly valued and strong arguments were made in favour of its continuation in the future. The CfBT-led consortium has made some impressive achievements in delivering on a wide range of targets, but has been hampered by an extended contracting period resulting in part from a complex strategic planning process. This involved marrying the needs of the regions with resources available nationally. This extended planning period led to some difficulties in delivery at grassroots level. The yearly stop/start nature of the initiative has militated against the development of a clear strategic approach to quality improvement, but the decision to shift responsibility for planning and delivery of the initiative to the LSC regional infrastructure has been widely welcomed and very successful. Initial ‘teething’ problems have been largely overcome and an effective regional infrastructure has developed, supported by good professional links

- 7 -

between the various key stakeholders and the development of trusting working relationships. Many practitioners in the Skills for Life workforce were unprepared for the growth in Skills for Life provision and some were not in a position to benefit from the emphasis on professional development central to the Skills for Life Quality Initiative. Many had no initial teacher training and there were a high proportion of part-time sessional members in the workforce. Despite this, professional development was seen as beginning to have a positive impact on practitioner confidence and skills. Improvements in the professional practice of the Skills for Life workforce will have benefits for the wider learning and skills sector. The goal of professionalising the workforce is closely linked with the Government’s agenda on the reform of teacher education and improvements in teaching Skills for Life will also support other Government priorities, such as the achievement of the level 2 qualification target. The goals of the Skills for Life Quality Initiative are far-reaching, involving the promotion of cultural change in provider organisations and stimulating changes in individual professional practice. These goals reflect the ambitious and challenging nature of the Skills for Life strategy and targets. Individual and organisational change are complex processes to influence and it takes time for change to take effect. However, positive organisational and individual change is perceived to be occurring, although across the sector the impact is variable. Further education and work based learning providers report a positive impact. Adult and community learning providers are less sure of the benefits of the initiative. In a national context of increasing personalisation and a developing ‘local solutions for local people’ philosophy, demand for further customisation of the Skills for Life Quality Initiative is likely to continue. Balanced against this is the reality that it is also necessary to establish national priorities that enable government and its agencies to shape the reach, quality and form of its services. In this scenario, the future for the delivery of the Skills for Life Quality Initiative may be in the establishment of strong central guidelines about key priorities, but giving providers more freedom to develop and deliver Skills for Life in their own way, subject to certain key principles. Key findings These findings are presented in the order in which they emerged from the data and are not intended to represent any order of importance.

• Comprehensive, high quality materials and support framework are in place

• Variable perception of the quality of facilitation; some facilitators are seen as very good while some are challenged by the specialist nature of providers and their specific requirements

- 8 -

• Impressive consortium achievements in reaching or exceeding targets in 2004-05

• The consortium has faced challenges in achieving the target for training in 2005-06, achieving 61% of its planned target of delegate numbers.

• The extended strategic planning period restricted the time available to deliver training

• The shortened time period for training delivery impacted on delivery of planned numbers.

• Impact of stop/start character of the initiative has militated against strategic development

• Shift to regional planning and delivery caused some initial problems, but was widely welcomed and seen as very positive

• Need for an improved planning process involving closer collaboration between national and regional planners, informed by more comprehensive feedback on activity

• Parts of the Skills for Life workforce remain in need of initial teacher training in order to maximise the benefit from the CPD offer in the Skills for Life Quality Initiative. This fragmented part-time workforce has been under pressure from growing numbers of learners

• The Skills for Life Quality Initiative operates within a crowded market of support initiatives; there is a need for rationalisation and a strategic overview of provision

• High aspirations have been established for the Initiative , involving cultural change and personal empowerment, which takes time to achieve

• The whole organisation approach is perceived to be influential in promoting change

• There is a perception that significant organisational change is taking place as a result of the initiative; changes have been observed in strategic planning, workforce development, senior management engagement and in embedding Skills for Life

• Professional development activity is having an impact upon practitioners; improvements have been reported in teacher confidence, skill development, joint working and communication

• Some parts of the sector are more positive than others about improvements: adult and community learning providers are less sure of the benefits of the Skills for Life Quality Initiative

• Respondents would like to see more evaluation focusing on the impact of the initiative on learning and achievement; without such information it is impossible to know whether the Initiative is benefiting those it is ultimately meant to benefit, namely learners.

Recommendations for action 2006-07 A number of recommendations for the next phase of the Skills for Life Quality Initiative are proposed which include:

• the initiative to continue to develop and strengthen. The majority of respondents wanted identified quality improvement support for Skills for

- 9 -

Life teachers and managers to continue. In particular, respondents identified the facilitation element as critical to further quality improvement

• evolution of a more strategic approach avoiding the stop/start nature of previous years

• development of a more joined up approach bringing together the various quality improvement initiatives under one umbrella

• continue customisation and allocate some resources direct to providers • review the menu of professional development provision for Skills for

Life staff, including ensuring that the right people get on the right courses, and the provision of appropriate professional development for vocational teachers. Close links should be forged between the Skills for Life Quality Initiative and the reform of teacher training currently being conducted by the DfES and LLUK.

• the Skills for Life Quality Initiative to work closely with LLUK and SVUK and to collaborate on the generation of better data on who is and who is not qualified, analysis of this data and feedback to the sector

• brief inspectors to look for clear evidence of appropriate Skills for Life developments

• re-prioritise ACL and ‘harder to reach’ providers • carry on the regionalisation and further customisation of the initiative.

This customisation could be supported by the differentiation of the sector workforce in terms of data collection about different elements of the workforce and their experience in delivering Skills for Life and of the Skills for Life Quality Initiative

• further develop embedding by focusing CPD on the different needs of vocational teachers and Skills for Life teachers

• facilitate peer coaching among Skills for Life trainers and between Skills for Life and vocational trainers

• promote more active sharing of good practice • further development of work on advice and guidance for Skills for Life

teachers to ensure that the Skills for Life Quality Initiative professional development activities supports teachers in gaining qualified teacher status

• promote the sustainability of the Initiative by considering how the wealth of resources produced through the Skills for Life Quality Initiative can be kept up to date and how they can be used effectively in initial teacher training

• establish a pool of quality assured consultants and approved trainers. This recommendation emerged from the regional quality improvement managers (RQIMs) who were aware that a list of Consortium approved trainers already existed. However, each region received some funding for development work and training delivery, managed within the regions that was outside the Consortium-led activities. RQIMs were aware that some trainers and consultants who had not been offered work by the Consortium had succeeded in obtaining work funded from the regional development funds.

• dovetail into the QIA framework with policy support from the DfES, and ownership by providers for sustainability

- 10 -

• promote more effective regional planning by ensuring systematic feedback of data collection to regions from central CfBT and ensuring a professional dialogue in regional planning taking account of research findings and intelligence from the ‘field’

• plan a follow up evaluation on an experimental model basis, and also an evaluation which will focus on the impact of the initiative at classroom level once the initiative has had more time to make an impact on teaching and learning

- 11 -

1. REVIEW OF DOCUMENTATION This section traces the origins of the various quality improvement initiatives launched since 1997. The development of the basic skills Skills for Life Quality Initiative and the Skills for Life Quality Initiative are described. These are set in the context of the quality of provision in the foundation programmes area, learner achievement in Skills for Life provision, and the range of other quality improvement initiatives offered across the learning and skills sector. 1.1 Origins of the quality improvement initiatives The origins of the Skills for Life Quality Initiative can be traced back to the Moser Report of 1999. The report emphasised the scale of basic skills problems, noting that up to seven million adults in England were unable to read or write to the standard expected of an eleven-year-old. Even greater numbers were suspected of having difficulties with numeracy. Moser attributed a substantial part of the cause of this problem to the inconsistent level, in terms of both quality and quantity, of basic skills training available throughout the country. Within the post school sector the report resonated with a number of pronouncements from the Further Education Funding Council (FEFC) on basic skills provision. In 1998, in its Curriculum Area Survey Report, FEFC (1998), stated: Many colleges find difficulty in applying college-wide quality assurance systems to courses in this programme area. Quality assurance of basic education lacks rigour. The report also noted that whilst teachers in the programme area were conscientious and committed to their work, they did not all have sufficient expertise and some lacked appropriate teaching qualifications. In order to address the quality of provision colleges needed to improve assessment of learner’s needs, arrangements for assuring quality, staff development and management support and teaching quality itself. The Chief Inspector’s Annual Report of 1998/99, FEFC (1999a), noted that in basic education, the profile of lesson grades was considerably worse than the average profile for all programme areas. Also in 1999, FEFC (1999b), the FEFC’s evaluation of entry and level 1 awards in literacy, numeracy and ESOL concluded that the number of awards in this area was unnecessarily large and that there was considerable overlap and duplication between them. In addition, some awards had no defined standards and the award of certificates for them had little meaning. There was clear resonance here with Moser’s advocacy of a new basic skills curriculum for adults with clear common standards at different levels. Already in 1997, the FEFC had launched its Inclusive Learning Quality Initiative (ILQI) in response to the Tomlinson Report on inclusive learning, FEFC (1997). The ILQI was divided into two stages. Stage 1 involved 94

- 12 -

colleges working in 9 partnerships to develop a set of staff and organisational development materials on a series of topics. For each partnership, one college, which had demonstrated good practice, led the development work of the partnership. Stage 2 of the ILQI, launched in 1998 was designed to make a significant contribution to quality improvement, including the improvement of standards of teaching and learning. The main outcomes proposed for stage 2 were: that all institutions will have improved their organisational capacity to match provision to the individual learning requirements of learners, in particular teaching and learning. FEFC (1998). The initiative also proposed the appointment and training of facilitators to work developmentally with colleges. Key elements of the ILQI, for example its use of facilitators and its emphasis on organisational development were also important to the current manifestation of the Skills for Life Quality Initiative. The Council announced plans for the evaluation of the ILQI in 1999. FEFC (1999c). The evaluation would concentrate on assessing the impact of the initiative in terms of the extent to which institutions have met the targets in their action plan, and how far the objectives and outcomes of the initiative had been achieved. The evaluation of stage 2 of the ILQI was published in 2000, FEDA (2000). The report concluded that the impact on staff had been greatest in terms of raising awareness of inclusive learning and that this had led to more effective approaches to teaching and learning. Tutoring and initial assessment procedures had improved and a clear impact had also been made on the range and nature of the curriculum. Significant changes were also reported to quality assurance systems. The report identifies a number of key factors in promoting inclusive learning, all of which are significant in terms of identifying crucial factors affecting institutional responses to major change initiatives:

• For all institutions ‘staff understanding and commitment to a culture of inclusive learning’ is identified as overwhelmingly the most important factor in promoting inclusive learning.

• This is closely followed in importance by ‘governor and senior manager commitment’.

• Respondents rank staff development as the next most important factor. • A key theme running through many responses is the importance of

linking inclusive learning activity into institutions systems and structures so it becomes an integral part of planning, quality assurance, and self assessment. That way it will become a central part of the institution’s core business.

If the phrase ‘quality improvement in basic skills’ is substituted for ‘inclusive learning’ in the key factors list above, it is unlikely anyone would disagree with these as a set of factors essential to the success of the Skills for Life strategy.

- 13 -

The report noted the most frequently mentioned barriers to change as being: lack of time and pressure on staff, the number of new initiatives to which institutions are expected to respond, and staff attitudes and resistance. 1.2 The Basic Skills Quality Initiative In November 1999, FEFC (1999d), the FEFC announced the introduction of a Basic Skills Quality Initiative (BSQI). This was part of strand 3 of the Standards Fund, which was introduced to underpin the government’s drive to raise standards in the FE sector. The fund consisted of four strands. Strand 1 involved targeted intervention in colleges causing concern, Strand 2 consisted of post-inspection support for other colleges as appropriate, Strand 3 focused on the provision of training and development for college principals, governors, lecturers and the BSQI, and Strand 4 involved the dissemination of good practice. The BSQI was designed to improve the quality of provision of basic skills and to lead to improvements in the standard of basic skills teaching, better learning experiences and improvements in people’s capacity to work and progress in education and employment. It involved the production of staff development materials and the use of trained facilitators. The BSQI was central to the Council’s response to the Moser report and was similar in conception and purpose to the ILQI. The improvement model, based on facilitators, pioneered in the ILQI was clearly highly influential in the design of the BSQI. In 2001 the LSC took over the funding of the BSQI at a time of increased interest in basic skills following the publication of the Skills for Life national strategy, DfEE (2001). The national strategy pledged £1.5 billion expenditure on basic skills provision over the subsequent three years. The scope of the BSQI was extended to include provision at level 2 and in ESOL. In addition, the BSQI was widened to include work based learning providers. Skills for Life proposed a wide-ranging strategy. Adult basic skills education was organised into a new learning infrastructure, which consisted of new national standards, teaching resources and a curriculum. New national targets for literacy, numeracy and language were also set. Whilst an evaluation of the further education standards fund was conducted between 2001 and 2003, there is no published report available which could provide an insight into the success of the BSQI between these years. However, the LSC noted that in 2000-2001 over a 150 facilitators were trained, £1.8 million was offered to providers, 3000 copies of BSQI materials were distributed, facilitators were allocated to work with providers and a training and development programme was started. LSC (2001). In 2001-2002 more facilitators were trained, further funding was allocated to providers and materials were revised to take account of work based learning providers. The impact of this expenditure and activity however is not clear.

- 14 -

In a report from Ofsted (2003), the Inspectorate concluded The Skills for Life initiative has been highly successful in increasing the number of literacy, numeracy and ESOL learners and in raising the profile of this area of learning, but there needs to be a sharper focus on the quality of the education and training that is available. This suggests that comprehensive benefits resulting from the expenditure and activity of the BSQI had yet to be felt. Ofsted noted a widespread increase in the number of learners, but a resultant serious shortage of qualified and experienced teachers. Its assessment that teachers and tutors lacked expertise to teach literacy and numeracy well, did not identify learners’ precise learning needs, that most provision was weak in terms of initial assessment and that there was a shortage of good learning materials echoed the proposals for training and support that constituted the Skills for Life Quality Initiative. When the government published its update on the Skills for Life strategy in 2003, DfES (2003), which focused on delivery to 2007, it stated: Over 15,000 teachers have been trained in the literacy and numeracy core curriculum and a further 5,000 ESOL teachers have been introduced to the ESOL curriculum. The LSC has taken responsibility for the funding and delivery of teacher training and continuing professional development. We estimate that 25,000 teachers will have made use of the programme by 2007 Other achievements listed included

• Publication of core curricula for literacy, numeracy and ESOL learning • Pre-entry curriculum framework • Literacy and numeracy materials for screening, initial assessment and

diagnostic assessment • National literacy and numeracy tests at levels 1 and 2

1.3 Skills for Life Quality Initiative The LSC became responsible for funding and co-ordinating the implementation of the DfES Skills for Life strategy in 2003. As part of this strategy, the Skills for Life Quality Initiative was introduced in 2003-04 with a budget of £9 million. Between 2003 and 2004, the management and delivery of the Skills for Life Quality Initiative was divided between two consortia. The main activities in 2003-04 were training and support for leaders and management, development of a facilitation network and professional development for teachers, trainers and support staff. The contractor responsible for the leadership and management programme and the development of the facilitation network was CfBT, in partnership with the

- 15 -

University of Wolverhampton. The contractor leading the programme of professional development was the Basic Skills Agency (BSA) and partners. The consultancy firm GHK evaluated this programme in 2004. The study evaluated the two main programmes and reported four main findings.

• The evidence showed that overall the Skills for Life Quality Initiative is adding valued activities and supporting improvements in existing literacy, numeracy and language provision.

• The surveys of leaders, managers, teachers and local LSCs showed that the Skills for Life Quality Initiative was contributing to improving the skills of the workforce

• The evaluation highlighted the strong potential to improve retention and achievement rates of learners on Skills for Life courses

• The Skills for Life Quality Initiative had the potential to impact on future policy and practice by strengthening links between strategic mechanisms and partnerships

The report concluded that the two programme elements had made significant progress against their operational targets and that overall opinion and experiences of the content were high and the activities of the initiative were rated very favourably. The main criticisms were levelled at a lack of clarity about practical implementation and delivery, lack of consultation and clear information and rushed timing. Facilitators varied in their skill levels and abilities and greater consultation with local LSCs would have been appreciated. The report made an important point in that it noted that a key lesson from the start-up phase was that contracting, negotiating and setting up a fully operational programme needed to be quick and efficient and was essential to allow partners to concentrate on roll-out activities. The positive tone of the GHK evaluation is repeated by the NAO (2004). They noted that: The Department has led the development of lasting improvements in the quality of literacy, numeracy and language learning. All the elements that support good learning-such as skill standards, curricula, good quality teaching and recognisable qualifications-were either non-existent or underdeveloped in 2001. The Department has introduced standards, curricula and new teaching materials to raise the quality of learning. It has brought in national tests and associated qualifications so that people know when they have achieved the standards. However, the NAO also recommended that more teachers with up-to-date training in teaching literacy, numeracy and language were needed to provide high quality learning and that some low performance was likely to be due to providers not energetically taking up the fruits of the new learning infrastructure.

- 16 -

The ALI (2005) also noted a positive impact of the Skills for Life Quality Initiative in their report on Skills for Life at work: Some providers have clear strategies for improving literacy, numeracy and language training at work. They have made good use of expert consultant support offered through the Skills for Life Quality Initiative. The contract to lead and develop the Skills for Life Quality Initiative in 2004-05 was awarded to a CfBT consortium. The two key aims for the initiative in this year were to develop further a professional, Skills for Life workforce and a ‘whole organisation’ approach to Skills for Life through an increased emphasis on regional development. This latter aim can be seen as a response to the concerns about lack of consultation with local LSCs noted in the evaluation of the previous year’s programme. The contract was delivered through three key strands: the development of a regional infrastructure, a professional development menu and a network of facilitators. The deliverables for the consortium were largely target driven and the consortium’s specific targets included:

• Train 2,500 leaders and managers • Deliver 10,000 days of facilitator time • Engage 3,000 teachers in undertaking courses or modules • Train 300 teacher trainers • Launch a Skills for Life Quality Initiative web-site • Develop additional level 4 modular materials

An evaluation of the Skills for Life Quality Initiative was carried out by the National Research and Development Centre for Adult Literacy and Numeracy (NRDC) based at the Institute of Education and the Centre for Education and Industry (CEI) at the University of Warwick. The aims of this evaluation were to examine and make recommendations to the LSC on the effectiveness of the impact of the Skills for Life Quality Initiative and its delivery model. In addition, the evaluation aimed to identify any changes to better support and develop the future success of the Skills for Life Quality Initiative. The evaluation demonstrated that the initiative had made significant progress over the year. Evidence highlighted progress in relation to raising the profile of the Skills for Life agenda, developing greater awareness about the regional ownership agenda, improving the quality of facilitators, developing strategy to encourage providers to implement a ‘whole organisation‘ approach, and expanding the modular professional development framework. It was also noted that the development and delivery of the second year of the initiative was inhibited by some contractual issues, including the length of the period to finalise contracts between the LSC and the CfBT consortium. This finding appears to be consistent with that of the previous evaluation and suggests that a key recommendation of GHK’s report had not been possible. It was also reported that enormous variation had occurred between the 9 LSC regions with regard to the effectiveness of Skills for Life Quality Initiative provision across the three elements of the delivery framework. Whilst positive

- 17 -

feedback was received about the effectiveness of facilitators, it was also evident that the overall quality of service provided by facilitators was patchy. The evaluation noted that embedding Skills for Life through a ‘whole organisation’ approach, remained the most challenging and underdeveloped ambition of the initiative. It also drew attention to the continuing debate around the future of the Skills for Life Quality Initiative. The emerging evidence from the evaluation would suggest that a great deal of debate continues to centre on the future of the Skills for Life Quality Initiative. These include discussions about the lifespan of the Skills for Life Quality Initiative, future funding and management, the inter-relationships between the Skills for Life Quality Initiative other policy initiatives and whether the current delivery mechanism was suitable to meet future needs. There was also a body of opinion which suggested that it was time to ‘take stock’ and to map the future of the initiative. In 2005-06 the consortium led by CfBT continued to manage and deliver the Skills for Life Quality Initiative. A significant change for the 2005-06 phase of the programme was that regions, building on the successes of 2004-05, were to take a leading role on planning and delivery. Each of the nine regions, led by the designated Skills for Life regional lead, had to draw up a regional delivery plan that formed the basis of activity for this phase. The areas of activity were:

• Support for regions to further develop the regional infrastructure for Skills for Life activities and help ‘join-up’ the Skills for Life agenda

• A programme of free professional training and resources for senior managers and managers, teacher educators, specialist literacy, numeracy and ESOL teachers, and vocational teachers. Training and resources were to focus on whole organisation approaches, embedded learning, leadership and management, e-learning, key skills and coaching and mentoring.

• A regional network of trained Skills for Life Quality Initiative facilitators to work with organisations selected by the regional and local LSCs to support providers in moving towards a whole organisation approach to Skills for Life.

Within these three areas, work focused on priorities identified by the LSC:

• Development of a whole organisation approach to Skills for Life, and supporting the planning and implementation of this within organisations

• Regional facilitation and continuing development of the Skills for Life Quality Initiative facilitator network

• Training in leadership and management skills specifically related to Skills for Life

• Supporting the embedding of Skills for Life for learners in all contexts • Raising the awareness of the relationship between key skills and Skills

for Life

- 18 -

• Support for developments relating to a continuous professional development (CPD) framework and the development of regional specialist Skills for Life information advice and guidance.

• Dissemination of good practice including the maintenance of websites. This major emphasis on developing the regional infrastructure involved an extensive planning process at regional level and detailed consultations between LSC national office and CfBT. The challenges this presented to all the key players and the time involved in completing complex planning arrangements, is important in understanding the issues raised later in the findings section of this report. In addition, this also helps explain the issues arising as a result of the protracted contracting process which emerged in the findings section of this report. Whilst this review has considered the development of the various Quality Initiatives since the launch of the Inclusive Learning Quality Initiative in 1997, the focus of this report is the Skills for Life Quality Initiative 2005-06. 1.4 The wider context for quality improvement in the learning and skills sector

It could be argued that the Skills for Life Quality Initiative operates within a crowded market for the development of quality in the learning and skills sector. A number of parallel initiatives have been operating since 1999 with similar approaches to the Skills for Life Quality Initiative. The Learning and Skills Development Agency (LSDA) has played a major role in the development and delivery of these initiatives. Perhaps the improvement programme most closely related to the Skills for Life Quality Initiative is the Key Skills Support Programme (KSSP).

Currently, the KSSP defines its role as helping with teaching and learning, organisation and support for key skills and increasingly Skills for Life. Since 1999, the programme has been offering a broad menu of support that includes materials for teaching and learning, free training, regional networks, consultancy and publications. More recently the programme has offered mentors to work with organisations, a complete continuing professional development course and on-line resources.

Also in 1999, the LSDA introduced the Raising Quality and Achievement programme (RQA). This provided a wide range of quality improvement activities including information and advice service, quality improvement networks, on-site support for providers, a benchmarking service, and development projects on strategies for improving retention and achievement and opportunities for sharing good practice.

In 2001 the programme published a report summarising the lessons learnt over the preceding two years, Stanton (2001). This emphasized a number of generic principles for quality improvement initiatives including:

- 19 -

• Taking a holistic rather than piecemeal approach to improvement in an organisation

• The importance of taking a strategic approach following an analysis of the organisation context and the tactics available to support improvement

• The need to develop a culture that favours quality improvement • The importance of learning from others and becoming adept at

transferring effective practice within and across organisations • The value of action planning, target setting and monitoring of the

effects of initiatives.

The RQA programme was succeeded in 2003 by Support for Success (s4s). This programme aimed to help providers embark on a programme of self-improvement and spread good practice. LSC funded providers were supported to facilitate quality improvement networks and projects. In addition, regional training events, consultancy days and customised support packages for organisations were offered.

Also 2003 saw the establishment of the Centre for Excellence in Leadership, with a remit to foster and support leadership improvement across the sector. The Centre pursued its aims through a programme of training and development activities, consultancy support, research, publications and the provision of virtual learning resources.

This brief summary of parallel initiatives operational in the sector alongside the Skills for Life Quality Initiative shows the operation of a number of comprehensive, centrally funded interventions targeted at practitioners, managers and leaders. These initiatives all used broadly similar strategies to target and influence individuals and organisations in the sector. In many cases, the target audience of leaders and managers was the same for each programme, resulting in a plethora of training, consultancy and resources on offer to both practitioners and their managers. The impact of this proliferation of support is likely to contribute to the often-heard assertion of practitioners and managers that they suffer from initiative overload and a deluge of information crossing their desks. In this context, the sheer volume of information and support on offer may ameliorate the impact of individual initiatives.

- 20 -

2 CONSORTIUM ACHIEVEMENT AGAINST TARGETS This section outlines the targets set under the Skills for Life Quality Initiative contract and the CfBT-led consortium’s achievement of these targets in 2004-05 and the achievement in 2005-06. These two years relate to the contract awarded to the CfBT consortium. Data was not available to the evaluation team for 2003-04. In 2004-05 the consortium was given a contract valued at £12.2 million to deliver eleven targets under the Skills for Life Quality Initiative programme. A variation of the contract was made during the latter part of the year which revised two of the targets and added an additional £5 million for the consortium to deliver three extra targets. The variation of contract meant that in the year 2004-05, the consortium was required to:

• train 1,000 Skills for Life practitioners in Key Skills; • train 300 teacher trainers; • develop a database of teacher trainers; • produce 25 professional development modules at level 4; • deliver short non-accredited training courses to 3,000 practitioners; • research into the development of a Level 7 qualification; • deliver monthly newsletters from September 2004; • ensure that 1,000 practitioners received online or paper based

guidance about Accreditation of Prior Learning(APL); • develop a website which would achieve 10,000 hits per month; • deliver 9,000 provider consultations; • train 2,000 leaders and managers; • deliver accredited training at Levels 2,3 and 4 for an estimated 1,700

practitioners; and • deliver non- accredited training to an additional 5,600 practitioners.

The following section outlines the achievement of the consortium in the delivery of the above targets. 2.1 Overall achievement of the consortium's targets A detailed review of the LSC’s Skills for Life Quality Initiative 2004-05 report shows that twelve out of the thirteen targets were either achieved or exceeded. The most successful outcome of the consortium's 2004-05 programme in terms of volume was the national website developed for this initiative. This achieved 30,000 hits per month compared to a target of 10,000, representing a two-fold increase over and above the required target. The target of delivering training to 2,000 leaders and managers was however not achieved. Reasons cited in the LSC’s Skills for Life Quality Initiative report state that the target had not been reached for a combination of reasons. This includes the extended regional consultation impacting on the training start date and obtaining provider details. This meant that there were delays in the

- 21 -

identification of potential providers and their delegates for the Leadership and Management training. Additional outcomes achieved by the consortium include:

• a new leadership and management module developed specifically for senior managers and delivered in collaboration with the Centre for Excellence in Leadership (CEL);

• 2,550 providers provided with new Skills for Life strategic action plans; • the revision of 16 Professional Development Modules produced last

year; • the development of a new introductory course and a more advanced

module to provide a more appropriate training route for vocational teachers;

• the development of a guidance document to support organisations in building capacity for mentoring and coaching within their workforce;

• the development of a toolkit providing a flexible resource for integrating Access for All in the workplace;

• the development of nine regional development plans to support the implementation of regional training delivery infrastructures;

• and the provision of assistance to regions in the development of nine regional websites.

2.2 Accredited training As part of the variation of contract, the consortium was required to provide additional training to 1,700 practitioners at levels 2, 3 and 4. This target was exceeded with a final figure of around 3,200 practitioners receiving accredited training. Information provided by the consortium shows this training was at the following levels: Level 4 29% Level 3 30% Level 2 41% of which: Full certificate 19% Unit 1 22% 2.3 Achievement of Skills for Life Quality Initiative priorities for 2005-06 This section charts the achievements of the consortium against the ten national priorities of the Skills for Life Quality Initiative for 2005-06:

• Skills for Life Whole Organisation Approach • Embedding Skills for Life • Leadership and Management for Skills for Life • Facilitator Network • E-learning • Key Skills

- 22 -

• Continuing Professional Development / Information, Advice and Guidance

• Dissemination of good practice • Data collection and analysis • Development of large impact measures

Each of these priorities contributed towards the overall goals of professionalising the workforce and developing a whole organisation approach to Skills for Life. This section focuses on activities such as training and facilitation events planned by the consortium, to meet regional plans, under each of the above. The information used in this section have been sourced from three main documents; the Learning and Skills Council's Skills for Life Quality Initiative e-bulletin for 2005-06, the Skills for Life Quality Initiative 2005-06 National Preliminary Final Report, 29th March 2006, and the Skills for Life Quality Initiative 2005-06 Final Report. In 2005-06 the contract was valued at £10.7 million. 2.4 Activities undertaken by the Consortium in 2005-06 Over 480 events classified as professional development training activities relating to seven of the above ten priorities;( excludes the Facilitator Network ) were made available by the consortium in 2005-06.These training events covered a range of professional courses such as train the trainer events, e-learning and leadership and management courses. For the facilitator network, the consortium was required to deliver just over 4,000 consultations to providers. The following sections outline the achievements under each of the eight national priorities. 2.4.1 Skills for Life Whole Organisational Approach The whole organisation approach encompassed the complete initiative. Activities under this strand focused on training providers and partners on how to adopt an organisational approach to Skills for Life. Various activities contributed to the approach, but were also priorities in their own right. These included leadership and management courses, a facilitator network, staff development, e-learning and advice and guidance systems. There were 9 events planned, which focused specifically on the whole organisation approach. The Consortium also developed a toolkit/guidance document which provided practical advice and suggestions for organisations in a wide range of settings for mainstreaming Skills for Life. 2.4.2 Embedding Skills for Life A range of training events (156) and support materials was developed for both teachers and managers to support this strand. These included train the trainer events and Skills for Life courses for vocational teachers. 1,718 delegates have been trained across the range of activities made available.

- 23 -

2.4.3 Leadership and Management Skills The leadership and management programme offered a variety of programmes including two day face-to-face programmes which allowed participants to examine their leadership qualities. It also offered half day leadership and management sessions and one day seminars for senior managers. Accreditation of leadership and management training was also available. Approximately 86 courses were planned across the nine regions under this strand. A total of 778 delegates participated in these events. 2.4.4 Facilitator Network 1479 organisations across the nine regions were nominated by the LSC regions for provider consultations. The aim was to deliver and achieve a target of just over 4,000 regional consultations in 2005-06. The target for facilitator consultations was exceeded and 1,432 providers received 5, 109 consultations. 2.4.5 E-Learning The consortium provided a range of professional development programmes across the nine regions aimed at supporting e-learning development. These included courses such as:

• one day introductory courses for practitioners to develop their competence and confidence to use technology in their day to day teaching;

• in house - modular training programmes for staff leading to accreditation; and

• one day courses that prepare teacher educators to integrate e- learning into professional programmes.

453 delegates received training through the introductory events and e-learning for teacher educators. In addition, 35 organisations completed in-house consultation and training. 2.4.6 Key Skills Awareness training sessions were delivered across the regions for Key Skills practitioners under this strand. 38 events were planned in these regions and included events that offered awareness training sessions to Skills for Life Quality Initiative facilitators to enable them to support organisations in developing strategic approaches. Take up was lower than anticipated in the regional planning process with 236 of the 945 places originally requested in regional plans actually taken up.

- 24 -

2.4.7 Continuing Professional Development/Information, Advice and Guidance 36 events were planned in this area across the regions for Skills for Life and vocational teachers who wanted to move into teacher training, observation and mentoring. Practitioners and managers from a broad spectrum were also attracted to these events. Courses included:

• two twelve-day courses for teacher educators, accredited at Masters level

• four-day residential courses; • coaching and mentoring courses, both accredited and non- accredited; • one day courses for observers and assessors; and • the provision of regional expert services to advise teachers on

professional development options. 2.4.8 Dissemination of good practice A series of events to disseminate good practice were planned in the regions. 14 dissemination events were held under this strand and featured presenters from organisations recognised as excellent in Skills for Life areas. 2.5 The Consortium’s progress on training activities A total, of 5,153 training places were delivered over the period of the contract, approximately 61% of the planned target of 8,458 delegates. The strands with the highest proportion of training completed were those relating to, Embedding Skills for Life, CPD/IAG and Dissemination of Good Practice. Those areas with the lowest number of completed delegates were Whole Organisation Approach (WOA) training for LSC Staff and Key Skills. WOA training suffered as a result of organisational changes occurring at LSC and some training events aimed at LSC organisational staff were cancelled at the request of some regions. 2.6 The Consortium's progress under the 'Facilitator Network Strand' The consortium was successful in exceeding its target of delivering 4,123 provider consultations across the nine regions. 1,432 providers received 5,109 facilitator consultations. The degree of success however varied across the nine regions, for example, in one region the number of provider consultations was less than 50% of the planned target. 2.7 Conclusions This section summarises the achievement against targets of the consortium in 2004-05 and its achievement against targets set for 2005-06.

- 25 -

The data suggests that the CfBT led consortium Skills for Life Quality Initiative programme for 2004-05 was successful. Almost all the targets that the consortium was required to deliver in that year were achieved or exceeded. The most successful outcome of the 2004-05 programme in terms of volume was the national website developed for this initiative which achieved 30,000 hits per month against a target of 10,000. However one area in which the consortium did not achieve its target was in relation to the provision of training to Skills for Life leaders and managers. Under this target, the consortium delivered training to just over 1,700 practitioners against a target of 2,000. The target was not reached for a combination of reasons. Additional outcomes achieved by the consortium in that year which did not count as part of the target but nevertheless had an impact on the Skills for Life Quality Initiative programme included the:

• development of a new management module; • provision of new Skills for Life strategic action plans; and • development of nine regional plans to support the implementation of

nine regional training delivery infrastructures Overall, the consortium's 2004-05 Skills for Life Quality Initiative programme was successful in ensuring that all the required deliverables were achieved. The consortium's progress towards priorities set out for it in 2005-06 was more variable. Under training activities covering seven out of the eight national priorities, the consortium between October 2005 and March 2006 delivered training to 5,153 delegates (approximately 61%) out of a planned target of 8,458. The facilitator network activities organised by the consortium however were very successful, exceeding its target of 4123 provider consultations by achieving 5,109 consultations. This varied success was due to a number of factors. First, the planning process involved in moving from a national plan to that meeting the needs of each region. The length of time required to finalise and agree regional plans led to a reduced period of time for training and impacted on the effectiveness of promoting events and the time available for providers to arrange release of staff to attend training events. Second, the reduced period of time for training also impacted on the time available to manage event organisation. This led to administrative problems which in turn affected the ability and motivation of participants to attend training. Despite the difficulties involved in ensuring the full take up of training places, there is good evidence available from the evaluations conducted and reported in the Final Report to the LSC for 2005-06, that the training that took place was high quality, relevant to the needs of the participants and that a significant proportion of attendees were from the more hard to reach provider organisations.

- 26 -

Of the 3,469 evaluation forms received, 98.7% of delegates reported that their overall opinion of the training they received was satisfactory or above.86.5% of these delegates rated the training as good or excellent. Similarly, 98.6% of delegates rated the relevance of the training they received as satisfactory or above, with 85.3% of these ratings at the level of good or excellent. Data was collected on whether delegates felt that they had been well advised in being recommended to attend training courses. 67.9% of the delegates who completed evaluation forms answered yes to this question suggesting that the training was appropriately targeted and that the right people had attended the training. Only 4% of delegates felt that they had not been well advised. In terms of the breakdown of delegates attending training courses from the different parts of the sector, data collected from the 3,469 evaluation forms received showed that 22.5% of delegates worked in adult and community learning providers, 29.2% in FE colleges and 13.6% in work based learning providers. These represented the three largest groups of providers and suggest a good degree of success in providing training for more hard to reach providers in adult and community learning organisations. A more detailed assessment of the Consortium’s achievements, and including its impact at regional and sectoral level, is included as Annex 4 to this report.

- 27 -

3. PERCEPTIONS OF THE SKILLS FOR LIFE QUALITY INITIATIVE

The data presented here are structured around the key themes of the evaluation:

• the impact of Skills for Life Quality Initiative upon resources • the impact of policy • the impact upon systems • the impact upon people

For each theme data are drawn from the various methods of data collection. Key points that emerged across the different sources of information were selected and illustrated with data in the form of quotations from interview transcripts, email responses and focus groups transcripts as well as quantitative data from the survey. In mapping data across these various sources of information it is possible to see a consistency of view emerge, that strengthens the conclusions drawn from the data. Data collected and analysed in this way can rarely claim to establish a direct relationship between cause and effect, in this case between the Skills for Life Quality Initiative and the change in behaviour of policy-makers, providers and practitioners. Many other influences will intervene, making it extremely difficult to definitively link the existence of the Initiative with changes in stakeholder attitudes and behaviours. However, the approach to data collection described in the section on methodology and the data presented here, are powerful in surfacing issues and perceptions about the work and success of the Skills for Life Quality Initiative. An important way of attempting to ensure the validity of the data collected in this evaluation was to use a strategy of ‘checking out’. This involved checking data collected from one source or method against those collected from others. This is a procedure usually referred to as triangulation, whereby data from interviews for example, are triangulated against the data collected from focus groups, and the survey. The quotations from interviewees included in this section are taken verbatim from interview transcripts. The exception was where asides had been made which interrupted the flow and sense of the comment. These asides have been omitted and this is indicated by a short series of dots. Quotations for inclusion have been chosen on the basis of their typicality or to illustrate a particularly important point which differs from the general view. Quotations from the transcripts of the focus groups have been used in the same way and are subject to the same treatment. The sources of the quotations are noted after each extract. With regard to the level of knowledge about the Skills for Life Quality Initiative possessed by interviewees and focus group members, these type of respondents had significant knowledge, having been chosen to participate in the evaluation because of their experience of the initiative as consortium members, regional leads, provider managers or practitioners. Similarly this

- 28 -

was the case with those who participated in the expert seminar. Those who submitted email responses were also very familiar with the initiative through their membership of Skills for Life research networks, and as provider managers and practitioners across the sector. A number of individuals who fell into other categories, such as LSC regional leads, CfBT regional quality improvement managers, and regional training organisers submitted email responses because they were unable to attend scheduled focus group sessions. When survey participants were asked about their knowledge of the Skills for Life Quality Initiative, all except one survey respondent had heard about the initiative. Two survey respondents had heard about the initiative, but didn’t know much about it. A significant proportion (78%) either knew a fair amount or a great deal. The majority of the survey respondents were linked to the Skills for Life Quality Initiative in some way. Just under one third (32%) had been involved in a Skills for Life Quality Initiative professional development activity. A similar figure (31%) had used the services of a Skills for Life Quality Initiative facilitator and 10% had used the online resources developed under the initiative. Five of the respondents were connected to the Skills for Life Quality Initiative via their membership of a regional network. 3.1 IMPACT UPON RESOURCES 3.1.1 Quantity and quality of training and materials One of the most visible achievements of the Skills for Life Quality Initiative had been the quantity and quality of resources produced, these included staff and organisational development materials, training programmes, a network of trained facilitators, the web-site and on-line resources. These resources had built upon those developed as part of preceding Quality Initiatives and ensured that there was a comprehensive resource available for providers and their staff to access. Perceptions of the quality of these resources were very positive as the following comments illustrated: I have been impressed with the quality of trainers and training offered Email response I like the web-site-there’s lots of really useful training material to use or adapt. What I have seen has been accurate, well structured and usable. Email response The embedded material has been very very useful Focus group A majority of the interviewees highly valued the resources and the training (described as ‘high quality’) provided under the Skills for Life Quality Initiative

- 29 -

programme. Some felt that the training sessions had helped tutors embed different aspects of Skills for Life into their teaching: The staff were quite impressed with that course because they were not quite sure how to embed the literacy and numeracy into the vocational area that they were delivering to the learner Others were impressed by how the courses had affected themselves and colleagues: For me it’s been an incredible learning curve, and for the other people that’s taking part in the courses, it’s just wonderful really These positive impressions of training are born out by the participant evaluation ratings given to the training. Delegates were asked to complete an evaluation form at the end of each event where they judged the criteria relating to elements such as training methods, materials used, relevance, and how well their expectations had been met as excellent, good, satisfactory or unsatisfactory. The number of delegates who were satisfied or more than satisfied with each aspect of the training exceeded 90% across all training programmes. The only exception to this related to the pre-course arrangements which averaged out at 87%. Possible reasons for this are noted later in section 4.1.3. This data is extracted from the Skills for Life Quality Initiative 2005-06 Final Report, April 2006. During the web survey, participants were asked about the usefulness of the various resources available under the initiative and the responses confirm the perceptions reported above. On average, 39% of the practitioners felt that all the resources had been ‘of some use’ and 20% that they had been ‘very useful’. However, some of the resources were found to be of more use than others. The two extremes in this case were the web-site and the good practice cluster groups. 52% said the web-site had been ‘of some use’ and 18% that it had been ‘very useful’. With regard to the good practice cluster groups 45% of respondents had no experience of these. Of those that did have experience, 25% felt that they had been ‘of some use’ and 13% that they had been ‘very useful’. The most useful resource was reported to be the Skills for Life Quality Initiative facilitators, 32% felt that this resource had been ‘very useful’ and 47% that it had been of ‘some use’. This was followed by the downloadable resources, which 28% of respondents reported as being ‘very useful’ and 44% as being of ‘some use’. Respondents reported the online training elements/modules as having the lowest ‘very useful’ rating at 11% with 38% reporting that they had been of ‘some use’. 3.1.2 Facilitation The majority of interviewees were impressed by the support received from their facilitators. Providers were said to have received targeted support at regular intervals on how to effectively manage the curriculum offered.

- 30 -

And I think that [name of facilitator] has very much moved with each organisation in a way that's been suitable to them The level of support received from facilitators within some regions was said to have been the key to the success of the programme. Without this support the programme would have been seen as a 'pick and mix' approach. I think if you haven't got the support that we've had of our regional Professional Development units then it would very much be,…a menu of…'pick-and-mix', 'what out of these would you like'. We've had our Professional Development Unit that have done an awful lot of work with providers, Skills for Life Quality Initiative has supported that and that has been the key to the success of it in our region. If training had been on its own, with this start and this end, then I think for providers it wouldn't have helped them with long term planning. It was felt that facilitators in particular were driving organisational change. I am not sure how effective the materials are in forging that change; I think it's been people, their expertise, and their relationships that have moved provider’s forwards The experience of working with facilitators was mixed. There was recognition that it was valuable, but there were also criticisms that included variation in the quality of facilitators. Criticisms focused on the following: too much emphasis on the facilitator’s definition of need rather than that of the providers. A pre-occupation with writing action plans, the difficulties individual facilitators had in grasping the nature of widely varying organisations and a feeling that the facilitator process was not flexible enough to take account of the needs of the individual organisations. From talking with colleagues and picking up what ….said about the quality of facilitators, I think that sometimes you can be lucky and sometimes you can be less lucky. The more you work with a facilitator, the more canny you get as to how far you can steer them to be bespoke and I think the trick is in learning how to manage your facilitator Focus group The pivotal role and varying quality of the facilitators was also discussed in the following focus group extract, which provided a view on how the initial drive to recruit facilitators may have resulted in a mixed quality cohort: And I think that has also been one of the weaknesses, the variation in the quality of the facilitators and that reflects totally in the starting point where we were. Where were you going to draw these people from in a field that had been neglected for many many years? Of course you have the opportunity to take some of the best, but because it’s national, because you have no time to build capacity, you were going out on a broad front which meant that you were

- 31 -

scooping up people who weren’t really experienced and well informed in order to ensure an even spread across the field. Focus group There was also a feeling that certainly in the first year of the contract, the Consortium was too target-driven and that this militated against a clear focus on quality in the provision of facilitation: The vast number of facilitator days, I don’t think that really helped, people were very focused on the targets rather than the quality of what was actually being delivered Focus group Overall, there was a view that the quality of facilitators themselves had improved, and that they were now dealing with a wider range of organisations each with specific, and to some extent unique, issues. Consequently facilitation remained a challenging activity requiring the process skills of consultation, detailed knowledge of Skills for Life and experience in getting to grips with provider organisational processes. 3.1.3 Achievement of targets relating to training and facilitation Given the extensive range of targets set it in the first year of the CfBT consortium’s operations (2004-05), the consortium made an impressive achievement in delivering training and consultations, development materials and the web-site (as noted in section 3). Improved performance has been achieved in relation to facilitator consultation targets, which were exceeded in 2005-06. However, progress towards the delivery of training in 2005-06 had been more problematic with 61%of training delivered. The reasons for this appeared to be twofold: First that the extended strategic planning period delayed the start of the training offer in the current year, and second that administrative difficulties had discouraged the take up of training opportunities: I think for me the biggest cause for concern is that if you look at this year we have gone all the way to January really before we started to roll out all this training and you know it is not what any of us in this room would have wanted in an ideal world to have started that late Focus group This important point about the need for speed in contracting was noted in the GHK evaluation of 2004: A key lesson from the start-up phase is that contracting, negotiating and setting up a fully operational programme needs to be quick and efficient and is essential to allow partners to concentrate on roll-out of activities. Last year’s evaluation also noted the need for efficacy in contracting (NRDC/CEI, 2005):

- 32 -

Delays to the start of funding Year 2 of the Skills for Life Quality Initiative had a negative impact on the image of the initiative at local and regional level. This issue was compounded by the extremely fast pace at which the roll-out of Year 2 of the initiative was expected to be delivered at grassroots level, once national contracts had been signed. It is important to note however, that the delays were also influenced by the decision to deliver the Skills for Life Quality Initiative through the LSC regional infrastructure, requiring agreed regional plans to inform the national contract. The result of the extended planning phase and consequent late contracting in 2005-06 was that a large number of training events needed to be delivered within a short space of time, making it difficult for providers to plan for and cover the release of staff. Despite the fact that some of the most important visible outputs of the initiative (e.g. the training) were not on offer until the spring term of 2006, a great deal of planning activity had been taking place in the regions, involving the LSCs, CfBT central administration and CfBT regional managers. However, concentration of delivery led to a perception of lack of administrative efficiency of the Consortium in arranging training. This perception was reinforced by hiccups in administering the delivery of training. The challenge of national co-ordination and local delivery caused a range of difficulties such as the resultant contracted period of time available to deliver training courses and the consequent time pressure on providers to release staff to attend training. It is likely that this contributed to a lower take up of training opportunities and to the lower rate of participant satisfaction with pre-course arrangements for training events reported in the previous section. This situation was similar to that of the preceding year, reported in last year’s evaluation, NRDC/CEI (2005), when it was asserted that insufficient notice was given to providers about the availability of training courses, difficulties existed in arranging cover for staff to attend at short notice and there was confusion about the range of courses and providers which existed within the Skills for Life Quality Initiative. Interviewees also expressed their frustration with this situation: Given something like four months to take up all of the training that’s there, it’s just impossible because obviously if you’ve got key people who you would probably put through three or four of the courses, there’s no way that you could suddenly release them in a block to take three or four different courses. So that’s been a real disappointment. Interviewees also looked to see a more strategic approach: I think if they are going to do that it would be nice to know they have maybe a twelve month timetable where they have planned courses out for each month, following one month from the next. I think that that’s the best thing they could do to improve everything so they do give plenty of notice. It would be nice to

- 33 -

have that twelve months planning, what courses are being offered and when and where they are running. When asked in the survey if the initiative had been well planned and delivered, 20% of respondents disagreed or strongly disagreed. 42% tended to agree and 9% strongly agreed. The remaining respondents either did not know or felt that they could neither agree nor disagree with this question. 3.2 IMPACT OF POLICY 3.2.1 Stop/start nature of the initiative One of the most important impacts of policy upon the Skills for Life Quality Initiative was the stop/start nature of the initiative: One of the key weaknesses and I am sure you will hear this over and over again has been the stop/start nature of this whole initiative. It should have been at least a 3 year project, I mean if you want to make an impact and improve quality you have got to take a longer view. That has been one of the most difficult things for all of us, not getting a contract sorted, you are six months in and so on and I think that was a major mistake not to see it as a longer term initiative. Focus group Interviewees noted this also: I think the initiative has been in danger of being a stop-start model instead of a flow-through…and the facilitation probably could have been smoother…because we may have taken a few steps back in terms of facilitation starting again-after a few months gap. I just think it’s been in danger of becoming this separate entity rather than a real quality improvement process. The award of contracts on a year by year basis militated against effective strategic planning and the delay in awarding contracts necessitated more expedient solutions to the provision of services at grassroots level. Despite having to adapt to these circumstances, the Consortium tried hard to adopt a more strategic approach and to emphasise the developmental and processual nature of the initiative. As the following comment notes, the first step in taking a more strategic approach was to establish the Consortium and provide a more coherent package of support: Up to the point at which the Skills for Life Quality Initiative came in, Skills for Life was very piecemeal and it was very operational and the Skills for Life Quality Initiative provided a framework that would allow it to become a national initiative, it was an opportunity to pull together the experts……..as a mechanism to drive up quality. Focus group

- 34 -

Emphasising the importance of sustainability as a key feature of the initiative was also necessary to counteract the effect of the stop/start nature of the funding: I think the idea that it is time-limited and time-bound is difficult; there should be more focus on sustainability at the start Focus group 3.2.2 Regional delivery A major policy decision was to shift the focus of the initiative from a national programme to one delivered on a regional basis through the regional LSC network. An important purpose was to enable the more effective tailoring of the initiative to the needs of providers in the regions and to develop greater ownership of the initiative among LSCs. This approach could be seen as part of a wider public sector imperative towards personalisation whereby services are provided more locally to meet the specific needs of users. Providing the initiative through a network of nine different LSC regions has led to variations in delivery. The way in which the initiative was delivered in the different regions was largely dependent upon a range of critical success factors such as the quality of regional leadership and vision with regard to the initiative, the state of the local infrastructure, and the degree of ownership by local LSCs across the region. Also significant were the effective use of virtual resources and the geographical lines of communication within large and diverse regions. Planning for the delivery of the Skills for Life Quality Initiative varied in the different regions. In essence the process involved the agreement of a national brochure/offer by LSC national office and a requirement for the regions to produce a regional plan based on the priorities in the national brochure. Some regions appeared not to be aware of the brochure and others had conducted training needs analyses in their regions and were already evolving their own plans before national guidelines were produced. In some regions the CfBT regional quality improvement managers (RQIMs) and regional training organisers (RTOs) were extensively involved in the production of the regional plans and in others this was not the case. The regional plans were revised following negotiations between the regions and CfBT before being finalised and agreed with LSC National Office. Once finalised, the LSC regional leads managed the implementation at regional level supported by the RQIMs and RTOs. Some regional interviewees were not clear about what was on offer under the initiative. They expressed the need to improve communication about the vision of the Skills for Life Quality Initiative and what it would cover at both national and regional level. In terms of a change programme I think it’s vital to have a vision. It’s vital to communicate that vision, but that’s only one per cent of the action necessary

- 35 -

to make that vision a reality. And don’t forget we’re working with deeply entrenched problems. It was also suggested that there was a need to have more clarity about what the national Skills for Life offer was: I think maybe some more structure to the planning so that there is some clarity over what the national offer is going to be and what facility the regions have to amend or utilise that national offer in the regions. I think you need the structures in place in the regions to be able to respond to that. Overall however, by the end of the second year, the benefits of the regional policy were perceived as considerable. In addition to the more targeted approach and the renewed sense of priority given to the initiative as illustrated in the following: Designated this year by the LSC, you know exactly who they wanted to nominate as providers, who they wanted to work with them…and really achieving some very high quality things Focus group Regional working resulted in: a focus on the building up of a regional infrastructure that has allowed for regional sharing of good practice Focus group and the development of greater ownership and collaboration: it’s allowed the players within the regions to have more of a say and I think that’s been really collaborative and made us think in a joined-up way, and allowed us to evaluate exactly what training we need, where our weaknesses were and where our gaps were Focus group However, achieving such benefits was more difficult in some regions because of their state of readiness: There were some regions that were right for it because the infrastructure is already well developed….but I think where you have got very fragmented regions like some of the ones that I am working in things are an uphill struggle, they are almost not ready for it and they are not ready to take on that responsibility. Focus group Aspects of the planning and delivery of the initiative were said to have been effective in some regions and less so in others. The overall planning and delivery was described as patchy by interviewees.

- 36 -

I think that overall the planning and delivery has been patchy, has been effective in some parts and not in others. I think one of the problems…from the DfES side of things is…the Skills for Life Quality Initiative is planned on a yearly cycle and it is not a two or three year programme….In terms of LSC national planning we started a little too late in bringing in regional planning. In addition, there was a lack of awareness among interviewees of the policy of targeting specific providers. Most of the practitioners interviewed were under the impression that all providers had been invited to participate in the initiative. Interviewees felt that the policy of targeting specific needs could have led to mixed messages and inconsistencies in the delivery of the programme regionally: I think it opens up a can of worms because unless there’s a clear strategic intent, which is managed consistently across the country, lots of mixed messages are going to develop and this will then lead to inconsistency. And it’ll devalue the perceived identity of the brand that the government is trying to create for this product. In the survey, when respondents were asked about the impact of delivering the Skills for Life Quality Initiative via LSC regional structures, 44% felt that the impact had been significant or substantial and another 40% reported that they perceived some impact. Only 6% felt that there had been very little impact. 3.2.3 Readiness of the workforce There was some perception that the initiative had been started before the workforce was ready to benefit from it. This is discussed in further detail in the section on the impact of the initiative upon people. The following comment clearly states the nature of the problem: This was launched on a world which was trying to come to terms with many different aspects of the Skills for Life strategy at once, not least the driving up of demand and before the Department’s commitment to qualifying the workforce. The cart, as it were, came before the horse. The demand was driven up before professionalisation and qualifications could catch up and this was happening with a workforce which was largely, indeed increasingly unqualified, because demand was getting larger. Focus group As noted earlier, an overall purpose of the Skills for Life Quality Initiative was to professionalise the workforce. A key part of this professionalisation involved driving up the quality of existing teachers by building on their pedagogical expertise through the addition of specialised training in Skills for Life. Unlike the schools sector however, many teachers of Skills for Life were without basic teaching qualifications. As a consequence there was a perception that the Skills for Life Quality Initiative continuing professional development activity was not always fit for purpose for unqualified teachers.

- 37 -

3.2.4 Range of initiatives Finally, it is important to note that the Skills for Life Quality Initiative was operating within a crowded market in terms of the range of initiatives the learning and skills sector was coming to terms with and the plethora of support programmes and materials available to teachers. The need for rationalisation across initiatives designed for the sector and a more coherent approach to quality improvement was evident and likely to have influenced the way in which the initiative was perceived. We should be thinking in terms of bigger pictures as well because there are lots of other quite significant initiatives out there that service the needs of the sector…..subject learning coaches for example. A lot of their subject areas will have lots of learners that need support with literacy and numeracy and at a strategic level there should be some very strong and practical interlinking between this sort of development (Skills for Life Quality Initiative) and that sort of development (subject learning coaches). Focus group A focus group of practitioners commented on the mass of information they received from various training and support agencies: We are getting envelopes through the door daily from people, CfBT are doing this, the key skills support programme are doing that, the NUT is doing this, Matrix is doing that. Obviously it is a market, they are going to find a niche but it becomes confusing and I think it would be better if there were some co-ordination of it or some way in which the competitors worked together Focus group 3.3 IMPACT ON SYSTEMS In analysing the impact of the Skills for Life Quality Initiative upon systems it is important to note that it takes time for a large scale reform to become embedded, consequently according to respondents the impact on systems was just beginning to become evident. 3.3.1 Organisational change It is useful to begin first with change and identify the model of change that was implicit in the initiative itself, before examining the approach to change encouraged in organisations participating in the initiative. Consortium managers argued the case for perceiving the Skills for Life Quality Initiative as developmental, funding existed to support the development of organisations and individuals and not to reward organisations for reaching targets. I would say it was a developmental initiative, I can’t say that enough times, it wasn’t a funding mechanism to get numbers of trainers….it was a developmental strategy which goes forward and back and forward…

- 38 -

Focus group More specifically the nature of this developmental approach to change was one which focused on changing the culture of providers: Just sending people onto training, however committed those individuals are, they need an infrastructure to go back to in order to be able to practise their skills, get other people using those skills and actually make an impact on the curriculum, and organisation, that will actually in the end result in lots of people improving their skills and literacy. That means organisational change, cultural change Focus group Also on promoting a sense of ownership: Organisations are now realising they need to work on the ground and that has really empowered them…. they are responsible for quality, not someone else coming in and doing it to them, that idea that you own it and you have to make it comes not from an event, it’s a whole process Focus group The emphasis of the Skills for Life Quality Initiative upon the provision of training, resources and facilitation, in addition to a desire to effect cultural change and promote ownership suggests the implicit strategy of change is a combination of a rational approach and a normative approach. Rational approaches rely on demonstrating a better way of doing something, through training and facilitation in this instance, which is then adopted because of its proven good sense and practical benefits. Normative approaches involve individuals changing their practise because they incorporate change as their own. In this instance, this occurs through taking ownership of quality improvement and promoting cultural change in their organisation from within. The two strategies are often interlinked, whereby a rational approach stimulates a normative response. Given the need to promote cultural change and ownership, the Consortium management viewed the transition from a national to a regional programme itself as part of the development process and something that could continue to move closer to providers themselves: More funding should go straight to the provider to self improvement action, whether it was action research based or whatever Focus group Consistent with the notion of culture change and ownership was the whole organisation approach which had been a key feature of the initiative. As a provider manager commented: The Skills for Life provision, I believe, will only work successfully if basic skills is totally embedded in the curriculum and lecturers and college departments

- 39 -

will only do that if there’s a very, very clear steer from Principals and from Governors Email response Emerging from respondents was a view that the whole organisation approach was beginning to impact upon providers: Skills for Life not taking place in a dimly lit corner, but recognised as part of a whole organisation was obviously a major part of the initiative and on the whole I think this has been taken on board. Although, having also been working with one large Borough Council I know it is going to take time for policies to be implemented at corporate level. Email response Interviewees felt that the whole organisation approach element of the initiative had allowed them to see the ‘bigger picture’ and had provided the opportunity to meet and network with other deliverers of Skills for Life. I think through the training that we have done…we are seeing a bigger picture and we have got opportunities to meet up with other people who are delivering Skills for Life…and the resources…we’re just understanding and coming to terms with what the whole organisation approach means for Skills for Life 3.3.2 Examples of impact Interviewees and focus group members pointed to a number of examples of where the Skills for Life Quality Initiative had made an impact upon organisational systems. These included the restructuring of internal processes such as strategic planning, workforce development, senior management engagement and embedding Skills for Life in vocational teaching. The Skills for Life Quality Initiative was seen as an initiative that had been embraced at all levels within provider organisations: The feedback I get…from the champions inside organisations, is that it is promoting change…here’s something we can do, our tutors can do, we can do for clients, we can do for our customers, and we can do for our workforce, and that seems to have been taken on at a very high level, from the boardroom down in most cases. Interviewees believed that the Skills for Life Quality Initiative had acted as a catalyst for providers and encouraged the restructuring of their internal processes in the development of their own Skills for Life strategies. The Skills for Life Quality Initiative has been a catalyst for us to focus on Skills for Life, which has bought it further up our priority list…Our in-house Skills for Life strategy is being written now whereas previously it’s just been informal policy papers linked to our provision. and

- 40 -

The impact on our organisation…it has given us training opportunities. It has helped us to grow in confidence as we develop our strategy. The following examples from two different regions point to success in changing practice in prisons: We have done some cluster consultation groups with prisons particularly in the ……and they have moved quite considerably to adapt their organisational practice, looking at the workforce across the prison service, and that’s been, despite the constraints of only having three facilitation days in rapid succession, quite successful Focus group Once again in prisons in………the work with them to develop a peer-partner programme is going very well Focus group Data in the Final Report to the LSC on the Skills for Life Quality Initiative supports the view that internal processes in provider organisations had been influenced by the Initiative. For example, the report states that the number of providers who have established or plan to establish a strategy for responding to future development of Key Skills and Skills for Life has increased from 20 to 59 per cent, with a further 21 per cent planning implementation. The following focus groups extract points to changes to senior management job descriptions: Most organisations that have engaged with the initiative now have Skills for Life either on senior managers’ job descriptions or a nominated senior manager with a responsibility for Skills for Life….and the qualifications profile of a typical Skills for Life deliverer is richer now. Focus group The Final Report to the LSC also notes that the number of providers who have identified or have sent relevant managers on leadership and management training has increased from 15 to 69 per cent, with a further 19 per cent planning to identify relevant managers. The impact of the embedded approach was also noted: With one facilitator we are working on embedded literacy techniques and I would say yes, that is having an effect on the vocational centres. With the other facilitator we are working on the whole organisational approach. What is being done is we are looking at how Skills for Life interacts across both the curriculum and business support processes. Bearing in mind the emphasis now on supporting Skills for Life and embedded teaching, we are looking at the way we recruit people, how we write job descriptions and how we support people to embed Skills for Life once they are in their role.

- 41 -

Again, the Final Report to the LSC supported the developing impact of the embedded approach with the number of providers who had established or planned to establish a clear strategy to embed Skills for Life in vocational areas having increased from 18 to 64 per cent, with a further 24 per cent planning change. In terms of the impact upon organisations themselves, there was also a sense of realism about how difficult it is to establish direct cause and effect: I think what is clear is that organisations have changed and shifted, how much of that is due to Skills for Life Quality Initiative I think is a difficulty because there have been so many other drivers, as well as organisations understanding the significance of literacy, language and numeracy. Focus group In addition, there was some caution about how far change had penetrated into organisations: The other thing I would say is that I see a lot of strategies and people working on strategies and reviewing strategies, what I don’t see very much of in reports are actions to show how strategies are being implemented, how it is impacting on learning programmes and on the learner. Focus group 3.3.3 Impact on other sub-systems Respondents to the survey were asked about the impact of the Skills for Life Quality Initiative upon the following:

• Professional development • Whole organisation approach to quality improvement • Levels of achievement • Teaching and learning • Organisational change

On average, around 40% agreed that there had been some impact across all of the above areas. The proportion was higher however, when respondents were asked about the impact of the initiative upon the quality of regional professional development activity, with 50% ‘tending to agree’ that it had and another 6% replying that they ‘strongly agree’. The level of agreement with the statements regarding the impact of the initiative upon these systems is summarised in Table 1 below.

Table 1 Impact of the Skills for Life Quality Initiative upon systems

Area of impact/degree of impact

Percentage tend to agree Skills for Life Quality Initiative has

Percentage strongly agree

- 42 -

had impact Professional development

50 6

Quality improvement

45 13

Levels of achievement

31 7

Teaching and learning

44 13

Organisational change

44 10

This shows that in all areas except the impact on learner achievement, more than half of all respondents believed that the Skills for Life Quality Initiative has had an impact upon systems of development, improvement, learning and change. Further evidence of this impact comes from the Final Report to the LSC which shows that the number of providers who:

• Have coaching and mentoring strategies to train and upskill staff, and share good practice has increased from 32 to 65 per cent

• Have prioritised the development of e-learning as a key Skill for Life has increased from 28 to 60 per cent

• Have established a process to evaluate changed teaching and learning practice as a result of Skills for Life training and resources has increased from 21 to 57 per cent

• Have established effective assessment procedures and processes for learners and evaluated their impact has increased from 41 to 74 per cent.

3.4 IMPACT ON PEOPLE As noted in the section on policy there was a perception that the Skills for Life Quality Initiative had been launched onto a workforce that was largely unprepared for it. This view is confirmed by practitioners who note that: Many tutors were insufficiently ready, especially vocational tutors. In addition, the nature of the Skills for Life workforce, with many part time tutors on sessional contracts makes it a fragmented workforce, difficult to reach Focus group 3.4.1 Benefits for practitioners Data from the focus groups and interviews showed that the impact of the Skills for Life Quality Initiative upon practitioners had been considerable. This impact had taken the form of enabling practitioners to reflect on the challenges of, and provide them with the skills to, deliver Skills for Life. In addition, it had raised the profile of Skills for Life, encouraged joint working

- 43 -

and improved communication between practitioners and between practitioners and provider management. Practitioners reported that the Skills for Life Quality Initiative has helped them think through the issues and difficulties involved in Skills for Life and helped to ‘up skill’ them to face the challenges provided in their teaching. I began to appreciate that when you embed Skills for Life properly into a programme of learning you can create another depth in what you offer learners. You can teach them vocational skills and explicitly introduce and talk through issues around numeracy and literacy, but in doing that you have to become more intimate with the learners. That very intimacy means that the learner is going to get a better experience. Provider managers also acknowledged this impact: It has enabled us to look more internally at our own staff and we have conducted staff audits and offered staff the opportunity to take level 2 qualifications, support staff and academic staff, so all of that has enabled us to move forward in a way that we wouldn’t have done without it. Most teachers reported positive experiences on training programmes and with facilitators. On a personal level it makes a huge difference, especially when you go to a good training event, you really take it on board and do something with it Focus group I think it allows us to raise the profile within the college, with senior management, that is certainly how we have had some benefit in the last couple of months. Because we had the external facilitators coming in, I have secured very meaningful discussions with 5 of the 6 senior management team, the facilitators have helped me get Skills for Life very high up on the agenda Focus group A large number of those interviewed felt that the programme had instilled confidence within teachers: It’s improved the confidence of our teachers and they absolutely loved it In addition, it provided them with the opportunity to reflect upon what they did as well as to help identify learners’ needs. It is very valuable when you are teaching to have that broad approach and not just confine your efforts to teaching somebody to read, but to look at all other aspects of their life including the networks they have, and all the organisations they come into contact with

- 44 -

The Skills for Life Quality Initiative was observed by interviewees to have provided a consistent message across the regions on the approach for delivering Skills for Life provision. This was said to have encouraged joint working among providers: I think the key element of this is that we’ve had a consistent message across the region so that we haven’t been developing local messages which were slightly off-centre. There’s been that joined-up ness and I think the Skills for Life Quality Initiative and its resources have made that possible. The initiative was also said to have improved communication amongst peers, managers and within whole organisations and heightened the importance of continuous professional development for practitioners involved in Skills for Life. Just talking to other people has been valuable because I think a lot of basic skills tutors and Skills for Life tutors feel quite isolated because their organisations aren’t aware of things like core curriculum levels and the network of trainers and tutors interacting with each other has been valuable Practitioners also reported feeling swamped by materials and offers of support and made a plea for more resources available ‘on the ground’ in the form of time to customise materials to the learning needs of their particular learners, or smaller classes to enable them to provide more individual attention to learners. This was consistent with the point made earlier, which noted that the next stage in the development of the initiative should be the provision of more resources direct to providers to enable them to work on changes to classroom practice. I don’t want to go on any more conferences, I don’t want any more materials….having the time to develop the materials to make it relevant to our cohort that is the thing….you have got to customise to every group that you have. Focus group However, the majority felt that it was too early to assess whether the Skills for Life Quality Initiative had led to higher levels of achievement and improvement of the learner experience, some felt that this would be the case, but accepted that there was no evidence to back this up: I don’t know that there’s evidence that it has…but I believe firmly that it does The Final Report to the LSC on the Skills for Life Quality Initiative 2005-06 notes a number of contributions of the Initiative to the improvement of the learner experience and to capacity building of teaching staff. The report suggests that the learner experience has been improved through the contribution of the Skills for Life Quality Initiative to improvement in provider standards, the encouragement of a more coherent approach to quality improvement and by extending the reach of the Skills for Life Quality Initiative to a wider range of providers. This has been achieved with a greater variety in

- 45 -

the type and amount of support provided, a more customised approach to the development of materials in specific sectors and by working closely with regions to develop alternative delivery models. With regard to capacity building, the report states that in most regions there were practitioners who completed accredited training programmes and that the Consortium had used a variety of approaches including training, briefing and shadowing to ensure the development of skills and expertise to deliver high quality teacher training. When asked about the overall benefit of the Skills for Life Quality Initiative to practitioners in the survey, 63% felt that the initiative had either been ‘of great benefit’ or ‘significant’ to those teaching, training teachers and managing Skills for Life. However, 27% did not think they had benefited that much from the programme. The initiative seems to have been of most benefit to those in the work based learning part of the sector and to those in further education colleges. Those that reported that they had least benefited, were participants from the ACL sector, with only 35% of respondents feeling that they had benefited greatly or significantly from the programme. When asked about the overall impact of the Skills for Life Quality Initiative over the last few years 5% reported that the impact had been ‘very substantial’, 37% that it had been ‘significant’, and 37% that it had ‘some impact’. Only 8% of respondents reported that it had ‘little impact’. 3.4.2 Maintaining progress For the Skills for Life Quality Initiative to be effective, interviewees felt that it had to focus more on the people planning the initiative by encouraging feedback and improving communication lines between providers and those planning and delivering the programme of support. More consultations were necessary with those involved, specifically with trainers, tutors and facilitators on issues such as how to develop any further phase of the initiative. A thorough consultation at this stage would be exceptionally helpful. Many of us have been delivering Skills for Life since its inception around 2000 and it’s probably an appropriate time to gather feedback and start to plan on the back of that. Regional planners would also like to see more formative feedback from central CfBT to enable more effective provision: Because the provision in the Skills for Life Quality Initiative for vocational teachers is only this year and because of the centralised system, unless you’ve intercepted it in some way, you don’t know how many vocational teachers from your area have actually gone on it until you get the reports back at the end of the year

- 46 -

Focus group Interviewees also felt that there was a need for a more flexible approach including focusing on the specific needs of individual organisations. Follow up support in the form of action plans needed to be included and evaluated on an ongoing basis. I think in terms of the limitations, it is the view that organisations have to move on at the same rate as I’ve said before. A little bit more flexibility would have helped the process, but generally it’s been very positive. This was echoed in focus groups where a concern to understand what was happening with learners was evident: Colleges have got their strategies, they’ve got their managers, and they have even got their working parties, but actually what is happening underneath this to impact on the learning? Where we need to focus more of what the Skills for Life Quality Initiative is doing in the future is helping organisations with translating strategy into action and impacting on learning programmes. Focus group Picking up on the theme of developing a further phase of the initiative, there was some concern about keeping momentum and learning from the rich collection of data available: With these final reports that are coming in, there is a box there, issues and actions to carry on post March. Facilitators are completing them in great detail which suggests that a lot of discussions have been going on. It needs somebody in a facilitator kind of role to move it along and keep the focus there, somebody needs to be managing teams of facilitators, to check those reports, check that they are on task, line up that one with that one over there and so on. Also managing the training and updating of the team and I think it’s very important, whether it is at CfBT level or whatever, it shouldn’t just be left to happen. Focus group In addition, the value of follow up contacts with providers was stressed: I think there should be plans for revisiting previous year’s clients…because the danger is you form an action plan and then something goes wrong, or someone leaves and sits on a shelf. I think the facilitator could take a more proactive approach year on year, so I think there ought to be opportunities for revisiting previous users and helping them, nudging them along to implement what they’ve agreed to do. The theme of a more informed dialogue between regional planners and central management of the initiative, with the purpose of continuous improvement, was strongly reiterated: Let us plan in partnership, let us as regions plan in partnership with the national bodies over the shape of what comes. There are only nine regions,

- 47 -

so it’s not impossible to have regional representation in the thinking about how to get the balance right between what’s national and what’s regional, what are the constraints Focus group I’d put the professional debate in that planning, a professional element, because I think what happens is that the Department and the LSC decide their priorities, fair enough because that’s where the budget comes from, but then I think that gets very quickly operationalised without a continuing professional element as the plans get formed. Focus group

- 48 -

5. CONCLUSIONS AND RECOMMENDATIONS The following conclusions and recommendations are summarised under the aims of the evaluation. 5.1 Conclusions and recommendations summarised under the aims of the evaluation Achievements since 2000

• quantity and quality of resources and support framework • development of the regional infrastructure for delivery • creation of a less piecemeal, more developmental, initiative • establishment of high aspirations involving cultural change in provider

organisations and the empowerment of providers • promotion of organisational change • improved quality of professional development activity • helped to up skill practitioners and prepare them for the challenges of

Skills for Life • instilled confidence within teachers and encouraged reflection upon

professional practice Strengths of planning and delivery

• the shift to regional planning and delivery • greater customisation of support and training with the involvement of

providers • high quality events • much facilitation activity highly valued

Weaknesses of planning and delivery

• stop/start nature of the initiative • variable quality of facilitation • too target-driven, possibility of lack of focus on the overall goals of

professionalising the workforce and developing a whole organisation approach, due to concentration on the achievement of individual targets

• delays to start of training provision • compressed time span available to deliver training • administrative difficulties in arranging and delivering training • lack of consistency in the process of developing regional plans • insufficient focus on sustainability • lack of coterminous financial and academic year planning periods

Extent to which targets have been met

• virtually all targets met or exceeded in 2004-05 • facilitation days targets exceeded in 2005-06

- 49 -

• training days target not met in 2005-06, achievement at 61% at the end of March 2006

Consequences for providers of delivery via LSC regional infrastructure

• more ‘tailored’ support • greater sense of control and ownership • evolution of regional support infrastructure • more collaborative relationships with LSCs • variable provision across LSC regions • initial ‘teething’ problems leading to lack of clarity about the nature of

the regional offer What more needs to be done The recommendations in this section and the following one have emerged from the evaluation data and from discussions in the expert seminar.

• the initiative to continue to develop and strengthen. The majority of respondents wanted identified quality improvement support for Skills for Life teachers and managers to continue. In particular, respondents identified the facilitation element as critical to further quality improvement

• evolution of a more strategic approach avoiding the stop/start nature of previous years

• development of a more joined up approach bringing together the various quality improvement initiatives under one umbrella

• continue customisation and allocate some resources direct to providers • review the menu of professional development provision for Skills for

Life staff, including ensuring that the right people get on the right courses, and the provision of appropriate professional development for vocational teachers. Close links should be forged between the Skills for Life Quality Initiative and the reform of teacher training currently being conducted by the DfES and LLUK.

• the Skills for Life Quality Initiative to work closely with LLUK and SVUK and to collaborate on the generation of better data on who is and who is not qualified, analysis of this data and feedback to the sector

• brief inspectors to look for clear evidence of appropriate Skills for Life developments

• re-prioritise ACL and ‘harder to reach’ providers • carry on the regionalisation and further customisation of the initiative.

This customisation could be supported by the differentiation of the sector workforce in terms of data collection about different elements of the workforce and their experience in delivering Skills for Life and of the Skills for Life Quality Initiative

• further develop embedding by focusing CPD on vocational teachers • facilitate peer coaching among Skills for Life trainers and between

Skills for Life and vocational trainers • promote more active sharing of good practice

- 50 -

• further development of work on advice and guidance for Skills for Life teachers to ensure that the Skills for Life Quality Initiative professional development activities supports teachers in gaining qualified teacher status

• promote the sustainability of the Initiative by considering how the wealth of resources produced through the Skills for Life Quality Initiative can be kept up to date and how they can be used effectively in initial teacher training

• establish a pool of quality assured consultants and approved trainers. This recommendation emerged from the regional quality improvement managers (RQIMs) who were aware that a list of Consortium approved trainers already existed. However, each region received some funding for development work and training delivery, managed within the regions that was outside the Consortium-led activities. RQIMs were aware that some trainers and consultants who had not been offered work by the Consortium had succeeded in obtaining work funded from the regional development funds.

• dovetail into the QIA framework with policy support from the DfES, and ownership by providers for sustainability

• promote more effective regional planning by ensuring systematic feedback of data collection to regions from central CfBT and ensuring a professional dialogue in regional planning taking account of research findings and intelligence from the ‘field’

• plan a follow up evaluation on an experimental model basis, and also an evaluation which will focus on the impact of the initiative at classroom level once the initiative has had more time to make an impact on teaching and learning

5.2 Overall summary of the Achievements of the Skills for Life Quality Initiative This evaluation has demonstrated that the Skills for Life Quality Initiative has been an influential intervention into the quality of Skills for Life provision. Large scale reform of the provision of Skills for Life involving changing the culture of individual provider organisations and the practice of individual teachers is a long term process, but significant progress has been made and evidence of organisational change is beginning to emerge. The consensus of all respondents was that a comprehensive range of high quality materials and support framework had been developed. Whilst respondents acknowledged the challenges facing facilitators working with a wide range of organisations within the different parts of the sector, overall this work was highly valued and strong arguments were made in favour of its continuation in the future. The CfBT consortium has made some impressive achievements in delivering on a wide range of targets, but has been hampered by an extended contracting period due to a complex strategic planning process involving marrying needs of the regions with resources available nationally. This extended planning period led to some difficulties in delivery at grassroots

- 51 -

level. These have been compounded by administrative issues in reconciling national and regional planning and delivery mechanisms. The stop/start nature of the initiative has militated against the development of a clear strategic approach to quality improvement, but the decision to shift responsibility for planning and delivery of the initiative to the LSC regional infrastructure has been widely welcomed and very successful. Initial ‘teething’ problems have been largely overcome and an effective regional infrastructure has developed, supported by good professional links between the various key stakeholders and the development of trusting working relationships. Many practitioners in the Skills for Life workforce were unprepared for the growth in Skills for Life provision and some were not in a position to benefit from the emphasis on professional development central to the Skills for Life Quality Initiative. Many had no initial teacher training and there were a high proportion of part-time sessional members in the workforce. Despite this, professional development was seen as beginning to have a positive impact on practitioner confidence and skills. Improvements in the professional practice of the Skills for Life workforce will have benefits for the wider learning and skills sector. The goal of professionalising the workforce is closely linked with the Government’s agenda on the reform of teacher education and improvements in teaching Skills for Life will also support other Government priorities, such as the achievement of the level 2 qualification target. The goals of the Skills for Life Quality Initiative are far-reaching, involving the promotion of cultural change in provider organisations and stimulating changes in individual professional practice. These goals reflect the ambitious and challenging nature of the Skills for Life strategy and targets. Individual and organisational change, are complex processes to influence and it takes time for change to take effect. However, positive organisational and individual change is perceived to be occurring, although across the sector the impact is variable. Further education and work based learning providers report a positive impact. Adult and community learning providers are less sure of the benefits of the initiative. The approach to change adopted for the Skills for Life Quality Initiative and noted in section 4.3.1, sits within a social interaction tradition which views change, in the form of better practise, as occurring where awareness is generated through personal contacts and group activity. This approach is more effective at explaining how change is communicated, rather than how it is implemented. In the context of the Skills for Life Quality Initiative more can be learnt about how it has been communicated and perceived than how it has been implemented, particularly at classroom level and its consequent impact on teaching and learning. Any further evaluation activity needs to focus on the impact of the Skills for Life Quality Initiative on teaching and learning.

- 52 -

5.3 The two-way implementation staircase This section employs the concept of the two-way implementation staircase developed at Lancaster University to assess the implementation of the Skills for Life Quality Initiative based on the data produced by this evaluation. The stakeholder groups occupying the different steps on the staircase are as follows:

• DfES • LSC national and regional offices • CfBT Consortium • RQIMs/RTOs • Facilitators/Trainers • Provider Organisations • Teachers/trainers/learning support • Learners

At each step on the staircase the likelihood of modification or adaptation of the initiative exists, as members of each stakeholder group assesses the policy and plays its part in its implementation. The consequence for the Skills for Life Quality Initiative is that as there are many stakeholders in the chain of implementation there will be many interpretations of the initiative. This is further complicated by the roll out of the initiative via the regional infrastructure, effectively ensuring that 9 versions of the Skills for Life Quality Initiative exist as each region will have modified and contextualised the initiative to fit it to their specific circumstances. As the evaluation has demonstrated, this scenario is widely supported within regions, allowing customisation of the initiative to the specific needs of regions and providers and also promoting the ownership of the initiative at grassroots level. This can create tensions between the different stakeholder groups on the staircase. In a national context of increasing personalisation and a developing ‘local solutions for local people’ philosophy, demand for further customisation of the Skills for Life Quality Initiative is likely to continue. Balanced against this is the reality that it is also necessary to establish national priorities that enable government and its agencies to shape the reach, quality and form of its services. In this scenario, the future for the delivery of the Skills for Life Quality Initiative may be in the establishment of strong central guidelines about key priorities, but giving providers more freedom to develop and deliver Skills for Life in their own way, subject to certain key principles.

- 53 -

- 54 -

ACKNOWLEDGEMENTS NRDC would like to thank all those who gave their time in focus groups, interviews and the completion of questionnaires. This final report has also benefited from comments following a presentation to the LSC attended by colleagues from CfBT, DfES, LSC and QIA.

- 55 -

ANNEXES ANNEX 1 References

ALI (2002) Annual Report of the Chief Inspector 2001-02 ALI (2003) Annual Report of the Chief Inspector 2002-03 ALI (2004) Annual Report of the Chief Inspector 2003-04 ALI (2005) Annual Report of the Chief Inspector 2004-05 ALI (2005) Skills for Life at Work: Making the most of training. June 2005 DfEE (1999) Improving literacy and numeracy: A fresh start. Report of the working group chaired by Sir Claus Moser DfEE (2001) Skills for Life: The national strategy for improving adult literacy and numeracy skills. DfES (2003) Skills for Life: Focus on delivery to 2007. FEDA (2000) Stage 2 of the ILQI Evaluation of Institutions progress towards their targets. Report for the FEFC August 2000 FEFC (1997) Inclusive Learning. Report of the learning difficulties and/or disabilities committee chaired by Professor John Tomlinson. FEFC (1998a) Basic education curriculum area survey report. Report from the Inspectorate. April 1998 FEFC (1998b) Circular 98/31 FEFC (1999a) Quality and Standards in FE in England 1998-99. Chief Inspectors Annual report FEFC (1999b) Numeracy, Literacy and ESOL: Evaluation of entry and level 1 awards. National report from the Inspectorate 1998-99. FEFC (1999c) Circular 99/50 FEFC (1999d) Circular 99/44 GHK (2004) Evaluation of the LSC Skills for Life Quality Initiative. Final Report March 2004

- 56 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

LSC (2001) Circular 01/15 NAO (2004) Skills for Life: Improving adult literacy and numeracy. December 2004 NRDC/CEI (2005) Evaluation of the Skills for Life Quality Initiative 2004-05. Final report April 2005. Ofsted/ALI (2003) Literacy, Numeracy and English for speakers of other languages: a survey of current practice in post 16 and adult provision.

Stanton, G (2001) Lessons learned on raising quality and achievement. LSDA.

- 57 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

ANNEX 2 Research methods

1.1 Influences and design issues

The methods used in this project have taken into account a number of influences and issues. These included:

• The need to achieve the aims of the evaluation • The short time-scale available to complete the evaluation • The need to provide evidence that is sufficiently robust, given the time-

scale • The desire to capture strong regional data and to gain a picture of

regional implementation • The number of achieved interviews necessary to form a representative

sample of the regions and the respondent groups involved • The key questions the evaluation should address as specified by the

Skills for Life Quality Initiative Senior Management Group • The two-way implementation staircase model developed by Lancaster

University, which provides a framework for exploring how policy intentions can be changed and modified as they move through the various stages of implementation (the ‘Chinese Whispers’ effect).

• The assumption that a change model, either of an explicit or implicit nature, has informed the development of the initiative

• The identification of professional development of the workforce, and the need for a more strategic approach to sector improvement, as key issues for the evaluation.

1.2 Evidence and rigour

Robustness of findings cannot be guaranteed by any of the sources of evidence used taken singly. For example, no one focus group provides representative evidence from the range of stakeholders sampled in that group. However, having taken an approach which involved gathering converging evidence from a range of sources, the accumulation of evidence showed where the different sources supported or contradicted each other. The comparison of the different sources acted as a form of triangulation. That is, for each of the evaluation questions we trawled the different sources of evidence, put the findings side by side, considered the similarities and differences, and took an informed view of why perspectives coincided or differed, in order to arrive at an overall judgement. In particular, opinions and perceptions gathered by qualitative means were checked against the quantitative evidence provided by CfBT.

- 58 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

It is important to acknowledge that this evaluation inherently cannot address questions of cause and effect. To do this would have required the authorisation, several years ago, of a different method of choosing which providers should participate (randomisation) in order to control for all the multifarious elements of wider activity and context and influences in the field. What this evaluation has addressed is the perceptions and judgements of a range of participants on the extent to which the Initiative has achieved its purposes, cross-checked where possible against quantitative data. 1.3 Self-evaluation

This evaluation has contained an important element of self-evaluation. The consortium led by CfBT was a key group of agencies and organisations involved in developing, implementing and supporting the Skills for Life infrastructure. As such, the consortium was a major stakeholder and constituted some key steps in an implementation staircase whereby the Skills for Life policy was received and transmitted to other implementers. In addition, the consortium acted to analyse Skills for Life policy and strategy and to send messages ‘back up’ the staircase to policy makers. The evaluation of the Skills for Life Quality Initiative is thus also an evaluation of the work of the consortium. 1.4 The research process

Each data gathering method employed as part of the evaluation is summarised below. An overall view of the research design is included as Table 2 below. 1.4.1 Desk research

This stage comprised:

• An analysis of quantitative data on progress towards targets set under the Skills for Life Quality Initiative contract, drawing on CfBT data.

• A review of documents relating to the history and development of the various Skills for Life Quality Initiatives since 2000 and their evaluation.

• An analysis of trends in participation and achievement, drawing on LSC ILR data.

1.4.2 Survey

A short on-line quantitative survey of participants was conducted between February and mid-March. The survey was published on the NRDC web-site and the Skills for Life Quality Initiative web-site. Links to the survey were sent round via NRDC contact lists and networks in order to stimulate participants to complete it.

- 59 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

Seventy respondents had completed the survey by the deadline date of 10th March. Responses to surveys as a form of data collection are often low and web-site surveys present their own problems of completion, given that potential respondents have to visit the web-site of their own volition, rather than receiving a direct mailing. However, the data collected from the survey provides a valuable addition to the data collected from the range of qualitative data gathering methods used. A copy of the survey is attached as Annex 3 to this report. 1.4.3 Interviews

Thirty individual key informant interviews were conducted. Respondents were drawn from five regions and from a range of organisations, concentrating on representatives from managers and teachers in provider organisations and the LSC Regional Leads. Interviews were conducted either face to face or by telephone. A common interview guide was used in each interview to provide a consistent framework for questioning. The regions chosen were the North- West, Yorkshire and Humberside, the East Midlands, the East of England and London. 1.4.4 Focus groups

Six focus groups were held to elicit perceptions of Skills for Life Quality Initiative policy development, design and delivery from:

• 2 meetings of CfBT Skills for Life Quality Initiative groups (i) Senior Partners Expert Group (ii) Consortium Group

• CfBT Regional Skills for Life Quality Initiative Managers. • Practitioners including managers, teacher-trainers and teachers (2

groups) • A semi-structured focus group was held during a residential training

event for participants on the NRDC Practitioner-led Research Initiative. The groups used a focus group guide to provide a common framework for the conduct of and questions for discussion. 1.4.5 Use of other NRDC data

(1) From the study of the Impact of the Skills for Life learning infrastructure on learners (the Learners Study) In addition to the methods specified above, Skills for Life stakeholders linked to the NRDC learners study were contacted.

- 60 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

Data was collected through a few face-to-face interviews, but most of the interviews were conducted over the phone. Short questionnaires completed and returned by e-mail were the other major data collection process. Interview notes and transcripts from the Skills for Life Learners Study were examined and references to the Skills for Life Quality Initiative were noted. The data from this exercise falls into two categories. The first category is the interview notes and transcripts from the Learners study as these were mainly collected during 2004 and the early part of 2005. Specific questions about the Skills for Life Quality Initiative were not asked so these are unprompted responses. 18 transcripts are included in the first category. The second category consists of those who were contacted specifically for the purposes of the Skills for Life Quality Initiative study and 28 people are included in this. Interviews and questionnaires were completed in January and February 2006. (2) From the Effective practice studies NRDC has recently completed five studies under this title, one each concerned with reading, writing, ESOL, numeracy and ICT. The five project teams were asked to search their data for any mentions of the Initiative; all reported no mentions. 1.4.6 Expert seminar

In order to address what still needed to be done to support the implementation of the Skills for Life strategy, an expert seminar was conducted. Seminar members consisted of 16 expert commentators drawn from the Skills for Life Quality Initiative management, LSC regions, CfBT Regional Quality Improvement Managers, provider managers, teachers and teacher-educators. This group met together for half a day on 22nd March to hear the emerging findings from the evaluation, to critically analyse what had been achieved, and to make proposals for what more still needed to be done.

- 61 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

Table 2: Overall research design

Aims Initial key questions Method Key respondent groups

Assess what has been achieved through the Skills for Life Quality Initiatives operational since 2000

• What was the Skills for Life Quality Initiative set up to do?

• Has it achieved this? • What obstacles have

been encountered? • What unintended

outcomes have occurred?

• What lessons have been learnt?

• Overall what difference have the Skills for Life Quality Initiatives made to the various stakeholders?

• Collation of quantitative data on targets and actual provision

• Desk research on history of the Skills for Life Quality Initiative and its development and impact as reported in earlier evaluations

Analyse the main strengths and weaknesses to inform the future development of the programme

• Have planning and delivery been efficient and effective?

• Has the ‘change model’ worked?

• What are the strengths and weaknesses of the Skills for Life Quality Initiative as a CPD programme?

• Has the Skills for Life Quality Initiative enabled teachers to be better teachers?

• Has the Skills for Life Quality Initiative led to higher levels of learner achievement?

• What has been the impact of ‘timing glitches’?

• Interviews • Focus groups • Survey • Extending the

scope of the NRDC learners study

• Skills for Life Quality Initiative managers

• LSC regional leads

• RQIMs • Provider

managers • Teachers • Teacher-

educators

To consider the consequences for providers of the delivery of the Skills for Life Quality Initiative through the LSC regional infrastructure

• What difference has been experienced by providers as a result of the delivery of the Skills for Life Quality Initiative through the regional infrastructure, as opposed to direct delivery to

• Interviews • Focus groups • Survey

• LSC regional leads

• Provider managers

• Teachers • RQIMs

- 62 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

organisations? • What has been the

impact on providers of Regional LSCs inviting specific providers to participate?

• What have been the consequences of targeting specific needs at a regional level resulting in a varying Skills for Life Quality Initiative offer between different regions?

Assess what still needs to be done to support the implementation of Skills for Life strategy to 2010

• What has been achieved and what is still to be done?

• How should CPD be prioritised in the coming period in the context of teacher education reforms?

• How can the programme move to adopt a more strategic approach?

• Expert seminar

• Skills for Life Quality Initiative managers

• RQIMs • Provider

managers • Teachers • Teacher-

educators

Produce guidelines on assessing impact and change in provider practice

• What guidance can be produced to enable providers to conduct self-assessments of the impact of the Skills for Life Quality Initiative?

• Guidance manual

Generate proposals for action for post March 2006

• What recommendations and proposals for the future should be made?

• Based on data gathered from methods identified above

- 63 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

ANNEX 3 Skills for Life Quality Initiative online survey

Have you been involved with and would like to comment on the Skills for Life Quality Initiative? This is a survey of participants about their perception of the Skills for Life Quality Initiative. The survey forms part of the evaluation of the Skills for Life Quality Initiative being conducted by NRDC for the LSC. If you would like to find out more about this survey, you can call Jenny Wedgbury at NRDC on 020 7612 6717

Please return all completed questionnaires to the following address: FAO Jenny Wedgbury NRDC, Room 709 Institute of Education University of London 20 Bedford Way London WC1H 0AL Respondent category

Q1 Which of the following

best describes you?

FE college teacher

FE college manager

Trainer in WBL organisation

WBL organisation manager

Tutor in adult and community learning provider organisation

ACL manager

Member of a national agency e.g. LSC/Ofsted/LSDA

Teacher –trainer

Other (please specify) Other

The Skills for Life Quality Initiative Q2 How much would you say

you know about the

Skills for Life

Know a great deal about it

Know a fair amount

Know just a little

- 64 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

Quality Initiative? Have heard of it, but know nothing about

it

Have never heard of it

best

for LInitiative?

Q3 Which of the following

describes your main

connection with the Skills

ife Quality

Participant in Skillelopment activity

s for Life Quality Initiative professional dev

User of on-line development materials

Member of a regional network

Used tlitator

he services of a Skills for Life Quality Initiative cifa

Other (please specify) Other

Perceptions of impact Q4 In your experience please indicate, in regard to each of the statements below, your view of the Skills for Life Quality Initiative.a

t

Init

infraha

) The impact of delivery of he Skills for Life Quality

iative via the LSC regional

structure s been

Very stantial sub

Significant

Some impact

Little impact

No impact

Don’t know

o

level rein a

diff

b) The consequences

f targeting specific needs

at regional sulting

varying offer in

erent regions has

been

Very stantial sub

Significant

Some impact

Little impact

No impact

Don’t know

forInit

the lasyears has

been

c) The impact of the Skills Life Quality

iative over t few

Very stantial sub

Significant

Some impact

Little impact

- 65 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

No impact

Don’t know

Q5 In your experience of the Skills for Life Quality Initiative, which of the following have you e?

a) Web-site

found useful, or of little us

Very useful

Of some use

Neither useful nor not useful

Of little use

Not useful

Not seen/attended b) Online

training elements/

modules

Very useful

Of some use

Neither useful nor not useful

Of little use

Not useful

Not seen/attended c)

Downloadable resources

Very useful

Of some use

Neither usefful

ul nor not use

Of little use

Not useful

Not seen/attended d) Longer

accredited courses

Very useful

Of some use

Neither useful nor not useful

Of little use

Not useful

Not seen/attended

- 66 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

e

ac

) One-day non-

credited courses

Very useful

Of some use

Neither useful nor not ful use

Of little use

Not useful

Not seen/attended f

Life Quality Initiative

facilitators

) Skills for Very useful

Of some use

Neither useful nor not useful

Of little use

Not useful

Not seen/attended

Networks g) Regional Very useful

Of some use

Neither useful nor not ful use

Of little use

Not useful

Not seen/attended h) Good

practice - Cluster groups

Very useful

Of some use

Neither useful nful

or not use

Of little use

Not useful

Not seen/attended Other

comments

Q6 ent or disagree that the Skills for Life Quality Initiative h

a) Been effective in promoting

To what ext do you agree as

Strongly agree

Tend to agree

- 67 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

organisational change? Neither useful nor not

useful

Tend to disagree

Strongly disagree

Don’t know b) Been well planned and

delivered?

Strongly agree

Tend to agree

Neither useful nor not useful

Tend to disagree

Strongly disagree

Don’t know c) helped improve

teaching and learning?

Strongly agree

Tend to agree

Neither useful nful

or not use

Tend to disagree

Strongly disagree

Don’t know d) led to

higher levels

acof learner

hievement?

Strongly agree

Tend to agree

Neither useful nor not useful

Tend to disagree

Strongly disagree

Don’t know e) promo

whole ted a

oa

i

rganisation pproach to

quality mprovement?

Strongly agree

Tend to agree

Neither useful nor not ful use

Tend to disagree

Strongly disagree

Don’t know

- 68 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

f)

professional development

activity 1?

helped improve the

quality of regional

Strongly agree

Tend to agree

Neither useful nor not useful

Tend to disagree

Strongly disagree

Don't know Overall

Q7 Overall, how much

benefit do you feel the Skills

for Life Quality Initiative has

for those teaching,

training teachers for,

and managing Skills for Life?

Of great benefit

Significant

Not very much

None at all

Don’t know

(Please keep responses to under 500 characters)

- 69 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

Q8 Please add any

comments you would like to make about the Skills for Life Quality

Initiative which will

assist in this evaluation

(please keep responses to

under 500 characters)

- 70 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

NB. Adding too much text to the text fields (more than 500 characters) can cause your submission to fail. If you have any technical problems, please contact k.sheard@ioe.ac.uk

- 71 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

ANNEX 4 CfBT led consortium’s progress towards targets set under the Skills for Life Quality Initiative contract: A report to the NRDC

CONTENTS

1.0 INTRODUCTION

2.0 THE CONSORTIUM'S PERFORMANCE TARGETS FOR 2004-05

2.1 Overall achievement of the consortium's targets

3.0 ACHIEVEMENT OF THE CONSORTIUM TARGETS ON A REGIONAL BASIS AND SECTORAL BASIS

3.1 Regional and Sectoral Impact of Provider consultations 3.1.1 Regional Impact 3.1.2 Sectoral impact

3.2 Accredited training 3.2.1 Regional Impact

4.0 CFBT'S PROGRESS TOWARDS SKILLS FOR LIFE QUALITY INITIATIVE PRIORITIES FOR 2005-06

4.1 Activities being undertaken by the Consortium in 2005-06 4.1.1 Skills for Life Whole Organisational Approach 4.1.2 Embedding Skills for Life 4.1.3 Leadership and Management Skills 4.1.4 Facilitator Network 4.1.5 E- Learning 4.1.6 Key Skills 4.1.7 Continuing Professional Development/Information, Advice and

Guidance 4.1.8 Dissemination of good practice

4.2 Consortium's progress under 'Element 2 Training' activities (excludes the Facilitator Network strand)

4.2.1 Regional Impact

4.3 Consortium's progress under the 'Facilitator Network Strand' 4.3.1 Sectoral Impact

5.0 CONCLUSION

5.1 Achievement towards the 2004-05 targets

5.2 Progress towards priorities set out in 2005-06

- 72 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

Table of figures Figure 1 CFBT's achievement against 2004-05 targets Figure 2 Provider consultations Figure 3 Accredited training Figure 4 Accredited training delivered across the nine regions Figure 5 Element 2 Training - Professional Development Training Figure 6 Provider consultations

Table of tables

Table 1 Achievement of performance targets set for CfBT – 2004-05 Table 2 Providers involved in consultations in 2004-05 Table 3 Providers involved in consultations in 2005-06

- 73 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

Introduction The Skills for Life Quality Initiative was launched by the Learning and Skills Council (LSC) in September 2003 as part of the Government’s national strategy for improving adult literacy, numeracy and language (ESOL). The two main aims of this initiative are to:

• develop a professional Skills for Life workforce; and • support organisations to move towards a whole organisational

approach to Skills for Life. Planned activities under this initiative are based on the following agreed national priorities:

• developing a whole organisational approach to Skills for Life and supporting the planning and implementation of this within organisations;

• developing a regional facilitation network; • providing training in Leadership and Management skills specifically

related to Skills for Life; • supporting the embedding of Skills for Life for learners in all contexts; • raising the awareness of the relationship between Key Skills and Skills

for Life; • supporting the development of a Continuous Professional

Development(CPD) framework; • developing a regional Specialist Skills for Life Information Advice and

Guidance; and • disseminating good practice including the maintenance of the Skills for

Life Quality Initiative websites. The Centre for British Teachers (CfBT) is the contractor and lead consortium member of the LSC’s Skills for Life Quality Initiative. It works with a diverse range of partners in developing programmes which address each of the above priorities. The rest of this report focuses on the achievement of the consortium towards targets set for them by the LSC in 2004-05 and its progress on priorities outlined for them in 2005-06.

- 74 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

The consortium's performance targets for 2004-05 In 2004-05, the CfBT consortium was given a contract valued at 12.2 million to deliver eleven targets under the Skills for Life Quality Initiative programme. A variation of the contract was then made during the latter part of the programme which revised two of the targets and added an additional £5 million for the consortium to deliver three extra targets. The variation of contract meant that in the year 2004-05, the CfBT consortium was required to:

• train 1,000 Skills for Life practitioners in Key Skills; • train 300 teacher trainers; • develop a database of teacher trainers; • produce 25 professional development modules at level 4; • deliver short non-accredited training courses to 3,000 practitioners; • research into the development of a Level 7 qualification; • deliver monthly newsletters from September 2004; • ensure that 1,000 practitioners received online or paper based

guidance about Accreditation of Prior Learning(APL); • develop a website which would achieve 10,000 hits per month; • deliver 9,000 provider consultations; • train 2,000 leaders and managers; • deliver accredited training at Levels 2,3 and 4 for an estimated 1,700

practitioners; and • deliver non- accredited training to an additional 5,600 practitioners.

The following section outlines the achievement of the consortium in the delivery of the above targets. Overall achievement of the consortium's targets A detailed review of the LSC’s Skills for Life Quality Initiative 2004-05 report suggests that twelve out of the thirteen targets were either achieved and or exceeded as outlined in Table 1.

Table 1 Achievement of performance targets set for CfBT – 2004-05 Targets set for CfBT Achieved

1. 1,000 SFL practitioners to receive training in Key Skills 1018

2. 300 teacher trainers to be trained 521

3. 1 database of teacher trainers to be developed 1

4. 25 professional development modules at Level 4 to be produced 37

5. 3,000 practitioners to receive short non-accredited training courses 8500(includes achievement of target 13)

6. the development of a Level 7 qualification to be researched Achieved

7. the delivery of monthly newsletters from September 2004 6

- 75 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

Targets set for CfBT Achieved

8. 1,000 practitioners to be able to receive online or paper-based guidance about Accreditation of Prior Learning(APL)

1470

9. The availability of a website achieving some 10,000 hits per month 30,000

10. 9,000 provider days to be achieved(revised) 9061

11. 2,000 leaders and managers to be trained(revised) 1706

12. Provide funding for an estimated additional 1,700 accredited training places at levels 2,3 and 4( additional)

3200

13. provide funding for an additional 5,600 non-accredited training places (linked to 5 – additional target)

Linked to 5

Source: Learning and Skills Council (LSC) Skills for Life Quality Initiative 2004-05 report The most successful outcome of the consortium's 2004-05 programme in terms of volume was the national website developed for this initiative. This achieved 30,000 hits per month compared to a target of 10,000, representing a three fold increase over and above the required target. The target of delivery training to 2,000 leaders and managers was however not achieved as shown in Table 1. Reasons cited in the LSC’s Skills for Life Quality Initiative report state that the target had not been reached for a combination of reasons. This includes the late start resulting from extended regional consultation and difficulties in obtaining accurate provider details from some local LSCs. This meant that there were delays in the identification of potential providers and their delegates for the Leadership and Management training. The figure below highlights the achievement of the targets set for the consortium in 2004-05.

- 76 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

Figure 1 CFBT's achievement against 2004-05 targets

0 5000 10000 15000 20000 25000 30000

Achievement

Key Skills Training

Teachers

Database

Development Modules

Non accredited training

APL

Website hits

Provider Days

Leaders and managers

Accredited trainingTa

rget

AchievedTarget

Achieved 1018 521 1 37 8500 1470 30,000 9061 1706 3200

Target 1000 300 1 25 8600 1000 10,000 9000 2000 1700

Key Skills Training Teachers Database Development

Modules

Non accredited

trainingAPL Website hits Provider Days Leaders and

managersAccredited

training

Note: Targets 5 and 13 in Table 1 have been combined in this figure

Note: Excludes targets 6 and 7

Additional outcomes achieved by the consortium include:

• a new leadership and management module developed specifically for senior managers and delivered in collaboration with the Centre for Excellence in Leadership (CEL);

• 2,550 providers provided with new Skills for Life strategic action plans; • the revision of 16 Professional Development Modules produced last

year; • the development of a new introductory course and a more advanced

module to provide a more appropriate training route for vocational teachers;

• the development of a guidance document to support organisations in building capacity for mentoring and coaching within their workforce;

• the development of a toolkit providing a flexible resource for integrating Access for All in the workplace;

• the development of nine regional development plans to support the implementation of regional training delivery infrastructures; and

• the provision of assistance to regions in the development of nine regional websites.

- 77 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

The following section provides detailed analysis of the achievement of the consortium's targets on a regional and sectoral basis. It is worth noting that not all the targets had information available on this basis. Achievement of the consortium targets on a regional basis and sectoral basis This section assesses the regional and sectoral impact of the Skills for Life initiative for two key targets; delivering provider consultations and accredited training at level 2, 3 and 4 to Skills for Life practitioners. The information used in this section has been sourced from the LSC's report titled Skills for Life Quality Initiative 2004-2005 Comparison of the Nine Regional Reports'. Regional and Sectoral Impact of Provider consultations As outlined in section 2, the consortium was required to deliver 9,000 consultations to providers across the nine regions in England with a regional allocation of 1,000 consultations per region. The consortium achieved a total of 9,061 consultations with an average of 1007 consultations per region.

• Regional Impact Further analysis of the information provided by the CfBT suggests that in a third of these regions, the allocation of 1000 consultations per region was exceeded. These were mainly in the:

• East Midlands in which 1,427 consultations were held representing a 43% increase over and above the allocated target;

• Northwest with 1,308 consultations representing an increase of 31%; and

• London with 1,278 consultations representing an increase of 28%. The figure below outlines the provider consultations achieved by the consortium across the nine regions.

- 78 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

Figure 2 Provider consultations

0 200 400 600 800 1000 1200 1400 1600

Regional target

E

EM

L

NE

NW

SE

SW

WM

YH

Provider consultations 1000 835 1427 1278 876 1308 975 645 874 843

Regional target E EM L NE NW SE SW WM YH

Sectoral impact During 2004 -05, over 2,500 providers across nineteen sectors received consultations. Over half were from the following sectors:

• Work-based learning providers not involved in Entry to Employment(E2E) schemes or Employer Training Pilots(ETPs) constituted just under a third(30%) of all the providers;

• 14% were Further Education(FE) providers; and • 12% were Adult Community Learning (ACL) providers.

The Table below outlines the breakdown of these consultations across the nineteen sectors.

- 79 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

Table 2 Providers involved in consultations in 2004-05

TOTAL E EM L NE NW SE SW WM YH AVG Adult & Community Learning (ACL) 337 34 47 64 33 34 42 34 27 22 37.44 Armed Forces (AF) 9 1 0 0 1 0 0 6 0 1 1 Agencies & Institutions (CRO) 0 0 0 0 0 0 0 0 0 0 0 Entry to Employment (E2E) 82 9 1 17 14 18 1 7 3 12 9.11 Emergency Services (ES) 2 0 0 0 1 0 1 0 0 0 0.22 Employer Training Pilots (ETP) 226 0 11 1 95 1 16 2 65 35 25.11 Further Education (FE) 389 34 29 46 15 68 61 44 59 33 43.22 Prisons (HMP) 112 18 19 9 9 13 7 10 12 15 12.44 Information, Advice & Guidance (IAG) 45 6 3 10 4 8 1 5 7 1 5 Jobcentre Plus (JCP) 104 11 6 4 7 17 48 6 3 2 11.56 Learning Difficulties & Disabilities (LDD) 58 9 8 3 7 12 4 3 3 9 6.44 Libraries (LIB) 12 7 1 0 0 1 0 2 1 0 1.33 National Health Service (NHS) 40 15 6 3 3 2 1 4 3 3 4.44 Probation (PRO) 71 10 7 5 12 8 6 16 1 6 7.89 University for Industry/Learndirect (UFI) 145 7 68 7 15 8 3 5 2 30 16.11 Unknown (UNK) 0 0 0 0 0 0 0 0 0 0 0 Voluntary & Community (V&C) 307 46 18 48 31 49 12 47 25 31 34.11 Work Based Learning not E2Es or ETPs (WB) 855 79 124 74 40 143 94 106 103 92 95 Youth Offending Institutes (YOI) 34 1 0 1 5 2 19 1 4 1 3.78 TOTAL 2828 287 348 292 292 384 316 298 318 293 314

Table 2 above also shows that the consortium consulted with an average of 300 providers which suggests that the number consulted across the nine regions were fairly evenly spread. Over a third (37%) of the providers came from the:

• the Northwest with14% of the share of providers; • East Midlands with 12%; and • South East with 11%.

However, the proportion of providers from the different sectors within each region varied greatly:

• almost all the providers from the Armed Forces (over two thirds, 67%) were from the South West region;

• only two providers were from the Emergency Services and these came from the North East and the South East regions;

• 42% of the ETPs providers were from the North East region; and • just under half (46%) of the Job Centre Plus providers came from the

South East. This suggests that the consortium was not as successful in engaging providers across different sectors within each region.

- 80 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

Accredited training As part of the variation of contract, the consortium was required to provide additional training to 1,700 practitioners at levels 2, 3 and 4. This target was exceeded with a final figure of around 3,200 practitioners receiving accredited training. Further analysis of the information provided by the consortium suggests that a significant proportion of this training was provided at Level 3 and above; just over half (59%) of the training was provided at this level. The figure below outlines the proportion of accredited training by qualification level.

Figure 3 Accredited training by qualification level

Level 2 Unit 122%

Level 2 Full Certificate19%

Level 330%

Level 429%

Regional Impact On a regional basis, the consortium achieved an average of 354 courses across the nine regions. Over half (51%) of the training was organised in the following regions:

• approximately 20%(603) of the training courses was organised in London;

• 19%(594) in the East Midlands; and • 13 %( 429) in the North West region.

The chart below outlines the number of training courses organised across the nine regions.

- 81 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

Figure 4 Accredited training delivered across the nine regions

0 100 200 300 400 500 600 700

E

EM

L

NE

NW

SE

SW

WM

YH

Training Places 117 594 603 278 429 261 317 370 218

E EM L NE NW SE SW WM YH

Around 5,500 additional central and local events were also organised by the nine regions under the Skills for Life Quality Initiative. The majority (83%) of these events were delivered locally.

- 82 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

CfBT's progress towards Skills for Life Quality Initiative priorities for 2005-06 This section charts the progress to date of the consortium against the eight national priorities of Skills for Life Quality Initiative:

• Skills for Life Whole Organisation Approach • Embedding Skills for Life • Leadership and Management for Skills for Life • Facilitator Network • E-learning • Key Skills • Continuing Professional Development / Information, Advice and

Guidance • Dissemination of good practice

It focuses on activities such as training and facilitation events planned by the consortium under each of the above and its progress to date. The information used in this section have been sourced from two main documents; the Learning and Skills Council's Skills for Life Quality Initiative e-bulletin for 2005-06 and the Skills for Life Quality Initiative 2005-06 National Preliminary Final Report, 29th March 2006. Activities being undertaken by the Consortium in 2005-06 Over 480 events classified as professional development training activities relating to seven of the ten priorities; excludes the Facilitator Network strand have been planned by the consortium in 2005-06.These training events cover a range of professional courses such as train the trainer events, e-learning and leadership and management courses For the facilitator network strand, the consortium is required to deliver just over 4,000 consultations to providers. The following sections outline the planned activities under each of the national priorities.

• Skills for Life Whole Organisational Approach The whole organisation approach encompassed the complete initiative. Activities under this strand focus on training providers and partners on how to adopt an organisational approach to Skills for Life. Various activities contributing to the approach, but which are also priorities in their own right include leadership and management courses, a facilitator network, staff development, e-learning and advice and guidance systems. There were 9 events planned for all the regions under this strand. Work is also in progress to develop a toolkit or guidance document which will provide practical and guidance solutions for organisations from a wide range of settings.

- 83 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

• Embedding Skills for Life

A range of training events (156) and support materials have been developed for both teachers and managers to support this strand. These include train the trainer events and Skills for Life courses for vocational teachers.

• Leadership and Management Skills The leadership and management programme offers a variety of programmes including two day face-face programmes which allow participants to examine their leadership qualities. It also offers half day leadership and management sessions and one day seminars for senior managers. Approximately 86 courses have been planned across the nine regions under this strand.

• Facilitator Network 1479 organisations across the nine regions have been nominated by the consortium for provider consultations. The consortium plans to achieve a target of just over 4,000 regional consultations in 2005-06.

• E- Learning The consortium is currently rolling out 100 E-learning programmes across the nine regions. These include various courses such as:

• one day courses for practitioners to develop their competence and confidence to use technology in their day to day teaching;

• in house - modular training programmes for staff leading to accreditation; and

• one day courses that courses that prepare teachers to integrate e- learning into professional programmes.

• Key Skills

Awareness training sessions for practitioners are being rolled out across the regions for Key Skills practitioners under this strand. 38 events have been planned in these regions and include events that offer awareness training sessions to Skills for Life Quality Initiative facilitators to enable them to support organisations in developing strategic approaches.

• Continuing Professional Development/Information, Advice and Guidance

36 events have been planned in this area across the regions for experienced Skills for Life teachers who want to move into teacher training, observation and mentoring. Practitioners and managers from a broad spectrum are also attracted to these events.

- 84 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

Planned courses include:

• two twelve week courses for teachers and educators, accredited at Masters level

• four day residential courses; • coaching and mentoring courses, accredited and non- accredited; • one day courses for observers and assessors; and • the provision of regional expert services to advice teachers on

professional development options.

• Dissemination of good practice A series of events to disseminate good practice have been planned in the southern regions. Around 14 events have been planned under this strand and will feature presenters from organisations recognised as excellent in Skills for Life areas. Consortium's progress on training activities (excludes the Facilitator Network strand) At the time of preparation of this report, the consortium, between October 2005 and 28 March 2006 predicted it will have provided training to just over 5,000 delegates, approximately 61% of its planned target of 8,458 delegates. Currently the consortium has provided training to just over 4,700 delegates, thus achieving 56% of its planned target. The strands with highest proportion of training completed are those relating to priority two, Embedding Skills for Life, priority seven CPD/IAG and priority eight Dissemination of Good Practice. Those areas with the lowest number of completed delegates are priority one, Whole Organisational Approach (WOA) and priority six Key Skills. WOA training suffered as a result of changes occurring at LSCs and training events were cancelled at the request of some regions.

• Regional Impact In four out of the nine regions, the consortium has completed over 70% of its training activities. In the West Midlands, the figure is slightly higher with 93% of the training activities completed. The uptake of training in the Southeast region has however been quite low; only a third of the planned training in that region has been taken up.

- 85 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

Figure 5 Training - Professional Development Training

0 1000 2000 3000 4000 5000 6000 7000 8000 9000

Delegates

TOTAL

EE

EM

LO

NE

NW

SE

SW

WM

YHR

egio

ns

Number of completed delegates 4705 282 605.5 491.5 485 544.5 777 595.5 314 610.5

Planned delegates on revised plan 8458 379 1250 646 783 750 2320 1013 339 978

TOTAL EE EM LO NE NW SE SW WM YH

Consortium's progress under the 'Facilitator Network Strand' The consortium has been successful in exceeding its target of delivering 4,123 provider consultations across the nine regions 4406 provider consultations had been delivered by 15th March 2006. In regions such as the East Midlands, London, Yorkshire and Humberside and the South East, the consortium has been particularly successful. Consultations planned for these regions have exceeded the target. The degree of success however varies across the nine regions, for example in the North East region the number of provider consultations has been less than 50% of target. Figure 6 below charts the progress of the consortium under this strand.

- 86 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

Figure 6 Provider consultations

0 500 1000 1500 2000 2500 3000 3500 4000 4500

Provider consultations

TOTAL

EE

EM

LO

NE

NW

SE

SW

WM

YH

Reg

ions

Number of provider consultations completed 4406 158 827 835 199 706 500 180 414 587

Facilitator time requested in regional plan 4123 180 550 600 500 750 355 198 490 500

TOTAL EE EM LO NE NW SE SW WM YH

• Sectoral Impact

Just over 1,300 providers across eighteen sectors in 2005-06 were involved in consultations. Over half (54%) came from the following three key sectors:

• Further Education (FE) providers accounted for 19% of the total number of providers;

• Workbased Learning providers not involved in E2Es or ETPs accounted for 20%; and

• Providers from the Voluntary sector accounted for 15%.

- 87 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

Table 3 Providers involved in consultations in 2005-06 Providers TOTAL EE EM LO NE NW SE SW WM YH % Adult & Community Learning (ACL) 155 11 12 28 13 28 23 11 10 19 11% Entry to Employment (E2E) 57 1 12 8 5 14 0 2 4 11 4% Armed Forces (AF) 2 1 1 0 0 0 0 0 0 0 0% Emergency Services (ES) 1 0 0 0 0 1 0 0 0 0 0% Employer Training Pilots (ETP) 2 0 1 0 0 1 0 0 0 0 0% Further Education (FE) 263 4 21 39 19 56 42 23 30 29 19% Prisons (HMP) 69 5 16 8 4 12 6 4 3 11 5% Information, Advice & Guidance (IAG) 6 0 1 0 0 1 0 0 2 2 0% Jobcentre Plus (JCP) 33 0 4 4 1 6 9 0 4 5 2% Learning Difficulties & Disabilities (LDD) 31 2 6 3 2 3 6 0 4 5 2% Libraries (LIB) 42 4 0 33 0 1 0 0 2 2 3% National Health Service (NHS) 19 7 1 1 1 6 0 0 2 1 1% Probation (PRO) 31 1 9 6 0 5 5 0 1 4 2% University for Industry/Learndirect (UFI) 33 0 6 10 8 5 2 2 0 0 2% Voluntary & Community (V&C) 208 4 87 24 6 45 2 3 9 28 15% Work Based Learning not E2Es or ETPs (WBL) 267 3 44 36 6 77 13 24 14 50 20% Youth Offending Institutes (YOI) 15 0 0 1 1 1 10 1 0 1 1% Other/TBC (TBC) 131 4 54 28 4 10 5 4 8 14 10% TOTAL 1365 47 275 229 70 272 123 74 93 182 N/A

Conclusion This section summarises the achievements of the consortium in 2004-05 and its progress towards priorities set for them in 2005-06. Achievement towards the 2004-05 targets The data suggests that the CfBT led consortium Skills for Life Quality Initiative programme for 2004-05 was successful. Almost all the targets that the consortium was required to deliver in that year have either been achieved and or exceeded. The most successful outcome of the 2004-05 programme in terms of volume was the national website developed for this initiative which achieved 30,000 hits per month against a target of 10,000. However one area in which the consortium did not achieve its target was is in relation to the provision of training to Skills for Life leaders and managers. Under this target, the consortium delivered training to just over 1,700 practitioners against a target of 2,000. This was said to be due to difficulties in obtaining accurate provider details from the LSC which led to delays in the identification of providers and their delegates. Additional outcomes achieved by the consortium in that year which did not count as part of the target but nevertheless had an impact on the Skills for Life Quality Initiative programme include the:

• development of a new management module; • provision of new Skills for Life strategic action plans; and • development of nine regional plans to support the implementation of

nine regional training delivery infrastructures and

- 88 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc

Overall, the consortium's 2004-05 Skills for Life Quality Initiative programme was successful in ensuring that all the required deliverables were achieved. Progress towards priorities set out in 2005-06 The consortium's progress towards priorities set out for it in 2005-06 has been more variable. Under training activities covering seven out of the ten national priorities, the consortium between October 2005 and March 2006 has delivered training to just over 5000 delegates (approximately 61%) out of a planned target of 8,458. The facilitator network activities organised by the consortium however has been very successful, exceeding its target of 4123 provider consultations by achieving 4406 consultations. This varied success may be due to the two factors; one that the sector has either reached a ceiling in terms of the number of people who require Element 2 Training or that the short space of time required by the consortium to deliver in the current year has not allowed enough time to set strategies in place to recruit practitioners for training in this area.

- 89 - SfLQI 2005-06 Final Evaluation Report FIN 31May06.doc