External Evaluation Report - Development of the LLL Concept at … External Evaluation 12... ·...

External Evaluation Report

Dr Lisa Cowey Independent Assessor

October 2012

DELLO: Development of the Lifelong Learning Concept at the University of Montenegro

Table of content

Foreword .................................................................................................................................................... 2

1.0 Introduction to the external assessment of the DeLLCo Project ........................................................ 3

1.1. Methodology of the assessment ........................................................................................................ 3

1.1.1. On-site appraisal ............................................................................................................................. 3

1.1.2. Individual reflective survey ............................................................................................................. 3

1.1.3. Additional evaluation activities ....................................................................................................... 3

2.0 Introduction to DeLLCo ....................................................................................................................... 4

2.1. Objectives of the DeLLCo Project ........................................................................................................ 4

2.2. Goals of the project ............................................................................................................................ 4

2.3. Project Methodology .......................................................................................................................... 5

2.3.1. Development of legislative framework for the introduction of LLL concept .................................. 5

2.3.2. Improvement of cooperation with enterprises in view of LLL concept and university-enterprise

cooperation (UM) ........................................................................................................................................... 5

2.3.3. Improvement of learning opportunities for adults ......................................................................... 5

2.3.1. Project partners .............................................................................................................................. 6

3.0 Independent evaluation: Attainment of the aims and goals of the Project ....................................... 6

3.1. WP1 Developing legislative framework for the introduction of LLL concept ..................................... 6

3.2. WP2 Improvement of cooperation with enterprises in view of LLL concept and University

enterprise cooperation ................................................................................................................................... 8

3.3. WP3 Establishment of connections between stakeholders to improve vertical and horizontal

access 9

3.4. WP4 Procedures for validation of non-formal and informal learning .............................................. 11

3.5. WP5 Dissemination ........................................................................................................................... 12

3.6. WP6 Project sustainability ................................................................................................................ 13

3.7. WP7 Management ............................................................................................................................ 14

3.8. WP8 Quality control and monitoring ................................................................................................ 15

4.0 Conclusions and recommendations: aims and goals ........................................................................ 16

5.0 Independent evaluation – impact and outcomes ............................................................................. 17

5.1. A. TEMPUS DeLLCo for you ............................................................................................................... 17

5.2. B. YOU FOR DeLLCo ........................................................................................................................... 18

5.3. C. DeLLCo FOR YOUR ORGANISATION .............................................................................................. 19

5.4. D. DeLLCo FOR THE WIDER NETWORK ............................................................................................. 20

5.5. E DeLLCo OUTCOMES ........................................................................................................................ 21


1

5.6. F. DeLLCo EXPERIENCE ...................................................................................................................... 23

5.7. G. LIFE AFTER DeLLCo ........................................................................................................................ 23

5.8. H DeLLCo AND THE FUTURE .............................................................................................................. 24

5.9. I. FINALLY........................................................................................................................................... 25

6.0 Conclusions and recommendations: impact and outcomes ............................................................. 26

Annex 1 TEMPUS DELLCO EXTERNAL EVALUATION QUESTIONNAIRE ......................................... 27

Annex 2 LOGICAL FRAMEWORK MATRIX – LFM ............................................................................... 31


2

Foreword

This external evaluating report has been produced at the end of the TEMPUS project Development of Lifelong Learning Concept at the University of Montenegro (DeLLCo). This was part of the Structural Measures Sub-Programme under the Action for Higher Education and Society. The original anticipated that an extension will be obtained until March 2013. The extension was requested to enable more time for implementation of pilot modules, (this was a proposal from the EU as the original project proposal did not envisaged starting the pilot courses implementation within project lifetime) and also the production of a promotional film and the final conference. However, this evaluation was completed in line with the associated work package (WP8) by 14th October 2012. The report reflects only the views of the author and the Commission cannot be held responsible for any use that might be may be made of the information contained in it.

Dr Lisa Lisa Cowey MBA PGCert IP October 2012


3

1.0 Introduction to the external assessment of the DeLLCo Project

The external evaluation of the TEMPUS DeLLCo project was undertaken as part of Work Package

8. Quality control and monitoring. This work package foresees three main actions:

8.1 Monitoring of project progress

8.2 External evaluation

8.3 Internal evaluation

This report contributes to the external evaluation activity.

This part of the assessment was conducted by an independent assessor with strong coordination

with the University of Montenegro (Grant holder) and the University of Alicante (WP8 Leader). In

addition, the independent consultant had positive and strong exchanges with the other EU

project partners and national social partners, coordinated through the University of Montenegro

1.1. Methodology of the assessment

The assessment was composed of two main activities:

1.1.1. On-site appraisal

On-site appraisal of the project’s aims and results was undertaken as part of the project meeting

on the 24th September 2012 in Montenegro. This session lasted around 2 hours and consisted of

facilitated discussions focusing on each work-package in turn. It enabled the project partners to

explore the extent to which they had achieved the original aims and goals, how activities had

been implemented, identification of Good Practice, the wider value generated by their activities

and lessons learned that might be of benefit to the partnership but also to the wider community.

1.1.2. Individual reflective survey

A survey questionnaire was developed (see Annex 1) and distributed to all partners before the

meeting on the 24th September. Partners were asked to fill this in individual and return it to the

independent assessor by the end of September for analysis.

The survey instrument was based on that used by the Socrates-Leonardo project called Tulip

“Trade Union and University Lifelong Learning in Partnership” (2007-1984/001-002). The

questions used for external evaluation of TULIP were amended to reflect the different

partnership composition and goals of DeLLCo. The survey focused on the wider impact of the

project and the partnership, (outcomes rather than outputs) and sought to capture the

quantitative aspects of a project that are not always reflected in more quantitative evaluations,

(lessons learned and unexpected results).

1.1.3. Additional evaluation activities

In addition, the independent evaluator spent time studying the written project outputs and

reports including the project proposal, interim report, study visit reports Labour market needs

analysis - Employer Survey Focus group results, Comparative analysis of LLL legislation in EU


4

partner countries and the Report from the ‘Workshop Lifelong Learning at the University of

Montenegro’.

The original Logical Framework was also up-dated with the help of the partners to include more

verifiable indicators (Annex 2) and this provided part of the appraisal framework used on the a

24th September.

2.0 Introduction to DeLLCo

The broad aim of the project Development of the Lifelong Learning Concept at the

University of Montenegro is to improve learning opportunities in Montenegrin society at large

and increase employability. This is to be achieved through the development of university LLL

strategy.

Taking into consideration the diversified needs and fast changing requirements at the labour

market, the idea of creating a LLL strategy becomes a priority in the higher education reform

agenda, with wider impact on the whole of society.

Currently in Montenegro there are no flexible learning paths for learners outside the formal

education. As a result of that, qualifications, certificates and diplomas obtained out of the formal

system of education cannot be recognized; knowledge, skills and competences cannot be

validated at all. Furthermore, adult education exists, but it does not have a systematic approach.

2.1. Objectives of the DeLLCo Project

The specific objective of the project is the creation of a favourable environment for enabling

flexible and adaptable courses and modules so as to keep pace with the fast changing labour

market needs by defining an institutional LLL strategy, developing model courses and

modules adapted to the needs of various learners and teacher training for delivery of

courses, creation of better links with the world of labor, raising of awareness and

promoting of the idea of LLL in the society in general, matching of courses offered

and the real needs existing at the labour market.

The project aims to develop the Lifelong Learning Strategy for the University, thus enabling

introduction of the new concept. Taking into consideration the activities envisaged, and the role

that the University of Montenegro has in general, there are going to be clear reflections to the

society as a whole, working towards establishing firm partnerships with the labour market and

industry, raising awareness on broader level, introducing new legal framework, etc.

2.2. Goals of the project

The broad objectives of the project are:

to enable wider access to higher education to incorporate non-formal and informal education into the overarching national

qualification framework


5

to enhance partnerships with between the University of Montenegro and relevant social partners

It was expected that throughout, by the end, and after the project implementation the following specific objectives would be achieved:

legislative framework for the introduction of LLL concept developed cooperation with enterprises in view of LLL concept and university-enterprise cooperation

improved

learning opportunities for adults improved connections between stakeholders improved to improve vertical and horizontal access basis for introducing procedures for validation of non-formal and informal learning

introduced

comprehensive dissemination campaign on LLL concept in society at large implemented conditions for the exploitation of the project results immediately and after the project end

developed

day-to-day and overall management process secured quality plan implemented

2.3. Project Methodology

The project utilized the following methodology:

2.3.1. Development of legislative framework for the introduction of LLL concept

Training visits and comparative overview of LLL laws in EU partner countries were intended to

provide the necessary knowledge to the members of Montenegrin working team, thus enabling

development of LLL Strategy as the overarching document on LLL at the University of

Montenegro.

2.3.2. Improvement of cooperation with enterprises in view of LLL concept and university-enterprise cooperation (UM)

The existing cooperation was to be improved, adding a new aspect - the LLL concept. This

implied elaboration of an analysis of labour market needs, development of a database on the

current profiles available at the labour market against those lacking, creation of an ICT

supported system for keeping track of graduates employability and maintenance of a web portal

for efficient communication between UoM and companies.

2.3.3. Improvement of learning opportunities for adults

Introduction of appropriate measures and mechanisms provides for flexible learning paths for

adults thus enabling vertical and horizontal access within the education system at the UoM.

Development of pilot modules for adult learners was planned and implemented under the project

addressing the education of adults in two identified fields (tourism and practical physiotherapy).

A basis was created for introducing procedures for validation of non-formal and informal

learning.


6

2.3.1. Project partners

The DeLLCo project involves the following national partners:

University of Montenegro

Ministry of Education and Sports (MPIN)

Chamber of Economy (PKCG)

Employment Agency of Montenegro (ZZZCG)

Center for Vocational Education (CSO)

The national partners have been supported by the European partners of:

University of Aarhus – Denmark

University in Leuven - Belgium

University of Alicante – Spain

World University Service WUS - Austria

3.0 Independent evaluation: Attainment of the aims and goals of the Project

The assessment below was based on the external evaluation meeting on 24th September 2012 at the University of Montenegro. The meeting was moderated by Lisa Cowey (Independent External

Evaluator) and Ester Boldrini (University of Alicante).

3.1. WP1 Developing legislative framework for the introduction of LLL concept

Indicators of

achievement and

or/performance

as indicated in

the project

proposal

University strategy adopted

Change to the University Statute

Activities

1.1 Training visits for Montenegrin working team

1.2 Comparative overview of LLL laws in EU partner countries

1.3 Drafting of university strategy

Evaluation of the

WP

This activity and deliverables are close to completion

The official WP leader was Leuven but the leadership was shared with

UoMontenegro. WUS provided extremely strong and beneficial steering.


The strategy for this activity was to aim for a big, diverse and consistent

team in the study visits. Representatives from all partners joined the visits.

Permanent staff were involved including lawyers, career advisors, people


7

responsible for future dissemination, vice-rectors, Rector, IT sector,

teachers. A corrective meeting was held in Copenhagen (see below) and

an external expert in strategic planning was engaged; this individual had

previously involved in strategic planning of research HEI—Cambridge UK.

The last draft of the strategy was successfully revised.

Added value: Development of competencies was strongly demonstrated

through the drafting and revision. Not only has knowledge been transferred

and adsorbed it can be seen to have been applied. The Chamber of

Commerce has been seen to be very active. It was commented several

times that the application of understanding has been widened and

deepened.


The comparable overview was made by WUS and comments were then

made to further improve the document. The improvement centered on the

provision of equal information from each country. The additional

information requested from each partner was made and the comparative

analysis was received.


The Quality Assurance (QA) activity was also started and contributions

received. A public consultation was held and this did struggle to generate

real interest. In June 2012 the final draft of the strategy was prepared

including the package of procedures, suggestions and guidelines for the HE

Council.

Discussion for the modules accreditation with the Ministry has been

completed and the strategy will be presented to the Senate in September.

It is anticipated that the new strategy will be adopted

successfully.

Added value: A great deal of know-how on LLL implementation has been

accrued and applied (see above) showing clear development of

competencies.

Great progress has been achieved for the University and wider benefit

observed including: overcoming initial scepticism; convincing people; raised

awareness; correct and efficient implementation of the action and strong

and activity involvement of the social partners.

Lessons Learnt

It was important to appreciate the limitations of existing laws.

Collaboration with the vocational training providers was invaluable.


8

Exploring vocational training was ‘eye opening’ for many at the University in

terms of learning more about this type of ‘education’ and blending of

models.

The project was ambitious – both in terms of goals and also time scale (2

years). The original idea had been to stop before the implementation.

Three years may have been a better duration but the pressure to finish in 2

years was positive as it kept the project in the fore-font of people’s

consciousness and continuing to full implementation revealed many aspects

that might otherwise have been overlooked e.g. the need to align pilot

module content with University strategy.

DeLLCo has been a valuable learning experience and a good basis

for developing and implementing future projects.

3.2. WP2 Improvement of cooperation with enterprises in view of LLL concept and University enterprise cooperation

Indicators of

achievement and

or/performance

as indicated in

the project

proposal

Analysis of labour market needs

Workshops on LLL concept

ICT supported system and web portal created

Activities

2.1 Elaboration of analysis of labour market needs

2.2. Organizing workshop on LLL concept involving all stakeholders

2.3 Creating of ICT supported system for keeping track of graduates

employability

2.4 Creation of web portal enabling efficient communication between

university and companies

Evaluation of the

WP

Labour market needs deliverable completed.

Bilateral agreement concluded with agencies to use the database. Financial

support for HUEI Career Development Centre. HEIs Webpage.

All workshops have been completed on schedule.

A decision was made to slightly change the approach and make the

information available on different locations rather than through a single

portal. This was seen to be an improvement on the original project proposal

by all concerned and originated as a result of discussions between the

different partners.


9

ICT supporting system is under development (with Employment Agency).

The portal for HEIs and company communication will be hosted by the

Quality Assurance (QA) centre.

Data the economy needs from universities & other material will be

developed by the Career Centres. This will be concluded within the week.

The activity will be completed within the lifetime of the project.

It is now intended to link strongly to alumni activities; these are still quite

under developed at UoM. The CONGRAD project focused on alumni and it

support system for the region QA & IT system for tracking graduates.

Close cooperation is necessary with the Employment Agency and formal

agreements have been signed.

3.3. WP3 Establishment of connections between stakeholders to improve vertical and horizontal access

Indicators of

achievement and

or/performance

as indicated in

the project

proposal

LLL modules and courses tailored to the statistics and conducted analysis

Improving learning opportunities for adults

Actions 3.1 Creation of flexible learning paths

3.2 Designing pilot module for one specific field evaluation

Evaluation of the

WP

Creation of 2 pilot courses: it was not so easy to involve the faculties and

employers. A focus group composed of employers was formed. But there

was a challenge in gathering together the people.

Guidelines for the module preparation came from WUS + University

Alicante. Training process was offered on how to create the modules.

“Good workshops”. But it was noted that here is a strong need to create

space to “enable people to work in their mother tongue” and not to stick to

the English of the project in such a workshop. “Language is very specific

and the results are improved in this way”.

Ultimately 2 very good and strong structures were developed.

A first draft of the modules took place, followed by revisions. The problem

was the preparation for accreditation as any other study programme. UoM

need to accredit the modules. The procedures for accreditation will be

accepted on the October Senate session. The accreditation must FOLLOW


10

the adoption of the accreditation procedures. This must be the order.

Flexible learning paths: these were added in the strategic documents

(defined and positioned). The UpM need to change their statute to be more

flexible. The Statute should be changed not only for LLL. Procedures have

been defined, but implementation not yet possible. A necessity is seen to

“modulise” the educational process. “Everything is presently too rigid”. This

is going to take time to implement but the structure will be in place.

Comment from the various Partners:

Aarhus: ‘It has been a process. There was a need to align the works

packages with the strategy. This was a bit new and very challenging’.

WUS: What is the long term strategy? This is still developing but the project

has contributed strongly to raising this question’.

WUS: An unexpected result was that change in education is being driven by

this sort of project. This was not anticipated when the project was

conceived.

All the partners agreed that this had been one of the most challenging WPs.

The challenge was compounded by the need to work in English on topics

(legal) where language needs to be very specific and English is not the

mother tongue. The previous experience of the EU partners’ was a more

flexible environment with more regulations allowing for more flexibility in

approach. E.g. regulations applying to both full time and part time students

being present at the beginning of the process.

In the long term the UoM sees a need for the somewhat rigid regulations to

be softened. This project fostered the modularization process. The project

came from the Vice Rectorate and so had influence at the HEI level. ‘This is

just the beginning of a longer process’, ‘It is also a society change. Just

10% of the HEI have been educated’.

This sort of activity ‘moves boundaries’. It is ‘changing society’. ‘Such a

project therefore has a wide impact’. DELLCO is the first experience in “HEI

reform” it is a national matter. There has been deep learning and high

impact.

This activity is also strategically very important as it has impact on

the EU accession process.


11

3.4. WP4 Procedures for validation of non-formal and informal learning

Indicators of achievement

and or/performance as

indicated in the project

proposal

Procedures for validation of non-formal and informal learning

defined

Actions

4.1 Study visits to EU partner institutions

4.2 Workshop on recognition mechanisms and methodologies

4.3 Preparation of guidelines for recognition of non-formal and informal learning

Evaluation of the WP

Two site visits to Copenhagen were carried out in the framework

of the project generating knowledge on the validation of

formal/informal learning. Materials have been translated, and

there has been high interest.

The focus has been on strategy and student recruitment –

recognition of non-formal learning (development of “recognition

mechanisms”).

The benefit was mutual (EU-PC) in terms of reflection on the

work to be done A lot of questions were raised on both sides –

“why do it this way?” This provides a strong challenge to the

status quo that was appreciated by the EU partners.

All the partners were involved and the same staff in order to

ensure the know-how could be transferred and applied. WP4

was strongly linked to WP3 and they deliberately involved same

individuals.

Awareness has been raised in the Council and Ministry for the

pilot courses implementation during the DeLLCo implementation.

It was noted that these were ambitious activities. The point was

made again that the highly ambitious nature of the project was a

good driver. It presented an opportunity to do a lot of (new)

things at once. “Quite unique”.

Many staff were involved and they were proactive. There was a

strong sense of commitment to outcomes and active

contribution. Strong professionalism. Activities very additive.


12

3.5. WP5 Dissemination




proposal

Developed promotional material

Dissemination campaign conducted

Actions

5.1 Development of printed promotional materials

5.2 Creation and maintenance of web site

5.3 TV promotional film

5.4 Round tables on the role of LLL in society

5.5 Media coverage

5.6 Inter-project coaching


Lead Partner UoM

The UoM deliberately waited until they has clear results before

they started the dissemination process. They did not want to

‘confuse the media’. They wanted to wait for ‘concrete outcomes’

and disseminate them. There has been regular media coverage

utilising ‘simple messages because there was not the existing

know-how in the society’. ‘It is hard to get a complex concept

across’. Articles have appears in magazines, for example

Magazine of Montenegro Econom, Speciailzed Magaizine for

Education .

A promotional film will be produced (with real students). They

have used internal communication tools: publication about

events on their websites, invitation emails. The EU project

partners are also publishing and dissemination via their own

web-sites and organizations so awareness of the activity goes

wider than the UoM and wider than the country.

The dissemination will be continued beyond the project lifetime.

Concrete results are needed for the attention of the media. For

this reason they are waiting for the Career Development Centre.

(inter project coaching). This will draw attention.

Good linkages have been established with other projects (added

value) e.g. TEMPUS INTERFACE (JP 511224-2010 ) and a

Serbian Tempus LLL.

Inter project coaching has occurred including with projects

Congrad and Information Literacy. WUS was also able to

engagement an expert. Examples of Good Practice will be


13

3.6. WP6 Project sustainability




proposal

6.1 Cooperation with the stakeholders established

6.1 Active participation of stakeholders in project activities achieved

6.2 LLL as part of overall education reform incorporated

6.3 Efficient procedures for incorporating modules into the system developed

Actions

6.1 Establishing cooperation between university, labour market and ministries

6.2 Institutional support of LLL including recognition of non-formal and informal education

6.3 Creation of efficient procedures for incorporating modules into the system

Good communication has been established between all the

national partners with regular (almost daily) interactions.

Multiple and diverse methods of communication are used

Official agreements have been signed (MoU) between the UoM

and both the Chamber of Commerce And Employment Agency.

These will foster activities and simplify procedures.

There has been a good and high quality cooperation; it will

continue (further projects are planned).

The adoption of the strategy by the HEI senate is a big

achievement and will contribute to the sustainability.

publicised at the Dubrovnik Tempus conference.

The project has also received publicity through the Annual

Report of the Employment Agency. It gains wider exposure

through the EU partners who regularly publish news in their

websites and disseminate though their own networks.

The project has also seen strong cooperation with private HEIS

in Montenegro as part of the EU funded Euraxess Researchers in

Motion projects for mobility. The private Universities are present

whenever they can and they will ultimately follow the lead and

adopt this system making the LLL a “national story”.

Full contractual requirements have been met on this WP

and strongly exceeded.


14

Companies are interested and want to participate. They are

following the process with interest. End user interest is therefore

established. This will also contribute to sustainability.

3.7. WP7 Management




proposal

Project meetings organized

WG established

Reports prepared

Day to day coordination activities carried out

Actions

7.1 Organization of kick off meeting

7.2 Coordination of the Working Group

7.3 Reporting to the EC

7.4 Financial reporting

7.5 Day to day management of the Working Group

7.6 Collection of reports on project progress from WP leaders

7.7 Organization of two coordination meetings


Working groups at the Rectorate Level have been of great

importance demonstrating the responsibility of the staff and a

high level of involvement. They have regular coordination

meetings. This is helpful as the University is composed of 22

legal entities scattered across the country.

The final conference will be at the project end (March 2013).

(presuming that extension is approved)

The working group has been kept well informed during the

project lifecycle.

Reporting requirements have been on a 6 monthly basis. The

report is not prepared on WP basis but this has not presented

any problems.

The intermediate report accepted. Coordination meting were

been reduced (KoM & FC) at the suggestion of EU and with the

agreement of all partners. Coordination meetings have been

integrated in all activities & visit.

Social partners and EU partners all spoke strongly and warmly of

how well the project has been managed, particularly the long

term planning. Very efficient. Very smooth. “Could not have

been better”.


15

3.8. WP8 Quality control and monitoring




proposal

Monitoring reports

Evaluation reports

Actions 8.1 Monitoring of project progress


8.3 Internal

The monitoring of the project has been efficient. Internal and

external evaluations are now being prepared.


16

4.0 Conclusions and recommendations: aims and goals

It is clear from the notes above and the discussion with project partners at the

University of September 24th that this has been a very successful project in terms of

achieving and exceeding its aims and goals.

The project was acknowledged to have been ambitions – both in scope and time frame.

However, this ambition is perceived to have been a positive driver and has been used to

good effect. The partners have recognised that the project would have even more

benefit it is included the implementation of the pilot modules. They have utilised the

project to realise this additional goal but very sensibly requested more time to enable

this to happen.

The original Log Frame did not include many Verifiable Indicators but their absence has

not prevented the partners from driving the project forwards toward its final goals.

The University and partners have kept clear sight of the strategic goal and balanced the

quantitative and qualitative aspects of the project. Support has been demonstrated from

the top of the University and reinforced by the Ministry and the critical strategic

partners. The additional importance of utilising these results in the EU accesion process

has been recognised and also the potential benefit to other HEIs in Montenegro and the

region. A clear precedent has been demonstrated but also a clear willingness to share

experience with others.

The project has been professionally, effectively and efficiently managed. Project

management experience has been developed and shared. This will provide a further

basis for implementing similar projects in the future.

Dissemination of projects results have been thoughtfully timed to gain maximum impact.

A good balance has been struck between the contractual requirements of all EU funded

projects and the special situation regarding the existing level of understanding of the

public target audience for the LLL project results.

The project has laid the foundation for long term sustainability. Relationships with the

social partners have been formalised through Memorandum of understanding and new

organisations have been established that will continue beyond the lifetime of the

project.. Support from the EU partners for further projects is clear and there is

commitment to the LLL concept from the University Rectorate and the Ministry.

It is strongly recommended that discussions continue on transferring the results and

knowledge to other institutes in Montenegro and the region so they can be utilised to

good effect by similar groups.


17

5.0 Independent evaluation – impact and outcomes

11 Questionnaires were returned by:

University of Montenegro (3)

Ministry of Education and Sports (MPIN)

Chamber of Economy (CEM/PKCG)

Employment Agency of Montenegro (ZZZCG)

Center for Vocational Education (CSO)

University of Aarhus – Denmark

University of Alicante – Spain

World University Service WUS – Austria

KU Leuven - Belgium

The main results of the survey are summarised below:

5.1. A. TEMPUS DeLLCo for you

The first set of questions was designed to explore the individual (personal and professional) motivations and expectations of the participants.

National motivations were diverse and ranged from those who had the freedom to decide on their participation, recognised the potential importance of the project and made a decision to take an “active role in achieving … very important goals of DeLLCo” to those who were “assigned”’ to the project or saw their participation as being decided by more senior people in their organisation and simply “part of the job”’. The motivations of the European Partners was unsurprisingly slightly different with emphasise being on “joining” the project with a desire to “share expertise and long-standing experience” however one university cited an existing lack of institutional links with UoM and saw this project as “a good opportunity to get to know the University of Montenegro”.

The professional benefit was clear to most of the participants in a range of ways including project management skills that seemed particularly valued by the national partners.

“It was the first job where I was responsible for monitoring everyday implementation and progression of project activities, even more so as this was the

first Tempus project coordinated by our University”

“I have learnt a lot about managing and functioning of TEMPUS projects, their structure and various aspects of working packages during the whole process.”

“This was an opportunity to further develop skills on EU project management, especially from the position of the overall project coordinator”.

However the very specific knowledge of this particular project was also well recognised.

“I have also got the precious knowledge about functioning of different educational institutions in partner countries, which gave me a broader picture on the LLL concept, as well as every other part of the educational system in those countries”.

“I have got additional knowledge of the functioning of LLL and became more aware of its real significance”….” LLL (in general and on HE level), recognition of prior

learning, flexible learning paths, etc.”


18

The degree of learning from each other was also recognised by the European Partners who stated that they had:

“learnt a lot from the other partners about their understanding and practise of lifelong learning”.

Wider benefit was also noted recognised including “the networking, the professional behaviour and commitment of all those involved”.

It was also noted that being involved in successful projects was an additional benefit with one

partner remaking with admirable honesty “one needs such projects from time to time ”

The degree to which partners have learned deeply and strongly about LLL was clear from their comments. The diversely of aspects related to LLL that had been observed and encountered through the project are made very clear from the comments below:

“I learnt how the introduction of the LLL concept has been realizing in EU partner countries, the problems they faced with, the manner in which these problems have been solved, the introducing procedure of LLL, establishing of LLL culture, the practical results of this concept, further steps for the purpose of advancing the concept itself”…. “It also provided me with great insights regarding adult education, and all opportunity such concept brings to the society, I also learned about untraditional education paths, which also brought new ideas for LLL development at the University”.

Wider learning and benefit was also reflected with participants noting that “besides learning about LLL…” they had also “learnt a lot about our national partners, what they do, how the organizations they come from function” and “a lot about international project management and financial management of Tempus projects”.

The EU partners clearly expected to learn less about a subject on which they brought strong expertise and experience but they too notes that they had learnt more about

“LLL in other countries, especially the Danish model” and that it had been “enriching to find out on other EU countries LLL systems”

When asked about the method of learning most participants focused strongly on the value of the study trips to partner institutions “where we had the opportunity to see the practice of and solutions partner institutions” but they also recognised that it had required “communication with the national and the EU partners”. One EU respondent summed it up simply: “Through active participation” and it was noted that “thanks to these forms of learning, we were able to get concise and important advice and information for our future work”.

5.2. B. YOU FOR DeLLCo

The second set of questions was designed to explore what the individuals felt they had been able to bring to the project including special expertise, existing networks or past projects and other ongoing projects.

In this area, the respondents, particularly the EU partners did look towards their specialist knowledge related to the core subject of the project (LLL). National partners recognised the value of existing knowledge about the higher education system in Montenegro and “knowledge of the legal legislation, especially of the national professional qualifications “ while the European Partners highlighted “expertise in curriculum development” and “Knowledge and experience with lifelong learning: recognition of prior learning and flexible learning and curriculum planning”.


19

However, it was encouraging that most participants also recognised that good projects require a diversity of skills including “experience in informing the various publics”, “the experience in project management” and “Organization of focus groups with employers”.

Participants were able to drawn in previous participation in previous TEMPUS projects and clearly recognised the value of both national and international networks including those that would assist in dissemination of the activities and those dedicated to LLL both inside and outside of the European Universities.

Existing TEMPUS projects, e.g. Enhancing the quality of distance learning at Western Balkan higher education institutions DL@WEB and also CONGRAD, Delfis and interface were clearly natural partners for DeLLCO but one participant remarked with perception that:

“The project itself is opening a completely new chapter in the University development. Having that in mind, it would be closer to truth to say that DeLLCo was

the project providing support to other initiatives”.

DELCCO is clearly seen as a project that can and will now support other anticipated projects e.g. TEMPUS projects: DEBUT-M.

5.3. C. DeLLCo FOR YOUR ORGANISATION

The next set of questions focused on how DeLLCo has been beneficial for the participant’s home organisations. These included considering the activities or features that were developed with the help of the DeLLCo project, how the findings were used internally, and the use of DeLLCo as a tool for staff development.

National partners were quick and generous to recognise that their organisations, and in particular the University had gained huge benefit from the project:

“Through Dellco we developed the entire framework for the offer of LLL courses. the LLL Strategy, the rules for the offer of LLL courses and we initiated the talk about the need to adapt our educational offer to the needs of the society we exist and live in”.

But again, the comments indicated that wider benefit was also well perceived:

“Dellco also helped us change the worldview of academic community of various aspects of educational system”…”First of all we became more aware of the

importance of proper and adequate prescribing of article of new Law on National Professional Qualifications related to LLL and in creation of Strategy of Development

of Higher Education in which one of strategic goals is development of LLL in accordance with the best practice”.

“Agreement on cooperation with the University of Montenegro, which allows for better and easier future cooperation between our institutions” and “better

established contacts with representatives of the Faculty”.

The national partners have used DeLLCo strongly as a tool for staff development and that this value will cascade down the organisations:

“New knowledge and experience gained from this project are good base for further staff development”.

The lasting legacy for such training was also recognised:

“I find particularly important the fact that trained people will form the base for LLL development once the project ends. That means that trained University

administration will contribute to effective implementation of LLL activities and ideas, even when University management changes”.


20

The answers in this section were particularly interesting to an external reader. The surveys all made it clear that organisations learning had been mutual and not limited to the national partners and that individuals had gained understanding and developed new professional skills and enriched their project management and interpersonal skills. However, the answers to this section of the questionnaire also indicated that the European Partners had perhaps expected to contribute rather than to directly benefit. In particular, participation in these projects is not seen as a tool for staff development by the more experienced Universities. In a number of cases the questions from the European side related to the benefit to their own organisation from project participation were answered with Not Applicable and Not Relevant although some did indicate that the results would not be “used directly” indicating that a wider benefit had been recognised. It could be that the European partner Universities should consider again the wider benefit of participating and deliberately use this as a tool for staff development, particularly in younger staff members who are likely to be hosting study visits and engaging but supporting their more experienced colleagues.

5.4. D. DeLLCo FOR THE WIDER NETWORK

This section explored how DeLLCo was of benefit to the wider network. In particular how other colleagues were involved into the project, how collages from partner institutions were involved, if DeLLCo had helped to strengthen exiting institutional links, how discussion was promoted and information disseminated.

The replies in this section indicated that both systematic and informal structure had been set up and utilised to the benefit of the project. The University has approach the issue in a very systematic way:

“The project management team was created on the central level, of the University Rectorate. Representatives of the University management (Rector, Vice-Rectors)

were involved, also the legal office and PR office. Later on, we involved the Quality Assurance Centre. In Year II of the project, starting to work on the concrete

modules, we also involved representatives of the faculties in charge of modules.

Broader academic public was informed and invited to project events through the project web-site and University web-site, and, also, though the University mailing

list, which is a comprehensive list of academic and administrative staff of the University”.

The involvement of the Quality Assurance Center and legal department was highlighted as having been systematic as was the involvement of the PR officer in dissemination activities. Regular meetings had been organised but cooperating amongst the different actors was also an important element of the networking.

Sometimes the importance of an organisation ensured that colleagues were motivated to engage:

“The Ministry is, by the nature of its responsibility, very important element in creation of procedure of the LLL concept , so our colleagues were very interested to actively participate and contribute, each from their own field of education for which

he or she is responsible”.

It was also possible to involve a wider network of colleagues and institutions through the study visits and the focus groups and workshops. However, some methods were also very simple “I reported about the project, posted it at our website”.

Colleagues in the partner institutions were not just passively informed however, they were also actively involved “we consulted them and used their advice in the planning and realization of all the activities”… “they played an advisory role helping us implement


21

the project activities. One partner (EUCEN) from the Interface-Tempus even came as an expert to the workshop in July 2012.

The national partners reported most benefit by strengthening concrete links with other organisations with the support of DeLLCo. In particular, the University of Montenegro highlighted that they had strengthened links with existing partners but also “established good cooperation with the Employment Agency and the Chamber of Commerce (formalized also through signing of Memoranda on Cooperation) and with the Vocational Education Centre”. The MoU is cited as having “already resulted in beneficial activities for society such as establishing of University Career Centre”.

The national partners also mentioned the continued or new links formed at EU level “that will hopefully be further utilized in new project proposals and other joint activities”. The benefit to the EU partners was perceived to be of less notable relevance with again some EU respondents indicating that this issue was ‘not relevant’ or ‘not applicable’. However, on Eu University stated that “This project has definitely broadened our own contacts and network” and cited examples of outcomes that had occurred as a eult of joining DeLLCo “several joint applications for projects have been submitted, we have received Erasmus Munduus Action 2 students from them and we are currently involved in a joint project on Inclusive Education. ...at the same time we got to know the other EU partners better in this project”.

Promotion of discussions and dissemination of information was thoughtful and strategic.

The University expressed a wish to:

“carefully select activities to be spread widely to the general public, with concrete results and stories that will make the public interested in LLL. We have to bear in

mind the fact that the society is highly unfamiliar with the concept, and that we need to make it as understandable as we can.”

A dissemination strategy has been developed involving promotion of courses and adoption of Strategy; the strategy was tailored to suit both TEMPUS and the actual situation in country. Dissemination among internal public and stakeholders, will continue to rely on various web presentations-University website, the International Relations Office website and the web presentations of the partner institutions, which “informed their publics on project activities”.

5.5. E DeLLCo OUTCOMES

Questions in this section explored the relative usefulness of the resources and outcomes developed by DeLLCo. Interestingly, most participants did not focus on hard outputs of documented resources but looked at the knowledge they had gained and how it might now be further applied:

“Most useful was our learning about LLL in Europe and our dialogue with social partners about the possibility to introduce it at UoM”.

They also recognised the value of the environment that had been created:

“Raising motivation and encouragement for the distance learning”

Others did see the value in the documents and procedures that had been created:

“The most useful outcome for the University and future efforts on LLL development is the LLL Strategy, the document that will create the solid ground for LLL introduction. Also, highly beneficial outcome is the introduction of pilot courses, which will provide

us with new concept of studying”.


22

But added:

“The stronger cooperation among relevant institutions in Montenegro is also very useful and important outcome”.

One of the European Partners noted that the difference between outputs and outcomes, remarking “I find all outcomes of this project useful, [but] the implementation will be critical”. This comment was echoes in those made about outputs and resources that were ‘less useful”

Few respondents wanted to find any of the resources not useful. But one person did venture the comment that value might take time to realise noting that “they are rather difficult to implement in the Montenegrin society. For example, these include highly flexible learning paths and LLL programmes that we are not capable of reaching at the moment”.

This comment is perhaps mitigated by one from a European Partner who commented: “at this stage all the activities and outcomes are very useful. Even if a lot must to be expected to change and adapt according to other finding at a longer –term and also according to the social and economical needs that are in constant change”.

There were also few suggestions for resources that might have been developed that were missing. Comments in this area were limited to a request for more of outputs that were valued e.g. “More examples of good practice”, “more pilot modules in order to have a broader basis for the future implementation of LLL measures”.

Finally, one person noted that what would have been the greatest missing resource would have been the lack of implementation of the Pilot courses. That this deficiency had been recognised and addressed was made clear, adding “For this reason an extension of the Project was requested”.

It is often interesting to investigate unexpected outcomes. Several of the respondents clearly felt that unexpected outcomes would be negative and would not have been possible on such a well organised and managed project. Other recognised that an unexpected result can be positive

“What we did not expect was development of such good relationship with the social partners and the opening of a Career Development Centre”.

This was a sentiment echoed by other national and EU partners:

“Such a grounded (deep) understanding of lifelong learning”

“That the project initiated a closer cooperation with partners outside the university and the possibility that the LLL strategy for the UoM might become the nationwide

LLL strategy”.

Others also saw that unexpected results can have added value:

“As repeatedly mentioned, there were added values to the project visible in the official establishment of inter-institutional cooperation with local partners and, of

course, establishment of the Career Development Centre. Also, the results of DeLLCo have become more visible on national level, through the involvement of the project coordinator in the negotiation process with the EU, for Montenegrin membership in the Union, where, according to initial meetings, LLL will be one of the crucial points

of negotiation”.


23

5.6. F. DeLLCo EXPERIENCE

This section of the questionnaire explored the methods and working practices that had been most useful or most novel methods in DeLLCo? It also asked if some methods and working practices had been less appreciated.

The over-whelming response was that the interactive approach that had been adopted was of huge benefit although it was not percieved to have been innovative.

“The methods used were not very innovative but I would stress that during the events we tried to use the interactive approach as much as possible in order to

enhance the learning experience”.

“Working practice in DeLLCo was common or better said very traditional, but at the other side gave me to be very productive in the acquisition and application of new

knowledge and experience”.

The need for interaction was also highlighted by one partner who remarked that the only example of a practice that has not been so useful had been group work that had “had to many questions to be a good discussion in the groups, it was more like answers to a questioner and not a discussion”.

Learning in “realistic conditions” e.g. the study visits and direct contact with the practice in the partner countries were also strongly highlighted as a very positive method. “Strong involvement of social partners/stakeholders and that it was always more or less the same group of people involved” was seen as a very good working practice.

5.7. G. LIFE AFTER DeLLCo

This section investigated if there were plans in place to continue the cooperation’s started under DeLLCO.

The national partners indicated that now only were they ready and willing to continue their partnership but that they had already formalised them through Memoranda of Understanding signed between the partners. They added that “the process of European integration and further reform of our Univeristy will also entail this” which added further emphasis to the activity. The benefit of the DeLLCO project was seen in future cooperation with one partner commenting

“external partners will be more respected and responsible in the creation of new programmes and modules”.

Concrete exchanges and activities are already identified including: “Statistical information exchange between the Institute and the UoM about the state of the labour market, labor supply, records of students and graduates, as well as future users of the program for lifelong learning based on the Portal under the Careers Centre and the logistical, supervisory group that is mentioned in the work center”.

The European partners suggested that the activities of DeLLCo might be continued “hopeful in new project which can support the implementing of the strategy of lifelong learning – continuing the network”. They also suggested that “the framework already established would help the activity to continue” and that ”a network with regional institution would be very useful.”

Some EU partners responded very thoughtfully and perhaps drawing on their own experience in this area, to the question of how could the DeLLCo project might continue its work, suggesting:

“Step by step during the course of the implementation of the LLL strategy; promotion of the pilot modules as good practice examples and start introducing more LLL


24

courses/modules etc. right away. Maintain the good cooperation with the social partners/stakeholders”

5.8. H DeLLCo AND THE FUTURE

The penultimate section of the questionnaire sought to identify Lessons learned, Good Practice and long lasting impact of the project. This was a section where all the organisations contributed and at some length. A selection of the comments have been summarized below.

One of the more important lessons learned from DeLLCo was felt to be that:

“for new ideas and concepts to be introduced the most difficult is to change the mindset of people and their way of thinking”

The need to change a traditional way of thinking was also noted by another partner who said:

“The most important lesson that I learned refers to the need of University’s adaptation to a changing society, which includes changing of worldviews affected by

closed and traditional systems”

The issue of mind-set recognised in the complexity of the introduction of LLL which:

“cannot be imagined without support of all relevant partners and experienced EU partners. It is typical joint mission process with precisely defined role of all involved

and responsible partners”

The need to involve all relevant partners and enable them to work more effectively was a point made by several people:

“the set up- involving the social partners and that in important discussion the participants can benefit when working in their own mother tongue”

Finally, the length of time needed to make changes in mindset and thinking was raised:

“That it cannot be for granted that there is a common/same understanding about the aim/content of a project and that it is important to reach this common

understanding at a certain point. e.g. in this project, even though LLL at the university was the topic from the very beginning, only after approx. one year people

really started to realise what the introduction of LLL at the UoM really means”.

Examples of Good Practice that could be transferred with benefit between organizations sighted the good way of working on the project several times:

“good Dellco practice of teamwork and efficient planning could bring benefits to the decision making process at the University” ….”the excellent project administration.“

Some examples of Good Practice were ‘reluctantly’ seen to be too advanced for transfer to local frameworks although not all:

“Many of the activities organised by our EU partners are of interest for potential implementation at our University. Reluctantly, one has to admit that some of them,

such as practices in recognition of prior learning and flexible programmes (offered by Aarhus) are still too early to consider. On the other hand, the way of organisation of

LLL Master (including the element of public-private partnership), then Third age university, organising system of continuing education on national level, are more

easily to be implemented already at this point.

And some, particularly the recognition of prior learning were seen as ready and ripe for transfer:

“Recognition of prior learning gained through work experience”…” the development of flexible learning programs and recognition of prior learning especially those

acquired through the experience of work”


25

The national and EU partners were united in their belief that DeLLCo will have lasting value

“I think Dellco is the beginning of a long lasting talk about the need to change the university and to adapt it to the environment if functions in. Though at the moment it may seem the changes are too far reaching and the staff will be reluctant to accept them, I think time will show the soundness of the ideas underlying the project”.

“Yes, it will. It contributed to the arising awareness of the importance of LLL and created new culture in that sense”.

However, some did see the need to continue to win the hearts and minds of the wider community but saw DeLLCo an important first step:

“Society as a whole will be responsible for lasting of LLL, it does not depend just on the University. LLL environment in Montenegro is yet to be constructed, Dellco was

the right beginning of that process”.

Others saw the likelihood that the LLL concept would not spread to the other Universities in country and in region:

“Experience of the project can be used for further work on the NQF that is in the form of law currently. LLL is one of the strategic orientations of the HE in the country, so, experience can easily be transferred to other HE institutions in the country”…” fully

to all potential users in the territory of Montenegro through a wide network of cooperation in all university units”.

A final work on the measureable effects and impact of the project is given to one of the EU partners who responded:

“Definitely! The awareness raised, the changed mindset of the people involved, the LLL strategy, the procedures, the amendment of the university statute, the influence

on the respective law, the pilot modules, the interest of the other 2 (private) universities in MNE in this strategy”

5.9. I. FINALLY

The survey concluded by asking participants what they had enjoyed most about the project.

It was notable that the responses in this section focused on how people had enjoyed working together, particularly in the working groups and learning from each other in an atmosphere of “unselfish help and great expertise”. Individuals expressed a hope that they would “succeed in our plans to continue working together through different initiatives”. The scope of the project and its success were also mentioned “I also enjoyed the whole process of implementing LLL activities, with the main idea of introducing something completely new and beneficial to the society”.

The reasons for this positive environment were all linked to the attitude of those involved - “a professional and pleasant group” …“the commitment and the professional attitude of all people involved in this project”.


26

6.0 Conclusions and recommendations: impact and outcomes

TEMPUS DeLLCo has produced huge impact on the present status of LLL in Montenegro.

The success may in part be attributed to the motivation of those joining the project who

were strongly aware of the strategic importance of the activity and committed to final

outcomes. It has also assembled the necessary mix of specialist knowledge and project

management skills to ensure the project not only had access to real knowledge, but was

able to adopt, adapt and refine in a well managed, efficient and effective project

environment.

Both individuals and their organisations have benefited from the project, some more

than they may have anticipated or been consciously aware. This is a notable point and it

is recommended that perhaps some partner organisations reassess their present

approach to using such projects for deliberate staff development.

DeLLCo has also been of benefit to the wide networks. These were particularly

recognised by the University of Montenegro. There are examples of deliberate and

systematised linkages and also of less planned but also valuable spill-over’s. This is

perhaps also an area where some partners might want to consider if they could try to

deliberately consider involving a wider circle of colleagues or organisations in some

activities of such projects.

The strong positive outcomes of DeLLCo are clear. These are both tangible and

intangible involving formal documents, procedures, knowledge and transferable

experiences. The positive outcomes reflect the highly positive experience of involvement

in the project, by all partners. It is also notable that this experience is not taken for

granted and was a welcome contrast with previous experience of some partners on

other projects.

The sustainability of the project outputs looks very positive – MoUs have been signed

and new organisations set up. Procedures are being formally adopted and new courses

implemented. The national partners are also realistic about the amount of work still

facing them and in particular the effort needed to change established mindset and

thinking. They have the support of the European Partners who have signalled their

willingness to help in the ‘step by step’ process and this will also contribute to the

sustainability.

Finally, people involved in this project have enjoyed working together and that

frequently signals professionalism, a wiliness to give and share and to respect the

contribution of all others. And while this assessment has focused on Dellco as a project it

is important to acknowledge that projects reflect people. The partners and individuals

involved in DeLLCo (Development of Lifelong Learning Concept at the University of

Montenegro) should all feel justifiably proud of themselves.


27

Annex 1 TEMPUS DELLCO EXTERNAL EVALUATION QUESTIONNAIRE

Lisa Cowey September 2012

Programme: TEMPUS

Sub-Programme Structural Measures Action: Higher Education and Society

Project Title Development of the Lifelong Learning Concept at the University of Montenegro Project Acronym DeLLCo


28

TEMPUS DeLLCo External Evaluation questionnaire

This questionnaire is designed to help the TEMPUS DeLLCo partners to understand how well their project has achieved its goals and objectives, and to try and judge the results, outcomes and added value of the project for both direct participants and the wider network of beneficiaries. Is also attempts to collect some Lessons Learned that can be factored in to future project planning and implementation. Identification of Good Practice is also of benefit. Please find a quiet period, turn you mobile phone off, resist the urge to check your emails, take your time and think about the answer to each question. Many thanks Lisa Cowey


29

Abut you

1) Your name

2) Your employer (your organization)

3) Your role in the organization (please don’t just give a job title. Explain what you are responsible for and

what you do)

Assessing the wider value and impact of the project.

A. TEMPUS DeLLCo for you

1. Why did you join the DeLLCo project?

2. How did you benefit from the DeLLCo project professionally?

3. What did you learn in the DeLLCo project?

4. How did you learn in the DeLLCo project?

B. YOU FOR DeLLCo

5. What elements of your special expertise did you use in the DeLLCo project?

6. Which existing networks or past projects did you make use of in the DeLLCo project?

7. Which other ongoing projects (in your institution, national or international) supported DeLLCo during

the project?

C. DeLLCo FOR YOUR ORGANISATION

8. What were the activities or features of your institution that you developed with the help of the DeLLCo project?

9. How have you used the findings of the DeLLCo project in your organization?

10. Was DeLLCo used as a tool for staff development?

D DeLLCo FOR THE WIDER NETWORK

11. How did you involve colleagues in your organization in the DeLLCo project?

12. How did you involve colleagues in the partner institutions?

13. What were the concrete links outside your organisarion that you strengthened with the support of

DeLLCo?

14. How did you promote discussion on employer-university cooperation?

15. How have you disseminated the products of the DeLLCo project?

E. DeLLCo OUTCOMES

16. Which of the DeLLCo resources and outcomes do you find most useful?

17. Which of the DeLLCo resources and outcomes do you not find so useful?

18. What would have been useful that was not produced?

19. Did DeLLCo produce some outcomes that you did not expect?

F. DeLLCo EXPERIENCE

20. What were the most useful and innovative methods and working practices in DeLLCo?


30

21. Which methods and practices did you not find so useful?

G. LIFE AFTER DeLLCo

22. How do you plan to continue the cooperation between the universities and the external Montenegrin

partners after the DeLLCo project? (A question for the Partners from Montenegro)

23. How could the DeLLCo project continue its work? (A question for the European partners)

H DeLLCo AND THE FUTURE

24. Can you identify any important lessons leaned from DeLLCo? (These could be specifically about Life Long Learning activities or just about international EU funded projects in general)

25. Have you seen any activities or actions during the project that you feel could be transferred with benefit to your own organization (Good Practice Examples?)

26. Do you think that DeLLCo will have lasting value? Can you identify any indicators that suggest to you that the project has had a real impact on the LLL environment in Montenegro (or in other locations)

I. FINALLY

27. What have you enjoyed most about participating in DeLLCo?


31

Annex 2 LOGICAL FRAMEWORK MATRIX – LFM

Wider Objective:

What is the overall broader objective, to which

the project will contribute?

Development of favourable

environment for enabling higher

employability

Indicators of progress:

What are the key indicators related to the wider

objective?

Key competences and skills

developed in compliance with labour market needs

HOW INDICATORS WILL BE MEASURED: What are the sources of information on these indicators?

Feedback from the labour market –

enterprises and public institutions

Specific Project Objective/s:

What are the specific objectives, which the project

shall achieve?

Development of LLL university

strategy for enabling wider access to higher education

Database on the current profiles

available at the labour market against those lacking

Incorporation of non-formal and

informal education into the overarching

national qualification framework Enhance partnerships with relevant

social partners


What are the quantitative and qualitative

indicators showing whether and to what extent

the project’s specific objectives are achieved?

University strategy adopted

LLL modules and courses tailored to

the statistics and conducted analysis

Validation of non-formal and informal

learning ICT system as an interface between

labour market and university

established

How indicators will be measured:

What are the sources of information that exist

and can be collected? What are the methods

required to get this information?

Feedback from public discussions and

roundtables

Increased employability

Number of access to database

Number of courses

developed/modified/upgraded Number of demands from labour

market through database

Assumptions & risks:

What are the factors and conditions not under the direct control of the project, which are necessary to achieve these objectives? What risks have to be considered?

Lack of willingness among

academic staff and social

partners to accept LLL concept

Readiness of ministry of

education and other public

institution to accept the LLL concept

Non-acceptance of the

concept by university units


32

Outputs (tangible) and Outcomes

(intangible):

Please provide the list of concrete outputs/outcomes grouped in Workpackages, leading to the specific objective/s. Please list here the Workpackages (use bullet points):


What are the indicators to measure whether and to what extent the project achieves the envisaged results and effects?

How indicators will be measured:

What are the sources of information on these indicators?

Assumptions & risks:

What external factors and conditions must be

realised to obtain the expected outcomes and

results on schedule?

Unwillingness of university

authorities to accept the

concept of LLL Lack of willingness among

academic staff and

especially social partners to accept LLL concept and

disbelief of employers

Accreditation of the new

modules Non-acceptance of the

concept by university units

Readiness of ministry of

education and other public institution to accept the LLL

concept

Low level interest of all

stakeholders to get familiarized with the project,

its activities and results, which could result from not

enough knowledge on the theme. Possible low

participation of all local

stakeholders.

WP 1 Developing legislative

framework for the introduction of

LLL concept (Leuven)

Activities (key activities and sequence to be

carried out in order to produce the expected

results?

1.1 Training visits for Montenegrin

working team

1.2 Comparative overview of LLL laws

in EU partner countries


1.1 Visits to EU partners organized

1.1 Team for engaging in LLL issues

trained

1.2 Comparative overview drafted and

published

1.3 The University LLL Strategy drafted

1.3 The LLL Strategy adopted by the

University Senate

1.1Number of visits

1.1 Number of staff trained

1.1 Number of institutions represented

in the team

1.2 Comparative overview document

1.3. University LLL Strategy document

1.3 University Senate Decision on

Approval

WP2 Improvement of cooperation

with enterprises in view of LLL

concept and university-enterprise

cooperation (UM)

2.1 Elaboration of analysis of labour

market needs

2.2. Organizing workshops on LLL

2.1 Labor market analysis conducted

2.2 Workshops on LLL concept

organized

2.2 Number of people and institutions

represented at the workshops

2.1 Labor market analysis paper

2.1 Workshop reports

2.2 List of participants

2.3 Web portal


33

concept involving all stakeholders

2.3 Creating of ICT supported system for keeping track of graduates

employability



2.3 ICT supported system for tracking

gradute employability established

2.4 Web portal developed

2.4 Web portal

WP 3 Establishment of connections

between stakeholders to improve

vertical and horizontal access (UM)

3.1 Creation of flexible learning paths

3.2 Designing pilot module for one specific field

3.1 More flexible access to higher

education enabled

3.2 Pilot module designed

3.1 University Statute amendments

3.2 Pilot modules material

WP 4 Procedures for validation of

non-formal and informal learning

(Aarchus)

4.1 Study visits to EU partner

institutions

4.2 Workshop on recognition

mechanisms and methodologies

4.3 Preparation of guidelines for recognition of non-formal and informal

learning

4.1 Study visits conducted

4.1 People from the university and

stakeholders trained

4.2 Workshop organized

4.3 Guidelines drafted

4.3 Guidelines adopted

4.1 Number of study visits

4.1 Number of people from different

organizations trained

4.2 Workshop report

4.3 Guidelines

4.3 Senate Decision on adoption of

Guidelines

WP 5 Dissemination (UM)

5.1 Development of printed promotional

5.1 Printed promotional material

designed and developed

5.1 Promotional material


34

materials

5.2 Creation and maintainance of web

site


5.4 Round tables on the role of LLL in

society

5.5 Media coverage


5.2 Web site created

5.2 Web site regularly updated

5.3 TV promotional film developed

5.3 TV promotional film broadcast

5.4 Round tables organized

5.5 Events and results of the project

covered by the media

5.6 Relevant on-going initiatves

identified

5.6 Exchange of good practice with

other initiatives achieved

5.2 Web site

5.2 Number of web site entries


5.4 Number of programmes in which

the film was shown

5.4 Round talbe reports

5.5 Press clippings

5.6 Number of identified initiatives

5.6 Number of joint activities

WP 6 Exploitation of results (UM)

6.1 Establishing cooperation between

university, labour market and ministries

6.2 Institutional support of LLL including

recognition of non-formal and informal

education

6.3 Creation of efficient procedures for

incorporating modules into the system

6.1 Cooperation with the stakeholders

established

6.1 Active participation of stakeholders

in project activities achieved

6.2 LLL as part of overall education

reform incorporated

6.3 Efficient procedures for


developed

6.1 Number of institutions involved

6.1 Number of people included

6.2 Amendments proposed for the

University Senate

6.2 Amendments proposed for national

higher education legislation

6.3 Procedures


35

WP 7 Management (UM)





7.5 Day to day management of the

Working Group

7.6 Collection of reports on project

progress from WP leaders

7.7 Organization of two coordination

meetings

7.1 Kick-off meeting organized

7.2 Working Group established

7.2 All major issues regarding project

implementation considered and agreed

on by the Working Group

7.3 Timely reporting to the EC achieved

7.4 Rules for periodic reporting withing

the consortium developed

7.4 Timely reporting by all project

partners achieved

7.5 Regular communication among the

partners established

7.6 Reports on project progress from

WP leaders prepared

7.7 Coordination meetings organized

7.1 Meeting report

7.2 Decision on the establishment of

the Working group

7.3 Reports

7.4 Rules in the Partnership Agreement

7.5 E-mails, meetings, telephone calls

7.6 WP reports

7.7 Minutes from the meetings

WP 8 Quality plan (Alicante)




8.1 Project progress regularly monitored

8.2 External evaluation procedures

defined

8.2 External evaluation conducted


procedures defined

8.1 Project monitoring reports

8.2 Procedures

8.2 External evaluation report

8.3 Procedures

8.3 Internal evaluation report


36

8.3 Internal evaluation conducted

Activities:

What are the key activities to be carried out and

in what sequence in order to produce the

expected results?




2.1 Elaboration of analysis of labour market needs

2.2. Organizing workshops on LLL concept involving all stakeholders

2.3 Creating of ICT supported system

for keeping track of graduates employability



3.1 Creation of flexible learning paths 3.2 Designing pilot module for one

specific field

Inputs:

What inputs are required to implement these

activities, e.g. staff time, equipment, mobilities,

publications etc.?

1.1 35 mobilities – WB-EU 1.1 6 staff EU x 4 days

1.2 6 EU staff x 8 days 6 WB staff x 20 days

1.3 6 WB staff x 15 days 1.3 Printing

2.1 6 WB staff x 10 days 2.2 2 WB administrative staff x 5 days

5 EU staff x 7 days Travel costs for EU staff 4 x 5 days

ME-ME mobilities 26 x 5 days

Printing

2.3 Equipment, software 1 server

9 laptops

20 desktops Printing/copying machines and

scanners database

2.4 Printing cost

Staff cost (maitanance) 3 WB staff x 20

days

Assumptions, risks and pre-

conditions:

What pre-conditions are required before the

project starts? What conditions outside the

project’s direct control have to be present for the

implementation of the planned activities?


37

4.1 Study visits to EU partner

institutions

4.2 Workshop on recognition

mechanisms and methodologies

4.3 Preparation of guidelines for


learning

5.1 Development of printed promotional

materials

5.2 Creation and maintainance of web

site


5.4 Round tables on the role of LLL in

society

5.5 Media coverage


3.1 6 WB staff x 10 days 3.1. 4 EU staff x 10 days

3.2 2 EU staff x 10 days

3.2 3 WB staff x 15 days

4.1 20 study visits WB-EU

4.1 4 EU staff x 7 days 4.2 17 mobilities ME-ME x 4 days

4.2 9 mobilities EU-WB

4.2 3 EU staff x 7 days 4.2 Printing

4.3 2 WB staff x 10 days 4.3 2 EU staff x 7 days

5.1 Printing 5.1 1 WB staff x 5 days

5.2 Publishing 5.2 2 WB technical staff x 10 days

5.2 1 WB administrative staff x 5 days 5.3 Publishing

5.4 Printing

5.5 Publishing 5.6 Printing

5.6 4 mobilities EU-WB 5.6 4 mobilities WB-WB

6.1 Equipment 1 server

2 projectors 2 smartboards

2 beamers

20 desktops for the classroom 2 colour printers

1 network printer


38

6.1 Establishing cooperation between

university, labour market and ministries

6.2 Institutional support of LLL including


education

6.3 Creation of efficient procedures for






7.5 Day to day management of the

Working Group

7.6 Collection of reports on project

progress from WP leaders

7.7 Organization of two coordination

meetings









7.5 1 manger WB staff x 10 days

7.5 1 administrative WB staff x 15 days

7.5 3 manager EU staff x 7 days

7.5 3 administrative EU staff x 10 days


7.7 6 mobilities WB-EU

7.7 4 mobilities EU-EU

7.7 Printing




External Evaluation Report - Development of the LLL Concept at … External Evaluation 12... ·...

Documents

Transcript of External Evaluation Report - Development of the LLL Concept at … External Evaluation 12... ·...