Evaluation Sustainability

54
1 http://vimeo.com/12357663

description

Evaluation Sustainability presentation at the 2nd annual LLiSA conference

Transcript of Evaluation Sustainability

Page 1: Evaluation Sustainability

1

http://vimeo.com/12357663

Page 3: Evaluation Sustainability

Evaluation, Sustainability, Quality

Management

in Relation to the Context and Situation

Lessons learned from:

▶ Active and Healthy

Ageing in EU

▶ Lessons learned in

Japan, Taiwan and China

▶ Laurea LivingLabs – Unit

of Excellence: FAO/UN

initiative

▶ European Network of

LivingLabs – criteria &

PPPP initiative

3/1/2011 Laurea University of Applied Sciences 3

Page 4: Evaluation Sustainability

Platform Breaking Down Barriers Between Key Players at Regional Level

Laurea LivingLabs Network with the Helsinki - Uusimaa region as

an open innovation ecosystem based on human centric RDI

European Innovation Partnerships – meeting societal challenges and reinforcing

competitiveness

Simulation Seminar for ERAC regarding the pilot "Active and Healthy Ageing”

Ways of identifying and tackling obstacles to Innovation

22nd of February, 2011

Berlaymont building, room Schuman, 200 rue de la Loi, Brussels

Tuija Hirvikoski, PhDDirector| Laurea University of Applied Sciences| Internationalisation of RDI | ENoLL council member and co-ordinator of the thematic domains | Sendai-

Finland Wellbeing Centre steering group member |

| [email protected] | www.laurea.fi | http://www.openlivinglabs.eu/ | http://vimeo.com/16424693| http://sendai.fwbc.jp/en/index.html |

EUROPEAN COMMISSIONDIRECTORATE GENERAL FOR RESEARCH & INNOVATION

Directorate C - Research and Innovation

Unit C.1 – Innovation Policy

Page 5: Evaluation Sustainability

New Collaborative Innovation Paradigm, Why?

• due to the multi-layered,

multidisciplinary, multi-

sector, multi-stakeholder

complexities

• we need to improve the

effectiveness and impact of

policies, programmes and

projects

• apart from new

technologies and

products, we also need

new production and

consumptions that will

renew local and global

services, markets and

industries and we want

to effect human

behaviour

3/1/2011 Laurea University of Applied Sciences 5

Page 6: Evaluation Sustainability

New Collaborative Innovation Paradigm

How?

LivingLabs =

enlarged Triple Helix model & open

Research, Development and

Innovation (RDI) infrastructures and

methodologies

Learning and Co-Innovating for

Development

local, regional and cross-

boarder collaboration

addressing the grand

challenges

3/1/2011 Laurea University of Applied Sciences 6

Hirvikoski (2009)

Page 7: Evaluation Sustainability

Lessons learned from EU

Active and Healthy ageing

wanted

• system level thinking,

holistic and systemic

solutions

• capacity to orchestrate

and scale the

innovation up from the

local level to the

national/system level

because of

• fragmentation of the

market /non-existing

market

• overlapping RDI

• need to boost the

transformation of the

professionals‟, clients‟ and

citizens‟ behaviour

• to boost societal innovation

3/1/2011 Laurea University of Applied Sciences 7

Page 8: Evaluation Sustainability

Lessons Learned in Japan,

China and Taiwan

• Sendai-Finland

Wellbeing Centre

• from push marketing

towards open RDI across

Finnish &Japanese

stakeholders

• Active Ageing in

Shanghai

• systemic, holistic

solution across 16

Finnish companies and

Chinese authorities

• Cultivating Talent and

Transformation of the

Innovation Ecosystem

of Taiwan

• high tech

• people driven (Biking

and social media in a

Hakka community)

• integrating tradition and

high tech (Dharma

Drum)

3/1/2011 Laurea University of Applied Sciences 8

Page 9: Evaluation Sustainability

Kanter (1988) Innovation is most likely in

organizations that

(a) have integrative structures,

(b) emphasize diversity,

(c) have multiple structural linkages inside and outside the organization,

(d) have intersecting territories,

(e) have collective pride and faith in people‟s talent, and

(f) emphasize collaboration and teamwork.

Page 10: Evaluation Sustainability

Type of, radicalism of and maturity

of innovation, e.g.

TH2009

Page 11: Evaluation Sustainability

Cultivating Smart Living Talents in

Higher Education

International Forum on

Smart Living Summer School Initiatives in Asia

January 14, 2011

Taipei, Taiwan

Tuija Hirvikoski, PhD, Laurea University of Applied Sciences

European Network of Living Labs, co-coordinator of the thematic domain

Laurea with multiple Centre of

Excellence awards

in the Helsinki Metropolitan Area

Page 12: Evaluation Sustainability

Smart Living/Cities

Challenges and Solutions in the rapidly

changing world1. What is needed for smart living in smart cities?

2. What kind of an innovation process are we taking

about?

3. Abilities and skills needed? (Professionals and

students)

4. Challenges for higher education (and

other stakeholders....)

5. The future in the world without

borders

January 14, 2011, Taipei , Taiwan International Forum on Smart Living Summer School Initiatives in Asia Tuija Hirvikoski 12

Page 13: Evaluation Sustainability

3/1/2011 Laurea University of Applied Sciences 13

About the Smart

City Solutions

based on Industrial

University

collaboration

See theHelsinki Smart City

Showcase

http://vimeo.com/16424693

and

http://www.openlivinglabs.eu/

Page 14: Evaluation Sustainability

The current status and the future trend on smart

living education

ENoLL, European Network of LivingLabs▶ http://www.openlivinglabs.eu/

▶ http://www.openlivinglabs.eu/news/enoll-5th-wave

▶ http://vimeo.com/17515676

▶ http://www-sop.inria.fr/teams/axis/LLSS2010/ecoleLL/

Laurea Learning by Developing (LbD) model with the best employment rate.

FINHEEC: “the model can set an example for the whole Finland” ▶ The Finnish Higher Education Evaluation Council http://www.kka.fi/index.phtml?l=en&s=82

Laurea University of Applied Sciences International Advisory Board▶ http://www.laurea.fi/internet/en/03_information_on_Laurea/02_group/06_News/97_2010/International_experts_invited_as_Laurea_s_Honorary

_Research_Fellows/index.jsp

and compared to what was learned from the colleagues in ITRI (i236), III,

and The Talent Cultivation Program for Smart Living Industry

• INSIGHT Suan-Lien Elderly Centre (Prof. Lin and Prof. Kang) – Care home LL

• Touch Centre (Prof. Jeng and Prof. Yang) Showroom & future learning environment LL

• Eco-City (Prof. Lin) Biking and social media supporting economical, social and cultural

development in rural area

January 14, 2011, Taipei , Taiwan International Forum on Smart Living Summer School Initiatives in Asia Tuija Hirvikoski 14

Page 16: Evaluation Sustainability

Laurea’s Operating Environment

• the Greater Helsinki Metropolitan Area produces

approximately 50% of Finland‟s gross domestic

product

• Uusimaa region consists of urban and rural areas• The social and health care sector is strongly influenced by demographic changes and

struggling with the challenge of ensuring equal services for rural and urban areas with

limited budgetary conditions. The ageing population, long distances and the possible

lack of qualified work force are common challenges in particularly in the archipelago

• In its operating environment, Laurea is specializing

in service innovations and focusing on regional

development of the metropolitan area

3/1/2011 Laurea University of Applied Sciences16

Helsinki Smart City Showcase

http://vimeo.com/16424693

Page 17: Evaluation Sustainability

Laurea’s LivingLabs Network

and the partners in the Uusimaa Region

3/1/2011 Laurea University of Applied Sciences 17

Strategic Intent 2015:

In 2015 Laurea will be

an internationally

acknowledged

university of applied

sciences specializing

in future expertise

and regional

development in the

metropolitan area.

Active Life Village

Service Innovation and Design Lab

Care Innovation and Design Hub

User Driven Innovation Centre

Medical and Care Simulation Centre

The focus of the LivingLabs is on welfare,

knowledge intensive business services, security

and social responsibility. The basis for Laurea’s

R&D&I is a holistic view of well-being which

provides sustainable direction for businesses and

for the development of entire service systemsCKIR /Aalto

Forum Virium

OtaniemiMarketing

Culminatum

Hospital AreaHyri and TechVilla

Uusimaa Regional Council

HUS, THL

Page 18: Evaluation Sustainability

Political

guidelines

Institutional

enviromentDesign

environment

Developer

Execution

enviroment

User

Personal

meanings

Utilizer

Business

environment

Strategic

environment

EnablersNew Technologies, products, services,

New production and consumption

spin-offs,

User behavior transformation

renewal of markets, industries and societies

orchestration

table

Learning by Developing (LbD) with an open RDI Ecosystem =

Laurea LivingLabs

professional development

Page 19: Evaluation Sustainability

LbD = student-centric research and

development work integrated into

learning

• LbD is a system of learning and human development that

mobilizes and harnesses human creative talent en masse. It is a

learning system that integrates education with research,

development and innovation operations (RDI); consequently, it

fuels our collective creativity. In addition, it provides each

student the tools to develop and utilize his or her creative

talents fully in order to generate a livelihood for him or herself

and to foster productivity for society as a whole• http://www.laurea.fi/internet/en/031_quality/01/05_development/02_operational_development/05_benchm

arking_evaluation_audit/centre_of_excellence_evaluation_2009.jsp

La http://www.slideshare.net/tuihirv/laurea-living-labs

January 14, 2011, Taipei , Taiwan International Forum on Smart Living Summer School Initiatives in Asia Tuija Hirvikoski 19

Page 20: Evaluation Sustainability

Learning by Developing – a

Unique Way to Study

• The working life oriented learning method Learning by Developing is a pedagogical innovation developed by Laurea.

• The LbD -projects are conducted in partnership with the world of industry and commerce, which means that authentic workplace issues are selected as subjects of studying.

• For students LbD is a new way to get the competence needed in working life: they grow from learners into experts with excellent employment opportunities within their specific fields.

3/1/2011 Laurea University of Applied Sciences 20

Page 21: Evaluation Sustainability

Feedback on Laurea Centre of

Excellence (FINHEEC)

• The students„ participation in R&D has been raised to a new level,

so that they are now the main activating forces in the process

• supported by Laurea's management and it is purposefully developed

throughout the organisation

• a vision-based management model - incentivising for staff -

communities of experts based on the interests of the parties

involved

• a regional operator and partner in the metropolitan area - in

continuous interaction with the environment

• many of the articles and conference presentations have involved

students

• Laurea has determined the core competences included in all of its

degree programmes, taken into account the context of each degree

in curriculum development, and improved competence evaluation

procedures3/1/2011 21

Page 22: Evaluation Sustainability

Feedback on Centre of Excellence

• Laurea has defined its core competence in relation to the operating

environment's competence needs in terms of areas of focus, from which it has

also derived the core competences for each degree programme

• The Learning by Developing core process strives to bring together employers

and the workplace, employment-related research and development, and

learning. Student-centric R&D is the connecting link between Laurea's three

main tasks, when the students' projects are aimed at developing employment

in the local region.

• In feedback, students especially mentioned the possibility of doing the right

tasks in projects, and the appropriateness of their studies. Another strength

that was mentioned was the improvement of team work and interaction skills,

although imbalances in the degree of participation appear to be a challenge

for teachers to tackle in the active student community and the commitment-

demanding study model.

• One of the particular strengths of R&D activities at Laurea University of

Applied Sciences is the role of students as central actors and takers of

responsibility.3/1/2011 22

Page 23: Evaluation Sustainability

Feedback on Centre of Excellence

• Despite the strong emphasis on R&D, theory and practice are well balanced

in the education. Student-centric R&D is particularly well suited to

universities of applied sciences, and Laurea can set an example for the whole

of Finland.

• The generally high level of results indicates that student-centric R&D is a

good choice for pedagogy

• Laurea produces the highest number of ECTS credits from R&D. In addition,

the students' participation in publication, project preparation and even

project management activities proves that they are central actors in Laurea's

R&D operations.

• Other strengths are Laurea's open interaction with its operating environment,

its agility in responding to the needs of the environment, and the apparent

functionality of its management model.

3/1/2011 23

Page 24: Evaluation Sustainability

Shared leadership based on interaction

Leaders and professors provide

• an empowering vision and

meaningful storylines

• support and trust those with ideas

and an entrepreneurial anxiety and

capabilities to make the ideas to

work in reality

• create platforms and trust based

opportunities for collaboration and

co-creation

Crewmembers (staff and students)

• participate into the interactive

leaderships e.g. by forecasting,

proposing, carrying the

responsibilities

• are curious about opportunities and

search for solutions

• empower their peers and leaders

January 14, 2011,

Taipei , Taiwan

International Forum on Smart Living Summer

School Initiatives in AsiaTuija Hirvikoski 24

True transformations evolve organically: new innovations emerge, new systems of technology and infrastructure are put in place, and new patterns of

living and working gradually take shape and begin to remake the economic landscape.

Page 25: Evaluation Sustainability

New Collaborative Innovation Paradigm, How?

Cross-boarder collaboration

In the world without boarders, the diversity of cultures, technologies

and business models nurtures innovation and learning

New opportunities emerge whilst ideas, knowledge and innovation

travel from country to country

Page 26: Evaluation Sustainability

New Collaborative Innovation

Paradigm. How?European Network of Living Labs

(ENoLL) was founded in 2006 under

the auspices of the Finnish EU-

presidency

• contributes to the creation of a

dynamic, multi-layered and

multidimensional, future European

innovation ecosystem

• supports co-creative, human-centric

and user-driven research,

development and innovation in order

to better cater for people‟s needs

3/1/2011 Laurea University of Applied Sciences 26

Hirvikoski (2009)

All current 212 Living Labs listed on

www.openlivinglabs.eu/livinglabs

Page 27: Evaluation Sustainability

Average growth of turnover in Danish firms, 2005-2007 (Source: Fora & Statistics Denmark, 2010)

Page 28: Evaluation Sustainability

New Collaborative Innovation Paradigm

How?The long term vision on Public-

Private-People Partnerships (PPPP)

for user-driven open innovation

includes:

• Living Labs as Open Access

Platforms for Smart Cities and

Regions

• Alliances with Local Authorities to

integrate Living Lab Innovation and

Sustainability Policies

• Cross-border and Cross-Thematic

Cooperation addressing the grand

challenges of our time

• Large-scale User Behaviour

Transformation through Social and

Societal Innovation

3/1/2011 Laurea University of Applied Sciences 28

Europe as a dynamic, multi-layered and

multi-dimensional European Innovation

Ecosystem

Page 29: Evaluation Sustainability

Living Lab Partnerships

City- and Regional

Governments

Social Networks

Citizens Groups

Universities

Research Institutions

SMEs

ICT Industries

International Organizations

FAO, UN, World Bank, OECD

International Partnerships

with Living Labs in other Regions

European Union

Page 30: Evaluation Sustainability

3/1/2011 Laurea University of Applied Sciences 30

The ENoLL 5th Wave

of Membership

Jesse Marsh

5wave Coordinator

Page 31: Evaluation Sustainability

01-03-2011 31Ghent, 14 December 2010 31The ENoLL 5th Wave of Membership

A non-profit association established under Belgian law –

AISBL

General Assembly of Effective Members

Elected Council of max 21 Members

President, Vice-Presidents, Treasurers, Secretary

elected by the Council

Thematic Domain Networks, Working Groups, etc. open

to participation by all members

ENoLL: a Legal Entity

29.02.2011 LLiSA

Page 32: Evaluation Sustainability

01-03-2011 32Ghent, 14 December 2010 32The ENoLL 5th Wave of Membership

Adherent members

▶ Living Labs having passed an Wave evaluation

Effective members

▶ Adherent members having paid a membership fee

Associate members

▶ Non-LLs (eg University, public authority, etc.) paying the

membership fee

An administration services fee may be introduced as of

2011

Membership in ENoLL

29.02.2011 LLiSA

Page 33: Evaluation Sustainability

01-03-2011 33Ghent, 14 December 2010 33The ENoLL 5th Wave of Membership

Send an email to [email protected] stating

▶ Organisation name

▶ Web address

▶ Applicant country

▶ Contact person details

You will receive a unique registration number and the

official Membership Application Form

Pre-registration

29.02.2011 LLiSA

Page 34: Evaluation Sustainability

01-03-2011 34Ghent, 14 December 2010 34The ENoLL 5th Wave of Membership

Evaluation teams constituted from ENoLL members and

LLs

3 members per team from different countries

Proposals divided by team excluding represented

countries

Team Leaders collaborate to ensure common criteria

X-Team cross-evaluates to ensure coherence

Evaluation process

29.02.2011 LLiSA

Page 35: Evaluation Sustainability

01-03-2011 35Ghent, 14 December 2010 35The ENoLL 5th Wave of Membership

Max 8 pages, max 1 Mb pdf, no annexes

Send to [email protected]

Deadline: 15 March 2011, 24:00 hours CET

Submission constitutes:

▶ Expression of interest to join ENoLL AISBL

▶ Permission to ENoLL to hold data for evaluation purposes

▶ Permission to ENoLL to make information public unless

otherwise notified by 15 March 2011

Submission

29.02.2011 LLiSA

Page 36: Evaluation Sustainability

01-03-2011 36Ghent, 14 December 2010 36The ENoLL 5th Wave of Membership

Basic facts

Membership motivation

Description and characteristics

Organisation

Openness

Resources

Users and Reality

Value

Direction and Plans for the Future

Application Form

29.02.2011 LLiSA

Page 37: Evaluation Sustainability

01-03-2011 37Ghent, 14 December 2010 37The ENoLL 5th Wave of Membership

Proposals are then evaluated against 20 criteria grouped

into 5 categories

▶ Organisation

▶ Openness

▶ Resources

▶ Users & Reality

▶ Value

No targets, no quotas: proposals evaluated strictly on

merit

Evaluation Criteria

29.02.2011 LLiSA

Page 38: Evaluation Sustainability

Enoll- Evaluation Criteria

• Evidence of co-created values from research,

development and innovation

• Values/Services offered/provided to LL actors

• Measures to involve users

• Reality of usage contexts, where the LL runs its

operations

• User-centricity within the entire service process

Page 39: Evaluation Sustainability

Enoll- Evaluation Criteria

Full product lifecycle support - capability & maturity

LL covers several entities within value-chain(s)

Quality of user-driven innovation methods and tools

Availability of required technology and/or testbeds

Evidence of expertise gained from the LivingLab

operations

Commitment to open processes

IPR principles supporting capability and openness

Page 40: Evaluation Sustainability

Enoll- Evaluation Criteria

• Openness towards new partners & investors

• Business-citizens-government partnership –strength &

maturity

• Organization of LL governance, management & operations

• Business model for LL sustainability

• Interest and capacity to be active in EU Innovation system

• International networking experience and capability

• Channels (web etc) supporting public visibility and

interaction

• People/Positions dedicated to LL management& operations

Page 41: Evaluation Sustainability

01-03-2011 41Ghent, 14 December 2010 41The ENoLL 5th Wave of Membership

Evaluation results submitted to ENoLL Council for

validation and final decisions

Candidates informed of evaluation results at least 30

days prior to official announcement

Launch event scheduled 16 May in Budapest under the

auspices of the Hungarian Presidency

Selection and announcement

29.02.2011 LLiSA

Page 42: Evaluation Sustainability

01-03-2011 42Ghent, 14 December 2010 42The ENoLL 5th Wave of Membership

Pre-register via email to obtain the registration number

and Application Form

Submit the completed application by

15 March 2011

Complex yet transparent and accountable evaluation

process taking over one month

Official launch of successful new members at Hungarian

Presidency event in Budapest

Summary

29.02.2011 LLiSA

Page 43: Evaluation Sustainability

01-03-2011 43Ghent, 14 December 2010 43The ENoLL 5th Wave of Membership

ENoLL Portal at www.openlivinglabs.eu

▶ In-depth description of ENoLL: organisation, working

groups, membership, etc.

▶ Info and FAQ section on 5° Wave call

▶ For ENoLL members, access to dissemination kit to

promote the 5° wave

▶ For everyone, subscription to the ENoLL newsletter for

the latest updates

For further information

29.02.2011 LLiSA

Page 44: Evaluation Sustainability

01-03-2011 44Ghent, 14 December 2010 44The ENoLL 5th Wave of Membership

Jesse Marsh, 5° wave coordinator

Álvaro Oliveira and Mikael Börjeson, Task Force core

group

Pieter Balloon and Anna Kivilehto, secretarial support

The ENoLL evaluation team

[email protected]

Thank you for your attention

29.02.2011 LLiSA

Page 45: Evaluation Sustainability

- Reviewers download all (~18) applications assigned to his/hers team

from web (see above)

- Review team first checks the eligibility criteria for each application.

- Each reviewer reads and grades their applications (each appl graded

by three reviewers).

- It is recommended to use the review result template (principals from

Jesse) in 4WB sheet "Grade"

- The team compiles results from all three reviewers and fill in the

result in sheet "Appl"

- The entire workbook with filled in EligCriteria and Grades (5 Cat/appl)

is sent to me by 23 Feb.

- Please "channel" remarks/questions to me via your team leader.

PROPOSED PRINCIPAL PROCESS

for the evaluators

Page 46: Evaluation Sustainability

Apollon - Homecare and Independent Living

Experiences of SMEs

▶ Evaluates the different services in another country – forces to re-think the initial business case

▶ Provides a “lessons learned” in transferring or setting-up cross border Homecare and Independent products – better value proposition and better product fit

▶ provides opportunity to discuss the Health care eco-system in another country to identity different needs and to explore new business opportunities

▶ provides insights into the requirements and operational issues related to transferring products to other markets outside the base market

Innovating, Televic

Page 47: Evaluation Sustainability

Elderly Welfare Promotion Group, National Taiwan University INSIGHT

Suan-Lien Living Lab

Innovations of Long-Term Care Services for the Elderly in Chuncheon,

Hallym University, Korea

Connecting the Dots of Excellence across European and Asian Aging Societies

Page 48: Evaluation Sustainability

East Asian civicsocietal model

Northern European

welfare model

What is the role of the citizens, users, and people ?How does the public, private, people partnership evolve?

LivingLabs/Testbeds role?

Comparative research

City/region level

analysis

scaling up the results

to the system level

=> new products, services and solutions to be scaled up from the micro level to the system level and to the global market

Connecting the Dots of Excellence across

European and Asian Aging Societies

Page 49: Evaluation Sustainability

Pre-Commercial Procurement Good Practice

Procure R&D in steps (solutions, prototypes, test

series) to reduce the risk and give SMEs a

chance

Risk-benefit sharing with Suppliers

Competing development with multiple Suppliers

Sharing R&D costs with other Procurers

Page 50: Evaluation Sustainability

PPPP for Active and Healthy Aging –

evaluation of inputs, process, results,

impacts

Action Through a Policy

Mix

Improving Co-ordination

of Key Players

Europe as a dynamic,

multi-layered and

multi-dimensional

Innovation Ecosystem:

impacts

3/1/2011 Laurea University of Applied Sciences 50

Page 51: Evaluation Sustainability

Lessons Learned - Action Through a Policy Mix

- enabling health and social care policy, educational policy,

industrial and innovation policy (management by vision,

evaluation and incentives)

integration of education and RDI to foster motivated professionals

integration of bottom-up (demand and human driven, self-organising)

and top-down (scalable, transferable solutions) approaches

parallel incremental development and systemic paradigm sift of

active and healthy aging (care systems, health behaviour, societal

innovation etc.)

enhancing innovation in pre-commercial public purchasing processes

& sustainable urban innovations through multidisciplinary R&D

cooperation (as The Innovative City® Program)

simultaneous innovative procurement and support for growth

companies => business expansion, generates new job opportunities

3/1/2011 Laurea University of Applied Sciences 51

Page 52: Evaluation Sustainability

Lessons Learned - Improving the Coordination of

Key Players by PPPP

• enabling structures and platforms (LivingLabs) to address

the multi-stakeholder, multi-layer, multi-disciplinary, multi-

sector complexities => effectiveness and impact of policies

• proactive platform activators => trust, coherent value

bases, good collaboration among stakeholders => strategic

agility

• empower the elderly, citizens, care professionals and

students => innovative solutions & diffusion of innovation

• connect the dots of excellence across European and Asian

aging societies

Page 53: Evaluation Sustainability

Scaling up the PPPP model: Europe as a dynamic,

multi-layered and multi-dimensional Innovation

Ecosystem - Consequences

3/1/2011 Laurea University of Applied Sciences 53

- involves creative and motivated professionals and citizens

to the development of Active and Healthy Ageing solutions

and the transformation of behaviour

- creates innovative solutions from the micro to the most

macro level of the Active and Healthy Ageing systems

- decreases innovation obstacles

- boosts Europe as a lead market of Active and Healthy Aging

- boosts new spin-offs and supports growth companies

- gives companies an opportunity to grow in the emerging

Asian (global) markets

- generates meaningful new jobs

Page 54: Evaluation Sustainability

January 14, 2011, Taipei , Taiwan International Forum on Smart Living Summer School Initiatives in Asia Tuija Hirvikoski 54

Image Tuija

Thank you