CriCoRM Project WP3: EVALUATION OF THE PROJECT Evaluation report.pdf2nd Evaluation report 4 This...

D8

CriCoRM Project WP3: EVALUATION OF THE PROJECT

Deliverable D8:

2nd EVALUATION REPORT

2nd Evaluation report

1

1 INTRODUCTION 3

2 EVALUATION FORM 5

2.1 Internal evaluation 6

2.2 Status of the targeted achievements 7

2.3 WP1: Coordination of the project 8

2.4 WP2: Dissemination of the project 9

2.5 WP3: Evaluation of the project 10

2.6 WP4: Analysis of the communication processes 10

2.7 WP5: Guidelines development 11

3 EVALUATION OF PROJECT MANAGEMENT 12

3.1 Project management 13

3.2 Communication aspect 13

3.3 Coordination aspect 13

3.4 Timeliness 13

3.5 Questionnaire to the CriCoRM partners 14

3.6 Month 18 of the project 14

3.6.1 ASL BRESCIA 14

3.6.2 UNIBS 15

3.6.3 LMU 16

3.6.4 UMU 17

3.6.5 INOVA+ 18


3.7.1 ASL BRESCIA 19

3.7.2 UNIBS 20

3.7.3 LMU 21

3.7.4 UMU 22

3.7.5 INOVA+ 23


2

3.8 Questionnaire to the CriCoRM WP leaders 24

3.9 Month18 of the project 25

3.9.1 WP1 - ASL BRESCIA 25


3.9.3 WP3 - UNIBS 27

3.9.4 WP4 – LMU 27




3.10.3 WP3 – UNIBS 30

3.10.4 WP4 – LMU 31

3.10.5 WP5 – UMU 32

4 COMMENT 33

4.1 Project management 34

4.2 Work package progress 34

4.3 General comment 34

5 ANNEXES: Independent Peer Assessment reports 35


3

1. Introduction


4

This document is the second evaluation report that covers the second year of the

Crisis Communication in the area of Risk Management (CriCoRM) project. The

evaluation was made according to the methodology presented in the Evaluation Plan

approved by the consortium partners. The evaluation report confirms that the specific

objectives set for the second year have been reached in addition to the performance

indicators, and self-evaluation of individual work package (WP) leaders and partners.

Equally to the first year, the self-evaluation made by the partners and their comments

on the project represented an important contribution during the development of the

report both as a form of monitoring and as a means to suggest further improvements.

Data used in the evaluation were collected using different questionnaires and through

e-mails with partners. This report is made up of two parts: the Evaluation Form,

setting out the specific objectives, and the Evaluation of Project Management, giving

the results of all the yearly questionnaires regarding the management of the project.

Peer Review reports are also provided. These were produced by the independent

expert responsible for giving a professional analysis and assessment of the level of

content of the second year project deliverables and of their alignment with the Grant

Agreement (see Task 3.3).

A general overview and peer review of process and project achievements and interim

technical quality assessment (representing other activities envisaged within Task 3.3

of the Grant Agreement) are documented as a further attachment in the form of a

peer review report of the second year Project Interim Report. This is to ensure an

independent, external professional quality assessment and quality assurance for the

whole project, complementing and reviewing the evaluation exercise carried out and

documented within this deliverable and Evaluation Report.


5

2. Evaluation form


6

2.1 Internal evaluation

Coordination of the project WP1

Dissemination of the project WP2

Evaluation of the project WP3

Analysis of the communication processes WP4

Development of guidelines WP5

Implementation of Web 2.0 tools for crisis WP6

Training WP7

Table 1: Table of the project’s specific objectives

In order to guarantee that each partner develops the specific objectives correctly, we

have chosen a set of indicators by which to evaluate each WP and the extent to which

each specific objective was successfully achieved. Most of the indicators were selected

specifically for each objective. We have drawn up a specific table of performance

indicators to evaluate the success of each specific objective. Each WP leader was able

to set the minimum (acceptable) and maximum (target) criteria at the beginning of

the WP. The status of all the indicators was monitored during the WP and was checked

on its completion. The proposed indicators for all the WPs/specific objectives were

divided into subsections. The WP achievements (i.e. scores given using individual

performance indicators) are described in three columns: Acceptable, Target and

Results. The Acceptable column represents minimum acceptable achievements that

are sufficient for the purpose of the WP. Target achievements are set as realistic goals

that could be accomplished within a given work package. For the indicators used to

monitor development of the objectives, we proposed a 3-level quality scale for

measuring the achievement of the criteria goals. A traffic-light type scale was used

where the colours represent project development: green would mean the indicator is

good or normal, yellow would indicate a minor departure from the established plans

(such as a postponed deliverable, or work that needs some further agreed

modifications), and red would indicate a potential problem for the objective (such as a

deliverable that misses a postponed deadline or an unsatisfactory result).


7

2.2 Status of the targeted achievements

This task monitored how deliverables were produced, how timelines were met, and

how progress was achieved. Every 6 months, based on specific targets described in

the evaluation plan, an evaluation form was completed with data from partners’

progress reports and the completed questionnaires. An evaluation form was sent to

each WP leader as part of a monitoring process aimed at detecting potential

weaknesses in the project that would require timely reaction and improvement. This

included a section on performance of horizontal WPs, which involved all partners, and

another section on specific WPs, which involved each WP leader. The table shows the

achievements at month 24 of the CriCoRM project according to the objectives

established in the WP3 Evaluation Plan. In the Results column, we asked each WP

leader to fill in the results at month 24 of their WP for every target as follows:

“Achieved” if the target was reached;

“On time” if the target was still ongoing but was running to time and, if possible, to

estimate the percentage of the work completed;

“Late” if the target was still ongoing but there were potential problems in meeting

the deadline and, if possible, to estimate the percentage of the work completed;

“Not applicable” (n.a.) if the target was not expected to be reached.


8

2.3 WP1: Coordination of the project

Achievements

Month Acceptable Target Results

24 1. Achievement of milestones M1.3

and D4

2. Assurance that the other partners can achieve the set

milestones and deliverables: M2.4

and M3.3

3. Timely resolution of

possible conflicts between partners of

the consortium

4. Successful

organisation of 1 consortium meeting

5. Proper periodic

communication with

EAHC officer

1. Achievement of milestones M1.3 and D4

2. Assurance that the other partners can achieve the set milestones and

deliverables: M2.4 and M3.3

3. Timely resolution of

possible conflicts between partners of the consortium

4. Successful organization of

PMB3

5. Proper periodic

communication with EAHC

officer

Achieved

Achieved

N. A.

Achieved

Achieved


9

2.4 WP2: Dissemination of the project

Achievements


24 1. Achievement of

milestone M2.4

2. Completion of D1 second report

3. 300 leaflets issued

4. 4 posters issued

5. 1 articles in

newspapers and specialist magazines

6. Participation in 4

meetings,

conferences, trade fairs and field events

7. Website updates and

visitors

1. Achievement of

milestone M2.4

2. Completion of D1 second report

3. 500 leaflets issued

4. 5 posters issued

5. 2 articles in

newspapers and specialist magazines

6. Participation in 6

meetings,

conferences, trade fairs and field

events

7. Website updates

and visitors

Achieved

Achieved

Achieved (n. 2115)

Achieved (n. 70) Achieved (n. 8)

Achieved (n. 22)

Achieved

(Uploading of documents in some

sections of CriCoRM web sites; Creating

some sections of the CriCoRM web site in consortium

languages).


10

2.5 WP3: Evaluation of the project

2.6 WP4: Analysis of the communication processes

Achievements Month Acceptable Target Results

24 1. Achievement of

milestone M3.3

2. 2 evaluation reports

3. Majority of the

partners satisfied with the evaluation reports

4. Data from 4

questionnaires administered to each partner

5. 1 feedback form

administered to key stakeholders

6. 4 brief reports on questionnaires

1. All planned milestones

achieved

2. 2 evaluation reports

3. All of the partners satisfied

with the evaluation reports

4. Complete data from 4

questionnaires submitted to each partner

5. 1 feedback form submitted

to key stakeholders

6. 4 brief reports on questionnaires

Achieved

Achieved

Achieved

Achieved

Postponed

Achieved

Achievements


24 1. Completion of data

analysis and interpretation

1. Completion of data analysis

and interpretation

Achieved

This WP starts at month 1 and ends at month 19


11

2.7 WP5: Guidelines development

Achievements


24 1. Achievement of milestones M5.1 and

M 5.2

2. Completion of deliverable D5

3. Identification of at least 60%

stakeholders involved in the dissemination of the

guidelines

4. 1 report on guidelines feedback

1. All planned milestones achieved

2. All planned deliverables finalized

3. Identification of all stakeholders involved in

the dissemination of the guidelines

4. 2 reports on guidelines feedback

In time (50% achieved)




This WP starts at month 17 and ends at month 28


12

3. Evaluation of project management


13

3.1 Project management

Project management was evaluated using specific questionnaires on the following

communication and coordination aspects:

timeliness of deliverables (measured as the number of deliverables delivered on

time, with an agreed delay, and with an exceptional delay) and accessibility to the

drafts of deliverables (measured as the number of deliverables available as internal

drafts);

cooperation/communication between WP leaders and the project manager

(evaluated through a questionnaire at 6, 12, 18, 24, 30 and 36 months);

cooperation/communication between partners within WPs (evaluated through a

questionnaire at 6, 12, 18, 24, 30 and 36 months)

3.2 Communication aspect

Communication was evaluated using a questionnaire that involves all the consortium

partners and regards cooperation and communication between WP leaders and the

project manager and among all the WP partners. Answers were rated on a scale of 1

(worst) to 10 (best).

3.3 Coordination aspect

Coordination was evaluated using a questionnaire that involves all the consortium

partners and regards coordination between WP leaders and the project manager and

among all the WP partners. Answers were rated on a scale of 1 (worst) to 10 (best).

3.4 Timeliness

Timeliness was evaluated using a questionnaire that involves all the consortium

partners and regards coordination between WP leaders and the project manager and

among all the WP partners. Answers were rated on a scale of 1 (worst) to 10 (best).


14

3.5 Questionnaire to the CriCoRM partners

Each partner was asked to evaluate the WP leaders they worked with during the

second year. Their answers concerning specific targets, described in the evaluation

plan as Timeliness, Coordination and Communication are shown in the following

tables.

Data were collected until 9thJanuary 2014 and until 28thMay 2014.

Answers were rated on a scale from 1 (worst) to 10 (best) or “NA” (not applicable)

3.6 Month 18 of the project

3.6.1 ASL BRESCIA:

TIMELINESS

Question: WP Leader Rate (1-10)

If your WP leader set you a deadline, do you think you had enough time to prepare the answers?

WP3 10

If score is under 7, please specify the kind of problem experienced

COORDINATION


How capable was the WP leader of coordinating the project?

WP3 10



When you asked your leader any questions, how

available was he or she to help, and how would you rate the timeliness and clarity of the answers you

were given?

WP3 10

If score is under 7, please specify the kind of problem

experienced

Question: WP Rate (1-10)

How far did your experience as a WP partner meet your expectations?

WP3 10



15

COMMUNICATION


How good was the level of communication with your

WP leader?

WP3 10


experienced

3.6.2 UNIBS:

TIMELINESS



WP1 10


COORDINATION



WP1 10




available was he or she to help, and how would you rate the timeliness and clarity of the answers you were given?

WP1 10




WP1 10


Question: Rate (1-10)

As a partner of a consortium, how well did ASL Brescia coordinate the

project as CRiCoRM project manager?

10


COMMUNICATION



WP leader?

WP1 10



16

3.6.3 LMU:

TIMELINESS



WP1 9

WP2 9

WP3 9


COORDINATION



WP1 9

WP2 9

WP3 9


experienced


When you asked your leader any questions, how available was he or she to help, and how would you

rate the timeliness and clarity of the answers you were given?

WP1 9

WP2 9

WP3 9



How far did your experience as a WP partner meet

your expectations?

WP1 9

WP2 9

WP3 9





10


COMMUNICATION



WP leader?

WP1 10

WP2 9

WP3 10



17

3.6.4 UMU:

TIMELINESS



WP1 10

WP2 10

WP3 10

WP4 10


experienced

COORDINATION


How capable was the WP leader of coordinating the

project?

WP1 10

WP2 10

WP3 10

WP4 10


experienced




WP1 10

WP2 10

WP3 10

WP4 10


experienced



WP1 10

WP2 10

WP3 10

WP4 10


experienced


As a partner of a consortium, how well did ASL Brescia coordinate the project as CRiCoRM project manager?

10


COMMUNICATION



WP leader?

WP1 10

WP2 10

WP3 10

WP4 10



18

3.6.5 INOVA+:

TIMELINESS



WP1 8

WP2 8

WP3 8


COORDINATION



WP1 8

WP2 8

WP3 8


experienced




WP1 8

WP2 8

WP3 8




your expectations?

WP1 8

WP2 8

WP3 8




8


COMMUNICATION



WP leader?

WP1 8

WP2 8

WP3 8



19


3.7.1 ASL BRESCIA:

TIMELINESS



WP3 10


COORDINATION



WP3 10





were given?

WP3 10


experienced



WP3 10


COMMUNICATION


How good was the level of communication with your WP leader?

WP3 10



20

3.7.2 UNIBS:

TIMELINESS



WP1 10


COORDINATION



WP1 10





were given?

WP1 10


experienced



WP1 10





10


COMMUNICATION


How good was the level of communication with your WP leader?

WP1 10



21

3.7.3 LMU:

TIMELINESS


If you had deadlines to meet by the WP leaders

you work with, do you think you had enough time to prepare the answers?

WP1 10

WP2 10

WP3 10

WP5 10 If score is less than 7, please specified the kind of the problem

COORDINATION


How do you rate the coordination capacity of the WP leader?

WP1 10

WP2 10

WP3 10



If you made any questions to your leaders, how do

you rate the availability, timeliness and clarity in the answers?

WP1 10

WP2 10

WP3 10

WP5 10 If score is less than 7, please specified the kind of the

problem


How do you rate the activity of your WP compared to your expectations?

WP1 9

WP2 9

WP3 9





10



22

COMMUNICATION


How do you rate the communication aspect with your

WP leader? WP1 9

WP2 9

WP3 9

WP5 9 If score is less than 7, please specified the kind of the

problem

3.7.4 UMU:

TIMELINESS



WP1 10

WP2 10

WP3 10

WP4 10


COORDINATION



WP1 10

WP2 10

WP3 10

WP4 10


experienced




WP1 10

WP2 10

WP3 10

WP4 10


experienced



WP1 10

WP2 10

WP3 10

WP4 10





10



23

COMMUNICATION



WP leader?

WP1 10

WP2 10

WP3 10

WP4 10


3.7.5 INOVA+:

TIMELINESS


If your WP leader set you a deadline, do you think

you had enough time to prepare the answers?

WP1 9

WP2 9

WP3 9


COORDINATION



WP1 9

WP2 9

WP3 9


experienced




WP1 9

WP2 9

WP3 9




your expectations?

WP1 9

WP2 9

WP3 9




9



24

COMMUNICATION



WP leader?

WP1 9

WP2 9

WP3 9


3.8 Questionnaire to the CriCoRM WP leaders

The questionnaires were sent to the WP leaders who had started their activities in the

first year of the project. In particular, each WP leader was asked to evaluate the level

of collaboration of their WP partners according to Timeliness, Coordination and

Communication, as described in the evaluation plan.

The tables show the answers given in the two WP leaders’ questionnaires. Data were

collected until 9th January 2014 and until 28th May 2014.


25


3.9.1 WP1 - ASL BRESCIA

TIMELINESS

COORDINATION


If you asked your partners any questions, how available were they to help, and how would you rate the timeliness and clarity of the answers you were given?

9



How far did the activity of the WP meet your

expectations?

9


COMMUNICATION


How good was the level of communication with your WP partners?

9



If you established any deadlines, did your partners meet

them?

8


experienced

Some partners didn’t

fulfill the established deadlines.


26


TIMELINES

COORDINATION


If you asked your partners any questions, how available were they to help, and how would you rate the timeliness

and clarity of the answers you were given?

9


experienced


How far did the activity of the WP meet your expectations?

9


COMMUNICATION



9


experienced



them?

8


experienced


27

3.9.3 WP3 - UNIBS

TIMELINES

COORDINATION


If you asked your partners any questions, how available

were they to help, and how would you rate the timeliness and clarity of the answers you were given?

9




9


experienced

COMMUNICATION


How good was the level of communication with your WP

partners?

9


3.9.4 WP4 - LMU

TIMELINESS


If you established any deadlines, did your partners meet them?

10


experienced

Some partners didn’t

fulfill the deadlines.



10


experienced


28

COORDINATION




8




expectations?

9


COMMUNICATION



9




TIMELINESS

COORDINATION




10



9



29




expectations?

10


COMMUNICATION



10



TIMELINESS

COORDINATION




10




expectations?

10


experienced



9


experienced


30

COMMUNICATION



partners?

10


experienced

3.10.3 WP3 – UNIBS

TIMELINES

COORDINATION




10




expectations?

10


experienced

COMMUNICATION



partners?

9


experienced



them?

9


experienced


31

3.10.4 WP4 - LMU

TIMELINESS

COORDINATION




10


experienced



10


experienced

COMMUNICATION



partners?

10




them?

N.A.



32

3.10.5 WP5 - UMU

TIMELINESS

COORDINATION




10




expectations?

10


COMMUNICATION



10




10


experienced


33

4. Comment


34

4.1 Project management

Results of the two questionnaires show that, overall, the partners were very satisfied

with the progress of the project and with the cooperation they received. Very good

results were achieved relating to communication and coordination. All the partner

confirmed their satisfaction with ASL Brescia as project manager.

4.2 Work package progress

The results of the second year of the project are on schedule or achieved according to

the targets established by the WP leaders. There is only a minor deviation to be noted

concerns the feedback form to be submitted to key stakeholders. This target has been

postponed after the end of WP5, expected at month 28 because it is necessary to

complete the implementation of guidelines before sending them to the stakeholders

and consequently have a feedback from them.

4.3 General comment

In general, the results of questionnaires sent at the 12th and 24th month are

satisfactory and there are no important problems to be underlined. There is an overall

increase in the scores for all the specific targets described in the evaluation plan such

as Timeliness, Coordination and Communication between the two periods of data

collection.


35

5. ANNEXES: Independent Peer

Assessment reports


36

Date: 30-06-2013

D8 – 2nd Evaluation Report

Peer review report

Author: Alberto Bonetti

Work package n° 3 – Evaluation of the project

Task 3.3 Periodic independent assessment

1. Basic Information

Deliverable Reference: D2

Deliverable Title: Evaluation Report (2nd version)

Main Author/Editor: University of Brescia

Peer Reviewer (Institution, Person): Studio di Ingegneria Bonetti Alberto

2. Length of the deliverable

Is the length of the deliverable justified?

The deliverable, as applicable already for its previous version at month 12, has been

prepared as a concise document, well balanced with the expected level of assessment and

evaluation results.

It reflects appropriately the content and guidelines given in the deliverable D1 – Evaluation

Plan.

Are there any parts that result superfluous, irrelevant, redundant, unspecific and/or are

written in flowery language? (considering principles of precision and conciseness)

The focused character of the deliverable does not leave any room for redundant or

irrelevant content. It’s precise and concise.


37

3. Content

Does the deliverable report meet the objectives of the deliverable as described in the

CriCoRM project Grant Agreement technical attachment (Description of the action)?

If not, parts where improvement is necessary should be indicated.

The deliverable includes evaluation findings deriving from the suitable application, in the

second year of the project, of the procedures and provisions for self-monitoring of the

achievements of the project that were already documented within the initial Evaluation

Plan.

The deliverable presents sound evidence of how the CriCoRM consortium has self-evaluated

the achieved objectives for the second year of the project and has carefully monitored and

detected any potential project weakness.

The deliverable represents a well-structured and appropriate evaluation report for the

second year of the CriCoRM project, including clear and quantitative indications on the level

of achievement of the project objectives, expressing suitable performance indicators and

reaching a comprehensive internal evaluation.

The time plan for evaluation of the project tasks has been duly respected in the second

year, witnessing a diligent and competent evaluation management.

The resulting evaluation report is effective and clear.

In terms of achievement of the relevant targets related to the second year of the project,

D8 offers evidence that most of the indicators and parameters reached a successful level of

achievement.

The combination of internal, external and independent peer reviewing has allowed for a

quite comprehensive self-assessment exercise, useful to identify possible areas for

improvement, problematic as well as successful aspects of the project.

The tabular reporting and presentation of the level of achievement of each single indicator

per workpackage allows also in this second year version of D8 (as already evident in the

previous release) a quick and synthetic understanding of the level of success and

achievements.

Quite extensive has been also in the second year the effort spent on the assessment of the

level of quality and internal appreciation of project coordination and management activities.

The sound and overall very positive level of response confirms also in the second period of

the project the high level of project coordination and continuous support to the project

deployment.

Is the content of the deliverable focused and presented in a precise and to-the-point

manner? If not, parts where improvement is necessary should be indicated.

The deliverable is complete, focused and addressing all relevant issues.

It is appreciated that, answering to the last year peer review reporting, more effort has

been devoted to a more critical analysis and understanding of the meaning of the results.

In this way, the deliverable does not only witness a diligent work of collection and

assessment of parameters and indicators, but also an initial analysis and critical

understanding of the evaluated parameters, although not particularly detailed.

This is useful in order to understand better what is behind partial achievements, although

deserves further effort in the next and last period of the project.

Focus is and should be even further in the next period on the analysis of what is behind the

successful achievement of targets, in order that the revealed good practice is well

understood by the constituency and by those who are interested to replicate the CriCoRM

experience and approach.

Does the deliverable require substantial revision or rewriting?

If yes, concrete suggestions how to improve the deliverable should be given.

There is no need of any revision or rewriting of this version.


38

However, still there is the need (already pointed out last year and only partially applied in

the new version of the deliverable) to reach a fully effective “narrative” character, in order

to “tell the story” more clearly. This is expected in the month 36 version of this deliverable.

Grammatical errors and/or typographical errors and/or incomprehensive sentences should

be identified.

The deliverable is carefully and well written, clear, understandable, simple and correct.

4. Review Summary

The overall rating of the deliverable is [ 4 to 5: good to excellent]:

1: poor

2: below average

3: average

4: good

5: excellent

The current version of the deliverable is [ 1 ]:

1: applicable and ready to be submitted to the CHAFEA, if required;

2: applicable, but requires minor revisions;

3: inapplicable and requires substantial revisions.

Is it necessary for the revised deliverables to be reviewed again before submitting it to the

EAHC (1:Yes, 2: No)? [ No ]

Other remarks:

The effort spent also in the second year of the project in terms of evaluation and assessment

procedure and of identification of objective and measurable parameters has to be well

appreciated and should be even strengthen in the next and final period of the project, in order

to meet fully the initial contractual expectations.

Particular attention should be put on the Web2.0 application assessment, that will deserve a

significant effort and benchmarking exercise in the final phase of the project.

The final version of the deliverable should also offer a fully substantiated set of lessons

learned, sustained through quantified, measurable and objective parameters that should

reflect the initial expectations and represent a relevant benchmark for other similar

programmes and projects.

Finally, it is essential to stress again that evaluation should primarily serve impact assessment,

preparing the field for exploitation and deployment of the CriCoRM results, aiming at

replication and continuous adoption of the approach, concept, methodology and tools.


39

Date: 05-06-2014

D1 – 2nd Dissemination Report

Peer review report


Work package n° 2 – Dissemination of the results



Deliverable Title: Dissemination Report (2nd version)

Main Author/Editor: ASL Brescia




The deliverable replicates nature, structure and content of its previous first version. It is

equally suitably focused and presents an appropriate length and level of content and detail,

matching well its intended scope.

It reflects the initial sharp and focused character of the very first deliverable D1 –

Dissemination Plan – based on which this deliverable has been conceived and prepared.

Relevant dissemination tools have been duly monitored and commented also in the second

year of the project (leaflet, posters, articles, website, conferences and events).



There are no irrelevant or redundant sections.


40

3. Content


CriCoRM project Annex to the Grant Agreement (Description of the action)? If not, parts

where improvement is necessary should be indicated.

The deliverable represents the monitoring report of all dissemination activities carried out

in the second year of the project. It represents a document that is expected to further

evolve in the last year of the project.

The achieved dissemination targets are correctly reported and matched to the plans and

estimated outcomes. There is a sound evidence of an appropriate level of achievements.

Mayor stakeholders, already duly identified in the previous period, have been further

addressed through variegate dissemination measures.

As recommended in the last year peer review assessment, initial elements of a critical

analysis and understanding of the level of achievements and lessons learned from

dissemination activities have been included within D1, although still with limited argoments

in terms of potential impact of the planned dissemination activities.

A specific chapter on dissemination evaluation is included, comparing achievements with

expectations, but not entering in great detail in dealing with impacts and lessons to be left

for future similar exercises.


manner?If not, parts where improvement is necessary should be indicated.

The deliverable is complete and focused.

There are no evident needed improvements to the actual version, although the authors are

invited to continue, also in the next period, to analyse even in higher detail and provide a

more comprehensive critical understanding of the effective impact of the various

dissemination measures deployed by the partners. Particular attention shall be put on

social media based dissemination initiatives, considering that this is the main focus of

CriCoRM.

Does the deliverable require substantial revision or rewriting? If yes, concrete suggestions

how to improve the deliverable should be given.

There is no need of any revision or rewriting at this stage.

There will be completions expected in the next scheduled Dissemination Report.


be identified.

The deliverable is overall well written and understandable. There is evidence that effort has

been put on proof-reading and polishing, well improved in respect to previous versions.

4. Review Summary

The overall rating of the deliverable is [ 4: good ]:

1: poor

2: below average

3: average

4: good

5: excellent




41





Other remarks:

The deployed dissemination activities have been of a good quality and well balanced with the

type of project. It is well appreciated the novel effort on appreciating initial elements linked to

the effective value and impact the dissemination measures will have on the addressed

constituency, although further effort should be devoted to these arguments, in the next period

and deliverable version.

This still represents, in fact, a relevant challenge to be faced also in the last year of the project

and duly reflected in the final version of this deliverable (scheduled by the end of the project).

Extra effort and focus should be put in the last year of the project to involve actively

stakeholders like end-user organisations and voluntary, consumers and patients’ associations,

considering the growing role this type of tertiary audience plays in the communication field and

in respect to the CriCoRM project in particular.

The final version of D1 should put particular evidence on a structured and detailed

understanding of the level of success dissemination activities have reached, in comparison to

initial expectations and reporting about each deviations commenting on reasons and

elaborating on any possible lesson learned, for the benefit of the partners but also of the Public

Health Programme constituency and future projects.


42

Date: 15-12-2013

D3 –State of the Art review report

List of key Stakeholder Identification

Data collection

Peer review report


Work package n° 4 – Analysis of communication processes



Deliverable Title: State of the Art review report - List of key Stakeholder Identification -

Data collection

Main Author/Editor: LMU (Ludwig-Maximilians-Universitat)




The deliverable’s length reflects appropriately the depth of the conducted analysis.

The level of detail is suitable for the scope and meets well the expectations.

Relevant literature references are included in support to an exhaustive and comprehensive

analysis of the state-of-the-art in crisis communication.



There are no irrelevant or redundant sections.


43

3. Content


CriCoRM project Annex to the Grant Agreement (Description of the action)? If not, parts

where improvement is necessary should be indicated.

The deliverable represents a final and comprehensive version of D3, and follows an initial

draft version released at month 12.

It presents, with a sound level of detail, the available literature on crisis communication,

includes best practices, theoretical frameworks and empirical studies in the field.

It discusses a great variety of topics from organizational to finance issues related health

crises and offers an interesting and competent overview.

D3 gives a complete “state of the art” summary of the literature on crisis communication in

health related areas of the last years. It starts with a theoretical foundation of crisis

communication and discusses the difference between risk and crisis communication. Then,

the specific characteristics of crisis communication concerning health issues are explained.

Afterwards, the media’s role as communicator and information channel is analysed,

followed by an examination of chances and limits of social media in the area of crisis

communication.

The deliverable includes also the description of important frameworks on crisis

communication and reviews relevant empirical studies dealing with health crises. There is a

focus on studies regarding the H1N1 pandemic, but also papers concerning SARS and avian

influenza are discussed. The part about the empirical investigation of H1N1 is divided into

grey literature, content analyses and stakeholder surveys. Finally, D3 summarizes the main

results and draws some initial concluding notes.

A specific focus is put on the specific characteristics of crisis communication in the area of

public health and to social media deployment, in particular.

In respect to the initial draft, this final version of the deliverable include further details

related to the critical analysis of the collected evidence, state-of-the-art and stakeholder

survey, in terms of crisis communication.

Applying appropriately what observed and recommended within the peer review report

related to the draft version, the actual version of D3 goes appropriately beyond a diligent

monitoring exercise and desk study, including also a critical view and analysis of the crisis

communication perspectives, in particular in terms of potential impact of the different

measures and available approaches.

The review of relevant literature revealed in fact several research gaps. First, the use of

social media in crisis communication has only been considered for the US, European

countries have not been taken into account yet. Second, focusing on risk and crisis

communication, no broad comparison of more than two countries could be found. Only

communication strategies of the US and China were compared. Third, no study has tracked

the crisis communication process through several stages. Not more than two channels (e.g.

press release and media coverage) have been considered so far. An analysis and

comparison of three different communication channels (such as press releases, news

coverage and online discussions) is still missing, even though this would be essential for

understanding the diffusion of information and the public opinion-building process in crises.

These are quite informative findings, witnessing a through conducted analysis.

However, the work has also revealed some limitations (duly recognized by the same

authors). These are mainly related to the online survey, where a comparison of different

cultural groups wasn’t possible because the participants were distributed unevenly across

the single countries. Further, the participants resulted to be very heterogeneous regarding

their nationality and the type of organization they belonged to. Nevertheless, the results

remain a valid indicator for the perspective of European stakeholders involved in crisis

communication.


44

Another aspect that revealed some limitations is the content analysis. Here, a definitive

proof of a causal effect of media coverage on vaccination behavior and mortality is not fully

provided. Furthermore, only ten European countries and two newspapers per country have

been examined. Thus, the results are somewhat limited and cannot be inferred to media

coverage in European countries in general (although the leading newspaper in each country

and countries covering more than 70% of the European population have been considered).

Similarly, only press releases of the most important health authority in one country have

been coded, neglecting information provided for example by local or regional health

authorities (although these might have been influenced by national press releases again).

Overall, despite the indicated methodological limitations, this study still provides clearly

important practical implications for communication in health crises, representing a solid

basis for next project phases.



The deliverable is complete, focused and addressing all relevant issues (although with few

limitations already argued in the previous section).

There are no evident needed improvements to the final version, considering the added

details, in respect to the previous draft version of month 12, and the included analysis and

critical understanding of the effective impact that the various crisis communication

approaches imply.



There is no need of any revision or rewriting.


be identified.

The deliverable is well written and understandable.

There is no need of further proof reading. It is accurate and suitable to be disseminated to

a professional but also to a wider non-specialistic audience. It is in fact sufficiently self-

standing and self-explanatory.

4. Review Summary

The overall rating of the deliverable is [ 4 to 5: excellent ]:

1: poor

2: below average

3: average

4: good

5: excellent








45

Other remarks:

The deployed activities, analysis of crisis communication state-of-the-art and stakeholders

survey are of an overall very good quality and well balanced with the expectations.

In the next period, the CriCoRM project should build appropriately on the findings from D3, in

particular in connection to the lessons learned from the appreciation and critical review of the

various possible communication strategies deployed within the deliverable, that should be used

to define the best approaches that the CriCoRM project should adopt and follow.

The main focus should be to ensure an effective communication, value and impact of the

project on the addressed constituency and community at large.


46

Date: 30 -06-2013

2nd Interim Report

Peer review report


Work package n° 1 – Coordination of the project

Task 3.3 Periodic independent assessment


Deliverable Reference:

Deliverable Title: Interim Report (2nd version)

Main Author/Editor: ASL Brescia




The deliverable is of an appropriate length. As for its previous version, also the second

interim report is well detailed, well-structured and offers a suitable level of detail in terms

of achievements of the project in the second year of operation of the project.

It is concise and with a level of detail well balanced with the expected level of management

activities.



The focused character of the deliverable does not leave any room for redundant or

irrelevant content. It’s precise and concise.


47

3. Content


CriCoRM project Grant Agreement technical attachment (Description of the action)? If not,

parts where improvement is necessary should be indicated.

The Interim Report

The deliverable includes appropriate details on achievements reached during the second

year of the project, comparing them accordingly with expectations and initial plans.

It represents an informative overview of the project status and results, also with a critical

assessment of results achieved.

The Interim Report is structured following appropriately the due template. The presentation

of achievements per workpackage is sufficient, although not very detailed.

There is no evidence of particular deviations in respect to the plans and neither of

significant delays or drawbacks.

The sound cooperative spirit within the consortium is confirmed also in the second year of

the project, witnessing an appropriate deployment of the respective roles and a diligent

fulfillment of the allocated duties.

The work done

The work has followed well the plans. A sound effort has been devoted to promotional and

dissemination activities also in the second year of the project, through the website,

newsletters and articles.

The project meetings revealed to be affective, well prepared and supportive for the project

deployment.

The effort

The claimed resources are overall well in line with the plans, well balanced with the level of

achievements and well in line with the workplan and the needs of the different tasks.

The remaining effort is appropriate for the scope and well balanced with the activities

needed to complete the project.

All necessary conditions for a successful completion of the project are in place, with a

suitable potential of attainment of the final envisaged objectives, although the last period is

quite demanding, in terms of activities to be completed and deliverables to be finalised.

Crucial and critical will be the validation activity of the CriCoRM tools. Most of the project

value, success and findings depend from this validation activity.



The deliverable is complete, focused and addressing all relevant issues.

There are no evident needed improvements.



There is no need of any revision or rewriting.

The deliverable will be followed by a similar final management report at month 36.


be identified.

The deliverable is well written, clear, understandable, simple and correct, and witnesses of

a further effort devoted to proof reading and polishing, as recommended within the peer

reviewing of last year management reporting.


48

4. Review Summary

The overall rating of the deliverable is [ 4: good ]:

1: poor

2: below average

3: average

4: good

5: excellent


1: applicable and ready to be submitted to the EAHC, if required;





Other remarks:

The project is entering into the final very demanding period. A wide piloting will represent the

major scope of the last part of the project.

It will be essential to pilot and validate accordingly the developed Web 2.0 tools for health

crisis communication, that should demonstrate in an objective manner to represent an

effective two-way communication with the public, other than an interesting real-time

assessment of the impact of public health messages.

In the end, CriCoRM should and has all the potential to demonstrate and validate a relevant

benchmark and best practice in the field, as expected and anticipated at the start of the

project.

The Final Report is expected to bring evidence and solid arguments, in this respect.

CriCoRM Project WP3: EVALUATION OF THE PROJECT Evaluation report.pdf2nd Evaluation report 4 This...

Documents

Transcript of CriCoRM Project WP3: EVALUATION OF THE PROJECT Evaluation report.pdf2nd Evaluation report 4 This...