LOSD Publication and Capacity Building · LOSD Publication and Capacity Building D5.2 Training and...

14
LOSD Publication and Capacity Building D5.2 Training and Capacity Building Roadmap Date: 25 June 2019 Web: www.derilinx.com Email: [email protected] Address: 11/12 Baggot Court, Dublin 2, D02 F891 Tel: +353 (0)1 254 4316 Mob: +353 (0)87 417 2318 Twitter: @derilinx

Transcript of LOSD Publication and Capacity Building · LOSD Publication and Capacity Building D5.2 Training and...

Page 1: LOSD Publication and Capacity Building · LOSD Publication and Capacity Building D5.2 Training and Capacity Building Roadmap Date: 25 June 2019 Web: Email: deirdre@derilinx.com Address:

LOSD Publication and Capacity Building

D5.2 Training and Capacity Building

Roadmap

Date: 25 June 2019

Web: www.derilinx.com

Email: [email protected]

Address: 11/12 Baggot Court,

Dublin 2, D02 F891

Tel: +353 (0)1 254 4316

Mob: +353 (0)87 417 2318

Twitter: @derilinx

Page 2: LOSD Publication and Capacity Building · LOSD Publication and Capacity Building D5.2 Training and Capacity Building Roadmap Date: 25 June 2019 Web: Email: deirdre@derilinx.com Address:

LOSD D5.2

2

Table of Contents

1 Introduction 3

1.1 Purpose and Target Group of the deliverable 3

1.2 Document outline 3

2 LOSD Capacity Building Assets 3

2.1 Capacity Building Platform 3

2.2 Learning Materials 5

3 LOSD Capacity Building Services 5

3.1 LOSD Webinars 5

3.2 LOSD Hackathons and Meetups 7

3.3 GitHub 8

3.4 Slack Channel 8

4 LOSD Capacity Building Lessons Learnt 9

4.1 Capacity Building Approach 9

4.1.1 Virtual Assistant 10

4.1.2 Tool Tours 10

4.2 Interaction with Stakeholders 11

5 General Recommendations 13

5.1 NSO Network 13

6 Conclusions 14

Page 3: LOSD Publication and Capacity Building · LOSD Publication and Capacity Building D5.2 Training and Capacity Building Roadmap Date: 25 June 2019 Web: Email: deirdre@derilinx.com Address:

LOSD D5.2

3

1 Introduction This Linked Open Statistical Data (LOSD) project is being delivered in the context of the European

Statistical System’s (ESS) DIGICOM project. DIGICOM contributes to two key areas of the ESS vision:

(i) focus on users, and (ii) improve dissemination and communication1. One of the four DIGICOM work

packages is Open Data Dissemination2. The objective of the Open Data Dissemination work package is

to facilitate automated access to European aggregate data for heavy users or re-disseminators and to

improve access to micro-data.

1.1 Purpose and Target Group of the deliverable

The objective of this deliverable is to present a basic roadmap for capacity building beyond the

duration of the project. In particular, we elaborate upon the assets developed and lessons learnt in

the project as well as service options available for future capacity building. We discuss the challenges

and opportunities and possible solutions that may improve capacity building services in the context of

LOSD project service delivery. The Training and Capacity Building Roadmap presented in this

deliverable are drawn from direct interactions with stakeholders during hackathons and events, over

online channels, such as webinars and on the slack channel, and from feedback forms and extracted

web-statistics.

1.2 Document outline

The remainder of this document is organised as follows:

● Section 2 presents the description of the LOSD Capacity building assets that can be leveraged

by the NSOs for training purposes

● Section 3 presents the on-demand service options available on the capacity building supplier

side

● Section 4 presents the lessons learnt from capacity building in the LOSD project

● Section 5 elaborates upon capacity building recommendations based on the feedback

received.

2 LOSD Capacity Building Assets Over the course of the development of the Linked Open Statistical Data (LOSD) Pipeline, several

reusable capacity-building assets were generated. These assets will support the ongoing use and

adoption of the LOSD Pipeline in the future.

2.1 Capacity Building Platform

We have leveraged the established e-learning platform framework, Moodle, for the LOSD Capacity

Building Platform. That is to provide maximum ease of use and support for future improvements. The

framework enables flexible assembly of courses and learning materials including multimedia and

interactive components, allows effective authentication and manages access control. We leveraged

1 http://ec.europa.eu/eurostat/web/ess/about-us/ess-vision-2020/implementation-portfolio#DIGICOM 2 http://ec.europa.eu/eurostat/web/ess/digicom

Page 4: LOSD Publication and Capacity Building · LOSD Publication and Capacity Building D5.2 Training and Capacity Building Roadmap Date: 25 June 2019 Web: Email: deirdre@derilinx.com Address:

LOSD D5.2

4

Moodle to deploy learning content within six dedicated courses - three introductory courses and three

hands-on courses. The courses are composed of text (explanatory content and instructions), images

(screenshots of tools) and videos (for demos and tutorials). Some of the courses included extra quizzes

to boost the user-engagement. Information about the platform has been advertised and disseminated

before the hackathons and webinars in order to ensure that the stakeholders have enough knowledge

and skills to effectively participate in the activities envisaged.

Moodle has been installed on a cloud based server and made available at this link

http://losd.staging.derilinx.com/moodle/

Figure 1: Screenshot from LOSD Capacity Building Platform (Moodle)

The platform has been running throughout the duration of the project and can be maintained beyond

the project if required. A dockerised version has been developed, so that it is easy to deploy and run

the platform in different locations in different contexts.

Page 5: LOSD Publication and Capacity Building · LOSD Publication and Capacity Building D5.2 Training and Capacity Building Roadmap Date: 25 June 2019 Web: Email: deirdre@derilinx.com Address:

LOSD D5.2

5

2.2 Learning Materials

During the course of the project, we have generated rich learning materials including the Introduction

to Linked Data and Linked Open Statistical Data, and LOSD Pipeline specific materials. All the materials

are available on the Capacity Building Platform. Moreover, the materials have been released in the

universal PDF format for download and offline exploration. These materials can be leveraged in the

future capacity building efforts and to replicate the training process conducted in the project with

different NSOs or other stakeholders.

The content on the platform is structured into specific courses targeted to stakeholders of each of the

use-cases:

1. Introduction to Linked Data - The course is intended to provide a basic introduction to the topic

of Linked Data.

2. Introduction to RDF This course is an introduction to RDF - The course is simply a test for this

training platform.

3. LOSD Standards and Tools - This course introduces participants to existing LOSD standards (such

as RDF, SKOS, DCAT, Data Cube etc.) and technologies (such as Virtuoso, SPARQL).

4. LOSD Pipeline: LOSD Mapping - The course is intended to cover the principles of using Juma for

creating data mappings.

5. LOSD Pipeline: Data Cataloguing - The course is intended for all group of users and deals with the

use of CKAN as a data catalogue platform

6. LOSD Pipeline: Data Visualisation - This course introduces the set of support services and

applications that are available as part of the LOSD platform. It is mainly focused on the analysis

and visualisation of LOSD.

3 LOSD Capacity Building Services

3.1 LOSD Webinars

The LOSD Webinar I and II were streamed live via the Facebook platform and now are also available

online on Facebook. The main goal for the webinars was to provide a step-by-step overview of

concepts and tools to the stakeholders in a narrative form. To view those materials there is no need

for having a Facebook account (this is public content).

Webinar I can be found at:

https://www.facebook.com/newsegov/videos/758473811173169/

Page 6: LOSD Publication and Capacity Building · LOSD Publication and Capacity Building D5.2 Training and Capacity Building Roadmap Date: 25 June 2019 Web: Email: deirdre@derilinx.com Address:

LOSD D5.2

6

Webinar II can be found at:

https://www.facebook.com/newsegov/videos/254843405400883/

Both webinar videos can be considered a good summary and training asset that complement the

material on the Moodle platform.

Page 7: LOSD Publication and Capacity Building · LOSD Publication and Capacity Building D5.2 Training and Capacity Building Roadmap Date: 25 June 2019 Web: Email: deirdre@derilinx.com Address:

LOSD D5.2

7

3.2 LOSD Hackathons and Meetups

In the project, we have supported number of hackathons and meetups:

● Hackathon in Dublin (February 2018)

The hackathon in Dublin was the first opportunity for the members of the consortium to meet

with different representatives of the NSO partners. At this meeting, the work that the

consortium would be partaking was specified and requirements gathered. Three use cases

were proposed and agreed upon and a subset of datasets was chosen for the consortium to

focus on.

● Hackathon in Paris (September 2018)

It was a mid-project hackathon with 50+ participants from various horizons. Six teams working

on producing and consuming Linked Open Statistical Data. Members of the consortium

attended for two days. There they participated in some hackathon activities and had the

opportunity to demonstrate and get some user feedback on the different tools used in the

first version of the LOSD Pipeline (http://losd-data.staging.derilinx.com). A dedicated room

was made available to the consortium for participants to test the platform, in parallel to other

hackathon activities. As a result, only some people choose to test the platform at this stage

and feedback received was very minor.

● Meetup in London (January 2019)

This two-day workshop theme was 'Linked Open Statistical Data in the European Union'.

Practitioners from statistical institutes, NGOs and the private sector came together to

exchange lessons learned and collective ways forward: (i) provide high-level view of the

progress on creating Linked Data ecosystems for government statistical data and (ii) discuss

the challenges and opportunities that arise from this work.

● Hackathon in Rome (March 2019)

Members of the consortium attended the Rome hackathon for one day. During this

hackathon, the consortium was provided with a dedicated slot in the agenda to present to all

participants. There, the consortium presented in detail all aspects of the LOSD Pipeline in the

context of the three use cases. While going through each use case, participants were

encouraged to follow along in some of the activities while using the platform. At the end of

the presentation, a fruitful question and answer session was had, where the functionalities

and limitations of the LOSD platform were discussed. This hackathon provided a great

opportunity for the consortium to not just present the work that was done to date, but also

get the hackathon participants to really engage with the platform and provide essential

feedback.

● Final workshop in Sofia (May 2019)

The final ESSnet conference in Sofia was a successful event, where the outcomes of the project

were presented. An overview of the LOSD Pipeline was shown, as well as additional

information around challenges that the NSOs faced, how these were addressed throughout

the project. We also discussed the key lessons learned and recommended next steps for the

LOSD Pipeline. The NSO teams that we have been working with, as well as the wider ESSnet

Page 8: LOSD Publication and Capacity Building · LOSD Publication and Capacity Building D5.2 Training and Capacity Building Roadmap Date: 25 June 2019 Web: Email: deirdre@derilinx.com Address:

LOSD D5.2

8

community from other NSOs, the Eurostat team, and the local Bulgarian Open Data Unit

attended the event.

3.3 GitHub

The LOSD Pipeline is available as Open Source and can be accessed at the LOSD GitHub repository

available at https://github.com/LOSD-Data. This enables the local adoption, use and extension of the

LOSD Pipeline, as is being done be ISTAT, the Italian NSO.

3.4 Slack Channel

Stakeholders involved in the project were using dedicated Slack channel for the communication and

announcements. It is available at https://los-essnet.slack.com.

The Slack channel provided a great means of communication for members of the project. Usage was

infrequent in the early stages of the project. However, as time went on and more NSO personnel began

Page 9: LOSD Publication and Capacity Building · LOSD Publication and Capacity Building D5.2 Training and Capacity Building Roadmap Date: 25 June 2019 Web: Email: deirdre@derilinx.com Address:

LOSD D5.2

9

interacting with the LOSD platform - the slack channel became an important place for the consortium

to interact with, answer questions and help the NSO personnel with the use of the platform.

4 LOSD Capacity Building Lessons Learnt

4.1 Capacity Building Approach

Our Capacity Building approach encompassed several different techniques. In particular we have

relied on the Capacity Building Platform as a major resource and convergence point for users to find

essential instructions and related materials. The second important component was direct interactions

with users - via email, a Slack Channel and face-to-face meetings where experts guided the users on

their journey with the LOSD Pipeline. The LOSD Capacity Building Platform has been well received by

the pilots and no major issues were reported in using the platform. We present some basic statistics

in terms of use of the platform below:

• Total Pages views: 2107

• Registered Users: 25

• Views per specific course:

o Introduction to Linked Data: 82

o Introduction to RDF: 107

o LOSD Pipeline: Data Cataloguing: 395

o LOSD Pipeline: Data Visualisation: 194

o LOSD Pipeline: LOSD Mapping: 441

o LOSD Standards and Tools: 220

Page 10: LOSD Publication and Capacity Building · LOSD Publication and Capacity Building D5.2 Training and Capacity Building Roadmap Date: 25 June 2019 Web: Email: deirdre@derilinx.com Address:

LOSD D5.2

10

From the statistics collected it is apparent that the LOSD Capacity Building Platform experienced rather

moderate use. The total page views of 2107 presents an average figure considering the number of

stakeholders in the project. The most popular content included the Data Mapping and Cataloguing

courses. These were followed by Standards and Visualisation. The least used resources were the

introductory courses. A direct conclusion can be drawn that users prefer hands-on practical material

rather than more theoretical introductory courses. Also, the most popular course - LOSD Mapping was

also the part of the project which stakeholders reported to be the most challenging and required the

most support from experts. Therefore, the most challenges identified with that part of capacity

building can also span from the biggest interest & popularity of that material.

Users pointed out that more detailed materials or more assistance (including automated assistance)

could be provided so that users do not need to look for expert support to work with the LOSD tools.

That could be achieved by:

1) More detailed materials - more distinct steps in the materials, more help videos and more

explicit, step-by step tutorials or supporting technologies such as chatbots or ‘Interactive

Tooltips’ implemented in the tools that could ensure the highest level of automated assistance

in learning new tools - discussed in further part of this section

2) More interactive approach - more Moodle interactive features could be also applied to the

courses. The standard courses and portable PDF versions showed to be insufficient and more

engaging mode of training would be desired. In particular, interactive elements could be

implemented to ensure more active & rewarding process of user engagement. Therefore,

users could be evaluated and their learning process could be validated as they explore the

learning materials provided.

4.1.1 Virtual Assistant

Drawing on the conclusions and user feedback received, in particular reflected in the need for expert

support, a major recommendation is the need for more explanatory capabilities. For example, we

could explore using emerging conversational technologies such as Chatbots and Virtual Assistants to

help participants to reach the materials they are looking for. These chatbots could simulate experts

and could serve as agents supplying relevant materials to the user. They could also directly answer

queries and in more advanced versions, process the natural language queries by the users and give

them explicit answers. That would require the application of more advanced, AI-driven solutions and

training sets for conversations.

4.1.2 Tool Tours

Another possible improvement to the learning experience could be ‘Tool Tours’ that would guide users

through an application using step-by-step navigation within the tools so that they can become familiar

Page 11: LOSD Publication and Capacity Building · LOSD Publication and Capacity Building D5.2 Training and Capacity Building Roadmap Date: 25 June 2019 Web: Email: deirdre@derilinx.com Address:

LOSD D5.2

11

with the workflow without needing to interact with a human agent. We present an example solution

below (Source: https://www.appcues.com/blog/tooltips)

The Interactive Tool Tours are one of the best ways to ensure step-by-step learning as user engages

with the tool. The approach alleviates the often faced disconnect between the tool and the learning

material. Instead of ‘jumping’ between the tool and the tutorial or readme file the user is guided from

within the tool and learns the workflow more effectively. That approach however requires a direct

implementation of learning materials and suggestions directly within the tool rather than separate

learning resources. That imposes significant extra development effort on the tool developers to design

a special ‘Happy tour path’ or several of those to ensure all the use scenarios covered. However, the

solution offers one of the best and universal learning experiences.

4.2 Interaction with Stakeholders

One of the major forms of interaction with stakeholders in the project was through webinars,

hackathons and events. While webinars apply digitally-mediated dialogue, the hackathons and events

ensure effective face-to-face interactions. The obvious limitation of the digital channel communication

is compensated by possibility to reaching out to broader audience and removing many of the

participation limitations - a need to travel.

Direct communication with participants proved to be very effective. The stakeholders appreciated the

help of the experts. In terms of digitally-mediated dialog we had experts interacting with stakeholders

via e-mail and the Slack channel. More importantly however, we have organised and held two

webinars scheduled six months apart. The first webinar dealt with the Introduction to Linked Data,

while the second one took the participants on a hands-on journey through LOSD Publishing Pipeline.

Page 12: LOSD Publication and Capacity Building · LOSD Publication and Capacity Building D5.2 Training and Capacity Building Roadmap Date: 25 June 2019 Web: Email: deirdre@derilinx.com Address:

LOSD D5.2

12

The separation in time was perceived by stakeholders as very positive approach marking major

milestones in the project. Here we present some statistics in terms of audience and participation:

● The first webinar attracted 27 participants registered via Eventbrite and generated over 540

views on Facebook.

● The second webinar attracted 71 (increase from 27 in first webinar) participants registered via

Eventbrite and generated over 1.3K views on Facebook.

The materials were also put on YouTube for greater availability however they did not generate

significant views (less than 100). The advantage of Facebook dissemination especially in the second

Webinar came with directed paid campaign where we explicitly disseminated materials to people

interested in Open Data in Ireland and Brussels.

The webinars were evaluated using relevant feedback forms - we leveraged popular Google Forms for

that. The solution even though easy registered very little feedback and we had to send several

reminders before collecting some critical mass of data (about 7 responses in total). We believe that

explicit introduction to webinar evaluation could help in the future for collecting richer results. In

particular giving the participants more ownership of the evaluation process should make the

stakeholder more engaged.

The major feedback received indicated that there is a strong need for more interaction with the

presenters and extra interactive session at the end. Therefore, future webinars would include more

interactive elements and dedicated Q & A at the end of the session. In terms of technical support for

interactivity the current setting using Facebook as the primary channel and Goto Meeting as a

secondary is sufficient. The interactivity can be provided via full-voice feature in Goto Meeting or chat

& comment in Facebook.

One practical lesson learnt is that the numbers of speakers/presenters in the webinars should be

limited. First, it is difficult to fit a large number of people in front of the camera. Second, if speakers

have little to say the speaker / presentation change creates too much overhead to justify extra speaker

participation.

Figure 2: Preparation to Webinar Transmission - Busy Green Screen Studio at Insight

Page 13: LOSD Publication and Capacity Building · LOSD Publication and Capacity Building D5.2 Training and Capacity Building Roadmap Date: 25 June 2019 Web: Email: deirdre@derilinx.com Address:

LOSD D5.2

13

While running the webinars we experienced some technical difficulties related to failing internet

connection. The network infrastructure interruptions created a major obstacle to webinars. Therefore,

it is essential to ensure stable internet connectivity at all times not to lose the engagement with the

audience (4G backup connectivity could be a good option).

Another improvement could be applied to ensure larger ‘general public’ participation. We started an

online campaign within two weeks or so before the webinars. The numbers are satisfactory however

there is still a room for improvement. In particular we the webinars could be scheduled much more in

advance to have relevant dissemination strategy implemented far ahead of the webinar to allow

enough time for participants to include it in their busy schedules.

5 General Recommendations Our main recommendation would be to make sure that the capacity building services are well aligned

to the specific stakeholder needs, but also that they can be easily adapted to changing needs. In

particular, the provider of the services must ensure maximum flexibility as the needs may shift during

the service delivery as stakeholders learn about the topic (such as LOSD) and new questions,

challenges and needs emerge. Therefore, the materials and services offered must be extensible and

portable.

The best example is the LOSD mapping exercises, which proved to be a difficult task, yet one of the

most critical elements of capacity building. Users required significant support from experts on those

exercises and presented important value to the stakeholders. The mapping tools appeared to be

difficult to use and the learning processes provided still required some human intervention. Therefore,

future capacity building services should identify the critical elements as soon as possible and ensure

very strong support for those issues. That support could be automated as in the examples given on

Interactive Tooltips and Chatbot. The initial extra resources spent on providing the training

automation included in the tools themselves will definitely pay off when more stakeholders and more

users are to be trained.

In terms of mode of communication such as face-to-face interactions, hackathons, e-mail, Slack

channel and webinars, the in-person interactions are always favoured. However, due to limited

resources the best trade-off in communication is probably a well-supported Slack Channel and

interactive webinars. This is mainly due to the synchronous nature of communication. Users require

assistance live, as they explore the specific topic or use the tool. The e-mail communication and

‘passive learning materials’ such as PDF can work well as additional support but should not be in the

core of the capacity building delivery. That is reflected in particular in the feedback received on strong

need for more interactivity with experts which is understandable considering the fast-changing needs

and requirements that emerge as stakeholders explore the topic.

5.1 NSO Network

We propose that following on from the LOSD project, an LOSD NSO Network should be established for

increased peer-to-peer learning. The concept of the NSO Network is to support a community of Linked

Data practitioners in NSOs, including those from the Central Statistics Office, Ireland; INSEE French

Statistical Office; ISTAT Italian Statistical Office; and National Statistical Institute, Bulgaria. Such a

group, focussing initially on skills development, knowledge sharing and practice development, will act

as an (informal) Linked Data practitioner community across NSOs. Activities could include

Page 14: LOSD Publication and Capacity Building · LOSD Publication and Capacity Building D5.2 Training and Capacity Building Roadmap Date: 25 June 2019 Web: Email: deirdre@derilinx.com Address:

LOSD D5.2

14

development of and participation in webinars, seminars, training, project design and planning. This

community will be the ‘go to’ people in various areas relevant to the LOSD Publishing Platform (CKAN,

JUMA, SPARQL, etc.) They will maintain relationships with non-NSO Linked Data and Semantic Web

stakeholders such as Derilinx and university-based researchers. They will be the key NSO capability

that will ensure the sustainable evolution of the Linked Data platforms

6 Conclusions In this document we presented the lessons learned from our experience in delivering the LOSD

capacity building services to project stakeholders. We have identified the strong points of delivery and

areas where there is space for improvement. We have also presented some further recommendations

on possible service delivery in the future.

In particular, we have elaborated upon the challenges in enabling the LOSD stakeholders (especially

statisticians and other individuals working with data) to use the LOSD Publishing Pipeline effectively.

Those relate mainly to the need for better alignment and merging of the training content and training

approach with the tools and more step-by-step flow to minimise the need for expert interventions.

The best candidate solution in that sense seems the ‘Interactive Tooltips’ or ‘Tool Tours’ integrated

and supplied with the tools to users. Also, automation in terms of FAQ would be also desired.

Additionally, more intense face-to-face interactivity would be encouraged in place of development of

‘passive learning materials’ and tutorials. The scope of the in-person interactions will be of course

limited by the funds available and digital training should be always considered an option. In that case,

interactive synchronous communication such as Slack Channel or Live-Webinars with strong

interactivity support are encouraged.