LOSD Publication and Capacity Building · LOSD Publication and Capacity Building D5.2 Training and...
Transcript of LOSD Publication and Capacity Building · LOSD Publication and Capacity Building D5.2 Training and...
LOSD Publication and Capacity Building
D5.2 Training and Capacity Building
Roadmap
Date: 25 June 2019
Web: www.derilinx.com
Email: [email protected]
Address: 11/12 Baggot Court,
Dublin 2, D02 F891
Tel: +353 (0)1 254 4316
Mob: +353 (0)87 417 2318
Twitter: @derilinx
LOSD D5.2
2
Table of Contents
1 Introduction 3
1.1 Purpose and Target Group of the deliverable 3
1.2 Document outline 3
2 LOSD Capacity Building Assets 3
2.1 Capacity Building Platform 3
2.2 Learning Materials 5
3 LOSD Capacity Building Services 5
3.1 LOSD Webinars 5
3.2 LOSD Hackathons and Meetups 7
3.3 GitHub 8
3.4 Slack Channel 8
4 LOSD Capacity Building Lessons Learnt 9
4.1 Capacity Building Approach 9
4.1.1 Virtual Assistant 10
4.1.2 Tool Tours 10
4.2 Interaction with Stakeholders 11
5 General Recommendations 13
5.1 NSO Network 13
6 Conclusions 14
LOSD D5.2
3
1 Introduction This Linked Open Statistical Data (LOSD) project is being delivered in the context of the European
Statistical System’s (ESS) DIGICOM project. DIGICOM contributes to two key areas of the ESS vision:
(i) focus on users, and (ii) improve dissemination and communication1. One of the four DIGICOM work
packages is Open Data Dissemination2. The objective of the Open Data Dissemination work package is
to facilitate automated access to European aggregate data for heavy users or re-disseminators and to
improve access to micro-data.
1.1 Purpose and Target Group of the deliverable
The objective of this deliverable is to present a basic roadmap for capacity building beyond the
duration of the project. In particular, we elaborate upon the assets developed and lessons learnt in
the project as well as service options available for future capacity building. We discuss the challenges
and opportunities and possible solutions that may improve capacity building services in the context of
LOSD project service delivery. The Training and Capacity Building Roadmap presented in this
deliverable are drawn from direct interactions with stakeholders during hackathons and events, over
online channels, such as webinars and on the slack channel, and from feedback forms and extracted
web-statistics.
1.2 Document outline
The remainder of this document is organised as follows:
● Section 2 presents the description of the LOSD Capacity building assets that can be leveraged
by the NSOs for training purposes
● Section 3 presents the on-demand service options available on the capacity building supplier
side
● Section 4 presents the lessons learnt from capacity building in the LOSD project
● Section 5 elaborates upon capacity building recommendations based on the feedback
received.
2 LOSD Capacity Building Assets Over the course of the development of the Linked Open Statistical Data (LOSD) Pipeline, several
reusable capacity-building assets were generated. These assets will support the ongoing use and
adoption of the LOSD Pipeline in the future.
2.1 Capacity Building Platform
We have leveraged the established e-learning platform framework, Moodle, for the LOSD Capacity
Building Platform. That is to provide maximum ease of use and support for future improvements. The
framework enables flexible assembly of courses and learning materials including multimedia and
interactive components, allows effective authentication and manages access control. We leveraged
1 http://ec.europa.eu/eurostat/web/ess/about-us/ess-vision-2020/implementation-portfolio#DIGICOM 2 http://ec.europa.eu/eurostat/web/ess/digicom
LOSD D5.2
4
Moodle to deploy learning content within six dedicated courses - three introductory courses and three
hands-on courses. The courses are composed of text (explanatory content and instructions), images
(screenshots of tools) and videos (for demos and tutorials). Some of the courses included extra quizzes
to boost the user-engagement. Information about the platform has been advertised and disseminated
before the hackathons and webinars in order to ensure that the stakeholders have enough knowledge
and skills to effectively participate in the activities envisaged.
Moodle has been installed on a cloud based server and made available at this link
http://losd.staging.derilinx.com/moodle/
Figure 1: Screenshot from LOSD Capacity Building Platform (Moodle)
The platform has been running throughout the duration of the project and can be maintained beyond
the project if required. A dockerised version has been developed, so that it is easy to deploy and run
the platform in different locations in different contexts.
LOSD D5.2
5
2.2 Learning Materials
During the course of the project, we have generated rich learning materials including the Introduction
to Linked Data and Linked Open Statistical Data, and LOSD Pipeline specific materials. All the materials
are available on the Capacity Building Platform. Moreover, the materials have been released in the
universal PDF format for download and offline exploration. These materials can be leveraged in the
future capacity building efforts and to replicate the training process conducted in the project with
different NSOs or other stakeholders.
The content on the platform is structured into specific courses targeted to stakeholders of each of the
use-cases:
1. Introduction to Linked Data - The course is intended to provide a basic introduction to the topic
of Linked Data.
2. Introduction to RDF This course is an introduction to RDF - The course is simply a test for this
training platform.
3. LOSD Standards and Tools - This course introduces participants to existing LOSD standards (such
as RDF, SKOS, DCAT, Data Cube etc.) and technologies (such as Virtuoso, SPARQL).
4. LOSD Pipeline: LOSD Mapping - The course is intended to cover the principles of using Juma for
creating data mappings.
5. LOSD Pipeline: Data Cataloguing - The course is intended for all group of users and deals with the
use of CKAN as a data catalogue platform
6. LOSD Pipeline: Data Visualisation - This course introduces the set of support services and
applications that are available as part of the LOSD platform. It is mainly focused on the analysis
and visualisation of LOSD.
3 LOSD Capacity Building Services
3.1 LOSD Webinars
The LOSD Webinar I and II were streamed live via the Facebook platform and now are also available
online on Facebook. The main goal for the webinars was to provide a step-by-step overview of
concepts and tools to the stakeholders in a narrative form. To view those materials there is no need
for having a Facebook account (this is public content).
Webinar I can be found at:
https://www.facebook.com/newsegov/videos/758473811173169/
LOSD D5.2
6
Webinar II can be found at:
https://www.facebook.com/newsegov/videos/254843405400883/
Both webinar videos can be considered a good summary and training asset that complement the
material on the Moodle platform.
LOSD D5.2
7
3.2 LOSD Hackathons and Meetups
In the project, we have supported number of hackathons and meetups:
● Hackathon in Dublin (February 2018)
The hackathon in Dublin was the first opportunity for the members of the consortium to meet
with different representatives of the NSO partners. At this meeting, the work that the
consortium would be partaking was specified and requirements gathered. Three use cases
were proposed and agreed upon and a subset of datasets was chosen for the consortium to
focus on.
● Hackathon in Paris (September 2018)
It was a mid-project hackathon with 50+ participants from various horizons. Six teams working
on producing and consuming Linked Open Statistical Data. Members of the consortium
attended for two days. There they participated in some hackathon activities and had the
opportunity to demonstrate and get some user feedback on the different tools used in the
first version of the LOSD Pipeline (http://losd-data.staging.derilinx.com). A dedicated room
was made available to the consortium for participants to test the platform, in parallel to other
hackathon activities. As a result, only some people choose to test the platform at this stage
and feedback received was very minor.
● Meetup in London (January 2019)
This two-day workshop theme was 'Linked Open Statistical Data in the European Union'.
Practitioners from statistical institutes, NGOs and the private sector came together to
exchange lessons learned and collective ways forward: (i) provide high-level view of the
progress on creating Linked Data ecosystems for government statistical data and (ii) discuss
the challenges and opportunities that arise from this work.
● Hackathon in Rome (March 2019)
Members of the consortium attended the Rome hackathon for one day. During this
hackathon, the consortium was provided with a dedicated slot in the agenda to present to all
participants. There, the consortium presented in detail all aspects of the LOSD Pipeline in the
context of the three use cases. While going through each use case, participants were
encouraged to follow along in some of the activities while using the platform. At the end of
the presentation, a fruitful question and answer session was had, where the functionalities
and limitations of the LOSD platform were discussed. This hackathon provided a great
opportunity for the consortium to not just present the work that was done to date, but also
get the hackathon participants to really engage with the platform and provide essential
feedback.
● Final workshop in Sofia (May 2019)
The final ESSnet conference in Sofia was a successful event, where the outcomes of the project
were presented. An overview of the LOSD Pipeline was shown, as well as additional
information around challenges that the NSOs faced, how these were addressed throughout
the project. We also discussed the key lessons learned and recommended next steps for the
LOSD Pipeline. The NSO teams that we have been working with, as well as the wider ESSnet
LOSD D5.2
8
community from other NSOs, the Eurostat team, and the local Bulgarian Open Data Unit
attended the event.
3.3 GitHub
The LOSD Pipeline is available as Open Source and can be accessed at the LOSD GitHub repository
available at https://github.com/LOSD-Data. This enables the local adoption, use and extension of the
LOSD Pipeline, as is being done be ISTAT, the Italian NSO.
3.4 Slack Channel
Stakeholders involved in the project were using dedicated Slack channel for the communication and
announcements. It is available at https://los-essnet.slack.com.
The Slack channel provided a great means of communication for members of the project. Usage was
infrequent in the early stages of the project. However, as time went on and more NSO personnel began
LOSD D5.2
9
interacting with the LOSD platform - the slack channel became an important place for the consortium
to interact with, answer questions and help the NSO personnel with the use of the platform.
4 LOSD Capacity Building Lessons Learnt
4.1 Capacity Building Approach
Our Capacity Building approach encompassed several different techniques. In particular we have
relied on the Capacity Building Platform as a major resource and convergence point for users to find
essential instructions and related materials. The second important component was direct interactions
with users - via email, a Slack Channel and face-to-face meetings where experts guided the users on
their journey with the LOSD Pipeline. The LOSD Capacity Building Platform has been well received by
the pilots and no major issues were reported in using the platform. We present some basic statistics
in terms of use of the platform below:
• Total Pages views: 2107
• Registered Users: 25
• Views per specific course:
o Introduction to Linked Data: 82
o Introduction to RDF: 107
o LOSD Pipeline: Data Cataloguing: 395
o LOSD Pipeline: Data Visualisation: 194
o LOSD Pipeline: LOSD Mapping: 441
o LOSD Standards and Tools: 220
LOSD D5.2
10
From the statistics collected it is apparent that the LOSD Capacity Building Platform experienced rather
moderate use. The total page views of 2107 presents an average figure considering the number of
stakeholders in the project. The most popular content included the Data Mapping and Cataloguing
courses. These were followed by Standards and Visualisation. The least used resources were the
introductory courses. A direct conclusion can be drawn that users prefer hands-on practical material
rather than more theoretical introductory courses. Also, the most popular course - LOSD Mapping was
also the part of the project which stakeholders reported to be the most challenging and required the
most support from experts. Therefore, the most challenges identified with that part of capacity
building can also span from the biggest interest & popularity of that material.
Users pointed out that more detailed materials or more assistance (including automated assistance)
could be provided so that users do not need to look for expert support to work with the LOSD tools.
That could be achieved by:
1) More detailed materials - more distinct steps in the materials, more help videos and more
explicit, step-by step tutorials or supporting technologies such as chatbots or ‘Interactive
Tooltips’ implemented in the tools that could ensure the highest level of automated assistance
in learning new tools - discussed in further part of this section
2) More interactive approach - more Moodle interactive features could be also applied to the
courses. The standard courses and portable PDF versions showed to be insufficient and more
engaging mode of training would be desired. In particular, interactive elements could be
implemented to ensure more active & rewarding process of user engagement. Therefore,
users could be evaluated and their learning process could be validated as they explore the
learning materials provided.
4.1.1 Virtual Assistant
Drawing on the conclusions and user feedback received, in particular reflected in the need for expert
support, a major recommendation is the need for more explanatory capabilities. For example, we
could explore using emerging conversational technologies such as Chatbots and Virtual Assistants to
help participants to reach the materials they are looking for. These chatbots could simulate experts
and could serve as agents supplying relevant materials to the user. They could also directly answer
queries and in more advanced versions, process the natural language queries by the users and give
them explicit answers. That would require the application of more advanced, AI-driven solutions and
training sets for conversations.
4.1.2 Tool Tours
Another possible improvement to the learning experience could be ‘Tool Tours’ that would guide users
through an application using step-by-step navigation within the tools so that they can become familiar
LOSD D5.2
11
with the workflow without needing to interact with a human agent. We present an example solution
below (Source: https://www.appcues.com/blog/tooltips)
The Interactive Tool Tours are one of the best ways to ensure step-by-step learning as user engages
with the tool. The approach alleviates the often faced disconnect between the tool and the learning
material. Instead of ‘jumping’ between the tool and the tutorial or readme file the user is guided from
within the tool and learns the workflow more effectively. That approach however requires a direct
implementation of learning materials and suggestions directly within the tool rather than separate
learning resources. That imposes significant extra development effort on the tool developers to design
a special ‘Happy tour path’ or several of those to ensure all the use scenarios covered. However, the
solution offers one of the best and universal learning experiences.
4.2 Interaction with Stakeholders
One of the major forms of interaction with stakeholders in the project was through webinars,
hackathons and events. While webinars apply digitally-mediated dialogue, the hackathons and events
ensure effective face-to-face interactions. The obvious limitation of the digital channel communication
is compensated by possibility to reaching out to broader audience and removing many of the
participation limitations - a need to travel.
Direct communication with participants proved to be very effective. The stakeholders appreciated the
help of the experts. In terms of digitally-mediated dialog we had experts interacting with stakeholders
via e-mail and the Slack channel. More importantly however, we have organised and held two
webinars scheduled six months apart. The first webinar dealt with the Introduction to Linked Data,
while the second one took the participants on a hands-on journey through LOSD Publishing Pipeline.
LOSD D5.2
12
The separation in time was perceived by stakeholders as very positive approach marking major
milestones in the project. Here we present some statistics in terms of audience and participation:
● The first webinar attracted 27 participants registered via Eventbrite and generated over 540
views on Facebook.
● The second webinar attracted 71 (increase from 27 in first webinar) participants registered via
Eventbrite and generated over 1.3K views on Facebook.
The materials were also put on YouTube for greater availability however they did not generate
significant views (less than 100). The advantage of Facebook dissemination especially in the second
Webinar came with directed paid campaign where we explicitly disseminated materials to people
interested in Open Data in Ireland and Brussels.
The webinars were evaluated using relevant feedback forms - we leveraged popular Google Forms for
that. The solution even though easy registered very little feedback and we had to send several
reminders before collecting some critical mass of data (about 7 responses in total). We believe that
explicit introduction to webinar evaluation could help in the future for collecting richer results. In
particular giving the participants more ownership of the evaluation process should make the
stakeholder more engaged.
The major feedback received indicated that there is a strong need for more interaction with the
presenters and extra interactive session at the end. Therefore, future webinars would include more
interactive elements and dedicated Q & A at the end of the session. In terms of technical support for
interactivity the current setting using Facebook as the primary channel and Goto Meeting as a
secondary is sufficient. The interactivity can be provided via full-voice feature in Goto Meeting or chat
& comment in Facebook.
One practical lesson learnt is that the numbers of speakers/presenters in the webinars should be
limited. First, it is difficult to fit a large number of people in front of the camera. Second, if speakers
have little to say the speaker / presentation change creates too much overhead to justify extra speaker
participation.
Figure 2: Preparation to Webinar Transmission - Busy Green Screen Studio at Insight
LOSD D5.2
13
While running the webinars we experienced some technical difficulties related to failing internet
connection. The network infrastructure interruptions created a major obstacle to webinars. Therefore,
it is essential to ensure stable internet connectivity at all times not to lose the engagement with the
audience (4G backup connectivity could be a good option).
Another improvement could be applied to ensure larger ‘general public’ participation. We started an
online campaign within two weeks or so before the webinars. The numbers are satisfactory however
there is still a room for improvement. In particular we the webinars could be scheduled much more in
advance to have relevant dissemination strategy implemented far ahead of the webinar to allow
enough time for participants to include it in their busy schedules.
5 General Recommendations Our main recommendation would be to make sure that the capacity building services are well aligned
to the specific stakeholder needs, but also that they can be easily adapted to changing needs. In
particular, the provider of the services must ensure maximum flexibility as the needs may shift during
the service delivery as stakeholders learn about the topic (such as LOSD) and new questions,
challenges and needs emerge. Therefore, the materials and services offered must be extensible and
portable.
The best example is the LOSD mapping exercises, which proved to be a difficult task, yet one of the
most critical elements of capacity building. Users required significant support from experts on those
exercises and presented important value to the stakeholders. The mapping tools appeared to be
difficult to use and the learning processes provided still required some human intervention. Therefore,
future capacity building services should identify the critical elements as soon as possible and ensure
very strong support for those issues. That support could be automated as in the examples given on
Interactive Tooltips and Chatbot. The initial extra resources spent on providing the training
automation included in the tools themselves will definitely pay off when more stakeholders and more
users are to be trained.
In terms of mode of communication such as face-to-face interactions, hackathons, e-mail, Slack
channel and webinars, the in-person interactions are always favoured. However, due to limited
resources the best trade-off in communication is probably a well-supported Slack Channel and
interactive webinars. This is mainly due to the synchronous nature of communication. Users require
assistance live, as they explore the specific topic or use the tool. The e-mail communication and
‘passive learning materials’ such as PDF can work well as additional support but should not be in the
core of the capacity building delivery. That is reflected in particular in the feedback received on strong
need for more interactivity with experts which is understandable considering the fast-changing needs
and requirements that emerge as stakeholders explore the topic.
5.1 NSO Network
We propose that following on from the LOSD project, an LOSD NSO Network should be established for
increased peer-to-peer learning. The concept of the NSO Network is to support a community of Linked
Data practitioners in NSOs, including those from the Central Statistics Office, Ireland; INSEE French
Statistical Office; ISTAT Italian Statistical Office; and National Statistical Institute, Bulgaria. Such a
group, focussing initially on skills development, knowledge sharing and practice development, will act
as an (informal) Linked Data practitioner community across NSOs. Activities could include
LOSD D5.2
14
development of and participation in webinars, seminars, training, project design and planning. This
community will be the ‘go to’ people in various areas relevant to the LOSD Publishing Platform (CKAN,
JUMA, SPARQL, etc.) They will maintain relationships with non-NSO Linked Data and Semantic Web
stakeholders such as Derilinx and university-based researchers. They will be the key NSO capability
that will ensure the sustainable evolution of the Linked Data platforms
6 Conclusions In this document we presented the lessons learned from our experience in delivering the LOSD
capacity building services to project stakeholders. We have identified the strong points of delivery and
areas where there is space for improvement. We have also presented some further recommendations
on possible service delivery in the future.
In particular, we have elaborated upon the challenges in enabling the LOSD stakeholders (especially
statisticians and other individuals working with data) to use the LOSD Publishing Pipeline effectively.
Those relate mainly to the need for better alignment and merging of the training content and training
approach with the tools and more step-by-step flow to minimise the need for expert interventions.
The best candidate solution in that sense seems the ‘Interactive Tooltips’ or ‘Tool Tours’ integrated
and supplied with the tools to users. Also, automation in terms of FAQ would be also desired.
Additionally, more intense face-to-face interactivity would be encouraged in place of development of
‘passive learning materials’ and tutorials. The scope of the in-person interactions will be of course
limited by the funds available and digital training should be always considered an option. In that case,
interactive synchronous communication such as Slack Channel or Live-Webinars with strong
interactivity support are encouraged.