Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public...

37
1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people PROJECT FINAL REPORT ONLINE PUBLIC VERSION (updated Oct 5 th 2017) Grant Agreement number: FP7 - 611621 Project acronym: BlindPAD Project title: Personal Assistive Device for BLIND and visually impaired people Funding Scheme: FP7-ICT-2013-10 Period covered: from Jan 2014 to May 2017 Name of the scientific representative of the project's co-ordinator, Title and Organisation: Dr. Luca Brayda Researcher Fondazione Istituto Italiano di Tecnologia Tel:+3901081 72 205 Fax: E-mail: [email protected] Project website address: http://www.blindpad.eu/

Transcript of Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public...

Page 1: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

1 BlindPAD - Final Report – online public version 30.09.2017

Personal Assistive Device for BLIND and

visually Impaired people

PROJECT FINAL REPORT

ONLINE PUBLIC VERSION (updated Oct 5th 2017)

Grant Agreement number: FP7 - 611621

Project acronym: BlindPAD

Project title: Personal Assistive Device for BLIND and visually impaired people

Funding Scheme: FP7-ICT-2013-10

Period covered: from Jan 2014 to May 2017

Name of the scientific representative of the project's co-ordinator, Title and Organisation:

Dr. Luca Brayda

Researcher

Fondazione Istituto Italiano di Tecnologia

Tel:+3901081 72 205

Fax:

E-mail: [email protected]

Project website address: http://www.blindpad.eu/

Page 2: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

2 BlindPAD - Final Report – online public version 30.09.2017

The BlindPAD project For visually impaired people it is difficult to digitally get graphical contents increasingly conveyed

through sight. The sense of touch can potentially bridge the gap, as it is crucial -in absence of vision

-for understanding abstract concepts and acquiring information about the surroundings. Examples

are learning at school and developing mental maps in orientation and mobility daily tasks. However,

available touch screens have limited or no tactile feedback at all. The potential and the market of

tactile displays are largely unexploited, although there is a clear demand from users: these devices

need to become more versatile, cheaper, portable and socially acceptable. This project has made

graphical contents accessible through touch by building and field-testing a Personal Assistive Device

for BLIND and visually impaired people (BlindPAD). BlindPAD puts veridical touch-based information

into the hands of users, exploiting and enhancing their residual sensory abilities. The BlindPAD has

explored several technologies, compared in terms of actuation force, resolution, safety, power

consumption and reliability.

By adopting a user-centred approach within an accessible and usable ecosystem, we have assessed,

with serious games, how the BlindPAD can help visually impaired people in two paramount use

cases: touch-based learning of symbolic content at school age; orientation and mobility skills indoor.

We have shown that our programmable tactile display increases, in persons with sensory deficits,

the spatial working memory, the mathematical abilities, the capacity to find one’s own position in

an unknown space and spatial knowledge in general, beyond current rehabilitation protocols.

BlindPAD will be a personal, portable and cheap solution to improve knowledge and independence,

thus increasing chances of employment, of social inclusion and, ultimately, of a better quality of life.

Page 3: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

3 BlindPAD - Final Report – online public version 30.09.2017

Innovation

Figure 1 What is BlindPAD: a system comprising three main components. Combining a haptic display, a software that delivers graphical content and a series of exercises has been never done before.

The output of the BlindPAD project that sums up the foreground is the virtuous and innovative

combination of three components (see Figure 1):

• a haptic display, that stimulates the sense of touch with 192 programmable moving pins

• a piece of software that allows to translate visual concepts into tactile representation

• a series of exercises about 1) geometry and 2) orientation that are implemented on the

haptic display through the tactile graphics software.

Together, these three elements are the unique novelty of the project, that no previous research or

industrial activity has led to such stage. We detail each piece in the following.

=

Haptic

display

Tactile graphics software

+

Rehabilitation exercises

+

Page 4: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

4 BlindPAD - Final Report – online public version 30.09.2017

1 Executive summary Visually impaired persons, especially children, are locked out from digital information and

communication because technologies are heavily oriented to visual user interfaces and provide

increasing amounts of information graphically. Blind people need to use the residual sense of touch

to understand information, something already achieved for digital text – thanks to Braille – but not

at all achieved for graphics. Therefore, the need of achievable tactile technologies facilitating their

better inclusion into modern society and work becomes very important. Fabricating a tactile tablet

for blind persons has been a challenge for decades: the idea being that of transforming a visual

concept into a tactile representation is well known as ‘sensory substitution’. Much like ‘pixels’, one

digital image can be formed by a grid of small tactile pins (‘taxels’) that can be programmed to be

‘up’ or ‘down’, therefore forming a bas-relief that can be sensed and understood with the hands.

However, making a dense array of taxels with sufficient force and displacement to be easily felt by

any user, and that is also low power, fast and compact is a major technological challenge. Due to

the complexity of drive electronics or the lack of performance of actuators, none of previous

technological solutions were shown to be scalable, have sufficient performance, and be portable.

Here, we have successfully built and field-tested a system comprising a new tactile tablet, a software

translating images into tactile representations and a series of exercises that together make digital

graphical information accessible to blind and visually impaired persons. Our tactile display is 12 cm

x 15 cm in size, consisting of 192 electromagnetic, independent taxels. Each taxel is very fast in

changing state (up/down), allowing static and moving patterns to be displayed. Together with our

software that renders tactile images, we verified that the BlindPAD system is effective on blind and

visually impaired people, especially at the developmental age. We considered two paramount use

cases: learning mathematics and learning unknown spaces from maps: we have shown that

BlindPAD successfully trains visuo-spatial working memory, complex mental operations,

mathematical concepts and helps to picture maps of unknown rooms, supporting the ability to find

one’s own position in a real environment. Our results contributes both to the research field of

material science, as we have shown that it is possible to build arrangements of small element able

to deliver high forces but with small components, to the field of experimental psychology and

cognitive neuroscience, in which the potential abilities of persons with sensory deficits are still

debated and to a certain extent underestimated or at least unexploited; to the field of computer-

human interaction, since tactile feedback on non-flat screens has deserved little attention; in

general to the field of computer-assisted rehabilitation, where no standards exists and the access

to digital information can be a breakthrough in making rehabilitation practices semi-autonomous.

The BlindPAD system was built in three years from scratch: it now comprises a hardware tactile

device, a software running on most PC and tablets, and a series of exercises that stimulate with

touch the abilities linked to spatial working memory, spatial processing, logics and mathematical

reasoning. The system, as it is, can be used in rehabilitation centers, with potential very high impact

on rehabilitation practices. Our prototypes are rather easy to reproduce, therefore the technology

can be exploited by transferring it to a start-up. With a relatively low effort in terms of engineering,

it can become a product.

Page 5: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

5 BlindPAD - Final Report – online public version 30.09.2017

2 Context and objectives The novelty of this proposal is the construction around users’ needs of the first, truly personalised

and portable tactile display able to deliver graphical content to hands and fingers with a veridical

three-dimensional tactile feedback. To learn graphical concepts, visually impaired people are

generally forced to use large sheets of swelled paper, bulky or heavy physical three-dimensional

models, large Braille bars to purposes they are not built for. In addition, none of the solutions used

today can be easily personalized. Similarly, no portable solution exists yet which delivers refreshable

tactile maps.

The project started from the idea to build a tactile surface as big as a tablet computer. To achieve

this, the approach was to put into competition several technologies. The strategy to investigate two

different promising technologies aimed to reduce the risk to achieve the target objective of a cost-

effective, light-weight tactile display. These two innovative technological approaches were based

on electro-active polymers and electromagnetic actuation: they shared many higher level

components (interface, addressing, etc.), and were designed from the start to allow scaling, in this

project, from hundreds to even thousands of taxels, with potential further up-scaling at relatively

low costs. Both relied on low-voltages, but used different physical actuation principles. By having

these two technologies, we significantly reduced risks with only a small increase in person/months.

After the first level of research on both technologies, the best suitable regarding portability and

usability was selected for further investigation at a defined decision point in the project. The final

adopted technology aims at helping blind and visually impaired persons in at least two fundamental

tasks of their lives. The first is education, because touchable graphical content will play a role similar

to what images do for sighted people in any learning activity involving abstractions and symbolic

representations, which are of paramount importance in the developing age. The second in

orientation and mobility (O&M), because touchable maps will help the development of cognitive

representation of the world, able to be updated, zoomed, and tailored to specific sensory needs.

Both scenarios are fundamental in social inclusion, since visually impaired young persons will be

potentially able to follow, at their fingertips, the same information visually presented to their

schoolmates on the blackboard, due to the portable nature of the device; adults, instead, will exploit

state-of-art geolocalization technologies and software to find and communicate with their sighted

fellows, either by building mental maps before leaving home, or directly carrying the device with

themselves.

To reach this goal, we decided that this portable stimulator might have the form factor of a tablet,

thus suitable for interaction with tactile objects using few fingers up to two hands. The

microactuated personal assistive device proposed, called BlindPAD (Personal Assistive Device for

BLIND and visually impaired subjects), allows the dynamic control of the tactile information by virtue

of an electronic board, wirelessly connectable to standard mobile devices, such as tablets and

smartphones. Graphical content, stored either locally or remotely, is then haptically rendered and

displayed. To enhance the usability of BlindPAD, the interaction mode can be extended with state-

of-the-art touch sensitive technology, allowing to read the fingers positions on top of the novel

surface. This approach opens the way for interactive applications making the information access for

visually impaired even more effective.

The specific objectives of the project are:

Page 6: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

6 BlindPAD - Final Report – online public version 30.09.2017

Objective 1: to investigate two novel and competing tactile feedback technologies and to select the

most effective in terms of 1) usability, 2) portability, 3) scalability for large array size and 4) suitability

for low-cost mass production for the development of the proposed BlindPAD display. The first

objective is technological. The BlindPAD display is composed of three parts: a tactile interface, i.e.

a bi-dimensional matrix of mm-scale taxels, a connected electronic board driving the actuators and

interfacing with a computer or smart phone, as well as a state-of-the art touch sensitive technology.

In addition, the position of one or more fingers is tracked to allow interaction with the touchable

objects. Among the different technologies, the best one is chosen to build the final demonstrator.

The partners refine their technology given portability (power, mass, robustness) constraints. The

limits of scaling are studied to understand realistic taxel size given both processing and drive circuit

constraints. For large arrays, attention must be paid to achieve uniform properties over the entire

array (both to have the same force for all taxels, but also to allow simpler drive circuits by not

requiring taxel by taxel calibration). The assembly scheme is planned to be engineered for

manufacturability. Lower-cost manufacturing technologies are studied, in view of an eventual

technology transfer to manufacturing partners.

Table 1 Technological objectives of the project BlindPAD

Har

dw

are

Build a single taxel interface: only one pin, but with several technologies

Build a low resolution grid of such interfaces: arrange several pins together, at

increasing resolutions (row and columns): 2x2, 4x4

Build a high resolution grid of such interfaces: arrange many pins together, at increasing

resolutions (row and columns): 12x16, 32x24

Choose the most affordable technology and build a tablet-sized haptic display

Build the electronics that drive the haptic display

Make the display portable

Build a sensing surface, to detect the position of the fingers on the haptic display

Soft

war

e

Build a human-computer interface allowing to draw arbitrary images on a grid

Allow the software to receive input from both scientific software tools and

unexperienced users, and send the output to the tablet-sized haptic display

Objective 2: The second objective is social inclusion, obtained by empowering rehabilitation

programs with a tool to increase spatial abilities faster and closer to that of sighted persons. The

second objective is to test the BlindPAD in two use cases.

The first use case was learning of basic graphical content in mathematics (geometry), with symbolic

dictionaries (object shapes, iconic and visual conventional signs translated into touchable content).

It was demonstrated that certain exercises can be performed with a tactile tablet at least as well as

current rehabilitation techniques, which have the drawback of requiring constant attention from

Page 7: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

7 BlindPAD - Final Report – online public version 30.09.2017

practitioners and a tremendous amount of time to prepare them (up to 50% of rehabilitators is

spent in preparing personalized stimuli for blind children). Such exercises include basic tasks that

are generally performed at the school age, where the need of graphical elements is stronger:

establishing which tactile symbols made by few raised pins can be well recognized, then recognizing

which tactile symbol among a set of distracting elements is correct, then memorizing one or more

spatial disposition of elementary tactile symbols, then performing the difficult task of mentally

rotating a shape (something that will be useful later on when studying a map), or finally estimating

distances among tactile symbols.

The second use case is Orientation and Mobility: visually impaired subjects are limited by the lack

of reference points: for every novel environment people must have a representation of space in

their mind (i.e. a mental map). The BlindPAD wants to offer a representation of simple top-view

maps of rooms. We answer to fundamental questions such as “Where am I?” and “How do I get

there?”. Here, BlindPAD is used as a complementary tool for O&M programs currently addressed to

deprived subjects who already developed geometrical skills necessary to process maps and

orientation instructions. Tests are intended to be measurable: way how well a tactile map can be

acquired and memorized? Can memorizing a tactile map be useful when exploring an unknown

space? Interaction with the real landmarks, previously touched on the BlindPAD, can be used to

check, with help from O&M experts and standard spatial tests, how well the mental map was

developed, therefore deriving how well the BlindPAD was able to display it.

Importantly, the serious games will be experienced not in labs but in contexts which will be familiar

for the subjects, and which will be comfortable and plausible for the end-users (e.g. rehabilitation

centres), thus emphasizing the peculiarity of the BlindPAD as a personal device.

Table 2 Cognitive objectives of the BlindPAD project

Per

cep

tio

n

and

co

gnit

ion

Establish minimum criteria to say that an array of pins is well perceivable

Compare the competing technologies and choose the best one in function of user needs

Compose a set of tactile images: a dictionary that can be used in serious games

Reh

abili

tati

on

Improve the learning of graphical concepts in blind and visually impaired people

Improve orientation and mobility skills in blind and visually impaired people

Comparing and complementing existing rehabilitation protocols with BlindPAD

Page 8: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

8 BlindPAD - Final Report – online public version 30.09.2017

3 S&T results/foregrounds

3.1 From many to one technology: the BlindPAD EM prototype

In this section we list the two main technologies that showed an advancement in the project (Shape-

Memory Polymers and Electro Magnetic) and were patented, together with the electronic

components that were part of the prototypes (electronic board and sensing surface). The final

adopted technology was the Electromagnetic because it granted high forces, high displacement and

reliability.

3.1.1 The Shape-Memory Polymer Technology

Shape Memory PolyUrethanes (SMPUs) are one type of shape memory polymers. Their main feature

is their ability to lock a temporary shape and recover back the permanent one anytime upon simple

thermal actuation. Shape-change temperature can be chosen from 35 to 85 °C.

For BlindPAD, the objective is to benefit from their enormous change in Young's modulus with

temperature to intrinsically implement latching and to selectively actuate individual taxels in the

array. The proposed unique and novel concept, which enables larger scale manufacturing and faster

refresh rate, is to integrate a stretchable heater on each taxel and to use a common pneumatic

source to move all taxels (i.e. one pump drives all the taxels, but only those that were heated move).

There are two stable states when the membrane is cold: “UP” or “DOWN”.

We emphasize that the SMP technology allows to change the tactile pattern in a differential fashion:

only the taxels which change from one tactile image to the next are eventually physically modified

(i.e. heated).

Page 9: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

9 BlindPAD - Final Report – online public version 30.09.2017

Figure 2: Pictures of each separated parts used to drive in the 32x24 SMP prototype. The hardware (actuators + pumps), the driver board and the software are show.

Figure 3: Picture of the 32x24 SMP prototype with the pin interface (from IIT) and the copper plate as it would be used for

perception tests to speed up the refresh time.

3.1.2 The ElectroMagnetic actuation

Electromagnetic (EM) actuation offers particularly appealing performance in terms of force,

deflection, bandwidth, scaling, integration, robustness and portability. This haptic display was

developed with low cost in mind, and consists of a matrix of small magnets between two printed

circuit boards (PCBs). We have successfully built a technology that exploits electromagnetic force to

move the single taxels of a tactile tablet.

4x4 arrays: The EM technology was then scaled to a 4x4 array, with further advancements: a

schematic view of a single taxel is presented in Figure 4a; and key assembly steps of the 4x4 EM

device are presented in Figure 4b to c (8mm case). The device consists of an array of EM-based

Page 10: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

10 BlindPAD - Final Report – online public version 30.09.2017

vertical actuators under a 3D-printed pin array interface. The moving part of each EM actuator

consists of pot-magnet suspended between two elastomer membranes over a multilayer planar coil.

Only the surface of the magnet facing the coil is unshielded. In this way, the magnetic flux from the

pot-magnet assembly is restricted mostly to the region below the magnet. The top and bottom

membranes support the pot-magnets and act as restoring springs.

Final: 12x16 array. Then the EM prototype that was scaled up is a 12 cm x 15 cm size haptic display,

consisting of 192 electromagnetic (EM) independent taxels that move up and down to present

graphical information that is explored by the user using his/her sense of fine touch. Each taxel

changes state (up/down) in under 10 milliseconds, allowing static and moving patterns to be

displayed. This dramatic step forward on the EM taxel technology combines several innovations

regarding magnet shielding method, coil placement, soft magnetic material arrangement, and

operating concept. All these factors taken together, enable the fabrication of a dense array of

electromagnetic actuators that can a) latch in 2 stable positions, b) offer strong holding force, c)

have no cross-talk, d) be fast (few ms), e) give mm-scale displacement, f) be compact, g) be low-

power and scalable to very large arrays.

Figure 4 Diagram and key elements assembly of the 4x4 haptic display. a) Schematic view of a single taxel and the main actuation components. b) Photo of the 6-layer PCB containing the array of planar coils. The PCB is placed on an aluminium plate

supported by four standoffs. c) Top view of he magnetic layer. It is formed by the 16 moving pot-magnets, a perimeter line of fixed pot-magnets, the top and bottom elastomer membranes (not visible) and an acrylic holder. d) A 3D printed pin interface

completes the device as a final layer, and is what the user touches.

The final demonstrator implements a novel working principle: the bi-stable EM actuators. In the

sequence a single taxel switches from the “UP” state to the “DOWN” state and vice versa. When the

taxel is up the lateral shielded magnet is attracted by the closest latching plate, i.e. the top one. Due

Page 11: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

11 BlindPAD - Final Report – online public version 30.09.2017

to this interaction between the magnet and the latching plate, the taxel remains in the up state

without any power consumption. To switch to down state, a short actuation pulse is applied to the

top and bottom coils, with the direction of the electrical current properly chosen so that the top coil

repels the magnet while the bottom coil attracts it. During the magnet way down, it’s interaction

with the bottom latching plate increases, up to the point where the actuator switches to the second

stable position. The transition takes only a few milliseconds, when the actuation current is on. Same

as in the up state, no electrical current is need to maintain the magnet in the down state. To switch

from the down to up taxel state, similar actuation pulse is applied but with the electrical current in

the inverse direction.

3.2 Everybody can draw tactile graphics: the PadDraw software

PadDraw is developed within the project and connected directly to the BlindPAD and haptic devices

(like Hyperbraille). As part of the negotiation process this software is discussed how to offer the

software. As part of the full service offer, PadDraw will be offered, supported and also training on

“How to use PadDraw for BlindPAD” are discussed. In another scenario a lightweight PadDraw will

be offered as free service with some basic API functionalities, to support educational training and a

creative vision exchange. The latest version of PadDraw runs on Windows and MacOS. Windows is

also available as tablet and mobile version. Therefore, PadDraw will also run on such device and can

be used in mobile scenarios. In PadDraw every object you can add to an image and every effect you

can apply to the image is basically a component. When a component is added to the image it is

stacked onto the other components that are already contained in the image. This way a component

can also be regarded as a layer inside the image.

Components are basically divided into two groups:

• Drawable components

• Effect components

Every component has two core properties with the following characteristics:

• ID: integer, unique, automatically set by PadDraw

• tag: string, may be assigned to multiple components, set by user1

The application mainly consists of four major modules describes below in more details:

• Tactile image composer module

• Tactile slideshow composer module

• Remote control module

• Hardware interaction module

1 Please note: tags can only be composed of alphanumerical characters (a - z, A -Z, 0 - 9). All other characters are forbidden!

Page 12: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

12 BlindPAD - Final Report – online public version 30.09.2017

Figure 5 Main Window of PadDraw with Components Inspector and Undo

3.3 Testing BlindPAD prototypes

We have compared different prototypes of BlindPAD, using a common technological setup. The goal

was to choose the most suitable technology, to be scaled up in the last year of the project. The setup

consisted in a matrix of 16 tactile actuators, organized as a 4x4 matrix. Perception tests involved

participants of different age and kind of visual disability. We compared the technology based on

ElectroMagnetic taxels to that based on ShapeMemoryPolymers. Tests involved perception and

recognition of tactile images under various constraints related to tactile vocabulary size and

cognitive resources involved, as well as usability questionnaires. Tests reported similar results

between technologies with results very significantly higher than chance when people were asked to

identify one over 12 tactile symbols. Interestingly, some symbols were perceived better than others,

with higher performance on more reminiscent symbols.

Success rate in detection tasks exhibited performance higher than 95%. Success rate in

discrimination of different tactile images exhibited performance of 90%. Success rate in a more

challenging identification task of tactile images on a wide vocabulary exhibited performance of

around 70%, i.e. 8 times higher than chance.

Although EM and SMP are very different in terms of maximum force and displacement (EM having

more displacement and less force, SMP having less displacement and more force), both

technologies were effectively perceived by participants, with personal preferences for either

technology.

Page 13: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

13 BlindPAD - Final Report – online public version 30.09.2017

3.4 Programmable tactile displays increase spatial abilities of visually impaired people

3.4.1 Case study 1: increasing mathematical abilities with BlindPAD

We involved populations of sighted, low vision and blind participants. As for tactile displays, we

adopted several technologies: paper-based raised line drawings (the state of the art in rehabilitation

centers), a commercial pin-array display (Hyperbraille) and our prototype of pin-array display

(BlindPAD).

A first study (see Figure 6) was motivated by the fact that blind people have severe issues in

manipulating and integrating visuospatial information, and in memorizing a high number of data

(see Figure 7) This is relevant for the success in different scholar disciplines (geometry, mathematics,

etc.) as well as some working careers (e.g. science). We verified whether visuospatial working

memory can be trained in sighted, blind and low-vision youngsters using pin-array tactile displays.

Our results suggest that visuospatial working memory can be successfully trained in visually

impaired youngsters at least in tasks characterized by a low cognitive load, with trainings lasting

about one hour, once per week for four weeks.

The results are part of the following publication (open access):

Leo, F., Cocchi, E., & Brayda, L. (2017). The Effect of Programmable Tactile Displays on Spatial

Learning Skills in Children and Adolescents of Different Visual Disability. IEEE Transactions on

Neural Systems and Rehabilitation Engineering, 25(7), 861-872.

Download the PDF here DOI: 10.1109/TNSRE.2016.2619742

Figure 6 A. Experimental setup with the Hyperbraille display on the left side and the PC running the PadDraw software on the right side. The picture shows an example of trial of the spatial memory task. B. An experimenter and a rehabilitation practitioner

giving instructions to a blind child. A rehabilitation practitioner was always present during the tests involving visually impaired youngsters.

Page 14: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

14 BlindPAD - Final Report – online public version 30.09.2017

Figure 7 A. Schematic of successive events within a trial for Single-matrix task. A speech synthetized voice indicated to the participant that the targets appeared on the screen. After a presentation time of 15 seconds, a voice (not reported in the figure)

asked the participant to remove the fingers from the display. The targets disappeared and a voice asked the participant to indicate target locations. After each answer, the experimenter started a new trial. B. Schematic of successive events within a trial for Double-matrix task. Task events were similar to the Single-matrix task but two different matrices were displayed in sequence. A 2-s interstimulus interval interleaved matrices presentation. Participants were asked to report where were the

targets in the two matrices by replicating the original temporal sequence (i.e. 1st matrix first, then 2nd matrix). After each answer, the experimenter started a new trial.

A second study clarified if Braille-based displays can be used also to convey graphical information.

We determined an ideal set of graphical symbols that can be easily recognized, in function of their

shape and size. Symbols as small as 5 mm in size can be discriminated easily enough to be used in

tactile maps and diagrams. However, our confusion matrices showed that certain shapes of symbols

could not be used without the risk of mistakes. This was the basis for the following studies, but also

for those concerning Orientation and Mobility.On our final BlindPAD prototype (EM 12x16), one

taxel only could be easily discriminated (here a taxel is more than twice larger than a Braille dot).

The final study targets mathematical abilities using the prototype of BlindPAD: a four-week training

was implemented (similar to Study 1), this time with a distance discrimination task on two-

dimensional spatial dispositions.

Page 15: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

15 BlindPAD - Final Report – online public version 30.09.2017

Figure 8: A blind volunteer testing the first approach of wireless EM haptic display at EPFL.

Page 16: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

16 BlindPAD - Final Report – online public version 30.09.2017

4 The potential impact (including the socio-economic impact and the wider

societal implications of the project so far) and the main dissemination

activities and exploitation of results.

4.1 Impact on technology

4.1.1 New actuation systems for haptics and beyond

The BlindPAD project has cast technological results well beyond the state of the art: our solutions

are based on the latching principle, that is the technique of maintaining tactile information without

spending power to deliver the desired force and displacements of the tactile actuators, a crucial

aspects for portability and power consumption issues. An ideal surface, should elicit static pressure

(without vibration), so that mechanical energy would result from the surface shape (possibly with

an on/off actuation) combined with active finger movement and skin stretch (Konio et al., 2003,

Dargahi et al, 2004). The idea behind the Shape-Memory Polymer is smart enough to lead to

solution outside the haptic field. The idea behind the shielding of the pot magnets can be used for

tactile stimulations but also for other actuation systems where multiple actuators are desired (e.g.

robotics).

Our latest prototype presents the following technical specifications:

Figure 9: 12x16 haptic display (15 cm x 12.5 cm), and Raspberry Pi (top right). The user touches smooth plastic taxels, driven by

an array of 192 high speed actuators.

• 192 (i.e. 12 x 16) independently controlled taxels

• 10 millisecond refresh time per taxel (enables rapid update of static images)

• 8 mm pitch between taxels

• 0.8 mm vertical travel

• 200 mN holding force

• latches in both up and down states

• controlled by Raspberry Pi

Page 17: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

17 BlindPAD - Final Report – online public version 30.09.2017

4.1.2 New information systems that translate visual information into tactile graphics

The BlindPAD specifically targets an education use case and a mobility use case, taken as two

representative examples where acquiring and manipulating graphical content is crucial. In fact,

being able to learn as much as others can do and the ability of travelling without barriers are at the

base of any sustainable social relationship. The BlindPAD aims at favouring communication between

visually impaired youngsters, their teachers and mates. At growing pedagogic interest in touch-

based interfaces, therefore relaxing the constraints on the way symbolic representations in scientific

subjects are taught today. Because content distributed over the internet, starting from educational

content which goes more and more paperless, is more and more involved with visual effects, this

project makes tactile rendering central to the information delivery process. The nature of touch

imposes touchable graphical content to be essential; it will foster research not to what is “nice” to

be depicted, but what is “necessary” to be displayed understood. Therefore, the choice of the final

resolution of the device (also driven by technological constraints) cannot be evaluated in absolute

terms, but relatively to what can be done with that specific resolution.

4.2 Main dissemination activities

In the BlindPAD project overall:

• We organized two scientific workshop dedicated to interaction and haptic technologies

• As of June 30th 2017, we published 9 journal papers, 10 conference/workshop papers, while

2 journals are under review and 5 other journal papers are in preparation

• We presented our results in 17 events (congresses, conferences, invited talks), of which 10

were national and 7 international

• We had a BlindPAD booth or presentations at 9 wide-public exhibitions, of which 4 were

national and 5 international. The exhibitions we chose reached a large audience both in the

scientific domain and in the general public domain. We estimated that at least 5000 people

have directly seen our presentations, or touched our demos, or took our flyers.

• In some of our dissemination activities we involved visually impaired persons to explain

children and adults what are their user needs in the modern society.

• The term “blindpad”, without spurious meanings, is indexed by Google as part of 13000 web

pages. Considering that the most famous tactile pin array ever existed – the Optacon – totals

up to 38000 hits, and that the state-of-art Hyperbraille pin array totals up to 4400 hits, we

estimate that this is a very good result.

• BlindPAD has an accessible website, which meets the requirements of W3C. The website

totalized 23000 visits in 3 years, i.e. more than 600 visits per month. Twitter and Facebook

sites were regularly updated.

Page 18: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

18 BlindPAD - Final Report – online public version 30.09.2017

Figure 10 Main dissemination activities: Festival della Scienza 2014, 2015 and 2016, ICT Lisbon 2015, CHI 2017, World of Haptics 2017, Makerstown 2017. Our exhibition were targeting an audience of all ages and particularly attracted children because of the

‘game-like’ attitude of our demos.

Our dissemination was centered on a “verify, then show” philosophy. We favored venues where

people could practically touch our prototypes and check that what we announce is what actually

works. We adopted the strategy: patent, then publish, then announce. This can explain the

relatively low number of followers on Twitter and Facebook.

Page 19: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

19 BlindPAD - Final Report – online public version 30.09.2017

5 List of beneficiaries

Participant no. Participant organization name Part.

short

name

Country

1 (Coordinator) Fondazione Istituto Italiano di Tecnologia IIT Italy

2 École Polytechnique Fédérale de Lausanne EPFL Switzerland

3 Geomobile GEO Germany

4 Istituto David Chiossone onlus CHIO Italy

5 Fundacja Instytut Rozwoju Regionalnego FIRR Poland

6 Ateknea Solutions ATEK Hungary

Figure 11 List of beneficiaries

6 Coordinator contact details

Company name: Fondazione Istituto Italiano di Tecnologia

Address: Via Morego 30, 16163 Genoa, Italy

Tel: +39 010 81 72 205

Project Coordinator: Luca Brayda, PhD

E-mail: [email protected]

7 Project logo

8 Project website

The public website is accessible through: http://www.blindpad.eu/

Facebook: @ blindpad https://www.facebook.com/blindpad

Twitter: # blindpad https://twitter.com/blindpad

Page 20: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

20 BlindPAD - Final Report – online public version 30.09.2017

9 Multimedia Promotional flyer: here

Poster: here.

Video about the motivation behind BlindPAD: https://youtu.be/JpsE1B14xRs

Video about the ‘Time in the dark’ laboratory https://youtu.be/O-JgiGNaPGA

Video about the Shape Memory Polymer technology: https://youtu.be/TMegjjbYLPA

Video about the Electromagnetic technology: https://youtu.be/BmjQZBPGHTQ

Video about the final BlindPAD prototype (1): https://youtu.be/eKKY1b59O6M

Video about the final BlindPAD prototype (2): https://youtu.be/dwaiPnoYf6Y

Video about what to display on the final prototype: https://youtu.be/vMXbKEAkzTw

Video about the overall project results: https://youtu.be/jW00wu7vtiw

Page 21: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

21 BlindPAD - Final Report – online public version 30.09.2017

Part II Use and dissemination of foreground

This section summarizes the plan for use and dissemination of foreground (including socio-economic

impact and target groups for the results of the research). Section A is public, while part of Section B

is confidential.

10 Section A: Dissemination (Public)

10.1 The ‘product’ to be disseminated:

We identified the following features, worth to be ‘sold ‘:

• The technological novelty behind BlindPAD: our approach has always been low-cost for

mass-production. Our message is that any technological effort to fabricate tactile tablets or

similar products must start from commercially available pieces.

• The concepts and material engineering novelties that led us to several technologies: having

patented them, we can disseminate how to build them to the scientific community.

• A novel ecosystematic way of working of scientific staff with rehabilitation practitioners,

where ideas are shared from the two different domains and benefit from diverse points of

view.

• The haptic tablet: our prototype can be 1) used for further research projects, 2) used in

rehabilitation institutes with the current state of the art, 3) re-engineered to obtain finer

resolutions or wider sizes. This is part of the foreground ready for exploitation.

• The need of standards in tactile graphics when accessing visual content: BlindPAD has

created PadDraw a human-computer interface that other researchers can use to create

tactile representation of visual content at various spatial resolution and temporal

constraints. This is part of the foreground ready for exploitation.

• The serious games: our games have demonstrated to increase spatial abilities. They can be

used by other research institutions and other rehabilitation centers. As well, The evaluation

metrics: adopted during our learning experiments (e.g. measuring reaction times, walking

path length, exploration strategies, performance improvements adapted to the person’s

ability). Specific focus goes on the personalization of the metric as we have done during the

experiments. This is part of the foreground ready for exploitation.

Page 22: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

22 BlindPAD - Final Report – online public version 30.09.2017

10.2 Target groups

The following groups are the main dissemination targets:

• Industry organizations and service providers: These are commercial organizations as well

as solution providers with special focus supporting blind and visually impaired people.

Rehabilitation institutes for visually impaired people and schools are important service

providers, since these are the main places where graphical/spatial information is taught both

to children and adults.

• Scientific and academic community: These are research and academic organizations,

universities, scientific journals, scientific conferences, associations, online communities and

other working groups in fields related to the BLINDPAD work.

• Policy-makers and funding institutions: These are decision-makers that influence policy-

making and relevant stakeholders for the uptake of the project’s results. Public health

systems, for example, provide financial support and evaluation tools for assistive aids.

• General public: These are the potential end-users of the BLINDPAD system: the main target

are blind and severely visually impaired persons, although the project results may be

extended also to lower visual impairment.

A second level of dissemination specifically targets a possible marketable product:

• Business angels and venture capitalists, banks, private funding bodies and companies: the

goal being of using further funding to make the current prototypes (from the cheapest to

the most expensive): 1) more robust, 2) certified according to standards (EMC) 3) re-

engineered to account for larger sizes 4) re-engineered to account for higher resolution.

• Public health systems: the goal being to sensitize policy makers toward more funding in

tactile graphics.

Page 23: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

23 BlindPAD - Final Report – online public version 30.09.2017

template A1: list of scientific (peer reviewed) publications, starting with the most important ones

NO. Title Main author

Title of the periodical or the series

Number, date or frequency

Publisher

Place of publication

Year of publication

Relevant pages

Permanent identifiers (if available)

Is/Will open access

provided to this publication?

1 Flexible active skin: large reconfigurable arrays of individually addressed shape memory polymer actuators

N.Besse Advanced Materials Technologies

weekly Wiley VCH

Germany 2017 In press In press yes

2 Using pot-magnets to enable stable and scalable electromagnetic tactile displays

J. Zarate IEEE Transactions on Haptics

quarterly IEEE Computer Society

Washington 2016 Vol 10, Issue 1

10.1109/TOH.2016.2591951 yes

3 The Effect of Programmable Tactile Displays on Spatial Learning Skills in Children and Adolescents of Different Visual Disability

F. Leo IEEE Transactions on Neural Systems and Rehabilitation Engineering

monthly IEEE Computer Society

Washington 2016 Vol: PP, Issue 99

10.1109/TNSRE.2016.2619742

yes

4 Highly Magneto-Responsive Elastomeric Films Created by a Two-Step Fabrication Process

S. Marchi ACS Appl. Mater. Interfaces

weekly Americal Chemical Society

Washington 2015 7 (34), pp 19112–19118

10.1021/acsami.5b04711 yes

5 The Influence of Tactile Cognitive Maps on Auditory Space Perception in Sighted Persons

A.Tonelli Frontiers in psychology

monthly Frontiers

Switzerland 2016 2016; 7: 1683

10.3389/fpsyg.2016.01683

yes

Page 24: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

24 BlindPAD - Final Report – online public version 30.09.2017

6 Designing a vibrotactile head-mounted display for spatial awareness in 3d spaces

V. Oliveira IEEE Transactions on Visualization and Computer Graphics

monthly IEEE Computer Society

Washington 2017 23(4), 1409-1417

10.1109/TVCG.2017.2657238

yes

7 Optimization of the force and power consumption of a microfabricated magnetic actuator for haptic applications

J. Zarate Sensors and Actuators A: Physical

monthly Elsevier Amsterdam 2015 Volume

234, 1 Pages 57-64

doi.org/10.1016/j.sna.2015.08.007

yes

8 Task-dependent calibration of auditory spatial perception through environmental visual observation

A.Tonelli Front Syst Neurosci

yearly Frontiers

Switzerland 2015 9: 84.

10.3389/fnsys.2015.00084 yes

9 The importance of visual experience, gender and emotion in the assessment of an assistive haptic mouse

M. Memeo IEEE Transactions on Haptics

quarterly IEEE Computer Society

Washington 2015 Volume: 8

Issue: 3

10.1109/TOH.2015.2426692

yes

Page 25: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

25 BlindPAD - Final Report – online public version 30.09.2017

template A2: list of dissemination activities

NO. Type of activities

Main leader Title Date/Period

Place Type of audience

Size of audience

Countries addressed

1 Conference G. Bubak Parylene coated carbon nanotube actuators for tactile stimulation, International conference SPIE Electroactive Polymer Actuators and Devices

April 2015

San Diego, CA, USA

Scientific Community (Research), Industry, Medias

300 Worldwide

2 Workshop W. Kunkhornsup Study of Static Tactile Detection Threshold via Pneumatically Driven Polydimethylsiloxane Membrane

2014 Dresden, Germany

Scientific Community (Research), Industry,

100 Worldwide

3 Conference J. Zarate Design and optimization of microfabricated planar coils for tactile displays”, in Proceedings of International Conference of Micro and Nano Engineering

2014 Lausanne, Switzerland

Scientific Community (Research), Industry

200 Worldwide

4 Conference N. Besse “4x4 SMP demo” Haptics Symposium 2016 conference

2016 Phyladelphia, USA

Scientific Community (Research), Industry,

500 Worldwide

5 Conference F. Leo Improving Visuo-Spatial abilities in blind youngsters using programmable tactile displays”, in Cognitive Neuroscience Society meeting

2017 S. Francisco, USA

Scientific Community (Research),

200 Worldwide

6 Conference F. Leo BlindPAD: proposta di uno strumento per incrementare le abilità visuo-spaziali nei non vedenti’, XXV Congresso Nazionale Airipa

2016 Turin, ITaly Scientific Community (Research),

200 Italy

7 Workshop F. Leo Recalling graphical traits with programmable tactile displays improves spatial abilities in young visually impaired persons’, The European Workshop on Imagery and Cognition

June 6, 2016

Paris, France Scientific Community (Research)

1000 EU

Page 26: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

26 BlindPAD - Final Report – online public version 30.09.2017

8 Conference J. Zarate Keep in Touch: Portable Haptic Display With 192 High Speed Taxels", CHI 2017

May 2017

Denver, USA Scientific Community (Research), Industry, Medias

3000 Worldwide

9 Worskhop L. Brayda Organization of Workshop on Multisensory Interaction and Assistive Technology, within International Conference of Tabletop and Surfaces 2015

Nov 2015

Madeira, Portugal

Scientific Community (Research), Industry, Medias

200 Worldwide

10 Workshop L. Brayda Organization of Workshop on Haptics Interfaces for Accessibility within World of Haptics Conference

June 2017

Furstenfeldbruck, Germany

Scientific Community (Research), Industry,

500 Worldwide

11 Workshop E. Cocchi Workshop on Complex Disabilities, Fondazione Mariani

March 2014

Florence, ITaly

Scientific Communitiy, Civil society, Policy Makers

100 Italy

12 Conference E. Cocchi National conference “La riabilitazione visiva in età evolutiva – centri italiani a confronto” (Visual rehabilitation in developmental age – comparison of Italian centers),

Nov 2014

Palermo, Italy

Scientific Communitiy, Civil society, Policy Makers

100 Italy

13 Workshop L .Brayda Workshop “Occhio della Mente“ (The Mind’s Eye

Dec 2014

Genoa, Italy Scientific Communitiy, Civil society, Policy Makers

200 Italy

14 Workshop F. Leo Workshop “Occhio della Mente“ (The Mind’s Eye

Nov 2015

Genoa, Italy Scientific Communitiy, Civil society, Policy Makers

200 Italy

15 Workshop E. Capris Workshop “Occhio della Mente“ (The Mind’s Eye

Nov 2016

Genoa, Italy Scientific Communitiy, Civil society, Policy Makers

200 Italy

16 Conference and Exhibition

M. Memeo Congress of the Italian Neurology Society Oct 2015

Genoa, Italy Scientific Community (Research), Industry, Medias, Policy Makers, Large Public

500 Italy

17 Exhibition L. Brayda ICT Lisbon: Innovate, Connect, Transform Oct 2015

Lisbon, Portugal

Scientific Community (Research), Industry, Medias, Policy Makers, Large Public

6000 EU

18 Conference P .Witek National Educational Conference April 2016

Krakow, Poland

Scientific Community , Policy Makers

200 Poland

19 Conference L. Brayda Metakoiné Nov 2016

Milan, Italy Industry, Medias, Policy Makers, Large Public

100 Italy

20 Workshop Waskyelewicz Pełno(S)prawny student Dec 2016

Krakow, Poland

Teachers, Students 200 Poland

Page 27: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

27 BlindPAD - Final Report – online public version 30.09.2017

21 Conference J .Zarate Demo: Portable pin array with 192 fast latching taxels

June 2017

Furstenfeldbruck, Germany

Scientific Community (Research), Industry,

500 Worldwide

22 Exhibition J. Meis Sight City 2014 May 2014

Frankfurt ,Germany

Industry, Medias, Policy Makers, Large Public

3000 EU

23 Exhibition J. Meis Sight City 2015 May 2015

Frankfurt ,Germany

Industry, Medias, Policy Makers, Large Public

3000 EU

24 Exhibition L. Brayda “Il Tempo al buio” (Time in the dark) at the Festival of Science

Oct 2014

Genoa, Italy Industry, Medias, Policy Makers, Large Public

3000 Italy

25 Conference F. Leo “Il Tempo al buio” (Time in the dark) at the Festival of Science

Oct 2015

Genoa, Italy Industry, Medias, Policy Makers, Large Public

3000 Italy

26 Conference P .Witek Silesia Commons Innovation Nov 2015

Katowice, Poland

Industry, Medias, Policy Makers, Large Public

400 Poland

27 Exhibition L. Brayda Abilitando Sept 2015

Bosco Marengo, Italy

Industry, Medias, Policy Makers, Large Public

3000 Italy

28 Conference L .Brayda EFARRI Awards 2016 Nov 2016

Bruxelles, Belgium

Scientific Community (Research), Industry, Medias, Policy Makers

300 EU

29 Exhibition L .Brayda Makerstown 2016 May 2016

Bruxelles, Belgium

Industry, Medias, Policy Makers, Large Public

2000 EU

30 Exhibition L .Brayda Makerstown 2016 June 2017

Bruxelles, Belgium

Industry, Medias, Policy Makers, Large Public

2000 EU

31 Conference

N. Besse Transducers’17

June 2017

Kaohsiung, Taiwan

Scientific Community (Research), Industry, 500

Worldwise

32

Conference

J. Zarate World of Haptics June 2017

Furstenfeldbruck, Germany

Scientific Community (Research), Industry,

300

EU

33 Conference

N. Besse SPIE-EAPAD 2016,

March 2016

Las Vegas, Nevada

Scientific Community (Research), Industry, 300

Worldwide

34 Press release

L .Brayda The EU project BlindPAD Jan 2014

Genoa, Italy Large Public 10000

EU

Page 28: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

28 BlindPAD - Final Report – online public version 30.09.2017

35 Press Release

H .Shea A touchable tablet to guide the visually impaired

May 2017

Neuchatel, Switzerland

Large Public 10000

EU

36 Flyers L .Brayda The EU project BlindPAD 2014 Genoa, Italy Large Public 1000 Worldwide

37 Web Article http://indianexpress.com

New touchscreen tablet to guide visually impaired 2014

www Large Public

Worldwide

38 Web Article

http://salute24.ilsole24ore.com/articles/19296 Teletatto, trasforma le immagini in tatto 2014

www Large Public

Worldwide

39 Web Article http://technowinki.onet.pl BlindPAD: tablet dla niewidomych 2015 www Large Public Worldwide

40 Web Article http://tg24.sky.it/cronaca/

Tecnologia, dall'Italia tablet e app per non vedenti 2014

www Large Public

Worldwide

41 Web Article http://www.diarioextra.com

Se imagina ver una tableta a traos de los dedos? 2015

www Large Public

Worldwide

42 Web Article http://www.disabilidoc.it/ Eye-Tech: un occhio di riguardo verso ciechi 2015 www Large Public Worldwide

43 Web Article http://www.efedocanalisis.com La tableta que se ve con los dedos 2016 www Large Public Worldwide

44 Web Article http://www.famigliacristiana.it/ Laboratori di make up per non vedenti 2016 www Large Public Worldwide

45 Web Article http://www.genovapost.com

Istituto Chiossone: Venerdì a Genova convegno su integrazione ... 2016

www Large Public

Worldwide

46 Web Article http://www.genovapost.com

Villa David Chiossone compie 10 anni: sabato apertura alla città 2015

www Large Public

Worldwide

47 Web Article

http://www.genovapost.com/Genova

Istituto Chiossone: domani convegno "Eye - Tech" 2015

www Large Public

Worldwide

48 Web Article http://www.ictjournal.ch/news/

L'EPFL dèveloppe une tablette tactile en relief pour les malvoyants 2017

www Large Public

Worldwide

49 Web Article http://www.primocanale.it/

Retina artificiale e tablet con mappe tattili, stretta sinergia tra IIT e ... 2015

www Large Public

Worldwide

50 Web Article

http://www.repubblica.it/tecnologia/

BlindPAD, anche i non vedenti useranno i tablet: sensazioni tattili da ... 2014

www Large Public

Worldwide

51 Web Article http://www.superando.it/ Disabilità visiva e inclusione scolastica 2016 www Large Public Worldwide

52 Web Article http://www.tendencias21.net

Crean una tableta tactil que refleja en relieve los planos del entorno 2016

www Large Public

Worldwide

53 Web Article https://techcrunch.com BlindPAD's tablet for the vision-impaired 2017 www Large Public Worldwide

54 Web Article https://www.beritateknologi.com

Sentuhan Detail Visual Khusus Pada Tablet BlindPAD Untuk ... 2016

www Large Public

Worldwide

55 Web Article https://www.engadget.com/

World's first braille smartwatch is an ebook reader and more 2015

www Large Public

Worldwide

Page 29: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

29 BlindPAD - Final Report – online public version 30.09.2017

56 Web Article https://www.psfk.com/

Tablet Makes Visual Information Tactile For The Visually Impaired 2017

www Large Public

Worldwide

57 Web Article https://www.rdmag.com/news/ Touchable Tablets Could Guide the Blind 2017 www Large Public Worldwide

58 Web Article https://www.thedigeon.com Il tablet Braille che aiuta gli ipovedenti 2015 www Large Public Worldwide

59 Web Article https://wwwhatsnew.com/

Una tablet capaz de representar informaci√on para personas con ... 2017

www Large Public

Worldwide

60 Web Article https://yourstory.com/

A taste of what it is like to 'Start Off as CEO': Five youngsters tell their ... 2016

www Large Public

Worldwide

11 Section B: Foreground exploitation

11.1 Part B1: patents (Public)

Template B1: List of applications for patents, trademarks, registered designs, etc.

Type of IP Rights:

Confidential Click on YES/NO

Foreseen embargo date dd/mm/yyyy

Application reference(s) (e.g. EP123456)

Subject or title of application

Applicant (s) (as on the application)

Patent

YES Aug 30th, 2016

IT 102016000088204

Attuatore bistabile basato sull’attrazione elettromagnetica F. Bertora, L.Brayda, G.Sandini

Page 30: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

0 BlindPAD - Final Report – online public version 30.09.2017

12 Report on societal implications

A General Information (completed automatically when Grant Agreement number is entered.

Grant Agreement Number: 611621

Title of Project: BlindPAD

Name and Title of Coordinator: Luca Brayda, PhD

B Ethics

1. Did your project undergo an Ethics Review (and/or Screening)?

• If Yes: have you described the progress of compliance with the relevant Ethics

Review/Screening Requirements in the frame of the periodic/final project reports?

Special Reminder: the progress of compliance with the Ethics Review/Screening

Requirements should be described in the Period/Final Project Reports under the Section

3.2.2 'Work Progress and Achievements'

No

2. Please indicate whether your project involved any of the following issues (tick

box) :

YES

Research on Humans

• Did the project involve children? Y

• Did the project involve patients? Y

• Did the project involve persons not able to give consent? N

• Did the project involve adult healthy volunteers? Y

• Did the project involve Human genetic material? N

• Did the project involve Human biological samples? N

Page 31: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

1 BlindPAD - Final Report – online public version 30.09.2017

• Did the project involve Human data collection? Y

Research on Human embryo/foetus

• Did the project involve Human Embryos? N

• Did the project involve Human Foetal Tissue / Cells? N

• Did the project involve Human Embryonic Stem Cells (hESCs)? N

• Did the project on human Embryonic Stem Cells involve cells in culture? N

• Did the project on human Embryonic Stem Cells involve the derivation of cells from

Embryos?

N

Privacy

• Did the project involve processing of genetic information or personal data (eg.

health, sexual lifestyle, ethnicity, political opinion, religious or philosophical conviction)?

Y

• Did the project involve tracking the location or observation of people? Y

Research on Animals

• Did the project involve research on animals? N

• Were those animals transgenic small laboratory animals? N

• Were those animals transgenic farm animals? N

• Were those animals cloned farm animals? N

• Were those animals non-human primates? N

Research Involving Developing Countries

• Did the project involve the use of local resources (genetic, animal, plant etc)? N

• Was the project of benefit to local community (capacity building, access to

healthcare, education etc)?

Y

Dual Use

• Research having direct military use N

• Research having the potential for terrorist abuse N

Page 32: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

2 BlindPAD - Final Report – online public version 30.09.2017

C Workforce Statistics

3. Workforce statistics for the project: Please indicate in the table below the number of people

who worked on the project (on a headcount basis).

Type of Position Number of Women Number of Men

Scientific Coordinator 1

Work package leaders 3 7

Experienced researchers (i.e. PhD holders) 10 18

PhD Students 6 4

Other 14 5

4. How many additional researchers (in companies and universities) were recruited

specifically for this project?

12

Of which, indicate the number of men:

6

D Gender Aspects

5. Did you carry out specific Gender Equality Actions under the project?

X

Yes

No

6. Which of the following actions did you carry out and how effective were they?

Not at all

effective

Very

effecti

ve

Design and implement an equal opportunity policy X

Set targets to achieve a gender balance in the

workforce

X

Organise conferences and workshops on gender X

Actions to improve work-life balance X

Other: Balancing experimental samples across gender

Page 33: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

3 BlindPAD - Final Report – online public version 30.09.2017

7. Was there a gender dimension associated with the research content – i.e. wherever people

were the focus of the research as, for example, consumers, users, patients or in trials, was the issue

of gender considered and addressed?

X Yes- please specify

No

E Synergies with Science Education

8. Did your project involve working with students and/or school pupils (e.g. open days,

participation in science festivals and events, prizes/competitions or joint projects)?

X Yes- please specify

No

9. Did the project generate any science education material (e.g. kits, websites, explanatory

booklets, DVDs)?

X Yes- please specify

No

F Interdisciplinarity

10. Which disciplines (see list below) are involved in your project?

Main discipline: Engineering and technology: chemical, electronic, mechanical,

material and computer science engineering

Associated discipline: Social

Sciences: Psychology, Educational

sciences

Associated discipline: Medical Sciences:

health sciences, public health services

G Engaging with Civil society and policy makers

11a Did your project engage with societal actors beyond the research

community? (if 'No', go to Question 14)

X

Yes

No

Pupils were involved in 3 Open days at IIT, 3 festivals, 3

competitions

The BlindPAD system (hardware and software) is an

education material per se. About 3000 brochures, 1

website.

Yes, samples of children and adults were balanced across

age

Page 34: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

4 BlindPAD - Final Report – online public version 30.09.2017

11b If yes, did you engage with citizens (citizens' panels / juries) or organised civil society (NGOs,

patients' groups etc.)?

No

X Yes- in determining what research should be performed

X Yes - in implementing the research

X Yes, in communicating /disseminating / using the results of the project

11c In doing so, did your project involve actors whose role is mainly to organise

the dialogue with citizens and organised civil society (e.g. professional mediator;

communication company, science museums)?

X

Yes

No

12. Did you engage with government / public bodies or policy makers (including international

organisations)

No

Yes- in framing the research agenda

Yes - in implementing the research agenda

X Yes, in communicating /disseminating / using the results of the project

13a Will the project generate outputs (expertise or scientific advice) which could be used by

policy makers?

X Yes – as a primary objective (please indicate areas below- multiple answers possible)

Yes – as a secondary objective (please indicate areas below - multiple answer

possible)

No

13b If Yes, in which fields?

Audiovisual and Media

Consumers

Education, Training, Youth

Information Society

Public Health

Research and Innovation

Transport

Page 35: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

5 BlindPAD - Final Report – online public version 30.09.2017

13c If Yes, at which level?

X Local / regional levels

X National level

X European level

International level

H Use and dissemination

14. How many Articles were published/accepted for publication in peer-

reviewed journals?

9

To how many of these is open access2 provided? 9

How many of these are published in open access journals? 9

How many of these are published in open repositories? 9

To how many of these is open access not provided? 0

Please check all applicable reasons for not providing open access:

publisher's licensing agreement would not permit publishing in a

repository

no suitable repository available

no suitable open access journal available

no funds available to publish in an open access journal

lack of time and resources

lack of information on open access

other3: ……………

15. How many new patent applications (‘priority filings’) have been made?

("Technologically unique": multiple applications for the same invention in different

jurisdictions should be counted as just one application of grant).

3

2 Open Access is defined as free of charge access for anyone via Internet. 3 For instance: classification for security project.

Page 36: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

6 BlindPAD - Final Report – online public version 30.09.2017

16. Indicate how many of the following Intellectual

Property Rights were applied for (give number in each

box).

Trademark 0

Registered design 0

Other 0

17. How many spin-off companies were created / are planned as a direct result

of the project?

1

Indicate the approximate number of additional jobs in these companies: 3

18. Please indicate whether your project has a potential impact on employment, in comparison

with the situation before your project:

x Increase in employment, or X In small & medium-sized enterprises

Safeguard employment, or X In large companies

Decrease in employment, None of the above / not relevant to the

project

Difficult to estimate / not possible to

quantify

19. For your project partnership please estimate the employment effect

resulting directly from your participation in Full Time Equivalent (FTE = one

person working fulltime for a year) jobs:

Difficult to estimate / not possible to quantify

Indicate figure:

3

I Media and Communication to the general public

20. As part of the project, were any of the beneficiaries professionals in communication or media

relations?

Yes X No

21. As part of the project, have any beneficiaries received professional media / communication

training / advice to improve communication with the general public?

X Yes No

22 Which of the following have been used to communicate information about your project to

the general public, or have resulted from your project?

X Press Release X Coverage in specialist press

Page 37: Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people

7 BlindPAD - Final Report – online public version 30.09.2017

X Media briefing X Coverage in general (non-specialist) press

X TV coverage / report X Coverage in national press

X Radio coverage / report X Coverage in international press

X Brochures /posters / flyers X Website for the general public / internet

DVD /Film /Multimedia X Event targeting general public (festival,

conference, exhibition, science café)

23 In which languages are the information products for the general public produced?

X Language of the coordinator X English

X Other language(s)