Accessibility of websites and
mobile apps
A study on the current practices regarding
accessibility statements, reporting mechanisms and
mobile monitoring methodologies
Draft Final Deliverable
A study prepared for the European Commission
DG Communications Networks, Content & Technology
by:
Digital
Single
Market
This study was carried out for the European Commission by
Capgemini Consulting and Dialogic
For more information about this paper, please contact: Directorate-General for Communications
Networks, Content and Technology G – Learning, Multilingualism & Accessibility Unit G.3
Åsa Barton – Policy Officer E-mail: [email protected]
Project team
Capgemini Consulting
Niels van der Linden – Principal Consultant
Executive lead
E-mail: [email protected] Remco van der Spiegel – Managing consultant
Project manager
E-mail: [email protected]
Wander Engbers – Consultant
Analyst
Christian le Clercq – Senior Consultant
Analyst
Sem Enzerink – Consultant
Analyst
Dialogic
Robbin te Velde – Principal Researcher
Quality assurance
Internal identification
Contract number: 30-CE-0631083/00-65
SMART number 2016/0089
DISCLAIMER
By the European Commission, Directorate-General of Communications Networks, Content & Technology.
The information and views set out in this publication are those of the author(s) and do not necessarily reflect the
official opinion of the Commission. The Commission does not guarantee the accuracy of the data included in this
study. Neither the Commission nor any person acting on the Commission’s behalf may be held responsible for
the use which may be made of the information contained therein.
© European Union, 2017. All rights reserved. Certain parts are licensed under conditions to the EU.
Reproduction is authorised provided the source is acknowledged.
3
European Commission Accessibility of websites and mobile apps – A study on the current
practices regarding accessibility statements, reporting mechanisms and mobile monitoring methodologies Luxembourg, Publications Office of the European Union
4
Executive Summary Technology, in particular connected technology, is rapidly changing our world. It changes every
industry, organisation and process. It also changes how citizens interact with their (local) public
sector: how they acquire information or how they use public services. To ensure all citizens, including
those with a disability, profit from digitisation the European Union enshrined Web Accessibility in its
legislation. Web Accessibility means that people with disabilities, such as blindness or motoric
limitations, can perceive, understand, navigate, interact with and contribute to the Web. With the
Web Accessibility Directive, it becomes mandatory for all Public Sector Bodies to have websites and
mobile applications adhering this principle.
Methodology: Examining the current accessibility practices in three studies To ensure that the Member States move towards an accessible web in a similar manner, the
Directive aims to harmonise accessibility legislation. In anticipation of the implementing acts
detailing this legislation, this study presents an overview of the current practices in three respective
fields: accessibility statements (study 1); reporting mechanisms (study 2); and, monitoring
methodologies for mobile applications (study 3). Based on the analyses of these practices and
discussions with experts, the study presents recommendations for the development of implementing
acts. The study collected the information via the following four means:
Landscaping of the European countries by asking them to participate in a questionnaire and
complementary interview. In total 28 Member States responded to the questionnaire and 25
representatives were interviewed.
Desk research to obtain a sound definition of the state-of-play on web accessibility
developments and to identify the key stakeholders, experts and interest groups.
Interviews with 25 international experts from the public, private and NGO sector. In
particular to discuss the possibilities to monitor the accessibility of mobile applications.
Survey addressing a global audience to identify existing monitoring initiatives covering the
accessibility of mobile applications for further examination. The survey did not identify any
existing initiatives.
Policy: Moving towards transposition, the currently heterogeneous position of
web accessibility Transcending the three individual studies, the report first examined the current legal position of web
accessibility and the (expected) progress in the transposition of the Directive. The legal position of
web accessibility strongly varies in Europe, ranging from no legislation, to general non-discrimination
acts, to binding legislation specifically on web accessibility. This heterogeneity affects the maturity of
the practices and the transposition preparations. To transpose the Directive, 21 countries started
preparations, ranging from composing commissions to drafting legislation. The countries mention
time and budget constraints as possible risks for a successful transposition. These constraints are
either the result of national developments, such as upcoming elections, or because of the absence of
implementing acts. In 24 of the surveyed countries, interest groups are involved in the policy
processes regarding digital accessibility. This ranges from ad-hoc advisory roles to executing
accessibility evaluations.
5
Study 1: Large variety of practices regarding accessibility statements At the moment, 54% of the reviewed countries have specific policies or regulations on accessibility
statements (implemented or in preparation). Member States miss overview on the use of statements
in their country. Only seven Member states could estimate the percentage of statements on public
sector body websites. Their estimations ranged from less than 25% to over 95%. This underlines that
the full use of accessibility statements has not been commonly achieved across all Member States.
Our analysis of 66 accessibility statements showed that statements generally include: a description of
inaccessible parts, a feedback mechanism, assistive user instructions, a statement date and the
achieved level of compliance. None of the statements provided a link to enforcement procedures or
the update frequency. The statements contained information relevant for end-users (35%),
monitoring purposes 9%) or a combination (16%). In 39% of the cases, the statements did not
sufficiently provide information to be helpful for end-users nor monitoring purposes.
The study found six practices how Member States develop (and update) accessibility statements:
1. Free formats that allow Public Sector Bodies to structure their own accessibility statements.
2. Content descriptions that prescribe the required topics of the accessibility statement.
3. Standard templates that define the format and structure of the accessibility statement so a
public sector body only provides particular content regarding its website.
4. Common Web Platform incorporations that automatically create an accessibility statement
within the commonly used (technical) web platform of public sector bodies.
5. Accessibility generators that collect particular information of the public sector bodies to
automatically generate an accessibility statement.
6. Statements based on a label that declare accessibility compliance by showing a logo and
further information related to that logo.
We examined the accessibility statement practices on their benefits and concerns, related to:
scalability, assumed impact, awareness, administrative burden and sustainability. The outcome of
this examination is displayed in Figure 1.
Free format (non-
standardised)
Content description (non-
standardised)
Standard template (standardised)
CWP incorporation (standardised)
Statement generator
(standardised)
Statements based on a label (standardised)
Be
ne
fits
Scalability
Administrative
burden
Sustainability
Scalability
Awareness
Administrative
burden
Sustainability
Scalability
Assumed impact
Administrative
burden
Sustainability
Assumed impact Scalability
Assumed impact
Administrative
burden
Sustainability
Assumed impact
Awareness
Co
nce
rns
Assumed
impact
Awareness
Assumed
impact
Awareness Scalability
Awareness
Administrative burden
Sustainability
Awareness Scalability
Administrative
burden
Sustainability
Figure 1: Benefits and concerns of practices regarding accessibility statements
The Directive proposes a harmonised accessibility statement model. Therefore, the benefits and
concerns of the standard template are particularly relevant, while gaining insights from the other
practices. On the level of individual accessibility statements, no single statement included all relevant
elements: containing all defined content items, combining end-user and monitoring purposes,
defining information on the update frequency, being easy to find and declaring accountability.
Therefore, the chapter provided good examples of different accessibility statements that cover
6
different accessibility statement elements. This selection of examples can inform further
development of a harmonised accessibility statement model.
This study recommends a standard statement in an accessible format with a fixed order of items that
serves as input for both end-users and monitoring purposes. It should include the mandatory items
defined in the Directive. The statement should also provide the following additional items: level of
compliance, planning of improvements and the test report. In order to be relevant, the statement
should contain as-is accessibility information meaning that the information provided should be
updated after each (internal or external) update with impact on the accessibility of the website or
mobile application. The statement should mention a senior executive or team that is political
accountable for the accessibility of the website or mobile application to show responsibility and
ensure ownership within the Public Sector Body. It is recommended to ensure findability, for
example by defining a uniform URL or location of the link to the statement. This link should be
available on a permanent body of pages of the website.
A standardised template to communicate the actual accessibility status of the PSB’s website, which serves end-users and monitoring purposes
Co
nte
nt
ite
ms
for
stan
dar
d t
em
pla
te
Use a harmonised standard template that serves as input for both end-users and monitoring.
Include additional content items: description of the level of compliance, planning of
improvements and a link to the test report, besides the mandatory items (descriptions and
explanation of inaccessible parts, alternatives, link to a feedback channel and a link to an
enforcement procedure).
The template should have a fixed order of items, taking into account the end-users point of
view.
Acc
ou
nta
bili
ty Include details of the senior executive or team accountable for accessibility in order to show
political ownership and sense of urgency.
Stan
dar
d
loca
tio
n Define a uniform location to ensure findability, for example by means of a uniform URL as
location for accessibility statements, or a uniform location of the link to the accessibility page,
and ensure visibility for search engines.
Fre
qu
en
cy o
f
up
dat
ing
The statement should reflect the as-is status and hence be updated at each update with
relevance to accessibility (change in the functioning of the website or app, this includes
website updates, browser and O.S. updates).
Acc
ess
ible
form
at
The standard template should be in an accessible (WAD mandatory) and machine-readable
format that can easily serve as input for monitoring.
Figure 2: Recommendations for accessibility statements
7
Study 2: How to collect information? Different ways to organise the reporting
on web accessibility In the absence of a policy on web accessibility in many countries, 11 out of 28 reviewed countries
have a formal reporting practice at this moment. With the addition of informal, planned or past
initiatives, the study identified and examined 16 reporting examples. Based on this examination, the
study found three types of reporting practices:
1. Decentralised transfer of data by the PSB via a (self-) declaration to a central body: a
reporting mechanism wherein it is the responsibility of the PSB to determine its level of
accessibility, via self- or third party evaluation, and to provide this information to a central
body.
2. Centralised monitoring based on third party evaluation: a reporting mechanism wherein a
third party evaluates the level of accessibility of multiple PSBs, either in commission of a
central government body or as NGO initiative.
3. Centralised monitoring based on a government agency evaluation: a reporting mechanism
wherein a central government body both monitors and reports on the level of accessibility of
PSBs.
Most of the existing reporting practices are organised according to the third type in which one body
measures and reports on the web accessibility of (a number of) public sector bodies. Of the
interviewed countries, 28% indicated to prefer or prepare decentralised reporting practices. The
current practices have a two-folded purpose: providing feedback to the audited entities and
reporting on overall developments using aggregated data. The practices regarding reporting were
also examined on their benefits and concerns in terms of scalability, assumed impact, awareness,
administrative burden and sustainability. This examination, displayed in Figure 3, showed that each
type has its own benefits and concerns. Nevertheless, decentralised and centralised reporting
primarily differ in scalability and administrative burden.
1. Decentralised reporting
2. Centralised reporting by third party
3. Centralised reporting by a central (government)
body
Be
ne
fits
Awareness
Administrative burden
Target group
involvement
Administrative burden
Scalability
Assumed impact
Co
nce
rns Assumed impact
Scalability
Sustainability
Scalability
Administrative burden
Figure 3: Benefits and concerns of practices regarding reporting
The study found three recurring trade-offs when examining the practices which pose relevant
questions for the development of a harmonised mechanism. The first trade-off concerns the use of
automated tools. Automated tools create efficiency gains and are complementary to manual
evaluation by end-users. The second is the trade-off between a detailed and valid but labour
intensive methodology and a more high-level but efficient methodology to measure the level of
accessibility. The third trade-off is between publishing the (individual) outcome of the reporting or
providing a safe-learning environment for the public sector bodies. Although not transparent, this
does enable continuous (peer-) learning.
8
Based on the current insights and preferences of Member States, the reporting mechanism will need
to strike a balance between flexibility and rigidity. Flexibility in the ‘how’, allowing Member States to
decide to organise the reporting centralised or decentralised. Rigidity in the ‘what’, to make sure
Member States collect the same data in a similar way via a harmonised monitoring methodology so
the data can be aggregated at the European Union level. We identified good practices to balance
flexibility and rigidity. The reporting could also make use of the other mandatory elements of the
Directive, in particular the accessibility statements. When these statements are valid and sound –
based on an approved and harmonised monitoring or evaluation methodology – the reporting body
could automatically harvest these for information. The report therefore recommends a form of
decentralised reporting scheme, entailing a strong role for the public sector bodies. This enhances
their awareness with web accessibility and provides them with better insights on their performance,
which could both help them improve. Furthermore, it allows for a bigger scale as all individual public
sector bodies are conducting evaluations. The figure below presents the recommendations that,
either as a framework or by themselves, provide the means to achieve a reporting mechanism with
the highest teleological impact.
Decentralised reporting scheme with a controlling sample
Fro
m M
em
ber
Stat
e t
o
Co
mm
issi
on
Harmonise the data by providing a fixed template for the Member States to report to the Commission on:
Monitoring data and enforcement procedure data (WAD mandatory)
Aggregated and segmented data in order to analyse differences in segments and allow for
Best practices and challenges to help Member States continuously improve
Market characteristics of accessibility products and services to analyse the impact of the
Directive on the web accessibility market
Wit
hin
Me
mb
er
Stat
e Monitoring based on decentralised evaluation by PSBs and transfer of data from PSB to
Member State’s monitoring body (self- of 3rd party evaluation)
Harmonised data by providing a fixed reporting template for the Public Sector Bodies to
report on:
o Level of accessibility
o Wider (planned) accessibility measures to keep track of policy developments
o End-user feedback
o Use automated tools for an efficient collection of data
Co
ntr
ol Centralised controlling function by the Monitoring Body, based on a random and purposive
sample to include sectors, services and functionalities that are most relevant to the end-users.
Consult end-users to determine the most relevant websites.
Fre
qu
en
cy Align the frequency of the transfer of data of PSBs to the Monitoring Body with the budgetary
cycle to encourage direct budget allocation.
Triennial reporting from Member State to EC (WAD mandatory).
Figure 4: Recommendations for a reporting mechanism
9
Study 3: heading towards a monitoring methodology for mobile applications The third study focused on methodologies to monitor the accessibility of native mobile applications
with the primary conclusion that there are currently no formal or structural initiatives monitoring the
accessibility of mobile applications, within nor outside the Europe Union. As an explanation, the
surveyed Member States and the global experts (from academia, public sector, private sector and
non-governmental organisations) indicated that the accessibility of mobile applications is not yet as
developed as the accessibility of websites.
Important conditions to stimulate the development of monitoring mobile applications in the future
are: an international mobile accessibility standard, a balance between automated and manual
evaluations, a clearly defined evaluation scope and the availability of source codes. Mobile
accessibility in general would benefit from: greater awareness among mobile app developers,
standardisation within the diversity of user interfaces and strengthened end-user feedback. The
experts were in near consensus that a monitoring methodology can only be successful when it covers
the entire life cycle of a mobile application. In other words, not only focussing on the accessibility
when the app goes live, but also in an earlier stage, during the development of the application.
Based on the gathered insights, Figure 5 presents five scenarios for a monitoring mechanism, with
respective benefits and concerns. The study outlines requirements for each scenario to implement it
as a monitoring methodology.
Development phase Live phase
Monitoring scenario 1: Mandatory self-evaluation. Developer is required to evaluate and document on accessibility during development process. Benefits: Developer can directly mitigate issues, (partly) use of automated tools, efficient evaluation. Concerns: Administrative burden for developer, dependent on uniform evaluations. Requirements: Reporting on accessibility by
developer, evaluation methodology.
Monitoring scenario 3: Evaluation by central body. All applications are evaluated by one central reporting body. Benefits: Consistent evaluation. Concerns: Labour-intensive and thus requires many resources from a central evaluation body. Requirements: Auditing expertise, evaluation
method.
Monitoring scenario 2: Process certification. Application owner is required to provide information that developer is certified. Benefits: Low costs, low administrative burden, promotional value. Concerns: Dependent on trust, no actual evaluations, missing insights on actual accessibility level. Requirements: Standards, certified developers, certification authority, authority to penalise.
Monitoring scenario 4: Evaluation by developer. Developer evaluates accessibility on request. Benefits: Developer has access to source code, developer can directly mitigate issues or during update. Concerns: Dependency on availability developer. Specific requirements: Evaluation method.
Scenario 5: Flagging by user community. Reactive enforcement based on user notification. Benefits: Only high priorities focused, target group involvement. Concerns: Not comparable, dependent on interest groups, no equal interest community. Requirements: Evaluation capability, enforcement
procedure, establish central contact points where
the feedback of end-users is being handled by
professionals.
Figure 5: Possible mobile monitoring scenarios
10
Based on these scenarios, the report recommends harmonised monitoring methodology combining
different scenarios. Mobile accessibility experts believe that evaluating the actual level of
accessibility is important, regardless of the scenario. Although the current automated evaluation
tools can measure different factors of mobile accessibility, none of these tools can currently evaluate
all accessibility factors. In existing evaluations automated and manual evaluations are
complementary. Depending on the monitoring scenario different tools can be used in different
phases of the application lifecycle: by developers in the development and testing phase, or by end-
users after deployment of an application. This translates into the recommendation to involve all
stakeholders in the monitoring process, from self-evaluation by the developers to end-user flagging
for focused evaluation. The study recommends a decentralised monitoring practice based on
mandatory developer evaluations in combination with control by a central body which is guided by
end-user feedback. The figure below presents the practical recommendations accompanying this
envisioned organisation.
A monitoring practice based on developer evaluation combined with central body control
Har
mo
nis
ed
mo
nit
ori
ng
me
tho
do
logy
Monitoring practice based on collection of decentralised evaluations. PSBs transfer the data
of the mandatory evaluation by the app developer to the Member State’s monitoring body.
Organise end-user feedback (post-market surveillance) as input for central body control.
Promote awareness on accessibility through Integrated Development Environments (IDEs).
Har
mo
nis
ed
eva
luat
ion
me
tho
do
logy
Clearly define the scope of the evaluation and use a modular design.
A combined evaluation approach where automated and manual evaluations are
complementary.
Clearly report on the evaluation process.
Fre
qu
en
cy Align the frequency of monitoring with the budgetary cycle of the app owner to encourage
direct budget allocation.
Align accessibility evaluations with the launch and major updates of mobile applications and
with updates of the operating system.
Sam
ple
Cover the entire population of mobile applications or use purposive samples.
Evaluate the full content of the app; only use internal sampling for extensive mobile
applications.
Figure 6: Recommendations for the monitoring of mobile applications
CA
TA
LO
GU
E N
UM
BER
Digital
Single
Market
Top Related