A macroscopic web accessibility evaluation at different processing phases

A macroscopic Web accessibility evaluation at different

processing phases

Nádia Fernandes, Luís Carriço

9th International Cross-Disciplinary Conference on Web Accessibility –16/17th April 2012 – Lyon, France.

Motivation


2

Introduction

• The Web is being used by all kinds of people;

• Web sites must be accessible;

• Modern Web development transcends static HTML;

• A more real evaluation is necessary.

39th International Cross-Disciplinary Conference on Web Accessibility –

16/17th April 2012 – Lyon, France.

Previous work


4

QualWeb evaluator


5

Before

After

• Accessibility evaluation of Web pages:

– using WCAG 2.0 ,

• 18 HTML techniques;

– at different processing phases

• BEFORE and AFTERBrowser processing;


6

QualWeb evaluator: execution

Some improvements...


7

• is a command-line tool that uses WebKit ,

• works like a WebKit-based Web browser (simulation);

• can be controlled using Javascript.


8

PhantomJs

1. It avoids data injection at the browser level;

2. Evaluated Web page before and after browser processing is exactly the same;

3. Integration with a crawler.


9

Problems Solved!

How do the macroscopic properties emerging from Web accessibility change in respect to the processing phase of

delivery?


10

Research Question

• We evaluated a set of Web pages from a list provided by the Portuguese Web Archive (version of 2008);

• We used the QualWeb evaluator.


11

Experimental Study

• Results of the evaluation are presented in terms of:

– PASS, WARN, and FAIL (Applicable = PASS + WARN + FAIL)

• Metrics used:

– rate conservative =

– rate optimistic =

– rate strict =

• The results are between accessible (100%) and not accessible (0%).


12

Metrics

• 24, 462 Web pages evaluated.

• Total Number of HTML elements:

– before processing - 24,918,720

– after processing - 41,967,072

– (Ratio ≈ 1.7).

• Average Number of HTML elements per page:

– before processing - 1010

– after processing – 1710

– (Ratio ≈1.7).


13

Results

The differences of an HTML document between both

Processing Phases were observed.


14

• Successes– before processing - 9 elements

– after processing - 87 elements

– (Ratio ≈ 9.7).

• Failures– before processing - 46 elements

– after processing – 176 elements

– (Ratio ≈ 3.8).

• Warnings– before processing - 262 elements

– after processing – 451 elements

– (Ratio ≈ 1.7).


15

Results: Average Outcomes

• Conservative rate:– The average quality increases after

processing.

– Accessibility quality between 60% and 90%disappeared after processing.

• Optimistic rate:– The average quality decreases after

processing.

– Some results lower than 20% disappearedafter processing.

– Decrease of higher accessibility quality,leading to a lower accessibility average.


16

Results: rates Before and After processing

• Strict rate:– The average increased after processing.

– Results higher than 85% disappearedafter processing.

– Worse pages before processing get higherscores after processing.

– Better pages before processing areranked lower after processing.


17

Results: rates Before and After processing

1. Techniques coverage

2. Dynamic content

3. Automatic evaluation


18

Limitations

• Evaluations before processing clearly is not the best option!

• We used/percept/interact with the after processed version.

• Considering the rates…

– Web pages possess higher uniformity

– That can be explained with reusable code


19

Discussion

• Impact on Designing Accessible Web Pages

– Importance of sharing reusable code;

– High quality reusable code produce better quality pages.

• Impact on the Perception of Accessibility

– Each metric identify different perspectives;

– It is important that evaluations assess what user perceive.


20

Discussion

• We presented a large-scale study of accessibility on the Web.

• We were able to characterize some accessibility properties of the Web, pointing some differences between processing phases.

• The results obtained on the evaluation of pages after browser processing tend to be more homogeneous than before.

• Considering that the end-user interacts with the after processed pages then most studies about Web quality should be redone.


21

Conclusion

1. Enlarge the coverage of WCAG 2.0 implemented tests

2. Evaluate Rich Internet applications

3. Perform a comparative set of studies


22

Future Work


23

[email protected]

A macroscopic web accessibility evaluation at different processing phases

Design

Transcript of A macroscopic web accessibility evaluation at different processing phases