Towards Automated Quality Towards Automated Quality Assurance for Socially Created Assurance for Socially Created Information Assets on the WebInformation Assets on the Web

David Davenport & Markus SchaalComputer Engineering Dept.,

Bilkent University, 06800 Ankara.

david@cs.bilkent.edu.tr & schaal@cs.bilkent.edu.tr

IntroductionIntroduction

• Quality in collaborative web-based projects • considered good • no guarantees!

Outline:• Quality & Quality Assurance• Info. Quality & User Expertise• From Actions to Assessments• A Research Testbed…

Outline:• Quality & Quality Assurance• Info. Quality & User Expertise• From Actions to Assessments• A Research Testbed…

Quality isQuality is

• something good, desirable!

• hard to definemeet or exceed user expectations

• easy to spot lack offails to meet spec.,poorly designed,difficult to use, unreliable, etc.

Quality isQuality is

an open-ended quest for

perfection

Quality AssuranceQuality Assurance

• Assuring improvements requires a process

Vision/goal

System/product/service

Make measurements & compare

Implement changes

What is Information Quality?What is Information Quality?

• Criteria: accessibility, coverage, correctness, consistency, recency, reliability, cost, etc.

• Information must “inform” / “educate”• success depends on recipient too!

• Measuring information quality• either observe user behaviour (individual & groups)• or ask their opinion… but users are not all equal.

Users are not all equalUsers are not all equal

experts

novices

& subversives!

Modulate user

opinion accordingly

But, on the web, how can you know your users?

From Actions to AssessmentsFrom Actions to Assessments

System model of user

User

users

Systemmodel

ofcontent

contents

updates

modulates

viewupdate

rate

Action

assessmentsof quality

Content & User AssessmentsContent & User Assessments

• Update content & user models using evidence• gathered from implicit observations

& by explicitly asking users, but• modulated by current assessment of user expertise/trust!

• Everything depends on users, encourage by• making it easy to contribute and rate content• providing incentives to do so, &• showing them the results!

• What evidence should be gathered & how?• What content & user models should be used?• How should the models be updated?• How should the results be reported?

Research questions

A Research Testbed…A Research Testbed…

• Target students taking CS101/2 “Java” course• Lots of Java / computing resources on the web

(some old, some too complex, some plain wrong!)• Task is to create a quality wiki-like resource.

• Module – for Moodle, Drupal, MediaWiki• Enable distributed users to collaboratively

build quality information assets on the web.

• Of note…• Log all interactions• Try out various user, content & update models• Show overall system quality & its history

To Conclude…To Conclude…

• Web is increasingly important info. resource• Improvements in Quality benefit all• Need way to evaluate quality,

identify opportunities for improvements, & track changes

• Our research aims to • produce a clear understanding

& common “quality” framework• add-on “quality” modules usable by others.

Thank you Thank you for your time & attentionfor your time & attention

contactdavid@cs.bilkent.edu.tr / schaal@cs.bilkent.edu.tr