CESSDA Question Databank Tender, results and future Maarten Hoogerwerf, CESSDA expert seminar 2009.

CESSDA Question Databank

Tender, results and future

Maarten Hoogerwerf, CESSDA expert seminar 2009

Introduction

• Data Archiving and Networked Services– Institute of both KNAW and NWO– Mission– Departments:• Archive and dissemination• Infrastructure• Software development


Outline

• Background• Question Bank Tender• Discussion of technical specifications• Conclusion• Approach


Background

• Cross-national survey programmes introduce comparability and harmonization issues.

• Supporting infrastructure: – Constructs, Classifications, Conversions Database

(CCCDB or CHARMCATS)– Question Database (QDB)

• Pre- and post harmonization


Tender

• Specification of tender– Requirements, use cases– Need for CESSDA-wide architecture

• Execution– Metadata Technology– Marratech Sessions– Involvement of architecture WP

• Report and review


Report

• General– QDB should not function stand alone• References to variables, questionnaire, etc.• DDI3 metadata model• Webservice architecture

– DDI v1 and v2 in use by CESSDA archives

• Discussion– Will tools be able to migrate to DDI v3?


Report

• Purpose and Functionality– Link questions via concepts, variables– Link additional survey metadata / physical data– Query questions based on references– QDB needs to include references

• Discussion– Either use DDI3– Use generic model


Report

• Architecture– Repositories povide content– Registry indexes content– 3CDB and QDB provide functionality– Increasing identification and communication

• Discussion– Question bank vs. QDB?– Identification designed for DDI3 context


Report

• Repository– Contains content from one or more archives– Contains one or more banks

• Studies, variables, concepts, universes, questions, ...– Dedicated or on top of existing systems– Additional administration, logs, etc.

• Discussion– Existing systems fall short (identification, version,...)– Quality essential for stability


Report

• Registry– Banks register content– Minimal metadata required for searching– Responsible for searching / locating, not for

retrieval– Use SDMX approach

• Discussion– How much metadata is needed for proper

functioning?


Report

• QDB– Function as repository for local questions and

proxy for non-local questions– Stores comparison information

• Discussion– Should QDB archive questions / comparison

information– Who is responsible for QDB (LTP)


Report

• Requirements and use cases– A ‘Gold Standard’ promotes the use of certain

proven objects and increases comparability– Use registry for searching

• Discussion– Assign to existing questions or define them

centrally?– Use registry or QDB for searching questions?


Report

• Metadata and technology overview– Many open source components– Database might require proprietary software

• Discussion– Start with open source database. Good design

allows replacement when needed.


Report

• Implementation– Start prototype implementations to demonstrate

functionality– Start improving legacy metadata– Use / extend SDMX registry

• Discussion– Deadlock-situation: get tools to improve metadata,

improve metadata to demonstrate functionality– How DDI3 is improved metadata from Nesstar without

workflow, versioning, identification? DDI3-ready?


Alternative Solution

• MT approach is similar / better than intuitive solution– DDI3 metadata

approach is essential– Web service is more

flexible than harvesting– MT approach is more

distributed


Conclusion

• DDI3 is an obvious choice, adopt it and improve it

• It will change workflow, infrastructure and responsibility

• How can archives justify, pay, risk and achieve this?

• What is the role of CESSDA?


Approach


Approach

• Phase 1: search, browse and access questions– Question text + response domain– Results in having some base material

• Phase 2: add references– To/from concepts and questionnaires– Implement registry to facilitate search– Explore organiation,publishing issues


Approach

• Phase 3: Add QDB/3CDB– What functions do these provide– What metadata functions do these require

• Etc.


CESSDA Question Databank Tender, results and future Maarten Hoogerwerf, CESSDA expert seminar 2009.

Documents

Transcript of CESSDA Question Databank Tender, results and future Maarten Hoogerwerf, CESSDA expert seminar 2009.