Quality Assurance at the European Monitoring Centre for Drugs and Drug Addiction Sandrine Sleiman...
-
Upload
aileen-allen -
Category
Documents
-
view
217 -
download
2
Transcript of Quality Assurance at the European Monitoring Centre for Drugs and Drug Addiction Sandrine Sleiman...
Quality Assurance at the European Monitoring Centre for Drugs and Drug Addiction
Sandrine SleimanEuropean Conference on Quality in Official Statistics, Rome8-11 July 2008
Contents
• Scope of EMCDDA activities
• Quality assurance: operating framework
• Challenges and results
Who we are
• A decentralised EU agency
• Formally established in 1993
• Based in Lisbon, Portugal (operating since 1995)
• The hub of drug-related information in the EU
Guiding principles and priorities: 2007-2009
Principles
• Scientific excellence• Partnership • Good governance and
efficiency
Priorities
• Consolidating monitoring and reporting activities
• Enhancing data analysis• Communicating more
effectively
Collecting and analysing existing data
Areas:
• Epidemiology — drug situation, monitoring new trends, five key indicators
• Interventions — health, social and criminal justice responses, best practice in demand and supply reduction
• Legislation — national and Community strategies and policies
• Policy — political and institutional framework, coordination
Five Key indicators • Drug use in the general population (students and young adults):
prevalence and patterns of drug use in the general population
• Drug users accessing treatment (TDI)
• Drug related infectious diseases (HIV, HCV, HBV)
• Drug related deaths and mortality of drug users
• Problem drug use: prevalence and patterns of problem drug use
Requires commitment from MS to invest in data collection on a routine basis
Type of data collected
• Annual registry and epidemiological data
• Descriptive and qualitative data (non-annual)
• Data from research studies
• Institutional data
• Database cases
• Coverage and availability of prevention, treatment and harm reduction
• Statistics from law enforcement sources
Collecting and analysing existing data
• Reitox national focal points (NFPs), national experts
• Annual reporting to EMCDDA (national reports, standard tables, questionnaires)
• Expert groups for developing indicators (nominated by NFPs)
• Ad hoc working groups
Reporting system: standard templates
• National reports and selected issues (based on detailed guidelines)
• Standard Tables (ST) for quantitative information
• Sturctured Questionnaires (SQ) for qualitative information supplementing ST and NR
Quality Management at the EMCDDA
Data collectioninstruments
Quality overviews
Quality Control
by Scientific
Units
Quality assurance at EMCDDA level
Data Validation
Data collection instruments, protocols and guidelines
National Focal Points
National Focal Points
&
National data providers
Data submission in FONTE, including quality control at
national level
National data
providers
Quality Assurance
by QA team
Main quality related outputs & reports
Annual Quality reports
• Implementation Needs Profiles on the 5 key indicators
• EU experts groups ad-hoc documents
• Regular reporting to EC on implementation of 5 KI and on quality of reporting
Annual Quality reports
Quality reports includes feedback on :
Quality criteria :
Completeness, insight, reliability, usefulness,
internal consistency
National reports
1. Global evaluation
2. Method. Aspects
3. By section : level of details, clarity, strong/ weak points, suggestions for improvements, discrepencies
4. Compliance with formal r equirements
5. Layout presentation incl. deadline
Standard tables C1 C2 C3 C4 M1 M2 M3 D1 + Number of tables uploaded; Date of 1 st upload; Number of uploads.
Structured questionnaires - Level of details (insuf., suff., good) - Conciseness (insuf., suff. , good) - Discrepencies with NR
Criteria for National reports• Completeness: the report contains all the necessary and existent
information in order to provide an overview of the situation
• Insight: the report includes complete and significant information, giving an interpretation to the reported information, according to social and political contexts.
• Reliability: the extent to which the information in the report allows comparisons (between different time periods).
• Usefulness: information oriented to the targets; acceptable and pertinent to the report objectives; no redundant information is presented.
• Internal consistency: the extent to have coherent information in the report or to describe the reasons for a lack of internal consistency.
Criteria for data checking in STs
CNDA: Not clear whether all data in the ST are new or not
CQUA: Quantitative information needed clarification
CQUAL: Qualitative information needed clarification
CREF: References needed clarification
DNR: Discrepancy between NR and ST
MDA: Quantitative information missing
MME: Methodological information missing
MRE: References missing
OTH: Others
Criteria for SQ
Level of details
Insufficient, sufficient, good
Conciseness
Insufficient, sufficient, good
Discrepencies with NR
Yes - No
General comment
Any other important feedback
Implementation Needs Profiles
• Concerns 5 key epidemiological indicators
• Focuses on activities related to indicators and partially on content
• Is a general analysis, helps to identify problematic areas, status quo or improvement in data collection
Criteria used
• Most recent data
• Reference
• Description
• Weaknesses
• Activities implemented or ongoing/planned
• Actions needed
Quality assurance: work in progress
Quality Assurance…
• is mentioned as a priority in the EU action plan• is mentioned in the external evaluation report• is part of the EMCDDA’s 3-year work programme
Quality documents should be …
• considered as real tools for the EMCDDA and the REITOX network (as well as for external actors)
• aimed at scientific excellence and not administrative purposes • an overview of strengths and weaknesses of data or collection
processes • …
Quality assurance: work in progress
• Production of 5 KI methodological packages • Definition of precise assessment criteria for 5 KI
based on :
• minimum implementation standards• specific to each indicator• sensitive to national contexts• somewhere between pragmatism and perfection
• Redefinition of templates and guidelines with adaptation of quality criteria
Quality assurance : work in progressUnderstand needs and capacity to
deliver from the Reitox network
Priorities for value delivery: revision of reporting guidelines
Promise what can be delivered
Change capability to match promise
Deliveries matching
standards and priorities
Change capability to match future
needs:
- Quality feedbacks
- Capacity development
activities
Case study: national reportsSuccess
• Recommendations for improvement are generally applied as regards adherence to guidelines:
- Document layout - References to standard tables and structured questionnaires- Sources and use of Harvard for bibliography- Methodological information is more precise- Efforts in writing the National report
Case study: national reports Gaps
• Deadlines missed• Conciseness• Lack of insight on trends and interpretation of results • In some countries, it is still too early to have
information on trends• Work on quality assurance depends on external
factors, such as Member States’ involvement in quality issues