Evidence in action – moving from guidance to review

Sophie Robinson, Toni Price, Caroline Miller

National Institute for Health and Clinical Excellence (NICE)

Health Libraries Group conference July 2010

What we’re going to talk about

• Role of information specialists in producing and reviewing NICE technology appraisals;– how information specialists are involved in the

scoping and review processes– information gathered by information specialists– Skills & challenges involved in supporting technology

appraisals

What is NICE?• The National Institute for Health and Clinical Excellence

(NICE) is the independent organisation set up in 1999 based in England responsible for

– providing national guidance on the promotion of good health and the prevention and treatment of ill health

– setting quality standards

– managing NHS Evidence

• NICE makes recommendations to the NHS on:

– new and existing medicines, treatments and procedures

– treating and caring for people with specific diseases and conditions.

• NICE makes recommendations to the NHS, local authorities and other organisations on:

– how to improve people’s health and prevent illness and disease.

The information services team at NICE

• A team of 20 information professionals based in London and Manchester

• Provide information support to meet the information needs of NICE staff and to help produce NICE guidance

• Support topic selection, appraisals scoping & reviews, short guidelines, interventional procedures, quality standards, quality & outcomes framework, devices & diagnostics, public health, NICE taxonomy etc

• Liaison leads for all teams in Institute• Library resources – journals, databases, books, ILLs,

information skills training, current awareness service

What is a technology appraisal?

• Technology appraisals are recommendations on the use of new and existing medicines and treatments within the NHS, such as:– medicines – medical devices (for example, hearing aids or

inhalers) – diagnostic techniques (tests used to identify

diseases) – surgical procedures (such as repairing hernias) – health promotion activities (for example, ways of

helping people with diabetes manage their condition).

Types of appraisal

• Single technology appraisals (STA) specifically designed for the appraisal of a single product, device or other technology with a single indication where most of the relevant evidence lies with one manufacturer– eg Everolimus for the prevention of organ rejection in kidney

transplantation (in progress)

• Multiple technology appraisal (MTA) - includes more than one product, device or technology or more than one indication or more than one manufacturer or sponsor– eg Immunosuppressive therapy (basiliximab, daclizumab,

tacrolimus, mycophenolate (mofetil and sodium) and sirolimus) for renal transplantation in children and adolescents TA99

Technology appraisals

• Technology appraisal recommendations are based on a review of clinical and economic evidence.

• Clinical evidence measures how well the medicine or treatment works.

• Economic evidence measures how well the medicine or treatment works in relation to how much it costs the NHS - does it represent value for money?

• Independent academic assessment groups carry out the systematic review process for appraisals

• Obligation for NHS organisations to fund and resource medicines and treatments recommended, usually within three months of NICE issuing guidance

The technology appraisal process

The role of the information specialist in the technology appraisal process

• Initial information support at topic selection stage – support with identifying related guidance for briefing notes for topic selection consideration panels

• Support for production of ‘draft scope’ – 2-4 page document setting out parameters of topic in question, used for consultation

• Collaborate with a technical analyst in production of draft scope and provide other information support needed throughout the production process

• Monitor topic right through to publication – licensing updates and key new trials

• New topics and review topics are ‘scoped’

Contributing to the draft scope

• Collate information on – Health condition/disease/behaviour in question– Population affected including epidemiological

information, equality issues– Technology in question including licensing

information – Evidence based guidance on health condition /

technology– Relevant key trials

•Not a systematic search at scoping stage•Record information via NICE intranet on a ‘scoping page’ which is updated

Information specialist skills used in production of technology appraisal

• Retrieval, interpretation, synthesis, summarising & presenting information

• Elements of critical appraisal to summarise key evidence

• Collaborative with technical analyst• Identifying issues for consideration - eg populations,

comparators, links with other appraisals or NICE guidance

Challenges of contributing to the scoping process

• Risk of working in advance of licensing and topics not coming to fruition

• Uncertainty around licensing requires updates to information

• Project management timelines• Understanding diseases, stages, lines of treatment• Identifying relevant key trials• Identifying key issues eg whether an appraisal is

feasible (faller’s clinics where no definition or evidence and topic did not progress)

• Knowing when to stop searching for information

Reviewing Technology Appraisals• All published technology appraisal guidance is given a

date (between 1-5 years) at which the guidance will be considered for review.

• These reviews are known as Review Proposal Projects or RPPs.

• The aim of the review process is to decide whether or not the guidance needs to be updated.

• Information services play a central role, by providing an evidence base upon which to make the decision.

RPP process overview

Information Services involvement

• IS staff carry out the bulk of the work for development of RPPs, gathering and evaluating information to see if there is significant evidence to warrant a review.

• This involves literature searching and writing recommendation papers for NICE’s Guidance Executive (senior management team) to consider.

• The information specialist has to work to specific timelines and processes developed and overseen by a Project Manager from Technology Appraisals.

Information gathering• The information specialist will familiarise themselves

thoroughly with the appraisal to be reviewed, with the disease area concerned and with the drug or device.

• They will also search for new evidence including:– New indications for drugs included in the original

guidance and related new products.– Information on the progress of ongoing randomised

controlled trials.– Update to the searches used in the original Health

Technology Assessment report for the appraisal.

Results• The results of the searches are sifted by the information

specialist to identify any new evidence that could trigger a review of the appraisal.

• For example, the IS may find a key clinical trial that could lead to the deferral of the review of the appraisal awaiting trial results.

• The published results of an RCT may contradict one of the recommendations in the original appraisal.

• A new indication or new related drug may mean the information in the appraisal is out of date.

Proposal paper for Guidance Executive (GE)

• Aims to summarise and collate relevant information found in a report template, which is presented to NICE’s senior management team.

• Recommends a proposal for what should happen to the appraisal (one from a list of standard options).

• A technical analyst will analyse and report on the results of the searches and make the final decision on the recommendation in conjunction with colleagues.

Possible recommendations

• Review of the appraisal should be: – planned into the appraisal work programme;– decision to review will be deferred until X (for

example, to await the results of a key clinical trial);– combined with the review of a related technology

appraisal or with a related new topic referred to NICE;

– incorporated or updated into an on-going clinical guideline;

– transferred to static guidance list.

Proposal paper for Guidance Executive

• The paper is agreed by an Associate Director from the appraisals team.

• The paper is then submitted to the senior management team (Guidance Executive) who consider the evidence and make the final proposal for the recommendation.

• NICE then consults on the proposal with consultees and commentators (manufacturers, patient groups, NHS Trusts, Royal Colleges and other interested parties).

• The proposal is posted on the NICE website.

Decision paper for Guidance Executive

• At the end of the consultation period a summary of consultation responses is collated.

• This is compiled by the information specialist with input from the technical analyst and project manager where necessary.

• Comments from consultees are tabulated and responses are given.

• The paper is agreed by an Associate Director and then goes to GE for consideration.

• Consultees are advised of the final decision and this is also posted on the NICE website.

Information Specialist work

• Timelines• Reading and understanding the existing guidance• Identifying the challenging areas for search purposes• Searching • Understanding the findings and the implications• Communication with analyst in advance of presenting

the findings• Identifying the best recommendation, according to the

findings

Challenges of TA99

• Sifting for relevant information - how relevant might data on adults be?

• Choosing what to recommend as the way forward – using and understanding the evidence base

• Information synthesis – making a robust case for review• Time – not unlimited• Transparency

RPP skills

• Process driven work• Searching – finding the key information• Judgement - understanding and interpreting the findings

(for example what’s pivotal, what will make a material difference to our existing guidance)

• Decision making based on the evidence – make the proposal

• Project management, and being an active team player• Communications skills

Summary

• Information specialists use a wide range of skills when supporting the scoping process and the reviews of health technology appraisals

• Information services team do 3 RPPs and 7 scopes per month in addition to work for other Directorates at NICE

• Any questions?• caroline.miller@nice.org.uk; toni.price@nice.org.uk

sophie.robinson@nice.org.uk;• www.nice.org.uk