1
BISCUIT FINAL REPORT
December 2012
Dr Lesley Wye
Research Fellow
Centre for Primary Health Care
School of Social and Community Medicine
University of Bristol
2
Executive summary
Increasingly, NHS commissioners need information about the costs of services to
help inform their decision making. Currently, commissioners rely almost entirely on
crude cost data to evaluate the economic impact of services, if any cost data are
available at all.
The aim of the BISCUIT study was to pilot approaches to economic evaluation of
homeopathic packages of care delivered at Bristol Homeopathic Hospital. We
wanted to test the methods and outcome tools to facilitate the development of a
larger study that could investigate the differences in resource use between those
who use NHS homeopathic packages of care and those who do not. Our research
question for this feasibility study was:
What are the best methods and best outcome tools for undertaking a large
scale economic evaluation of a cohort of homeopathic patients compared with
a matched control group not using NHS homeopathy?
To answer this, we used a cost utility model and carried out the first steps of a
discrete choice experiment.
Using a prospective matched controlled cohort design, we recruited 29 Bristol
Homeopathic Hospital patients from 315 possible candidates, but only received the
relevant permissions to access medical records and subsequently found matched
controls for nine. Nineteen matched controls returned data.
Qualitative interview data was collected to learn about resource use, experiences of
the study and views on the outcome tools for 23 participants. Quantitative data was
collected through GP medical record extraction of data such as GP and practice
nurse consultations, medications, hospital consultations, tests and investigations. In
addition, participant questionnaires were administered to capture wellbeing (Warwick
Edinburgh Wellbeing tool), quality of life (SF-36) and resource use (Client Service
Resource Inventory). This latter tool collected data on personal costs such as private
healthcare, over the counter products, travel and lost earnings. Qualitative data were
analysed using qualitative content analysis and a framework approach. Descriptive
analyses were carried out on the quantitative data by an independent statistician.
Key findings around feasibility were:
1. There were two major challenges in this feasibility study. The first was the low
level of recruitment of case and control participants. The approach adopted
whereby case participants were recruited directly from the Bristol Homeopathic
Hospital without prior recruitment of their GP practice meant that the pool of
potential candidates was substantially reduced when GP practice permission was
3
not forthcoming. The second challenge was achieving good enough matching of
cases and controls, which also affected the recruitment of controls.
2. Because only 9 of the original 315 potential candidates from the Bristol
Homeopathic Hospital were matched with controls, representativeness is
questionable.
3. The outcome tools were feasible, acceptable, sensitive to change and
appropriate.
4. Cases and controls were well matched at baseline for quality of life and
wellbeing, but not for resource usage.
5. Quality of life for case participants improved compared to controls. Wellbeing for
case participants also changed significantly for cases compared with controls.
However because this was not a randomised controlled study, we do not know if
the improvements seen were due to the Bristol Homeopathic Hospital service.
6. At baseline, costs were substantially higher for case participants. There was no
difference in resource utilisation in the 12 months after baseline between cases
and controls.
7. If these study results were to apply to other homeopathic case and matched
control populations, then 90 participants (1:1 matched) would be needed in a
larger study to demonstrate that the quality of life and wellbeing improvements
generated by homeopathic packages of care were worth their cost. But because
case and control participants were not well matched at baseline for resource use,
we do not have complete confidence in this figure.
8. Individual variability in resource utilisation for both cases and controls was
substantial.
Given these findings, results should be taken with caution.
In answering our research question, we found that the prospective controlled cohort
design produced difficulties in recruitment and matching case and control
participants. But the outcomes tools of the SF-36, Warwick Edinburgh Wellbeing and
Client Service Resource inventory were acceptable and useful.
The other aspect of this study was to identify key attributes of homeopathic
packages of care for a discrete choice model. From interviews with the Bristol
Homeopathic Hospital participants, we found the most valued aspects of the
treatment were:
positive impact on health
the questions that the homeopaths asked
personalised approach
The most valued aspects of the service were:
extended length of the consultation
convenient location of the clinic
not having to pay for the service
4
In thinking about designing a larger study, we considered the use of a traditional
randomised controlled trial design, which would address known and unknown
confounders arising in through matching and lead to the possibility of uncovering a
causal connection between the Bristol Homeopathic Hospital service and any
changes brought over time. But this type of methodology is difficult to use in
evaluating a long-standing service (rather than a treatment) as it does not allow for
real life decision-making on behalf of patients. There are also ethical issues about
denying patients access to an existing, publically funded service, once patients have
requested referral.
An alternative could be the use of a Zelens design. In this type of design, the top 20
practices referring into the Bristol Homeopathic Hospital would be recruited first,
thereby avoiding the experience in this study of the loss of 40% of participants from
GP practice refusal. Then, we would recruit a cohort of patients experiencing the top
five conditions treated by the Bristol Homeopathic Hospital from these 20 practices.
As patients are referred into the Bristol Homeopathic Hospital and recruited, they
would be matched with community controls from any of the 20 practices on the basis
of condition, symptom, age within 5 years and a resource use proxy measure of GP
consultations (within 2 consultations) in the past year.
The advantages of this approach is that it reflects real life by not tampering with
patient decision-making and would hopefully lead to much closer matching at
baseline between case and control participants. The disadvantages are that without
randomisation, known and unknown confounders could affect any observed changes
over time and there is no blinding. Moreover, we know that individual doctors do vary
substantially in their referral patterns, which would impact on patient resource use,
and this needs to be taken into account in this design. Nonetheless, this Zelens
approach would be a step closer to developing a robust study of economic
evaluation of the Bristol Homeopathic Hospital.
Although undeniably challenging, the importance of designing good quality economic
evaluations for generalist services like the Bristol Homeopathic Hospital is important.
Therefore, we would encourage future researchers to work through the
methodological difficulties of conducting economic evaluations of services to ensure
NHS commissioners - and the wider public - have the information necessary to make
appropriate judgments.
5
Acknowledgements
We thank our funders, Blackie Foundation, the British Homeopathic Association,
South West GP Trust and United Hospitals Bristol Above and Beyond Fund.
We thank all of the case and control participants who have taken the time to respond
to questionnaires. We also would like to thank the GP practice staff who gave
permission to access patient records and those who carried out the searches. In
addition, Susan Barron, departmental manager at the Bristol Homeopathic Hospital,
and the Homeopathic Hospital staff were instrumental in recruiting case participants.
Vicky Burrows, a fifth year medical student, dedicated a period of self-study to help
set this project up. Patricia Martens continued with the administrative tasks. Hallam
Wye checked data entry. Tom Griffin was the independent statistician. Lorna Duncan
conducted the majority of participant interviews and acted as research assistant.
Thanks also go to our steering group and wider advisors. The steering group
consisted of Dr Elizabeth Thompson, lead clinician at the Homeopathic Hospital,
Professor William Hamilton (Peninsula Medical School) and Professor Debbie Sharp
(University of Bristol). Many economists contributed their time and expertise
including Dr
Top Related