A modular self-education tool for tracking patient response to treatment changes: A naturalistic

study of ADHD treatment in Adolescents.

1

igme@shaw.ca**To whom correspondence should be addressed

4/24/23

2

Table of ContentsTABLE OF CONTENTS................................................................................................3ABSTRACT..................................................................................................................4INTRODUCTION.........................................................................................................5BACKGROUND............................................................................................................5METHODS...................................................................................................................6RESULTS.....................................................................................................................8

RELIABILITY.................................................................................................................8VALIDITY.....................................................................................................................8DISCUSSION...............................................................................................................12

REFERENCES............................................................................................................14HOW TO DOWNLOAD TRACT.................................................................................16World Health Organization: Compendium of Emerging Technologies Link.................16

3

AbstractObjectivesMeasuring clinical and functional outcomes in mental health is becoming an increasingly complex and costly endeavour. The purpose of this paper is to describe a means of measuring functional and clinical outcomes in in clinical practice that may be used for self-assessment and education.

MethodsRetrospective data was collected 24 Attention Deficit and Hyperactivity Disorder (ADHD) case files over the treatment period for each case. The design constituted a naturalistic cross-over trial based on a national 6 month suspension of one drug (Adderall XR). Clinically trained raters, who were blind to the cases and medication regimes, were given instruction on how to rate the visit data extracted from each case. The ratings provided the basis to examine inter-rater agreement (Kappa) and validity of the extracted clinical material (counts within treatment categories). ResultsOverall level of agreement between two independent raters was excellent (Kappa = .75). Summed ratings across subjects classified medication regimes demonstrating discriminant validity.

ConclusionsClinicians were able to use a simple modular approach to measuring complex phenomena with valid and reliable results. In the field measurement the effect of multiple interventions in the treatment of complex disorders may be useful in identifying and evaluating adherence to evidence-based practices.

4

IntroductionStandard approaches to physicians monitoring patients’ responses to treatment and clinical intervention are critical to understanding the effect of clinical education and teaching in respect to dissemination of and adherence to evidence-based practices. There are few systems currently available that are flexible enough to adapt to the range of clinical practices around which physicians may want to develop self-monitoring protocols. To this end, we have developed a flexible computer-based tool that physicians may use over time to define, document, and monitor a range of clinical variables and measures specific to the objectives of their clinical practice.

In this paper, we describe the utility of this clinical feedback system to monitor patient response to standard treatments of Attention Deficit and Hyperactivity Disorders (ADHD) in adolescent patients. It was possible to examine the utility of the self-monitoring protocol through retrospective examination of patients’ responses to this specific change in ADHD treatment.

BackgroundResearch related to the identification, assessment and treatment of attention deficit and hyperactivity disorder suggests that the rate of ADHD in general population is high (American Academy of Pediatrics, 2001; Brownell & Yogendran, 2001; Charach, Cao, Schachar & To, 2006; Child Health Surveillance Project Data Group, 2005; Kotowycz, Crampton, & Steele, 2006; Romano, Baillargeon, Wu, Robaey, & Tremblay 2002). Primary work in the last two decades has been in the area of developing optimal practices for the treatment of ADHD (MTA Cooperative Group, 1999a, b). From such studies, researchers have learned the importance of the role of medication in ameliorating the

5

acute symptoms of ADHD, together with a consistent and integrated multi-disciplinary approach to long-term treatment. For example, the MTA study identified the difference between evidence-based treatment and treatment as usual at community-level intervention. Nevertheless, to date, little is known about the level of adherence and fidelity to practice guidelines among physicians in monitoring the progress of their ADHD patients. This paper describes one approach that may be an aid to physician self-monitoring and thereby improve adherence to evidence-based care of ADHD and, for that matter, other disorders, given the modular nature of the measurement methods.

To the end of establishing a standardized approach to physician self-monitoring it is important to examine the reliability and validity of the approach. To test the reliability and validity of the modular standardized approach to self-monitoring, we undertook to retrospectively measure patient response over time to variations in ADHD treatment. In terms of reliability, we expected an acceptable level of agreement between raters blinded to all grouping data but the sequential visit data summaries for each patient, which permitted the two raters to assess and judge the patient response to treatment at each sequential visit. To examine validity, we examined patient response to different types of medication treatment, under the hypothesis that there would be differences. The study was organized around the Feb 2005 discontinuation and September 2005 reinstatement of AdderallXR in Canada as this permitted a naturalistic open cross-over design in the medications taken by the some of the patients.

6

MethodsThe charts of 24 patients receiving treatment for Attention Deficit and Hyperactivity Disorders (ADHD) were retrospectively reviewed for the purpose of developing a form for physicians to self-monitor patient response to treatment. The form consisted of demographic variables (age and family composition), and variables representing family history of mental illness, diagnosis, co-morbidity and dual diagnosis (e.g., comorbid substance disorder). Treatment variables included medications (including medication start and stop dates, route and dose medications); additional variables included coding of life events (e.g., suicide attempts, abortion, parental separation) and compliance issues. Accordingly, retrospective data extracted from 24 case files of patients who had received treatment for Attention Deficit and Hyperactivity Disorders (ADHD) between the years 2000 and 2007.

To examine the reliability of the treatment response coding scheme, two independent raters coded the clinical visit data according to their judgment using a standardized scale (range –1 for a more negative response at the time of the visit to +1 for a more positive response at the time of the visit, with zero representing no change in treatment response status). The visit data coded by independent raters was used to test the reliability of the judgment coding results.

To examine the validity of the coding scheme, the coded data was analyzed in terms of patients’ responses over time to medications. Standardized rating of patient response to treatment on specific visit dates was made by assigning a value of –1 for poor treatment response or function, 0 for no change and +1 for improved treatment response or function. Forms based on the physician notes from sequential visits were subsequently rated by two independent raters, who were blinded to the type of medication prescribed at the time of

7

the visit. Based on the ratings of individual sessions, a tally of ratings for all sequential visits was calculated over the course of treatment. Hence, a patient might be judged as functioning or responding poorly on visit #1 (value = -1), better in terms of symptoms on visit #2 (value = +1) and better socially (+1) and the same in terms of symptoms (value = 0 for no change) on visit #3 (value = +1 + 0 for that visit). The cumulative frequencies in each category are shown in Table 1. Two illustrative case examples are presented in time in Figure 1. The pooled data were also represented as the ratio over all visits of total positive to negative responses (e.g., Figure 2). There were no substantial differences between responses to medications before or after the dates related to the Heath Canada discontinuation February 2005 and reinstatement in September 2005 of AdderallXR (20 of 24 cases), hence these data were pooled. Graphs and tables are used to describe the summary data. Stepwise regression analysis was used to examine the effect of demographic and treatment variables on function and response to treatment. Figure 3 in the appendix shows the user interface of the tool.

Results

ReliabilityThe overall level of agreement between two independent raters with clinical experience in clinical psychiatry across a total 630 visits for 24 cases was excellent (Kappa = 0.75).

ValidityTable 1 shows responses to treatment ratings for 630 visits by 24 patients being treated for ADHD disorders. Note that the neutral or no change indicators tended to follow positively judged responses to

8

treatment. Hence, a positive response to treatment tends to saturate. Note the raters were blind to the groupings when judging treatment responses across the categories shown in Table 1.

9

Table 1: Treatment ratings for 630 visits by 24 patients being treated for ADHD disorders.

Naturalistic Study GroupNegative Response

Neutral or no change

Positive Response Total

No Medication 4 4 1 9AdderallXR 17 13 22 52AdderallXR + Other Medication 22 27 50 99Other ADHD Medication 66 74 67 207Other ADHD Medication + Other Medication 58 69 72 199Other Non-ADHD Medication 30 17 17 64Total 197 204 229 630

Figure 1 below shows two case examples. Stars and drop lines represent medication changes and the level represents the type of medication change. The abscissa represents each consecutive visit. Corresponding points on the ordinate represent the impact at the time of the visit on the cumulative change in clinical and or social function to that point.

10

Case 1

-0.50

0.51

1.52

2.53

3.54

4.5

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43

Visit

Cummulative Response Medication Change

Case 7

-3

-2

-1

0

1

2

3

4

5

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43

Visit

Cummulative Response Medication Change

Figure 1: Graphic representations of visit ratings for two case examples.

11

Two cases without life events were selected for illustrative purposes. The stars and drop lines represent changes in medication and are defined by the corresponding ordinate coordinate: 0 no medication, 1 AdderallXR only, 2 AdderallXR and other medication, 3 Other ADHD medication only, 4 Other ADHD Medication and additional medication, 5 other non-ADHD medication). Circles represent the cumulative visit response judgment as judged by raters blind to the groupings. The figure shows that each patient has a unique response profile that was judged independently to change with change in medication regime.

Figure 2 shows the ratio of positive to negatively judged visits. Patients treated with Adderall were rated overall as having better function and response to treatment.

Ratio of positive to negative responses for patient visits

0 0.5 1 1.5 2 2.5

No Medication

AdderallXR

AdderallXR + Other Medication

Other ADHD Medication

Other ADHD Medication + another Medication

Other Non-ADHD Medciation

Figure 2: Ratio of total positive to negative responses to treatment by class of medication. Ratios less than 1 indicate a preponderance overall of negative visit ratings (e.g., No medication, Other Non-ADHD Medication).

12

DiscussionEvidence-based practice in the assessment and treatment of ADHD is well described in the literature by the American Academy of Pediatrics, 2001 and this has led to the development of organizations in North America that exist to disseminate this information related to evidence-based practice to physicians and families. Two examples are Canadian Attention Deficit Hyperactivity Disorder Resource Alliance (CADDRA) and Children and Adults with Attention Deficit Hyperactivity Disorder (CHADD). These organizations keep physicians in the community abreast of optimal evidence-based treatments specifically for ADHD, especially with reference to the use of medication. Ideally, a family can move from Alaska to Florida and have a means to ensure that any family doctor has access to the most current evidence-based information about treatment. Furthermore, physicians also have access to information and strategies to improve their level of practice.i

Even given the level of development of evidence-based practice around ADHD, it remains unknown the degree to which evidence-based practices are actually used by the physicians identifying, assessing and treating this disorder in the community. Some studies describe the level of adherence to evidence-based treatment (Kotowycz et al., 2005, Kuschel et al., 2006). This is particularly important, given the level of community treatment (American Academy of Pediatrics, 2001; Brownell & Yogendran, 2001; Charach, Cao, Schachar & To, 2006; Child Health Surveillance Project Data Group, 2005; Kotowycz, Crampton, & Steele, 2006; Romano, Baillargeon, Wu, Robaey, & Tremblay 2002). It remains important to know the level of fidelity maintained by community physicians. Furthermore, there is no requirement for a physician identifying, assessing, and treating ADHD or prescribing

i http://www.reachinstitute.net/

13

medication for the treatment of ADHD to provide for families evidence that they have even the basic knowledge required for employing evidence-based practices that exist.

The current study provides evidence that a simple tool might be used at the level of community physicians not only to monitor the effectiveness of their treatment, but also to track a range of variables that may influence treatment outcomes. In the present example we have shown that the tool is sensitive enough to detect differences in medication regimes. Furthermore, the results accumulated over time may be used to illustrate graphically to families the effect of a range of salient variables representing events and factors that are known to influence treatment outcomes. The tool’s application is wide given that its economy and flexibility and that the tool may be used to track visits in a standardized way. In fact, even the results of standardized measures may be embedded in the tool and compared to a clinician’s self-rating judgments, permitting empirical examination of interventions and treatments for a range of disorders. As such, physicians and allied professionals may monitor the degree to which they are effective in adhering to evidence-based practice guidelines and the degree to which evidence-based practice guidelines are effective in the field.

14

ReferencesBlew, H., Kenny, G. (2006). Attention deficit hyperactivity disorder: the current debate and neglected dimensions. J Child Health Care,10(3), 251-263.

Brownell MD, Yogendran MS. Attention-deficit hyperactivity disorder in Manitoba children: medical diagnosis and psychostimulant treatment rates. Can J Psychiatry. 2001 Apr;46(3):264-72.

Cawthorpe D, Wilkes T, Rahman A, Smith D, Conner-Spady B, McGurran J, Noseworthy T. (2007). Priority-Setting for Children’s Mental Health: Clinical Usefulness and Validity of the Priority Criteria Score, Journal of the Canadian Academy of Child and Adolescent Psychiatry. 16(1): 17-25.

Charach A, Cao H, Schachar R, To T. Correlates of methylphenidate use in Canadian children: a cross-sectional study. Can J Psychiatry. 2006 Jan;51(1):17-26.

Child Health Surveillance Project Data Group. (2005). Alberta Child Health Surveillance Report 2005. Edmonton, Ab. Alberta Health and Wellness.

Jensen, P.S., Knapp, P., Mrazek, D. (2006). Towards a new diagnostic system for child psychopathology: moving beyond the DSM. New York: Guilford Press

Kotowycz N, Crampton S, Steele M. Assessing the standard of care for child and adolescent attention-deficit hyperactivity disorder in Elgin County, Ontario: a pilot study. Can J Rural Med. 2005 Summer;10(3):149-54.

MTA Cooperative Group. (1999a). A 14-Month randomized clinical trial of treatment strategies for attention-deficit/hyperactivity disorder. Archives of General Psychiatry, 56, 1073-1086.

MTA Cooperative Group. (1999b). Moderators and mediators of treatment response for children with attention-deficit/hyperactivity disorder: the multimodal treatment study of children with attention-deficit hyperactivity disorder. Archives of General Psychiatry, 56, 1088-1096.

Romano E, Baillargeon RH, Wu HX, Robaey P, Tremblay RE. Prevalence of methylphenidate use and change over a two-year period: a

15

nationwide study of 2- to 11-year-old Canadian children. J Pediatr. 2002 Jul;141(1):71-5.

Wilkes, T.C.R., Cawthorpe, D. (2005). Brief child & family phone interview validates provisional diagnostic categories. Poster presentation at Mental Health Research Showcase. Banff, Alberta.

American Academy of Pediatrics. Subcommittee on Attention-Deficit/Hyperactivity Disorder and Committee on Quality Improvement. (2001). Clinical practice guideline: treatment of the school-aged child with attention-deficit/hyperactivity disorder. Pediatrics, 108(4),1033-1044.

Kotowycz, N., Crampton, S., Steele, M. (2005). Assessing the standard of care for child and adolescent attention-deficit hyperactivity disorder in Elgin County, Ontario: a pilot study. Can J Rural Med, 10(3), 149-154.

Kuschel, A., Stander, D., Bertram, H., Heinrichs, N., Naumann, S., Hahlweg, K. (2006). The prevalence of ADHD and attention problems in preschool-aged children. A comparison of two diagnostic instruments. Z Kinder Jugendpsychiatr Psychother, 34(4), 275-84.

The Centre for the Advancement of Children’s Mental Health (CACMH), Columbia University, New York, Mental Health: A report of the Surgeon General. (1999) (Accessed March,2007). www.surgeongeneral.gov/library/mentalhealth/toc.html

Kutcher, S., Davidson, S.,(2007). Mentally ill youths: meeting service needs. Canadian Medical Association Journal., 176(4): 417

16

How to download TRACTTreatment Response Application for

Client Tracking(And For Client and Clinician Education)

Manual Version 1.4 Download information at

http://internationalgme.org/TRACT/IGME_TRACT_2015.htm

This application may be used to track and represent individual patient responses to treatments over time. Additionally, patients may be assigned to specific treatment groups. User defined variables representing treatment and response parameters may be defined across clinically relevant domains. Client notes may be kept to provide details the context of the visit in relation to patient progress or deterioration as this pertains to the scaled clinical judgment across variables defined for the group. (Note that you are required to define domains for rating across all patients – notes are designed draw attention to individual differences or client context that you want track in a specific manner over time). In future versions, some response parameters will be user-defined in terms of range in order to enter the results of standardized tests, which may be compared to scaled clinical judgments for the purpose of validation. In this version, you will need to enter standardized test results in the case note dialogue box. This will permit you the additional advantage of validating your own clinical judgments. Different clinicians may keep separate records based on observations of the same patient in order to compare the reliability of their clinical observations over time. TRACT is an accountability, research and teaching tool.

World Health Organization: Compendium of Emerging Technologies LinkThis application has been included in the following WHO publication:

Cawthorpe, D., Lauchlan, M., (2011) Treatment Response Application for Client Tracking, WHO compendium of new and emerging health technologies: Health Data Monitoring. 20. http://whqlibdoc.who.int/hq/2011/WHO_HSS_EHT_DIM_11.02_eng.pdf

17