TCSHSS Evaluation Report

Tarrant County School Health Surveillance Pilot Project

First Year Evaluation Report

Prepared for Tarrant County Public Health and

The Southwest Center for Advanced Public Health Practice (APC)

Submitted May 30, 2008

By Tabatha Powell, MPH

of 57

About this Document: Readers of this report should begin with awareness of the following considerations:

1. Compliance with Research Rules and Usage Restrictions:

Authority: The School Health Surveillance System Pilot Project proposal and its contents have been submitted and approved under the EXEMPT status through the University of North Texas Health Science Center School of Public Health Office for the Protection of Human Subjects, IRB #2007-111. The staffs of Tarrant County Public Health and the Southwest Center for Advanced Public Health Practice support the conduct of the project and its evaluation. Limitations: Please note that the data provided in this report is presented in summary format and is limited so as to protect the confidentiality and identity of the individuals, schools, districts, and organizations that participated in the online evaluation survey and interviews. Independent School District and campus-specific participation rates, reporting frequencies, or percentages will not be shared at a level that would identify the reporting jurisdiction(s). Usage restrictions: Any unauthorized use of the information provided in this report or data collected using the School Health Surveillance System without the approval of Tarrant County Public Health and the Southwest Center for Advanced Public Health Practice is prohibited.

2. Author’s Role:

The primary author for this report has also contributed to the development and implementation of the project. The evaluation framework was developed by graduate students (primary author as team leader) at the University of North Texas Health Science Center, School of Public Health as a course requirement for HMAP 5450 Public Health Program Planning and Evaluation during the fall semester of 2007.

3. Source Citations and References:

This report follows established conventions concerning source citations. For the reader’s convenience, each in-text note or reference is a hyperlink to the precise location in the notes and references section where that reference information appears. Simply click on each of the superscripted and hyperlinked numerals to access that particular note or reference. Also, each section title in the table of contents is a hyperlink that readers can use to access the various sections of the report.

of 57

TABLE OF CONTENTS

I. EXECUTIVE SUMMARY ……………………….…………………………............... 4

II. BACKGROUND ………………………….…………………………………………... 5

Project Participants and Roles ……………………………………………………… 5

Project Purposes and Goals ………………………………………………………… 6

System Description …………………………………………………………………. 8

III. EVALUATION ……………………………………………………………………….. 11

Overview and Methodology ………………………………………………………... 11

Part 1: System Performance ………………………………………………………... 15

Part 2: Recruitment and Training …………………………………………………... 19

Part 3: System Use …………………………………………………………………. 20

Part 4: Data Management …………………………………………………………... 21

Part 5: Public Health Response ……………………………………………………... 23

IV. RECOMMENDATIONS ……………………………………………………………... 24

V. CONCLUSIONS ……………………………………………………………………… 27

VI. ACKNOWLEDGEMENTS …………………………………………………………... 27

NOTES AND REFERENCES ………………………………………………………... 28

APPENDIX …………………………………………………………………………… 32

APPENDIX A: Biography of Tabatha Powell, MPH ……………………………… 32

APPENDIX B: School Health Surveillance System Pilot Project Logic Model …... 33

APPENDIX C: Evaluation Matrix with Observed Outcomes ……………………… 34

APPENDIX D: School Nurse Communications Pilot Project Fact Sheet ………….. 41

APPENDIX E: School Nurse Survey: Frequent/Sporadic Use of System ………… 42

APPENDIX F: School Nurse Survey: Non-User of System ……………………….. 47

APPENDIX G: Key Informant Interview Questions ………………………………. 50

APPENDIX H: Screen Shot of Share Your School’s Health Data Report Form …... 51

APPENDIX I: Screen Shot of ISD Home Page ……………………………………. 52

APPENDIX J: Screen Shot of Flu Prevention Resources Overview ………………. 53

APPENDIX K: Screen Shot of Regional Outbreak Maps for Influenza-Like Illness 54

APPENDIX L: Screen Shot of Project Status and Background Overview ………… 55

APPENDIX M: Guidance for Parents for Reporting Flu to Their School …………. 56

APPENDIX N: Guidance for Parents for Reporting Flu to Their School, Spanish ... 57

of 57

I. EXECUTIVE SUMMARY

This evaluation report assesses the progress of Tarrant County Public Health (TCPH) and the Southwest Center for Advanced Public Health Practice (APC) upon completion of the first year of a pilot project intended to enhance school health surveillance primarily via development and implementation of a Web-based communications portal. The report describes and assesses the system components, reviews the measures used to assess the system’s effectiveness, covers intended outcomes and actual results, and offers recommendations for the communications portal and public health’s partnership with schools. This report follows guidance from the Centers for Disease Control and Prevention (CDC) on evaluating public health surveillance systems1 to evaluate operational aspects of the system. It also reflects individual assessments, obtained via “key informant” interviews, of system developers, users and others. Key findings: • The first year of the pilot project can be considered mostly successful because many, though not

all, of the program objectives were achieved.

• Several challenges encountered were either resolved during the first year of activity or are targeted for resolution as the program evolves. Short-term evolution (second program year) is expected to include expansion to schools in both Dallas and Denton counties, continued efforts to engage more schools in Tarrant County, and the addition of school-based health clinics and child care centers in Tarrant County.

• The Web-based communications portal is a viable system for the automated collection of school health data. TPCH and the APC trained more than 200 school nurses to use the system, showing them how to share their school’s health data by using an online report form that they could complete in less than five minutes.

• The work needed to automatically analyze the data received using syndromic surveillance software was not completed in the first year. This factor, coupled with the low rate of initial participation in online school health reporting, made the system incapable of serving immediately as a stand-alone method for the early detection of health events such as a potential pandemic influenza strain.

• The system’s current limitations for early event detection do not mean it has no value. The system has shown potential, if used in combination with other biosurveillance systems and data, to help public health monitor community health trends and respond to findings in a focused, more effective partnership with schools, where infectious diseases often emerge first and spread rapidly.

• Once school health data is automatically analyzed using syndromic surveillance software and if

school participation rates improve, the system will be more useful as a means of health event detection and tracking; its efficacy for these and related purposes can then be more fully evaluated than is possible now.

• Because the Web-based communications portal uses open source software (no licensing fee required), it is an affordable, replicable system other public health agencies can consider to strengthen their own school partnerships and biosurveillance capabilities.

of 57

II. BACKGROUND Project Participants and Roles Key participants of the pilot project that this evaluation report addresses include:

• Tarrant County Public Health (TCPH): One of the Southwest’s leading public health agencies since its inception in the 1950s, TCPH has been a valuable local resource by providing services to all Tarrant County residents aimed at promoting, achieving and maintaining a healthy standard of living. With a diverse client base and scope of services, a staff of more than 300, and annual funding resources exceeding $33 million, TCPH fulfills the CDC’s 10 Essential Public Health Services.2 TCPH’s Epidemiology Division, led by Dr. Anita Kurian, works closely with schools and was heavily involved in all aspects of the pilot project.

• Southwest Center for Advanced Public Health Practice (APC): The APC, a grant-funded unit of TCPH, is an emerging national leader in public health informatics and emergency preparedness and readiness assessment. The APC’s surveillance network is advancing health situational awareness.3 Its exercises and training materials address the unique challenges of pandemic influenza and radiological and chemical threats. APC Manager Bill Stephens, Regional Surveillance Coordinator Dean Lampman, and Surveillance Database Manager Dave Heinbaugh were all key contributors to planning and implementation of the pilot project.

• Tarrant County school nurses: Students in Tarrant County, with its population of 1.7 million4, attend more than 500 schools in 16 Independent School Districts (ISDs) and portions of four additional ISDs, and in nearly 100 private schools5. The pilot project seeks to collect and analyze health data from these schools through the support of school nurses. In the future, recruiting is expected to be extended to school-based health clinics and child care facilities.

• Texas Department of State Health Services (DSHS): To improve health and safety in Texas, DSHS “uses the principles of epidemiology to prevent and reduce infectious disease morbidity and mortality” and “prepares Texas to effectively detect, investigate, control and prevent natural and manmade public health disasters.”6 A model for the School Health Surveillance System evaluated here was first piloted in East Texas. DSHS Health Service Region (HSR) 4/5N worked with schools to collect data on absenteeism and Influenza-Like Illness (ILI) from at least one school in each of the region’s 35 counties and analyzed it with a rudimentary syndromic surveillance system. Lessons learned there influenced the Tarrant County project.

• University of North Texas Health Science, School of Public Health (UNTHSC SPH) and graduate student Tabatha Powell, MPH: Through doctoral residency programs and masters-level internships, UNTHSC SPH has partnered with TCPH and the APC to support research-based applied public health opportunities for its public health students. Tabatha Powell, a doctoral-level epidemiology student at UNTHSC SPH, first began working in school-based influenza surveillance activities in 2003 with DSHS HSR 4/5N as a Public Health Prevention Specialist with the Centers for Disease Control and Prevention (CDC). She helped conceive and implement the Tarrant County pilot project during her doctoral residency with the APC and is lead author of this report. As noted in the acknowledgements, other UNTHSC SPH graduate students and faculty also worked on the evaluation and report, as did APC staff. Appendix A is Ms. Powell’s full biography.

of 57

Project Purposes and Goals In March 2007, the APC initiated development and design of a Web-based communications portal it hoped would leverage an existing regional syndromic surveillance system, strengthen an already productive partnership between TCPH and Tarrant County school nurses, and facilitate enhanced surveillance of absenteeism and ILI in schools. Specifically, the project was intended to:

• Slow the spread of flu (and reduce its magnitude and impact) throughout Tarrant County by enhancing information exchange between TCPH and local schools on flu and other matters of public health importance.

• Support early detection of ILI by making it easier for schools to report absenteeism data and ILI among school-aged children and youth.

• Provide easy access to various resources that can help school nurses promote disease prevention and health promotion activities among nurses, staff, students, and parents in Tarrant County.

• Focus public health resources in response to early detection of increased ILI and student absenteeism rates. Capturing data on ILI yields insight on how the student population is being affected throughout the school year and especially at the height of the flu season. This data, coupled with laboratory findings, Sentinel Provider Surveillance Network (SPSN) data, and hospital emergency department syndromic surveillance data, can help epidemiologists more precisely characterize flu and its community impact and help public health leaders target limited resources to control its spread.

The project’s focus on school-aged children reflects their significant role in the transmission cycle of influenza and other diseases. School-aged children are common vehicles for transmitting influenza to other persons in their households, including those categorized into high-risk groups (e.g., the elderly, young children, and immuno-compromised individuals).7 Detecting outbreaks of flu and other diseases in school populations can facilitate the development and implementation of timely public health recommendations to prevent the spread of disease to high-risk populations. The project’s focus on flu reflects the importance of containing seasonal flu, which results in approximately 36,000 deaths and 200,000 hospitalizations each year8, as well as the potential for pandemic influenza. Influenza epidemics have significant negative effects on community health and can also hinder the economy.9 The severity of the disease and its duration causes increased work and school absenteeism.10 Another factor that has fueled increased interest in influenza surveillance is growing recognition of the importance of early detection to facilitate effective response to a pandemic.11 Responding to the threat of pandemic flu and new and emerging diseases such as Severe Acute Respiratory Syndrome (SARS), the U.S. Department of Health and Human Services, in collaboration with the Centers for Disease Control and Prevention (CDC), has provided resources to facilitate public health preparedness planning tailored to a school setting. An example is the School District (K-12) Pandemic Influenza Planning Checklist, which recommends that schools should “consider developing, in concert with the local health department, a surveillance system that would alert the local health department to a substantial increase in absenteeism among students.” 12

of 57

While the Flu Planning Checklist recommendations are helpful, they don’t go quite far enough. That’s because research has indicated that absenteeism data itself is not enough to assist the health department in assessing the causes for the increases in student absences.13 Public health staff planning appropriate response to health patterns must also know the reasons for student absences. Unfortunately, schools are not equipped to obtain specimens for laboratory testing, nor do they have access to physician diagnoses to confirm cases of influenza or other conditions. They can, however, classify a child as having ILI. The list of ILI symptoms is short, reasonably clear, and certainly understandable to school nurses (who, while not always physicians or nurses, are usually trained health professionals). Recognizing these realities, the APC and TCPH saw the potential for developing a system that would enable local health departments to collect school absenteeism and additional health data. With help from a subject matter expert, Tabatha Powell, MPH, the APC began to formulate plans for a pilot project in which:

• Schools would be asked to share their data electronically, more frequently and more completely than they had previously14, but be given the means to do so in the form of a Web-based communications portal with a report form designed to allow school nurses to share their school’s health data within five minutes or less.

• Data from schools would be analyzed using the school absenteeism module of a syndromic surveillance software system, the Electronic Surveillance System for the Early Notification of Community-based Epidemics (ESSENCE), as established in Tarrant County.15 (The CDC defines syndromic surveillance as “using health-related data that precede diagnosis and signal a sufficient probability of a case or an outbreak to warrant further public health response.” 16)

• Absenteeism would not be the only data shared. (TCPH did have a few individual schools reporting ILI and total absenteeism data via fax and e-mail in the 2005-2006 school year, and also early on in the 2007-2008 via those same methods. However with only a few schools reporting, the data was not representative of all Tarrant County schools. This led to a decision by TCPH’s Epidemiology Division to collect data weekly at the ISD level, still by fax and e-mail, for generalization of findings.) The system the APC envisioned would seek absenteeism data daily, as well as information about students with ILI, and would ask for nurse perceptions of the level of flu activity seen on campuses—with specific measures of ‘increasing,’ ‘decreasing,’ or ‘unchanged’. As such, the APC’s planned system would seek more data, at a more granular level, and on a timelier basis, than any methods TCPH had used previously.

• Dialogue between public health and school nurses would be accommodated via an online venue that would also conveniently organize and serve up disease prevention and health promotion resources appropriate for use with students, parents and school faculty.

• A flu prevention campaign would be developed to provide materials for school nurses, students, teachers, and the media with pertinent health promotion and disease prevention information made available through the Web portal.

of 57

The following is a summary of intended outcomes of the pilot project: Intended outcome goals: Description: Integrate syndromic surveillance data into schools

Syndromic surveillance augments traditional surveillance by showing where and how seasonal flu is emerging and spreading.17 Its use with schools can help public health focus its efforts with schools to coordinate responses.

Extend the current public health community of practice in North Texas.

This will facilitate healthcare problem solving with school nurses and other facilities throughout the region while also facilitating communications between area public health departments by providing a common toolset for regional health surveillance with a focus on seasonal flu.

Add a critical surveillance data and health intervention point

Since school-aged children are highly vulnerable to flu, they are often the primary vehicles for transmission of flu in a community. Influenza viruses cause disease among all age groups, but according to the Advisory Committee on Immunization Practices (ACIP), rates of infection are highest among children.18

Implement a comprehensive flu surveillance system in North Texas and improve preparedness for pandemic influenza.

While the project’s focus is on seasonal influenza, many of the same interventions and best practices recommended for mitigating seasonal flu also apply to pandemic influenza. Schools will also be able to meet the recommendations for planning and coordination with the local health department to develop a surveillance system that would alert the local health department to a substantial increase in absenteeism among students as described in the HHS Pandemic Influenza Checklist for Schools (K-12).12

Strengthen the quality of surveillance data in North Texas.

The current syndromic surveillance data (largely focused on hospital ED chief complaints, as well as pharmacy sales data) will become even more valuable when combined with school and other key data points.

Serve as a model for expansion with similar initiatives in Texas

The methods used are both cost effective and highly replicable.

System Description The School Health Surveillance Pilot Project was initiated by the Southwest Center for Advanced Public Health Practice (APC) and Tarrant County Public Health (TCPH) in March 2007. System development began in June 2007. A School Nurse Advisory Committee (SNAC) consisting of school nurses who attended the Education Service Center Region XI Spring Workshop in May 2007, was formed and convened in September 2007, to provide feedback to the APC on system layout, design, and content. Due to an unforeseen delay in funding allocations for the APC’s project activities, the APC developed the system layout and design without use of a contractor. It chose to build the system using a royalty-free, Web-based, open-source portal platform called DotNetNuke.19 After completing several periods of system review and revisions, TCPH launched the system on November 5, 2007 to school nurses throughout Tarrant County who were notified via e-mail of the system’s availability. Starting November 5, TCPH began to use the system to collect data from nurses. System log reports also began to show the number of reports submitted by nurses, the frequency with which nurses viewed the various Web pages available to them, the frequency of user login problems, and other useful data.

of 57

A primary focus of the system is to collect daily student absenteeism data from school nurses. Another key data field that distinguishes this project from others is the collection of the number of students absent and seen by the nurse with ILI. The case definition of ILI, as defined by CDC, is “a person demonstrating a measured temperature greater than 100 degrees Fahrenheit AND cough AND/OR sore throat. Other symptoms include malaise, muscle aches, runny nose, chills, and/or headache.”20 To better describe the project activities and intended outcomes, a logic model was developed to serve as a guide in the development and conduct of the system and project evaluation (Appendix B). This report’s author developed the logic model using CDC guidance and in consultation with students and professors from UNTHSC SPH. After reviewing the current method used for reporting absenteeism from the Independent School Districts (ISDs) in Tarrant County, TCPH and the APC determined that there were five primary areas in which school-based influenza surveillance could be enhanced:

1. Reporting method

2. Timeliness of data

3. Data specifications

4. Communication tools

5. Public health response Prior to the pilot project, TCPH had for several years collected attendance data from 15 of the 16 ISDs in Tarrant County. Lead district nurses reported the total number of students absent for the week by close of business every Tuesday for the preceding week sending the data via fax or e-mail. This report included an aggregate number of student absences from all school campuses in participating ISDs. Data was collected by school nurses using a district-wide attendance system or by other means. As part of its system launch communications, TCPH and the APC developed and distributed a fact sheet that compared the previous methods used to the new, enhanced methods. Here is an excerpt of the fact sheet (the full version is provided as Appendix D). Program parameter Current method New method Reporting frequency Weekly Daily Reporting entity School district Individual schools Reporting method to TCPH Fax, with some via e-mail Web-based upload of report form Data analysis by TCPH Manual (via Excel & SPSS) Automatic and extended in ESSENCE Reporting to schools Manual process in which

influenza activity reports are sent weekly via email; additional e-mail or phone calls are placed as needed

Automatic and extensive via access to ESSENCE, with online postings of influenza activity reports weekly and with additional reports provided online or via e-mail or phone when warranted

Tools provided to schools Limited to fact sheets and reports issued as needed

Adds access to school nurse view of the ESSENCE system and new, valuable resources in a comprehensive campaign

of 57

Implementing the School Health Surveillance System with schools was key to addressing barriers that had been identified. To address those barriers and meet the needs the APC and TCPH had identified, the School Health Surveillance Web portal was designed to include five primary Web pages that system users (school nurses) can access:

1. Share Your School’s Health Data, which is the default landing page (see Appendix H)

2. Independent School District home page (see Appendix I)

3. Flu Prevention Resources (see Appendix J)

4. Flu and ILI Outbreak Maps (see Appendix K)

5. About ESSENCE. Below is a description of each of these pages: Share Your School’s Health Data: After users log in to the system, they are directed to the Share Your School’s Health Data Web page (see Appendix H). This is the report form TCPH uses to collect enhanced influenza surveillance data from school nurses. They report campus-level data regarding student absences due to all causes and student absences due to ILI on a daily basis. Data fields that are pre-filled include school name, ISD, enrollment, and report date. The primary data of interest include: total number of students absent, total absent due to ILI, number of students seen in nurse’s office, number of students seen with ILI, perception of ILI, and school closure. Supplemental data include faculty absences and faculty absences due to ILI. Independent School District (ISD) Home Page: Each ISD has its own “home page” (see sample for the Birdville ISD in Appendix I), which includes a banner that displays the district logo. For private schools, the Education Service Region XI logo is displayed. All schools have access to and view the same modules, links, and files on the Web page. The only difference from district to district is the ISD banner and action items that may be posted that apply to a specific district. The three modules that compose this Web page are: 1) The Tarrant County Public Health News and Analysis module, which TCPH staff update with the Tarrant County Weekly Influenza Report, current news article summaries, and pertinent health advisories. 2) Helpful Resources module, which provides a link to the Flu Prevention Resource Web page, an archive of weekly flu reports, Nurse-to-Nurse articles, Public Health Notes, and links to the Health Alert Network, TCPH home page, and CDC main flu page. 3) About This Portal, which offers a link to the project background page to give users the current status and an overview of the pilot project. This page has two important purposes: 1) School nurses who miss School Nurse Advisory Meetings or other pertinent briefings can go to this page to access the valuable information they missed at these meetings; and 2) The page provides TCPH and APC stakeholders and funders with a similar ability to review what the APC and TCPH have been working on and saying to the school community. Users can also click on a link that directs them to an e-mail format to provide comments or feedback to TCPH.

Flu Prevention Resources: There are three primary modules on this Webpage (see Appendix J). Flu Prevention Resources spotlights those resources appropriate for immediate use during the current sub-season of the annual flu cycle.21 This module also provides a drop-down menu of all resources listed alphabetically that are available to users.

of 57

Resources in Spanish is a list of all available resources that are written in Spanish. Tools to Share with Teachers is a subset of the alphabetical list of all resources that have been reviewed to determine age appropriateness. Most flu prevention resources posted are those tailored to school nurses and parents; some are items nurses can share with teachers. Flu and ILI Outbreak Maps: Maps generated in ESSENCE using hospital emergency department chief complaint admittance data are displayed on this Web page (see Appendix K), although plans ultimately call for the page to convey school absenteeism data as presented in ESSENCE. Users can view snapshots of ILI activity in the DFW region, Tarrant County, and four county quadrants (Northeast, Southeast, Northwest, and Southwest Tarrant County). An archive of previously posted maps is also accessible. Maps are updated daily and reflect data from the previous day. About ESSENCE: ESSENCE is the syndromic surveillance system that will analyze the school surveillance data and this page describes the system. This page provides a description of the system. The School Health Surveillance System also provides a direct link to access ESSENCE for users who request that capability and obtain their individual log-in credentials. III. EVALUATION Overview and Methodology The system was evaluated to determine whether a syndromic surveillance system with a Web-based communications portal for information exchange between public health and schools could effectively and efficiently detect potential influenza outbreaks and provide the health department with data to appropriately direct and focus public health resources. In addition, the evaluation also assessed whether the system was used effectively and efficiently by school nurses as a communications tool and instrument for reporting health data to the local health department. Through the analysis and monitoring of ILI and absenteeism data (seen as a proxy for a clinical diagnosis of influenza illness), the evaluation assessed whether the data was used to focus resources for targeted specimen sampling, vaccinations, and public health messaging to prevent and reduce the spread of the disease to the community in Tarrant County. This evaluation used the CDC’s Updated Guidelines for Evaluating Public Health Surveillance Systems22 as a framework for evaluating the components of the pilot project relevant to operational aspects of the system. This evaluation of the pilot project assesses its success in the first year and offers recommendations intended to ensure that the system becomes sustainable and achieves all of its objectives and intended outcomes. The primary focus of the evaluation was assessing the progress of the project to meet the process objectives by measuring the inputs and outputs of the system. The evaluation assesses the potential of the system with regards to certain aspects that can improve and enhance surveillance to more effectively and appropriately allocate public health resources, detect potential flu outbreaks, and provide public health disease prevention and health promotion information. Assumptions Due to limited resources for conducting the evaluation, the evaluation assesses informally whether certain underlying assumptions that affected the design, development and implementation of the project proved to be accurate and whether these were addressed effectively.

of 57

Specific assumptions involved:

• Transfer accountability of the system from design to implementation: The APC was responsible for designing and developing the School Health Surveillance System. They also provided technical guidance and support to manage the Web portal. Upon launch of the system, TCPH assumed responsibility for updating informational resources, action items, and flu prevention resource materials. Analysis of the routine surveillance data and public health response actions became the responsibility of TCPH (transfer accountability). Separately, and in a future expansion of the project, the APC and TCPH (in collaboration with the UNTHSC SPH student with whom both groups have worked) will conduct primary studies using data collected in the system to assess the system’s potential for early detection of increased ILI/absenteeism/flu in schools at the campus level in addition to ancillary studies that will expand the analysis based on preliminary hypotheses gained from assessing the aggregate (ecological) level data.

• System access and competency and skills of system participants: A survey administered to Tarrant County school nurses in May 2007 indicated that most nurses had access to a computer with Internet capabilities and that nurses felt comfortable accessing the Internet. Based on this information, TCPH and the APC assumed Tarrant County schools nurses have access to a computer with Internet capabilities, could therefore use a Web-based system to share their data and would be willing to do so without extensive training.

Methodology Methods used to collect the data and information for the evaluation included surveys, key informant interviews, and system reports. Surveys were administered to school nurses who were categorized as non-users and sporadic or frequent users of the system (see Appendix B and Appendix C). Interviews with key stakeholders were also conducted to assess the usefulness of the system at county, regional, and state levels (see Appendix G). Data on student absenteeism and absences due to ILI were submitted by school nurses using the “Report Your School’s Health Data” form (see Appendix H). System logs and reports were run to obtain other related information. Both primary and secondary data were used in the analysis. Primary data collection focused on the new data points created in the system. Data were abstracted from the surveillance system using the administrative event viewer, site log, and report form functions. Surveys administered to school nurses and other program participants provided information regarding system design and output capabilities. Logs, tracking tools, and other electronic recordings were used to document program activities and were a resource and reference for data analysis in the evaluation. Through the use and analysis of system data, surveys, and key informant interviews, information regarding the focus areas was collated and reviewed. A summary of the findings follows; the information may be helpful to those considering the system’s utility for other health departments. As noted in the listed limitations on page 2 of this report, the data provided here is presented in summary format and limited to protect the confidentiality and identity of the individuals, schools, districts, and organizations that participated in the online evaluation survey and interviews. ISD and campus-specific participation rates, reporting frequencies, or percentages will not be shared at a level that would identify the reporting jurisdiction(s). All data from the evaluation will be kept secure and confidential in the domains of TCPH and APC servers and files.

of 57

The objectives for the pilot project evaluation focus on achieving process activities. Potential short-term, intermediate, and long-term outcomes and impacts will be evaluated in following years when the project has been implemented long enough that more baseline data is available to assess the project’s impact outcomes and sustainability. An evaluation matrix (Appendix B) was developed to provide project staff with an overview of the anticipated objectives and activities that would be conducted through the course of the project’s pilot year and measured during its evaluation. The matrix includes both intended and measured (actual) project activity outcomes and outlines the evaluation questions for each major component of the pilot project. Each question includes the indicator and data source used to measure the intended outcome of the objective. A column labeled ‘Measured Outcome(s)’ records the actual value or outcome achieved. The evaluation of the project compares expected measures to observed measures for each objective according to the activities and outputs recorded throughout the project’s first year of implementation. Those intended outcomes that address or project moderate to long-term impacts will not be assessed in the evaluation of the pilot project’s first year. These outcomes are more appropriately addressed following the first year of the program, when it may be possible to measure and evaluate outcomes with extended impacts. The data collected in the first year of the pilot will be used as baseline data to be compared with future years’ data. Although the matrix will not be reviewed in detail, the report highlights those outcomes that were relevant to the assessment of several main focus areas. There are five main focus areas under which the project was evaluated. The first four focus areas concentrate primarily on process objectives and outcomes while the last area addresses whether the use of the system and the data collected and information resulted in changes of behavior(s), which are considered impact outcomes. Each of the activities in these focus areas are described in further detail throughout this report. The five main focus areas are:

1. System performance

2. Recruitment and training

3. System use

4. Data management

5. Public health response

To assess the system and pilot project’s performance in these areas, numerous data sources were accessed and referenced. For system performance and system use measures, the site log and event viewer reports, found in the administrative area of the Web portal, were run to assess system use and log-in difficulties. Project and TCPH staff files and records were accessed to assess recruitment and training, data management, and public health response.

of 57

Survey Results Two online surveys (see Appendix E and Appendix F) were developed using on online survey instrument23 to assist in the evaluation of the system. The first survey was designed for those school nurses who had not yet begun to report their school data through the system. A second survey was developed for those nurses who have used the system frequently or infrequently. Frequency of reporting was defined as those nurses who had submitted data at least once per week during the course of the 2007-2008 school year. Frequent users submitted data more than one time per week. An e-mail composed by TCPH staff that included links to both surveys was sent to all 16 lead district nurses on April 17, 2008. The e-mail directed school nurses to complete the survey for non-users if they had not yet begun to use the system. For those nurses who had used the system to report, whether frequently or infrequently, they were directed to the survey for system users. The survey was closed on May 16, 2008. School nurses had one month to complete it. There were 11 responses to the survey for system users and 14 responses for non-users of the system. Nurses from seven ISDs participated. Of the nurses who responded, a larger percentage was from elementary schools campuses rather than middle and high school campuses. Readers are cautioned that, with fewer than 30 total responses and no responses from some participating ISDs, these survey results are not statistically meaningful. They have still been included because, while only anecdotal, the data is nevertheless interesting and of some value. Here are some key findings:

• Stated interest in flu prevention resources did not translate to strong use of such resources: School nurses who used the system indicated that they log in to report their data, on average, three days per week, but they “rarely” view the flu prevention resources and other public health information accessible through the Web portal. They attribute this to daily time constraints. Yet nurses who have not used the system expressed interest in the type of information that has been made available through the system.

• Use of free educational video typically delayed: Many school nurses who received the DVD “Why Don’t We Do It in Our Sleeves?” indicated that they have not used the DVD during the 2007-2008 school year, but plan to use it during the 2008-2009 year.24 They anticipating doing so by sharing it with their schools’ administrations, showing it to students, and sharing it with teachers (e.g. health, physical education teachers) contingent upon approval from their school’s administration. Of the nurses who did use the DVD during the 2007-2008 school year, these nurses shared it with their administration, teachers (primarily health, homeroom, and physical education teachers), and students.

• Interest high in other flu-related data from public health: Practically all nurses who responded to the survey were somewhat to very interested in having access to the county’s weekly influenza report (90 percent), ILI maps (73 percent), action items (90 percent), and the ability to provide comments (81 percent). All nurses expressed some degree of interest in having access to the supplemental resources made available through the Web portal, although non-users tended to indicate a higher level of interest in these resources (see Appendix E to view the supplemental resources).

of 57

• Methods of collecting absenteeism data vary: There was an equivalent division between methods for collecting absenteeism data. Half of the nurses who responded indicated that they collect absenteeism data from attendance clerks; the other half use an internal district-wide system from which the data can be downloaded. Most nurses (58 percent) reported that it took them less than five minutes to collect the attendance data and separately less than two minutes to enter it into the system. All nurses preferred using an online system to report data to the health department and more than 90 percent of nurses reported having access to public health information from TCPH with little to no difficulties logging in to the system.

• Nurse office visits generally are not captured electronically: The primary method school nurses use to record reasons for students visit to their office is via paper. Computerized methods were not the primary means for recording this information, although half of nurses indicated that they use a district system for maintaining student health records and tracking reasons for student visits to their offices.

• Nurses want TCPH to be more active on the front line: Nurses were asked to provide feedback regarding what they feel would make the system more useful. These comments focused on modifying the system to enable nurses to see their own data, so they can review the data report log to verify the data and date sent. School nurses also indicated that TCPH could better partner with schools by continuing to promote flu vaccination in school-aged children and focusing efforts on promoting prevention measures (hand hygiene, cover your cough, etc.).

• Non-participants are aware of the system but reluctant to start using it: For those nurses who had not yet begun to use the system, the survey provided them with a description of the system. Nurses expressed interest in learning more about the system’s capabilities and features but were less interested in participating in the system and receiving training on how to use the system, although almost 90 percent of the nurses who completed the survey were aware of the system’s existence and use in Tarrant County as a result of communication with TCPH staff or e-mail communications received from their lead district nurses.

Part 1: System Performance The evaluation of the system’s performance attempts to review each aspect of the surveillance system using the CDC’s Guidelines for Evaluating Surveillance Systems1 to determine the system’s usefulness and potential effectiveness. As is described in the CDC guidance, the evaluation includes an assessment of the system’s attributes, which are addressed in detail below. Level of Usefulness In its current form, the system provides a moderate to high level of usefulness. The system meets one of the recommendations from the U.S. Department of Health and Human Services and CDC for pandemic flu planning and preparedness.12 The system can, even with the limitations of frequency, consistency, and completeness of data, help users detect increases in student absenteeism rates and absences due to ILI among students who visited the nurse’s office. TCPH staff members have used data from the system, in conjunction with other active and passive surveillance data, to direct and focus public health resources such as the testing of patients with ILI.

of 57

TPCH staff members have viewed absenteeism data from the system as complementary data that supplements other flu surveillance data from hospital emergency departments, SPSN participating physician practices, and laboratory findings. In key informant interviews and other dialogue throughout the first year of the pilot project, TCPH staff leaders have noted that the team of epidemiologists is using surveillance data to more effectively target limited public health resources. This includes, but is not limited to, the selection of patients targeted for rapid flu tests and other forms of laboratory tests. Previously, patients were selected for testing based mostly on historical flu season patterns. Now they are chosen largely based on whether they reside in local “hot zones” where increased ILI is seen as occurring or emerging based on current surveillance data. System Attributes The CDC’s Guidelines for Evaluating Surveillance Systems call for addressing a system’s simplicity, ease and method of training, flexibility, acceptability to users, sensitivity, representativeness, timeliness and stability. The remainder of Part 1 (presented on this page, 16, and through page 19) provides an evaluation of the School Health Surveillance system in each of these areas. Simplicity The system has a simple design for the reporting of the requested data and, generally, is easy to use. School nurses indicated it took less than five minutes, on average, to enter and submit daily flu reports. Little data is requested from nurses and most of it can be collected in less than 30 minutes, depending on the type of system the nurse is using (i.e., contacting attendance clerk or using a district-wide attendance system). School nurses have expressed difficulty in obtaining data on the reason for absence to report “total absences due to ILI.” On the other hand, nurses have reported “number of students with ILI seen in their office” because this data seems to be more easily ascertained than total ILI for the student population and does not require as much time to obtain. The CDC’s case definition for ILI is displayed at the bottom of the reporting form. Nurses have been advised to only count students who meet this case definition before entering them into the system. By making this information available on the system’s default landing page, the APC and TPCH hope to increase the system’s predictive value positive (discussed later) and increase the system’s ease of use. The data collected are strictly aggregate numbers and do not contain Private Health Information (PHI) as defined in the Health Information Portability and Accountability Act (HIPAA), such as demographic data, that would reveal the identify of specific students. Reports can be updated by resubmitting the information into the system. At this time, the system is not designed as to enable nurses to review previously submitted reports. To simplify the reporting process further, the APC is considering options and methods for integrating the Texas Education Agency (TEA) Public Education Information Management System (PEIMS) because nurses have indicated that that PEIMS system is widely used to collect and report school attendance data. In addition, the APC is further exploring the possibility of integrating student medical record data that some school nurses have said that they use to track student health.

of 57

Training TCPH staff members were trained to use the system in a 60-minute session. An APC staff member led this training session and provided detailed instructions on the management of the system, which included adding and removing resource documents, displaying ILI maps, and running data reports. School nurses were trained to use the system in a 30-minute session; each received individualized PowerPoint presentation slides with their user log-in and password information. A concise step-by-step demonstration of the system was presented to nurses who attended training sessions conducted with several ISDs. One disadvantage of this method is that nurses did not have an opportunity to participate in a hands-on use of the system during the training session. This may have resulted in increased follow-up time and troubleshooting with nurses who had difficulties using the system. This activity partially meets the intended outcomes as defined in Objective 1.9 (Appendix B) Key informant interviews (see Appendix G) revealed that TCPH staff spends approximately 12 hours per week updating resources on the Web portal. Data cleaning, management, and analysis require fewer than 20 hours per week, on average. This time is anticipated to be reduced significantly once the data is automatically exported into ESSENCE and analyzed (see the data management section of this report). Even then, though, there will be a testing period to ensure that the system is analyzing the data as intended. Meanwhile, mechanisms are in place to ensure that data is routinely backed up. Flexibility The system has shown flexibility to adapt to changing information needs and operating conditions. Modifications to the layout and design occurred as a result of feedback received from TCPH staff and users. For example, a registration link that enabled nurses to register and change existing account data was initially available following the system’s launch. This link was removed when it became apparent that nurses were creating duplicate accounts. TCPH staff also moved the location of the Action Items module from the ISD home page to the portal landing page (ShareYour School’s Health Data form) to make that information more readily visible to school nurses as they share their school health data. This system is also flexible because additional data fields can be added to the report form to capture other health events. Consider, for example, what might be possible with the system if there were a case of food contamination in a school cafeteria. The school nurse could begin to report number of students ill with gastrointestinal symptoms (e.g. diarrhea, vomiting, nausea) followed by communications from TCPH in the Action Items module. Letters to parents and other informational resources could be made available on a resource page viewable only by the district or schools affected by the outbreak. Another potential use for the system is during public health preparedness exercises. This provides an opportunity for schools to use the system in a simulated environment. This work can occur with few or no changes to the data fields and Web pages used for routine school surveillance purposes, ensuring that the system maintains comparable data over time. Acceptability There is strong acceptability for the use of the system at TCPH and regionally. TCPH exhibited willingness, even eagerness, to use the system as a communications bridge between public health and schools, facilitating enhanced influenza surveillance at multiple levels.

of 57

Acceptability of the system among school nurses is mixed. Once schools have been recruited to use the system, use increases significantly among schools within the district recruited. A similar system implemented in Montgomery County, Maryland also had low participation rates initially. Staff responsible for school recruiting in that region indicated their system required a 12- to 18-month timeframe to gain widespread use (see also the recruitment and training and system use sections of this report.) Sensitivity Assessing sensitivity focuses on determining what proportion of the cases of ILI the system detects and its ability to detect an outbreak by monitoring changes in the number of cases over time. In its current form and participation level, the system’s sensitivity is low with respect to both case reporting and the ability to detect outbreaks. The consistent receipt of more ILI data is necessary to distinguish increased absences due to other causes since total absenteeism is not an appropriate proxy or early indicator for flu outbreaks. Increasing the system’s sensitivity and Predictive Value Positive (PVP) would require:

1. Ensuring that the case definition for ILI is followed to reduce disease misclassification. 2. Developing and implementing a process for tracking ILI from those who collect attendance data. 3. Informing parents about the importance of reporting their child(ren)’s reasons for absences. 4. Conducting active surveillance or ancillary studies to compare the number of students absent

with ILI reported through the system and the true frequency or actual number of students with ILI who were absent through validation of the data (physician notes, follow-up with parents after student returns to ascertain reason for absence, etc.)

A more refined distinction between total absences and the proportion of absences attributable to ILI provides public health with information it can use to focus its limited resources more appropriately. As is stated in the CDC guidelines, “a public health surveillance system that does not have high sensitivity can still be useful in monitoring trends as long as the sensitivity remains reasonably constant over time.”22 Moreover, there are opportunities to implement recommendations provided as part of this evaluation that can make the system more sensitive. A more sensitive system will be one that can be used to monitor trends in diseases over time without relying solely on nurse-confirmed reports of ILI. The PVP is the proportion of reported cases that actually have ILI. At this point, the PVP is difficult to assess because, in launching the Web communications portal, TCPH did not rapidly secure full school nurse participation among all targeted ISDs. Even at the end of the first full year of the program, the level of reporting was not sufficiently robust to allow for a proper assessment of the system’s PVP. It is reasonable to assert, though, that the PVP for the system at year end would probably be rather high, since there were likely to be more students who actually had ILI than the number reported. Representativeness According to the CDC, a system is representative if it accurately describes the occurrence of a health event over time and its distribution in the population by place and person. As with PVP, assessing whether the School Health Surveillance System is representative is premature at this point.

of 57

To be worthwhile, an assessment of whether the system is representative will require collecting more data. A future approach could include comparing the reports of ILI by SPSN providers in the county to ILI data reported by schools. It is reasonable to assume the system will be representative when all 16 IDS are reporting data through the system with high participation rates. For now, a dual data collection method is in place, including the long-standing practice of weekly reporting, aggregated for each ISD, and the new School Health Surveillance System. TCPH staff leaders have indicated that the agency will consider halting use of the weekly reporting method if a minimum of 80 percent of schools within each district share their data daily using the School Health Surveillance System. Timeliness To ensure that the data it collects is sufficiently timely to have value, TCPH has recommended that school nurses report their data no later than three days from the actual date of collection. On average, the data collected from nurses in year one were two days old. The average range was one to three days after removing outliers. Although this cannot be considered near real-time data reporting, the data was still sufficiently timely as to have value for public health analysis purposes. TCPH has not insisted on same-day receipt of data from nurses because it recognizes that, for various reasons, some nurses cannot report their campus data in a timely fashion. Nurses who work at more than one school, for example, are among those who indicated they would find it difficult or impossible to share same-day data for multiple schools. Stability The system itself is reliable for collecting and storing the data. Downtime was limited to only one incident of short duration, which was attributed to problems with the county server. The system has been available and operational when needed and stability is satisfactory.

Part 2: Recruitment and Training School nurses, principals, and superintendents were notified of the system’s development and anticipated implementation date in September 2007 via dissemination of letters (Objective 1.1). Each lead district nurse received user log-in and password information for each of his/her school nurses via e-mail. TCPH requested that the lead district nurses forwarded the launch materials to each of their district’s campus-level nurses. This either did not occur or was not effective because use of the system following the launch communications was sparse at best. In response, TCPH staff scheduled meetings with selected district lead nurses to further discuss the program, obtain support for it, and provide training. Although these in-person recruiting activities were largely successful, they didn’t generate the anticipated results until after the school session winter break. The time required for recruitment was underestimated as defined in Objectives 1.5 and 1.6 in the evaluation matrix. APC and TCPH staff benefited from discussions with staff members from the Montgomery County (Maryland) Department of Mental Health and Hygiene, who have implemented a similar program with school nurses. Through these discussions, the APC and TCPH staff learned that delays in recruitment were normal and that the full recruitment of schools in Montgomery County took approximately 12 to 18 months of effort in that region — despite the impact of a mandatory reporting requirement not yet proposed or approved in Texas.

of 57

During the peak of the flu season (February 2008), more than 100 schools from five ISDs reported student absenteeism data. Participation rates increased following TCPH training sessions in which school nurses received individualized slides with their user log-in and passwords. TCPH demonstrated the log-in process, how to use the report form, and the content available on various pages. Nurses got answers to questions on system use, ILI case definitions, and other issues they deemed important. Following the release of user log-in information and passwords to nurses in one school district, log-in difficulties occurred due to confusion about the registration process. During the first few months of the project, a registration link was available for nurses to access and update their contact and school information (i.e., enrollment numbers). Some nurses did not follow APC instructions to update only their enrollment data. Instead, they re-registered their accounts, creating duplicates. When this problem was discovered, the APC addressed it by working with individual nurses to resolve their account discrepancies and to remove the duplicate accounts. In addition, the APC decided to remove the registration link from the log-in page to prevent further instances of the problem. Clearly, the registration difficulties slowed down the progress of the program and may have increased, to some extent, the level of resistance to the program among some school nurses. Part 3: System Use The system software provides several metrics on system use. The following is a summary of use by school nurses since the system launch (November 2007). Reporting trends: School nurses first began logging into the system on November 5, 2007 following e-mails sent to lead district nurses notifying them of the system’s availability. The first report on student absenteeism data was filed on in the third week of November, at which time only one school was using the system to report its data. In the two weeks prior to winter break, there were nearly 100 reports filed by nurses from 29 schools. After returning to school and TCPH staff began implementing their recruitment strategy in full strength, 70 schools across three school districts including a private school submitted approximately 700 reports in January. As flu season peaked in Tarrant County and the state of Texas in February, more than 1,000 reports were submitted from nearly 100 schools crossing four school districts. The total number of reports began to decrease in March with approximately a 10 percent decrease in the total number of schools reporting, although one more ISD began to participate in the program. By April and May, seven ISDs were reporting data through the system, although there were fewer schools within the ISDs reporting data. Reporting trends of ILI data: Although “reason for student’s absence” has been reported by school nurses as difficult to ascertain, school nurses have reported this data, total number of students absent with ILI, beginning in December 2007 and throughout the 2007-2008 school year. Half of school nurses (~60 percent) reported total number of students absent from all causes and total number of students absent from ILI for the two weeks preceding winter break in December. A large percentage (70 percent) of these reports also included faculty absences due to all causes and faculty absences due to ILI (both supplemental data). As flu cases spiked upward in January, peaking in February, school nurses continued to report total absences due to ILI; more than 60 percent did so in both January and February. Nearly 95 percent of school nurses reported the total number of students seen in their office with ILI in both months. By April and May, the number of school nurses reporting total number of students will ILI had decreased by almost half (to 35 percent), while the percentage of nurses reporting the “number of students seen in nurse’s office with ILI” remained high (80 percent).

of 57

An option TCPH has considered involves treating ILI data as supplemental data to increase participation rates and the system’s acceptability with school nurses. However, there seems to be limited benefit in moving this data field to the supplemental data section of the report form. By doing so, there may be less likelihood that nurses will report this information. More research is needed to assess the utility of using “number of students seen in nurse’s office” as a means for extrapolating total ILI in the student population or using this as a proxy for that measure. Defining students absent with ILI more accurately defines the student population that’s ill. This reduces disease misclassification and improves the system’s PVP, sensitivity and specificity, key measures of its efficacy. Without ILI data, the system tracks absenteeism, as PEIMS does, but is no longer a syndromic surveillance system that provides early detection of flu-like syndromes and facilitates effective use of public health resources. Popularity of other system resources: During the height of the flu season in February, there were an estimated 3,000 views of the Share Your School’s Health Data report form after excluding views made by administrative personnel conducting system maintenance. This large number of views can be expected because the report form is the default landing page and the primary purpose of the portal. The flu and ILI maps page was second in number of views per page (approximately 270) followed by the flu prevention resources page (90). It’s encouraging that these pages attracted some use, given their purposes: informing nurses of health trends and facilitating disease prevention. TCPH could generate greater use of these pages by implementing certain recommendations in this report. Part 4: Data Management Management of the system data includes data cleaning, review, manipulation, and analysis. The system was designed with an assumption that the data submitted using the online report form would be exported automatically into ESSENCE via a process that leverages a sophisticated Extract, Transform and Load (ETL) tool (see Figure 1). Analysis of the data was envisioned to occur in ESSENCE, with the expectation that TCPH staff (and ultimately school nurses, if they were interested) could view and save maps depicting school-level increased ILI and absenteeism data by zip code or other defined geographic boundaries in the county. Unfortunately, the data has not yet been imported into ESSENCE for syndromic surveillance analysis using that system’s disease detection algorithms. The APC and TCPH still plan to use ESSENCE for analysis. Their first-year delay in doing so was attributed to:

• A shift in priorities, which occurred when the system launch did not proceed as planned. More time was required to both build the system and engage school nurses in its use than was envisioned. Those efforts took priority over providing an automated analytical capability.

• A temporary lack of technical support for implementation of this portion of the project. Specifically, the APC determined it could not implement the school absenteeism module of ESSENCE without technical support from the system developer, the Johns Hopkins University Applied Physics Lab (JHU APL). Toward the end of the first year of the pilot project, the APC submitted and JHU APL signed a contract to secure the necessary technical support for installing the latest version of ESSENCE and also activating the school absenteeism module.

• Recognition of ESSENCE’s limitations. The school module available in ESSENCE currently only analyzes total student absences. It does not account for all of the additional data variables collected in the School Health Surveillance System. This underscored the decision of TCPH and the APC to focus their initial attention on successful deployment of the Web portal, including the recruiting of the targeted ISDs who were asked to begin sharing campus-specific health data.

of 57

The project to install the new version of ESSENCE is nearing completion, so during the 2008-2009 school year, school data will be able to flow automatically to ESSENCE for near-real-time analysis.

Figure 1: Depiction of Data Flow within School Health Surveillance System

School nurses enter data in the School Health Surveillance System, built on the open source Dot Net Nuke platform19, using on an online form (see Appendix H)that takes five minutes or less to complete. Their data is captured in an Extensible Markup Language (XML) database, where it’s managed and moved using an open source Extract Transform and Load (ETL) tool called Mirth25. The school data is then loaded in the royalty-free ESSENCE database, where it becomes accessible to ESSENCE application users. The methods being employed in North Texas are both affordable and replicable. Moreover, this approach to data gathering, analysis and sharing is planned to be taken soon with daycare centers and school-based clinics.

Data Quality The quality of data obtained through the system is highly variable and difficult to reasonably assess given the still low level of reporting, which has yielded data that is not satisfactory in terms of being complete and representative. When schools participating in the project report total student absences, that data is of high quality. However, inconsistent reporting remains sufficiently problematic that the overall quality of the project’s data, including but not limited to total absenteeism, remains low. In the future, efforts to assess the quality of attendance data could compare system data to TEA PEIMS system. Data quality, completeness and validity should be a focus of future studies. Because the system uses a case definition for ILI that has been accepted by the public health community (defined by CDC), the specificity of the system is increased and there’s a potential reduction in information bias leading to differential disease misclassification. Consistent use of this case definition also improves potential comparability of the data among other reporting systems used across the state and nation.

of 57

As noted in the recommendations accompanying this report, it’s possible to improve data quality. Continued trainings for users and providing nurses with the case definition for ILI (to emphasize its importance and to reduce disease misclassification) are steps that can be taken to increase data quality. Part 5: Public Health Response The level of usefulness section of this report (page 15) described how TCPH has altered and enhanced its use of public health resources in response to the availability of its various surveillance data, including school health surveillance data. Increasingly, TCPH has come to view its surveillance data as pieces of a living puzzle. Some puzzles are easily solved while others are not. To better understand the dynamics of any specific community health pattern or disease outbreak, the more puzzle pieces available for study, the better the chance of seeing and responding appropriately to the emerging (and often rapidly changing) picture. It helps if the puzzle pieces are individually understandable and if their connection to each other is clear or easily ascertained, but that’s not always the case. When it isn’t, the traditional shoe-leather investigative aspects of epidemiology come to the forefront. No single computerized system, or even a collection of such systems, will ever replace those critical components of the art and science of epidemiology. In addition to addressing how TCPH used the School Health Surveillance System in its daily work, this evaluation report considers the efforts of the APC and TCPH to evangelize the system at the local, regional and national levels. With that in mind, the following is a list of outreach activities that the APC and TCPH conducted to increase awareness of the pilot project and the system that supported it. The list encompasses disease prevention and health promotion activities and information sharing activities regarding the project’s design, development, and implementation. Initial Outreach The APC first approached school nurses to discuss a possible School Health Surveillance System during the 2007 Education Service Center Region XI Spring Workshop, held on May 9, 2007. At that event, the APC enlisted the interest of school nurses to participate in a School Nurse Advisory Committee (SNAC) to discuss the content and design of the system, which then was not yet defined but which was anticipated to be a Web portal. Fifteen school nurses representing eight school districts and one private school ultimately joined the SNAC (Objectives 1.1. and 1.2, Appendix B). Development of the Web portal was delayed when budgeted funds were not available in the timeframe initially planned. This led to a delayed system launch, though the system ultimately was available by roughly the time promised in early communications to school nurses, superintendents, and principals. By the second week of November, e-mails were sent to district nurses notifying them of the system’s activation. Some nurses then began to report their data, albeit sporadically. In another early outreach effort, the APC participated in the Carnaval de Salud26 on October 13, 2007. APC set up a booth to offer information, in English and Spanish, promoting hand hygiene. Booth visitors saw a short video clip of “Why Don’t We Do It in Our Sleeves? 24” Those who first successfully demonstrated the preferred method of sneezing and coughing as described in the video received an inflatable soccer ball bearing the TCPH seal and the words, “Mete un gol y quitate la gripe,” or “Score a goal; don’t get the flu.” Participants kicked these soccer balls into a portable soccer goal. They left with their own soccer ball and flu prevention materials that informed them about the importance of influenza vaccination and following proper hand hygiene practices.

of 57

Recruitment of school districts also involved providing the school nurses who participated in the training with the “Why Don’t We Do It in Our Sleeves?” DVD.24 Intended outcomes were only partially met (Objective 2.1, Appendix B). The surveillance system and project also was promoted to increase awareness among public health and medical professionals in Tarrant County. An article describing the project was published in Tarrant County Physician, the Tarrant County Medical Society (TCMS) journal.27 Subsequent Outreach An update to the TCMS article was published in EYE on EPI, a journal of TCPH Epidemiology Division.27 In addition, project status and updates were shared with local epidemiologists at regional epidemiology meetings and the April 10, 2008 state Epidemiology Response Team (ERT) meeting.28 Similar information was shared with local hospital infection control practitioners and epidemiologists at meetings of the Dallas-Fort Worth Association for Professionals in Infection Control (APIC).29 The APC continued to promote the system as the first year of the pilot project drew to a close. Specifically, the APC attended and presented information on the status of the surveillance system pilot project at the 2008 Education Service Center Region XI Spring Workshop held at the ESC Region XI office in Fort Worth, Texas.30 On behalf of the APC and TCPH, this report’s author presented information on the School Health Surveillance System at the The 58th meeting of The James Steele Conference on Diseases in Nature Transmissible to Man on April 24, 2008 in Galveston, Texas.31 As a result of the author’s participation at that conference, numerous inquiries have been made requesting additional information on the system design, development, and implementation. For example, the University of Texas School of Public Health in Houston has requested that staff from the APC and TCPH present a two-day, four-hour session on the system and its components.32 In its various efforts to promote the system and underlying surveillance program, the APC and TCPH have taken time to point out that the system’s success hinges primarily on the support of school nurses. With that in mind, TCPH has added a component to its annual program in which it thanks school nurses and honors them for their contributions. TCPH this year is recognizing school nurses, schools, and school districts who participated in the pilot year of the project by giving them certificates of appreciation. All ISDs participating in the system received a certificate, as did all schools that provided school health data at least 45 percent of the time. IV. RECOMMENDATIONS The data and information from the evaluation tools have been interpreted to provide the APC and TCPH with appropriate recommendations regarding the system’s functionality and the project’s management and expansion. These findings and recommendations will be shared with TCPH staff, APC staff, and other stakeholders. Identified problems and recommended solutions will be explored in depth based on the findings of the evaluation to ensure that every avenue is exhausted in both assessing the initial success of the program and ensuring its continued successful evolution. School nurses provided both positive and negative feedback. Much of the negative feedback focused the difficulty school nurses faced in obtaining the reason for student absences. Research supports the need to seek reason for absence to distinguish ILI from other reasons for absence; its availability can help guide appropriate public health response and use of resources.

of 57

It’s noteworthy that the APC and TCPH have responded positively to the feedback they’ve received from school nurses who have used the system. As one example, the suggested school nurse “action items” module was moved from the school ISD home page to the Report Your School’s Health Data page. This was done to ensure that nurses are viewing this important information while reducing the need for nurses to have to search through the portal to find it. The APC has also established an intranet-style page (a page where access is limited to internal users) to let TCPH and APC staff members submit and track requests for system enhancements. The page already lists several planned enhancements that, if implemented successfully, could improve system use and functionality. The following recommendations encompass those on the Website and the report author’s additional ideas:

• The APC team and TCPH staff must remain engaged in the project and maintain continued discussions with system users. The system must continue to evolve to meet user needs and user dialogue is the only want to ensure that this occurs.

• School nurses and administrators must be informed of and show responsiveness to the importance of the system. More research to further validate the use of ILI in school surveillance is needed but it cannot be conducted if school nurses are unable or not willing to collect and share this data. Obtaining all of the data requested on the data entry form, including total ILI, should not be seen as optional. In many states, reporting is mandatory; consideration should be given as to whether mandatory reporting could / should occur in Texas.

• Data collected from schools must begin to flow automatically into ESSENCE and be accessible to school nurses. The 2008-2009 school year should begin with the provision of reports from TCPH that give school nurses a retrospective review of the 2007-2008 flu season as reflected in school data submitted during the first year of the program and analyzed in ESSENCE. This will represent a promise fulfilled.

• Problems with log-in procedures must be addressed. Consideration should be given to allowing less-restrictive user log-ins and passwords (i.e. no required use of capital letters on first letters of names and spaces between first and last names). Consideration should be given to adopting each ISD’s unique school identifier as the password for each school nurse’s account. The log-in page should have contact information, including a name, e-mail address and phone number of a technical support person, to assist those who have trouble logging in. If the program is expanded regionally, contact information specific to each county should be available on the log-in page to help direct users to the appropriate person.

• Problems with inappropriate data entry must be addressed. To reduce the amount of data cleaning required, limitations on the data field requirements should be defined to prevent characters from being entered into numerical data fields. This will ease the data cleaning and analysis processes, saving time for TCPH staff members currently conducting these activities.

• Special attention must be given to the problem of obtaining total ILI data. School nurses must be encouraged to more accurately assess total absences, but the APC and TCPH must identify and implement ways to help them. One idea discussed in the first year of the program involves issuing communications that call on parents to report flu to their attendance clerks and that ask attendance clerks to then track and share in a timely manner the flu and total attendance data they receive with school nurses (or enter it into the system themselves). This idea has merit and should be implemented in the second year as aggressively as possible.

of 57

• More education and training must be provided to school nurses using or considering use of the system. To report accurately, nurses must know and use the CDC’s ILI case definition.

• Students and parents must become aware of and involved in the system. Consideration should be given to efforts to engage the media in promoting awareness of the system and particularly the recommendation that parents report flu to their school attendance clerks.

• Site maintenance must remain a priority. Resources such as the flu prevention resources page will require continuous review and updates to remain as valuable as possible.

• Further research on system efficacy must be planned and conducted as soon as is feasible.

To assess the level of sensitivity and PVP of the system, consideration should be given to conducting active surveillance or ancillary studies to compare the number of students absent with ILI reported through the system and the true frequency or actual number of students with ILI who were absent through validation of the data (physician notes, follow-up with parents after student returns to ascertain reason for absence, etc.) can be performed.

Recommendations Regarding Expansion The preceding recommendations address the system as currently implemented. The following additional recommendations reflect input received through the evaluation process and focus on addressing issues that may arise as the system is expanded in Tarrant County and regionally:

• Documentation must be addressed. A standard operating procedures manual should be developed to assist new users and organizations in portal implementation and maintenance.

• Technical support requirements must not be understated. Although the portal framework itself is available at no cost, technical support is needed to maintain the system and to address any issues that arise regarding the technical nature of the system.

• The importance of recruitment must be emphasized. Recruitment is an ongoing process that involves continued communication with system users. It’s critical that communication between TCPH and the participating schools and school districts continue on a frequent basis.

• The TEA should become involved. Ideally, the School Health Surveillance System should be integrated with current systems used to collect attendance data. The APC should facilitate discussions with the TEA to further explore the options for such integration.

• Flu should not be the only health issue reported and tracked. Since the design of the School Health Surveillance System is not necessarily limited to sharing data on influenza only, resources should be broadened to include changing immunization practices, tuberculosis prevention and control information, etc. TCPH may need to periodically request information from other programs, but should be able to establish an efficient process for collecting and posting the additional data. For example, a resource page based on topics of interest as identified through the comments section of the report form could be created and linked from ISD home pages. Form letters that nurses can download and send to teachers or parents to address these topics could also be accessible on those pages. Topics of interest to nurses may not be reportable conditions, but information from the health department on common problems, such as head lice and Methicillin-Resistant Staphylococcus Aureus (MRSA), could be included on this page and provide value to nurses.

of 57

V. CONCLUSIONS This evaluation focused on measuring process outcomes with intended outcomes defined by process objectives and the activities of the project. However, the evaluation is limited significantly because the system evaluated is not yet mature at generating targeted levels of data. Similarly, an assessment of impact or effect outcomes, which measure changes in behavior, practices, or beliefs for a system such as that evaluated here, is also limited in this case. Such changes typically occur over a longer period of time than only a year, so assessing such changes after only a year is of questionable value. Overall, the School Surveillance System has been successful to at least some extent in every area for which it can be reasonably assessed at this point. Although not every project objective was met with 100 percent satisfaction, improvement was demonstrated at each level and throughout the duration of the project’s first year. Because recruitment of schools is a lengthy process that involves continued communication and relationships, the amount and quality of the data cannot be properly assessed at this early stage. Following a complete year of reporting from at least a large proportion of schools within each school district, analysis of the data and additional research studies can assess the quality, validity, and meaningfulness of the data. Even without this baseline level of data, however, TCPH was able to review the data that was reported, consider it in relationship to other available surveillance data, and take appropriate public health actions. The usefulness of the system is promising. In their survey responses, school nurses indicated that they’re able to share their data in the portal easily and they’re interested in the information available on the portal. Further program expansion will yield additional insight on the effectiveness of the system. VI. ACKNOWLEDGEMENTS This report could not have been written and compiled without the guidance and expertise of those who contributed to its development:

• Special recognition is afforded to APC team member Dean Lampman for enhancing this report’s content and formatting with invaluable contributions, insight and technical expertise.

• Without the continued support of the APC team, including Center Manager Bill Stephens and

Database Manager Dave Heinbaugh, this project would not have been the success it now is.

• Special thanks and recognition is given to Dr. Anita Kurian and the TCPH Epidemiology Division for their support of the project, for achieving (and sometimes exceeding) expectations for the project’s first year, and for their strong expertise in infectious disease surveillance.

• Special thanks and recognition is extended to TCPH Epidemiology staff members Liz Izaguirre and

Angie Strickler for their extraordinary dedication to achieving the program’s ambitious recruiting and participation goals, for forging partnerships with school nurses and administrators, and for their work in managing and analyzing incoming school health surveillance data.

• Special recognition is given to Dr. Douglas Mains, UNTHSC SPH professor of HMAP 5450 Public Health Program Planning and Evaluation, and course team members Carmen Salone, Sheniqua Michael, and Efua Opoku who developed the evaluation framework for this project.

of 57

NOTES / REFERENCES:

1. CDC. Guidelines for Evaluating Public Health Surveillance Systems. MMWR Supplements. May 6, 1988. 37(S-5); 1-18. Accessed 10/05/2008.

2. Association of Schools of Public Health. Ten Essential Public Health Services. www.asph.org/document.cfm?page=300. Accessed 05/26/2008.

3. According to the CDC, “Health situational awareness is the ability to utilize detailed, real-time

health data to confirm, refute and to provide an effective response to the existence of an outbreak. It also is used to monitor an outbreak’s magnitude, geography, rate of change and life cycle.” www.cdc.gov/biosense/publichealth.htm. Accessed 5/27/2008.

4. Tarrant County’s estimated population for July 1, 2007 was 1,717,435 according to the U.S.

Census Bureau county population estimates; www.census.gov/popest/counties/CO-EST2007-01.html. Accessed 05/29/2008.

5. A list of Tarrant County’s 16 school districts including portions of four additional districts

whose jurisdiction cross neighboring counties is available through the Texas Education Agency at http://deleon.tea.state.tx.us/sdl/Forms/. Accessed 05/29/08. The Texas Private School Accreditation Commission provides a list of private schools accredited through their commission at www.tepsac.org/index.cfm. Accessed 05/29/08. The estimates given for number of schools, both public and private, are from Tarrant County Public Health and are based on the agency’s ongoing work with schools. The precise number of schools fluctuates annually as schools open or close.

6. The Texas Department of State Health Services. Service Descriptions.

www.dshs.state.tx.us/services/descriptions.pdf. Accessed 05/26/2008.

7. Principi, N. et al. Burden of Influenza in Healthy Children and their Households. Arch. Dis. Child. 2004; 89: 1002-1007.

8. Key Facts about Influenza and Influenza Vaccine. Fact Sheet. Centers for Disease Control and

Prevention. www.cdc.gov/flu/keyfacts.htm. Accessed 09/10/2006.

9. Kilmov A., Simonsen L, Fukuda K., Cox N. Surveillance and Impact of Influenza in the United States. Vaccine. 1999;17: S42-S46.

10. Lenaway, D., Ambler, A. “Evaluation of a School-Based Influenza Surveillance System.” Public Health Reports. May-June 1995; 110(3): 333-337.

11. Sebastiani et al. A Bayesian dynamic model for influenza surveillance. Statistics in Medicine.

2006; 25: 1803-1816.

12. U.S. Department of Health and Human Services. School District (K-12) Pandemic Influenza Planning Checklist. www.pandemicflu.gov/plan/school/schoolchecklist.html. Accessed 05/03/2008.

13. Besculides, M. et al. Evaluation of School Absenteeism Data for Early Outbreak Detection, New York City. BMC Public Health. 2005; 5: 105

of 57

www.asph.org/document.cfm?page=300

http://www.cdc.gov/biosense/publichealth.htm.

www.census.gov/popest/counties/CO-EST2007-01.html

www.census.gov/popest/counties/CO-EST2007-01.html

http://deleon.tea.state.tx.us/sdl/Forms/

http://www.tepsac.org/index.cfm

www.dshs.state.tx.us/services/descriptions.pdf

www.cdc.gov/flu/keyfacts.htm

www.pandemicflu.gov/plan/school/schoolchecklist.html

14. Previous county-level influenza surveillance data from schools focused on the collection of total students absences reported and aggregated to the district level on a weekly basis. The differences between the prior method and the system evaluated in this report are summarized in a fact sheet that the APC and TCPH distributed to school nurses (see Appendix D).

15. ESSENCE, a system developed by the Johns Hopkins University Applied Physics Lab, is one

of three syndromic surveillance systems used in North Texas; the others are the Real-Time Outbreak and Disease Surveillance System (RODS) and BioSense, developed by the CDC. To learn more about these three systems, readers can visit the following Web pages:

• ESSENCE: http://essence.jhuapl.edu/ESSENCE/ Accessed 05/30/2008 • RODS: www.rods.pitt.edu/site/ Accessed 05/30/2008 • BioSense: www.cdc.gov/biosense/index.htm Accessed 05/30/2008

16. Syndromic Surveillance: An Applied Approach to Outbreak Detection. Centers for Disease

Control and Prevention. www.cdc.gov/EPO/dphsi/syndromic.htm. Accessed 10/12/2006.

17. For insight on the efficacy of syndromic surveillance, see a ground-breaking report published by the APC in the summer of 2006: “Buckeridge, D., Verma, A., Siegrist, D, “Tarrant County Evaluation Study: Final Report,” which showed that the RODS system as established in North Central Texas outperformed disease detection by clinician-only methods in a simulated anthrax outbreak with a disease detection time savings of 35 percent to 47 percent, depending on the number of infected, which ranged from as few as 100 (where the largest time savings was seen) to as many as 5,000 (where still a 35 percent time savings was seen). The report is at www.texasapc.net/Portals/0/Surveillance/Report-Final.pdf . Accessed 05/30/2008. A summary is at www.texasapc.net/News/tabid/62/ArticleType/ArticleView/ArticleID/625/Default.aspx. Accessed 05/30/2008.

18. CDC. Prevention and Control of Influenza: Recommendations of the Advisory Committee on

Immunization Practices (ACIP). MMWR. 2003;52 (No.RR-8):1-34

19. For information about DotNetNuke or to download the software, visit www.dotnetnuke.com. Accessed 05/30/2008. The software is free, but readers are reminded that software runs on hardware and requires maintenance support and technical expertise, none of which is free.

20. The case definition is available from the CDC as part of its “Overview of Influenza

Surveillance in the United States,” www.cdc.gov/flu/weekly/fluactivity.htm. Accessed 05/30/2008.

21. Pilot project program leaders identified four periods of flu activity: pre-season (August,

September and early October), emergence (mid-October to mid-November), rise and peak (mid-November to late February), and decline (March through May). For each period, certain flu prevention resources are especially appropriate and their use should be encouraged. The APC worked with its subject matter expert from UNT HSC SPH to develop a flu campaign planning toolkit that TCPH staff used during the first year of the program. Team leaders are now evaluating what changes, if any, are needed for the toolkit to be more widely useful. When any required updates are completed, the toolkit will likely be offered via the APC Toolbox, accessible on the National Association of County and City Health Officials (NACCHO) Website at http://www.naccho.org/toolbox/program.cfm?id=9. Accessed 05/30/2008.

of 57

http://essence.jhuapl.edu/ESSENCE/

http://www.rods.pitt.edu/site/

http://www.cdc.gov/biosense/index.htm

www.cdc.gov/EPO/dphsi/syndromic.htm

http://www.texasapc.net/Portals/0/Surveillance/Report-Final.pdf

http://www.texasapc.net/News/tabid/62/ArticleType/ArticleView/ArticleID/625/Default.aspx

http://www.dotnetnuke.com/

http://www.cdc.gov/flu/weekly/fluactivity.htm

http://www.naccho.org/toolbox/program.cfm?id=9

22. CDC. Updated Guidelines for Evaluating Public Health Surveillance Systems: Recommendations from the Guidelines Working Group. MMWR. July 27, 2001. 50(RR13); 1-35. www.cdc.gov/mmwr/preview/mmwrhtml/rr5013a1.htm. Accessed 05/15/2008.

23. The online survey instrument TCPH and the APC used is the professional version of

SurveyMonkey, described at: www.surveymonkey.com. Accessed 05/30/2008.

24. Using a small portion of its grant funding from NACCHO, the APC purchased about 300 copies of the DVD, “Why Don’t We Do It in Our Sleeves?” The DVD is a popular educational resource addressing flu prevention techniques in an entertaining manner. The APC gave the DVDS to TCPH staff, who in turn provided them to district lead school nurses at several ISDs as part of its effort to recruit school nurse participation in the School Health Surveillance System. You can view the DVD on YouTube at: http://youtube.com/watch?v=Wju7F5ytk6M. You can order the DVD at the product developer’s Website: www.coughsafe.com/media.html. Accessed 05/30/2008.

25. The Extensible Markup Language (XML) is a general-purpose specification for creating

custom markup languages. It is classified as an extensible language because it lets users define their own elements. Its primary purpose is to facilitate the sharing of structured data across different information systems. Learn more at: http://en.wikipedia.org/wiki/XML. Accessed 05/30/2008. Mirth is open source software (no licensing fee) that’s described as a "Swiss army knife" integration engine. It provides the necessary tools for developing, deploying, and monitoring interfaces. It can perform filtering on incoming messages using the default message filters it provides or custom filtering scripts. Mirth’s Web interface makes it easy to write new scripts by proving a template and editing field that lets users easily validate their script code. Learn more at: www.mirthproject.org. Accessed 05/30/2008.

26. TCPH hosted Carnaval de Salud as part of the seventh annual Binational Health Week, a

collaboration of the Mexican Ministry of Health, the Mexican Consulate of Dallas, and other local agencies that focuses on health care and wellness among Hispanics. The goal is to promote sustainable partnerships that address health problems in the U.S., Mexico and Canada. See TCPH press release: www.tarrantcounty.com/ehealth/cwp/view.asp?A=762&Q=464488. Accessed 05/30/2008.

27. The more than 3,000 physicians, residents and medical students of the TCMS “work together to

provide a supportive practice environment for all physicians and foster quality health care for the people of Tarrant County,” per their Website: www.tcms.org/Default.aspx?tabid=1. Accessed 05/30/2008. View the APC and TCPH articles:

• Tarrant County Physician article: www.texasapc.net/Portals/0/Surveillance/Schools/TCMS_article_fall_2007.pdf. Accessed 05/30/2008

• EYE on EPI article: www.tarrantcounty.com/ehealth/lib/ehealth/Eye_on_Epi_Spr_08.pdf. Accessed 05/30/2008

of 57

http://www.cdc.gov/mmwr/preview/mmwrhtml/rr5013a1.htm

http://www.surveymonkey.com/

http://www.coughsafe.com/media.html

http://en.wikipedia.org/wiki/Markup_language

http://en.wikipedia.org/wiki/Extensible_language

http://en.wikipedia.org/wiki/XML

www.mirthproject.org

http://www.tarrantcounty.com/ehealth/cwp/view.asp?A=762&Q=464488

http://www.tcms.org/Default.aspx?tabid=1

http://www.texasapc.net/Portals/0/Surveillance/Schools/TCMS_article_fall_2007.pdf

http://www.tarrantcounty.com/ehealth/lib/ehealth/Eye_on_Epi_Spr_08.pdf

28. The Texas Department of State Health Services hosted its Spring 2008 Statewide ERT meeting

at its Region 2/3 facility (Arlington, TX). The APC presentation addressed school surveillance in the context of building a public health community of practice that leverages the benefits of biosurveillance. Slides: www.texasapc.net/Portals/0/Surveillance/Schools/ERT.pdf Accessed 05/30/2008.

29. The D/FW chapter of APIC dates to 1976 and is one of the first five APIC chapters established

in the U.S. The APC’s presentation at the group’s March 2008 meeting is similar but not identical to its DSHS ERT presentation (note 28). View the materials:

• Presentation: www.texasapc.net/Portals/0/Surveillance/Schools/APIC.pdf Accessed 05/30/2008

• Meeting minutes: www.apicdfw.org/minutes.08.03.html Accessed 05/30/2008.

30. As noted on its Website (www.esc11.net/escrxi/site/default.asp Accessed 05/30/2008), ESC Region XI is one of 20 education service centers established by the Texas State Legislature in 1967. As an intermediary educational agency, ESC Region XI provides information, services, staff development, and support to 77 public school districts and 22 charter schools in a 10-county area in North Texas. The Region XI area includes all of Tarrant County’s school and, in total, includes 53,000 educators and nearly a half million students. The APC presentation at the ESC Region XI Spring Workshop provided a brief review of the pilot project’s progress. Slides: www.texasapc.net/Portals/0/Surveillance/Schools/ESC.pdf Accessed 05/30/2008

31. The 58th meeting of the James Steele Conference on Diseases in Nature Transmissible to Man

was held from April 22-24, 2008, in Galveston, TX. Veterinarians, epidemiologists, public health and medical professionals are frequent participants in the conference, which is intended to be a forum for sharing research findings and epidemiological data on current and emerging zoonotic diseases. More information is at http://diseasesinnature.googlepages.com/. Accessed on 05/30/2008. The School Health Surveillance System presentation at the conference provided a brief review of the pilot project’s progress and background on school surveillance generally. Slides: www.texasapc.net/Portals/0/Surveillance/Schools/DIN.pdf Accessed 05/30/2008

32. The APC and TCPH were honored to accept the invitation of the Center for Biosecurity and Public Health Preparedness at the University of Texas School of Public Health (in Houston) to share information on the pilot project and its potential to facilitate preparedness for pandemic influenza. The Center’s Summer Institute session agenda provides a brief description of the APC and TCPH presentation: www.sph.uth.tmc.edu/biosecurity/FrontLine.cfm. Accessed 05/30/2008. The APC and TCPH presentations for this training will be available on the APC Website (www.texasapc.net Accessed 05/30/2008) following the conference, which is scheduled for June 18 and 19, 2008.

of 57

http://www.texasapc.net/Portals/0/Surveillance/Schools/ERT.pdf

http://www.texasapc.net/Portals/0/Surveillance/Schools/APIC.pdf

http://www.apicdfw.org/minutes.08.03.html

http://(www.esc11.net/escrxi/site/default.asp

http://www.texasapc.net/Portals/0/Surveillance/Schools/ESC.pdf

http://diseasesinnature.googlepages.com/

http://www.texasapc.net/Portals/0/Surveillance/Schools/DIN.pdf

http://www.sph.uth.tmc.edu/biosecurity/FrontLine.cfm

http://www.texasapc.net/

of 57

VII. APPENDIX APPENDIX A: Biography of Tabatha Powell, MPH Tabatha Powell is currently pursuing a Doctorate of Public Health degree in Epidemiology at the University of North Texas Health Science Center, School of Public Health in Fort Worth, Texas. Her research interests include infectious disease epidemiology and syndromic surveillance systems. Ms. Powell is currently collaborating with the Southwest Center for Advanced Public Health Practice to implement a regional school syndromic surveillance system for the early detection of influenza-like illness in school-aged children. She has also worked as an epidemiologist for the Texas Department of State Health Services, Health Service Region 2/3 in Arlington where her primary responsibilities were conducting outbreak and disease case investigations in addition to leading regional influenza surveillance activities. Ms. Powell has also had the opportunity to work for the Centers for Disease Control and Prevention (CDC) as a member of the 2002 Public Health Prevention Service. During her three years with CDC, Ms. Powell participated as an assessment team member in the CDC’s Eight-City Enhanced Terrorism Surveillance Project. She also contributed to the development of A National Action Plan for Cancer Survivorship: Advancing Public Health Strategies and facilitated in the formation of a new national partnership with the Lance Armstrong Foundation to foster the implementation of cancer awareness and education activities. During her final year in the program, she was assigned to the Texas Department of State Health Services (DSHS), Health Service Region 4/5N in Tyler, Texas where she worked in the Zoonosis Control Program and Epi Response Team (ERT). Ms. Powell earned her Bachelor of Science from the University of Maryland at College Park and a Master of Public Health in Epidemiology at the George Washington University in Washington, D.C.

mailto:[email protected]

APPENDIX B: School Health Surveillance System Pilot Project - Logic Model

of 57

APPENDIX C: Evaluation Matrix with Observed Outcomes

GOAL I: Operational School Health Surveillance System

To develop an operational syndromic surveillance system and communication portal for schools for TCPH to monitor and track

the presence of ILI in the school community in Tarrant County. Evaluation Question

Indicator/Measure Source Intended Outcome Measured Outcome

Objective 1.1: By May 31, 2007, the APC will initiate contact with ESC Region XI and school nurses regarding the development and implementation of the School Health Surveillance system. Did the APC initiate contact with ESC Region XI and school nurses regarding the development and implementation of the SHSS by May 31, 2007?

Date of communication Email, phone logs ESC Region XI and school nurses will be informed of the pilot project.

ESC Region XI and school nurses were informed of the pilot project at the 2007 ESC Region XI Workshop on May 9, 2007.

Objective 1.2: By September 31, 2007, the APC will form a School Nurse Committee with members from Tarrant County schools. Did the APC form a School Nurse Advisory Committee with members from Tarrant County schools by September 31, 2007?

The formation of the SNAC is documented.

Emails, interest sheet signed at 2007 ESC Region XI Spring Workshop; excel spreadsheet with names of members

The SNAC was formed. School nurses were asked to participate in the SNAC during the 2007 ESC Region XI Spring Workshop.

Objective 1.3: By October 1, 2008, the APC will meet with the School Nurse Advisory Committee regarding the status of the project and request for feedback on system components. Did the APC meet with the SNAC regarding the status of the project and in order to obtain feedback on the proposed system components by October 1, 2007?

Date of first meeting of the SNAC.

Emails, sign-in sheet, agenda, meeting minutes

First meeting with SNAC is held prior to October 1, 2007.

First meeting with SNAC was held on September 13, 2007.

Objective 1.4: By December 31, 2007, APC staff will develop a Web portal that can be accessed and used by both public and private schools throughout Tarrant County to report campus-level ILI and student absenteeism data. Was the Web portal Date of Web portal Screen shots of Web All public and private school Partially met; system was

of 57

developed by December 31, 2007 for use by both private and public school nurses?

implementation. portal, emails communication (sent receipts)

nurses in Tarrant County will receive an email with account registration and login instructions to access the Web portal.

operational in Nov. 2007; not all private schools were contacted via email. All public schools were contacted via lead district nurse for the 16 ISDs in Tarrant County.

Objective 1.5: By May 31, 2008, TCPH and the APC will hold discussions with the 16 independent school districts (ISDs) in Tarrant County regarding the implementation of the system in the district and the training of school nurses. Were meetings held with district lead nurses in Tarrant County to discuss the implementation and training of the system by May 31, 2008?

Date of communication with each of the 16 ISD lead nurses

Copy of email communication or phone log

All 16 ISD lead nurses were contacted regarding implementation and training.

Yes; multiple emails were sent to all 16 ISDs (~4) in addition to phone calls. See email logs and documentation of TCPH flu surveillance coordinator.

Objective 1.6: By May 31, 2008, 80% of schools within a school district will be using the Web portal to report campus-level ILI and student absenteeism data in order to transition or transfer reporting methods from district level to campus level. What percentage of schools within each of the 16 ISDs is using the Web portal to report campus-level ILI and student absenteeism data by May 31, 2008?

[Number of schools in ‘X’ ISD using the system] divided by [Number of school in ‘X’ district]

Organizational system record, tracking tools

80% of schools within ‘X’ ISD are using the system

50% of ISDs, 3 ISDs are reporting sporadically as provided by TCPH flu surveillance administrative assistant.

Objective 1.7: By December 31, 2007, the TCPH influenza surveillance coordinator and other key TCPH staff who will be responsible for maintaining and updating the information on the Web portal will be trained on the components of the system. Were key staff at TCPH that are responsible for maintaining and updating the system trained on the Web portal components by December 31, 2007?

[Number of required staff who completed training] divided by [Number of staff required to receive training]

Meeting minutes, sign-in sheet

100% of TCPH key staff trained on system.

100% of TCPH staff were trained; included flu surv. coordinator, chief epi, epi, flu surv. admin assist

Objective 1.8: By May 31, 2008, the APC and TCPH will conduct an evaluation of the School Health Surveillance System to assess the success of the pilot year, the utility of the system, and replicability to other local health departments throughout DSHS HSR 2/3 and the state. Was the evaluation of the SHSS pilot project completed by May 31, 2008?

Evaluation report of the School Health Surveillance System.

Report: electronic and hard copy.

Evaluation report will be completed by May 31, 2008.

Evaluation report was submitted for internal review on May 19, 2008.

Objective 1.9: By May 31, 2008, 50% of school nurses will receive training on the TCPH School Health Surveillance System. What percentage of nurses has received training on the

[Number of school nurses trained] divided

Email communications, sign-in sheets, training

50% of school nurses received training on the system by May 31,

Partially met; School nurses from five districts received training on

of 57

system by May 31, 2008? by [Number of school

nurses in Tarrant County] agenda 2008. how to use the system and its

features. Objective 1.10: By May 31, 2008, 80% of schools in 12 of 16 ISDs will be accessing the Web portal at least one time per week. Of the total number of school nurses in Tarrant County, what percentage has accessed the Web portal at least once per week?

[Number of nurses who access portal at least once per week] divided by [Total number of school nurses]

Organizational system record, tracking tools

By May 31, 2008, 80% of the school nurses in 12 of 16 ISDs access the Web portal at least one time per week (which could be the time that they report)

Participation rates vary. This is not easily determined or assessed.

After the initial notification of the reporting system was disseminated, how long did it take for school nurses to begin reporting on a daily basis consistently?

[Date when 80% of school nurses within each district have reported daily data at least once a week for at least a month] – [Date of training meeting with nurses in district]

Sign-in sheet; training agenda; email communications

80% of school nurses in at least 12 ISDs are reporting at least once a week for at least a month , 2 months following the individual training with the district nurses

Within the first day, nurses began to access the system, although it was not until the nurses received training on the system that reporting data began.

Objective 1.11: By May 31, 2008, the APC will facilitate communications between TCPH and the attendance clerks in order to initiate the development of a system to address the reporting of attendance data to the health department Did the APC facilitate communications between TCPH and attendance clerks in order to initiate the development of a system to address the reporting of attendance data by May 31, 2008?

Communications between TCPH and attendance clerks facilitated by the APC; development of strategy to address attendance data reporting issues.

Emails, phone logs, documents displaying strategy.

The APC and TCPH are working with the attendance clerks to develop a strategy for addressing the attendance data reporting issues.

Internally, the APC has developed a decision tree to assist attendance clerks in classifying cases of ILI; The APC has developed an information sheet for parents to encourage reporting symptomology.

Objective 1.12: Between June 1, 2007 and May 31, 2008, the APC will participate in meetings with SNAC, ESC Region XI school nurses, TCPH staff, the Tarrant County Medical Association, and DSHS HSR 2/3 to facilitate communications regarding the pilot project. Did the APC participate in meetings with the SNAC, ESC Region XI, etc. to facilitate communications regarding the pilot project between June 1, 2007, and May 31, 2008?

Meetings were held. Emails, sign-in sheets, agendas, meeting minutes, additional communications

Meetings were held with the aforementioned stakeholders between June 1, 2007, and May 31, 2008.

Meetings were held with the SNAC on; the APC presented project status updates to ESC Region XI nurses on April 19, 2008; the APC presented at the statewide and regional meeting of epidemiologists to provide updates on the status of the project.

Objective 1.13: By May 31, 2008, the APC and TCPH will develop a survey for school nurses in Tarrant County to collect data regarding the project evaluation. Results will be disseminated and shared with stakeholders by June 31, 2008.

of 57

Did the APC and TCPH develop a survey for school nurses to assess the acceptability and usability of the system by May 31, 2008?

Survey developed and administered to school nurses.

Survey instrument; notification emails to school nurses

Survey was developed and administered prior to May 31, 2008 and results were shared in with stakeholders prior to June 31, 2008.

The survey was launched on April 14, 2008 and terminated on May 16, 2008. Results from the surveys will be shared in the evaluation report.

GOAL II:

Information Exchange

To increase information exchange between Tarrant County school nurses and Tarrant County Public Health Evaluation Question


Objective 2.2: By July 31, 2007, the APC will develop a flu prevention campaign with resources that will be shared with school nurses via the School Health Surveillance system. Was a flu prevention campaign for use in the system developed by July 31, 2007?

Flu prevention campaign document.

Emails, copy of flu prevention campaign document with list of resources

Flu prevention campaign was developed by July 31, 2007.

Flu prevention campaign was developed by July 1, 2007.

Objective 2.3: By May 31, 2008, 60% of school nurses in Tarrant County will visit the flu prevention campaign Web page. Did 60% of school nurses who were recruited to use the system visit the flu prevention campaign Web page?

Visits to flu prevention campaign Web page.

System log files 60% of school nurses in Tarrant County will visit the flu prevention campaign Web page.

Partially met; a limited number of school nurses have begun to look at the material provided on the flu prevention campaign Web page.

Objective 2.4: By May 31, 2008, 60% of school nurses in Tarrant County will download at least one file from the flu prevention resources Web page to increase their knowledge. Did the school nurses access the information provided on the flu prevention resources to increase their knowledge or to share information with teachers, staff, students, or parents?

[Number of school nurses who have visited the flu prevention campaign and/or downloaded information from the page] divided by [Total number of school nurses registered on the system]

System tracking tools, feedback from nurses, survey results

By May 31, 2008, 60% of school nurses have visited the flu prevention resources Web page and/or downloaded at least one file from the Website.

Unable to assess in completeness using system logs and tracking tools. It is inferred that this objective has not been met based on the findings in the system logs (limited access of prevention and other resources).

Objective 2.1: By December 31, 2007, 250 private and public schools will receive the “Why Don’t We Do It in Our Sleeves?” DVD. Did both private and public schools receive and use the

[Number of schools that requested and used

Survey results, email communications

Approximately 250 private and public schools will receive and use

Public schools only (through ISD recruitment and training); see DVD

of 57

“Why Don’t We Do It in Our Sleeves” DVD?

DVD] divided by [Number of schools that requested DVD]

the DVD. log

GOAL III:

Change in Behavior / Increase Knowledge

To increase public awareness of the public health importance of implementing preventive measures for the prevention and control of influenza. Evaluation Question


Objective 3.1: TCPH will increase the proportion of vaccinations (# used/ordered) administered to those in the community by 5% from the previous year. Was there an increase in the proportion of flu vaccine used following the release of the flu prevention resources and implementation of the surveillance system?

[Number of remaining flu vaccine from 2008] divided by [Total number of vaccine ordered] subtracted from the [Number of remaining flu vaccine from 2007] divided by [Total number of vaccine ordered]

Internal records and order logs

TCPH will increase the proportion of vaccinations (# used/ordered) administered to those in the community by 5% from the previous year

Not met; this is most appropriately addressed in year 2 of the project since there was a low participation rate by schools and incomplete data.

Objective 3.2: By May 31, 2008, 60% of school nurses in Tarrant County have downloaded at least one file from the flu prevention resources Web page for use in the school. Has more than half of the nurses who are using the system downloaded at least one file from the flu prevention resources page?

[Number of nurses who have downloaded one file from flu prev resource page] divided by [average weekly number of nurses who have used the system]

System logs 60% of school nurses using the system have downloaded at least one file from the flu prevention resources page.

Unknown; system logs do not record this data; only number of views to a specific page.

Objective 3.3: Increased testing or more targeted or focused testing based on information gained from system. Was testing for flu and flu-like illness based on the information gained through the surveillance system?

Correlate testing with increases in student absenteeism and syndromic data

System logs, testing records, email communications

TCPH will focus testing on those areas of the county with increased rates of reported ILI through the pilot surveillance system

Partially; school absenteeism data supplemented other surveillance data (hospital ED data, SPSN, and laboratory findings)

Objective 3.4: By May 31, 2008, there will be a 5% decrease in the proportion of remaining flu vaccine during decline sub-season from the

of 57

previous year’s supply. Was there a 5% decrease in the proportion of remaining flu vaccine during the decline sub-season from the previous year’s supply?

[% of remaining flu vaccine from 2007] minus [% remaining flu vaccine from 2008]

Immunization program records

5% decrease in the proportion of remaining flu vaccine during the decline sub-season from previous year’s supply.

Not measured during year 1; more appropriately assessed in year 2 during impact evaluation.

Objective 3.5: By May 31, 2008, at least 50% of school nurses indicate that participating in the pilot project through reporting and accessing health resources, has increased their awareness of the health status of the students in their schools. Has participation in the pilot project increased school nurses’ awareness of the health status of students in their schools?

[Number of nurses state that their awareness has increased] divided by [Number of school nurses using system]

Survey results 50% of school nurses indicate that participating in the pilot project has increased their awareness of the health status of students in their schools.


Objective 3.6: By May 31, 2008, at least 50% of school nurses indicate that participating in the pilot project through reporting and accessing health resources, has affected their response to an increase in ILI through the use of information available through the Web portal. Has participation in the pilot project changed their response to an increase in ILI through the use of information available on the Web portal?

[Number of nurses that state that having access to ILI data affected their response] divided by [Number of nurses using system]

Survey results 50% of school nurses indicate that participating in the pilot project has affected their response to an increase in ILI through the use of information available through the Web portal.


Objective 3.7: By May 31, 2008, at least 50% of school nurses indicate that they have changed their behaviors or have heard that there has been a change in behavior of the school population (students, teachers, parents) with regards to the use of prevention and health promotion methods. Has participation in the project changed behaviors of school nurses or of the school population?

[Number of nurses that state that there has been some change in behaviors] divided by [total number of nurses using the system]

Survey results 50% of school nurses indicate that they have changed their behaviors or have heard that there has been a change in behavior of the school population.


Objective 3.8: There is a 1% decrease in the peak of total student absences during the 2007 – 2008 influenza season as compared to the 2006 – 2007 influenza season. Was there a decrease in student absenteeism rates during the peaks of the 2007-2008 flu season as compared to the 2006-2007 flu season?

[Student absenteeism rates from 2007-2008 season] subtracted from [Student absenteeism rates from 2007-2008]

TCPH data records, school nurse reports

There is a 1% decrease in the peak of total student absences during the 2007 – 2008 influenza season as compared to the 2006 – 2007 influenza season.

This decrease cannot be attributed to the system’s implementation but will provide baseline data for further analysis

Objective 3.9: Attendance clerks: 50% of ISDs we have addressed the issues of attendance.

of 57

of 57

Were attendance clerks involved in the process for reporting student absenteeism and ILI syndromic data through the Web portal?

List of clerks who attended the training meetings and discussions within each of the 16 ISDs

Sign-in sheet, agenda, meeting minutes, email communications

50% of ISDs have addressed the issues of attendance by including the attendance clerk in the reporting process.

Partially; Example: one district informed attendance clerks to share attendance data with school nurses

APPENDIX D: School Nurse Communications Pilot Project Fact Sheet, 6/23/2008 Program parameter: Current method: New method: Reporting frequency Weekly Daily Reporting entity School district Individual schools Reporting method to TCPH Fax, with some via e-mail Web-based upload of report form Data analysis by TCPH Manual (via Excel & SPSS) Automatic and extended in ESSENCE Reporting to schools Manual process in which

influenza activity reports are sent weekly via e-mail; additional e-mail or phone calls are placed as needed

Automatic and extensive via access to ESSENCE, with online postings of influenza activity reports weekly and with additional reports provided online or via e-mail or phone when warranted

Tools provided to schools Limited to fact sheets and reports issued as needed

Adds access to school nurse view of the ESSENCE system and new, valuable resources in a comprehensive campaign

Benefits of the new method:

Reporting frequency: More timely information sharing will help detect the presence of flu faster and allow an efficient, coordinated, comprehensive response. This benefit is further enhanced by focusing the response to the zip codes (neighborhoods) where flu is found to be emerging.* Reporting entity: Gathering data from individual schools makes it easier to isolate the presence of flu at the neighborhood level and helps TCPH provide timely information and instructions to schools for mitigating flu when it emerges within individual neighborhoods and spreads throughout the county. This level of granularity is not feasible if data is reported in aggregate form by large districts. Reporting method to TCPH: By providing report forms that can accessed and uploaded in a Web portal, it becomes faster and easier for schools to report their information and for TCPH to gather and analyze it. Prior problems with illegible handwriting and missing data fields are also avoided. Data analysis by TCPH: By pushing data collected from the Web portal into the ESSENCE syndromic surveillance system, TCPH can avoid all of the manual labor associated with re-keying data from faxes and analyzing it with limited tools such as Microsoft Excel. Report forms and graphs are generated with a few clicks in the software rather than as separate manual processes. Reporting to schools: School nurses will be given direct access to the ESSENCE system so they can see their own data and monitor community health patterns across the county. (Previously, they lacked such access and only gained insight to community health patterns through phone calls or e-mail messages sent periodically by TCPH.) A two-way exchange of information will be more evident because TCPH reports will now be available to all nurses in the same portal where nurses will be asked to upload their data. Tools provided to schools: In addition to the reports published in the Web portal, TCPH will provide additional resources to school nurses such as the Why Don’t We Do It in Our Sleeves DVD (see video) and other materials that can be used to educate students and parents. *Evidence of the emergence of flu will be found in ESSENCE analysis of chief complaint data from 14 Tarrant County hospitals and from school nurse data, results of samples tested at the North Texas Regional Laboratory within TCPH, and reports obtained through the TCPH sentinel flu surveillance program, which collects data from area physicians, pediatricians, universities, businesses, etc.

of 57

http://www.texasapc.net/Surveillance/ESSENCE/tabid/76/Default.aspx

http://www.texasapc.net/Surveillance/ESSENCE/tabid/76/Default.aspx

http://video.google.com/videoplay?docid=-8574515984097771637&q=ben+lounsbury

APPENDIX E: Tarrant County School Health Surveillance System School Nurse Survey

Frequent/Sporadic Use of System

The Tarrant County School Surveillance System is a tool that school nurses use to report student absenteeism data for their individual school AND to obtain pertinent health promotion and disease prevention materials. The information is updated periodically, and alerts regarding outbreaks affecting North Texas, the state, and the nation are posted on this Website. In addition, an influenza report for Tarrant County is posted every week with current data. The following questions ask about your knowledge and use of the Tarrant County School Health Surveillance System. SYSTEM USE 1. How often do you log onto the system to report your school’s student absenteeism data during the

week?

a. Rarely b. 1-2 days a week c. 3-4 days a week

d. Every day

2. During the week, how often do you log onto the system to look at the other information available

through the Web portal (e.g. prevention materials, weekly flu report, zip-code level maps of ILI, news items, action items, etc.)?

a. Never (Q3) b. Rarely (Q3)

c. Once every two weeks (Q3)

d. Once or twice a week (Q4)

e. Three or more times a week (Q4)

of 57

3. Please indicate the reasons for not accessing or accessing infrequently the health promotion and

disease prevention information available on the Website (Please check all that apply).

a. Unaware that the information was available. b. Did not know how to access the information from the reporting Web page. c. Information is not easy to find. d. Information was not of interest to me. e. Only have a limited amount of time during the day to look at this type of information (time

constraints) f. My computer takes too long to download files or move from Web page to Web page. g. Other:________________________

Reporting your school health data (student absences and ILI data) can be a two-step process. The first process involves collecting the data on the number of students absent (either yourself or from an attendance clerk) at your school. 4. Please indicate below how you obtain student attendance data. Please check all that apply.

a. I contact the attendance clerk.

b. I access the data using an internal district-wide system.

c. I receive calls from parents regarding their childs’ absences. d. Other:_____________

5. How long does it take you to collect the data in order to be able to enter it into the system?

a. Less than 5 minutes b. Between 5 and 15 minutes c. Between 15 and 30 minutes

d. More than 30 minutes

e. Other:________________________-

The second step involves logging onto the Tarrant County School Health Surveillance System and entering the data into the report form. 6. How long does it take you to enter the data into the system?

a. Less than 2 minutes

of 57

b. Between 2 and 5 minutes

c. Between 5 and 10 minutes

d. More than 10 minutes

e. Other:________________________-

SYSTEM FEATURES Please indicate to what extent that you agree with the following statements.

Strongly Disagree

Disagree Agree Strongly Agree

7. I have no problems logging into the system. 8. I like using an online system for reporting my data. 9. I like having access to public health information from Tarrant

County Public Health using the Web portal.

10. I can obtain data on student absences from the attendance clerk or someone else at my school.

11. I keep a computerized log of reasons for student visits to my office.

12. I keep a paper-based log of reasons for student visits to my office.

13. I use a district system for tracking student visits to my office and other student health records.

How interested would you be in continuing to have access to the following primary resources through the Web portal?

Not Interested

Somewhat Interested

Very Interested

14. Tarrant County Weekly Influenza Surveillance Report. 15. Ability to view maps of flu-like illness as reported by hospitals

and schools in Tarrant county, DFW, and north central Texas region by zip code.

16. Flu prevention resources available for school nurses. 17. A place where school nurses can access suggested action items

from their health department on a homepage customized for them.

18. The ability to provide comments and suggestions on health-related issues.

of 57

How interested would you be in continuing to have access to the following supplemental resources through the Web portal?

Not Interested

Somewhat Interested

Very Interested

19. Current public health information that is in the news. 20. A Website that contains action-oriented resources such as form

letters to educate parents (such as MRSA, flu, etc.)

21. “Nurse to Nurse” column with current public health information serving nurses in the county (written by Tarrant County’s nurse epidemiologist)

22. “Public Health Notes” column from the Tarrant County Physician (written by Tarrant County’s Health Authority).

23. Current recommendations for pandemic flu planning in schools. 24. Links to Websites such as the Tarrant County Public Health

(TCPH) and Centers for Disease Control and Prevention (CDC).

PREVENTION INFORMATION 25. Did you receive the DVD, “Why Don’t You Do It in Your Sleeves?”?

a. Yes b. No (Q28)

26. How have you used the DVD since you received the copy? (Check all that apply)

a. Shared it with my school’s administration. (Q28) b. Shared it with teachers. (If so, specify teacher type such as “health teacher”) (Q28)

c. Shown it to students. (Q28)

d. I have not used it yet, but plan to use it before the end of the school year. (Q27)

e. I have not used it yet, but plan to use it next school year. (Q27)

27. If you plan to use the DVD by the end of this school year or during next school year, please

indicate how you plan to use the DVD. (Check all that apply).

a. Will share it with my school’s administration.

of 57

b. Will share it with teachers. (If so, specify teacher type such as “health teacher”)

c. Will show it to students.

d. Other:__________________________________

28. What comments or suggestions do you have about how we could make the School Health

Surveillance System more useful to you? 29. Besides enhancing the School Health Surveillance System, what else could Tarrant County Public

Health do to partner more effectively with you in the future? DEMOGRAPHICS We ask that you share your information below so that we can better serve you by addressing your needs and concerns. Your information is confidential and will only be shared with persons conducting the survey. Summary information regarding responses will be used in the evaluation of the system and will not be identifiable to individual schools or school nurses. 30. Please select your Independent School District

a. Arlington j. Grapevine-Colleyville b. Azle k. Hurst-Euless-Bedford c. Birdville l. Keller d. Carroll m. Kennedale e. Castleberry n. Lake Worth f. Crowley o. Mansfield g. Eagle Mt-Saginaw p. White Settlement h. Everman q. Private school i. Fort Worth r. Other:__________

31. Please indicate the school type. Please check all that apply.

a. Primary b. Elementary c. Intermediate d. Middle / Junior High e. High / Senior High

of 57

APPENDIX F: Tarrant County School Health Surveillance System School Nurse Survey

For Non-Users of the System

SCHOOL HEALTH SURVEILLANCE SYSTEM The Tarrant County School Surveillance System is a tool that school nurses use to report student absenteeism data for their individual school AND to obtain pertinent health promotion and disease prevention materials. The information is updated periodically and alerts regarding outbreaks affecting North Texas, the state, and the nation are posted on this Website. In addition, the weekly influenza report for Tarrant County is posted every week with current data. Please indicate below to what extent you agree with the following statements:

Strongly Disagree

Disagree Agree Strongly Agree

1. I am interested in participating in this system. 2. I am aware that this system is already in use in Tarrant County. Q5 Q5 3. I would like to learn more about the system and its capabilities. 4. I would like to attend a training to learn how to use this system.

If the respondent answered to A or SA to Q2, then refer to the following question 5. How did you become aware of the Tarrant County School Health Surveillance System?

a. Someone from the health department spoke to me about it. b. My lead nurse emailed or spoke with me about it.

c. I heard about it from other nurses in the district.

d. I heard about it from nurses outside of my district.

e. Other:__________________________________

How interested would you be in having access to the following primary resources through the Web portal?

Not Interested

Somewhat Interested

Very Interested

6. Tarrant County Weekly Influenza Surveillance Report. 7. Ability to view maps of flu-like illness as reported by hospitals

and schools in Tarrant county, DFW, and north central Texas region by zip code.

of 57

8. Flu prevention resources available for school nurses. 9. A place where school nurses can access suggested action items

from their health department on a homepage customized for them.

10. The ability to provide comments and suggestions on health-related issues.

How interested would you be in having access to the following supplemental resources through the Web portal?

Not Interested

Somewhat Interested

Very Interested

11. Current public health information that is in the news. 12. A Website that contains action-oriented resources such as form

letters to educate parents (such as MRSA, flu, etc.)

13. “Nurse to Nurse” column with current public health information serving nurses in the county (written by Tarrant County’s nurse epidemiologist)

14. “Public Health Notes” column from the Tarrant County Physician (written by Tarrant County’s Health Authority).

15. Current recommendations for pandemic flu planning in schools. 16. Links to Websites such as the Tarrant County Public Health

(TCPH) and Centers for Disease Control and Prevention (CDC).

DEMOGRAPHICS We ask that you share your information below so that we can better serve you by addressing your needs and concerns. Your information is confidential and will only be shared with persons conducting the survey. Summary information regarding responses will be used in the evaluation of the system and will not be identifiable to individual schools or school nurses. 32. Please select your Independent School District

a. Arlington j. Grapevine-Colleyville b. Azle k. Hurst-Euless-Bedford c. Birdville l. Keller d. Carroll m. Kennedale e. Castleberry n. Lake Worth f. Crowley o. Mansfield g. Eagle Mt-Saginaw p. White Settlement h. Everman q. Private school i. Fort Worth r. Other:__________

33. Please indicate the school type. Please check all that apply.

a. Primary b. Elementary c. Intermediate d. Middle / Junior High e. High / Senior High

of 57

APPENDIX G: School Health Surveillance System Key Informant Interviews

(15-30 minutes)

Explain to interviewees that participation in the interview is strictly voluntary and that they can choose not to participate at any time during the interview. “Your information is confidential and will only be shared with persons conducting the survey. Summary information regarding responses will be used in the evaluation of the system and will not be identifiable to individuals or their respective organizations or institutions. Participation in this survey or your input will not affect your employment at your respective organization or institution.” Provide interviewees with the following information regarding the system and the goals of the pilot project as a lead into the interview questions. Brief Description of the System The Tarrant County School Surveillance System is a tool that school nurses use to report student absenteeism data for their individual school AND to obtain pertinent health promotion and disease prevention materials. The information is updated periodically and alerts regarding outbreaks affecting North Texas, the state, and the nation are posted on this Website. In addition, the weekly influenza report for Tarrant County is posted every week with current data. Overarching Goals of Pilot Project Through the development of an operational, electronic, Web-based syndromic surveillance system with communication portal and a renewed partnership with school nurses, the Southwest Center for Advanced Public Health Practice (APC) and Tarrant County Public Health (TCPH) will (1) Support early detection of influenza-like illness (ILI) through enhancement of school-based

ILI and student absenteeism reporting. (2) Provide public health resources to promote disease prevention and health promotion

activities for the school population (nurses, staff, students, parents) in Tarrant County. In order to (1) Slow the spread of flu throughout the community in Tarrant County. (2) Reduce the magnitude of flu outbreaks in Tarrant County.

of 57

Questions 1. Based on the system’s potential for gathering data for early detection of ILI in school and daycare

based populations and the potential for using this information to direct and focus public health resources more appropriately, please provide us with your thoughts regarding the:

a. Benefits of the system.

b. hallenges of implementing a system such as this (If no comment, prompt with the question below).

i. Our research included a discussion with Montgomery County…standard of theirs as compared to ours…..Since recruitment of schools and daycares (potentially) has and may be a challenge, what approaches do you think would or have been effective for engaging this community currently or in the past?

c. Possible next steps for the system

2. Is there or do you feel that there will be support for this type of system in the county, region, state

for its expansion into a statewide system for monitoring influenza-like illness in schools?

a. What concerns will others have regarding using this type of system in their health department?

b. What changes do you foresee as being necessary for the system to be applied in health departments across the state?

3. Is there anything you would like to add, emphasize, or say in summary about

the school health surveillance system and the pilot project?"

of 57

APPENDIX H: Screen Shot of Share Your School’s Health Data Report Form

of 57

APPENDIX I: Screen Shot of ISD Home Page

of 57

APPENDIX J: Screen Shot of Flu Prevention Resources Overview

of 57

APPENDIX K: Screen Shot of Regional Outbreak Maps for Influenza-Like Illness

of 57

APPENDIX L: Screen Shot of Project Status and Background Overview

of 57

APPENDIX M: Guidance for Parents for Reporting Flu to Their School

If Your Child Sick With Flu, Report It To Your School! What To Say If your son or daughter is sick with flu, it’s important that you report this reason of illness when you speak with your school’s attendance clerk. Don’t tell the attendance clerk only that your child will be absent from school. Instead say, “My child is sick with flu.” When To Say It You need not wait for a doctor’s diagnosis to know your child has flu. You should report that your child is sick with flu if they have a cough or a sore throat and if, when you take their body temperature using a thermometer, the reading measures 100 degrees or greater. Additional symptoms that often indicate flu include malaise, muscle aches, runny nose, sore throat, chills, and headache. If your child has these symptoms, don’t send them to school. Call your school and say, “My child is sick with flu.” Why It Matters If you report flu as the reason for your child’s absence, your school can share that valuable information with Tarrant County Public Health. We’ve asked school nurses to report school absenteeism due to influenza to get a clearer view of how flu is emerging. This can help us take action to slow the spread of flu in our region. We take flu seriously — and you should, too — because it can lead to serious illnesses, school and business closures, and even deaths. In fact, the Centers for Disease Control and Prevention (CDC) reports that, in the U.S., flu causes more than 200,000 hospitalizations annually and about 36,000 deaths. Next Steps Once you report flu to your school attendance clerk, consider making an appointment with your child’s pediatrician to verify the presence of flu and obtain a prescription for helpful medications. Don’t send your child back to school until all of your child’s flu symptoms have subsided or your pediatrician says it’s acceptable for your child to return to school.

of 57

of 57

APPENDIX N: Guidance for Parents for Reporting Flu to Their School, Spanish Version

Si su hijo esta enfermo con la gripe, reportelo a su escuela

Que decir Si su hijo/hija esta enfermo con gripe, es importante que usted reporte esta enfermedad cuando usted hable con el encargado de asistencias de su escuela. No le diga al encargado de asistencia solamente que su hijo/hija esta ausente de la escuela, sino en vez tiene que decir, “Mi hijo/hija está enfermo con gripe.” Cuando decirlo Usted no necesita esperar el diagnóstico del médico para saber si su hijo/hija tiene gripe. Usted puede reportar que su hijo/hija esta enfermo con gripe cuando el/ella tiene tos, irritación de garganta o cuando tienen fiebre superior a los 100 grados. Otros síntomas adicionales que indican que su hijo/hija tienen gripe es malestar de cuerpo, dolores musculares, congestión de nariz, irritación de garganta, escalofríos, y dolores de cabeza. Si su hijo/hija tiene estos síntomas, no lo mande a la escuela. Llame a la escuela y diga que esta enfermo con gripe. Porqué importa Si usted reporta que su hijo/hija se ha ausentado de la escuela porque esta enfermo con gripe, la escuela puede compartir esta valiosa información con el departamento de Salud Pública del condado de Tarrant. Hemos pedido que las enfermeras de las escuelas reporten que la razón porque su hijo/hija se ha ausentado de la escuela es debido a la gripe, y así poner darnos cuenta como la gripe está emergiendo. Esto puede ayudarnos a tomar una decisión par aver cual es la extensión de la gripe. Nosotros nos tomamos la gripe muy en serio y usted también, porque esto puede llevar a enfermedades más serias e incluso la escuela y otras compañías pueden cerrar y también en algunos casos hasta la muerte. De hecho, los centros para el Control de Enfermedad y Prevención (CDC) han reportado que en los Estados Unidos, la gripe ha causado más de 200.000 hopitalizaciones anuales y cerca de 36.000 muertes. Pasos a tomar Una vez que usted ha reportado la gripe al encargado de asistencias de la escuela, usted puede hacer una cita con el médico pediatra de su hijo/hija para verificar la enfermedad de la gripe y así obtener una prescripción adecuada. No mande a us hijo/hija a la escuela hasta que los síntomas de la gripen se hayan aliviado o el medico pediatra le diga que su hijo/hija esta bien para volver a la escuela.

TCSHSS Evaluation Report

Documents

Transcript of TCSHSS Evaluation Report