Proceedings 19th Triennial Congress of the IEA, Melbourne 9-14 August 2015  

1  

Development of an integrated web application to build an overall Patient Safety profile of local healthcare trusts

Michela Tanzini, Riccardo Tartaglia, Francesco Ranzani, Tommaso Bellandi, Laura Arrigoni

Centre for Clinical Risk Management and Patient Safety, Florence, Italy; Estav Nord Ovest, Pisa, Italy

The main objective of the project is to design a regional web application to manage all data on patient safety and quality of care with an integrated and systemic approach. The technological solution consists of four main modules with restricted access for healthcare workers, which gather and manage information related to different aspects of patient safety and quality of care: reporting and learning system, claims and litigation, patient safety practices and institutional accreditation. The web application key factor is a control panel, which focuses on processing, and determining an overall safety profile of a local healthcare trust thanks to all the information managed in the single modules. According to an ergonomic approach, a representative set of end users have been involved in requirements identification, prototype development, usability testing adopting an iterative design perspective.

Keywords: regional patient safety web application, process mapping, user involvement, user testing, healthcare trust safety profile

1. Scope

The main objective of the project is to design a regional web application to manage all data on patient safety and quality of care with an integrated and systemic approach. The technological solution consists of four main modules with restricted access for healthcare workers: 1. reporting and learning system (RLS); 2. claims and litigation (CL); 3. patient safety practices 4. institutional accreditation system.

The web application key factor is a control panel, which focuses on processing, and determining an overall safety profile of a local healthcare trust thanks to all the information managed in the single modules. Two modules with public access are also implemented to promote patient safety initiatives: a module to announce all training activities of the healthcare facilities related to patient safety and quality of care and another one to promote specific patient safety projects achieved, such as patient identification, adoption of surgical checklist, etc. 2. Project organization

The project started in September 2013. A multidisciplinary team was formed to identify users’ requirements and to develop the first technological prototype: a software engineer, two developers, a designer, two process analysts and an ergonomics specialist. The first six month of the project has been focused on users’ needs identification and survey about the tools, which has been used up now to manage all these data, thanks to ethnographic interviews and observation. All relevant team members of the local healthcare trust were involved in requirements’ analysis: administrative staff, maintenance technician, coroner, lawyer, clinical risk manager, patient safety manager, nurses and clinical staff. The gathered information has been used to map the main working process related to the data to be tracked in each module, which has been developed through a series of iterative stages. Each module has been implemented within at least three months. Once the prototype was ready to use, some usability testing were planned in order to define any troubles in terms of ease of use, clarity of information, completeness and thoroughness. The last few months have been devoted to clarify what problems the technology is designed to help tackle and to build consensus (Cresswell, 2013). The final release of the web application is still under construction and will be tested again before deploying expected by the June 2015. The design team commitment will be continuing during the maintenance and upgrade phase, by observing how the technological solution will be actually used in the context of use. New needs and new adaptations may be necessary and appropriate in order to respond more adequately to end users’ needs and requirements in performing their tasks.

Proceedings 19th Triennial Congress of the IEA, Melbourne 9-14 August 2015  

2  

3. Human factors topics covered

The project has addressed various aspects related to the human factor. First, the user’s centrality in the design of a technological solution designed to support its working activities. The end user, in fact, has been involved in all design’s phases and in the evaluation of the first prototypes. An ergonomic approach was used in order to adopt an iterative design method: analysis, implementation, testing and redesign based on the observed feedback. Usability tests have represented an additional tool to involve user and to verify, one hand, the working process analysis and the related designed workflow, on the other, the accuracy of information requirements for each individual task.  4. Project phases

4.1 Mapping of different working process related to management of patient safety practices with a participatory approach

Central to the design of this technological solution is a clear understanding of what users actually want to do: what are their tasks? what is the nature of those tasks? (Crystal, Ellington 2004). Each module stands for an existing organizational healthcare process regarding patient safety: events reporting, claims and litigations, enforcement of patient safety practices. In order to design each module effectively, a discussion and analysis of corporate procedures, work practices and tools used up to now were required. The design team focuses on analyzing in depth all the activities and tasks performed by each healthcare workers involved in the process. The team decided to use task analysis as a method to analyze the working process.

Hierarchical Task Analysis (HTA) is useful for decomposing complex tasks: it provides a framework for the investigation of existing practices to facilitate the design of complex systems, enabling designers to envision the goals, tasks, subtasks, operations, and plans essential to users’  activities. Task analysis makes it possible to design and allocate tasks appropriately within the new system. Using HTA, all processes have been redesigned using an open source software, which translates the flowchart in Business Process Model and Notation (BPMN), a graphical representation for specifying organizational processes (see Figure 1). This tool represented furthermore an additional aid to transmit that information to the developers.

Figure 1. Reporting Learning System Flowchart.

4.2 Requirements’ detection and users’ needs thanks to ethnographic interviews and observation

HTA does not provide unfortunately systematic way for dealing with the rich social and physical context in which activities are embedded. Similarly, HTA fails to support the components needed to understand system flows and dynamics. These limitations necessitate the use of additional method to develop a more complete understanding of human activity. Therefore, in addition to the analysis process, it was decided:

• to observe the healthcare workers performing their tasks; • to analyze the organizational procedures and protocols; • to interview all healthcare workers and stakeholders to detect their information needs and expectations

to be designed in new web application.

Proceedings 19th Triennial Congress of the IEA, Melbourne 9-14 August 2015  

3  

To keep track of all this information, an interview template has been implemented, structured as follows: • job description (role played inside the local healthcare trust, responsibilities, activity types, etc.); • purpose (goals to be achieved related to the organizational role); • problems (main difficulties connected to the tasks to be achieved, way of solving and managing

problems, etc.); • opportunities (expectations, new goals, tip for troubleshooting, etc.) (see Figure 2).

Figure 2. Interview template.

All interviews has been recorded. The process analysts were therefore able to clarify some aspects not

sufficiently covered in the task analysis, thus integrating information concerning the single actions execution to achieve the identified tasks. 4.3 Design and implementation of modules’ prototype

Once a set of goals and issues were assembled, the two process analysts began drawing up plans to specify how the goals should be achieved, including contingencies, decision points, concurrent operations and cycling (Shepherd, 2001). A description task template was implemented in order to define each action to be fulfilled to achieve a specific task, structured as follow:

• task type (compilation or confirm task); • task description (every label to be displayed in order to fill the form); • form type (multiple or single selection, free text); • compilation type (mandatory, already filled); • control (what kind of control has to be done in order to fulfill the goal); • notes (see Figure 3).

Proceedings 19th Triennial Congress of the IEA, Melbourne 9-14 August 2015  

4  

Figure 3. Detailed description task template. 4.4 End users’ involvement in the interface design

A new technological solution needs to fulfill a range of requirements on a variety of levels: it needs to be usable for end users (not cumbersome for clinicians and beneficial for patients), cost-effective for organizations, and interoperable to allow secondary uses of data (Cresswell, 2013). It should be both fit for organizational purpose and fit for clinical practice. End users involvement in the design and in the development phase is strategic. The graphical user interface has been based on the requirements specification to support end users with their task at hand (Nielsen, 1994). Efforts promoting the participation of end users by actively identifying early sketchy and simple prototypes have the highest potential to achieve successful solution.

According to what discovered during healthcare workers interviews, the graphical user interface should be characterized as follow:

• functions customization (i.e. opportunity to manage end user own access page, by choosing which most frequently used functions to display);

• focus on the functions associated to end user role, according to a specific task to be achieved; • display information and contextual functions to aim the task at runtime; • consistency in visualizing main and ancillary functions of the module; • focus on reporting and data processing functions in order to give immediate feedback to end users; •   different information sources integration of the various modules to identify local healthcare trust,

department profile and so on (see Figure 4).

Figure 4. First home page proposal.

Unfortunately, HTA provides no systematic way for dealing with the rich social and physical context in which activities are embedded. Similarly, HTA fails to support the components needed to analyse system flows and dynamics. These limitations necessitate the use of additional theoretical structures to develop a more complete understanding of human activity. 4.5 Usability testing

According to an ergonomic approach, a representative set of end users has been involved in some usability testing once the first prototype has been implemented. The usability testing included a series of tasks to achieve a specific goal (eg the compilation of a never event report). The user interaction while performing the task was recorded. It was asked to the user furthermore to think aloud in order to understand the logical reasoning, reflections and hesitations he had in achieving the given goal. Two ergonomic observed the user and annotated the number of hesitation, the number of errors, requests for help, and the number of clicks with the aid of a counter.  

The time to complete the required tasks and the reliability of the decision-making proved to be satisfactory, both according to the observed functional measures and according to the subjective

Proceedings 19th Triennial Congress of the IEA, Melbourne 9-14 August 2015  

5  

satisfaction’s score. According to these preliminary results, the system responds to the user needs and facilitate both the recording and the monitoring activities on patient safety data.    4.6 Consensus building

The organizational and social factor play a strategic role in the planning and requirements analysis of the project: getting the organization ready to change, engaging general manager and satisfying the realistic expectations have an effective impact on building shared vision across the healthcare facility on how the new web application could support in determining an improved quality of care (see Table 1).

Professional, managerial and administrative consensus needs to be build around the strategic vision. Efforts in promoting the participation of all end users in identifying an inclusive solution have the highest potential to create technological solutions which are easy to use and represent a real added value to daily activities. Table 1. Factors associated with effective implementation identified in the literature adapted from Cresswell, 2013.

Technical Social Organizational Wider socio-political

Usability Attitudes and concerns Getting the organization ready for change planning

Other healthcare organizations

System performance

Resistance Leadership and management

Industry

Integration Workaround Realistic expectations Policy Interoperability Expectations User ownership Professional groups Benefits/values and

motivations Teamwork and communication

Independent bodies

Engagement and user input in design

Learning and evaluation The wider economic environment

Training support International development Integration with existing

workpractices

4.7 Training

At the moment, all the different types of end users are involved in training. Specific functionalities, all the detailed data have been explained while simulating the use of the web application. The training involves a limited number of healthcare workers (two or three) who will have the responsibility to train their colleagues. From an organizational point of view, it is impossible to be able to train all the users in charge of managing data in the web application (currently about 3000). Additional training tools will be available to end users to ensure training also autonomously: faq, video tutorials, module’s workflow, etc.

Despite the numbers, the training provides, however, the opportunity to involve a greater number of users, even those who have not been involved in the design phase: a debate with the latter, in fact, offers further ideas for improvement:

• clarify doubts and explain in detail the choices and constraints shared by the working group and the users involved in the design;

• identify new information needs not addressed to integrate; • get immediate feedback on the ease of use of the web application; • observe any critical interaction.

4.8 Evaluation over the time

After deploying, the team's aim is to observe and understand how the technological solution will be actually used in the context of use. New adaptations may be necessary and appropriate in order to respond more adequately to end users’ needs and requirements in performing their tasks. The design team commitment will be continuing during the maintenance and upgrade phase.

Proceedings 19th Triennial Congress of the IEA, Melbourne 9-14 August 2015  

6  

5. Conclusion

The application will be deployed in June 2015. Observing the early feedback of the training courses, it seems that there will be some difficulties linked to the introduction of a new technological solution for managing patient safety data. Users who have not been involved in the whole development process, in fact, are more critical and less available to adopt the tool. Users who have participated to the implementation instead are satisfied with the outcome and eager to start using the web application. A further limitation of the project is related to usability testing, which were unable to investigate all aspects of each module. It was necessary a selection of some features and tasks to check; more extensive usability tests would have been probably helpful. At the deployment, a phone number will be given to support users in explaining difficult interaction or other criticalities. It will help to assess whether the training realized in the healthcare trust facilities, run by trained healthcare workers, has been effective. It would be also useful to understand if there are any functionalities, which seem critical and difficult to understand for the user. It would be interesting to see what type of user will have most reluctant to adopt the tool, and if this information will confirm the initial hypothesis. The challenge is still continuing by observing the interaction in the real contest of use. References Creswell J.C. 2013. “Research design: Qualitative, quantitative, and mixed methods approaches.” Sage Institute of Medicine 1999. “To err is human: building a safer health system” Washington DC: National Academy Press. Latif A., Rawat N., Pustavoitau A. et al. 2013. “National study on the distribution, causes, and consequences of

voluntarily reported medication errors between the ICU and non-ICU settings.” Crit Care Med 41:389-98 Pham J.C., Girard T., Pronovost P.J. 2013. “What to do with healthcare.” Incident Reporting Systems Journal of Public

Health Research volume 2:e27 Pham J.C., Aswani M.S., Rosen M. et al. 2012. “Reducing medical errors and adverse events.” Ann Rev Med 63:447-63 Sari A.B., Sheldon T.A., Cracknell A., Turnbull A. 2007. “Sensitivity of routine system for reporting patient safety

incidents in an NHS hospital: retrospective patient case note review.” BMJ 334:79 Kingston M.J., Evans S.M., Smith B.J., Berry J.G. 2004. “Attitudes of doctors and nurses towards incident reporting: a

qualitative analysis.” Med J Aust 181:36-9 Crystal A., Ellington B. 2004. “Task analysis and human-computer interaction: approaches, techniques, and levels of

analysis.” In Proceedings of the Tenth Americas Conference on Information Systems. A comparison of hierarchical and cognitive task analysis, New York, August 2004.

Caroll J.M. 2002. “Making use is more than a matter of task analysis.” Interacting with Computers 14: 619-62 Shepherd, A. 2001. “Hierarchical task analysis.” New York: Taylor & Francis Nielsen, J. 1994. “Usability engineering.” Academic