Improving health system responses when patients are harmed: a protocol for a multistage mixed-methods study

Purpose

We will undertake a qualitative deductive content analysis37 of 300 patient safety investigation reports. The aim is to assess their quality using four existing assessment frameworks and determine the incident types associated with the investigation reports. Stage 2 is aligned with research question 2.

Setting

The study setting is health services within the three of the four partner jurisdictions—NSW, Queensland and Victoria. Legislation in the ACT precludes sharing of the investigation reports within a research environment. The investigation reports will be accessed via the partner organisations.

Data sources

Health services undertake investigations which inform reports (‘investigation reports’) when there is an adverse event. The inclusion criteria will be investigation reports undertaken on adverse events designated as the highest level of severity within each jurisdiction. We will analyse 100 each from NSW, Victoria and Queensland. A sample of 100 at state level will allow CIs of+/−10% points around a point estimate of 50%.

Data in the investigation reports include details such as the incident type, investigation type, date, state, health service and location within the health service. The main body of the report is in a narrative form outlining the nature of the adverse event, the contributing factors and the risk reduction action plans/recommendations, which propose actions to reduce the risk of adverse event recurrence.

Sampling

We will request a list, from the partners, of all investigation reports completed within the three states during 2022 and 2023. The lists will contain several fields to facilitate the selection of a representative sample of investigation reports from each jurisdiction. These fields include:

• Record/investigation number (to identify and select those to request for analysis).

• Whether or not the investigation was protected by legislation.

• The investigation method (eg, root cause analysis, London Protocol38).

• The health service location where the adverse event took place (to identify the size and rurality of the facility).

• Medical specialty related to the adverse event.

• Whether the adverse event was a notifiable and/or sentinel event.

On receipt of the list, the research team will stratify the investigation reports into five categories specified above (protected by legislation, investigation method, health service location, specialty, notifiable/sentinel event) and select randomly until we have a representative sample (100 Investigation Reports for each state) based on these variables. We intend to oversample lesser used investigation types and smaller health facilities to increase our confidence in the results of our findings in these subsamples; the overall result will be appropriately weighted to account for over-sampling.

Once the investigation reports being included in the sample are identified, we will formally request the selected reports to be transferred to a Macquarie University secure SharePoint portal. The 300 investigation reports will then be allocated a unique study ID. Any small cell sizes with variables or characteristics with the potential to be reidentified will be aggregated into broader categories such that the reported data are non-identifiable.

Data Collection

The incident type related to the investigation will be classified according to the high-level classes of a modified version of the WHO’s International Classification for Patient Safety.39 Demographic information related to the patient (eg, age (recorded in age bands, eg, 60–69 years) and gender) will be extracted from the reports.

The quality assessment review, using a deductive content analysis, involves reading and coding the investigation reports and extracting information from them guided by four quality assessment frameworks:

1. The Healthcare Inspectorate tool from the Netherlands40 assesses adherence to guidance for patient safety investigation methods (eg, investigation process and team, adverse event reconstruction, analysis, conclusion, recommendations, aftercare and board responses).

2. The US Department of Veteran Affairs’ recommendations effectiveness scale41 assesses recommendations’ relative ‘strength’ according to their type, effectiveness and sustainability. According to the criteria, effective reports should have clear causal statements and strong actions. Strong actions are most likely to be system-based according to human factors engineering principles.42

3. Accimaps and SEIPS.36 43 The SEIPS framework identifies work system descriptors that might have contributed to an adverse event, also known as ‘contributing factors’. SEIPS categorises these into external influences, organisation of work factors, task factors, person factors, tools and technology and the physical environment. These apply at different levels of a healthcare system. Accimaps is a system-based incident analysis technique. An output of an Accimap is a layered causal diagram, which shows the relationships of contributing factors across the different levels of a system. By combining the SEIPS categories with simplified Accimaps levels, the assessment can identify the breadth of healthcare system levels considered during each investigation.

4. The Learning Response Review and Improvement Tool from NHS Scotland and HSSIB.25 The tool rates the investigation according to criteria considered good practice for systems investigations to identify areas for improvement and to act as a quality assurance process for monitoring and evaluating the standard of organisational learning response reports. The tool includes criteria related to whether both patients (and their families and carers) and staff have been actively listened to and emotionally supported where required.25

The data items within each of the four quality assessment frameworks against which the text in the investigation reports will be coded are shown in online supplemental material 1. The research team will develop data collection tools in a Microsoft Access database to record the quality assessment framework data items. The databases will be securely stored on the Macquarie University SharePoint platform.

Experienced safety science researchers will undertake the coding. They will independently code the first five investigations and then meet, compare, assess and discuss discrepancies. From that point, every 10th investigation will be double coded, recorded and discussed.