Background
To ensure the therapeutic success of contemporary radiotherapy modalities, consistent quality across all procedural steps is imperative.1 2 Mistakes in therapy planning and execution can result in adverse events, including patient fatalities.3–5 In Germany, akin to other European nations, the Radiation Protection Act and Ordinance stipulate guidelines for thresholds and quality assurance measures concerning the use of ionising radiation in medical applications. These regulations mandate prospective risk assessments and the reporting of critical radiation-related events. However, the exact methodologies for effective risk management can vary significantly across individual facilities.6 Moreover, the absence of centralised guidelines necessitates considerable effort to establish facility-specific safety protocols.
Epidemiological data, primarily based on reported adverse events, likely underestimate the true risk to patient safety across healthcare settings.2 Additional methods such as monitoring by indicators are therefore favoured.7 These are designed to identify and quantify situations most likely to result in adverse events. While various sets of patient safety indicators (PSIs) are available for general healthcare, those are difficult to adapt for radiation oncology. Existing safety indicators in the healthcare sector often focus on inpatient scenarios like fall or pressure ulcer prevention, which are not directly applicable to the predominantly outpatient-based radiation oncology treatments.8 Oncological treatments, including radiotherapy, entail unique risks that demand specialised expertise for effective risk assessment and the identification of potential therapy side effects. For radiation oncology, several groups reported on the related topic of quality indicators, which focuses not only on effectivity, effectiveness and reproducibility but also on safety of radiation therapy. Nevertheless, none of those encompass all aspects of patient care including supportive care and medication alongside with safety culture, risk management and aspects of personal resources.9–12
Objectives
The primary objectives of this study are: (1) to develop PSIs tailored specifically for radiation oncology, (2) to design and psychometrically validate an instrument for assessing patient safety in German radiation oncology facilities and (3) to evaluate the feasibility of implementing this instrument in routine clinical practice.
Methods
This investigation is a 3-year, mixed methods study, structured into three distinct phases. An illustrative overview of the study design is provided in figure 1. The study commenced in July 2023 and is planned to conclude in June 2026, with each project phase lasting approximately 1 year. A detailed overview of the aimed schedule is available in the study synopsis in online supplemental material.
Supplemental material
Overview of study design and samples. *Experts include all five relevant professional groups: Physicians, medical physicists, radiation therapists (RTTs), nurses and administrative staff.
Phase I: identification of PSIs
In this initial phase, we aim to identify PSIs pertinent to radiation oncology, with an emphasis on the quality of processes and structures and, moreover, on possible quantitative data representing benchmarks associated with safe patient care. An extended panel of experts will discuss these indicators, initially in focus groups, culminating in a consensus through a Delphi study.
A comprehensive review for relevant PSIs in radiation oncology will be conducted using PubMed as well as national and international recommendations from affiliated societies. We will include all publications from 1990 until today in German or English which cover relevant keywords concerning radiation oncology and patient safety. Data from incident reporting systems as well as from published risk analyses are also included in our screening if they underline certain risks specifically preventable through safety measures. Further, we will include papers on oncology and systemic tumour therapies, which cover both patient safety and supportive care. Publications for in vitro data or animal models as well as publications covering only specific therapy options (like proton therapy) will be excluded. Examples of suitable PSIs include measures in the treatment process (eg, patient identification), compliance with legal requirements for machine-based quality assurance (eg, calibration intervals) and adherence to dose limits. Existing PSIs from known questionnaires are also considered. English-language PSIs identified in the literature search are translated following the TRAPD (translation, review, adjudication, pretest and documentation) protocol.13 The objective is to capture indicators as broadly as possible. Based on the literature review, we will also compile different dimensions of patient safety specific to radiotherapy.
To refine the initial set of indicators, experts will participate in focus groups where they discuss the relevance of identified PSIs and can propose additional ones. The focus group discussions aim to streamline the total number of indicators for clarity. These sessions will be audio recorded, transcribed and anonymised. We will analyse them using qualitative content analysis, employing both deductive and inductive approaches.14
Further, we will conduct semistructured interviews with patients who are receiving or have received radiotherapy. In those interviews, we will discuss all PSIs that concern interactions with patients. We will discuss the patients’ experiences with patient safety throughout their radiotherapy and will explore whether they want to add any PSIs. Again, the interviews will be audio recorded, transcribed, anonymised and analysed using qualitative content analysis.14
In order to design the instrument efficiently, the time frame and effort required to use it must be limited, which requires a selection of indicators. This will be achieved through a two-round Delphi study15 using an online survey administered via LimeSurvey (LimeSurvey GmbH, Hamburg, Germany). Participants will rate the relevance and accessibility of the indicators and a free-text field will allow for the addition of other relevant indicators. In the second round, participants will receive feedback on the first round’s results and have the opportunity to revise their ratings.15
Sample
Given that radiation oncology is an interdisciplinary field, our panel of experts will comprise five professional groups: physicians, medical physicists, radiation therapists, nurses and administrative staff. Participants will be recruited via email lists from professional societies, supported by collaborating departments. Each professional group will be represented by at least two participants. Two focus groups, each consisting of 8–10 participants, will be conducted, which include 8–12 patients covering different ages, different diseases and types of therapy indication (palliative vs curative vs prophylactic). The Delphi study will be carried out in two rounds with a minimum of 10 participants who will be recruited through email, supported by professional societies and partner institutions.
Phase II: psychometric testing
The PSIs selected through the Delphi process will be transformed into questionnaire items, which will be evaluated using a scale (eg, degree of agreement). By categorising these safety indicators under their respective dimensions, a structured questionnaire with subsections will be developed.
The questionnaire’s comprehensibility will be initially assessed through cognitive interviews.16 These interviews will be audio recorded, transcribed, anonymised and subsequently analysed. If item revisions are required, the process will be repeated until no changes are necessary.
The finalised questionnaire will then be pilot-tested among the participating professional groups at collaborating institutions, using an online survey administered via LimeSurvey (LimeSurvey GmbH, Hamburg, Germany).
Based on the gathered data, we will perform psychometric analyses. Item difficulty will be calculated to examine ceiling and floor effects, thereby ensuring sufficient variability in the responses.17 Missing values will serve as a proxy for item acceptability. Inter-item and item-total correlations will be assessed to eliminate redundancy. Structural validity will be confirmed through factor analysis, modelling the dimensions identified in phase I. Model fit will be evaluated based on several fit indices including root mean square error of approximation, standardised root mean square residual, Tucker-Lewis index and comparative fit index.18 The final questionnaire items will be selected in a team discussion, considering both psychometric properties and content relevance.
Sample
Participants for the interviews will again represent all five professional groups involved in patient care and will be recruited similarly to phase I. Each questionnaire item will be tested in at least five cognitive interviews per round. The sample size will vary depending on the questionnaire’s size and the number of testing rounds (eg, n=5–10). For the psychometric evaluation, a sample size of 3–5 persons per question is deemed suitable, although the exact number of questions will be determined in phase I. An average of three participating institutions, each with an average of three staff members, is planned for each of the 17 state medical associations (‘Ärztekammer Kreis’) (n=153). Recruitment will again be conducted via email, supported by professional societies and affiliated institutions.
Phase III: test for feasibility in routine care
The final questionnaire will be introduced into routine healthcare settings for feasibility testing. Participating facilities will have the opportunity to self-evaluate using the indicators, calculate their scores and, where necessary, identify measures to enhance patient safety. Comprehensive support will be extended to the facilities to determine the most effective means of integrating the questionnaire into their existing workflows and structures. Options include using a web app or paper formats, along with guidelines specifying who should complete the questionnaire and when.
To gain insights into user experiences for the experts of the phase before, semistructured interviews will be conducted with individual users. The focus will be on determining factors that either facilitate or hinder the effective use of the questionnaire and the application of its results. The interview guide will be structured around established implementation frameworks, such as the Consolidated Framework For Implementation Research.19
Subsequently, an expert workshop will be held to discuss the findings and explore options for broader implementation and utilisation of the questionnaire, as well as future studies comparing PSI with clinical metrics and outcome.
Web app development
Concurrent with phases II and III, an external provider will design and develop a web app to host the questionnaire. Once launched, users will have immediate access to category-specific evaluations. Optionally, the collected data can be shared with the project team or future users for further scientific exploration.
Sample
The final testing phase will involve at least four healthcare facilities. Within each facility, a minimum of three interviews will be conducted across different professional groups. An expert workshop will gather participants from across Germany, including professionals in radiotherapy, patient representatives and other experts, amounting to approximately 30 participants in total.
Patient and public involvement
We will involve patients by forming a patients’ advisory board, which holds multiple meetings throughout the project (at least two per year, more if necessary). Four patients of the university hospital have agreed to advise the project. Possible tasks in general are discussions of the current state of the project and in particular advisements for the semistructured interviews with patients. Here, the patient advisory board can identify ways to recruit patients for the interviews, elaborate the interview guide to ensure comprehensibility of the task and relevance for the patients as well as contribute to the interpretation of the results. Furthermore, the patient advisory board will contribute to the dissemination of information about the project. Here, the patients will assist in writing and revising lay information about our project which will be available online. Finally, the patient advisory board will be invited to our expert workshop and offered the possibility to report their own work and experiences within the project.
Discussion
This study aims to create the first instrument specifically designed to systematically assess patient safety in the field of radiation oncology in Germany. Beyond an extensive literature review, the study incorporates expert recommendations from all pertinent professional groups. The formation of a patient’s advisory board offers the opportunity to include the patient’s point of view through direct involvement. Additionally, the developed questionnaire will undergo psychometric evaluation across multiple institutions. The study will also investigate potential barriers and facilitators for its implementation through interviews and an expert workshop.
The finalised questionnaire will serve as a digital self-assessment tool for radiation oncology departments. This tool will enable these departments to evaluate their performance in various facets of patient safety, prioritise interventions to increase patient safety and track outcomes over time. The final tool can serve to monitor and improve patient safety on a broad basis in conjunction with incident reporting and patient follow-up data. One limitation of this project is that no data comparison is planned for these quantitative metrics against the degree of implementation of safety indicators. Subsequent projects might show quantitative improvements through comparing those metrics against each other.
Moreover, this initiative contributes to the broader discourse on safety awareness within healthcare. It provides individual facilities the opportunity to benchmark their processes against national and international guidelines, thereby fostering a culture of continuous improvement in patient safety.
Ethics and dissemination
This study will adhere to the most recent version of the World Medical Association’s Declaration of Helsinki, upholding the principles of good scientific practice. Participation in the study is entirely voluntary and poses no foreseeable risks to the participants. All participants will be fully informed of the study’s objectives, data collection procedures and the intended use of the collected data. Prior to their involvement, written informed consent will be obtained from each participant. Rigorous measures will be implemented to ensure compliance with data sensitivity, data protection and confidentiality standards. Ethical approval for this study has been granted by the Hamburg Ethics Committee (Approval Number: 2023-101018-BO-ff).
To maintain transparency and engagement, a biannual newsletter will be distributed to study participants, collaboration partners and the interested public, updating them on the study’s progress and key findings. The study’s outcomes will be submitted for publication in scientific journals, with a preference for open access formats to maximise accessibility.
In addition to scholarly publication, the results will be disseminated during the implementation workshop in phase III, which will involve a diverse group of stakeholders. Moreover, findings will be showcased at relevant national and international conferences to broaden their impact within the professional community.
The questionnaire developed through this study will be made available online, ensuring its universal accessibility and paving the way for its widespread adoption in improving patient safety in radiation oncology.
Acknowledgments
The authors would like to thank the associated institutions for support during study planning and recruitment: PD Dr Cihan Gani, Tübingen; Professor Dr. Hans Christiansen, Hannover; Professor Dr. Hans-Theodor Eich, Münster; Professor K. A. Hartmann, Düsseldorf; PD Dr. med. David Kaul, Berlin; PD Dr. med. Gunther Klautke, Chemnitz; Professor Dr. Heinz Schmidtberger, Mainz; Professor Dr Dirk Vordermark, Halle; Professor Dr. Andrea Wittig-Sauerwein, Würzburg; Professor Dr. Florian Würschmidt, Hamburg; Dr. med. Christina Woywod, Pinneberg; PD Dr. Jan Kriz, Münster; Christiana Lütter, Köln; Professor Dr. Ursula Nestle, Mönchengladbach; Dr. Laith Samhouri, Lüdenscheid; Professor Dr. Markus Buchgeister, DGMP; Angelika Kleine, OVIRO; Gunthard Kissinger, M.U.N.D.; Helmuth Wolken, Selbsthilfe Lungenkrebs Hamburg; Birgit Schlömp, VMTRO; Silke Wischer, BVPRO; Dr. Michael Baehr, UKE Klinikapotheke; Avin Hell, UKE UCCH Patient:innenbeirat; and Professor Dr. Levente Kriston, UKE Hamburg.
We acknowledge financial support from the Open Access Publication Fund of UKE – Universitätsklinikum Hamburg-Eppendorf.
This post was originally published on https://bmjopen.bmj.com