STRENGTHS AND LIMITATIONS OF THIS STUDY
-
This study will use mixed methods approach, which combines quantitative and qualitative research methods, to develop outcome sets for stroke in a multidimensional value assessment of traditional Chinese medicine (TCM).
-
This study will explore the feasibility of incorporating TCM syndromes into clinical and economic evaluations of TCM drugs to better inform decision-making.
-
Despite the efforts we will make to include a large number of stakeholders from diverse backgrounds, we cannot exclude the possibility that the outcome measures are not extensive or generalisable.
Introduction
Stroke is ranked as the leading cause of death in China and the second leading cause of death globally. Stroke is associated with the highest loss of disability-adjusted life years (DALYs) among all diseases.1 Due to cerebral blood circulation disorders, it is critical to ensure blood supply to the cerebral tissue during the acute stage of stroke, particularly for acute ischaemic stroke (AIS).2 Traditional Chinese medicine (TCM) and integrative medicine, including acupuncture, Tuina (Chinese massage), fumigation and washing therapy, play a vital role in the treatment of AIS in China.3 4 Using TCM alone or in combination with western medicines, such as thrombolysis, antiplatelet therapy, anticoagulation and improved cerebral circulation, can effectively improve patient outcomes and reduce stroke-related morbidity and mortality.5
According to the TCM theory, or the so-called ‘Bian Zheng Shi Zhi’, the diagnosis and treatment should be based on disease differentiation and syndrome differentiation. In contrast to western medicine, which focuses on vital signs or laboratory tests (eg, blood glucose), the diagnosis in TCM is based on clinical syndromes such as deficiency of vital energy, blood stasis and damp-heat syndrome, which largely rely on clinicians’ clinical experience. Recently, various instruments have been validated to quantify clinical syndromes to modernise TCM.6
Extensive clinical trials and systematic reviews have been conducted to assess the effectiveness, safety and economics of TCM/integrative medicines in the treatment of stroke. However, the number of clinical studies that can provide high-quality evidence remains limited7 mainly because of substantial variations in the outcome measures used in clinical studies.8–10 For western medicine, the Core Outcome Measures in Effectiveness Trials (COMET) initiative has promoted the application of core outcome sets (COS),11 to improve the consensus on the selection of outcome measurements used in clinical trials. Generally, the COS identify the minimum set of outcomes that should be measured and reported in clinical trials of a specific disease. To address the heterogeneity in measuring and reporting outcomes in TCM, some COS have been developed specifically for TCM clinical trials but with limited emphasis on TCM syndromes.8 9
Despite efforts to develop outcome sets for clinical studies, insufficient attention has been paid to the choice of outcomes accepted for health technology assessment (HTA). HTA is defined as a multidisciplinary process that determines the value of health technology (eg, medicine, devices, vaccines, procedures and programmes) with the primary objective of informing decision-making to achieve an equitable, efficient and high-quality health system.12 Similar outcomes have been evaluated in clinical trials and HTA; however, discrepancies are apparent because the purposes of clinical trials and HTA decision-making are different. Premarketing clinical trials generally determine whether a medicine is effective with acceptable safety profiles, whereas HTA agencies focus on the economic value, clinical effectiveness and safety of a medicine in real-world settings.13 HTA agencies indicate a stronger preference for endpoint outcome measures, whereas regulators tend to accept intermediate outcomes.13 In a recent study assessing the similarity between COS and HTA outcomes, only 43% of the core outcomes overlapped in both the HTA database and the COS, indicating heterogeneity between the outcome sets used in clinical studies and HTA.14 TCM (eg, Ginkgo biloba Damo injection) is often used as an adjuvant therapy for AIS patients in China but its clinical and economic value has not been well justified owing to the lack of methods, especially outcome sets that incorporate TCM syndromes.
Given the increasing number of innovative TCM drugs approved and available in the market, many have been included in the National Reimbursement Drug List (NRDL) through price negotiations. The price negotiation process generally requires an HTA. However, the outcomes that should be used to determine the efficacy of TCM drugs remain unclear. Hence, obtaining a comprehensive list of outcomes for TCM value assessments is crucial. Therefore, the main objective of this protocol was to develop an outcome set used in multidimensional value assessment for the TCM treatment of stroke.15
Methods
Aim and scope
The aim of this protocol is to develop outcome sets that accept HTA decision-making in TCM for stroke. The scope of the outcome sets was based on the criteria recommended by the Core Outcome Measures in Effectiveness Trials (COMET).16 This protocol followed the Core Outcome Set Standards for Reporting Statements.17 The scope of the outcome sets for HTA included all TCM pharmacological therapies in both controlled trials and observational studies for adult patients with AIS. We will evaluate the outcomes of controlled trials and observational studies separately because they are generally conducted in different settings with different logistical support, which might influence the selection of outcome measures.18 The study was planned to be implemented from 1 January 2023 to 31 December 2025.
This protocol was reviewed and approved by the Institutional Review Board of Minhang Hospital of Fudan University.
Overview of outcome set development
The outcome set will be developed in four phases summarised in figure 1. First, we will perform a systematic literature review to identify candidate outcomes that have been previously measured in published high-quality studies and/or are important for stakeholders. Second, we will develop a comprehensive list of outcome measures within each HTA domain by conducting multistakeholder, semistructured interviews. Third, we will conduct two rounds of Delphi surveys to prioritise the outcomes for each HTA domain. Finally, the outcome sets will be finalised by holding virtual ratification meetings with multiple stakeholder groups.
Four phases of the outcome set development.
Phase I: systematic literature review
We will conduct a systematic review to assess the outcome measures used in randomised controlled trials (RCTs) and observational studies on stroke, including all TCM pharmacological therapies. In addition, we reviewed previously published COS and HTA reports. This systematic review was registered on the PROSPER platform.
Search strategy
We will search both English and Chinese databases, including Medline, Embase, Cochrane Library, China National Knowledge Infrastructure (CNKI), WANFANG data and China Biology Medicine (CBM), for all relevant literature published between 1 January 2012 and 31 December 2022. To ensure a more comprehensive review, we will also search clinical trial registries, including the Chinese Clinical Trial Registry, ClinicalTrials.gov and COS databases. The languages will be limited to English and Chinese.
Eligibility criteria
We will include publications that meet the inclusion and exclusion criteria listed in table 1.
Inclusion and exclusion criteria of systematic literature review
Literature selection
Duplicate records will be removed using Endnote X9 literature management software. Two researchers will screen the articles independently. First, the titles and abstracts of all records will be screened and the full text will be reviewed to confirm whether the records met the eligibility criteria. A third senior researcher will independently screen the records if any disagreements existed.
Quality assessment
For studies meeting the eligibility criteria, a quality assessment of the studies will also be conducted using the Cochrane Risk of Bias 2 (RoB 2) tool, the modified Jadad scale for randomised trials19 and the Cochrane Risk of Bias In Non-randomised Studies of Interventions (ROBINS-I) tool for observational studies.20 21 For RoB 2 tool, each domain was classified as yes, no or unclear on the basis of risk of bias. Jadad scale was used to quantitatively grade the method quality of studies, which was categorised as high quality (≥4 points) and low quality(<4 points).
Data extraction
Two researchers will extract data from the studies that met the eligibility criteria, including authors, title, year of publication, type of study design (RCT, cohort studies, case-control studies, etc), HTA decision-making, sample size, characteristics of participants (age, sex, Chinese medicine syndromes), types of pharmacological interventions (drug name, duration of therapies) and outcome measures (name, definition, measurement instrument/method). Because Chinese medicine syndrome types may determine the selection of outcome measures, two researchers will extract the syndrome types, including the syndrome name, signs, tongue, pulse and other syndromes. Chinese medicine syndromes will be categorised according to the TCM guidelines for stroke. We will extract evaluation scales designed specifically for Chinese medicine such as the ischaemic stroke TCM syndrome diagnostic scale.22 In addition, the name of the Chinese medicine syndrome will be standardised. Any disagreements will be discussed with a third senior researcher to reach a consensus.
Data analysis
We will develop a list of candidate outcomes based on the existing literature, with a special focus on those used in HTA decision-making. Outcome measures were grouped into HTA domains, including effectiveness, safety, economics, access to care and drug quality. The grouping of the outcome measurements will be independently performed by two researchers. The outcomes will be ranked according to their frequency in each HTA domain.
Phase II: semistructured interviews
The inputs of stakeholders, including doctors, policymakers, health economists and patients, are crucial for the development of the outcome set. We will recruit multiple stakeholders to participate in semistructured interviews.
Sampling and recruitment strategy
Our research team will collect the names and contact details of all potential participants using our professional networks. Participants will be recruited through purposive sampling to ensure the inclusion of key individuals who are suitable to contribute to this study.23 24 To ensure that they are suitable for our study, we will evaluate their background regarding their expertise and research experience before sending out the invitation. Potential participants will first be invited via social media (eg, WeChat) or email. If there are no replies, we will make phone calls or text messages were sent. Snowball sampling was used when more participants were required. For example, potential participants were asked to refer to other individuals who might also be suitable for the interview. The sample size will be sufficient to achieve data saturation according to previous research.25 Sampling and recruitment strategy is shown in table 2
Eligibility criteria of semistructured interviews
Development of interview guide
A semistructured interview guide will be developed based on a review of the published literature on HTA decision-making for stroke therapy grounded in the HTA value framework.26 The theoretical framework is used to ensure that all relevant domains that can determine the value of therapy will be deliberated on, including effectiveness, safety, economics, innovation, quality and access. The interview guide will cover questions related to participants’ views on the domain of value assessment for HTA decision-making and the outcome measures in each domain. To ensure the quality of the interview guide, pilot interviews will be conducted to evaluate the dependability of different stakeholders on the research team. Pilot interviews will not be included in the analysis.
Data collection
If they agreed, they will be asked to provide written informed consent before commencing. One researcher trained in qualitative research will lead the interviews. The interviews will be audio-recorded and transcribed. Two researchers will independently review the transcripts to ensure credibility and dependability. We will also collect data on participants’ characteristics.
Data analysis
Thematic analysis will be carried out using the general inductive approach based on the steps outlined by Thomas DR.27 All transcripts will be input into NVivo software based on the priori themes including HTA value domains and additional themes emerging from the data; two researchers will read the transcripts from interviews and construct a thematic framework that captures data related to (1) stakeholder’s views about existing standardised outcome measures and their priorities and (2) conceptualisations of outcomes that are not covered in the existing outcome measures. Another researcher will review the analysis to ensure credibility.
Phase III: Delphi survey
To prioritise the outcome measures, a two-round Delphi survey will be conducted to gain consensus among more stakeholders with diverse backgrounds. Compared with interviews, the Delphi survey has the advantage of encouraging junior participants to express their opinions and collecting stakeholder opinions more efficiently. The survey will comprise two rounds of online questionnaires.
Selection of stakeholder
Healthcare professionals including doctors specialising in stroke, clinical experts in TCM, health economists, policymakers and health outcome methodologists participated in two rounds of the Delphi survey. The eligibility criteria are listed in table 3. We will use the sampling and recruitment strategy specified in ‘Phase II: semistructured interviews’. To include more stakeholders, those participating in the semistructured interviews were not invited. Stakeholders participate in two rounds of Delphi surveys.
Eligibility criteria of Delphi survey
Sample size estimation
Because there is no robust method for calculating the sample size for the Delphi approach in the development of a COS,28 the number of stakeholders is generally based on previous studies29 with at least 12 participants.16 Therefore, we plan to recruit 15 stakeholders for each round of Delphi or a total of 30 participants, based on the COMET initiative guidelines and existing research.30 31
Delphi survey
The Delphi survey will be designed to obtain the importance scores of domains and outcome measures for HTA decision-making. The online questionnaire will be sent to the participants via social media or email containing weblinks (Wenjuanxing).
The questionnaire will be based on information identified from systematic reviews and semistructured multistakeholder interviews. Considering language barriers, the outcome measures in English will be translated into Chinese. According to the Grading of Recommendations Assessment, Development and Evaluation Working Group definitions, each item (including domains or outcome measures) will be ranked on a nine-point Likert scale. The higher the score, the more important the item is; scores of 1–3 points indicate limited importance; scores of 4–6 points indicate importance but not critical; and scores of 7–9 points indicate critical importance. In addition, the option of ‘unknown or unable to decide’ will be also available. The questionnaire will be first reviewed and piloted by a small group of researchers consisting of healthcare professionals, outcome methodologists and health economists.
All candidate outcomes will be included in the questionnaire used in the first round of the Delphi survey. In addition to the nine-point Likert scale, participants will be offered the opportunity to suggest additional outcome measures that are deemed important. We plan to complete the first-round Delphi survey within 4 weeks. Participants will receive reminder messages via email or social media at the end of the third week.
Stakeholders who completed the first round of the Delphi survey will be invited to participate in the second round survey. The questionnaire will only include outcome measures that are scored ≥4 by over 70% of the participants in the first round of the survey. Participants will also be offered 4 weeks to complete the questionnaire.
Phase IV: consensus meeting
An online consensus meeting will be held after completing the two-round Delphi survey to confirm the domains and outcomes of the HTA assessment for TCM. Because there is no robust method to estimate the sample size for a consensus meeting, we plan to invite at least 10 stakeholders, following previous studies. The participants will mainly consist of stakeholders who participated in semistructured interviews and Delphi surveys. For outcome measures with less agreement, the stakeholders will vote anonymously. Table 4 presents the definitions of consensus.
Definition of consensus
Ethics and dissemination
This protocol was reviewed and approved by the Institutional Review Board of Minhang Hospital of Fudan University. Our findings will be disseminated to academic conferences and peer-reviewed publications. Where applicable, we aim to communicate relevant results to policymakers, contributing to evidence-based decision-making.
Patient and public involvement
No patient is involved.
Discussion
Although several COS have been developed for stroke,8 9 32–34 most have focused on western medicine32–34 or evaluated the clinical effectiveness of TCM therapy.8 9 Because clinical trials and HTA have very different purposes, there are substantial discrepancies in the selection of outcome set. No outcome set has been specifically developed for value assessment of TCM. To the best of our knowledge, this is the first protocol to develop a standardised outcome set for HTA that will improve the consistency and transparency of value assessments in TCM. Therefore, we recommend that the proposed outcome set should be measured and reported as the minimum set of outcomes for HTA assessment in TCM treating stroke. The development of this protocol followed the Core Outcome Set Handbook 1.0.16
The candidate domains and outcomes will be obtained from the existing literature through a systematic literature review and from multistakeholder conceptualisation through semistructured interviews. To reach a consensus on the final set of outcomes, we will hold two-round Delphi surveys and consensus meetings to rank the importance of the outcome measures.
In addition to value assessment, Chinese medicine syndromes—a unique characteristic of TCM—will also be incorporated in the development of the outcome set. Based on our preliminary analysis, the Stroke Syndrome of TCM (SSTCM) scale covers visual, facial palsy, motor arm and leg, and language domains, which are also covered by the NIH Stroke Scale (NIHSS), suggesting the similarities between the TCM syndrome scale and instruments developed for western medicine. Although many TCM trials have used endpoints from western medicine, Chinese medicine syndromes play a critical role in the evaluation of the clinical effectiveness of TCM. With regard to incorporating the syndrome scale into HTA, we will explore the possibility of assigning different weights to various outcome measures using multicriteria decision analysis (MCDA).35 Recently, MCDA has been used by many HTA agencies to evaluate the value of medicines.36 Therefore, this well-designed protocol considers the distinguishing nature of TCM in value assessment.
Ethics statements
Patient consent for publication
This post was originally published on https://bmjopen.bmj.com