Introduction
A recently published research article by Soeorg et al highlighted that 80% of European secondary data sources commonly used in pharmacoepidemiology lack information on the therapeutic indication of dispensed medicinal products.1 The inherent lack of information on the therapeutic indication exposes pharmacoepidemiologists to the risk of introducing biases when conducting epidemiological research using secondary data sources.2 Confounding by indication and misclassification of exposure are the main biases that can be introduced in epidemiological research when information on the therapeutic indication of medicine is missing.3–5 The lack of information on the therapeutic indication makes it challenging to adjust for confounding by indication, even when using advanced methods for confounding adjustment in real-world data.6 7 Currently, in observational studies, conventional methods have not been demonstrated to fully resolve confounding by indication.8
Biases due to the lack of information on the therapeutic indication of pharmacological treatments are expected to be high, especially for drugs with multiple approved therapeutic indications, such as antiseizure medications (ASMs) which are also known as antiepileptics drugs (AEDs) (online supplemental table S1). A representative example of the role of confounding by indication for AEDs and its impact on epidemiological estimates is provided by the study conducted by Battey et al.4 The study showed increased mortality and increased risk of intracerebral haemorrhage among patients taking AEDs (online supplemental figure S2). The role of confounding by indication in relation to prognosis among individuals treated with AEDs was examined in a study by Renoux et al that investigated how the therapeutic indication of AEDs affected the association between the AEDs treatment and clinical outcomes.9 Renoux et al observed that the bias identified in this study was only solved when AEDs were compared among individuals using AEDs for the same therapeutic indication. This was demonstrated by showing that statistical adjustment was insufficient to rule out the bias in the observed associations and that stratification by therapeutic indication was needed to control for the baseline risk that differed between individuals using AEDs for different therapeutic indications.9
Supplemental material
Recent epidemiological data from the Danish Epilepsy Centre, the national institution for the diagnosis and treatment of epilepsy in Denmark, has highlighted that in 2020 approximately 50 000 resident individuals in Denmark have been diagnosed with epilepsy.10 However, an up-to-date analysis of data on drug consumption of AEDs in Denmark using the web-based Danish Medical Statistics (MEDSTAT) in the period 2000–2020, revealed that in 2020, in Denmark, a total of 83 820 individuals aged 65 or older redeemed a prescription of an AED.11 As mentioned earlier, AEDs have multiple approved therapeutic indications, which can expose researchers to the risk of misclassifying the therapeutic indication of AEDs (online supplemental table S2). Older individuals exposed to AEDs are the ones at higher risk of misclassification of exposure due to the higher chances of having AEDs used for other therapeutic indications due to the intrinsic association between age and concurrent diseases for which AEDs are approved.12 Recently conducted systematic reviews have highlighted the lack of valid methods to predict the therapeutic indication of AEDs when such information is missing in secondary data sources (online supplemental appendix 1). To overcome this gap in knowledge, we developed a new data-driven method to predict the therapeutic indication of redeemed prescriptions in secondary data sources using AEDs users among individuals aged ≥65 identified in Danish registries as proof-of-concept drug group. Additionally, this study aimed to compare the similarity in baseline characteristics between individuals predicted to use AEDs for epilepsy versus those predicted for other therapeutic indications. Furthermore, it sought to assess their overall survival to investigate the impact of misclassification of the therapeutic indication on mortality.
Supplemental material
Methods
Study design and setting
This study is a registry-based cohort of AEDs users in Denmark aged ≥65 with a confirmed diagnosis of epilepsy, identified using the incident new-user study design. The study period was from January 2005 to December 2017. The study population was followed for 365 days from the index date (ie, date of first redeemed AED following the diagnosis of epilepsy) until one of the following events happened: (1) ending of the follow-up year, (2) death from any cause, (3) emigration or (4) discontinuing the treatment with AEDs. Discontinuing the treatment with AEDs was defined as two times the median of the inter-arrival distribution of the redeemed AED at the index date.
Data sources
Data on patient demographics, vital status, current and previous medication treatments and coexisting medical conditions were obtained from Danish nationwide medical, and administrative databases/registries. In Denmark, each Danish resident is assigned a unique Danish civil registration number (CPR) at birth or immigration, which enables accurate linkage of information from all Danish registries and the tracking of medical events recorded in these registers across time.
Danish Civil Registration System
The Danish Civil Registration System (CRS) was initiated in 1968 to administrate a personal identification number system, which is a valid individual-level linkage with other Danish registries. The CRS was used to retrieve information on the date of birth, sex, emigration date and the CPR number for individuals included in the study population.
Danish National Patient Registry
The Danish National Patient Registry (LPR) has collected information on each Danish resident’s somatic hospital encounters since 1977, and from 1995, information from psychiatric and emergency departments and outpatient visits.13 The LPR was used to assemble received diagnoses (ie, epilepsy and comorbidities) during hospital admissions along with the date of diagnosis and extract information on hospital records (eg, surgical procedures).
Danish Medicinal Product Statistics Registry
The Danish Medicinal Product Statistics Registry (DNPR) provides complete information on all redeemed prescriptions in Danish community pharmacies (since 1995).14 The DNPR was used to extract information on redeemed prescriptions of older individuals diagnosed with epilepsy. In this study, we focused exclusively on prescription claims for individuals aged 65 and above from the DNPR as this study is part of the project titled ‘Potentially inappropriate prescriptions to elderly patients: identification and risk minimization’.
Danish Register of Cause of Death
The Danish Register of Cause of Death (DRCR) collects information on all Danish death certificates, including the CPR number, date of death and the cause of death (by ICD-10 codes since 1994). The DRCR was used for censoring the follow-up period at death.
Laboratory Information System databases
The Clinical Laboratory Information System database at Aarhus University Hospital and the nationwide Register of Laboratory Results for Research are the two Danish laboratory databases with different geographical and temporal coverage, which collect individual-level biomarker results from hospital encounters and general practitioners.15 Both laboratory databases were used to ascertain therapeutic drug monitoring (TDM) results of AEDs for individuals included in the study population.
Study population
The study population was composed of all Danish older individuals ≥65 years with first-ever redeemed AED prescriptions after receiving the diagnosis of epilepsy without prior use of any other AEDs (ie, incident new users). A washout period of 12 months was applied, in which individuals did not have at least one AED prescription redeemed regardless of whether or not it matched the AEDs redeemed at the index date. The washout period of 12 months was chosen to reduce the level of misclassification of ‘true’ AEDs incident new users associated with a relatively shorter washout period.16
The ‘washout period’ refers to a designated span of time during which participants are not exposed to AEDs. This interval allows for the clearance or ‘washout’ of any lingering effects of prior treatments, medications or interventions that could potentially confound the study results.
Study outcomes
The primary outcomes of the study were the assessment of the performance of the algorithm for predicting the therapeutic indication of AEDs by the proportion of individuals for which it was possible to predict the therapeutic indication of AEDs. Additionally, the sensitivity and misclassification of the algorithm were measured using a subset of data of redeemed prescriptions with recorded therapeutic indications. As a secondary outcome, we measured the proportion of overlapping density functions of propensity score (PS) between individuals predicted to use AEDs for epilepsy versus other therapeutic indications. A Cox hazard model was used to compute the cumulative hazard of the overall survival among individuals predicted to use AEDs for epilepsy versus other therapeutic indications to investigate the impact of misclassification of the therapeutic indication on mortality.
Developed algorithm
An overview of the developed algorithm is provided in figure 1 (a detailed description of the strategies implemented in the algorithm is provided in online supplemental appendix 2). The strategies were arranged based on a hierarchy of certainty for the predicted therapeutic indication of redeemed AEDs. The hierarchy was determined by the perceived level of evidence provided by measured events before and during the follow-up period (online supplemental figure S2). Each step in the hierarchy corresponded to one of the approaches used in the algorithm and referred to the steps presented in figure 1.
Overview of the steps for the developed algorithm. AEDs, antiepileptics drugs; TDM, therapeutic drug monitoring. Some of the strategies used in the algorithm required the use of a specific study design to avoid epidemiological biases. For example, in steps 5 to 7, it is necessary to look at events that occurred during the entire follow-up period. Therefore, to avoid conditioning on a future event,32 it was not possible to use a cohort design. This should not be seen as a limitation of the algorithm as currently, the evidence-based pyramid of evidence as explained by Murad al. emphasizes comparability of evidence between cohort and case-control studies.33
Data analysis
Comorbidities included in the Elixhauser comorbidity index,17 age, the first redeemed AED, sex and year of inclusion in the study population were identified at the index date. Continuous variables were shown as means and SD for variables for which the assumption of normality of the underlying distribution was not violated and as median and IQR otherwise. Categorical variables were presented as frequencies and proportions (%). A histogram of the year of inclusion of the study population was plotted. The median duration for the epilepsy-diagnosis hospitalisation was computed along with the IQR as the median duration from the hospital discharge to the first redeemed prescription of AEDs.
In step 1 of the algorithm, the frequency of individuals with epilepsy that redeemed AEDs and lacked information on the therapeutic indication was described and plotted as a pie chart. For those individuals with an available therapeutic indication, the quota of misclassification in using AEDs using step 1 of the algorithm was assessed.
In step 2 of the algorithm, the frequency of individuals with epilepsy that redeemed AEDs according to the guidelines was presented. For those individuals with an available therapeutic indication, the quota of misclassification in using AED using step 2 of the algorithm was assessed.
For these steps 3–6 of the algorithm, the frequency of individuals classifiable as using AEDs for epilepsy according to the algorithm was assessed. Additionally, based on the considerations provided in online supplemental appendix 2, in step 4, dosing regimens for each AEDs, along with their approved therapeutic indications were presented using a Forrest plot to evaluate the overlapping of dosing regimens for each AEDs. Finally, based on the considerations provided in online supplemental appendix 2, in step 5, the machine learning algorithm described by Breitenstein et al was used to classify individuals that redeemed multiple AEDs during the study period as using these AEDs as therapeutic substitution or add-ons.18 This has been performed using an externally validated dataset from Danish registers and according to the procedure described by Pazzagli et al.19 An overview of the study variables generated for each step is provided in online supplemental appendix 3.
Performance assessment
The proportion of the study population for which it was possible to predict the therapeutic indication of AEDs with a high degree of certainty using the developed algorithm was calculated. For a subset of the redeemed prescriptions from the DNPR, we had information regarding the therapeutic indications of AEDs. By applying the algorithm to these individuals, we could assess whether the predicted indication was epilepsy and then compare it with the recorded indication in the DNPR. The prediction performance of the developed algorithm was evaluated using confusion matrix performance metrics. The evaluation measures metrics included the number of true positives (TP) and false negatives (FN) from which the sensitivity was calculated. Sensitivity is a statistical measure, also known as the true positive rate.20 In the context of this study, sensitivity is calculated as the proportion of true positives (ie, individuals who redeemed AEDs for epilepsy) among all individuals who actually redeemed AEDs for epilepsy. Sensitivity was assessed by comparing the predicted therapeutic indication of epilepsy with the ‘true’ recorded therapeutic indication for AEDs. The 95% CI of the sensitivity was calculated using the following formula: 
.
Misclassification was defined by the false negative rate, which expressed the number of individuals that were incorrectly predicted using redeemed AEDs for therapeutic Indications other than epilepsy when in reality they were using them for epilepsy. The sensitivity, misclassification and the number of individuals in the study population classified by the algorithm were plotted for steps 3–7. In this study, we focused exclusively on predicting epilepsy as the therapeutic indication. Therefore, we were able to correctly calculate TP and FN, but not false positives (FP) and true negatives (TN). Therefore, it was not possible to assess specificity, positive predictive value (PPV) and negative predictive value (NPV), as they require FP and TN for their calculation (PPV=TP/TP+FP, NPV=TN/(TN+FN) and specificity=TN/(FP+TN)).
According to the secondary outcome, PS was calculated to assess exchangeability between individuals predicted to use AEDs for epilepsy and those predicted to use AEDs for other therapeutic indications using a logistic regression model with the following covariates: comorbidities described in the Elixhauser comorbidity index, age, sex and year of inclusion in the study cohort. The kernel density estimates of the PS at the index date were plotted in both analyses and the area of overlap between the density functions was computed. Exchangeability was considered to reach if the overlap of the PS was greater than 75%.21
Cox regression models were used to compute adjusted HRs and CI for the overall survival between individuals which were predicted using AEDs for epilepsy versus individuals with other therapeutic indications, adjusting for the PS. The proportionality of hazard assumptions was checked using Schofield residuals. Cumulative hazards for overall mortality were plotted. For the comparison of the cumulative hazard functions, the Grey test was used as a statistical test.
All statistical analyses and data management were performed using R statistics (V.4.0.0, R Development Core Team) and SAS statistical software (V.9.4 SAS Institute, Cary, NC).
Patient and public involvement
None.
Discussion
We found that the number of individuals aged ≥65 years that were incident hospitalised in the study period due to epilepsy was 1215. This result is in line with reports from the Danish Epilepsy Association that claim that on average 1000 older individuals aged ≥65 years are diagnosed with epilepsy each year.22 There was a higher proportion of men compared with women with epilepsy consistent with the gender-specific incident rate for incident epilepsy in Denmark.23 The sociodemographic characteristics of the study population were consistent with those observed in other studies in the scientific literature conducted in Scandinavian countries.24 The median duration of epilepsy was also in line with estimates reported in other epidemiological studies of epilepsy in older individuals that were conducted in Denmark.18 A high prevalence of comorbidities among the study population was not surprising considering the age of individuals with epilepsy. Regarding cardiovascular comorbidities, the prevalence observed was higher than that observed in the general population at similar age. This is related to the aetiology of epilepsy in older individuals which, in almost 50% of newly diagnosed patients is related to cardiovascular problems.25 The majority of the study population reported diagnoses of unknown/unspecified seizures, followed by focal seizures. Focal seizures were expected to be the most prevalent type of epilepsy considering that half of all epilepsy in older individuals are caused by stroke which scarring process lead to focal epilepsy.26 For the unknown and unspecified seizures, it can be that most of the individuals admitted to the hospital may already be epilepsy and not in a status epilepticus when reaching the hospital. Therefore, it may be challenging for healthcare operators to identify the type of seizures.
Valproate, lamotrigine and levetiracetam were the drugs mostly used within the study population which is not surprising considering that these AEDs are the first-line treatment for seizures in older individuals.27 Regarding valproate, this drug is mostly prescribed for older individuals with different seizure types and is available in various formulations which are suitable for older individuals. The low frequency of redeemed phenytoin, phenobarbital carbamazepine and topiramate reflected the increased interaction risks with medications mostly used in older individuals with comorbidities. The aforementioned AEDs are hepatic enzyme-inducers and, therefore, unadvisable as first-line monotherapy for older individuals, except for carbamazepine which is approved only for focal and unknown seizure.28
AEDs were mostly taken in agreement with the clinical guidelines for the treatment of seizure as a first-line monotherapy. The prescribing patterns for AEDs normally consider the individual’s comorbidities and comedications, which is particularly challenging in elderlies presenting extensive comorbidities and comedications.29 This explains the fraction of individuals using AEDs in contrast with clinical guidelines. This could also be related to those using AEDs for other therapeutic indications: as observed in step 1, 4.3% of the incident new users redeemed AEDs that are not approved as first-line monotherapies for the treatment of seizures. This hypothesis is also supported by the amount of misclassification reported in step 2, where 5.4% of individuals who were considered taking AEDs in agreement with clinical guidelines, redeemed AEDs at the index date for non-epileptic therapeutic indications.
The step 5 approach using switch and add-on therapies to predict the therapeutic indication was particularly promising with a sensitivity of 73.2%. However, only 254 incident new users had performed a therapeutic switch and/or add-on of AEDs during the first year of follow-up. This result is in contrast with previous studies showing a high frequency of AEDs users who changed AEDs treatments due to the high risk of therapeutic inefficacy and/or high adverse events during the first year of follow-up.30 The low incidence of switches and add-on therapies among the incident new users could also be due to the short follow-up period or the high overall mortality among older individuals during the first year of treatment with AEDs.31
TDM approach was also promising in predicting the use of AEDs for epilepsy (sensitivity of 78.0%). This approach offers an analytical advantage in Denmark, where this procedure is usually performed when AEDs are used for epilepsy.32 33 However, it may not be epilepsy to other settings/countries where recommendations on the use of TDM for AEDs differ. Similar to step 5 approach, only 663 patients underwent TDM, limiting its usefulness.
The approach using epileptic surgical procedures and examinations to predict AEDs’ therapeutic showed the lowest sensitivity (62.8%). This could be due to the temporal distance between the diagnosis, the redemption of a pharmacological prescription and the occurrence of surgery/procedures. In particular, individuals referred to surgical intervention are not always required to receive AEDs. Therefore, in those individuals, AEDs prescribed immediately after hospital discharge are likely used for therapeutic indications other than epilepsy.
The relative low sensitivity of the approach evaluating the medical history of the individuals was unexpected (67.5%). No reasonable explanation for such result was found, except for the duration of the covariate assessment window which may have precluded a more holistic evaluation of individuals’ medical history.
We did not observe a statistically significant increase in overall mortality among individuals misclassified as using AEDs for other therapeutic indications suggesting that the misclassification of the therapeutic indication should not have a significant impact on mortality. Additionally, the results of our analysis indicate that there is exchangeability between individuals predicted to use AEDs for epilepsy and those using AEDs for other therapeutic indications. This is supported by the observation that 86% of the density functions overlapped between the two groups of individuals. These results suggest that in this specific setting the misclassification introduced by not having information on the therapeutic indication for which AEDs are redeemed may not significantly impact mortality outcomes during the initial year of treatment.
In our opinion, the algorithm holds promise for application to other similar data sources to predict the therapeutic indication of AEDs, and the same approach can be expanded to other therapeutic indications to mitigate the impact of misclassification induced by the lack of information on therapeutic indication. However, further research is needed to fully assess the usability and value of the algorithm. Its applicability extends beyond the performance observed within the confines of the current dataset, necessitating exploration of its adaptability to other datasets. Evaluating how effectively the algorithm can be transferred and implemented across diverse datasets is crucial for understanding its versatility and potential impact in various healthcare contexts. This examination yields valuable insights into the algorithm’s robustness, scalability and generalisability, shedding light on both its benefits and limitations across different populations and healthcare systems. Therefore, continued investigation is essential to comprehensively assess the algorithm’s usability and value in advancing healthcare knowledge and improving patient outcomes.
Finally, the significance of our algorithm lies in its potential to mitigate confounding by indication and improve the accuracy of epidemiological research, not only in neurology but also in other therapeutic areas where similar challenges exist. By providing more accurate information on the therapeutic indications of medications, our algorithm can enhance the quality and reliability of real-world evidence, ultimately contributing to better-informed clinical decision-making and patient care across various therapeutic domains.
Limitations
The performance of the algorithm relied on the validity of the strategies to predict the therapeutic indication of AEDs along with assumptions for each strategy. Their violation may have contributed to the observed sensitivity. Another limitation of this study is related to data quality. The validity of epilepsy diagnoses in Denmark is not 100%. According to the newest study of validation for epilepsy diagnosis in the LPR, the PPV for epilepsy was shown to have a moderate to high value of 81% (95% CI 75% to 86%).34 Therefore, it may not be excluded that sensitivity may be affected by the misclassification of epilepsy diagnosis. Thus, the control over incorrectly coded diagnoses is still challenging. Another limitation of this study is that we were not able to use the predicted daily dosing regimen to predict the therapeutic indication of AEDs. This is an indirect consequence of overlapping posological regimens among AEDs’ recommended therapeutic approaches. We believe that adding this approach may have strongly improved the sensitivity of the algorithm. The overall performance of the algorithm was 65.3%; however, it might be improved by removing the unsuccessful approaches that we mentioned above. This may preclude the usefulness of the algorithm as the particularly promising steps apply to a few individuals. Further analyses are needed to improve the overall sensitivity of the algorithm and guarantee its use to the largest fraction of individuals included in the study population. One limitation of our study is that we did not consider the off-label use of AEDs. Off-label prescribing, which involves the use of medications for purposes other than those approved by regulatory agencies, can occur for AEDs in clinical practice. By not accounting for off-label use, our study may not capture the full spectrum of therapeutic indications for AEDs. Another limitation of our study is that the algorithm uses pain diagnosis codes (such as migraine, neuropathic pain, fibromyalgia) to assess clinical events preceding the redeemed prescriptions. However, it is important to acknowledge that case-finding algorithms for various pain conditions are frequently characterised by poor validity.35
Finally, in pharmacoepidemiology research using secondary data sources, it is essential to address situations where biases, particularly misclassification biases, may persist despite the use of algorithms. While algorithms offer a systematic approach to analyse complex datasets and predict therapeutic indications, their effectiveness may be hindered by inherent biases and limitations. It is crucial to evaluate the reliability of algorithmic predictions in light of these limitations and to critically assess their applicability to real-world clinical scenarios. Researchers must engage in transparent discussions about the strengths and weaknesses of using algorithms in pharmacoepidemiological studies. By acknowledging the potential biases and limitations associated with algorithmic approaches, researchers can make informed decisions about the interpretation and generalisability of their findings. Furthermore, this awareness can guide the development of more robust study designs and analytical methods that effectively leverage secondary data sources while minimising bias and maximising the validity of research outcomes.
Future work
It should be emphasised that the stepwise approach used in the algorithm could be adapted for other therapeutic areas. Future work is needed in this regard.
This post was originally published on https://bmjopen.bmj.com