STRENGTHS AND LIMITATIONS OF THIS STUDY
-
The Victorian Emergency Minimum Dataset (VEMD) collects data from 39 public hospitals in Victoria with 24-hour emergency services. It should be noted that only public hospitals with a designated, 24-hour emergency department (ED) report to the VEMD: other hospitals and rural urgent care centres are not included.
-
To examine potential injury data quality limitations in the VEMD, we compared VEMD injury presentations with Victorian Admitted Episodes Dataset (VAED) injury admissions, in terms of patient sociodemographical factors and injury profile.
-
Hospital-admitted injury patients who arrived through the ED were more numerous in the VAED (admissions) than in the VEMD (ED presentations that were subsequently admitted).
-
The patterns of VEMD under-representation of injury cases with specific characteristics that are described in this study, such as older patients and those with falls or transport injuries, should be taken into account in ED-based injury incidence reporting.
-
This study is limited to unlinked data analysis: linked data studies are recommended to explore patient pathways, in particular, non-injury ED presentations that resulted in injury admission.
Introduction
The emergency department (ED) is a first point of contact for serious injury that requires immediate treatment. It, therefore, provides a useful resource for tracking population-based injury statistics, such as injury type and cause, patient profile and injury incidence trends over time. In the USA, the National Electronic Injury Surveillance System (NEISS) was established in 1972 for monitoring consumer product-related injuries through ED patient attendance data; NEISS captures a nationally representative sample of hospitals.1
In Australia, several jurisdictions carry out injury surveillance through ED data but dedicated injury surveillance data fields related to injury cause, intent (unintentional, assault, self-harm), place of occurrence, activity when injured and injury narrative, are only collected in Victoria. Victoria has an injury surveillance unit (VISU), dedicated to analysing, interpreting and disseminating state-wide injury surveillance data, to be used to inform injury prevention policy and practice. With over 30 years experience, VISU serves well as a source of information and expertise, a platform for testing injury surveillance methodologies, and a resource for potentially testing and evaluating machine-learning applied to text narrative. It is, however, imperative to fully understand the strengths and limitations of the current injury data in the Victorian Emergency Minimum Dataset (VEMD).2 The VEMD captures ED presentation data from all 39 public hospitals in Victoria that have dedicated, 24-hour ED services. Not all emergency services are thus captured: regional areas of Victoria are under-represented.3 Small rural hospitals and Emergency Care Centres are not represented in the VEMD, and this impacts injury surveillance statistics, as shown through the Rural Acute Hospital Data Registry, which captures data from ten rural hospitals in South West Victoria.4 5 It is, therefore, recommended that additional data sources such as urgent care centre data and GP clinic data, are used for injury incidence reporting in outer regional and remote areas of Australia.
Another limitation of using 24-hour ED data for injury surveillance is inherent to emergency care: time spent with a patient is generally limited and full capture and recording of information related to the injury circumstances is not the first priority. In 2020/2021, VISU conducted a project to identify and address some of the barriers experienced in EDs in achieving high-quality injury data.6 As some of the data quality issues are intrinsic to the ED setting, a better understanding of how this impacts injury statistics is not only vital for Victorian injury surveillance but will also benefit ED-based injury surveillance in other Australian jurisdictions and beyond.
The aim of this study is to determine the data quality of the VEMD as data source for establishing hospital-treated injury rates in Victoria. Specifically, to explore to what extent injury coding quality limitations, which are inherent to the ED where time is limited, impact ED-based injury statistics. This information will help to understand which injury types and causes may be under-reported in the ED; potentially, these patterns may be generalisable beyond the Victorian context. The aim is addressed through a comparison of VEMD injury presentations with Victorian Admitted Episodes Dataset (VAED)7 injury admissions, in terms of patient sociodemographical factors and injury profile. To ensure a meaningful comparison, only VEMD cases that were subsequently admitted and only VAED cases that arrived through the ED were included (figure 1). Only hospitals that contribute to both the VEMD and VAED are included. The VAED contain comprehensive ICD-10-AM (International Statistical Classification of Diseases and Related Health Problems, 10th Revision, Australian Modification) coded inpatient data that are subject to regular review and auditing; therefore, mismatches between the VAED and VEMD resulting from the analysis are considered indicative of data quality issues in the VEMD.
(A) A schematic diagram illustrating the potential mismatching between ED injury cases and admissions injury cases (B) a schematic diagram illustrating the study focus injury ED cases for comparison between VEMD and VAED. ED, emergency department; VAED, Victorian Admitted Episodes Dataset; VEMD, Victorian Emergency Minimum Dataset.
Methods
Data sources
An analysis of hospital admissions and ED presentations data for a 5-year period from July 2014 to June 2019 was carried out retrospectively. The data were extracted from the VAED and the VEMD. The objective was to understand the data quality of the VEMD as a reliable injury data source, by aligning it with hospital admissions data in the VAED, which is more comprehensive. All 39 public hospitals in Victoria, which provide dedicated 24-hour ED services and contribute to both the VEMD and VAED, were selected. The study objective was addressed by identifying the differences between emergency injury presentations that were subsequently admitted to a ward (as reported in the VEMD) and hospital injury admissions that came through an ED (as reported in the VAED).
Case selection
Victorian Emergency Minimum Dataset
VEMD-recorded ED presentations from 1 July 2014 to 30 June 2019 were included if they contained an ICD-10-AM diagnosis code related to injury, poisoning and certain other consequences of external causes (S00–T98) in the primary diagnosis field. Selection was further limited to cases that were subsequently admitted to a ward/procedure room. This included the following: admission to ward not elsewhere described; to short stay observation unit; to medical assessment and planning unit; to intensive care unit—this campus; to coronary care unit—this campus; to mental health observation/assessment unit; to other mental health bed—this campus; to cardiac catheter laboratory; to other operating theatre/procedure room. Transfers to other hospitals were not included. We excluded all cases of medical injuries in the primary diagnosis field (T80–T88) as well as cases where cause codes were assigned to medical injuries and late effects.
Victorian Admitted Episodes Dataset
Hospital admissions with an ICD-10-AM principal diagnosis codes related to injury, poisoning and certain other consequences of external codes (S00–T98), were selected for the same period. Cases were limited to emergency admissions that came through the ED of the same campus. Records in principal diagnosis codes related to medical injuries (T80–T88) and/or the ICD-10-AM diagnosis code for first occurring external cause codes between Y40 and Y89 (Complications of medical and surgical care and sequelae of external causes of morbidity and mortality) were removed. To capture incident admissions, and therefore, prevent overcounting, admissions that were statistical separations (internal transfers in the same hospital) or inward transfers from another hospital were excluded. Finally, only VAED-recorded admissions in hospitals that also contributed to the VEMD (39 public hospitals in Victoria) were selected.
Study variables
Sociodemographic variables
Demographic factors such as age, sex, financial year and SEIFA (Socio-Economic Index for Areas), ARIA+ (The Accessibility/Remoteness Index of Australia Plus) were considered as covariates for the statistical model. ‘Age’ was categorised into nine groups: 0–14, 25–24, 25–34, 35–44, 45–54, 55–64, 65–74, 75–84 and 85+ years (10-year brackets except for children and older ages). The ‘year’ variable consisted of five financial years between 2014/2015 and 2018/2019. ‘SEIFA’ was classified into 10 deciles (the state-based index rankings and quantiles) using the Index of Relative Socio-economic Advantage and Disadvantage.8 For simplicity, SEIFA deciles were collapsed into five groups each containing two adjacent deciles. The ‘ARIA’ values were converted to remoteness categories as defined by the Australian Statistical Geography Standard.9 10
Injury variables
The difference between VAED and VEMD in injury coding is that the VAED is entirely ICD-10-AM based whereas the VEMD uses (a subset of) ICD-10-AM codes for the injury diagnosis but has (not ICD-10-AM coded) injury surveillance data items for cause, intent, activity and place. The VEMD also collects injury narrative text. The coded activity, place and cause items in the VEMD are loosely based on ICD but are generally less comprehensive and sometimes also structured differently. For example, in the VEMD, cause and intent are captured in separate variables whereas in the ICD-10-AM in the VAED, intent is intrinsic in the cause codes (ie, the intent can be determined from the cause codes used).
‘Injury type’ was constructed based on the primary diagnosis in the VEMD and the principal diagnosis in the VAED. The other injury information of interest that was common in both the VEMD and VAED datasets were cause, intent group and activity: this information was contained in the injury surveillance items in the VEMD and in the 40 ICD-10-AM diagnosis code fields in the VAED. Both cause and intent group in the VAED were obtained from ICD-10-AM external cause codes (these were in the range of V00–Y89) and were matched with the categories of cause and intent group in the VEMD.
Patient and public involvement
Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Statistical methods
The analytical approach was based on the assumption that the VAED is a comprehensive dataset of all admitted injury cases whereas in the VEMD, not all injury cases are recorded as such: some injury cases may erroneously have a non-injury diagnosis in the VEMD. Furthermore, injury cause may be unspecified in the VEMD (due to time constraints in the ED), resulting in undercounting of specified cause groups (such as falls, transport, poisoning). However, in some circumstances, the injury cases in the VEMD could outnumber the VAED cases, for example, when a patient presents to the ED with an injury but is subsequently admitted for an underlying or resulting morbidity such as diabetes dysregulation or a cardiac condition. The modelling would need to accommodate both scenarios. Therefore, to determine the difference between (admitted) injury presentations to EDs in Victoria captured in the VEMD and (emergency) injury hospital admissions captured in the VAED, negative binomial regression was used to model injury ED presentations with injury admissions as model offset. The offset was used to model ED presentations in the VEMD proportional to the VAED: effectively, this models the rate rather than count. Demographic and injury variables were included in the modelling to better understand VEMD and VAED differences. Residual plots and goodness-of-fit test were investigated to determine the appropriate model. This model was compared with a Poisson regression with and without adjustment for overdispersion. The results indicated that the negative binomial model met the assumptions, allowed for overdispersion and the statistical significance was more conservative in comparison to Poisson models. The interaction effects between age and gender were not included in the model as there was no statistically significant association between them, despite a visual association. Incident rate ratios (IRRs) were reported as indicators of effect sizes along with their 95% CI. Analyses were conducted in SAS V.9.4.
Results
There were 404 608 injury cases in the VEMD sample and 414 630 in the VAED: a ratio of 0.98. Over the 5 years, there was a slight rise in the annual number of cases for both VAED and VEMD datasets. However, the ratio of VEMD presentations to VAED admissions decreased non-significantly, dropping from 1.01 to 0.92 (p=0.66), as shown in table 1. The number of ED presentations varied by age group: patients less than 15 years old had the highest (56 016) and patients aged between 55 and 64 years had the lowest numbers (37 850), noting that the former spanned a wider age bracket. The number of admissions was highest in the 85+ age group (53 888) and lowest in the 35–54 years age group. The youngest age group had the highest and the oldest age group had the lowest VEMD/VAED ratio. Both VEMD and VAED injury cases numbers were higher in males than females; males also had higher VEMD/VAED ratios than females. In terms of socioeconomic index, both VEMD and VAED injury cases increased steeply with SEIFA deciles from low to high, with the exception of a slight reduction observed in decile 5 (compared with decile 4). Patterns in VEMD/VAED ratio per SEIFA decile were unremarkable. When considering the remoteness index, there was a significant decline in both VEMD and VAED injury cases as one moved from major cities to outer regional areas. However, the ratio of VEMD presentations to VAED admissions increased, rising from 0.97 to 1.04.
Characteristics of injury presentations that were subsequently admitted to a ward (as reported in the VEMD) and hospital injury admissions that came through an ED (as reported in the VAED) limited to VEMD contributing hospitals
Figure 1A provides a schematic overview of the potential mismatching between ED injury cases and admissions injury cases. Some cases may not be coded as primary diagnosis is an injury in the VEMD, even though this is the principal diagnosis for subsequent admission (VAED). Conversely, not all injury-related ED presentations are admitted primarily in relation to injury: the principal diagnosis for admission could be non-injury. Figure 1B provides a schematic overview illustrating the study focus injury ED cases for comparison between VEMD and VAED.
Figure 2 shows the difference between the number of presentations and admissions by age and sex over 5 years. This figure indicates that older females and, to a lesser extent, older males were over-represented in the VAED injury cases whereas younger males, and to a lesser extent, younger females were over-represented in the VEMD injury cases.
Difference between emergency injury presentations that were subsequently admitted to a ward (as reported in the VEMD) and hospital injury admissions that came through an ED (as reported in the VAED) over 5 years (2014/2015–2018/2019) by age-group and sex. ED, emergency department; VAED, Victorian Admitted Episodes Dataset; VEMD, Victorian Emergency Minimum Dataset.
Table 2 presents the number of VEMD admitted presentations and VAED emergency admissions from July 2014 to June 2019 by injury types, cause group, intent group and activity for principal diagnosis. Although the total number of injury cases is similar in the two datasets, there are differences in the distribution across injury types. In both data sources, the most common injury type was head injury (18.9% VEMD cases; 22.2% VAED cases). Injuries to wrist and hand were second-most common in the VEMD (11.0%) whereas knee and lower leg injuries were second place in the VAED (9.4%). The greatest VEMD VAED difference was observed in injuries involving multiple body regions (T00–T07) which constituted 8.2% of VEMD but <1% of VAED injury cases. The next largest difference was observed in Other and unspecified effects of external causes (T66–T78) with 5.1% and 1.7% of VEMD and VAED cases, respectively. Certain early complications of trauma (T79) constituted 0.3% of VAED injury cases but did not feature in VEMD cases. The most notable difference in cause group was in other or undetermined intent, which made up 16.7% of VEMD injury cases but only 1.4% of VAED injury cases. The greatest VEMD and VAED differences in specified cause groups were observed in falls (37.4% and 46%, respectively) and transport (9.1% and 14.9%, respectively). With the exception of hit/struck/crush, all specified cause groups listed in table 2 had greater representation in the VAED than in the VEMD, suggesting that in some cases, the cause may only be specified and recorded as such after admission. The intent group rates presented in table 2 indicate that all specified Intent groups were more common in VAED emergency admissions than VEMD admitted presentations whereas other and undetermined intent was much more common in the VEMD (16.7%) than VAED (1.4%). Presenting VEMD and VAED cases by Activity of injured shows a different pattern to the other injury variables: Unspecified activity was much more common in the VAED (60.4%) than in the VEMD (36.4%). Activity when injured was more commonly coded as leisure in the VEMD (34.5%) than in the VAED (4.5%).
The number of VEMD presentations that admitted to ward at the same campus and VAED emergency admissions through ED at the same campus over 5 years (2014/2015–2018/2019) by injury types, cause group, intent group and activity based on the principal diagnosis only
To better understand differences between the VAED and VEMD in terms of patient demographics, VEMD presentations were modelled proportional to VAED admissions, by age, sex, year, SEIFA and ARIA as shown in table 3. VAED admissions, which were considered to be comprehensive and complete in terms of capture and coding, were therefore modelled as an offset exposure to examine the effect of the rate of VEMD presentations per unit exposure. The coefficient for the offset variable was set to 1. Effectively this modelling approach provides a comparison between the ED presentation patient profile and the hospital admission patient profile, for emergency injury admissions that arrive through the ED. The output of the model indicates that the number of ED presentations in financial year 2014/2015 was 1.08 (95% CI 1.07 to 1.09; p≤0.001) times higher than 2018/2019 (reference group), after accounting for differences in patient demographics and holding all other variables constant. The IRR of males was 1.01 (95% CI 1.00 to 1.02; p=0.003), indicating a slight over-representation of males in the VEMD relative to the VAED. In terms of age, the results showed a relative over-representation of younger people in the VEMD relative to the VAED and a concomitant under-representation of older people, that is, high-to-low IRR gradient was observed with increasing age. The socioeconomic index pattern was more complex, with the lowest IRR observed in the middle SEIFA group (deciles 5–6) and the highest IRR observed in the lowest (deciles 1–2) as well as in the highest (deciles 9–10) SEIFA groups. This pattern reflects a relative under-representation of patients with mid-range socioeconomic index in the ED data. The remoteness index showed a pronounced over-representation of outer regional areas (95% CI 1.06 to 1.09; p<0.0001) and a comparatively over-representation of inner regional (95% CI 1.01 to 1.03; p<0.0001) in the VEMD in comparison to the VAED.
Incident rate ratios (IRRs) for demographic characteristics using negative binomial regression
Discussion
This study was designed to compare patterns of injury coding and completeness in ED data with admissions data, selecting a matched set of hospitals and selecting only (subsequently) admitted cases for a fair comparison. Although situated in the Australian state of Victoria, the findings of this study may be reflective of ED coding practices and patterns that could be generalisable to other jurisdictions. We found that the overall number of cases in the two data sources was similar but in the most recent years of data, admissions outnumbered ED presentations, suggesting recent under-reporting of injury cases in the ED. An over-representation of children in the ED data and under-representation of older people also points to differences in injury diagnosis coding practices in the two settings. Other key findings related to injury cause, which was more likely to be undetermined in the ED data and more likely to be coded as falls or transport injury in the admissions data; this has implications for injury surveillance. Self-harm and assault injury were more commonly recorded in the admissions data.
Outnumbering of injury cases in the admissions data compared with the ED data can only be explained by injury codes being assigned a non-injury diagnosis code in the ED, as the datasets were matched by hospitals and included only cases that journeyed through the ED to subsequent admission. Any explanation of how and why this issue occurred is speculative, but the comparisons of characteristics prevented in table 1 provide some insight. First, the outnumbering of injury cases in the admissions versus ED presentations occurred in the more recent years of data and there appeared to be an increase in this ratio over time, suggesting a gradual onset. Second, the differences in age profile of ED patients vesus admitted patients suggest that injuries are relatively over-reported in children and under-reported in older patients in the ED, particularly those aged 85 years and above. This could be related to limitations in the permitted number of diagnosis codes in the ED (a maximum of three, compared with 40 diagnosis code fields in the admissions data). For example, a patient with a fall due to neurogenic syncope, cardiac syncope or hypoglycaemia could be recorded in the ED data with a cardiac, neurological or endocrine diagnosis. However, if the fall resulted in a mild head injury, the latter may have been the principal reason for admission to ward for overnight observation. This scenario is an example of how a patient can be missing from the ED injury case selection but appear in the injury admissions case selection (corresponding with B and D, respectively, in figure 1A). In children, the reverse could be the case: in children presenting to the ED with relatively minor injuries, a principal reason for admission could be to rule out underlying pathology such as epilepsy. This scenario is an example of how a case can appear in the ED injury case selection but not in the injury admissions case selection (corresponding with A and C, respectively, in figure 1A).
An alternative explanation for the undercapture of injury cases in the ED relative to admissions, which must be considered, is related to the impact of the injury surveillance system on ED staff: an injury diagnosis triggers a suite of injury surveillance data items in the ED patient data entry system.6 Potentially, busy staff who are not motivated to participate in injury surveillance could select a non-injury diagnosis, where possible, to avoid additional administrative load. To determine if this happens and if so, on what scale, these various scenarios can be tested and further explored using a linked dataset of ED presentations and admissions, allowing for a case to be tracked from the former to the latter to explore any changes in diagnosis and to quantify the pathways between (A) and (B) to (C) and (D) as indicated in figure 1A. A linked data study will also help to shed light on the relative over-representation of injury cases in the VEMD in regional areas: possibly injury surveillance data collected in the ED is of higher quality in regional versus metropolitan areas.
The busy setting in the ED might not allow for a thorough exploration of the injury intent, and it is, therefore, unsurprising that intentional injury cases in the admissions outnumbered those in the ED presentations. An even more skewed ratio was observed in assault, maltreatment and neglect cases, which were almost double in the admissions data compared with the ED. This is an important finding as it demonstrates that ED statistics provide a substantial underestimate of the actual case numbers for hospital-treated injury resulting from assault (a ratio of 0.56) and self-harm (ratio of 0.78). ED data have been used to estimate rates of intentional injury in other studies11 12 and for future studies, these under-representation estimates can be acknowledged and where feasible, under-representation should be measured and statistically adjusted for.
Another noteworthy difference between ED presentations and hospital admitted injury cases grouped by cause relates to the number of falls injuries and transport injuries: both are much higher in the admissions that in the ED presentations, both in absolute numbers and proportionally. The most plausible explanation for this discrepancy relates to the high number of ‘other or undetermined’ cases: 17% of ED presentations vs 6% of admissions. Some of these cases may have been specified as caused by transport or a fall, after admission: this could occur, for example, if the patient was unconscious or confused when they arrived in the ED and the cause of the injury was only established or confirmed after admission. As noted for intentional self-harm, these differences in case numbers point to under-reporting in the ED: this needs to be taken into account when presenting ED-based transport or falls injury rates.13–17
This study has limitations that need to be acknowledged. First, the study was based on the 39 hospitals with 24-hour ED, reporting to the VEMD: other emergency care settings that do not report to the VEMD are out of scope: the limitations of this have been reported elsewhere.3 Second, hospital admissions data are used as the gold standard for injury coding. Although the diagnoses in the VAED are audited and of excellent data quality, there are scenarios where a patient may present to the ED as a non-injury patient but subsequently be admitted to ward as an injury patient and vice versa, as outlined earlier in the discussion. This limitation is inherent to the study design. A linked data study is recommended to fully test and explore patient pathways and further clarify and quantify the potential impact of data quality issues in the ED on injury statistics and surveillance. Furthermore, the VAED, while a good source of diagnosis and cause codes, is limited in its capture of ICD-10-AM activity when injured codes: over 60% of admissions have ‘unspecified’ activity coding. This means that the VAED is unsuited as a gold standard for data quality in this injury item and further studies using surveys and other data collection methods are required to test and validate VEMD activity when injured coding. Finally, it should be noted that case selection was based on VEMD departure status (admissions to ward) and VAED admission type (arrival through the ED). Both these data elements in their respective datasets can contain a small level of coding error, which may have contributed to the mismatch in VEMD-VAED injury case numbers presented in this study.
In conclusion, this study of injury data quality limitations in the ED data shows that hospital-admitted injury patients who arrived through the ED are more numerous in the VAED (admissions) than in the VEMD (ED presentations) even though the number is expected to be equal, based on hospital and case selection. This could be attributed to patients being admitted for an injury even though the principal condition they presented within the ED was not an injury; particularly in older patients. Falls injuries, transport injuries and intentional injuries (self-harm and assault) were under-represented in the ED: these patterns of under-representation should be taken into account in ED-based injury incidence reporting. Studies linking ED presentations and hospital admissions are recommended to further explore, qualify and quantify patient pathways and diagnostic practices in the ED.
Data availability statement
Data may be obtained from a third party and are not publicly available. Aggregated injury data are accessible through the Victorian Injury Surveillance Unit (VISU). However, VISU cannot provide unit record data due to its data use conditions. For unit record hospital data, interested parties may request access through the Victorian Agency for Health Information, subject to ethics approval and review and approval of the data application by the data custodian.
Ethics statements
Patient consent for publication
Ethics approval
Analysis and reporting of hospital admissions data and emergency department data by the Victorian Injury Surveillance Unit was reviewed and approved by the Monash University Human Research Ethics Committee (the project id is 21427).
Acknowledgments
The authors would like to acknowledge the Victorian Department of Health as the source of VEMD and VAED data for this study. They would also like to express their gratitude to Dr Dasamal Tharanga Fernando for their valuable insights in shaping the content of this work
This post was originally published on https://bmjopen.bmj.com