Introduction
Community-acquired pneumonia (CAP) is an increasing cause of hospitalisation and mortality, especially among elderly patients.1–5 Early diagnosis and accurate treatment at the emergency department are essential to avoid serious complications such as bacteraemia, sepsis, organ failure and death6 and to fight antimicrobial resistance.7
The diagnosis of CAP generally requires a new infiltrate on a chest X-ray with a clinically compatible syndrome (eg, fever, dyspnoea, cough and sputum production).8 These symptoms are not sufficient to diagnose or exclude CAP, as they overlap with other diseases8 and can be subtle in patients with advanced age and/or impaired immune systems.9 10 The chest X-ray is an imprecise diagnostic tool for CAP, risking under/overdiagnosis11 12 and might not be the optimal reference standard for CAP. This variability in clinical signs and symptoms combined with non-specific diagnostic tools,12 biomarkers13 14 and time-consuming microbiological tests9 challenges physicians in differentiating CAP from other infections.10 15
The CAP population today has also changed with increased ageing,16 multimorbidities17 and immunomodulatory treatments. Therefore, our knowledge of CAP symptoms and signs needs to be adapted to the actual population.
Previously, prediction models for the diagnosis of CAP were developed on prognostic factors, including severity assessment,18 19 observations in a primary care setting only20–22 or a reference diagnosis based solely on the registered discharge diagnosis in the medical record or positive chest X-ray findings.22 23 A valid outcome diagnosis was essential. However, in pragmatic studies, an expert panel using available information has been deemed a better reference standard.11
Therefore, there is a need to describe the clinical characteristics of the current population of patients admitted with suspected CAP and develop a diagnostic model that includes physical examination, blood tests, vital signs, patient medical history and healthcare expertise. Given the current diagnostic tool inaccuracies, an expert-panel-based diagnostic model was expected to surpass the emergency department (ED) physicians’ initial accuracy.
Hypothesis and objectives
We hypothesised that a diagnostic prediction model based on well-defined clinical characteristics could assist an ED physician to make an earlier, more precise CAP diagnosis. Therefore, the aim was to identify the clinical characteristics of adults admitted with CAP and evaluate the performance of these characteristics in a prediction model.
The objectives were as follows:
-
To compare clinical characteristics of patients with a CAP diagnosis from (1) all patients admitted with suspected infection and (2) patients suspected of CAP.
-
To develop and evaluate a diagnostic model to identify patients with CAP among ED patients suspected of infection and to compare the performance of the model to the initial assessment of the ED physician.
Methods
The study was reported following ‘The Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis’ (TRIPOD) statement.24 The protocol was registered by the Danish Data Protection Agency (No 20/60508) and in ClinicalTrials.gov (NCT04681963).
Study design, source of data and setting
This study had an analytical, cross-sectional, multicentre design. The data were collected prospectively and originated from the INfectious DisEases in Emergency Departments (INDEED) study. The published study protocol provides further detailed information.25 Four Danish medical EDs participated, with a catchment area of around 775 000 inhabitants, between 1 March 2021 and 28 February 2022.
In Denmark, patients can be directed to various specialties within the ED, for example, medical, gastrointestinal surgery, cardiology, orthopaedics, gynaecology, psychiatry and intensive care.26 Suspected infection cases are usually assigned to a medical ED.
Participants
Adult patients (≥18 years) admitted to a medical ED were eligible to participate. Patients were included if the ED physician suspected infection and the patient could provide verbal and written consent. The exclusion criteria were as follows: (1) need for urgent, life-saving treatment, (2) transferal to intensive care, (3) admission within the last fortnight, (4) verified SARS-CoV-2 infection at the time of admission or within 14 days before admission, (5) severe immunodeficiencies (HIV positive, with a cluster of differentiation 4 cell count <200) or treatment with immunosuppressive medicine (Anatomical Therapeutic Chemical classification L04A), corticosteroids (>20 mg/day prednisone or equivalent for >14 days within the last 30 days) or chemotherapy within 30 days.
Recruitment and data collection
Six project assistants with healthcare backgrounds (three physicians, one physiotherapist and two final-year medical students) were responsible for inclusion and data collection from Mondays to Fridays, 08:00 to 20:00. The population was a convenient sample of eligible patients consecutively identified from the patient management system by a project assistant. Immediately following the initial clinical assessment, the project assistant asked the ED physician whether an infection was suspected and the most likely infection focus (CAP, urinary tract infection or unknown origin). Generally, the clinical assessment took place within the first 30 min of admission before blood tests or imaging were ordered, and therefore, the ED physician often relied only on information from the patient’s signs, symptoms and vital parameters. The study assistant collected verbal and written consent from eligible patients. All data collected were registered in the electronic study database REDCap (Research Electronic Data Capture).27
Reference diagnosis
The reference diagnosis was the diagnosis of CAP assessed by an expert panel. The expert panel consisted of eight clinical experts at consultant level in the fields of infectious diseases and emergency medicine working in pairs. They conducted a patient file audit and determined the final diagnosis based on all clinical information registered within the first week of ED admission. The information included routine laboratory tests of blood, urine and sputum. In addition, PCR tests of sputum, urine flow cytometry, chest X-ray and chest CT were available for some patients. The experts had access to all images, including the radiologist’s interpretation and documentation. The experts were blinded to each other and independently registered their assessments in a standardised electronic template27 in the study database. In case of disagreement, the two specialists re-evaluated the medical record and collectively reached a consensus.
Predictors
All clinical characteristics were collected on arrival at the ED. Symptoms, demographic data and lifestyle factors were registered during a standardised bedside interview with the patient. In addition, information about vital parameters, comorbidities, medical treatment and blood tests was collected from the patient’s medical record. The project assistants collecting data were blinded to the final diagnosis.
70 candidate predictors were selected from the literature and discussed with the specialists and project group.20 28–37 The prespecified potential predictors with measurement units, groups, cut-offs and considerations/assumptions of inclusion were selected (see online supplemental table S1).
Supplemental material
-
Demographic information, lifestyle factors and comorbidities: age, sex, civil status, employment, nursing home residence, smoking, alcohol consumption, Body Mass Index (BMI), level of physical activity, activities of daily living score, dementia, respiratory, neurological, cardiovascular, endocrinological, nephrological and gastrointestinal comorbidities were collected.
-
Patient symptoms 2 weeks before admission: malaise, fatigue, headache, dizziness, altered mental status, for example, confusion, dyspnoea, malnutrition, cough, secretions from the respiratory tract, sore throat, common cold, fever feeling, chest pain, peripheral oedema, nausea, vomiting, decreased appetite, abdominal pain, diarrhoea and pain in muscles and joints including back pain were collected.
-
Severity assessment, clinical parameters with cut-offs based on National Early Warning Score38 used at the arrival of the ED and the use of medications: CURB-65 ≥3 (confusion, uraemia, respiratory rate, blood pressure, age >65 years), triage,39 Glasgow Coma Scale, oxygen saturation <96%, heart rate <51 or >90 beats/min, blood pressure (systolic <111 or >219, diastolic ≤60 mm Hg), respiratory rate >20 breaths/min, temperature >38°C, abnormal chest auscultation, abdominal tenderness, polypharmacy (≥5 medications), use of analgesics and vaccination status (SARS-CoV-2, pneumococcus, influenza) were recorded.
-
Blood tests with cut-offs routinely applied at our institutions: haematocrit (%), haemoglobin (mmol/L), leucocytes (109/L), platelets (109/L), neutrophils (109/L), lymphocytes (109/L), albumin g/L, creatinine (µmol/L), blood urea nitrogen (mmol/L), sodium (mmol/L), prothrombin, bilirubin (µmol), glucose (mmol/L) and C reactive protein (CRP) (mg/L) were recorded.
Statistical methods
The study sample size was estimated using data from the University Hospital of Southern Denmark. We estimated a need for at least 700 patients admitted with suspected infection. Of those, 400 patients should have suspected CAP and 200 patients should have verified CAP to complete a reasonable multivariable regression analysis. Descriptive statistics for baseline characteristics of the patients were conducted for the 70 potential predictors based on the data from the INDEED study.25 Data were presented as means and SDs, or medians and IQRs for continuous variables, and numbers (n) and percentages (%) for categorical and binary variables. Extensive univariate logistic regression analyses were performed to examine the unadjusted association between each candidate predictor and the outcome CAP. Results of univariate analyses were reported with OR, 95% CIs and statistical significance levels were two-sided reported with a p value of <0.05 to present a descriptive overview of the individual’s associations in the population. Complete case analyses were performed, and the predictors were dichotomised or categorised and presented with percentages (%) for inclusion in the final model. The least absolute shrinkage and selection operator (LASSO) multivariable regression was performed with a random split-sample to develop and validate the model, using 20% of the data for internal cross-validation. The model calibration was assessed using a likelihood ratio test, and recalibration was done based on the calibration belt and the optimal predicted proportion. In the model, age (≥75 years old) was considered an effect modifier based on several studies showing differences in symptoms and signs of a CAP diagnosis in older adults.33 40–42 An exploratory approach was conducted for the clinical characteristics to achieve the model with the best predictive performance, testing performances with continuous, dichotomous or categorical variables. In addition, the receiver-operator characteristic (ROC) curve was created to estimate the model’s accuracy, and the area under the curve (AUC) visualised any discrimination between true positives and negatives. The sensitivity, specificity and positive and negative predictive values with 95% CI were calculated using the best threshold criteria of the predicted probability of the ROC curve. The same threshold was implemented in developing a CAP score, including the predictor variables. A CAP score >0 represents the presence of CAP, and <0 indicates the absence of CAP. Sensitivity, specificity and positive and negative predictive values with 95% CI were calculated from the initial diagnosis made by the ED physician. Analyses were performed using STATA V.17.0 (Texas, USA).
Patient and public involvement
Patients and/or the public were not directly involved in this study.
Results
Participants
We recruited 954 patients admitted to the ED with suspected infection, representing 43% of the population screened for eligibility. Of those, the attending physician suspected that 402 (42%) had a CAP diagnosis. The expert panel verified a CAP diagnosis in 265 (28%) of the recruited patients (figure 1). The evaluation of 332 chest CT scans showed that 188 (57%) patients had verified pneumonia, and from those, 148 (76%) had CAP assessed by the expert panel and confirmed by a chest CT scan. Most patients (65%) with CAP were discharged to an internal medicine ward, while 29% of the patients diagnosed with CAP by the expert panel were discharged directly home. There were 2.5%, 2.5% and 1.0% of the population with CAP that were discharged to the intensive care unit (ICU), surgical and other wards, respectively.
Trial population, green boxes showing the numbers of patients with CAP. CAP, community-acquired pneumonia.
Characteristics of patients with suspected infections
We compared the clinical characteristics of patients with verified CAP to patients with suspected infection (954). Median age for patients with verified CAP was 75 years (IQR 63.5; 82.0), and over half admitted with suspected infection were males (53.8%). Univariate analysis revealed that patients with verified CAP were more often previous smokers (OR 1.83 (CI 1.30 to 2.57), p<0.001) with smoking history compared with suspected infection cases. Strongly independent predictors for CAP were symptoms such as dyspnoea, cough, expectoration, chest pain and cold symptoms (all p<0.001). Compared with patients without CAP, the risk of having CAP increased fivefold if the patient had chest auscultation abnormalities (OR 5.67 (CI 4.15 to 7.75), p<0.001) and decreased by half in case of abdominal tenderness by palpation (OR 0.52 (CI 0.35 to 0.78), p=0.002]. Patients with CAP often had comorbidities related to other pulmonary diseases (p<0.001) and often had previous CAP infections (p<0.001). These patients were more acutely ill when assessed by triage (p<0.001), with fever >38°C (p=0.036), higher respiratory rate (median 20.0 (IQR 18.0; 24.0), p<0.001), higher heart rate (mean 93.2 (SD 18.9), p<0.001) and lower oxygen saturation (median 95.0 (IQR 93.0; 97.0), p<0.001). Patients with verified CAP had a median CRP of 125.0 (IQR 57.0; 203.5) versus 82.0 (IQR 19.0; 172.0) (p<0.001) compared with the rest of the population and higher levels of neutrophils (p<0.001) and leucocytes (p<0.001). Furthermore, lymphocytes yielded a p value of 0.018. Patients with verified CAP were more often vaccinated against SARS-CoV-2 (p=0.033) and influenza (p=0.025), but no differences were found regarding pneumococcal vaccination. Table 1 presents the characteristics of the population with statistically significant results of the unadjusted association between each predictor for patients with verified and not verified CAP. See online supplemental table S2 for the 70 exploratory results from continuous, dichotomous and categorical variables tested in the diagnostic prediction model.
Characteristics of the population with suspected infection (n=954)
Characteristics of patients suspected of CAP
Using the 70 candidate predictors, we compared the clinical characteristics of patients with verified CAP to patients with suspected but not verified CAP (402).
Statistically significant differences are shown in table 2. Of the 402 patients with suspected CAP, half of the patients, 229 (57%) had verified CAP. Patients with suspected CAP had a median age of 74.0 (IQR 62.0; 82.0), and half were male (52.7%). Patients with verified CAP reported more respiratory symptoms, such as cough (p=0.009) and expectoration (p=0.037), and more gastrointestinal symptoms, such as nausea (p=0.033) and loss of appetite (p=0.030), compared with those without CAP. Fewer patients with verified CAP had a CURB-65 ≥3 (p=0.047), and more patients had oxygen saturation <96% (p<0.001), a heart rate of <51 or >100 beats/min (p=0.045) and fever>38°C (p=0.011). Elevated infection biomarkers (leucocytes, neutrophils, CRP, all p<0.001) and plasma natrium (p<0.001) were highly associated with CAP. Fewer patients with CAP had plasma bilirubin values of <5 or >25 mmol/L (p=0.045) (table 2).
Characteristics of the population with suspected CAP (n=402) by the physician at admission
Model development and performance
We developed a prediction model for diagnosing pneumonia in patients admitted with suspected infection (n=954) and compared it with the clinician’s presumptive diagnosis. Online supplemental table S3 presents the characteristics of the population randomised in the training and validation sets.
The predictors associated with CAP in our final model are presented in table 3.
The complete diagnostic model, including the intercept
The model performance yielded an AUC of 0.85 (CI 0.77 to 0.92), and the calibration of the model yielded p=0.227 after recalibration, demonstrating a good prediction of the proportion of patients with CAP in the test sample (online supplemental figures S1 and S2).
Based on a lambda result of λ=0.0402856 and a probability threshold of 0.35, the LASSO calculation with characteristics predictive of CAP and the calculation of the final model with a cut-off value greater than 0 indicating the diagnosis CAP are presented in supplemental material (online supplemental formulas S1 and S2).
At the optimal cut-off of 0.35, the prediction model yielded an 86.1% sensitivity and 64.1% specificity. Based on the trial population (figure 1), the sensitivity of the prediction model was comparable to the initial diagnosis made by the ED physicians. However, the specificity and positive predictive value were significantly lower (table 4).
Performance of the predictive model compared with the initial diagnosis made by the ED physicians.
Model specification
The final model did not include the following possible predictors: lymphocytes, SARS-CoV-2 and BMI. The reasons were a high percentage of missings (lymphocytes 66.3%), clinical relevance and statistical performance (BMI and SARS-CoV-2). These considerations are described in detail in online supplemental material.
Discussion
More than every fourth patient with suspected infection was diagnosed with CAP (28%). The ED physicians suspected CAP in almost half (42%) of patients admitted with suspected infection. Patients with suspected CAP included 57% with a final expert diagnosis of CAP and 43% without CAP. We have identified 27 clinical characteristics for patients diagnosed with CAP among those admitted suspected of infection. Patients with CAP were characterised more often with a history of smoking, previous CAP, respiratory symptoms, abnormal lung auscultation, worse triage and abnormal levels of infection biomarkers. Fewer clinic characteristics (13) were identified for patients diagnosed with CAP among patients suspected of CAP by the ED physician and included typical respiratory symptoms but also gastrointestinal symptoms, abnormal vital signs, increased blood markers and lower CURB-65 scores. The final diagnostic prediction model yielded 13 diagnostic predictors for CAP recognised by the literature. The model performance was similar to the diagnosis made by the ED physicians regarding sensitivity and negative predictive value but not as good in determining the specificity and positive predictive values.
Our prediction model had a good performance (AUC 0.85) and calibration (p=0.227), and with the best cut-off at 35%, the sensitivity reached 86.1% and specificity 64.1%. Therefore, the model could be tested externally at other sites, especially where clinicians are not always available due to the lack of resources, and contribute to the initial management of CAP, guiding further clinical investigation. In this study, ED physicians relied on the patient’s history and the results from a simple clinical examination to diagnose CAP with a comparable negative predictive value (93% vs 94%) and a better positive predictive value (57% vs 42%). Even though our model is not entirely comparable to the initial diagnosis made by the ED physicians due to the difference in the prevalence of CAP, our results are similar to a recent systematic review.43 Other studies reported that ED physicians’ accuracy in diagnosing CAP ranged from 76% to 96%,44 and artificial intelligence predicted the presence of pneumonia with a sensitivity of 94% and specificity of 50%.45 These results show that there is room for improvement in diagnosing CAP. It could be achieved by including additional predictors such as biomarkers, for example, procalcitonin, YKL-40 and surfactant protein-D,46 47 molecular detection of respiratory pathogens,48 and/or improved imaging modalities.12 14
This prospective study highlights the challenges in identifying patients with CAP based on patient history, vital signs and symptoms on admission.20 22 46 An initial CAP diagnosis may often differ from the discharge diagnosis.10 49 A plausible cause for uncertainty in diagnosing CAP was the heterogenic presentation of symptoms overlapping with other diseases. We found that patients with verified CAP often had gastrointestinal symptoms, whereas patients not verified with CAP sometimes presented with typical respiratory symptoms and had more severe conditions measured by CURB-65. Typical respiratory symptoms could explain some CAP misclassification. Misclassification of CAP may lead to unnecessary or ineffective antibiotic treatment, increased healthcare costs, delayed diagnosis, increased mortality and increased risk of bacterial resistance.44 50
The predictors of CAP identified in this study are strongly discussed in the literature.9 20 36 37 42 46 49 Most prediction models for ED patients with CAP aim to predict prognostic outcomes such as disease severity and mortality.51 Prior studies have either included few diagnostic predictors or very selected patients.20 22 52 The main reason for including several potential predictors and having age as a cross-factor in the development of our model was the expectation of finding predictors not represented in the literature and predictors specific for older patients (≥75 years). This is considered very relevant as the population worldwide ages.4 16 An age of ≥75 interacted with the symptoms of cough, blood glucose levels and peripheral oedema. Peripheral oedema was associated with an absence of CAP, and symptoms may be explained by other infections, such as erysipelas or heart failure. In addition, hyperglycaemia has been recognised as a predictor associated with poorer patient outcomes for elderly patients with CAP, regardless of their history of diabetes.53 54
Even though the literature highlights malnutrition as a strong prognostic predictor for CAP,33 35 55 we excluded BMI from our final model. Measuring weight and height is not a priority in acute settings where vital parameters, symptoms and point-of-care biomarkers are the primary observations in the diagnostic process. Another concern was that BMI was missing in 26.3% of the population, and bias may arise due to systematic differences between subjects with complete datasets and subjects with missing data. Patients with missing BMI data may be more frail, incapable or difficult to transfer. A model including BMI could be a better choice in a primary care setting, where patients are not necessarily as acutely ill and may be able to weigh themselves.
A major strength of this study is the completeness of data from medical charts and patient interviews combined with CAP diagnoses assigned by a panel of experts. The experts had a range of information from the patient’s medical records, including chest X-ray, chest CT for patients suspected of CAP and microbiology results. In addition to identifying possible predictors, we included many relevant and easily accessible clinical parameters. Finally, we excluded patients infected with SARS-CoV-2 from the study to increase the potential generalisability for patients with CAP after the pandemic.
This study has limitations. Multiple testing and mass significance are potentially a problem in this study. Methods, such as Bonferroni-Holm correction, could have been applied to counteract this problem.56 However, the univariate analyses were conducted for exploratory and descriptive purposes only. Therefore, these results should be interpreted cautiously, and the findings should be used as hypothesis-generating rather than conclusive. Another concern is that even though the reference standard of CAP was the same for the model performance and the initial diagnosis of the ED physicians, the expert panel may have had better opportunities to diagnose CAP in patients with suspected CAP due to the availability of results from imaging and microbiological tests, and better register of patient’s symptoms. This could lead to differential verification bias overestimating the ED physician’s accuracy in diagnosing CAP.57 This assumption was supported by the higher specificity of CAP diagnoses from ED physicians.
Another limitation is the selected population of patients allocated to the internal medicine specialty that may have masked atypical predictors from patients assigned to other specialities. Furthermore, some patients with atypical clinical presentation may have an infection that the ED physician had not suspected on admission and, therefore, was not included in our study. Patients with severe conditions or acute cognitive impairment who could not consent were excluded. Furthermore, the inclusion of patients took place during work hours and weekdays, which may have reduced the number of severe cases as admission during out-of-hours and weekends are associated with increased mortality and ICU admissions.58 Therefore, our results can only be generalised to patients suspected of CAP and admitted on weekdays during the daytime.
A broader patient inclusion may contribute to a model that identifies other predictors to diagnose CAP as the clinical presentation may differ from those admitted with suspected CAP and capable of consent. Another limitation was the pragmatic choice of cut-offs for blood tests routinely used in our institutions, which reflected our clinical practice. However, it does raise questions about the applicability in other settings that apply different cut-offs.
This population cohort could be applicable as a test validation cohort for future models as the data collection of these well-known predictors of CAP is reproducible across EDs. The development of automatic extraction for a prediction model from electronic medical records using artificial intelligence could be of great value in a busy ED. In conclusion, typical respiratory symptoms combined with abnormal vital signs and elevated infection biomarkers are predictors for CAP on admission to an ED. A diagnostic prediction model based on these predictors is of limited value. Future prediction models should include novel diagnostic tools, imaging, PCR analysis and/or serological markers not routinely used in clinical practice to improve model performance and diagnose CAP more accurately in the ED.
This post was originally published on https://bmjopen.bmj.com