Cerebrospinal fluid CXCL13 concentration for diagnosis of neurosyphilis: a systematic review and meta-analysis

STRENGTHS AND LIMITATIONS OF THIS STUDY

  • Rigorous methodology was used in this study, including explicit eligibility criteria, extensive database searches, study selection by two reviewers working independently and risk of bias assessment.

  • We used subgroup analysis and meta-regression analysis to assess the potential cause of heterogeneity.

  • The diagnostic thresholds of cerebrospinal fluid CXCL13 displayed large differences among different studies, potentially impacting the accuracy of the combined results.

  • The limited number of included studies may lead to an overall risk of bias or insufficient evidence.

Introduction

Neurosyphilis, caused by the invasion of central nervous system (CNS) by Treponema pallidum, can potentially lead to irreversible damage to CNS if not administered treatment early. In recent years, increased cases of neurosyphilis were reported worldwide. For instance, in Columbia, Canada, the incidence of neurosyphilis increased from 0.03 per 100 000 to 0.8 per 100 000 between 1992 and 2012.1 In Australia, the annual incidence of neurosyphilis was 2.47 per 100 000 between 2007 and 2016.2 In Guangdong, China, the reported incidence of neurosyphilis increased by 8.1% annually from 0.21 per 100 000 to 0.31 per 100 000 between 2009 and 2014.3 In a recent survey conducted in five cities in China, it was discovered that the annual incidence of neurosyphilis increased by 48.11% from 2017 to 2019.4 Hence, early detection and treatment were urgently needed to address the threat posed by the rapid growth of neurosyphilis.

Diagnosis of neurosyphilis depends on a comprehensive judgement combining cerebrospinal fluid (CSF) tests (eg, abnormal CSF white cell count, and (or) protein concentration and reactive CSF-Venereal Disease Research Laboratory (VDRL)/rapid plasma reagin (RPR)/T. pallidum particle agglutination (TPPA)) with neurological signs and symptoms. Early detection of neurosyphilis remains a great challenge in clinical settings due to the lack of specific and sensitive diagnostic tests. The non-treponemal serological tests in CSF showed low sensitivity (VDRL: 49–87%, RPR: 51.5–81.8%, toluidine red unheated serum test: 58.9–82.5%), while treponemal serological tests (fluorescent treponemal antibody absorption/TPPA) lack specificity (55–100% and 85–100%, respectively).5 CSF pleocytosis and elevated protein concentrations can serve as an important but non-specific indicator for the diagnosis.6 In recent years, there has been an increased focus on the investigation of diagnostic biomarkers for neurosyphilis, such as CXC chemokine family members (CXCL13, CXCL10, CXCL8), macrophage migration inhibition factor, neurofilament light chain, glial fibrillar acidic protein, ubiquitin C-terminal hydrolase-L1, metabolites, exosomal miRNA in CSF, CD8+ IFN-g+ cells and antibody index for the intrathecal synthesis of specific anti-treponemal IgG in peripheral blood. These biomarkers have been demonstrated to be predictive of neurosyphilis,7 8 with CXCL13 being the most extensively researched among them.

CXCL13 is primarily produced by follicular dendritic cells and plays a role in CNS injury by attracting B cells through chemotaxis.9 Despite severe dysfunction of the blood–brain barrier, CXCL13 does not enter the CSF via blood circulation. Therefore, elevated levels of CXCL13 in CSF may originate directly in the CNS, indicating its potential as a biomarker for diagnosing neurosyphilis.10 According to an updated review, CXCL13 has a sensitivity and specificity of 41–90% and 37–90%, respectively, for the diagnosis of neurosyphilis.5 The Chinese National Guidelines recognise CXCL13 as an important reference indicators for neurosyphilis diagnosis.11 However, due to varying research qualities and diagnostic values of CXCL13 across studies, a systematic evaluation is urgently needed. To comprehensively assess its testing characteristics, we conducted a systematic review and meta-analysis of studies investigating the CSF CXCL13 concentration as a diagnostic biomarker for neurosyphilis.

Methods

This systematic review and meta-analysis has adhered to the Preferred Reporting Items for a Systematic Review and Meta-Analysis (PRISMA) of diagnostic test accuracy studies, and the protocol was registered in the PROSPERO database (CRD42023414212).

Search strategy

We performed a search for eligible articles on PubMed, Embase, Cochrane Library and Web of Science from database inception up to 1 May 2023. The search terms were ‘syphilis’ or ‘neurosyphilis’ and ‘CXCL13’; details of the search strategy are provided in the online supplemental file. There were no restrictions on language, and all references of included studies were hand-searched to find additional relevant articles.

Supplemental material

Eligibility and exclusion criteria

All studies that met the following criteria were identified: (1) defined syphilis and neurosyphilis; (2) evaluated the value of CXCL13 testing in CSF for the diagnosis of neurosyphilis; (3) the data of true positive (TP), false positive (FP), false negative (FN) and true negative (TN) of CSF CXCL13 testing in the diagnosis of neurosyphilis are provided or can be calculated; (4) cross-sectional design diagnostic tests and case–control design diagnostic tests.

The exclusion criteria were as follows: (1) reviews, case reports, correspondence and comments; (2) studies with incomplete data or no relevant outcome.

Study selection

A total of 61 articles met our search terms. After removing the duplicates, two authors (F-ZD and XZ) independently screened the remaining studies based on their titles and abstract and identified all eligible studies. Later, each of them independently read the full texts of the eligible studies and finally identified all included studies. If there was disagreement, discussions were conducted with a third researcher (R-LZ) until a consensus was reached.

Data extraction and quality assessment

Two researchers (F-ZD and XZ) extracted data independently from all finally included articles. Disagreements were discussed and resolved from a third researcher’s point of view (X-LZ). The extracted information included the year of article publication, author, country, clinical subtype of neurosyphilis, HIV infection status, sample size, threshold value of CXCL13 testing in CSF, TP, FP, FN and TN results in each included study. Two authors (F-ZD and XZ) assessed the quality of the included studies using the updated Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool.12

Statistical analysis

All data were analysed using Review Manager V.5.3, Stata MP (Multiprocessor computers) V.16.0 and Meta-disc V.1.4 software. The quantitative synthesis was performed by using a bivariate random-effects model. TP, FP, FN and TN will be used to describe various indicators in the studies. Spearman correlation coefficient was measured to assess the potential threshold effect. The results of the combination of the studies will be expressed by sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR) and area under the curve (AUC). Forest maps will be used to describe the 95% CI of sensitivity, specificity, PLR and NLR. According to the Cochrane Handbook, I2 is divided into 0.25, 0.50 and 0.75, representing mild, moderate and high heterogeneity, respectively.13 If I2>50%, the source of heterogeneity will be further analysed by meta-regression analysis, sensitivity analysis and subgroup analysis. Prior and posterior probability tests will be used to further evaluate the diagnostic value of CSF CXCL13 testing. Deek’s funnel plot asymmetry test will be used to reflect literature publication bias.

Patient and public involvement

None.

Results

Search results

As shown in the flow chart for study selection in figure 1, a total of 61 articles were identified from four databases under the search terms (PubMed 24, Embase 34, Cochrane Library 1, Web of Science 2), of which 24 duplicate records were removed. Then, 37 articles (35 published in English, 1 published in German and 1 published in Russian) were screened by title and abstract, of which 12 irrelevant studies, 9 reviews, correspondence, comments, etc, and 1 case report were removed. Finally, seven articles published in English were included in the meta-analysis by full-text assessed, after eight articles were removed due to the unavailable abstract (1) and data (6) and being mechanism research (1). The study selection process is illustrated as a PRISMA flow diagram (figure 1).

Figure 1
Figure 1

Preferred Reporting Items for a Systematic Review and Meta-Analysis flow diagram.

Characteristics of studies

The main characteristics of the seven included articles were summarised in table 1. There were 1582 subjects included in the final meta-analysis, which consisted of 1152 patients with syphilis (non-neurosyphilis) and 430 patients with neurosyphilis. Among these studies, four were conducted in China, two in the USA and one in the Netherlands. Four studies focused on HIV-negative patients, while one study specifically examined HIV-positive patients. The diagnostic thresholds of CXCL13 testing in CSF range from 4.871 pg/mL to 256.4 pg/mL.

Table 1

Characteristics of the included studies

Quality assessment

As shown in online supplemental figure 1, the QUADAS-2 assessment tool was used to evaluate the quality of included seven articles. Four studies14–17 avoided the case–control study design and were deemed to have a low risk of bias in patient selection. All studies were defined as high risk of bias in index tests because their diagnostic threshold of CSF CXCL13 testing was not prespecified. Two studies15 17 indicated a high risk of bias in reference standards because their diagnostic criteria of neurosyphilis did not follow the latest Chinese National Guidelines, while four studies16 18–20 were defined with uncertain risk of bias because it was unclear whether the interpretation of the results was performed without knowledge of the trial’s outcomes to be evaluated. One study15 did not include all patients in the analysis and was therefore classified as having an uncertain risk of bias. The remaining studies were at low risk of bias. Overall, the quality of the included literature can be considered moderate to high, which enhances the credibility of the results from all studies.

Supplemental material

Sensitivity analysis

As depicted in online supplemental figure 2A,B, all data were primarily focused on the diagonal, demonstrating that the statistical findings we obtained were relatively robust from goodness-of-fit and bivariate normality analyses. Influence analysis identified two influential observations, while outlier detection depicted one outlier study. When removing the two datasets (one database was overlapped), only the pooled sensitivity and AUC increased minimally in the overall effect (sensitivity: 0.76 vs 0.78; specificity: 0.83 vs 0.82; AUC: 0.84 vs 0.87), indicating that these outliers did not influence our findings.

Diagnostic performance

We summarised eight datasets from seven studies and calculated the pooled sensitivity as 0.76 (95% CI: 0.64 to 0.85), the pooled specificity as 0.83 (95% CI: 0.80 to 0.85), the pooled PLR as 4.43 (95% CI: 3.62 to 5.43), the pooled NLR as 0.29 (95% CI: 0.19 to 0.44), the pooled AUC as 0.84 (95% CI: 0.81 to 0.87) and the combined Diagnostic Odds Ratio (DOR) as 15.19 (95% CI: 8.43 to 27.37). Results are shown in figure 2 and online supplemental figure 3.

Figure 2
Figure 2

Pooled sensitivity, pooled specificity and summary ROC (SROC) curve. (A) Forest plots of pooled sensitivity, (B) forest plots of pooled specificity, (C) SROC curve and its area under the curve (AUC). ROC, receiver operating characteristic.

Based on the meta-analysis of all the data, we set the pretest probability of the prevalence of neurosyphilis as 20%.21 According to the Fagan plot analysis, the positive post-test probabilities increased to 53%, while the negative post-test probabilities decreased to 7% (online supplemental figure 4).

Meta-regression and subgroup analysis

Spearman test results indicated that there was no threshold effect in this study (Spearman correlation coefficient=0.357, p=0.385). The inconsistency I2 and Cochran Q test of heterogeneity of sensitivity and specificity showed that I2=82%, Q=38.88 with p<0.01 and I2=32.29%, Q=10.34 with p=0.17, respectively, indicating the high heterogeneity in sensitivity of this study. Meta-regression analysis was performed according to the following seven characteristics: predesign, region, sample size, diagnostic thresholds of CSF CXCL13 testing, publish year, clinical subtype and HIV infection status, to explore the source of heterogeneity. As shown in figure 3 and online supplemental table 1, studies from different regions were the main source of heterogeneity in sensitivity analysis.

Supplemental material

Figure 3
Figure 3

Meta-regression analysis to identify the source of heterogeneity.

Further subgroup analysis was conducted according to these seven characteristics. As shown in table 2, the diagnostic value of CSF CXCL13 testing in studies among the Chinese population was better than that among the non-Chinese population (sensitivity: 0.84 (95% CI: 0.79 to 0.89) vs 0.64 (95% CI: 0.45 to 0.79), specificity: 0.83 (95% CI: 0.77 to 0.88) vs 0.83 (95% CI: 0.80 to 0.86), PLR: 5.1 (95% CI: 3.6 to 7.1) vs 3.7 (95% CI: 2.8 to 5.0), NLR: 0.19 (95% CI: 0.14 to 0.26) vs 0.44 (95% CI: 0.27 to 0.70) and AUC: 0.87 (95% CI: 0.84 to 0.90) vs 0.83 (95% CI: 0.80 to 0.86)). There was no obvious difference in diagnostic value observed between the cut-off value and publish year subgroups. In addition, the diagnostic value of CSF CXCL13 testing in the group with sample size ≥200, unclassified clinical subtype and HIV-negative group was better than the combined value of all studies. The highest pooled AUC (0.90 (95% CI: 0.87 to 0.92)) was observed in the unclassified clinical subtype.

Table 2

Subgroup analysis

Publication bias assessment

Deek’s funnel plot asymmetry test was used to test the publication bias of the included studies, and the result was p=0.24, indicating that there was no potential publication bias (shown in online supplemental figure 5).

Discussion

The diagnosis of neurosyphilis, particularly asymptomatic and presumptive neurosyphilis which mainly relies on CSF abnormalities, poses a great challenge for clinicians. Therefore, it is essential to have optimised tests or valuable biomarkers for early detection and definitive diagnosis, ultimately improving the prognosis for patients. Recent research focused on the exploration of new biomarkers, with particular emphasis on the evaluation of CXCL13 concentration in CSF. Wang et al
20 evaluated the sensitivity and specificity of CSF CXCL13 for the diagnosis of neurosyphilis as 85.4% and 89.1%, respectively, while the sensitivity and specificity of CSF CXCL13 were 41% and 79% from Marra et al’s17 study. The variation in diagnostic value and research quality among these studies prompted us to conduct this meta-analysis. To our knowledge, it is the first meta-analysis to summarise data from all relevant articles to systematically evaluate the value of CSF CXCL13 testing in diagnosing neurosyphilis.

In this meta-analysis, the pooled sensitivity, specificity and AUC of the CSF CXCL13 testing were 0.76 (95% CI: 0.64 to 0.85), 0.83 (95% CI: 0.80 to 0.85) and 0.84 (95% CI: 0.81 to 0.87), respectively. Sensitivity analysis confirmed the stability of the combined results, indicating a good diagnostic accuracy. However, its sensitivity is lower than that of the treponemal serological tests (75.6–95%), and the specificity is lower than non-treponemal serological tests (81.8–100%), indicating that the CSF CXCL13 testing was not sufficient to replace the syphilitic serological tests in clinical settings. Furthermore, previous research has demonstrated that CSF CXCL13 also holds diagnostic significance in various other CNS disorders, such as Lyme neuroborreliosis, multiple sclerosis and anti-N-methyl-D-aspartate receptor encephalitis,22–24 resulting in the absence of specificity of CXCL13 abnormalities in CSF when other CNS diseases cannot be ruled out.

Considering the high heterogeneity of pooled sensitivity, a meta-regression analysis was conducted to identify the source of this variation. The results revealed that studies conducted in different regions are the main source of heterogeneity. Further subgroup analysis showed that studies conducted in China had superior diagnostic value with low heterogeneity. Four of seven included studies were conducted in China, indicating a growing trend towards independent research on biomarkers for the diagnosis of neurosyphilis in China. Since 2019, the Chinese Center for Disease Control and Center for Sexually Transmitted Disease Control conducted a clinical sentinel surveillance and build a research alliance for nationwide neurosyphilis to comprehend the incidence of neurosyphilis and address key issues concerning clinical management and scientific research of this condition.7 Hence, it is speculated that the focus of the Chinese government on the prevention and control of neurosyphilis may be accountable for the increased number of research studies in China. Due to the limited number of included studies in certain subgroups, such as those with prospective design, sample sizes less than 200, asymptomatic and symptomatic neurosyphilis, and HIV-infected individuals, it is not possible to calculate the combined diagnostic value. The study showed that the combined diagnostic value in the retrospective design subgroup was comparable with the total combined value. Additionally, it was found that the combined value in sample sizes larger than 200, as well as in unclassified neurosyphilis and HIV-negative subgroups, exhibited superiority over the total combined value. Diagnosing asymptomatic neurosyphilis is more challenging than diagnosing symptomatic neurosyphilis, and further studies are needed to assess the diagnostic value of CSF CXCL13 testing.20 In addition, Mothapo et al
16 found that HIV-infected patients benefited more from CSF CXCL13 as an added marker for diagnosis of neurosyphilis. Cepok et al
25 also demonstrated that HIV infection triggers an early profound B cell response in CNS, which serves as the primary virus-related B cell subset in the CSF, and B cells are the main source of CSF CXCL13. Consequently, patients with coinfected neurosyphilis experience an increased release of CXCL13 in their CSF. The diagnostic thresholds of CSF CXCL13 testing differed between each study due to varied CXCL13 test kits used in different studies. Subgroup analyses failed to identify differences in the diagnostic value of the CSF CXCL13 testing across groups with different cut-off values. Nonetheless, the development and evaluation of CXCL13 test kits that can be consistently used in clinical settings remain a significant challenge. In summary, further studies are necessary to assess the diagnostic value of CSF CXCL13 testing in HIV infection and asymptomatic neurosyphilis.

The presence of publication bias was not indicated by the superimposed regression line and slope coefficient. However, it is important to note that assessments of publication bias, such as Deek’s approach, have not been specifically designed for diagnostic accuracy studies and have limited power. Ideally, Deek’s funnel plot requires at least 10 studies for reliability.26 The limited study count may undermine the test results’ significance due to insufficient power. Therefore, they can produce misleading results, especially when sensitivities and specificities are heterogeneous.27 28 As a result, the possibility of publication bias in this meta-analysis cannot be completely ruled out.

Admittedly, there are several limitations in this study. First, the meta-analysis included a limited number of publications, most of which were retrospective and case–control studies. This inevitably led to information and population selection biases. Second, there is a significant variation in the diagnostic thresholds of CSF CXCL13 testing among different studies. Although the Spearman correlation coefficient indicates no threshold effect, these variations could still impact the accuracy of the combined results. Additionally, there is a lack of sufficient studies available for pooling the diagnostic value of asymptomatic neurosyphilis and HIV-infected neurosyphilis subgroups in the subgroup analysis. As a result, it is not possible to compare the differences in the combined value between these subgroups.

In conclusion, the results demonstrated that CSF CXCL13 testing had a reasonable accuracy in diagnosing neurosyphilis. Diagnostic accuracy in Chinese studies is superior to that in non-Chinese studies. However, based on current evidence, the pooled sensitivity and specificity indicate that the CSF CXCL13 testing is not sufficiently suitable for clinical application. Further multicentre prospective diagnostic trials with large sample sizes, especially in patients with asymptomatic neurosyphilis and HIV-infected neurosyphilis, are expected to provide more evidence for the clinical application of CSF CXCL13 testing.

Data availability statement

Data sharing not applicable as no datasets generated and/or analysed for this study.

Ethics statements

Patient consent for publication

Ethics approval

Not applicable.

This post was originally published on https://bmjopen.bmj.com