Comparing COVID-19 vaccination coverage, adverse reactions and impact of social determinants of health on vaccine hesitancy in ADRD/MCI and non-ADRD/MCI population: protocol for a retrospective cross-sectional study

Data source: All of Us (AoU) Researcher Workbench

The data source for this study protocol will be the AoU Researcher Workbench, which is a secure and comprehensive source of biomedical datasets enrolled with a broad, diverse group of US populations.25 26 The AoU Research Workbench seeks to engage individuals from under-represented demographic groups, promoting diversity in research.27 The AoU research programme encompasses various data elements, including participants’ basic demographic information, responses to health surveys, physical measurements, biospecimen collection (including blood, urine and saliva samples stored in the secure AoU biobank), structured electronic health records (EHRs) and Fitbit tracker data collected from one million participants across the USA.28 The EHR data are available since AoU participants enrolled in the programme in 2018. Each participant has completed informed consent for sharing their EHR data with the data and research centre, and they provide survey responses covering various domains on an ongoing basis.28 29 Specifically, we will extract relevant data fields from sources such as the Demographics, Basic Survey, COVID-19 Vaccine Survey, Health Access & Utilization, Social Determinants of Health and EHR conditions data. Enrolment for the AoU programme commenced in 2018 and is anticipated to continue for at least 10 years. By 2023, the AoU research initiative successfully extended invitations to one million individuals nationwide.30 The diverse population and data integration from multiple sources within the AoU Research Workbench allow for a comparative analysis of vaccination rates, adverse reactions and the influence of social determinants of health on vaccine hesitancy between populations with ADRD/MCI and non-ADRD/MCI populations with a sequential follow-up. The deidentified data are accessible through the AoU Researcher Workbench (https://workbench.researchallofus.org) under institutional data use agreements.31 All analyses will be conducted within a secure platform provided by AoU Researcher Workbench. The study protocol and AoU materials have received approval from the Institutional Review Board at Florida State University Office of Research.

Workspace on the AoU Researcher Workbench secure platform: ADRD/MCI and COVID-19 vaccination

We have established a workspace named ‘ADRD/MCI and COVID-19 vaccination’ within the AoU Researcher Workbench. This cloud-based platform grants authorised researchers access to and the ability to analyse data from the AoU. The platform offers two levels of data access: the registered tier and the controlled tier. We have been granted access to the controlled data tier. Within this controlled tier, we have constructed two distinct cohorts: one consists of individuals with ADRD/MCI and the other consists of those without ADRD/MCI. Cohort formation was based on participants’ EHRs, including International Classification of Diseases (ICD) codes related to ADRD/MCI as original source concepts as well as standard concepts group based on the ICD coding and classification. We applied appropriate logical operators such as ‘AND’ and ‘OR’ to combine the key inclusion and exclusion criteria. To facilitate our research, we used the dataset builder tool to construct the datasets. Subsequently, we exported the acquired data to Jupyter Notebooks for analysis, leveraging the R programming language V.4.3 (R Foundation for Statistical Computing) and Python V.3.12 (Python Software Foundation).

Patient and public involvement

There was no patient or public involvement in this research.

Outcome variables

COVID-19 vaccination rates, adverse reactions and vaccine hesitancy will be assessed through EHR drug exposures and COVID-19 Vaccine Survey Questionnaires administered throughout the summer, fall, winter of 2021 and new year 2022. The vaccine survey questionnaires contain inquiries about receiving the first dose/second dose/boosters of COVID-19 vaccination, adverse reactions after vaccination (eg, swelling, fever, headache, muscle pain, etc) and how likely people are to get vaccinated and reasons that make people hesitate to get the vaccine. The respondents will indicate how likely or unlikely they would be vaccinated on a 5-point Likert scale, ranging from ‘very likely’ to ‘very unlikely’ (table 1).

Independent variables

Social determinants of health at individual, interpersonal and community-level factors (table 1).

The study protocol encompasses social determinants of health at three levels: individual, interpersonal and community. Individual-level factors include individual demographics, chronic conditions and mental health status. Chronic conditions related to ADRD/MCI and COVID-19 vaccination rates include32 33 hypertension, cerebrovascular disease, cerebral infarction, overweight and obesity, diabetes, coronary artery disease, heart failure, myocardial infarction, chronic obstructive pulmonary disease, chronic kidney disease, chronic liver disease, cancer and mental health disorders.

Interpersonal-level factors for ADRD/MCI patients consist of sociocultural environment and social support, as assessed in the AoU Basic Survey and AoU Social Determinants of Health Survey, with affirmative responses to the following questions: (a) ‘Do you own or rent the place where you live?’ (b) ‘Where are you currently living?’ (c) ‘How many years have you lived at your current address?’ (d) ‘What is the main type of housing in your neighbourhood?’ (e) ‘Not including yourself, how many other people live at home with you?’ (f) ‘how much you agree or disagree that people around here are willing to help their neighbour?’ and (g) ‘how much you agree or disagree that people in your neighbourhood generally get along with each other?’.

Community-level factors include healthcare facility accessibility and utilisation, as assessed in the AoU Health Access & Utilization Survey, with affirmative responses to the following questions: (a) ‘During the past 12 months, were you told by a healthcare provider or doctor’s office that they did not accept your healthcare coverage?’ (b) ‘What kind of place that you usually go to when you are sick or need advice about your health?’ (c) ‘Have you seen any of the following doctors or healthcare providers about your own health in the last 12 months?’—a general doctor; a nurse practitioner; a doctor specialised in women’s health; a mental health professional; an optometrist; a podiatrist; a chiropractor; a physical therapist; a dentist; a medical doctor; traditional healers; (d) ‘Have you delayed getting care for any of the following reasons in the past 12 months?’—did not have transportation; live in the rural area where distance to healthcare provider is too far; nervous about seeing a healthcare provider; could not get time off work; could not get child care; could not get elderly care; could not afford the copay; could not afford the deductible; had to pay out of pocket for some or all of the procedure.

Samples

We have set up study cohorts and preliminary datasets within the AoU Researcher Workbench to assess the availability of essential variables for this study protocol. After conducting an initial screening within the AoU Researcher Workbench, we have identified a sample size of 157 281 individuals with COVID-19 vaccination information in either EHR drug exposures domain or the COVID-19 vaccine survey. Among these individuals, 9718 (6%) were diagnosed with ADRD/MCI conditions, and the remaining 110 355 without ADRD/MCI conditions; 37 208 individuals without EHR data are excluded. Through 1:1 propensity score matching, 9718 individuals without ADRD/MCI are paired with 9718 individuals with ADRD/MCI via balancing the demographics, chronic diseases and mental health status (figure 1 and table 2).

Figure 1

Study flowchart of participants in ADRD/MCI and non-ADRD/MCI cohorts. ADRD, Alzheimer’s disease and related dementias; AoU All of Us; EHR, Electronic Health Record; MCI, mild cognitive impairment.

Sample size and power calculation

A two-group retrospective cohort design is used in this study protocol. The sample size calculation is powered by the outcomes. The changes in vaccination rate and vaccination hesitancy will be compared between the ADRD/MCI groups and individuals without ADRD/MCI. Hence, there will be a total of two statistical tests conducted. We will follow the methodology proposed by Kelley and Maxell to estimate the sample size for survey questions.34 Based on the study parameters, a sample size of 2000 was determined sufficient to achieve a conjunctive power of 0.8 or higher at a significance level 0.05.21

Statistical analysis plan

Demographic variables will be summarised using descriptive statistics (eg, Mean±SD or median with IQR) as appropriate for continuous variables. Frequency and percentage will be summarised for categorical variables. Data quality will be checked, including steps like outlier detection.35 The distribution of all variables will be examined to check the validity of distribution assumptions before subsequent analyses, using univariate/multivariate Shapiro-Wilk test and a visual inspection of histograms and quantile-quantile plots. If the normal assumption of continuous variables is not met, appropriate data transformations or alternative data analysis procedures (eg, nonparametric, bootstrapping) will be employed. The baseline demographics and survey answers of the ADRD/MCI and without ADRD/MCI cohorts will be summarised in forms of tables and figures, with reporting the corresponding p values from independent samples t-test (or Kruskal-Wallis test when appropriate) for continuous variables and χ² test for categorical variables. Significance level at 0.05 sets as threshold.

In our planned analysis of vaccination rates between ADRD/MCI individuals and those without ADRD/MCI, we use propensity score matching to evaluate the vaccination rate difference between the two cohorts while controlling the selected covariates. Propensity score matching is a statistical matching technique used with observational data, which attempts to construct a balanced intervention and control group by matching each intervention unit with a controlled unit of similar characteristics.36 The propensity score is a balancing score, the distribution of baseline covariates will be similar between cohorts.37 In this retrospective observational study, the cohorts of ADRD/MCI and non-ADRD/MCI individuals are not randomly selected, leading to an imbalance sample sizes and uncontrolled baseline characteristics. This imbalance may cause confounding effects on the vaccination rates. To control the confounding effects, we generate propensity scores using logistic regression to identify the effects of confounding covariates, with the nearest available Mahalanobis metric matching applied to obtain 1:1 matched pair between ADRD/MCI and non-ADRD/MCI cohorts. The balanced covariates in the propensity score matching include age, gender, sex at birth, race, ethnicity, chronic diseases and mental health status. The goal of propensity score matching is to achieve a balanced distribution of covariates between the ADRD/MCI and non-ADRD/MCI groups. We assess the effectiveness of matching by comparing the effective size of each included covariate before and after matching, with a decrease in effective size indicating balanced pairs. The results of propensity logit will be presented in the histogram, and standardised mean differences across covariates before and after matching are reported. The study uses the ‘psmpy 0.3.13’ propensity score matching package for propensity logits and graphical representations of matching outcomes as well as matched pairs and standardised mean differences across covariates.38

Vaccine hesitancy will be computed and compared across cohorts using χ² tests. To analyse the association of vaccine hesitancy (ie, individuals reported how likely to get vaccinated) with covariates, we will employ a binary logistic regression with vaccine hesitancy as the outcome (5-point Likert scale recategorised to binary outcome: likely or unlikely) and covariates as controlled variables. This approach will enable us to compute the adjusted ORs (AOR) to determine the likelihood of vaccine hesitancy.39 An AOR greater than 1 indicates higher odds for vaccine hesitancy, meaning individuals who are unlikely to take the vaccination have high vaccine hesitancy.39 Of 95% CIs for each coefficient’s AORs will be calculated. The statistical analysis is performed on Jupyter Notebooks, leveraging the R programming language V.4.3 (R Foundation for Statistical Computing) and Python V.3.12 (Python Software Foundation).

Data management and safety

All datasets will be stored on AoU Research Benchwork. Data filtering will be performed on the AoU Research platform. Data will be analysed using R or Python. Our research team has experience using Python programming language and has pilot cohorts developed in the AoU Research platform and used for analysis. We will adhere to AoU research benchwork data protection regulations and data use guidelines.