Estimated Effectiveness of the BNT162b2 XBB Vaccine Against COVID-19

Key Points

Question  Does receiving the BNT162b2 XBB vaccine offer additional protection against COVID-19 hospital admissions and ambulatory visits for US adults compared with not receiving a BNT162b2 XBB vaccine of any kind, and do older versions of the COVID-19 vaccine still provide any protection compared with being unvaccinated?

Findings  In this case-control study among 2854 cases and 15 345 controls, the BNT162b2 XBB vaccine provided statistically significant additional protection against a range of COVID-19 outcomes during the early part of the 2023 to 2024 viral respiratory season. Older versions of COVID-19 vaccines offered little, if any, additional protection compared with being unvaccinated, including against COVID-19 hospital admissions, regardless of the number or type of prior doses received.

Meaning  These findings reaffirm current recommendations for broad age-based use of annually updated COVID-19 vaccines.

Importance  Data describing the early additional protection afforded by the recently recommended BNT162b2 XBB vaccine (Pfizer-BioNTech; 2023-2024 formulation) are limited.

Objective  To estimate the association between receipt of the BNT162b2 XBB vaccine and medically attended COVID-19 outcomes among US adults 18 years and older.

Design, Setting, and Participants  This test-negative case-control study was performed to estimate the effectiveness of the BNT162b2 XBB vaccine against COVID-19–associated hospitalization and emergency department (ED) or urgent care (UC) encounters among adults in the Kaiser Permanente Southern California health system between October 10, 2023, and December 10, 2023. Cases were those presenting with an acute respiratory illness and who had a positive SARS-CoV-2 polymerase chain reaction test; controls had an acute respiratory illness but tested negative for SARS-CoV-2.

Exposure  The primary exposure was receipt of the BNT162b2 XBB vaccine compared with not receiving an XBB vaccine of any kind, regardless of prior COVID-19 vaccination or SARS-CoV-2 infection history. Receipt of prior (non-XBB) versions of COVID-19 vaccines was also compared with being unvaccinated to estimate remaining protection from older vaccines.

Main Outcomes and Measures  Analyses for cases and controls were conducted separately for COVID-19 hospital admissions and ED/UC encounters. Adjusted odds ratios and 95% CIs were estimated from multivariable logistic regression models that were adjusted for patient demographic and clinical characteristics. Estimation of vaccine effectiveness was calculated as 1 − odds ratio × 100%.

Results  Among 2854 cases and 15 345 controls (median [IQR] age, 56 [37-72] years; 10 658 [58.6%] female), adjusted estimation of effectiveness of the BNT162b2 XBB vaccine received a median of 34 days prior vs not having received an XBB vaccine of any kind was 62% (95% CI, 32%-79%) against COVID-19 hospitalization and 58% (95% CI, 48%-67%) for ED/UC visits. Compared with being unvaccinated, those who had received only older versions of COVID-19 vaccines did not show statistically significant reduced risk of COVID-19 outcomes, including hospital admission.

Conclusions and Relevance  Findings of this case-control study reaffirm current recommendations for broad age-based use of annually updated COVID-19 vaccines given that (1) the BNT162b2 XBB vaccine provided statistically significant additional protection against a range of COVID-19 outcomes and (2) older versions of COVID-19 vaccines offered little, if any, long-term protection, including against hospital admission, regardless of the number or type of prior doses received.

XBB and its sublineages were the predominant circulating SARS-CoV-2 strains in the US between January and December 2023.¹ XBB sublineages are antigenically and phylogenetically distinct from the Omicron BA.4 and BA.5 sublineages that were predominant in late 2022. Thus, on September 11, 2023, the US Food and Drug Administration authorized or approved updated monovalent messenger RNA (mRNA) COVID-19 vaccines targeting the XBB sublineage for individuals aged 6 months through 11 years and 12 years and older, respectively.² This included use of XBB vaccines for all recommended doses, including primary series and booster doses. XBB vaccines were made broadly available in the US on September 15, 2023, following Centers for Disease Control and Prevention (CDC) recommendations for use in all individuals 6 months and older in preparation for the winter respiratory virus season.³ Studies evaluating the association between receipt of XBB vaccines and the development of clinically relevant COVID-19 end points are needed.

Similar to our previous reports,^4-8 we performed a test-negative case-control study to estimate the effectiveness of the BNT162b2 XBB vaccine (Pfizer-BioNTech; 2023-2024 formulation). We included patients 18 years and older at Kaiser Permanente Southern California (KPSC) who were diagnosed with acute respiratory infection (ARI; eTable 1 in Supplement 1) and tested for SARS-CoV-2 using polymerase chain reaction (PCR) during a (1) hospital admission or (2) emergency department (ED) or urgent care (UC) encounter without a subsequent hospital admission for the same index encounter from October 10, 2023, through December 10, 2023. The study start date corresponded to 14 days after the date that XBB vaccines were made available in KPSC (September 25, 2023).

Participants were required to have 1 year or more of health plan membership (allowing for a 31-day gap in membership to account for delays in membership renewal) to determine comorbidities and medical history. Encounters in which the patient had the following were excluded: (1) another positive SARS-CoV-2 test within 90 days, (2) received any type of XBB vaccine other than the BNT162b2 XBB vaccine, (3) received a BNT162b2 XBB vaccine within 2 months after a prior COVID-19 dose, (4) received a BNT162b2 XBB vaccine within 14 days prior to the encounter, (5) received any other non-XBB booster doses (eg, BA.4/5 bivalent or wild-type boosters) outside of CDC-recommended dosing intervals (recommendations were defined as receipt of any mRNA BA.4/5 bivalent dose between August 31, 2022, and September 11, 2023, with ≥8 weeks [≥56 days] since their most recent dose of original wild-type COVID-19 mRNA vaccine received with a minimal required interval of ≥28 days between a second and subsequent wild-type dose), (6) received nirmatrelvir/ritonavir or any other COVID-19 outpatient antiviral or monoclonal antibody (ie, molnupiravir, remdesivir, bebtelovimab, bamlanivimab, casirivimab, cilgavimab, sotrovimab, tixagevimab) in the 30 days prior to a COVID-19 encounter, or (7) a hospital admission that, despite having an ARI diagnosis with a positive SARS-CoV-2 test, was determined to be likely unrelated to COVID-19 or clearly related to another cause based on medical record review conducted by trained research staff who were blinded to vaccination status and later validated by a blinded physician investigator (B.K.A.; eAppendix in Supplement 1). For patients who had multiple encounters, we included only the first encounter to maintain independence of outcome events. This study was approved by the KPSC institutional review board, which waived the requirement for informed consent in accordance with the Common Rule (45 CFR §46.116).

Cases were those with a positive SARS-CoV-2 PCR test associated with a hospital admission or ED/UC encounter with ARI. Controls had ARI and a negative SARS-CoV-2 PCR test result. Hospital admission and ED/UC outcomes were mutually exclusive. SARS-CoV-2 PCR tests among cases and controls were restricted to those administered within 14 days prior to the initial ARI encounter through no more than 3 days after the encounter.

All KPSC members were eligible for COVID-19 vaccines at no cost based on indications authorized or approved by the US Food and Drug Administration. KPSC electronic health records captured all vaccinations administered within the health system. Records were supplemented with vaccine administration data from the California Immunization Registry, to which all health care professionals are required by law to report COVID-19 vaccinations within 24 hours. To be considered vaccinated, the dose had to occur more than 14 days before testing for SARS-CoV-2.

For the primary analysis, the odds of receipt of a BNT162b2 XBB vaccine were compared with the odds of not receiving a XBB vaccine of any kind (including unvaccinated persons) across cases and controls, regardless of prior COVID-19 vaccination or SARS-CoV-2 infection history. Secondary analyses compared the odds of receipt of a BNT162b2 XBB vaccine vs the odds of (1) receipt of 1 or more doses of a BA.4/5 bivalent vaccine but no XBB vaccine of any kind, (2) receipt of 3 or more or 2 or more doses of an original wild-type mRNA vaccine but no variant-adapted vaccines of any kind (eg, XBB or BA.4/5 bivalent doses), and (3) being unvaccinated across cases and controls. Among those who did not receive an XBB vaccine of any kind, we also compared the odds of receiving 1 or more doses of a BA.4/5 bivalent vaccine or receiving 3 or more or 2 or more original wild-type COVID-19 vaccine doses without a bivalent vaccine of any kind with the odds of being unvaccinated across cases and controls. These comparisons were used to estimate remaining protection from prior non-XBB vaccines (ie, either BA.4/5 or wild-type doses) during the study period.

Odds ratios and 95% CIs calculated using the Wald method were derived from multivariable logistic regression models that included week of encounter, age (18-49, 50-64, and ≥65 years), sex (female, male), self-reported race and ethnicity (non-Hispanic African American or Black, non-Hispanic Asian or Pacific Islander, Hispanic or Latinx, non-Hispanic White, other [including individuals who identified as American Indian or multiple or other races and ethnicities], and unknown), body mass index (calculated as weight in kilograms divided by height in meters squared; <18.5, 18.5-24.9, 25.0-29.9, 30.0-34.9, ≥35.0, and unknown), Charlson Comorbidity Index (0, 1, 2, 3, and ≥4), receipt of influenza vaccine in the year before admission (yes or no), receipt of pneumococcal vaccine in the 5 years before admission (yes or no), health care utilization in the year before admission (ie, number of hospital admissions and ED or outpatient visits), and documentation of previous SARS-CoV-2 infection confirmed by PCR or antigen test (ever vs never) for pre-Delta, Delta, and Omicron periods. Missing values were treated as separate categories for all variables in all analyses.

Analyses were completed separately for ARI-associated hospital admissions and ED/UC encounters, as well as separately by the exposure comparisons defined herein. Estimation of vaccine effectiveness (VE) was calculated as 1 − odds ratio × 100%. Primary analyses were further stratified by age group (18-64 years vs ≥65 years). In sensitivity analyses, we examined the effect on the primary analysis by including patients who received COVID-19 antiviral or monoclonal antibody treatment. Analyses were performed using SAS, version 9.4 (SAS Institute).

Of 20 015 ARI encounters among adults 18 years and older with continuous enrollment and an eligible SARS-CoV-2 PCR test, 18 199 met study selection criteria (2977 [16.4%] hospital admissions and 15 222 [83.6%] ED/UC encounters; Figure 1). Overall, 148 of 592 (25.0%) hospitalizations with a positive SARS-CoV-2 test were determined to be unrelated to COVID-19 on physician medical record review and were excluded. The median (IQR) age among patients was 56 (37-72) years. Of the 18 199 included patients, 2854 (15.7%) tested positive for SARS-CoV-2 and 1146 (6.3%) received the BNT162b2 XBB vaccine (Table and Figure 1). In analyses of hospital admissions among 391 cases and 2586 controls, 13 (3.3%) and 169 (6.5%), respectively, received the BNT162b2 XBB vaccine. Of 2463 cases and 12 759 controls in analyses of ED/UC encounters, 95 (3.9%) and 869 (6.8%), respectively, received the BNT162b2 XBB vaccine. A total of 17 053 patients (93.7%) never received an XBB COVID-19 vaccine of any kind and 2140 (11.8%) never received a COVID-19 vaccine of any kind. Among those who received the BNT162b2 XBB vaccine, the median (range) time since receipt of their most recent previous dose of a COVID-19 vaccine was 363 (63-956) days. Overall, median (IQR) time since receipt of a BNT162b2 XBB vaccine was 34 (23-49) days. Median (IQR) time since last dose for those who received a bivalent booster but no XBB dose was 358 (311-392) days. Median (IQR) times since last dose for those with more than 3 and more than 2 doses with no variant-adapted boosters were 627 (425-691) days and 675 (513-763) days, respectively.

Adjusted estimation of VE of the BNT162b2 XBB vaccine (vs not having received an XBB vaccine of any kind) was 62% (95% CI, 32%-79%) against COVID-19 hospital admission and 58% (95% CI, 48%-67%) against ED/UC visits (Figure 2 and eTables 2 and 5 in Supplement 1). In secondary analyses, the estimation of VE of the BNT162b2 XBB vaccine was similar regardless of comparison group, including those who (1) received 1 or more doses of the BA.4/5 bivalent vaccine and no XBB vaccine, (2) received 3 or more or 2 or more doses of the original wild-type vaccine without any variant-adapted boosters of any kind (eg, BA.1 or BA.4/5 bivalent vaccines or XBB vaccines), and (3) were unvaccinated, across all settings of care (ie, hospital admission and ED/UC visits) (Figure 2 and eTable 2 in Supplement 1). With the exception of hospitalization outcomes among the 18- to 64-year olds, for which 95% CIs were wide, estimation of VE appeared generally similar across age groups 18 to 64 years and 65 years and older (eTables 2 and 3 in Supplement 1). In sensitivity analyses, results were similar when patients who received antiviral or monoclonal antibody treatment in the 30 days prior to their COVID-19 encounter (n = 950) were included (eTable 4 in Supplement 1). Finally, compared with unvaccinated individuals, those who had not received an XBB vaccine of any kind but had received older versions of COVID-19 vaccines (ie, ≥1 BA.4/5 bivalent dose or ≥3 or ≥2 original wild-type doses and no variant-adapted vaccines of any kind) did not show a statistically significant reduced risk of COVID-19 outcomes, including hospital admission, during the study period (Figure 3).

In this test-negative case-control study conducted in a large US health care system during the early part of the 2023 to 2024 viral respiratory season, the estimation of VE for the BNT162b2 XBB vaccine was 62% (95% CI, 32%-79%) against COVID-19 hospital admission and 58% (95% CI, 48%-67%) against COVID-19 ED/UC encounters after a median of 34 days since receipt of the BNT162b2 XBB dose compared with those who did not receive an XBB vaccine. For the week ending December 9, 2023, there were 23 432 new COVID-19 hospital admissions in the US—the highest rate since winter 2022.⁹ This still represents a large public health burden and is roughly 3 times higher than the number of new weekly influenza hospitalizations that occurred during the same period (n = 7090),¹⁰ albeit fewer than the approximately 150 000 weekly hospitalizations seen at the peak of the Omicron BA.1 wave.⁹ With this context, the present findings help reaffirm current recommendations for broad age-based use of annually updated COVID-19 vaccines in the US to improve protection against COVID-19 each year prior to likely winter peaks in disease activity.¹¹

The BNT162b2 XBB vaccine provided similar additional protection in adults regardless of age group and the number of prior COVID-19 vaccine doses received for all COVID-19 outcomes. This latter finding was consistent with results suggesting that prior receipt of only older versions of COVID-19 vaccines (ie, receipt of a BA.4/5 bivalent vaccine or ≥3 or ≥2 original wild-type doses but no XBB vaccine) provided little, if any, current additional protection compared with being unvaccinated against COVID-19 outcomes, including hospital admission. Median time since administration of these older vaccines was between 1 and 2 years ago, whereas median time since receipt of a BNT162b2 XBB vaccine was 1 month. Thus, analogous to influenza, although older versions of COVID-19 vaccines once provided high levels of protection, the combination of waning vaccine-induced immunity and continuous SARS-CoV-2 strain evolution eventually renders prior versions of vaccines ineffective. This, in turn, warrants routine updates to COVID-19 vaccines—also like influenza—so long as SARS-CoV-2 continues to circulate and cause disease.^9-11

An earlier study showed that among adults 65 years and older in Denmark, receipt of an XBB vaccine (90% of which were the BNT162b2 XBB vaccine in the study population) led to a 76% (95% CI, 62%-85%) reduction in risk of COVID-19 hospital admission over an average follow-up time of 10 days compared with those who did not receive an XBB vaccine¹²—an estimate that was comparable to the present, albeit slightly higher, against the same outcome and comparison group. However, this study had shorter follow-up time and included data only through the end of October 2023, which was prior to the emergence and rapid growth of the JN.1 strain. Another recent study conducted in the Netherlands also showed similar reductions in risk of hospital admission (71%; 95% CI, 67%-74%) among adults 60 years and older who were previously vaccinated.¹³ The present study helps confirm these early global findings but also describes the association between BNT162b2 XBB vaccine receipt and the development of COVID-19 across a broader range of outcomes and age groups, in a more diverse study population, and during a more recent time period that included when the JN.1 strain was rapidly increasing in prevalence in the US.

Uptake of XBB vaccines in the US to date has been low. As of December 22, 2023, only 19% and 37% of all adults 18 years and older and 65 years and older, respectively, had received an XBB vaccine.¹⁴ Current COVID-19 vaccine coverage considerably lags that of seasonal influenza vaccines, despite both vaccines being made available during the autumn and winter and current CDC guidelines that support co-administration of the 2 vaccines.¹⁵ Reasons for low COVID-19 vaccine uptake likely include reduced concern about COVID-19 in the general population over time and, as pandemic declarations ended, annual COVID-19 vaccination not yet being seen as a routine health activity, confusion about risk level regarding COVID-19, and continued skepticism in some populations about the safety and effectiveness of mRNA vaccines.¹⁶ In addition, XBB vaccines were not made available until the latter half of September 2023, which is notably later than when influenza vaccines are made available each year. Thus, there may have been missed opportunities for co-administering COVID-19 vaccines with influenza vaccines during the month of September—a time when many influenza vaccines are given.

This study has limitations. Although we controlled for key sociodemographic and clinical characteristics, there may be residual confounding associated with unaccounted-for differences in the likelihood of exposure or severity of SARS-CoV-2 infection between vaccinated and unvaccinated individuals. Although individuals who are more likely to get vaccinated against COVID-19 may also be more likely to seek care or testing for SARS-CoV-2, the test-negative design of this study and the focus on ARI events occurring in the hospital and ED/UC settings help mitigate against bias caused by differences in health care–seeking behavior, including the propensity to test.^17-19 We also controlled for prior health care utilization, age, and underlying comorbid illness to help mitigate the potential for healthy vaccinee bias. A second limitation is that median time since receipt of a BNT162b2 XBB vaccine was only 34 days, and future studies are needed to evaluate durability of protection. In addition, this study was conducted during a period when XBB sublineages were predominant but JN.1 was also co-circulating and rapidly increasing in prevalence across the US and California.¹ However, we did not have genotype information available for all of the included cases, nor were we able to estimate sublineage-specific estimates. Thus, future studies describing the association between receipt of XBB vaccines and development of strain-specific BA2.86 sublineage-related disease (eg, JN.1) are needed. While high health care utilization in this study population may mitigate underascertainment of prior infections, misclassification of previous infections is likely, particularly as home testing has increased and overall rates of testing have gone down compared with earlier in the pandemic. If undocumented previous infection was more likely in unvaccinated individuals, for example, this could contribute to underestimation of vaccine protection. It remains possible that some health care encounters were “with COVID-19” rather than “for COVID-19,” and this could lead to underestimation of the protective effects of vaccination against medically attended disease. To help mitigate this bias, we (1) used medical record review to exclude hospital admissions that were unrelated to COVID-19 and (2) restricted the analyses to patients presenting with ARI for all outcomes. Lastly, uptake of the BNT162b2 XBB vaccine was low overall and was most frequent in individuals who were older and who had comorbidities, which may affect the generalizability of the findings, lead to the underestimation of VE, or both.

In this case-control study, individuals who did not receive an XBB vaccine and had received only older versions of COVID-19 vaccines had little, if any, additional protection compared with unvaccinated individuals against COVID-19 end points, including hospital admission, regardless of the number or type of prior doses received. Receipt of a BNT162b2 XBB vaccine, however, was associated with statistically significant reduced risk of developing a range of COVID-19 outcomes during the early part of the 2023 to 2024 viral respiratory season—with the strongest protective effects seen against hospital admission. These 2 findings help reaffirm current recommendations for broad age-based use of annually updated COVID-19 vaccines. Uptake of this 2023-2024 formulation of COVID-19 vaccines, however, remains low, and targeted and tailored interventions to continuously improve annual COVID-19 uptake are warranted.

Accepted for Publication: March 22, 2024.

Published Online: June 24, 2024. doi:10.1001/jamainternmed.2024.1640

Open Access: This is an open access article distributed under the terms of the CC-BY-NC-ND License. © 2024 Tartof SY et al. JAMA Internal Medicine.

Corresponding Author: Sara Y. Tartof, PhD, MPH, Department of Research & Evaluation, Kaiser Permanente Southern California, 100 S Los Robles, 2nd Floor, Pasadena, CA 91101 (sara.y.tartof@kp.org).

Author Contributions: Dr Slezak and Ms Hong had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Tartof, Slezak, Puzniak, Simmons, Jodar, McLaughlin.

Acquisition, analysis, or interpretation of data: Tartof, Slezak, Frankland, Puzniak, Hong, Ackerson, Stern, Zamparo, Jodar, McLaughlin.

Drafting of the manuscript: Tartof, Frankland, Puzniak, McLaughlin.

Critical review of the manuscript for important intellectual content: All authors.

Statistical analysis: Slezak, Frankland, Hong.

Obtained funding: Tartof, Puzniak, Zamparo, Jodar, McLaughlin.

Administrative, technical, or material support: Frankland, Ackerson, Stern, Zamparo, Simmons, Jodar, McLaughlin.

Supervision: Tartof, Ackerson, Zamparo, Jodar, McLaughlin.

Conflict of Interest Disclosures: Dr Tartof reported institutional grants from Pfizer and GSK outside the submitted work. Dr Slezak reported institutional grants from Pfizer and Dynavax outside the submitted work. Mr Frankland reported a research contract with Pfizer and previous stock in Pfizer. Dr Puzniak reported personal fees from Pfizer during the conduct of the study. Ms Hong reported institutional grants from Pfizer during the conduct of the study. Dr Ackerson reported institutional grants from Pfizer, Moderna, GSK, and Dynavax outside the submitted work. Dr Stern reported grants from GSK, Sanofi, and Moderna outside the submitted work. Dr Jodar reported salary from and stock in Pfizer outside the submitted work. Dr McLaughlin reported salary from and stock in Pfizer during the conduct of the study. No other disclosures were reported.

Funding/Support: This study was sponsored by Pfizer.

Role of the Funder/Sponsor: The study design was jointly developed by Kaiser Permanente Southern California and Pfizer. Kaiser Permanente Southern California collected and analyzed the data. Pfizer did not participate in the collection or analysis of data. Kaiser Permanente Southern California and Pfizer participated in the interpretation of data, the writing of the report, and the decision to submit the manuscript for publication.

Data Sharing Statement: See Supplement 2.

Additional Contributions: We thank Harpreet Takhar, MPH, from the Kaiser Permanente Southern California Department of Research & Evaluation for administrative support during this study. No additional compensation was provided.