Open Access
Article

Can commercial automated immunoassays be utilized to predict neutralizing antibodies after SARS-CoV-2 infection? A comparative study between three different assays

Ahmed Ismail1,Farah M. Shurrab2,Hadeel T. Al-Jighefee2,3,Duaa W. Al-Sadeq2,4,Hamda Qotba5,6,Ibrahim Abdu Al-Shaar1,Hadi. M. Yassine2,3,Laith J. Abu-Raddad7,8,9,Gheyath K. Nasrallah2,3,*
1
Laboratory Section, Medical Commission Department, Ministry of Public Health, 42 Doha, Qatar
2
Biomedical Research Center, Qatar University, 2713 Doha, Qatar
3
Department of Biomedical Science, College of Health Sciences, Member of QU Health, Qatar University, 2713 Doha, Qatar
4
College of Medicine, Member of QU Health, Qatar University, 2713 Doha, Qatar
5
Department of Clinical Research, Primary Health Care Centers, 26555 Doha, Qatar
6
Department of Pathology, Sidra Medicine, 26999 Doha, Qatar
7
Infectious Disease Epidemiology Group, Weill Cornell Medicine-Qatar, Cornell University, Qatar Foundation—Education City, 24144 Doha, Qatar
8
World Health Organization Collaborating Centre for Disease Epidemiology Analytics on HIV/AIDS, Sexually Transmitted Infections, and Viral Hepatitis, Weill Cornell Medicine—Qatar, Cornell University, Qatar Foundation—Education City, 24144 Doha, Qatar
9
Department of Healthcare Policy and Research, Weill Cornell Medicine, Cornell University, New York, NY 10065, USA
DOI: 10.52586/4934 Volume 26 Issue 7, pp.198-206
Submited: 19 April 2021 Revised: 22 May 2021
Accepted: 04 June 2021 Published: 30 July 2021
*Corresponding Author(s):  
Gheyath K. Nasrallah
E-mail:  
gheyath.nasrallah@qu.edu.qa
Copyright: © 2021 The author(s). Published by BRI. This is an open access article under the CC BY 4.0 license (https://creativecommons.org/licenses/by/4.0/).
Abstract

Background: High-throughput assays that can infer neutralizing activity against SARS-CoV-2 are of great importance for assessing the immunity induced by natural infection and COVID-19 vaccines. We aimed to evaluate the performance and degree of correlation of three fully automated anti-SARS-CoV-2 immunoassays with neutralization activity using a surrogate virus-neutralizing test (sVNT) from GenScript, targeting the receptor-binding domain. Methods: 110 sera collected from PCR-confirmed asymptomatic COVID-19 individuals were tested for neutralizing antibodies (nAbs) using the sVNT. Positive samples were tested on three automated immunoassays targeting different viral antigens: Mindray CL-900i®, Abbott Architect, and Ortho VITROS®. The diagnostic sensitivity, specificity, agreement, and correlation with the sVNT were assessed. Receiver operating characteristic (ROC) curve analysis was performed to determine optimal thresholds for predicting the presence of neutralizing activity by each assay. Results: All three assays showed 100% specificities. The highest sensitivity was 99.0%, demonstrated by VITROS®, followed by 94.3%, for CL-900i®, and 81.0%, for Architect. Both VITROS® and CL-900i® had the strongest correlation with the sVNT (ρ = 0.718 and ρ = 0.712, respectively), while Architect showed a moderate correlation (ρ = 0.618). ROC curve analysis indicated that the manufacturer’s recommended cutoff values are adequate for predicting the presence of nAbs and providing a strong correlation with the sVNT. Conclusion: VITROS® and CL-900i® serological assays, which detect antibodies against SARS-CoV-2 spike protein, could serve as reliable assays to predict neutralization activity after infection or vaccination.

Key words

SARS-CoV-2; COVID-19; Serology; Automated immunoassay; CLIA; Neutralizing antibodies; Surrogate virus neutralization test (sVNT)

2. Introduction

Following the emergence of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in China, the infectious agent of Coronavirus Disease 2019 (COVID-19), the world faced a global health crisis with over 121 million confirmed cases and 2.68 million deaths worldwide [1]. Since the release of the SARS-CoV-2 genome sequence in January 2020, all efforts were directed towards the development of effective anti-viral drugs and vaccines [2]. Up to date, more than 180 vaccines have been developed and many of which have moved into phase III of clinical trials and received the emergency use authorization (EUA) by the U.S. Food and Drug Administration (FDA) [3]. Consequently, mass immunization campaigns were launched worldwide to ease some of the restrictions that were enforced by COVID-19. Even so, a substantial ongoing surge in cases and fatalities was still seen across many countries, indicating that the rollout of vaccination probably came out too late in these countries [4].

Neutralizing antibodies (nAbs) against SARS-CoV-2 play a key role in the quest for protective immunity and have been the reference method for assessing protection against several viruses after vaccination, including smallpox, polio, and influenza viruses [5, 6, 7, 8]. The antibody immune responses against SARS-CoV-2 have been extensively described in literature, where the limited pre-existing immunity was suggested to account for the remarkable rise in cases worldwide [4]. It was shown in several studies that SARS-CoV-2 infected individuals develop protective humoral immunity that persists for at least six months following infection [9, 10, 11, 12]. Although not suited for early diagnosis, serological assays have been widely used to detect antibody responses against SARS-CoV-2 and identify the disease prevalence in populations [1, 13]. Also, serological assays are important for measuring the efficacy of containment measures and screening convalescent sera for therapeutic and prophylactic purposes [12].

The clinical performance of the commercially available serology assays detecting Abs against SARS-CoV-2 has been extensively evaluated by numerous studies [12, 14, 15, 16]. In these studies, immunoassays that target the spike (S) protein of SARS-CoV-2 or the receptor-binding domain (RBD) have shown the best performance in terms of sensitivity and specificity [10]. However, measuring the antibody response against the SARS-CoV-2 is not sufficient for assessing protective immunity since most of these Abs are binding Abs that do not neutralize the virus [17]. On the other hand, nAbs are more indicative of protective immunity as they block the interaction between the virus and the host cells [18]. Methods for detecting nAbs include virus neutralization assays (VNAs) which are the gold standard for measuring protective immunity following infection or vaccination [17, 18, 19, 20]. However, VNAs are cumbersome, time-consuming, require biosafety level 3 (BSL-3) facilities, and cannot be implemented in routine practice [21]. Thus, correlation studies between high-throughput commercial SARS-CoV-2 serology platforms with nAbs are needed. In this regard, few studies have investigated the correlation between VNAs and SARS-CoV-2 immunoassays [9, 16, 22, 23, 24]. A recent study comparing six SARS-CoV-2 immunoassays with microneutralization assay showed variable performances depending on the target antigen with a sensitivity and specificity ranging between 43.8–87.8% and 68.3–97.5%, respectively. Thus, due to variability of reported performances among the assays, further investigation is needed to accurately identify reliable assays that provide the best correlation with neutralizing activity against SARS-CoV-2. Such assays are crucial to evaluate the effectiveness of the currently approved vaccines, determine the durability of the produced nAbs, and identify whether the vaccines are effective against the recently emerged SARS-CoV-2 variants [25, 26].

In this study, we aimed to evaluate the performance characteristics of three fully automated immunoassays (CL-900i®, Architect, and VITROS®) targeting different SARS-CoV-2 antigens in comparison with a surrogate virus-neutralizing test (sVNT) that detects nAbs against SARS-CoV-2 RBD.

3. Materials and methods

3.1 Study design, ethical approval, and clinical samples

The performance of the following three CE-marked SARS-CoV-2 chemiluminescent immunoassays was evaluated: CL-900i® which detects IgG Abs against viral S and nucleocapsid (N) proteins; Architect i4000SR which detects IgG Abs against the N protein; and VITROS® which detects total Abs (IgG, IgA, and IgM) against S1 antigen. The sera used were collected from volunteer individuals between July 26 and September 9, 2020, as a nationwide survey sub-study [27]. The study was approved by the Institutional Review Boards at Qatar University (QU-IRB 1492-E/21).

A total of 110 sera samples from RT-PCR-confirmed asymptomatic individuals were used to determine the sensitivity of each immunoassay was assessed in comparison to the sVNT. Details regarding the demographic and clinical characteristics of the participants are shown in Supplementary Table 1. Nasopharyngeal swab specimens from all participants were tested for SARS-CoV-2 using the Superscript III One-Step RT-PCR reaction mix with PlatinumR Taq DNA polymerase (Cat No. 12594100, ThermoFisher, Waltham, MA, USA) after RNA extraction using Qiagen RNA extraction kit. Each sample was tested using three sets of primers: one set targeting the E gene for screening and the other two sets targeting the RdRp gene for confirmation as described in [28]. Quant Studio 6 Flex real-time PCR System was used, and cycle threshold (CT) values below 30 were considered positive, while CT values between 30–33 were considered reactive.

To determine the specificity of the immunoassays and investigate cross-reactivity, we used a group of 70 pre-pandemic plasma samples collected from blood donors before 2019 and used in previous studies we conducted [12, 14, 29, 30, 31, 32]. The panel comprised of plasma samples seropositive for (a) dengue virus (n = 13), (b) influenza (n = 15), (c) West Nile Virus and Parvovirus B19 (n = 9), (d) non-respiratory viruses (n = 29) and (e) antinuclear antibodies (ANAs) (n = 4). More details about the characteristics of the pre-pandemic control samples are shown in Supplementary Table 2.

3.2 Automated chemiluminescent immunoassays

Three commercial automated serology platforms from different companies were used to detect anti-SARS-CoV-2 Abs in COVID-19 patients and pre-pandemic sera samples. These assays are: (i) CL-900i® SARS-CoV-2 IgG (Mindray, Shenzhen, China), (ii) Architect SARS-CoV-2 IgG (Abbott Laboratories, USA), and (iii) VITROS® Anti-SARS-CoV-2 Total Ab (Ortho Clinical Diagnostics, USA). All tests were carried out according to the manufacturers’ instructions. The characteristics of the immunoassays, including detection method, targeted antigens, detected antibody, and result interpretation, are summarized in Table 1 (Ref. [33, 34, 35, 36, 37, 38]).

Table 1: Characteristics of the automated analyzers used for anti-SARS-CoV-2 antibodies detection.
Manufacturer Immunoassay name Automated system Detection method/assay type Detected antibody Targeted SARS-CoV-2 antigen (s) Result interpretation Reference
Mindray Bio-Medical Electronics Co., Ltd. CL-900i® SARS-CoV-2 IgG CL-900i® system CLIA* IgG S and N* <10 U/mL: Negative [33, 34]
10 AU/mL: Positive
Abbott Laboratories Architect SARS-CoV-2 IgG ARCHITECT® i4000SR CMIA* IgG N <1.4 S/C Negative [35, 36, 37]
1.4 S/C: Positive
Ortho Clinical Diagnostics VITROS® Anti-SARS-CoV-2 Total Ab VITROS® ECiQ CLIA IgG, IgM, and IgA S (S1 subunit) * <1.0 S/C: Negative [38]
1.0 S/C: Positive
*CLIA, chemiluminescence immunoassay; CMIA, chemiluminescent microparticle immunoassay; S: spike protein; N: nucleocapsid protein; S1: subunit of the spike protein.

3.3 Neutralization assay (sVNT)

A SARS-CoV-2 surrogate virus neutralization test (sVNT) was used as a reference test in this study (Cat. No. L00847, GenScript Biotech, NJ, USA) for detecting nAbs against SARS-CoV-2. This assay utilizes the same format as enzyme-linked immunosorbent assay (ELISA) by detecting Abs that competitively inhibit the interaction between recombinant SARS-CoV-2 RBD-HRP fusion protein and recombinant ACE2 protein coated on a 96-well plate [39]. The assay was shown to have a high degree of correlation with the conventional pseudovirus neutralization test (pVNT, R2 = 0.84) and demonstrated high specificity (99.9%) and sensitivity (95.0–100%) [21]. In this study, all sera samples were tested for nAbs against the RBD using the sVNT. According to the manufacturer’s instructions, a percent inhibition of 20% was considered positive (nAbs were detected), and a percent inhibition of <20% was considered negative (nAbs were not detected).

3.4 Statistical analysis

Using the GenScript sVNT as the reference standard, sensitivity, specificity, overall percent agreement, and Cohen’s Kappa coefficient were calculated to assess the performance of each automated immunoassay. Cohen’s kappa coefficient (κ) is a robust statistical measure of agreement used to test inter-rater reliability and to assess the possibility of agreement occurrence by chance [40, 41]. Kappa value can be interpreted as follows: a value of 0 indicates no agreement, 0.01–0.20 is a poor agreement, 0.21–0.40 is a fair agreement, 0.41–0.60 is a moderate agreement, 0.61–0.80 is substantial agreement, and 0.81–1.00 is an almost perfect agreement [40]. Correlation and linear regression analysis between each automated immunoassay and the sVNT percent inhibition were performed. Non-parametric Spearman’s correlation coefficient (ρ) was calculated with 95% confidence interval (95% CI), where a coefficient of <0.3 suggests no or negligible correlation, 0.3–0.5 is a weak correlation, 0.5–0.7 is a moderate correlation, 0.7–0.9 is a strong correlation, and >0.9 is a very strong correlation [42]. Receiver operating characteristic (ROC) curve analysis and Youden index were used to assess the assays thresholds (cutoff indices) and identify optimized ones. A nonparametric ROC analysis was performed for each immunoassay to calculate the area under the curve (AUC). The bigger the AUC, the more accurate a tool in terms of diagnostic performance. The relation between AUC and diagnostic accuracy applies as follows: an AUC of <0.5 suggests no discrimination (ability to diagnose patients with and without the disease or condition based on the test), 0.7–0.8 is deemed to be acceptable, 0.8–0.9 is deemed excellent, and >0.9 is deemed outstanding [43, 44]. Youden’s index is often used as a summary measure of the ROC curve to evaluate the overall discriminative power of a diagnostic procedure and to compare it with other tests [45]. Youden’s index (J) was calculated using the formula: J = max (sensitivity + specificity) – 1 to help determine the optimal thresholds for each assay [46, 47]. All statistical analyses were performed using Microsoft Excel 365 software and GraphPad Prism software (Version 9.0.0, San Diego, CA, USA).

4. Results

4.1 Sample selection and nAbs screening using sVNT

A total of 110 samples from SARS-CoV-2-confirmed asymptomatic individuals and the 70-pre-pandemic samples were screened for the presence of nAbs using the sVNT. Out of the 110 samples, 105 were positive (95.5%). All pre-pandemic samples were negative, indicating a 100% specificity of the sVNT assay. Then, the positive 105 samples were used to evaluate the performance and correlation of the three immunoassays in comparison with the sVNT.

4.2 Diagnostic performance of the immunoassays using sVNT as a reference test

The diagnostic performance of each automated immunoassay using the sVNT as a reference test is summarized in Table 2. The specificity of all three assays was 100% (95% CI: 94.8–100.0). Architect showed the lowest sensitivity in detecting Abs that correlates with neutralizing activity at 81.0% (95% CI: 77.2–84.7), followed by CL-900i® at 94.3% (95% CI: 89.8–98.7), and VITROS® which showed the highest sensitivity at 99.0% (95% CI: 93.8–100.0). The overall agreement with the sVNT ranged from 88.6% (95% CI: 83.9–93.3, kappa = 0.77), for Architect, to 99.3% (95% CI: 97.9–100.0, kappa = 0.98) for VITROS®. CL-900i® showed an overall agreement of 96.6% (95% CI: 93.9–99.3, kappa = 0.93).

Table 2: Diagnostic assessment of the three automated SARS-CoV-2 immunoassays using sVNT as a reference test for nAbs detection.
Automated assay Mindray CL-900i® SARS-CoV-2 IgG Abbott architect SARS-CoV-2 IgG VITROS® anti-SARS-CoV-2 total test
Sensitivity % (95% CI) 94.3 (89.8–98.7) 81.0 (77.2–84.7) 99.0 (93.8–100.0)
Specificity % (95% CI) 100.0 (94.8–100.0) 100.0 (94.8–100.0) 100.0 (94.8–100.0)
Overall agreement % (95% CI) 96.6 (93.9–99.3) 88.6 (83.9–93.3) 99.3 (97.9–100.0)
Cohen’s kappa coefficient κ (95% CI) 0.93 (0.89–0.97) 0.77 (0.71–0.84) 0.98 (0.96–1.0)
ROC curves optimized cut-off index >7.860 >0.8650 >0.91
Sensitivity using optimized cut-off indices % (95% CI) 98.1 (93.3–99.7) 87.6 (79.9– 92.6) 99.0 (94.7–99.9)
Specificity using optimized cut-off indices % (95% CI) 100.0 (94.8–100.0) 100.0 (94.8–100.0) 100.0 (90.4–100.0)

The correlation and linear regression analysis between the readings obtained from each automated immunoassay and neutralizing activity (percent inhibition) from the sVNT are illustrated in Fig. 1. Spearman’s correlation coefficients (ρ) showed a statistically significant positive correlation for all three automated assays with the sVNT (p < 0.001). The strongest correlation was shown by immunoassays targeting the S protein; VITROS® (ρ = 0.718, Fig. 1C), followed by CL-900i® (ρ = 0.712, Fig. 1A). Architect immunoassay, which targets the N protein, demonstrated the weakest correlation compared to the other two assays (ρ = 0.618, Fig. 1B). Linear regression analysis showed that all constructed models could statistically significantly predict the dependent variable (% inhibition) based on the cutoff index (COI) generated by each immunoassay with a predication precision ranging between 13.9–15.6% (standard error of estimate, SEE). The best regression model fitting the data was shown by VITROS® (r2 = 403, SEE = 13.9%), followed by Architect (r2 = 0.308, SEE = 15.1%) and Mindray (r2 = 0.275, SEE = 15.6%).

. Spearman’s correlation coefficient (

Figure 1: Correlation and linear regression analysis between each SARS-CoV-2 immunoassay ratio results and the surrogate virus neutralization test (sVNT) percent inhibition in asymptomatic COVID-19 individuals (n = 105). Spearman’s correlation coefficient (ρ), r2, and p-value are shown for each model. Rho (ρ) is shown with 95% confidence interval (95% CI), a correlation value of <0.3 is negligible, 0.3–0.5 is weak, 0.5–0.7 is moderate, 0.7–0.9 is strong and >0.9 is very strong [34]. The dashed red line corresponds to the cutoff value of each assay (CL-900i® 10.0, Architect 1.4, VITROS® 1.0). The significance level was set at 0.05. COI, cutoff index; S/C, signal/cutoff.

4.3 Receiver operating characteristics (ROC) curve analysis

As shown in Fig. 2, ROC curve analyses showed that all three assays had an AUC that exceeded 0.99, indicating an excellent performance for all assays (CL-900i®: 0.9976, Architect: 0.9966, VITROS®: 0.9997, p < 0.0001). Optimized thresholds (cutoff indices) for detecting anti-SARS-CoV-2 Abs that correlate with the sVNT were obtained based on these ROC curves and the calculated Youden’s index. The derived cut-off indices were >7.860, >0.8650, and >0.91 for CL-900i®, Architect, and VITROS®, respectively, compared to the manufacturer’s suggested cut-offs which were 10.0, 1.4, and 1.0, respectively. By applying these new cutoff values, all assays showed an improved sensitivity (CL-900i®: 98.1%, Architect: 87.6%), except for VITROS®, which maintained the same sensitivity (99.0%). The new cutoff values did not affect the specificity of the assays, which remained at 100%.

. The sensitivity and specificity values correspond to the plotted points in the graphs which were used to calculate the area under the curve (AUC) and

Figure 2: Empirical Receiver Operating Characteristic (ROC) curve analysis for each automated immunoassay to estimate the optimal threshold levels for predicting the presence of neutralizing antibodies against SARS-CoV-2 in asymptomatic COVID-19 individuals (n = 105). The sensitivity and specificity values correspond to the plotted points in the graphs which were used to calculate the area under the curve (AUC) and p-value for each curve plot. Based on the area under the ROC curve, the Youden Index cutoff values that maximize the sum of sensitivity and specificity were determined. The significance level was set at 0.05.

5. Discussion

The ongoing COVID-19 pandemic requires high-throughput serological assays that can provide strong correlations with nAbs against SARS-CoV-2 and enable studying the immunity induced by natural infection and vaccines. Although conventional virus neutralization tests (cVNT) are the gold standard for detecting nAbs, they are cumbersome, time-consuming, require BSL-3 facilities, and cannot be implemented in routine practice [21]. On the other hand, sVNT is a quick and straightforward test that does not require specialized facilities and has shown an excellent performance compared with the cVNT in several studies [21, 47, 48]. However, this assay is not available yet in high throughput automated format for screening large populations compared to commercial automated assays. Therefore, we aimed to evaluate the performance of three automated immunoassays (CL-900i®, Architect, and VITROS®) using the GenScript sVNT as a reference test. Also, we aimed to assess the correlation of these assays with neutralizing activity against SARS-CoV-2. A panel of 110 samples collected from RT–PCR confirmed individuals and 70 pre-pandemic sera were used.

Following the initial screening of all samples using the sVNT, nAbs were detectable in 95.5% (105/110) of the sera samples. The sVNT did not show any cross-reactivity with the pre-pandemic samples, neither did any of the three automated assays, with 100% specificities. VITROS® demonstrated the highest sensitivity (99.0%), followed by CL-900i® (94.3%) and Architect (81.0%). These findings were concordant with a previous study that reported a 100% sensitivity for VITROS® using a microneutralization test as a reference test [49]. A possible explanation of the superior sensitivity demonstrated by VITROS® could be attributed to the fact that it detects the total Abs (IgM, IgA, and IgG) against the virus, compared to the other two assays (CL-900i® and Architect) that only detect IgG Abs. Thus, targeting multiple types of Abs could increase the chance of detecting Abs that correlate with neutralizing activity, and enhance the assay’s sensitivity. Further, although a different sample cohort was used, the sensitivity obtained by CL-900i® was comparable to the one reported in a previous study we conducted (~96%), at which the performance of three immunoassays was assessed using the sVNT test [15]. In line with our findings, Architect also showed a comparable performance in another study that reported a sensitivity of 80.5% using a microneutralization assay as the reference test [49, 50].

The correlation between the ratios obtained from each automated assay and the percent inhibition of nAbs targeting the RBD was also examined. Both CL-900i® and VITROS® demonstrated a strong correlation with the sVNT compared to Architect, which showed a moderate correlation (Fig. 1). This could be because CL-900i® detects Abs against both the S and N proteins, while VITROS® solely targets Abs against S1 subunit, which also contains the RBD. Similar studies have shown that the RBD is a potent target for nAbs and that serology assays that detect Abs against the S1 subunit or the RBD alone strongly correlate with neutralization activity [21, 48, 51, 52, 53]. A study comparing the performance of five serology assays with neutralization test showed that assays targeting the S protein including DiaSorin CLIA, Beckman Coulter CLIA, and Euroimmun ELISA had the highest correlation coefficients (rho = 0.72, 0.68, and 0.63, respectively), similar to our findings (Fig. 1). On the other hand, the correlation between Architects and nAbs was weaker due to the fact it only targets Abs against the N protein. An earlier study also demonstrated a modest correlation between nAb titers and immunoassays detecting Abs against the N protein [51]. In that study, Pearson correlation coefficients were calculated between SARS-CoV-2 neutralization titers (EC50) and the ratios reported by Roche and Abbot immunoassays. The correlation coefficient was 0.29 for Roche and 0.47 for Abbot, denoting a moderate correlation with nAbs for both assays.

ROC curve analysis was performed to determine the optimal cutoff indices for each automated immunoassay that could predict the presence of nAbs against SARS-CoV-2 (Fig. 2). The calculated cutoff values showed that the manufacturer’s recommended thresholds are adequate for predicting the presence of nAbs and providing a strong correlation with the sVNT. Further, these cutoff indices can be adjusted depending on the clinical setting or research context in which they can be lowered to improve the sensitivity without affecting the specificity in high prevalence settings.

Our study had some limitations, including the small sample size and the fact that all samples were collected from asymptomatic male individuals. However, these samples were collected as part of a previous population-based study we conducted on the craft and manual workers in Qatar [27] who represent an important section of Qatar by comprising 60% of the total population. Also, the selected samples were obtained from various time points (7–35 days) after the positive RT-PCR test to account for variability and delay of elicited immune response among different individuals. Further, other studies have shown that male individuals produce weaker immune responses, particularly nAbs, than females [54, 55]. Hence, identifying the thresholds of protective immunity in male individuals is essential to avoid false-negative results and accurately estimate herd immunity. Another limitation is that our pre-pandemic sera samples did not include seropositive samples for other human coronaviruses, potentially leading to an overestimated specificity. However, our control group did include Dengue and ANA seropositive samples which have been reported to cause cross-reactivity with SARS-CoV-2 in other studies [56, 57].

In conclusion, our findings showed that both CL-900i® and VITROS® immunoassays demonstrated excellent performance in terms of sensitivity, specificity, and overall correlation with the sVNT, suggesting that they could serve as reliable and high-throughput assays for predicting the presence of protective nAbs. The identification of such reliable assays is of particular importance following the mass vaccination campaigns taking place worldwide. They could be used for screening vaccinated populations and recovered COVID-19 patients to measure the effectiveness of the developed vaccines and ensure efficient herd immunity.

6. Author contributions

Conceptualization: GKN and LJA, HMY; Methodology: AI, FMS, HAJ, DWA, IAA; Formal Analysis: HAJ; Validation: GKN, HAJ, FMS; Investigation: AI, FMS, HQ, GKN Resources: GKN; Data Curation: AI, GKN, HAJ, FMS; Writing—Original Draft Preparation: AI, FMS, HAJ; Writing, Review & Editing: HMY, GKN, LJA, AI, HQ; Visualization: HAJ, FMS, GKN; Supervision: GKN, DWA; Project Administration: GKN, DWA; Funding Acquisition: GKN. All authors have read and agreed to the published version of the manuscript.

7. Ethics approval and consent to participate

This project was approved by the Institutional Review Boards at QU (QU-IRB 1492-E/21).

8. Acknowledgment

We would like to thank the medical technologist at the Medical Commission, Qatar for their help in performing the Vitros and the Architect immunoassay.

9. Funding

This research was funded by Qatar National Research Fund (a member of Qatar Foundation), grant No. RRC-2-032 and Qatar University internal grant QUERG-CMED-2020-2.

10. Conflict of interest

The authors declare no conflict of interest.

Supplementary material

Supplementary material associated with this article can be found, in the online version, at https://www.fbscience.com/Landmark/articles/10.52586/4934.

Abbreviations

sVNT, surrogate virus-neutralizing test; ROC, Receiver operating characteristic; nAb, neutralizing antibodies; RBD, receptor-binding domain; COVID-19, Coronavirus Disease 2019; VNAs, virus neutralization assays; BSL-3, biosafety level 3; CT, Cycle threshold; CLIA, chemiluminescence immunoassay; CMIA, chemiluminescent microparticle immunoassay; ELISA, enzyme-linked immunosorbent assay; AUC, area under the curve; RT-PCR, Real-time polymerase chain reaction.

References
  • [1] Carbone M, Lednicky J, Xiao S, Venditti M, Bucci E. Coronavirus 2019 Infectious Disease Epidemic: where we are, what can be done and Hope for. Journal of Thoracic Oncology. 2021; 16: 546–571.
  • [2] Kyriakidis NC, López-Cortés A, González EV, Grimaldos AB, Prado EO. SARS-CoV-2 vaccines strategies: a comprehensive review of phase 3 candidates. NPJ Vaccines. 2021; 6: 28.
  • [3] Krammer F. SARS-CoV-2 vaccines in development. Nature. 2020; 586: 516–527.
  • [4] Post N, Eddy D, Huntley C, van Schalkwyk MCI, Shrotri M, Leeman D, et al. Antibody response to SARS-CoV-2 infection in humans: A systematic review. PLoS ONE. 2021; 15: e0244126.
  • [5] Jiang S, Hillyer C, Du L. Neutralizing Antibodies against SARS-CoV-2 and other Human Coronaviruses. Trends in Immunology. 2020; 41: 355–359.
  • [6] Zinkernagel RM. On natural and artificial vaccinations. Annual Review of Immunology. 2003; 21: 515–546.
  • [7] Fenner F. Mouse-pox (infectious ectromelia of mice): a review. Journal of Immunology. 1949; 63: 341–373.
  • [8] Sabin AB. Oral Poliovirus Vaccine: History of its Development and Use and Current Challenge to Eliminate Poliomyelitis from the World. Journal of Infectious Diseases. 1985; 151: 420–436.
  • [9] Duysburgh E, Mortgat L, Barbezange C, Dierick K, Fischer N, Heyndrickx L, et al. Persistence of IgG response to SARS-CoV-2. Lancet Infectious Diseases. 2021; 21: 163–164.
  • [10] Dan JM, Mateus J, Kato Y, Hastie KM, Faliti C, Ramirez SI, et al. Immunological memory to SARS-CoV-2 assessed for greater than six months after infection. BioRxiv. 2021; 371: eabf4063.
  • [11] Abu-Raddad LJ, Chemaitelly H, Coyle P, Malek JA, Ahmed AA, Mohamoud YA, et al. SARS-CoV-2 reinfection in a cohort of 43,000 antibody-positive individuals followed for up to 35 weeks. MedRxiv. 2021.
  • [12] Al-Jighefee HT, Yassine HM, Nasrallah GKJP. Evaluation of antibody response in symptomatic and asymptomatic covid-19 patients and diagnostic assessment of new IgM/IgG elisa kits. Pathogens. 2021; 10: 161.
  • [13] Padoan A, Bonfante F, Pagliari M, Bortolami A, Negrini D, Zuin S, et al. Analytical and clinical performances of five immunoassays for the detection of SARS-CoV-2 antibodies in comparison with neutralization activity. EBioMedicine. 2020; 62: 103101.
  • [14] Yassine HM, Al-Jighefee H, Al-Sadeq DW, Dargham SR, Younes SN, Shurrab F, et al. Performance evaluation of five ELISA kits for detecting anti-SARS-COV-2 IgG antibodies. International Journal of Infectious Diseases. 2021; 102: 181–187.
  • [15] Younes S, Al-Jighefee H, Shurrab F, Al-Sadeq DW, Younes N, Dargham SR, et al. Diagnostic Efficiency of Three Fully Automated Serology Assays and Their Correlation with a Novel Surrogate Virus Neutralization Test in Symptomatic and Asymptomatic SARS-COV-2 Individuals. Microorganisms. 2021; 9: 245.
  • [16] Patel EU, Bloch EM, Clarke W, Hsieh Y, Boon D, Eby Y, et al. Comparative Performance of Five Commercially Available Serologic Assays to Detect Antibodies to SARS-CoV-2 and Identify Individuals with High Neutralizing Titers. Journal of Clinical Microbiology. 2020; 2020.08.31.20184788.
  • [17] Wu F, Wang A, Liu M, Wang Q, Chen J, Xia S, et al. Neutralizing antibody responses to SARS-CoV-2 in a COVID-19 recovered patient cohort and their implications. MedRxiv. 2020.
  • [18] Bisht H, Roberts A, Vogel L, Subbarao K, Moss B. Neutralizing antibody and protective immunity to SARS coronavirus infection of mice induced by a soluble recombinant polypeptide containing an N-terminal segment of the spike glycoprotein. Virology. 2005; 334: 160–165.
  • [19] Wang S, Liao H, Zhang J, Cheng TR, Wong C. Development of a universal influenza vaccine using hemagglutinin stem protein produced from Pichia pastoris. Virology. 2019; 526: 125–137.
  • [20] Wang X, Guo X, Xin Q, Pan Y, Hu Y, Li J, et al. Neutralizing Antibody Responses to Severe Acute Respiratory Syndrome Coronavirus 2 in Coronavirus Disease 2019 Inpatients and Convalescent Patients. Clinical Infectious Diseases. 2020; 71: 2688–2694.
  • [21] Tan CW, Chia WN, Qin X, Liu P, Chen MI, Tiu C, et al. A SARS-CoV-2 surrogate virus neutralization test based on antibody-mediated blockage of ACE2–spike protein–protein interaction. Nature Biotechnology. 2020; 38: 1073–1078.
  • [22] Dogan, M., et al., SARS-CoV-2 specific antibody and neutralization assays reveal the wide range of the humoral immune response to virus. Communications Biology, 2021. 4(1): p. 129.
  • [23] Valdivia A, Torres I, Latorre V, Francés-Gómez C, Albert E, Gozalbo-Rovira R, et al. Inference of SARS-CoV-2 spike-binding neutralizing antibody titers in sera from hospitalized COVID-19 patients by using commercial enzyme and chemiluminescent immunoassays. European Journal of Clinical Microbiology & Infectious Diseases. 2021; 40: 485–494.
  • [24] Suhandynata RT, Hoffman MA, Huang D, Tran JT, Kelner MJ, Reed SL, et al. Commercial Serology Assays Predict Neutralization Activity against SARS-CoV-2. Clinical Chemistry. 2021; 67: 404–414.
  • [25] Zhou G, Zhao Q. Perspectives on therapeutic neutralizing antibodies against the Novel Coronavirus SARS-CoV-2. International Journal of Biological Sciences. 2020; 16: 1718–1723.
  • [26] Sewell HF, Agius RM, Stewart M, Kendrick D. Cellular immune responses to COVID-19. British Medical Journal Publishing Group. 2020; 370: m3018.
  • [27] Al-Thani MH, Farag E, Bertollini R, Al Romaihi HE, Abdeen S, Abdelkarim A, et al. Seroprevalence of SARS-CoV-2 infection in the craft and manual worker population of Qatar. MedRxiv. 2020.
  • [28] Corman VM, Landt O, Kaiser M, Molenkamp R, Meijer A, Chu DK, et al. Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Eurosurveillance. 2020; 25: 2000045.
  • [29] Nasrallah GK, Dargham SR, Sahara AS, Elsidiq MS, Abu-Raddad LJJJoCV. Performance of four diagnostic assays for detecting herpes simplex virus type 2 antibodies in the Middle East and North Africa. Journal of Clinical Virology. 2019; 111: 33–38.
  • [30] Al Kahlout RA, Nasrallah GK, Farag EA, Wang L, Lattwein E, Müller MA, et al. Comparative Serological Study for the Prevalence of Anti-MERS Coronavirus Antibodies in High- and Low-Risk Groups in Qatar. Journal of Immunology Research. 2019; 2019: 1–8.
  • [31] Smatti MK, Nasrallah GK, Al Thani AA, Yassine HM. Measuring influenza hemagglutinin (HA) stem-specific antibody-dependent cellular cytotoxicity (ADCC) in human sera using novel stabilized stem nanoparticle probes. Vaccine. 2020; 38: 815–821.
  • [32] Nasrallah GK, Dargham SR, Mohammed LI, Abu-Raddad LJ. Estimating seroprevalence of herpes simplex virus type 1 among different Middle East and North African male populations residing in Qatar. Journal of Medical Virology. 2018; 90: 184–190.
  • [33] Nuccetelli M, Pieri M, Grelli S, Ciotti M, Miano R, Andreoni M, et al. SARS-CoV-2 infection serology: a useful tool to overcome lockdown? Cell Death Discovery. 2020; 6: 38.
  • [34] IgG(CLIA) MS-C-. Mindray. SARS-CoV-2 IgG(CLIA). 2020. Available at: https://www.mindray.com/en/product/CL-900i.html (Accessed: 2 April 2021).
  • [35] Bryan A, Pepper G, Wener MH, Fink SL, Morishima C, Chaudhary A, et al. Performance Characteristics of the Abbott Architect SARS-CoV-2 IgG Assay and Seroprevalence in Boise, Idaho. Journal of Clinical Microbiology. 2020; 58: e00941-20.
  • [36] Chew KL, Tan SS, Saw S, Pajarillaga A, Zaine S, Khoo C, et al. Clinical evaluation of serological IgG antibody response on the Abbott Architect for established SARS-CoV-2 infection. Clinical Microbiology and Infection. 2020; 26: 1256.e9–1256.e11.
  • [37] Meschi S, Colavita F, Bordi L, Matusali G, Lapa D, Amendola A, et al. Performance evaluation of Abbott ARCHITECT SARS-CoV-2 IgG immunoassay in comparison with indirect immunofluorescence and virus microneutralization test. Journal of Clinical Virology. 2020; 129: 104539.
  • [38] Use VIPA-S-C-TRP-If. VITROS Immunodiagnostic Products Anti-SARS-CoV-2 Total Reagent Pack—Instructions for Use. 2021. Available at: https://www.fda.gov/media/136967/download (Accessed: 5 April 2022).
  • [39] Meyer B, Reimerink J, Torriani G, Brouwer F, Godeke G, Yerly S, et al. Validation and clinical evaluation of a SARS-CoV-2 surrogate virus neutralisation test (sVNT). Emerging Microbes & Infections. 2020; 9: 2394–2403.
  • [40] McHugh ML. Interrater reliability: the kappa statistic. Biochemia Medica. 2012; 22: 276–282.
  • [41] Bendavid A. Comparison of classification accuracy using Cohen’s Weighted Kappa. Expert Systems with Applications. 2008; 34: 825–832.
  • [42] Mukaka MM. Statistics corner: A guide to appropriate use of correlation coefficient in medical research. Malawi Medical Journal. 2012; 24: 69–71.
  • [43] Mandrekar JN. Receiver Operating Characteristic Curve in Diagnostic Test Assessment. Journal of Thoracic Oncology. 2010; 5: 1315–1316.
  • [44] Schisterman EF, Faraggi D, Reiser B, Hu J. Youden Index and the optimal threshold for markers with mass at zero. Statistics in Medicine. 2008; 27: 297–315.
  • [45] Unal I. Defining an Optimal Cut-Point Value in ROC Analysis: an Alternative Approach. Computational and Mathematical Methods in Medicine. 2017; 2017: 1–14.
  • [46] Fluss R, Faraggi D, Reiser B. Estimation of the Youden Index and its Associated Cutoff Point. Biometrical Journal. 2005; 47: 458–472.
  • [47] Michiels J, Heyndrickx L, Nkuba-Ndaye A, Ceulemans A, Bartholomeeusen KK, Madinga J, et al. Evaluation of a surrogate virus neutralization test for high-throughput serosurveillance of SARS-CoV-2. MedRxiv. 2021.
  • [48] Valcourt EJ, Manguiat K, Robinson A, Chen JC, Dimitrova K, Philipson C, et al. Evaluation of a commercially-available surrogate virus neutralization test for severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2). Diagnostic Microbiology and Infectious Disease. 2021; 99: 115294.
  • [49] Walker GJ, Naing Z, Ospina Stella A, Yeang M, Caguicla J, Ramachandran V, et al. SARS Coronavirus-2 microneutralisation and commercial serological assays correlated closely for some but not all enzyme immunoassays. Viruses. 2021; 13: 247.
  • [50] Jääskeläinen AJ, Kuivanen S, Kekäläinen E, Ahava MJ, Loginov R, Kallio-Kokko H, et al. Performance of six SARS-CoV-2 immunoassays in comparison with microneutralisation. Journal of Clinical Virology. 2020; 129: 104512.
  • [51] Tang MS, Case JB, Franks CE, Chen RE, Anderson NW, Henderson JP, et al. Association between SARS-CoV-2 neutralizing antibodies and commercial serological assays. Clinical Chemistry. 2020; 66: 1538–1547.
  • [52] Hörber S, Soldo J, Relker L, Jürgens S, Guther J, Peter S, et al. Evaluation of three fully-automated SARS-CoV-2 antibody assays. Clinical Chemistry and Laboratory Medicine. 2020; 58: 2113–2120.
  • [53] Okba NMA, Müller MA, Li W, Wang C, GeurtsvanKessel CH, Corman VM, et al. Severe Acute Respiratory Syndrome Coronavirus 2-Specific Antibody Responses in Coronavirus Disease Patients. Emerging Infectious Diseases. 2020; 26: 1478–1488.
  • [54] Bunders MJ, Altfeld M. Implications of Sex Differences in Immunity for SARS-CoV-2 Pathogenesis and Design of Therapeutic Interventions. Immunity. 2020; 53: 487–495.
  • [55] Zhao G, Xu Y, Li J, Cui X, Tan X, Zhang H, et al. Sex differences in immune responses to SARS-CoV-2 in patients with COVID-19. Bioscience Reports. 2021; 41: BSR20202074.
  • [56] Masyeni S, Santoso MS, Widyaningsih PD, Asmara DW, Nainu F, Harapan H, et al. Serological cross-reaction and coinfection of dengue and COVID-19 in Asia: Experience from Indonesia. International Journal of Infectious Diseases. 2021; 102: 152–154.
  • [57] Vojdani A, Vojdani E, Kharrazian DJFiI. Reaction of human monoclonal antibodies to SARS-CoV-2 proteins with tissue antigens: Implications for autoimmune diseases. Frontiers in Immunology. 2021; 11: 3679.
Materials
Landmark/articles/materials/Landmark4934.docx
Share and Cite
Ahmed Ismail, Farah M. Shurrab, Hadeel T. Al-Jighefee, Duaa W. Al-Sadeq, Hamda Qotba, Ibrahim Abdu Al-Shaar, Hadi. M. Yassine, Laith J. Abu-Raddad, Gheyath K. Nasrallah. Can commercial automated immunoassays be utilized to predict neutralizing antibodies after SARS-CoV-2 infection? A comparative study between three different assays. Frontiers in Bioscience-Landmark. 2021. 26(7); 198-206.