Read on Twitter
We set out to identify the sensitivity & specificity of common tests for #COVID19 along w/ @dan_diekema , @Anthony98947615 w/ @CDCgov support
A simple enough task, right?
I’ve seen tweets by @DrSidMukherjee @drjohnm @BenMazer @PaulSaxMD @BradSpellberg and others interested
1/n
A simple enough task, right?
I’ve seen tweets by @DrSidMukherjee @drjohnm @BenMazer @PaulSaxMD @BradSpellberg and others interested
1/n
Looking for comments/criticism
What are we missing? No industry adverts please!
Important papers?
2/n
What are we missing? No industry adverts please!
Important papers?
2/n
The @US_FDA has a test comparison site that is incomprehensible to me… but @ASMicrobiology types tell me it reports on analytical sensitivity and LoD for tests
3/n
3/n
First, terms:
As a clinician, I want to know if a patient does or doesn’t have #covid19.
I’m less concerned by internal laboratory QA unless it translates
We wanted what has been called CLINICAL or diagnostic sensitivity/ specificity NOT analytical sen/sp.
4/n
As a clinician, I want to know if a patient does or doesn’t have #covid19.
I’m less concerned by internal laboratory QA unless it translates
We wanted what has been called CLINICAL or diagnostic sensitivity/ specificity NOT analytical sen/sp.
4/n
Described in 1997:
Clinical “diagnostic sensitivity is defined by the percentage of persons who have the disorder of interest who have positive results on the assay”
And more recently early in COVID by @akesselheim
https://www.nejm.org/doi/full/10.1056/NEJMp2015897#.X-DkelzIGHM.twitter
5/n
Clinical “diagnostic sensitivity is defined by the percentage of persons who have the disorder of interest who have positive results on the assay”
And more recently early in COVID by @akesselheim
https://www.nejm.org/doi/full/10.1056/NEJMp2015897#.X-DkelzIGHM.twitter
5/n
Another note, PCR has largely been the laboratory gold standard.
So, most reports are for sen/sp relative to PCR—not to a clinical gold standard.
We adjusted reported results to reflect:
e.g. PCR Sen 90% x Antigen Sen of 50% = overall antigen Sen of 45%
6/n
So, most reports are for sen/sp relative to PCR—not to a clinical gold standard.
We adjusted reported results to reflect:
e.g. PCR Sen 90% x Antigen Sen of 50% = overall antigen Sen of 45%
6/n
Finally, we are NOT talking about being infectious for #covid19.
That is another topic with even LESS data I hope to share soon.
Suffice to say, many people with #covid19 disease are NOT infectious, especially after a week or 10 days, but some tests still +, esp PCR
7/n
That is another topic with even LESS data I hope to share soon.
Suffice to say, many people with #covid19 disease are NOT infectious, especially after a week or 10 days, but some tests still +, esp PCR
7/n
PCR—the gold standard
Clinical sensitivity ~ 90% on day 4 of symptoms w/ common instruments
https://www.sciencedirect.com/science/article/pii/S0163445320305776
https://dx.plos.org/10.1371/journal.pone.0242958
https://jcm.asm.org/content/58/8/e00995-20
https://academic.oup.com/ofid/article-abstract/7/8/ofaa315/5876007#.X-DpCI8IKbk.twitter
8/n
Clinical sensitivity ~ 90% on day 4 of symptoms w/ common instruments
https://www.sciencedirect.com/science/article/pii/S0163445320305776
https://dx.plos.org/10.1371/journal.pone.0242958
https://jcm.asm.org/content/58/8/e00995-20
https://academic.oup.com/ofid/article-abstract/7/8/ofaa315/5876007#.X-DpCI8IKbk.twitter
8/n
-earlier reports of lower PCR sensitivity had atypical gold standard or were not reporting optimal sampling on day 4
-Sensitivity IS lower before/ after day 4 of symptoms per https://www.acpjournals.org/doi/10.7326/M20-1495#.X-DnQ9bVCk0.twitter
9/n
-Sensitivity IS lower before/ after day 4 of symptoms per https://www.acpjournals.org/doi/10.7326/M20-1495#.X-DnQ9bVCk0.twitter
9/n
Reasons for PCR false-negatives (10% false – rate)
lack of virus in sampling site
inadequate sampling
lack of instrument optimization or variation between instruments
10/n
lack of virus in sampling site
inadequate sampling
lack of instrument optimization or variation between instruments
10/n
PCR Clinical Specificity ~99%
this was hard!
Scant data, authors suggest between 95%-99.5%
https://www.bmj.com/content/369/bmj.m1808
https://www.gov.uk/government/publications/tfms-consensus-statement-on-mass-testing-27-august-2020
https://www.cdc.gov/csels/dls/locs/2020/report-false-negatives-and-false-positives-from-covid-19-testing.html
http://m.koreaherald.com/view.php?ud=20200429000724
https://jcm.asm.org/content/58/8/e00743-20
11/n
this was hard!
Scant data, authors suggest between 95%-99.5%
https://www.bmj.com/content/369/bmj.m1808
https://www.gov.uk/government/publications/tfms-consensus-statement-on-mass-testing-27-august-2020
https://www.cdc.gov/csels/dls/locs/2020/report-false-negatives-and-false-positives-from-covid-19-testing.html
http://m.koreaherald.com/view.php?ud=20200429000724
https://jcm.asm.org/content/58/8/e00743-20
11/n
Reasons for PCR false-positives (~1% false + rate)
-past infection with residual RNA
-differences in testing between instruments
-glitches in instrument reading of Ct/Cq values
-lack of laboratory optimization normally required by the FDA
-contamination
12/n
-past infection with residual RNA
-differences in testing between instruments
-glitches in instrument reading of Ct/Cq values
-lack of laboratory optimization normally required by the FDA
-contamination
12/n
Next, the Point of Care NAAT tests
Limited information published and in some ways very similar to lab based PCR
Abbott IDNow seems less sensitivity/more specific than Cepheid Xpert Xpress/Roche Cobas
13/n
Limited information published and in some ways very similar to lab based PCR
Abbott IDNow seems less sensitivity/more specific than Cepheid Xpert Xpress/Roche Cobas
13/n
POC NAAT tests
—Abbott IDNow
Clinical Sensitivity 54% / Clinical specificity 97.5%
(final numbers after adjustment for comparison to PCR)
Misses high CT value PCR + which are often NON-infectious. But no data vs. cell culture
https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD013705/information#.X-DtNgTrpF8.twitter
amazing work @deeksj
14/n
—Abbott IDNow
Clinical Sensitivity 54% / Clinical specificity 97.5%
(final numbers after adjustment for comparison to PCR)
Misses high CT value PCR + which are often NON-infectious. But no data vs. cell culture
https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD013705/information#.X-DtNgTrpF8.twitter
amazing work @deeksj
14/n
other POC NAAT sources
https://jcm.asm.org/content/58/8/e01136-20
https://jcm.asm.org/content/58/8/e00938-20
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7470790/
https://www.sciencedirect.com/science/article/pii/S073288932030585X
15/n
https://jcm.asm.org/content/58/8/e01136-20
https://jcm.asm.org/content/58/8/e00938-20
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7470790/
https://www.sciencedirect.com/science/article/pii/S073288932030585X
15/n
We estimated
Cepheid Xpert Xpress /Roche
Overall
Clinical sensitivity 90%
Clinical Specificity 95%
https://jcm.asm.org/content/58/8/e00772-20
https://pubmed.ncbi.nlm.nih.gov/32417674/
(and other refs for IDNow)
16/n
Cepheid Xpert Xpress /Roche
Overall
Clinical sensitivity 90%
Clinical Specificity 95%
https://jcm.asm.org/content/58/8/e00772-20
https://pubmed.ncbi.nlm.nih.gov/32417674/
(and other refs for IDNow)
16/n
Last, but not least...
Antigen tests! The great promise.
I would say useful but not great.
Overall
Clinical sensitivity 45%
Clinical specificity 97%
https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD013705/information#.X-DtNgTrpF8.twitter
https://www.sciencedirect.com/science/article/pii/S1386653220302420
https://www.sciencedirect.com/science/article/pii/S1386653220301979
17/n
Antigen tests! The great promise.
I would say useful but not great.
Overall
Clinical sensitivity 45%
Clinical specificity 97%
https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD013705/information#.X-DtNgTrpF8.twitter
https://www.sciencedirect.com/science/article/pii/S1386653220302420
https://www.sciencedirect.com/science/article/pii/S1386653220301979
17/n
False +s with Antigen tests are real
https://www.cleveland.com/coronavirus/2020/08/are-false-positives-from-antigen-tests-ratcheting-up-ohio-coronavirus-case-numbers-doctors-correct-testing-myths.html
http://dpbh.nv.gov/uploadedFiles/dpbhnvgov/content/Resources/Directive%20to%20Discontinue%20Use%20of%20Antigen%20POC_10.02.2020_ADA_Compliant.pdf
https://www.ahcancal.org/Data-and-Research/Center-for-HPE/Documents/Report-Discordant-COVID-Test-Results.pdf#search=Discordant%20Results%20between%20COVID%2D19%20Point%20of%20Care%20Antigen%20and%20PCR%20Tests%20in%20Nursing%20Homes
18/n
https://www.cleveland.com/coronavirus/2020/08/are-false-positives-from-antigen-tests-ratcheting-up-ohio-coronavirus-case-numbers-doctors-correct-testing-myths.html
http://dpbh.nv.gov/uploadedFiles/dpbhnvgov/content/Resources/Directive%20to%20Discontinue%20Use%20of%20Antigen%20POC_10.02.2020_ADA_Compliant.pdf
https://www.ahcancal.org/Data-and-Research/Center-for-HPE/Documents/Report-Discordant-COVID-Test-Results.pdf#search=Discordant%20Results%20between%20COVID%2D19%20Point%20of%20Care%20Antigen%20and%20PCR%20Tests%20in%20Nursing%20Homes
18/n
Antigen tests better detect live virus but VERY limited data vs. cell culture @michaelmina_lab
And newer antigen tests will likely be better (but need clinical data, not just lab comparisons)
19/n
And newer antigen tests will likely be better (but need clinical data, not just lab comparisons)
19/n
So, in conclusion, a group of @IDSAInfo docs, epidemiologists and @ASMicrobiology estimated the following CLINICAL Sensitivity & specificity for #COVID19 diagnostic tests
--feedback welcome
--feedback welcome
