Validation: Search for the Gold Standard
HIV tests should be validated against direct isolation of the virus from people that test positive, and lack of isolation in people who test negative. Instead of just comparing people with AIDS against healthy people, tests should also be validated on people with non-AIDS diseases that may cross-react (such as leprosy or malaria) and with other conditions that may generate a higher load than normal of antibodies (e.g. recent vaccination, pregnancy, autoimmune disease). Isolation should be validated by purification of HIV particles, followed by microscope verification of purity, and then by analysis of the constituent proteins and genetic material. This suite of tests is sometimes referred to as a 'Gold Standard' and has never been performed for HIV, not even once.
The patient was a 28-year-old male who was referred for HIV testing after a needle stick injury from a needle contaminated by blood from a newly diagnosed HIV-positive patient. He was given postexposure prophylaxis consisting of zidovudine, lamivudine, and nelfinavir for 4 weeks. Baseline and 6-week HIV-1/2 antibody enzyme immunoassay (EIA) testing produced negative results. An HIV viral load assay was ordered at the same time as the 6-week EIA, for which the patients blood was collected in a Greiner K2 EDTA tube. This tube was centrifuged and then frozen at 70íC within 65 min of collection. The following day, the Versant HIV-1 RNA 3.0 bDNA assay was performed according to the manufacturers instructions, using the Bayer System 340 analyzer, and produced a result of 955 copies/ml. No error code was generated by the analyzer during this test. Immediately after this test was performed, a Bayer technician examined the analyzer by prior arrangement (see below). It was discovered that the read head of the instrument was not adequately compressing the tops of the sample wells, potentially resulting in light leakage between wells. Maintenance was performed to correct this problem. A repeat viral load bDNA assay on the same serum 2 days later produced a result of 75 copies/ml. Additional EIAs performed 3 days and 45 days after the initial viral load assay were repeatedly negative (7 and 12 weeks following the needle stick exposure). An additional viral load assay using new serum collected 2 days after the first viral load also produced a result of 75 copies/ml. In the 4.5 months since the needle stick exposure, the patient continues to be asymptomatic, with negative HIV testing, and is presumed to be HIV negative. [presumed to be negative because there is no way to know for sure which test is right and which is wrong. In cases like this, the majority usually rules, but the majority is not always right]
Marinovich A et al. False-positive result from a Bayer Versant human immunodeficiency virus type 1 branched-DNA viral load assay, with a possible role for light leakage after inadequate maintenance of the analyzer. J Clin Microbiol. 2006 Nov;44(11):4288-9.
A Reactive result indicates that HIV-1 RNA was detected. For a specimen that is repeatedly reactive on an HIV-1 antibody test and reactive in the APTIMA HIV-1 RNA Qualitative Assay, the individual is considered confirmed infected with HIV-1
A specimen that is reactive in the APTIMA HIV-1 RNA Qualitative Assay but that has not been tested in an HIV-1 antibody test should be further tested using a licensed test for HIV-1 antibodies.
Aptima HIV-1 RNA qualitative assay. Gen-Probe. 2006 Oct
The only way of proving that antibody reactivity is caused by a retroviral infection is to compare the presence or absence of reactivity with the presence or absence of the retrovirus. In other words, as with other tests used in clinical practice, the test must be validated against a gold standard. 8 In a test for HIV infection, the gold standard can only be HIV itself, as proven by HIV isolation. Yet, no such data have been reported a fact acknowledged by manufacturers of antibody tests: At present there is no recognized standard for establishing the presence or absence of HIV-1 antibody in human blood. 9 Instead, specificity is determined using the clinical diagnosis of AIDS as a gold standard. However, AIDS cannot be a substitute gold standard because: (i) AIDS-indicator diseases are caused by agents other than HIV; (ii) the evidence HIV experts present that HIV is the cause of AIDS is the reaction between the antibodies in patient sera and the test kit antigens. To then claim AIDS proves that the antibodies are HIV is a circular argument. Furthermore, if AIDS is a gold standard for HIV infection, all seropositive individuals who do not have AIDS, that is, the vast majority, must be false positives.
Papadopulos-Eleopulos E et al. No proof HIV antibodies are caused by a retroviral infection. Emerg Med Australas. 2006 Jun;18(3):308-9; author reply309-10.
At present, there is no recognized standard for establishing the presence or absence of antibodies to HIV-1 and HIV-2 in human blood.
Compared with the Abbott [antibody] assay, the Vironostika [antibody] assay had a sensitivity of 98% and a specificity of 94.7% [showing how one test that has never been properly validated is used to validate another similar test, by showing that most of the time they produce the same result]
Gouws E et al. High Incidence of HIV-1 in South Africa Using a Standardized Algorithm for Recent HIV Seroconversion. J Acquir Immune Defic Syndr. 2002 Apr 15;29(5):531-535.
The performance characteristics of the home-made PCR assay were firstly evaluated by instituting an inter-laboratory quality control.  Samples (six negative and ten positive) tested in the Laboratory of Virology of Hospital Necker using the home-made PCR assay were shipped to the CeDReS where they were assessed by the same technique. All specimens tested by these two laboratories gave concordant results. Afterwards, the sensitivity of nested PCR amplification was validated for each run by using calibrated DNA from 8E5 cells as positive controls with a detection level of five copies of proviral DNA per million cells. All tests were carried out in duplicate, with positive and negative controls, and distilled water as an internal control. In samples that were negative by the two pol primer pairs, DNA quality was verified by PCR with HLA-Dq-alpha primers. The PCR was classified as positive when a positive signal was obtained for at least two of the three primer pairs [but this technique is so sensitive that all three primer pairs should always be detected]
Rouet F et al. Early diagnosis of paediatric HIV-1 infection among African breast-fed children using a quantitative plasma HIV RNA assay. AIDS. 2001 Sep 28;15(14):1849-56.
Primary infection was defined as a confirmed positive virologic test result with either a negative HIV antibody assay result or an indeterminate Western blot. Because there is no virologic gold standard, we assumed that levels of plasma HIV RNA had a sensitivity of 100% for diagnosing primary infection [bonus marks for detecting the flaw in this logic]. False-positive HIV RNA measurements were defined as those that were negative on repeated testing and those obtained in patients who did not undergo seroconversion [note the contradiction with the previous sentence]...Eight of 303 uninfected patients (2.6%) had false-positive results on HIV RNA testing [note that in a population that is mostly uninfected, this could result in several times more false positives than true positives]
Daar ES et al. Diagnosis of primary HIV-1 infection. Ann Intern Med. 2001 Jan 2;134(1):25-9.
Sera from 111,639 men were tested following [an algorithm involving two EIA antibody tests from different manufacturers followed by a Western Blot antibody test]. A total of 2839 (2.5%) sera were reactive in EIA no. 1; 2322 (82%) of these were also reactive in EIA no. 2. HIV-1 Western blot of these dually reactive sera yielded 2195 reactive, 62 non-reactive and 65 indeterminate results. Defining true positivity as reactive in all three assays [!], the prevalence of HIV-1 antibodies in this serum set was 2.0% (2195/111639). The frequency of false positivity of two sequential EIA was 0.11% (127/111639). The positive predictive value of one EIA was 77.3%, of two EIA 94.5%. [The problem is that if antibodies can cross-react, the probability of them cross-reacting on 3 different antibody tests is still quite high, consequently the assumption that if all three tests are positive, that all three tests are meaningful, is quite arbitrary]
Chanbancherd P et al. Frequency of HIV false positivity from two sequential enzyme immunoassays in 111,639 sera. AIDS. 1999 Oct 22;13(15):2182-3.
The only way to obtain specific reagents is to isolate the virus, that is, obtain viral particles separate from everything else. If this is not done it is impossible to say which reagents (proteins) originate from the virus and which are contaminants. Only then can the viral proteins be used as specific reagents with which to perform antibody tests. Even then, because a given antigen can react with antibodies directed against other antigens (cross-reactions), the specificity of the reactions must be determined by using viral isolation as a gold standard. However, instead of using this procedure, the only one scientifically valid, Gallo and his colleagues cultured a leukaemic cell line (HT) with tissues with tissues derived from AIDS patients. Proteins derived from the culture supernatants (but without proof of origin from a retrovirus or even particles, viral or non-viral, or even from the patients), were incubated with sera from AIDS patients or those at risk.
At present there is no recognized standard for establishing the presence and absence of HIV-1 antibody in human blood.
The evaluation of the sensitivity and specificity of PCR for the diagnosis of HIV infection in infants is particularly difficult because there is no reference or gold standard test that determines unequivocally the true infection status of the patient [but antibody tests, generally agreed to be inapplicable to infants, hardly qualify as a gold standard, so the same problem occurs with all people being tested for HIV]...a single positive PCR test result does not provide definitive evidence of HIV infection in infants...This finding holds even for more recent studies published after PCR technology had matured
Owens DK et al. A Meta-analytic Evaluation of the Polymerase Chain Reaction for the Diagnosis of HIV Infection in Infants. JAMA. 1996 May 1;275(17):1342-1348.
Based on an assumed 100% prevalence [of HIV] in antibody positive individuals, the sensitivity of the neutralization test was 99.5% (855/859) for repeatedly reactive specimens with a valid neutralization test and 100% (855/855) for specimens that were stored and tested according to recommendations.
Although serologic assays are capable of identifying prior exposure to human immunodeficiency virus (HIV), they cannot alone demonstrate whether an individual is currently harboring the virus. The first method used to ascertain if a blood specimen contained HIV was co-cultivation with stimulated primary human lymphocytes or continuous human T cell lines and monitoring the culture supernatants for the presence of reverse transcriptase. Although viral isolation has proved to be a poor diagnostic tool because of its relative insensitivity [about half of HIV-antibody-positive people are culture negative], high costs, and lengthy time requirements, culture has served as the standard by which all other diagnostic tests have been judged and established. Furthermore, virus culture remains the steadfast route by which new variants are identified, isolated and initially characterised.
Rayfield M et al. HIV culture. Chapter 7 in AIDS Testing: a comprehensive guide to technical, medical, social, legal and management issues. Springer Verlag. 1994;129.
This study assumed that the investigated samples were from non-HIV infected individuals. While we were unable to unequivocally prove this, samples were obtained from normal blood donors who signed a declaration form stating that they had not engaged in any HIV-related risk behaviour. Because these individuals were healthy enough to present for blood donation, it is unlikely that their indeterminant WB reactivities could have resulted from the loss of anti-core antibodies, which has been associated with late-stage AIDS
the use of WB interpretive criteria, such as those proposed by the World Health Organization [would] allow individuals reactive to two glycoprotein bands to be considered anti-HIV-1-seropositive, and would inappropriately classify 11 out of the 13 samples described in this study as anti-HIV-1-seropositive by the DB [Diagnostic Biotechnology] WB
Healey DS, Bolton WV. Apparent HIV-1 glyco-protein reactivity on Western Blot in uninfected blood donors. AIDS. 1993 May;7(5):655-8.
Alloimmune mice...were shown to make antibodies against gp120 and p24 of human immunodeficiency virus (HIV), and mice of [two] autoimmune strains...made antibodies against gp120. This is surprising because the mice were not exposed to HIV. [i.e. HIV proteins are found in uninfected mice!!]
Kion TA, Hoffmann GW. Anti-HIV and anti-anti-MHC antibodies in alloimmune and autoimmune mice. Science. 1991 Sep 6;253:1138-40.
The currently licensed Du Pont Western blot test specifies that the test result should be interpreted as positive only when the detected bands include p24 and p31, and gp41 or gp120/160. Conversely, a negative Du Pont Western blot test result requires the absence of any and all bandsnot just viral-bands. All other patterns are regarded as indeterminate
Alternative criteria have been proposed by various groups. ASTPHLD has proposed that a positive test result be defined by the presence of any two of the following bands: p24, gp41, and gp120/160 (13). The Consortium for Retrovirus Serology Standardization (CRSS) has defined a positive test result as the presence of either p24 or p31, plus a diffuse envelope band (i.e., gp41 or gp120/160) (14). The American Red Cross has defined a positive test result as greater than or equal to 1 band from each of the GAG, POL, and ENV gene-product groups (15). These three groups and DuPont all agree that an indeterminate result is the presence of any other band or bands that fail to meet the positive criteria, and that a negative result is the absence of all bands
For all three categories with repeatedly reactive EIA test results, the Western blot results demonstrate that the ASTPHLD definition gives the highest percentage of positive and the lowest percentage of indeterminate results [note that this has nothing to do with the accuracy of the test but
] On the basis of the results described above, CDC concurred with the ASTPHLD criteria and recommends their use in public health and clinical practice.
Interpretation and use of the western blot assay for serodiagnosis of human immunodeficiency virus type 1 infections. MMWR Morb Mortal Wkly Rep. 1989 Jul 21;38(Suppl 7):1-7.
One commonly used reference standard, a Food and Drug Administration panel often to 18 serum samples from patients with clinically diagnosed AIDS, suffers from the problems of spectrum bias and small sample size.
Schwartz JS, Dans PE, Kinosian BP. Human immunodeficiency virus test evaluation, performance, and use. Proposals to make good tests better. JAMA. 1988 May 6;259(17):2574-9.
Serologic assays identify persons with prior exposure to human immunodeficiency virus (HIV-1), they do not specifically determine current infection...The number of peripheral blood lymphocytes expressing viral RNA, as detected by in situ hybridization in an infected person is less than 1 in 10,000 cells...Defective provirus would be detected by the PCR technique provided the region targeted for amplification was preserved [i.e. antibody tests do not prove current infection, and viral load/PCR tests cannot distinguish between defective and infectious HIV!]
Ou CY et al. DNA amplification for direct detection of HIV-1 in DNA of peripheral blood mononuclear cells. Science. 1988 Jan 15;239(4837):295-7.
for HIV infection, there is no independent, unequivocal way of identifying a group of individuals who are all assuredly infected or uninfected
Cleary PD et al. Compulsory premarital screening for the human immunodeficiency virus: Technical and public health considerations. JAMA. 1987 Oct 2;258(13):1757-62.
The meaning of positive tests will depend on the joint [ELISA/WB] false positive rate. Because we lack a gold standard, we do not know what that rate is now. We cannot know what it will be in a large-scale screening program
Meyer KB, Pauker SG. Screening for HIV: can we afford the false positive rate?. N Engl J Med. 1987;317(4):238-41.
Evaluation of a new test requires an established or known standard for comparison. At this point, however, no established standard exists for identifying HTLV-III [HIV] infection in asymptomatic people. Current culture methods for [HIV] identify virus in only 36% to 85% of persons with AIDS or related conditions and cannot be used as an absolute standard for HTLV-III/LAV [HIV] infection. For this reason, we defined [note: not proved] specimens positive on Western blot or culture as positive for infection with [HIV]
Ward JW et al. Laboratory and epidemiologic evaluation of an enzyme immunoassay for antivodies to HTLV-III. JAMA. 1986 Jul 18;256(3):357-61.
in the interpretation of the results it is assumed that these 11 [HIV-negative samples from people in high risk groups] and all the BD [blood donor] and PFP [potentially false-positive] specimens were negative and that the remaining 72 HRG [high risk group] specimens were the only true positive ones
Mortimer PP et al. Which anti-HTLV-III/LAV assays for screening and confirmatory testing?. Lancet. 1985 Oct 19;326(8460):873-7.
We evaluated the validity of the test by determining whether the test could distinguish known patients with AIDS from the normal population and from groups that might pose cross-reactivity problems [but not against actual detection of a virus]...Since the ELISA [antibody test] ratio [indicating the intensity of the antibody reaction] was less than 5.0 in approximately 99% of these controls, serum samples with ratios of 5.0 or greater were defined as positive for HTLV-III [HIV] antibodies
Weiss SH et al. Screening test for HTLV-III (AIDS agent) antibodies: specificity, sensitivity, and applications. JAMA. 1985 Jan 11;253(2):221-5.
Sensitivity and specificity of the HIV-1 Western Blot Kit was determined in comparative studies with a previously licensed HIV-1 Western blot using EIA repeatedly reactive samples from high AIDS risk and low risk populations respectively [i.e. without a gold standard such as virus purification] [Specificity can only be tested with EIA negative specimens, of which there were none].
Human Immunodeficiency Virus Type 1 (HIV-1) HIV-1 Western Blot Kit. Epitope.