Covid-19 test validation — a backstory

Suresh S
7 min readJul 3, 2020

--

Developing and validating a clinical test is always difficult, but doing it on a timeline dictated by a pandemic is even harder.

Many commercial, academic, and clinical labs have been working tirelessly to come up with efficient methods to test patients for Covid-19 infections — also referred to as SARS-CoV-2 infections. The company I work at has also been involved in this effort. My role, way back in April 2020 (a lifetime ago), was to help set up processes, decide test conditions and to validate the entire system. This mainly involved taking a few samples with known results (i.e. 30 positive and 30 negative samples) and determining whether the system we set up generated the expected results. In other words when we completed the test, did all the 30 known positives actually yield a positive result for Covid-19, and were all 30 known negatives Covid-19 negative? The answer to both questions was yes. In other words, there were no false positives or false negatives in our test — the bane of any diagnostic or screening test. In the time of a pandemic this becomes even more critical, because you don’t want a person who is carrying the virus to be walking around thinking they aren’t carriers, based on a false negative result they received from a lab that tested their sample.

Our lab moved to Pittsburgh just before we learnt about Covid-19, so I had to travel to Pittsburgh three times to setup and implement the testing process. I opted to drive the 1000 mile round trip distance rather than fly because I felt I had better control over my environment in a pandemic. In addition to my regular luggage, I took a sleeping bag, pillow, bed sheets, plastic ware, cups, cleaning supplies, the works! As soon as I entered the hotel room, I wore gloves and cleaned every surface with germicidal wipes — door knobs, faucet handles, desk and table surfaces, light switches, toilet handle — everything! I spread my bed sheet over the hotel bed, opened my sleeping bag over that and slept on that so I didn’t have to touch the bed. I even shut off the AC because I didn’t know what the fan was blowing into the room. Because I stayed for at least a week each time, I carried a large zip lock bag and laundry soap to wash and rinse my clothes in it and then hung the clothes to dry on hangers I brought from home. I swear if the room was big enough I would have pitched a tent in it.

Each of the three trips had side adventures; on the first one, I had two flat tires when I reached Pittsburgh and it was a genuine headache finding an auto shop during a lock-down to fix them. On the second visit, I was maybe one of only three guests in the entire 11-floor hotel, and in the night a huge thunderstorm caused a power outage. Five minutes later the hotel’s emergency generator blew out and I was plunged in total darkness. I felt like I was in the movie “The Shining” and expected a maniacal murderer knocking on my door at any moment. Finally, on my return trip last week I experienced driving for 10 miles in the most amazing hailstorm. The sky went from blue, to grey to very dark in just a few minutes. At first it was just rain, but soon hail stones the size of marbles were pelting the car and blocking out visibility almost completely. It ended as abruptly as I started and soon I was driving under blue skies again. I’m sure there’s a moral behind these adventures and if you can find one in them please let me know!

Before any clinical test can be rolled out to the public it has to be approved by the FDA. To secure approval each step of the test has to be rigorously evaluated and validated. Then, all the data you collect has to be assembled in the prescribed FDA format and sent off to them. An FDA scientist is assigned to your submission and after thorough review you either get an approval or rejection. The work that I did when I was in Pittsburgh went through the FDA submission process, and our company was approved as a Covid-19 or SARS-CoV-2 testing site.

On the surface, Covid-19 testing is not terribly different from lab to lab — the nuanced differences are typically in the chemical reagents and equipment used, and the checkpoints that a particular lab puts in place to ensure integrity of the process. Here’s a picture to show you what the generic workflow looks like in most labs.

The most under-appreciated part of the testing is sample acquisition. Designing a fool-proof method to ensure that the chain of custody of each sample is faithfully retained from the moment the sample arrives in our lab, to the point the result is provided to the patient, was almost as important as establishing the actual testing protocol. In our lab, each day begins with the mail delivering hundreds of boxes of samples. Each box contains a tube in a small baggie. Each tube is barcoded and has the patient’s name, date of birth and other identifiers. Each tube is entered into our lab information management system (LIMS) and given an additional barcode that will follow the sample during its transformation, by several technicians and robots, from specimen to a result on a report that goes back to the patient.

The first step in the patient’s sample (most commonly a nasopharyngeal swab immersed in a liquid inside a tube) travels through the lab is to take the liquid from the tube and place it in a “well” of a plate we call the “sample plate” that contains 96 such wells, each with a unique address — horizontally they are labelled A-H, and vertically 1–12; therefore wells will have addresses like A1 (topmost at the left), A2 (the one to its right) and so on (see infographic). Each sample plate is given a barcode and will contain up to 94 patient samples (2 wells are used for what are known as process controls which are used to monitor the process from start to finish). We identify each patient’s sample location by the sample plate’s barcode and also by the well address — for example, in sample plate number 1, patient 1 will be in well A1, patient 2 will be in well A2, and so on.

The filled plate is then taken to a robot which is capable of extracting the genetic material from everything in the sample — human cells, microorganisms, and viruses including the SARS-CoV-2 virus if it is present in the sample. For this test, we have programmed the robot to extract a type of genetic material called RNA — the type of genetic material that is carried inside the SARS-CoV-2 virus.

The robot does this step in about an hour, and transfers the RNA from each patient sample to a new previously identically barcoded 96 well plate called the RNA plate; the robot faithfully maintains the well addresses from the sample plate. Therefore, in the RNA plate, all the RNA from patient 1 will be in well A1, and from patient 2 will be in well A2, and so on.

Next we take the RNA plate to another robot which “sips” a bit of the RNA from each well of the RNA plate and puts it into corresponding well addresses of a third identically barcoded plate we call the “assay plate”. The robot then adds enzymes and SARS-CoV-2 specific chemical reagents to each well of the assay plate.

The assay plate is taken to the final instrument which reads the signals from each well of the assay plate. The test is almost binary in nature; if the original sample had SARS-CoV-2 virus present then that well in the assay plate will emit light of specific color which is measured by this instrument. If the sample had no SARS-CoV-2 present it will remain dark. This SARS-CoV-2 present or absent result is then sent to the laboratory management system which remembers all the plate barcodes and well addresses and traces the information all the way back to the patient’s barcode on the tube and prints out the test result along with the patient identifiers (name, date of birth).

We process 1000 samples in an 8-hour shift. The team of heroes consists of two people who accession the sample (the ones who enter all the patient info into the lab management system), two sample extractors — the ones responsible for getting the RNA out of each sample, two assay specialists who do the actual test to see if the sample is positive or negative for SARS-CoV-2. Finally, there are two data analysts who stay up late at night and review the data to ensure that sample custody and data integrity has been maintained from start to finish. After all this massive team effort, the patient receives test results through their physician. It can’t be emphasized enough, that behind the result a patient receives, is an invisible team of dedicated laboratory technicians and scientists who are working on extremely demanding schedules — without their commitment we would be worse off than we are now.

--

--