Flowify AI: Improved Accuracy due to Time Tracking of Rapid Tests
Flowify AI is an advanced software platform that enables the digitization of rapid diagnostic tests (e.g. lateral flow assays (LFAs)). Most importantly, it empowers users to use their smartphone for the automatic result interpretation of rapid tests. Our state-of-the-art AI image evaluation algorithm analyzes the camera stream of the device and computes an accurate result. However, it is very important that a user performs the test according to the instructions for use (IFU) to obtain a valid and precise result. In this article, we study the time sensitivity of LFAs and the importance of following the specified time intervals. Moreover, we outline how the Flowify AI mobile application supports a user with respect to those challenges.
Rapid Test Execution Phases
The execution of a rapid test is subject to distinct phases (see Fig. 1). First, the user needs the prepare the sample (e.g. obtain capillary blood, mix it with a solution) and drop it on the rapid test (preparation phase). Directly afterwards starts the development phase of the rapid test. The manufacturer specifies a certain time interval (runtime) during which the sample flows through the test and the signals (e.g. test line and control line) develop. The runtime of LFA rapid tests depends on the product and is mostly 5, 10 or 15 minutes long. The reading phase starts directly after the development phase. At this time, the signals are fully developed and the test result can be interpreted. The reading time is mostly between 2 and 5 minutes long, but for some products it is even recommended to read the result at an exact time point, i.e., directly after the development phase.
It is important to note that the signal is not stable after the development phase as the reaction on the test continues and the test dries up. After the reading phase, the rapid test is still in a visual state that allows to read the result, but the result is no longer valid and can be inaccurate.
Time Sensitivity
We study the effect of time on the result of a rapid test by monitoring the raw signal values during all phases of the rapid test execution. Flowify AI is able to compute a signal value for each line of an LFA rapid test. The signal values fall into the range from 0.00 to 1.00, whereby the signal value is zero if a line is invisible and greater than or equal to 0.01 if a line is visible. A higher signal value indicates a stronger signal intensity. For the purpose of this article, it is sufficient to analyze signal values as they are input for quantitative results and subsequent operations can distort the underlying effect.
In the following, we present two examples: First, we provide a basic example using a single Vitamin D rapid test. Second, we employ multiple CRP tests to investigate the effect for different concentration levels.
Example 1
The first example is a Vitamin D test that was performed with a capillary blood sample. The runtime of the test is 15 minutes and the reading time is two minutes. Hence, the result has to be interpreted between 15 and 17 minutes after the sample was dropped on the test.
We took an image of the test every 10 seconds for 25 minutes and used Flowify AI to determine the signal values of the test line and the control line of each image.
It is noticeable that the test looks already readable after about four minutes, but the signal value of the test line more than doubles until the start of the reading time. Hence, if a user does not observe the time, it is possible that the result interpretation happens too early.
In general, we notice that the signal values of both lines increase over time. However, the control line is saturated after nine minutes and its value slightly drops as the color of the LFA becomes a bit yellowish (i.e. darker than white) and the contrast decreases. This effect is not noticeable for the test line at the present concentration level as the reaction continues to run and the discoloration also shines through the signal line.
The signal values at the center of the reading time are T=0.11 and C=0.48. If the result is determined five minutes too early, the value of the test line is lower (T=0.08) and unchanged for the control line (C=0.48). The values hardly change if the result is interpreted five minutes too late (T=0.12, C=0.47). Hence, if the time of the read-out is only slightly delayed, the deviations are not significant for this specific sample.
Example 2
The next example is a CRP test having a runtime of 10 minutes and a reading time of two minutes. We used antigen to study the time sensitivity for seven distinct concentration levels. Similar to the first example, we took an image of each test every 20 seconds for 20 minutes and computed the signal values using Flowify AI.
The images of the tests also indicate that the test looks already readable way before the specified reading time. The signals of the test lines continue to increase until the end of image capturing, i.e., 20 minutes after test execution, but tend to plateau towards the end. This is not only caused by the continuing reaction, but the test also dries up and becomes whiter which leads to a higher contrast. Compared to the test from the first example, which was performed with a capillary blood sample, we do not observe a discoloration over time.
We also notice that the signal values correlate with the concentration levels as the levels are clearly distinguishable but follow a comparable trend. However, the degree of the signal value changes during the reading phase (and also afterwards) depends on the concentration level.
For low concentrations, the signal values do hardly increase (e.g. 10 ng/ml and lower), but for higher concentrations (25 ng/ml and 40 ng/ml) the change is more noticeable as the signals increase by 0.04 (see Tab. 1). For the highest concentration level, the change is not as high (only 0.02) as the signal is already quite saturated at the beginning of the reading time interval.
Results
Based on our experiments, we can report the following findings. The signal values tend to increase over time which means that an early read-out leads to an underestimated concentration and due to a late read-out the concentration will be overestimated. A late read-out can also lead to a false positive result if the test line becomes visible after the reading time. Hence, the result is only valid if it is obtained within the reading phase. An early read-out is possible if a user does not observe the time as the test looks already readable before the start of the reading phase. While the signals increase over time, we also notice that they tend to plateau at some point. Hence, it might be reasonable for a manufacturer to specify a longer runtime, but this is often not possible for qualitative tests as it might lead to false positive results.
Time Tracking Assistance
The Flowify AI mobile app assists the user during the execution of a rapid test and also incorporates features that ensure that the specified runtime and reading time are respected. For this purpose, it accompanies the general process of a test execution and integrates several checks (see Fig. 7).
1. Test Registration
At the beginning, the test needs to be registered. The Flowify AI algorithm checks if the test is still new. Moreover, the correct product and batch can be automatically identified if the test has a barcode identifier that offers this information. Alternatively, the user can manually select the product. Subsequently, all parameters that are required for the test execution (e.g. runtime and reading time) and the result interpretation are retrieved. Moreover, the user can enter an identifier for the sample that is about to be tested.
2. Test Execution
In the following step, the user can drop the prepared sample onto the test and start a timer.
3. (Multi-)timer
The duration of a timer (development phase) complies with the product instructions. Once a timer is finished, the user receives a notification and can evaluate the test. More precisely, a second timer covering the reading phase is started. If the scan process is not terminated or started within the reading time, the user is still able to scan the test, but the violation of the test instructions will be tracked.
In order to facilitate fast and accurate testing in professional settings that are subject to a high throughput of tests, Flowify AI incorporates a multi-timer functionality. Hence, it is possible to run multiple timers in parallel. This is often required as the runtime of a rapid test can be rather long (e.g. 10–15 minutes) which allows to execute several tests in sequence. Each timer can be associated with a test based on the entered sample identifier and/or an unique identifier of the test (e.g. encoded in a barcode).
4. Result Interpretation & Presentation
After a timer is finished, the user is able to scan the test. Our Flowify AI image analysis algorithm analyzes the camera stream and automatically computes the result. The result will be presented to the user and persisted on the device and optionally to a cloud service. If the test has a unique identifier, it can be checked if the correct test for a specific timer is scanned. Additionally, it will be tracked if a test is not evaluated within the specified reading time interval.
Conclusion
We analyzed the time sensitivity of LFA rapid tests and provided support that it is very important to consider the specified reading time. A too early or too late result read-out can lead to an invalid, inaccurate, wrong (e.g. false negative or false positive) and inconsistent result.
The Flowify AI mobile application assists the user during the execution of a rapid test. The integrated time tracking feature ensures and monitors that a rapid test is evaluated at the right time which is a requirement for an accurate and valid test result. Moreover, the mobile application has precise reading capabilities using a sophisticated image evaluation algorithm that enables a reliable and objective test result interpretation. Hence, it is a convenient tool that minimizes human error and also supports testing a large number of tests as it is often required in professional settings.
