Stop TB Partnership

AI-Powered Computer-Aided Detection (CAD) Software

Artificial intelligence (AI) technologies offer unprecedented opportunities within a healthcare context. The use of AI technologies to aid in the diagnosis of diseases such as cancer and TB has attracted much attention. AI, specifically the application of deep-learning neural networks, is increasingly being applied in the field of medical imaging for the computer-aided detection (CAD) of disease. Artificial neural networks mimic human neural networks and are able to learn supervised or unsupervised from training datasets. Multiple computer-aided reading software that utilize deep neural networks for the recognition of TB-related abnormalities from CXRs are now commercially available. These technologies hold promise as a TB screening and triage tool, and the potential to expand diagnostic capabilities, particularly in rural and low-resource contexts, is tremendous.

In December 2020, the World Health Organization (WHO) released a Rapid Communication, recommending for the first time that CAD may be used as an alternative to human reader interpretation of CXR for screening and triage for TB. The Stop TB Partnership is working through the vast network of partners to collate information on AI products commercially available, to evaluate the products as an independent body and to support implementers to conduct pilot studies and projects.

Collation of commercially available AI solutions

With many commercial CAD products available on the market, it is hard for country program, implementing partners, and civil society and communities to learn to be updated with the market of CAD products and be informed of new development.

Stop TB invites you to access the newly released www.ai4hlth.org, co-developed with the Foundation for Innovative New Diagnostics (FIND), which provides the first extensive landscape overview of available CAD products that can interpret CXR images for TB and with detailed technical specifications, including:

  • Certification
  • Price
  • Development Stage
  • Deployment Method: cloud vs offline
  • Hardware and Software requirements
  • Input requirements
  • Output format
  • Target Setting
  • Current market
  • Input and Output requirements
  • Product development and training

As of Sept 1, 2020, there are eight CAD products specific to TB which are available on the market, CAD4TB from Delft imaging (Netherlands), DxTB from DeepTek (India), InferRead DR Chest from Infervision (China), JF CXR-1 from JF Healthcare (China), JVIEWER-X from JLK, Lunit INSIGHT CXR from Lunit (South Korea), qXR from Qure.ai (India), XrayAME from Epcon (Belgium). Solutions with CE certification are CAD4TB, InferRead DR Chest, JVIEWER-X, Lunit INSIGHT CXR, and qXR. A further three products are currently in a validation stage: AXIR from Radisen (South Korea), Dr CADx from Dr CADx (Zimbabwe), and T-Xnet from Artelus (India).

Read more

Virtual Innovations Spotlights on AI-Powered CAD Software

In order to support country programmes, healthcare providers, the communities and people affected by TB and our partners during COVID-19, the TB REACH team presented current work on CAD at the Stop TB’s virtual innovation spotlights (VIS). The presentation and recording can be accessed:

Artificial Intelligence (AI)/Computer Aided Detection (CAD) - Breaking with Tradition: The Value and Use of AI/CAD for TB Detection powered by Stop TB Partnership - TB REACH

Webinar recording

Presentation

More Virtual Innovations AI Spotlights can be found here.

We evaluate commercial AI/CAD products

We evaluate commercial AI product using the TB REACH-CXR Archive, a privately held and de-identified evaluation database of chest radiographs for independent and external validation of commercial CAD products. The TB REACH-CXR Archive is large dataset of 30,957 posterior-anterior chest x-ray images for 30,957 patients collected through 4 projects funded by the Stop TB Partnership’s TB REACH Initiative between 2015-2020. All images have been de-identified to protect patient privacy. Randomly generated identifiers are used to group distinct reports and patients. Each project study was conducted in a different setting and geographic regions.

Our evaluation report includes:

  1. Area under the Receiver Operating Characteristic and Precision Recall Curves
  2. Comparison with human radiologist
  3. AI accuracy histogram
  4. Threshold Score selection (including measures of accuracy and cost-effectiveness)

We're developing an independent online platform for the automated evaluation of AI software products using our large and diverse data sets.

Publications from the TB REACH-CXR Archive can be found below.

Contact us to be evaluated using the Stop TB chest x-ray archive: dhthub@stoptb.org.

TB REACH’s AI / CAD Projects

TB REACH has funded and supported the implementation of CAD products across different countries (Peru, Pakistan, Zambia, India, Myanmar, Cambodia, Vietnam, Cameroon, Kenya, Moldova and Bangladesh), targeting different populations. The AI/CAD tools include CAD4TB, qXR, and Lunit.

Read more in our article about artifical intelligence and TB diagnostics here.

Recent Publications

We conducted a landscape analysis to collect information from developers known to have, or soon to have, a CAD product for TB. We identified 27 CAD developers and 11 completed our survey with details about the certification, deployment, operational characteristics, input requirements, output format, pricing, and data privacy of their latest product version. For each response, a summary product profile was created based on the information provided and these were published on an open-access website: ai4hlth.org. Online deployment was most common, but offline versions were found to be available for settings without stable network connection. Almost all CAD products are agnostic to brand and model of the digital x-ray platform and almost all integrate with legacy systems. Input is commonly in DICOM and/or JPEG, PNG, TIFF format. Output is often a heatmap and numeric abnormality score for TB. This study provided, for the first time, an extensive overview of the available CAD products that can interpret chest x-ray images for TB. By making information more accessible and searchable, we hope to better brief TB implementers on the variety of products available and facilitate informed decision making when using this technology to ultimately serve more people with TB.

Read more

We conducted a retrospective evaluation of three DL systems (CAD4TB, Lunit INSIGHT, and qXR) for detecting TB-associated abnormalities in chest radiographs from outpatients in Nepal and Cameroon. All 1196 individuals received a Xpert MTB/RIF assay and a CXR read by two groups of radiologists and the DL systems. Xpert was used as the reference standard. The area under the curve of the three systems was similar: Lunit (0.94, 95% CI: 0.93–0.96), qXR (0.94, 95% CI: 0.92–0.97) and CAD4TB (0.92, 95% CI: 0.90–0.95). When matching the sensitivity of the radiologists, the specificities of the DL systems were significantly higher except for one. Using DL systems to read CXRs could reduce the number of Xpert MTB/RIF tests needed by 66% while maintaining sensitivity at 95% or better. Using a universal cutoff score resulted different performance in each site, highlighting the need to select scores based on the population screened. These DL systems should be considered by TB programs where human resources are constrained, and automated technology is available.

Download PDF

We evaluated five AI software platforms specific to TB: CAD4TB (v6), InferReadDR (v2), Lunit INSIGHT for Chest Radiography (v4.9.0) , JF CXR-1 (v2) by and qXR (v3) by on an unseen dataset of chest X-rays collected in three TB screening center in Dhaka, Bangladesh. The 23,566 individuals included in the study all received a CXR read by a group of three Bangladeshi board-certified radiologists. A sample of CXRs were re-read by US board-certified radiologists. Xpert was used as the reference standard. All five AI platforms significantly outperformed the human readers. The areas under the receiver operating characteristic curves are qXR: 0.91 (95% CI:0.90-0.91), Lunit INSIGHT CXR: 0.89 (95% CI:0.88-0.89), InferReadDR: 0.85 (95% CI:0.84-0.86), JF CXR-1: 0.85 (95% CI:0.84-0.85), CAD4TB: 0.82 (95% CI:0.81-0.83). We also proposed a new analytical framework that evaluates a screening and triage test and informs threshold selection through tradeoff between cost efficiency and ability to triage. Further, we assessed the performance of the five AI algorithms across the subgroups of age, use cases, and prior TB history, and found that the threshold scores performed differently across different subgroups. The positive results of our evaluation indicate that these AI products can be useful screening and triage tools for active case finding in high TB-burden regions.

Read more