Given the socioeconomic disparities and scarce healthcare resources, aggressive measures to combat the spread of Covid-19 such as personal protection, widespread contact tracing, isolation and mass movement restrictions were challenging for low-middle-income countries (LMICs) to adopt [3]. As a densely populated country with roughly 1.4 billion people, Covid infection rates in India quickly rose to a stark 850,000 by July 2020 [4].

Rising cases unsurprisingly swamped clinical services, including imaging departments where CXRs became the first-line imaging modality for Covid-19 screening. Accurate diagnosis of CXRs requires radiological expertise and chronological reporting becomes a tedious task when patient numbers become very large. This eluded the potential for AI solutions that can act as diagnostic assistances to be implemented and alleviate the burden on clinicians.

Behold’s red dot® V1 platform is a Class I CE marked software as a medical device (SaaMD). Red dot® processes adult ≥18 years frontal Chest X-Ray (CXR) examinations to help radiologists prioritise which chest plain film should be reviewed more urgently.

In March 2020, Behold.ai announced a partnership with Apollo Radiology International, part of the Apollo Hospitals Group (a network of 71 hospitals in India) and a leading provider of interpretative and other clinical radiological services [5]. The Behold.ai red dot® service was implemented in a pilot programme for the quick triage of patients with suspected COVID-19. The goal of this programme was to speed up the diagnosis of this disease and allow a better allocation of resources, thus supporting India’s response to the pandemic.

This programme was supported by the Amazon Web Services (AWS) Diagnostic Development Initiative – the aim of this Initiative is to accelerate research and innovation on COVID-19 and infectious diseases [6]. Through the support of projects on early disease detection, diagnostics, prognosis and public health genomics, the AWS Diagnostic Development Initiative seeks to achieve a better understanding of this and other infectious diseases, which can lead to effective mitigation and prevention strategies.

The ’instant triage’ of CXRs provided by Behold’s AI-based solution allowed to classify CXR exams as either Normal (meaning no visible signs of COVID-19 or other abnormalities) or Abnormal. In a period of 19 months (from May 2020 to December 2021), 82,093 CXR exams were analysed by red dot®.

An overview of the algorithm performance can be found below.

To evaluate algorithm performance, a subset of 1,289 CXRs were labelled by an FRCR Consultant Radiologist and compared with the classification generated by the algorithm. In the classification of Abnormal exams, red dot® showed an AUC of 0.91. In the classification of CXRs with visible signs of COVID-19 (based on the presence of pulmonary consolidation), red dot® showed an AUC of 0.92.

Below shows a CXR classified as Abnormal by the red dot® algorithm, along with a heatmap generated by the algorithm which shows the location of suspicious areas.

Below shows another CXR classified by the red dot® algorithm as Normal, which was subsequently confirmed by the FRCR Consultant Radiologist.

The work carried out by Behold.ai in partnership with Apollo Hospitals and with the support of the AWS Diagnostic Development Initiative showed that the red dot® platform can effectively support the triage and diagnosis of COVID-19. The work of Behold.ai contributed to tackling a health crisis, akin to the work the company is doing to tackle excess deaths from lung cancer, which have been exacerbated by the COVID-19 pandemic. The work also showed that the performance of the red dot® algorithm generalises well to populations outside of the NHS.

Future collaborations and partnerships between AI companies such as Behold.ai, hospital sites and technology initiatives will continue to pave the way for an accelerated diagnosis of diseases, including infectious diseases such as COVID-19.