AN AI-BASED VISUAL EXPLORATION FRAMEWORK FOR NEUROSCIENCE AND HISTOPATHOLOGY

Abstract

The recent past has seen an ever-increasing trend to base clinical decisions on medical imaging. Fueled by advances in imaging technology, the availability of imaging devices, and the computational power to process such images, most diagnostic tasks nowadays rely on a variety of scanning technologies to detect objects and provide evidence-driven conclusions. As image sizes continue to increase beyond what humans can understand from traditional images, computer-aided visualization and feature extraction have gained traction. This work studies how recent advances in visualization driven by artificial intelligence can impact the workflows in histopathology and neuroscience. These two fields routinely generate massive data sets, ranging from several tens of billions of samples (histopathology) to trillions or even quadrillions (neuroscience). Clearly, tasks such as extracting features using localization or segmentation and their classification become very repetitive tasks with the potential to impose high cognitive loads for domain experts. Combined with the lack of time to give each case full consideration, we postulate that both fields benefit from novel software tools to support experts and assist them in focusing on the scientific question as opposed to tedious and time-consuming menial tasks. In five related publications, we present preliminary insights: (a) how "engineered" shape feature descriptors can be used in two and three dimensions to describe and classify cell nuclei in medical images. We show that our shape descriptors can serve as input features for visual analysis using shallow and deep learning supervised and unsupervised interpretation based on shape characteristics of different cell nuclear envelopes in histopathology and neuroscience imaging. (b) how recent advances in deep learning localization and classification technologies can be used for visual annotation of large images in histopathology. Our visual annotations and cell nuclei quantification allow histopathologists to work at both the nucleus and the tissue level, a feature appreciated by domain experts in a qualitative user study. (c) how a novel GPU-based data structure that we call the Mixture Graph can be utilized to store, visualize, and query segmented (nominal) volume data arising from neuro- and material science across all levels of detail. To prove the efficiency of our Mixture Graph data structure, three categorical volume data sets are factorized for efficient, pre-filtered rendering at interactive frame rates. The Mixture Graph also provides an efficient way to compute pre-filtered volume lighting correctly and to interactively explore segments based on shape, geometry, and orientation using multi-dimensional transfer functions. Qualitative and quantitative evaluation experiments show that our visual exploration solutions provide a set of features that can be used to augment workflows of neuroscience and histopathology.

Date of Award	2022
Original language	American English
Awarding Institution	HBKU College of Science and Engineering