Automated Analysis and Collaborative Exploration of Brain Ultrastructure from Electron Microscopy

Abstract

Understanding brain organization at nanometer scales requires efficient analysis of electron microscopy (EM) data and intuitive access to complex three-dimensional reconstructions. This dissertation presents an integrated framework that addresses these challenges through systematic benchmarking, automated segmentation, immersive visualization, and emerging morphological analysis. First, we conduct a comprehensive systematic review and meta-analysis of deep learning methods for brain EM segmentation, covering 60 studies and quantitatively evaluating 27 across 10 datasets. Results show that foundation models outperform traditional CNNs by 13--35\%, providing evidence-based guidelines for model selection. Guided by these insights, we introduce SAM4EM, a lightweight adapter for the Segment Anything Model 2 that enables automated, prompt-free 3D segmentation of brain ultrastructure. The method features a two-stage decoder, LoRA fine-tuning for limited-data scenarios, and a 3D memory attention mechanism for temporal consistency. SAM4EM achieves Dice scores of 70.5\% for glial cells, 80.7\% for mitochondria, and 53.8\% for synapses---improving state-of-the-art performance by 1.8, 6.4, and 11.5 points, respectively. To connect computational results with practical use, we develop NeuroVerse, an immersive metaverse platform that converts segmented EM data into interactive 3D environments for collaborative neuroanatomy and research. User studies with students and neuroscientists show enhanced understanding of complex structures and support effective remote collaboration. Finally, we outline extensions toward automated 3D morphological analysis. Early results from NeuroShape demonstrate the feasibility of deep learning for geometry-based shape analysis, while ongoing GraPHFormer work incorporates topological and graph-based representations for richer ultrastructural characterization. Together, these contributions show how foundation models, efficient fine-tuning, and immersive visualization can overcome major bottlenecks in EM data analysis, enabling scalable biological discovery and broadening access to advanced neuroanatomical tools.

Date of Award	2025
Original language	American English
Awarding Institution	HBKU College of Science and Engineering