Multi-Modal Medical Imaging Diagnostic Assistant (X-Ray + Clinical Notes)
Dual-stream neural network fusing Vision Transformer (ViT) chest X-ray features with patient clinical text for pneumonia & effusion detection.
Project Overview
Fuses convolutional/transformer image embeddings from chest radiographs with unstructured physician notes using cross-attention mechanisms. Achieves superior diagnostic sensitivity compared to vision-only models while generating highlighted Grad-CAM visual heatmaps.
Fuses convolutional/transformer image embeddings from chest radiographs with unstructured physician notes using cross-attention mechanisms. Achieves superior diagnostic sensitivity compared to vision-only models while generating highlighted Grad-CAM visual heatmaps.
Dual-stream neural network fusing Vision Transformer (ViT) chest X-ray features with patient clinical text for pneumonia & effusion detection.