New model, built on AWS, outperforms other public models in matching MRI scans with textual descriptions in image retrieval tasks
GE HealthCare has been a pioneer in medical imaging for more than a century, including in magnetic resonance imaging (MRI). GE HealthCare, which developed the first high-field 1.5T MRI scanner in 1983, is continuing to push the boundaries of innovation into the new era of artificial intelligence (AI) with the announcement of the industry’s first full-body 3D MRI research foundation model.[1] The model is designed to enable developers to build applications for tasks such as image retrieval, classification, operational efficiency, and report generation. We developed this technology by training the first-generation of the model on a dataset of more than 173,000 MRI images from more than 19,000 studies, using only MRI images. It is the latest innovation as part of GE HealthCare’s AI Innovation Lab, an initiative designed to accelerate early-concept AI innovations within the company.
While other foundation models have relied on a series of 2D images, this new research project is the first to train a foundation model with 3D MRI data. Additionally, the model has also been trained using multi-modal data which enables added functionality such as image-to-text searches, linking words to images, segmentation, and disease classification. The new 3D model allows for more comprehensive analysis of complex anatomical structures. The vision is for this model to ultimately help care teams see more in one scan than ever before with the goal of eventually supporting enhanced diagnoses and better treatment. The model was built on Amazon Web Services (AWS) and is a continuation of our strategic collaboration announced in July.
By combining our deep MRI experience and track record of innovation in AI such as solutions like AIR™ Recon DL, Sonic DL™, and more, GE HealthCare is pioneering a new approach with its full-body 3D MRI foundation model. Compared to other publicly available research foundation models, this model showed enhanced performance due to its MRI-specific dataset, advanced 3D functionality, specialized architecture, and pre-training methods. In fact, preliminary testing has demonstrated that this new model outperforms other publicly available models in tasks such as classification of prostate cancer and Alzheimer’s disease of images with reports, even when training data is limited. For example, this model showed up to 30% accuracy in matching MRI scans with textual descriptions in image retrieval tasks — a significant improvement over the 3% capability demonstrated by similar models demonstrated in our internal testing. To go in-depth on benchmarking, read our technical post here.
Additionally, the self-supervised nature of this model means that the model can be fine-tuned with minimal image labeling, a time-consuming, costly, and challenging task for specialized medical imaging.
The ultimate goal of this research is to build a powerful model that can serve as a launch pad for a variety of new applications. Our vision is to put this model into the hands of the technical teams working within health systems and academic medical centers globally, to give them a powerful new tool for developing research and clinical applications faster, better, and more cost-effectively than ever before.
Mass General Brigham will be an early evaluator of the model. GE HealthCare and Mass General Brigham have been working closely on AI solutions since announcing their 10-year commitment in 2017 to explore the use of AI across a broad range of diagnostic and treatment paradigms through sustainable AI development.
Dr. Keith Dreyer, Chief Data Science Officer at Mass General Brigham and leader of the Mass General Brigham AI business, observed that “while general AI is getting closer to performance levels that may soon be acceptable for broad applications in medicine, in the intervening period, our research shows that foundation models fined tuned for specific medical applications can help accelerate performance gains needed to build confidence in the technology and demonstrate near-term ROI”.
“This model is a great example of how GE HealthCare continues to pioneer MRI and AI technology, thinking expansively about how to apply the latest technologies to facilitate more accurate diagnosis and better care,” said Roland Rott, President and CEO of GE HealthCare Imaging. “We see significant potential for this model to enhance interventions such as with biopsies, radiation therapy, or robotic surgery by analyzing 3D MRI data in real time, which could improve the overall success of the procedure.”
“GE HealthCare has been pioneering the development of foundation models in healthcare. We are excited by the strong performance of our newest 3D native MRI foundation model, and the overall potential this model holds to reveal new structures and accelerate research, and eventually diagnosis, and treatment,” said Dr. Taha Kass-Hout, Global Chief Science and Technology Officer for GE HealthCare. “The vision for this research project is to provide health systems with a new, powerful AI-powered tool they can customize for the clinical use cases that make the most sense for their organization. This includes building applications for tasks such as image retrieval, classification, localization, and report generation from text prompts.”
Developing advanced foundation models requires extensive compute power and sophisticated cloud-based tools. To build this model, we used a suite of AWS services to help us enhance the efficiency, scalability, and performance of the model. These services supported both the data storage and the intensive compute requirements of large-scale machine learning workloads. This included the use of Amazon SageMaker, a fully managed machine learning service, which enabled us to access ultra-high-speed networking, scale our usage up or down quickly, deploy distributed training strategies, and monitor resource utilization effectively. SageMaker’s integrated debugging and profiling tools also helped us with efficient resource usage and reduced training time by allowing us to identify bottlenecks in real-time.
“It’s exciting to be working with GE Healthcare to bring purpose-built models like this to fruition,” said Dan Sheeran, General Manager, Health Care and Life Sciences, AWS. “This is an example of how applying generative AI could both improve existing protocols and workflows as well as provide entirely new approaches to drive better experiences and care. The ability to fine-tune the model, leveraging AWS processing power and security, can maximize its impact across clinical and research applications.”
While the model is currently in the research phase, GE HealthCare is exploring ways to make the model available to customers. With its flexibility and reliability, this model has the potential to be a valuable resource for researchers, developers, and clinicians, helping improve patient outcomes. To learn more, read our technical blog.
[1] Concept only. This work is in concept phase and may never become a product. Not for Sale. Any reported results are preliminary and subject to change. Not cleared or approved by the U.S. FDA or any other global regulator for commercial availability.
