Skip to main navigation Skip to search Skip to main content

Bigger Isn’t Always Better: Towards a General Prior for Medical Image Reconstruction

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

Abstract

Diffusion models have been successfully applied to many inverse problems, including MRI and CT reconstruction. Researchers typically re-purpose models originally designed for unconditional sampling without modifications. Using two different posterior sampling algorithms, we show empirically that such large networks are not necessary. Our smallest model, effectively a ResNet, performs almost as good as an attention U-Net on in-distribution reconstruction, while being significantly more robust towards distribution shifts. Furthermore, we introduce models trained on natural images and demonstrate that they can be used in both MRI and CT reconstruction, out-performing model trained on medical images in out-of-distribution cases. As a result of our findings, we strongly caution against simply re-using very large networks and encourage researchers to adapt the model complexity to the respective task. Moreover, we argue that a key step towards a general diffusion-based prior is training on natural images.

Original languageEnglish
Title of host publicationPattern Recognition - 46th DAGM German Conference, DAGM GCPR 2024, Proceedings
EditorsDaniel Cremers, Zorah Lähner, Michael Moeller, Matthias Nießner, Björn Ommer, Rudolph Triebel
PublisherSpringer, Cham
Pages275-291
Number of pages17
ISBN (Print)9783031851803
DOIs
Publication statusE-pub ahead of print - 23 Apr 2025
Event46th Annual Conference of the German Association for Pattern Recognition, DAGM-GCPR 2024 - Munich, Germany
Duration: 10 Sept 2024 → 13 Sept 2024

Publication series

NameLecture Notes in Computer Science
Volume15297 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference46th Annual Conference of the German Association for Pattern Recognition, DAGM-GCPR 2024
Country/TerritoryGermany
CityMunich
Period10/09/24 → 13/09/24

Fields of Expertise

  • Information, Communication & Computing

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science

Keywords

  • Data-driven prior
  • Medical image reconstruction
  • Score-based model

Cite this