Image Synthesis with Transformer: Brain MR Image to PET Image
University of Wisconsin-Madison, USA
2022.9 - present
Vikas Singh, PH. D, Professor in the Department of Biostatistics
Yong Jae Lee, PH. D, Associate Professor in the Department of Computer Sciences
To reduce the financial burden on the patient and to speed up the diagnostic process, we propose a novel unsupervised GAN framework to achieve the goal of using magnetic resonance imaging (MRI), an inexpensive and easy-to-use technique, to generate certain types of images of positron emission to mography (PET), which is an expensive and complex time-consuming technique.
Our key idea is to use CycleGAN to map the input image to a specified one in the output domain and use a combination of MobileNetV2 and self-attention in the process to achieve global and local feature capture of the input photo. We present experiments on the Alzheimers Disease Neuroimaging Initiative (ADNI) dataset with 726 training and 80 testing subjects and obtain acceptable per- formance in PET image synthesis. We also use various metrics to evaluate the generated images.
GitHub LINK: https://github.com/GeofrreyLi/U-TransCyGan
Swapping Auto-Encoder presentation: Slides