Simulator-Based Self-Supervision for Learned 3D Tomography Reconstruction


We propose a deep learning method for 3D volumetric reconstruction in low-dose helical cone-beam computed tomography. Prior machine learning approaches require reference reconstructions computed by another algorithm for training. In contrast, we train our model in a fully self-supervised manner using only noisy 2D X-ray data. This is enabled by incorporating a fast differentiable CT simulator in the training loop. As we do not rely on reference reconstructions, the fidelity of our results is not limited by their potential shortcomings. We evaluate our method on real helical cone-beam projections and simulated phantoms. Our results show significantly higher visual fidelity and better PSNR over techniques that rely on existing reconstructions. When applied to full-dose data, our method produces high-quality results orders of magnitude faster than iterative techniques.

Interactive figures

Interactive versions of the paper figures. See paper for details.

Use mouse wheel to zoom in/out, click and drag to pan. Press keys [1], [2], ... to switch between different methods, and left and right arrow keys to scroll through nearby slices.


We thank Timo Aila for discussions and comments on the manuscript; Samuli Siltanen, Alexander Meaney, Antti Korvenoja, Mika Kortesniemi, Juha Järveläinen and Jussi Hirvonen for discussions; David Luebke and Kimmo Kaski for general support; and Tero Kuosmanen, Janne Hellsten and Samuel Klenberg for compute infrastructure support. The project made use of computational resources provided by the Aalto Science-IT project and the Finnish IT Center for Science (CSC). Onni Kosomaa's NVIDIA Research internship notwithstanding, he was funded by the Finnish Center for Artificial Intelligence (FCAI), a part of the Academy of Finland Flagship programme.