Architecture and Hyperparameter Search for Super-Resolution Networks Operating on Medical Images
Research topic and goals
Diagnosis of pathologies in the human respiratory system have recently included results of computational fluid dynamics (CFD) simulations, which allow numerical quantification of respiratory flows by the pressure loss, the temperature distribution, etc. (Lintermann, Meinke, and Schröder 2013). Highly resolved computational meshes based on computer tomography (CT) images are necessary to accurately simulate the respiratory flow. However, clinical CT image resolution is often limited by radiation dose and acquisition time. Super-resolution networks (SRNs) have the potential to increase the resolution of images a posterior the recording. In this project, SRNs are employed and optimized to recover high-resolution (HR) from low-resolution (LR) CT images. SRNs predictions are validated by comparing the results of CFD simulations, carried out with the highly scalable lattice-Boltzmann method (LBM) (Lintermann, Meinke, and Schröder 2020).
The performance of SRNs is highly dependent on the hyperparameters, either related to model architecture or optimizer. Finding the optimal architectures and hyperparameters is limited by computational resources as the search space is often too large to explore exhaustively. The DeepHyper (DH) framework aims to tackle these challenges by employing an asynchronous Bayesian optimization (BO) approach for hyperparameter and architecture search at HPC scale (Balaprakash et al. 2018). Another limitation of existing SRNs is that they provide forecasts without any uncertainty estimates. In this project, the developers will build on DeepHyper’s automated deep ensemble for uncertainty quantification capability (AutoDEUQ). DeepHyper/AutoDEUQ estimates aleatoric and epistemic uncertainties by: automatically generating a catalog of neural networks models through joint neural architecture and hyperparameter search, wherein each model is trained to model the distribution of the data; and selecting a set of high-performing models to construct the ensembles, and estimating aleatoric and epistemic uncertainties from the generated model ensembles.
It is the aim of the proposed cooperation to investigate architecture and hyperparameter search algorithms for SRNs for enhancing resolution of CT images. This includes performance, scalability, and accuracy analyses of DH. The findings will be juxtaposed to those obtained employing similar tools for distributed hyperparameter optimization such as Ray Tune. JSC brings in its knowledge about SRNs, medical data, and CFD simulations, and ANL contributes with its expertise in architecture and hyperparameter search in general, and in employing DH on HPC systems.
Results for 2021/2022
The dataset consists of CT head recording from 65 patients. HR images were 1mm thick axial slices with matrix size of 512*512. LR images were generated by taking average of 3 adjacent HR slices, and the middle HR slice was taken as ground truth. The architecture of original U-net (Ronneberger, Fischer, and Brox 2015) was built with batch normalization, trained with LR image as input and middle HR slice as label. For comparison, resolution was also resumed using spline interpolation. The trained network was tested on 3 patients, and the U-net prediction outperforms interpolation results by comparing peak signal-to-noise ratio. Surfaces were generated using U-net prediction, HR, LR images and interpolation results. LBM simulation was carried out (Lintermann, Meinke, and Schröder 2020) for one of the patients and the averaged pressure loss between inlets and outlet was 9.695 Pa for HR, 8.965 Pa (-8.5 %) for LR, 7.706 Pa (-20.5 %) for interpolated, 9.849 Pa (+1.6 %) for U-net prediction. The above results are used as baseline for the current project.
Results for 2023/2024
In 2023, we started a collaboration with IAS-8 (Data Analytics and Machine Learning) in Forschungszentrum Jülich, to include a data efficient training method (Quercia et al. 2023) to bias SGD towards more relevant image regions, leading to faster convergence and more stable results. By the end of 2023, we summarized current results and submitted to the JLESC special issue. In addition, a webpage applction has been under development using the JSC Coud computing resources, to allow users upload their coarse CT head data and make prediction of fine CT data uisng the SRN developed in this project.
Visits and meetings
- 28-30, Sep 2022: Mario Rüttgers participated in 14th JLESC Workshop at The University of Illinois at Urbana-Champaign and started the collaboration
- Project members had online kick-off meetings for detailed plans, and meet regularly to discuss progress
- 21-23, Mar 2023: Xin Liu participated in 15th JLESC Workshop at INRIA Bordeaux to present project progress
Impact and publications
Future plans
- Collection of best-practice methods for architecture and hyperparameter search for SRNs using different frameworks
- Deployment of DH as a standard module for HPC systems at JSC
- Manuscript for the special issue journal publication in the framework of the 15th JLESC workshop in Bordeaux, France
- Develop and maintain the webpage application
References
-
Quercia, Alessio, Abigail Morrison, Hanno Scharr, and Ira Assent. 2023. “Sgd Biased towards Early Important Samples for Efficient Training.” In IEEE International Conference on Data Mining (ICDM). IEEE.
@inproceedings{quercia2023, title = {Sgd biased towards early important samples for efficient training}, author = {Quercia, Alessio and Morrison, Abigail and Scharr, Hanno and Assent, Ira}, booktitle = {IEEE International Conference on Data Mining (ICDM)}, pages = {}, year = {2023}, organization = {IEEE} } -
Lintermann, Andreas, Matthias Meinke, and Wolfgang Schröder. 2020. “Zonal Flow Solver (ZFS): a Highly Efficient Multi-Physics Simulation Framework.” International Journal of Computational Fluid Dynamics 34 (7-8). Taylor & Francis: 458–85.
@article{lintermann2020zonal, title = {Zonal Flow Solver (ZFS): a highly efficient multi-physics simulation framework}, author = {Lintermann, Andreas and Meinke, Matthias and Schr{\"o}der, Wolfgang}, journal = {International journal of computational fluid dynamics}, volume = {34}, number = {7-8}, pages = {458--485}, year = {2020}, publisher = {Taylor \& Francis} } -
Balaprakash, Prasanna, Michael Salim, Thomas D Uram, Venkat Vishwanath, and Stefan M Wild. 2018. “DeepHyper: Asynchronous Hyperparameter Search for Deep Neural Networks.” In 2018 IEEE 25th International Conference on High Performance Computing (HiPC), 42–51. IEEE.
@inproceedings{balaprakash2018deephyper, title = {DeepHyper: Asynchronous hyperparameter search for deep neural networks}, author = {Balaprakash, Prasanna and Salim, Michael and Uram, Thomas D and Vishwanath, Venkat and Wild, Stefan M}, booktitle = {2018 IEEE 25th international conference on high performance computing (HiPC)}, pages = {42--51}, year = {2018}, organization = {IEEE} } -
Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. 2015. “U-Net: Convolutional Networks for Biomedical Image Segmentation.” In International Conference on Medical Image Computing and Computer-Assisted Intervention, 234–41. Springer.
@inproceedings{ronneberger2015u, title = {U-net: Convolutional networks for biomedical image segmentation}, author = {Ronneberger, Olaf and Fischer, Philipp and Brox, Thomas}, booktitle = {International Conference on Medical image computing and computer-assisted intervention}, pages = {234--241}, year = {2015}, organization = {Springer} } -
Lintermann, Andreas, Matthias Meinke, and Wolfgang Schröder. 2013. “Fluid Mechanics Based Classification of the Respiratory Efficiency of Several Nasal Cavities.” Computers in Biology and Medicine 43 (11). Elsevier: 1833–52.
@article{lintermann2013fluid, title = {Fluid mechanics based classification of the respiratory efficiency of several nasal cavities}, author = {Lintermann, Andreas and Meinke, Matthias and Schr{\"o}der, Wolfgang}, journal = {Computers in biology and medicine}, volume = {43}, number = {11}, pages = {1833--1852}, year = {2013}, publisher = {Elsevier} }