Performance-portability and in-situ visualisation with SERGHEIi - HPC solutions

Research topic and goals

This project is concerned with facilitating the performance-portability, usability and HPC site portability of SERGHEI, in particular aiming towards demonstrating its potential applicability in flash flood warning systems. SERGHEI is performance-portable via Kokkos, but porting and deploying on different HPC systems still requires a certain effort. In order for the prototype to be usable, portability should be demonstrated and require minimal effort. Additionally, for flash flood emergency assessment, in situ visualisation is required to generate information for decision making on the fly, in particular because very frequent output is required, which if written to disk and then visualised would create large bottlenecks, potentially rendering the early warning prototype too slow for use. Finally, in this project we will explore multi-site workflows in which SERGHEI may be deployed for the same purpose on multiple HPC sites (internationally, between Germany, USA, and Costa Rica), operating, for example at different resolutions, or with different forcings, which would feed the same early warning simulation workflow.

This project aims to:

• demonstrate and evaluate performance portability of all modules in SERGHEI
• investigate portable optimisations for SERGHEI-SWE, and potentially other modules
• prototype of in-situ visualisation for SERGHEI-SWE, exploring the use of CINEMA and OpeanSeaDragon for interactive data explorationI
• exploring multi-site, M-to-N workflows coupling geographically distant HPC resources

Results for 2025/2026

Achieved performance-portability on all thre GPU vendors on pre- and exascale machines (JUWELS, JEDI, Aurora, Forntier).

Visits and meetings

• May 2025: JLESC Workshop lead to establishing the project

Impact and publications

Contributed a paper to SBAC-PAD (Towards Portability at Scale: A Cross-Architecture Performance Evaluation of a GPU-enabled Shallow Water Solver)(Villalobos et al. 2025).

Future plans

Performance optimisations. Re-take in-situ visualisation

References

1. Villalobos2025

   BibTeX Abstract

   Villalobos, Johansell, Daniel Caviedes-Voullième, Silvio Rizzi, and Esteban Meneses. 2025. “Towards Portability at Scale: A Cross-Architecture Performance Evaluation of a GPU-Enabled Shallow Water Solver.” In 2025 IEEE/SBC 37th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD), 180–91. https://doi.org/10.1109/SBAC-PAD66369.2025.00025.

   @inproceedings{Villalobos2025,
     author = {Villalobos, Johansell and Caviedes-Voullième, Daniel and Rizzi, Silvio and Meneses, Esteban},
     booktitle = {2025 IEEE/SBC 37th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)},
     title = {Towards Portability at Scale: A Cross-Architecture Performance Evaluation of a GPU-enabled Shallow Water Solver},
     year = {2025},
     volume = {},
     number = {},
     pages = {180-191},
     keywords = {Performance evaluation;Codes;Scalability;Memory management;Graphics processing units;Mathematical models;Numerical models;Kernel;Optimization;Arithmetic;Performance evaluation;Graphical Processing Units;Shallow Water Equations},
     doi = {10.1109/SBAC-PAD66369.2025.00025}
   }
   

2. Caviedes2023

   BibTeX Abstract

   Caviedes-Voullième, Daniel, Mario Morales-Hernández, Matthew R. Norman, and Ilhan Özgen-Xian. 2023. “SERGHEI (SERGHEI-SWE) v1.0: a Performance-Portable High-Performance Parallel-Computing Shallow-Water Solver for Hydrology and Environmental Hydraulics.” Geoscientific Model Development 16 (3): 977–1008. https://doi.org/10.5194/gmd-16-977-2023.

   @article{Caviedes2023,
     author = {Caviedes-Voulli\`eme, Daniel and Morales-Hern\'andez, Mario and Norman, Matthew R. and \"Ozgen-Xian, Ilhan},
     journal = {Geoscientific Model Development},
     title = {{SERGHEI} ({SERGHEI}-{SWE}) v1.0: a performance-portable high-performance parallel-computing shallow-water solver for hydrology and environmental hydraulics},
     year = {2023},
     month = feb,
     number = {3},
     pages = {977--1008},
     volume = {16},
     doi = {10.5194/gmd-16-977-2023},
     file = {:Caviedes2023.pdf:PDF},
     groups = {Surface flows},
     publisher = {Copernicus {GmbH}}
   }