Dynamic load balancing with Pampa in Alya

Research topic and goals

The objective of this project is to use a runtime repartitioning library PAMPA (Lachat, Dobrzynski, and Pellegrini 2014), (Mbengoue et al. 2013) to dynamically load balance a high performance computational mechanics code Alya (Houzeaux et al. 2009), (Houzeaux, Aubry, and Vázquez 2011), (Vázquez et al. 2015). The main application is the particle transport in a fluid.

In Alya, the transport of particles in a fluid is achieved using two parallel instances of the code. One instance is in charge of solving the flow equation and the other one of transporting the particles. The velocity field is sent from the first to the second one at the end of each time step through MPI. If a classical mesh partitioning based on the balancing of the number of elements is performed, the particles are likely to be located in very few subdomains, which obviously leads to a very poor load balance. In this project we aim at implementing the Pampa library in Alya in order to redistribute the elements among the MPI processes in order to dynamically balance the computation of particle transport.

Results for 2015/2016

The work have just started. The data structure required by Pampa was coded in Alya and Pampa was interfaced with Alya.

Results for 2016/2017

Initially, bridges from Alya to Pampa and Pampa to Alya were created. To do so, the data structures of Alya were adapted to Pampa’s data structures. In fact, the parallelization strategy of Alya is based on a partition into disjoint element sets, where interface nodes on the interfaces are duplicated. Thus, Alya does not require halos. However, Pampa was based on disjoint sets of nodes, which made the interface between the two codes a bit complex and heavy to manage. As the Alya data structure is quite common in the finite element community, we decided it was more convenient to adapt Pampa to this format. This work has been carried out in the first months of this year and is now being tested in Alya.

Visits and meetings

  • Guillaume Houzeaux (BSC) and Mariano Vázquez (BSC) met INRIA team at Bordeaux, 14-15 Oct. 2016.
  • Guillaume Houzeaux (BSC) and Vishal Mehta (BSC) met INRIA team at Bordeaux, 24-26 Feb. 2016.
  • Cédric Lachat (INRIA) met BSC team at Barcelona, 7-9- Dec. 2016.

Impact and publications

The first objective of this project is to dynamically load balance the particle transport. To achieve this, many efforts are required to reorganize the data structure in Alya to fit with Pampa’s. But among other features, Pampa allows parallel mesh adaptation and redistribution. Therefore all these efforts will enable to further use Pampa to perform mesh adaptation in Alya:

  • Dynamic load balance for other physical modules
  • Mesh adaptivity

Future plans

  • Implementation
  • Continuous validation
  • Application to particle transport
  • Scalability tests and performance analysis

References

  1. Vázquez, M., G. Houzeaux, S. Koric, A. Artigues, J. Aguado-Sierra, Arı́s R., D. Mira, et al. 2015. “Alya: Multiphysics Engineering Simulation Towards Exascale.” J. Comput. Sci. 
    @article{VazquezEtAl2015,
  author = {V\'azquez, M. and Houzeaux, G. and Koric, S. and Artigues, A. and Aguado-Sierra, J. and Ar\'{\i}s, R. and Mira, D. and Calmet, H. and Cucchietti, F. and Owen, H. and Taha, A. and Burness, E.D. and Cela, J.M. and Valero, M.},
  journal = {J. Comput. Sci.},
  keywords = {Alya},
  title = {Alya: Multiphysics Engineering Simulation Towards Exascale},
  year = {2015}
}
  2. Lachat, Cédric, Cécile Dobrzynski, and François Pellegrini. 2014. “Parallel Mesh Adaptation Using Parallel Graph Partitioning.” In 5th European Conference on Computational Mechanics (ECCM V), 3:2612–23. CIMNE-International Center for Numerical Methods in Engineering. 
    @inproceedings{LachatEtAl2014,
  title = {Parallel mesh adaptation using parallel graph partitioning},
  author = {Lachat, C{\'e}dric and Dobrzynski, C{\'e}cile and Pellegrini, Fran{\c{c}}ois},
  booktitle = {5th European Conference on Computational Mechanics (ECCM V)},
  volume = {3},
  pages = {2612--2623},
  year = {2014},
  organization = {CIMNE-International Center for Numerical Methods in Engineering}
}
  3. Mbengoue, Dragan Amenga, Damien Genet, Cedric Lachat, Emeric Martin, Maxime Mogé, Vincent Perrier, Florent Renac, François Rue, and Mario Ricchiuto. 2013. “Comparison of Algorithm in Aerosol and Aghora for Compressible Flows.” 
    @article{MbengoueEtAl2013,
  title = {Comparison of algorithm in Aerosol and Aghora for compressible flows},
  author = {Mbengoue, Dragan Amenga and Genet, Damien and Lachat, Cedric and Martin, Emeric and Mog{\'e}, Maxime and Perrier, Vincent and Renac, Florent and Rue, Fran{\c{c}}ois and Ricchiuto, Mario},
  year = {2013}
}
  4. Houzeaux, G, R Aubry, and M Vázquez. 2011. “Extension of Fractional Step Techniques for Incompressible Flows: The Preconditioned Orthomin(1) for the Pressure Schur Complement.” Cf 44: 297–313. 
    @article{HouzeauxEtAl2011,
  author = {Houzeaux, G and Aubry, R and V\'azquez, M},
  journal = cf,
  keywords = {Orthomin(1) iteration},
  pages = {297-313},
  title = {Extension of fractional step techniques for incompressible flows: The preconditioned Orthomin(1) for the pressure Schur complement},
  volume = {44},
  year = {2011}
}
  5. Houzeaux, G, M Vázquez, R Aubry, and JM Cela. 2009. “A Massively Parallel Fractional Step Solver for Incompressible Flows.” Jcp 228 (17): 6316–32. 
    @article{HouzeauxEtAl2009,
  author = {Houzeaux, G and V\'azquez, M and Aubry, R and Cela, JM},
  journal = jcp,
  number = {17},
  pages = {6316-6332},
  title = {A Massively Parallel Fractional Step Solver for Incompressible Flows},
  volume = {228},
  year = {2009}
}