TRUST Framework Webinar

The meeting focused on the status and possible Exa-MA integration of the CEA TRUST framework, with emphasis on:

Pierre Ledac presented TRUST as a CEA simulation framework used for fluid mechanics and thermo-hydraulic applications, with a significant application ecosystem. The framework is being progressively ported to GPU using Kokkos.

The GPU migration is already effective for several kernels and applications, but the port is still hybrid in places. Some parts of the computation remain on CPU, in particular some matrix construction or initialization steps, while GPU-resident updates and kernels are progressively introduced.

A key point is that solver and preconditioner performance is now one of the main bottlenecks for large GPU runs. TRUST currently relies on a solver ecosystem involving PETSc, HYPRE, AMGX, and potentially Ginkgo and HPDDM.

Utpal Kiran raised a question about matrix assembly. The answer was that TRUST currently has several assembly paths, depending on the application and discretization.

The current situation is hybrid:

This is not seen as a blocker for the first solver-oriented mini-apps, but it is important for future work on fully GPU-resident workflows and nonlinear/time-dependent simulations.

Pierre Jolivet asked what alternative Ginkgo solvers could perform better than the current TRUST solver stack.

Pierre Ledac answered that, at this stage, he is not yet confident that Ginkgo performs better than the current combinations used in TRUST, in particular PETSc/HYPRE and AMGX. The comparison still needs systematic benchmarking, including convergence behavior, robustness, and time-to-solution.

The solver candidates to compare include:

A specific discussion concerned structured meshes. TRUST is already using HYPRE successfully on tetrahedral/unstructured cases. However, for structured meshes, algebraic multigrid may be too costly, and structured multigrid could be a better option.

The open technical question is how to expose TRUST structured mesh information to HYPRE:

Pierre Jolivet noted that he was not sure whether the necessary PETSc plumbing exists between DMDA and HYPRE SStruct solvers, and said he would check.

Christophe Prud’homme emphasized the need for mini-apps.

The first realistic TRUST mini-apps should likely be solver-oriented rather than full discretization mini-apps. TRUST can provide representative matrices and linear systems, together with PETSc option files, allowing comparison of several preconditioners and solvers.

Mixed precision was identified as an important future direction. Once the Kokkos/GPU porting work has matured, reducing memory traffic and bandwidth pressure becomes a major optimization target.

The main motivations are:

HYPRE already seems to have some support for lower precision or single precision in parts of the solver stack, but it is not yet clear how easily this can be accessed from PETSc or how well it is documented.

A robust mixed-precision activity would require:

This could become a good Exa-MA mini-app or demonstrator topic, especially if connected to WP3 solver activities and AMD GPU work.

A possible link between TRUST and Uranie was discussed in the context of WP6 and uncertainty quantification.

Christophe recalled that Exa-MA may be able to support approximately one year of engineer time. The engineer would likely be recruited at Ecole Polytechnique but would work closely with CEA, with possible interactions around both Uranie and TRUST.

The exact roadmap still needs to be defined. Candidate activities include:

The group agreed that there is time to define the work, but that the roadmap should converge within roughly six months in order to prepare the recruitment profile.

The GENCI Grand Challenge on MI300 GPUs was discussed.

TRUST is interested, especially because access to JUPITER may be uncertain and TRUST has a large simulation campaign that could benefit from MI300 resources. Pierre Ledac estimated that TRUST might need around 100,000 GPU-hours.

Exa-MA is preparing a broader request involving several frameworks and groups. The open strategic question is whether TRUST should:

Christophe will check the best strategy and come back to Pierre. TRUST indicated that it is ready to start, including during the September-October access window.

The discussion also covered PETSc and Kokkos version management.

The main conclusion is that TRUST needs to follow GPU-related updates closely, because new versions often include important bug fixes and performance improvements. However, updates can also introduce regressions or numerical changes that are costly to analyze.

Pierre emphasized the importance of having many regression tests covering all relevant solver configurations. Such tests may become essential several years later when a PETSc, Kokkos, or solver-stack update changes behavior.

One CPU-side PETSc update was mentioned as causing blocking issues in parallel regression tests; the exact version should be checked before documenting this externally.

Pierre Ledac asked about a previous AMD-related discussion mentioning possible solver developments.

Christophe clarified that the items discussed in the AMD workshop were mostly proposals or possible co-design directions, not necessarily existing AMD solver products. Examples included GPU-resident solver paths, SpMV, smoothers, Krylov methods, collective reductions, mixed precision, convergence behavior, and memory footprint.

Christophe will check where the AMD/NumPEx discussions stand and whether TRUST could be connected to future AMD co-design activities.

TRUST appears to be a strong candidate framework for Exa-MA WP3 activities because it provides realistic, production-grade solver and preconditioner challenges on modern GPU architectures.

The most pragmatic short-term objective is not yet to define a full discretization mini-app, but rather to create a solver-oriented TRUST mini-app suite. This suite should expose representative systems, allow controlled solver/preconditioner comparisons, and provide reproducible performance and robustness metrics.

A good medium-term trajectory would be: