TOUGH4 uses standard distributed CSR format matrix for storage of linear equation systems generated by the Newton-Raphson iteration. This provides great flexibility for calling any third-party linear solvers which use the standard CSR or BSR format matrix as input (A function for transforming CSR to BSR format is provided in the code). TOUGH4 includes interface with several well-known solver packages including AMGCL, VIENNACL, FASP, PETSC, TRILINOS, AMGX and ROCALUTION. The original serial linear solver of TOUGH code has become obsolete as it does not provide any advantage for modern computing hardware. In TOUGH4, AMGCL (Demidov, 2019) was designed as the default linear solver. AMGCL supports OPENMP, MPI and GPU parallel computing and allows the use of efficient AMG-CPR preconditioner. All the other linear equation solving packages have their own advantages. Table 3 summarizes the advantages and disadvantages of different packages.
Table 3 Comparison of linear solver packages
AMGCL
Include CPR preconditioner, support OPENMP, MPI and GPU parallel computing.
ILU is only available using OPENMP. MPI parallel computing is not very efficient.
VIENNACL
Allow use iterative solvers on both AMD and Nvidia GPU.
Only allow single GPU, no MPI.
FASP
Support Fast Auxiliary Space Preconditioning, support OPENMP.
Do not support MPI and GPU.
PETSC
Allow using many external packages, Include ILU, Jacobi, additive Schwarz preconditioner, allow user to build CPR preconditioner. MPI parallel computing is very efficient.
No or very limit support to hybrid parallel computing, limit support to GPU, no parallel ILU.
TRILINOS
Include ILU, Jacobi, additive Schwarz, and AMG, sometime better performance than PETSC.
No CPR preconditioner, difficult to compile on WINDOWS PC.
AMGX
Allow multiple GPUs.
For Nvidia GPU only.
ROCALUTION
Allow multiple GPUs, support OPENCL for different type of GPUs.
Limit efficient preconditioner.
TOUGH4 offers the option to link selected solver packages (except the default AMGCL package which is always included). Selection of the solver package, solving method, preconditioner, and input of other computing parameters are through keyword "".
