Computational & Technology Resources
an online resource for computational,
engineering & technology publications
Civil-Comp Proceedings
ISSN 1759-3433
CCP: 95
PROCEEDINGS OF THE SECOND INTERNATIONAL CONFERENCE ON PARALLEL, DISTRIBUTED, GRID AND CLOUD COMPUTING FOR ENGINEERING
Edited by:
Paper 77

Coupling HPC and Numerical Validation: Accurate and Efficient Simulation of Large-scale Hydrodynamic Events

C. Moulinec1, C. Denis2, N. Durand3, R.W. Barber1, D.R. Emerson1, X.J. Gu1, E. Razafindrakoto3, R. Issa3 and J.-M. Hervouet3

1STFC Daresbury Laboratory, Warrington, United Kingdom
2EDF R&D, SINETICS, Clamart, France
3EDF R&D, LNHE, Chatou, France

Full Bibliographic Reference for this paper
C. Moulinec, C. Denis, N. Durand, R.W. Barber, D.R. Emerson, X.J. Gu, E. Razafindrakoto, R. Issa, J.-M. Hervouet, "Coupling HPC and Numerical Validation: Accurate and Efficient Simulation of Large-scale Hydrodynamic Events", in , (Editors), "Proceedings of the Second International Conference on Parallel, Distributed, Grid and Cloud Computing for Engineering", Civil-Comp Press, Stirlingshire, UK, Paper 77, 2011. doi:10.4203/ccp.95.77
Keywords: hydrodynamics, HPC, numerical validation, numerical simulation, round-off error propagation, flooding.

Summary
suite is TELEMAC-2D, an open-source two-dimensional finite element shallow water equation code which has been specifically tailored for free-surface flows. TELEMAC-2D has undergone significant improvements over the years and has been optimised for high performance computing (HPC) applications. However, whilst its performance has been tested on several high-end machines, very few in-depth studies have been made with meshes beyond several million elements.

The paper initially considers the important issue of round-off error propagation in HPC applications. Several methods have been developed over the years to analyse the effects of round-off error propagation. Among existing validation tools, the CADNA (Control of Accuracy and Debugging for Numerical Applications) library [2], developed by the Laboratoire d'Informatique de Paris 6, appears to be the most promising approach for industrial codes. The CADNA library makes use of discrete stochastic arithmetic in order to estimate the accuracy of each floating point arithmetic operation. Unfortunately, the original CADNA library is only available for sequential programs. However, it is important to be able to quantify round-off error propagation in parallel codes since the order of the floating point operations will often differ between the serial and parallel versions of the code. In addition, round-off error propagation is exacerbated in HPC applications due to the huge increase in the number of floating point operations that are carried out. Consequently, a new library called CADNA_MPI has been developed by EDF R&D. The library is based on adapting CADNA to work with MPI programs written in Fortran 77, Fortran 90, C and C++ and uses new MPI types and reduction operators so that it is compliant with CADNA stochastic datatypes.

The paper then demonstrates the capabilities of TELEMAC-2D using a 12-million element test case of the Gironde Estuary in southwest France. The simulations were performed on four separate meshes of successively increasing resolution. The original mesh of the estuary containing approximately 0.2 million elements has been successively refined by splitting each triangular element into four sub-elements to obtain meshes containing 0.8, 3 and 12 million elements, respectively. The results show that mesh-independent predictions can be achieved in certain parts of the estuary after the first level of grid refinement. However, in other parts of the estuary, the surface elevation predictions appear to be affected by the mesh resolution even after the third level of grid refinement. This highlights the importance of being able to provide mesh-independent predictions and demonstrates the obvious benefits of high performance computing.

References