Keywords: high-performance computing, successive-over-relaxation, conjugate gradient method, incomplete Cholesky decomposition, MPIOM, climate simulation, mathematical modeling, ocean model, barotropic subsystem.

In this work we present some results of the "Scalable-Earth-System-Models for high productivity climate simulations" (ScalES) project funded by the German Bundesministerium für Bildung und Forschung (BMBF No. 01IH08004E). We show the efforts of improving the linear solver of the barotropic subsystem [3], discretized by finite differences, in the MPIOM by means of modern mathematical methods.

The traditional solver uses the successive-over-relaxation (SOR) method with red-black numbering and a relaxation parameter that is estimated by a number of test calculations and measurements of the rate of convergence. We implemented the conjugate gradient (CG) method with several preconditioners, most notably incomplete Cholesky decomposition with fill-in p (ICC(p)). We demonstrate different aspects of the implementation, especially all challenges in parallelization and exploitation of the IBM POWER6 architecture [4] to achieve high scalability.

The CG method is then used in combination with several preconditioners to solve the barotropic subsystem in MPIOM and compared to SOR in terms of the number of iterations and runtime. We conclude that the CG method with an appropriate preconditioner such as ICC(p) greatly reduces the necessary number of iterations, communication and eventually the runtime, even compared with the SOR method with a highly optimized relaxation parameter. Finally we discuss possible ways of further improvement with respect to the obtained results.

1

S. Marsland, H. Haak, J. Jungclaus, M. Latif, F. Röske, "The Max-Planck-Institute Global Ocean/Sea Ice Model with Orthogonal Curvilinear Coordinates", Ocean Modelling, 5, 91-127, 2003. doi:10.1016/S1463-5003(02)00015-X

2

A. Arakawa, V. Lamb, "Computational Design of the Basic Dynamical Processes of the UCLA General Circulation Model", Methods in Computational Physics, 17, 173-265, 1977.

3

E. Simonnet, T.T. Medjo, R. Temam, "Barotropic-Baroclinic Formulation of the Primitive Equations of the Ocean", Applicable Analysis, 82(5), 439-456, 2003. doi:10.1080/0003681031000094591

4

H.Q. Le, W.J. Starke, J.S. Fields, F.P. O'Connell, D.Q. Nguyen, B.J. Ronchetti, W.M. Sauer, E.M. Schwarz, M.T. Vaden, "IBM POWER6 microarchitecture", IBM J. Res. Dev., 51(6), 639-662, 2007. doi:10.1147/rd.516.0639

purchase the full-text of this paper (price £20)

go to the previous paper
go to the next paper
return to the table of contents
return to the book description
purchase this book (price £85 +P&P)