It has been shown that thePRESB preconditioning method applied for two-by-two block matrix systems with square blocks can outperform other methods, such as the block diagonally preconditioned MINRES method. The PRESB method can be accelerated by the GMRES method, which results in a superlinear rate of convergence.
Since in some problems the eigenvalue bounds are known and often tight, one can as an alternative method use a Chebyshev acceleration which doesn’t give a superlinear convergence but saves computational vector inner products and therefore saves wasted elapsed computer times for global communications between processors.
Acknowledgements.
The research of O. Axelsson has been supported by the Ministry of Education, Youth and Sports from the National Programme of Sustainability (NPU II) project ”IT4 Inno-vations excellence in science LQ1602”.
The research of J. Kar´atson has been supported by the BME NC TKP2020 grant of NKFIH Hungary and also carried out in the ELTE Institutional Excellence Program (1783-3/2018/FEKUTSRAT) supported by the Hungarian Ministry of Human Capaci-ties, and further, it was supported by the Hungarian Scientific Research Fund OTKA SNN125119.
References
[1] R.S. Varga, Matrix Iterative Analysis. Prentice-Hall, Inc., Englewood Cliffs, N. J., 1962.
[2] D. M. Young, Iterative Solution of Large Linear Systems,Academic Press, New York, 1971.
[3] O. Axelsson, V.A. Barker, Finite Element Solution of Boundary Value Problems.
Theory and Computation. Academic Press, Inc. Orlanda, Fl. 1984. Reprinted in SIAM’s Classical series in Applied Mathematics, Philadelphia, PA, USA, 2001.
[4] L.A. Hageman, D.M. Young, Applied Iterative Methods. Academic Press, Inc. San Diego, 1981. Anabridged Republication, Dover Publications, Inc. Mineola, New York, 2004.
[5] O. Axelsson, Iterative Solution Methods. Cambridge University Press, Cambridge 1994.
[6] Y. Saad, Iterative Methods for Sparse Linear Systems. PWS Publishing Company, Boston, 1996. Society for Industrial and Applied Mathematics, 2 edition, 2003.
[7] O. Axelsson, M. Neytcheva, B. Ahmad. A comparison of iterative methods to solve complex valued linear algebraic systems. Numerical Algorithms 66 (2014), 811-841.
[8] Z.-Z. Bai, On preconditioned iteration methods for complex linear systems. J. Eng.
Math. 93(2015), 41-60.
[9] Z. Zhong, G.-F. Zhang and M.-Z. Zhu, A block alternating splitting iteration method for classical block two-by-two complex linear systems,J.Comput. Appl. Math.
288(2015), 203-214.
[10] J. Wang, X. Guo, H. Zhong, Accelerated GPMHSS Method for Solving Complex Systems of Linear Equations. East Asian J. Appl. Math.7(2017), 143-155.
[11] O. Axelsson, P.S. Vassilevski, A black box generalized conjugate gradient solver with inner iterations and variable-step preconditioning.SIAM. J. Matrix Anal. Appl.
12(1991), 625-644.
[12] Y. Saad, A flexible inner-outer preconditioned GMRES algorithm. SIAM J. Sci.
Comput. 14(1993), 461-469.
[13] O. Axelsson, S. Farouq, M. Neytcheva, Comparison of preconditioned Krylov sub-space iteration methods for PDE-constrained optimization problems. Poisson and convection-diffusion control. Numerical Algorithms 73(2016), 631-663.
[14] O. Axelsson, M. Neytcheva, Operator splittings for solving nonlinear, coupled multi-physics problems with an application to the numerical solution of an interface prob-lem. TR 2011-009, Department of Information Technology, Uppsala University, April 2011.
[15] O. Axelsson, A. Kucherov, Real valued iterative methods for solving complex sym-metric linear systems. Numer. Linear Algebra Appl.7(2000), 197-218.
[16] P. Boyanova, M. Do-Quang, M. Neytcheva, Efficient preconditioners for large scale binary Cahn-Hilliard Models. Computational Methods in Applied Mathemat-ics 12(2012), 1-22.
[17] O. Axelsson, P. Boyanova, M. Kronbichler, M. Neytcheva, X. Wu, Numerical and computational efficiency of solvers for two-phase problems. Computers & Mathemat-ics with Applications, 65(2013), 301-314.
[18] P. Boyanova, M. Neytcheva, Efficient numerical solution of discrete multi-component Cahn-Hilliard systems Computers and Math. Appl., 67 (2014), 106–121.
[19] O. Axelsson, D. Luk´aˇs, Preconditioning methods for eddy current optimally con-trolled time-harmonic electromagnetic problems, J. Numer. Math.27(2019), 121.
[20] O. Axelsson, D. Luk´aˇs, Preconditioners for time-harmonic optimal control eddy-current problems, in: Lirkov I., Margenov S. (eds), Large-Scale Scientific Computing, LSSC 2017,Lecture Notes in Computer Science, vol. 10665, Springer, Cham, pp. 47-54.
[21] Z.-Z. Liang, O. Axelsson, M. Neytcheva, A robust structured preconditioner for time-harmonic parabolic optimal control problems,Numer. Algorithms 79(2018), 575-596.
[22] O. Axelsson, M. Neytcheva, Z.-Z. Liang, Parallel solution methods and precondition-ers for evolution equations. Math Model Anal., 23 (2018), 287–308.
[23] J. Pearson, A. Wathen, A new approximation of the Schur complement in precon-ditioners for PDE-constrained optimization. Numer. Linear Algebra Appl. 19(2012), 816-829.
[24] T. Rees, M. Stoll, Block-triangular preconditioners for PDE-constrained optimiza-tion, Numer. Linear Algebra Appl. 17 (2010), 977-996.
[25] Z.-Z. Bai, Block preconditioners for elliptic PDE-constrained optimization problems.
Computing 91(2011), 379-395.
[26] M. Stoll, A. Wathen, Preconditioning for partial differential equation constrained optimization with control constraints. Numer. Linear Algebra Appl. 19 (2012), 53-71.
[27] V. Simoncini, Reduced order solution of structured linear systems arising in certain PDE-constrained optimization problems, Comput Optim Appl. 53 (2012), 591-617.
[28] M. Kolmbauer, U. Langer, A robust preconditioned MINRES solver for dis-tributed time-periodic eddy current optimal control problems.SIAM J. Sci. Comput.
34(2012), B785-B809.
[29] M. Kollmann, W. Zulehner, A robust preconditioner for distributed optimal control for Stokes flow with control constraints. Numer. Math. Adv. Appl. 2011, 771-779.
Springer, Berlin, Heidelberg, 2013.
[30] J.-W. Pearson, M. Stoll, A.-J. Wathen, Preconditioners for state-constrained optimal control problems with Moreau-Yosida penalty function.Numer. Linear Algebra Appl.
21(2014), 81-97.
[31] B. Morini, V. Simoncini, M. Tani, A comparison of reduced and unreduced KKT systems arising from interior point methods, Comput Optim Appl. 68 (2017), 1-27.
[32] Y.-F. Ke, Ch.-F. Ma, Some preconditioners for elliptic PDE-constrained optimization problems, Comput. Math. Appl., 75(2018), 2795-2813.
[33] W. Zulehner, Efficient solvers for saddle point problems with applications to PDE-constrained optimization. Advanced finite element methods and applications, 197-216, Lect. Notes Appl. Comput. Mech., 66, Springer, Heidelberg, 2013.
[34] Z.-Z. Bai, G. Golub, M. Ng, Hermitian and skew-Hermitian splitting methods for non-Hermitian positive definite linear systems, SIAM J. Matrix Anal. Appl. 24 (2003), 603-626.
[35] Y. Dong, C. Gu, On PMHSS iteration methods for continuous Sylvester equations, J. Comput. Math. 35 (2017), 600-619.
[36] C. C. Paige, M. A. Saunders. Solution of sparse indefinite systems of linear equations, SIAM J. Numerical Analysis, 12 (1975), 617–629.
[37] Z.-Z. Bai, M. Benzi, Regularized HSS iteration methods for saddle-point linear sys-tems. BIT Numer. Math. 57(2017), 287-311.
[38] Z.-Z. Bai, M Benzi, F Chen, On preconditioned MHSS iteration methods for complex symmetric linear systems, Numer. Algorithms 56 (2011), 297-317.
[39] Z.-Z. Bai, G. Golub, Accelerated Hermitian and skew-Hermitian splitting iteration methods for saddle-point problems. IMA J. Numer. Anal. 27 (2007), 1-23.
[40] J. Sch¨oberl, W. Zulehner, Symmetric indefinite preconditioners for saddle point prob-lems with applications to PDE-constrained optimization probprob-lems. SIAM J Matrix Anal Appl. 29 (2007), 752-773.
[41] Z.-Z. Bai, M. Benzi, F. Chen, Z.-Q. Wang, Preconditioned MHSS iteration methods for a class of block two-by-two linear systems with applications to distributed control problems. IMA J. Numer. Anal. 33(2013), 343-369.
[42] A. Battermann, E. Sachs, Block preconditioners for KKT systems in PDE-governed optimal control problems, Schulz V. (eds) Fast Solution of Discretized Optimization Problems. ISNM International Series of Numerical Mathematics, vol 138. Birkhuser, Basel, 2000, 1-18.
[43] J.-W. Pearson, M. Stoll, A.-J. Wathen, Regularization-Robust Preconditioners for Time-Dependent PDE-Constrained Optimization Problems, SIAM J Matrix Anal Appl., 33 (2012), 1126-1152.
[44] Yi-Fen Ke, Chang-Feng Ma, Some preconditioners for elliptic PDE-constrained op-timization problems, Comput Math Appl. 75 (2018), 2795-2813.
[45] R. Becker, B. Vexler, Optimal control of the convection-diffusion equation using stabilized finite element methods, Numer Math. 106, 349-367 (2007).
[46] O. Axelsson, S. Farouq, M. Neytcheva. Comparison of preconditioned Krylov sub-space iteration methods for PDE-constrained optimization problems. Stokes control.
Numerical Algorithms, 74(2017), 19-37.
[47] E. Haber, U.M. Ascher, Preconditioned all-at-once methods for large, sparse param-eter estimation problems, Inverse Problems 17 (2001), 1847-1864.
[48] O. Axelsson, R. Blaheta, M. B´ereˇs, A boundary optimal control identification prob-lem. In preparation.
[49] A.T. Barker, T. Rees, M. Stoll, A Fast Solver for anH1Regularized PDE-Constrained Optimization Problems, Comm. Comput. Physics, 19(2016), 143-167.
[50] O. Axelsson, S. Farouq, M. Neytcheva, A preconditioner for optimal control problems constrained by Stokes equation with a time-harmonic control.J. Comp. Appl. Math.
310(2017), 5-18.
[51] A. Greenbaum, V. Pt´ak, Z. Strakoˇs. Any nonincreasing convergence curve is possible for GMRES. SIAM J. Matrix Anal. Appl. 17(1996), 465-469.
[52] O. Axelsson, Z.-Z. Liang, Parameter modified versions of preconditioning and ite-rative inner product free refinement methods for two-by-two block matrices. Lin.
Algebra Appl., 582 (2019), 403-429.
[53] M. Kolmbauer, The Multiharmonic Finite Element and Boundary Element Method for Simulation and Control of Eddy Current Problems. Ph.D. thesis,Johannes Kepler Universit¨at, Linz, Austria (2012).
[54] S.-M. Cao, W. Feng, Z.-Q. Wang, On a type of matrix splitting preconditioners for a class of block two-by-two linear systems. Appl. Math. Lett. 79(2018), 205-210.
[55] Z.-Z. Bai. Rotated block triangular preconditioning based on PMHSS, Science China Mathematics, 56:12(2013), 2523–2538.
[56] T. Rossi, J. Toivanen. A parallel fast direct solver for block tridiagonal systems with separable matrices of arbitrary dimension, SIAM J. Sci. Comput. 20 (1999), no. 5, 1778–1796 (electronic).
[57] Z.-Q. Wang, On a Chebyshev accelerated splitting iteration method with application to two-by-two block linear systems, Numer. Linear Algebra Appl. 2018; 25:e2172.
https://doi.org/10.1002/nla.2172.
[58] O. Axelsson, D.K. Salkuyeh, A new version of a preconditioning method for certain two-by-two block matrices with square blocks.BIT Numer. Math.59 (2019), 321-342.
[59] I. Moret, A note on the superlinear convergence of GMRES,SIAM J. Numer. Anal.
34 (1997), 513–516.
[60] I. Gohberg, S. Goldberg, M.A. Kaashoek, Classes of Linear Operators, Vol. I., Ope-rator Theory: Advances and Applications, 49, Birkh¨auser Verlag, Basel, 1990.
[61] C.I. Goldstein, T.A. Manteuffel, S.V. Parter, Preconditioning and boundary condi-tions withoutH2estimates: L2condition numbers and the distribution of the singular values, SIAM J. Numer. Anal. 30 (1993), no. 2, 343–376.
[62] O. Axelsson, J. Kar´atson, F. Magoules, Superlinear convergence using block precon-ditioners for the real system formulation of complex Helmholtz equations. Comput.
Appl. Math., 2018, DOI 10.1016/j.cam.2018.01.029.
[63] O. Axelsson, J. Kar´atson, F. Magoules, Superlinear convergence under complex shifted Laplace preconditioners for Helmholtz equations.
www.cs.elte.hu/˜karatson/Helmholtz-preprint.pdf
[64] O. Axelsson, J. Kar´atson, Superlinear convergence of the GMRES for PDE-constrained optimization problems, Numer. Funct. Anal. Optim. 39 (2018), no. 9, 921–936.
[65] O. Axelsson, M. Neytcheva, A. Str¨om, An efficient preconditioning method for the state box-constrained optimal control problem, J. Num. Math. 26(2018), 185-207.
[66] R. Herzog and E. Sachs, Preconditioned conjugate gradient method for optimal control problems with control and state constraints, SIAM J. Matrix Anal. Appl., 31(2010), 2291-2317.
[67] R. Winter, Some superlinear convergence results for the conjugate gradient method, SIAM J. Numer. Anal., 17 (1980), 14-17.
[68] O. Axelsson, J. Kar´atson, Mesh independent superlinear PCG rates via compact-equivalent operators, SIAM J. Numer. Anal., 45 (2007), No.4, pp. 1495-1516.
[69] O. Axelsson, J. Kar´atson, Superlinearly convergent CG methods via equivalent pre-conditioning for nonsymmetric elliptic operators, Numer. Math. 99 (2004), No. 2, 197-223.
[70] O. Axelsson, J. Kar´atson, Equivalent operator preconditioning for linear elliptic prob-lems, Numer. Algorithms, 50 (2009), Issue 3, p. 297-380.
[71] O. Axelsson, J. Kar´atson, Superlinear Convergence of the GMRES for PDE-Constrained Optimization Problems, Numerical Functional Analysis and Optimiza-tion, 39:9 ( 2018), 921-936.
[72] K. Ito, K. Kunisch, Semi-smooth Newton methods for state-constrained optimal control problems. Systems & Control Letters 50 (2003), 221-228.
[73] M. Hinterm¨uller, M. Hinze, Moreau-Yosida regularization in state constrained elliptic control problems: error estimates and parameter adjustment. SIAM J Numer Anal.
47 (2009), 1666-1683.
[74] R. Herzog, E. Sachs, Preconditioned conjugate gradient method for optimal control problems with control and state constraints SIAM J Matrix Anal Appl. 31 (2010), 2291-2317.
[75] M. Porcelli, V. Simoncini, M. Tani, Preconditioning of active-set Newton methods for PDE-constrained optimal control problems. SIAM J Sci Comput. 37 (2015), S472-S502.
[76] V. Faber, T. Manteuffel, S.V. Parter, On the theory of equivalent operators and ap-plication to the numerical solution of uniformly elliptic partial differential equations, Adv. in Appl. Math., 11 (1990), 109-163.