{ "cells": [ { "cell_type": "code", "execution_count": 1, "id": "f9a57656", "metadata": { "execution": { "iopub.execute_input": "2026-10-02T14:42:50.435193Z", "iopub.status.busy": "2026-10-02T14:42:50.434984Z", "iopub.status.idle": "2026-10-02T14:42:50.439206Z", "shell.execute_reply": "2026-10-02T14:42:50.438394Z" }, "papermill": { "duration": 0.00733, "end_time": "2026-10-02T14:42:50.439993+00:00", "exception": false, "start_time": "2026-10-02T14:42:50.432663+00:00", "status": "completed" }, "tags": [ "remove-input", "active-ipynb", "remove-output" ] }, "outputs": [], "source": [ "try:\n", " from openmdao.utils.notebook_utils import notebook_mode # noqa: F401\n", "except ImportError:\n", " !python -m pip install openmdao[notebooks]" ] }, { "cell_type": "markdown", "id": "60104937", "metadata": { "papermill": { "duration": 0.001296, "end_time": "2026-10-02T14:42:50.647603+00:00", "exception": false, "start_time": "2026-10-02T14:42:50.646307+00:00", "status": "completed" }, "tags": [] }, "source": [ "# PETScDirectSolver\n", "\n", "PETScDirectSolver is a linear solver that assembles the system Jacobian and solves the linear system with LU factorization and back substitution using the `petsc4py` library. It can handle any system topology. Since it assembles a global Jacobian for all of its subsystems, any linear solver that is assigned in any of its subsystems does not participate in this calculation (though they may be used in other ways such as in subsystem Newton solves). Unlike the standard DirectSolver which always uses SuperLU (for sparse) or LAPACK (dense) through scipy, the PETScDirectSolver has access to multiple DirectSolvers available in PETSc.\n", "\n", "PETSc is more general and has more options than scipy for direct solvers, but due to the generality PETSc provides a thicker wrapper around its solvers. So when considering whether to use `DirectSolver` or `PETScDirectSolver`, one should consider the size of their problem and sparsity pattern. Typically PETSc will be more beneficial for larger matrices where the factorization / solving speedup from a different solver is larger than the slowdown due to overhead from the heavier wrapper. For more info about the solvers exposed by the PETScDirectSolver, see this summary table of direct solvers from the [PETSc documentation](https://petsc.org/release/overview/linear_solve_table/#direct-solvers).\n", "\n", "Of the available sparse direct solvers, SuperLU, KLU, UMFPACK, and PETSc can only be used in serial. The MUMPS and SuperLU-Dist solvers can be used in serial, but will also be run distributed if the script is run with MPI. For dense matrices the solver will ignore the sparse algorithms and automatically use LAPACK, which only runs in serial.\n", "\n", "As an example, here we calculate the total derivatives of the Sellar system objective with respect to the design variable 'z', using the \"klu\" direct solver in the PETScDirectSolver." ] }, { "cell_type": "code", "execution_count": 2, "id": "382d2172", "metadata": { "execution": { "iopub.execute_input": "2026-10-02T14:42:50.749904Z", "iopub.status.busy": "2026-10-02T14:42:50.749716Z", "iopub.status.idle": "2026-10-02T14:42:52.803831Z", "shell.execute_reply": "2026-10-02T14:42:52.803202Z" }, "papermill": { "duration": 2.126101, "end_time": "2026-10-02T14:42:52.804359+00:00", "exception": false, "start_time": "2026-10-02T14:42:50.678258+00:00", "status": "completed" }, "tags": [ "remove-input", "remove-output" ] }, "outputs": [ { "data": { "application/papermill.record/text/html": "
class SellarDerivatives(om.Group):\n    """\n    Group containing the Sellar MDA. This version uses the disciplines with derivatives.\n    """\n\n    def setup(self):\n        self.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])\n        self.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])\n\n        self.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)', obj=0.0,\n                                                  x=0.0, z=np.array([0.0, 0.0]), y1=0.0, y2=0.0),\n                           promotes=['obj', 'x', 'z', 'y1', 'y2'])\n\n        self.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1', con1=0.0, y1=0.0),\n                           promotes=['con1', 'y1'])\n        self.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0', con2=0.0, y2=0.0),\n                           promotes=['con2', 'y2'])\n\n        self.set_input_defaults('x', 1.0)\n        self.set_input_defaults('z', np.array([5.0, 2.0]))\n
\n", "application/papermill.record/text/latex": "\\begin{Verbatim}[commandchars=\\\\\\{\\}]\n\\PY{k}{class}\\PY{+w}{ }\\PY{n+nc}{SellarDerivatives}\\PY{p}{(}\\PY{n}{om}\\PY{o}{.}\\PY{n}{Group}\\PY{p}{)}\\PY{p}{:}\n\\PY{+w}{ }\\PY{l+s+sd}{\\PYZdq{}\\PYZdq{}\\PYZdq{}}\n\\PY{l+s+sd}{ Group containing the Sellar MDA. This version uses the disciplines with derivatives.}\n\\PY{l+s+sd}{ \\PYZdq{}\\PYZdq{}\\PYZdq{}}\n\n \\PY{k}{def}\\PY{+w}{ }\\PY{n+nf}{setup}\\PY{p}{(}\\PY{n+nb+bp}{self}\\PY{p}{)}\\PY{p}{:}\n \\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{add\\PYZus{}subsystem}\\PY{p}{(}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{d1}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{n}{SellarDis1withDerivatives}\\PY{p}{(}\\PY{p}{)}\\PY{p}{,} \\PY{n}{promotes}\\PY{o}{=}\\PY{p}{[}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{x}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{z}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{y1}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{y2}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{]}\\PY{p}{)}\n \\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{add\\PYZus{}subsystem}\\PY{p}{(}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{d2}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{n}{SellarDis2withDerivatives}\\PY{p}{(}\\PY{p}{)}\\PY{p}{,} \\PY{n}{promotes}\\PY{o}{=}\\PY{p}{[}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{z}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{y1}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{y2}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{]}\\PY{p}{)}\n\n \\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{add\\PYZus{}subsystem}\\PY{p}{(}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{obj\\PYZus{}cmp}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{n}{om}\\PY{o}{.}\\PY{n}{ExecComp}\\PY{p}{(}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{obj = x**2 + z[1] + y1 + exp(\\PYZhy{}y2)}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{n}{obj}\\PY{o}{=}\\PY{l+m+mf}{0.0}\\PY{p}{,}\n \\PY{n}{x}\\PY{o}{=}\\PY{l+m+mf}{0.0}\\PY{p}{,} \\PY{n}{z}\\PY{o}{=}\\PY{n}{np}\\PY{o}{.}\\PY{n}{array}\\PY{p}{(}\\PY{p}{[}\\PY{l+m+mf}{0.0}\\PY{p}{,} \\PY{l+m+mf}{0.0}\\PY{p}{]}\\PY{p}{)}\\PY{p}{,} \\PY{n}{y1}\\PY{o}{=}\\PY{l+m+mf}{0.0}\\PY{p}{,} \\PY{n}{y2}\\PY{o}{=}\\PY{l+m+mf}{0.0}\\PY{p}{)}\\PY{p}{,}\n \\PY{n}{promotes}\\PY{o}{=}\\PY{p}{[}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{obj}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{x}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{z}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{y1}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{y2}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{]}\\PY{p}{)}\n\n \\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{add\\PYZus{}subsystem}\\PY{p}{(}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{con\\PYZus{}cmp1}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{n}{om}\\PY{o}{.}\\PY{n}{ExecComp}\\PY{p}{(}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{con1 = 3.16 \\PYZhy{} y1}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{n}{con1}\\PY{o}{=}\\PY{l+m+mf}{0.0}\\PY{p}{,} \\PY{n}{y1}\\PY{o}{=}\\PY{l+m+mf}{0.0}\\PY{p}{)}\\PY{p}{,}\n \\PY{n}{promotes}\\PY{o}{=}\\PY{p}{[}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{con1}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{y1}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{]}\\PY{p}{)}\n \\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{add\\PYZus{}subsystem}\\PY{p}{(}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{con\\PYZus{}cmp2}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{n}{om}\\PY{o}{.}\\PY{n}{ExecComp}\\PY{p}{(}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{con2 = y2 \\PYZhy{} 24.0}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{n}{con2}\\PY{o}{=}\\PY{l+m+mf}{0.0}\\PY{p}{,} \\PY{n}{y2}\\PY{o}{=}\\PY{l+m+mf}{0.0}\\PY{p}{)}\\PY{p}{,}\n \\PY{n}{promotes}\\PY{o}{=}\\PY{p}{[}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{con2}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{y2}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{]}\\PY{p}{)}\n\n \\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{set\\PYZus{}input\\PYZus{}defaults}\\PY{p}{(}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{x}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{l+m+mf}{1.0}\\PY{p}{)}\n \\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{set\\PYZus{}input\\PYZus{}defaults}\\PY{p}{(}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{z}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{,} \\PY{n}{np}\\PY{o}{.}\\PY{n}{array}\\PY{p}{(}\\PY{p}{[}\\PY{l+m+mf}{5.0}\\PY{p}{,} \\PY{l+m+mf}{2.0}\\PY{p}{]}\\PY{p}{)}\\PY{p}{)}\n\\end{Verbatim}\n", "application/papermill.record/text/plain": "class SellarDerivatives(om.Group):\n \"\"\"\n Group containing the Sellar MDA. This version uses the disciplines with derivatives.\n \"\"\"\n\n def setup(self):\n self.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])\n self.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])\n\n self.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)', obj=0.0,\n x=0.0, z=np.array([0.0, 0.0]), y1=0.0, y2=0.0),\n promotes=['obj', 'x', 'z', 'y1', 'y2'])\n\n self.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1', con1=0.0, y1=0.0),\n promotes=['con1', 'y1'])\n self.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0', con2=0.0, y2=0.0),\n promotes=['con2', 'y2'])\n\n self.set_input_defaults('x', 1.0)\n self.set_input_defaults('z', np.array([5.0, 2.0]))" }, "metadata": { "scrapbook": { "mime_prefix": "application/papermill.record/", "name": "code_src23" } }, "output_type": "display_data" } ], "source": [ "from openmdao.utils.notebook_utils import get_code\n", "from myst_nb import glue\n", "glue(\"code_src23\", get_code(\"openmdao.test_suite.components.sellar_feature.SellarDerivatives\"), display=False)" ] }, { "cell_type": "markdown", "id": "1f3b8cc7", "metadata": { "papermill": { "duration": 0.011428, "end_time": "2026-10-02T14:42:52.817009+00:00", "exception": false, "start_time": "2026-10-02T14:42:52.805581+00:00", "status": "completed" }, "tags": [] }, "source": [ ":::{dropdown} `SellarDerivatives` class definition \n", "\n", "{glue:}`code_src23`\n", ":::" ] }, { "cell_type": "code", "execution_count": 3, "id": "3c4a3b5d", "metadata": { "execution": { "iopub.execute_input": "2026-10-02T14:42:52.819628Z", "iopub.status.busy": "2026-10-02T14:42:52.819475Z", "iopub.status.idle": "2026-10-02T14:42:54.091000Z", "shell.execute_reply": "2026-10-02T14:42:54.090243Z" }, "papermill": { "duration": 1.273766, "end_time": "2026-10-02T14:42:54.091687+00:00", "exception": false, "start_time": "2026-10-02T14:42:52.817921+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[1790952174.042954] [runnervm8df0l:7361 :0] ib_iface.c:1269 UCX ERROR mana_0: iface 0x562693f034a0 failed to create UD QP TX wr:256 sge:6 inl:64 resp:0 RX wr:4096 sge:1 resp:0 failed: Operation not supported\n", "[1790952174.043198] [runnervm8df0l:7361 :0] ucp_worker.c:1412 UCX ERROR uct_iface_open(ud_verbs/mana_0:1) failed: Input/output error\n", "NL: NLBGS Converged in 8 iterations\n", "9.610010556989947\n", "1.7844853356313648\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[runnervm8df0l:07361] pml_ucx.c:313 Error: Failed to create UCP worker\n", "/home/runner/work/OpenMDAO/OpenMDAO/.pixi/envs/dev/lib/python3.13/site-packages/openmdao/utils/relevance.py:1232: OpenMDAOWarning:The top level group has a nonlinear solver that computes gradients, so the entire model will be included in the optimization iteration.\n" ] } ], "source": [ "import openmdao.api as om\n", "from openmdao.test_suite.components.sellar_feature import SellarDerivatives\n", "\n", "prob = om.Problem()\n", "model = prob.model = SellarDerivatives()\n", "\n", "model.nonlinear_solver = om.NonlinearBlockGS()\n", "model.linear_solver = om.PETScDirectSolver(sparse_solver_name='klu')\n", "\n", "prob.setup()\n", "prob.run_model()\n", "\n", "wrt = ['z']\n", "of = ['obj']\n", "\n", "J = prob.compute_totals(of=of, wrt=wrt, return_format='flat_dict')\n", "print(J['obj', 'z'][0][0])\n", "print(J['obj', 'z'][0][1])" ] }, { "cell_type": "code", "execution_count": 4, "id": "747672eb", "metadata": { "execution": { "iopub.execute_input": "2026-10-02T14:42:54.141038Z", "iopub.status.busy": "2026-10-02T14:42:54.140645Z", "iopub.status.idle": "2026-10-02T14:42:54.146154Z", "shell.execute_reply": "2026-10-02T14:42:54.145544Z" }, "papermill": { "duration": 0.053404, "end_time": "2026-10-02T14:42:54.146732+00:00", "exception": false, "start_time": "2026-10-02T14:42:54.093328+00:00", "status": "completed" }, "tags": [ "remove-input", "remove-output" ] }, "outputs": [ { "data": { "text/plain": [ "np.float64(2.4481205496170354e-09)" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from openmdao.utils.assert_utils import assert_near_equal\n", "\n", "assert_near_equal(J['obj', 'z'][0][0], 9.61001056, .00001)\n", "assert_near_equal(J['obj', 'z'][0][1], 1.78448534, .00001)" ] }, { "cell_type": "markdown", "id": "83a83f04", "metadata": { "papermill": { "duration": 0.003411, "end_time": "2026-10-02T14:42:54.265490+00:00", "exception": false, "start_time": "2026-10-02T14:42:54.262079+00:00", "status": "completed" }, "tags": [] }, "source": [ "## PETScDirectSolver Options" ] }, { "cell_type": "code", "execution_count": 5, "id": "90858480", "metadata": { "execution": { "iopub.execute_input": "2026-10-02T14:42:54.270218Z", "iopub.status.busy": "2026-10-02T14:42:54.269934Z", "iopub.status.idle": "2026-10-02T14:42:54.274857Z", "shell.execute_reply": "2026-10-02T14:42:54.274040Z" }, "papermill": { "duration": 0.008338, "end_time": "2026-10-02T14:42:54.275417+00:00", "exception": false, "start_time": "2026-10-02T14:42:54.267079+00:00", "status": "completed" }, "tags": [ "remove-input" ] }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", " \n", "\n", "\n", "

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
OptionDefaultAcceptable ValuesAcceptable TypesDescription
assemble_jacTrue[True, False]['bool']Activates use of assembled jacobian by this solver.
err_on_singularTrue[True, False]['bool']Raise an error if LU decomposition is singular. Must always be 'True' for the PETScDirectSolver. This option is only maintained for compatibility with parent solver methods.
iprint1N/A['int']whether to print output
rhs_checkingFalseN/A['bool', 'dict']If True, check RHS vs. cache and/or zero to avoid some solves.Can also be set to a dict of options for the LinearRHSChecker to allow finer control over it. Allowed options are: ('check_zero', 'rtol', 'atol', 'max_cache_entries', 'collect_stats', 'auto', 'verbose')
sparse_solver_namesuperlu['superlu', 'klu', 'umfpack', 'petsc', 'mumps', 'superlu_dist']N/ADirect solver algorithm from PETSc that will be used for the LU factorization and solve if the matrix is sparse. For a dense matrix, this option will be ignored and LAPACK will be automatically used.
\n", "\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "om.show_options_table(\"openmdao.solvers.linear.petsc_direct_solver.PETScDirectSolver\")" ] }, { "cell_type": "markdown", "id": "4b24297d", "metadata": { "papermill": { "duration": 0.002009, "end_time": "2026-10-02T14:42:54.279094+00:00", "exception": false, "start_time": "2026-10-02T14:42:54.277085+00:00", "status": "completed" }, "tags": [] }, "source": [ "## PETScDirectSolver Constructor\n", "\n", "The call signature for the `PETScDirectSolver` constructor is:\n", "\n", "```{eval-rst}\n", " .. automethod:: openmdao.solvers.linear.petsc_direct_solver.PETScDirectSolver.__init__\n", " :noindex:\n", "```" ] } ], "metadata": { "celltoolbar": "Tags", "kernelspec": { "display_name": "petsc-test", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.14" }, "orphan": true, "papermill": { "default_parameters": {}, "duration": 5.429499, "end_time": "2026-10-02T14:42:55.198070+00:00", "environment_variables": {}, "exception": null, "input_path": "/home/runner/work/OpenMDAO/OpenMDAO/openmdao/docs/openmdao_book/features/building_blocks/solvers/petsc_direct_solver.ipynb", "output_path": "/home/runner/work/OpenMDAO/OpenMDAO/openmdao/docs/_executed_book/features/building_blocks/solvers/petsc_direct_solver.ipynb", "parameters": {}, "start_time": "2026-10-02T14:42:49.768571+00:00", "version": "2.7.0" } }, "nbformat": 4, "nbformat_minor": 5 }