{ "cells": [ { "cell_type": "code", "execution_count": 1, "id": "07024984", "metadata": { "execution": { "iopub.execute_input": "2026-10-02T14:44:31.016941Z", "iopub.status.busy": "2026-10-02T14:44:31.016609Z", "iopub.status.idle": "2026-10-02T14:44:31.022420Z", "shell.execute_reply": "2026-10-02T14:44:31.021719Z" }, "papermill": { "duration": 0.009231, "end_time": "2026-10-02T14:44:31.023232+00:00", "exception": false, "start_time": "2026-10-02T14:44:31.014001+00:00", "status": "completed" }, "tags": [ "remove-input", "active-ipynb", "remove-output" ] }, "outputs": [], "source": [ "try:\n", " from openmdao.utils.notebook_utils import notebook_mode # noqa: F401\n", "except ImportError:\n", " !python -m pip install openmdao[notebooks]" ] }, { "cell_type": "markdown", "id": "e50a8314", "metadata": { "papermill": { "duration": 0.000752, "end_time": "2026-10-02T14:44:31.025045+00:00", "exception": false, "start_time": "2026-10-02T14:44:31.024293+00:00", "status": "completed" }, "tags": [] }, "source": [ "# Instance-Based Call Tracing\n", "\n", "The `openmdao trace` command can be used to print a trace of each instance method call. For example:\n", "\n", "``` bash\n", " openmdao trace \n", "```\n", "\n", "Whenever a method is called that matches the search criteria, the following will be written to the console, indented based on its location in the call stack. :\n", "\n", "1. Pathname of the object instance (if available)\n", "2. Class\n", "3. Instance ID\n", "4. Method name\n", "\n", "For example:\n", "```\n", " Problem#1.Problem.__init__\n", " Group#1.Group.__init__\n", " Group#1.System.__init__\n", " DictionaryJacobian#1.Jacobian.__init__\n", " Group#1().System.initialize\n", " NonlinearRunOnce#1.Solver.__init__\n", " NonlinearRunOnce#1.Solver._declare_options\n", " LinearRunOnce#1.LinearSolver.__init__\n", " LinearRunOnce#1.Solver.__init__\n", " LinearRunOnce#1.Solver._declare_options\n", " Driver#1.Driver.__init__\n", " IndepVarComp#1.IndepVarComp.__init__\n", " IndepVarComp#1.ExplicitComponent.__init__\n", " IndepVarComp#1.Component.__init__\n", " IndepVarComp#1.System.__init__\n", " DictionaryJacobian#2.Jacobian.__init__\n", " IndepVarComp#1().System.initialize\n", " Propulsor#1.Group.__init__\n", " Propulsor#1.System.__init__\n", " DictionaryJacobian#3.Jacobian.__init__\n", " Propulsor#1().System.initialize\n", " NonlinearRunOnce#2.Solver.__init__\n", " NonlinearRunOnce#2.Solver._declare_options\n", " LinearRunOnce#2.LinearSolver.__init__\n", " LinearRunOnce#2.Solver.__init__\n", " LinearRunOnce#2.Solver._declare_options\n", " Problem#1.Problem.set_solver_print\n", " Group#1().System.set_solver_print\n", " LinearRunOnce#2.Solver._set_solver_print\n", " NonlinearRunOnce#2.Solver._set_solver_print\n", " Problem#1.Problem.setup\n", " Group#1().System._setup\n", " Group#1().System._get_initial_procs\n", " Group#1().Group._setup_procs\n", " IndepVarComp#1().System._setup_procs\n", " Propulsor#1().Group._setup_procs\n", " FlightConditions#1.Group.__init__\n", " FlightConditions#1.System.__init__\n", " DictionaryJacobian#4.Jacobian.__init__\n", " FlightConditions#1().FlightConditions.initialize\n", " NonlinearRunOnce#3.Solver.__init__\n", " NonlinearRunOnce#3.Solver._declare_options\n", " LinearRunOnce#3.LinearSolver.__init__\n", " LinearRunOnce#3.Solver.__init__\n", " LinearRunOnce#3.Solver._declare_options\n", "\n", "...\n", "```\n", "\n", "\n", "Note that we must always include the class name and instance ID, even when the instance has a pathname attribute, because there are times early in execution where either the pathname attribute doesn’t exist yet, as in the beginning of `__init__` method, or pathname exists but still has the default value of “” instead of its eventual value, as in the `_setup_procs` method.\n", "\n", "For more verbose output, which includes values of function locals and return values, as well as the number of times a function has been called, use the -*v* arg. For example:\n", "\n", "``` bash\n", " openmdao trace -v \n", "```\n", "\n", "Which will result in output that looks like this:\n", "\n", "```\n", "Problem#1.Problem.__init__ (1)\n", " comm=None\n", " model=None\n", " root=None\n", " self=\n", " Group#1.Group.__init__ (1)\n", " kwargs={}\n", " self=\n", " Group#1.System.__init__ (1)\n", " kwargs={}\n", " self=\n", " DictionaryJacobian#1.Jacobian.__init__ (1)\n", " kwargs={}\n", " self=\n", " <-- DictionaryJacobian#1.Jacobian.__init__\n", " Group#1().System.initialize (1)\n", " self=\n", " <-- Group#1().System.initialize\n", " <-- Group#1().System.__init__\n", " NonlinearRunOnce#1.Solver.__init__ (1)\n", " kwargs={}\n", " self=NL: RUNONCE\n", " NonlinearRunOnce#1.Solver._declare_options (1)\n", " self=NL: RUNONCE\n", " <-- NonlinearRunOnce#1.Solver._declare_options\n", " <-- NonlinearRunOnce#1.Solver.__init__\n", " LinearRunOnce#1.LinearSolver.__init__ (1)\n", " kwargs={}\n", " self=LN: RUNONCE\n", " LinearRunOnce#1.Solver.__init__ (1)\n", " kwargs={}\n", " self=LN: RUNONCE\n", " LinearRunOnce#1.Solver._declare_options (1)\n", " self=LN: RUNONCE\n", " <-- LinearRunOnce#1.Solver._declare_options\n", " <-- LinearRunOnce#1.Solver.__init__\n", " <-- LinearRunOnce#1.LinearSolver.__init__\n", " <-- Group#1().Group.__init__\n", " Driver#1.Driver.__init__ (1)\n", " self=\n", " <-- Driver#1.Driver.__init__\n", "<-- Problem#1.Problem.__init__\n", "\n", "...\n", "```\n", "\n", "By default, a pre-defined set of general OpenMDAO functions will be included in the trace, but that can be changed using the -*g* option. For example, in order to trace only `setup`-related functions, do the following:\n", "\n", "``` bash\n", " openmdao trace -v -g setup\n", "```\n", "\n", "The tracer can also display the change in memory usage from the time a function is called to the time it returns. To show memory usage, use the -*m* option, for example:\n", "\n", "``` bash\n", " openmdao trace -m \n", "``` \n", "\n", "will result in output like this:\n", "\n", "``` \n", "...\n", "\n", "Group#1().Group._setup_procs\n", " DistribOuptutImplicit#0().System._setup_procs\n", " PETScKrylov#0.PETScKrylov.__init__\n", " PETScKrylov#0.LinearSolver.__init__\n", " PETScKrylov#0.Solver.__init__\n", " PETScKrylov#0.PETScKrylov._declare_options\n", " <-- PETScKrylov#0.PETScKrylov._declare_options (time: 0.06384) (total: 75.445 MB)\n", " <-- PETScKrylov#0.Solver.__init__ (time: 0.06391) (total: 75.445 MB)\n", " <-- PETScKrylov#0.LinearSolver.__init__ (time: 0.06397) (total: 75.445 MB)\n", " <-- PETScKrylov#0.PETScKrylov.__init__ (time: 0.06402) (total: 75.445 MB)\n", " <-- DistribOuptutImplicit#0(aero.icomp).System._setup_procs (time: 0.06519) (total: 77.371 MB) (diff: +4772 KB)\n", " DistribInputExplicit#0().System._setup_procs\n", " <-- DistribInputExplicit#0(aero.ecomp).System._setup_procs (time: 0.06738) (total: 79.281 MB) (diff: +1956 KB)\n", "<-- Group#1(aero).Group._setup_procs (time: 0.06746) (total: 79.281 MB)\n", "\n", "...\n", "```\n", "\n", "Note that total memory usage and elapsed time is shown on each function return line. Those function returns where a difference in memory was found will display the difference at the end of the line.\n", "\n", "The tracer can also be used to help track down memory leaks. Using the -*l* option, it will display a list of object types and their counts that were created since the function was called and not garbage collected after the function returned. For example:\n", "\n", "``` bash\n", " openmdao trace -l -g solver\n", "```\n", "\n", "will result in output like this:\n", "\n", "```\n", "...\n", "\n", "LinearRunOnce#0.LinearRunOnce.solve\n", " LinearRunOnce#0.LinearBlockGS._iter_execute\n", " LinearRunOnce#1.LinearRunOnce.solve\n", " LinearRunOnce#1.LinearBlockGS._iter_execute\n", " PETScKrylov#0.PETScKrylov.solve\n", " <-- PETScKrylov#0.PETScKrylov.solve\n", " <-- LinearRunOnce#1.LinearBlockGS._iter_execute\n", " <-- LinearRunOnce#1.LinearRunOnce.solve\n", " Recording +1\n", " <-- LinearRunOnce#0.LinearBlockGS._iter_execute\n", "<-- LinearRunOnce#0.LinearRunOnce.solve\n", " Recording +1\n", "\n", "...\n", "```\n", "\n", "This output shows that in LinearRunOnce\\#1.LinearRunOnce.solve, a Recording object was created and not garbage collected. Note that this does not always indicate a memory leak, as there are some functions that intentionally create new objects that are intended to last beyond the life of the function. This tool merely gives you a place to look in the code where a memory leak *might* exist.\n", "\n", "To see a list of the available pre-defined sets of functions to trace, look at the usage info for the -*g* command that can be obtained as follows:\n", "\n", "``` bash\n", " openmdao trace -h\n", "```" ] }, { "cell_type": "code", "execution_count": 2, "id": "e4199f7a", "metadata": { "execution": { "iopub.execute_input": "2026-10-02T14:44:31.052877Z", "iopub.status.busy": "2026-10-02T14:44:31.052637Z", "iopub.status.idle": "2026-10-02T14:44:35.365242Z", "shell.execute_reply": "2026-10-02T14:44:35.364212Z" }, "papermill": { "duration": 4.316733, "end_time": "2026-10-02T14:44:35.366001+00:00", "exception": false, "start_time": "2026-10-02T14:44:31.049268+00:00", "status": "completed" }, "tags": [ "remove-input" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[1790952274.703376] [runnervm8df0l:8652 :0] ib_iface.c:1269 UCX ERROR mana_0: iface 0x55acbd4ddca0 failed to create UD QP TX wr:256 sge:6 inl:64 resp:0 RX wr:4096 sge:1 resp:0 failed: Operation not supported\r\n", "[1790952274.703710] [runnervm8df0l:8652 :0] ucp_worker.c:1412 UCX ERROR uct_iface_open(ud_verbs/mana_0:1) failed: Input/output error\r\n", "[runnervm8df0l:08652] pml_ucx.c:313 Error: Failed to create UCP worker\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "usage: openmdao trace [-h] [-g METHODS] [-v] [--ptrs] [-m] [-l]\r\n", " [--show_returns] [-r RANK] [-o OUTFILE] [-f FILTERS]\r\n", " file\r\n", "\r\n", "positional arguments:\r\n", " file Python file to be traced.\r\n", "\r\n", "options:\r\n", " -h, --help show this help message and exit\r\n", " -g, --group METHODS Determines which group of methods will be traced.\r\n", " Default is \"openmdao\". Options are: ['apply_linear',\r\n", " 'coloring', 'dataflow', 'driver', 'jac', 'linear',\r\n", " 'openmdao', 'openmdao_all', 'setup', 'solver',\r\n", " 'transfer']\r\n", " -v, --verbose Show function locals and return values.\r\n", " --ptrs Show addresses of printed objects.\r\n", " -m, --memory Show memory usage.\r\n", " -l, --leaks Show objects that are not garbage collected after each\r\n", " function call.\r\n", " --show_returns Show return for each function.\r\n", " -r, --rank RANK MPI rank where output is desired. Default is all\r\n", " ranks.\r\n", " -o, --outfile OUTFILE\r\n", " Output file. Defaults to stdout.\r\n", " -f, --filter FILTERS An expression. If it evaluates to True for any\r\n", " matching trace function, that function will be\r\n", " displayed in the trace. One expression can be added\r\n", " for each class.\r\n" ] } ], "source": [ "!openmdao trace -h" ] } ], "metadata": { "celltoolbar": "Tags", "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.14" }, "orphan": true, "papermill": { "default_parameters": {}, "duration": 5.231776, "end_time": "2026-10-02T14:44:35.582825+00:00", "environment_variables": {}, "exception": null, "input_path": "/home/runner/work/OpenMDAO/OpenMDAO/openmdao/docs/openmdao_book/features/debugging/profiling/inst_call_tracing.ipynb", "output_path": "/home/runner/work/OpenMDAO/OpenMDAO/openmdao/docs/_executed_book/features/debugging/profiling/inst_call_tracing.ipynb", "parameters": {}, "start_time": "2026-10-02T14:44:30.351049+00:00", "version": "2.7.0" } }, "nbformat": 4, "nbformat_minor": 5 }