{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2e39f5eb",
   "metadata": {},
   "source": [
    "# MPI Debugging with VS Code\n",
    "\n",
    "In order to debug a program running under MPI, a debugger must be able to attach to multiple running processes.  VS Code, and other IDEs based on VS Code, like Cursor for example, can do this using a `launch.json` file that specifies a unique debug configuration for each process.  Setting this up manually isn't much fun, so OpenMDAO has automated the process.\n",
    "\n",
    "OpenMDAO injects new configurations into `launch.json` and attempts not to overwrite any of your manual changes to it\n",
    "\n",
    "\n",
    "## Activating the Debugger\n",
    "\n",
    "The automated MPI debugging process uses the `debugpy` package, so before you start, install that into your python environment using your preferred method, e.g., `pip`, `conda`, `pixi`, etc.\n",
    "\n",
    "### Run Your Program\n",
    "\n",
    "The first step is to run your program.  This can be done using various commands depending on whether you're debugging a test or a standalone python script.  The key point is that in order to allow MPI debugging to happen, you must set the `VSCODE_DBG` environment variable.\n",
    "\n",
    "For example, let's say you are running a simple python script using 2 MPI processes.  You would normally run it like this:\n",
    "\n",
    "```bash\n",
    "    mpirun -n 2 python myscript.py\n",
    "```\n",
    "\n",
    "In order to activate the automated MPI debugging setup, just do the following to set the environment variable:\n",
    "\n",
    "```bash\n",
    "    VSCODE_DBG=1 mpirun -n 2 python myscript.py\n",
    "```\n",
    "\n",
    "If, on the other hand, you want to run an MPI test on two processes, you could use the `run_om_test` command to do it like this:\n",
    "\n",
    "```bash\n",
    "    mpirun -n 2 run_om_test test_distribcomp.py:MPITests.test_overlapping_inputs_idxs\n",
    "```\n",
    "\n",
    "To activate debugging in this case, just add `VSCODE_DBG=1` before the command as before:\n",
    "\n",
    "```bash\n",
    "    VSCODE_DBG=1 mpirun -n 2 run_om_test test_distribcomp.py:MPITests.test_overlapping_inputs_idxs\n",
    "```\n",
    "\n",
    "\n",
    "Note that the `VSCODE_DBG` environment variable processing occurs in the `openmdao.api` file, so if that file is not included in your script or test file, the MPI debugging will not activate.  If you don't want to include the entire `openmdao.api` file, you can import the `setup_dbg` function directly from `openmdao.utils.general_utils` and call it directly, for example:\n",
    "\n",
    "```python\n",
    "    from openmdao.utils.general_utils import setup_dbg\n",
    "    setup_dbg()\n",
    "```\n",
    "\n",
    "When your program runs, assuming you have imported `openmdao.api` or called `setup_dbg` directly, you should see a message from `debugpy` on each rank, for example:\n",
    "\n",
    "```console\n",
    "    Rank 0: Debugger listening on port 51111\n",
    "    Rank 1: Debugger listening on port 51112\n",
    "```\n",
    "\n",
    "Your program will block until you attach to the running processes with (VS Code / Cursor)\n",
    "\n",
    "\n",
    "### Set Your Breakpoint(s)\n",
    "\n",
    "Set up your breakpoints as you normally would when debugging in your IDE, typically by clicking on the red dot in the far left column of the editor window containing the file where you want to set the breakpoint.\n",
    "\n",
    "\n",
    "### Attach to Your Running MPI Processes\n",
    "\n",
    "In your (VS Code / Cursor) IDE, select `Run and Debug`.  The location for this differs slightly depending on which IDE your're using, but it's generally on the left side in the same place as `Explorer` and `Search`.\n",
    "\n",
    "Once you've selected `Run and Debug`, you'll see a dropdown menu near the top on the left.  Select `MPI Debug`. Finally, under the top level `Run` menu, select `Start Debugging`, or just hit the `F5` key.\n",
    "\n",
    "### Debugging\n",
    "\n",
    "Once you've attached to the running processes and have stopped at a breakpoint, you can debug as you normally would with a single process.  The main difference is that now the call stack will show separate call trees for each MPI rank configuration.  The configurations are named `_rank_?_config`, where `?` is the rank number.  Also, near the top of the editor window, the little toolbar with choices like `Continue`, `Step Over`, `Step Into`, etc. will have a dropdown menu next to it that allows you to choose which rank you want to step through.  It also has a `Disconnect` option to disconnect from the current rank's process.\n",
    "\n",
    "The `Debug Console` tab of the terminal below the editor window also has a dropdown menu to allow you to switch between rank processes so you can interact with each specific process.\n"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}