{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "active-ipynb",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    from openmdao.utils.notebook_utils import notebook_mode\n",
    "except ImportError:\n",
    "    !python -m pip install openmdao[notebooks]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# NonlinearBlockGS\n",
    "\n",
    "NonlinearBlockGS applies Block Gauss-Seidel (also known as fixed-point iteration) to the\n",
    "components and subsystems in the system. This is mainly used to solve cyclic connections. You\n",
    "should try this solver for systems that satisfy the following conditions:\n",
    "\n",
    "1. System (or subsystem) contains a cycle, though subsystems may.\n",
    "2. System does not contain any implicit states, though subsystems may.\n",
    "\n",
    "NonlinearBlockGS is a block solver, so you can specify different nonlinear solvers in the subsystems and they\n",
    "will be utilized to solve the subsystem nonlinear problem.\n",
    "\n",
    "Note that you may not know if you satisfy the second condition, so choosing a solver can be a trial-and-error proposition. If\n",
    "NonlinearBlockGS doesn't work, then you will need to use [NewtonSolver](../../../_srcdocs/packages/solvers.nonlinear/newton).\n",
    "\n",
    "Here, we choose NonlinearBlockGS to solve the Sellar problem, which has two components with a\n",
    "cyclic dependency, has no implicit states, and works very well with Gauss-Seidel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.notebook_utils import get_code\n",
    "from myst_nb import glue\n",
    "glue(\"code_src33\", get_code(\"openmdao.test_suite.components.sellar.SellarDis1withDerivatives\"), display=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::{Admonition} `SellarDis1withDerivatives` class definition \n",
    ":class: dropdown\n",
    "\n",
    "{glue:}`code_src33`\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.notebook_utils import get_code\n",
    "from myst_nb import glue\n",
    "glue(\"code_src34\", get_code(\"openmdao.test_suite.components.sellar.SellarDis2withDerivatives\"), display=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::{Admonition} `SellarDis2withDerivatives` class definition \n",
    ":class: dropdown\n",
    "\n",
    "{glue:}`code_src34`\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import openmdao.api as om\n",
    "from openmdao.test_suite.components.sellar import SellarDis1withDerivatives, SellarDis2withDerivatives\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model\n",
    "\n",
    "model.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])\n",
    "model.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)',\n",
    "                                           z=np.array([0.0, 0.0]), x=0.0),\n",
    "                    promotes=['obj', 'x', 'z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1'), promotes=['con1', 'y1'])\n",
    "model.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0'), promotes=['con2', 'y2'])\n",
    "\n",
    "model.nonlinear_solver = om.NonlinearBlockGS()\n",
    "\n",
    "prob.setup()\n",
    "\n",
    "prob.set_val('x', 1.)\n",
    "prob.set_val('z', np.array([5.0, 2.0]))\n",
    "\n",
    "prob.run_model()\n",
    "\n",
    "print(prob.get_val('y1'))\n",
    "print(prob.get_val('y2'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.assert_utils import assert_near_equal\n",
    "\n",
    "assert_near_equal(prob.get_val('y1'), 25.58830273, .00001)\n",
    "assert_near_equal(prob.get_val('y2'), 12.05848819, .00001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This solver runs all of the subsystems each iteration, passing data along all connections\n",
    "including the cyclic ones. After each iteration, the iteration count and the residual norm are\n",
    "checked to see if termination has been satisfied.\n",
    "\n",
    "You can control the termination criteria for the solver using the following options:\n",
    "\n",
    "# NonlinearBlockGS Options"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input"
    ]
   },
   "outputs": [],
   "source": [
    "om.show_options_table(\"openmdao.solvers.nonlinear.nonlinear_block_gs.NonlinearBlockGS\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## NonlinearBlockGS Constructor\n",
    "\n",
    "The call signature for the `NonlinearBlockGS` constructor is:\n",
    "\n",
    "```{eval-rst}\n",
    "    .. automethod:: openmdao.solvers.nonlinear.nonlinear_block_gs.NonlinearBlockGS.__init__\n",
    "        :noindex:\n",
    "```\n",
    "\n",
    "## Aitken relaxation\n",
    "\n",
    "This solver implements Aitken relaxation, as described in Algorithm 1 of this paper on aerostructual design [optimization](http://www.umich.edu/~mdolaboratory/pdf/Kenway2014a.pdf).\n",
    "The relaxation is turned off by default, but it may help convergence for more tightly coupled models.\n",
    "\n",
    "## Residual Calculation\n",
    "\n",
    "The `Unified Derivatives Equations` are formulated so that explicit equations (via `ExplicitComponent`) are also expressed\n",
    "as implicit relationships, and their residual is also calculated in \"apply_nonlinear\", which runs the component a second time and\n",
    "saves the difference in the output vector as the residual. However, this would require an extra call to `compute`, which is\n",
    "inefficient for slower components. To eliminate the inefficiency of running the model twice every iteration the NonlinearBlockGS\n",
    "driver saves a copy of the output vector and uses that to calculate the residual without rerunning the model. This does require\n",
    "a little more memory, so if you are solving a model where memory is more of a concern than execution time, you can set the\n",
    "\"use_apply_nonlinear\" option to True to use the original formulation that calls \"apply_nonlinear\" on the subsystem.\n",
    "\n",
    "\n",
    "## NonlinearBlockGS Option Examples\n",
    "\n",
    "**maxiter**\n",
    "\n",
    "  `maxiter` lets you specify the maximum number of Gauss-Seidel iterations to apply. In this example, we\n",
    "  cut it back from the default, ten, down to two, so that it terminates a few iterations earlier and doesn't\n",
    "  reach the specified absolute or relative tolerance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.sellar import SellarDis1withDerivatives, SellarDis2withDerivatives\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model\n",
    "\n",
    "model.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])\n",
    "model.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)',\n",
    "                                           z=np.array([0.0, 0.0]), x=0.0),\n",
    "                    promotes=['obj', 'x', 'z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1'), promotes=['con1', 'y1'])\n",
    "model.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0'), promotes=['con2', 'y2'])\n",
    "\n",
    "prob.setup()\n",
    "nlbgs = model.nonlinear_solver = om.NonlinearBlockGS()\n",
    "\n",
    "#basic test of number of iterations\n",
    "nlbgs.options['maxiter'] = 1\n",
    "prob.run_model()\n",
    "print(model.nonlinear_solver._iter_count)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert(model.nonlinear_solver._iter_count == 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nlbgs.options['maxiter'] = 5\n",
    "prob.run_model()\n",
    "print(model.nonlinear_solver._iter_count)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert(model.nonlinear_solver._iter_count == 5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#test of number of iterations AND solution after exit at maxiter\n",
    "prob.set_val('x', 1.)\n",
    "prob.set_val('z', np.array([5.0, 2.0]))\n",
    "\n",
    "nlbgs.options['maxiter'] = 3\n",
    "prob.set_solver_print()\n",
    "prob.run_model()\n",
    "\n",
    "print(prob.get_val('y1'))\n",
    "print(prob.get_val('y2'))\n",
    "print(model.nonlinear_solver._iter_count)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert_near_equal(prob.get_val('y1'), 25.58914915, .00001)\n",
    "assert_near_equal(prob.get_val('y2'), 12.05857185, .00001)\n",
    "assert(model.nonlinear_solver._iter_count == 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**atol**\n",
    "\n",
    "  Here, we set the absolute tolerance to a looser value that will trigger an earlier termination. After\n",
    "  each iteration, the norm of the residuals is calculated one of two ways. If the \"use_apply_nonlinear\" option\n",
    "  is set to False (its default), then the norm is calculated by subtracting a cached previous value of the\n",
    "  outputs from the current value.  If \"use_apply_nonlinear\" is True, then the norm is calculated by calling\n",
    "  apply_nonlinear on all of the subsystems. In this case, `ExplicitComponents` are executed a second time.\n",
    "  If this norm value is lower than the absolute tolerance `atol`, the iteration will terminate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.sellar import SellarDis1withDerivatives, SellarDis2withDerivatives\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model\n",
    "\n",
    "model.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])\n",
    "model.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)',\n",
    "                                           z=np.array([0.0, 0.0]), x=0.0),\n",
    "                    promotes=['obj', 'x', 'z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1'), promotes=['con1', 'y1'])\n",
    "model.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0'), promotes=['con2', 'y2'])\n",
    "\n",
    "nlbgs = model.nonlinear_solver = om.NonlinearBlockGS()\n",
    "nlbgs.options['atol'] = 1e-4\n",
    "\n",
    "prob.setup()\n",
    "\n",
    "prob.set_val('x', 1.)\n",
    "prob.set_val('z', np.array([5.0, 2.0]))\n",
    "\n",
    "prob.run_model()\n",
    "\n",
    "print(prob.get_val('y1'))\n",
    "print(prob.get_val('y2'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert_near_equal(prob.get_val('y1'), 25.5882856302, .00001)\n",
    "assert_near_equal(prob.get_val('y2'), 12.05848819, .00001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**rtol**\n",
    "\n",
    "  Here, we set the relative tolerance to a looser value that will trigger an earlier termination. After\n",
    "  each iteration, the norm of the residuals is calculated one of two ways. If the \"use_apply_nonlinear\" option\n",
    "  is set to False (its default), then the norm is calculated by subtracting a cached previous value of the\n",
    "  outputs from the current value.  If \"use_apply_nonlinear\" is True, then the norm is calculated by calling\n",
    "  apply_nonlinear on all of the subsystems. In this case, `ExplicitComponents` are executed a second time.\n",
    "  If the ratio of the currently calculated norm to the initial residual norm is lower than the relative tolerance\n",
    "  `rtol`, the iteration will terminate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.notebook_utils import get_code\n",
    "from myst_nb import glue\n",
    "glue(\"code_src35\", get_code(\"openmdao.test_suite.components.sellar.SellarDerivatives\"), display=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::{Admonition} `SellarDerivatives` class definition \n",
    ":class: dropdown\n",
    "\n",
    "{glue:}`code_src35`\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.sellar import SellarDis1withDerivatives, SellarDis2withDerivatives, SellarDerivatives\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model\n",
    "\n",
    "model.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])\n",
    "model.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)',\n",
    "                                           z=np.array([0.0, 0.0]), x=0.0),\n",
    "                    promotes=['obj', 'x', 'z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1'), promotes=['con1', 'y1'])\n",
    "model.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0'), promotes=['con2', 'y2'])\n",
    "\n",
    "nlbgs = model.nonlinear_solver = om.NonlinearBlockGS()\n",
    "nlbgs.options['rtol'] = 1e-3\n",
    "\n",
    "prob.setup()\n",
    "\n",
    "prob.set_val('x', 1.)\n",
    "prob.set_val('z', np.array([5.0, 2.0]))\n",
    "\n",
    "prob.run_model()\n",
    "\n",
    "print(prob.get_val('y1'), 25.5883027, .00001)\n",
    "print(prob.get_val('y2'), 12.05848819, .00001)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert_near_equal(prob.get_val('y1'), 25.5883027, .00001)\n",
    "assert_near_equal(prob.get_val('y2'), 12.05848819, .00001)"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.1"
  },
  "orphan": true
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
