{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "78271fb9",
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output",
     "active-ipynb"
    ]
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    from openmdao.utils.notebook_utils import notebook_mode\n",
    "except ImportError:\n",
    "    !python -m pip install openmdao[notebooks]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af488f90",
   "metadata": {},
   "source": [
    "# Restarting Linear Solutions for Expensive Linear Solves\n",
    "\n",
    "When using iterative linear solvers, it is often desirable to use the converged solution from a previous linear solve as the initial guess for the current one. There is some memory cost associated with this feature, because the solution for each quantity of interest will be saved separately. However, the benefit is reduced computational cost for the subsequent linear solves.\n",
    "\n",
    "```{Note}\n",
    "This feature should not be used when using the [DirectSolver](../../building_blocks/solvers/direct_solver) at the top level of your model. It won’t offer any computational savings in that situation.\n",
    "```\n",
    "\n",
    "To use this feature, provide `cache_linear_solution=True` as an argument to [add_design_var()](../adding_desvars_cons_objs/adding_design_variables), [add_objective()](../adding_desvars_cons_objs/adding_objective) , or [add_constraint()](../adding_desvars_cons_objs/adding_constraint).\n",
    "\n",
    "If you are using one of the OpenMDAO iterative solvers ([ScipyKrylov](../../building_blocks/solvers/scipy_iter_solver), [PETScKrylov](../../building_blocks/solvers/petsc_krylov), [LinearBlockGS](../../building_blocks/solvers/linear_block_gs), or [LinearBlockJac](../../building_blocks/solvers/linear_block_jac), then simply adding that argument is enough to activate this feature.\n",
    "\n",
    "If you have implemented the `solve_linear`() method for an [ImplicitComponent](../working_with_components/implicit_component), then you will need to make sure to use the provided guess solution in your implementation. The cached solution will be put into the solution vector for you to use as an initial guess. Note that you will override those values with the final solution. In `fwd` mode, the guess will be in the `d_outputs` vector. In `rev` mode, the guess will be in the `d_residuals` vector.\n",
    "\n",
    "Below is a toy example problem that illustrates how the restart vectors should work. The restart is passed in via the `x0` argument to gmres."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "66d4ef5f",
   "metadata": {},
   "outputs": [],
   "source": [
    "from distutils.version import LooseVersion\n",
    "import numpy as np\n",
    "import scipy\n",
    "from scipy.sparse.linalg import gmres\n",
    "\n",
    "import openmdao.api as om\n",
    "\n",
    "\n",
    "class QuadraticComp(om.ImplicitComponent):\n",
    "    \"\"\"\n",
    "    A Simple Implicit Component representing a Quadratic Equation.\n",
    "\n",
    "    R(a, b, c, x) = ax^2 + bx + c\n",
    "\n",
    "    Solution via Quadratic Formula:\n",
    "    x = (-b + sqrt(b^2 - 4ac)) / 2a\n",
    "    \"\"\"\n",
    "\n",
    "    def setup(self):\n",
    "        self.add_input('a', val=1.)\n",
    "        self.add_input('b', val=1.)\n",
    "        self.add_input('c', val=1.)\n",
    "        self.add_output('states', val=[0,0])\n",
    "\n",
    "    def setup_partials(self):\n",
    "        self.declare_partials(of='*', wrt='*')\n",
    "\n",
    "    def apply_nonlinear(self, inputs, outputs, residuals):\n",
    "        a = inputs['a']\n",
    "        b = inputs['b']\n",
    "        c = inputs['c']\n",
    "        x = outputs['states'][0]\n",
    "        y = outputs['states'][1]\n",
    "\n",
    "        residuals['states'][0] = a * x ** 2 + b * x + c\n",
    "        residuals['states'][1] = a * y + b\n",
    "\n",
    "    def solve_nonlinear(self, inputs, outputs):\n",
    "        a = inputs['a']\n",
    "        b = inputs['b']\n",
    "        c = inputs['c']\n",
    "        outputs['states'][0] = (-b + (b ** 2 - 4 * a * c) ** 0.5) / (2 * a)\n",
    "        outputs['states'][1] = -b/a\n",
    "\n",
    "    def linearize(self, inputs, outputs, partials):\n",
    "        a = inputs['a'][0]\n",
    "        b = inputs['b'][0]\n",
    "        c = inputs['c'][0]\n",
    "        x = outputs['states'][0]\n",
    "        y = outputs['states'][1]\n",
    "\n",
    "        partials['states', 'a'] = [[x**2],[y]]\n",
    "        partials['states', 'b'] = [[x],[1]]\n",
    "        partials['states', 'c'] = [[1.0],[0]]\n",
    "        partials['states', 'states'] = [[2*a*x+b, 0],[0, a]]\n",
    "\n",
    "        self.state_jac = np.array([[2*a*x+b, 0],[0, a]])\n",
    "\n",
    "    def solve_linear(self, d_outputs, d_residuals, mode):\n",
    "\n",
    "        if mode == 'fwd':\n",
    "            print(\"incoming initial guess\", d_outputs['states'])\n",
    "            d_outputs['states'] = gmres(self.state_jac, d_residuals['states'], x0=d_outputs['states'])[0]\n",
    "\n",
    "        elif mode == 'rev':\n",
    "            d_residuals['states'] = gmres(self.state_jac, d_outputs['states'], x0=d_residuals['states'])[0]\n",
    "\n",
    "p = om.Problem()\n",
    "p.driver = om.ScipyOptimizeDriver()\n",
    "p.driver.options['optimizer'] = 'SLSQP'\n",
    "\n",
    "p.model.set_input_defaults('b', val=4.)\n",
    "p.model.add_subsystem('quad', QuadraticComp(), promotes_inputs=['a', 'b', 'c'], promotes_outputs=['states'])\n",
    "p.model.add_subsystem('obj', om.ExecComp('y = (x[1]-x[0])**2', x=np.ones(2)))\n",
    "p.model.connect('states', 'obj.x')\n",
    "\n",
    "p.model.add_design_var('a', upper=4., cache_linear_solution=True)\n",
    "p.model.add_constraint('states', upper=10)\n",
    "p.model.add_objective('obj.y')\n",
    "\n",
    "p.setup(mode='fwd')\n",
    "p.run_driver()\n",
    "\n",
    "print(p['a'], p['b'], p['c'])\n",
    "print(p['states'])\n",
    "print(p['obj.y'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c416398f",
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.assert_utils import assert_near_equal\n",
    "import numpy as np\n",
    "\n",
    "assert_near_equal(p['a'], 3.99999858, tolerance=1e-3)\n",
    "assert_near_equal(p['b'], 4.0, tolerance=1e-3)\n",
    "assert_near_equal(p['c'], 1.0, tolerance=1e-3)\n",
    "assert_near_equal(p['states'], np.array([-0.49970278, -1.00000035]), tolerance=1e-3)\n",
    "assert_near_equal(p['obj.y'], 0.25000053, tolerance=1e-3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d130963",
   "metadata": {},
   "source": [
    "```{Note}\n",
    "In the example shown above, the model is solving for derivatives in ‘fwd’ mode, specified by `p.setup(mode=’fwd’)`. In that case the `cache_linear_solution` arg may be passed when adding design variables as shown above with `p.model.add_design_var(‘a’, cache_linear_solution=True)`. However, if the model were to solve for derivatives in ‘rev’ mode instead, then the `cache_linear_solution` arg would be applied to the objective and/or the constraint variables. For example, `p.model.add_constraint(‘states’, upper=10, cache_linear_solution=True)`.\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  },
  "orphan": true
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
