{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "active-ipynb",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    from openmdao.utils.notebook_utils import notebook_mode\n",
    "except ImportError:\n",
    "    !python -m pip install openmdao[notebooks]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ArmijoGoldsteinLS\n",
    "\n",
    "ArmijoGoldsteinLS checks bounds and backtracks to a point that satisfies them. From there,\n",
    "further backtracking is performed, until the termination criteria are satisfied.\n",
    "The main termination criteria is the Armijo-Goldstein condition, which checks for a sufficient\n",
    "decrease from the initial point by measuring the slope. There is also an iteration maximum.\n",
    "\n",
    "Here is a simple example where ArmijoGoldsteinLS is used to backtrack during the Newton solver's iteration on\n",
    "a system that contains an implicit component with 3 states that are confined to a small range of values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.notebook_utils import get_code\n",
    "from myst_nb import glue\n",
    "glue(\"code_src19\", get_code(\"openmdao.test_suite.components.implicit_newton_linesearch.ImplCompTwoStatesArrays\"), display=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::{Admonition} `ImplCompTwoStatesArrays` class definition \n",
    ":class: dropdown\n",
    "\n",
    "{glue:}`code_src19`\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import openmdao.api as om\n",
    "\n",
    "from openmdao.test_suite.components.implicit_newton_linesearch import ImplCompTwoStatesArrays\n",
    "\n",
    "top = om.Problem()\n",
    "top.model.add_subsystem('comp', ImplCompTwoStatesArrays(), promotes_inputs=['x'])\n",
    "\n",
    "top.model.nonlinear_solver = om.NewtonSolver(solve_subsystems=False)\n",
    "top.model.nonlinear_solver.options['maxiter'] = 10\n",
    "top.model.linear_solver = om.ScipyKrylov()\n",
    "\n",
    "top.model.nonlinear_solver.linesearch = om.ArmijoGoldsteinLS()\n",
    "\n",
    "top.setup()\n",
    "top.set_val('x', np.array([2., 2, 2]).reshape(3, 1))\n",
    "# Test lower bounds: should go to the lower bound and stall\n",
    "top.set_val('comp.y', 0.)\n",
    "top.set_val('comp.z', 1.6)\n",
    "top.run_model()\n",
    "\n",
    "for ind in range(3):\n",
    "    print(top.get_val('comp.z', indices=ind))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.assert_utils import assert_near_equal\n",
    "\n",
    "for ind in range(3):\n",
    "    assert_near_equal(top.get_val('comp.z', indices=ind), [1.5], 1e-8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ArmijoGoldsteinLS Options"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input"
    ]
   },
   "outputs": [],
   "source": [
    "om.show_options_table(\"openmdao.solvers.linesearch.backtracking.ArmijoGoldsteinLS\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ArmijoGoldsteinLS Constructor\n",
    "\n",
    "The call signature for the `ArmijoGoldsteinLS` constructor is:\n",
    "\n",
    "```{eval-rst}\n",
    "    .. automethod:: openmdao.solvers.linesearch.backtracking.ArmijoGoldsteinLS.__init__\n",
    "        :noindex:\n",
    "```\n",
    "\n",
    "## ArmijoGoldsteinLS Option Examples\n",
    "\n",
    "**bound_enforcement**\n",
    "\n",
    "ArmijoGoldsteinLS includes the `bound_enforcement` option in its options dictionary. This option has a dual role:\n",
    "\n",
    "1. Behavior of the non-bounded variables when the bounded ones are capped.\n",
    "2. Direction of the further backtracking.\n",
    "\n",
    "There are three different acceptable values for bounds-enforcement schemes available in this option.\n",
    "\n",
    "With \"scalar\" bounds enforcement, only the variables that violate their bounds are pulled back to feasible values; the\n",
    "remaining values are kept at the Newton-stepped point. This changes the direction of the backtracking vector so that\n",
    "it still moves in the direction of the initial point.\n",
    "\n",
    "![BT2](images/BT2.jpg)\n",
    "\n",
    "With \"vector\" bounds enforcement, the solution in the output vector is pulled back to a point where none of the\n",
    "variables violate any upper or lower bounds. Further backtracking continues along the Newton gradient direction vector back towards the\n",
    "initial point.\n",
    "\n",
    "![BT1](images/BT1.jpg)\n",
    "\n",
    "With \"wall\" bounds enforcement, only the variables that violate their bounds are pulled back to feasible values; the\n",
    "remaining values are kept at the Newton-stepped point. Further backtracking only occurs in the direction of the non-violating\n",
    "variables, so that it will move along the wall.\n",
    "\n",
    "![BT3](images/BT3.jpg)\n",
    "\n",
    "Here are examples of each acceptable value for the **bound_enforcement** option:\n",
    "\n",
    "- bound_enforcement: vector\n",
    "\n",
    "  The `bound_enforcement` option in the options dictionary is used to specify how the output bounds\n",
    "  are enforced. When this is set to \"vector\", the output vector is rolled back along the computed gradient until\n",
    "  it reaches a point where the earliest bound violation occurred. The backtracking continues along the original\n",
    "  computed gradient.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.implicit_newton_linesearch import ImplCompTwoStatesArrays\n",
    "\n",
    "top = om.Problem()\n",
    "top.model.add_subsystem('comp', ImplCompTwoStatesArrays(), promotes_inputs=['x'])\n",
    "\n",
    "top.model.nonlinear_solver = om.NewtonSolver(solve_subsystems=False)\n",
    "top.model.nonlinear_solver.options['maxiter'] = 10\n",
    "top.model.linear_solver = om.ScipyKrylov()\n",
    "\n",
    "top.model.nonlinear_solver.linesearch = om.ArmijoGoldsteinLS(bound_enforcement='vector')\n",
    "\n",
    "top.setup()\n",
    "\n",
    "top.set_val('x', np.array([2., 2, 2]).reshape(3, 1))\n",
    "\n",
    "# Test lower bounds: should go to the lower bound and stall\n",
    "top.set_val('comp.y', 0.)\n",
    "top.set_val('comp.z', 1.6)\n",
    "top.run_model()\n",
    "\n",
    "for ind in range(3):\n",
    "    print(top.get_val('comp.z', indices=ind))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "for ind in range(3):\n",
    "    assert_near_equal(top.get_val('comp.z', indices=ind), [1.5], 1e-8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- bound_enforcement: scalar\n",
    "\n",
    "  The `bound_enforcement` option in the options dictionary is used to specify how the output bounds\n",
    "  are enforced. When this is set to \"scaler\", then the only indices in the output vector that are rolled back\n",
    "  are the ones that violate their upper or lower bounds. The backtracking continues along the modified gradient."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.implicit_newton_linesearch import ImplCompTwoStatesArrays\n",
    "\n",
    "top = om.Problem()\n",
    "top.model.add_subsystem('comp', ImplCompTwoStatesArrays(), promotes_inputs=['x'])\n",
    "\n",
    "top.model.nonlinear_solver = om.NewtonSolver(solve_subsystems=False)\n",
    "top.model.nonlinear_solver.options['maxiter'] = 10\n",
    "top.model.linear_solver = om.ScipyKrylov()\n",
    "\n",
    "ls = top.model.nonlinear_solver.linesearch = om.ArmijoGoldsteinLS(bound_enforcement='scalar')\n",
    "\n",
    "top.setup()\n",
    "top.set_val('x', np.array([2., 2, 2]).reshape(3, 1))\n",
    "top.run_model()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Test lower bounds: should stop just short of the lower bound\n",
    "top.set_val('comp.y', 0.)\n",
    "top.set_val('comp.z', 1.6)\n",
    "top.run_model()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- bound_enforcement: wall\n",
    "\n",
    "  The `bound_enforcement` option in the options dictionary is used to specify how the output bounds\n",
    "  are enforced. When this is set to \"wall\", then the only indices in the output vector that are rolled back\n",
    "  are the ones that violate their upper or lower bounds. The backtracking continues along a modified gradient\n",
    "  direction that follows the boundary of the violated output bounds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.implicit_newton_linesearch import ImplCompTwoStatesArrays\n",
    "\n",
    "top = om.Problem()\n",
    "top.model.add_subsystem('comp', ImplCompTwoStatesArrays(), promotes_inputs=['x'])\n",
    "\n",
    "top.model.nonlinear_solver = om.NewtonSolver(solve_subsystems=False)\n",
    "top.model.nonlinear_solver.options['maxiter'] = 10\n",
    "top.model.linear_solver = om.ScipyKrylov()\n",
    "\n",
    "top.model.nonlinear_solver.linesearch = om.ArmijoGoldsteinLS(bound_enforcement='wall')\n",
    "\n",
    "top.setup()\n",
    "\n",
    "top.set_val('x', np.array([0.5, 0.5, 0.5]).reshape(3, 1))\n",
    "\n",
    "# Test upper bounds: should go to the upper bound and stall\n",
    "top.set_val('comp.y', 0.)\n",
    "top.set_val('comp.z', 2.4)\n",
    "top.run_model()\n",
    "\n",
    "print(top.get_val('comp.z', indices=0))\n",
    "print(top.get_val('comp.z', indices=1))\n",
    "print(top.get_val('comp.z', indices=2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert_near_equal(top.get_val('comp.z', indices=0), [2.6], 1e-8)\n",
    "assert_near_equal(top.get_val('comp.z', indices=1), [2.5], 1e-8)\n",
    "assert_near_equal(top.get_val('comp.z', indices=2), [2.65], 1e-8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**maxiter**\n",
    "\n",
    "  The \"maxiter\" option is a termination criteria that specifies the maximum number of backtracking steps to allow.\n",
    "\n",
    "**alpha**\n",
    "\n",
    "  The \"alpha\" option is used to specify the initial length of the Newton step. Since NewtonSolver assumes a\n",
    "  step size of 1.0, this value usually shouldn't be changed.\n",
    "\n",
    "**rho**\n",
    "\n",
    "  The \"rho\" option controls how far to backtrack in each successive backtracking step. It is applied as a multiplier to\n",
    "  the step, so a higher value (approaching 1.0) is a very small step, while a low value takes you close to the initial\n",
    "  point. The default value is 0.5.\n",
    "\n",
    "**c**\n",
    "\n",
    "  In the `ArmijoGoldsteinLS`, the \"c\" option is a multiplier on the slope check. Setting it to a smaller value means a more\n",
    "  gentle slope will satisfy the condition and terminate.\n",
    "\n",
    "**print_bound_enforce**\n",
    "\n",
    "  When the \"print_bound_enforce\" option is set to True, the line-search will print the name and values of any variables\n",
    "  that exceeded their lower or upper bounds and were drawn back during bounds enforcement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.implicit_newton_linesearch import ImplCompTwoStatesArrays\n",
    "\n",
    "top = om.Problem()\n",
    "top.model.add_subsystem('comp', ImplCompTwoStatesArrays(), promotes_inputs=['x'])\n",
    "\n",
    "newt = top.model.nonlinear_solver = om.NewtonSolver(solve_subsystems=False)\n",
    "top.model.nonlinear_solver.options['maxiter'] = 2\n",
    "top.model.linear_solver = om.ScipyKrylov()\n",
    "\n",
    "ls = newt.linesearch = om.BoundsEnforceLS(bound_enforcement='vector')\n",
    "ls.options['print_bound_enforce'] = True\n",
    "\n",
    "top.set_solver_print(level=2)\n",
    "\n",
    "\n",
    "top.setup()\n",
    "top.set_val('x', np.array([2., 2, 2]).reshape(3, 1))\n",
    "\n",
    "# Test lower bounds: should go to the lower bound and stall\n",
    "top.set_val('comp.y', 0.)\n",
    "top.set_val('comp.z', 1.6)\n",
    "top.run_model()\n",
    "\n",
    "for ind in range(3):\n",
    "    print(top.get_val('comp.z', indices=ind))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "for ind in range(3):\n",
    "    assert_near_equal(top.get_val('comp.z', indices=ind), [1.5], 1e-8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- retry_on_analysis_error\n",
    "\n",
    "  By default, the ArmijoGoldsteinLS linesearch will backtrack if the model raises an AnalysisError, which can happen if the component explicitly raises it, or a subsolver hits its iteration limit with the 'err_on_non_converge' option set to True. If you would rather terminate on an AnalysisError, you can set this option to False.\n"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.1"
  },
  "orphan": true
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
