{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2ab75e47",
   "metadata": {},
   "source": [
    "# rf612_recoverFromInvalidParameters\n",
    "Likelihood and minimization: Recover from regions where the function is not defined.\n",
    "\n",
    "We demonstrate improved recovery from disallowed parameters. For this, we use a polynomial PDF of the form\n",
    "  \\mathrm{Pol2} = \\mathcal{N} \\left( c + a_1 \\cdot x + a_2 \\cdot x^2 + 0.01 \\cdot x^3 \\right),\n",
    "where $ \\mathcal{N} $ is a normalisation factor. Unless the parameters are chosen carefully,\n",
    "this function can be negative, and hence, it cannot be used as a PDF. In this case, RooFit passes\n",
    "an error to the minimiser, which might try to recover.\n",
    "\n",
    "Before ROOT 6.24, RooFit always passed the highest function value that was encountered during the minimisation\n",
    "to the minimiser. If a parameter is far in a disallowed region, the minimiser has to blindly test various\n",
    "values of the parameters. It might find the correct values by chance, but often be unable to recover from bad\n",
    "starting values. Here, we use a model with such bad values.\n",
    "\n",
    "Starting with ROOT 6.24, the minimiser receives more information. For example, when a PDF is negative,\n",
    "the magnitude of the \"undershoot\" is passed to the minimiser. The minimiser can use this to compute a\n",
    "gradient, which will eventually lead it out of the disallowed region. The steepness of this gradient\n",
    "can be chosen using RooFit::RecoverFromUndefinedRegions(double). A value of zero is equivalent to RooFit\n",
    "before ROOT 6.24. Positive values activate the recovery. Values between 1. and 10. were found to be a\n",
    "good default. If no argument is passed, RooFit uses 10.\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "**Author:** Stephan Hageboeck  \n",
    "<i><small>This notebook tutorial was automatically generated with <a href= \"https://github.com/root-project/root/blob/master/documentation/doxygen/converttonotebook.py\">ROOTBOOK-izer</a> from the macro found in the ROOT repository  on Tuesday, May 19, 2026 at 08:33 PM.</small></i>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ce15e47",
   "metadata": {},
   "source": [
    "Create a fit model:\n",
    "The polynomial is notoriously unstable, because it can quickly go negative.\n",
    "Since PDFs need to be positive, one often ends up with an unstable fit model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b4fc3db1",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2026-05-19T20:33:39.753901Z",
     "iopub.status.busy": "2026-05-19T20:33:39.753791Z",
     "iopub.status.idle": "2026-05-19T20:33:40.281116Z",
     "shell.execute_reply": "2026-05-19T20:33:40.280462Z"
    }
   },
   "outputs": [],
   "source": [
    "RooRealVar x(\"x\", \"x\", -15, 15);\n",
    "RooRealVar a1(\"a1\", \"a1\", -0.5, -10., 20.);\n",
    "RooRealVar a2(\"a2\", \"a2\", 0.2, -10., 20.);\n",
    "RooRealVar a3(\"a3\", \"a3\", 0.01);\n",
    "RooPolynomial pdf(\"pol3\", \"c + a1 * x + a2 * x*x + 0.01 * x*x*x\", x, RooArgSet(a1, a2, a3));"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1140acf1",
   "metadata": {},
   "source": [
    "Create toy data with all-positive coefficients:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "f8cbd6d9",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2026-05-19T20:33:40.282556Z",
     "iopub.status.busy": "2026-05-19T20:33:40.282440Z",
     "iopub.status.idle": "2026-05-19T20:33:40.491407Z",
     "shell.execute_reply": "2026-05-19T20:33:40.490869Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "input_line_53:2:2: warning: 'data' shadows a declaration with the same name in the 'std' namespace; use '::data' to reference this declaration\n",
      " std::unique_ptr<RooDataSet> data(pdf.generate(x, 10000));\n",
      " ^\n"
     ]
    }
   ],
   "source": [
    "std::unique_ptr<RooDataSet> data(pdf.generate(x, 10000));"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46ae7e61",
   "metadata": {},
   "source": [
    "For plotting.\n",
    "We create pointers to the plotted objects. We want these objects to leak out of the function,\n",
    "so we can still see them after it returns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "c1208439",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2026-05-19T20:33:40.492645Z",
     "iopub.status.busy": "2026-05-19T20:33:40.492510Z",
     "iopub.status.idle": "2026-05-19T20:33:40.729019Z",
     "shell.execute_reply": "2026-05-19T20:33:40.718687Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "input_line_54:4:1: error: reference to 'data' is ambiguous\n",
      "data->plotOn(frame, RooFit::Name(\"data\"));\n",
      "^\n",
      "input_line_53:2:30: note: candidate found by name lookup is 'data'\n",
      " std::unique_ptr<RooDataSet> data(pdf.generate(x, 10000));\n",
      "                             ^\n",
      "/usr/lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/range_access.h:344:5: note: candidate found by name lookup is 'std::data'\n",
      "    data(initializer_list<_Tp> __il) noexcept\n",
      "    ^\n",
      "/usr/lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/range_access.h:312:5: note: candidate found by name lookup is 'std::data'\n",
      "    data(_Container& __cont) noexcept(noexcept(__cont.data()))\n",
      "    ^\n",
      "/usr/lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/range_access.h:323:5: note: candidate found by name lookup is 'std::data'\n",
      "    data(const _Container& __cont) noexcept(noexcept(__cont.data()))\n",
      "    ^\n",
      "/usr/lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/range_access.h:334:5: note: candidate found by name lookup is 'std::data'\n",
      "    data(_Tp (&__array)[_Nm]) noexcept\n",
      "    ^\n"
     ]
    }
   ],
   "source": [
    "TCanvas* c = new TCanvas();\n",
    "RooPlot* frame = x.frame();\n",
    "data->plotOn(frame, RooFit::Name(\"data\"));"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90a708af",
   "metadata": {},
   "source": [
    "Plotting a PDF with disallowed parameters doesn't work. We would get a lot of error messages.\n",
    "Therefore, we disable plotting messages in RooFit's message streams:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7a2db67a",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2026-05-19T20:33:40.730420Z",
     "iopub.status.busy": "2026-05-19T20:33:40.730287Z",
     "iopub.status.idle": "2026-05-19T20:33:40.938681Z",
     "shell.execute_reply": "2026-05-19T20:33:40.938228Z"
    }
   },
   "outputs": [],
   "source": [
    "RooMsgService::instance().getStream(0).removeTopic(RooFit::Plotting);\n",
    "RooMsgService::instance().getStream(1).removeTopic(RooFit::Plotting);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb512a04",
   "metadata": {},
   "source": [
    "RooFit before ROOT 6.24\n",
    "--------------------------------\n",
    "Before 6.24, RooFit wasn't able to recover from invalid parameters. The minimiser just errs around\n",
    "the starting values of the parameters without finding any improvement."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3db32dd5",
   "metadata": {},
   "source": [
    "Set up the parameters such that the PDF would come out negative. The PDF is now undefined."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "dc24ba7e",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2026-05-19T20:33:40.940284Z",
     "iopub.status.busy": "2026-05-19T20:33:40.940178Z",
     "iopub.status.idle": "2026-05-19T20:33:41.148340Z",
     "shell.execute_reply": "2026-05-19T20:33:41.147893Z"
    }
   },
   "outputs": [],
   "source": [
    "a1.setVal(10.);\n",
    "a2.setVal(-1.);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "902d9b24",
   "metadata": {},
   "source": [
    "Perform a fit:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "8e9989df",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2026-05-19T20:33:41.149858Z",
     "iopub.status.busy": "2026-05-19T20:33:41.149746Z",
     "iopub.status.idle": "2026-05-19T20:33:41.356249Z",
     "shell.execute_reply": "2026-05-19T20:33:41.355867Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "input_line_57:2:62: error: reference to 'data' is ambiguous\n",
      " std::unique_ptr<RooFitResult> fitWithoutRecovery{pdf.fitTo(*data, RooFit::Save(),\n",
      "                                                             ^\n",
      "input_line_53:2:30: note: candidate found by name lookup is 'data'\n",
      " std::unique_ptr<RooDataSet> data(pdf.generate(x, 10000));\n",
      "                             ^\n",
      "/usr/lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/range_access.h:344:5: note: candidate found by name lookup is 'std::data'\n",
      "    data(initializer_list<_Tp> __il) noexcept\n",
      "    ^\n",
      "/usr/lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/range_access.h:312:5: note: candidate found by name lookup is 'std::data'\n",
      "    data(_Container& __cont) noexcept(noexcept(__cont.data()))\n",
      "    ^\n",
      "/usr/lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/range_access.h:323:5: note: candidate found by name lookup is 'std::data'\n",
      "    data(const _Container& __cont) noexcept(noexcept(__cont.data()))\n",
      "    ^\n",
      "/usr/lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/range_access.h:334:5: note: candidate found by name lookup is 'std::data'\n",
      "    data(_Tp (&__array)[_Nm]) noexcept\n",
      "    ^\n",
      "input_line_57:7:12: error: use of undeclared identifier 'frame'\n",
      "pdf.plotOn(frame, RooFit::LineColor(kRed), RooFit::Name(\"noRecovery\"));\n",
      "           ^\n"
     ]
    }
   ],
   "source": [
    "std::unique_ptr<RooFitResult> fitWithoutRecovery{pdf.fitTo(*data, RooFit::Save(),\n",
    "    RooFit::RecoverFromUndefinedRegions(0.), // This is how RooFit behaved prior to ROOT 6.24\n",
    "    RooFit::PrintEvalErrors(-1), // We are expecting a lot of evaluation errors. -1 switches off printing.\n",
    "    RooFit::PrintLevel(-1))};\n",
    "\n",
    "pdf.plotOn(frame, RooFit::LineColor(kRed), RooFit::Name(\"noRecovery\"));"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "586125a5",
   "metadata": {},
   "source": [
    "RooFit since ROOT 6.24\n",
    "--------------------------------\n",
    "The minimiser gets information about the \"badness\" of the violation of the function definition. It uses this\n",
    "to find its way out of the disallowed parameter regions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d7dc142c",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2026-05-19T20:33:41.357411Z",
     "iopub.status.busy": "2026-05-19T20:33:41.357306Z",
     "iopub.status.idle": "2026-05-19T20:33:41.565693Z",
     "shell.execute_reply": "2026-05-19T20:33:41.565301Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\n",
      "-------------- Starting second fit ---------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "std::cout << \"\\n\\n\\n-------------- Starting second fit ---------------\\n\\n\" << std::endl;"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3db386ab",
   "metadata": {},
   "source": [
    "Reset the parameters such that the PDF is again undefined."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "9bf79d6a",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2026-05-19T20:33:41.566853Z",
     "iopub.status.busy": "2026-05-19T20:33:41.566745Z",
     "iopub.status.idle": "2026-05-19T20:33:41.775053Z",
     "shell.execute_reply": "2026-05-19T20:33:41.774598Z"
    }
   },
   "outputs": [],
   "source": [
    "a1.setVal(10.);\n",
    "a2.setVal(-1.);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59dc5b7a",
   "metadata": {},
   "source": [
    "Fit again, but pass recovery information to the minimiser:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "8ceb20ba",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2026-05-19T20:33:41.776624Z",
     "iopub.status.busy": "2026-05-19T20:33:41.776495Z",
     "iopub.status.idle": "2026-05-19T20:33:41.982825Z",
     "shell.execute_reply": "2026-05-19T20:33:41.982439Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "input_line_60:2:59: error: reference to 'data' is ambiguous\n",
      " std::unique_ptr<RooFitResult> fitWithRecovery{pdf.fitTo(*data, RooFit::Save(),\n",
      "                                                          ^\n",
      "input_line_53:2:30: note: candidate found by name lookup is 'data'\n",
      " std::unique_ptr<RooDataSet> data(pdf.generate(x, 10000));\n",
      "                             ^\n",
      "/usr/lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/range_access.h:344:5: note: candidate found by name lookup is 'std::data'\n",
      "    data(initializer_list<_Tp> __il) noexcept\n",
      "    ^\n",
      "/usr/lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/range_access.h:312:5: note: candidate found by name lookup is 'std::data'\n",
      "    data(_Container& __cont) noexcept(noexcept(__cont.data()))\n",
      "    ^\n",
      "/usr/lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/range_access.h:323:5: note: candidate found by name lookup is 'std::data'\n",
      "    data(const _Container& __cont) noexcept(noexcept(__cont.data()))\n",
      "    ^\n",
      "/usr/lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/range_access.h:334:5: note: candidate found by name lookup is 'std::data'\n",
      "    data(_Tp (&__array)[_Nm]) noexcept\n",
      "    ^\n",
      "input_line_60:8:12: error: use of undeclared identifier 'frame'\n",
      "pdf.plotOn(frame, RooFit::LineColor(kBlue), RooFit::Name(\"recovery\"));\n",
      "           ^\n"
     ]
    }
   ],
   "source": [
    "std::unique_ptr<RooFitResult> fitWithRecovery{pdf.fitTo(*data, RooFit::Save(),\n",
    "    RooFit::RecoverFromUndefinedRegions(1.), // The magnitude of the recovery information can be chosen here.\n",
    "                                             // Higher values mean more aggressive recovery.\n",
    "    RooFit::PrintEvalErrors(-1), // We are still expecting a few evaluation errors.\n",
    "    RooFit::PrintLevel(0))};\n",
    "\n",
    "pdf.plotOn(frame, RooFit::LineColor(kBlue), RooFit::Name(\"recovery\"));"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e947c192",
   "metadata": {},
   "source": [
    "Collect results and plot.\n",
    "--------------------------------\n",
    "We print the two fit results, and plot the fitted curves.\n",
    "The curve of the fit without recovery cannot be plotted, because the PDF is undefined if a2 < 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "5dc0305b",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2026-05-19T20:33:41.983962Z",
     "iopub.status.busy": "2026-05-19T20:33:41.983855Z",
     "iopub.status.idle": "2026-05-19T20:33:42.192271Z",
     "shell.execute_reply": "2026-05-19T20:33:42.191894Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "input_line_62:2:3: error: use of undeclared identifier 'fitWithoutRecovery'\n",
      " (fitWithoutRecovery->Print())\n",
      "  ^\n",
      "Error in <HandleInterpreterException>: Error evaluating expression (fitWithoutRecovery->Print())\n",
      "Execution of your code was aborted.\n"
     ]
    }
   ],
   "source": [
    "fitWithoutRecovery->Print();\n",
    "std::cout << \"Without recovery, the fitter encountered \" << fitWithoutRecovery->numInvalidNLL()\n",
    "    << \" invalid function values. The parameters are unchanged.\" << std::endl;\n",
    "\n",
    "fitWithRecovery->Print();\n",
    "std::cout << \"With recovery, the fitter encountered \" << fitWithRecovery->numInvalidNLL()\n",
    "    << \" invalid function values, but the parameters are fitted.\" << std::endl;\n",
    "\n",
    "TLegend* legend = new TLegend(0.5, 0.7, 0.9, 0.9);\n",
    "legend->SetBorderSize(0);\n",
    "legend->SetFillStyle(0);\n",
    "legend->AddEntry(\"data\", \"Data\", \"P\");\n",
    "legend->AddEntry(\"noRecovery\", \"Without recovery (cannot be plotted)\", \"L\");\n",
    "legend->AddEntry(\"recovery\", \"With recovery\", \"L\");\n",
    "frame->Draw();\n",
    "legend->Draw();\n",
    "c->Draw();"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5644d5c",
   "metadata": {},
   "source": [
    "Draw all canvases "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "77aa9a4f",
   "metadata": {
    "collapsed": false,
    "execution": {
     "iopub.execute_input": "2026-05-19T20:33:42.193393Z",
     "iopub.status.busy": "2026-05-19T20:33:42.193284Z",
     "iopub.status.idle": "2026-05-19T20:33:42.422405Z",
     "shell.execute_reply": "2026-05-19T20:33:42.421950Z"
    }
   },
   "outputs": [],
   "source": [
    "%jsroot on\n",
    "gROOT->GetListOfCanvases()->Draw()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "ROOT C++",
   "language": "c++",
   "name": "root"
  },
  "language_info": {
   "codemirror_mode": "text/x-c++src",
   "file_extension": ".C",
   "mimetype": " text/x-c++src",
   "name": "c++"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}