{
"cells": [
{
"cell_type": "markdown",
"id": "67564b31",
"metadata": {},
"source": [
"# TMVA_Higgs_Classification\n",
"Classification example of TMVA based on public Higgs UCI dataset\n",
"\n",
" The UCI data set is a public HIGGS data set , see http://archive.ics.uci.edu/ml/datasets/HIGGS\n",
"used in this paper: Baldi, P., P. Sadowski, and D. Whiteson. “Searching for Exotic Particles in High-energy Physics\n",
" with Deep Learning.” Nature Communications 5 (July 2, 2014).\n",
"\n",
"\n",
"\n",
"\n",
"**Author:** Harshal Shende \n",
"This notebook tutorial was automatically generated with ROOTBOOK-izer from the macro found in the ROOT repository on Tuesday, May 19, 2026 at 08:22 PM."
]
},
{
"cell_type": "markdown",
"id": "0a72dc53",
"metadata": {},
"source": [
" Declare Factory"
]
},
{
"cell_type": "markdown",
"id": "70e90ba6",
"metadata": {},
"source": [
" Create the Factory class. Later you can choose the methods\n",
" whose performance you'd like to investigate."
]
},
{
"cell_type": "markdown",
"id": "44fd788d",
"metadata": {},
"source": [
" The factory is the major TMVA object you have to interact with. Here is the list of parameters you need to pass"
]
},
{
"cell_type": "markdown",
"id": "e8226ea1",
"metadata": {},
"source": [
" - The first argument is the base of the name of all the output\n",
" weightfiles in the directory weight/ that will be created with the\n",
" method parameters"
]
},
{
"cell_type": "markdown",
"id": "7cd6bf20",
"metadata": {},
"source": [
" - The second argument is the output file for the training results"
]
},
{
"cell_type": "markdown",
"id": "2159b88a",
"metadata": {},
"source": [
" - The third argument is a string option defining some general configuration for the TMVA session. For example all TMVA output can be suppressed by removing the \"!\" (not) in front of the \"Silent\" argument in the option string"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "64844072",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:22:56.384891Z",
"iopub.status.busy": "2026-05-19T20:22:56.384721Z",
"iopub.status.idle": "2026-05-19T20:22:57.390510Z",
"shell.execute_reply": "2026-05-19T20:22:57.390034Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import os\n",
"\n",
"import ROOT\n",
"\n",
"TMVA = ROOT.TMVA\n",
"TFile = ROOT.TFile\n",
"\n",
"TMVA.Tools.Instance()"
]
},
{
"cell_type": "markdown",
"id": "18eb0c57",
"metadata": {},
"source": [
"options to control used methods"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "d1491412",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:22:57.392464Z",
"iopub.status.busy": "2026-05-19T20:22:57.392322Z",
"iopub.status.idle": "2026-05-19T20:22:57.569831Z",
"shell.execute_reply": "2026-05-19T20:22:57.569261Z"
}
},
"outputs": [],
"source": [
"useLikelihood = True # likelihood based discriminant\n",
"useLikelihoodKDE = False # likelihood based discriminant\n",
"useFischer = True # Fischer discriminant\n",
"useMLP = False # Multi Layer Perceptron (old TMVA NN implementation)\n",
"useBDT = True # Boosted Decision Tree\n",
"useDL = True # TMVA Deep learning ( CPU or GPU)\n",
"useKeras = True # Use Keras Deep Learning via PyMVA\n",
"\n",
"if ROOT.gSystem.GetFromPipe(\"root-config --has-tmva-pymva\") == \"yes\":\n",
" TMVA.PyMethodBase.PyInitialize()\n",
"else:\n",
" useKeras = False # cannot use Keras if PYMVA is not available\n",
"\n",
"if useKeras:\n",
" try:\n",
" pass\n",
" except:\n",
" ROOT.Warning(\"TMVA_Higgs_Classification\", \"Skip using Keras since tensorflow is not available\")\n",
" useKeras = False\n",
"\n",
"outputFile = TFile.Open(\"Higgs_ClassificationOutput.root\", \"RECREATE\")\n",
"factory = TMVA.Factory(\n",
" \"TMVA_Higgs_Classification\", outputFile, V=False, ROC=True, Silent=False, Color=True, AnalysisType=\"Classification\"\n",
")"
]
},
{
"cell_type": "markdown",
"id": "0fa61a41",
"metadata": {},
"source": [
" Setup Dataset(s)"
]
},
{
"cell_type": "markdown",
"id": "05d47486",
"metadata": {},
"source": [
"Define now input data file and signal and background trees"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "332e1240",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:22:57.571828Z",
"iopub.status.busy": "2026-05-19T20:22:57.571698Z",
"iopub.status.idle": "2026-05-19T20:22:57.704741Z",
"shell.execute_reply": "2026-05-19T20:22:57.704158Z"
}
},
"outputs": [],
"source": [
"inputFileName = str(ROOT.gROOT.GetTutorialDir()) + \"/machine_learning/data/Higgs_data.root\"\n",
"\n",
"inputFile = TFile.Open(inputFileName)\n",
"if inputFile is None:\n",
" raise FileNotFoundError(\"Input file is not found - exit\")"
]
},
{
"cell_type": "markdown",
"id": "f099694e",
"metadata": {},
"source": [
"--- Register the training and test trees"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "01c7cdc6",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:22:57.707227Z",
"iopub.status.busy": "2026-05-19T20:22:57.707094Z",
"iopub.status.idle": "2026-05-19T20:22:57.841328Z",
"shell.execute_reply": "2026-05-19T20:22:57.840793Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"******************************************************************************\n",
"*Tree :sig_tree : tree *\n",
"*Entries : 10000 : Total = 1177229 bytes File Size = 785298 *\n",
"* : : Tree compression factor = 1.48 *\n",
"******************************************************************************\n",
"*Br 0 :Type : Type/F *\n",
"*Entries : 10000 : Total Size= 40556 bytes File Size = 307 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 130.54 *\n",
"*............................................................................*\n",
"*Br 1 :lepton_pT : lepton_pT/F *\n",
"*Entries : 10000 : Total Size= 40581 bytes File Size = 30464 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.32 *\n",
"*............................................................................*\n",
"*Br 2 :lepton_eta : lepton_eta/F *\n",
"*Entries : 10000 : Total Size= 40586 bytes File Size = 28650 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.40 *\n",
"*............................................................................*\n",
"*Br 3 :lepton_phi : lepton_phi/F *\n",
"*Entries : 10000 : Total Size= 40586 bytes File Size = 30508 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.31 *\n",
"*............................................................................*\n",
"*Br 4 :missing_energy_magnitude : missing_energy_magnitude/F *\n",
"*Entries : 10000 : Total Size= 40656 bytes File Size = 35749 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.12 *\n",
"*............................................................................*\n",
"*Br 5 :missing_energy_phi : missing_energy_phi/F *\n",
"*Entries : 10000 : Total Size= 40626 bytes File Size = 36766 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.09 *\n",
"*............................................................................*\n",
"*Br 6 :jet1_pt : jet1_pt/F *\n",
"*Entries : 10000 : Total Size= 40571 bytes File Size = 32298 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.24 *\n",
"*............................................................................*\n",
"*Br 7 :jet1_eta : jet1_eta/F *\n",
"*Entries : 10000 : Total Size= 40576 bytes File Size = 28467 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.41 *\n",
"*............................................................................*\n",
"*Br 8 :jet1_phi : jet1_phi/F *\n",
"*Entries : 10000 : Total Size= 40576 bytes File Size = 30399 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.32 *\n",
"*............................................................................*\n",
"*Br 9 :jet1_b-tag : jet1_b-tag/F *\n",
"*Entries : 10000 : Total Size= 40586 bytes File Size = 5087 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 7.88 *\n",
"*............................................................................*\n",
"*Br 10 :jet2_pt : jet2_pt/F *\n",
"*Entries : 10000 : Total Size= 40571 bytes File Size = 31561 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.27 *\n",
"*............................................................................*\n",
"*Br 11 :jet2_eta : jet2_eta/F *\n",
"*Entries : 10000 : Total Size= 40576 bytes File Size = 28616 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.40 *\n",
"*............................................................................*\n",
"*Br 12 :jet2_phi : jet2_phi/F *\n",
"*Entries : 10000 : Total Size= 40576 bytes File Size = 30547 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.31 *\n",
"*............................................................................*\n",
"*Br 13 :jet2_b-tag : jet2_b-tag/F *\n",
"*Entries : 10000 : Total Size= 40586 bytes File Size = 5031 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 7.97 *\n",
"*............................................................................*\n",
"*Br 14 :jet3_pt : jet3_pt/F *\n",
"*Entries : 10000 : Total Size= 40571 bytes File Size = 30642 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.31 *\n",
"*............................................................................*\n",
"*Br 15 :jet3_eta : jet3_eta/F *\n",
"*Entries : 10000 : Total Size= 40576 bytes File Size = 28955 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.38 *\n",
"*............................................................................*\n",
"*Br 16 :jet3_phi : jet3_phi/F *\n",
"*Entries : 10000 : Total Size= 40576 bytes File Size = 30433 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.32 *\n",
"*............................................................................*\n",
"*Br 17 :jet3_b-tag : jet3_b-tag/F *\n",
"*Entries : 10000 : Total Size= 40586 bytes File Size = 4879 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 8.22 *\n",
"*............................................................................*\n",
"*Br 18 :jet4_pt : jet4_pt/F *\n",
"*Entries : 10000 : Total Size= 40571 bytes File Size = 29189 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.37 *\n",
"*............................................................................*\n",
"*Br 19 :jet4_eta : jet4_eta/F *\n",
"*Entries : 10000 : Total Size= 40576 bytes File Size = 29311 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.37 *\n",
"*............................................................................*\n",
"*Br 20 :jet4_phi : jet4_phi/F *\n",
"*Entries : 10000 : Total Size= 40576 bytes File Size = 30525 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.31 *\n",
"*............................................................................*\n",
"*Br 21 :jet4_b-tag : jet4_b-tag/F *\n",
"*Entries : 10000 : Total Size= 40586 bytes File Size = 4725 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 8.48 *\n",
"*............................................................................*\n",
"*Br 22 :m_jj : m_jj/F *\n",
"*Entries : 10000 : Total Size= 40556 bytes File Size = 34991 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.15 *\n",
"*............................................................................*\n",
"*Br 23 :m_jjj : m_jjj/F *\n",
"*Entries : 10000 : Total Size= 40561 bytes File Size = 34460 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.16 *\n",
"*............................................................................*\n",
"*Br 24 :m_lv : m_lv/F *\n",
"*Entries : 10000 : Total Size= 40556 bytes File Size = 32232 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.24 *\n",
"*............................................................................*\n",
"*Br 25 :m_jlv : m_jlv/F *\n",
"*Entries : 10000 : Total Size= 40561 bytes File Size = 34598 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.16 *\n",
"*............................................................................*\n",
"*Br 26 :m_bb : m_bb/F *\n",
"*Entries : 10000 : Total Size= 40556 bytes File Size = 35012 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.14 *\n",
"*............................................................................*\n",
"*Br 27 :m_wbb : m_wbb/F *\n",
"*Entries : 10000 : Total Size= 40561 bytes File Size = 34493 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.16 *\n",
"*............................................................................*\n",
"*Br 28 :m_wwbb : m_wwbb/F *\n",
"*Entries : 10000 : Total Size= 40566 bytes File Size = 34410 *\n",
"*Baskets : 1 : Basket Size= 1500672 bytes Compression= 1.16 *\n",
"*............................................................................*\n"
]
}
],
"source": [
"signalTree = inputFile.Get(\"sig_tree\")\n",
"backgroundTree = inputFile.Get(\"bkg_tree\")\n",
"signalTree.Print()"
]
},
{
"cell_type": "markdown",
"id": "fdd0e7bb",
"metadata": {},
"source": [
" Declare DataLoader(s)"
]
},
{
"cell_type": "markdown",
"id": "2056cbe3",
"metadata": {},
"source": [
"The next step is to declare the DataLoader class that deals with input variables\n",
"Define the input variables that shall be used for the MVA training\n",
"note that you may also use variable expressions, which can be parsed by TTree::Draw( \"expression\" )]"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "fc2e9ae0",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:22:57.843275Z",
"iopub.status.busy": "2026-05-19T20:22:57.843148Z",
"iopub.status.idle": "2026-05-19T20:22:57.980701Z",
"shell.execute_reply": "2026-05-19T20:22:57.962343Z"
}
},
"outputs": [],
"source": [
"loader = TMVA.DataLoader(\"dataset\")\n",
"\n",
"loader.AddVariable(\"m_jj\")\n",
"loader.AddVariable(\"m_jjj\")\n",
"loader.AddVariable(\"m_lv\")\n",
"loader.AddVariable(\"m_jlv\")\n",
"loader.AddVariable(\"m_bb\")\n",
"loader.AddVariable(\"m_wbb\")\n",
"loader.AddVariable(\"m_wwbb\")"
]
},
{
"cell_type": "markdown",
"id": "5259a2da",
"metadata": {},
"source": [
"We set now the input data trees in the TMVA DataLoader class\n",
"global event weights per tree (see below for setting event-wise weights)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "e76951bf",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:22:57.982580Z",
"iopub.status.busy": "2026-05-19T20:22:57.982441Z",
"iopub.status.idle": "2026-05-19T20:22:58.096013Z",
"shell.execute_reply": "2026-05-19T20:22:58.086038Z"
}
},
"outputs": [],
"source": [
"signalWeight = 1.0\n",
"backgroundWeight = 1.0"
]
},
{
"cell_type": "markdown",
"id": "671c00cf",
"metadata": {},
"source": [
"You can add an arbitrary number of signal or background trees"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "b65cd68e",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:22:58.097718Z",
"iopub.status.busy": "2026-05-19T20:22:58.097553Z",
"iopub.status.idle": "2026-05-19T20:22:58.216210Z",
"shell.execute_reply": "2026-05-19T20:22:58.212536Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"DataSetInfo : [dataset] : Added class \"Signal\"\n",
" : Add Tree sig_tree of type Signal with 10000 events\n",
"DataSetInfo : [dataset] : Added class \"Background\"\n",
" : Add Tree bkg_tree of type Background with 10000 events\n"
]
}
],
"source": [
"loader.AddSignalTree(signalTree, signalWeight)\n",
"loader.AddBackgroundTree(backgroundTree, backgroundWeight)"
]
},
{
"cell_type": "markdown",
"id": "665ac15c",
"metadata": {},
"source": [
"Set individual event weights (the variables must exist in the original TTree)\n",
" for signal : factory->SetSignalWeightExpression (\"weight1*weight2\");\n",
" for background: factory->SetBackgroundWeightExpression(\"weight1*weight2\");\n",
" loader->SetBackgroundWeightExpression( \"weight\" );"
]
},
{
"cell_type": "markdown",
"id": "a1fdcbf0",
"metadata": {},
"source": [
"Apply additional cuts on the signal and background samples (can be different)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "70d2ad8e",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:22:58.217837Z",
"iopub.status.busy": "2026-05-19T20:22:58.217712Z",
"iopub.status.idle": "2026-05-19T20:22:58.330112Z",
"shell.execute_reply": "2026-05-19T20:22:58.326718Z"
}
},
"outputs": [],
"source": [
"mycuts = ROOT.TCut(\"\") # for example: TCut mycuts = \"abs(var1)<0.5 && abs(var2-0.5)<1\";\n",
"mycutb = ROOT.TCut(\"\") # for example: TCut mycutb = \"abs(var1)<0.5\";"
]
},
{
"cell_type": "markdown",
"id": "e9626102",
"metadata": {},
"source": [
"Tell the factory how to use the training and testing events\n",
"\n",
"If no numbers of events are given, half of the events in the tree are used\n",
"for training, and the other half for testing:\n",
" loader->PrepareTrainingAndTestTree( mycut, \"SplitMode=random:!V\" );\n",
"To also specify the number of testing events, use:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "fe0c4ab1",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:22:58.331921Z",
"iopub.status.busy": "2026-05-19T20:22:58.331795Z",
"iopub.status.idle": "2026-05-19T20:22:58.440272Z",
"shell.execute_reply": "2026-05-19T20:22:58.439642Z"
}
},
"outputs": [],
"source": [
"loader.PrepareTrainingAndTestTree(\n",
" mycuts, mycutb, nTrain_Signal=7000, nTrain_Background=7000, SplitMode=\"Random\", NormMode=\"NumEvents\", V=False\n",
")"
]
},
{
"cell_type": "markdown",
"id": "0bc9b27a",
"metadata": {},
"source": [
" Booking Methods"
]
},
{
"cell_type": "markdown",
"id": "720d3be8",
"metadata": {},
"source": [
"Here we book the TMVA methods. We book first a Likelihood based on KDE (Kernel Density Estimation), a Fischer discriminant, a BDT\n",
"and a shallow neural network\n",
"Likelihood (\"naive Bayes estimator\")"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "6c902e04",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:22:58.445733Z",
"iopub.status.busy": "2026-05-19T20:22:58.445583Z",
"iopub.status.idle": "2026-05-19T20:22:58.584719Z",
"shell.execute_reply": "2026-05-19T20:22:58.584134Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Factory : Booking method: \u001b[1mLikelihood\u001b[0m\n",
" : \n"
]
}
],
"source": [
"if useLikelihood:\n",
" factory.BookMethod(\n",
" loader,\n",
" TMVA.Types.kLikelihood,\n",
" \"Likelihood\",\n",
" H=True,\n",
" V=False,\n",
" TransformOutput=True,\n",
" PDFInterpol=\"Spline2:NSmoothSig[0]=20:NSmoothBkg[0]=20:NSmoothBkg[1]=10\",\n",
" NSmooth=1,\n",
" NAvEvtPerBin=50,\n",
" )"
]
},
{
"cell_type": "markdown",
"id": "a0297af9",
"metadata": {},
"source": [
"Use a kernel density estimator to approximate the PDFs"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "9bd76fc0",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:22:58.586595Z",
"iopub.status.busy": "2026-05-19T20:22:58.586465Z",
"iopub.status.idle": "2026-05-19T20:22:58.690055Z",
"shell.execute_reply": "2026-05-19T20:22:58.689497Z"
}
},
"outputs": [],
"source": [
"if useLikelihoodKDE:\n",
" factory.BookMethod(\n",
" loader,\n",
" TMVA.Types.kLikelihood,\n",
" \"LikelihoodKDE\",\n",
" H=False,\n",
" V=False,\n",
" TransformOutput=False,\n",
" PDFInterpol=\"KDE\",\n",
" KDEtype=\"Gauss\",\n",
" KDEiter=\"Adaptive\",\n",
" KDEFineFactor=0.3,\n",
" KDEborder=None,\n",
" NAvEvtPerBin=50,\n",
" )"
]
},
{
"cell_type": "markdown",
"id": "d7292971",
"metadata": {},
"source": [
"Fisher discriminant (same as LD)"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "eddbf683",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:22:58.691911Z",
"iopub.status.busy": "2026-05-19T20:22:58.691786Z",
"iopub.status.idle": "2026-05-19T20:22:58.801569Z",
"shell.execute_reply": "2026-05-19T20:22:58.801052Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Factory : Booking method: \u001b[1mFisher\u001b[0m\n",
" : \n"
]
}
],
"source": [
"if useFischer:\n",
" factory.BookMethod(\n",
" loader,\n",
" TMVA.Types.kFisher,\n",
" \"Fisher\",\n",
" H=True,\n",
" V=False,\n",
" Fisher=True,\n",
" VarTransform=None,\n",
" CreateMVAPdfs=True,\n",
" PDFInterpolMVAPdf=\"Spline2\",\n",
" NbinsMVAPdf=50,\n",
" NsmoothMVAPdf=10,\n",
" )"
]
},
{
"cell_type": "markdown",
"id": "682e9e4a",
"metadata": {},
"source": [
"Boosted Decision Trees"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "56085ec9",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:22:58.803525Z",
"iopub.status.busy": "2026-05-19T20:22:58.803401Z",
"iopub.status.idle": "2026-05-19T20:22:58.951469Z",
"shell.execute_reply": "2026-05-19T20:22:58.941881Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Factory : Booking method: \u001b[1mBDT\u001b[0m\n",
" : \n",
" : Rebuilding Dataset dataset\n",
" : Building event vectors for type 2 Signal\n",
" : Dataset[dataset] : create input formulas for tree sig_tree\n",
" : Building event vectors for type 2 Background\n",
" : Dataset[dataset] : create input formulas for tree bkg_tree\n",
"DataSetFactory : [dataset] : Number of events in input trees\n",
" : \n",
" : \n",
" : Number of training and testing events\n",
" : ---------------------------------------------------------------------------\n",
" : Signal -- training events : 7000\n",
" : Signal -- testing events : 3000\n",
" : Signal -- training and testing events: 10000\n",
" : Background -- training events : 7000\n",
" : Background -- testing events : 3000\n",
" : Background -- training and testing events: 10000\n",
" : \n",
"DataSetInfo : Correlation matrix (Signal):\n",
" : ----------------------------------------------------------------\n",
" : m_jj m_jjj m_lv m_jlv m_bb m_wbb m_wwbb\n",
" : m_jj: +1.000 +0.777 +0.010 +0.107 +0.036 +0.517 +0.532\n",
" : m_jjj: +0.777 +1.000 +0.006 +0.083 +0.157 +0.682 +0.669\n",
" : m_lv: +0.010 +0.006 +1.000 +0.111 -0.026 +0.011 +0.023\n",
" : m_jlv: +0.107 +0.083 +0.111 +1.000 +0.325 +0.550 +0.555\n",
" : m_bb: +0.036 +0.157 -0.026 +0.325 +1.000 +0.463 +0.347\n",
" : m_wbb: +0.517 +0.682 +0.011 +0.550 +0.463 +1.000 +0.912\n",
" : m_wwbb: +0.532 +0.669 +0.023 +0.555 +0.347 +0.912 +1.000\n",
" : ----------------------------------------------------------------\n",
"DataSetInfo : Correlation matrix (Background):\n",
" : ----------------------------------------------------------------\n",
" : m_jj m_jjj m_lv m_jlv m_bb m_wbb m_wwbb\n",
" : m_jj: +1.000 +0.804 +0.017 +0.125 +0.007 +0.381 +0.394\n",
" : m_jjj: +0.804 +1.000 +0.025 +0.159 +0.153 +0.535 +0.520\n",
" : m_lv: +0.017 +0.025 +1.000 +0.114 +0.042 +0.064 +0.069\n",
" : m_jlv: +0.125 +0.159 +0.114 +1.000 +0.286 +0.592 +0.542\n",
" : m_bb: +0.007 +0.153 +0.042 +0.286 +1.000 +0.623 +0.441\n",
" : m_wbb: +0.381 +0.535 +0.064 +0.592 +0.623 +1.000 +0.878\n",
" : m_wwbb: +0.394 +0.520 +0.069 +0.542 +0.441 +0.878 +1.000\n",
" : ----------------------------------------------------------------\n",
"DataSetFactory : [dataset] : \n",
" : \n"
]
}
],
"source": [
"if useBDT:\n",
" factory.BookMethod(\n",
" loader,\n",
" TMVA.Types.kBDT,\n",
" \"BDT\",\n",
" V=False,\n",
" NTrees=200,\n",
" MinNodeSize=\"2.5%\",\n",
" MaxDepth=2,\n",
" BoostType=\"AdaBoost\",\n",
" AdaBoostBeta=0.5,\n",
" UseBaggedBoost=True,\n",
" BaggedSampleFraction=0.5,\n",
" SeparationType=\"GiniIndex\",\n",
" nCuts=20,\n",
" )"
]
},
{
"cell_type": "markdown",
"id": "81b665e2",
"metadata": {},
"source": [
"Multi-Layer Perceptron (Neural Network)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "3d03f668",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:22:58.953749Z",
"iopub.status.busy": "2026-05-19T20:22:58.953612Z",
"iopub.status.idle": "2026-05-19T20:22:59.067623Z",
"shell.execute_reply": "2026-05-19T20:22:59.058761Z"
}
},
"outputs": [],
"source": [
"if useMLP:\n",
" factory.BookMethod(\n",
" loader,\n",
" TMVA.Types.kMLP,\n",
" \"MLP\",\n",
" H=False,\n",
" V=False,\n",
" NeuronType=\"tanh\",\n",
" VarTransform=\"N\",\n",
" NCycles=100,\n",
" HiddenLayers=\"N+5\",\n",
" TestRate=5,\n",
" UseRegulator=False,\n",
" )"
]
},
{
"cell_type": "markdown",
"id": "af642a25",
"metadata": {},
"source": [
" Here we book the new DNN of TMVA if we have support in ROOT. We will use GPU version if ROOT is enabled with GPU"
]
},
{
"cell_type": "markdown",
"id": "afbd9a8a",
"metadata": {},
"source": [
" Booking Deep Neural Network"
]
},
{
"cell_type": "markdown",
"id": "35c15377",
"metadata": {},
"source": [
"Here we define the option string for building the Deep Neural network model."
]
},
{
"cell_type": "markdown",
"id": "ed52a86e",
"metadata": {},
"source": [
"## 1. Define DNN layout"
]
},
{
"cell_type": "markdown",
"id": "becbbd5a",
"metadata": {},
"source": [
"The DNN configuration is defined using a string. Note that whitespaces between characters are not allowed."
]
},
{
"cell_type": "markdown",
"id": "24cb27cc",
"metadata": {},
"source": [
"We define first the DNN layout:"
]
},
{
"cell_type": "markdown",
"id": "bd69d42f",
"metadata": {},
"source": [
"- **input layout** : this defines the input data format for the DNN as ``input depth | height | width``.\n",
"In case of a dense layer as first layer the input layout should be ``1 | 1 | number of input variables`` (features)\n",
"- **batch layout** : this defines how are the input batch. It is related to input layout but not the same.\n",
"If the first layer is dense it should be ``1 | batch size ! number of variables`` (features)"
]
},
{
"cell_type": "markdown",
"id": "4d2e411b",
"metadata": {},
"source": [
"*(note the use of the character `|` as separator of input parameters for DNN layout)*"
]
},
{
"cell_type": "markdown",
"id": "83fb9ef9",
"metadata": {},
"source": [
"note that in case of only dense layer the input layout could be omitted but it is required when defining more\n",
"complex architectures"
]
},
{
"cell_type": "markdown",
"id": "7f846102",
"metadata": {},
"source": [
"- **layer layout** string defining the layer architecture. The syntax is\n",
"- layer type (e.g. DENSE, CONV, RNN)\n",
"- layer parameters (e.g. number of units)\n",
"- activation function (e.g TANH, RELU,...)"
]
},
{
"cell_type": "markdown",
"id": "c4e81026",
"metadata": {},
"source": [
"*the different layers are separated by the ``\",\"`` *"
]
},
{
"cell_type": "markdown",
"id": "e294572b",
"metadata": {},
"source": [
"## 2. Define Training Strategy"
]
},
{
"cell_type": "markdown",
"id": "2917bc4d",
"metadata": {},
"source": [
"We define here the training strategy parameters for the DNN. The parameters are separated by the ``\",\"`` separator.\n",
"One can then concatenate different training strategy with different parameters. The training strategy are separated by\n",
"the ``\"|\"`` separator."
]
},
{
"cell_type": "markdown",
"id": "a2705386",
"metadata": {},
"source": [
"- Optimizer\n",
"- Learning rate\n",
"- Momentum (valid for SGD and RMSPROP)\n",
"- Regularization and Weight Decay\n",
"- Dropout\n",
"- Max number of epochs\n",
"- Convergence steps. if the test error will not decrease after that value the training will stop\n",
"- Batch size (This value must be the same specified in the input layout)\n",
"- Test Repetitions (the interval when the test error will be computed)"
]
},
{
"cell_type": "markdown",
"id": "fe37630d",
"metadata": {},
"source": [
"## 3. Define general DNN options"
]
},
{
"cell_type": "markdown",
"id": "35f569db",
"metadata": {},
"source": [
"We define the general DNN options concatenating in the final string the previously defined layout and training strategy.\n",
"Note we use the ``\":\"`` separator to separate the different higher level options, as in the other TMVA methods.\n",
"In addition to input layout, batch layout and training strategy we add now:"
]
},
{
"cell_type": "markdown",
"id": "d169cc8c",
"metadata": {},
"source": [
"- Type of Loss function (e.g. CROSSENTROPY)\n",
"- Weight Initizalization (e.g XAVIER, XAVIERUNIFORM, NORMAL )\n",
"- Variable Transformation\n",
"- Type of Architecture (e.g. CPU, GPU, Standard)"
]
},
{
"cell_type": "markdown",
"id": "e9bc7b4b",
"metadata": {},
"source": [
"We can then book the DL method using the built option string"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "5c68c90a",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:22:59.070651Z",
"iopub.status.busy": "2026-05-19T20:22:59.070498Z",
"iopub.status.idle": "2026-05-19T20:22:59.206503Z",
"shell.execute_reply": "2026-05-19T20:22:59.205999Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Factory : Booking method: \u001b[1mDNN_CPU\u001b[0m\n",
" : \n",
" : Parsing option string: \n",
" : ... \"!H:V:ErrorStrategy=CROSSENTROPY:VarTransform=G:WeightInitialization=XAVIER:InputLayout=1|1|7:BatchLayout=1|128|7:Layout=DENSE|64|TANH,DENSE|64|TANH,DENSE|64|TANH,DENSE|64|TANH,DENSE|1|LINEAR:TrainingStrategy=LearningRate=1e-3,Momentum=0.9,ConvergenceSteps=10,BatchSize=128,TestRepetitions=1,MaxEpochs=20,WeightDecay=1e-4,Regularization=None,Optimizer=ADAM,ADAM_beta1=0.9,ADAM_beta2=0.999,ADAM_eps=1.E-7,DropConfig=0.0+0.0+0.0+0.:Architecture=CPU\"\n",
" : The following options are set:\n",
" : - By User:\n",
" : \n",
" : - Default:\n",
" : Boost_num: \"0\" [Number of times the classifier will be boosted]\n",
" : Parsing option string: \n",
" : ... \"!H:V:ErrorStrategy=CROSSENTROPY:VarTransform=G:WeightInitialization=XAVIER:InputLayout=1|1|7:BatchLayout=1|128|7:Layout=DENSE|64|TANH,DENSE|64|TANH,DENSE|64|TANH,DENSE|64|TANH,DENSE|1|LINEAR:TrainingStrategy=LearningRate=1e-3,Momentum=0.9,ConvergenceSteps=10,BatchSize=128,TestRepetitions=1,MaxEpochs=20,WeightDecay=1e-4,Regularization=None,Optimizer=ADAM,ADAM_beta1=0.9,ADAM_beta2=0.999,ADAM_eps=1.E-7,DropConfig=0.0+0.0+0.0+0.:Architecture=CPU\"\n",
" : The following options are set:\n",
" : - By User:\n",
" : V: \"True\" [Verbose output (short form of \"VerbosityLevel\" below - overrides the latter one)]\n",
" : VarTransform: \"G\" [List of variable transformations performed before training, e.g., \"D_Background,P_Signal,G,N_AllClasses\" for: \"Decorrelation, PCA-transformation, Gaussianisation, Normalisation, each for the given class of events ('AllClasses' denotes all events of all classes, if no class indication is given, 'All' is assumed)\"]\n",
" : H: \"False\" [Print method-specific help message]\n",
" : InputLayout: \"1|1|7\" [The Layout of the input]\n",
" : BatchLayout: \"1|128|7\" [The Layout of the batch]\n",
" : Layout: \"DENSE|64|TANH,DENSE|64|TANH,DENSE|64|TANH,DENSE|64|TANH,DENSE|1|LINEAR\" [Layout of the network.]\n",
" : ErrorStrategy: \"CROSSENTROPY\" [Loss function: Mean squared error (regression) or cross entropy (binary classification).]\n",
" : WeightInitialization: \"XAVIER\" [Weight initialization strategy]\n",
" : Architecture: \"CPU\" [Which architecture to perform the training on.]\n",
" : TrainingStrategy: \"LearningRate=1e-3,Momentum=0.9,ConvergenceSteps=10,BatchSize=128,TestRepetitions=1,MaxEpochs=20,WeightDecay=1e-4,Regularization=None,Optimizer=ADAM,ADAM_beta1=0.9,ADAM_beta2=0.999,ADAM_eps=1.E-7,DropConfig=0.0+0.0+0.0+0.\" [Defines the training strategies.]\n",
" : - Default:\n",
" : VerbosityLevel: \"Default\" [Verbosity level]\n",
" : CreateMVAPdfs: \"False\" [Create PDFs for classifier outputs (signal and background)]\n",
" : IgnoreNegWeightsInTraining: \"False\" [Events with negative weights are ignored in the training (but are included for testing and performance evaluation)]\n",
" : RandomSeed: \"0\" [Random seed used for weight initialization and batch shuffling]\n",
" : ValidationSize: \"20%\" [Part of the training data to use for validation. Specify as 0.2 or 20% to use a fifth of the data set as validation set. Specify as 100 to use exactly 100 events. (Default: 20%)]\n",
"DNN_CPU : [dataset] : Create Transformation \"G\" with events from all classes.\n",
" : \n",
" : Transformation, Variable selection : \n",
" : Input : variable 'm_jj' <---> Output : variable 'm_jj'\n",
" : Input : variable 'm_jjj' <---> Output : variable 'm_jjj'\n",
" : Input : variable 'm_lv' <---> Output : variable 'm_lv'\n",
" : Input : variable 'm_jlv' <---> Output : variable 'm_jlv'\n",
" : Input : variable 'm_bb' <---> Output : variable 'm_bb'\n",
" : Input : variable 'm_wbb' <---> Output : variable 'm_wbb'\n",
" : Input : variable 'm_wwbb' <---> Output : variable 'm_wwbb'\n",
" : Will now use the CPU architecture with BLAS and IMT support !\n"
]
}
],
"source": [
"if useDL:\n",
" useDLGPU = ROOT.gSystem.GetFromPipe(\"root-config --has-tmva-gpu\") == \"yes\"\n",
"\n",
" # Define DNN layout\n",
" # Define Training strategies\n",
" # one can catenate several training strategies\n",
" training1 = ROOT.TString(\n",
" \"LearningRate=1e-3,Momentum=0.9,\"\n",
" \"ConvergenceSteps=10,BatchSize=128,TestRepetitions=1,\"\n",
" \"MaxEpochs=20,WeightDecay=1e-4,Regularization=None,\"\n",
" \"Optimizer=ADAM,ADAM_beta1=0.9,ADAM_beta2=0.999,ADAM_eps=1.E-7,\" # ADAM default parameters\n",
" \"DropConfig=0.0+0.0+0.0+0.\"\n",
" )\n",
" # training2 = ROOT.TString(\"LearningRate=1e-3,Momentum=0.9\"\n",
" # \"ConvergenceSteps=10,BatchSize=128,TestRepetitions=1,\"\n",
" # \"MaxEpochs=20,WeightDecay=1e-4,Regularization=None,\"\n",
" # \"Optimizer=SGD,DropConfig=0.0+0.0+0.0+0.\")\n",
"\n",
" # General Options.\n",
" dnnMethodName = ROOT.TString(\"DNN_CPU\")\n",
"\n",
" if useDLGPU:\n",
" arch = \"GPU\"\n",
" dnnMethodName = \"DNN_GPU\"\n",
" else:\n",
" arch = \"CPU\"\n",
"\n",
" factory.BookMethod(\n",
" loader,\n",
" TMVA.Types.kDL,\n",
" dnnMethodName,\n",
" H=False,\n",
" V=True,\n",
" ErrorStrategy=\"CROSSENTROPY\",\n",
" VarTransform=\"G\",\n",
" WeightInitialization=\"XAVIER\",\n",
" InputLayout=\"1|1|7\",\n",
" BatchLayout=\"1|128|7\",\n",
" Layout=\"DENSE|64|TANH,DENSE|64|TANH,DENSE|64|TANH,DENSE|64|TANH,DENSE|1|LINEAR\",\n",
" TrainingStrategy=training1,\n",
" Architecture=arch,\n",
" )"
]
},
{
"cell_type": "markdown",
"id": "ec979360",
"metadata": {},
"source": [
"Keras DL"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "82de9a1f",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:22:59.208155Z",
"iopub.status.busy": "2026-05-19T20:22:59.208030Z",
"iopub.status.idle": "2026-05-19T20:22:59.312577Z",
"shell.execute_reply": "2026-05-19T20:22:59.312051Z"
}
},
"outputs": [],
"source": [
"if useKeras:\n",
" ROOT.Info(\"TMVA_Higgs_Classification\", \"Building Deep Learning keras model\")\n",
" # create Keras model with 4 layers of 64 units and relu activations\n",
" from tensorflow.keras.layers import Dense\n",
" from tensorflow.keras.models import Sequential\n",
" from tensorflow.keras.optimizers import Adam\n",
"\n",
" model = Sequential()\n",
" model.add(Dense(64, activation=\"relu\", input_dim=7))\n",
" model.add(Dense(64, activation=\"relu\"))\n",
" model.add(Dense(64, activation=\"relu\"))\n",
" model.add(Dense(64, activation=\"relu\"))\n",
" model.add(Dense(2, activation=\"sigmoid\"))\n",
" model.compile(loss=\"binary_crossentropy\", optimizer=Adam(learning_rate=0.001), weighted_metrics=[\"accuracy\"])\n",
" model.save(\"model_higgs.keras\")\n",
" model.summary()\n",
"\n",
" if not os.path.exists(\"model_higgs.keras\"):\n",
" raise FileNotFoundError(\"Error creating Keras model file - skip using Keras\")\n",
" else:\n",
" # book PyKeras method only if Keras model could be created\n",
" ROOT.Info(\"TMVA_Higgs_Classification\", \"Booking Deep Learning keras model\")\n",
" factory.BookMethod(\n",
" loader,\n",
" TMVA.Types.kPyKeras,\n",
" \"PyKeras\",\n",
" H=True,\n",
" V=False,\n",
" VarTransform=None,\n",
" FilenameModel=\"model_higgs.keras\",\n",
" FilenameTrainedModel=\"trained_model_higgs.keras\",\n",
" NumEpochs=20,\n",
" BatchSize=100,\n",
" )"
]
},
{
"cell_type": "markdown",
"id": "3eaca06b",
"metadata": {},
"source": [
" GpuOptions=\"allow_growth=True\",\n",
" ) # needed for RTX NVidia card and to avoid TF allocates all GPU memory"
]
},
{
"cell_type": "markdown",
"id": "6fadb770",
"metadata": {},
"source": [
" Train Methods"
]
},
{
"cell_type": "markdown",
"id": "adaaf7ec",
"metadata": {},
"source": [
"Here we train all the previously booked methods."
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "4b36f4d0",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:22:59.314536Z",
"iopub.status.busy": "2026-05-19T20:22:59.314411Z",
"iopub.status.idle": "2026-05-19T20:23:06.106464Z",
"shell.execute_reply": "2026-05-19T20:23:06.105913Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Factory : \u001b[1mTrain all methods\u001b[0m\n",
"Factory : [dataset] : Create Transformation \"I\" with events from all classes.\n",
" : \n",
" : Transformation, Variable selection : \n",
" : Input : variable 'm_jj' <---> Output : variable 'm_jj'\n",
" : Input : variable 'm_jjj' <---> Output : variable 'm_jjj'\n",
" : Input : variable 'm_lv' <---> Output : variable 'm_lv'\n",
" : Input : variable 'm_jlv' <---> Output : variable 'm_jlv'\n",
" : Input : variable 'm_bb' <---> Output : variable 'm_bb'\n",
" : Input : variable 'm_wbb' <---> Output : variable 'm_wbb'\n",
" : Input : variable 'm_wwbb' <---> Output : variable 'm_wwbb'\n",
"TFHandler_Factory : Variable Mean RMS [ Min Max ]\n",
" : -----------------------------------------------------------\n",
" : m_jj: 1.0352 0.65399 [ 0.14661 13.098 ]\n",
" : m_jjj: 1.0218 0.36964 [ 0.34201 7.3920 ]\n",
" : m_lv: 1.0497 0.16065 [ 0.26679 3.6823 ]\n",
" : m_jlv: 1.0126 0.39935 [ 0.38441 6.5831 ]\n",
" : m_bb: 0.98070 0.53223 [ 0.093482 7.8598 ]\n",
" : m_wbb: 1.0338 0.35968 [ 0.38503 4.5425 ]\n",
" : m_wwbb: 0.96049 0.31009 [ 0.43228 4.0728 ]\n",
" : -----------------------------------------------------------\n",
" : Ranking input variables (method unspecific)...\n",
"IdTransformation : Ranking result (top variable is best ranked)\n",
" : -------------------------------\n",
" : Rank : Variable : Separation\n",
" : -------------------------------\n",
" : 1 : m_bb : 9.114e-02\n",
" : 2 : m_wwbb : 4.330e-02\n",
" : 3 : m_wbb : 4.241e-02\n",
" : 4 : m_jjj : 2.875e-02\n",
" : 5 : m_jlv : 1.905e-02\n",
" : 6 : m_jj : 3.432e-03\n",
" : 7 : m_lv : 2.855e-03\n",
" : -------------------------------\n",
"Factory : Train method: Likelihood for Classification\n",
" : \n",
" : \n",
" : \u001b[1m================================================================\u001b[0m\n",
" : \u001b[1mH e l p f o r M V A m e t h o d [ Likelihood ] :\u001b[0m\n",
" : \n",
" : \u001b[1m--- Short description:\u001b[0m\n",
" : \n",
" : The maximum-likelihood classifier models the data with probability \n",
" : density functions (PDF) reproducing the signal and background\n",
" : distributions of the input variables. Correlations among the \n",
" : variables are ignored.\n",
" : \n",
" : \u001b[1m--- Performance optimisation:\u001b[0m\n",
" : \n",
" : Required for good performance are decorrelated input variables\n",
" : (PCA transformation via the option \"VarTransform=Decorrelate\"\n",
" : may be tried). Irreducible non-linear correlations may be reduced\n",
" : by precombining strongly correlated input variables, or by simply\n",
" : removing one of the variables.\n",
" : \n",
" : \u001b[1m--- Performance tuning via configuration options:\u001b[0m\n",
" : \n",
" : High fidelity PDF estimates are mandatory, i.e., sufficient training \n",
" : statistics is required to populate the tails of the distributions\n",
" : It would be a surprise if the default Spline or KDE kernel parameters\n",
" : provide a satisfying fit to the data. The user is advised to properly\n",
" : tune the events per bin and smooth options in the spline cases\n",
" : individually per variable. If the KDE kernel is used, the adaptive\n",
" : Gaussian kernel may lead to artefacts, so please always also try\n",
" : the non-adaptive one.\n",
" : \n",
" : All tuning parameters must be adjusted individually for each input\n",
" : variable!\n",
" : \n",
" : \n",
" : \u001b[1m================================================================\u001b[0m\n",
" : \n",
" : Filling reference histograms\n",
" : Building PDF out of reference histograms\n",
" : Elapsed time for training with 14000 events: 0.0746 sec \n",
"Likelihood : [dataset] : Evaluation of Likelihood on training sample (14000 events)\n",
"Likelihood : [dataset] : Evaluation of Likelihood on training sample (14000 events)\n",
" : Elapsed time for evaluation of 14000 events: 0.0105 sec \n",
" : Elapsed time for evaluation of 14000 events: 0.0107 sec \n",
" : Creating xml weight file: \u001b[0;36mdataset/weights/TMVA_Higgs_Classification_Likelihood.weights.xml\u001b[0m\n",
" : Creating standalone class: \u001b[0;36mdataset/weights/TMVA_Higgs_Classification_Likelihood.class.C\u001b[0m\n",
" : Higgs_ClassificationOutput.root:/dataset/Method_Likelihood/Likelihood\n",
"Factory : Training finished\n",
" : \n",
"Factory : Train method: Fisher for Classification\n",
" : \n",
" : \n",
" : \u001b[1m================================================================\u001b[0m\n",
" : \u001b[1mH e l p f o r M V A m e t h o d [ Fisher ] :\u001b[0m\n",
" : \n",
" : \u001b[1m--- Short description:\u001b[0m\n",
" : \n",
" : Fisher discriminants select events by distinguishing the mean \n",
" : values of the signal and background distributions in a trans- \n",
" : formed variable space where linear correlations are removed.\n",
" : \n",
" : (More precisely: the \"linear discriminator\" determines\n",
" : an axis in the (correlated) hyperspace of the input \n",
" : variables such that, when projecting the output classes \n",
" : (signal and background) upon this axis, they are pushed \n",
" : as far as possible away from each other, while events\n",
" : of a same class are confined in a close vicinity. The \n",
" : linearity property of this classifier is reflected in the \n",
" : metric with which \"far apart\" and \"close vicinity\" are \n",
" : determined: the covariance matrix of the discriminating\n",
" : variable space.)\n",
" : \n",
" : \u001b[1m--- Performance optimisation:\u001b[0m\n",
" : \n",
" : Optimal performance for Fisher discriminants is obtained for \n",
" : linearly correlated Gaussian-distributed variables. Any deviation\n",
" : from this ideal reduces the achievable separation power. In \n",
" : particular, no discrimination at all is achieved for a variable\n",
" : that has the same sample mean for signal and background, even if \n",
" : the shapes of the distributions are very different. Thus, Fisher \n",
" : discriminants often benefit from suitable transformations of the \n",
" : input variables. For example, if a variable x in [-1,1] has a \n",
" : a parabolic signal distributions, and a uniform background\n",
" : distributions, their mean value is zero in both cases, leading \n",
" : to no separation. The simple transformation x -> |x| renders this \n",
" : variable powerful for the use in a Fisher discriminant.\n",
" : \n",
" : \u001b[1m--- Performance tuning via configuration options:\u001b[0m\n",
" : \n",
" : \n",
" : \n",
" : \n",
" : \u001b[1m================================================================\u001b[0m\n",
" : \n",
"Fisher : Results for Fisher coefficients:\n",
" : -----------------------\n",
" : Variable: Coefficient:\n",
" : -----------------------\n",
" : m_jj: -0.051\n",
" : m_jjj: +0.187\n",
" : m_lv: +0.037\n",
" : m_jlv: +0.065\n",
" : m_bb: -0.207\n",
" : m_wbb: +0.532\n",
" : m_wwbb: -0.743\n",
" : (offset): +0.125\n",
" : -----------------------\n",
" : Elapsed time for training with 14000 events: 0.00553 sec \n",
"Fisher : [dataset] : Evaluation of Fisher on training sample (14000 events)\n",
"Fisher : [dataset] : Evaluation of Fisher on training sample (14000 events)\n",
" : Elapsed time for evaluation of 14000 events: 0.000775 sec \n",
" : Elapsed time for evaluation of 14000 events: 0.000891 sec \n",
" : Separation from histogram (PDF): 0.085 (0.000)\n",
" : Dataset[dataset] : Evaluation of Fisher on training sample\n",
" : Creating xml weight file: \u001b[0;36mdataset/weights/TMVA_Higgs_Classification_Fisher.weights.xml\u001b[0m\n",
" : Creating standalone class: \u001b[0;36mdataset/weights/TMVA_Higgs_Classification_Fisher.class.C\u001b[0m\n",
"Factory : Training finished\n",
" : \n",
"Factory : Train method: BDT for Classification\n",
" : \n",
"BDT : #events: (reweighted) sig: 7000 bkg: 7000\n",
" : #events: (unweighted) sig: 7000 bkg: 7000\n",
" : Training 200 Decision Trees ... patience please\n",
" : Elapsed time for training with 14000 events: 0.376 sec \n",
"BDT : [dataset] : Evaluation of BDT on training sample (14000 events)\n",
"BDT : [dataset] : Evaluation of BDT on training sample (14000 events)\n",
" : Elapsed time for evaluation of 14000 events: 0.0472 sec \n",
" : Elapsed time for evaluation of 14000 events: 0.0474 sec \n",
" : Creating xml weight file: \u001b[0;36mdataset/weights/TMVA_Higgs_Classification_BDT.weights.xml\u001b[0m\n",
" : Creating standalone class: \u001b[0;36mdataset/weights/TMVA_Higgs_Classification_BDT.class.C\u001b[0m\n",
" : Higgs_ClassificationOutput.root:/dataset/Method_BDT/BDT\n",
"Factory : Training finished\n",
" : \n",
"Factory : Train method: DNN_CPU for Classification\n",
" : \n",
" : Preparing the Gaussian transformation...\n",
"TFHandler_DNN_CPU : Variable Mean RMS [ Min Max ]\n",
" : -----------------------------------------------------------\n",
" : m_jj: 0.0042139 0.99787 [ -3.2801 5.7307 ]\n",
" : m_jjj: 0.0043508 0.99784 [ -3.2805 5.7307 ]\n",
" : m_lv: 0.0051672 1.0008 [ -3.2813 5.7307 ]\n",
" : m_jlv: 0.0044388 0.99830 [ -3.2803 5.7307 ]\n",
" : m_bb: 0.0041864 0.99765 [ -3.2793 5.7307 ]\n",
" : m_wbb: 0.0046426 0.99950 [ -3.2802 5.7307 ]\n",
" : m_wwbb: 0.0044594 0.99873 [ -3.2802 5.7307 ]\n",
" : -----------------------------------------------------------\n",
" : Start of deep neural network training on CPU using MT, nthreads = 1\n",
" : \n",
"TFHandler_DNN_CPU : Variable Mean RMS [ Min Max ]\n",
" : -----------------------------------------------------------\n",
" : m_jj: 0.0042139 0.99787 [ -3.2801 5.7307 ]\n",
" : m_jjj: 0.0043508 0.99784 [ -3.2805 5.7307 ]\n",
" : m_lv: 0.0051672 1.0008 [ -3.2813 5.7307 ]\n",
" : m_jlv: 0.0044388 0.99830 [ -3.2803 5.7307 ]\n",
" : m_bb: 0.0041864 0.99765 [ -3.2793 5.7307 ]\n",
" : m_wbb: 0.0046426 0.99950 [ -3.2802 5.7307 ]\n",
" : m_wwbb: 0.0044594 0.99873 [ -3.2802 5.7307 ]\n",
" : -----------------------------------------------------------\n",
" : ***** Deep Learning Network *****\n",
"DEEP NEURAL NETWORK: Depth = 5 Input = ( 1, 1, 7 ) Batch size = 128 Loss function = C\n",
"\tLayer 0\t DENSE Layer: \t ( Input = 7 , Width = 64 ) \tOutput = ( 1 , 128 , 64 ) \t Activation Function = Tanh\n",
"\tLayer 1\t DENSE Layer: \t ( Input = 64 , Width = 64 ) \tOutput = ( 1 , 128 , 64 ) \t Activation Function = Tanh\n",
"\tLayer 2\t DENSE Layer: \t ( Input = 64 , Width = 64 ) \tOutput = ( 1 , 128 , 64 ) \t Activation Function = Tanh\n",
"\tLayer 3\t DENSE Layer: \t ( Input = 64 , Width = 64 ) \tOutput = ( 1 , 128 , 64 ) \t Activation Function = Tanh\n",
"\tLayer 4\t DENSE Layer: \t ( Input = 64 , Width = 1 ) \tOutput = ( 1 , 128 , 1 ) \t Activation Function = Identity\n",
" : Using 11200 events for training and 2800 for testing\n",
" : Compute initial loss on the validation data \n",
" : Training phase 1 of 1: Optimizer ADAM (beta1=0.9,beta2=0.999,eps=1e-07) Learning rate = 0.001 regularization 0 minimum error = 1.07937\n",
" : --------------------------------------------------------------\n",
" : Epoch | Train Err. Val. Err. t(s)/epoch t(s)/Loss nEvents/s Conv. Steps\n",
" : --------------------------------------------------------------\n",
" : Start epoch iteration ...\n",
" : 1 Minimum Test error found - save the configuration \n",
" : 1 | 0.657783 0.623939 0.276961 0.0211467 43531.6 0\n",
" : 2 Minimum Test error found - save the configuration \n",
" : 2 | 0.607098 0.60001 0.277024 0.0211726 43525.3 0\n",
" : 3 Minimum Test error found - save the configuration \n",
" : 3 | 0.584142 0.58876 0.276944 0.0211574 43536.2 0\n",
" : 4 Minimum Test error found - save the configuration \n",
" : 4 | 0.578295 0.576617 0.278125 0.0212892 43358.4 0\n",
" : 5 | 0.572791 0.577254 0.277789 0.0211118 43385.3 1\n",
" : 6 | 0.569817 0.583492 0.277877 0.0211453 43376.1 2\n",
" : 7 Minimum Test error found - save the configuration \n",
" : 7 | 0.566148 0.572966 0.277628 0.0212719 43439.7 0\n",
" : 8 | 0.562927 0.576096 0.277773 0.0211932 43401.7 1\n",
" : 9 | 0.55942 0.586573 0.278112 0.0212025 43346 2\n",
" : 10 | 0.560123 0.577589 0.278694 0.0213076 43265.7 3\n",
" : 11 | 0.556311 0.574743 0.279673 0.0213087 43101.9 4\n",
" : 12 | 0.553991 0.574257 0.279603 0.0212815 43109.1 5\n",
" : 13 Minimum Test error found - save the configuration \n",
" : 13 | 0.553097 0.572593 0.280035 0.0214929 43072.3 0\n",
" : 14 | 0.549818 0.575582 0.279743 0.0214718 43117.4 1\n",
" : 15 | 0.550372 0.573492 0.28055 0.0213149 42957.2 2\n",
" : 16 | 0.550004 0.574016 0.280566 0.0214129 42970.7 3\n",
" : 17 | 0.546114 0.576607 0.279431 0.0213047 43141.7 4\n",
" : 18 | 0.544893 0.573499 0.279747 0.021351 43096.6 5\n",
" : 19 | 0.543521 0.576985 0.279901 0.0213519 43071.2 6\n",
" : 20 | 0.54291 0.580613 0.279993 0.0213966 43063.3 7\n",
" : \n",
" : Elapsed time for training with 14000 events: 5.63 sec \n",
"DNN_CPU : [dataset] : Evaluation of DNN_CPU on training sample (14000 events)\n",
" : Evaluate deep neural network on CPU using batches with size = 128\n",
" : \n",
"DNN_CPU : [dataset] : Evaluation of DNN_CPU on training sample (14000 events)\n",
" : Elapsed time for evaluation of 14000 events: 0.113 sec \n",
" : Elapsed time for evaluation of 14000 events: 0.114 sec \n",
" : Creating xml weight file: \u001b[0;36mdataset/weights/TMVA_Higgs_Classification_DNN_CPU.weights.xml\u001b[0m\n",
" : Creating standalone class: \u001b[0;36mdataset/weights/TMVA_Higgs_Classification_DNN_CPU.class.C\u001b[0m\n",
"Factory : Training finished\n",
" : \n",
" : Ranking input variables (method specific)...\n",
"Likelihood : Ranking result (top variable is best ranked)\n",
" : -------------------------------------\n",
" : Rank : Variable : Delta Separation\n",
" : -------------------------------------\n",
" : 1 : m_bb : 4.649e-02\n",
" : 2 : m_wbb : 3.677e-02\n",
" : 3 : m_wwbb : 3.283e-02\n",
" : 4 : m_lv : -7.659e-04\n",
" : 5 : m_jj : -3.645e-03\n",
" : 6 : m_jjj : -3.674e-03\n",
" : 7 : m_jlv : -1.422e-02\n",
" : -------------------------------------\n",
"Fisher : Ranking result (top variable is best ranked)\n",
" : ---------------------------------\n",
" : Rank : Variable : Discr. power\n",
" : ---------------------------------\n",
" : 1 : m_bb : 1.180e-02\n",
" : 2 : m_wwbb : 7.816e-03\n",
" : 3 : m_wbb : 2.085e-03\n",
" : 4 : m_jlv : 5.619e-04\n",
" : 5 : m_jjj : 2.327e-04\n",
" : 6 : m_lv : 3.319e-05\n",
" : 7 : m_jj : 1.479e-05\n",
" : ---------------------------------\n",
"BDT : Ranking result (top variable is best ranked)\n",
" : ----------------------------------------\n",
" : Rank : Variable : Variable Importance\n",
" : ----------------------------------------\n",
" : 1 : m_bb : 2.045e-01\n",
" : 2 : m_wwbb : 1.687e-01\n",
" : 3 : m_jlv : 1.638e-01\n",
" : 4 : m_jjj : 1.413e-01\n",
" : 5 : m_wbb : 1.356e-01\n",
" : 6 : m_jj : 1.080e-01\n",
" : 7 : m_lv : 7.813e-02\n",
" : ----------------------------------------\n",
" : No variable ranking supplied by classifier: DNN_CPU\n",
"TH1.Print Name = TrainingHistory_DNN_CPU_trainingError, Entries= 0, Total sum= 11.3096\n",
"TH1.Print Name = TrainingHistory_DNN_CPU_valError, Entries= 0, Total sum= 11.6157\n",
"Factory : === Destroy and recreate all methods via weight files for testing ===\n",
" : \n",
" : Reading weight file: \u001b[0;36mdataset/weights/TMVA_Higgs_Classification_Likelihood.weights.xml\u001b[0m\n",
" : Reading weight file: \u001b[0;36mdataset/weights/TMVA_Higgs_Classification_Fisher.weights.xml\u001b[0m\n",
" : Reading weight file: \u001b[0;36mdataset/weights/TMVA_Higgs_Classification_BDT.weights.xml\u001b[0m\n",
" : Reading weight file: \u001b[0;36mdataset/weights/TMVA_Higgs_Classification_DNN_CPU.weights.xml\u001b[0m\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"0%, time left: unknown\n",
"7%, time left: 0 sec\n",
"13%, time left: 0 sec\n",
"19%, time left: 0 sec\n",
"25%, time left: 0 sec\n",
"32%, time left: 0 sec\n",
"38%, time left: 0 sec\n",
"44%, time left: 0 sec\n",
"50%, time left: 0 sec\n",
"57%, time left: 0 sec\n",
"63%, time left: 0 sec\n",
"69%, time left: 0 sec\n",
"75%, time left: 0 sec\n",
"82%, time left: 0 sec\n",
"88%, time left: 0 sec\n",
"94%, time left: 0 sec\n",
"0%, time left: unknown\n",
"7%, time left: 0 sec\n",
"13%, time left: 0 sec\n",
"19%, time left: 0 sec\n",
"25%, time left: 0 sec\n",
"32%, time left: 0 sec\n",
"38%, time left: 0 sec\n",
"44%, time left: 0 sec\n",
"50%, time left: 0 sec\n",
"57%, time left: 0 sec\n",
"63%, time left: 0 sec\n",
"69%, time left: 0 sec\n",
"75%, time left: 0 sec\n",
"82%, time left: 0 sec\n",
"88%, time left: 0 sec\n",
"94%, time left: 0 sec\n",
"0%, time left: unknown\n",
"7%, time left: 0 sec\n",
"13%, time left: 0 sec\n",
"19%, time left: 0 sec\n",
"25%, time left: 0 sec\n",
"32%, time left: 0 sec\n",
"38%, time left: 0 sec\n",
"44%, time left: 0 sec\n",
"50%, time left: 0 sec\n",
"57%, time left: 0 sec\n",
"63%, time left: 0 sec\n",
"69%, time left: 0 sec\n",
"75%, time left: 0 sec\n",
"82%, time left: 0 sec\n",
"88%, time left: 0 sec\n",
"94%, time left: 0 sec\n",
"0%, time left: unknown\n",
"7%, time left: 0 sec\n",
"13%, time left: 0 sec\n",
"19%, time left: 0 sec\n",
"25%, time left: 0 sec\n",
"32%, time left: 0 sec\n",
"38%, time left: 0 sec\n",
"44%, time left: 0 sec\n",
"50%, time left: 0 sec\n",
"57%, time left: 0 sec\n",
"63%, time left: 0 sec\n",
"69%, time left: 0 sec\n",
"75%, time left: 0 sec\n",
"82%, time left: 0 sec\n",
"88%, time left: 0 sec\n",
"94%, time left: 0 sec\n",
"0%, time left: unknown\n",
"7%, time left: 0 sec\n",
"13%, time left: 0 sec\n",
"19%, time left: 0 sec\n",
"25%, time left: 0 sec\n",
"32%, time left: 0 sec\n",
"38%, time left: 0 sec\n",
"44%, time left: 0 sec\n",
"50%, time left: 0 sec\n",
"57%, time left: 0 sec\n",
"63%, time left: 0 sec\n",
"69%, time left: 0 sec\n",
"75%, time left: 0 sec\n",
"82%, time left: 0 sec\n",
"88%, time left: 0 sec\n",
"94%, time left: 0 sec\n"
]
}
],
"source": [
"factory.TrainAllMethods()"
]
},
{
"cell_type": "markdown",
"id": "ffd8bfe1",
"metadata": {},
"source": [
" Test all methods"
]
},
{
"cell_type": "markdown",
"id": "7d7b1f1d",
"metadata": {},
"source": [
"Now we test and evaluate all methods using the test data set"
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "a0b6c591",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:23:06.108265Z",
"iopub.status.busy": "2026-05-19T20:23:06.108139Z",
"iopub.status.idle": "2026-05-19T20:23:07.749929Z",
"shell.execute_reply": "2026-05-19T20:23:07.749403Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Factory : \u001b[1mTest all methods\u001b[0m\n",
"Factory : Test method: Likelihood for Classification performance\n",
" : \n",
"Likelihood : [dataset] : Evaluation of Likelihood on testing sample (6000 events)\n",
"Likelihood : [dataset] : Evaluation of Likelihood on testing sample (6000 events)\n",
" : Elapsed time for evaluation of 6000 events: 0.00484 sec \n",
" : Elapsed time for evaluation of 6000 events: 0.00495 sec \n",
"Factory : Test method: Fisher for Classification performance\n",
" : \n",
"Fisher : [dataset] : Evaluation of Fisher on testing sample (6000 events)\n",
"Fisher : [dataset] : Evaluation of Fisher on testing sample (6000 events)\n",
" : Elapsed time for evaluation of 6000 events: 0.000421 sec \n",
" : Elapsed time for evaluation of 6000 events: 0.000519 sec \n",
" : Dataset[dataset] : Evaluation of Fisher on testing sample\n",
"Factory : Test method: BDT for Classification performance\n",
" : \n",
"BDT : [dataset] : Evaluation of BDT on testing sample (6000 events)\n",
"BDT : [dataset] : Evaluation of BDT on testing sample (6000 events)\n",
" : Elapsed time for evaluation of 6000 events: 0.0183 sec \n",
" : Elapsed time for evaluation of 6000 events: 0.0184 sec \n",
"Factory : Test method: DNN_CPU for Classification performance\n",
" : \n",
"DNN_CPU : [dataset] : Evaluation of DNN_CPU on testing sample (6000 events)\n",
" : Evaluate deep neural network on CPU using batches with size = 1000\n",
" : \n",
"TFHandler_DNN_CPU : Variable Mean RMS [ Min Max ]\n",
" : -----------------------------------------------------------\n",
" : m_jj: 0.029995 0.98065 [ -3.1064 5.7307 ]\n",
" : m_jjj: 0.030151 0.98464 [ -2.9982 5.7307 ]\n",
" : m_lv: 0.011988 1.0066 [ -3.2274 5.7307 ]\n",
" : m_jlv: 0.0049774 1.0015 [ -3.0644 5.7307 ]\n",
" : m_bb: -0.036143 1.0111 [ -5.7307 5.7307 ]\n",
" : m_wbb: -0.0056377 1.0239 [ -3.0260 5.7307 ]\n",
" : m_wwbb: 0.0023364 1.0091 [ -3.1905 5.7307 ]\n",
" : -----------------------------------------------------------\n",
"DNN_CPU : [dataset] : Evaluation of DNN_CPU on testing sample (6000 events)\n",
" : Elapsed time for evaluation of 6000 events: 0.047 sec \n",
" : Elapsed time for evaluation of 6000 events: 0.0576 sec \n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"0%, time left: unknown\n",
"7%, time left: 0 sec\n",
"13%, time left: 0 sec\n",
"19%, time left: 0 sec\n",
"25%, time left: 0 sec\n",
"32%, time left: 0 sec\n",
"38%, time left: 0 sec\n",
"44%, time left: 0 sec\n",
"50%, time left: 0 sec\n",
"57%, time left: 0 sec\n",
"63%, time left: 0 sec\n",
"69%, time left: 0 sec\n",
"75%, time left: 0 sec\n",
"82%, time left: 0 sec\n",
"88%, time left: 0 sec\n",
"94%, time left: 0 sec\n",
"0%, time left: unknown\n",
"7%, time left: 0 sec\n",
"13%, time left: 0 sec\n",
"19%, time left: 0 sec\n",
"25%, time left: 0 sec\n",
"32%, time left: 0 sec\n",
"38%, time left: 0 sec\n",
"44%, time left: 0 sec\n",
"50%, time left: 0 sec\n",
"57%, time left: 0 sec\n",
"63%, time left: 0 sec\n",
"69%, time left: 0 sec\n",
"75%, time left: 0 sec\n",
"82%, time left: 0 sec\n",
"88%, time left: 0 sec\n",
"94%, time left: 0 sec\n",
"0%, time left: unknown\n",
"7%, time left: 0 sec\n",
"13%, time left: 0 sec\n",
"19%, time left: 0 sec\n",
"25%, time left: 0 sec\n",
"32%, time left: 0 sec\n",
"38%, time left: 0 sec\n",
"44%, time left: 0 sec\n",
"50%, time left: 0 sec\n",
"57%, time left: 0 sec\n",
"63%, time left: 0 sec\n",
"69%, time left: 0 sec\n",
"75%, time left: 0 sec\n",
"82%, time left: 0 sec\n",
"88%, time left: 0 sec\n",
"94%, time left: 0 sec\n",
"0%, time left: unknown\n",
"7%, time left: 0 sec\n",
"13%, time left: 0 sec\n",
"19%, time left: 0 sec\n",
"25%, time left: 0 sec\n",
"32%, time left: 0 sec\n",
"38%, time left: 0 sec\n",
"44%, time left: 0 sec\n",
"50%, time left: 0 sec\n",
"57%, time left: 0 sec\n",
"63%, time left: 0 sec\n",
"69%, time left: 0 sec\n",
"75%, time left: 0 sec\n",
"82%, time left: 0 sec\n",
"88%, time left: 0 sec\n",
"94%, time left: 0 sec\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Factory : \u001b[1mEvaluate all methods\u001b[0m\n",
"Factory : Evaluate classifier: Likelihood\n",
" : \n",
"Likelihood : [dataset] : Loop over test events and fill histograms with classifier response...\n",
" : \n",
" : maximum iterations (100) reached before convergence\n",
" : maximum iterations (100) reached before convergence\n",
"TFHandler_Likelihood : Variable Mean RMS [ Min Max ]\n",
" : -----------------------------------------------------------\n",
" : m_jj: 1.0368 0.66752 [ 0.16310 16.132 ]\n",
" : m_jjj: 1.0272 0.38070 [ 0.41899 8.9401 ]\n",
" : m_lv: 1.0522 0.17017 [ 0.29757 3.2605 ]\n",
" : m_jlv: 1.0135 0.40315 [ 0.41660 5.8195 ]\n",
" : m_bb: 0.96616 0.53867 [ 0.080986 8.2551 ]\n",
" : m_wbb: 1.0344 0.37776 [ 0.42068 6.4013 ]\n",
" : m_wwbb: 0.96122 0.31782 [ 0.44118 4.5350 ]\n",
" : -----------------------------------------------------------\n",
"Factory : Evaluate classifier: Fisher\n",
" : \n",
"Fisher : [dataset] : Loop over test events and fill histograms with classifier response...\n",
" : \n",
" : Also filling probability and rarity histograms (on request)...\n",
"TFHandler_Fisher : Variable Mean RMS [ Min Max ]\n",
" : -----------------------------------------------------------\n",
" : m_jj: 1.0368 0.66752 [ 0.16310 16.132 ]\n",
" : m_jjj: 1.0272 0.38070 [ 0.41899 8.9401 ]\n",
" : m_lv: 1.0522 0.17017 [ 0.29757 3.2605 ]\n",
" : m_jlv: 1.0135 0.40315 [ 0.41660 5.8195 ]\n",
" : m_bb: 0.96616 0.53867 [ 0.080986 8.2551 ]\n",
" : m_wbb: 1.0344 0.37776 [ 0.42068 6.4013 ]\n",
" : m_wwbb: 0.96122 0.31782 [ 0.44118 4.5350 ]\n",
" : -----------------------------------------------------------\n",
"Factory : Evaluate classifier: BDT\n",
" : \n",
"BDT : [dataset] : Loop over test events and fill histograms with classifier response...\n",
" : \n",
"TFHandler_BDT : Variable Mean RMS [ Min Max ]\n",
" : -----------------------------------------------------------\n",
" : m_jj: 1.0368 0.66752 [ 0.16310 16.132 ]\n",
" : m_jjj: 1.0272 0.38070 [ 0.41899 8.9401 ]\n",
" : m_lv: 1.0522 0.17017 [ 0.29757 3.2605 ]\n",
" : m_jlv: 1.0135 0.40315 [ 0.41660 5.8195 ]\n",
" : m_bb: 0.96616 0.53867 [ 0.080986 8.2551 ]\n",
" : m_wbb: 1.0344 0.37776 [ 0.42068 6.4013 ]\n",
" : m_wwbb: 0.96122 0.31782 [ 0.44118 4.5350 ]\n",
" : -----------------------------------------------------------\n",
"Factory : Evaluate classifier: DNN_CPU\n",
" : \n",
"DNN_CPU : [dataset] : Loop over test events and fill histograms with classifier response...\n",
" : \n",
" : Evaluate deep neural network on CPU using batches with size = 1000\n",
" : \n",
"TFHandler_DNN_CPU : Variable Mean RMS [ Min Max ]\n",
" : -----------------------------------------------------------\n",
" : m_jj: 0.0042139 0.99787 [ -3.2801 5.7307 ]\n",
" : m_jjj: 0.0043508 0.99784 [ -3.2805 5.7307 ]\n",
" : m_lv: 0.0051673 1.0008 [ -3.2813 5.7307 ]\n",
" : m_jlv: 0.0044388 0.99830 [ -3.2803 5.7307 ]\n",
" : m_bb: 0.0041864 0.99765 [ -3.2793 5.7307 ]\n",
" : m_wbb: 0.0046426 0.99950 [ -3.2802 5.7307 ]\n",
" : m_wwbb: 0.0044594 0.99873 [ -3.2802 5.7307 ]\n",
" : -----------------------------------------------------------\n",
"TFHandler_DNN_CPU : Variable Mean RMS [ Min Max ]\n",
" : -----------------------------------------------------------\n",
" : m_jj: 0.029995 0.98065 [ -3.1064 5.7307 ]\n",
" : m_jjj: 0.030151 0.98464 [ -2.9982 5.7307 ]\n",
" : m_lv: 0.011988 1.0066 [ -3.2274 5.7307 ]\n",
" : m_jlv: 0.0049774 1.0015 [ -3.0644 5.7307 ]\n",
" : m_bb: -0.036143 1.0111 [ -5.7307 5.7307 ]\n",
" : m_wbb: -0.0056377 1.0239 [ -3.0260 5.7307 ]\n",
" : m_wwbb: 0.0023364 1.0091 [ -3.1905 5.7307 ]\n",
" : -----------------------------------------------------------\n",
" : \n",
" : Evaluation results ranked by best signal efficiency and purity (area)\n",
" : -------------------------------------------------------------------------------------------------------------------\n",
" : DataSet MVA \n",
" : Name: Method: ROC-integ\n",
" : dataset DNN_CPU : 0.769\n",
" : dataset BDT : 0.758\n",
" : dataset Likelihood : 0.700\n",
" : dataset Fisher : 0.654\n",
" : -------------------------------------------------------------------------------------------------------------------\n",
" : \n",
" : Testing efficiency compared to training efficiency (overtraining check)\n",
" : -------------------------------------------------------------------------------------------------------------------\n",
" : DataSet MVA Signal efficiency: from test sample (from training sample) \n",
" : Name: Method: @B=0.01 @B=0.10 @B=0.30 \n",
" : -------------------------------------------------------------------------------------------------------------------\n",
" : dataset DNN_CPU : 0.122 (0.149) 0.403 (0.434) 0.684 (0.705)\n",
" : dataset BDT : 0.080 (0.095) 0.394 (0.394) 0.674 (0.685)\n",
" : dataset Likelihood : 0.066 (0.080) 0.313 (0.334) 0.585 (0.593)\n",
" : dataset Fisher : 0.017 (0.014) 0.128 (0.141) 0.500 (0.529)\n",
" : -------------------------------------------------------------------------------------------------------------------\n",
" : \n",
"Dataset:dataset : Created tree 'TestTree' with 6000 events\n",
" : \n",
"Dataset:dataset : Created tree 'TrainTree' with 14000 events\n",
" : \n",
"Factory : \u001b[1mThank you for using TMVA!\u001b[0m\n",
" : \u001b[1mFor citation information, please visit: http://tmva.sf.net/citeTMVA.html\u001b[0m\n"
]
}
],
"source": [
"factory.TestAllMethods()\n",
"\n",
"factory.EvaluateAllMethods()"
]
},
{
"cell_type": "markdown",
"id": "4d1e87d5",
"metadata": {},
"source": [
"after we get the ROC curve and we display"
]
},
{
"cell_type": "code",
"execution_count": 19,
"id": "cf574054",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:23:07.751370Z",
"iopub.status.busy": "2026-05-19T20:23:07.751246Z",
"iopub.status.idle": "2026-05-19T20:23:07.974303Z",
"shell.execute_reply": "2026-05-19T20:23:07.973790Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
"
\n",
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"c1 = factory.GetROCCurve(loader)\n",
"c1.Draw()"
]
},
{
"cell_type": "markdown",
"id": "d9c1fec5",
"metadata": {},
"source": [
"at the end we close the output file which contains the evaluation result of all methods and it can be used by TMVAGUI\n",
"to display additional plots"
]
},
{
"cell_type": "code",
"execution_count": 20,
"id": "e703f4dc",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:23:07.975803Z",
"iopub.status.busy": "2026-05-19T20:23:07.975679Z",
"iopub.status.idle": "2026-05-19T20:23:08.090052Z",
"shell.execute_reply": "2026-05-19T20:23:08.089532Z"
}
},
"outputs": [],
"source": [
"outputFile.Close()"
]
},
{
"cell_type": "markdown",
"id": "5fbb030d",
"metadata": {},
"source": [
"Draw all canvases "
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "d5d6dbb9",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2026-05-19T20:23:08.091718Z",
"iopub.status.busy": "2026-05-19T20:23:08.091576Z",
"iopub.status.idle": "2026-05-19T20:23:08.213644Z",
"shell.execute_reply": "2026-05-19T20:23:08.213146Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
"
\n",
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from ROOT import gROOT \n",
"gROOT.GetListOfCanvases().Draw()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.12"
}
},
"nbformat": 4,
"nbformat_minor": 5
}