==> Start TMVAMulticlass --- TMVAMulticlass: Using input file: /github/home/ROOT-CI/build/tutorials/machine_learning/data/tmva_multiclass_example.root DataSetInfo : [dataset] : Added class "Signal" : Add Tree TreeS of type Signal with 2000 events DataSetInfo : [dataset] : Added class "bg0" : Add Tree TreeB0 of type bg0 with 2000 events DataSetInfo : [dataset] : Added class "bg1" : Add Tree TreeB1 of type bg1 with 2000 events DataSetInfo : [dataset] : Added class "bg2" : Add Tree TreeB2 of type bg2 with 2000 events : Dataset[dataset] : Class index : 0 name : Signal : Dataset[dataset] : Class index : 1 name : bg0 : Dataset[dataset] : Class index : 2 name : bg1 : Dataset[dataset] : Class index : 3 name : bg2 Factory : Booking method: BDTG : : the option NegWeightTreatment=InverseBoostNegWeights does not exist for BoostType=Grad : --> change to new default NegWeightTreatment=Pray : Rebuilding Dataset dataset : Building event vectors for type 2 Signal : Dataset[dataset] : create input formulas for tree TreeS : Building event vectors for type 2 bg0 : Dataset[dataset] : create input formulas for tree TreeB0 : Building event vectors for type 2 bg1 : Dataset[dataset] : create input formulas for tree TreeB1 : Building event vectors for type 2 bg2 : Dataset[dataset] : create input formulas for tree TreeB2 DataSetFactory : [dataset] : Number of events in input trees : : : : : Number of training and testing events : --------------------------------------------------------------------------- : Signal -- training events : 1000 : Signal -- testing events : 1000 : Signal -- training and testing events: 2000 : bg0 -- training events : 1000 : bg0 -- testing events : 1000 : bg0 -- training and testing events: 2000 : bg1 -- training events : 1000 : bg1 -- testing events : 1000 : bg1 -- training and testing events: 2000 : bg2 -- training events : 1000 : bg2 -- testing events : 1000 : bg2 -- training and testing events: 2000 : DataSetInfo : Correlation matrix (Signal): : ---------------------------------------- : var1 var2 var3 var4 : var1: +1.000 +0.385 +0.621 +0.838 : var2: +0.385 +1.000 +0.698 +0.723 : var3: +0.621 +0.698 +1.000 +0.849 : var4: +0.838 +0.723 +0.849 +1.000 : ---------------------------------------- DataSetInfo : Correlation matrix (bg0): : ---------------------------------------- : var1 var2 var3 var4 : var1: +1.000 +0.413 +0.612 +0.833 : var2: +0.413 +1.000 +0.728 +0.753 : var3: +0.612 +0.728 +1.000 +0.855 : var4: +0.833 +0.753 +0.855 +1.000 : ---------------------------------------- DataSetInfo : Correlation matrix (bg1): : ---------------------------------------- : var1 var2 var3 var4 : var1: +1.000 +0.423 +0.619 +0.846 : var2: +0.423 +1.000 +0.705 +0.730 : var3: +0.619 +0.705 +1.000 +0.855 : var4: +0.846 +0.730 +0.855 +1.000 : ---------------------------------------- DataSetInfo : Correlation matrix (bg2): : ---------------------------------------- : var1 var2 var3 var4 : var1: +1.000 -0.658 +0.032 -0.004 : var2: -0.658 +1.000 -0.000 +0.014 : var3: +0.032 -0.000 +1.000 -0.048 : var4: -0.004 +0.014 -0.048 +1.000 : ---------------------------------------- DataSetFactory : [dataset] : : Factory : Booking method: MLP : MLP : Building Network. : Initializing weights Factory : Booking method: PDEFoam : Factory : Booking method: DL_CPU : : Parsing option string: : ... "!H:V:ErrorStrategy=CROSSENTROPY:VarTransform=N:WeightInitialization=XAVIERUNIFORM:Architecture=GPU:Layout=TANH|100,TANH|50,TANH|10,LINEAR:TrainingStrategy=Optimizer=ADAM,LearningRate=1e-3,TestRepetitions=1,ConvergenceSteps=10,BatchSize=100,MaxEpochs=20" : The following options are set: : - By User: : : - Default: : Boost_num: "0" [Number of times the classifier will be boosted] : Parsing option string: : ... "!H:V:ErrorStrategy=CROSSENTROPY:VarTransform=N:WeightInitialization=XAVIERUNIFORM:Architecture=GPU:Layout=TANH|100,TANH|50,TANH|10,LINEAR:TrainingStrategy=Optimizer=ADAM,LearningRate=1e-3,TestRepetitions=1,ConvergenceSteps=10,BatchSize=100,MaxEpochs=20" : The following options are set: : - By User: : V: "True" [Verbose output (short form of "VerbosityLevel" below - overrides the latter one)] : VarTransform: "N" [List of variable transformations performed before training, e.g., "D_Background,P_Signal,G,N_AllClasses" for: "Decorrelation, PCA-transformation, Gaussianisation, Normalisation, each for the given class of events ('AllClasses' denotes all events of all classes, if no class indication is given, 'All' is assumed)"] : H: "False" [Print method-specific help message] : Layout: "TANH|100,TANH|50,TANH|10,LINEAR" [Layout of the network.] : ErrorStrategy: "CROSSENTROPY" [Loss function: Mean squared error (regression) or cross entropy (binary classification).] : WeightInitialization: "XAVIERUNIFORM" [Weight initialization strategy] : Architecture: "GPU" [Which architecture to perform the training on.] : TrainingStrategy: "Optimizer=ADAM,LearningRate=1e-3,TestRepetitions=1,ConvergenceSteps=10,BatchSize=100,MaxEpochs=20" [Defines the training strategies.] : - Default: : VerbosityLevel: "Default" [Verbosity level] : CreateMVAPdfs: "False" [Create PDFs for classifier outputs (signal and background)] : IgnoreNegWeightsInTraining: "False" [Events with negative weights are ignored in the training (but are included for testing and performance evaluation)] : InputLayout: "0|0|0" [The Layout of the input] : BatchLayout: "0|0|0" [The Layout of the batch] : RandomSeed: "0" [Random seed used for weight initialization and batch shuffling] : ValidationSize: "20%" [Part of the training data to use for validation. Specify as 0.2 or 20% to use a fifth of the data set as validation set. Specify as 100 to use exactly 100 events. (Default: 20%)] DL_CPU : [dataset] : Create Transformation "N" with events from all classes. : : Transformation, Variable selection : : Input : variable 'var1' <---> Output : variable 'var1' : Input : variable 'var2' <---> Output : variable 'var2' : Input : variable 'var3' <---> Output : variable 'var3' : Input : variable 'var4' <---> Output : variable 'var4'  : CUDA backend not enabled. Please make sure you have CUDA installed and it was successfully detected by CMAKE by using -Dtmva-gpu=On  : Will now use instead the CPU architecture ! : Will now use the CPU architecture with BLAS and IMT support ! Factory : Train all methods Factory : [dataset] : Create Transformation "I" with events from all classes. : : Transformation, Variable selection : : Input : variable 'var1' <---> Output : variable 'var1' : Input : variable 'var2' <---> Output : variable 'var2' : Input : variable 'var3' <---> Output : variable 'var3' : Input : variable 'var4' <---> Output : variable 'var4' Factory : [dataset] : Create Transformation "D" with events from all classes. : : Transformation, Variable selection : : Input : variable 'var1' <---> Output : variable 'var1' : Input : variable 'var2' <---> Output : variable 'var2' : Input : variable 'var3' <---> Output : variable 'var3' : Input : variable 'var4' <---> Output : variable 'var4' Factory : [dataset] : Create Transformation "P" with events from all classes. : : Transformation, Variable selection : : Input : variable 'var1' <---> Output : variable 'var1' : Input : variable 'var2' <---> Output : variable 'var2' : Input : variable 'var3' <---> Output : variable 'var3' : Input : variable 'var4' <---> Output : variable 'var4' Factory : [dataset] : Create Transformation "G" with events from all classes. : : Transformation, Variable selection : : Input : variable 'var1' <---> Output : variable 'var1' : Input : variable 'var2' <---> Output : variable 'var2' : Input : variable 'var3' <---> Output : variable 'var3' : Input : variable 'var4' <---> Output : variable 'var4' Factory : [dataset] : Create Transformation "D" with events from all classes. : : Transformation, Variable selection : : Input : variable 'var1' <---> Output : variable 'var1' : Input : variable 'var2' <---> Output : variable 'var2' : Input : variable 'var3' <---> Output : variable 'var3' : Input : variable 'var4' <---> Output : variable 'var4' TFHandler_Factory : Variable Mean RMS [ Min Max ] : ----------------------------------------------------------- : var1: 0.052185 1.0190 [ -4.0592 3.2645 ] : var2: 0.33312 1.0446 [ -3.6891 3.7877 ] : var3: 0.10463 1.1205 [ -3.6296 3.9200 ] : var4: -0.078123 1.2764 [ -4.8486 4.3625 ] : ----------------------------------------------------------- : Preparing the Decorrelation transformation... TFHandler_Factory : Variable Mean RMS [ Min Max ] : ----------------------------------------------------------- : var1: 0.089502 1.0000 [ -3.4349 2.7570 ] : var2: 0.38543 1.0000 [ -3.3765 3.1055 ] : var3: 0.052636 1.0000 [ -2.8007 3.1004 ] : var4: -0.20867 1.0000 [ -3.0012 2.5822 ] : ----------------------------------------------------------- : Preparing the Principle Component (PCA) transformation... TFHandler_Factory : Variable Mean RMS [ Min Max ] : ----------------------------------------------------------- : var1:-2.3297e-10 1.8127 [ -7.2691 6.3617 ] : var2:-3.1381e-10 0.89464 [ -2.7283 2.6323 ] : var3:-2.2463e-10 0.73955 [ -2.6363 2.4256 ] : var4:-9.8869e-11 0.61727 [ -1.7822 2.2327 ] : ----------------------------------------------------------- : Preparing the Gaussian transformation... : Preparing the Decorrelation transformation... TFHandler_Factory : Variable Mean RMS [ Min Max ] : ----------------------------------------------------------- : var1: 0.0071986 1.0000 [ -2.5427 5.8540 ] : var2: 0.0087421 1.0000 [ -2.8611 4.9796 ] : var3: 0.0090897 1.0000 [ -2.9572 5.6365 ] : var4: 0.0084612 1.0000 [ -3.0233 5.7479 ] : ----------------------------------------------------------- : Ranking input variables (method unspecific)... Factory : Train method: BDTG for Multiclass classification : : Training 1000 Decision Trees ... patience please : Elapsed time for training with 4000 events: 2.81 sec : Dataset[dataset] : Create results for training : Dataset[dataset] : Multiclass evaluation of BDTG on training sample : Dataset[dataset] : Elapsed time for evaluation of 4000 events: 0.633 sec : Creating multiclass response histograms... : Creating multiclass performance histograms... : Creating xml weight file: dataset/weights/TMVAMulticlass_BDTG.weights.xml : Creating standalone class: dataset/weights/TMVAMulticlass_BDTG.class.C : TMVAMulticlass.root:/dataset/Method_BDT/BDTG Factory : Training finished : Factory : Train method: MLP for Multiclass classification : : Training Network : : Elapsed time for training with 4000 events: 11.3 sec : Dataset[dataset] : Create results for training : Dataset[dataset] : Multiclass evaluation of MLP on training sample : Dataset[dataset] : Elapsed time for evaluation of 4000 events: 0.00451 sec : Creating multiclass response histograms... : Creating multiclass performance histograms... : Creating xml weight file: dataset/weights/TMVAMulticlass_MLP.weights.xml : Creating standalone class: dataset/weights/TMVAMulticlass_MLP.class.C : Write special histos to file: TMVAMulticlass.root:/dataset/Method_MLP/MLP Factory : Training finished : Factory : Train method: PDEFoam for Multiclass classification : : Build up multiclass foam 0 : Elapsed time: 0.344 sec : Build up multiclass foam 1 : Elapsed time: 0.329 sec : Build up multiclass foam 2 : Elapsed time: 0.328 sec : Build up multiclass foam 3 : Elapsed time: 0.209 sec : Elapsed time for training with 4000 events: 1.29 sec : Dataset[dataset] : Create results for training : Dataset[dataset] : Multiclass evaluation of PDEFoam on training sample : Dataset[dataset] : Elapsed time for evaluation of 4000 events: 0.0558 sec : Creating multiclass response histograms... : Creating multiclass performance histograms... : Creating xml weight file: dataset/weights/TMVAMulticlass_PDEFoam.weights.xml : writing foam MultiClassFoam0 to file : writing foam MultiClassFoam1 to file : writing foam MultiClassFoam2 to file : writing foam MultiClassFoam3 to file : Foams written to file: dataset/weights/TMVAMulticlass_PDEFoam.weights_foams.root : Creating standalone class: dataset/weights/TMVAMulticlass_PDEFoam.class.C Factory : Training finished : Factory : Train method: DL_CPU for Multiclass classification : TFHandler_DL_CPU : Variable Mean RMS [ Min Max ] : ----------------------------------------------------------- : var1: 0.12276 0.27828 [ -1.0000 1.0000 ] : var2: 0.075909 0.27943 [ -1.0000 1.0000 ] : var3: -0.010745 0.29684 [ -1.0000 1.0000 ] : var4: 0.035804 0.27714 [ -1.0000 1.0000 ] : ----------------------------------------------------------- : Start of deep neural network training on CPU using MT, nthreads = 1 : TFHandler_DL_CPU : Variable Mean RMS [ Min Max ] : ----------------------------------------------------------- : var1: 0.12276 0.27828 [ -1.0000 1.0000 ] : var2: 0.075909 0.27943 [ -1.0000 1.0000 ] : var3: -0.010745 0.29684 [ -1.0000 1.0000 ] : var4: 0.035804 0.27714 [ -1.0000 1.0000 ] : ----------------------------------------------------------- : ***** Deep Learning Network ***** DEEP NEURAL NETWORK: Depth = 4 Input = ( 1, 1, 4 ) Batch size = 100 Loss function = C Layer 0 DENSE Layer: ( Input = 4 , Width = 100 ) Output = ( 1 , 100 , 100 ) Activation Function = Tanh Layer 1 DENSE Layer: ( Input = 100 , Width = 50 ) Output = ( 1 , 100 , 50 ) Activation Function = Tanh Layer 2 DENSE Layer: ( Input = 50 , Width = 10 ) Output = ( 1 , 100 , 10 ) Activation Function = Tanh Layer 3 DENSE Layer: ( Input = 10 , Width = 4 ) Output = ( 1 , 100 , 4 ) Activation Function = Identity : Using 3200 events for training and 800 for testing : Compute initial loss on the validation data : Training phase 1 of 1: Optimizer ADAM (beta1=0.9,beta2=0.999,eps=1e-07) Learning rate = 0.001 regularization 0 minimum error = 0.710211 : -------------------------------------------------------------- : Epoch | Train Err. Val. Err. t(s)/epoch t(s)/Loss nEvents/s Conv. Steps : -------------------------------------------------------------- : Start epoch iteration ... : 1 Minimum Test error found - save the configuration : 1 | 0.607445 0.534943 0.0333528 0.00270563 104414 0 : 2 Minimum Test error found - save the configuration : 2 | 0.511023 0.492358 0.0338992 0.0027481 102725 0 : 3 Minimum Test error found - save the configuration : 3 | 0.476035 0.463099 0.0343908 0.00279736 101287 0 : 4 Minimum Test error found - save the configuration : 4 | 0.447034 0.437636 0.034649 0.0028411 100604 0 : 5 Minimum Test error found - save the configuration : 5 | 0.423442 0.418354 0.0349074 0.00286325 99862.3 0 : 6 Minimum Test error found - save the configuration : 6 | 0.40537 0.403925 0.0350433 0.00288971 99522.3 0 : 7 Minimum Test error found - save the configuration : 7 | 0.391463 0.392031 0.035167 0.00290244 99180.2 0 : 8 Minimum Test error found - save the configuration : 8 | 0.379038 0.380902 0.035257 0.00289209 98872.5 0 : 9 Minimum Test error found - save the configuration : 9 | 0.368608 0.370453 0.0355438 0.00291611 98076.3 0 : 10 Minimum Test error found - save the configuration : 10 | 0.359225 0.361299 0.0356062 0.00290241 97848 0 : 11 Minimum Test error found - save the configuration : 11 | 0.351391 0.354149 0.0356157 0.0029071 97833.6 0 : 12 Minimum Test error found - save the configuration : 12 | 0.342495 0.345616 0.0357834 0.00294672 97452.1 0 : 13 Minimum Test error found - save the configuration : 13 | 0.335143 0.338739 0.0357703 0.00294124 97474.5 0 : 14 Minimum Test error found - save the configuration : 14 | 0.328215 0.331769 0.0360133 0.00296195 96819 0 : 15 Minimum Test error found - save the configuration : 15 | 0.321904 0.324195 0.0360492 0.00296667 96727.7 0 : 16 Minimum Test error found - save the configuration : 16 | 0.315592 0.319982 0.0360826 0.0029745 96653 0 : 17 Minimum Test error found - save the configuration : 17 | 0.310906 0.312788 0.0361925 0.00298899 96375.4 0 : 18 Minimum Test error found - save the configuration : 18 | 0.304855 0.310077 0.0363615 0.00304418 96046.1 0 : 19 Minimum Test error found - save the configuration : 19 | 0.300983 0.304615 0.0362679 0.00299896 96185.7 0 : 20 Minimum Test error found - save the configuration : 20 | 0.296955 0.301251 0.036272 0.00299772 96170.4 0 : : Elapsed time for training with 4000 events: 0.722 sec : Dataset[dataset] : Create results for training : Dataset[dataset] : Multiclass evaluation of DL_CPU on training sample : Dataset[dataset] : Elapsed time for evaluation of 4000 events: 0.0977 sec : Creating multiclass response histograms... : Creating multiclass performance histograms... : Creating xml weight file: dataset/weights/TMVAMulticlass_DL_CPU.weights.xml : Creating standalone class: dataset/weights/TMVAMulticlass_DL_CPU.class.C Factory : Training finished : : Ranking input variables (method specific)... BDTG : Ranking result (top variable is best ranked) : -------------------------------------- : Rank : Variable : Variable Importance : -------------------------------------- : 1 : var4 : 3.060e-01 : 2 : var1 : 2.473e-01 : 3 : var2 : 2.400e-01 : 4 : var3 : 2.067e-01 : -------------------------------------- MLP : Ranking result (top variable is best ranked) : ----------------------------- : Rank : Variable : Importance : ----------------------------- : 1 : var4 : 5.440e+01 : 2 : var1 : 2.568e+01 : 3 : var2 : 2.223e+01 : 4 : var3 : 7.204e+00 : ----------------------------- PDEFoam : Ranking result (top variable is best ranked) : -------------------------------------- : Rank : Variable : Variable Importance : -------------------------------------- : 1 : var4 : 2.756e-01 : 2 : var1 : 2.691e-01 : 3 : var2 : 2.402e-01 : 4 : var3 : 2.151e-01 : -------------------------------------- : No variable ranking supplied by classifier: DL_CPU TH1.Print Name = TrainingHistory_DL_CPU_trainingError, Entries= 0, Total sum= 7.57712 TH1.Print Name = TrainingHistory_DL_CPU_valError, Entries= 0, Total sum= 7.49818 Factory : === Destroy and recreate all methods via weight files for testing === : : Reading weight file: dataset/weights/TMVAMulticlass_BDTG.weights.xml : Reading weight file: dataset/weights/TMVAMulticlass_MLP.weights.xml MLP : Building Network. : Initializing weights : Reading weight file: dataset/weights/TMVAMulticlass_PDEFoam.weights.xml : Read foams from file: dataset/weights/TMVAMulticlass_PDEFoam.weights_foams.root : Reading weight file: dataset/weights/TMVAMulticlass_DL_CPU.weights.xml Factory : Test all methods Factory : Test method: BDTG for Multiclass classification performance : : Dataset[dataset] : Create results for testing : Dataset[dataset] : Multiclass evaluation of BDTG on testing sample : Dataset[dataset] : Elapsed time for evaluation of 4000 events: 0.43 sec : Creating multiclass response histograms... : Creating multiclass performance histograms... Factory : Test method: MLP for Multiclass classification performance : : Dataset[dataset] : Create results for testing : Dataset[dataset] : Multiclass evaluation of MLP on testing sample : Dataset[dataset] : Elapsed time for evaluation of 4000 events: 0.0047 sec : Creating multiclass response histograms... : Creating multiclass performance histograms... Factory : Test method: PDEFoam for Multiclass classification performance : : Dataset[dataset] : Create results for testing : Dataset[dataset] : Multiclass evaluation of PDEFoam on testing sample : Dataset[dataset] : Elapsed time for evaluation of 4000 events: 0.0469 sec : Creating multiclass response histograms... : Creating multiclass performance histograms... Factory : Test method: DL_CPU for Multiclass classification performance : : Dataset[dataset] : Create results for testing : Dataset[dataset] : Multiclass evaluation of DL_CPU on testing sample : Dataset[dataset] : Elapsed time for evaluation of 4000 events: 0.102 sec : Creating multiclass response histograms... : Creating multiclass performance histograms... Factory : Evaluate all methods : Evaluate multiclass classification method: BDTG : Creating multiclass response histograms... : Creating multiclass performance histograms... : Creating multiclass response histograms... : Creating multiclass performance histograms... TFHandler_BDTG : Variable Mean RMS [ Min Max ] : ----------------------------------------------------------- : var1: 0.065615 1.0061 [ -4.0592 3.5808 ] : var2: 0.29707 1.0658 [ -3.6952 3.7877 ] : var3: 0.13183 1.1245 [ -4.5727 4.5640 ] : var4: -0.071010 1.2654 [ -4.8486 5.0412 ] : ----------------------------------------------------------- : Evaluate multiclass classification method: MLP : Creating multiclass response histograms... : Creating multiclass performance histograms... : Creating multiclass response histograms... : Creating multiclass performance histograms... TFHandler_MLP : Variable Mean RMS [ Min Max ] : ----------------------------------------------------------- : var1: 0.065615 1.0061 [ -4.0592 3.5808 ] : var2: 0.29707 1.0658 [ -3.6952 3.7877 ] : var3: 0.13183 1.1245 [ -4.5727 4.5640 ] : var4: -0.071010 1.2654 [ -4.8486 5.0412 ] : ----------------------------------------------------------- : Evaluate multiclass classification method: PDEFoam : Creating multiclass response histograms... : Creating multiclass performance histograms... : Creating multiclass response histograms... : Creating multiclass performance histograms... TFHandler_PDEFoam : Variable Mean RMS [ Min Max ] : ----------------------------------------------------------- : var1: 0.065615 1.0061 [ -4.0592 3.5808 ] : var2: 0.29707 1.0658 [ -3.6952 3.7877 ] : var3: 0.13183 1.1245 [ -4.5727 4.5640 ] : var4: -0.071010 1.2654 [ -4.8486 5.0412 ] : ----------------------------------------------------------- : Evaluate multiclass classification method: DL_CPU : Creating multiclass response histograms... : Creating multiclass performance histograms... : Creating multiclass response histograms... : Creating multiclass performance histograms... TFHandler_DL_CPU : Variable Mean RMS [ Min Max ] : ----------------------------------------------------------- : var1: 0.12643 0.27476 [ -1.0000 1.0864 ] : var2: 0.066267 0.28510 [ -1.0016 1.0000 ] : var3: -0.0035395 0.29791 [ -1.2498 1.1706 ] : var4: 0.037349 0.27475 [ -1.0000 1.1474 ] : ----------------------------------------------------------- TFHandler_DL_CPU : Variable Mean RMS [ Min Max ] : ----------------------------------------------------------- : var1: 0.12643 0.27476 [ -1.0000 1.0864 ] : var2: 0.066267 0.28510 [ -1.0016 1.0000 ] : var3: -0.0035395 0.29791 [ -1.2498 1.1706 ] : var4: 0.037349 0.27475 [ -1.0000 1.1474 ] : ----------------------------------------------------------- : : 1-vs-rest performance metrics per class : ------------------------------------------------------------------------------------------------------- : : Considers the listed class as signal and the other classes : as background, reporting the resulting binary performance. : A score of 0.820 (0.850) means 0.820 was acheived on the : test set and 0.850 on the training set. : : Dataset MVA Method ROC AUC Sig eff@B=0.01 Sig eff@B=0.10 Sig eff@B=0.30 : Name: / Class: test (train) test (train) test (train) test (train) : : dataset BDTG : ------------------------------ : Signal 0.967 (0.980) 0.496 (0.616) 0.910 (0.953) 0.994 (0.997) : bg0 0.908 (0.927) 0.201 (0.331) 0.729 (0.777) 0.924 (0.944) : bg1 0.945 (0.955) 0.413 (0.429) 0.833 (0.860) 0.970 (0.973) : bg2 0.974 (0.984) 0.600 (0.677) 0.926 (0.973) 0.995 (0.998) : : dataset MLP : ------------------------------ : Signal 0.975 (0.976) 0.591 (0.609) 0.931 (0.940) 0.997 (0.994) : bg0 0.930 (0.934) 0.279 (0.389) 0.781 (0.789) 0.960 (0.951) : bg1 0.963 (0.964) 0.494 (0.462) 0.889 (0.900) 0.990 (0.994) : bg2 0.971 (0.977) 0.653 (0.697) 0.901 (0.900) 0.994 (1.000) : : dataset PDEFoam : ------------------------------ : Signal 0.924 (0.931) 0.274 (0.374) 0.760 (0.781) 0.950 (0.963) : bg0 0.843 (0.853) 0.113 (0.167) 0.596 (0.613) 0.824 (0.833) : bg1 0.899 (0.909) 0.287 (0.290) 0.682 (0.740) 0.914 (0.920) : bg2 0.971 (0.968) 0.488 (0.436) 0.934 (0.913) 0.996 (0.999) : : dataset DL_CPU : ------------------------------ : Signal 0.960 (0.963) 0.208 (0.299) 0.921 (0.921) 0.991 (0.990) : bg0 0.919 (0.909) 0.244 (0.288) 0.764 (0.738) 0.942 (0.921) : bg1 0.951 (0.949) 0.341 (0.343) 0.865 (0.859) 0.984 (0.976) : bg2 0.913 (0.927) 0.564 (0.552) 0.744 (0.765) 0.886 (0.924) : : ------------------------------------------------------------------------------------------------------- : : : Confusion matrices for all methods : ------------------------------------------------------------------------------------------------------- : : Does a binary comparison between the two classes given by a : particular row-column combination. In each case, the class : given by the row is considered signal while the class given : by the column index is considered background. : : === Showing confusion matrix for method : BDTG : (Signal Efficiency for Background Efficiency 0.01%) : --------------------------------------------------- : Signal bg0 bg1 bg2 : test (train) test (train) test (train) test (train) : Signal - 0.489 (0.430) 0.864 (0.764) 0.784 (0.472) : bg0 0.311 (0.181) - 0.207 (0.132) 0.694 (0.611) : bg1 0.830 (0.834) 0.288 (0.339) - 0.668 (0.511) : bg2 0.708 (0.593) 0.684 (0.593) 0.625 (0.600) - : : (Signal Efficiency for Background Efficiency 0.10%) : --------------------------------------------------- : Signal bg0 bg1 bg2 : test (train) test (train) test (train) test (train) : Signal - 0.901 (0.852) 0.996 (0.993) 0.956 (0.892) : bg0 0.810 (0.763) - 0.643 (0.601) 0.924 (0.904) : bg1 0.984 (0.984) 0.716 (0.677) - 0.898 (0.843) : bg2 0.983 (0.928) 0.982 (0.953) 0.948 (0.897) - : : (Signal Efficiency for Background Efficiency 0.30%) : --------------------------------------------------- : Signal bg0 bg1 bg2 : test (train) test (train) test (train) test (train) : Signal - 0.981 (0.960) 1.000 (0.999) 0.999 (0.998) : bg0 0.963 (0.927) - 0.852 (0.814) 0.990 (0.986) : bg1 0.999 (0.998) 0.915 (0.888) - 0.984 (0.984) : bg2 0.999 (0.996) 0.998 (0.995) 0.998 (0.993) - : : === Showing confusion matrix for method : MLP : (Signal Efficiency for Background Efficiency 0.01%) : --------------------------------------------------- : Signal bg0 bg1 bg2 : test (train) test (train) test (train) test (train) : Signal - 0.456 (0.481) 0.936 (0.943) 0.645 (0.548) : bg0 0.421 (0.278) - 0.302 (0.229) 0.604 (0.477) : bg1 0.913 (0.925) 0.261 (0.400) - 0.566 (0.602) : bg2 0.675 (0.662) 0.710 (0.669) 0.696 (0.641) - : : (Signal Efficiency for Background Efficiency 0.10%) : --------------------------------------------------- : Signal bg0 bg1 bg2 : test (train) test (train) test (train) test (train) : Signal - 0.875 (0.891) 0.999 (1.000) 0.920 (0.894) : bg0 0.766 (0.755) - 0.696 (0.707) 0.909 (0.905) : bg1 0.997 (0.992) 0.780 (0.790) - 0.901 (0.867) : bg2 0.880 (0.890) 0.944 (0.933) 0.891 (0.886) - : : (Signal Efficiency for Background Efficiency 0.30%) : --------------------------------------------------- : Signal bg0 bg1 bg2 : test (train) test (train) test (train) test (train) : Signal - 0.974 (0.972) 1.000 (1.000) 0.995 (0.998) : bg0 0.954 (0.960) - 0.914 (0.914) 0.995 (0.992) : bg1 0.999 (0.999) 0.958 (0.944) - 0.998 (0.996) : bg2 0.999 (0.990) 1.000 (0.996) 0.999 (0.989) - : : === Showing confusion matrix for method : PDEFoam : (Signal Efficiency for Background Efficiency 0.01%) : --------------------------------------------------- : Signal bg0 bg1 bg2 : test (train) test (train) test (train) test (train) : Signal - 0.273 (0.124) 0.423 (0.441) 0.405 (0.491) : bg0 0.156 (0.076) - 0.126 (0.077) 0.512 (0.496) : bg1 0.480 (0.405) 0.197 (0.220) - 0.410 (0.350) : bg2 0.462 (0.507) 0.462 (0.577) 0.401 (0.412) - : : (Signal Efficiency for Background Efficiency 0.10%) : --------------------------------------------------- : Signal bg0 bg1 bg2 : test (train) test (train) test (train) test (train) : Signal - 0.673 (0.666) 0.826 (0.851) 0.835 (0.796) : bg0 0.569 (0.562) - 0.528 (0.477) 0.814 (0.809) : bg1 0.825 (0.811) 0.539 (0.520) - 0.808 (0.784) : bg2 0.925 (0.934) 0.948 (0.969) 0.874 (0.876) - : : (Signal Efficiency for Background Efficiency 0.30%) : --------------------------------------------------- : Signal bg0 bg1 bg2 : test (train) test (train) test (train) test (train) : Signal - 0.908 (0.894) 0.973 (0.965) 0.972 (0.962) : bg0 0.781 (0.793) - 0.725 (0.740) 0.947 (0.931) : bg1 0.955 (0.949) 0.840 (0.835) - 0.935 (0.928) : bg2 0.999 (0.996) 0.999 (0.998) 0.999 (0.996) - : : === Showing confusion matrix for method : DL_CPU : (Signal Efficiency for Background Efficiency 0.01%) : --------------------------------------------------- : Signal bg0 bg1 bg2 : test (train) test (train) test (train) test (train) : Signal - 0.448 (0.499) 0.930 (0.952) 0.158 (0.117) : bg0 0.413 (0.255) - 0.200 (0.192) 0.491 (0.300) : bg1 0.895 (0.925) 0.187 (0.251) - 0.298 (0.268) : bg2 0.551 (0.580) 0.564 (0.571) 0.478 (0.539) - : : (Signal Efficiency for Background Efficiency 0.10%) : --------------------------------------------------- : Signal bg0 bg1 bg2 : test (train) test (train) test (train) test (train) : Signal - 0.882 (0.898) 0.994 (0.996) 0.821 (0.747) : bg0 0.766 (0.775) - 0.673 (0.654) 0.756 (0.813) : bg1 0.995 (0.992) 0.758 (0.792) - 0.794 (0.766) : bg2 0.808 (0.787) 0.754 (0.728) 0.749 (0.733) - : : (Signal Efficiency for Background Efficiency 0.30%) : --------------------------------------------------- : Signal bg0 bg1 bg2 : test (train) test (train) test (train) test (train) : Signal - 0.983 (0.979) 0.999 (1.000) 0.989 (0.983) : bg0 0.924 (0.947) - 0.896 (0.916) 0.957 (0.959) : bg1 0.999 (1.000) 0.944 (0.953) - 0.956 (0.966) : bg2 0.942 (0.919) 0.926 (0.901) 0.897 (0.854) - : : ------------------------------------------------------------------------------------------------------- : Dataset:dataset : Created tree 'TestTree' with 4000 events : Dataset:dataset : Created tree 'TrainTree' with 4000 events : Factory : Thank you for using TMVA! : For citation information, please visit: http://tmva.sf.net/citeTMVA.html ==> Wrote root file: TMVAMulticlass.root ==> TMVAMulticlass is done!