Logo ROOT  
Reference Guide
 
Loading...
Searching...
No Matches
TMVA_RNN_Classification.py File Reference

Namespaces

namespace  TMVA_RNN_Classification
 

Detailed Description

View in nbviewer Open in SWAN
TMVA Classification Example Using a Recurrent Neural Network

This is an example of using a RNN in TMVA. We do classification using a toy time dependent data set that is generated when running this example macro

DataSetInfo : [dataset] : Added class "Signal"
: Add Tree sgn of type Signal with 2000 events
DataSetInfo : [dataset] : Added class "Background"
: Add Tree bkg of type Background with 2000 events
Factory : Booking method: ␛[1mTMVA_LSTM␛[0m
:
: Parsing option string:
: ... "!H:V:ErrorStrategy=CROSSENTROPY:VarTransform=None:WeightInitialization=XAVIERUNIFORM:ValidationSize=0.2:RandomSeed=1234:InputLayout=10|30:Layout=LSTM|10|30|10|0|1,RESHAPE|FLAT,DENSE|64|TANH,LINEAR:TrainingStrategy=LearningRate=1e-3,Momentum=0.0,Repetitions=1,ConvergenceSteps=5,BatchSize=100,TestRepetitions=1,WeightDecay=1e-2,Regularization=None,MaxEpochs=10Optimizer=ADAM,DropConfig=0.0+0.+0.+0.:Architecture=CPU"
: The following options are set:
: - By User:
: <none>
: - Default:
: Boost_num: "0" [Number of times the classifier will be boosted]
: Parsing option string:
: ... "!H:V:ErrorStrategy=CROSSENTROPY:VarTransform=None:WeightInitialization=XAVIERUNIFORM:ValidationSize=0.2:RandomSeed=1234:InputLayout=10|30:Layout=LSTM|10|30|10|0|1,RESHAPE|FLAT,DENSE|64|TANH,LINEAR:TrainingStrategy=LearningRate=1e-3,Momentum=0.0,Repetitions=1,ConvergenceSteps=5,BatchSize=100,TestRepetitions=1,WeightDecay=1e-2,Regularization=None,MaxEpochs=10Optimizer=ADAM,DropConfig=0.0+0.+0.+0.:Architecture=CPU"
: The following options are set:
: - By User:
: V: "True" [Verbose output (short form of "VerbosityLevel" below - overrides the latter one)]
: VarTransform: "None" [List of variable transformations performed before training, e.g., "D_Background,P_Signal,G,N_AllClasses" for: "Decorrelation, PCA-transformation, Gaussianisation, Normalisation, each for the given class of events ('AllClasses' denotes all events of all classes, if no class indication is given, 'All' is assumed)"]
: H: "False" [Print method-specific help message]
: InputLayout: "10|30" [The Layout of the input]
: Layout: "LSTM|10|30|10|0|1,RESHAPE|FLAT,DENSE|64|TANH,LINEAR" [Layout of the network.]
: ErrorStrategy: "CROSSENTROPY" [Loss function: Mean squared error (regression) or cross entropy (binary classification).]
: WeightInitialization: "XAVIERUNIFORM" [Weight initialization strategy]
: RandomSeed: "1234" [Random seed used for weight initialization and batch shuffling]
: ValidationSize: "0.2" [Part of the training data to use for validation. Specify as 0.2 or 20% to use a fifth of the data set as validation set. Specify as 100 to use exactly 100 events. (Default: 20%)]
: Architecture: "CPU" [Which architecture to perform the training on.]
: TrainingStrategy: "LearningRate=1e-3,Momentum=0.0,Repetitions=1,ConvergenceSteps=5,BatchSize=100,TestRepetitions=1,WeightDecay=1e-2,Regularization=None,MaxEpochs=10Optimizer=ADAM,DropConfig=0.0+0.+0.+0." [Defines the training strategies.]
: - Default:
: VerbosityLevel: "Default" [Verbosity level]
: CreateMVAPdfs: "False" [Create PDFs for classifier outputs (signal and background)]
: IgnoreNegWeightsInTraining: "False" [Events with negative weights are ignored in the training (but are included for testing and performance evaluation)]
: BatchLayout: "0|0|0" [The Layout of the batch]
: Will now use the CPU architecture with BLAS and IMT support !
Factory : Booking method: ␛[1mTMVA_DNN␛[0m
:
: Parsing option string:
: ... "!H:V:ErrorStrategy=CROSSENTROPY:VarTransform=None:WeightInitialization=XAVIER:RandomSeed=0:InputLayout=1|1|300:Layout=DENSE|64|TANH,DENSE|TANH|64,DENSE|TANH|64,LINEAR:TrainingStrategy=LearningRate=1e-3,Momentum=0.0,Repetitions=1,ConvergenceSteps=10,BatchSize=256,TestRepetitions=1,WeightDecay=1e-4,Regularization=None,MaxEpochs=20DropConfig=0.0+0.+0.+0.,Optimizer=ADAM:CPU"
: The following options are set:
: - By User:
: <none>
: - Default:
: Boost_num: "0" [Number of times the classifier will be boosted]
: Parsing option string:
: ... "!H:V:ErrorStrategy=CROSSENTROPY:VarTransform=None:WeightInitialization=XAVIER:RandomSeed=0:InputLayout=1|1|300:Layout=DENSE|64|TANH,DENSE|TANH|64,DENSE|TANH|64,LINEAR:TrainingStrategy=LearningRate=1e-3,Momentum=0.0,Repetitions=1,ConvergenceSteps=10,BatchSize=256,TestRepetitions=1,WeightDecay=1e-4,Regularization=None,MaxEpochs=20DropConfig=0.0+0.+0.+0.,Optimizer=ADAM:CPU"
: The following options are set:
: - By User:
: V: "True" [Verbose output (short form of "VerbosityLevel" below - overrides the latter one)]
: VarTransform: "None" [List of variable transformations performed before training, e.g., "D_Background,P_Signal,G,N_AllClasses" for: "Decorrelation, PCA-transformation, Gaussianisation, Normalisation, each for the given class of events ('AllClasses' denotes all events of all classes, if no class indication is given, 'All' is assumed)"]
: H: "False" [Print method-specific help message]
: InputLayout: "1|1|300" [The Layout of the input]
: Layout: "DENSE|64|TANH,DENSE|TANH|64,DENSE|TANH|64,LINEAR" [Layout of the network.]
: ErrorStrategy: "CROSSENTROPY" [Loss function: Mean squared error (regression) or cross entropy (binary classification).]
: WeightInitialization: "XAVIER" [Weight initialization strategy]
: RandomSeed: "0" [Random seed used for weight initialization and batch shuffling]
: Architecture: "CPU" [Which architecture to perform the training on.]
: TrainingStrategy: "LearningRate=1e-3,Momentum=0.0,Repetitions=1,ConvergenceSteps=10,BatchSize=256,TestRepetitions=1,WeightDecay=1e-4,Regularization=None,MaxEpochs=20DropConfig=0.0+0.+0.+0.,Optimizer=ADAM" [Defines the training strategies.]
: - Default:
: VerbosityLevel: "Default" [Verbosity level]
: CreateMVAPdfs: "False" [Create PDFs for classifier outputs (signal and background)]
: IgnoreNegWeightsInTraining: "False" [Events with negative weights are ignored in the training (but are included for testing and performance evaluation)]
: BatchLayout: "0|0|0" [The Layout of the batch]
: ValidationSize: "20%" [Part of the training data to use for validation. Specify as 0.2 or 20% to use a fifth of the data set as validation set. Specify as 100 to use exactly 100 events. (Default: 20%)]
: Will now use the CPU architecture with BLAS and IMT support !
Running with nthreads = 4
--- RNNClassification : Using input file: time_data_t10_d30.root
number of variables is 300
vars_time0[0]
vars_time0[1]
vars_time0[2]
vars_time0[3]
vars_time0[4]
vars_time0[5]
vars_time0[6]
vars_time0[7]
vars_time0[8]
vars_time0[9]
vars_time0[10]
vars_time0[11]
vars_time0[12]
vars_time0[13]
vars_time0[14]
vars_time0[15]
vars_time0[16]
vars_time0[17]
vars_time0[18]
vars_time0[19]
vars_time0[20]
vars_time0[21]
vars_time0[22]
vars_time0[23]
vars_time0[24]
vars_time0[25]
vars_time0[26]
vars_time0[27]
vars_time0[28]
vars_time0[29]
vars_time1[0]
vars_time1[1]
vars_time1[2]
vars_time1[3]
vars_time1[4]
vars_time1[5]
vars_time1[6]
vars_time1[7]
vars_time1[8]
vars_time1[9]
vars_time1[10]
vars_time1[11]
vars_time1[12]
vars_time1[13]
vars_time1[14]
vars_time1[15]
vars_time1[16]
vars_time1[17]
vars_time1[18]
vars_time1[19]
vars_time1[20]
vars_time1[21]
vars_time1[22]
vars_time1[23]
vars_time1[24]
vars_time1[25]
vars_time1[26]
vars_time1[27]
vars_time1[28]
vars_time1[29]
vars_time2[0]
vars_time2[1]
vars_time2[2]
vars_time2[3]
vars_time2[4]
vars_time2[5]
vars_time2[6]
vars_time2[7]
vars_time2[8]
vars_time2[9]
vars_time2[10]
vars_time2[11]
vars_time2[12]
vars_time2[13]
vars_time2[14]
vars_time2[15]
vars_time2[16]
vars_time2[17]
vars_time2[18]
vars_time2[19]
vars_time2[20]
vars_time2[21]
vars_time2[22]
vars_time2[23]
vars_time2[24]
vars_time2[25]
vars_time2[26]
vars_time2[27]
vars_time2[28]
vars_time2[29]
vars_time3[0]
vars_time3[1]
vars_time3[2]
vars_time3[3]
vars_time3[4]
vars_time3[5]
vars_time3[6]
vars_time3[7]
vars_time3[8]
vars_time3[9]
vars_time3[10]
vars_time3[11]
vars_time3[12]
vars_time3[13]
vars_time3[14]
vars_time3[15]
vars_time3[16]
vars_time3[17]
vars_time3[18]
vars_time3[19]
vars_time3[20]
vars_time3[21]
vars_time3[22]
vars_time3[23]
vars_time3[24]
vars_time3[25]
vars_time3[26]
vars_time3[27]
vars_time3[28]
vars_time3[29]
vars_time4[0]
vars_time4[1]
vars_time4[2]
vars_time4[3]
vars_time4[4]
vars_time4[5]
vars_time4[6]
vars_time4[7]
vars_time4[8]
vars_time4[9]
vars_time4[10]
vars_time4[11]
vars_time4[12]
vars_time4[13]
vars_time4[14]
vars_time4[15]
vars_time4[16]
vars_time4[17]
vars_time4[18]
vars_time4[19]
vars_time4[20]
vars_time4[21]
vars_time4[22]
vars_time4[23]
vars_time4[24]
vars_time4[25]
vars_time4[26]
vars_time4[27]
vars_time4[28]
vars_time4[29]
vars_time5[0]
vars_time5[1]
vars_time5[2]
vars_time5[3]
vars_time5[4]
vars_time5[5]
vars_time5[6]
vars_time5[7]
vars_time5[8]
vars_time5[9]
vars_time5[10]
vars_time5[11]
vars_time5[12]
vars_time5[13]
vars_time5[14]
vars_time5[15]
vars_time5[16]
vars_time5[17]
vars_time5[18]
vars_time5[19]
vars_time5[20]
vars_time5[21]
vars_time5[22]
vars_time5[23]
vars_time5[24]
vars_time5[25]
vars_time5[26]
vars_time5[27]
vars_time5[28]
vars_time5[29]
vars_time6[0]
vars_time6[1]
vars_time6[2]
vars_time6[3]
vars_time6[4]
vars_time6[5]
vars_time6[6]
vars_time6[7]
vars_time6[8]
vars_time6[9]
vars_time6[10]
vars_time6[11]
vars_time6[12]
vars_time6[13]
vars_time6[14]
vars_time6[15]
vars_time6[16]
vars_time6[17]
vars_time6[18]
vars_time6[19]
vars_time6[20]
vars_time6[21]
vars_time6[22]
vars_time6[23]
vars_time6[24]
vars_time6[25]
vars_time6[26]
vars_time6[27]
vars_time6[28]
vars_time6[29]
vars_time7[0]
vars_time7[1]
vars_time7[2]
vars_time7[3]
vars_time7[4]
vars_time7[5]
vars_time7[6]
vars_time7[7]
vars_time7[8]
vars_time7[9]
vars_time7[10]
vars_time7[11]
vars_time7[12]
vars_time7[13]
vars_time7[14]
vars_time7[15]
vars_time7[16]
vars_time7[17]
vars_time7[18]
vars_time7[19]
vars_time7[20]
vars_time7[21]
vars_time7[22]
vars_time7[23]
vars_time7[24]
vars_time7[25]
vars_time7[26]
vars_time7[27]
vars_time7[28]
vars_time7[29]
vars_time8[0]
vars_time8[1]
vars_time8[2]
vars_time8[3]
vars_time8[4]
vars_time8[5]
vars_time8[6]
vars_time8[7]
vars_time8[8]
vars_time8[9]
vars_time8[10]
vars_time8[11]
vars_time8[12]
vars_time8[13]
vars_time8[14]
vars_time8[15]
vars_time8[16]
vars_time8[17]
vars_time8[18]
vars_time8[19]
vars_time8[20]
vars_time8[21]
vars_time8[22]
vars_time8[23]
vars_time8[24]
vars_time8[25]
vars_time8[26]
vars_time8[27]
vars_time8[28]
vars_time8[29]
vars_time9[0]
vars_time9[1]
vars_time9[2]
vars_time9[3]
vars_time9[4]
vars_time9[5]
vars_time9[6]
vars_time9[7]
vars_time9[8]
vars_time9[9]
vars_time9[10]
vars_time9[11]
vars_time9[12]
vars_time9[13]
vars_time9[14]
vars_time9[15]
vars_time9[16]
vars_time9[17]
vars_time9[18]
vars_time9[19]
vars_time9[20]
vars_time9[21]
vars_time9[22]
vars_time9[23]
vars_time9[24]
vars_time9[25]
vars_time9[26]
vars_time9[27]
vars_time9[28]
vars_time9[29]
prepared DATA LOADER
Building recurrent keras model using a LSTM layer
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
reshape (Reshape) (None, 10, 30) 0
lstm (LSTM) (None, 10, 10) 1640
flatten (Flatten) (None, 100) 0
dense (Dense) (None, 64) 6464
dense_1 (Dense) (None, 2) 130
=================================================================
Total params: 8234 (32.16 KB)
Trainable params: 8234 (32.16 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
Factory : Booking method: ␛[1mPyKeras_LSTM␛[0m
:
: Setting up tf.keras
: Using TensorFlow version 2
: Use Keras version from TensorFlow : tf.keras
: Applying GPU option: gpu_options.allow_growth=True
: Loading Keras Model
: Loaded model from file: model_LSTM.h5
Factory : Booking method: ␛[1mBDTG␛[0m
:
: the option NegWeightTreatment=InverseBoostNegWeights does not exist for BoostType=Grad
: --> change to new default NegWeightTreatment=Pray
: Rebuilding Dataset dataset
: Building event vectors for type 2 Signal
: Dataset[dataset] : create input formulas for tree sgn
: Using variable vars_time0[0] from array expression vars_time0 of size 30
: Using variable vars_time1[0] from array expression vars_time1 of size 30
: Using variable vars_time2[0] from array expression vars_time2 of size 30
: Using variable vars_time3[0] from array expression vars_time3 of size 30
: Using variable vars_time4[0] from array expression vars_time4 of size 30
: Using variable vars_time5[0] from array expression vars_time5 of size 30
: Using variable vars_time6[0] from array expression vars_time6 of size 30
: Using variable vars_time7[0] from array expression vars_time7 of size 30
: Using variable vars_time8[0] from array expression vars_time8 of size 30
: Using variable vars_time9[0] from array expression vars_time9 of size 30
: Building event vectors for type 2 Background
: Dataset[dataset] : create input formulas for tree bkg
: Using variable vars_time0[0] from array expression vars_time0 of size 30
: Using variable vars_time1[0] from array expression vars_time1 of size 30
: Using variable vars_time2[0] from array expression vars_time2 of size 30
: Using variable vars_time3[0] from array expression vars_time3 of size 30
: Using variable vars_time4[0] from array expression vars_time4 of size 30
: Using variable vars_time5[0] from array expression vars_time5 of size 30
: Using variable vars_time6[0] from array expression vars_time6 of size 30
: Using variable vars_time7[0] from array expression vars_time7 of size 30
: Using variable vars_time8[0] from array expression vars_time8 of size 30
: Using variable vars_time9[0] from array expression vars_time9 of size 30
DataSetFactory : [dataset] : Number of events in input trees
:
:
: Number of training and testing events
: ---------------------------------------------------------------------------
: Signal -- training events : 1600
: Signal -- testing events : 400
: Signal -- training and testing events: 2000
: Background -- training events : 1600
: Background -- testing events : 400
: Background -- training and testing events: 2000
:
Factory : ␛[1mTrain all methods␛[0m
Factory : Train method: TMVA_LSTM for Classification
:
: Start of deep neural network training on CPU using MT, nthreads = 4
:
: ***** Deep Learning Network *****
DEEP NEURAL NETWORK: Depth = 4 Input = ( 10, 1, 30 ) Batch size = 100 Loss function = C
Layer 0 LSTM Layer: (NInput = 30, NState = 10, NTime = 10 ) Output = ( 100 , 10 , 10 )
Layer 1 RESHAPE Layer Input = ( 1 , 10 , 10 ) Output = ( 1 , 100 , 100 )
Layer 2 DENSE Layer: ( Input = 100 , Width = 64 ) Output = ( 1 , 100 , 64 ) Activation Function = Tanh
Layer 3 DENSE Layer: ( Input = 64 , Width = 1 ) Output = ( 1 , 100 , 1 ) Activation Function = Identity
: Using 2560 events for training and 640 for testing
: Compute initial loss on the validation data
: Training phase 1 of 1: Optimizer ADAM (beta1=0.9,beta2=0.999,eps=1e-07) Learning rate = 0.001 regularization 0 minimum error = 0.707058
: --------------------------------------------------------------
: Epoch | Train Err. Val. Err. t(s)/epoch t(s)/Loss nEvents/s Conv. Steps
: --------------------------------------------------------------
: Start epoch iteration ...
: 1 Minimum Test error found - save the configuration
: 1 | 0.70278 0.693865 0.596742 0.0400638 4490.92 0
: 2 Minimum Test error found - save the configuration
: 2 | 0.69287 0.685182 0.597184 0.0397349 4484.71 0
: 3 Minimum Test error found - save the configuration
: 3 | 0.685469 0.681006 0.599617 0.039634 4464.42 0
: 4 Minimum Test error found - save the configuration
: 4 | 0.675425 0.673416 0.587199 0.0393696 4563.46 0
: 5 | 0.662939 0.6736 0.577989 0.0393349 4641.2 1
: 6 Minimum Test error found - save the configuration
: 6 | 0.649297 0.670376 0.577836 0.0395373 4644.26 0
: 7 Minimum Test error found - save the configuration
: 7 | 0.641764 0.655104 0.558261 0.0392463 4816.81 0
: 8 Minimum Test error found - save the configuration
: 8 | 0.625596 0.643419 0.569494 0.0396779 4718.61 0
: 9 Minimum Test error found - save the configuration
: 9 | 0.609922 0.632299 0.565489 0.0404186 4761.26 0
: 10 Minimum Test error found - save the configuration
: 10 | 0.59561 0.625754 0.560007 0.0396188 4804.1 0
:
: Elapsed time for training with 3200 events: 5.84 sec
: Evaluate deep neural network on CPU using batches with size = 100
:
TMVA_LSTM : [dataset] : Evaluation of TMVA_LSTM on training sample (3200 events)
: Elapsed time for evaluation of 3200 events: 0.211 sec
: Creating xml weight file: ␛[0;36mdataset/weights/TMVAClassification_TMVA_LSTM.weights.xml␛[0m
: Creating standalone class: ␛[0;36mdataset/weights/TMVAClassification_TMVA_LSTM.class.C␛[0m
Factory : Training finished
:
Factory : Train method: TMVA_DNN for Classification
:
: Start of deep neural network training on CPU using MT, nthreads = 4
:
: ***** Deep Learning Network *****
DEEP NEURAL NETWORK: Depth = 4 Input = ( 1, 1, 300 ) Batch size = 256 Loss function = C
Layer 0 DENSE Layer: ( Input = 300 , Width = 64 ) Output = ( 1 , 256 , 64 ) Activation Function = Tanh
Layer 1 DENSE Layer: ( Input = 64 , Width = 64 ) Output = ( 1 , 256 , 64 ) Activation Function = Tanh
Layer 2 DENSE Layer: ( Input = 64 , Width = 64 ) Output = ( 1 , 256 , 64 ) Activation Function = Tanh
Layer 3 DENSE Layer: ( Input = 64 , Width = 1 ) Output = ( 1 , 256 , 1 ) Activation Function = Identity
: Using 2560 events for training and 640 for testing
: Compute initial loss on the validation data
: Training phase 1 of 1: Optimizer ADAM (beta1=0.9,beta2=0.999,eps=1e-07) Learning rate = 0.001 regularization 0 minimum error = 0.923998
: --------------------------------------------------------------
: Epoch | Train Err. Val. Err. t(s)/epoch t(s)/Loss nEvents/s Conv. Steps
: --------------------------------------------------------------
: Start epoch iteration ...
: 1 Minimum Test error found - save the configuration
: 1 | 0.800628 0.758422 0.192653 0.0157468 14470.9 0
: 2 Minimum Test error found - save the configuration
: 2 | 0.711458 0.697061 0.192076 0.0156086 14506.9 0
: 3 Minimum Test error found - save the configuration
: 3 | 0.68502 0.692941 0.195265 0.0155131 14241.9 0
: 4 Minimum Test error found - save the configuration
: 4 | 0.685064 0.68697 0.195149 0.0157552 14270.3 0
: 5 Minimum Test error found - save the configuration
: 5 | 0.684515 0.681631 0.192511 0.0155061 14462.8 0
: 6 | 0.681062 0.686013 0.192316 0.0150816 14444.2 1
: 7 | 0.686073 0.68553 0.19302 0.0153508 14408.8 2
: 8 | 0.689494 0.687685 0.193342 0.015139 14365.6 3
: 9 Minimum Test error found - save the configuration
: 9 | 0.677964 0.676494 0.192456 0.0155373 14470 0
: 10 Minimum Test error found - save the configuration
: 10 | 0.673215 0.675339 0.193247 0.0155907 14409.8 0
: 11 | 0.66488 0.682329 0.192278 0.0151575 14453.4 1
: 12 Minimum Test error found - save the configuration
: 12 | 0.66478 0.672006 0.192674 0.0156035 14457.5 0
: 13 | 0.668681 0.685484 0.192683 0.0152245 14425.9 1
: 14 | 0.666995 0.680032 0.190634 0.0150497 14579.9 2
: 15 | 0.664743 0.701315 0.190794 0.0152132 14580.2 3
: 16 | 0.656504 0.68456 0.190861 0.0151994 14573.5 4
: 17 | 0.665009 0.681945 0.190325 0.0149276 14595.4 5
: 18 | 0.677906 0.683282 0.190455 0.0152069 14607.9 6
: 19 | 0.67405 0.689419 0.190389 0.0150727 14602.2 7
: 20 | 0.662402 0.675381 0.190493 0.0150978 14595.6 8
:
: Elapsed time for training with 3200 events: 3.86 sec
: Evaluate deep neural network on CPU using batches with size = 256
:
TMVA_DNN : [dataset] : Evaluation of TMVA_DNN on training sample (3200 events)
: Elapsed time for evaluation of 3200 events: 0.1 sec
: Creating xml weight file: ␛[0;36mdataset/weights/TMVAClassification_TMVA_DNN.weights.xml␛[0m
: Creating standalone class: ␛[0;36mdataset/weights/TMVAClassification_TMVA_DNN.class.C␛[0m
Factory : Training finished
:
Factory : Train method: PyKeras_LSTM for Classification
:
:
: ␛[1m================================================================␛[0m
: ␛[1mH e l p f o r M V A m e t h o d [ PyKeras_LSTM ] :␛[0m
:
: Keras is a high-level API for the Theano and Tensorflow packages.
: This method wraps the training and predictions steps of the Keras
: Python package for TMVA, so that dataloading, preprocessing and
: evaluation can be done within the TMVA system. To use this Keras
: interface, you have to generate a model with Keras first. Then,
: this model can be loaded and trained in TMVA.
:
:
: <Suppress this message by specifying "!H" in the booking option>
: ␛[1m================================================================␛[0m
:
: Split TMVA training data in 2560 training events and 640 validation events
: Training Model Summary
saved recurrent model model_LSTM.h5
Booking Keras model LSTM
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
reshape (Reshape) (None, 10, 30) 0
lstm (LSTM) (None, 10, 10) 1640
flatten (Flatten) (None, 100) 0
dense (Dense) (None, 64) 6464
dense_1 (Dense) (None, 2) 130
=================================================================
Total params: 8234 (32.16 KB)
Trainable params: 8234 (32.16 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
: Option SaveBestOnly: Only model weights with smallest validation loss will be stored
Epoch 1/10
1/26 [>.............................] - ETA: 42s - loss: 0.6988 - accuracy: 0.5400␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
8/26 [========>.....................] - ETA: 0s - loss: 0.6995 - accuracy: 0.5263 ␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
16/26 [=================>............] - ETA: 0s - loss: 0.6964 - accuracy: 0.5294␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
24/26 [==========================>...] - ETA: 0s - loss: 0.6941 - accuracy: 0.5342
Epoch 1: val_loss improved from inf to 0.68460, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 3s 38ms/step - loss: 0.6940 - accuracy: 0.5340 - val_loss: 0.6846 - val_accuracy: 0.5750
Epoch 2/10
1/26 [>.............................] - ETA: 0s - loss: 0.6886 - accuracy: 0.4700␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
9/26 [=========>....................] - ETA: 0s - loss: 0.6819 - accuracy: 0.5833␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
17/26 [==================>...........] - ETA: 0s - loss: 0.6770 - accuracy: 0.5812␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
25/26 [===========================>..] - ETA: 0s - loss: 0.6772 - accuracy: 0.5768
Epoch 2: val_loss improved from 0.68460 to 0.67811, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 9ms/step - loss: 0.6777 - accuracy: 0.5758 - val_loss: 0.6781 - val_accuracy: 0.5641
Epoch 3/10
1/26 [>.............................] - ETA: 0s - loss: 0.6683 - accuracy: 0.5900␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
9/26 [=========>....................] - ETA: 0s - loss: 0.6571 - accuracy: 0.6111␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
17/26 [==================>...........] - ETA: 0s - loss: 0.6559 - accuracy: 0.6176␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
24/26 [==========================>...] - ETA: 0s - loss: 0.6562 - accuracy: 0.6246
Epoch 3: val_loss improved from 0.67811 to 0.65092, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 10ms/step - loss: 0.6561 - accuracy: 0.6234 - val_loss: 0.6509 - val_accuracy: 0.6297
Epoch 4/10
1/26 [>.............................] - ETA: 0s - loss: 0.6312 - accuracy: 0.6900␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
9/26 [=========>....................] - ETA: 0s - loss: 0.6476 - accuracy: 0.6444␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
17/26 [==================>...........] - ETA: 0s - loss: 0.6398 - accuracy: 0.6488␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - ETA: 0s - loss: 0.6287 - accuracy: 0.6641
Epoch 4: val_loss improved from 0.65092 to 0.63772, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 9ms/step - loss: 0.6287 - accuracy: 0.6641 - val_loss: 0.6377 - val_accuracy: 0.6453
Epoch 5/10
1/26 [>.............................] - ETA: 0s - loss: 0.6176 - accuracy: 0.6900␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
10/26 [==========>...................] - ETA: 0s - loss: 0.6037 - accuracy: 0.6720␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
19/26 [====================>.........] - ETA: 0s - loss: 0.5936 - accuracy: 0.6821
Epoch 5: val_loss improved from 0.63772 to 0.57409, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 8ms/step - loss: 0.5915 - accuracy: 0.6883 - val_loss: 0.5741 - val_accuracy: 0.7078
Epoch 6/10
1/26 [>.............................] - ETA: 0s - loss: 0.5766 - accuracy: 0.6900␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
11/26 [===========>..................] - ETA: 0s - loss: 0.5526 - accuracy: 0.7400␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
21/26 [=======================>......] - ETA: 0s - loss: 0.5446 - accuracy: 0.7376
Epoch 6: val_loss improved from 0.57409 to 0.55457, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 8ms/step - loss: 0.5440 - accuracy: 0.7387 - val_loss: 0.5546 - val_accuracy: 0.7109
Epoch 7/10
1/26 [>.............................] - ETA: 0s - loss: 0.4654 - accuracy: 0.7900␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
11/26 [===========>..................] - ETA: 0s - loss: 0.5059 - accuracy: 0.7636␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
20/26 [======================>.......] - ETA: 0s - loss: 0.5070 - accuracy: 0.7650
Epoch 7: val_loss improved from 0.55457 to 0.51088, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 9ms/step - loss: 0.5083 - accuracy: 0.7621 - val_loss: 0.5109 - val_accuracy: 0.7484
Epoch 8/10
1/26 [>.............................] - ETA: 0s - loss: 0.4671 - accuracy: 0.7600␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
11/26 [===========>..................] - ETA: 0s - loss: 0.4772 - accuracy: 0.7718␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
21/26 [=======================>......] - ETA: 0s - loss: 0.4807 - accuracy: 0.7790
Epoch 8: val_loss did not improve from 0.51088
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 7ms/step - loss: 0.4828 - accuracy: 0.7773 - val_loss: 0.5321 - val_accuracy: 0.7406
Epoch 9/10
1/26 [>.............................] - ETA: 0s - loss: 0.4301 - accuracy: 0.8300␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
11/26 [===========>..................] - ETA: 0s - loss: 0.4840 - accuracy: 0.7636␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
21/26 [=======================>......] - ETA: 0s - loss: 0.4723 - accuracy: 0.7719
Epoch 9: val_loss improved from 0.51088 to 0.49599, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 8ms/step - loss: 0.4665 - accuracy: 0.7746 - val_loss: 0.4960 - val_accuracy: 0.7563
Epoch 10/10
1/26 [>.............................] - ETA: 0s - loss: 0.5259 - accuracy: 0.7300␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
12/26 [============>.................] - ETA: 0s - loss: 0.4711 - accuracy: 0.7725␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
23/26 [=========================>....] - ETA: 0s - loss: 0.4553 - accuracy: 0.7861
Epoch 10: val_loss did not improve from 0.49599
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 7ms/step - loss: 0.4532 - accuracy: 0.7871 - val_loss: 0.4988 - val_accuracy: 0.7594
: Getting training history for item:0 name = 'loss'
: Getting training history for item:1 name = 'accuracy'
: Getting training history for item:2 name = 'val_loss'
: Getting training history for item:3 name = 'val_accuracy'
: Elapsed time for training with 3200 events: 4.71 sec
: Setting up tf.keras
: Using TensorFlow version 2
: Use Keras version from TensorFlow : tf.keras
: Applying GPU option: gpu_options.allow_growth=True
: Disabled TF eager execution when evaluating model
: Loading Keras Model
: Loaded model from file: trained_model_LSTM.h5
PyKeras_LSTM : [dataset] : Evaluation of PyKeras_LSTM on training sample (3200 events)
: Elapsed time for evaluation of 3200 events: 0.262 sec
: Creating xml weight file: ␛[0;36mdataset/weights/TMVAClassification_PyKeras_LSTM.weights.xml␛[0m
: Creating standalone class: ␛[0;36mdataset/weights/TMVAClassification_PyKeras_LSTM.class.C␛[0m
Factory : Training finished
:
Factory : Train method: BDTG for Classification
:
:
: ␛[1m================================================================␛[0m
: ␛[1mH e l p f o r M V A m e t h o d [ BDTG ] :␛[0m
:
: ␛[1m--- Short description:␛[0m
:
: Boosted Decision Trees are a collection of individual decision
: trees which form a multivariate classifier by (weighted) majority
: vote of the individual trees. Consecutive decision trees are
: trained using the original training data set with re-weighted
: events. By default, the AdaBoost method is employed, which gives
: events that were misclassified in the previous tree a larger
: weight in the training of the following tree.
:
: Decision trees are a sequence of binary splits of the data sample
: using a single discriminant variable at a time. A test event
: ending up after the sequence of left-right splits in a final
: ("leaf") node is classified as either signal or background
: depending on the majority type of training events in that node.
:
: ␛[1m--- Performance optimisation:␛[0m
:
: By the nature of the binary splits performed on the individual
: variables, decision trees do not deal well with linear correlations
: between variables (they need to approximate the linear split in
: the two dimensional space by a sequence of splits on the two
: variables individually). Hence decorrelation could be useful
: to optimise the BDT performance.
:
: ␛[1m--- Performance tuning via configuration options:␛[0m
:
: The two most important parameters in the configuration are the
: minimal number of events requested by a leaf node as percentage of the
: number of training events (option "MinNodeSize" replacing the actual number
: of events "nEventsMin" as given in earlier versions
: If this number is too large, detailed features
: in the parameter space are hard to be modelled. If it is too small,
: the risk to overtrain rises and boosting seems to be less effective
: typical values from our current experience for best performance
: are between 0.5(%) and 10(%)
:
: The default minimal number is currently set to
: max(20, (N_training_events / N_variables^2 / 10))
: and can be changed by the user.
:
: The other crucial parameter, the pruning strength ("PruneStrength"),
: is also related to overtraining. It is a regularisation parameter
: that is used when determining after the training which splits
: are considered statistically insignificant and are removed. The
: user is advised to carefully watch the BDT screen output for
: the comparison between efficiencies obtained on the training and
: the independent test sample. They should be equal within statistical
: errors, in order to minimize statistical fluctuations in different samples.
:
: <Suppress this message by specifying "!H" in the booking option>
: ␛[1m================================================================␛[0m
:
BDTG : #events: (reweighted) sig: 1600 bkg: 1600
: #events: (unweighted) sig: 1600 bkg: 1600
: Training 100 Decision Trees ... patience please
: Elapsed time for training with 3200 events: 1.68 sec
BDTG : [dataset] : Evaluation of BDTG on training sample (3200 events)
: Elapsed time for evaluation of 3200 events: 0.0191 sec
: Creating xml weight file: ␛[0;36mdataset/weights/TMVAClassification_BDTG.weights.xml␛[0m
: Creating standalone class: ␛[0;36mdataset/weights/TMVAClassification_BDTG.class.C␛[0m
: data_RNN_CPU.root:/dataset/Method_BDT/BDTG
Factory : Training finished
:
: Ranking input variables (method specific)...
: No variable ranking supplied by classifier: TMVA_LSTM
: No variable ranking supplied by classifier: TMVA_DNN
: No variable ranking supplied by classifier: PyKeras_LSTM
BDTG : Ranking result (top variable is best ranked)
: --------------------------------------------
: Rank : Variable : Variable Importance
: --------------------------------------------
: 1 : vars_time7 : 2.240e-02
: 2 : vars_time8 : 1.988e-02
: 3 : vars_time7 : 1.965e-02
: 4 : vars_time8 : 1.839e-02
: 5 : vars_time0 : 1.827e-02
: 6 : vars_time8 : 1.802e-02
: 7 : vars_time7 : 1.774e-02
: 8 : vars_time9 : 1.769e-02
: 9 : vars_time9 : 1.761e-02
: 10 : vars_time9 : 1.686e-02
: 11 : vars_time6 : 1.577e-02
: 12 : vars_time9 : 1.557e-02
: 13 : vars_time8 : 1.523e-02
: 14 : vars_time6 : 1.522e-02
: 15 : vars_time7 : 1.516e-02
: 16 : vars_time6 : 1.480e-02
: 17 : vars_time6 : 1.457e-02
: 18 : vars_time8 : 1.363e-02
: 19 : vars_time5 : 1.301e-02
: 20 : vars_time6 : 1.289e-02
: 21 : vars_time7 : 1.268e-02
: 22 : vars_time6 : 1.259e-02
: 23 : vars_time8 : 1.256e-02
: 24 : vars_time9 : 1.211e-02
: 25 : vars_time5 : 1.168e-02
: 26 : vars_time7 : 1.165e-02
: 27 : vars_time8 : 1.157e-02
: 28 : vars_time8 : 1.154e-02
: 29 : vars_time8 : 1.147e-02
: 30 : vars_time1 : 1.143e-02
: 31 : vars_time7 : 1.141e-02
: 32 : vars_time7 : 1.130e-02
: 33 : vars_time5 : 1.098e-02
: 34 : vars_time9 : 1.083e-02
: 35 : vars_time6 : 1.032e-02
: 36 : vars_time7 : 1.022e-02
: 37 : vars_time9 : 1.017e-02
: 38 : vars_time3 : 1.013e-02
: 39 : vars_time4 : 9.936e-03
: 40 : vars_time8 : 9.879e-03
: 41 : vars_time8 : 9.870e-03
: 42 : vars_time0 : 9.728e-03
: 43 : vars_time9 : 9.580e-03
: 44 : vars_time0 : 9.373e-03
: 45 : vars_time1 : 9.283e-03
: 46 : vars_time0 : 8.933e-03
: 47 : vars_time8 : 8.775e-03
: 48 : vars_time5 : 8.566e-03
: 49 : vars_time9 : 8.500e-03
: 50 : vars_time7 : 8.483e-03
: 51 : vars_time1 : 8.472e-03
: 52 : vars_time9 : 8.410e-03
: 53 : vars_time9 : 8.400e-03
: 54 : vars_time2 : 8.245e-03
: 55 : vars_time7 : 8.226e-03
: 56 : vars_time0 : 8.099e-03
: 57 : vars_time1 : 7.689e-03
: 58 : vars_time1 : 7.669e-03
: 59 : vars_time1 : 7.643e-03
: 60 : vars_time5 : 7.480e-03
: 61 : vars_time6 : 7.450e-03
: 62 : vars_time7 : 7.311e-03
: 63 : vars_time1 : 7.268e-03
: 64 : vars_time6 : 7.219e-03
: 65 : vars_time5 : 7.150e-03
: 66 : vars_time1 : 6.977e-03
: 67 : vars_time8 : 6.950e-03
: 68 : vars_time7 : 6.895e-03
: 69 : vars_time7 : 6.895e-03
: 70 : vars_time0 : 6.872e-03
: 71 : vars_time6 : 6.812e-03
: 72 : vars_time9 : 6.811e-03
: 73 : vars_time0 : 6.635e-03
: 74 : vars_time5 : 6.542e-03
: 75 : vars_time8 : 6.531e-03
: 76 : vars_time7 : 6.381e-03
: 77 : vars_time6 : 6.369e-03
: 78 : vars_time1 : 6.318e-03
: 79 : vars_time0 : 6.064e-03
: 80 : vars_time3 : 5.872e-03
: 81 : vars_time4 : 5.789e-03
: 82 : vars_time9 : 5.595e-03
: 83 : vars_time5 : 5.502e-03
: 84 : vars_time7 : 5.436e-03
: 85 : vars_time6 : 5.397e-03
: 86 : vars_time2 : 5.354e-03
: 87 : vars_time5 : 5.344e-03
: 88 : vars_time8 : 5.288e-03
: 89 : vars_time5 : 5.185e-03
: 90 : vars_time4 : 5.082e-03
: 91 : vars_time1 : 5.081e-03
: 92 : vars_time9 : 5.063e-03
: 93 : vars_time1 : 5.016e-03
: 94 : vars_time6 : 4.972e-03
: 95 : vars_time4 : 4.884e-03
: 96 : vars_time7 : 4.690e-03
: 97 : vars_time1 : 4.562e-03
: 98 : vars_time4 : 4.467e-03
: 99 : vars_time4 : 4.460e-03
: 100 : vars_time8 : 4.425e-03
: 101 : vars_time7 : 4.296e-03
: 102 : vars_time4 : 4.200e-03
: 103 : vars_time2 : 3.835e-03
: 104 : vars_time1 : 3.751e-03
: 105 : vars_time7 : 3.704e-03
: 106 : vars_time5 : 3.350e-03
: 107 : vars_time8 : 3.122e-03
: 108 : vars_time4 : 2.963e-03
: 109 : vars_time1 : 2.811e-03
: 110 : vars_time9 : 2.781e-03
: 111 : vars_time0 : 0.000e+00
: 112 : vars_time0 : 0.000e+00
: 113 : vars_time0 : 0.000e+00
: 114 : vars_time0 : 0.000e+00
: 115 : vars_time0 : 0.000e+00
: 116 : vars_time0 : 0.000e+00
: 117 : vars_time0 : 0.000e+00
: 118 : vars_time0 : 0.000e+00
: 119 : vars_time0 : 0.000e+00
: 120 : vars_time0 : 0.000e+00
: 121 : vars_time0 : 0.000e+00
: 122 : vars_time0 : 0.000e+00
: 123 : vars_time0 : 0.000e+00
: 124 : vars_time0 : 0.000e+00
: 125 : vars_time0 : 0.000e+00
: 126 : vars_time0 : 0.000e+00
: 127 : vars_time0 : 0.000e+00
: 128 : vars_time0 : 0.000e+00
: 129 : vars_time0 : 0.000e+00
: 130 : vars_time0 : 0.000e+00
: 131 : vars_time0 : 0.000e+00
: 132 : vars_time0 : 0.000e+00
: 133 : vars_time1 : 0.000e+00
: 134 : vars_time1 : 0.000e+00
: 135 : vars_time1 : 0.000e+00
: 136 : vars_time1 : 0.000e+00
: 137 : vars_time1 : 0.000e+00
: 138 : vars_time1 : 0.000e+00
: 139 : vars_time1 : 0.000e+00
: 140 : vars_time1 : 0.000e+00
: 141 : vars_time1 : 0.000e+00
: 142 : vars_time1 : 0.000e+00
: 143 : vars_time1 : 0.000e+00
: 144 : vars_time1 : 0.000e+00
: 145 : vars_time1 : 0.000e+00
: 146 : vars_time1 : 0.000e+00
: 147 : vars_time1 : 0.000e+00
: 148 : vars_time1 : 0.000e+00
: 149 : vars_time2 : 0.000e+00
: 150 : vars_time2 : 0.000e+00
: 151 : vars_time2 : 0.000e+00
: 152 : vars_time2 : 0.000e+00
: 153 : vars_time2 : 0.000e+00
: 154 : vars_time2 : 0.000e+00
: 155 : vars_time2 : 0.000e+00
: 156 : vars_time2 : 0.000e+00
: 157 : vars_time2 : 0.000e+00
: 158 : vars_time2 : 0.000e+00
: 159 : vars_time2 : 0.000e+00
: 160 : vars_time2 : 0.000e+00
: 161 : vars_time2 : 0.000e+00
: 162 : vars_time2 : 0.000e+00
: 163 : vars_time2 : 0.000e+00
: 164 : vars_time2 : 0.000e+00
: 165 : vars_time2 : 0.000e+00
: 166 : vars_time2 : 0.000e+00
: 167 : vars_time2 : 0.000e+00
: 168 : vars_time2 : 0.000e+00
: 169 : vars_time2 : 0.000e+00
: 170 : vars_time2 : 0.000e+00
: 171 : vars_time2 : 0.000e+00
: 172 : vars_time2 : 0.000e+00
: 173 : vars_time2 : 0.000e+00
: 174 : vars_time2 : 0.000e+00
: 175 : vars_time2 : 0.000e+00
: 176 : vars_time3 : 0.000e+00
: 177 : vars_time3 : 0.000e+00
: 178 : vars_time3 : 0.000e+00
: 179 : vars_time3 : 0.000e+00
: 180 : vars_time3 : 0.000e+00
: 181 : vars_time3 : 0.000e+00
: 182 : vars_time3 : 0.000e+00
: 183 : vars_time3 : 0.000e+00
: 184 : vars_time3 : 0.000e+00
: 185 : vars_time3 : 0.000e+00
: 186 : vars_time3 : 0.000e+00
: 187 : vars_time3 : 0.000e+00
: 188 : vars_time3 : 0.000e+00
: 189 : vars_time3 : 0.000e+00
: 190 : vars_time3 : 0.000e+00
: 191 : vars_time3 : 0.000e+00
: 192 : vars_time3 : 0.000e+00
: 193 : vars_time3 : 0.000e+00
: 194 : vars_time3 : 0.000e+00
: 195 : vars_time3 : 0.000e+00
: 196 : vars_time3 : 0.000e+00
: 197 : vars_time3 : 0.000e+00
: 198 : vars_time3 : 0.000e+00
: 199 : vars_time3 : 0.000e+00
: 200 : vars_time3 : 0.000e+00
: 201 : vars_time3 : 0.000e+00
: 202 : vars_time3 : 0.000e+00
: 203 : vars_time3 : 0.000e+00
: 204 : vars_time4 : 0.000e+00
: 205 : vars_time4 : 0.000e+00
: 206 : vars_time4 : 0.000e+00
: 207 : vars_time4 : 0.000e+00
: 208 : vars_time4 : 0.000e+00
: 209 : vars_time4 : 0.000e+00
: 210 : vars_time4 : 0.000e+00
: 211 : vars_time4 : 0.000e+00
: 212 : vars_time4 : 0.000e+00
: 213 : vars_time4 : 0.000e+00
: 214 : vars_time4 : 0.000e+00
: 215 : vars_time4 : 0.000e+00
: 216 : vars_time4 : 0.000e+00
: 217 : vars_time4 : 0.000e+00
: 218 : vars_time4 : 0.000e+00
: 219 : vars_time4 : 0.000e+00
: 220 : vars_time4 : 0.000e+00
: 221 : vars_time4 : 0.000e+00
: 222 : vars_time4 : 0.000e+00
: 223 : vars_time4 : 0.000e+00
: 224 : vars_time4 : 0.000e+00
: 225 : vars_time4 : 0.000e+00
: 226 : vars_time5 : 0.000e+00
: 227 : vars_time5 : 0.000e+00
: 228 : vars_time5 : 0.000e+00
: 229 : vars_time5 : 0.000e+00
: 230 : vars_time5 : 0.000e+00
: 231 : vars_time5 : 0.000e+00
: 232 : vars_time5 : 0.000e+00
: 233 : vars_time5 : 0.000e+00
: 234 : vars_time5 : 0.000e+00
: 235 : vars_time5 : 0.000e+00
: 236 : vars_time5 : 0.000e+00
: 237 : vars_time5 : 0.000e+00
: 238 : vars_time5 : 0.000e+00
: 239 : vars_time5 : 0.000e+00
: 240 : vars_time5 : 0.000e+00
: 241 : vars_time5 : 0.000e+00
: 242 : vars_time5 : 0.000e+00
: 243 : vars_time5 : 0.000e+00
: 244 : vars_time5 : 0.000e+00
: 245 : vars_time6 : 0.000e+00
: 246 : vars_time6 : 0.000e+00
: 247 : vars_time6 : 0.000e+00
: 248 : vars_time6 : 0.000e+00
: 249 : vars_time6 : 0.000e+00
: 250 : vars_time6 : 0.000e+00
: 251 : vars_time6 : 0.000e+00
: 252 : vars_time6 : 0.000e+00
: 253 : vars_time6 : 0.000e+00
: 254 : vars_time6 : 0.000e+00
: 255 : vars_time6 : 0.000e+00
: 256 : vars_time6 : 0.000e+00
: 257 : vars_time6 : 0.000e+00
: 258 : vars_time6 : 0.000e+00
: 259 : vars_time6 : 0.000e+00
: 260 : vars_time6 : 0.000e+00
: 261 : vars_time6 : 0.000e+00
: 262 : vars_time7 : 0.000e+00
: 263 : vars_time7 : 0.000e+00
: 264 : vars_time7 : 0.000e+00
: 265 : vars_time7 : 0.000e+00
: 266 : vars_time7 : 0.000e+00
: 267 : vars_time7 : 0.000e+00
: 268 : vars_time7 : 0.000e+00
: 269 : vars_time7 : 0.000e+00
: 270 : vars_time7 : 0.000e+00
: 271 : vars_time7 : 0.000e+00
: 272 : vars_time7 : 0.000e+00
: 273 : vars_time8 : 0.000e+00
: 274 : vars_time8 : 0.000e+00
: 275 : vars_time8 : 0.000e+00
: 276 : vars_time8 : 0.000e+00
: 277 : vars_time8 : 0.000e+00
: 278 : vars_time8 : 0.000e+00
: 279 : vars_time8 : 0.000e+00
: 280 : vars_time8 : 0.000e+00
: 281 : vars_time8 : 0.000e+00
: 282 : vars_time8 : 0.000e+00
: 283 : vars_time8 : 0.000e+00
: 284 : vars_time8 : 0.000e+00
: 285 : vars_time8 : 0.000e+00
: 286 : vars_time9 : 0.000e+00
: 287 : vars_time9 : 0.000e+00
: 288 : vars_time9 : 0.000e+00
: 289 : vars_time9 : 0.000e+00
: 290 : vars_time9 : 0.000e+00
: 291 : vars_time9 : 0.000e+00
: 292 : vars_time9 : 0.000e+00
: 293 : vars_time9 : 0.000e+00
: 294 : vars_time9 : 0.000e+00
: 295 : vars_time9 : 0.000e+00
: 296 : vars_time9 : 0.000e+00
: 297 : vars_time9 : 0.000e+00
: 298 : vars_time9 : 0.000e+00
: 299 : vars_time9 : 0.000e+00
: 300 : vars_time9 : 0.000e+00
: --------------------------------------------
TH1.Print Name = TrainingHistory_TMVA_LSTM_trainingError, Entries= 0, Total sum= 6.54167
TH1.Print Name = TrainingHistory_TMVA_LSTM_valError, Entries= 0, Total sum= 6.63402
TH1.Print Name = TrainingHistory_TMVA_DNN_trainingError, Entries= 0, Total sum= 13.6404
TH1.Print Name = TrainingHistory_TMVA_DNN_valError, Entries= 0, Total sum= 13.7638
TH1.Print Name = TrainingHistory_PyKeras_LSTM_'accuracy', Entries= 0, Total sum= 6.92539
TH1.Print Name = TrainingHistory_PyKeras_LSTM_'loss', Entries= 0, Total sum= 5.70292
TH1.Print Name = TrainingHistory_PyKeras_LSTM_'val_accuracy', Entries= 0, Total sum= 6.8375
TH1.Print Name = TrainingHistory_PyKeras_LSTM_'val_loss', Entries= 0, Total sum= 5.81788
Factory : === Destroy and recreate all methods via weight files for testing ===
:
: Reading weight file: ␛[0;36mdataset/weights/TMVAClassification_TMVA_LSTM.weights.xml␛[0m
: Reading weight file: ␛[0;36mdataset/weights/TMVAClassification_TMVA_DNN.weights.xml␛[0m
: Reading weight file: ␛[0;36mdataset/weights/TMVAClassification_PyKeras_LSTM.weights.xml␛[0m
: Reading weight file: ␛[0;36mdataset/weights/TMVAClassification_BDTG.weights.xml␛[0m
Factory : ␛[1mTest all methods␛[0m
Factory : Test method: TMVA_LSTM for Classification performance
:
: Evaluate deep neural network on CPU using batches with size = 800
:
TMVA_LSTM : [dataset] : Evaluation of TMVA_LSTM on testing sample (800 events)
: Elapsed time for evaluation of 800 events: 0.0496 sec
Factory : Test method: TMVA_DNN for Classification performance
:
: Evaluate deep neural network on CPU using batches with size = 800
:
TMVA_DNN : [dataset] : Evaluation of TMVA_DNN on testing sample (800 events)
: Elapsed time for evaluation of 800 events: 0.0201 sec
Factory : Test method: PyKeras_LSTM for Classification performance
:
: Setting up tf.keras
: Using TensorFlow version 2
: Use Keras version from TensorFlow : tf.keras
: Applying GPU option: gpu_options.allow_growth=True
: Disabled TF eager execution when evaluating model
: Loading Keras Model
: Loaded model from file: trained_model_LSTM.h5
PyKeras_LSTM : [dataset] : Evaluation of PyKeras_LSTM on testing sample (800 events)
: Elapsed time for evaluation of 800 events: 0.235 sec
Factory : Test method: BDTG for Classification performance
:
BDTG : [dataset] : Evaluation of BDTG on testing sample (800 events)
: Elapsed time for evaluation of 800 events: 0.00612 sec
Factory : ␛[1mEvaluate all methods␛[0m
Factory : Evaluate classifier: TMVA_LSTM
:
TMVA_LSTM : [dataset] : Loop over test events and fill histograms with classifier response...
:
: Evaluate deep neural network on CPU using batches with size = 1000
:
: Dataset[dataset] : variable plots are not produces ! The number of variables is 300 , it is larger than 200
Factory : Evaluate classifier: TMVA_DNN
:
TMVA_DNN : [dataset] : Loop over test events and fill histograms with classifier response...
:
: Evaluate deep neural network on CPU using batches with size = 1000
:
: Dataset[dataset] : variable plots are not produces ! The number of variables is 300 , it is larger than 200
Factory : Evaluate classifier: PyKeras_LSTM
:
PyKeras_LSTM : [dataset] : Loop over test events and fill histograms with classifier response...
:
: Dataset[dataset] : variable plots are not produces ! The number of variables is 300 , it is larger than 200
Factory : Evaluate classifier: BDTG
:
BDTG : [dataset] : Loop over test events and fill histograms with classifier response...
:
: Dataset[dataset] : variable plots are not produces ! The number of variables is 300 , it is larger than 200
:
: Evaluation results ranked by best signal efficiency and purity (area)
: -------------------------------------------------------------------------------------------------------------------
: DataSet MVA
: Name: Method: ROC-integ
: dataset PyKeras_LSTM : 0.872
: dataset BDTG : 0.836
: dataset TMVA_LSTM : 0.713
: dataset TMVA_DNN : 0.641
: -------------------------------------------------------------------------------------------------------------------
:
: Testing efficiency compared to training efficiency (overtraining check)
: -------------------------------------------------------------------------------------------------------------------
: DataSet MVA Signal efficiency: from test sample (from training sample)
: Name: Method: @B=0.01 @B=0.10 @B=0.30
: -------------------------------------------------------------------------------------------------------------------
: dataset PyKeras_LSTM : 0.305 (0.255) 0.650 (0.663) 0.845 (0.864)
: dataset BDTG : 0.180 (0.315) 0.552 (0.647) 0.813 (0.861)
: dataset TMVA_LSTM : 0.060 (0.060) 0.315 (0.341) 0.605 (0.635)
: dataset TMVA_DNN : 0.065 (0.041) 0.219 (0.182) 0.505 (0.435)
: -------------------------------------------------------------------------------------------------------------------
:
Dataset:dataset : Created tree 'TestTree' with 800 events
:
Dataset:dataset : Created tree 'TrainTree' with 3200 events
:
Factory : ␛[1mThank you for using TMVA!␛[0m
: ␛[1mFor citation information, please visit: http://tmva.sf.net/citeTMVA.html␛[0m
nthreads = 4
# TMVA Classification Example Using a Recurrent Neural Network
# This is an example of using a RNN in TMVA.
# We do the classification using a toy data set containing a time series of data sample ntimes
# and with dimension ndim that is generated when running the provided function `MakeTimeData (nevents, ntime, ndim)`
import ROOT
num_threads = 4 # use max 4 threads
# do enable MT running
if "imt" in ROOT.gROOT.GetConfigFeatures():
ROOT.EnableImplicitMT(num_threads)
# switch off MT in OpenBLAS to avoid conflict with tbb
ROOT.gSystem.Setenv("OMP_NUM_THREADS", "1")
print("Running with nthreads = {}".format(ROOT.GetThreadPoolSize()))
else:
print("Running in serial mode since ROOT does not support MT")
TMVA = ROOT.TMVA
TFile = ROOT.TFile
import os
import importlib
## Helper function to generate the time data set
## make some time data but not of fixed length.
## use a poisson with mu = 5 and truncated at 10
def MakeTimeData(n, ntime, ndim):
# ntime = 10;
# ndim = 30; // number of dim/time
fname = "time_data_t" + str(ntime) + "_d" + str(ndim) + ".root"
v1 = []
v2 = []
for i in range(ntime):
v1.append(ROOT.TH1D("h1_" + str(i), "h1", ndim, 0, 10))
v2.append(ROOT.TH1D("h2_" + str(i), "h2", ndim, 0, 10))
f1 = ROOT.TF1("f1", "gaus")
f2 = ROOT.TF1("f2", "gaus")
sgn = ROOT.TTree("sgn", "sgn")
bkg = ROOT.TTree("bkg", "bkg")
f = TFile(fname, "RECREATE")
x1 = []
x2 = []
for i in range(ntime):
x1.append(ROOT.std.vector["float"](ndim))
x2.append(ROOT.std.vector["float"](ndim))
for i in range(ntime):
bkg.Branch("vars_time" + str(i), "std::vector<float>", x1[i])
sgn.Branch("vars_time" + str(i), "std::vector<float>", x2[i])
sgn.SetDirectory(f)
bkg.SetDirectory(f)
ROOT.gRandom.SetSeed(0)
mean1 = ROOT.std.vector["double"](ntime)
mean2 = ROOT.std.vector["double"](ntime)
sigma1 = ROOT.std.vector["double"](ntime)
sigma2 = ROOT.std.vector["double"](ntime)
for j in range(ntime):
mean1[j] = 5.0 + 0.2 * ROOT.TMath.Sin(ROOT.TMath.Pi() * j / float(ntime))
mean2[j] = 5.0 + 0.2 * ROOT.TMath.Cos(ROOT.TMath.Pi() * j / float(ntime))
sigma1[j] = 4 + 0.3 * ROOT.TMath.Sin(ROOT.TMath.Pi() * j / float(ntime))
sigma2[j] = 4 + 0.3 * ROOT.TMath.Cos(ROOT.TMath.Pi() * j / float(ntime))
for i in range(n):
if i % 1000 == 0:
print("Generating event ... %d", i)
for j in range(ntime):
h1 = v1[j]
h2 = v2[j]
h1.Reset()
h2.Reset()
f1.SetParameters(1, mean1[j], sigma1[j])
f2.SetParameters(1, mean2[j], sigma2[j])
h1.FillRandom("f1", 1000)
h2.FillRandom("f2", 1000)
for k in range(ntime):
# std::cout << j*10+k << " ";
x1[j][k] = h1.GetBinContent(k + 1) + ROOT.gRandom.Gaus(0, 10)
x2[j][k] = h2.GetBinContent(k + 1) + ROOT.gRandom.Gaus(0, 10)
sgn.Fill()
bkg.Fill()
if n == 1:
c1 = ROOT.TCanvas()
c1.Divide(ntime, 2)
for j in range(ntime):
c1.cd(j + 1)
v1[j].Draw()
for j in range(ntime):
c1.cd(ntime + j + 1)
v2[j].Draw()
ROOT.gPad.Update()
if n > 1:
sgn.Write()
bkg.Write()
sgn.Print()
bkg.Print()
f.Close()
## macro for performing a classification using a Recurrent Neural Network
## @param use_type
## use_type = 0 use Simple RNN network
## use_type = 1 use LSTM network
## use_type = 2 use GRU
## use_type = 3 build 3 different networks with RNN, LSTM and GRU
use_type = 1
ninput = 30
ntime = 10
batchSize = 100
maxepochs = 10
nTotEvts = 2000 # total events to be generated for signal or background
useKeras = True
useTMVA_RNN = True
useTMVA_DNN = True
useTMVA_BDT = False
tf_spec = importlib.util.find_spec("tensorflow")
if tf_spec is None:
useKeras = False
ROOT.Warning("TMVA_RNN_Classificaton","Skip using Keras since tensorflow is not installed")
rnn_types = ["RNN", "LSTM", "GRU"]
use_rnn_type = [1, 1, 1]
if 0 <= use_type < 3:
use_rnn_type = [0, 0, 0]
use_rnn_type[use_type] = 1
useGPU = True # use GPU for TMVA if available
useGPU = "tmva-gpu" in ROOT.gROOT.GetConfigFeatures()
useTMVA_RNN = ("tmva-cpu" in ROOT.gROOT.GetConfigFeatures()) or useGPU
if useTMVA_RNN:
ROOT.Warning(
"TMVA_RNN_Classification",
"TMVA is not build with GPU or CPU multi-thread support. Cannot use TMVA Deep Learning for RNN",
)
archString = "GPU" if useGPU else "CPU"
writeOutputFile = True
rnn_type = "RNN"
if "tmva-pymva" in ROOT.gROOT.GetConfigFeatures():
else:
useKeras = False
inputFileName = "time_data_t10_d30.root"
fileDoesNotExist = ROOT.gSystem.AccessPathName(inputFileName)
# if file does not exists create it
if fileDoesNotExist:
MakeTimeData(nTotEvts, ntime, ninput)
inputFile = TFile.Open(inputFileName)
if inputFile is None:
raise ROOT.Error("Error opening input file %s - exit", inputFileName.Data())
print("--- RNNClassification : Using input file: {}".format(inputFile.GetName()))
# Create a ROOT output file where TMVA will store ntuples, histograms, etc.
outfileName = "data_RNN_" + archString + ".root"
outputFile = None
if writeOutputFile:
outputFile = TFile.Open(outfileName, "RECREATE")
## Declare Factory
# Create the Factory class. Later you can choose the methods
# whose performance you'd like to investigate.
# The factory is the major TMVA object you have to interact with. Here is the list of parameters you need to
# pass
# - The first argument is the base of the name of all the output
# weightfiles in the directory weight/ that will be created with the
# method parameters
# - The second argument is the output file for the training results
#
# - The third argument is a string option defining some general configuration for the TMVA session.
# For example all TMVA output can be suppressed by removing the "!" (not) in front of the "Silent" argument in
# the option string
# // Creating the factory object
factory = TMVA.Factory(
"TMVAClassification",
outputFile,
V=False,
Silent=False,
Color=True,
DrawProgressBar=True,
Transformations=None,
Correlations=False,
AnalysisType="Classification",
ModelPersistence=True,
)
dataloader = TMVA.DataLoader("dataset")
signalTree = inputFile.Get("sgn")
background = inputFile.Get("bkg")
nvar = ninput * ntime
## add variables - use new AddVariablesArray function
for i in range(ntime):
dataloader.AddVariablesArray("vars_time" + str(i), ninput)
dataloader.AddSignalTree(signalTree, 1.0)
dataloader.AddBackgroundTree(background, 1.0)
# check given input
datainfo = dataloader.GetDataSetInfo()
vars = datainfo.GetListOfVariables()
print("number of variables is {}".format(vars.size()))
for v in vars:
print(v)
nTrainSig = 0.8 * nTotEvts
nTrainBkg = 0.8 * nTotEvts
# Apply additional cuts on the signal and background samples (can be different)
mycuts = "" # for example: TCut mycuts = "abs(var1)<0.5 && abs(var2-0.5)<1";
mycutb = ""
# build the string options for DataLoader::PrepareTrainingAndTestTree
dataloader.PrepareTrainingAndTestTree(
mycuts,
mycutb,
nTrain_Signal=nTrainSig,
nTrain_Background=nTrainBkg,
SplitMode="Random",
SplitSeed=100,
NormMode="NumEvents",
V=False,
CalcCorrelations=False,
)
print("prepared DATA LOADER ")
## Book TMVA recurrent models
# Book the different types of recurrent models in TMVA (SimpleRNN, LSTM or GRU)
if useTMVA_RNN:
for i in range(3):
if not use_rnn_type[i]:
continue
rnn_type = rnn_types[i]
## Define RNN layer layout
## it should be LayerType (RNN or LSTM or GRU) | number of units | number of inputs | time steps | remember output (typically no=0 | return full sequence
rnnLayout = str(rnn_type) + "|10|" + str(ninput) + "|" + str(ntime) + "|0|1,RESHAPE|FLAT,DENSE|64|TANH,LINEAR"
## Defining Training strategies. Different training strings can be concatenate. Use however only one
trainingString1 = "LearningRate=1e-3,Momentum=0.0,Repetitions=1,ConvergenceSteps=5,BatchSize=" + str(batchSize)
trainingString1 += ",TestRepetitions=1,WeightDecay=1e-2,Regularization=None,MaxEpochs=" + str(maxepochs)
trainingString1 += "Optimizer=ADAM,DropConfig=0.0+0.+0.+0."
## define the inputlayout string for RNN
## the input data should be organize as following:
##/ input layout for RNN: time x ndim
## add after RNN a reshape layer (needed top flatten the output) and a dense layer with 64 units and a last one
## Note the last layer is linear because when using Crossentropy a Sigmoid is applied already
## Define the full RNN Noption string adding the final options for all network
rnnName = "TMVA_" + str(rnn_type)
factory.BookMethod(
dataloader,
TMVA.Types.kDL,
rnnName,
H=False,
V=True,
ErrorStrategy="CROSSENTROPY",
VarTransform=None,
WeightInitialization="XAVIERUNIFORM",
ValidationSize=0.2,
RandomSeed=1234,
InputLayout=str(ntime) + "|" + str(ninput),
Layout=rnnLayout,
TrainingStrategy=trainingString1,
Architecture=archString
)
## Book TMVA fully connected dense layer models
if useTMVA_DNN:
# Method DL with Dense Layer
# Training strategies.
trainingString1 = ROOT.TString(
"LearningRate=1e-3,Momentum=0.0,Repetitions=1,"
"ConvergenceSteps=10,BatchSize=256,TestRepetitions=1,"
"WeightDecay=1e-4,Regularization=None,MaxEpochs=20"
"DropConfig=0.0+0.+0.+0.,Optimizer=ADAM:"
) # + "|" + trainingString2
# General Options.
trainingString1.Append(archString)
dnnName = "TMVA_DNN"
factory.BookMethod(
dataloader,
TMVA.Types.kDL,
dnnName,
H=False,
V=True,
ErrorStrategy="CROSSENTROPY",
VarTransform=None,
WeightInitialization="XAVIER",
RandomSeed=0,
InputLayout="1|1|" + str(ntime * ninput),
Layout="DENSE|64|TANH,DENSE|TANH|64,DENSE|TANH|64,LINEAR",
TrainingStrategy=trainingString1
)
## Book Keras recurrent models
# Book the different types of recurrent models in Keras (SimpleRNN, LSTM or GRU)
if useKeras:
for i in range(3):
if use_rnn_type[i]:
modelName = "model_" + rnn_types[i] + ".h5"
trainedModelName = "trained_" + modelName
print("Building recurrent keras model using a", rnn_types[i], "layer")
# create python script which can be executed
# create 2 conv2d layer + maxpool + dense
from tensorflow.keras.models import Sequential
from tensorflow.keras.optimizers import Adam
# from keras.initializers import TruncatedNormal
# from keras import initializations
from tensorflow.keras.layers import Input, Dense, Dropout, Flatten, SimpleRNN, GRU, LSTM, Reshape, BatchNormalization
model = Sequential()
model.add(Reshape((10, 30), input_shape=(10 * 30,)))
# add recurrent neural network depending on type / Use option to return the full output
if rnn_types[i] == "LSTM":
model.add(LSTM(units=10, return_sequences=True))
elif rnn_types[i] == "GRU":
model.add(GRU(units=10, return_sequences=True))
else:
model.add(SimpleRNN(units=10, return_sequences=True))
# m.AddLine("model.add(BatchNormalization())");
model.add(Flatten()) # needed if returning the full time output sequence
model.add(Dense(64, activation="tanh"))
model.add(Dense(2, activation="sigmoid"))
model.compile(loss="binary_crossentropy", optimizer=Adam(learning_rate=0.001), weighted_metrics=["accuracy"])
model.save(modelName)
model.summary()
print("saved recurrent model", modelName)
if not os.path.exists(modelName):
useKeras = False
print("Error creating Keras recurrent model file - Skip using Keras")
else:
# book PyKeras method only if Keras model could be created
print("Booking Keras model ", rnn_types[i])
factory.BookMethod(
dataloader,
TMVA.Types.kPyKeras,
"PyKeras_" + rnn_types[i],
H=True,
V=False,
VarTransform=None,
FilenameModel=modelName,
FilenameTrainedModel="trained_" + modelName,
NumEpochs=maxepochs,
BatchSize=batchSize,
GpuOptions="allow_growth=True",
)
# use BDT in case not using Keras or TMVA DL
if not useKeras or not useTMVA_BDT:
useTMVA_BDT = True
## Book TMVA BDT
if useTMVA_BDT:
factory.BookMethod(
dataloader,
TMVA.Types.kBDT,
"BDTG",
H=True,
V=False,
NTrees=100,
MinNodeSize="2.5%",
BoostType="Grad",
Shrinkage=0.10,
UseBaggedBoost=True,
BaggedSampleFraction=0.5,
nCuts=20,
MaxDepth=2,
)
## Train all methods
factory.TrainAllMethods()
print("nthreads = {}".format(ROOT.GetThreadPoolSize()))
# ---- Evaluate all MVAs using the set of test events
factory.TestAllMethods()
# ----- Evaluate and compare performance of all configured MVAs
factory.EvaluateAllMethods()
# check method
# plot ROC curve
c1 = factory.GetROCCurve(dataloader)
c1.Draw()
if outputFile:
outputFile.Close()
Option_t Option_t TPoint TPoint const char GetTextMagnitude GetFillStyle GetLineColor GetLineWidth GetMarkerStyle GetTextAlign GetTextColor GetTextSize void char Point_t Rectangle_t WindowAttributes_t Float_t Float_t Float_t Int_t Int_t UInt_t UInt_t Rectangle_t Int_t Int_t Window_t TString Int_t GCValues_t GetPrimarySelectionOwner GetDisplay GetScreen GetColormap GetNativeEvent const char const char dpyName wid window const char font_name cursor keysym reg const char only_if_exist regb h Point_t winding char text const char depth char const char Int_t count const char ColorStruct_t color const char Pixmap_t Pixmap_t PictureAttributes_t attr const char char ret_data h unsigned char height h Atom_t Int_t ULong_t ULong_t unsigned char prop_list Atom_t Atom_t Atom_t Time_t format
A ROOT file is composed of a header, followed by consecutive data records (TKey instances) with a wel...
Definition TFile.h:53
static TFile * Open(const char *name, Option_t *option="", const char *ftitle="", Int_t compress=ROOT::RCompressionSetting::EDefaults::kUseCompiledDefault, Int_t netopt=0)
Create / open a file.
Definition TFile.cxx:4070
static Config & Instance()
static function: returns TMVA instance
Definition Config.cxx:98
This is the main MVA steering class.
Definition Factory.h:80
static void PyInitialize()
Initialize Python interpreter.
static Tools & Instance()
Definition Tools.cxx:71
void EnableImplicitMT(UInt_t numthreads=0)
Enable ROOT's implicit multi-threading for all objects and methods that provide an internal paralleli...
Definition TROOT.cxx:537
UInt_t GetThreadPoolSize()
Returns the size of ROOT's thread pool.
Definition TROOT.cxx:575
th1 Draw()
Author
Harshal Shende

Definition in file TMVA_RNN_Classification.py.