Logo ROOT  

Reference Guide
 
Loading...
Searching...
No Matches
TMVA_RNN_Classification.C File Reference
Tutorials » TMVA tutorials

Detailed Description

View in nbviewer Open in SWAN
TMVA Classification Example Using a Recurrent Neural Network

This is an example of using a RNN in TMVA. We do classification using a toy time dependent data set that is generated when running this example macro

Running with nthreads = 4
--- RNNClassification : Using input file: time_data_t10_d30.root
DataSetInfo : [dataset] : Added class "Signal"
: Add Tree sgn of type Signal with 2000 events
DataSetInfo : [dataset] : Added class "Background"
: Add Tree bkg of type Background with 2000 events
number of variables is 300
vars_time0[0],vars_time0[1],vars_time0[2],vars_time0[3],vars_time0[4],vars_time0[5],vars_time0[6],vars_time0[7],vars_time0[8],vars_time0[9],vars_time0[10],vars_time0[11],vars_time0[12],vars_time0[13],vars_time0[14],vars_time0[15],vars_time0[16],vars_time0[17],vars_time0[18],vars_time0[19],vars_time0[20],vars_time0[21],vars_time0[22],vars_time0[23],vars_time0[24],vars_time0[25],vars_time0[26],vars_time0[27],vars_time0[28],vars_time0[29],vars_time1[0],vars_time1[1],vars_time1[2],vars_time1[3],vars_time1[4],vars_time1[5],vars_time1[6],vars_time1[7],vars_time1[8],vars_time1[9],vars_time1[10],vars_time1[11],vars_time1[12],vars_time1[13],vars_time1[14],vars_time1[15],vars_time1[16],vars_time1[17],vars_time1[18],vars_time1[19],vars_time1[20],vars_time1[21],vars_time1[22],vars_time1[23],vars_time1[24],vars_time1[25],vars_time1[26],vars_time1[27],vars_time1[28],vars_time1[29],vars_time2[0],vars_time2[1],vars_time2[2],vars_time2[3],vars_time2[4],vars_time2[5],vars_time2[6],vars_time2[7],vars_time2[8],vars_time2[9],vars_time2[10],vars_time2[11],vars_time2[12],vars_time2[13],vars_time2[14],vars_time2[15],vars_time2[16],vars_time2[17],vars_time2[18],vars_time2[19],vars_time2[20],vars_time2[21],vars_time2[22],vars_time2[23],vars_time2[24],vars_time2[25],vars_time2[26],vars_time2[27],vars_time2[28],vars_time2[29],vars_time3[0],vars_time3[1],vars_time3[2],vars_time3[3],vars_time3[4],vars_time3[5],vars_time3[6],vars_time3[7],vars_time3[8],vars_time3[9],vars_time3[10],vars_time3[11],vars_time3[12],vars_time3[13],vars_time3[14],vars_time3[15],vars_time3[16],vars_time3[17],vars_time3[18],vars_time3[19],vars_time3[20],vars_time3[21],vars_time3[22],vars_time3[23],vars_time3[24],vars_time3[25],vars_time3[26],vars_time3[27],vars_time3[28],vars_time3[29],vars_time4[0],vars_time4[1],vars_time4[2],vars_time4[3],vars_time4[4],vars_time4[5],vars_time4[6],vars_time4[7],vars_time4[8],vars_time4[9],vars_time4[10],vars_time4[11],vars_time4[12],vars_time4[13],vars_time4[14],vars_time4[15],vars_time4[16],vars_time4[17],vars_time4[18],vars_time4[19],vars_time4[20],vars_time4[21],vars_time4[22],vars_time4[23],vars_time4[24],vars_time4[25],vars_time4[26],vars_time4[27],vars_time4[28],vars_time4[29],vars_time5[0],vars_time5[1],vars_time5[2],vars_time5[3],vars_time5[4],vars_time5[5],vars_time5[6],vars_time5[7],vars_time5[8],vars_time5[9],vars_time5[10],vars_time5[11],vars_time5[12],vars_time5[13],vars_time5[14],vars_time5[15],vars_time5[16],vars_time5[17],vars_time5[18],vars_time5[19],vars_time5[20],vars_time5[21],vars_time5[22],vars_time5[23],vars_time5[24],vars_time5[25],vars_time5[26],vars_time5[27],vars_time5[28],vars_time5[29],vars_time6[0],vars_time6[1],vars_time6[2],vars_time6[3],vars_time6[4],vars_time6[5],vars_time6[6],vars_time6[7],vars_time6[8],vars_time6[9],vars_time6[10],vars_time6[11],vars_time6[12],vars_time6[13],vars_time6[14],vars_time6[15],vars_time6[16],vars_time6[17],vars_time6[18],vars_time6[19],vars_time6[20],vars_time6[21],vars_time6[22],vars_time6[23],vars_time6[24],vars_time6[25],vars_time6[26],vars_time6[27],vars_time6[28],vars_time6[29],vars_time7[0],vars_time7[1],vars_time7[2],vars_time7[3],vars_time7[4],vars_time7[5],vars_time7[6],vars_time7[7],vars_time7[8],vars_time7[9],vars_time7[10],vars_time7[11],vars_time7[12],vars_time7[13],vars_time7[14],vars_time7[15],vars_time7[16],vars_time7[17],vars_time7[18],vars_time7[19],vars_time7[20],vars_time7[21],vars_time7[22],vars_time7[23],vars_time7[24],vars_time7[25],vars_time7[26],vars_time7[27],vars_time7[28],vars_time7[29],vars_time8[0],vars_time8[1],vars_time8[2],vars_time8[3],vars_time8[4],vars_time8[5],vars_time8[6],vars_time8[7],vars_time8[8],vars_time8[9],vars_time8[10],vars_time8[11],vars_time8[12],vars_time8[13],vars_time8[14],vars_time8[15],vars_time8[16],vars_time8[17],vars_time8[18],vars_time8[19],vars_time8[20],vars_time8[21],vars_time8[22],vars_time8[23],vars_time8[24],vars_time8[25],vars_time8[26],vars_time8[27],vars_time8[28],vars_time8[29],vars_time9[0],vars_time9[1],vars_time9[2],vars_time9[3],vars_time9[4],vars_time9[5],vars_time9[6],vars_time9[7],vars_time9[8],vars_time9[9],vars_time9[10],vars_time9[11],vars_time9[12],vars_time9[13],vars_time9[14],vars_time9[15],vars_time9[16],vars_time9[17],vars_time9[18],vars_time9[19],vars_time9[20],vars_time9[21],vars_time9[22],vars_time9[23],vars_time9[24],vars_time9[25],vars_time9[26],vars_time9[27],vars_time9[28],vars_time9[29],
prepared DATA LOADER
Factory : Booking method: ␛[1mTMVA_LSTM␛[0m
:
: Parsing option string:
: ... "!H:V:ErrorStrategy=CROSSENTROPY:VarTransform=None:WeightInitialization=XAVIERUNIFORM:ValidationSize=0.2:RandomSeed=1234:InputLayout=10|30:Layout=LSTM|10|30|10|0|1,RESHAPE|FLAT,DENSE|64|TANH,LINEAR:TrainingStrategy=LearningRate=1e-3,Momentum=0.0,Repetitions=1,ConvergenceSteps=5,BatchSize=100,TestRepetitions=1,WeightDecay=1e-2,Regularization=None,MaxEpochs=20,Optimizer=ADAM,DropConfig=0.0+0.+0.+0.:Architecture=CPU"
: The following options are set:
: - By User:
: <none>
: - Default:
: Boost_num: "0" [Number of times the classifier will be boosted]
: Parsing option string:
: ... "!H:V:ErrorStrategy=CROSSENTROPY:VarTransform=None:WeightInitialization=XAVIERUNIFORM:ValidationSize=0.2:RandomSeed=1234:InputLayout=10|30:Layout=LSTM|10|30|10|0|1,RESHAPE|FLAT,DENSE|64|TANH,LINEAR:TrainingStrategy=LearningRate=1e-3,Momentum=0.0,Repetitions=1,ConvergenceSteps=5,BatchSize=100,TestRepetitions=1,WeightDecay=1e-2,Regularization=None,MaxEpochs=20,Optimizer=ADAM,DropConfig=0.0+0.+0.+0.:Architecture=CPU"
: The following options are set:
: - By User:
: V: "True" [Verbose output (short form of "VerbosityLevel" below - overrides the latter one)]
: VarTransform: "None" [List of variable transformations performed before training, e.g., "D_Background,P_Signal,G,N_AllClasses" for: "Decorrelation, PCA-transformation, Gaussianisation, Normalisation, each for the given class of events ('AllClasses' denotes all events of all classes, if no class indication is given, 'All' is assumed)"]
: H: "False" [Print method-specific help message]
: InputLayout: "10|30" [The Layout of the input]
: Layout: "LSTM|10|30|10|0|1,RESHAPE|FLAT,DENSE|64|TANH,LINEAR" [Layout of the network.]
: ErrorStrategy: "CROSSENTROPY" [Loss function: Mean squared error (regression) or cross entropy (binary classification).]
: WeightInitialization: "XAVIERUNIFORM" [Weight initialization strategy]
: RandomSeed: "1234" [Random seed used for weight initialization and batch shuffling]
: ValidationSize: "0.2" [Part of the training data to use for validation. Specify as 0.2 or 20% to use a fifth of the data set as validation set. Specify as 100 to use exactly 100 events. (Default: 20%)]
: Architecture: "CPU" [Which architecture to perform the training on.]
: TrainingStrategy: "LearningRate=1e-3,Momentum=0.0,Repetitions=1,ConvergenceSteps=5,BatchSize=100,TestRepetitions=1,WeightDecay=1e-2,Regularization=None,MaxEpochs=20,Optimizer=ADAM,DropConfig=0.0+0.+0.+0." [Defines the training strategies.]
: - Default:
: VerbosityLevel: "Default" [Verbosity level]
: CreateMVAPdfs: "False" [Create PDFs for classifier outputs (signal and background)]
: IgnoreNegWeightsInTraining: "False" [Events with negative weights are ignored in the training (but are included for testing and performance evaluation)]
: BatchLayout: "0|0|0" [The Layout of the batch]
: Will now use the CPU architecture with BLAS and IMT support !
Factory : Booking method: ␛[1mTMVA_DNN␛[0m
:
: Parsing option string:
: ... "!H:V:ErrorStrategy=CROSSENTROPY:VarTransform=None:WeightInitialization=XAVIER:RandomSeed=0:InputLayout=1|1|300:Layout=DENSE|64|TANH,DENSE|TANH|64,DENSE|TANH|64,LINEAR:TrainingStrategy=LearningRate=1e-3,Momentum=0.0,Repetitions=1,ConvergenceSteps=10,BatchSize=256,TestRepetitions=1,WeightDecay=1e-4,Regularization=None,MaxEpochs=20DropConfig=0.0+0.+0.+0.,Optimizer=ADAM:CPU"
: The following options are set:
: - By User:
: <none>
: - Default:
: Boost_num: "0" [Number of times the classifier will be boosted]
: Parsing option string:
: ... "!H:V:ErrorStrategy=CROSSENTROPY:VarTransform=None:WeightInitialization=XAVIER:RandomSeed=0:InputLayout=1|1|300:Layout=DENSE|64|TANH,DENSE|TANH|64,DENSE|TANH|64,LINEAR:TrainingStrategy=LearningRate=1e-3,Momentum=0.0,Repetitions=1,ConvergenceSteps=10,BatchSize=256,TestRepetitions=1,WeightDecay=1e-4,Regularization=None,MaxEpochs=20DropConfig=0.0+0.+0.+0.,Optimizer=ADAM:CPU"
: The following options are set:
: - By User:
: V: "True" [Verbose output (short form of "VerbosityLevel" below - overrides the latter one)]
: VarTransform: "None" [List of variable transformations performed before training, e.g., "D_Background,P_Signal,G,N_AllClasses" for: "Decorrelation, PCA-transformation, Gaussianisation, Normalisation, each for the given class of events ('AllClasses' denotes all events of all classes, if no class indication is given, 'All' is assumed)"]
: H: "False" [Print method-specific help message]
: InputLayout: "1|1|300" [The Layout of the input]
: Layout: "DENSE|64|TANH,DENSE|TANH|64,DENSE|TANH|64,LINEAR" [Layout of the network.]
: ErrorStrategy: "CROSSENTROPY" [Loss function: Mean squared error (regression) or cross entropy (binary classification).]
: WeightInitialization: "XAVIER" [Weight initialization strategy]
: RandomSeed: "0" [Random seed used for weight initialization and batch shuffling]
: Architecture: "CPU" [Which architecture to perform the training on.]
: TrainingStrategy: "LearningRate=1e-3,Momentum=0.0,Repetitions=1,ConvergenceSteps=10,BatchSize=256,TestRepetitions=1,WeightDecay=1e-4,Regularization=None,MaxEpochs=20DropConfig=0.0+0.+0.+0.,Optimizer=ADAM" [Defines the training strategies.]
: - Default:
: VerbosityLevel: "Default" [Verbosity level]
: CreateMVAPdfs: "False" [Create PDFs for classifier outputs (signal and background)]
: IgnoreNegWeightsInTraining: "False" [Events with negative weights are ignored in the training (but are included for testing and performance evaluation)]
: BatchLayout: "0|0|0" [The Layout of the batch]
: ValidationSize: "20%" [Part of the training data to use for validation. Specify as 0.2 or 20% to use a fifth of the data set as validation set. Specify as 100 to use exactly 100 events. (Default: 20%)]
: Will now use the CPU architecture with BLAS and IMT support !
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
reshape (Reshape) (None, 10, 30) 0
lstm (LSTM) (None, 10, 10) 1640
flatten (Flatten) (None, 100) 0
dense (Dense) (None, 64) 6464
dense_1 (Dense) (None, 2) 130
=================================================================
Total params: 8234 (32.16 KB)
Trainable params: 8234 (32.16 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
(TString) "python3"[7]
Factory : Booking method: ␛[1mPyKeras_LSTM␛[0m
:
: Setting up tf.keras
: Using TensorFlow version 2
: Use Keras version from TensorFlow : tf.keras
: Applying GPU option: gpu_options.allow_growth=True
: Loading Keras Model
: Loaded model from file: model_LSTM.h5
Factory : Booking method: ␛[1mBDTG␛[0m
:
: the option NegWeightTreatment=InverseBoostNegWeights does not exist for BoostType=Grad
: --> change to new default NegWeightTreatment=Pray
: Rebuilding Dataset dataset
: Building event vectors for type 2 Signal
: Dataset[dataset] : create input formulas for tree sgn
: Using variable vars_time0[0] from array expression vars_time0 of size 30
: Using variable vars_time1[0] from array expression vars_time1 of size 30
: Using variable vars_time2[0] from array expression vars_time2 of size 30
: Using variable vars_time3[0] from array expression vars_time3 of size 30
: Using variable vars_time4[0] from array expression vars_time4 of size 30
: Using variable vars_time5[0] from array expression vars_time5 of size 30
: Using variable vars_time6[0] from array expression vars_time6 of size 30
: Using variable vars_time7[0] from array expression vars_time7 of size 30
: Using variable vars_time8[0] from array expression vars_time8 of size 30
: Using variable vars_time9[0] from array expression vars_time9 of size 30
: Building event vectors for type 2 Background
: Dataset[dataset] : create input formulas for tree bkg
: Using variable vars_time0[0] from array expression vars_time0 of size 30
: Using variable vars_time1[0] from array expression vars_time1 of size 30
: Using variable vars_time2[0] from array expression vars_time2 of size 30
: Using variable vars_time3[0] from array expression vars_time3 of size 30
: Using variable vars_time4[0] from array expression vars_time4 of size 30
: Using variable vars_time5[0] from array expression vars_time5 of size 30
: Using variable vars_time6[0] from array expression vars_time6 of size 30
: Using variable vars_time7[0] from array expression vars_time7 of size 30
: Using variable vars_time8[0] from array expression vars_time8 of size 30
: Using variable vars_time9[0] from array expression vars_time9 of size 30
DataSetFactory : [dataset] : Number of events in input trees
:
:
: Number of training and testing events
: ---------------------------------------------------------------------------
: Signal -- training events : 1600
: Signal -- testing events : 400
: Signal -- training and testing events: 2000
: Background -- training events : 1600
: Background -- testing events : 400
: Background -- training and testing events: 2000
:
Factory : ␛[1mTrain all methods␛[0m
Factory : Train method: TMVA_LSTM for Classification
:
: Start of deep neural network training on CPU using MT, nthreads = 4
:
: ***** Deep Learning Network *****
DEEP NEURAL NETWORK: Depth = 4 Input = ( 10, 1, 30 ) Batch size = 100 Loss function = C
Layer 0 LSTM Layer: (NInput = 30, NState = 10, NTime = 10 ) Output = ( 100 , 10 , 10 )
Layer 1 RESHAPE Layer Input = ( 1 , 10 , 10 ) Output = ( 1 , 100 , 100 )
Layer 2 DENSE Layer: ( Input = 100 , Width = 64 ) Output = ( 1 , 100 , 64 ) Activation Function = Tanh
Layer 3 DENSE Layer: ( Input = 64 , Width = 1 ) Output = ( 1 , 100 , 1 ) Activation Function = Identity
: Using 2560 events for training and 640 for testing
: Compute initial loss on the validation data
: Training phase 1 of 1: Optimizer ADAM (beta1=0.9,beta2=0.999,eps=1e-07) Learning rate = 0.001 regularization 0 minimum error = 0.706511
: --------------------------------------------------------------
: Epoch | Train Err. Val. Err. t(s)/epoch t(s)/Loss nEvents/s Conv. Steps
: --------------------------------------------------------------
: Start epoch iteration ...
: 1 Minimum Test error found - save the configuration
: 1 | 0.701708 0.69817 0.652208 0.0430245 4103.85 0
: 2 Minimum Test error found - save the configuration
: 2 | 0.693475 0.695759 0.660752 0.0417971 4039.07 0
: 3 | 0.687753 0.698077 0.658556 0.0416252 4052.32 1
: 4 Minimum Test error found - save the configuration
: 4 | 0.678409 0.691142 0.638542 0.0411255 4184.69 0
: 5 Minimum Test error found - save the configuration
: 5 | 0.666726 0.677079 0.627646 0.0412981 4263.68 0
: 6 Minimum Test error found - save the configuration
: 6 | 0.648882 0.656136 0.599676 0.0400878 4467.57 0
: 7 Minimum Test error found - save the configuration
: 7 | 0.628427 0.646569 0.593822 0.0399642 4513.79 0
: 8 Minimum Test error found - save the configuration
: 8 | 0.613436 0.630771 0.596476 0.0404368 4496.09 0
: 9 Minimum Test error found - save the configuration
: 9 | 0.58733 0.613073 0.608133 0.040516 4404.38 0
: 10 Minimum Test error found - save the configuration
: 10 | 0.56648 0.591079 0.604132 0.0403027 4433.96 0
: 11 Minimum Test error found - save the configuration
: 11 | 0.551819 0.582653 0.59773 0.0399924 4482.39 0
: 12 | 0.539671 0.588457 0.588352 0.039286 4553.19 1
: 13 | 0.526355 0.586975 0.568633 0.0381541 4712.72 2
: 14 Minimum Test error found - save the configuration
: 14 | 0.514895 0.556662 0.554544 0.0387423 4846.83 0
: 15 Minimum Test error found - save the configuration
: 15 | 0.495319 0.536454 0.544486 0.0383503 4939.38 0
: 16 Minimum Test error found - save the configuration
: 16 | 0.479113 0.536217 0.535457 0.0382094 5027.68 0
: 17 Minimum Test error found - save the configuration
: 17 | 0.471206 0.519126 0.532107 0.0382202 5061.89 0
: 18 | 0.455934 0.519491 0.532372 0.0380526 5057.46 1
: 19 Minimum Test error found - save the configuration
: 19 | 0.443945 0.503518 0.533845 0.0382866 5044.81 0
: 20 Minimum Test error found - save the configuration
: 20 | 0.43639 0.500718 0.539201 0.0384222 4992.22 0
:
: Elapsed time for training with 3200 events: 11.8 sec
: Evaluate deep neural network on CPU using batches with size = 100
:
TMVA_LSTM : [dataset] : Evaluation of TMVA_LSTM on training sample (3200 events)
: Elapsed time for evaluation of 3200 events: 0.205 sec
: Creating xml weight file: ␛[0;36mdataset/weights/TMVAClassification_TMVA_LSTM.weights.xml␛[0m
: Creating standalone class: ␛[0;36mdataset/weights/TMVAClassification_TMVA_LSTM.class.C␛[0m
Factory : Training finished
:
Factory : Train method: TMVA_DNN for Classification
:
: Start of deep neural network training on CPU using MT, nthreads = 4
:
: ***** Deep Learning Network *****
DEEP NEURAL NETWORK: Depth = 4 Input = ( 1, 1, 300 ) Batch size = 256 Loss function = C
Layer 0 DENSE Layer: ( Input = 300 , Width = 64 ) Output = ( 1 , 256 , 64 ) Activation Function = Tanh
Layer 1 DENSE Layer: ( Input = 64 , Width = 64 ) Output = ( 1 , 256 , 64 ) Activation Function = Tanh
Layer 2 DENSE Layer: ( Input = 64 , Width = 64 ) Output = ( 1 , 256 , 64 ) Activation Function = Tanh
Layer 3 DENSE Layer: ( Input = 64 , Width = 1 ) Output = ( 1 , 256 , 1 ) Activation Function = Identity
: Using 2560 events for training and 640 for testing
: Compute initial loss on the validation data
: Training phase 1 of 1: Optimizer ADAM (beta1=0.9,beta2=0.999,eps=1e-07) Learning rate = 0.001 regularization 0 minimum error = 1.04941
: --------------------------------------------------------------
: Epoch | Train Err. Val. Err. t(s)/epoch t(s)/Loss nEvents/s Conv. Steps
: --------------------------------------------------------------
: Start epoch iteration ...
: 1 Minimum Test error found - save the configuration
: 1 | 0.869958 0.725553 0.201207 0.0162921 13844.2 0
: 2 Minimum Test error found - save the configuration
: 2 | 0.738276 0.725224 0.201176 0.0162335 13842.2 0
: 3 Minimum Test error found - save the configuration
: 3 | 0.708737 0.698406 0.2011 0.0165245 13869.6 0
: 4 Minimum Test error found - save the configuration
: 4 | 0.688126 0.665377 0.193568 0.0156066 14385.2 0
: 5 | 0.678687 0.681129 0.192416 0.0152716 14451.5 1
: 6 | 0.667565 0.680621 0.192483 0.0153285 14450.7 2
: 7 | 0.65751 0.67596 0.192848 0.0152565 14415.1 3
: 8 | 0.657112 0.679343 0.193272 0.0153278 14386.5 4
: 9 | 0.649843 0.675151 0.199081 0.0154285 13939.4 5
: 10 | 0.658999 0.668509 0.194365 0.0152984 14296.3 6
: 11 | 0.648508 0.682569 0.193279 0.0152966 14383.5 7
: 12 | 0.647254 0.674864 0.192638 0.0152186 14429.1 8
: 13 | 0.638651 0.690955 0.192711 0.0152971 14429.5 9
: 14 | 0.632114 0.666363 0.194642 0.0152683 14271.9 10
: 15 | 0.644693 0.667062 0.19469 0.0163687 14356.1 11
:
: Elapsed time for training with 3200 events: 2.95 sec
: Evaluate deep neural network on CPU using batches with size = 256
:
TMVA_DNN : [dataset] : Evaluation of TMVA_DNN on training sample (3200 events)
: Elapsed time for evaluation of 3200 events: 0.105 sec
: Creating xml weight file: ␛[0;36mdataset/weights/TMVAClassification_TMVA_DNN.weights.xml␛[0m
: Creating standalone class: ␛[0;36mdataset/weights/TMVAClassification_TMVA_DNN.class.C␛[0m
Factory : Training finished
:
Factory : Train method: PyKeras_LSTM for Classification
:
: Split TMVA training data in 2560 training events and 640 validation events
: Training Model Summary
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
reshape (Reshape) (None, 10, 30) 0
lstm (LSTM) (None, 10, 10) 1640
flatten (Flatten) (None, 100) 0
dense (Dense) (None, 64) 6464
dense_1 (Dense) (None, 2) 130
=================================================================
Total params: 8234 (32.16 KB)
Trainable params: 8234 (32.16 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
: Option SaveBestOnly: Only model weights with smallest validation loss will be stored
Epoch 1/20
1/26 [>.............................] - ETA: 43s - loss: 0.7340 - accuracy: 0.5400␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
8/26 [========>.....................] - ETA: 0s - loss: 0.7145 - accuracy: 0.5125 ␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
16/26 [=================>............] - ETA: 0s - loss: 0.7054 - accuracy: 0.5238␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
24/26 [==========================>...] - ETA: 0s - loss: 0.7019 - accuracy: 0.5250
Epoch 1: val_loss improved from inf to 0.69184, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 3s 38ms/step - loss: 0.7012 - accuracy: 0.5223 - val_loss: 0.6918 - val_accuracy: 0.5281
Epoch 2/20
1/26 [>.............................] - ETA: 0s - loss: 0.6811 - accuracy: 0.5500␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
7/26 [=======>......................] - ETA: 0s - loss: 0.6884 - accuracy: 0.5500␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
15/26 [================>.............] - ETA: 0s - loss: 0.6857 - accuracy: 0.5500␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
22/26 [========================>.....] - ETA: 0s - loss: 0.6818 - accuracy: 0.5664
Epoch 2: val_loss improved from 0.69184 to 0.68245, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 11ms/step - loss: 0.6793 - accuracy: 0.5738 - val_loss: 0.6825 - val_accuracy: 0.5641
Epoch 3/20
1/26 [>.............................] - ETA: 0s - loss: 0.6624 - accuracy: 0.6100␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
7/26 [=======>......................] - ETA: 0s - loss: 0.6633 - accuracy: 0.6143␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
12/26 [============>.................] - ETA: 0s - loss: 0.6707 - accuracy: 0.5967␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
20/26 [======================>.......] - ETA: 0s - loss: 0.6657 - accuracy: 0.6055
Epoch 3: val_loss improved from 0.68245 to 0.67275, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 12ms/step - loss: 0.6687 - accuracy: 0.5930 - val_loss: 0.6727 - val_accuracy: 0.5781
Epoch 4/20
1/26 [>.............................] - ETA: 0s - loss: 0.6343 - accuracy: 0.6300␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
8/26 [========>.....................] - ETA: 0s - loss: 0.6552 - accuracy: 0.6225␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
14/26 [===============>..............] - ETA: 0s - loss: 0.6555 - accuracy: 0.6164␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
22/26 [========================>.....] - ETA: 0s - loss: 0.6524 - accuracy: 0.6195
Epoch 4: val_loss improved from 0.67275 to 0.66059, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 11ms/step - loss: 0.6549 - accuracy: 0.6109 - val_loss: 0.6606 - val_accuracy: 0.6156
Epoch 5/20
1/26 [>.............................] - ETA: 0s - loss: 0.6337 - accuracy: 0.6400␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
8/26 [========>.....................] - ETA: 0s - loss: 0.6514 - accuracy: 0.6100␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
17/26 [==================>...........] - ETA: 0s - loss: 0.6351 - accuracy: 0.6412␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
24/26 [==========================>...] - ETA: 0s - loss: 0.6382 - accuracy: 0.6383
Epoch 5: val_loss improved from 0.66059 to 0.63905, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 10ms/step - loss: 0.6370 - accuracy: 0.6363 - val_loss: 0.6391 - val_accuracy: 0.6281
Epoch 6/20
1/26 [>.............................] - ETA: 0s - loss: 0.5953 - accuracy: 0.7300␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
10/26 [==========>...................] - ETA: 0s - loss: 0.6092 - accuracy: 0.6790␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
18/26 [===================>..........] - ETA: 0s - loss: 0.6121 - accuracy: 0.6683
Epoch 6: val_loss improved from 0.63905 to 0.62025, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 9ms/step - loss: 0.6125 - accuracy: 0.6648 - val_loss: 0.6202 - val_accuracy: 0.6656
Epoch 7/20
1/26 [>.............................] - ETA: 0s - loss: 0.5801 - accuracy: 0.7300␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
10/26 [==========>...................] - ETA: 0s - loss: 0.5870 - accuracy: 0.6930␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
19/26 [====================>.........] - ETA: 0s - loss: 0.5832 - accuracy: 0.6995
Epoch 7: val_loss improved from 0.62025 to 0.61529, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 9ms/step - loss: 0.5858 - accuracy: 0.6980 - val_loss: 0.6153 - val_accuracy: 0.6406
Epoch 8/20
1/26 [>.............................] - ETA: 0s - loss: 0.5015 - accuracy: 0.7300␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
9/26 [=========>....................] - ETA: 0s - loss: 0.5797 - accuracy: 0.6911␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
17/26 [==================>...........] - ETA: 0s - loss: 0.5749 - accuracy: 0.7012␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - ETA: 0s - loss: 0.5702 - accuracy: 0.7047
Epoch 8: val_loss improved from 0.61529 to 0.58292, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 9ms/step - loss: 0.5702 - accuracy: 0.7047 - val_loss: 0.5829 - val_accuracy: 0.7000
Epoch 9/20
1/26 [>.............................] - ETA: 0s - loss: 0.5262 - accuracy: 0.7000␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
10/26 [==========>...................] - ETA: 0s - loss: 0.5485 - accuracy: 0.7180␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
18/26 [===================>..........] - ETA: 0s - loss: 0.5388 - accuracy: 0.7294␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - ETA: 0s - loss: 0.5372 - accuracy: 0.7309
Epoch 9: val_loss improved from 0.58292 to 0.54673, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 9ms/step - loss: 0.5372 - accuracy: 0.7309 - val_loss: 0.5467 - val_accuracy: 0.7375
Epoch 10/20
1/26 [>.............................] - ETA: 0s - loss: 0.5478 - accuracy: 0.7200␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
11/26 [===========>..................] - ETA: 0s - loss: 0.5131 - accuracy: 0.7582␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
20/26 [======================>.......] - ETA: 0s - loss: 0.5088 - accuracy: 0.7540
Epoch 10: val_loss improved from 0.54673 to 0.52583, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 9ms/step - loss: 0.5097 - accuracy: 0.7480 - val_loss: 0.5258 - val_accuracy: 0.7500
Epoch 11/20
1/26 [>.............................] - ETA: 0s - loss: 0.4824 - accuracy: 0.7200␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
11/26 [===========>..................] - ETA: 0s - loss: 0.4900 - accuracy: 0.7627␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
21/26 [=======================>......] - ETA: 0s - loss: 0.4841 - accuracy: 0.7690
Epoch 11: val_loss improved from 0.52583 to 0.50905, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 8ms/step - loss: 0.4942 - accuracy: 0.7594 - val_loss: 0.5090 - val_accuracy: 0.7609
Epoch 12/20
1/26 [>.............................] - ETA: 0s - loss: 0.4493 - accuracy: 0.7800␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
10/26 [==========>...................] - ETA: 0s - loss: 0.4866 - accuracy: 0.7510␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
20/26 [======================>.......] - ETA: 0s - loss: 0.4725 - accuracy: 0.7690
Epoch 12: val_loss improved from 0.50905 to 0.48350, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 8ms/step - loss: 0.4697 - accuracy: 0.7738 - val_loss: 0.4835 - val_accuracy: 0.7906
Epoch 13/20
1/26 [>.............................] - ETA: 0s - loss: 0.4898 - accuracy: 0.7500␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
11/26 [===========>..................] - ETA: 0s - loss: 0.4677 - accuracy: 0.7582␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
20/26 [======================>.......] - ETA: 0s - loss: 0.4613 - accuracy: 0.7680
Epoch 13: val_loss improved from 0.48350 to 0.48221, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 9ms/step - loss: 0.4495 - accuracy: 0.7801 - val_loss: 0.4822 - val_accuracy: 0.7906
Epoch 14/20
1/26 [>.............................] - ETA: 0s - loss: 0.3689 - accuracy: 0.8600␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
10/26 [==========>...................] - ETA: 0s - loss: 0.4453 - accuracy: 0.7990␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
20/26 [======================>.......] - ETA: 0s - loss: 0.4566 - accuracy: 0.7910
Epoch 14: val_loss improved from 0.48221 to 0.47486, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 8ms/step - loss: 0.4562 - accuracy: 0.7867 - val_loss: 0.4749 - val_accuracy: 0.7922
Epoch 15/20
1/26 [>.............................] - ETA: 0s - loss: 0.4179 - accuracy: 0.8100␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
12/26 [============>.................] - ETA: 0s - loss: 0.4133 - accuracy: 0.8150␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
22/26 [========================>.....] - ETA: 0s - loss: 0.4225 - accuracy: 0.7973
Epoch 15: val_loss improved from 0.47486 to 0.45124, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 8ms/step - loss: 0.4258 - accuracy: 0.7992 - val_loss: 0.4512 - val_accuracy: 0.7922
Epoch 16/20
1/26 [>.............................] - ETA: 0s - loss: 0.4175 - accuracy: 0.7900␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
11/26 [===========>..................] - ETA: 0s - loss: 0.4055 - accuracy: 0.8055␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
21/26 [=======================>......] - ETA: 0s - loss: 0.4176 - accuracy: 0.7995
Epoch 16: val_loss improved from 0.45124 to 0.44947, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 8ms/step - loss: 0.4215 - accuracy: 0.7996 - val_loss: 0.4495 - val_accuracy: 0.8047
Epoch 17/20
1/26 [>.............................] - ETA: 0s - loss: 0.4596 - accuracy: 0.8300␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
12/26 [============>.................] - ETA: 0s - loss: 0.3961 - accuracy: 0.8283␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
22/26 [========================>.....] - ETA: 0s - loss: 0.4167 - accuracy: 0.8082
Epoch 17: val_loss improved from 0.44947 to 0.43981, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 8ms/step - loss: 0.4200 - accuracy: 0.8074 - val_loss: 0.4398 - val_accuracy: 0.7953
Epoch 18/20
1/26 [>.............................] - ETA: 0s - loss: 0.3658 - accuracy: 0.8300␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
11/26 [===========>..................] - ETA: 0s - loss: 0.4273 - accuracy: 0.8045␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
20/26 [======================>.......] - ETA: 0s - loss: 0.4145 - accuracy: 0.8080
Epoch 18: val_loss improved from 0.43981 to 0.43832, saving model to trained_model_LSTM.h5
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 8ms/step - loss: 0.4121 - accuracy: 0.8113 - val_loss: 0.4383 - val_accuracy: 0.8094
Epoch 19/20
1/26 [>.............................] - ETA: 0s - loss: 0.3486 - accuracy: 0.8200␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
9/26 [=========>....................] - ETA: 0s - loss: 0.3886 - accuracy: 0.8178␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
17/26 [==================>...........] - ETA: 0s - loss: 0.4146 - accuracy: 0.8047␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
25/26 [===========================>..] - ETA: 0s - loss: 0.4050 - accuracy: 0.8104
Epoch 19: val_loss did not improve from 0.43832
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 8ms/step - loss: 0.4032 - accuracy: 0.8121 - val_loss: 0.4421 - val_accuracy: 0.7984
Epoch 20/20
1/26 [>.............................] - ETA: 0s - loss: 0.4000 - accuracy: 0.7800␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
11/26 [===========>..................] - ETA: 0s - loss: 0.4009 - accuracy: 0.8064␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
22/26 [========================>.....] - ETA: 0s - loss: 0.3917 - accuracy: 0.8164
Epoch 20: val_loss did not improve from 0.43832
␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈␈
26/26 [==============================] - 0s 7ms/step - loss: 0.3920 - accuracy: 0.8160 - val_loss: 0.4400 - val_accuracy: 0.7969
: Getting training history for item:0 name = 'loss'
: Getting training history for item:1 name = 'accuracy'
: Getting training history for item:2 name = 'val_loss'
: Getting training history for item:3 name = 'val_accuracy'
: Elapsed time for training with 3200 events: 7.19 sec
: Setting up tf.keras
: Using TensorFlow version 2
: Use Keras version from TensorFlow : tf.keras
: Applying GPU option: gpu_options.allow_growth=True
: Disabled TF eager execution when evaluating model
: Loading Keras Model
: Loaded model from file: trained_model_LSTM.h5
PyKeras_LSTM : [dataset] : Evaluation of PyKeras_LSTM on training sample (3200 events)
: Elapsed time for evaluation of 3200 events: 0.286 sec
: Creating xml weight file: ␛[0;36mdataset/weights/TMVAClassification_PyKeras_LSTM.weights.xml␛[0m
: Creating standalone class: ␛[0;36mdataset/weights/TMVAClassification_PyKeras_LSTM.class.C␛[0m
Factory : Training finished
:
Factory : Train method: BDTG for Classification
:
BDTG : #events: (reweighted) sig: 1600 bkg: 1600
: #events: (unweighted) sig: 1600 bkg: 1600
: Training 100 Decision Trees ... patience please
: Elapsed time for training with 3200 events: 1.67 sec
BDTG : [dataset] : Evaluation of BDTG on training sample (3200 events)
: Elapsed time for evaluation of 3200 events: 0.0187 sec
: Creating xml weight file: ␛[0;36mdataset/weights/TMVAClassification_BDTG.weights.xml␛[0m
: Creating standalone class: ␛[0;36mdataset/weights/TMVAClassification_BDTG.class.C␛[0m
: data_RNN_CPU.root:/dataset/Method_BDT/BDTG
Factory : Training finished
:
: Ranking input variables (method specific)...
: No variable ranking supplied by classifier: TMVA_LSTM
: No variable ranking supplied by classifier: TMVA_DNN
: No variable ranking supplied by classifier: PyKeras_LSTM
BDTG : Ranking result (top variable is best ranked)
: --------------------------------------------
: Rank : Variable : Variable Importance
: --------------------------------------------
: 1 : vars_time7 : 2.192e-02
: 2 : vars_time7 : 2.038e-02
: 3 : vars_time8 : 1.908e-02
: 4 : vars_time7 : 1.901e-02
: 5 : vars_time9 : 1.895e-02
: 6 : vars_time6 : 1.821e-02
: 7 : vars_time6 : 1.818e-02
: 8 : vars_time9 : 1.786e-02
: 9 : vars_time8 : 1.679e-02
: 10 : vars_time8 : 1.634e-02
: 11 : vars_time7 : 1.631e-02
: 12 : vars_time8 : 1.571e-02
: 13 : vars_time9 : 1.558e-02
: 14 : vars_time0 : 1.500e-02
: 15 : vars_time0 : 1.470e-02
: 16 : vars_time8 : 1.435e-02
: 17 : vars_time1 : 1.389e-02
: 18 : vars_time7 : 1.381e-02
: 19 : vars_time5 : 1.381e-02
: 20 : vars_time9 : 1.315e-02
: 21 : vars_time9 : 1.286e-02
: 22 : vars_time4 : 1.228e-02
: 23 : vars_time5 : 1.223e-02
: 24 : vars_time5 : 1.151e-02
: 25 : vars_time6 : 1.137e-02
: 26 : vars_time8 : 1.122e-02
: 27 : vars_time8 : 1.098e-02
: 28 : vars_time8 : 1.076e-02
: 29 : vars_time9 : 1.073e-02
: 30 : vars_time6 : 1.058e-02
: 31 : vars_time7 : 1.030e-02
: 32 : vars_time0 : 1.015e-02
: 33 : vars_time0 : 9.995e-03
: 34 : vars_time4 : 9.881e-03
: 35 : vars_time9 : 9.708e-03
: 36 : vars_time9 : 9.664e-03
: 37 : vars_time6 : 9.345e-03
: 38 : vars_time6 : 9.322e-03
: 39 : vars_time8 : 9.189e-03
: 40 : vars_time9 : 9.172e-03
: 41 : vars_time6 : 9.143e-03
: 42 : vars_time1 : 8.999e-03
: 43 : vars_time6 : 8.916e-03
: 44 : vars_time9 : 8.730e-03
: 45 : vars_time6 : 8.129e-03
: 46 : vars_time4 : 8.043e-03
: 47 : vars_time1 : 7.960e-03
: 48 : vars_time7 : 7.941e-03
: 49 : vars_time6 : 7.755e-03
: 50 : vars_time7 : 7.687e-03
: 51 : vars_time6 : 7.632e-03
: 52 : vars_time8 : 7.623e-03
: 53 : vars_time5 : 7.535e-03
: 54 : vars_time1 : 7.518e-03
: 55 : vars_time6 : 7.289e-03
: 56 : vars_time1 : 7.239e-03
: 57 : vars_time7 : 7.198e-03
: 58 : vars_time4 : 7.131e-03
: 59 : vars_time4 : 7.107e-03
: 60 : vars_time0 : 7.073e-03
: 61 : vars_time8 : 6.985e-03
: 62 : vars_time9 : 6.905e-03
: 63 : vars_time1 : 6.899e-03
: 64 : vars_time0 : 6.818e-03
: 65 : vars_time9 : 6.735e-03
: 66 : vars_time7 : 6.720e-03
: 67 : vars_time6 : 6.714e-03
: 68 : vars_time7 : 6.706e-03
: 69 : vars_time3 : 6.667e-03
: 70 : vars_time0 : 6.656e-03
: 71 : vars_time7 : 6.451e-03
: 72 : vars_time6 : 6.422e-03
: 73 : vars_time5 : 6.412e-03
: 74 : vars_time9 : 6.390e-03
: 75 : vars_time8 : 6.275e-03
: 76 : vars_time8 : 6.264e-03
: 77 : vars_time5 : 6.233e-03
: 78 : vars_time1 : 6.219e-03
: 79 : vars_time0 : 6.114e-03
: 80 : vars_time9 : 6.024e-03
: 81 : vars_time1 : 5.917e-03
: 82 : vars_time7 : 5.860e-03
: 83 : vars_time1 : 5.832e-03
: 84 : vars_time0 : 5.792e-03
: 85 : vars_time6 : 5.725e-03
: 86 : vars_time9 : 5.708e-03
: 87 : vars_time8 : 5.706e-03
: 88 : vars_time9 : 5.706e-03
: 89 : vars_time4 : 5.671e-03
: 90 : vars_time5 : 5.543e-03
: 91 : vars_time3 : 5.531e-03
: 92 : vars_time8 : 5.356e-03
: 93 : vars_time9 : 5.295e-03
: 94 : vars_time0 : 5.289e-03
: 95 : vars_time4 : 5.256e-03
: 96 : vars_time9 : 5.248e-03
: 97 : vars_time0 : 5.236e-03
: 98 : vars_time7 : 4.975e-03
: 99 : vars_time5 : 4.928e-03
: 100 : vars_time2 : 4.812e-03
: 101 : vars_time2 : 4.784e-03
: 102 : vars_time0 : 4.589e-03
: 103 : vars_time7 : 4.452e-03
: 104 : vars_time7 : 4.272e-03
: 105 : vars_time8 : 4.233e-03
: 106 : vars_time4 : 4.163e-03
: 107 : vars_time1 : 4.128e-03
: 108 : vars_time1 : 3.937e-03
: 109 : vars_time2 : 3.906e-03
: 110 : vars_time7 : 3.715e-03
: 111 : vars_time9 : 3.549e-03
: 112 : vars_time6 : 3.511e-03
: 113 : vars_time5 : 3.443e-03
: 114 : vars_time4 : 3.391e-03
: 115 : vars_time1 : 3.008e-03
: 116 : vars_time0 : 0.000e+00
: 117 : vars_time0 : 0.000e+00
: 118 : vars_time0 : 0.000e+00
: 119 : vars_time0 : 0.000e+00
: 120 : vars_time0 : 0.000e+00
: 121 : vars_time0 : 0.000e+00
: 122 : vars_time0 : 0.000e+00
: 123 : vars_time0 : 0.000e+00
: 124 : vars_time0 : 0.000e+00
: 125 : vars_time0 : 0.000e+00
: 126 : vars_time0 : 0.000e+00
: 127 : vars_time0 : 0.000e+00
: 128 : vars_time0 : 0.000e+00
: 129 : vars_time0 : 0.000e+00
: 130 : vars_time0 : 0.000e+00
: 131 : vars_time0 : 0.000e+00
: 132 : vars_time0 : 0.000e+00
: 133 : vars_time0 : 0.000e+00
: 134 : vars_time1 : 0.000e+00
: 135 : vars_time1 : 0.000e+00
: 136 : vars_time1 : 0.000e+00
: 137 : vars_time1 : 0.000e+00
: 138 : vars_time1 : 0.000e+00
: 139 : vars_time1 : 0.000e+00
: 140 : vars_time1 : 0.000e+00
: 141 : vars_time1 : 0.000e+00
: 142 : vars_time1 : 0.000e+00
: 143 : vars_time1 : 0.000e+00
: 144 : vars_time1 : 0.000e+00
: 145 : vars_time1 : 0.000e+00
: 146 : vars_time1 : 0.000e+00
: 147 : vars_time1 : 0.000e+00
: 148 : vars_time1 : 0.000e+00
: 149 : vars_time1 : 0.000e+00
: 150 : vars_time1 : 0.000e+00
: 151 : vars_time1 : 0.000e+00
: 152 : vars_time2 : 0.000e+00
: 153 : vars_time2 : 0.000e+00
: 154 : vars_time2 : 0.000e+00
: 155 : vars_time2 : 0.000e+00
: 156 : vars_time2 : 0.000e+00
: 157 : vars_time2 : 0.000e+00
: 158 : vars_time2 : 0.000e+00
: 159 : vars_time2 : 0.000e+00
: 160 : vars_time2 : 0.000e+00
: 161 : vars_time2 : 0.000e+00
: 162 : vars_time2 : 0.000e+00
: 163 : vars_time2 : 0.000e+00
: 164 : vars_time2 : 0.000e+00
: 165 : vars_time2 : 0.000e+00
: 166 : vars_time2 : 0.000e+00
: 167 : vars_time2 : 0.000e+00
: 168 : vars_time2 : 0.000e+00
: 169 : vars_time2 : 0.000e+00
: 170 : vars_time2 : 0.000e+00
: 171 : vars_time2 : 0.000e+00
: 172 : vars_time2 : 0.000e+00
: 173 : vars_time2 : 0.000e+00
: 174 : vars_time2 : 0.000e+00
: 175 : vars_time2 : 0.000e+00
: 176 : vars_time2 : 0.000e+00
: 177 : vars_time2 : 0.000e+00
: 178 : vars_time2 : 0.000e+00
: 179 : vars_time3 : 0.000e+00
: 180 : vars_time3 : 0.000e+00
: 181 : vars_time3 : 0.000e+00
: 182 : vars_time3 : 0.000e+00
: 183 : vars_time3 : 0.000e+00
: 184 : vars_time3 : 0.000e+00
: 185 : vars_time3 : 0.000e+00
: 186 : vars_time3 : 0.000e+00
: 187 : vars_time3 : 0.000e+00
: 188 : vars_time3 : 0.000e+00
: 189 : vars_time3 : 0.000e+00
: 190 : vars_time3 : 0.000e+00
: 191 : vars_time3 : 0.000e+00
: 192 : vars_time3 : 0.000e+00
: 193 : vars_time3 : 0.000e+00
: 194 : vars_time3 : 0.000e+00
: 195 : vars_time3 : 0.000e+00
: 196 : vars_time3 : 0.000e+00
: 197 : vars_time3 : 0.000e+00
: 198 : vars_time3 : 0.000e+00
: 199 : vars_time3 : 0.000e+00
: 200 : vars_time3 : 0.000e+00
: 201 : vars_time3 : 0.000e+00
: 202 : vars_time3 : 0.000e+00
: 203 : vars_time3 : 0.000e+00
: 204 : vars_time3 : 0.000e+00
: 205 : vars_time3 : 0.000e+00
: 206 : vars_time3 : 0.000e+00
: 207 : vars_time4 : 0.000e+00
: 208 : vars_time4 : 0.000e+00
: 209 : vars_time4 : 0.000e+00
: 210 : vars_time4 : 0.000e+00
: 211 : vars_time4 : 0.000e+00
: 212 : vars_time4 : 0.000e+00
: 213 : vars_time4 : 0.000e+00
: 214 : vars_time4 : 0.000e+00
: 215 : vars_time4 : 0.000e+00
: 216 : vars_time4 : 0.000e+00
: 217 : vars_time4 : 0.000e+00
: 218 : vars_time4 : 0.000e+00
: 219 : vars_time4 : 0.000e+00
: 220 : vars_time4 : 0.000e+00
: 221 : vars_time4 : 0.000e+00
: 222 : vars_time4 : 0.000e+00
: 223 : vars_time4 : 0.000e+00
: 224 : vars_time4 : 0.000e+00
: 225 : vars_time4 : 0.000e+00
: 226 : vars_time4 : 0.000e+00
: 227 : vars_time4 : 0.000e+00
: 228 : vars_time5 : 0.000e+00
: 229 : vars_time5 : 0.000e+00
: 230 : vars_time5 : 0.000e+00
: 231 : vars_time5 : 0.000e+00
: 232 : vars_time5 : 0.000e+00
: 233 : vars_time5 : 0.000e+00
: 234 : vars_time5 : 0.000e+00
: 235 : vars_time5 : 0.000e+00
: 236 : vars_time5 : 0.000e+00
: 237 : vars_time5 : 0.000e+00
: 238 : vars_time5 : 0.000e+00
: 239 : vars_time5 : 0.000e+00
: 240 : vars_time5 : 0.000e+00
: 241 : vars_time5 : 0.000e+00
: 242 : vars_time5 : 0.000e+00
: 243 : vars_time5 : 0.000e+00
: 244 : vars_time5 : 0.000e+00
: 245 : vars_time5 : 0.000e+00
: 246 : vars_time5 : 0.000e+00
: 247 : vars_time5 : 0.000e+00
: 248 : vars_time5 : 0.000e+00
: 249 : vars_time6 : 0.000e+00
: 250 : vars_time6 : 0.000e+00
: 251 : vars_time6 : 0.000e+00
: 252 : vars_time6 : 0.000e+00
: 253 : vars_time6 : 0.000e+00
: 254 : vars_time6 : 0.000e+00
: 255 : vars_time6 : 0.000e+00
: 256 : vars_time6 : 0.000e+00
: 257 : vars_time6 : 0.000e+00
: 258 : vars_time6 : 0.000e+00
: 259 : vars_time6 : 0.000e+00
: 260 : vars_time6 : 0.000e+00
: 261 : vars_time6 : 0.000e+00
: 262 : vars_time6 : 0.000e+00
: 263 : vars_time7 : 0.000e+00
: 264 : vars_time7 : 0.000e+00
: 265 : vars_time7 : 0.000e+00
: 266 : vars_time7 : 0.000e+00
: 267 : vars_time7 : 0.000e+00
: 268 : vars_time7 : 0.000e+00
: 269 : vars_time7 : 0.000e+00
: 270 : vars_time7 : 0.000e+00
: 271 : vars_time7 : 0.000e+00
: 272 : vars_time7 : 0.000e+00
: 273 : vars_time7 : 0.000e+00
: 274 : vars_time7 : 0.000e+00
: 275 : vars_time7 : 0.000e+00
: 276 : vars_time8 : 0.000e+00
: 277 : vars_time8 : 0.000e+00
: 278 : vars_time8 : 0.000e+00
: 279 : vars_time8 : 0.000e+00
: 280 : vars_time8 : 0.000e+00
: 281 : vars_time8 : 0.000e+00
: 282 : vars_time8 : 0.000e+00
: 283 : vars_time8 : 0.000e+00
: 284 : vars_time8 : 0.000e+00
: 285 : vars_time8 : 0.000e+00
: 286 : vars_time8 : 0.000e+00
: 287 : vars_time8 : 0.000e+00
: 288 : vars_time8 : 0.000e+00
: 289 : vars_time8 : 0.000e+00
: 290 : vars_time9 : 0.000e+00
: 291 : vars_time9 : 0.000e+00
: 292 : vars_time9 : 0.000e+00
: 293 : vars_time9 : 0.000e+00
: 294 : vars_time9 : 0.000e+00
: 295 : vars_time9 : 0.000e+00
: 296 : vars_time9 : 0.000e+00
: 297 : vars_time9 : 0.000e+00
: 298 : vars_time9 : 0.000e+00
: 299 : vars_time9 : 0.000e+00
: 300 : vars_time9 : 0.000e+00
: --------------------------------------------
TH1.Print Name = TrainingHistory_TMVA_LSTM_trainingError, Entries= 0, Total sum= 11.3873
TH1.Print Name = TrainingHistory_TMVA_LSTM_valError, Entries= 0, Total sum= 12.0281
TH1.Print Name = TrainingHistory_TMVA_DNN_trainingError, Entries= 0, Total sum= 10.186
TH1.Print Name = TrainingHistory_TMVA_DNN_valError, Entries= 0, Total sum= 10.2571
TH1.Print Name = TrainingHistory_PyKeras_LSTM_'accuracy', Entries= 0, Total sum= 14.4285
TH1.Print Name = TrainingHistory_PyKeras_LSTM_'loss', Entries= 0, Total sum= 10.5006
TH1.Print Name = TrainingHistory_PyKeras_LSTM_'val_accuracy', Entries= 0, Total sum= 14.3391
TH1.Print Name = TrainingHistory_PyKeras_LSTM_'val_loss', Entries= 0, Total sum= 10.8483
Factory : === Destroy and recreate all methods via weight files for testing ===
:
: Reading weight file: ␛[0;36mdataset/weights/TMVAClassification_TMVA_LSTM.weights.xml␛[0m
: Reading weight file: ␛[0;36mdataset/weights/TMVAClassification_TMVA_DNN.weights.xml␛[0m
: Reading weight file: ␛[0;36mdataset/weights/TMVAClassification_PyKeras_LSTM.weights.xml␛[0m
: Reading weight file: ␛[0;36mdataset/weights/TMVAClassification_BDTG.weights.xml␛[0m
nthreads = 4
Factory : ␛[1mTest all methods␛[0m
Factory : Test method: TMVA_LSTM for Classification performance
:
: Evaluate deep neural network on CPU using batches with size = 800
:
TMVA_LSTM : [dataset] : Evaluation of TMVA_LSTM on testing sample (800 events)
: Elapsed time for evaluation of 800 events: 0.0517 sec
Factory : Test method: TMVA_DNN for Classification performance
:
: Evaluate deep neural network on CPU using batches with size = 800
:
TMVA_DNN : [dataset] : Evaluation of TMVA_DNN on testing sample (800 events)
: Elapsed time for evaluation of 800 events: 0.0213 sec
Factory : Test method: PyKeras_LSTM for Classification performance
:
: Setting up tf.keras
: Using TensorFlow version 2
: Use Keras version from TensorFlow : tf.keras
: Applying GPU option: gpu_options.allow_growth=True
: Disabled TF eager execution when evaluating model
: Loading Keras Model
: Loaded model from file: trained_model_LSTM.h5
PyKeras_LSTM : [dataset] : Evaluation of PyKeras_LSTM on testing sample (800 events)
: Elapsed time for evaluation of 800 events: 0.238 sec
Factory : Test method: BDTG for Classification performance
:
BDTG : [dataset] : Evaluation of BDTG on testing sample (800 events)
: Elapsed time for evaluation of 800 events: 0.00586 sec
Factory : ␛[1mEvaluate all methods␛[0m
Factory : Evaluate classifier: TMVA_LSTM
:
TMVA_LSTM : [dataset] : Loop over test events and fill histograms with classifier response...
:
: Evaluate deep neural network on CPU using batches with size = 1000
:
: Dataset[dataset] : variable plots are not produces ! The number of variables is 300 , it is larger than 200
Factory : Evaluate classifier: TMVA_DNN
:
TMVA_DNN : [dataset] : Loop over test events and fill histograms with classifier response...
:
: Evaluate deep neural network on CPU using batches with size = 1000
:
: Dataset[dataset] : variable plots are not produces ! The number of variables is 300 , it is larger than 200
Factory : Evaluate classifier: PyKeras_LSTM
:
PyKeras_LSTM : [dataset] : Loop over test events and fill histograms with classifier response...
:
: Dataset[dataset] : variable plots are not produces ! The number of variables is 300 , it is larger than 200
Factory : Evaluate classifier: BDTG
:
BDTG : [dataset] : Loop over test events and fill histograms with classifier response...
:
: Dataset[dataset] : variable plots are not produces ! The number of variables is 300 , it is larger than 200
:
: Evaluation results ranked by best signal efficiency and purity (area)
: -------------------------------------------------------------------------------------------------------------------
: DataSet MVA
: Name: Method: ROC-integ
: dataset PyKeras_LSTM : 0.889
: dataset BDTG : 0.851
: dataset TMVA_LSTM : 0.844
: dataset TMVA_DNN : 0.587
: -------------------------------------------------------------------------------------------------------------------
:
: Testing efficiency compared to training efficiency (overtraining check)
: -------------------------------------------------------------------------------------------------------------------
: DataSet MVA Signal efficiency: from test sample (from training sample)
: Name: Method: @B=0.01 @B=0.10 @B=0.30
: -------------------------------------------------------------------------------------------------------------------
: dataset PyKeras_LSTM : 0.195 (0.275) 0.698 (0.681) 0.880 (0.888)
: dataset BDTG : 0.150 (0.290) 0.572 (0.676) 0.838 (0.891)
: dataset TMVA_LSTM : 0.195 (0.175) 0.540 (0.590) 0.840 (0.855)
: dataset TMVA_DNN : 0.027 (0.030) 0.170 (0.193) 0.425 (0.426)
: -------------------------------------------------------------------------------------------------------------------
:
Dataset:dataset : Created tree 'TestTree' with 800 events
:
Dataset:dataset : Created tree 'TrainTree' with 3200 events
:
Factory : ␛[1mThank you for using TMVA!␛[0m
: ␛[1mFor citation information, please visit: http://tmva.sf.net/citeTMVA.html␛[0m
/***
# TMVA Classification Example Using a Recurrent Neural Network
This is an example of using a RNN in TMVA.
We do the classification using a toy data set containing a time series of data sample ntimes
and with dimension ndim that is generated when running the provided function `MakeTimeData (nevents, ntime, ndim)`
**/
#include<TROOT.h>
#include "TMVA/Factory.h"
#include "TMVA/DataLoader.h"
#include "TMVA/DataSetInfo.h"
#include "TMVA/Config.h"
#include "TMVA/MethodDL.h"
#include "TFile.h"
#include "TTree.h"
/// Helper function to generate the time data set
/// make some time data but not of fixed length.
/// use a poisson with mu = 5 and truncated at 10
///
void MakeTimeData(int n, int ntime, int ndim )
{
// const int ntime = 10;
// const int ndim = 30; // number of dim/time
TString fname = TString::Format("time_data_t%d_d%d.root", ntime, ndim);
std::vector<TH1 *> v1(ntime);
std::vector<TH1 *> v2(ntime);
int i = 0;
for (int i = 0; i < ntime; ++i) {
v1[i] = new TH1D(TString::Format("h1_%d", i), "h1", ndim, 0, 10);
v2[i] = new TH1D(TString::Format("h2_%d", i), "h2", ndim, 0, 10);
}
auto f1 = new TF1("f1", "gaus");
auto f2 = new TF1("f2", "gaus");
TTree sgn("sgn", "sgn");
TTree bkg("bkg", "bkg");
TFile f(fname, "RECREATE");
std::vector<std::vector<float>> x1(ntime);
std::vector<std::vector<float>> x2(ntime);
for (int i = 0; i < ntime; ++i) {
x1[i] = std::vector<float>(ndim);
x2[i] = std::vector<float>(ndim);
}
for (auto i = 0; i < ntime; i++) {
bkg.Branch(Form("vars_time%d", i), "std::vector<float>", &x1[i]);
sgn.Branch(Form("vars_time%d", i), "std::vector<float>", &x2[i]);
}
sgn.SetDirectory(&f);
bkg.SetDirectory(&f);
gRandom->SetSeed(0);
std::vector<double> mean1(ntime);
std::vector<double> mean2(ntime);
std::vector<double> sigma1(ntime);
std::vector<double> sigma2(ntime);
for (int j = 0; j < ntime; ++j) {
mean1[j] = 5. + 0.2 * sin(TMath::Pi() * j / double(ntime));
mean2[j] = 5. + 0.2 * cos(TMath::Pi() * j / double(ntime));
sigma1[j] = 4 + 0.3 * sin(TMath::Pi() * j / double(ntime));
sigma2[j] = 4 + 0.3 * cos(TMath::Pi() * j / double(ntime));
}
for (int i = 0; i < n; ++i) {
if (i % 1000 == 0)
std::cout << "Generating event ... " << i << std::endl;
for (int j = 0; j < ntime; ++j) {
auto h1 = v1[j];
auto h2 = v2[j];
h1->Reset();
h2->Reset();
f1->SetParameters(1, mean1[j], sigma1[j]);
f2->SetParameters(1, mean2[j], sigma2[j]);
h1->FillRandom("f1", 1000);
h2->FillRandom("f2", 1000);
for (int k = 0; k < ndim; ++k) {
// std::cout << j*10+k << " ";
x1[j][k] = h1->GetBinContent(k + 1) + gRandom->Gaus(0, 10);
x2[j][k] = h2->GetBinContent(k + 1) + gRandom->Gaus(0, 10);
}
}
// std::cout << std::endl;
sgn.Fill();
bkg.Fill();
if (n == 1) {
auto c1 = new TCanvas();
c1->Divide(ntime, 2);
for (int j = 0; j < ntime; ++j) {
c1->cd(j + 1);
v1[j]->Draw();
}
for (int j = 0; j < ntime; ++j) {
c1->cd(ntime + j + 1);
v2[j]->Draw();
}
gPad->Update();
}
}
if (n > 1) {
sgn.Write();
bkg.Write();
sgn.Print();
bkg.Print();
f.Close();
}
}
/// macro for performing a classification using a Recurrent Neural Network
/// @param nevts = 2000 Number of events used. (increase for better classification results)
/// @param use_type
/// use_type = 0 use Simple RNN network
/// use_type = 1 use LSTM network
/// use_type = 2 use GRU
/// use_type = 3 build 3 different networks with RNN, LSTM and GRU
void TMVA_RNN_Classification(int nevts = 2000, int use_type = 1)
{
const int ninput = 30;
const int ntime = 10;
const int batchSize = 100;
const int maxepochs = 20;
int nTotEvts = nevts; // total events to be generated for signal or background
bool useKeras = true;
bool useTMVA_RNN = true;
bool useTMVA_DNN = true;
bool useTMVA_BDT = false;
std::vector<std::string> rnn_types = {"RNN", "LSTM", "GRU"};
std::vector<bool> use_rnn_type = {1, 1, 1};
if (use_type >=0 && use_type < 3) {
use_rnn_type = {0,0,0};
use_rnn_type[use_type] = 1;
}
bool useGPU = true; // use GPU for TMVA if available
#ifndef R__HAS_TMVAGPU
useGPU = false;
#ifndef R__HAS_TMVACPU
Warning("TMVA_RNN_Classification", "TMVA is not build with GPU or CPU multi-thread support. Cannot use TMVA Deep Learning for RNN");
useTMVA_RNN = false;
#endif
#endif
TString archString = (useGPU) ? "GPU" : "CPU";
bool writeOutputFile = true;
const char *rnn_type = "RNN";
#ifdef R__HAS_PYMVA
TMVA::PyMethodBase::PyInitialize();
#else
useKeras = false;
#endif
int num_threads = 4; // use by default all threads
gSystem->Setenv("OMP_NUM_THREADS", "1"); // switch off MT in OpenBLAS
// do enable MT running
if (num_threads >= 0) {
ROOT::EnableImplicitMT(num_threads);
}
TMVA::Config::Instance();
std::cout << "Running with nthreads = " << ROOT::GetThreadPoolSize() << std::endl;
TString inputFileName = "time_data_t10_d30.root";
bool fileExist = !gSystem->AccessPathName(inputFileName);
// if file does not exists create it
if (!fileExist) {
MakeTimeData(nTotEvts,ntime, ninput);
}
auto inputFile = TFile::Open(inputFileName);
if (!inputFile) {
Error("TMVA_RNN_Classification", "Error opening input file %s - exit", inputFileName.Data());
return;
}
std::cout << "--- RNNClassification : Using input file: " << inputFile->GetName() << std::endl;
// Create a ROOT output file where TMVA will store ntuples, histograms, etc.
TString outfileName(TString::Format("data_RNN_%s.root", archString.Data()));
TFile *outputFile = nullptr;
if (writeOutputFile) outputFile = TFile::Open(outfileName, "RECREATE");
/**
## Declare Factory
Create the Factory class. Later you can choose the methods
whose performance you'd like to investigate.
The factory is the major TMVA object you have to interact with. Here is the list of parameters you need to
pass
- The first argument is the base of the name of all the output
weightfiles in the directory weight/ that will be created with the
method parameters
- The second argument is the output file for the training results
- The third argument is a string option defining some general configuration for the TMVA session.
For example all TMVA output can be suppressed by removing the "!" (not) in front of the "Silent" argument in
the option string
**/
// Creating the factory object
TMVA::Factory *factory = new TMVA::Factory("TMVAClassification", outputFile,
"!V:!Silent:Color:DrawProgressBar:Transformations=None:!Correlations:"
"AnalysisType=Classification:ModelPersistence");
TMVA::DataLoader *dataloader = new TMVA::DataLoader("dataset");
TTree *signalTree = (TTree *)inputFile->Get("sgn");
TTree *background = (TTree *)inputFile->Get("bkg");
const int nvar = ninput * ntime;
/// add variables - use new AddVariablesArray function
for (auto i = 0; i < ntime; i++) {
dataloader->AddVariablesArray(Form("vars_time%d", i), ninput);
}
dataloader->AddSignalTree(signalTree, 1.0);
dataloader->AddBackgroundTree(background, 1.0);
// check given input
auto &datainfo = dataloader->GetDataSetInfo();
auto vars = datainfo.GetListOfVariables();
std::cout << "number of variables is " << vars.size() << std::endl;
for (auto &v : vars)
std::cout << v << ",";
std::cout << std::endl;
int nTrainSig = 0.8 * nTotEvts;
int nTrainBkg = 0.8 * nTotEvts;
// build the string options for DataLoader::PrepareTrainingAndTestTree
TString prepareOptions = TString::Format("nTrain_Signal=%d:nTrain_Background=%d:SplitMode=Random:SplitSeed=100:NormMode=NumEvents:!V:!CalcCorrelations", nTrainSig, nTrainBkg);
// Apply additional cuts on the signal and background samples (can be different)
TCut mycuts = ""; // for example: TCut mycuts = "abs(var1)<0.5 && abs(var2-0.5)<1";
TCut mycutb = "";
dataloader->PrepareTrainingAndTestTree(mycuts, mycutb, prepareOptions);
std::cout << "prepared DATA LOADER " << std::endl;
/**
## Book TMVA recurrent models
Book the different types of recurrent models in TMVA (SimpleRNN, LSTM or GRU)
**/
if (useTMVA_RNN) {
for (int i = 0; i < 3; ++i) {
if (!use_rnn_type[i])
continue;
const char *rnn_type = rnn_types[i].c_str();
/// define the inputlayout string for RNN
/// the input data should be organize as following:
//// input layout for RNN: time x ndim
TString inputLayoutString = TString::Format("InputLayout=%d|%d", ntime, ninput);
/// Define RNN layer layout
/// it should be LayerType (RNN or LSTM or GRU) | number of units | number of inputs | time steps | remember output (typically no=0 | return full sequence
TString rnnLayout = TString::Format("%s|10|%d|%d|0|1", rnn_type, ninput, ntime);
/// add after RNN a reshape layer (needed top flatten the output) and a dense layer with 64 units and a last one
/// Note the last layer is linear because when using Crossentropy a Sigmoid is applied already
TString layoutString = TString("Layout=") + rnnLayout + TString(",RESHAPE|FLAT,DENSE|64|TANH,LINEAR");
/// Defining Training strategies. Different training strings can be concatenate. Use however only one
TString trainingString1 = TString::Format("LearningRate=1e-3,Momentum=0.0,Repetitions=1,"
"ConvergenceSteps=5,BatchSize=%d,TestRepetitions=1,"
"WeightDecay=1e-2,Regularization=None,MaxEpochs=%d,"
"Optimizer=ADAM,DropConfig=0.0+0.+0.+0.",
batchSize,maxepochs);
TString trainingStrategyString("TrainingStrategy=");
trainingStrategyString += trainingString1; // + "|" + trainingString2
/// Define the full RNN Noption string adding the final options for all network
TString rnnOptions("!H:V:ErrorStrategy=CROSSENTROPY:VarTransform=None:"
"WeightInitialization=XAVIERUNIFORM:ValidationSize=0.2:RandomSeed=1234");
rnnOptions.Append(":");
rnnOptions.Append(inputLayoutString);
rnnOptions.Append(":");
rnnOptions.Append(layoutString);
rnnOptions.Append(":");
rnnOptions.Append(trainingStrategyString);
rnnOptions.Append(":");
rnnOptions.Append(TString::Format("Architecture=%s", archString.Data()));
TString rnnName = "TMVA_" + TString(rnn_type);
factory->BookMethod(dataloader, TMVA::Types::kDL, rnnName, rnnOptions);
}
}
/**
## Book TMVA fully connected dense layer models
**/
if (useTMVA_DNN) {
// Method DL with Dense Layer
TString inputLayoutString = TString::Format("InputLayout=1|1|%d", ntime * ninput);
TString layoutString("Layout=DENSE|64|TANH,DENSE|TANH|64,DENSE|TANH|64,LINEAR");
// Training strategies.
TString trainingString1("LearningRate=1e-3,Momentum=0.0,Repetitions=1,"
"ConvergenceSteps=10,BatchSize=256,TestRepetitions=1,"
"WeightDecay=1e-4,Regularization=None,MaxEpochs=20"
"DropConfig=0.0+0.+0.+0.,Optimizer=ADAM");
TString trainingStrategyString("TrainingStrategy=");
trainingStrategyString += trainingString1; // + "|" + trainingString2
// General Options.
TString dnnOptions("!H:V:ErrorStrategy=CROSSENTROPY:VarTransform=None:"
"WeightInitialization=XAVIER:RandomSeed=0");
dnnOptions.Append(":");
dnnOptions.Append(inputLayoutString);
dnnOptions.Append(":");
dnnOptions.Append(layoutString);
dnnOptions.Append(":");
dnnOptions.Append(trainingStrategyString);
dnnOptions.Append(":");
dnnOptions.Append(archString);
TString dnnName = "TMVA_DNN";
factory->BookMethod(dataloader, TMVA::Types::kDL, dnnName, dnnOptions);
}
/**
## Book Keras recurrent models
Book the different types of recurrent models in Keras (SimpleRNN, LSTM or GRU)
**/
if (useKeras) {
for (int i = 0; i < 3; i++) {
if (use_rnn_type[i]) {
TString modelName = TString::Format("model_%s.h5", rnn_types[i].c_str());
TString trainedModelName = TString::Format("trained_model_%s.h5", rnn_types[i].c_str());
Info("TMVA_RNN_Classification", "Building recurrent keras model using a %s layer", rnn_types[i].c_str());
// create python script which can be executed
// create 2 conv2d layer + maxpool + dense
TMacro m;
m.AddLine("import tensorflow");
m.AddLine("from tensorflow.keras.models import Sequential");
m.AddLine("from tensorflow.keras.optimizers import Adam");
m.AddLine("from tensorflow.keras.layers import Input, Dense, Dropout, Flatten, SimpleRNN, GRU, LSTM, Reshape, "
"BatchNormalization");
m.AddLine("");
m.AddLine("model = Sequential() ");
m.AddLine("model.add(Reshape((10, 30), input_shape = (10*30, )))");
// add recurrent neural network depending on type / Use option to return the full output
if (rnn_types[i] == "LSTM")
m.AddLine("model.add(LSTM(units=10, return_sequences=True) )");
else if (rnn_types[i] == "GRU")
m.AddLine("model.add(GRU(units=10, return_sequences=True) )");
else
m.AddLine("model.add(SimpleRNN(units=10, return_sequences=True) )");
// m.AddLine("model.add(BatchNormalization())");
m.AddLine("model.add(Flatten())"); // needed if returning the full time output sequence
m.AddLine("model.add(Dense(64, activation = 'tanh')) ");
m.AddLine("model.add(Dense(2, activation = 'sigmoid')) ");
m.AddLine(
"model.compile(loss = 'binary_crossentropy', optimizer = Adam(learning_rate = 0.001), weighted_metrics = ['accuracy'])");
m.AddLine(TString::Format("modelName = '%s'", modelName.Data()));
m.AddLine("model.save(modelName)");
m.AddLine("model.summary()");
m.SaveSource("make_rnn_model.py");
// execute python script to make the model
auto ret = (TString *)gROOT->ProcessLine("TMVA::Python_Executable()");
TString python_exe = (ret) ? *(ret) : "python";
gSystem->Exec(python_exe + " make_rnn_model.py");
if (gSystem->AccessPathName(modelName)) {
Warning("TMVA_RNN_Classification", "Error creating Keras recurrent model file - Skip using Keras");
useKeras = false;
} else {
// book PyKeras method only if Keras model could be created
Info("TMVA_RNN_Classification", "Booking Keras %s model", rnn_types[i].c_str());
factory->BookMethod(dataloader, TMVA::Types::kPyKeras,
TString::Format("PyKeras_%s", rnn_types[i].c_str()),
TString::Format("!H:!V:VarTransform=None:FilenameModel=%s:tf.keras:"
"FilenameTrainedModel=%s:GpuOptions=allow_growth=True:"
"NumEpochs=%d:BatchSize=%d",
modelName.Data(), trainedModelName.Data(), maxepochs, batchSize));
}
}
}
}
// use BDT in case not using Keras or TMVA DL
if (!useKeras || !useTMVA_BDT)
useTMVA_BDT = true;
/**
## Book TMVA BDT
**/
if (useTMVA_BDT) {
factory->BookMethod(dataloader, TMVA::Types::kBDT, "BDTG",
"!H:!V:NTrees=100:MinNodeSize=2.5%:BoostType=Grad:Shrinkage=0.10:UseBaggedBoost:"
"BaggedSampleFraction=0.5:nCuts=20:"
"MaxDepth=2");
}
/// Train all methods
factory->TrainAllMethods();
std::cout << "nthreads = " << ROOT::GetThreadPoolSize() << std::endl;
// ---- Evaluate all MVAs using the set of test events
factory->TestAllMethods();
// ----- Evaluate and compare performance of all configured MVAs
factory->EvaluateAllMethods();
// check method
// plot ROC curve
auto c1 = factory->GetROCCurve(dataloader);
c1->Draw();
if (outputFile) outputFile->Close();
}
f
#define f(i)
Definition RSha256.hxx:104
Info
void Info(const char *location, const char *msgfmt,...)
Use this function for informational messages.
Definition TError.cxx:218
Error
void Error(const char *location, const char *msgfmt,...)
Use this function in case an error occurred.
Definition TError.cxx:185
Warning
void Warning(const char *location, const char *msgfmt,...)
Use this function in warning situations.
Definition TError.cxx:229
x2
Option_t Option_t TPoint TPoint const char x2
Definition TGWin32VirtualXProxy.cxx:70
x1
Option_t Option_t TPoint TPoint const char x1
Definition TGWin32VirtualXProxy.cxx:70
gROOT
#define gROOT
Definition TROOT.h:406
gRandom
R__EXTERN TRandom * gRandom
Definition TRandom.h:62
Form
char * Form(const char *fmt,...)
Formats a string in a circular formatting buffer.
Definition TString.cxx:2468
gSystem
R__EXTERN TSystem * gSystem
Definition TSystem.h:560
gPad
#define gPad
Definition TVirtualPad.h:304
TCanvas
The Canvas class.
Definition TCanvas.h:23
TCut
A specialized string object used for TTree selections.
Definition TCut.h:25
TF1
1-Dim function class
Definition TF1.h:213
TF1::SetParameters
virtual void SetParameters(const Double_t *params)
Definition TF1.h:649
TFile
A ROOT file is composed of a header, followed by consecutive data records (TKey instances) with a wel...
Definition TFile.h:54
TFile::Open
static TFile * Open(const char *name, Option_t *option="", const char *ftitle="", Int_t compress=ROOT::RCompressionSetting::EDefaults::kUseCompiledDefault, Int_t netopt=0)
Create / open a file.
Definition TFile.cxx:4070
TFile::Close
void Close(Option_t *option="") override
Close a file.
Definition TFile.cxx:933
TGraph::Draw
void Draw(Option_t *chopt="") override
Draw this graph with its current attributes.
Definition TGraph.cxx:809
TH1D
1-D histogram with a double per channel (see TH1 documentation)}
Definition TH1.h:620
TH1F::Reset
void Reset(Option_t *option="") override
Reset.
Definition TH1.cxx:9982
TH1::FillRandom
virtual void FillRandom(const char *fname, Int_t ntimes=5000, TRandom *rng=nullptr)
Fill histogram following distribution in function fname.
Definition TH1.cxx:3516
TH1::GetBinContent
virtual Double_t GetBinContent(Int_t bin) const
Return content of bin number bin.
Definition TH1.cxx:5028
TMVA::Config::Instance
static Config & Instance()
static function: returns TMVA instance
Definition Config.cxx:98
TMVA::DataLoader::AddVariablesArray
void AddVariablesArray(const TString &expression, int size, char type='F', Double_t min=0, Double_t max=0)
user inserts discriminating array of variables in data set info in case input tree provides an array ...
Definition DataLoader.cxx:504
TMVA::DataLoader::AddSignalTree
void AddSignalTree(TTree *signal, Double_t weight=1.0, Types::ETreeType treetype=Types::kMaxTreeType)
number of signal events (used to compute significance)
Definition DataLoader.cxx:371
TMVA::DataLoader::PrepareTrainingAndTestTree
void PrepareTrainingAndTestTree(const TCut &cut, const TString &splitOpt)
prepare the training and test trees -> same cuts for signal and background
Definition DataLoader.cxx:632
TMVA::DataLoader::AddBackgroundTree
void AddBackgroundTree(TTree *background, Double_t weight=1.0, Types::ETreeType treetype=Types::kMaxTreeType)
number of signal events (used to compute significance)
Definition DataLoader.cxx:402
TMVA::DataLoader::GetDataSetInfo
DataSetInfo & GetDataSetInfo()
Definition DataLoader.cxx:137
TMVA::DataSetInfo::GetListOfVariables
std::vector< TString > GetListOfVariables() const
returns list of variables
Definition DataSetInfo.cxx:406
TMVA::Factory
This is the main MVA steering class.
Definition Factory.h:80
TMVA::Factory::TrainAllMethods
void TrainAllMethods()
Iterates through all booked methods and calls training.
Definition Factory.cxx:1114
TMVA::Factory::BookMethod
MethodBase * BookMethod(DataLoader *loader, TString theMethodName, TString methodTitle, TString theOption="")
Book a classifier or regression method.
Definition Factory.cxx:352
TMVA::Factory::TestAllMethods
void TestAllMethods()
Evaluates all booked methods on the testing data and adds the output to the Results in the corresponi...
Definition Factory.cxx:1271
TMVA::Factory::EvaluateAllMethods
void EvaluateAllMethods(void)
Iterates over all MVAs that have been booked, and calls their evaluation methods.
Definition Factory.cxx:1376
TMVA::Factory::GetROCCurve
TGraph * GetROCCurve(DataLoader *loader, TString theMethodName, Bool_t setTitles=kTRUE, UInt_t iClass=0, Types::ETreeType type=Types::kTesting)
Argument iClass specifies the class to generate the ROC curve in a multiclass setting.
Definition Factory.cxx:912
TMVA::PyMethodBase::PyInitialize
static void PyInitialize()
Initialize Python interpreter.
Definition PyMethodBase.cxx:153
TMacro
Class supporting a collection of lines with C++ code.
Definition TMacro.h:31
TRandom::Gaus
virtual Double_t Gaus(Double_t mean=0, Double_t sigma=1)
Samples a random number from the standard Normal (Gaussian) Distribution with the given mean and sigm...
Definition TRandom.cxx:274
TRandom::SetSeed
virtual void SetSeed(ULong_t seed=0)
Set the random generator seed.
Definition TRandom.cxx:608
TString
Basic string class.
Definition TString.h:139
TString::Data
const char * Data() const
Definition TString.h:380
TString::Format
static TString Format(const char *fmt,...)
Static method which formats a string using a printf style format descriptor and return a TString.
Definition TString.cxx:2357
TSystem::Exec
virtual Int_t Exec(const char *shellcmd)
Execute a command.
Definition TSystem.cxx:640
TSystem::AccessPathName
virtual Bool_t AccessPathName(const char *path, EAccessMode mode=kFileExists)
Returns FALSE if one can access a file using the specified access mode.
Definition TSystem.cxx:1283
TSystem::Setenv
virtual void Setenv(const char *name, const char *value)
Set environment variable.
Definition TSystem.cxx:1634
TTree
A TTree represents a columnar dataset.
Definition TTree.h:79
ROOT::VecOps::cos
RVec< PromoteType< T > > cos(const RVec< T > &v)
Definition RVec.hxx:1815
ROOT::VecOps::sin
RVec< PromoteType< T > > sin(const RVec< T > &v)
Definition RVec.hxx:1814
c1
return c1
Definition legend1.C:41
n
const Int_t n
Definition legend1.C:16
h1
TH1F * h1
Definition legend1.C:5
f1
TF1 * f1
Definition legend1.C:11
ROOT::EnableImplicitMT
void EnableImplicitMT(UInt_t numthreads=0)
Enable ROOT's implicit multi-threading for all objects and methods that provide an internal paralleli...
Definition TROOT.cxx:537
ROOT::GetThreadPoolSize
UInt_t GetThreadPoolSize()
Returns the size of ROOT's thread pool.
Definition TROOT.cxx:575
TMVA_RNN_Classification
Definition TMVA_RNN_Classification.py:1
TMath::Pi
constexpr Double_t Pi()
Definition TMath.h:37
v2
@ v2
Definition rootcling_impl.cxx:3683
v
@ v
Definition rootcling_impl.cxx:3680
v1
@ v1
Definition rootcling_impl.cxx:3682
m
TMarker m
Definition textangle.C:8
Author
Lorenzo Moneta

Definition in file TMVA_RNN_Classification.C.