Logo ROOT   6.10/09
Reference Guide
List of all members | Public Types | Public Member Functions | Private Attributes | List of all members
TMVA::DNN::TNet< Architecture_t, Layer_t > Class Template Reference

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
class TMVA::DNN::TNet< Architecture_t, Layer_t >

Generic neural network class.

This generic neural network class represents a concrete neural network through a vector of layers and coordinates the forward and backward propagation through the net.

The net takes as input a batch from the training data given in matrix form, with each row corresponding to a certain training event.

On construction, the neural network allocates all the memory required for the training of the neural net and keeps it until its destruction.

The Architecture type argument simply holds the architecture-specific data types, which are just the matrix type Matrix_t and the used scalar type Scalar_t.

Template Parameters
ArchitectureThe Architecture type that holds the
Layer_tThe type used for the layers. Can be either Layer<Architecture> or SharedWeightLayer<Architecture>. datatypes for a given architecture.

Definition at line 49 of file Net.h.

Public Types

using LayerIterator_t = typename std::vector< Layer_t >::iterator
 
using Matrix_t = typename Architecture_t::Matrix_t
 
using Scalar_t = typename Architecture_t::Scalar_t
 

Public Member Functions

 TNet ()
 
 TNet (const TNet &other)
 
template<typename OtherArchitecture_t >
 TNet (size_t batchSize, const TNet< OtherArchitecture_t > &)
 
 TNet (size_t batchSize, size_t inputWidth, ELossFunction fJ, ERegularization fR=ERegularization::kNone, Scalar_t fWeightDecay=0.0)
 Construct a neural net for a given batch size with given output function * and regularization. More...
 
void AddLayer (size_t width, EActivationFunction f, Scalar_t dropoutProbability=1.0)
 Add a layer of the given size to the neural net. More...
 
template<typename SharedLayer >
void AddLayer (SharedLayer &layer)
 Add a layer which shares its weights with another TNet instance. More...
 
template<typename SharedLayer_t >
void AddLayer (SharedLayer_t &layer)
 
void Backward (const Matrix_t &X, const Matrix_t &Y)
 Compute the weight gradients in the net from the given training samples X and training labels Y. More...
 
void Clear ()
 Remove all layers from the network. More...
 
TNet< Architecture_t, TSharedLayer< Architecture_t > > CreateClone (size_t batchSize)
 Create a clone that uses the same weight and biases matrices but potentially a difference batch size. More...
 
void Forward (Matrix_t &X, bool applyDropout=false)
 Forward a given input through the neural net. More...
 
size_t GetBatchSize () const
 
size_t GetDepth () const
 
size_t GetInputWidth () const
 
Layer_t & GetLayer (size_t i)
 
const Layer_t & GetLayer (size_t i) const
 
ELossFunction GetLossFunction () const
 
Scalar_t GetNFlops ()
 
Matrix_tGetOutput ()
 
size_t GetOutputWidth () const
 
ERegularization GetRegularization () const
 
Scalar_t GetWeightDecay () const
 
void Initialize (EInitialization m)
 Initialize the weights in the net with the initialization method. More...
 
void InitializeGradients ()
 Initialize the gradients in the net to zero. More...
 
LayerIterator_t LayersBegin ()
 Iterator to the first layer of the net. More...
 
LayerIterator_t LayersEnd ()
 Iterator to the last layer of the net. More...
 
Scalar_t Loss (const Matrix_t &Y, bool includeRegularization=true) const
 Evaluate the loss function of the net using the activations that are currently stored in the output layer. More...
 
Scalar_t Loss (Matrix_t &X, const Matrix_t &Y, bool applyDropout=false)
 Propagate the input batch X through the net and evaluate the error function for the resulting activations of the output layer. More...
 
void Prediction (Matrix_t &Y_hat, Matrix_t &X, EOutputFunction f)
 Compute the neural network predictionion obtained from forwarding the batch X through the neural network and applying the output function f to the activation of the last layer in the network. More...
 
void Prediction (Matrix_t &Y_hat, EOutputFunction f) const
 Compute the neural network rediction obtained from applying the output function f to the activation of the last layer in the network. More...
 
void Print ()
 
void SetBatchSize (size_t batchSize)
 
void SetDropoutProbabilities (const std::vector< Double_t > &probabilities)
 
void SetInputWidth (size_t inputWidth)
 
void SetLossFunction (ELossFunction J)
 
void SetRegularization (ERegularization R)
 
void SetWeightDecay (Scalar_t weightDecay)
 

Private Attributes

size_t fBatchSize
 Batch size for training and evaluation of the Network. More...
 
Matrix_t fDummy
 Empty matrix for last step in back propagation. More...
 
size_t fInputWidth
 Number of features in a single input event. More...
 
ELossFunction fJ
 The loss function of the network. More...
 
std::vector< Layer_t > fLayers
 Layers in the network. More...
 
ERegularization fR
 The regularization used for the network. More...
 
Scalar_t fWeightDecay
 The weight decay factor. More...
 

#include <TMVA/DNN/Net.h>

Member Typedef Documentation

◆ LayerIterator_t

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
using TMVA::DNN::TNet< Architecture_t, Layer_t >::LayerIterator_t = typename std::vector<Layer_t>::iterator

Definition at line 54 of file Net.h.

◆ Matrix_t

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
using TMVA::DNN::TNet< Architecture_t, Layer_t >::Matrix_t = typename Architecture_t::Matrix_t

Definition at line 52 of file Net.h.

◆ Scalar_t

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
using TMVA::DNN::TNet< Architecture_t, Layer_t >::Scalar_t = typename Architecture_t::Scalar_t

Definition at line 53 of file Net.h.

Constructor & Destructor Documentation

◆ TNet() [1/4]

template<typename Architecture_t , typename Layer_t >
TMVA::DNN::TNet< Architecture_t, Layer_t >::TNet ( )

Definition at line 159 of file Net.h.

◆ TNet() [2/4]

template<typename Architecture_t , typename Layer_t >
TMVA::DNN::TNet< Architecture_t, Layer_t >::TNet ( const TNet< Architecture_t, Layer_t > &  other)

Definition at line 169 of file Net.h.

◆ TNet() [3/4]

template<typename Architecture_t , typename Layer_t >
template<typename OtherArchitecture_t >
TMVA::DNN::TNet< Architecture_t, Layer_t >::TNet ( size_t  batchSize,
const TNet< OtherArchitecture_t > &  other 
)

Definition at line 180 of file Net.h.

◆ TNet() [4/4]

template<typename Architecture_t , typename Layer_t >
TMVA::DNN::TNet< Architecture_t, Layer_t >::TNet ( size_t  batchSize,
size_t  inputWidth,
ELossFunction  fJ,
ERegularization  fR = ERegularization::kNone,
Scalar_t  fWeightDecay = 0.0 
)

Construct a neural net for a given batch size with given output function * and regularization.

Definition at line 198 of file Net.h.

Member Function Documentation

◆ AddLayer() [1/3]

template<typename Architecture_t , typename Layer_t >
void TMVA::DNN::TNet< Architecture_t, Layer_t >::AddLayer ( size_t  width,
EActivationFunction  f,
Scalar_t  dropoutProbability = 1.0 
)

Add a layer of the given size to the neural net.

Definition at line 224 of file Net.h.

◆ AddLayer() [2/3]

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
template<typename SharedLayer >
void TMVA::DNN::TNet< Architecture_t, Layer_t >::AddLayer ( SharedLayer &  layer)

Add a layer which shares its weights with another TNet instance.

◆ AddLayer() [3/3]

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
template<typename SharedLayer_t >
void TMVA::DNN::TNet< Architecture_t, Layer_t >::AddLayer ( SharedLayer_t &  layer)
inline

Definition at line 246 of file Net.h.

◆ Backward()

template<typename Architecture_t , typename Layer_t >
void TMVA::DNN::TNet< Architecture_t, Layer_t >::Backward ( const Matrix_t X,
const Matrix_t Y 
)
inline

Compute the weight gradients in the net from the given training samples X and training labels Y.

Definition at line 284 of file Net.h.

◆ Clear()

template<typename Architecture_t , typename Layer_t >
void TMVA::DNN::TNet< Architecture_t, Layer_t >::Clear ( )

Remove all layers from the network.

Definition at line 238 of file Net.h.

◆ CreateClone()

template<typename Architecture_t , typename Layer_t >
auto TMVA::DNN::TNet< Architecture_t, Layer_t >::CreateClone ( size_t  batchSize)

Create a clone that uses the same weight and biases matrices but potentially a difference batch size.

Definition at line 211 of file Net.h.

◆ Forward()

template<typename Architecture_t , typename Layer_t >
void TMVA::DNN::TNet< Architecture_t, Layer_t >::Forward ( Matrix_t X,
bool  applyDropout = false 
)
inline

Forward a given input through the neural net.

Computes all layer activations up to the output layer

Definition at line 272 of file Net.h.

◆ GetBatchSize()

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
size_t TMVA::DNN::TNet< Architecture_t, Layer_t >::GetBatchSize ( ) const
inline

Definition at line 137 of file Net.h.

◆ GetDepth()

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
size_t TMVA::DNN::TNet< Architecture_t, Layer_t >::GetDepth ( ) const
inline

Definition at line 136 of file Net.h.

◆ GetInputWidth()

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
size_t TMVA::DNN::TNet< Architecture_t, Layer_t >::GetInputWidth ( ) const
inline

Definition at line 142 of file Net.h.

◆ GetLayer() [1/2]

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
Layer_t& TMVA::DNN::TNet< Architecture_t, Layer_t >::GetLayer ( size_t  i)
inline

Definition at line 138 of file Net.h.

◆ GetLayer() [2/2]

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
const Layer_t& TMVA::DNN::TNet< Architecture_t, Layer_t >::GetLayer ( size_t  i) const
inline

Definition at line 139 of file Net.h.

◆ GetLossFunction()

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
ELossFunction TMVA::DNN::TNet< Architecture_t, Layer_t >::GetLossFunction ( ) const
inline

Definition at line 140 of file Net.h.

◆ GetNFlops()

template<typename Architecture_t , typename Layer_t >
auto TMVA::DNN::TNet< Architecture_t, Layer_t >::GetNFlops ( )

Definition at line 350 of file Net.h.

◆ GetOutput()

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
Matrix_t& TMVA::DNN::TNet< Architecture_t, Layer_t >::GetOutput ( )
inline

Definition at line 141 of file Net.h.

◆ GetOutputWidth()

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
size_t TMVA::DNN::TNet< Architecture_t, Layer_t >::GetOutputWidth ( ) const
inline

Definition at line 143 of file Net.h.

◆ GetRegularization()

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
ERegularization TMVA::DNN::TNet< Architecture_t, Layer_t >::GetRegularization ( ) const
inline

Definition at line 144 of file Net.h.

◆ GetWeightDecay()

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
Scalar_t TMVA::DNN::TNet< Architecture_t, Layer_t >::GetWeightDecay ( ) const
inline

Definition at line 145 of file Net.h.

◆ Initialize()

template<typename Architecture_t , typename Layer_t >
void TMVA::DNN::TNet< Architecture_t, Layer_t >::Initialize ( EInitialization  m)
inline

Initialize the weights in the net with the initialization method.

Definition at line 253 of file Net.h.

◆ InitializeGradients()

template<typename Architecture_t , typename Layer_t >
void TMVA::DNN::TNet< Architecture_t, Layer_t >::InitializeGradients ( )
inline

Initialize the gradients in the net to zero.

Required if net is used to store velocities of momentum-based minimization techniques.

Definition at line 262 of file Net.h.

◆ LayersBegin()

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
LayerIterator_t TMVA::DNN::TNet< Architecture_t, Layer_t >::LayersBegin ( )
inline

Iterator to the first layer of the net.

Definition at line 95 of file Net.h.

◆ LayersEnd()

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
LayerIterator_t TMVA::DNN::TNet< Architecture_t, Layer_t >::LayersEnd ( )
inline

Iterator to the last layer of the net.

Definition at line 98 of file Net.h.

◆ Loss() [1/2]

template<typename Architecture_t , typename Layer_t >
auto TMVA::DNN::TNet< Architecture_t, Layer_t >::Loss ( const Matrix_t Y,
bool  includeRegularization = true 
) const
inline

Evaluate the loss function of the net using the activations that are currently stored in the output layer.

Definition at line 305 of file Net.h.

◆ Loss() [2/2]

template<typename Architecture_t , typename Layer_t >
auto TMVA::DNN::TNet< Architecture_t, Layer_t >::Loss ( Matrix_t X,
const Matrix_t Y,
bool  applyDropout = false 
)
inline

Propagate the input batch X through the net and evaluate the error function for the resulting activations of the output layer.

Definition at line 321 of file Net.h.

◆ Prediction() [1/2]

template<typename Architecture_t , typename Layer_t >
void TMVA::DNN::TNet< Architecture_t, Layer_t >::Prediction ( Matrix_t Y_hat,
Matrix_t X,
EOutputFunction  f 
)
inline

Compute the neural network predictionion obtained from forwarding the batch X through the neural network and applying the output function f to the activation of the last layer in the network.

Definition at line 332 of file Net.h.

◆ Prediction() [2/2]

template<typename Architecture_t , typename Layer_t >
void TMVA::DNN::TNet< Architecture_t, Layer_t >::Prediction ( Matrix_t Y_hat,
EOutputFunction  f 
) const
inline

Compute the neural network rediction obtained from applying the output function f to the activation of the last layer in the network.

Definition at line 342 of file Net.h.

◆ Print()

template<typename Architecture_t , typename Layer_t >
void TMVA::DNN::TNet< Architecture_t, Layer_t >::Print ( )

Definition at line 395 of file Net.h.

◆ SetBatchSize()

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
void TMVA::DNN::TNet< Architecture_t, Layer_t >::SetBatchSize ( size_t  batchSize)
inline

Definition at line 147 of file Net.h.

◆ SetDropoutProbabilities()

template<typename Architecture_t , typename Layer_t >
void TMVA::DNN::TNet< Architecture_t, Layer_t >::SetDropoutProbabilities ( const std::vector< Double_t > &  probabilities)

Definition at line 381 of file Net.h.

◆ SetInputWidth()

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
void TMVA::DNN::TNet< Architecture_t, Layer_t >::SetInputWidth ( size_t  inputWidth)
inline

Definition at line 148 of file Net.h.

◆ SetLossFunction()

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
void TMVA::DNN::TNet< Architecture_t, Layer_t >::SetLossFunction ( ELossFunction  J)
inline

Definition at line 150 of file Net.h.

◆ SetRegularization()

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
void TMVA::DNN::TNet< Architecture_t, Layer_t >::SetRegularization ( ERegularization  R)
inline

Definition at line 149 of file Net.h.

◆ SetWeightDecay()

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
void TMVA::DNN::TNet< Architecture_t, Layer_t >::SetWeightDecay ( Scalar_t  weightDecay)
inline

Definition at line 151 of file Net.h.

Member Data Documentation

◆ fBatchSize

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
size_t TMVA::DNN::TNet< Architecture_t, Layer_t >::fBatchSize
private

Batch size for training and evaluation of the Network.

Definition at line 57 of file Net.h.

◆ fDummy

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
Matrix_t TMVA::DNN::TNet< Architecture_t, Layer_t >::fDummy
private

Empty matrix for last step in back propagation.

Definition at line 62 of file Net.h.

◆ fInputWidth

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
size_t TMVA::DNN::TNet< Architecture_t, Layer_t >::fInputWidth
private

Number of features in a single input event.

Definition at line 58 of file Net.h.

◆ fJ

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
ELossFunction TMVA::DNN::TNet< Architecture_t, Layer_t >::fJ
private

The loss function of the network.

Definition at line 63 of file Net.h.

◆ fLayers

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
std::vector<Layer_t> TMVA::DNN::TNet< Architecture_t, Layer_t >::fLayers
private

Layers in the network.

Definition at line 60 of file Net.h.

◆ fR

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
ERegularization TMVA::DNN::TNet< Architecture_t, Layer_t >::fR
private

The regularization used for the network.

Definition at line 64 of file Net.h.

◆ fWeightDecay

template<typename Architecture_t, typename Layer_t = TLayer<Architecture_t>>
Scalar_t TMVA::DNN::TNet< Architecture_t, Layer_t >::fWeightDecay
private

The weight decay factor.

Definition at line 65 of file Net.h.


The documentation for this class was generated from the following file: