ROOT 6.14/05 Reference Guide |
Generic Deep Neural Network class.
This classs encapsulates the information for all types of Deep Neural Networks.
Architecture | The Architecture type that holds the architecture-specific data types. |
Public Types | |
using | Matrix_t = typename Architecture_t::Matrix_t |
using | Scalar_t = typename Architecture_t::Scalar_t |
Public Member Functions | |
TDeepNet () | |
Default Constructor. More... | |
TDeepNet (size_t BatchSize, size_t InputDepth, size_t InputHeight, size_t InputWidth, size_t BatchDepth, size_t BatchHeight, size_t BatchWidth, ELossFunction fJ, EInitialization fI=EInitialization::kZero, ERegularization fR=ERegularization::kNone, Scalar_t fWeightDecay=0.0, bool isTraining=false) | |
Constructor. More... | |
TDeepNet (const TDeepNet &) | |
Copy-constructor. More... | |
~TDeepNet () | |
Destructor. More... | |
TBasicRNNLayer< Architecture_t > * | AddBasicRNNLayer (size_t stateSize, size_t inputSize, size_t timeSteps, bool rememberState=false) |
Function for adding Recurrent Layer in the Deep Neural Network, with given parameters. More... | |
void | AddBasicRNNLayer (TBasicRNNLayer< Architecture_t > *basicRNNLayer) |
Function for adding Vanilla RNN when the layer is already created. More... | |
TConvLayer< Architecture_t > * | AddConvLayer (size_t depth, size_t filterHeight, size_t filterWidth, size_t strideRows, size_t strideCols, size_t paddingHeight, size_t paddingWidth, EActivationFunction f, Scalar_t dropoutProbability=1.0) |
Function for adding Convolution layer in the Deep Neural Network, with a given depth, filter height and width, striding in rows and columns, the zero paddings, as well as the activation function and the dropout probability. More... | |
void | AddConvLayer (TConvLayer< Architecture_t > *convLayer) |
Function for adding Convolution Layer in the Deep Neural Network, when the layer is already created. More... | |
TDenseLayer< Architecture_t > * | AddDenseLayer (size_t width, EActivationFunction f, Scalar_t dropoutProbability=1.0) |
Function for adding Dense Connected Layer in the Deep Neural Network, with a given width, activation function and dropout probability. More... | |
void | AddDenseLayer (TDenseLayer< Architecture_t > *denseLayer) |
Function for adding Dense Layer in the Deep Neural Network, when the layer is already created. More... | |
TMaxPoolLayer< Architecture_t > * | AddMaxPoolLayer (size_t frameHeight, size_t frameWidth, size_t strideRows, size_t strideCols, Scalar_t dropoutProbability=1.0) |
Function for adding Pooling layer in the Deep Neural Network, with a given filter height and width, striding in rows and columns as well as the dropout probability. More... | |
void | AddMaxPoolLayer (CNN::TMaxPoolLayer< Architecture_t > *maxPoolLayer) |
Function for adding Max Pooling layer in the Deep Neural Network, when the layer is already created. More... | |
TReshapeLayer< Architecture_t > * | AddReshapeLayer (size_t depth, size_t height, size_t width, bool flattening) |
Function for adding Reshape Layer in the Deep Neural Network, with a given height and width. More... | |
void | AddReshapeLayer (TReshapeLayer< Architecture_t > *reshapeLayer) |
Function for adding Reshape Layer in the Deep Neural Network, when the layer is already created. More... | |
void | Backward (std::vector< Matrix_t > &input, const Matrix_t &groundTruth, const Matrix_t &weights) |
Function that executes the entire backward pass in the network. More... | |
void | Clear () |
Remove all layers from the network. More... | |
void | Forward (std::vector< Matrix_t > &input, bool applyDropout=false) |
Function that executes the entire forward pass in the network. More... | |
size_t | GetBatchDepth () const |
size_t | GetBatchHeight () const |
size_t | GetBatchSize () const |
Getters. More... | |
size_t | GetBatchWidth () const |
size_t | GetDepth () const |
EInitialization | GetInitialization () const |
size_t | GetInputDepth () const |
size_t | GetInputHeight () const |
size_t | GetInputWidth () const |
Layer_t * | GetLayerAt (size_t i) |
Get the layer in the vector of layers at poistion i. More... | |
const Layer_t * | GetLayerAt (size_t i) const |
std::vector< Layer_t * > & | GetLayers () |
const std::vector< Layer_t * > & | GetLayers () const |
ELossFunction | GetLossFunction () const |
size_t | GetOutputWidth () const |
ERegularization | GetRegularization () const |
Scalar_t | GetWeightDecay () const |
void | Initialize () |
DAE functions. More... | |
bool | IsTraining () const |
Scalar_t | Loss (const Matrix_t &groundTruth, const Matrix_t &weights, bool includeRegularization=true) const |
Function for evaluating the loss, based on the activations stored in the last layer. More... | |
Scalar_t | Loss (std::vector< Matrix_t > &input, const Matrix_t &groundTruth, const Matrix_t &weights, bool applyDropout=false, bool includeRegularization=true) |
Function for evaluating the loss, based on the propagation of the given input. More... | |
void | ParallelBackward (std::vector< TDeepNet< Architecture_t, Layer_t >> &nets, std::vector< TTensorBatch< Architecture_t >> &batches, Scalar_t learningRate) |
Function for parallel backward in the vector of deep nets, where the master net is the net calling this function and getting the updates from the other nets. More... | |
void | ParallelBackwardMomentum (std::vector< TDeepNet< Architecture_t, Layer_t >> &nets, std::vector< TTensorBatch< Architecture_t >> &batches, Scalar_t learningRate, Scalar_t momentum) |
Function for parallel backward in the vector of deep nets, where the master net is the net calling this function and getting the updates from the other nets, following the momentum strategy. More... | |
void | ParallelBackwardNestorov (std::vector< TDeepNet< Architecture_t, Layer_t >> &nets, std::vector< TTensorBatch< Architecture_t >> &batches, Scalar_t learningRate, Scalar_t momentum) |
Function for parallel backward in the vector of deep nets, where the master net is the net calling this function and getting the updates from the other nets, following the Nestorov momentum strategy. More... | |
void | ParallelForward (std::vector< TDeepNet< Architecture_t, Layer_t >> &nets, std::vector< TTensorBatch< Architecture_t >> &batches, bool applyDropout=false) |
Function for parallel forward in the vector of deep nets, where the master net is the net calling this function. More... | |
void | Prediction (Matrix_t &predictions, EOutputFunction f) const |
Prediction based on activations stored in the last layer. More... | |
void | Prediction (Matrix_t &predictions, std::vector< Matrix_t > input, EOutputFunction f) |
Prediction for the given inputs, based on what network learned. More... | |
void | Print () const |
Print the Deep Net Info. More... | |
void | SetBatchDepth (size_t batchDepth) |
void | SetBatchHeight (size_t batchHeight) |
void | SetBatchSize (size_t batchSize) |
Setters. More... | |
void | SetBatchWidth (size_t batchWidth) |
void | SetInitialization (EInitialization I) |
void | SetInputDepth (size_t inputDepth) |
void | SetInputHeight (size_t inputHeight) |
void | SetInputWidth (size_t inputWidth) |
void | SetLossFunction (ELossFunction J) |
void | SetRegularization (ERegularization R) |
void | SetWeightDecay (Scalar_t weightDecay) |
void | Update (Scalar_t learningRate) |
Function that will update the weights and biases in the layers that contain weights and biases. More... | |
Private Member Functions | |
size_t | calculateDimension (int imgDim, int fltDim, int padding, int stride) |
bool | isInteger (Scalar_t x) const |
Private Attributes | |
size_t | fBatchDepth |
The depth of the batch used for training/testing. More... | |
size_t | fBatchHeight |
The height of the batch used for training/testing. More... | |
size_t | fBatchSize |
Batch size used for training and evaluation. More... | |
size_t | fBatchWidth |
The width of the batch used for training/testing. More... | |
EInitialization | fI |
The initialization method of the network. More... | |
size_t | fInputDepth |
The depth of the input. More... | |
size_t | fInputHeight |
The height of the input. More... | |
size_t | fInputWidth |
The width of the input. More... | |
bool | fIsTraining |
Is the network training? More... | |
ELossFunction | fJ |
The loss function of the network. More... | |
std::vector< Layer_t * > | fLayers |
The layers consisting the DeepNet. More... | |
ERegularization | fR |
The regularization used for the network. More... | |
Scalar_t | fWeightDecay |
The weight decay factor. More... | |
#include <TMVA/DNN/DeepNet.h>
using TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::Matrix_t = typename Architecture_t::Matrix_t |
using TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::Scalar_t = typename Architecture_t::Scalar_t |
TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::TDeepNet | ( | ) |
TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::TDeepNet | ( | size_t | BatchSize, |
size_t | InputDepth, | ||
size_t | InputHeight, | ||
size_t | InputWidth, | ||
size_t | BatchDepth, | ||
size_t | BatchHeight, | ||
size_t | BatchWidth, | ||
ELossFunction | fJ, | ||
EInitialization | fI = EInitialization::kZero , |
||
ERegularization | fR = ERegularization::kNone , |
||
Scalar_t | fWeightDecay = 0.0 , |
||
bool | isTraining = false |
||
) |
TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::TDeepNet | ( | const TDeepNet< Architecture_t, Layer_t > & | deepNet | ) |
TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::~TDeepNet | ( | ) |
TBasicRNNLayer< Architecture_t > * TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::AddBasicRNNLayer | ( | size_t | stateSize, |
size_t | inputSize, | ||
size_t | timeSteps, | ||
bool | rememberState = false |
||
) |
void TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::AddBasicRNNLayer | ( | TBasicRNNLayer< Architecture_t > * | basicRNNLayer | ) |
TConvLayer< Architecture_t > * TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::AddConvLayer | ( | size_t | depth, |
size_t | filterHeight, | ||
size_t | filterWidth, | ||
size_t | strideRows, | ||
size_t | strideCols, | ||
size_t | paddingHeight, | ||
size_t | paddingWidth, | ||
EActivationFunction | f, | ||
Scalar_t | dropoutProbability = 1.0 |
||
) |
Function for adding Convolution layer in the Deep Neural Network, with a given depth, filter height and width, striding in rows and columns, the zero paddings, as well as the activation function and the dropout probability.
Based on these parameters, it calculates the width and height of the convolutional layer.
void TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::AddConvLayer | ( | TConvLayer< Architecture_t > * | convLayer | ) |
TDenseLayer< Architecture_t > * TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::AddDenseLayer | ( | size_t | width, |
EActivationFunction | f, | ||
Scalar_t | dropoutProbability = 1.0 |
||
) |
void TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::AddDenseLayer | ( | TDenseLayer< Architecture_t > * | denseLayer | ) |
TMaxPoolLayer< Architecture_t > * TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::AddMaxPoolLayer | ( | size_t | frameHeight, |
size_t | frameWidth, | ||
size_t | strideRows, | ||
size_t | strideCols, | ||
Scalar_t | dropoutProbability = 1.0 |
||
) |
Function for adding Pooling layer in the Deep Neural Network, with a given filter height and width, striding in rows and columns as well as the dropout probability.
The depth is same as the previous layer depth. Based on these parameters, it calculates the width and height of the pooling layer.
void TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::AddMaxPoolLayer | ( | CNN::TMaxPoolLayer< Architecture_t > * | maxPoolLayer | ) |
TReshapeLayer< Architecture_t > * TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::AddReshapeLayer | ( | size_t | depth, |
size_t | height, | ||
size_t | width, | ||
bool | flattening | ||
) |
void TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::AddReshapeLayer | ( | TReshapeLayer< Architecture_t > * | reshapeLayer | ) |
auto TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::Backward | ( | std::vector< Matrix_t > & | input, |
const Matrix_t & | groundTruth, | ||
const Matrix_t & | weights | ||
) |
|
private |
|
inline |
auto TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::Forward | ( | std::vector< Matrix_t > & | input, |
bool | applyDropout = false |
||
) |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
auto TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::Initialize | ( | ) |
|
inlineprivate |
|
inline |
auto TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::Loss | ( | const Matrix_t & | groundTruth, |
const Matrix_t & | weights, | ||
bool | includeRegularization = true |
||
) | const |
auto TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::Loss | ( | std::vector< Matrix_t > & | input, |
const Matrix_t & | groundTruth, | ||
const Matrix_t & | weights, | ||
bool | applyDropout = false , |
||
bool | includeRegularization = true |
||
) |
auto TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::ParallelBackward | ( | std::vector< TDeepNet< Architecture_t, Layer_t >> & | nets, |
std::vector< TTensorBatch< Architecture_t >> & | batches, | ||
Scalar_t | learningRate | ||
) |
auto TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::ParallelBackwardMomentum | ( | std::vector< TDeepNet< Architecture_t, Layer_t >> & | nets, |
std::vector< TTensorBatch< Architecture_t >> & | batches, | ||
Scalar_t | learningRate, | ||
Scalar_t | momentum | ||
) |
auto TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::ParallelBackwardNestorov | ( | std::vector< TDeepNet< Architecture_t, Layer_t >> & | nets, |
std::vector< TTensorBatch< Architecture_t >> & | batches, | ||
Scalar_t | learningRate, | ||
Scalar_t | momentum | ||
) |
auto TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::ParallelForward | ( | std::vector< TDeepNet< Architecture_t, Layer_t >> & | nets, |
std::vector< TTensorBatch< Architecture_t >> & | batches, | ||
bool | applyDropout = false |
||
) |
auto TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::Prediction | ( | Matrix_t & | predictions, |
EOutputFunction | f | ||
) | const |
auto TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::Prediction | ( | Matrix_t & | predictions, |
std::vector< Matrix_t > | input, | ||
EOutputFunction | f | ||
) |
auto TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::Print | ( | ) | const |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
|
inline |
auto TMVA::DNN::TDeepNet< Architecture_t, Layer_t >::Update | ( | Scalar_t | learningRate | ) |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |