ROOT 6.14/05 Reference Guide |
Generic layer class.
This generic layer class represents a dense layer of a neural network with a given width n and activation function f. The activation function of each layer is given by \(\mathbf{u} = \mathbf{W}\mathbf{x} + \boldsymbol{\theta}\).
In addition to the weight and bias matrices, each layer allocates memory for its activations and the corresponding first partial fDerivatives of the activation function as well as the gradients of the weights and biases.
The layer provides member functions for the forward propagation of activations through the given layer.
Definition at line 55 of file DenseLayer.h.
Public Types | |
using | Matrix_t = typename Architecture_t::Matrix_t |
using | Scalar_t = typename Architecture_t::Scalar_t |
Public Member Functions | |
TDenseLayer (size_t BatchSize, size_t InputWidth, size_t Width, EInitialization init, Scalar_t DropoutProbability, EActivationFunction f, ERegularization reg, Scalar_t weightDecay) | |
Constructor. More... | |
TDenseLayer (TDenseLayer< Architecture_t > *layer) | |
Copy the dense layer provided as a pointer. More... | |
TDenseLayer (const TDenseLayer &) | |
Copy Constructor. More... | |
~TDenseLayer () | |
Destructor. More... | |
virtual void | AddWeightsXMLTo (void *parent) |
Writes the information and the weights about the layer in an XML node. More... | |
void | Backward (std::vector< Matrix_t > &gradients_backward, const std::vector< Matrix_t > &activations_backward, std::vector< Matrix_t > &inp1, std::vector< Matrix_t > &inp2) |
Compute weight, bias and activation gradients. More... | |
void | Forward (std::vector< Matrix_t > &input, bool applyDropout=false) |
Compute activation of the layer for the given input. More... | |
EActivationFunction | GetActivationFunction () const |
const std::vector< Matrix_t > & | GetDerivatives () const |
std::vector< Matrix_t > & | GetDerivatives () |
Matrix_t & | GetDerivativesAt (size_t i) |
const Matrix_t & | GetDerivativesAt (size_t i) const |
Scalar_t | GetDropoutProbability () const |
Getters. More... | |
ERegularization | GetRegularization () const |
Scalar_t | GetWeightDecay () const |
void | Print () const |
Printing the layer info. More... | |
virtual void | ReadWeightsFromXML (void *parent) |
Read the information and the weights about the layer from XML node. More... | |
Public Member Functions inherited from TMVA::DNN::VGeneralLayer< Architecture_t > | |
VGeneralLayer (size_t BatchSize, size_t InputDepth, size_t InputHeight, size_t InputWidth, size_t Depth, size_t Height, size_t Width, size_t WeightsNSlices, size_t WeightsNRows, size_t WeightsNCols, size_t BiasesNSlices, size_t BiasesNRows, size_t BiasesNCols, size_t OutputNSlices, size_t OutputNRows, size_t OutputNCols, EInitialization Init) | |
Constructor. More... | |
VGeneralLayer (size_t BatchSize, size_t InputDepth, size_t InputHeight, size_t InputWidth, size_t Depth, size_t Height, size_t Width, size_t WeightsNSlices, std::vector< size_t > WeightsNRows, std::vector< size_t > WeightsNCols, size_t BiasesNSlices, std::vector< size_t > BiasesNRows, std::vector< size_t > BiasesNCols, size_t OutputNSlices, size_t OutputNRows, size_t OutputNCols, EInitialization Init) | |
General Constructor with different weights dimension. More... | |
VGeneralLayer (VGeneralLayer< Architecture_t > *layer) | |
Copy the layer provided as a pointer. More... | |
VGeneralLayer (const VGeneralLayer &) | |
Copy Constructor. More... | |
virtual | ~VGeneralLayer () |
Virtual Destructor. More... | |
void | CopyBiases (const std::vector< Matrix_t > &otherBiases) |
Copies the biases provided as an input. More... | |
void | CopyWeights (const std::vector< Matrix_t > &otherWeights) |
Copies the weights provided as an input. More... | |
const std::vector< Matrix_t > & | GetActivationGradients () const |
std::vector< Matrix_t > & | GetActivationGradients () |
Matrix_t & | GetActivationGradientsAt (size_t i) |
const Matrix_t & | GetActivationGradientsAt (size_t i) const |
size_t | GetBatchSize () const |
Getters. More... | |
const std::vector< Matrix_t > & | GetBiases () const |
std::vector< Matrix_t > & | GetBiases () |
const Matrix_t & | GetBiasesAt (size_t i) const |
Matrix_t & | GetBiasesAt (size_t i) |
const std::vector< Matrix_t > & | GetBiasGradients () const |
std::vector< Matrix_t > & | GetBiasGradients () |
const Matrix_t & | GetBiasGradientsAt (size_t i) const |
Matrix_t & | GetBiasGradientsAt (size_t i) |
size_t | GetDepth () const |
size_t | GetHeight () const |
EInitialization | GetInitialization () const |
size_t | GetInputDepth () const |
size_t | GetInputHeight () const |
size_t | GetInputWidth () const |
const std::vector< Matrix_t > & | GetOutput () const |
std::vector< Matrix_t > & | GetOutput () |
Matrix_t & | GetOutputAt (size_t i) |
const Matrix_t & | GetOutputAt (size_t i) const |
const std::vector< Matrix_t > & | GetWeightGradients () const |
std::vector< Matrix_t > & | GetWeightGradients () |
const Matrix_t & | GetWeightGradientsAt (size_t i) const |
Matrix_t & | GetWeightGradientsAt (size_t i) |
const std::vector< Matrix_t > & | GetWeights () const |
std::vector< Matrix_t > & | GetWeights () |
const Matrix_t & | GetWeightsAt (size_t i) const |
Matrix_t & | GetWeightsAt (size_t i) |
size_t | GetWidth () const |
void | Initialize () |
Initialize the weights and biases according to the given initialization method. More... | |
bool | IsTraining () const |
void | ReadMatrixXML (void *node, const char *name, Matrix_t &matrix) |
void | SetBatchSize (size_t batchSize) |
Setters. More... | |
void | SetDepth (size_t depth) |
void | SetHeight (size_t height) |
void | SetInputDepth (size_t inputDepth) |
void | SetInputHeight (size_t inputHeight) |
void | SetInputWidth (size_t inputWidth) |
void | SetIsTraining (bool isTraining) |
void | SetWidth (size_t width) |
void | Update (const Scalar_t learningRate) |
Updates the weights and biases, given the learning rate. More... | |
void | UpdateBiases (const std::vector< Matrix_t > &biasGradients, const Scalar_t learningRate) |
Updates the biases, given the gradients and the learning rate. More... | |
void | UpdateBiasGradients (const std::vector< Matrix_t > &biasGradients, const Scalar_t learningRate) |
Updates the bias gradients, given some other weight gradients and learning rate. More... | |
void | UpdateWeightGradients (const std::vector< Matrix_t > &weightGradients, const Scalar_t learningRate) |
Updates the weight gradients, given some other weight gradients and learning rate. More... | |
void | UpdateWeights (const std::vector< Matrix_t > &weightGradients, const Scalar_t learningRate) |
Updates the weights, given the gradients and the learning rate,. More... | |
void | WriteMatrixToXML (void *node, const char *name, const Matrix_t &matrix) |
void | WriteTensorToXML (void *node, const char *name, const std::vector< Matrix_t > &tensor) |
helper functions for XML More... | |
Private Attributes | |
std::vector< Matrix_t > | fDerivatives |
First fDerivatives of the activations of this layer. More... | |
Scalar_t | fDropoutProbability |
Probability that an input is active. More... | |
EActivationFunction | fF |
Activation function of the layer. More... | |
ERegularization | fReg |
The regularization method. More... | |
Scalar_t | fWeightDecay |
The weight decay. More... | |
Additional Inherited Members | |
Protected Attributes inherited from TMVA::DNN::VGeneralLayer< Architecture_t > | |
std::vector< Matrix_t > | fActivationGradients |
Gradients w.r.t. the activations of this layer. More... | |
size_t | fBatchSize |
Batch size used for training and evaluation. More... | |
std::vector< Matrix_t > | fBiases |
The biases associated to the layer. More... | |
std::vector< Matrix_t > | fBiasGradients |
Gradients w.r.t. the bias values of the layer. More... | |
size_t | fDepth |
The depth of the layer. More... | |
size_t | fHeight |
The height of the layer. More... | |
EInitialization | fInit |
The initialization method. More... | |
size_t | fInputDepth |
The depth of the previous layer or input. More... | |
size_t | fInputHeight |
The height of the previous layer or input. More... | |
size_t | fInputWidth |
The width of the previous layer or input. More... | |
bool | fIsTraining |
Flag indicatig the mode. More... | |
std::vector< Matrix_t > | fOutput |
Activations of this layer. More... | |
std::vector< Matrix_t > | fWeightGradients |
Gradients w.r.t. the weights of the layer. More... | |
std::vector< Matrix_t > | fWeights |
The weights associated to the layer. More... | |
size_t | fWidth |
The width of this layer. More... | |
#include <TMVA/DNN/DenseLayer.h>
using TMVA::DNN::TDenseLayer< Architecture_t >::Matrix_t = typename Architecture_t::Matrix_t |
Definition at line 58 of file DenseLayer.h.
using TMVA::DNN::TDenseLayer< Architecture_t >::Scalar_t = typename Architecture_t::Scalar_t |
Definition at line 57 of file DenseLayer.h.
TMVA::DNN::TDenseLayer< Architecture_t >::TDenseLayer | ( | size_t | BatchSize, |
size_t | InputWidth, | ||
size_t | Width, | ||
EInitialization | init, | ||
Scalar_t | DropoutProbability, | ||
EActivationFunction | f, | ||
ERegularization | reg, | ||
Scalar_t | weightDecay | ||
) |
Constructor.
Definition at line 126 of file DenseLayer.h.
TMVA::DNN::TDenseLayer< Architecture_t >::TDenseLayer | ( | TDenseLayer< Architecture_t > * | layer | ) |
Copy the dense layer provided as a pointer.
Definition at line 138 of file DenseLayer.h.
TMVA::DNN::TDenseLayer< Architecture_t >::TDenseLayer | ( | const TDenseLayer< Architecture_t > & | layer | ) |
Copy Constructor.
Definition at line 147 of file DenseLayer.h.
TMVA::DNN::TDenseLayer< Architecture_t >::~TDenseLayer | ( | ) |
Destructor.
Definition at line 156 of file DenseLayer.h.
|
virtual |
Writes the information and the weights about the layer in an XML node.
Implements TMVA::DNN::VGeneralLayer< Architecture_t >.
Definition at line 215 of file DenseLayer.h.
|
virtual |
Compute weight, bias and activation gradients.
Uses the precomputed first partial derviatives of the activation function computed during forward propagation and modifies them. Must only be called directly a the corresponding call to Forward(...).
Implements TMVA::DNN::VGeneralLayer< Architecture_t >.
Definition at line 176 of file DenseLayer.h.
|
virtual |
Compute activation of the layer for the given input.
The input must be in 3D tensor form with the different matrices corresponding to different events in the batch. Computes activations as well as the first partial derivative of the activation function at those activations.
Implements TMVA::DNN::VGeneralLayer< Architecture_t >.
Definition at line 163 of file DenseLayer.h.
|
inline |
Definition at line 116 of file DenseLayer.h.
|
inline |
Definition at line 110 of file DenseLayer.h.
|
inline |
Definition at line 111 of file DenseLayer.h.
|
inline |
Definition at line 113 of file DenseLayer.h.
|
inline |
Definition at line 114 of file DenseLayer.h.
|
inline |
Getters.
Definition at line 108 of file DenseLayer.h.
|
inline |
Definition at line 117 of file DenseLayer.h.
|
inline |
Definition at line 118 of file DenseLayer.h.
|
virtual |
Printing the layer info.
Implements TMVA::DNN::VGeneralLayer< Architecture_t >.
Definition at line 199 of file DenseLayer.h.
|
virtual |
Read the information and the weights about the layer from XML node.
Implements TMVA::DNN::VGeneralLayer< Architecture_t >.
Definition at line 233 of file DenseLayer.h.
|
private |
First fDerivatives of the activations of this layer.
Definition at line 61 of file DenseLayer.h.
|
private |
Probability that an input is active.
Definition at line 63 of file DenseLayer.h.
|
private |
Activation function of the layer.
Definition at line 65 of file DenseLayer.h.
|
private |
The regularization method.
Definition at line 66 of file DenseLayer.h.
|
private |
The weight decay.
Definition at line 67 of file DenseLayer.h.