Logo ROOT   6.21/01
Reference Guide
TMVA::DNN::TLayer< Architecture_t > Class Template Reference

template<typename Architecture_t>
class TMVA::DNN::TLayer< Architecture_t >

Generic layer class.

This generic layer class represents a layer of a neural network with a given width n and activation function f. The activation function of each layer is given by \(\mathbf{u} = \mathbf{W}\mathbf{x} + \boldsymbol{\theta}\).

In addition to the weight and bias matrices, each layer allocates memory for its activations and the corresponding first partial fDerivatives of the activation function as well as the gradients of the fWeights and fBiases.

The layer provides member functions for the forward propagation of activations through the given layer.

Definition at line 52 of file Layer.h.

Public Types

using Matrix_t = typename Architecture_t::Matrix_t
 
using Scalar_t = typename Architecture_t::Scalar_t
 
using Tensor_t = typename Architecture_t::Tensor_t
 

Public Member Functions

 TLayer (size_t BatchSize, size_t InputWidth, size_t Width, EActivationFunction f, Scalar_t dropoutProbability)
 
 TLayer (const TLayer &)
 
void Backward (Matrix_t &gradients_backward, const Matrix_t &activations_backward, ERegularization r, Scalar_t weightDecay)
 Compute weight, bias and activation gradients. More...
 
void Forward (Matrix_t &input, bool applyDropout=false)
 Compute activation of the layer for the given input. More...
 
EActivationFunction GetActivationFunction () const
 
Matrix_tGetActivationGradients ()
 
const Matrix_tGetActivationGradients () const
 
size_t GetBatchSize () const
 
Matrix_tGetBiases ()
 
const Matrix_tGetBiases () const
 
Matrix_tGetBiasGradients ()
 
const Matrix_tGetBiasGradients () const
 
size_t GetDropoutProbability () const
 
size_t GetInputWidth () const
 
Matrix_tGetOutput ()
 
const Matrix_tGetOutput () const
 
Matrix_tGetWeightGradients ()
 
const Matrix_tGetWeightGradients () const
 
Matrix_tGetWeights ()
 
const Matrix_tGetWeights () const
 
size_t GetWidth () const
 
void Initialize (EInitialization m)
 Initialize fWeights according to the given initialization method. More...
 
void Print () const
 
void SetDropoutProbability (Scalar_t p)
 

Private Attributes

Matrix_t fActivationGradients
 Gradients w.r.t. the activations of this layer. More...
 
size_t fBatchSize
 Batch size used for training and evaluation. More...
 
Matrix_t fBiases
 The bias values of this layer. More...
 
Matrix_t fBiasGradients
 Gradients w.r.t. the bias values of this layer. More...
 
Matrix_t fDerivatives
 First fDerivatives of the activations of this layer. More...
 
Scalar_t fDropoutProbability
 Probability that an input is active. More...
 
EActivationFunction fF
 Activation function of the layer. More...
 
size_t fInputWidth
 Number of neurons of the previous layer. More...
 
Matrix_t fOutput
 Activations of this layer. More...
 
Matrix_t fWeightGradients
 Gradients w.r.t. the weigths of this layer. More...
 
Matrix_t fWeights
 The fWeights of this layer. More...
 
size_t fWidth
 Number of neurons of this layer. More...
 

#include <TMVA/DNN/Layer.h>

Member Typedef Documentation

◆ Matrix_t

template<typename Architecture_t>
using TMVA::DNN::TLayer< Architecture_t >::Matrix_t = typename Architecture_t::Matrix_t

Definition at line 57 of file Layer.h.

◆ Scalar_t

template<typename Architecture_t>
using TMVA::DNN::TLayer< Architecture_t >::Scalar_t = typename Architecture_t::Scalar_t

Definition at line 56 of file Layer.h.

◆ Tensor_t

template<typename Architecture_t>
using TMVA::DNN::TLayer< Architecture_t >::Tensor_t = typename Architecture_t::Tensor_t

Definition at line 58 of file Layer.h.

Constructor & Destructor Documentation

◆ TLayer() [1/2]

template<typename Architecture_t >
TMVA::DNN::TLayer< Architecture_t >::TLayer ( size_t  BatchSize,
size_t  InputWidth,
size_t  Width,
EActivationFunction  f,
Scalar_t  dropoutProbability 
)

Definition at line 227 of file Layer.h.

◆ TLayer() [2/2]

template<typename Architecture_t >
TMVA::DNN::TLayer< Architecture_t >::TLayer ( const TLayer< Architecture_t > &  layer)

Definition at line 243 of file Layer.h.

Member Function Documentation

◆ Backward()

template<typename Architecture_t >
auto TMVA::DNN::TLayer< Architecture_t >::Backward ( Matrix_t gradients_backward,
const Matrix_t activations_backward,
ERegularization  r,
Scalar_t  weightDecay 
)
inline

Compute weight, bias and activation gradients.

Uses the precomputed first partial derviatives of the activation function computed during forward propagation and modifies them. Must only be called directly a the corresponding call to Forward(...).

Definition at line 287 of file Layer.h.

◆ Forward()

template<typename Architecture_t >
auto TMVA::DNN::TLayer< Architecture_t >::Forward ( Matrix_t input,
bool  applyDropout = false 
)
inline

Compute activation of the layer for the given input.

The input must be in matrix form with the different rows corresponding to different events in the batch. Computes activations as well as the first partial derivative of the activation function at those activations.

Definition at line 269 of file Layer.h.

◆ GetActivationFunction()

template<typename Architecture_t>
EActivationFunction TMVA::DNN::TLayer< Architecture_t >::GetActivationFunction ( ) const
inline

Definition at line 115 of file Layer.h.

◆ GetActivationGradients() [1/2]

template<typename Architecture_t>
Matrix_t& TMVA::DNN::TLayer< Architecture_t >::GetActivationGradients ( )
inline

Definition at line 123 of file Layer.h.

◆ GetActivationGradients() [2/2]

template<typename Architecture_t>
const Matrix_t& TMVA::DNN::TLayer< Architecture_t >::GetActivationGradients ( ) const
inline

Definition at line 124 of file Layer.h.

◆ GetBatchSize()

template<typename Architecture_t>
size_t TMVA::DNN::TLayer< Architecture_t >::GetBatchSize ( ) const
inline

Definition at line 108 of file Layer.h.

◆ GetBiases() [1/2]

template<typename Architecture_t>
Matrix_t& TMVA::DNN::TLayer< Architecture_t >::GetBiases ( )
inline

Definition at line 121 of file Layer.h.

◆ GetBiases() [2/2]

template<typename Architecture_t>
const Matrix_t& TMVA::DNN::TLayer< Architecture_t >::GetBiases ( ) const
inline

Definition at line 122 of file Layer.h.

◆ GetBiasGradients() [1/2]

template<typename Architecture_t>
Matrix_t& TMVA::DNN::TLayer< Architecture_t >::GetBiasGradients ( )
inline

Definition at line 125 of file Layer.h.

◆ GetBiasGradients() [2/2]

template<typename Architecture_t>
const Matrix_t& TMVA::DNN::TLayer< Architecture_t >::GetBiasGradients ( ) const
inline

Definition at line 126 of file Layer.h.

◆ GetDropoutProbability()

template<typename Architecture_t>
size_t TMVA::DNN::TLayer< Architecture_t >::GetDropoutProbability ( ) const
inline

Definition at line 111 of file Layer.h.

◆ GetInputWidth()

template<typename Architecture_t>
size_t TMVA::DNN::TLayer< Architecture_t >::GetInputWidth ( ) const
inline

Definition at line 109 of file Layer.h.

◆ GetOutput() [1/2]

template<typename Architecture_t>
Matrix_t& TMVA::DNN::TLayer< Architecture_t >::GetOutput ( )
inline

Definition at line 117 of file Layer.h.

◆ GetOutput() [2/2]

template<typename Architecture_t>
const Matrix_t& TMVA::DNN::TLayer< Architecture_t >::GetOutput ( ) const
inline

Definition at line 118 of file Layer.h.

◆ GetWeightGradients() [1/2]

template<typename Architecture_t>
Matrix_t& TMVA::DNN::TLayer< Architecture_t >::GetWeightGradients ( )
inline

Definition at line 127 of file Layer.h.

◆ GetWeightGradients() [2/2]

template<typename Architecture_t>
const Matrix_t& TMVA::DNN::TLayer< Architecture_t >::GetWeightGradients ( ) const
inline

Definition at line 128 of file Layer.h.

◆ GetWeights() [1/2]

template<typename Architecture_t>
Matrix_t& TMVA::DNN::TLayer< Architecture_t >::GetWeights ( )
inline

Definition at line 119 of file Layer.h.

◆ GetWeights() [2/2]

template<typename Architecture_t>
const Matrix_t& TMVA::DNN::TLayer< Architecture_t >::GetWeights ( ) const
inline

Definition at line 120 of file Layer.h.

◆ GetWidth()

template<typename Architecture_t>
size_t TMVA::DNN::TLayer< Architecture_t >::GetWidth ( ) const
inline

Definition at line 110 of file Layer.h.

◆ Initialize()

template<typename Architecture_t >
auto TMVA::DNN::TLayer< Architecture_t >::Initialize ( EInitialization  m)

Initialize fWeights according to the given initialization method.

Definition at line 260 of file Layer.h.

◆ Print()

template<typename Architecture_t >
void TMVA::DNN::TLayer< Architecture_t >::Print ( ) const

Definition at line 314 of file Layer.h.

◆ SetDropoutProbability()

template<typename Architecture_t>
void TMVA::DNN::TLayer< Architecture_t >::SetDropoutProbability ( Scalar_t  p)
inline

Definition at line 113 of file Layer.h.

Member Data Documentation

◆ fActivationGradients

template<typename Architecture_t>
Matrix_t TMVA::DNN::TLayer< Architecture_t >::fActivationGradients
private

Gradients w.r.t. the activations of this layer.

Definition at line 75 of file Layer.h.

◆ fBatchSize

template<typename Architecture_t>
size_t TMVA::DNN::TLayer< Architecture_t >::fBatchSize
private

Batch size used for training and evaluation.

Definition at line 63 of file Layer.h.

◆ fBiases

template<typename Architecture_t>
Matrix_t TMVA::DNN::TLayer< Architecture_t >::fBiases
private

The bias values of this layer.

Definition at line 70 of file Layer.h.

◆ fBiasGradients

template<typename Architecture_t>
Matrix_t TMVA::DNN::TLayer< Architecture_t >::fBiasGradients
private

Gradients w.r.t. the bias values of this layer.

Definition at line 74 of file Layer.h.

◆ fDerivatives

template<typename Architecture_t>
Matrix_t TMVA::DNN::TLayer< Architecture_t >::fDerivatives
private

First fDerivatives of the activations of this layer.

Definition at line 72 of file Layer.h.

◆ fDropoutProbability

template<typename Architecture_t>
Scalar_t TMVA::DNN::TLayer< Architecture_t >::fDropoutProbability
private

Probability that an input is active.

Definition at line 67 of file Layer.h.

◆ fF

template<typename Architecture_t>
EActivationFunction TMVA::DNN::TLayer< Architecture_t >::fF
private

Activation function of the layer.

Definition at line 77 of file Layer.h.

◆ fInputWidth

template<typename Architecture_t>
size_t TMVA::DNN::TLayer< Architecture_t >::fInputWidth
private

Number of neurons of the previous layer.

Definition at line 64 of file Layer.h.

◆ fOutput

template<typename Architecture_t>
Matrix_t TMVA::DNN::TLayer< Architecture_t >::fOutput
private

Activations of this layer.

Definition at line 71 of file Layer.h.

◆ fWeightGradients

template<typename Architecture_t>
Matrix_t TMVA::DNN::TLayer< Architecture_t >::fWeightGradients
private

Gradients w.r.t. the weigths of this layer.

Definition at line 73 of file Layer.h.

◆ fWeights

template<typename Architecture_t>
Matrix_t TMVA::DNN::TLayer< Architecture_t >::fWeights
private

The fWeights of this layer.

Definition at line 69 of file Layer.h.

◆ fWidth

template<typename Architecture_t>
size_t TMVA::DNN::TLayer< Architecture_t >::fWidth
private

Number of neurons of this layer.

Definition at line 65 of file Layer.h.


The documentation for this class was generated from the following file: