Logo ROOT  
Reference Guide
TMVA::DNN::TSGD< Architecture_t, Layer_t, DeepNet_t > Class Template Reference

template<typename Architecture_t, typename Layer_t = VGeneralLayer<Architecture_t>, typename DeepNet_t = TDeepNet<Architecture_t, Layer_t>>
class TMVA::DNN::TSGD< Architecture_t, Layer_t, DeepNet_t >

Stochastic Batch Gradient Descent Optimizer class.

This class represents the Stochastic Batch Gradient Descent Optimizer with options for applying momentum and nesterov momentum.

Definition at line 45 of file SGD.h.

Public Types

using Matrix_t = typename Architecture_t::Matrix_t
 
using Scalar_t = typename Architecture_t::Scalar_t
 
- Public Types inherited from TMVA::DNN::VOptimizer< Architecture_t, VGeneralLayer< Architecture_t >, TDeepNet< Architecture_t, VGeneralLayer< Architecture_t > > >
using Matrix_t = typename Architecture_t::Matrix_t
 
using Scalar_t = typename Architecture_t::Scalar_t
 

Public Member Functions

 TSGD (Scalar_t learningRate, DeepNet_t &deepNet, Scalar_t momentum)
 Constructor. More...
 
 ~TSGD ()=default
 Destructor. More...
 
Scalar_t GetMomentum () const
 Getters. More...
 
std::vector< std::vector< Matrix_t > > & GetPastBiasGradients ()
 
std::vector< Matrix_t > & GetPastBiasGradientsAt (size_t i)
 
std::vector< std::vector< Matrix_t > > & GetPastWeightGradients ()
 
std::vector< Matrix_t > & GetPastWeightGradientsAt (size_t i)
 
- Public Member Functions inherited from TMVA::DNN::VOptimizer< Architecture_t, VGeneralLayer< Architecture_t >, TDeepNet< Architecture_t, VGeneralLayer< Architecture_t > > >
 VOptimizer (Scalar_t learningRate, TDeepNet< Architecture_t, VGeneralLayer< Architecture_t > > &deepNet)
 Constructor. More...
 
virtual ~VOptimizer ()=default
 Virtual Destructor. More...
 
size_t GetGlobalStep () const
 
VGeneralLayer< Architecture_t > * GetLayerAt (size_t i)
 
std::vector< VGeneralLayer< Architecture_t > * > & GetLayers ()
 
Scalar_t GetLearningRate () const
 Getters. More...
 
void IncrementGlobalStep ()
 Increments the global step. More...
 
void SetLearningRate (size_t learningRate)
 Setters. More...
 
void Step ()
 Performs one step of optimization. More...
 

Protected Member Functions

void UpdateBiases (size_t layerIndex, std::vector< Matrix_t > &biases, const std::vector< Matrix_t > &biasGradients)
 Update the biases, given the current bias gradients. More...
 
void UpdateWeights (size_t layerIndex, std::vector< Matrix_t > &weights, const std::vector< Matrix_t > &weightGradients)
 Update the weights, given the current weight gradients. More...
 
virtual void UpdateBiases (size_t layerIndex, std::vector< Matrix_t > &biases, const std::vector< Matrix_t > &biasGradients)=0
 Update the biases, given the current bias gradients. More...
 
virtual void UpdateWeights (size_t layerIndex, std::vector< Matrix_t > &weights, const std::vector< Matrix_t > &weightGradients)=0
 Update the weights, given the current weight gradients. More...
 

Protected Attributes

Scalar_t fMomentum
 The momentum used for training. More...
 
std::vector< std::vector< Matrix_t > > fPastBiasGradients
 The sum of the past bias gradients associated with the deep net. More...
 
std::vector< std::vector< Matrix_t > > fPastWeightGradients
 The sum of the past weight gradients associated with the deep net. More...
 
- Protected Attributes inherited from TMVA::DNN::VOptimizer< Architecture_t, VGeneralLayer< Architecture_t >, TDeepNet< Architecture_t, VGeneralLayer< Architecture_t > > >
TDeepNet< Architecture_t, VGeneralLayer< Architecture_t > > & fDeepNet
 The reference to the deep net. More...
 
size_t fGlobalStep
 The current global step count during training. More...
 
Scalar_t fLearningRate
 The learning rate used for training. More...
 

#include <TMVA/DNN/SGD.h>

Inheritance diagram for TMVA::DNN::TSGD< Architecture_t, Layer_t, DeepNet_t >:
[legend]

Member Typedef Documentation

◆ Matrix_t

template<typename Architecture_t , typename Layer_t = VGeneralLayer<Architecture_t>, typename DeepNet_t = TDeepNet<Architecture_t, Layer_t>>
using TMVA::DNN::TSGD< Architecture_t, Layer_t, DeepNet_t >::Matrix_t = typename Architecture_t::Matrix_t

Definition at line 47 of file SGD.h.

◆ Scalar_t

template<typename Architecture_t , typename Layer_t = VGeneralLayer<Architecture_t>, typename DeepNet_t = TDeepNet<Architecture_t, Layer_t>>
using TMVA::DNN::TSGD< Architecture_t, Layer_t, DeepNet_t >::Scalar_t = typename Architecture_t::Scalar_t

Definition at line 48 of file SGD.h.

Constructor & Destructor Documentation

◆ TSGD()

template<typename Architecture_t , typename Layer_t , typename DeepNet_t >
TMVA::DNN::TSGD< Architecture_t, Layer_t, DeepNet_t >::TSGD ( Scalar_t  learningRate,
DeepNet_t &  deepNet,
Scalar_t  momentum 
)

Constructor.

Definition at line 85 of file SGD.h.

◆ ~TSGD()

template<typename Architecture_t , typename Layer_t = VGeneralLayer<Architecture_t>, typename DeepNet_t = TDeepNet<Architecture_t, Layer_t>>
TMVA::DNN::TSGD< Architecture_t, Layer_t, DeepNet_t >::~TSGD ( )
default

Destructor.

Member Function Documentation

◆ GetMomentum()

template<typename Architecture_t , typename Layer_t = VGeneralLayer<Architecture_t>, typename DeepNet_t = TDeepNet<Architecture_t, Layer_t>>
Scalar_t TMVA::DNN::TSGD< Architecture_t, Layer_t, DeepNet_t >::GetMomentum ( ) const
inline

Getters.

Definition at line 71 of file SGD.h.

◆ GetPastBiasGradients()

template<typename Architecture_t , typename Layer_t = VGeneralLayer<Architecture_t>, typename DeepNet_t = TDeepNet<Architecture_t, Layer_t>>
std::vector< std::vector< Matrix_t > > & TMVA::DNN::TSGD< Architecture_t, Layer_t, DeepNet_t >::GetPastBiasGradients ( )
inline

Definition at line 76 of file SGD.h.

◆ GetPastBiasGradientsAt()

template<typename Architecture_t , typename Layer_t = VGeneralLayer<Architecture_t>, typename DeepNet_t = TDeepNet<Architecture_t, Layer_t>>
std::vector< Matrix_t > & TMVA::DNN::TSGD< Architecture_t, Layer_t, DeepNet_t >::GetPastBiasGradientsAt ( size_t  i)
inline

Definition at line 77 of file SGD.h.

◆ GetPastWeightGradients()

template<typename Architecture_t , typename Layer_t = VGeneralLayer<Architecture_t>, typename DeepNet_t = TDeepNet<Architecture_t, Layer_t>>
std::vector< std::vector< Matrix_t > > & TMVA::DNN::TSGD< Architecture_t, Layer_t, DeepNet_t >::GetPastWeightGradients ( )
inline

Definition at line 73 of file SGD.h.

◆ GetPastWeightGradientsAt()

template<typename Architecture_t , typename Layer_t = VGeneralLayer<Architecture_t>, typename DeepNet_t = TDeepNet<Architecture_t, Layer_t>>
std::vector< Matrix_t > & TMVA::DNN::TSGD< Architecture_t, Layer_t, DeepNet_t >::GetPastWeightGradientsAt ( size_t  i)
inline

Definition at line 74 of file SGD.h.

◆ UpdateBiases()

template<typename Architecture_t , typename Layer_t , typename DeepNet_t >
auto TMVA::DNN::TSGD< Architecture_t, Layer_t, DeepNet_t >::UpdateBiases ( size_t  layerIndex,
std::vector< Matrix_t > &  biases,
const std::vector< Matrix_t > &  biasGradients 
)
protectedvirtual

Update the biases, given the current bias gradients.

Implements TMVA::DNN::VOptimizer< Architecture_t, VGeneralLayer< Architecture_t >, TDeepNet< Architecture_t, VGeneralLayer< Architecture_t > > >.

Definition at line 135 of file SGD.h.

◆ UpdateWeights()

template<typename Architecture_t , typename Layer_t , typename DeepNet_t >
auto TMVA::DNN::TSGD< Architecture_t, Layer_t, DeepNet_t >::UpdateWeights ( size_t  layerIndex,
std::vector< Matrix_t > &  weights,
const std::vector< Matrix_t > &  weightGradients 
)
protectedvirtual

Update the weights, given the current weight gradients.

Implements TMVA::DNN::VOptimizer< Architecture_t, VGeneralLayer< Architecture_t >, TDeepNet< Architecture_t, VGeneralLayer< Architecture_t > > >.

Definition at line 113 of file SGD.h.

Member Data Documentation

◆ fMomentum

template<typename Architecture_t , typename Layer_t = VGeneralLayer<Architecture_t>, typename DeepNet_t = TDeepNet<Architecture_t, Layer_t>>
Scalar_t TMVA::DNN::TSGD< Architecture_t, Layer_t, DeepNet_t >::fMomentum
protected

The momentum used for training.

Definition at line 51 of file SGD.h.

◆ fPastBiasGradients

template<typename Architecture_t , typename Layer_t = VGeneralLayer<Architecture_t>, typename DeepNet_t = TDeepNet<Architecture_t, Layer_t>>
std::vector<std::vector<Matrix_t> > TMVA::DNN::TSGD< Architecture_t, Layer_t, DeepNet_t >::fPastBiasGradients
protected

The sum of the past bias gradients associated with the deep net.

Definition at line 55 of file SGD.h.

◆ fPastWeightGradients

template<typename Architecture_t , typename Layer_t = VGeneralLayer<Architecture_t>, typename DeepNet_t = TDeepNet<Architecture_t, Layer_t>>
std::vector<std::vector<Matrix_t> > TMVA::DNN::TSGD< Architecture_t, Layer_t, DeepNet_t >::fPastWeightGradients
protected

The sum of the past weight gradients associated with the deep net.

Definition at line 53 of file SGD.h.


The documentation for this class was generated from the following file: