|
|
Low-level functions required for the forward propagation of activations through the network.
|
static void | MultiplyTranspose (TCudaMatrix< AFloat > &output, const TCudaMatrix< AFloat > &input, const TCudaMatrix< AFloat > &weights) |
| Matrix-multiply input with the transpose of and write the results into output . More...
|
|
static void | AddRowWise (TCudaMatrix< AFloat > &output, const TCudaMatrix< AFloat > &biases) |
| Add the vectors biases row-wise to the matrix output. More...
|
|
|
Low-level functions required for the forward propagation of activations through the network.
|
static void | Backward (TCudaMatrix< AFloat > &activationGradientsBackward, TCudaMatrix< AFloat > &weightGradients, TCudaMatrix< AFloat > &biasGradients, TCudaMatrix< AFloat > &df, const TCudaMatrix< AFloat > &activationGradients, const TCudaMatrix< AFloat > &weights, const TCudaMatrix< AFloat > &activationBackward) |
| Perform the complete backward propagation step. More...
|
|
static void | ScaleAdd (TCudaMatrix< AFloat > &A, const TCudaMatrix< AFloat > &B, Scalar_t beta=1.0) |
| Adds a the elements in matrix B scaled by c to the elements in the matrix A. More...
|
|
static void | Copy (TCudaMatrix< AFloat > &B, const TCudaMatrix< AFloat > &A) |
| Copy the elements of matrix A into matrix B. More...
|
|
|
For each activation function, the low-level interface contains two routines.
One that applies the acitvation function to a matrix and one that evaluate the derivatives of the activation function at the elements of a given matrix and writes the results into the result matrix.
|
static void | Identity (TCudaMatrix< AFloat > &B) |
|
static void | IdentityDerivative (TCudaMatrix< AFloat > &B, const TCudaMatrix< AFloat > &A) |
|
static void | Relu (TCudaMatrix< AFloat > &B) |
|
static void | ReluDerivative (TCudaMatrix< AFloat > &B, const TCudaMatrix< AFloat > &A) |
|
static void | Sigmoid (TCudaMatrix< AFloat > &B) |
|
static void | SigmoidDerivative (TCudaMatrix< AFloat > &B, const TCudaMatrix< AFloat > &A) |
|
static void | Tanh (TCudaMatrix< AFloat > &B) |
|
static void | TanhDerivative (TCudaMatrix< AFloat > &B, const TCudaMatrix< AFloat > &A) |
|
static void | SymmetricRelu (TCudaMatrix< AFloat > &B) |
|
static void | SymmetricReluDerivative (TCudaMatrix< AFloat > &B, const TCudaMatrix< AFloat > &A) |
|
static void | SoftSign (TCudaMatrix< AFloat > &B) |
|
static void | SoftSignDerivative (TCudaMatrix< AFloat > &B, const TCudaMatrix< AFloat > &A) |
|
static void | Gauss (TCudaMatrix< AFloat > &B) |
|
static void | GaussDerivative (TCudaMatrix< AFloat > &B, const TCudaMatrix< AFloat > &A) |
|
|
Loss functions compute a scalar value given the output of the network for a given training input and the expected network prediction Y that quantifies the quality of the prediction.
For each function also a routing that computes the gradients (suffixed by Gradients) must be provided for the starting of the backpropagation algorithm.
|
static AFloat | MeanSquaredError (const TCudaMatrix< AFloat > &Y, const TCudaMatrix< AFloat > &output) |
|
static void | MeanSquaredErrorGradients (TCudaMatrix< AFloat > &dY, const TCudaMatrix< AFloat > &Y, const TCudaMatrix< AFloat > &output) |
|
static AFloat | CrossEntropy (const TCudaMatrix< AFloat > &Y, const TCudaMatrix< AFloat > &output) |
| Sigmoid transformation is implicitly applied, thus output should hold the linear activations of the last layer in the net. More...
|
|
static void | CrossEntropyGradients (TCudaMatrix< AFloat > &dY, const TCudaMatrix< AFloat > &Y, const TCudaMatrix< AFloat > &output) |
|
static AFloat | SoftmaxCrossEntropy (const TCudaMatrix< AFloat > &Y, const TCudaMatrix< AFloat > &output) |
| Softmax transformation is implicitly applied, thus output should hold the linear activations of the last layer in the net. More...
|
|
static void | SoftmaxCrossEntropyGradients (TCudaMatrix< AFloat > &dY, const TCudaMatrix< AFloat > &Y, const TCudaMatrix< AFloat > &output) |
|
|
Output functions transform the activations output of the output layer in the network to a valid prediction YHat for the desired usage of the network, e.g.
the identity function for regression or the sigmoid transformation for two-class classification.
|
static void | Sigmoid (TCudaMatrix< AFloat > &YHat, const TCudaMatrix< AFloat > &) |
|
static void | Softmax (TCudaMatrix< AFloat > &YHat, const TCudaMatrix< AFloat > &) |
|
|
For each regularization type two functions are required, one named <Type>Regularization that evaluates the corresponding regularization functional for a given weight matrix and the Add<Type>RegularizationGradients , that adds the regularization component in the gradients to the provided matrix.
|
static AFloat | L1Regularization (const TCudaMatrix< AFloat > &W) |
|
static void | AddL1RegularizationGradients (TCudaMatrix< AFloat > &A, const TCudaMatrix< AFloat > &W, AFloat weightDecay) |
|
static AFloat | L2Regularization (const TCudaMatrix< AFloat > &W) |
|
static void | AddL2RegularizationGradients (TCudaMatrix< AFloat > &A, const TCudaMatrix< AFloat > &W, AFloat weightDecay) |
|
|
For each initialization method, one function in the low-level interface is provided.
The naming scheme is
Initialize<Type>
for a given initialization method Type.
|
static void | InitializeGauss (TCudaMatrix< AFloat > &A) |
|
static void | InitializeUniform (TCudaMatrix< AFloat > &A) |
|
static void | InitializeIdentity (TCudaMatrix< AFloat > &A) |
|
static void | InitializeZero (TCudaMatrix< AFloat > &A) |
|
|
static void | Dropout (TCudaMatrix< AFloat > &A, AFloat p) |
| Apply dropout with activation probability p to the given matrix A and scale the result by reciprocal of p . More...
|
|
|
Additional arithmetic on CUDA matrices used to implement the low-level interface.
|
static void | Multiply (TCudaMatrix< AFloat > &C, const TCudaMatrix< AFloat > &A, const TCudaMatrix< AFloat > &B) |
| Standard multiplication of two matrices A and B with the result being written into C. More...
|
|
static void | TransposeMultiply (TCudaMatrix< AFloat > &output, const TCudaMatrix< AFloat > &input, const TCudaMatrix< AFloat > &Weights) |
| Matrix multiplication of two matrices A and B^T (transposed) with the result being written into C. More...
|
|
static void | Hadamard (TCudaMatrix< AFloat > &A, const TCudaMatrix< AFloat > &B) |
| In-place Hadamard (element-wise) product of matrices A and B with the result being written into A . More...
|
|
static void | SumColumns (TCudaMatrix< AFloat > &B, const TCudaMatrix< AFloat > &A) |
| Sum columns of (m x n) matrixx A and write the results into the first m elements in A . More...
|
|
static AFloat | Sum (const TCudaMatrix< AFloat > &A) |
| Compute the sum of all elements in A . More...
|
|