Clip a Gradient Matrix by its Frobenius Norm

This function prevents exploding gradients by rescaling any gradient whose Frobenius norm exceeds a specified threshold. The direction of the gradient is preserved.

clip_gradient_norm(gradient, threshold = 1)

Arguments

gradient

The gradient matrix to be clipped.

threshold

The maximum allowed Frobenius norm for the gradient.

Value

A gradient matrix whose Frobenius norm is guaranteed to be less than or equal to the threshold.