nn_gelu function

GELU module

Applies the Gaussian Error Linear Units function: [REMOVE_ME]$\mbox{GELU}(x) = x * \Phi(x) [REMOVE_ME_2]$

nn_gelu(approximate = "none")

Arguments

• approximate: the gelu approximation algorithm to use: 'none' or 'tanh'. Default: 'none'.

Description

Applies the Gaussian Error Linear Units function:

Details

where $\Phi(x)$ is the Cumulative Distribution Function for Gaussian Distribution.

Shape

• Input: $(N, *)$ where * means, any number of additional dimensions
• Output: $(N, *)$, same shape as the input

Examples

if (torch_is_installed()) {
m <- nn_gelu()
input <- torch_randn(2)
output <- m(input)
}