Should GELU Handle -inf Gracefully?

[lucianyao]

Hi, I noticed that GELU may return NaN for -inf inputs due to the x * Φ(x) formulation. In cases like MaxPool padding, -inf can be passed into GELU, and I’ve seen it lead to NaN during testing. Curious if you think it's worth explicitly guarding for this case?

[laggui]

This is not a numerical stability issue where an alternative set of operations might be more suitable, simply that the input is not really in a valid range.

Adding such guards typically adds unnecessary overhead especially on GPU, so perhaps it is better to leave it to the user to clamp the input values e.g.

gelu(tensor.clamp(NEG_MAX, POS_MAX))