[D] Layernorm is just two projections and can be improved

I was thinking on how to visualize the layernorm for a vector, and figured out that it's just applying two projections when the learned parameters are null (if not, then they just add a rescaling and translation).

You project onto the hyperplane where the mean (or sum) of components is null (e.g. in 3D x + y + z = 0) by removing the mean and then project onto the sphere of radius sqrt(D) by dividing by the standard deviation. …

components hyperplane machinelearning mean null parameters project thinking translation vector