Kevin Delnoye François Fleuret It's not the original post norm
PostNorm: LN(x + f(x))
Swinv2: x + LN(f(x))
PreNorm: x + f(LN(x))
Avihu Dekel I expected it in the last layer, but anyway there something different in the last layer, it has another postnorm or something like that, maybe there is another layer after