robinzixuan/Phi4_attn_residual
Text Generation • 4B • Updated • 274 • 1
OASIS
-
-
robinzixuan/Phi4_attn_OASIS
Text Generation • 4B • Updated • 259 -
robinzixuan/Phi4_attn_AoS
Text Generation • 4B • Updated • 262 -
Attention Sinks and Outliers in Attention Residuals
Paper • 2605.17887 • Published • 5 -
robinzixuan/qwen3-0.6b-oasis
Text Generation • 0.6B • Updated -
robinzixuan/qwen3-0.6b-vanilla
Text Generation • 0.6B • Updated -
robinzixuan/llama-3.2-1b-vanilla
Text Generation • 1B • Updated -
robinzixuan/llama-3.2-1b-oasis
Text Generation • 1B • Updated -
robinzixuan/oasis_llama3.2_1b_6k
Text Generation • 1B • Updated • 59 -
robinzixuan/vanilla_llama3.2_1b_6k
Text Generation • 1B • Updated • 176 -
robinzixuan/oasis_qwen3_0.6b_6k
Text Generation • 0.6B • Updated • 53 -
robinzixuan/vanilla_qwen3_0.6b_6k
Text Generation • 0.6B • Updated • 234 -
robinzixuan/sink_logit_llama3.2_1b
Text Generation • 1B • Updated • 175 -
robinzixuan/sink_token_llama3.2_1b
Text Generation • 1B • Updated • 195 -
robinzixuan/s0_softmax1_noroute_llama3.2_1b
Text Generation • 1B • Updated • 164