从 GPT-2 到 Kimi K3：22,580 倍参数增长背后的架构演化
ali 用 22 张图和 17 组代码串联 GPT-2、线性注意力、DeltaNet、Gated DeltaNet、Kimi Delta Attention、混合 MLA/MoE、SiTU 与 Attention Residuals，说明 Kimi K3 的进步不只是扩大参数量，而是围绕记忆写入、遗忘、路由和检索逐步重构架构。
22580: From GPT2 to Kimi3, Explained
Ali traces the path from GPT-2 to Kimi K3 with 22 diagrams and 17 code examples, covering linear attention, DeltaNet, Gated DeltaNet, Kimi Delta Attention, hybrid MLA/MoE, SiTU, and Attention Residuals. The article argues that Kimi K3 is not blind scaling: each step adds a specific mechanism for memory, forgetting, routing, or retrieval.
https://x.com/waterloo_intern/status/2081762065392541951
X (formerly Twitter)
ali (@waterloo_intern) on X
22580: From GPT2 to Kimi3, Explained
July 29, 2026 15
