KV Sharing, MHC, and Compressed Attention(magazine.sebastianraschka.com)

34 分 | 作者 gmays 23小时前

2 条评论

  • nibab 20小时前
    cool stuff. my comp sci major feels almost completely redundant in this new vibecoding era and i feel like the only way to stay relevant as a programmer is to learn all these compute primitives and become an LLM systems guy.
  • redwood 19小时前
    Has anyone seen a similar deep dive but that looks a little bit more closely at the infrastructure building blocks that power each of the components. I mean something a bit more physically grounded like how much compute would go to each portion to serve a Frontier Model?