TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters The paper introduces Tokenformer. The architecture leverages the attention mechanism to facilitate not only inter-token computations but also interactions between tokens and model parameters. The authors replace all linear …
November 8, 2024 609 8
