Training and Inference Dynamics of PLDR-LLMs: Row-Map Collapse, Renormalization, and Predictive Reduction
Paper • 2609.34130 • Published • 1
Large Language Models, Transformers, Natural Language Processing
Training and Inference Dynamics of PLDR-LLMs: Row-Map Collapse, Renormalization, and Predictive Reduction
Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference