Articles tagged
Transformers
2 articles
Efficient & Geometrically-Smart: Linear Memory SE(2)-Invariant Attention Explained
Fourier-encoded relative pose lets SE(2)-invariant attention use linear rather than quadratic memory, with error below 0.001. Best paper at the RSS 2025 workshop.
Machine LearningTransformersGeometric AI
Modern Methods in Associative Memory
Hopfield networks store only 0.14N patterns; dense variants reach polynomial capacity and modern ones exponential, with a Lagrangian view linking them to attention.
Associative MemoryHopfield NetworksTransformers