Nystromformer
Upgrade for Vision Transformers: Improved Efficiency for Vision Transformers
Vision Transformer and models like it use a lot of computation and memory when processing images. New work modifies these architectures to run more efficiently while adopting helpful properties from convolutions.