Sparse Transformers
Know a great resource on Sparse Transformers? Send it to us at contactbackprop@gmail.com!
Tags: Transformers
Posts
Generative Modeling with Sparse Transformers
Blog post by OpenAI about Sparse Transformers
Sparse Attention Matrix Factorization (Sparse Transformers)
Quick look at sparse transformers by Lilian Weng
Code Examples
Code with sparse attention primitives provided by OpenAI with their paper
An implementation of Sparse Transformers
Distribution Augmentation for Generative Modeling
Sparse Transformer code that achieved state-of-the-art performance on cifar-10
Paper corresponding to this repo
APIs
A package containing Tensorflow Ops and corresponding GPU kernels for block-sparse matrix multiplication used for implementing sparse transformers
Know a great resource on Sparse Transformers? Send it to us at contactbackprop@gmail.com!