Papers

Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context

• Original paper introducing the Transformer-XL model

Posts

Transformer-XL: Unleashing the Potential of Attention Models

• Blog post by the authors of Transformer-XL summarizing their ideas

Longer Attention Span (Transformer-XL)

• Quick look at Transformer-XL from Lilian Weng’s blog

Code Examples

Transformer-XL

• Code by the authors of Tensorflow1 and Pytorch versions of Transformer-XL

APIs

Transformers - Transformer-XL API

• An api for Natural Language Processing tasks in pytorch and tensorflow

Paper for the api