Papers

RoBERTa: A Robustly Optimized BERT Pretraining Approach

• Paper introducing RoBERTa

Videos

RoBERTa Paper Walkthrough

• By Yannic Kilcher

Posts

RoBERTa: A Robustly Optimized BERT Pretraining Approach

• Short blog post about RoBERTa by Facebook AI

RoBERTa

• Short section on RoBERTa in a Lilian Weng blog post

Code Examples

RoBERTa Code

• Code released with the RoBERTa paper

APIs

Transformers - RoBERTa

• API for RoBERTa from the transformers package