XLNet

The U.S. military enlisted natural language processing to combat disinformation. Primer, a San Francisco startup, is developing a system for the Department of Defense that sifts through news, social media, research, and reports to spot propaganda campaigns.

How is it that many people learn to drive a car fairly safely in 20 hours of practice, while current imitation learning algorithms take hundreds of thousands of hours, and reinforcement learning algorithms take millions of hours? Clearly we’re missing something big.

Less than a month after XLNet overtook BERT, the pole position in natural language understanding changed hands again. RoBERTa is an improved BERT pretraining recipe that beats its forbear, becoming the new state-of-the-art language model — for the moment.