Slopfest #91 · 2017 · Vienna

Attention Is All You Need

A Google paper proposed that attention was all you need. The committee agreed and asked for nothing else.

#91The only copy was taped to a Würstelstand. It rained.

Seating plan in coloured pencil: lines from every seat to every other seat, eight coloured groups labelled 'heads', titled SLOPFEST #91

Museum status: Pending acquisition by a museum of modern art. Pending since the day it was made.

Venue: Austria Center Vienna, a very large room
Attendance: 48,300
Fun fact: 8 attention heads, 0 explanations

What happened

The festival was arranged so that every guest paid attention to every other guest at once. This was exhausting, and by 10 p.m. the guests had split into eight groups, each attending to something different. The committee called these heads. Nobody could explain what the groups were attending to, and this remains the state of the art.

Elsewhere in 2017

Google researchers publish 'Attention Is All You Need', introducing the Transformer, the basis of modern large language models.

Visit the 2017 website

Tip: use ← and → to flip through the archive.