Is Bigger Always Better? The Surprising (and dangerous?) Science Behind Large Language Models
How does a model trained to predict the next word become capable of translating languages, writing code, and tackling unfamiliar tasks? And can we keep making AI better simply by making it bigger? This talk explores the ideas behind the rise of large language models, from scaling laws (model size, data, and computing power) to learning from just a few examples in a prompt. We will investigate why some abilities seem to appear suddenly and perhaps dangerously unpredictably, whether these breakthroughs are as mysterious as they look, and how bigger models or more data can sometimes make performance worse. Along the way, we will encounter models that seemingly memorise before learning to generalise, and discuss what these surprising behaviours reveal about machine learning.