We don't really understand AI

Despite the breathtaking progress to date, we don't fully understand how generative AI works.

We don't really understand AI

Despite the breathtaking progress to date, we don't fully understand how generative AI works.

I made this remark at a closed-door panel discussion I moderated last week, alongside representatives from Microsoft and Nvidia.

Nobody disagreed.

๐Ÿ˜… ๐—ง๐—ต๐—ฒ ๐˜๐—ฟ๐˜‚๐˜๐—ต ๐—ฎ๐—ฏ๐—ผ๐˜‚๐˜ ๐—”๐—œ

While hardly news to those dabbling in AI, many will probably be surprised by this revelation.

Over the weekend, I came across an article in MIT Technology Review that offered the clearest explanation of the mystery of LLMs yet.

Here's my summary.

๐ŸŒ€ ๐Ÿญ/ ๐—”๐—œ ๐—บ๐˜†๐˜€๐˜๐—ฒ๐—ฟ๐˜†

By accident, two researchers at OpenAI discovered a phenomenon where models would seemingly fail to learn a task, then get it all of a sudden.

But this wasn't how deep learning was supposed to work; it was but one of several odd behaviours that had AI researchers scratching their heads.

โ€œObviously, weโ€™re not completely ignorant. But our theoretical analysis is so far off what these models can do... I think this is very mysterious,โ€ said a computer scientist.

An AI researcher was quoted saying: "My assumption was that scientists know what theyโ€™re doing. Like, theyโ€™d get the theories and then theyโ€™d build the models. That wasnโ€™t the case at all."

โ€œIt was like, here is how you train these models and then hereโ€™s the result. But it wasnโ€™t clear why this process leads to models that are capable of doing these amazing things.โ€

๐ŸŒ  ๐Ÿฎ/ ๐—ง๐—ผ๐—ผ ๐—ฐ๐—ผ๐—บ๐—ฝ๐—น๐—ฒ๐˜… ๐˜๐—ผ ๐—บ๐—ผ๐—ฑ๐—ฒ๐—น

The rapid advances over the last decade came more from trial and error than from understanding, as researchers copied what worked and added their own innovations.

Said another computer scientist: โ€œWe have a lot of experimental results that we donโ€™t completely understand, and often when you do an experiment it surprises you.โ€

For now, the biggest models are now so complex that researchers are experimenting on smaller varieties of statistical models as a proxy.

And each time that the basics get nailed down, breakthroughs would disprove it: "And then people made models that could speak 100 languages and it was like, okay, we understand nothing at all.โ€

โ€œIt turned out we werenโ€™t even scratching the surface.โ€

โ›‘๏ธ ๐Ÿฏ/ ๐—ฆ๐—ฎ๐—ณ๐—ฒ๐˜๐˜† ๐—ถ๐˜€ ๐˜๐—ต๐—ฒ ๐—ถ๐˜€๐˜€๐˜‚๐—ฒ

Ultimately, the greatest issue is safety. Without a complete understanding of the theory behind the science, how can we predict the capabilities that may emerge?

Put another way, one won't know what GPT-5 or GPT-6 is capable of until it is trained and tested. But what if it goes rogue? How would we detect it and stop it in time?

For now, work continues apace to explain the capabilities of our existing models.

And yes, a "superalignment" team at OpenAI was also set up to figure out how we might stop a hypothetical superintelligence.

๐—œ๐—บ๐—ฎ๐—ด๐—ฒ ๐—–๐—ฟ๐—ฒ๐—ฑ๐—ถ๐˜: DALL-E 3