We don't really understand AI

Despite the breathtaking progress to date, we don't fully understand how generative AI works.

I made this remark at a closed-door panel discussion I moderated last week, alongside representatives from Microsoft and Nvidia.

Nobody disagreed.

😅 𝗧𝗵𝗲 𝘁𝗿𝘂𝘁𝗵 𝗮𝗯𝗼𝘂𝘁 𝗔𝗜

While hardly news to those dabbling in AI, many will probably be surprised by this revelation.

Over the weekend, I came across an article in MIT Technology Review that offered the clearest explanation of the mystery of LLMs yet.

Here's my summary.

🌀 𝟭/ 𝗔𝗜 𝗺𝘆𝘀𝘁𝗲𝗿𝘆

By accident, two researchers at OpenAI discovered a phenomenon where models would seemingly fail to learn a task, then get it all of a sudden.

But this wasn't how deep learning was supposed to work; it was but one of several odd behaviours that had AI researchers scratching their heads.

“Obviously, we’re not completely ignorant. But our theoretical analysis is so far off what these models can do... I think this is very mysterious,” said a computer scientist.

An AI researcher was quoted saying: "My assumption was that scientists know what they’re doing. Like, they’d get the theories and then they’d build the models. That wasn’t the case at all."

“It was like, here is how you train these models and then here’s the result. But it wasn’t clear why this process leads to models that are capable of doing these amazing things.”

🌠 𝟮/ 𝗧𝗼𝗼 𝗰𝗼𝗺𝗽𝗹𝗲𝘅 𝘁𝗼 𝗺𝗼𝗱𝗲𝗹

The rapid advances over the last decade came more from trial and error than from understanding, as researchers copied what worked and added their own innovations.

Said another computer scientist: “We have a lot of experimental results that we don’t completely understand, and often when you do an experiment it surprises you.”

For now, the biggest models are now so complex that researchers are experimenting on smaller varieties of statistical models as a proxy.

And each time that the basics get nailed down, breakthroughs would disprove it: "And then people made models that could speak 100 languages and it was like, okay, we understand nothing at all.”

“It turned out we weren’t even scratching the surface.”

⛑️ 𝟯/ 𝗦𝗮𝗳𝗲𝘁𝘆 𝗶𝘀 𝘁𝗵𝗲 𝗶𝘀𝘀𝘂𝗲

Ultimately, the greatest issue is safety. Without a complete understanding of the theory behind the science, how can we predict the capabilities that may emerge?

Put another way, one won't know what GPT-5 or GPT-6 is capable of until it is trained and tested. But what if it goes rogue? How would we detect it and stop it in time?

For now, work continues apace to explain the capabilities of our existing models.

And yes, a "superalignment" team at OpenAI was also set up to figure out how we might stop a hypothetical superintelligence.

𝗜𝗺𝗮𝗴𝗲 𝗖𝗿𝗲𝗱𝗶𝘁: DALL-E 3