AI can’t learn forever

Artificial intelligence may be smart, but trusting your money with AI may not be. AI enthusiasts have proposed that investors might soon rely on stock trading algorithms that could constantly learn from changes in the market to update price predictions. But while they seem a promising use of new technology, new research highlights a feature of AI models that could cost users money.

The problem, according to a study out of the University of Alberta, is that as certain types of AI models train on new tasks indefinitely, their ability to learn gets worse.

This decline in ability to learn outlined by Shibhansh Dohare, a doctoral student at the University of Alberta, and his team specifically affects models built using deep learning methods. These complex programs mimic the architecture of a brain, with artificial neurons that act as waypoints for data.

If a deep learning model is left on its own to forever categorize items in pictures, its accuracy will decrease with every new type of object it has to learn how to recognize. That is, every time a new category is introduced — like a cat, a house or a stop sign — the model identifies it correctly fewer times than it did when it learned the previous object. 

In this experiment, the deep learning model had around 85% accuracy in categorizing objects at first, but around 60% accuracy by only its 5,000th identification. The problem doesn’t come from the amount of information the model needs to ingest, but rather the continual method it uses to train. 

“We are unsure about the root causes of the phenomenon,” Dohare said. 

What is clear is that “continual learning is a critical problem to overcome for deep learning systems,” said Keiland Cooper, a doctoral candidate studying sequential and continual learning at the University of California, Irvine, who was not involved in the study. 

Popular AI models already on the market, like ChatGPT, could be more powerful and cost-effective if they relied on continual learning, but they often don’t because of this problem. 

If asked about something current, like an upcoming election, ChatGPT might respond with, “I don’t have access to specific election data from last month or any updates beyond my last training cut-off.”

ChatGPT’s response to a question about recent election data. [Credit: Perri Thaler]

The model responds that way because it was only trained on data from before it was released and purposefully can’t train on real-time data. That means that it gets more and more outdated as time goes on. If it were to train on new data all the time, it would run into the problem outlined by this study — it would get worse at learning over time.

Instead, when the developers want ChatGPT or similar models to be up to date, they need to wipe the model clean and start training from scratch, which happens every few months or years. 

“At ChatGPT scale or even larger, it costs hundreds of millions of dollars for each training cycle,” Dohare said.

Cooper noted that this retraining also carries environmental concerns. Finding a replacement for the repeated resetting and retraining “may lower costs in terms of time, money and energy, among other benefits” he said. 

Even at a smaller scale, this continual learning problem can arise in scenarios where AI models need to constantly adapt and learn. 

Systems that recognize and respond to changes in human behavior, like those proposed for stock trading, may suffer from worse learning with time. Time and geography sensitive models, like weather forecasts, also need to continually learn to continue improving.

Dohare and his coauthors suggest a solution in their paper: finding the artificial neurons in the model that are least useful and resetting them back to their initial states. 

This approach works by balancing the trade-off between maintaining old, useful knowledge and benefitting from the abundance of learning that happens when some neurons begin to train.

While this technique has generally garnered positive feedback according to Dohare, there’s some concern about flexibility for bigger projects.

“The jury’s still out on whether this particular method will be scalable,” said Clare Lyle, a research scientist at Google DeepMind. “It’s an open question” whether this method will create instability in larger models completing more challenging tasks, like AI made to play chess or Go, she explained. 

Regardless of how researchers adjust deep learning models to embrace continual learning, eventually they will need to do so to create more productive and advanced AI models.

 “One hallmark of intelligence is its ability to adapt to changes,” Dohare said, so “AI will be missing a critical aspect of intelligence until this continual learning challenge is solved.”