Why neural networks struggle with the Game of Life
Why neural networks struggle with the Game of Life
Why neural networks struggle with the Game of Life
By Ben Dickson
September 16, 2020
Summary
Despite its simplicity the Game of Life remains a challenge to artificial neural networks, AI researchers at Swarthmore College and the Los Alamos National Laboratory have shown in a recent paper.
Titled, "It's Hard for Neural Networks To Learn the Game of Life," their research investigates how neural networks explore the Game of Life and why they often miss finding the right solution.
With neural networks being very good prediction machines, the researchers wanted to find out whether deep learning models could learn the underlying rules of the Game of Life.
There are a few reasons the Game of Life is an interesting experiment for neural networks.
Unlike domains such as computer vision or natural language processing, if a neural network has learned the rules of the Game of Life it will reach 100 percent accuracy.
They initialized the parameters to random values and trained the neural network on 1 million randomly generated examples of the Game of Life.
"The lottery ticket hypothesis proposes that when training a convolutional neural network, small lucky subnetworks quickly converge on a solution," the authors of the Game of Life paper write.
"While Conway's Game of Life itself is a toy problem and has few direct applications, the results we report here have implications for similar tasks in which a neural network is trained to predict an outcome which requires the network to follow a set of local rules with multiple hidden steps," the AI researchers write in their paper.
"Given the difficulty that we have found for small neural networks to learn the Game of Life, which can be expressed with relatively simple symbolic rules, I would expect that most sophisticated symbol manipulation would be even more difficult for neural networks to learn, and would require even larger neural networks," Springer said.
"Our result does not necessarily suggest that neural networks cannot learn and execute symbolic rules to make decisions it suggests that these types of systems may be very difficult to learn, especially as the complexity of the problem increases."
Reference
Dickson, B. (2020, September 16). Why neural networks struggle with the Game of Life. Retrieved September 21, 2020, from https://bdtechtalks.com/2020/09/16/deep-learning-game-of-life/