AI smashes video game high scores by remembering its past success
AI smashes video game high scores by remembering its past success
AI smashes video game high scores by remembering its past success
By Matthew Sparkes
February 24, 2021
Summary
An artificial intelligence that can remember its previous successes and use them to create new strategies has achieved record high scores on some of the hardest video games on classic Atari consoles.
Many AI systems use reinforcement learning, in which an algorithm is given positive or negative feedback on its progress towards a particular goal after each step it takes, encouraging it towards a particular solution.
Adrien Ecoffet at Uber AI Labs and OpenAI in California and his colleagues hypothesised that such algorithms often stumble upon encouraging avenues but then jump to another area in the hunt for something more promising, leaving better solutions overlooked.
The software stores screen grabs from a game as it plays to remember what it has tried, grouping together similar-looking images to identify points in the game it should return to as a jumping-off point.
The algorithm's aim is to maximise its score and it updates its record of a starting point when it is used to reach a new high score with a new screen grab from that part of the game.
This meant the algorithm could begin from any point without having to play the game from the start.
The team set the algorithm to playing a collection of 55 Atari games that has become a standard benchmark for reinforcement learning algorithms.
In one particularly complex game, Montezuma's Revenge, the algorithm scored higher than the previous record for reinforcement learning software and also beat the human world record.
Once the algorithm had reached a sufficiently high score, the researchers used the solution it came up with to train a neural network to replicate the strategy and play the game the same way, doing away with the need for reloading save states with an emulator.
This alternative approach turned out to be more computationally intensive, as the neural network version of the algorithm created billions of screen grabs while solving each game.
Reference
Sparkes, M. (2021, February 24). AI smashes video game high scores by remembering its past success. New Scientist. https://www.newscientist.com/article/2269085-ai-smashes-video-game-high-scores-by-remembering-its-past-success/