A group of researchers based at the University of Freiburg have been testing basic machine-learning programs – ‘evolutionary algorithms’ – by seeing how well they perform at classic Atari games.

This hands-on approach to machine learning involves generating multiple algorithms, testing them to identify the best, adjusting these and searching for further improvements. This strategy is an alternative to common ‘reinforcement learning’ algorithms, which involve learning entirely by trial-and-error.

In this case, the researchers used an updated version of Atari’s Q*bert – along with seven other games, including Pong and Space Invaders – in order to make it easier for their machine-learning algorithm to experiment with different strategies. They found that the machine-learning program playing Q*bert identified two unusual strategies after just five hours of training.

Q*bert, first released in 1982, is an arcade game in which the player attempts to switch the colour of every cube in a pyramid as an avatar which can jump on top of the cubes while being attacked by enemies.

The AI exploited a bug in the game which generated points for the player when Q*bert jumps cube to cube – apparently at random – and causing them to start blinking. A second strategy involved the avatar baiting an enemy to follow it, then jumping to its death, followed by the enemy. Killing the enemy generated enough points to gain another life, such that the game can continue.

Similarly unusual strategies arose as the machine-learning program played other Atari games. During Space Invaders and Alien, for instance, it focused on gaining a higher number of points, even at the cost of the main game objective.

According to the researchers, their work suggests that evolution strategies could be a viable alternative to standard approaches to reinforcement learning.

Machine-learning programs have been let loose on video games before, as they provide controlled and easily accessible environments which can be explored over and over. A better performance is marked clearly with a higher score or a win.

In 2015, it was reported that a machine-learning algorithm named MarI/0 was able to learn how to finish the platform game Super Mario Bros, being able to complete an entire level of the platform game in just 34 tries by jumping through the entire game. Last year, it was reported that driverless cars were using Grand Theft Auto 5 in order to learn road safety, thanks to the many obstacles and other chaos that the game presents.

A group of researchers based at the University of Freiburg have been testing basic machine-learning programs – ‘evolutionary algorithms’ – by seeing how well they perform at classic Atari games.

This hands-on approach to machine learning involves generating multiple algorithms, testing them to identify the best, adjusting these and searching for further improvements. This strategy is an alternative to common ‘reinforcement learning’ algorithms, which involve learning entirely by trial-and-error.

In this case, the researchers used an updated version of Atari’s Q*bert – along with seven other games, including Pong and Space Invaders – in order to make it easier for their machine-learning algorithm to experiment with different strategies. They found that the machine-learning program playing Q*bert identified two unusual strategies after just five hours of training.

Q*bert, first released in 1982, is an arcade game in which the player attempts to switch the colour of every cube in a pyramid as an avatar which can jump on top of the cubes while being attacked by enemies.

The AI exploited a bug in the game which generated points for the player when Q*bert jumps cube to cube – apparently at random – and causing them to start blinking. A second strategy involved the avatar baiting an enemy to follow it, then jumping to its death, followed by the enemy. Killing the enemy generated enough points to gain another life, such that the game can continue.

Similarly unusual strategies arose as the machine-learning program played other Atari games. During Space Invaders and Alien, for instance, it focused on gaining a higher number of points, even at the cost of the main game objective.

According to the researchers, their work suggests that evolution strategies could be a viable alternative to standard approaches to reinforcement learning.

Machine-learning programs have been let loose on video games before, as they provide controlled and easily accessible environments which can be explored over and over. A better performance is marked clearly with a higher score or a win.

In 2015, it was reported that a machine-learning algorithm named MarI/0 was able to learn how to finish the platform game Super Mario Bros, being able to complete an entire level of the platform game in just 34 tries by jumping through the entire game. Last year, it was reported that driverless cars were using Grand Theft Auto 5 in order to learn road safety, thanks to the many obstacles and other chaos that the game presents.