The sad record of an artificial intelligence in the classic pitfall: zero points – so far. They did, in any case, find out about the scientists of the vehicle service provider after having successfully completed their own KI. Even in Montezuma's Revenge they have achieved best performances, but KIs could play a very good role in this play by Atari.
The researchers wanted to try out new approaches to solving problem solving and how to write in their blog on the basis of the two games. The findings could, amongst other things, help in the robotics.
Particularly challenging were the problems of pitfall. The course of this game works poorly with classical deep learning, because there are only a few recognizable rewards that allow the algorithms to optimize in ever-new ways. Just 32 such gifts are hidden in over 255 environments.
Instead, would "Many actions lead to small negative rewards, which keep algorithms from exploring the environment", write the scientists. For example, the main character is sinking in a sump or being eaten by a crocodile. For this reason, all the experiments of a genuine KI have fallen short of failure after the start – without a single point.
In order to get better results, the researchers have combined a few new approaches. So, their KI – which they call go-exploration – have been taught to recapture new levels of exploration in new environments, combine results, and then look for fresh approaches.
In addition, the programmers have found new ways to give KI a quick overview of yet unknown terrain by scaling down their structure to a deliberately simple way and taking the result into the archive.
In addition, "first solve the tasks and, if necessary, make the solution more robust later"instead of beginning with optimizing. All of this has led Go-Explore to collect over 21,000 points in pitfall, which is far more than what would be a good human player.