Game-changing game changes

All it took to rewrite the rules of understanding the evolution of cooperation was a series of chance encounters among Martin Nowak, Krishnendu Chatterjee, and Christian Hilbe.

During his first visit to Harvard in 2008, Chatterjee, a computer-science professor at IST Austria, mentioned stochastic games — games that can change based on players’ actions — and the idea sent Nowak down a years-long path to merge the concept with evolutionary dynamics.

“People who study evolution of cooperation do not use stochastic games,” said Nowak, who developed the new framework in collaboration with Chatterjee, Hilbe, a postdoctoral fellow in Chatterjee’s group at IST, and Stepan Simsa of Charles University in Prague. “Instead, in a sequence of repeated encounters, it is assumed that the same game with the same payoff matrix is played again and again. In a stochastic game, the game itself can change probabilistically depending on the players’ actions.”

That new approach, described in a July 4 paper published in the journal Nature, describes a system that can model the evolution of cooperation based on repeated stochastic games. In the simplest form, only two games are played, one for a larger reward, the other for a smaller one. In each round, players must choose whether to cooperate or to defect.

Both games work the same way: If both players cooperate, both receive a reward. If one defects while the other cooperates, the defector collects a larger reward while the other player gets nothing. If both defect, both receive a reward, though it’s smaller than the reward for cooperation. Under normal conditions cooperation rarely emerges, because the most logical decision is for players to defect in an effort to maximize their reward. The innovation finds that whether players cooperate affects which game they subsequently play.

The players begin with the higher-value game. As long as both players cooperate, they continue to play that game, but if either defects they move to the lower-value game. Once they again cooperate, they might return to the higher-value game.

“Amazingly, even if both games are set up so that cooperation does not evolve, when we put them together, we get cooperation,” Nowak said. “It’s almost like a paradox.” The key to making the system work, Nowak said, is the difference in value between the two games.

“If we defect, we are destroying something, but if we cooperate we are building something,” he said. “So when we cooperate, we play subsequently for something that is more valuable, and if we defect, we play subsequently for something that is less valuable.

“That’s what makes the new approach exciting. The idea is so simple, and yet it changes everything. If you defect in the first game, you lose out twice, because your opponent will retaliate and you have to play a less-valuable game.”

The stochastic framework can also be applied to multiplayer games, including public-goods games, where players can resolve what is known as the “tragedy of the commons.”

“Humans are exploiting the environment in a public-goods game,” Nowak said. “In the old framework, we decide to cooperate or defect, but the next day we play the same game again, and the state of the environment is always the same. But in our new theory, if we exploit the environment badly, in the next round it may be deteriorated, and then we face a less-valuable public good.”

If the environment deteriorates sharply in response to defection, then there is a strong incentive to maintain cooperation: the knowledge of being at a breaking point. If the environment deteriorates slowly or not at all, then achieving cooperation is, paradoxically, more difficult.

The concept can also be used by public officials and policymakers to design programs that empower cooperation. If players cooperate, they have the chance to move to more valuable games.

“This has interesting implications for some of the major problems humans are facing, such as climate change, environmental destruction, and migration,” Nowak said. “If people understand that defection today means that we will play a game with a lower payoff tomorrow, then cooperation becomes a winning strategy.”

The research was supported with funding from the European Research Council, Graph Games, the Austrian Science Fund, an FWF NFN grant, the Office of Naval Research, and the John Templeton Foundation.