Evolutionary Games

Spring 2017

PSC/ANB 290

Evolutionary Games

The analytical tractability of game theory comes in large part by abstracting strategies from the agents that use them.  Once we put strategies back into agents and let them play more than once, all bets are off about which strategies will exhibit stability if any.

Evolutionary game theory situates games in evolutionary contexts and seeks to determine the dynamics and stability conditions of strategies.  There are roughly two approaches to evolutionary game theory.  The first is more mathematical, typically using differential equation to describe change in strategy frequency in a population. Strategies are still disembodied and generally, individuals in a population are assumed to be infinite in number and to randomly interact.  The second approach is the agent-based modeling approach, which place strategies back into agents, focuses on agent interactions, and strategy dynamics and stability are emergent properties of agent interactions.

There is no canonical way to create evolutionary game-theory models using agent-based modeling.  My preference is to create generically but minimally biologically realistic agents.  That is agents that are in space, can move, can aggregate, can play games with neighbors, reproduce with a mechanism of mutation, and invest in offspring.  Many more features can be added to such agents, but this is my starting point.

In evolutionary games, strategies evolve if agents using them tend to receive higher payoffs than agents that do not.  In the agent-based model that we develop, payoffs are turned into offspring that inherit a parent’s strategy. Agents that are more successful with their strategy produce more offspring and so the frequency of strategy use in a population increases.

Today we will create an evolutionary agent-based model for the prisoner’s dilemma.  We will begin by creating five classes.

List of classes:

Agent extends Replicator extends Aggregator extends RandomWalker implements Steppable

We will start with Replicator, which has the capability of movement, aggregation, and replication.

Environnment extends ReplicatorEnvironment extends AggregatorEnvironment extends RandomWalkerEnvironment extends SimStateSweep

Each environment has parameters required for the functionality of its corresponding agents.

Experimenter extends Observer

The experimenter will collect and save any data and perform any manipulation or control on agents that we need to do simultaneously.

AgentsWithGUI extends GUIStateSweep

Provides a graphical user interface.

Probe

When we used the CorrelationProbe in the Mate Choice model, it functioned as a probe for collecting data.  In general, I like to create probes that collect data where needed.  A probe is created in the Environment and passed to each agent as well as to the Experimenter.  The Experimenter then reads the probe and saves the desired data.

The code below is in the archived project EvolutionaryGT -290.zip and requires the archived project MASONex.zip.

The Agent class

package gameAgents;

import replicator.Replicator;
import sim.engine.SimState;
import sim.util.Bag;
import sim.util.Int2D;
import sweep.SimStateSweep;


public class Agent extends Replicator{

	public int maxAgents;
	public double cooperatorsPayoff;
	public double temptationPayoff;
	public double suckersPayoff;
	public double defectorsPayoff;
	public int playerSearchRadius;
	public boolean played = false;
	public boolean cooperator;
	public Probe probe;
	public double mutationRate;
	public Asexual reproductionMethod;
	public int replicationRate;
	public int lastReproduced = 1;



	public Agent(SimStateSweep state, int x, int y,int walkRuleNumber, int age,  boolean cooperator, double resources) {
		super(state,x,y,age,walkRuleNumber);
		Environment eState = (Environment)state;
		maxAgents = eState.maxAgents;
		cooperatorsPayoff = eState.cooperatorsPayoff;
		temptationPayoff = eState.temptationPayoff;
		suckersPayoff = eState.suckersPayoff;
		defectorsPayoff = eState.defectorsPayoff;
		playerSearchRadius = eState.playerSearchRadius;
		probe = eState.probe;
		if(cooperator)
			setColor(state,(float) 0, (float)0, (float)1, (float)1);
		else
			setColor(state,(float) 1, (float)0, (float)0, (float)1);
		this.cooperator = cooperator;
		this.resources = resources;
		mutationRate = eState.mutationRate;
		reproductionMethod = new Asexual();
		replicationRate = eState.replicationRate;
		lastReproduced = state.random.nextInt(replicationRate)+1;
	}
	/**
	 * Handles a 2-person prisoner's dilemma game.  There is a cost in obtaining a cooperative
	 * benefit.  If two plaeyers cooperate and share the gain, then each receives of payoff of
	 * benefit/2 - cost.  If one player defects from sharing the benefit, then its payoff is the 
	 * benefit - cost and the other player is -cost.  If both do not cooperate, the payoff the
	 * non-cooperative payoff (i.e., typically 0). 
	 * @param state
	 */

	public void play(Environment state){
		if(played){  //already played this round
			return;
		}
		Bag neighbors = state.sparseSpace.getMooreNeighbors(x, y, playerSearchRadius, mode, false);
		//next find a random player who has not played
		Agent other = null;
		if(neighbors == null || neighbors.numObjs == 0){//there are no players nearby
			return;  
		}
		Bag players = new Bag();
		for(int i=0;i<neighbors.numObjs;i++){
			Agent a = (Agent)neighbors.objs[i];
			if(!a.played){
				players.add(a);
			}
			players.add(a);
		}
		if(players.numObjs != 0){
			other = (Agent)players.objs[state.random.nextInt(players.numObjs)];
		}
		
		if(other == null){//we did our best
			return;
		}
		//now to the game
		if(this.cooperator && other.cooperator){
			this.resources += cooperatorsPayoff;//cooperative payoff
			other.resources += other.cooperatorsPayoff;

		}
		else if(this.cooperator && !other.cooperator){ //cooperator/defector
			this.resources += suckersPayoff;//the cost, i.e. sucker's payoff
			other.resources += other.temptationPayoff;
		}
		else if(!this.cooperator && other.cooperator){ //defector/cooperator
			other.resources += other.suckersPayoff;//the cost, i.e. sucker's payoff
			this.resources += temptationPayoff;
		}
		else if(!this.cooperator && !other.cooperator){ //both defectors
			other.resources += other.defectorsPayoff;
			this.resources += defectorsPayoff;
		}
	}

	public class Asexual implements Reproducible{
		public Replicator replicate(SimStateSweep state, Int2D location){
			final double parentalContribution = resources * parentalInvestment;
			resources -= parentalContribution; //deduct the parental investment
			//next any mutations
			Agent offspring = new Agent(state, location.x, location.y, 0,walkRuleNumber, cooperator, parentalContribution);
			if(state.random.nextBoolean(mutationRate)){
				offspring.cooperator = !offspring.cooperator; //flip it
			}
			
			return offspring;
		}
	}

	public void die(SimStateSweep state){
		super.die(state);
		if(cooperator){
			probe.setCooperatorEnergyAtDeath(resources);
			probe.setCooperatorsDied(1);
		}
		else{
			probe.setDefectorEnergyAtDeath(resources);
			probe.setDefectorsDied(1);
		}
	}
	
	public void randomDeath(Environment state){
		Bag agents = state.sparseSpace.getAllObjects();
		Agent a = (Agent)agents.objs[state.random.nextInt(agents.numObjs)];
		a.die(state);
	}
	
	public void adjustResources(){
		if(resources > maxResources){
			resources = maxResources;
		}
	}


	public void step(SimState state){
		super.step(state);
		if(age >= lifeSpan || resources <= 0 || state.random.nextBoolean(probabilityOfDying)){
			die((Environment)state); 
			return; //nothing more to do
		}
		play((Environment)state);
		adjustResources();
		if(lastReproduced >= replicationRate){
			if(reproduceRandom((Environment)state, reproductionMethod)){
                             if(((Environment)state).sparseSpace.allObjects.numObjs > maxAgents)
				  randomDeath((Environment)state);
				lastReproduced = 1;
			}	
		}
		else{
			lastReproduced++;
		}

		if(cooperator)
			probe.setCooperators(1);	
		else
			probe.setDefectors(1);
		}

}

The Environment class

package gameAgents;


import randomWalker.RandomWalkMechanics;
import replicator.ReplicatorEnvironment;


public class Environment extends ReplicatorEnvironment {
	public int cooperators = 500;
	public int defectors = 500;
	public double cooperatorsPayoff = 3;
	public double temptationPayoff = 5;
	public double suckersPayoff = -1;
	public double defectorsPayoff = 1;
	public int playerSearchRadius = 1;
	public double mutationRate = 0.01;
	public int replicationRate = 1;
	public Probe probe;
	
	

	public int getCooperators() {
		return cooperators;
	}

	public void setCooperators(int cooperators) {
		this.cooperators = cooperators;
	}

	public int getDefectors() {
		return defectors;
	}

	public void setDefectors(int defectors) {
		this.defectors = defectors;
	}

	public double getCooperatorsPayoff() {
		return cooperatorsPayoff;
	}

	public void setCooperatorsPayoff(double cooperatorsPayoff) {
		this.cooperatorsPayoff = cooperatorsPayoff;
	}

	public double getTemptationPayoff() {
		return temptationPayoff;
	}

	public void setTemptationPayoff(double temptationPayoff) {
		this.temptationPayoff = temptationPayoff;
	}

	public double getSuckersPayoff() {
		return suckersPayoff;
	}

	public void setSuckersPayoff(double suckersPayoff) {
		this.suckersPayoff = suckersPayoff;
	}

	public double getDefectorsPayoff() {
		return defectorsPayoff;
	}

	public void setDefectorsPayoff(double defectorsPayoff) {
		this.defectorsPayoff = defectorsPayoff;
	}

	public int getPlayerSearchRadius() {
		return playerSearchRadius;
	}

	public void setPlayerSearchRadius(int playerSearchRadius) {
		this.playerSearchRadius = playerSearchRadius;
	}

	public double getMutationRate() {
		return mutationRate;
	}

	public void setMutationRate(double mutationRate) {
		this.mutationRate = mutationRate;
	}

	public int getReplicationRate() {
		return replicationRate;
	}

	public void setReplicationRate(int replicationRate) {
		this.replicationRate = replicationRate;
	}

	public Environment(long seed, Class observer) {
		super(seed, observer);
		// TODO Auto-generated constructor stub
	}
	
	public void createAgents(){
		int locations = gridWidth * gridHeight;
		if(uniqueLocation && cooperators + defectors > locations){
			System.out.println("Too many agents! Please reduce the number of cooperators and defectors");
			return;
		}
		else{
			for(int i = 0;i< cooperators;i++){
				int x = random.nextInt(gridWidth);
				int y = random.nextInt(gridHeight);
				int age = random.nextInt(averagelifeSpan);
				double resources = random.nextDouble() * this.reproductionResources;
				Agent a = new Agent(this,x, y, age, RandomWalkMechanics.classicRules[rule], true, resources);
				place(a);
				a.event = schedule.scheduleRepeating(a);
			}
			
			for(int i = 0;i< defectors;i++){
				int x = random.nextInt(gridWidth);
				int y = random.nextInt(gridHeight);
				int age = random.nextInt(averagelifeSpan);
				double resources = random.nextDouble() * this.reproductionResources;
				Agent a = new Agent(this,x, y, age, RandomWalkMechanics.classicRules[rule], false, resources);
				place(a);
				a.event = schedule.scheduleRepeating(a);
			}
		}
	}
	
	public void start(){
		super.start();
		make2DSpace(false, gridWidth, gridHeight);
		probe = new Probe();
		createAgents();
		if(observer != null){
			if(objSpace)
				observer.initObjectGrid(objectSpace);
			else
				observer.initSparseGrid(sparseSpace);
			((Experimenter)observer).setProbe(probe);
		}
	}

}

The Experimenter class

package gameAgents;

import observer.Observer;
import sim.engine.SimState;
import sweep.ParameterSweeper;
import sweep.SimStateSweep;

public class Experimenter extends Observer {

	Probe probe;
	
	public Experimenter(String fileName, String folderName, SimStateSweep state, ParameterSweeper sweeper,
			String precision, String[] headers) {
		super(fileName, folderName, state, sweeper, precision, headers);
		probe = ((Environment)state).probe;
	}
	
	public Probe getProbe() {
		return probe;
	}

	public void setProbe(Probe probe) {
		this.probe = probe;
	}

	public boolean nextInterval(){
		data.add(probe.cooperators + probe.defectors);
		data.add(probe.cooperators);
		data.add(probe.defectors);
		data.add(probe.getCooperatorEnergyAtDeath());
		data.add(probe.getDefectorEnergyAtDeath());
		
		return true;
	}
	
	public void step(SimState state){
		super.step(state);
		if(getdata){
			nextInterval();
			probe.resetAll();
		}
		else{
			probe.resetCoopDef();
		}
		
		for(int i=0;i<agents.numObjs;i++){
			Agent a = (Agent)agents.objs[i];
			a.played = false; //set all to false for the next time step
		}
		if(agents.numObjs == 0){
			state.schedule.reset();
		}
	}

}

The Probe class

package egtAgents;

public class Probe {
	int cooperators = 0;
	int defectors = 0;
	int cooperatorsDied = 0;
	int defectorsDied = 0;
	double cooperatorResourcesAtDeath = 0;
	double defectorResourcesAtDeath = 0;
	
	public int getCooperators() {
		return cooperators;
	}
	public void setCooperators(int cooperators) {
		this.cooperators += cooperators;
	}
	public void resetCooperators(){
		cooperators = 0;
	}
	
	public int getDefectors() {
		return defectors;
	}
	public void setDefectors(int defectors) {
		this.defectors += defectors;
	}
	public void resetDefectors(){
		defectors = 0;
	}
	
	public int getCooperatorsDied() {
		return cooperatorsDied;
	}
	public void setCooperatorsDied(int cooperatorsDied) {
		this.cooperatorsDied += cooperatorsDied;
	}
	public void resetCooperatorsDied() {
		this.cooperatorsDied =0;
	}
	
	public int getDefectorsDied() {
		return defectorsDied;
	}
	public void setDefectorsDied(int defectorsDied) {
		this.defectorsDied += defectorsDied;
	}
	public void resetDefectorsDied() {
		this.defectorsDied =0;
	}
	
	
	public double getCooperatorEnergyAtDeath() {
		double average = 0;
		if(cooperatorsDied > 0){
			average = cooperatorResourcesAtDeath/cooperatorsDied;
		}
		return average;
	}
	public void setCooperatorEnergyAtDeath(double cooperatorEnergyAtDeath) {
		this.cooperatorResourcesAtDeath += cooperatorEnergyAtDeath;
	}
	public void resetCooperatorEnergyAtDeath() {
		this.cooperatorResourcesAtDeath =0;
	}
	
	public double getDefectorEnergyAtDeath() {
		double average = 0;
		if(defectorsDied > 0){
			average = defectorResourcesAtDeath/defectorsDied;
		}
		return average;
	}
	public void setDefectorEnergyAtDeath(double defectorEnergyAtDeath) {
		this.defectorResourcesAtDeath += defectorEnergyAtDeath;
	}
	public void resetDefectorEnergyAtDeath() {
		this.defectorResourcesAtDeath = 0;
	}
	
   public void resetAll(){
	   resetCooperators();
	   resetCooperatorEnergyAtDeath();
	   resetCooperatorsDied();
	   resetDefectors();
	   resetDefectorsDied();
	   resetDefectorEnergyAtDeath() ;
   }
   
   public void resetDied(){
	   resetCooperatorEnergyAtDeath();
	   resetCooperatorsDied();
	   resetDefectorsDied();
	   resetDefectorEnergyAtDeath() ;
   }
   
   public void resetCoopDef(){
	   resetCooperators();
	   resetDefectors();
   }
   
   public void printData(){
	   System.out.print(cooperators + defectors +"\t");
	   System.out.print( cooperators +"\t");
	   System.out.print(defectors +"\t");
	   System.out.print(getCooperatorEnergyAtDeath()+"\t");
	   System.out.println(getDefectorEnergyAtDeath());
   }
}

The AgentsWithGUI class

package gameAgents; 

import java.awt.*;
import sweep.GUIStateSweep;
import sweep.SimStateSweep;

public class AgentsWithGUI extends GUIStateSweep {
	public static void main(String[] args) {
		initialize (Environment.class,Experimenter.class,AgentsWithGUI.class, 400, 400, Color.WHITE, Color.BLUE,false,false);
	}

	public AgentsWithGUI(SimStateSweep state, int gridWidth, int gridHeight, Color backdrop, Color agentDefaultColor,
			boolean agentPortrayal) {
		super(state, gridWidth, gridHeight, backdrop, agentDefaultColor, agentPortrayal);
		// TODO Auto-generated constructor stub
	}
}