Evolutionarily stable strategy, stable state, periodic cycle and chaos in a simple discrete time two

J .theor .Biol .(1997)188,21–270022–5193/97/170021+07$25.00/0/jt97045271997Academic Press LimitedEvolutionarily Stable Strategy,Stable State,Periodic Cycle and Chaosin a Simple Discrete Time Two-Phenotype ModelT Y *,Y Q - ,J Z - W Z -Department of Conservational Biology ,Institute of Zoology ,Academia Sinica ,19Zhongguancun Lu ,Haidian ,Beijing ,100080,China (Received on 30July 1996,Accepted in revised form on 31March 1997)A simple discrete time two-phenotype matrix game model is investigated.In this model,according to the suggestion of Vincent &Fisher (1988,Evolutionary Ecology 2,321–337),the fitness of an individual is defined to be an exponential function of its expected pay-offvalue.The results show that:(i)in our model,the static conditions of ESS are only dependent on the properties of pay-offmatrix,but not on the specific form of fitness function.This result implies that the ESS conditions in our model are completely identical with the conditions in the two-phenotype model with linear fitness function.(ii)In our model,the relationship between the static conditions of ESS and the dynamic properties of the pure strategy model is that if the interior fixed point of the pure strategy model is not an ESS-equilibrium,then it must be unstable;conversely,if the interior fixed point of the pure strategy model is an ESS-equilibrium,then it can be stable or unstable,and an unstable ESS-equilibrium must correspond to the cyclic or chaotic behaviour of the population state.71997Academic Press Limited1.IntroductionMaynard Smith (1974,1982)introduced the funda-mental notion of an evolutionarily stable strategy (ESS)in order to explain the evolution of genetically determined social behaviour within a single animal species.Over the past two decades,the concept of ESS has not only proved of practical use in the study of animal behaviour but also has generated enormous theoretical research in this area.The matrix game model is one of the most important theoretical models in the evolutionary game theory.The original static matrix game model has given a dynamic approach to the evolution of animal behaviour (strategies)by many authors (e.g.Taylor &Jonker,1978;Zeeman,1980,1981;Hofbauer &Sigmund,1988;Cressman,1992).Thus the evolution can be defined as change in strategy frequency in the evolutionary game theory (Vincent &Brown,1988).In 1982,Maynard Smith defined a basic linear relationship between the Darwinian fitness of an individual and its expected pay-offvalue in the matrix game model.Maynard Smith assumed that all individuals have a fitness W 0before the contest and the expected pay-offvalue of an individual should be a gain in its fitness (where gain means that the expected pay-offvalue can be positive or negative),thus the fitness of an individual is the sum of W 0and its expected pay-offvalue.This definition is also called the linearity assumption of the fitness function (Hofbauer &Sigmund,1988;Cressman,1992).For the discrete time matrix game model,if we accept the linearity assumption of the fitness function,then W 0should be a sufficiently large positive value in order to guarantee that the fitness is greater than zero.Nevertheless,if the density-de-pendent mechanism is introduced in to this model (Cressman &Dash,1987,and Cressman,1988;who first introduced the density-dependent mechanism into the continuous time two-phenotype model),we*Author to whom correspondence should be addressed.. . 22willfind that the linearity assumption of thefitnessfunction may not be appropriate.As the simplestcase,if W0is density-dependent and the pay-offmatrix is density-independent(Cressman&Dash,1987),then when population density is sufficientlylarge,W0will be very small,and at the sametime,if the expected pay-offvalue of an individualis negative,then thefitness of this individualmay be negative.Obviously,this cannot be permit-ted.For the discrete time evolutionary game modelVincent&Fisher(1988)suggested that the individ-ualfitness should be an exponential function ofpopulation density and strategy frequency in orderto guarantee that thefitness is non-negative.According to this suggestion,let all individuals haveafitness exp4W05before the contest,where W0is only a parameter,then the effect of interactionbetween a pair of individuals on thefitness shouldbe mainly effected in the change of parameter W0.Thus,for the discrete time matrix game model,wecan define that the expected pay-offvalue is a gainin the parameter W0and thefitness of an individualis an exponential function of its expected pay-offvalue.Since Hofbauer et al.(1979)and Zeeman(1980,1981)discussed the relationship between the con-ditions of ESS and dynamic stability in thecontinuous time matrix game model with the linearfitness function,the viewpoint,that the populationstate corresponding to ESS(it can also be called anESS-equilibrium of the population state(Cressman,1992))must be asymptotically stable,has beenextensively accepted(Hofbauer&Sigmund,1988;Cressman,1992).Nevertheless,for the discrete timematrix game model with linearfitness function,ifonly two pure strategies are possible,the conditionsof ESS are compatible with the dynamics ofstrategy frequency,but if more than two purestrategies are possible,the conditions of ESS areneither necessary nor sufficient to guarantee thestability of the dynamics of strategy frequency(Maynard Smith,1982;Cressman,1992).Forexample,Cressman(1992)gave a simple three-phe-notype model,and an ESS-equilibrium is unstablein this model.In this paper,a simple discrete time two-pheno-type matrix game model with nonlinearfitnessfunction will be investigated.The main purpose ofthis paper is to illustrate the relationship betweenthe static conditions of ESS and the possiblenonlinear dynamic behaviours of the strategyfrequency dynamics.2.Basic Assumptions and the Basic ModelIn this paper,we make the following basic assumptions:(1)The generations of the population are discreteand non-overlapping.(2)The population is a haploid population,whichmeans that the offspring have completelyidentical phenotypic strategies with theirparents.(3)There are only two different pure strategies,R1and R2,in the population and the frequenciesof the pure strategies R1and R2in thepopulation at generation t are p t and(1−p t),respectively.(4)The individuals complete randomly in pairwisecontests,and the outcome(pay-offvalue)ofeach contest is determined by strategies used(Maynard Smith,1974,1982).(5)The pay-offmatrix is given byA=$a11a12a21a22%,(1)where a ij is the pay-offvalue for the purestrategy R i played against the pure strategy R j(i,j=1,2)(Maynard Smith,1974,1982;Hofbauer et al.,1979).(6)Let F and f be thefitness of an individual andits expected pay-offvalue,respectively,then thefitness function of this individual is given byF=exp4W0+f5,(2) where W0is a parameter and all individualshave afitness exp4W05before the contest(Vincent&Fisher,1988;see also the definitionin Section1).From the basic assumptions,let f1(t)and f2(t)be the expected pay-offvalues of pure strategists R1and R2at generation t,respectively,then we havef1(t)=p t a11+(1−p t)a12f2(t)=p t a21+(1−p t)a22.(3) Thus,thefitness functions of pure strategists R1and R2at generation t can be written asF1(t)=exp4W0+f1(t)5F2(t)=exp4W0+f2(t)5,(4) respectively.- 23 From Maynard Smith(1982),the frequency of thepure strategy R1at generation t+1can be given byp t+1=p t F1(t)p t F1(t)+(1−p t)F2(t)=p t exp4W0+f1(t)5p t exp4W0+f1(t)5+(1−p t)exp4W0+f2(t)5=p tp t+(1−p t)exp4f2(t)−f1(t)5.(5)Equation(5)can be considered to be a basic pure strategy model with the nonlinearfitness function. If an individual uses a mixed strategy S=[x,(1−x)],where x and1−x are the probabil-ities that this individual expresses the pure strategies R1and R2,respectively,in a population whose average strategy is(p t,1−p t),then its pay-offvalue isf S(t)=xf1(t)+(1−x)f2(t)(6)and so itsfitness isF S(t)=exp4W0+f S(t)5.(7)3.The Static Conditions of EvolutionarilyStable StrategyMaynard Smith(1982)pointed out that an evolutionarily stable strategy is a strategy such that, if all the members of a population adopt it,then no mutant strategy could invade the population under the influence of natural selection.In our model,if the population consists of the S I-strategist and S J-strategist,where S I and S J are the two different strategies,which areS I=[x I,(1−x I)]and S J=[x J,(1−x J)],(8)and the frequencies of S I-strategist and S J-strategist at generation t are1−e(t)and e(t),respectively,then the frequency of the pure strategy R1can be given byp t=[1−e(t)]x I+e(t)x J.(9)From the basic assumption(6),eqn(6)and eqn(7), we know that thefitness functions of S I-strategist and S J-strategist can be given byF SI(t)=exp4W0+x I f1(t)+(1−x I)f2(t)5F SJ(t)=exp4W0+x J f1(t)+(1−x J)f2(t)5,(10)respectively.If S I is an ESS,then according to the Maynard Smith’s(1982)definition,we must haveF SI (t)q F SJ(t)for all S J$S I,(11a)orx I f1(t)+(1−x I)f2(t)q x J f1(t)+(1−x J)f2(t)for all S J$S I(11b)when e(t)is sufficiently small.This means that if S I isan ESS and the population only consists ofS I-strategists,then no S J-strategist could invade thepopulation for all S J$S I when the number of theS J-strategist is sufficiently small.From eqn(3)and eqn(9),inequality(11)can berewritten as[1−e(t)](S I·A S I−S J·A S I)+e(t)(S I·A S J−S J·A S J)q0,(12)whereS i·A S j=x i[x j a11+(1−x j)a12]+(1−x i)[x j a21+(1−x j)a22]for all i,j=I,J(see also Hofbauer&Sigmund,1988).Since e(t)is sufficiently small,inequality(12)impliesthat S I is an ESS if and only if the following twoconditions are satisfied:(I)S J·A S I E S I·A S I for all S J$S I(13a)(II)if S J$S I and S J·A S I=S I·A S I,then S J·A S J Q S I·A S J.(13b)Obviously,the conditions(I)and(II)arecompletely identical with the conditions in the matrixgame model with linearfitness function(Hofbauer&Sigmund,1988).Thus,from Maynard Smith’s(1982)results in the classical two-phenotype matrix gamemodel,the ESS conditions in our model can be givenby(1)The pure strategy R1is an ESS if and only ifpay-offmatrix A satisfies eithera11−a21q0,ora11=a21and a12q a22;(2)The pure strategy R2is an ESS if and only ifpay-offmatrix A satisfies eithera22−a12q0,ora22=a12and a21q a11;. . 24(3)If pay-offmatrix A satisfiesa12−a22q0a21−a11q0,then the mixed strategy S I=[x I,(1−x I)],wherex I=a12−a22a12−a22+a21−a11$(0,1),is a mixed evolutionarily stable strategy.The above results clearly show that in our model, the static conditions of ESS are only dependent on the properties of pay-offmatrix,but not on the specific form offitness function.4.Dynamic Properties of The Pure Strategy Model Let p*be an interiorfixed point of the pure strategy model,eqn(5),which means p*$(0,1),then p*must satisfyf2(t)−f1(t)=0.(14) From eqn(14),it is easy to obtainp*=a12−a22a12−a22+a21−a11.(15)Obviously,in order to guarantee p*$(0,1),the pay-offmatrix A must satisfya12−a22q0a21−a11q0,(16a) ora12−a22Q0a21−a11Q0.(16b) Suppose the pay-offmatrix A satisfies inequality (16)and letg=a12−a22+a21−a11,(17)then eqn(5)can be rewritten asp t+1=8g(p t)0p tp t+(1−p t)exp4g(p t−p*)5,(18)where in order to clearly illustrate the dynamic properties of eqn(18),we can assume that p*is a given constant and is not dependent on g.Fromd8g(p t) dp t =exp4g(p t−p*)5[1−p t(1−p t)g][p t+(1−p t)exp4g(p t−p*)5]2=0,(19)we know that when g q4,8g(p t)has two critical points denoted by c and c,which arec=12−X14−1g and c=12+X14−1g,(20)and8g(c)and8g(c)are maximum and minimum of 8g(p t),respectively.Thus8g(p t)is a one-dimensional map with two critical points for any g q4.The frequency of the pure strategy R1at generation t+n can be given byp t+n=8n g(p t)0p tp t+(1−p t)exp6g0s n−1i=0p t+i−np*17.(21)It is easy to determine the stability of p*in eqn(18). Since the eigenvalue of eqn(18)at the point p*can be given bydp t+1dp t b p t=p*=1−p*(1−p*)g,(22)from the theory of difference equation p*is stable if and only if=1−p*(1−p*)g=Q1.(23) Thus we have that1.if g Q0,then p*must be unstable;2.if0Q g Q2/p*(1−p*),then p*is stable;3.if g q2/p*(1−p*),then p*is unstable andg cr=2/p*(1−p*)is bifurcation valueIt is not difficult to demonstrate that eqn(18) satisfies the theorem of period-doubling bifurcation when p t=p*and g=2/p*(1−p*)(Hao,1993).For the situation of g q2/p*(1−p*),we have to consider the following two basic cases.1 *=0.5In this case,eqn(18)can be rewritten asp t+1=8g(p t)0p tt t t,(24) and when g q8,p*=0.5is unstable.Since we have8g(p t)=1−8g(1−p t),(25) Equation(24)is an anti-symmetric map about p t=0.5.On the other hand,from eqn(21),the frequency of the pure strategy R1at generation t+2can be- 25given byp t+2=82g(p t)0p tp t+(1−p t)exp4g(p t+1+p t−1)5.(26)It is easy to note that thefixed point condition ofeqn(26)isp t+1+p t=1.(27)Obviously,p*=0.5is an unstable trivial solution ofeqn(27).Since g cr=8is a period-doubling bifurcationvalue,from the theorem of period-doubling bifur-cation,we know that there also exist two non-trivialsolutions in eqn(27),which can be denoted by p1andp2,and p1and p2must satisfyp1+p2=1.(28)It is easy to obtaindp t+2 dp t =dp t+2dp t+1·dp t+1dp t=exp4g(p t+1+p t−1)5[1−p t(1−p t)]×[1−p t+1(1−p t+1)][p t+(1−p t)exp4g(p t+1+p t−1)5]2,(29)thus we havedp t+2dp t b p t=p1=dp t+2dp t b p t=p2=(1−p1p2g)2.(30) Equation(30)shows that p1and p2have same stability.For conveniently discussing,we only need to investigate the stability of p1,and let p1q0.5(or p1$(0.5,1)).From eqn(30),we know that p1is stable if and only ifp1(1−p1)g Q2p1$(0.5,1).(31) In the following,we will demonstrate that inequality(31)always holds for any g ing the fixed point condition eqn(27)and eqn(28),we can obtaing=42p1−1lnp11−p1p1$(0.5,1).(32)Substituting eqn(32)for g in inequality(31),we havep1(1−p1)lnp11−p1Q p1−0.5p1$(0.5,1).(33)LetY1(p1)=p1(1−p1)lnp11−p1Y2(p1)=p1−0.5p1$(0.5,1),(34)then it is easy to obtainY1(0.5)=Y2(0.5)=0dY1(p1)dp1=(1−2p1)lnp11−p1+1dY2(p1)dp1=1.(35)Thus we havedY1(p1)dp1Q dY2(p1)dp1p1$(0.5,1).(36)Inequality(36)implies that inequality(31)alwaysholds for any g q8,which means that p1and p2arealways stablefixed points of eqn(26).The above result shows that for any g q8,eqn(24)must have a stable periodic two-cycle attractor.It isnecessary to point out that eqn(24)should beconsidered to be a special case because the generalanti-symmetric map with two critical points can havecomplex dynamic behaviour,for example theanti-symmetric cubic map has the property ofsymmetry breaking(Zhang et al.,1987).2 *$0.5In this case,when g q2/p*(1−p*),p*is anunstablefixed point of eqn(18),and since p*$0.5,the map function8g(p t)is not anti-symmetric aboutp t=0.5.From the theory of applied symbolicdynamics(Zheng&Hao,1994),we know that it isvery important to illustrate the relationship betweenthe dynamic invariant subinterval(Hao,1993)andthe critical points of the map function8g(p t)in orderto understand the nonlinear dynamic properties of8g(p t).On the other hand,since the bifurcation valueg cr=2/p*(1−p*)is symmetric about p*=0.5,weonly need to discuss the situation of p*Q0.5.Thesituation of p*q0.5is similar to p*Q0.5.Let U be the dynamic invariant subinterval of8g(p t),then from eqn(20),U can be given byU=[8g(c),8g(c)].(37)From the definition of the dynamic invariantsubinterval of the one-dimensional map function(Hao,1993),we know that for a given starting valueof eqn(18),p0(where p0$(0,1)),if p0$U,then we. . 26must have p t$U for all t=1,2,...;conversely,ifp0(U,then there must exist a positive integer l(wherel is alwaysfinite),which makes p i(U for i=1,2,...land p l+t$U for all t=1,2,....This means that any sequence of eqn(18)that stars in(0,1)becomes attracted by the closed interval U.Since p*Q0.5,we always have either8g(c)+8g(c)Q1,(38)or2c(g)−p*lnc(g)1−c(g)Q2c(g)−p*lnc(g)1−c(g).(39)Thus there are three possible relationships between U and the critical points(c and c),which are(1)if g satisfies inequalityg Q2c(g)−p*lnc(g)1−c(g),(40)then8g(c)Q c and8g(c)q c,or c(U and c(U;(2)if g satisfies inequality2lnc(g)Q g Q2ln c(g),(41)then8g(c)Q c and8g(c)Q c,or c$U and c(U;(3)if g satisfies inequalityg q2ln c(g),(42)then8g(c)q c and8g(c)Q c,or c$U and c$U.From the above relationships,we can obtain that: (a)If g satisfies inequality(40),then8g(p t)is astrictly decreasing function in U,or we always have d8g(p t)/dp t Q0for all p t$U,thus eqn(18)must have a stable periodic two-cycle attractor.(b)If g satisfies inequality(41),then8g(p t)is anunimodal map function in U,thus the dynamic properties of eqn(18)must be similar to the unimodal map,for example the logistic map (May,1976;Schuster,1988).This means that the periodic and chaotic behaviours of eqn(18) in U are all possible.(c)If g satisfies inequality(42),then8g(p t)hastwo critical points in U,thus the dynamic properties of eqn(18)are similar to the general cubic map(Hao,1989,Zheng&Hao,1994).For the one-dimensional map function with two critical points,the symbolic dynamics of one-dimensional map shows that the map withtwo critical points can have a new trajectory,but a new scaling property with universalitydoes not exist as compared with the unimodalmap(Hao,1989;Zheng&Hao,1994).Thisalso means that if g satisfies inequality(42),then the periodic and chaotic behaviours ofeqn(18)in U are all possible.It is easy tofind that when g q0,the interiorfixed point p*of eqn(18)must correspond to a mixed evolutionarily stable strategy,which can be denoted by S I,where S I=[x I,(1−x I)]and x I0p*=(a12−a22)/ g.From the discussions in Section3,we know that if S Iis a mixed evolutionarily stable strategy and the population only consists of S I-strategists,then no S J-strategist could invade the population for all S J$S I when the number of the S J-strategist is sufficiently small.According to Cressman’s(1992) definition,the interiorfixed point p*of eqn(18)can be called an ESS-equilibrium if and only if g q0.Thus for the pure strategy model eqn(18),the relationship between the static conditions of ESS and dynamics of strategy frequency can be given by(1)if the interiorfixed point of eqn(18)is not anESS-equilibrium,then it must be unstable; (2)if the interiorfixed point of eqn(18)is anESS-equilibrium,then it can be stable orunstable;(3)an unstable ESS-equilibrium of eqn(18)mustcorrespond to the cyclic or chaotic behaviour ofeqn(18).5.ConclusionIn this paper,we investigated a simple discrete time two-phenotype matrix game model with nonlinear fitness function.In this model,thefitness of an individual is defined to be an exponential function of its expected pay-offvalue(Vincent&Fisher,1988). We analysed the static conditions of ESS,the dynamic properties of the pure strategy model and the relationship between the ESS condition and the possible dynamic behaviour of the pure strategy model.Our result shows that:(1)In our model,the static conditions of ESS onlydepend on the properties of the pay-offmatrix,but do not on the specific form of thefitnessfunction.This result implies that the ESSconditions in our model are completelyidentical with the conditions in the two-pheno-type matrix game model with linearfitnessfunction.It is necessary to point out that thisresult should be considered to be model-depen-dent.- 27(2)The pure strategy model can have verycomplex nonlinear dynamic properties,and these properties can be easily understood in mathematics.The main dynamic properties of the pure strategy model eqn(18)can be given by(a)if g Q0,then the interiorfixed point p*ofeqn(18)must be unstable;(b)p*is stable if and only if0Q g Q g cr,where g cr=2/p*(1−p*)is a bifurcationvalue;(c)for all g q g cr,p*is unstable;(d)when p*=0.5,for any g q8,eqn(18)must have a stable periodic two-cycleattractor;(e)when p*$0.5,if g q g cr,then the periodicand chaotic behaviours of eqn(18)are allpossible.(3)In our model,the relationship between thestatic conditions of ESS and the dynamic properties of the pure strategy model can be given by(a)if g q0,then the interiorfixed point p*ofeqn(18)must correspond to a mixedevolutionarily stable strategy and thisfixed point can be called an ESS-equi-librium(Cressman,1992);(b)if an interiorfixed point of eqn(18)is notan ESS-equilibrium,then it must beunstable;(c)if an interiorfixed point of eqn(18)is anESS-equilibrium,then it can be stable orunstable;(d)an unstable ESS-equilibrium of eqn(18)must correspond to the cyclic or chaoticbehaviour of eqn(18).We would like to thank the anonymous referees for constructive criticisms on the manuscript.REFERENCESC ,R.(1988).Frequency-and density-dependent selection: the two-phenotype model.Theor.pop.Biol.34,378–398.C ,R.(1992).The Stability Concept of Evolutionary Game Theory.New York;Heidelberg:Springer-Verlag.C ,R.&D ,A.T.(1987).Density dependence of evolutionarily stable strategies.J.theor.Biol.126,393–406.H ,B.L.(1989).Elementary Symbolic Dynamics and Chaos in Dissipative System.Singapore:World Scientific.H ,B.L.(1993).Starting with Parabolas—An Introduction to Chaotic Dynamics[Chinese].Shanghai:Shanghai Scientific and Technological Education Publishing House.H ,J.,S ,P.&S ,K.(1979).A note on evolutionarily stable strategies and game dynamics.J.theor. Biol.81,609–612.H ,J.&S ,K.(1988).The Theory of Evolution and Dynamical Systems.Cambridge:Cambridge University Press.M ,R.M.(1976).Simple mathematical models with very complicated dynamics.Nature,261,459–467.M S ,J.(1974).The theory of games and the evolution of animal conflicts.J.theor.Biol.47,209–221.M S ,J.(1982)Evolution and the Theory of Games. Cambridge:Cambridge University Press.S ,H.G.(1988).Deterministic Chaos:An Introduction,2nd edn.VCH Weinheim.T ,P.&J ,L.(1978).Evolutionarily stable strategies and game dynamics:Math.Biosci.40,145–156.V ,T.L.&B ,J.S.(1988).The evolution of ESS theory. Ann.Rev.Ecol.Syst.19,423–443.V ,T.L.&F ,M.E.(1988).Evolutionarily stable strategies in differential and difference equation models. Evol.Ecol.2,321–337.Z ,H.K.,D ,J.H.,W ,P.Y.,J ,C.D.&H ,B.L. (1987).Analytical study of a bimodal map related to optical mun.Theor.Phys.8(3),281–294.Z ,E.C.(1980).Population dynamics from game theory.In: Global Theory of Dynamical Systems.Springer Lecture Notes in Mathematics819.Z ,E.C.(1981).Dynamics of the evolution of animal conflicts.J.theor.Biol.89,249–270.Z ,W.M.&H ,B.L.(1994).Applied Symbolic Dynamics [Chinese].Shanghai:Shanghai Scientific and Technological Education Publishing House.。