Feedback Equilibria fora General Class of Non-linear Differential ...

Feedback Equilibria for a General Class of Non-linear Differential
Games with Application to Environmental and Resource
Management.
K. G. Mäler∗, A. Xepapadeas†, A de Zeeuw‡
Abstract
As it is well known in differential games, the open-loop Nash equilibrium (OLNE)
concept with an infinite period of commitment is weakly time-consistent but not strongly
time-consistent. On the other hand, the feedback Nash equilibrium (FBNE) is Markov
perfect by construction and thus a more satisfactory solution concept, but solutions are
usually very difficult to derive. Explicit solutions can be obtained for the so-called linear-
quadratic differential game. However, the transition equations in realistic differential
game models for environmental and resource management contain non-linear feedbacks,
so that the linear-quadratic structure of the game is lost. Thus the attempt to make
the natural system more realistic complicates the use of the FBNE concept, since the
non-linear structure of the differential game does not allow the standard determination of
feedback equilibrium strategies. The purpose of this paper is to develop an algorithm to
solve a non-linear-quadratic differential game and to explicitly determine the non-linear
feedback equilibrium strategies. In particular we consider a class of non-linear differential
games which are often encountered in environmental and resource management problems.
At the first stage we analyze the cooperative solution and the OLNE of the underlying
differential game. These solutions can be regarded as benchmark cases that expose the
existence of multiple equilibria and of “good” and “bad” basins of attraction. At the
second stage we analyze the FBNE of the non-linear differential game and we construct
a procedure that determines the feedback equilibrium strategies numerically. Since the
non-linearity of the problem induces multiple equilibria, our procedure determines the
∗The Beijer International Institute of Ecological Economics, The Royal Swedish Academy of
Sciences, karl@beijer.kva.se
†Department of Economics, University of Crete, University Campus, 74 100 Rethymno, Crete,
GREECE, tel +30 831 0 77861, fax:+30 831 0 77860 xepapad@econ.soc.uoc.gr
‡Department of Economics and CentER, Tilburg University, A.J.deZeeuw@kub.nl

---

Feedback Equilibria
2
feedback equilibrium strategies locally in the basin of attraction of the corresponding
open-loop Nash equilibrium.
K eyw ords: Non-linear differential gam es, non-linear feedback strategies, multiple equilibria, shallow
lakes.
JE L classifi cation : Q 2, C61
1.
Introduction
Differential games have been extensively used during the recent decades to an-
alyze economic problems in areas such as industrial organization, environmental
and resource economics or macroeconomic policy.1 In the differential game for-
malism two main solution concepts are the most often used: the open-loop Nash
equilibrium (OLNE) concept where controls depend on the initial state of the sys-
tem under investigation and time, and the feedback Nash equilibrium (FBNE)
concept where attention is restricted to Markov perfect strategies with controls
depending on the current state of the system.
As it is well known, the OLNE with an infinite period of commitment is weakly
time-consistent but not strongly time-consistent (Basar, 1989). Therefore it does
not possess the Markov perfect property and is not robust against unexpected
changes in the state of the system. On the other hand, the FBNE is Markov per-
fect by construction and thus a more satisfactory solution concept, but solutions
are usually very difficult to derive. Explicit solutions can be obtained by using
a dynamic programming formulation for the so-called linear-quadratic differential
game where the objective function is quadratic and the dynamics are linear. In
this case a quadratic value function exists and this allows for an explicit solu-
tion of the problem. 2 When we depart from the linear-quadratic formulation,

---

Feedback Equilibria
3
an explicit solution is not possible except for some cases where due to the spe-
cific structure of the game the feedback problem can be reduced to an open-loop
problem (Fershtman, 1987).
Recent advances in environmental and resource economics emphasize the need
for a realistic representation of the natural system in unified economic ecological
models. Realistic modeling of natural systems in most cases indicates that the use
of linear dynamics to model natural processes might not be a good approximation
and that a non-linear formulation might be more appropriate. Non-linearities in
the transition equations associated with the evolution of natural systems relate
mainly to the existence of non-linear feedbacks which are physical processes that
further impact an initial change of the system under investigation. Feedbacks
could be positive if the impact is such that the initial perturbation is enhanced,
or negative if the initial perturbation is reduced. For example in the study of
climate change a positive feedback exists when an increase in temperature, say
due to increased accumulation of greenhouse gases, increases evaporation from the
oceans, which brings more water vapor into the atmosphere and finally enhances
greenhouse effects.3 In the analysis of shallow lake eutrophication, positive feed-
backs are related to the release of phosphorus that has been slowly accumulated in
sediments and submerged vegetation. Ignoring these non-linearities might obscure
very important characteristics that we observe in reality such as bifurcations of
the natural system to alternative equilibrium states, irreversibilities or hysteresis.
It is important to note that the design of policies without taking into account the
impact of non-linearities might lead to erroneous results and non-desirable states
of the ecosystem.
If, however, non-linear feedbacks are introduced into the transition equations

---

Feedback Equilibria
4
of differential game models for environmental and resource management, the often
used linear-quadratic structure of the game is lost. This implies that the quadratic
form of the value function cannot be used to determine the feedback equilibrium
strategies. Thus the attempt to make the natural system more realistic complicates
the use of the FBNE concept, since the non-linear structure of the differential game
does not allow the standard determination of the feedback equilibrium strategies.
We are not aware of many successful attempts to solve a non-linear-quadratic dif-
ferential game and to explicitly determine the non-linear equilibrium strategies.
An exception is the (stochastic) algorithm by Pakes and McGuire (1994, 2001)
that computes Markov perfect equilibria to describe the dynamics of an industry
with differentiated products. However, the algorithm presented in our paper is
simpler and less computationally demanding by making use of two aspects. First,
the specific structure of the type of problems we consider allows a transformation
of the dynamic programming equation, so that we can iterate on the steady state
of the FBNE instead of the space of equilibrium strategies and value functions.
Second, considering the Hamiltonian system and the sufficiency conditions allows
us to limit the area of search, so that we only need a local analysis. The purpose
of this paper is to present and apply this algorithm. In particular we consider a
class of non-linear differential games which are often encountered in environmen-
tal and resource management problems. In these games many agents influence
with their actions a common pool resource whose evolution is characterized by
non-linear dynamics. For example, we could have individual countries emitting
greenhouse gases and contributing to global warming, or farmers increasing phos-
phorus loadings to a shallow lake and contributing to its eutrophication. In both
cases the realistic representation of the natural system requires the introduction

---

Feedback Equilibria
5
of non-linear feedbacks in the system’s transition equation.
At the first stage we analyze the cooperative solution and an OLNE solution of
the underlying differential game. These solutions can be regarded as benchmark
cases that expose the existence of multiple equilibria and of “good” and “bad”
basins of attraction.
At the second stage we analyze the FBNE of the non-linear differential game.
Using a transformation of the dynamic programming equation and insights from
the Hamiltonian representation of the differential game, we are able to construct
a procedure that determines the feedback equilibrium strategy numerically. Since
the non-linearity of the problem induces multiple equilibria, our procedure deter-
mines the feedback equilibrium strategies locally in the basin of attraction of the
corresponding open-loop Nash equilibrium. Thus our result also indicates that
feedback equilibrium strategies differ depending on the relevant basin of attrac-
tion. 4 We also show that when the sufficient conditions for the Hamiltonian
representation of the differential game are satisfied, then the FBNE steady state
is “worse” than OLNE steady states and thus worse than the cooperative one.
This is a result contrary to the one obtained by Dockner and Long (1993) where
it is stated that non-linear feedback strategies for a linear-quadratic differential
game can reproduce the cooperative steady state. Our interpretation of this dis-
crepancy is that in the Dockner-Long game the non-linear strategy is not a priori
restricted so that the sufficient conditions of the Hamiltonian representation of the
differential game are satisfied.

---

Feedback Equilibria
6
2.
Cooperative and Open-Loop Nash Equilibria
We consider a set of n agents, which could be countries or communities, that
undertake a certain action ai ∈ A ⊂ R+, i = 1,...,n, with which they affect the
state of a natural system that is shared by all the agents. The action could be, for
example, phosphorus loadings into a lake due to agricultural activities or emissions
of greenhouse gases due to industrial activities. The action ai undertaken at time
t generates benefits according to a strictly increasing and concave utility function
U (ai), which is assumed to be the same for all agents: U 0 (ai) > 0, U00 (ai) < 0,
limai→0 U0 (ai) = +∞.
The evolution of pollutant in the natural system is described by the non-linear
transition equation
n
˙x = Xai−bx+f(x), x(0) =x0, x∈ X ⊂R+
(1)
i=1
The state variable x could be interpreted, for example, as accumulated phos-
phorus in a lake or accumulated greenhouse gases. In (1) the function f (x) is
an increasing non-linear function of the state variable x reflecting the presence of
non-linear feedbacks. We assume that f (0) = 0, f 0 (0) = 0, limx→∞ f0 (0) = 0
with a unique xm = argmax f 0 (x) so that f (x) is a convex-concave function, that
is f 00 (x) > 0 for x < x
5
m and f 00 (x) < 0 for x > xm . The accumulation of the
state variable causes environmental damage, or equivalently reduces the flow of
useful services generated by the natural system, according to a strictly increasing
and convex damage function D(x), which is also assumed to be the same for all
agents: D (0) = 0, D0 (x) > 0, D00 (x) > 0. Thus the flow of benefits accruing to

---

Feedback Equilibria
7
each agent at any instant of time is given by
U (ai) − D (x)
Each agent is choosing an action ai to maximize the present value of net benefits
over an infinite time horizon, or
max Ji = Z ∞e−ρt[U(ai(t))−D(x(t))]dt, i = 1,...,n
(2)
{ai(t)}
0
subject to (1), where ρ > 0 is a discount rate, common for all agents.
A cooperative solution to the problem (2) is to choose the time paths {ai (t)}
to maximize the sum of individual net benefits or,
n
max
XJi=Z ∞e−ρt" nXU(ai(t))−nD(x(t))#dt
(3)
{a1(t),...an(t)} i=1
0
i=1
subject to (1). The current value Hamiltonian for problem (3) is defined as
n
n
H = XU(ai)−nD(x)+λ[a−bx+f(x)] , a = Xai
i=1
i=1
Pontryagin’s maximum principle implies the following necessary conditions:6
U 0 (ai) = −λ, i = 1,...,n
(4)
˙
∂
λ = ρλ − H =
∂x
³ρ+b−f0(x)´λ+nD0(x)
(5)
˙x = a − bx + f (x), x(0) = x0
(6)
along with the transversality conditions at infinity
lim e−ρtλ (t) = 0, lim e−ρtλ (t) x (t) = 0
t→∞
t→∞

---

Feedback Equilibria
8
Solving (4) for ai we obtain ai = U 0−1 (−λ) for all i. Substituting this solution
into (1) we obtain for the state equation:
˙x = nU 0−1 (−λ) − bx + f (x)
(7)
Equations (7) and (5) form the Modified Hamiltonian Dynamic System (MHDS)
for the optimal control problem associated with the cooperative equilibrium. A
steady state for this system is defined as ³−x,¯λ´ : ˙x = 0, ˙λ = 0with −ai = U0−1(−¯λ).
A MHDS and a cooperative equilibrium can be equivalently defined in the
state-control space. Differentiating (4) with respect to time we obtain
U 00 (ai) ˙ai = −˙λ
Substituting this into (5) we obtain
1
˙ai = U00 (ai) h³ρ+b−f0 (x)´U0 (ai)−nD0 (x)i, i = 1,...,n
(8)
The system of differential equations (6) (8) is the MHDS in the state-control space.
A steady state for this system characterizing the cooperative equilibrium is defined
as ³−x,−a´ : ˙x = 0, ˙ai = 0.
Assume a utility function with a constant elasticity of marginal utility
U 00 (a
−θ =
i) ai
U 0 (ai)
which is specialized to
a1−θ
U (a
i
i)
=
, 0 < θ < 1
(9)
1 − θ
U (ai) = log ai , θ = 1
(10)

---

Feedback Equilibria
9
In order to get a characterization of the cooperative benchmark that is inde-
pendent of n, we take θ = 1. After multiplying with n, the differential equation
for the control variable (8) under symmetry (so that ai is the same for all i) can
be expressed in terms of total loading a:
˙a = −h³ρ+b−f0 (x)´a−D0 (x)ai
(11)
Then the steady state ³−x,−a´ in the state-control space is obtained as the solution
of the system
n
a |˙x=0 = bx − f (x), a = Xai, ai the same for all i
(12)
i=1
³ρ+b−f0(x) 
D0 (x)
´
a |˙a=0 = 

(13)
As shown in Brock and Starrett (1999), under the assumptions made on the
U (ai) , f (x) , and D (x) functions, the system of (7) and (5) or equivalently (12)
and (13) has in general an odd number of steady states. Brock and Starrett also
show that locally unstable steady states, with possibly complex eigenvalues, lie
between two locally stable steady states. The first and the last steady state are
locally stable. Furthermore, the locally stable steady states have the saddle-point
property with a one-dimensional globally stable manifold.
The open-loop Nash equilibrium is determined by considering that each agent
i behaves non-cooperatively and that by taking the actions of the other agents
j 6= i as fixed, each agent maximizes the present value of its own net benefits (2)
subject to (1). The current value Hamiltonian is defined as:
 n

Gi = U (ai) − D (x) + µi ai+X¯aj−bx+f(x)
l6=i

(14)

---

Feedback Equilibria
10
The necessary conditions of the maximum principle are
U 0 (ai) = −µi, i = 1,...,n
(15)
∂
˙µ
Gi
i = ρµi −
=
∂x
³ρ+b−f0(x)´µi+D0(x), i=1,...,n
(16)
˙x = a − bx + f (x), x(0) = x0
(17)
along with the transversality conditions at infinity,
lim e−ρtµi (t) = 0, lim e−ρtµi (t) x (t) = 0
t→∞
t→∞
for all i.
Using (15) to eliminate µi from (16) as in the cooperative case and the as-
sumptions on the utility function, symmetry and multiplication by n yields:
1
˙a = −·³ρ+b−f0 (x)´a− D0 (x)a2¸
(18)
n
The OLNE is characterized by (17) and (18). OLNE steady states are determined
in the same way as in the cooperative case and they have the same properties
in terms of the number of steady states and the stability properties. The steady
states are solutions of the system
n³ρ+b−f0(x) 
D0 (x)
´
a |˙a=0 = 
 and (12)
(19)
Comparing the systems characterizing the cooperative and the OLNE steady
states, that is comparing (19) with (13), it can be easily seen that the a ˙a=0 curve in
the open-loop Nash case shifts upwards relative to the a ˙a=0 curve in the cooperative
case. This implies, for example, in the case of eutrophication of shallow lakes that
the pollutant accumulation for the locally stable OLNE steady states is higher

---