Citation: Sébastien Bernacchi, Michaela Weissgram, Walter Wukovits, Christoph Herwig. Process efficiency simulation for key process parameters in biological methanogenesis[J]. AIMS Bioengineering, 2014, 1(1): 53-71. doi: 10.3934/bioeng.2014.1.53
[1] | Sébastien Bernacchi, Simon Rittmann, Arne H. Seifert, Alexander Krajete, Christoph Herwig . Experimental methods for screening parameters influencing the growth to product yield (Y(x/CH4)) of a biological methane production (BMP) process performed with Methanothermobacter marburgensis. AIMS Bioengineering, 2014, 1(2): 72-87. doi: 10.3934/bioeng.2014.2.72 |
[2] | Karthikeyan D. Ramachandriya, Dimple K. Kundiyana, Ashokkumar M. Sharma, Ajay Kumar, Hasan K. Atiyeh, Raymond L. Huhnke, Mark R. Wilkins . Critical factors affecting the integration of biomass gasification and syngas fermentation technology. AIMS Bioengineering, 2016, 3(2): 188-210. doi: 10.3934/bioeng.2016.2.188 |
[3] | Adam B Fisher, Stephen S Fong . Lignin biodegradation and industrial implications. AIMS Bioengineering, 2014, 1(2): 92-112. doi: 10.3934/bioeng.2014.2.92 |
[4] | Velislava N Lyubenova, Maya N Ignatova . On-line estimation of physiological states for monitoring and control of bioprocesses. AIMS Bioengineering, 2017, 4(1): 93-112. doi: 10.3934/bioeng.2017.1.93 |
[5] | Daniel Borchert, Diego A. Suarez-Zuluaga, Yvonne E. Thomassen, Christoph Herwig . Risk assessment and integrated process modeling–an improved QbD approach for the development of the bioprocess control strategy. AIMS Bioengineering, 2020, 7(4): 254-271. doi: 10.3934/bioeng.2020022 |
[6] | Vered Tzin, Ilana Rogachev, Sagit Meir, Michal Moyal Ben Zvi, Tania Masci, Alexander Vainstein, Asaph Aharoni, Gad Galili . Altered Levels of Aroma and Volatiles by Metabolic Engineering of Shikimate Pathway Genes in Tomato Fruits. AIMS Bioengineering, 2015, 2(2): 75-92. doi: 10.3934/bioeng.2015.2.75 |
[7] | Islam Uddin, Salman A. AlQahtani, Sumaiya Noor, Salman Khan . Deep-m6Am: a deep learning model for identifying N6, 2′-O-Dimethyladenosine (m6Am) sites using hybrid features. AIMS Bioengineering, 2025, 12(1): 145-161. doi: 10.3934/bioeng.2025006 |
[8] | Urooj Ainuddin, Maria Waqas . Finite state machine and Markovian equivalents of the lac Operon in E. coli bacterium. AIMS Bioengineering, 2022, 9(4): 400-419. doi: 10.3934/bioeng.2022029 |
[9] | Zongyuan Zhu, Rachael Simister, Susannah Bird, Simon J. McQueen-Mason, Leonardo D. Gomez, Duncan J. Macquarrie . Microwave assisted acid and alkali pretreatment of Miscanthus biomass for biorefineries. AIMS Bioengineering, 2015, 2(4): 449-468. doi: 10.3934/bioeng.2015.4.449 |
[10] | Maria Waqas, Urooj Ainuddin, Umar Iftikhar . An analog electronic circuit model for cAMP-dependent pathway—towards creation of Silicon life. AIMS Bioengineering, 2022, 9(2): 145-162. doi: 10.3934/bioeng.2022011 |
Time inconsistency in dynamic decision making is often observed in social systems and daily life. Motivated by practical applications, especially in mathematical economics and finance, time-inconsistency control problems have recently attracted considerable research interest and efforts attempting to seek equilibrium, instead of optimal controls. At a conceptual level, the idea is that a decision made by the controller at every instant of time is considered as a game against all the decisions made by the future incarnations of the controller. An "equilibrium" control is therefore one such that any deviation from it at any time instant will be worse off. The study on time inconsistency by economists can be dated back to Stroz [1] and Phelps [2,3] in models with discrete time (see [4] and [5] for further developments), and adapted by Karp [6,7], and by Ekeland and Lazrak [8,9,10,11,12,13] to the case of continuous time. In the LQ control problems, Yong [14] studied a time-inconsistent deterministic model and derived equilibrium controls via some integral equations.
It is natural to study time inconsistency in the stochastic models. Ekeland and Pirvu [15] studied the non-exponential discounting which leads to time inconsistency in an agent's investment-consumption policies in a Merton model. Grenadier and Wang [16] also studied the hyperbolic discounting problem in an optimal stopping model. In a Markovian systems, Björk and Murgoci [17] proposed a definition of a general stochastic control problem with time inconsistent terms, and proposed some sufficient condition for a control to be solution by a system of integro-differential equations. They constructed some solutions for some examples including an LQ one, but it looks very hard to find not-to-harsh condition on parameters to ensure the existence of a solution. Björk, Murgoci and Zhou [18] also constructed an equilibrium for a mean-variance portfolio selection with state-dependent risk aversion. Basak and Chabakauri [19] studied the mean-variance portfolio selection problem and got more details on the constructed solution. Hu, Jin and Zhou [20,21] studied the general LQ control problem with time inconsistent terms in a non-Markovian system and constructed an unique equilibrium for quite general LQ control problem, including a non-Markovian system.
To the best of our knowledge, most of the time-inconsistent problems are associated with the control problems though we use the game formulation to define its equilibrium. In the problems of game theory, the literatures about time inconsistency is little [22,23]. However, the definitions of equilibrium strategies in the above two papers are based on some corresponding control problems like before. In this paper, we formulate a general stochastic LQ differential game, where the objective functional of each player include both a quadratic term of the expected state and a state-dependent term. These non-standard terms each introduces time inconsistency into the problem in somewhat different ways. We define our equilibrium via open-loop controls. Then we derive a general sufficient condition for equilibrium strategies through a system of forward-backward stochastic differential equations (FBSDEs). An intriguing feature of these FBSDEs is that a time parameter is involved; so these form a flow of FBSDEs. When the state process is scalar valued and all the coefficients are deterministic functions of time, we are able to reduce this flow of FBSDEs into several Riccati-like ODEs. Comparing to the ODEs in [20], though the state process is scalar valued, the unknowns are matrix-valued because of two players. Therefore, such ODEs are harder to solve than those of [20]. Under some more stronger conditions, we obtain explicitly an equilibrium strategy, which turns out to be a linear feedback. We also prove that the equilibrium strategy we obtained is unique.
The rest of the paper is organized as follows. The next section is devoted to the formulation of our problem and the definition of equilibrium strategy. In Section 3, we apply the spike variation technique to derive a flow of FBSEDs and a sufficient condition of equilibrium strategies. Based on this general results, we solve in Section 4 the case when the state is one dimensional and all the coefficients are deterministic. The uniqueness of such equilibrium strategy is also proved in this section.
Let $ T > 0 $ be the end of a finite time horizon, and let $ (W_t)_{0\le t\le{T}} = (W_t^1, ..., W_t^d)_{0\le t\le{T}} $ be a $ d $-dimensional Brownian motion on a probability space $ (\Omega, \mathcal{F}, \mathbb{P}) $. Denote by $ (\mathcal{F}_t) $ the augmented filtration generated by $ (W_t) $.
Let $ \mathbb{S}^n $ be the set of symmetric $ n\times n $ real matrices; $ L_{\mathcal{F}}^2(\Omega, \mathbb{R}^l) $ be the set of square-integrable random variables; $ L_{\mathcal{F}}^2(t, T;\mathbb{R}^n) $ be the set of $ \{\mathcal{F}_s\}_{s\in[t, T]} $-adapted square-integrable processes; and $ L_{\mathcal{F}}^2(\Omega; C(t, T;\mathbb{R}^n)) $ be the set of continuous $ \{\mathcal{F}_s\}_{s\in[t, T]} $-adapted square-integrable processes.
We consider a continuous-time, $ n $-dimensional nonhomogeneous linear controlled system:
$ dX_s = [A_sX_s+B_{1,s}'u_{1,s}+B_{2,s}'u_{2,s}+b_s]ds+\sum\limits_{j = 1}^d[C_s^jX_s+D_{1,s}^ju_{1,s}+D_{2,s}^ju_{2,s}+\sigma_s^j]dW_s^j, \\ X_0 = x_0. $ | (2.1) |
Here $ A $ is a bounded deterministic function on $ [0, T] $ with value in $ \mathbb{R}^{n\times n} $. The other parameters $ B_1, B_2, C, D_1, D_2 $ are all essentially bounded adapted processes on $ [0, T] $ with values in $ \mathbb{R}^{l\times n}, \mathbb{R}^{l\times n}, \mathbb{R}^{n\times n}, \mathbb{R}^{n\times l}, \mathbb{R}^{n\times l} $, respectively; $ b $ and $ \sigma^j $ are stochastic processes in $ L^2_{\mathcal{F}}(0, T;\mathbb{R}^n) $. The processes $ u_i\in L^2_{\mathcal{F}}(0, T;\mathbb{R}^l), \ i = 1, 2 $ are the controls, and $ X $ is the state process valued in $ \mathbb{R}^{n} $. Finally, $ x_0\in\mathbb{R}^{n} $ is the initial state. It is obvious that for any controls $ u_i\in L^2_{\mathcal{F}}(0, T;\mathbb{R}^l), \ i = 1, 2 $, there exists a unique solution $ X\in L^2_{\mathcal{F}}(\Omega, C(0, T;\mathbb{R}^n)) $.
As time evolves, we need to consider the controlled system starting from time $ t\in[0, T] $ and state $ x_t\in L^2_{\mathcal{F}_t}(\Omega; \mathbb{R}^n) $:
$ dX_s = [A_sX_s+B_{1,s}'u_{1,s}+B_{2,s}'u_{2,s}+b_s]ds+\sum\limits_{j = 1}^d[C_s^jX_s+D_{1,s}^ju_{1,s}+D_{2,s}^ju_{2,s}+\sigma_s^j]dW_s^j, \\ X_t = x_t. $ | (2.2) |
For any controls $ u_i\in L^2_{\mathcal{F}}(0, T;\mathbb{R}^l), \ i = 1, 2 $, there exists a unique solution $ X^{t, x_t, u_1, u_2}\in L^2_{\mathcal{F}}(\Omega, C(0, T;\mathbb{R}^n)) $.
We consider a two-person differential game problem. At any time $ t $ with the system state $ X_t = x_t $, the $ i $-th ($ i = 1, 2 $) person's aim is to minimize her cost (if maximize, we can times the following function by $ -1 $):
$ Ji(t,xt;u1,u2)=12Et∫Tt[⟨Qi,sXs,Xs⟩+⟨Ri,sui,s,ui,s⟩]ds+12Et[⟨GiXT,XT⟩]−12⟨hiEt[XT],Et[XT]⟩−⟨λixt+μi,Et[XT]⟩ $ | (2.3) |
over $ u_1, u_2\in L_{\mathcal{F}}^2(t, T;\mathbb{R}^l) $, where $ X = X^{t, x_t, u_1, u_2} $, and $ \mathbb{E}_t[\cdot] = \mathbb{E}[\cdot|\mathcal{F}_t] $. Here, for $ i = 1, 2 $, $ Q_i $ and $ R_i $ are both given essentially bounded adapted process on $ [0, T] $ with values in $ \mathbb{S}^n $ and $ \mathbb{S}^l $, respectively, $ G_i, h_i, \lambda_i, \mu_i $ are all constants in $ \mathbb{S}^n $, $ \mathbb{S}^n $, $ \mathbb{R}^{n\times n} $ and $ \mathbb{R}^n $, respectively. Furthermore, we assume that $ Q_i, R_i $ are non-negative definite almost surely and $ G_i $ are non-negative definite.
Given a control pair $ (u_1^*, u_2^*) $. For any $ t\in[0, T), \epsilon > 0 $, and $ v_1, v_2\in L_{\mathcal{F}_t}^2(\Omega, \mathbb{R}^l) $, define
$ ut,ϵ,vii,s=u∗i,s+vi1s∈[t,t+ϵ),s∈[t,T], i=1,2. $ | (2.4) |
Because each person at time $ t > 0 $ wants to minimize his/her cost as we claimed before, we have
Definition 2.1. Let $ (u_1^*, u_2^*)\in L_{\mathcal{F}}^2(0, T;\mathbb{R}^l)\times L_{\mathcal{F}}^2(0, T;\mathbb{R}^l) $ be a given strategy pair, and let $ X^* $ be the state process corresponding to $ (u_1^*, u_2^*) $. The strategy pair $ (u_1^*, u_2^*) $ is called an equilibrium if
$ limϵ↓0J1(t,X∗t;ut,ϵ,v11,u∗2)−J1(t,X∗t;u∗1,u∗2)ϵ≥0, $ | (2.5) |
$ limϵ↓0J2(t,X∗t;u∗1,ut,ϵ,v22)−J2(t,X∗t;u∗1,u∗2)ϵ≥0, $ | (2.6) |
where $ u_i^{t, \epsilon, v_i}, i = 1, 2 $ are defined by (2.4), for any $ t\in[0, T) $ and $ v_1, v_2\in L_{\mathcal{F}_t}^2(\Omega, \mathbb{R}^l) $.
Remark. The above definition means that, in each time $ t $, the equilibrium is a static Nash equilibrium in a corresponding game.
Let $ (u_1^*, u_2^*) $ be a fixed strategy pair, and let $ X^* $ be the corresponding state process. For any $ t\in[0, T) $, as a similar arguments of Theorem 5.1 in pp. 309 of [24], defined in the time interval $ [t, T] $, there exist adapted processes $ (p_i(\cdot; t), (k_i^j(\cdot; t)_{j = 1, 2, ..., d}))\in L_{\mathcal{F}}^2(t, T;\mathbb{R}^n)\times(L_{\mathcal{F}}^2(t, T;\mathbb{R}^n))^d $ and $ (P_i(\cdot; t), (K_i^j(\cdot; t)_{j = 1, 2, ..., d}))\in L_{\mathcal{F}}^2(t, T;\mathbb{S}^n)\times(L_{\mathcal{F}}^2(t, T;\mathbb{S}^n))^d $ for $ i = 1, 2 $ satisfying the following equations:
$ {dpi(s;t)=−[A′spi(s;t)+∑dj=1(Cjs)′kji(s;t)+Qi,sX∗s]ds+∑dj=1kji(s;t)dWjs,s∈[t,T],pi(T;t)=GiX∗T−hiEt[X∗T]−λiX∗t−μi, $ | (3.1) |
$ {dPi(s;t)=−{A′sPi(s;t)+Pi(s;t)As+Qi,s+∑dj=1[(Cjs)′Pi(s;t)Cjs+(Cjs)′Kji(s;t)+Kji(s;t)Cjs]}ds+∑dj=1Kji(s;t)dWji,s∈[t,T],Pi(T;t)=Gi, $ | (3.2) |
for $ i = 1, 2 $. From the assumption that $ Q_i $ and $ G_i $ are non-negative definite, it follows that $ P_i(s; t) $ are non-negative definite for $ i = 1, 2 $.
Proposition 1. For any $ t\in[0, T), \epsilon > 0 $, and $ v_1, v_2\in L_{\mathcal{F}_t}^2(\Omega, \mathbb{R}^l) $, define $ u_i^{t, \epsilon, v_i}, i = 1, 2 $ by (2.4). Then
$ J1(t,X∗t;ut,ϵ,v11,u∗2)−J1(t,X∗t;u∗1,u∗2)=Et∫t+ϵt{⟨Λ1(s;t),v1⟩+12⟨H1(s;t)v1,v1⟩}ds+o(ϵ), $ | (3.3) |
$ J2(t,X∗t;u∗1,ut,ϵ,v22)−J2(t,X∗t;u∗1,u∗2)=Et∫t+ϵt{⟨Λ2(s;t),v2⟩+12⟨H2(s;t)v2,v2⟩}ds+o(ϵ), $ | (3.4) |
where $ \Lambda_i(s; t) = B_{i, s}p_i(s; t)+\sum_{j = 1}^d(D_{i, s}^j)'k_i^j(s; t)+R_{i, s}u_{i, s}^* $ and $ H_i(s; t) = R_{i, s}+\sum_{j = 1}^d(D_{i, s}^j)'P_i(s; t)D_{i, s}^j $ for $ i = 1, 2 $.
Proof. Let $ X^{t, \epsilon, v_1, v_2} $ be the state process corresponding to $ u_i^{t, \epsilon, v_i}, i = 1, 2 $. Then by standard perturbation approach (cf. [20,25] or pp. 126-128 of [24]), we have
$ Xt,ϵ,v1,v2s=X∗s+Yt,ϵ,v1,v2s+Zt,ϵ,v1,v2s,s∈[t,T], $ | (3.5) |
where $ Y\equiv Y^{t, \epsilon, v_1, v_2} $ and $ Z\equiv Z^{t, \epsilon, v_1, v_2} $ satisfy
$ {dYs=AsYsds+∑dj=1[CjsYs+Dj1,sv11s∈[t,t+ϵ)+Dj2,sv21s∈[t,t+ϵ)]dWjs,s∈[t,T],Yt=0, $ | (3.6) |
$ {dZs=[AsZs+B′1,sv11s∈[t,t+ϵ)+B′2,sv21s∈[t,t+ϵ)]ds+∑dj=1CjsZsdWjs,s∈[t,T],Zt=0. $ | (3.7) |
Moreover, by Theorem 4.4 in [24], we have
$ Et[sups∈[t,T)|Ys|2]=O(ϵ),Et[sups∈[t,T)|Zs|2]=O(ϵ2). $ | (3.8) |
With $ A $ being deterministic, it follows from the dynamics of $ Y $ that, for any $ s\in[t, T] $, we have
$ Et[Ys]=∫stEt[AsYτ]dτ=∫stAsEt[Yτ]dτ. $ | (3.9) |
Hence we conclude that
$ Et[Ys]=0s∈[t,T]. $ | (3.10) |
By these estimates, we can calculate
$ Ji(t,X∗t;ut,ϵ,v11,ut,ϵ,v22)−Ji(t,X∗t;u∗1,u∗2)=12Et∫Tt[<Qi,s(2X∗s+Ys+Zs),Ys+Zs>+<Ri,s(2u∗i+vi),vi>1s∈[t,t+ϵ)]ds+Et[<GiX∗T,YT+ZT>]+12Et[<Gi(YT+ZT),YT+ZT>]−<hiEt[X∗T]+λiX∗t+μi,Et[YT+ZT]>−12<hiEt[YT+ZT],Et[YT+ZT]>=12Et∫Tt[<Qi,s(2X∗s+Ys+Zs),Ys+Zs>+<Ri,s(2u∗i+vi),vi>1s∈[t,t+ϵ)]ds+Et[<GiX∗T−hiEt[X∗T]−λiX∗t−μi,YT+ZT>+12<Gi(YT+ZT),YT+ZT>]+o(ϵ). $ | (3.11) |
Recalling that $ (p_i(\cdot; t), k_i(\cdot; t)) $ and $ (P_i(\cdot; t), K_i(\cdot; t)) $ solve, respectively, BSDEs (3.1) and (3.2) for $ i = 1, 2 $, we have
$ Et[<GiX∗T−hiEt[X∗T]−λiX∗t−μi,YT+ZT>]=Et[<pi(T;t),YT+ZT>]=Et[∫Ttd<pi(s;t),Ys+Zs>]=Et∫Tt[<pi(s;t),As(Ys+Zs)+B′1,sv11s∈[t,t+ϵ)+B′2,sv21s∈[t,t+ϵ)>−<A′spi(s;t)+d∑j=1(Cjs)′kji(s;t)+Qi,sX∗s,Ys+Zs>+d∑j=1<kji(s;t),Cjs(Ys+Zs)+Dj1,sv11s∈[t,t+ϵ)+Dj2,sv21s∈[t,t+ϵ)>]ds=Et∫Tt[<−Qi,sX∗s>+⟨B1,spi(s;t)+d∑j=1(Dj1,s)′kji(s;t),v11s∈[t,t+ϵ)⟩+⟨B2,spi(s;t)+d∑j=1(Dj2,s)′kji(s;t),v21s∈[t,t+ϵ)⟩]ds $ | (3.12) |
and
$ Et[12<Gi(YT+ZT),YT+ZT>]=Et[12<Pi(T;t)(YT+ZT),YT+ZT>]=Et[∫Ttd<Pi(s;t)(Ys+Zs),Ys+Zs>]=Et∫Tt{<Pi(s;t)(Ys+Zs),As(Ys+Zs)+B′1,sv11s∈[t,t+ϵ)+B′2,sv21s∈[t,t+ϵ)>+<Pi(s;t)[As(Ys+Zs)+B′1,sv11s∈[t,t+ϵ)+B′2,sv21s∈[t,t+ϵ)],Ys+Zs>−<[A′sPi(s;t)+Pi(s;t)As+Qi,s+d∑j=1((Cjs)′Pi(s;t)Cjs+(Cjs)′Kji(s;t)+Kji(s;t)Cjs)](Ys+Zs),Ys+Zs>+d∑j=1<Kji(s;t)(Ys+Zs),Cjs(Ys+Zs)+Dj1,sv11s∈[t,t+ϵ)+Dj2,sv21s∈[t,t+ϵ)>+d∑j=1<Kji(s;t)[Cjs(Ys+Zs)+Dj1,sv11s∈[t,t+ϵ)+Dj2,sv21s∈[t,t+ϵ)],Ys+Zs>+d∑j=1<Pi(s;t)[Cjs(Ys+Zs)+Dj1,sv11s∈[t,t+ϵ)+Dj2,sv21s∈[t,t+ϵ)],Cjs(Ys+Zs)+Dj1,sv11s∈[t,t+ϵ)+Dj2,sv21s∈[t,t+ϵ)>}ds=Et∫Tt[−<Qi,s(Ys+Zs),Ys+Zs>+d∑j=1<Pi(s;t)[Dj1,sv1+Dj2,sv2],Dj1,sv1+Dj2,sv2>1s∈[t,t+ϵ)]ds+o(ϵ) $ | (3.13) |
Combining (3.11)-(3.13), we have
$ Ji(t,X∗t;ut,ϵ,v11,ut,ϵ,v22)−Ji(t,X∗t;u∗1,u∗2)=Et∫Tt[12<Ri,s(2u∗i+vi),vi>1s∈[t,t+ϵ)+⟨B1,spi(s;t)+d∑j=1(Dj1,s)′kji(s;t),v11s∈[t,t+ϵ)⟩+⟨B2,spi(s;t)+d∑j=1(Dj2,s)′kji(s;t),v21s∈[t,t+ϵ)⟩+12d∑j=1<Pi(s;t)[Dj1,sv1+Dj2,sv2],Dj1,sv1+Dj2,sv2>1s∈[t,t+ϵ)]ds+o(ϵ). $ | (3.14) |
Take $ i = 1 $, we let $ v_2 = 0 $, then $ u_2^{t, \epsilon, v_2} = u_2^* $, from (3.14), we obtain
$ J1(t,X∗t;ut,ϵ,v11,u∗2)−J1(t,X∗t;u∗1,u∗2)=Et∫Tt{⟨R1,su∗1+B1,sp1(s;t)+d∑j=1(Dj1,s)′kj1(s;t),v11s∈[t,t+ϵ)⟩+12⟨[R1,s+d∑j=1(Dj1,s)′P1(s;t)Dj1,s]v1,v1⟩}ds=Et∫t+ϵt{<Λ1(s;t),v1>+12<H1(s;t)v1,v1>}ds+o(ϵ). $ | (3.15) |
This proves (3.3), and similarly, we obtain (3.4).
Because of $ R_{i, s} $ and $ P_i(s; t), i = 1, 2 $ are non-negative definite, $ H_i(s; t), \ i = 1, 2 $ are also non-negative definite. In view of (3.3)-(3.4), a sufficient condition for an equilibrium is
$ Et∫Tt|Λi(s;t)|ds<+∞,lims↓tEt[Λi(s;t)]=0 a.s. ∀t∈[0,T],i=1,2. $ | (3.16) |
By an arguments similar to the proof of Proposition 3.3 in [21], we have the following lemma:
Lemma 3.1. For any triple of state and control processes $ (X^*, u_1^*, u_2^*) $, the solution to BSDE (3.1) in $ L^2(0, T;\mathbb{R}^n)\times (L^2(0, T;\mathbb{R}^n))^d $ satisfies $ k_i(s; t_1) = k_i(s; t_2) $ for a.e. $ s\ge\max\{t_1, t_2\}, \; i = 1, 2 $. Furthermore, there exist $ \rho_i\in L^2(0, T;\mathbb{R}^l) $, $ \delta_i\in L^2(0, T;\mathbb{R}^{l\times n}) $ and $ \xi_i\in L^2(\Omega; C(0, T;\mathbb{R}^n)) $, such that
$ Λi(s;t)=ρi(s)+δi(s)ξi(t),i=1,2. $ | (3.17) |
Therefore, we have another characterization for equilibrium strategies:
Proposition 2. Given a strategy pair $ (u_1^*, u_2^*)\in L^2(0, T;\mathbb{R}^l)\times L^2(0, T;\mathbb{R}^l) $. Denote $ X^* $ as the state process, and $ (p_i(\cdot; t), (k_i^j(\cdot; t)_{j = 1, 2, ..., d}))\in L_{\mathcal{F}}^2(t, T;\mathbb{R}^n)\times(L_{\mathcal{F}}^2(t, T;\mathbb{R}^n))^d $ as the unique solution for the BSDE (3.1), with $ k_i(s) = k_i(s; t) $ according to Lemma 3.1 for $ i = 1, 2 $ respectively. For $ i = 1, 2 $, letting
$ Λi(s,t)=Bi,spi(s;t)+d∑j=1(Dj,s)′k(s;t)j+Ri,su∗i,s,s∈[t,T], $ | (3.18) |
then $ u^* $ is an equilibrium strategy if and only if
$ Λi(t,t)=0,a.s.,a.e.t∈[0,T],i=1,2. $ | (3.19) |
Proof. From (3.17), we have $ \Lambda_1(s; t) = \rho_1(s)+\delta_1(s)\xi_1(t) $. Since $ \delta_1 $ is essentially bounded and $ \xi_1 $ is continuous, we have
$ \lim\limits_{\epsilon\downarrow0}\mathbb{E}_t\left[{1\over\epsilon}\int_t^{t+\epsilon}|\delta_1(s)(\xi_1(s)-\xi_1(t))|ds\right] \le c\lim\limits_{\epsilon\downarrow0}{1\over\epsilon}\int_t^{t+\epsilon}\mathbb{E}_t[|\xi_1(s)-\xi_1(t)|]ds = 0, $ |
and hence
$ \lim\limits_{\epsilon\downarrow0}{1\over\epsilon}\int_t^{t+\epsilon}\mathbb{E}_t[\Lambda_1(s;t)]ds = \lim\limits_{\epsilon\downarrow0}{1\over\epsilon}\int_t^{t+\epsilon}\mathbb{E}_t[\Lambda_1(s;s)]ds. $ |
Therefore, if (3.19) holds, we have
$ \lim\limits_{\epsilon\downarrow0}{1\over\epsilon}\int_t^{t+\epsilon}\mathbb{E}_t[\Lambda_1(s;t)]ds = \lim\limits_{\epsilon\downarrow0}{1\over\epsilon}\int_t^{t+\epsilon}\mathbb{E}_t[\Lambda_1(s;s)]ds = 0. $ |
When $ i = 2 $, we can prove (3.19) similarly.
Conversely, if (3.16) holds, then $ \lim_{\epsilon\downarrow0}{1\over\epsilon}\int_t^{t+\epsilon}\mathbb{E}_t[\Lambda_i(s; s)]ds = 0, i = 1, 2 $ leading to (3.19) by virtue of Lemma 3.4 of [21].
The following is the main general result for the stochastic LQ differential game with time-inconsistency.
Theorem 3.2. A strategy pair $ (u_1^*, u_2^*)\in L_{\mathcal{F}}^2(0, T;\mathbb{R}^l)\times L_{\mathcal{F}}^2(0, T;\mathbb{R}^l) $ is an equilibrium strategy pair if the following two conditions hold for any time $ t $:
(i) The system of SDEs
$ {dX∗s=[AsX∗s+B′1,su∗1,s+B′2,su∗2,s+bs]ds+∑dj=1[CjsX∗s+Dj1,su∗1,s+Dj2,su∗2,s+σjs]dWjs,X∗0=x0,dp1(s;t)=−[A′sp1(s;t)+∑dj=1(Cjs)′kj1(s;t)+Q1,sX∗s]ds+∑dj=1kj1(s;t)dWjs,s∈[t,T],p1(T;t)=G1X∗T−h1Et[X∗T]−λ1X∗t−μ1,dp2(s;t)=−[A′sp2(s;t)+∑dj=1(Cjs)′kj2(s;t)+Q2,sX∗s]ds+∑dj=1kj2(s;t)dWjs,s∈[t,T],p2(T;t)=G2X∗T−h2Et[X∗T]−λ2X∗t−μ2, $ | (3.20) |
admits a solution $ (X^*, p_1, k_1, p_2, k_2) $;
(ii) $ \Lambda_i(s; t) = R_{i, s}u_{i, s}^*+B_{i, s}p_i(s; t)+\sum_{j = 1}^d(D_{i, s}^j)'k_i^j(s; t), i = 1, 2 $ satisfy condition (3.19).
Proof. Given a strategy pair $ (u_1^*, u_2^*)\in L_{\mathcal{F}}^2(0, T;\mathbb{R}^l)\times L_{\mathcal{F}}^2(0, T;\mathbb{R}^l) $ satisfying (i) and (ii), then for any $ v_1, v_2\in L_{\mathcal{F}_t}^2(\Omega, \mathbb{R}^l) $, define $ \Lambda_i, H_i, i = 1, 2 $ as in Proposition 1. We have
$ limϵ↓0J1(t,X∗t;ut,ϵ,v11,u∗2)−J1(t,X∗t;u∗1,u∗2)ϵ=limϵ↓0Et∫t+ϵt{<Λ1(s;t),v1>+12<H1(s;t)v1,v1>}dsϵ≥limϵ↓0Et∫t+ϵt<Λ1(s;t),v1>dsϵ=0, $ | (3.21) |
proving the first condition of Definition 2.1, and the proof of the second condition is similar.
Theorem 3.2 involve the existence of solutions to a flow of FBSDEs along with other conditions. The system (3.20) is more complicated than system (3.6) in [20]. As declared in [20], "proving the general existence for this type of FBSEs remains an outstanding open problem", it is also true for our system (3.20).
In the rest of this paper, we will focus on the case when $ n = 1 $. When $ n = 1 $, the state process $ X $ is a scalar-valued rocess evolving by the dynamics
$ dX_s = [A_sX_s+B_{1,s}'u_{1,s}+B_{2,s}'u_{2,s}+b_s]ds+[C_sX_s+D_{1,s}u_{1,s}+D_{2,s}u_{2,s}+\sigma_s]'dW_s, \\ X_0 = x_0, $ | (3.22) |
where $ A $ is a bounded deterministic scalar function on $ [0, T] $. The other parameters $ B, C, D $ are all essentially bounded and $ \mathcal{F}_t $-adapted processes on $ [0, T] $ with values in $ \mathbb{R}^l, \mathbb{R}^d, \mathbb{R}^{d\times l} $, respectively. Moreover, $ b\in L_{\mathcal{F}}^2(0, T;\mathbb{R}) $ and $ \sigma\in L_{\mathcal{F}}^2(0, T;\mathbb{R}^d) $.
In this case, the adjoint equations for the equilibrium strategy become
$ {dpi(s;t)=−[A′spi(s;t)+(Cs)′ki(s;t)+Qi,sX∗s]ds+ki(s;t)′dWs,s∈[t,T],pi(T;t)=GiX∗T−hiEt[X∗T]−λiX∗t−μi, $ | (3.23) |
$ {dPi(s;t)=−[(2As+|Cs|2)Pi(s;t)+2C′sK(s;t)+Qi,s]ds+Ki(s;t)′dWs,s∈[t,T],Pi(T;t)=Gi, $ | (3.24) |
for $ i = 1, 2 $. For convenience, we also state here the $ n = 1 $ version of Theorem 3.2:
Theorem 3.3. A strategy pair $ (u_1^*, u_2^*)\in L_{\mathcal{F}}^2(0, T;\mathbb{R}^l)\times L_{\mathcal{F}}^2(0, T;\mathbb{R}^l) $ is an equilibrium strategy pair if, for any time $ t\in[0, T) $,
(i) The system of SDEs
$ {dX∗s=[AsX∗s+B′1,su∗1,s+B′2,su∗2,s+bs]ds+[CsX∗s+D1,su∗1,s+D2,su∗2,s+σs]′dWs,X∗0=x0,dp1(s;t)=−[Asp1(s;t)+(Cs)′k1(s;t)+Q1,sX∗s]ds+k1(s;t)′dWs,s∈[t,T],p1(T;t)=G1X∗T−h1Et[X∗T]−λ1X∗t−μ1,dp2(s;t)=−[Asp2(s;t)+(Cs)′k2(s;t)+Q2,sX∗s]ds+k2(s;t)′dWs,s∈[t,T],p2(T;t)=G2X∗T−h2Et[X∗T]−λ2X∗t−μ2, $ | (3.25) |
admits a solution $ (X^*, p_1, k_1, p_2, k_2) $;
(ii) $ \Lambda_i(s; t) = R_{i, s}u_{i, s}^*+B_{i, s}p_i(s; t)+(D_{i, s})'k_i(s; t), i = 1, 2 $ satisfy condition (3.19).
The unique solvability of (3.25) remains a challenging open problem even for the case $ n = 1 $. However, we are able to solve this problem when the parameters $ A, B_1, B_2, C, D_1, D_2, b, \sigma, Q_1, Q_2, R_1 $ and $ R_2 $ are all deterministic functions.
Throughout this section we assume all the parameters are deterministic functions of $ t $. In this case, since $ G_1, G_2 $ have been also assumed to be deterministic, the BSDEs (3.24) turns out to be ODEs with solutions $ K_i\equiv0 $ and $ P_i(s; t) = G_ie^{\int_s^T(2A_u+|C_u|^2)du}+\int_s^Te^{\int_s^T(2A_u+|C_u|^2)du}Q_{i, v}dv $ for $ i = 1, 2 $. Hence, the equilibrium strategy will be characterized through a system of coupled Riccati-type equations.
As in classical LQ control, we attempt to look for a linear feedback equilibrium strategy pair. For such purpose, motivated by [20], given any $ t\in[0, T] $, we consider the following process:
$ pi(s;t)=Mi,sX∗s−Ni,sEt[X∗s]−Γi,sX∗t+Φi,s,0≤t≤s≤T,i=1,2, $ | (4.1) |
where $ M_i, N_i, \Gamma_i, \Phi_i $ are deterministic differentiable functions with $ \dot{M}_i = m_i, \dot{N}_i = n_i, \dot{\Gamma}_i = \gamma_i $ and $ \dot\Phi_i = \phi_i $ for $ i = 1, 2 $. The advantage of this process is to separate the variables $ X_s^*, \mathbb{E}_t[X_s^*] $ and $ X_t^* $ in the solutions $ p_i(s; t), i = 1, 2 $, thereby reducing the complicated FBSDEs to some ODEs.
For any fixed $ t $, applying Ito's formula to (4.1) in the time variable $ s $, we obtain, for $ i = 1, 2 $,
$ dp_i(s;t) = \{M_{i,s}(A_sX_s^*+B_{1,s}'u_{1,s}^*+B_{2,s}'u_{2,s}^*+b_s)\\+m_{i,s}X_s^*-N_{i,s}\mathbb{E}_t[A_sX_s^*+B_{1,s}'u_{1,s}^*+B_{2,s}'u_{2,s}^*+b_s] \\ \qquad -n_{i,s}\mathbb{E}_t[X_s^*]-\gamma_{i,s}X_t^*+\phi_{i,s}\}ds+M_{i,s}(C_sX_s^*+D_{1,s}u_{1,s}^*+D_{2,s}u_{2,s}^*+\sigma_s)'dW_s. $ | (4.2) |
Comparing the $ dW_s $ term of $ dp_i(s; t) $ in (3.25) and (4.2), we have
$ ki(s;t)=Mi,s[CsX∗s+D1,su∗1,s+D2,su∗2,s+σs],s∈[t,T],i=1,2. $ | (4.3) |
Notice that $ k(s; t) $ turns out to be independent of $ t $.
Putting the above expressions (4.1) and (4.3) of $ p_i(s; t) $ and $ k_i(s; t), i = 1, 2 $ into (3.19), we have
$ R_{i,s}u_{i,s}^*+B_{i,s}[(M_{i,s}-N_{i,s}-\Gamma_{i,s})X_s^*+\Phi_{i,s}]+D_{i,s}'M_{i,s}[C_sX_s^*+D_{1,s}u_{1,s}^*+D_{2,s}u_{2,s}^*+\sigma_s]\\ = 0, \;s\in[0,T], $ | (4.4) |
for $ i = 1, 2 $. Then we can formally deduce
$ u∗i,s=αi,sX∗s+βi,s,i=1,2. $ | (4.5) |
Let $ M_s = {{{\text{diag}}}}(M_{1, s}I_l, M_{2, s}I_l), N_s = {{{\text{diag}}}}(N_{1, s}I_l, N_{2, s}I_l), \Gamma_s = {{{\text{diag}}}}(\Gamma_{1, s}I_l, \Gamma_{2, s}I_l), \Phi_s = {{{\text{diag}}}}(\Phi_{1, s}I_l, \Phi_{2, s}I_l) $, $ R_s = {{{\text{diag}}}}(R_{1, s}, R_{2, s}), B_s = \left(B1,sB2,s\right), D_s = \left(D1,s,D2,s\right) $, $ u_s^* = \left(u∗1,su∗2,s\right), \alpha_s = \left(α1,sα2,s\right) $ and $ \beta_s = \left(β1,sβ2,s\right) $. Then from (4.4), we have
$ Rsu∗s+[(Ms−Ns−Γs)X∗s+Φs]Bs+MsD′s[CsX∗s+Ds(αsX∗s+βs)+σs]=0,s∈[0,T] $ | (4.6) |
and hence
$ αs=−(Rs+MsD′sDs)−1[(Ms−Ns−Γs)Bs+MsD′sCs], $ | (4.7) |
$ βs=−(Rs+MsD′sDs)−1(ΦsBs+MsD′sσs). $ | (4.8) |
Next, comparing the $ ds $ term of $ dp_i(s; t) $ in (3.25) and (4.2) (we supress the argument $ s $ here), we have
$ M_i[AX^*+B'(\alpha X^*+\beta)+b]+m_iX^*-N_i\{A\mathbb{E}_t[X^*]+\\ B'\mathbb{E}_t[\alpha X^*+\beta]+b\} -n_i\mathbb{E}_t[X^*]-\gamma_iX_t^*+\phi_i \\ = -[A(M_iX^*-N_i\mathbb{E}_t[X^*]-\Gamma_iX_t^*+\Phi_i)+M_iC'(CX^*+D(\alpha X^*+\beta)+\sigma)]. $ | (4.9) |
Notice in the above that $ X^* = X_s^* $ and $ \mathbb{E}_t[X^*] = \mathbb{E}_t[X_s^*] $ due to the omission of $ s $. This leads to the following equations for $ M_i, N_i, \Gamma_i, \Phi_i $:
$ \left\{ ˙Mi=−(2A+|C|2)Mi−Qi+Mi(B′+C′D)(R+MD′D)−1[(M−N−Γ)B+MD′C],s∈[0,T],Mi,T=Gi; \right. $ | (4.10) |
$ {˙Ni=−2ANi+NiB′(R+MD′D)−1[(M−N−Γ)B+MD′C],s∈[0,T],Ni,T=hi; $ | (4.11) |
$ {˙Γi=−AΓi,s∈[0,T],Γi,T=λi; $ | (4.12) |
$ {˙Φi=−{A−[B′(M−N)+C′DM](R+MD′D)−1B}Φi−(Mi−Ni)b−MiC′σ−[(Mi−Ni)B′+MiC′D](R+MD′D)−1MD′σ,s∈[0,T],Φi,T=−μi. $ | (4.13) |
Though $ M_i, N_i, \Gamma_i, \Phi_i, i = 1, 2 $ are scalars, $ M, N, \Gamma, \Phi $ are now matrices because of two players. Therefore, the above equations are more complicated than the similar equations (4.5)–(4.8) in [20]. Before we solve the equations (4.10)–(4.13), we first prove that, if exist, the equilibrium constructed above is the unique equilibrium. Indeed, we have
Theorem 4.1. Let
$ L1={X(⋅;⋅):X(⋅;t)∈L2F(t,T;R),supt∈[0,T]E[sups≥t|X(s;t)|2]<+∞} $ | (4.14) |
and
$ L2={Y(⋅;⋅):Y(⋅;t)∈L2F(t,T;Rd),supt∈[0,T]E[∫Tt|X(s;t)|2ds]<+∞}. $ | (4.15) |
Suppose all the parameters $ A, B_1, B_2, C, D_1, D_2, b, \sigma, Q_1, Q_2, R_1 $ and $ R_2 $ are all deterministic.
When $ (M_i, N_i, \Gamma_i, \Phi_i), i = 1, 2 $ exist, and for $ i = 1, 2 $, $ (p_i(s; t), k_i(s; t))\in\mathcal{L}_1\times \mathcal{L}_2 $, the equilibrium strategy is unique.
Proof. Suppose there is another equilibrium $ (X, u_1, u_2) $, then the BSDE (3.1), with $ X^* $ replaced by $ X $, admits a solution $ (p_i(s; t), k_i(s), u_{i, s}) $ for $ i = 1, 2 $, which satisfies $ B_{i, s}p_i(s; s)+D_{i, s}'k_i(s)+R_{i, s}u_{i, s} = 0 $ for a.e. $ s\in[0, T] $. For $ i = 1, 2 $, define
$ ˉpi(s;t)≜pi(s;t)−[Mi,sXs−Ni,sEt[Xs]−Γi,s+Φi,s], $ | (4.16) |
$ ˉki(s;t)≜ki(s)−Mi,s(CsXs+D1,su1,s+D2,su2,s+σs), $ | (4.17) |
where $ k_i(s) = k_i(s; t) $ by Lemma 3.1.
We define $ p(s; t) = {{{\text{diag}}}}(p_1(s; t)I_l, p_2(s; t)I_l) $, $ \bar{p}(s; t) = {{{\text{diag}}}}(\bar{p}_1(s; t)I_l, \bar{p}_2(s; t)I_l) $, and $ u = \left(u1,su2,s\right) $. By the equilibrium condition (3.19), we have
$ 0 = \left(B1,sp1(s;s)+D′1,sk1(s)+R1,su1,sB2,sp2(s;s)+D′2,sk2(s)+R2,su2,s\right) \\ = p(s;s)B_s+\left(D′1,sk1(s)D′2,sk2(s)\right)+R_su_s \\ = [\bar{p}(s;s)+X_s(M_s-N_s-\Gamma_s)+\Phi_s]B_s+\left(D′1,sˉk1(s)D′2,sˉk2(s)\right) +\\M_sD_s'(C_sX_s+D_su_s+\sigma_s)+R_su_s \\ = \bar{p}(s;s)B_s+\left(D′1,sˉk1(s)D′2,sˉk2(s)\right) +X_s[(M_s-N_s-\Gamma_s)B_s+M_sD_s'C_s]+\Phi_sB_s+M_sD_s'\sigma_s \\ +(R_s+M_sD_s'D_s)u_s. $ | (4.18) |
Since $ R_s+M_sD_s'D_s $ is invertible, we have
$ us=−(Rs+MsD′sDs)−1{ˉp(s;s)Bs+(D′1,sˉk1(s)D′2,sˉk2(s))+Xs[(Ms−Ns−Γs)Bs+MsD′sCs]+ΦsBs+MsD′sσs}, $ | (4.19) |
and hence for $ i = 1, 2 $,
$ d\bar{p}_i(s;t) = dp_i(s;t)-d[M_{i,s}X_s-N_{i,s}\mathbb{E}_t[X_s]-\Gamma_{i,s}+\Phi_{i,s}] \\ = -[A_sp_i(s;t)+C_{s}'k_i(s)+Q_{i,s}X_s]ds+k_i'(s)dW_s-d[M_{i,s}X_s-N_{i,s}\mathbb{E}_t[X_s]-\Gamma_{i,s}X_t+\Phi_{i,s}] \\ = -\bigg\{A_s\bar{p}_i(s;t)+C_s'\bar{k}_i(s)+A_s(M_{i,s}X_s-N_{i,s}\mathbb{E}_t[X_s]-\Gamma_{i,s}X_t+\Phi_{i,s}) \\ \ +C_s'M_{i,s}(C_sX_s+D_{1,s}u_{1,s}+D_{2,s}u_{2,s}+\sigma_s)\bigg\}ds \\ \ +[\bar{k}_i(s)-M_{i,s}(C_sX_s+D_{1,s}u_{1,s}+D_{2,s}u_{2,s}+\sigma_s)]'dW_s \\ -\bigg\{M_{i,s}[A_sX_s+B_s'u_s+b_s]+m_{i,s}X_s-N_{i,s}(A_s\mathbb{E}_t[X_s]+B_s'\mathbb{E}_t[u_s]+b_s) \\ \ -n_{i,s}\mathbb{E}_t[X_s]-\gamma_{i,s}X_t+\phi_{i,s}\bigg\}ds \\ -M_{i,s}[C_sX_s+D_su_s+\sigma_s]'dW_s \\ = -\Bigg\{A_s\bar{p}_i(s;t)+C_s'\bar{k}_i(s)-M_{i,s}(B_s'+C_s'D_s)(R_s+M_sD_s'D_s)^{-1}\left[B_s\bar{p}(s;s)+\left(D′1,sˉk1(s)D′2,sˉk2(s)\right)\right] \\ \qquad N_{i,s}B_s'(R_s+M_sD_s'D_s)^{-1}\mathbb{E}_t\left[B_s\bar{p}(s;s)+\left(D′1,sˉk1(s)D′2,sˉk2(s)\right)\right]\Bigg\}ds+\bar{k}_i(s)'dW_s, $ | (4.20) |
where we suppress the subscript $ s $ for the parameters, and we have used the equations (4.10)–(4.13) for $ M_i, N_i, \Gamma_i, \Phi_i $ in the last equality. From (4.16) and (4.17), we have $ (\bar{p}_i, \bar{k}_i)\in\mathcal{L}_1\times\mathcal{L}_2 $. Therefore, by Theorem 4.2 of [21], we obtain $ \bar{p}(s; t)\equiv0 $ and $ \bar{k}(s)\equiv0 $.
Finally, plugging $ \bar{p}\equiv\bar{k}\equiv0 $ into $ u $ of (4.19), we get $ u $ being the same form of feedback strategy as in (4.5), and hence $ (X, u_1, u_2) $ is the same as $ (X^*, u_1^*, u_2^*) $ which defined by (4.5) and (3.25).
The solutions to (4.12) is
$ Γi,s=λie∫TsAtdt, $ | (4.21) |
for $ i = 1, 2 $. Let $ \tilde{N} = N_1/ N_2 $, from (4.11), we have $ \dot{\tilde{N}} = 0 $, and hence
$ ˜N≡h1h2,N2≡h2h1N1. $ | (4.22) |
Equations (4.10) and (4.11) form a system of coupled Riccati-type equations for $ (M_1, M_2, N_1) $:
$ {˙M1=−[2A+|C|2+B′Γ(R+MD′D)−1(B+D′C)]M1−Q1+(B+D′C)′(R+MD′D)−1M(B+D′C)M1−B′N(R+MD′D)−1(B+D′C)M1,M1,T=G1;˙M2=−[2A+|C|2+B′Γ(R+MD′D)−1(B+D′C)]M2−Q2+(B+D′C)′(R+MD′D)−1M(B+D′C)M2−B′N(R+MD′D)−1(B+D′C)M2,M2,T=G2;˙N1=−2ANi+NiB′(R+MD′D)−1[(M−N−Γ)B+MD′C],N1,T=h1. $ | (4.23) |
Finally, once we get the solution for $ (M_1, M_2, N_1) $, (4.13) is a simple ODE. Therefore, it is crucial to solve (4.23).
Formally, we define $ \tilde{M} = {M_1\over M_2} $ and $ J_1 = {M_1\over N_1} $ and study the following equation for $ (M_1, \tilde{M}, J_1) $:
$ {˙M1=−[2A+|C|2+B′Γ(R+MD′D)−1(B+D′C)]M1−Q1+(B+D′C)′(R+MD′D)−1M(B+D′C)M1−B′N(R+MD′D)−1(B+D′C)M1,M1,T=G1;˙˜M=−(Q1M1−Q2M1˜M)˜M,˜MT=G1G2;˙J1=−[|C|2−C′D(R+MD′D)−1M(B+D′C)+B′Γ(R+MD′D)−1D′C+Q1M1]J1−C′D(R+MD′D)−1Mdiag(Il,h2h1˜MIl)B,J1,T=G1h1, $ | (4.24) |
where $ M = {{{\text{diag}}}}(M_1I_l, {M_1\over\tilde M}I_l), N = {{{\text{diag}}}}({M_1\over J_1}I_l, {h_2\over h_1}{M_1\over J_1}I_l) $ and $ \Gamma = {{{\text{diag}}}}(\lambda_1e^{\int_s^T A_tdt}I_l, \lambda_2e^{\int_s^T A_tdt}I_l) $.
By a direct calculation, we have
Proposition 3. If the system (4.24) admits a positive solution $ (M_1, \tilde{M}, J_1) $, then the system (4.23) admits a solution $ (M_1, M_2, N_1) $.
In the following, we will use the truncation method to study the system (4.24). For convenienc, we use the following notations:
$ a∨b=max{a,b},∀a,b∈R, $ | (4.25) |
$ a∧b=min{a,b},∀a,b∈R. $ | (4.26) |
Moreover, for a matrix $ M\in\mathbb{R}^{m\times n} $ and a real number $ c $, we define
$ (M∨c)i,j=Mi,j∨c,∀1≤i≤m,1≤j≤n, $ | (4.27) |
$ (M∧c)i,j=Mi,j∧c,∀1≤i≤m,1≤j≤n. $ | (4.28) |
We first consider the standard case where $ R-\delta{I}\succeq0 $ for some $ \delta > 0 $. We have
Theorem 4.2. Assume that $ R-\delta{I}\succeq0 $ for some $ \delta > 0 $ and $ G\ge h > 0 $. Then (4.24), and hence (4.23) admit unique solution if
(i) there exists a constant $ \lambda\ge0 $ such that $ B = \lambda D'C $;
(ii) $ \frac{|C|^2}{2l}D'D-(\lambda+1)D'CC'D\succeq0 $.
Proof. For fixed $ c > 0 $ and $ K > 0 $, consider the following truncated system of (4.24):
$ {˙M1=−[2A+|C|2+B′Γ(R+M+cD′D)−1(B+D′C)]M1−Q1+(B+D′C)′(R+M+cD′D)−1(M+c∧K)(B+D′C)M1−B′(N+c∧K)(R+M+cD′D)−1(B+D′C)M1,M1,T=G1;˙˜M=−(Q1M1∨c−Q2M1∨c˜M∧K)˜M,˜MT=G1G2;˙J1=−λ(1)J1−C′D(R+M+cD′D)−1(M+c∧K)diag(Il,h2h1(˜M∧K)Il)B,J1,T=G1h1, $ | (4.29) |
where $ M_{c}^+ = {{{\text{diag}}}}((M_1\vee0)I_l, {{M_1\vee0}\over\tilde M\vee c}I_l) $, $ N_c^+ = {{{\text{diag}}}}({{M_1\vee0}\over J_1\vee c}I_l, {h_2\over h_1}{{M_1\vee0}\over J_1\vee c}I_l) $ and
$ λ(1)=|C|2−C′D(R+M+cD′D)−1(M+c∧K)(B+D′C)+B′Γ(R+M+cD′D)−1D′C+Q1M1∨c. $ | (4.30) |
Since $ R-\delta I\succeq0 $, the above system (4.29) is locally Lipschitz with linear growth, and hence it admits a unique solution $ (M_1^{c, K}, \tilde{M}^{c, K}, J_1^{c, K}) $. We will omit the superscript $ (c, K) $ when there is no confusion.
We are going to prove that $ J_1\ge1 $ and that $ M_1, \tilde{M}\in[L_1, L_2] $ for some $ L_1, L_2 > 0 $ independent of $ c $ and $ K $ appearing in the truncation functions. We denote
$ λ(2)=(2A+|C|2+B′Γ(R+M+cD′D)−1(B+D′C))−(B+D′C)′(R+M+cD′D)−1(M+c∧K)(B+D′C)−B′(N+c∧K)(R+M+cD′D)−1(B+D′C). $ | (4.31) |
Then $ \lambda^{(2)} $ is bounded, and $ M_1 $ satisfies
$ ˙M1+λ(2)M1+Q1=0,M1,T=G1. $ | (4.32) |
Hence $ M_1 > 0 $. Similarly, we have $ \tilde{M} > 0 $.
The equation for $ \tilde{M} $ is
$ {−˙˜M=(Q1M1∨c˜M−Q2M1∨c(˜M∧K)˜M,˜MT=G1G2; $ | (4.33) |
hence $ \tilde{M} $ admits an upper bound $ L_2 $ independent of $ c $ and $ K $. Choosing $ K = L_2 $ and examining again (4.33), we deduce that there exists $ L_1 > 0 $ independent of $ c $ and $ K $ such that $ \tilde{M}\ge L_1 $. Indeed, we can choose $ L_1 = \min_{0\le t\le T}{Q_{1, t}\over Q_{2, t}}\wedge{G1\over G_2} $ and $ L_2 = \max_{0\le t\le T}{Q_{1, t}\over Q_{2, t}}\vee{G1\over G_2} $. As a result, choosing $ c < L_1 $, the terms $ M_c^+ $ can be replaced by $ M = {{{\text{diag}}}}(M_1I_l, {M_1\over\tilde M}I_l) $, respectively, in (4.29) without changing their values.
Now we prove $ J\ge1 $. Denote $ \tilde{J} = J_1-1 $, then $ \tilde{J} $ satisfies the ODE:
$ ˙˜J=−λ(1)˜J−[λ(1)+C′D(R+MD′D)−1(M∧K)diag(Il,h2h1˜MIl)B]=−λ(1)˜J−a(1), $ | (4.34) |
where
$ a^{(1)} = \lambda^{(1)}+C'D(R+MD'D)^{-1}(M\wedge K){{{\text{diag}}}}(I_l,{h_2\over h_1}\tilde{M}I_l)B \\ = |C|^2-(\lambda+1)C'D(R+MD'D)^{-1}(M\wedge K)D'C+C'D\Gamma(R+MD'D)^{-1}(M\wedge K)D'C++{Q_1\over M_1\vee c} \\ +C'D(R+MD'D)^{-1}(M\wedge K){{{\text{diag}}}}(I_l,{h_2\over h_1}\tilde{M}I_l)D'C \\ \ge|C|^2-(\lambda+1)C'D(R+MD'D)^{-1}MD'C+C'D\Gamma(R+MD'D)^{-1}(M\wedge K)D'C++{Q_1\over M_1\vee c} \\ = tr\left\{(R+MD'D)^{-1}{|C|^2+Q_1/(M_1\vee c)\over2l}(R+MD'D)\right\}-(\lambda+1)tr\{(R+MD'D)^{-1}D'CC'DM\} \\ = tr\left\{(R+MD'D)^{-1}H\right\} $ | (4.35) |
with $ H = {|C|^2+Q_1/(M_1\vee c)\over2l}(R+D'DM)-(\lambda+1)D'CC'DM $.
When $ c $ is small enough such that $ R-cD'D\succeq0 $, we have
$ Q1M1∨c(R+MD′D)≥Q1L2D′D. $ | (4.36) |
Hence,
$ H⪰(|C|22lD′D−(λ+1)D′CC′D)M⪰0, $ | (4.37) |
and consequently $ a^{(1)}\ge tr\{(R+MD'D)^{-1}H\}\ge0 $. We then deduce that $ \tilde{J}\ge0 $, and hence $ J_1\ge1 $. The boundness of $ M_1 $ can be proved by a similar argument in the proof of Theorem 4.2 in [20].
Similarly, for the singular case $ R\equiv0 $, we have
Theorem 4.3. Given $ G_1\ge h_1\ge1, R\equiv0 $, if $ B = \lambda D'C $ and $ |C|^2-(\lambda+1)C'D(D'D)^{-1}D'C\ge0 $, then (4.24) and (4.23) admit a unique positive solution.
Concluding the above two theorems, we can present our main results of this section:
Theorem 4.4. Given $ G_1\ge h_1\ge1 $ and $ B = \lambda D'C $. The (4.23) admits a unique positive solution $ (M_1, M_2, N_1) $ in the following two cases:
(i) $ R-\delta I\succeq0 $ for some $ \delta > 0 $, $ \frac{|C|^2}{2l}D'D-(\lambda+1)D'CC'D\succeq0 $;
(ii) $ R\equiv0 $, $ |C|^2-(\lambda+1)C'D(D'D)^{-1}D'C\ge0 $.
Proof. Define $ p_i(s; t) $ and $ k_i(s; t) $ by (4.1) and (4.3), respectively. It is straightforward to check that $ (u_1^*, u_2^*, X^*, p_1, p_2, k_1, k_2) $ satisfies the system of SDEs (3.25). Moreover, in the both cases, we can check that $ \alpha_{i, s} $ and $ \beta_{i, s} $ in (4.5) are all uniformly bounded, and hence $ u_i^*\in L_{\mathcal{F}}^2(0, T;\mathbb{R}^l) $ and $ X^*\in L^2(\Omega; C(0, T;\mathbb{R})) $.
Finally, denote $ \Lambda_i(s; t) = R_{i, s}u_{i, s}^*+p_i(s; t)B_{i, s}+(D_{i, s})'k_i(s; t), i = 1, 2 $. Plugging $ p_i, k_i, u_i^* $ define in (4.1), (4.3) and (4.5) into $ \Lambda_i $, we have
$ \Lambda_i(s;t) = R_{i,s}u_{i,s}^*+(M_{i,s}X_s^*-N_{i,s}\mathbb{E}_t[X_s^*]-\Gamma_{i,s}X_t^*+\Phi_{i,s})B_{i,s} +\\ M_{i,s}D_{i,s}'[C_sX_s^*+D_{1,s}u_{1,s}^*+D_{2,s}u_{2,s}^*+\sigma_s] $ | (4.38) |
and hence,
$ \Lambda(t;t) \triangleq \left(Λ1(t;t)Λ2(t;t)\right) \\ = (R_t+M_tD_t'D_t)u_t^*+M_t(B_t+D_t'C_t)X_t^* -N_tB_t\mathbb{E}_t[X_t^*]-\Gamma_tB_tX_t^*+(\Phi_tB_t+M_tD_t'\sigma_t) \\ = -[(M_t-N_t-\Gamma_t)B_t+M_tD_t'C_t]X_t^*-(\Phi_tB_t+M_tD_t'\sigma_t) \\ \quad+M_t(B_t+D_t'C_t)X_t^*-N_tB_tX_t^*-\Gamma_tB_tX_t^*+(\Phi_tB_t+M_tD_t'\sigma_t) \\ = 0. $ | (4.39) |
Therefore, $ \Lambda_i $ satisfies the seond condition in (3.19).
We investigate a general stochastic linear-quadratic differential game, where the objective functional of each player include both a quadratic term of the expected state and a state-dependent term. As discussed in detail in Björk and Murgoci [17] and [18], the last two terms in each objective functional, respectively, introduce two sources of time inconsistency into the differential game problem. That is to say, the usual equilibrium aspect is not a proper way when the players at 0 cannot commit the players at all intermediate times to implement the decisions they have planed. With the time-inconsistency, the notion "equilibrium" needs to be extended in an appropriate way. We turn to adopt the concept of equilibrium strategy between the players at all different times, which is at any time, an equilibrium "infinitesimally" via spike variation. By applying the spike variation technique, We derive a sufficient condition for equilibrium strategies via a system of forward-backward stochastic differential equation. The unique solvability of such FBSDEs remains a challenging open problem.
For a special case, when the state is one-dimensional and the coefficients are all deterministic, the equilibrium strategy will be characterized through a system of coupled Riccati-type equations. At last, we find an explicit equilibrium strategy, which is also proved be the unique equilibrium strategy.
The research of the first author was partially supported by NSFC (No.12171426), the Natural Science Foundation of Zhejiang Province (No. Y19A010020) and the Fundamental Research Funds for the Central Universities (No. 2021FZZX001-01). The research of the second author was partially supported by NSFC (No. 11501325, No.71871129) and the China Postdoctoral Science Foundation (Grant No. 2018T110706, No.2018M642641). The authors would like to thank sincerely the referees and the associate editor for their helpful comments and suggestions.
The authors declare there is no conflicts of interest.
[1] | Klell M, (2010) Handbook of Hydrogen Storage. WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim. |
[2] | Schill N, van Gulik WM, Voisard D, et al. (1996) Continuous Cultures Limited by a Gaseous Substrate: Development of a Simple, Unstructured Mathematical Model and Experimental Verification with Methanobacteriumthermoautotrophicum. Biotechnol Bioeng 51: 645-658. |
[3] | Martinez Porqueras, E, Rittmann S, Herwig C (2012) Biofuels and CO2 neutrality: an opportunity. Biofuels 3(4): 413-426. |
[4] | Aspen Technology I, Aspen Plus V7.1. In. Aspen Technology, Inc, Burlington, USA, 2008. |
[5] |
Xu Y, Chuang KT, Sanger AR (2002) Design of a process for purification of Isopropyl Alcohol by Hydration of Propylene in a catalytic distillation column. J Trans Icheme 80:686-694. doi: 10.1205/026387602760312908
![]() |
[6] |
Rittmann S, Seifert A, Herwig C (2012) Quantitative analysis of media dilution rate effects on Methanothermobactermarburgensis grown in continuous culture on H2 and CO2. Biomass Bioenerg 36: 293-301. doi: 10.1016/j.biombioe.2011.10.038
![]() |
[7] |
Foglia D, Ljunggren M, Wukovits W, et al. (2010) Integration studies on a two-stage fermentation process for the production of biohydrogen. J Clean Prod 18: s72-s80. doi: 10.1016/j.jclepro.2010.06.022
![]() |
[8] | Foglia DWW, Friedl A, de Vrije T, et al. (2011) Fermentative Hydrogen Production: Influence of Application of Mesophilic and Thermophilic Bacteria on Mass and Energy Balances. Chem Eng Trans 25: 815-820. |
[9] |
Rittmann S, Herwig C (2012) A comprehensive and quantitative review of dark fermentative biohydrogen production. Microbial Cell Factories 11:115. doi: 10.1186/1475-2859-11-115
![]() |
[10] | Dutta A, Dowe N, Ibsen K N, et al. (2009) An Economic Comparison of Different Fermentation Configurations to Convert Corn Stover to Ethanol Using Z. mobilis and Saccharomyces. Biotechnol Prog 26: 64-72. |
[11] | Chen CC, Evans LB (1986) A local composition model for the excess Gibbs energy of aqueous electrolyte systems. Aiche Journal 32(3): 444-454. |
[12] |
dePoorter LMI, Geerts WJ, Keltjens JT (2007) Coupling of methanothermobacter thermautotrophicus methane formation and growth in fed-batch and continuous cultures under different H2gassing regimens†. Appl Environ Microbiol 73: 740-749. doi: 10.1128/AEM.01885-06
![]() |
[13] |
Prapaitrakul W, Shwikhat A, King Jr AD (1987) The influence of pH on gas solubilities in aqueous solutions of sodium octanoate at 25°C. J. colloid. Interface Sci 115: 443-449. doi: 10.1016/0021-9797(87)90061-0
![]() |
[14] | DVGW, 2011. Hydrogen in the natural gas grid. Results of the workshop “Concepts for energy storage”, Essen, January 2011, DBI Gas- und Umwelttechnik (in German). |
[15] | Enertrag, Combined cycle power plant Uckermark, 2009. Available from: https://www.enertrag.com/download/prospekt/wirkungsgrade.pdf (last access 29. Apr. 2013). |
[16] | FVEE annual meeting, New Routes for the Production of Substitute Natural Gas (SNG) from Renewable Energy, Berlin (in German) 2010. Available from: http://www.fvee.de/fileadmin/publikationen/Themenhefte/th2009/th2009_05_06.pdf. |
[17] | Sterner M, (2009) Bioenergy and renewable power methane in integrated 100% renewable energy systems. Limiting global warming by transforming energy systems. Kassel: Kassel University press. |
[18] | Sterner M, Jentsch M, Holzhammer U (2011) Energy economical and ecological evaluation of a wind-gas project (in German). Available from: http://www.greenpeace-energy.de/fileadmin/docs/sonstiges/ Greenpeace_Energy_Gutachten_Windgas_Fraunhofer_Sterner.pdf (last access 12.Jul.2013) |
[19] |
Dolfing J, Larter SR, Head IM (2008) Thermodynamic constraints on methanogenic crude oil biodegradation. J ISME 2: 442-452. doi: 10.1038/ismej.2007.111
![]() |
[20] | Cerbe G (2008) Basic principles of gas technology, gas procurement - gas distribution - gas utilization, 7 ed. Carl HanserVerlag GmbH & Co. KG. |
[21] | Betterton EA (ed.) (1992) Henry's law constants of soluble and moderately soluble organic gases: Effects in aqueous phase chemistry. Wiley, New York, NY |
[22] | Clegg S, Whitfield M (1995) A chemical model of seawater including dissolved ammonia and the stoichiometric dissociation constant of ammonia in estuarine water and seawater from -2 to 40°C. Geochim Cosmochim Ac 59(12): 2403-2421. |
[23] |
Riebesell U, et al. (2000) Reduced calcification of marine phytoplankton in response to increased atmospheric CO2. Nature 407: 364-367. doi: 10.1038/35030078
![]() |
[24] |
Harned H, Davis R (1943) The Ionization Constant of Carbonic Acid in Water and the Solubility of Carbon Dioxide in Water and Aqueous Salt Solutions from 0 to 50°. J Am Chem Soc 65: 2030-2037. doi: 10.1021/ja01250a059
![]() |
[25] | Ronnow PH, Gunnarson LAH (1981) Sulfide-Dependent Methane Production and Growth of a ThermophilicMethanogenic Bacterium. App Environ Microbiol 42: 580-584. |
[26] | Bulatovic SM (2007) Chapter 8: Interaction of Inorganic Regulating Reagents. In: Handbook of Flotation Reagents. Elsevier 153-184. |
[27] |
Seifert AH, Rittmann S, Bernacchi S, et al. (2013) Method for assessing the impact of emission gasses on physiology and productivity in biological methanogenesis. Bioresour. Technol 136: 747-751. doi: 10.1016/j.biortech.2013.03.119
![]() |
[28] | Herwig C, Seifert AH (2014) Selektive Abtrennung von Wasserbeigleichzeltigerbiomasseund Medienkomponentenretention. Patentschrift AT 513 378 B1 2014-04-15 |
[29] | Meyer HP, Schmidhalter D (2014) Industrial Scale Suspension Culture of Living Cells. Chapter: Bacterial process by Sagmeister et al., Wiley-Blackwel. |
[30] | DNV KEMA Energy & Sustainability, Final report: System analyses power to gas. Tecnology review. Available from: http://www.dnv.com/binaries/DNV%20KEMA%20%282013%29%20-%20Systems%20Anal yses%20Power%20to%20Gas%20-%20Technology%20Review_tcm4-567461.pdf |
[31] | Keener J, Sneyd J (2009). Mathematical Physiology: I and Cellular Physiology (2nd edition.). Springer 547. |
[32] | Bernhardt G, Jaenicke R, Lüdemann HD, et al. (1988) High Pressure Enhances the Growth Rate of the Thermophilic Archaebacterium Methanococcus thermolithotrophicus without Extending Its Temperature Range. Appl Environ Microbiol 54: 1258-1261. |
[33] | Seifert AH, Rittmann S, Herwig C (2014) Analysis of process related factors to increase volumetric productivity and quality of biomethane with Methanothermobacter marburgensis Appl Energ 132: 155–162. |
1. | Bernhard Lecker, Lukas Illi, Andreas Lemmer, Hans Oechsner, Biological hydrogen methanation – A review, 2017, 245, 09608524, 1220, 10.1016/j.biortech.2017.08.176 | |
2. | Yan Rafrafi, Léa Laguillaumie, Claire Dumas, Biological Methanation of H2 and CO2 with Mixed Cultures: Current Advances, Hurdles and Challenges, 2020, 1877-2641, 10.1007/s12649-020-01283-z | |
3. | Sébastien Bernacchi, Simon Rittmann, Arne H. Seifert, Alexander Krajete, Christoph Herwig, Experimental methods for screening parameters influencing the growth to product yield (Y(x/CH4)) of a biological methane production (BMP) process performed with Methanothermobacter marburgensis, 2014, 1, 2375-1495, 72, 10.3934/bioeng.2014.2.72 | |
4. | Birgit Hoff, Jens Plassmeier, Matthew Blankschien, Anne‐Catrin Letzel, Lauralynn Kourtz, Hartwig Schröder, Walter Koch, Oskar Zelder, Unlocking Nature's Biosynthetic Power—Metabolic Engineering for the Fermentative Production of Chemicals, 2021, 133, 0044-8249, 2288, 10.1002/ange.202004248 | |
5. | Annalisa Abdel Azim, Christian Pruckner, Philipp Kolar, Ruth-Sophie Taubner, Debora Fino, Guido Saracco, Filipa L. Sousa, Simon K.-M.R. Rittmann, The physiology of trace elements in biological methane production, 2017, 241, 09608524, 775, 10.1016/j.biortech.2017.05.211 | |
6. | Grazia Leonzio, Process analysis of biological Sabatier reaction for bio-methane production, 2016, 290, 13858947, 490, 10.1016/j.cej.2016.01.068 | |
7. | Lisa-Maria Mauerhofer, Patricia Pappenreiter, Christian Paulik, Arne H. Seifert, Sébastien Bernacchi, Simon K.-M. R. Rittmann, Methods for quantification of growth and productivity in anaerobic microbiology and biotechnology, 2019, 64, 0015-5632, 321, 10.1007/s12223-018-0658-4 | |
8. | Johanny Pestalozzi, Claudia Bieling, Dirk Scheer, Cordula Kropp, Integrating power-to-gas in the biogas value chain: analysis of stakeholder perception and risk governance requirements, 2019, 9, 2192-0567, 10.1186/s13705-019-0220-5 | |
9. | Simon K.-M.R. Rittmann, Arne H. Seifert, Sébastien Bernacchi, Kinetics, multivariate statistical modelling, and physiology of CO2-based biological methane production, 2018, 216, 03062619, 751, 10.1016/j.apenergy.2018.01.075 | |
10. | Simon K.-M. R. Rittmann, Arne H. Seifert, Alexander Krajete, Biomethanisierung — ein Prozess zur Ermöglichung der Energiewende?, 2014, 20, 0947-0867, 816, 10.1007/s12268-014-0521-3 | |
11. | Birgit Hoff, Jens Plassmeier, Matthew Blankschien, Anne‐Catrin Letzel, Lauralynn Kourtz, Hartwig Schröder, Walter Koch, Oskar Zelder, Unlocking Nature's Biosynthetic Power—Metabolic Engineering for the Fermentative Production of Chemicals, 2021, 60, 1433-7851, 2258, 10.1002/anie.202004248 | |
12. | Sébastien Bernacchi, Alexander Krajete, Christoph Herwig, Experimental workflow for developing a feed forward strategy to control biomass growth and exploit maximum specific methane productivity of Methanothermobacter marburgensis in a biological methane production process (BMPP), 2016, 2, 2471-1888, 262, 10.3934/microbiol.2016.3.262 | |
13. | Joanna Kazimierowicz, Marcin Dębowski, Marcin Zieliński, Effectiveness of Hydrogen Production by Bacteroides vulgatus in Psychrophilic Fermentation of Cattle Slurry, 2022, 4, 2571-8797, 806, 10.3390/cleantechnol4030049 | |
14. | Eduardo Sánchez Nocete, Javier Pérez Rodríguez, A Simple Methodology for Estimating the Potential Biomethane Production in a Region: Application in a Case Study, 2022, 14, 2071-1050, 15978, 10.3390/su142315978 | |
15. | Eike Janesch, Joana Pereira, Peter Neubauer, Stefan Junne, Phase Separation in Anaerobic Digestion: A Potential for Easier Process Combination?, 2021, 3, 2673-2718, 10.3389/fceng.2021.711971 | |
16. | Pauls P. Argalis, Kristine Vegere, Perspective Biomethane Potential and Its Utilization in the Transport Sector in the Current Situation of Latvia, 2021, 13, 2071-1050, 7827, 10.3390/su13147827 | |
17. | Monika Vítězová, Vladimír Onderka, Iva Urbanová, Anna Molíková, Nikola Hanišáková, Iva Buriánková, Tomáš Vítěz, David Novák, Jan Lochman, Markéta Machálková, Jakub Javůrek, In situ field experiment shows the potential of methanogenic archaea for biomethane production from underground gas storage in natural rock environment, 2023, 23521864, 103253, 10.1016/j.eti.2023.103253 | |
18. | 2024, 9783527352753, 1, 10.1002/9783527843954.ch1 | |
19. | Zohreh Safari, Rouhollah Fatehi, Reza Azin, Developing a numerical model for microbial methanation in a depleted hydrocarbon reservoir, 2024, 09601481, 120426, 10.1016/j.renene.2024.120426 |