Corrigendum to "(Almost) everything you always wanted to know about deterministic control problems in stratified domains"

1.   Introduction

The aim of this note is to report and correct an error that we have found in [1]. We illustrate the problem we are facing by producing an explicit counter-example to the comparison result but we also solve this difficulty by updating the definition of subsolutions (no modification is needed for the supersolution condition). To give the reader a quick (yet precise) formulation of the corrections we develop hereafter, let us summarize them as

1. All the results in [1] are valid, as they are formulated, if the subsolutions are assumed to be continuous functions in $\mathbb{R}^N$.

2. All the results in [1] are also valid in the case of upper semi-continuous subsolutions, provided we assume that they satisfy an Ishii's subsolution condition in addition to the "stratified" definition found in [1].

Let us now give more details. In [1], we are considering deterministic control problems whose dynamics and costs $(b,l)$ at any point $(x,t) \in \mathbb{R}^N \times [0,T]$ are chosen in a bounded, closed and convex set $\boldsymbol{\rm{BL}}(x,t)$. The classical Hamiltonian is defined by

and the aim is to give a suitable sense and study the associated Hamilton-Jacobi-Bellman Equation which, for classical problems, reads

If, for such classical problems, $\boldsymbol{\rm{BL}}(x,t)$ and $H$ are continuous everywhere, having a stratified problem means on the contrary that they can have discontinuities but with a particular structure.

More precisely, $\mathbb{R}^N$ can be decomposed as

where, for any $k$, $\boldsymbol{\rm{M}}^{k}$ is a $k$-dimensional submanifold of $\mathbb{R}^N$. The manifolds $\boldsymbol{\rm{M}}^{0},\boldsymbol{\rm{M}}^{1},\dots,\boldsymbol{\rm{M}}^{N-1}$ are disjoints (but they may have several connected components) and they are the locations where either $\boldsymbol{\rm{BL}}(x,t)$ or $H$ can have discontinuities, which essentially means (taking into account the assumptions made on $\boldsymbol{\rm{BL}}$) that we have more dynamics and costs on each $\boldsymbol{\rm{M}}^{k}$, which can be seen as specific control problems on $\boldsymbol{\rm{M}}^{k}$.

To treat in a proper way these specific control problems on $\boldsymbol{\rm{M}}^{k}$, it is necessary to introduce the Hamiltonians which are defined on $\boldsymbol{\rm{M}}^{k}$ by

where $T_x\boldsymbol{\rm{M}}^{k}$ is the tangent space to $\boldsymbol{\rm{M}}^{k}$ at $x$ and the associated Hamilton-Jacobi-Bellman Equation

The first aim of [1] was to provide a definition of viscosity sub and supersolution for Hamilton-Jacobi-Bellman Equations in Stratified Domain [(HJB-SD) for short], namely (1)-(2).

Before going further, we point out that we assume throughout this short note that the natural assumptions for a stratified problem are always satisfied: (ⅰ) $\mathbb{M} = (\boldsymbol{\rm{M}}^{k})_{k = 0..N}$ is a regular stratification of $\mathbb{R}^N$, (ⅱ) the Hamiltonians (or BL) satisfy the key assumptions of [1], namely (TC) (tangential continuity), (NC) (normal controllability) and (LP) (Lipschitz continuity).

The definition of super and subsolutions in [1] follows the ones of Bressan & Hong [2]: a lower semi-continuous function $v$ is a supersolution of (HJB-SD) if it is a supersolution of (1) in the classical Ishii's sense, while an upper semi-continuous function $u$ is a subsolution of (HJB-SD) if it is a subsolution (again in the classical Ishii's sense) of each equation (2) for any $k = 0,\cdots,N$. And it is worth pointing out that these $H^k$-inequalities are really inequalities on $\boldsymbol{\rm{M}}^{k}\times(0,T]$, $i.e.$ they are obtained by considering maxima of $u-\phi$ on $\boldsymbol{\rm{M}}^{k}\times(0,T]$ for any smooth test-function $\phi$.

Unfortunately, this way of defining subsolutions only in terms of the $H^k$'s is not sufficent since it treats all the $\boldsymbol{\rm{M}}^{k}$ separately without linking them and this allows $u$ to have artificial values on certain manifolds $\boldsymbol{\rm{M}}^{k}$. Let us mention that this difficulty does not appear in Bressan & Hong [2] since the subsolutions are assumed to be continuous.

A counter-example — Consider in $\mathbb{R}^N$ the equation

for which the "control" solution is given, in $\overline{B(0,1)}$ by

Now we can consider the stratification where $\boldsymbol{\rm{M}}^{N-1} = S(0,1) = \{x : |x| = 1\}$ and $\boldsymbol{\rm{M}}^{N} = \mathbb{R}^N\setminus \boldsymbol{\rm{M}}^{N-1}$. Then the above equation can be reformulated in terms of this stratified domain: for subsolutions, we have the same equation in $\boldsymbol{\rm{M}}^{N}$ while

where $D_T u$ stands for the tangent derivative of $u$ on the sphere $S(0,1)$. For this stratified formulation,

is obviously a subsolution which is upper semi-continuous since $U(x) = \exp(-1)<1$ on $S(0,1)$. But $v: = U$ in all $\mathbb{R}^N$ is a also a supersolution and of course $u(x)\leq v(x)$ does not hold in $\mathbb{R}^N$ because on $S(0,1)$, $u(x) = 1>v(x) = \exp(-1)$.

As we said above, the key fact in this counter-example to comparison is that we can put artificial values on $\boldsymbol{\rm{M}}^{N-1}$ for the subsolution $u$ since these values are unrelated with those of $u$ on $\boldsymbol{\rm{M}}^{N}$. If $u$ is assumed to be continuous, then the only solution is $U$.

We also point out that we could have provided a more pathological counter-example: if $x_0\in S(0,1)$, we can set $\boldsymbol{\rm{M}}^{0} = \{x_0\}$ and $\boldsymbol{\rm{M}}^{N} = \mathbb{R}^N\setminus \boldsymbol{\rm{M}}^{0}$. Then the condition on $\boldsymbol{\rm{M}}^{0}$ reduces to

and a pathological subsolution can be built by changing only the value of $U$ at $x_0$ by setting $u(x_0) = \alpha \in (\exp(-1),1]$, this interval of values ensuring the upper-semi-continuity of $u$.

2.   Correcting the definition of subsolutions

As we already mention it above, a simple way to correct our results is just to assume the subsolutions to be continuous. In this case, no extra requirement in the definition of subsolutions is needed and all the results in [1] -in particular the comparison result- apply readily as they are formulated. From another point of view, properties $(a)-(b)$ below which are needed for having a comparison result with an upper semi-continuous subsolution $u$, are obviously true for continuous subsolutions.

In order to have the right properties for upper semi-continuous subsolutions (namely $(a)-(b)$ below), without stating them as assumptions, it is enough to add a global Ishii type requirement for subsolutions, namely

Definition 2.1. An upper semi-continuous function $u$ is a stratified subsolution of (HJB-SD) if it satisfies

and for any $k = 0..N$,

We recall that

is the lower semi-continuous enveloppe of $H$ in $\mathbb{R}^N\times[0,T]\times\mathbb{R}^N$. This way we ensure that the control problem on $\boldsymbol{\rm{M}}^{k}$ (with perhaps low costs) is really seen in terms of HJB Equations. The supersolution condition is unchanged.

3.   Correcting the approximation argument

A key step in the comparison proof in [1] is to regularize $u$ by a tangent sup-convolution and then by a standard convolution. We claim in [1,Lemma 5.5] that, by doing so, we obtain a sequence of subsolutions $(u^\epsilon)_\epsilon$ which are continuous but the above counter-example shows that this statement was wrong in general. The reason is that if $x\in\boldsymbol{\rm{M}}^{k}$ and

then the tangential sup-convolution need not be continuous in the normal direction to $\boldsymbol{\rm{M}}^{k}$. The fix consists in making sure that the following property is fulfilled

$(a)$ For any $x \in \boldsymbol{\rm{M}}^{k}$, $k<N-1$, $u(x) = \limsup\{u(y),\ y\to x,\ y \notin \boldsymbol{\rm{M}}^{k}\}$.

$(b)$ For any $x \in \boldsymbol{\rm{M}}^{N-1}$,

where, for $r>0$ small enough, $U_+,U_-\subset\boldsymbol{\rm{M}}^{N}\cap B(x,r)$ are the locally disjoint connected components of $\big(\mathbb{R}^N\setminus\boldsymbol{\rm{M}}^{N-1}\big)\cap B(x,r)$.

Lemma 5.5 in [1] can now be corrected by assuming $(a)$ and $(b)$

Lemma 3.1. Let $x\in\boldsymbol{\rm{M}}^{k}$ and $t,h>0$. There exists $r'>0$ such that if $u$ is a subsolution of (HJB-SD) in $B(x,r')\times (t-h,t)$ satisfying in addition conditions $(a)$ and $(b)$, then for any $a\in(0,r')$, there exists a sequence of Lipschitz continuous functions $(u^\epsilon)_\epsilon$ in $\overline{B}(x,r'-a) \times (t-h/2,t)$ satisfying

$(i)$ the $u^\epsilon$ are subsolutions of (HJB-SD) in $B(x,r'-a) \times (t-h/2,t)$,

$(ii)$ $ {\rm{lim}}\;{\sup ^ * }\;{u^\epsilon } = u $.

$(iii)$ The restriction of $u^\epsilon$ to $\boldsymbol{\rm{M}}^{k}\cap \Big[B(x,r'-a) \times (t-h/2,t)\Big]$ is $C^1$.

Proof. The proof is exactly the one given in [1] but we need a little additional argument. We first reduce to the case of a flat stratification through a change of variables. Without loss of generality, we can assume that $x = 0$, and writing the coordinates in $\mathbb{R}^N$ as $(y_1,y_2)$ with $y_1\in \mathbb{R}^k$, $y_2 \in \mathbb{R}^{N-k}$ we may assume that $\boldsymbol{\rm{M}}^{k}: = \{(y_1,y_2):\ y_2 = 0\}$.

Then we perform a sup-convolution in the $\boldsymbol{\rm{M}}^{k}$ directions (and also the time direction) by setting

for some large enough constant $K>0$ (as explained in [1], for $k = 0$ it is enough to do only a time sup-convolution).

Here is the place where we introduce an additional regularity argument: the sup-convolution is clearly is Lipschitz continuous in the $y_1$ and $s$ variables and the normal controllability implies that it is also Lipschitz continuous in the $y_2$-variable for $y_2\neq 0$. It remains to connect the values of $u^{\epsilon_1, \alpha_1}_1(y_1,y_2,s)$ for $y_2\neq 0$ and $y_2 = 0$; this is where we need conditions $(a)$ and $(b)$. With this addition, the rest of the proof remains identical.

Let us notice that in particular, Lemma 3.1 is valid if $u$ is continuous since $(a)$ and $(b)$ are obviously satisfied. This is why there is no problem at all in [1] if we consider continuous subsolutions. Now, in the case of upper semi-continuous subsolutions, a more usable assumption than $(a)-(b)$ is to consider viscosity subsolutions in the sense of Ishii.

Lemma 3.2. Let $u$ be an upper semi-continuous function satisfying (3). Then $(a)$ and $(b)$ hold.

Proof. In order to prove $(a)$ we can assume that we are in the stationary case (for simplicity) and that $\boldsymbol{\rm{M}}^{k}$ if flat (this reduction is done in [1] using the regularity of $\boldsymbol{\rm{M}}^{k}$). Consider the function

where $e$ is any unit vector normal to $\boldsymbol{\rm{M}}^{k}$. If $u(x) > \limsup\{u(y),\ y\to x,\ y \notin \boldsymbol{\rm{M}}^{k}\}$, the maximum of this function is necessarily achieved on $\boldsymbol{\rm{M}}^{k}$ at ${y_\epsilon }$. Using the normal controllabillity in $\boldsymbol{\rm{M}}^{N}$ we deduce that $H_*$ is coercive in any normal direction to $\boldsymbol{\rm{M}}^{k}$ so that if $C$ is large enough we reach a contradiction in $H_*({y_\epsilon } ,u({y_\epsilon } ),D\phi({y_\epsilon }))\leq0$, which gives the desired property $(i)$.

In the case of $\boldsymbol{\rm{M}}^{N-1}$, the same proof works but this is not enough since, locally, $\mathbb{R}^N\setminus \boldsymbol{\rm{M}}^{N-1}$ has two connected components, $U_+$ and $U_-$ and we have to show that the property is true separately for both connected components. Here, we assume again that $\boldsymbol{\rm{M}}^{N-1}$ is an hyperplane and we take the same test-function but with $e = +n$ or $-n$ where $n$ is a normal vector to $\boldsymbol{\rm{M}}^{N-1}$.

If $u(x) > \limsup\{u(y),\ y\to x,\ y \notin \boldsymbol{\rm{M}}^{N-1}, n\cdot (y-x)>0\}$, we consider

The maximum cannot be achieved in the domain where $n\cdot (y-x)>0$ because of the above hypothesis on $u(x)$. But in the complementary of this set, the term $+Cn\cdot (y-x)$ has the right sign ($i.e.$, it is non-positive), allowing to show that a maximum is achieved and is converging to $x$ as $\epsilon\to0$. Therefore we can choose $C\gg \epsilon^{-1}$ and again the normal controllability of $H_*$ allows to get the contradiction. The argument is the same for the other connected component of $\boldsymbol{\rm{M}}^{N-1}$.

4.   Comparison and stability results

The comparison result [1,Theorem 5.2] relies on local arguments, a descending induction and the tangential regularization of the subsolution. With the corrected version of the regularization (Lemma 3.1 above), all the arguments that are used apply for either continuous, or stratified solutions.

Theorem 4.1. For any open subset $\Omega$ of $\mathbb{R}^N$ and for any $0\leq t_1<t_2\leq T$, we have a comparison result for (HJB-SD) in $Q = \Omega \times (t_1,t_2)$, i.e. for any bounded upper semi-continuous stratified subsolution $u$ of (HJB-SD) in $Q$ and any bounded lower semi-continuous supersolution $v$ of (HJB-SD) in $Q$, then

where $\partial_p Q$ denotes the parabolic boundary of $Q$, i.e. $\partial_p Q: = \partial \Omega \times [t_1,t_2] \cup \overline \Omega \times \{t_1\}$.

The immediate Corollary is that there is a unique stratified solution of the problem. Concerning the stability result [1,Theorem 6.2], we need only to modify the subsolution part as follows

Theorem 4.2. Assume that $({\rm{HJB-SD}})_\epsilon$ is a sequence of stratified problems associated to sequences of regular stratifications $(\mathbb{M}_\epsilon)_\epsilon$ and of Hamiltonians $(H_\epsilon,H^k_\epsilon)_\epsilon$. If $\mathbb{M}_\epsilon \mathop {RS}\limits_ \longrightarrow \mathbb{M}$, then the following holds

$(i)$ if for all $\epsilon >0$, $v_\epsilon$ is a lower semi-continuous supersolution of $({\rm{HJB-SD}})_\epsilon$, then $ \underline{v} = \liminf_* v_\epsilon $ is a lower semi-continuous supersolution of (HJB-SD), the HJB problem associated with $H = {\rm{lim}}\;{\sup ^ * } \,H_\epsilon$.

$(ii)$ If for $\epsilon > 0$, $u_\epsilon$ is a stratified upper semi-continuous subsolution of $({\rm{HJB-SD}})_\epsilon$ and if the Hamiltonians $(H^k_\epsilon)_{k = 0..N}$ satisfy (NC) and (TC) with uniform constants, then $\bar{u} = \limsup^*u_\epsilon$ is a stratified upper semi-continuous subsolution of (HJB-SD) with $H^k = {\rm{lim}}\;{\inf _ * } \;H^k_\epsilon$ for any $k = 0..N$.

Proof. The proof is identical to the one in [1], the only new argument we need to add concerns the fact that the Ishii condition is stable as $\epsilon\to0$: if $u_\epsilon$ is the subsolution satisfying $(H_\epsilon)_*(x,u_\epsilon,Du_\epsilon)\leq0$ in $\mathbb{R}^N$, then $u: = \limsup^* u_\epsilon$ also satisfies the limit Ishii condition $H_*(x,u,Du)\leq0$ in $\mathbb{R}^N$.

Then, Corollary 1 in [1] immediately follows in the class of stratified solutions.