Modeling path-dependent state transitions by a recurrent neural network

1.   Introduction

Rating transition models are widely used in the financial industry for credit risk evaluations, including stress testing and IFRS 9 expected credit loss evaluation ^[1,2,3,4], under the assumption that a rating transition is a time-homogeneous Markov process, depending only on the current rating and covariates. However, it is not uncommon that a Markov model deteriorates quickly after projecting for a few periods. This is because the Markov condition is generally not satisfied. A rating transition for a credit portfolio is generally path dependent. A test for this assumption is required ^[5,6] for the use of these Markov models.

There are various methods for path-dependent credit risk modeling ^[7], including regime-switching models ^[8] and the conditional methods ^[9,10]. The latter is comparative to a cohort analysis.

In this paper, we propose a recurrent neural network (RNN) model for modeling path-dependent rating transition. An RNN is a type of artificial neural network where connections between nodes form a directed graph along a temporal sequence. There are neurons for input and output at each time-period. The RNN is informed by past behaviours along the path. Information learned from previous periods propagates to future periods ^{[11,12,13,14,15,16,17]}.

The network structure for the proposed RNN model is described in Section 2 by using (2.1)–(2.4). This RNN model was implemented in Python. The experiments show that this RNN model is robust, compared to Markov transition models. Applications of this RNN model include the following, wherever path dependence is relevant:

(a) Path-dependent asset evaluation or credit risk evaluation

(b) Decisioning for account management

(c) Forecasting loss for stress testing, expected credit loss for IFRS 9 projects

(d) Estimating conditional probability of default for survival analysis

The paper is organized as follows. In Section 2, we setup the proposed RNN model. In Section 3, we calculate the partial derivatives for the network cost function. In Section 4, we present the experimental results for this proposed RNN model, as benchmarked with the time-homogeneous and time-inhomogeneous rating transition models.

2.   Recurrent neural network models for multiperiod state transition

In this section, we describe, the proposed RNN model, as given in (2.1)–(2.4), for modeling path-dependent rating transition. Given an observation horizon with $T$ periods:

our goal is to estimate at each $i\ge 0$ the probability of transitioning to a rating at time ${t}_{i+1}$ given the rating and covariates at time ${t}_{i}$.

Traditional rating transition models assume the Markov condition, i.e., the transition probability depends only on the current rating and covariates. The type of Markov transition models includes the following:

(a) Time-homogeneous Markov rating transition, as represented by one single transition model for all periods

(b) Time-inhomogeneous Markov rating transition, as represented by one transition model for each period

It is not uncommon that a time-homogeneous Markov model will deteriorate quickly after projecting only for a few periods. This is because the Markov condition is generally not satisfied. A rating transition for a credit portfolio is generally path dependent.

An RNN is an artificial neural network where connections between nodes form a directed graph along a temporal sequence. The chart below depicts the structure of an RNN. At the ${i}^{th}$ time period of the temporal sequence, the input neurons, the hidden neurons and the output neurons are respectively labeled ${x}^{\left(i\right)},$ ${h}^{\left(i\right)}$ and ${y}^{\left(i\right)}:$

An RNN shares the advantages of common neural networks; particularly, information learned at a point is propagated back and forward to all periods. It is path dependent.

Let ${\left\{{R}_{i}\right\}}_{i = 1}^{n}$ denote the $n$ ratings for a credit portfolio. For a loan portfolio, we reserve ${R}_{n-1}$ as the withdrawal rating and ${R}_{n}$ as the default ratings. Both the default and withdrawal ratings are assumed to be absorbed states, which means that a loan rated by a default or withdrawal rating will be excluded from the sample for future subsequent observations. Rating labels are observable at the beginning and the end of a period.

Let ${r}_{j}^{\left(i\right)}$ denote the indicator with a value of 1 if the rating for a loan at the end of the ${i}^{th}$ period is ${R}_{j}$ and 0 otherwise. Let ${n}_{p}$ denote the number of non-absorbed ratings. An input at the ${i}^{th}$ period is denoted as

where the first ($m-{n}_{p})$ input components ${(x}_{1}^{\left(i\right)}, {x}_{2}^{\left(i\right)}, \dots, {x}_{m-{n}_{p}}^{\left(i\right)}$) denote the covariates observable at the beginning of the ${i}^{th}$ period, and the remaining ${n}_{p}$ components are the rating indicators for non-absorbed ratings observed at the end of the ${(i-1)}^{th}$ period:

That is, the non-absorbed rating observed at the end of the ${(i-1)}^{th}$ period is used as the input for the next period. The output at the ${i}^{th}$ period is denoted as

where

The structure for this RNN is described as shown in (2.1)–(2.4) below. Initially, for the first period, we have

(a) Input: ${x}^{\left(1\right)} = ({x}_{1}^{\left(1\right)}, {x}_{2}^{\left(1\right)}, \dots, {x}_{m}^{\left(1\right)})$;

(b) Output: ${y}^{\left(1\right)}$ = ${(y}_{1}^{\left(1\right)}, {y}_{2}^{\left(1\right)}, \dots, {y}_{n}^{\left(1\right)})$, i.e., a unit vector where all components are zero except for one, which has a value of 1, and it is a random realization generated by the multinomial probability ${{p}_{1} = (p}_{11, }{p}_{12}, \dots, {p}_{1n}),$ where

and

Vector (${v}_{1}^{\left(1\right)}, {v}_{2}^{\left(1\right)}, \dots, {v}_{n}^{\left(1\right)})$ in (2.1) and (2.2) represents the information learned during the 1^st period, which is stored in hidden neurons ${h}^{\left(1\right)} = ({h}_{1}^{\left(1\right)}, {h}_{2}^{\left(1\right)}, \dots, {h}_{n}^{\left(1\right)})$ in the 1^st period.

In general, given the vector (${v}_{1}^{\left(i-1\right)}, {v}_{2}^{\left(i-1\right)}, \dots, {v}_{n}^{\left(i-1\right)})$ for the ${\left(i-1\right)}^{th}$ period ($i\ge 2)$, we have the following at the ${i}^{th}$ period:

(c) Input: ${x}^{\left(i\right)} = ({x}_{1}^{\left(i\right)}, {x}_{2}^{\left(i\right)}, \dots, {x}_{m}^{\left(i\right)})$;

(d) Output: ${y}^{\left(i\right)} = {(y}_{1}^{\left(i\right)}, {y}_{2}^{\left(i\right)}, \dots, {y}_{n}^{\left(i\right)}$), i.e., a unit vector, which is a random realization generated by multinomial probability ${{p}_{i} = (p}_{i1, }{p}_{i2}, \dots, {p}_{in}),$ where

and

As observed, ${v}_{j}^{\left(i\right)}$ consists of two parts, one from the current input, and the other from history, i.e., (${v}_{1}^{\left(i-1\right)}, {v}_{2}^{\left(i-1\right)}, \dots, {v}_{n}^{\left(i-1\right)})$, corresponding to the information learned up to the ${(i-1)}^{th}$ period. Similarly, the vector (${v}_{1}^{\left(i\right)}, {v}_{2}^{\left(i\right)}, \dots, {v}_{n}^{\left(i\right)})$ represents the information learned so far up to ${i}^{th}$ period, which is stored in hidden neurons ${h}^{\left(i\right)} = {(h}_{1}^{\left(i\right)}, {h}_{2}^{\left(i\right)}, \dots, {h}_{n}^{\left(i\right)})$.

This RNN model works, at the ${i}^{th}$ stage, in the way as described below:

1) Collects the input ${x}^{\left(i\right)},$ and information ${h}^{(i-1)}$ learned up to the end of $(i-1)$

2) Learns from ${x}^{\left(i\right)}$ and ${h}^{(i-1)}$ and stores the learned information in hidden neurons ${h}^{\left(i\right)}$

3) Derives the multinomial probability $({p}_{i1}, {p}_{i2}, \dots, {p}_{in})$, where ${p}_{ij}$ is the probability that the event will transition to ${j}^{th}$ rating.

4) Output: ${y}^{\left(i\right)}$ is a random multinomial realization, given the multinomial probability distribution $({p}_{i1}, {p}_{i2}, \dots, {p}_{in})$

Remark 2.1. The formulation of (2.4) does not come with a bias (i.e., intercept). An intercept can be inserted by adding a covariate with a constant value of 1, whenever necessary.

3.   Training the RNN rating transition model

Let ${y}^{\left(k\right)} = {(y}_{1}^{\left(k\right)}, {y}_{2}^{\left(k\right)}, \dots, {y}_{n}^{\left(k\right)})$ be the observed outcome at the ${k}^{th}$ period. The cost function at the ${k}^{th}$ period is denoted by ${L}_{k}$, which is given as (for one single data point):

This is the negative log-likelihood for observing the multinomial outcome ${(y}_{1}^{\left(k\right)}, {y}_{2}^{\left(k\right)}, \dots, {y}_{n}^{\left(k\right)})$. The total cost function to be minimized for this recurrent neural network is

summing over entire training sample.

3.1.   Partial derivatives of $L$ with respect to network weights

Training a neural network involves a series of gradient descent searches. Evaluation of partial derivatives is essential. In this sub-section, we calculate the partial derivatives for the network cost function with respect to network weights.

3.1.1.   Partial derivatives of ${L}_{k}$ with respect to ${v}_{i}^{(k-r)}$

Let ${d}_{i}^{(k, r)}$ denote the partial derivative of ${L}_{k}$ with respect to ${v}_{i}^{(k-r)}$, $0\le r\le k-1.$ By (2.1) and (2.3), we have the partial derivative $\frac{\partial {p}_{ki}}{\partial {v}_{j}^{\left(k\right)}}$ as:

Hence, by (3.1) and (3.3), we have the following for $1\le i\le n$:

where ${\sum }_{j = 1}^{n}{y}_{j}^{\left(k\right)} = 1$ is used.

Given ${d}_{j}^{(k, 0)}, 1\le j\le n,$ we can calculate ${d}_{i}^{(k, 1)}$ from top-down for $k > 1$ and $1\le i\le n$ by using (2.4):

Inductively, we have the following for $k > r$ and $0\le i\le n$:

3.1.2.   Partial derivatives of $L={\sum }_{k=1}^{T}{L}_{k}$ with respect to ${v}_{i}^{\left(r\right)}$

We will use the following fact:

Let ${D}_{i}^{T-r}$ denote the partial derivative of $L$ with respect to ${v}_{i}^{(T-r)}$. Given ${\{d}_{i}^{(k, 0)}| 1\le i\le n, 1\le k\le T\},$ we can calculate ${\{D}_{i}^{T-r}\left|1\le i\le n, 0\le r < T\right\}$ top-down. Initially, at the top period, we have the following by (3.6):

Next, backward from the top period, we have ${D}_{i}^{T-1}$ for $T > 1$ and $1\le i\le n$ as follows:

where (3.6) is used for 2^nd and ${5}^{th}$ equality signs. Inductively, we have ${D}_{i}^{T-r}$ for $T > r$ and $1\le i\le n$ from top-down as follows:

where (3.6) is used for 2^nd and ${4}^{th}$ equality signs.

3.1.3.   Partial derivatives of $L={\sum }_{k=1}^{T}{L}_{k}$ with respect to ${a}_{ij}^{\left(r\right)}$ and ${b}_{ij}^{\left(r\right)}$

Given ${\{D}_{i}^{r}\}$, i.e., the partial derivatives of the cost function $L$ with respect to ${v}_{i}^{\left(r\right)};$ we can now find the partial derivatives of $L$ with respect network weights ${a}_{ij}^{\left(r\right)}$ and ${b}_{ij}^{\left(r\right)}$ as defined in (2.2) and (2.4).

Let ${\delta }_{ij}^{r}$ and ${\sigma }_{ij}^{r}$ denote respectively the partial derivatives of $L$ with respect to ${a}_{ij}^{\left(r\right)}$ and ${b}_{ij}^{\left(r\right)}.$ By (2.4), at the top time period $r = T$, we have

If general, we have

3.2.   Initialization of network weights

A good initialization of the network weights ${a}_{ij}^{\left(r\right)}$ and ${b}_{ij}^{\left(r\right)}$ speeds up the convergence for the network training. In this sub-section, we propose an algorithm for initializing the network weights.

Let ${w}_{j}^{\left(r\right)}$ denote the vector of weights in (2.4) for ${v}_{j}^{\left(r\right)}$ at the ${r}^{th}$ time period, i.e.,

for $r > 1,$ and for $r = 1,$

Then the weight matrix for the network at the ${r}^{th}$ time-period is given by

Algorithm 3.1 (Initialization). Initialize network weights ${a}_{ij}^{\left(r\right)}$ and ${b}_{ij}^{\left(r\right)}$ as follows, step-by-step, starting from the first time period:

(a) Find ${w}_{j}^{\left(1\right)}, 1\le j\le n,$ by running a linear (or logistic if more sensitivity is required for some ${y}_{j}^{\left(1\right)}\text{'}s$) regression against the binary target ${y}_{j}^{\left(1\right)}$ with ${x}^{\left(1\right)}$ as the explanatory variable. Derive ${v}_{j}^{\left(1\right)}$ by (2.2).

(b) Given ${x}^{\left(2\right)} = {(x}_{1}^{\left(2\right)}, {x}_{2}^{\left(2\right)}, \dots,$ ${x}_{m}^{\left(2\right)})$ and ${v}^{\left(1\right)} = {(v}_{1}^{\left(1\right)}, {v}_{2}^{\left(1\right)}, \dots, {v}_{n}^{\left(1\right)}),$ find ${w}_{j}^{\left(2\right)}, 1\le j\le n$ by running a linear (or logistic if more sensitivity is required for some ${y}_{j}^{\left(2\right)}\text{'}s$) regression against ${y}_{j}^{\left(2\right)}$ with the components of ${x}^{\left(2\right)}$ and ${v}^{\left(1\right)}$ as explanatory variables. Derive ${v}_{j}^{\left(2\right)}$ by (2.4).

(c) Repeat (b) to obtain the initial weights for ${w}_{j}^{\left(r\right)}$ at the ${r}^{th}$ time period for $1\le j\le n$ and $1\le r\le T.$

3.3.   Training the recurrent neural network

Given initial weights, network training involves a series of gradient descent searches, as described in the next algorithm. Let ${w}^{\left(r\right)}$ be the weight matrix as in (3.11) for the network at the ${r}^{th}$ time period, i.e.,

Algorithm 3.2 (Network training). Update network weights ${w}^{\left(r\right)}, 1\le r\le T,$ step-by-step, as described below

(a) Forward scoring: Randomly select a small batch of examples (1–10 loan accounts, for example) from the time series of the training sample and calculate ${p}_{rj}$ by (2.3) using the current weights for $1\le r\le T$ and $1\le j\le n.$

(b) Select a time period $r,$ from $1$ to $T$ in sequence. At the ${r}^{th}$ time period, find the partial derivatives of $L$ with respect to ${a}_{ij}^{\left(r\right)}$ and ${b}_{ij}^{\left(r\right)}$ by (3.8) and (3.9); then, calculate $\mathrm{\Delta }{w}_{j}^{\left(r\right)}$ as follows

which is the average of the partial derivatives of $L$ over the batch; hence, obtain the following weight matrix:

(c) Select a learning rate $\eta$ from a grid of values such that the update of ${w}^{\left(r\right)}$ is

which gives rise to the biggest decrease for the cost function described by (3.2) over the entire training sample. Execute the update for ${w}^{\left(r\right)}$ by (3.12).

(d) Steps (a)–(c) are repeated until no material improvement is possible.

With the partial derivatives being evaluated over only one small batch of examples, these partial derivatives are called the mini-batch stochastic gradient for the cost function. This gradient can go off in a direction far from the batch gradient (i.e., the gradient over the entire training sample). Nevertheless, this noisiness is what we need for non-convex optimization ^[18,19] to escape from saddle points or local minima (Theorem 6 in ^[19]). The disadvantage is that more iterations are required to reach a good solution.

Remark 3.3. For Step (c) in Algorithm 3.2, there are better approaches for selecting a value for the learning rate $\eta$, rather than exhausting all possible values in the grid. For example, let ${\eta }_{i}$ be the ${i}^{th}$ value in the grid from 1 downward, and assume that currently ${\eta }_{i}$ is the best learning rate so far, and it leads to a decrease for the cost function; stop the search for the learning rate and use ${\eta }_{i}$ as the best learning rate, if ${\eta }_{i+1}$ does not lead to a bigger decrease for the cost function than ${\eta }_{i}$.

4.   Experimental results

In this section, we present the experimental results for the proposed RNN model, as benchmarked with two other Markov rating transition models.

The data we used constituted a synthetic sample, simulating a commercial loan portfolio with seven ratings ${\left\{{R}_{i}\right\}}_{i = 1}^{7}$ over seven quarters (periods). At the end of each quarter, accounts are rated by one of seven ratings, with ratings ${R}_{6}$ and ${R}_{7}$ being, respectively, the withdrawal and default ratings. Both the default and withdrawal ratings are absorbed ratings that were excluded from later quarters for observation. For simplicity, we included only three covariates, which simulated the following drivers for a loan:

(a) Debt service coverage ratio

(b) Debt to tangible net worth ratio

(c) Current ratio

The sample contained 10,000 accounts. It was split 50:50 into training and validation. We focued on the following three models:

1) Model 1—The proposed RNN rating transition model;

2) Model 2—Time-inhomogeneous Markov transition model, with one separate transition model for each period;

3) Model 3—Time-homogeneous Markov transition model, with one single transition model for all periods.

All three model use the same covariates.

Let ${y}_{j}$ denote a binary variable for a loan with a value of 1 if the loan has the rating ${R}_{j}$ at the quarter end, and 0 otherwise. Let ${(p}_{1, }{p}_{2}, \dots, {p}_{7})$ be the multinomial probabilities for a loan estimated by a rating transition model at the beginning of a quarter, with ${p}_{j}$ being the probability of transitioning to ${R}_{j}$ at the quarter end.

Tables 1 and 2 below show the Gini coefficients, over the training and validation samples respectively, for each of the above three models for ranking each of these seven ratings individually. For example, in Table 1, for the RNN transition model over the training sample, it had a Gini of 0.84 for the ranking rating ${R}_{1}$. This Gini was calculated by using ${p}_{1}$ to predict ${y}_{1}$ over the entire training sample. The results shown in these two tables demonstrate strong performance for the RNN transition model over the other two models.

In the remainder of this section, we focus on the robustness of a model in terms of predicting the default event, and the quality of using ${p}_{7}$ to predict ${y}_{7}$, the default indicator. Tables 3 and 4 below show the Gini coefficients period by period, over the training and validation samples respectively, for ranking the default indicator over each of seven periods. Again, the RNN transition model significantly outperformed the other two benchmark models across all periods.

The following six tables show the actual and predicted default rates for each model by decile over the training and validation samples. For example, Table 5 shows the actual and predicted default rates over the training sample for the RNN rating transition model. These values in the table were calculated by first sorting ${p}_{7}$ ascendingly, and then dividing the sample into 10 buckets, each of which was about 10%. The averages of the actual and predicted default rates over each bucket were taken.

These results demonstrate a significant improvement for the RNN model over the other two models, either on training or validation.

5.   Conclusions

A rating transition for a credit portfolio is generally path dependent. A Markov rating transition model, either homogeneous or inhomogeneous, usually does not perform well after projecting for a few periods. The RNN model proposed in this paper provides a solution for modeling state transitions under non-Markov settings. This RNN is informed by the information history along the path. The experiments show that this proposed RNN model significantly outperforms Markov models where path dependence is relevant.

Acknowledgements

The author thanks Biao Wu for many valuable discussions in the last 3 years in deep machine learning, Python financial engineering, as well as his insights and comments. Thanks also go to Felix Kan for many valuable comments, to Zunwei Du, Kaijie Cui, and Glenn Fei for many valuable conversations.

Conflict of interest

The views expressed in this article are not necessarily those of the banks the author works with. Please direct any comments to the author Bill Huajian Yang at: h_y02@yahoo.ca.