State Space Model: Partial Derivatives, Kalman Filter

A state space model is one which models a system (i.e. one comprising input, output and state variables) by means of first-order differential equations.

In a state space model, parameters are actually changing rather than remaining fixed. This is where the model differs from the more standard methods of forecasting time series, and is more flexible in adapting to sudden shocks in a time series, e.g. a sudden economic downturn.

Derivatives

A derivative allows us to measure the rate of change in a variable at a specific point on a curve. Specifically, partial differentation is especially relevant to the area of state-space modelling because it allows us to analyse the effect of a rate of change in one variable while holding all other variables constant.

For instance, suppose that we wish to examine the effect of the rate of change of a variable f with respect to x and y. To be able to do this, we need to use partial derivatives. Essentially, we are differentiating with respect to x while holding y constant, and also differentiating with respect to y while holding x constant.

In the case of a hypothetical equation 2x2 – xy2, we differentiate as follows:

∂/∂x = 4x – y2

∂/∂y = 2x2 - 2xy

In this sense, the partial derivative is essentially a regression line across a certain point on the non-linear curve which allows us to calculate the rate of change at that particular point on the curve.

State Space Model: Prediction, Filtering, and Smoothing

When it comes to forecasting a time series, a state-space model (such as the Kalman Filter) will incorporate the following three elements into the process:

Prediction: Forecasting the future values of the state-space

Filtering: Making the best prediction of the current values of the state from the given observations

Smoothing: Making the best prediction of the past values of the state from the given observations

The Kalman Filter and how it works

The Kalman Filter is one of the main state-space models employed to analyse a dynamic time series, as per the process above. The model is commonly employed in economics to forecast variables such as inflation, GDP, and others.

Another example could be the battery life of a smartphone. Due to wear and tear, the lifecycle of a smartphone battery is not typically constant.

There are many variables that can affect the rate of change of battery life, such as temperature, memory usage, network connectivity, and so on. Given that such variables are rarely constant based on usage, it also makes sense that battery life will also not be constant.

The state-space model allows us to account for the fact that the rate of change in variables are not constant, and hence neither is the state.

The beauty of the Kalman Filter is that the model also takes into account variables that are not directly observable. For instance, let’s suppose that we have an inflation variable xt, and we wish to calculate future inflation values of xt+1..

xt and yt are vectors n x 1 and m x 1, and wt and yt are independent i.i.d. process. P and Z are random coefficients for the variable xt.

t+1 = (P − KtZ)x̂t + Ktyt, where

Kt = PΣtZ'(ZΣtZ' + R)-1,

Σt+1 = PΣtP' + Q − PΣtZ'

(ZΣtZ' + R)-1tP,

and Σt = E[(xt − x̂t)(xt − x̂t)'

We see that the Kalman Filter gives us the prediction for the variable in question (in this case, inflation), taking into account the independent i.i.d processes wt and vt which are normally distributed with mean zero, where Q is the variance of wt and R is the variance of vt, given by E [wtwt] = Q and E [vtvt] = R.

Example

One of the biggest applications of the Kalman Filter is in adjusting the forecast of a time series when taking into account an economic shock. For instance, a sudden plunge in the price index fundamentally changes the nature of the time series.

state space model

A Kalman Filter would be an ideal model in accounting for such a shift in the trend.

In this regard, the inherent advantage of state-space models is that they are more adaptive than regular time-series models, and are therefore more suitable for real-time implementation. Therefore, in a situation where a time series shows a sudden change from the mean trend, using a state-space model is likely ideal under these circumstances.

P.S. I will soon be publishing a follow-up article on use of the Kalman Filter in R with a more extended example. To get notified of the article, feel free to subscribe to my mailing list below.