Challenge context

Every now and then, central bankers issue a speech that is supposed to convey their analysis of the world’s financial situation. These speeches are carefully followed on a global basis by all financial actors, and therefore have a strong influence over the evolution of financial markets, and to a larger extent, the overall economy. Indeed, central bank communications may impact various key economic factors, from interest rates to monetary policy, inflation expectations, credit, debt and overall financial leverage for private and public sectors alike.

In that these speeches may impact key macro-economic factors and move financial markets, the ability to correctly decipher and interpret “central bank lingo” has become a key area of focus for all sorts of financial analysts and economic actors. We invite the participants of this challenge to exploit embeddings of these speeches that are computed with powerful new tools recently made available by Google and others: natural language transformers. The application of this technique transforms a speech into a vector of 768 real numbers that should represent all key information conveyed in the speech.

Challenge goals

Your goal is to understand how financial markets react when central bankers deliver official speeches.

We do not provide the speeches themselves - otherwise the participants would quickly find out the date and the market moves ! - but we provide a transformed version. They were processed by a predefined BERT-style transformer, and this gives the input of the problem. The output is the mean price evolution of a collection of 39 different time series; these time series correspond to 13 different markets mesured at 3 different time scales.

We have computed the difference between closing prices of these 13 markets at 3 different maturities and the price of these markets at the closing time of the date of the speech. We are not interessed in very short time effects (between the beginning of the speech and the closing of the same day) and leaking effects (trading occuring because of information leakage before the beginning of the speech). A few tests have given us an indication that if a speech has an effect on the markets, it seems to intervene before the end of 2 weeks following the date of the speech: we have chosen 1 day lag, 1 week lag and 2 week lag to measure the possible effects on the markets.

As expected, at first sight it was difficult to distinguish an effect. We have therefore developped a technique to boost the response of the transformer using numerical NLP techniques. We deliver here the result of these boost. It is not miraculous, and the small number of points in the dataset is a real handicap.

The 13 markets are the following :

1. VIX: Index for the volatility of US stocks.
2. V2X: Index for the volatility of european stocks.
3. EURUSD: Change Rate EURO - US Dollars.
4. EUROUSDV1M: Volatility of at the money 1 month options on the EURUSD.
5. SPX: Index of the US Stocks.
6. SX5E: Index of Euro Stocks.
7. SRVIX: Swap Rate Volatility Index , it is an interest rate volatility index.
8. CVIX: Crypto Volatility Index, it is a crypto-currency volatility index.
9. MOVE: developed by Merrill Lynch, measures fear within the bond market.
10. USGG2YR: US Bonds 2 years.
11. USGG10YR: US Bonds 10 years.
12. GDBR2YR: German Bonds 2 years.
13. GDBR10YR: German Bonds 2 years.

Data description

The training data consists in 2000 transformed speeches and their subsequent market moves. More precisely:

• x_train.csv is a (2024, 768) array. Each line represents one speech issued at some date by some central banker (FED or ECB); for obvious reasons, they are anonymous. They were transformed into a feature vector of dimension 768 using a Bert-style transformer. These transformer are known to be extremely successfull for conveying the semantics of a natural language text.
• y_train.csv is a (2024, 39) array. Line i represents the market variations consecutive to the i-th speech whose transformed version is in the i-th line of the x_train file. These variations are relative.

The test data is as follows: 

• x_test.csv is a (415, 768) array. Each line represents one speech, at dates that are different than for the x_train.

The participant must submit a (415, 39) csv file for the predicted values of the market moves.

The metric is simply the L2 metric between the predicted market moves and the real market moves:

math \mathrm{loss}(y_{pred}, y) = \sqrt{\frac{1}{415*39}\sum_{i=1}^{415}\sum_{m = 1}^{39} |\hat{y}_{i,m} - y_{i,m}|^2}.

Warning: this rmse is different from the rmse defined by:

sklearn.mean_squared_error(a,b, squared=False)

which computes the average over the 39 dimensions of the output of the RMSE over the dataset.

Benchmark description

The benchmark is implemented using the xgboost tool available in the sklearn library. A good description of this grandient-boosted tree-based algorithm is available here.

For the maximum efficiency of the benchmark, we handle every dimension independantly and for every dimension the parameters that are used are:

1. n_estimators=100,
2. max_depth=5,
3. eta=0.1,
4. subsample=0.7,
5. colsample_bytree=0.8
6. scoring='neg_mean_squared_error'

With these parameters, we obtain an accuracy close to 20; on the public part of the test dataset, the accuracy is 18.39. Will you do better ?

Note: We see that the RMSE over the test set is very high, compared to the RMSE over the training set which is close to 10. This is expected because the number of samples of the training set is only of 2000 for a number of dimensions of the input being 768. So, the information brought by the training set is still too small to have a good predicting power.