Videos marked with an (*) contain discussion of new content in ICRFS-Plus™ 12.
If for any reason you are unable to view the training or demonstration videos, please contact our support staff at firstname.lastname@example.org and we will arrange to send you a copy of the videos on CD-ROM. You will be able to run the videos from the CD.
The training videos should be used for hands on training. We suggest you run the videos on a separate computer using a data projector, and train as a group.
The only way you will learn all the new concepts and be able to exploit all the immense benefits is by using the system. Experiential learning is imperative.
It is important that you study the videos in sequential order as set out below.
- 15.1 Introduction to the Bootstrap
- 15.2 Overview of the Mack method and the PTF modelling framework
- 15.3 Bootstrap TG ABC BS
- 15.4 Bootstrap TG LR High BS
15. The Bootstrap: how it shows the Mack method doesn't work
"To kill an error is as good a service as, and sometimes even better than, the establishing of a new truth or fact!"
|- Charles Darwin|
Bootstrap samples of the Mack method provide another compelling reason, amongst the numerous others, that it does not work. That is, it gives grossly inaccurate assessment of the risks.
The Mack method is a regression formulation of volume weighted average link ratios, the latter also known as the chain ladder method.
The idea behind the bootstrap is an old one. It is a re-sampling technique popularized by Brad Efron (1979) in his celebrated Annals of Statistics paper. Efron drew our attention to its considerable promise and gave it its name.
The bootstrap technique is used to calculate standard errors of parameters, confidence intervals, distributions of forecasts and so on. Typically, it is used when the sample size is small so that distributional assumptions cannot be tested and asymptotic results are not applicable. It also has applications to large sample sizes where distributional and model assumptions can be tested but the mathematics for computing forecast distributions is intractable.
For a paper on the bootstrap and the Mack Method click here.
The bootstrap technique is not a model and it does not make a bad model good.
Bootstrap samples are generated subsequent to a model being fitted to the data. A bootstrap sample (pseudo-data) has the same features as the real data only if the model satisfies assumptions supported by the data.
Accordingly the bootstrap technique can be used to test whether the model is appropriate for data.
In these video chapters we compare bootstrap samples for the Mack method versus bootstrap samples based on the optimal PTF model. We find that bootstrap samples (pseudo data) based on the Mack method (and related methods) do not reflect features in the real data - you can easily distinguish between the real data and the bootstrap samples. However, you cannot distinguish between bootstrap samples based on the optimal PTF model and the real data!
If the bootstrap samples do not replicate the features in the real data then the model is bad.
We study two LOBs;
- Triangle Group (TG) "ABC BS"
- Triangle Group (TG) "LRHigh BS"
Both datasets are real with changing calendar year trends. Moreover, the incremental paid losses in the "LRHigh BS" TG are heteroscedastic versus development period. That is, percentage variability varies by development period. This is another feature that the Mack method cannot capture, as shown by the Mack bootstrap samples.
In each case it is shown that the Mack method does not capture calendar year trends and the corresponding bootstrap samples bear no resemblance to the real data. This is not the case with the optimal PTF model.
15.1 Introduction to the Bootstrap
This video provides an introduction to the bootstrapping re-sampling technique using a PowerPoint presentation. It is emphasized that (i) standardized residuals residuals represent trends in the data minus trends estimated by the method; (ii) bootstrap samples based on a good model have the same salient features as the real data, and (iii) the bootstrap technique works if the weighted standardized residuals of a model come from the same distribution. If there is any structure in the residuals corresponding bootstrap samples do not resemble features in the real data. Accordingly, the bootstrap technique can be used to test the validity of the model for the (real) data.
15.2 Overview of the Mack method and the PTF modelling framework
The Mack method is a regression formulation of the link-ratio technique termed volume weighted averages. We use a real data set to explain the Mack method and how to calculate residuals. An extensive study of the Mack method and its relatives that all belong to the Extended Link Ratio Family (ELRF) modelling framework is given in video chapter 1.2 The Link Ratio Techniques (LRT) and the Extended Link Ratio Family (ELRF) modelling frameworks. Examples of Mack and other related methods fitted to real data is given in video chapter 2. Applications of the PTF and ELRF modelling frameworks.
An overview of the Probabilistic Trend Family (PTF) modelling framework is also given using a simulated data set. A more extensive study of the PTF modelling framework and its applications to real data is given in Chapter 3.1.
The Mack method does capture calendar year trends. Here also by way of a simulation we show that when we have data with a 10% calendar year trend the Mack method does capture the trend but there are no descriptors of it.
15.3 Bootstrap TG ABC BS
These data have major calendar year trend shifts that are quantified by the optimal PTF model.
We first create a bootstrap sample of the triangle values assuming they all come from the same distribution, that is, we randomly reshuffle the values into the different cells. This is done by setting all fitted parameters to zero. In this case bootstrapping the residuals is the same as bootstrapping the observations. Naturally the bootstrap triangle has very different structure to the real data. Most practitioners would argue that this is a silly thing to do. We agree! Furthermore, it is just as silly to bootstrap the residuals if the residuals of a model have any type of structure in them. That is, the scaled residuals are not random from the same distribution.
The Mack method applied to the corresponding cumulative array has residuals that exhibit calendar year trend changes (structure). That is, the residuals are not random from the same distribution. Bootstrap samples based on the Mack method are easily distinguishable from the real data, yet bootstrap samples based on the optimal PTF model are indistinguishable from the real data.
15.4 Bootstrap TG LR High BS
The residuals of the Mack method apply to these data exhibit a very strong negative trend. This means that the trends estimated by the (Mack) method are much higher than that in the data. Accordingly, the answers are biased upwards by about a factor of two. Bootstrap samples based on the Mack method are easily distinguishable from the real data, yet bootstrap samples based on the optimal PTF model are indistinguishable from the real data. The real incremental data has major calendar year trend shifts, and the quantity of process variation (on a log scale) varies by development period. Neither of these features are captured by the Mack method.