GI ROC reserving study - Effectiveness of Reserving Methods Working Party

In 2008 the General Insurance Reserve Oversight Committee (GI ROC, UK) initiated a research project on the effectiveness of reserving methods.

Click here for the Institute of Actuaries UK Working Party page which talks about the research project and mentions ICRFS-Plus™ as one of the "reserving methods" to be tested.

As part of this actuaries were invited to respond to a practical survey in which they would analyse and forecast one of eight datasets (A-H) drawn from real company data using any combination of a number of pre-given standard methods. The procedure was set up in Excel spreadsheets which were specially designed so that the data was presented in stages corresponding to annual reports. In the initial stage there was historic data for six years and only after fully formulating a reserving procedure and forecasts on the basis of this level of historic experience was the participating actuary given the following year's data. This continued for between three and five stages depending on the dataset. The data in each case consisted of Paid Losses(PL), Incurred Losses (IL) and Outstandings (OS) a.k.a. Case Reserve Estimates or CREs. There was usually one or more exposure vectors and in some cases considerable additional information relating to claim counts and costs at closure. In all cases a brief verbal description of the line of business was also given.

The exercise was thus designed to approximate the experience of an actuary working with limited data and having to refine their understanding in stages as they gained a deeper historic overview of claims developments.

Insureware was very happy to be invited to participate in this exercise, but since our modelling approach does not work via traditional methods or development factors we could not present our results in the interactive Excel spreadsheets that were originally used.

Accordingly we created heuristic reports on two of the datasets, A and C, and have structured these reports to follow what we believe to be the spirit of the exercise. We show how we would typically respond at each stage of the process while in possession of only the data available at that time.

Dataset A eventually runs to five years of data but only provides PL, IL and OS and one Exposure measure. It is for for a line of business Aeroplane Parts Liability, about which few actuaries would be expected to possess a great deal of experience.

Dataset C has only three years ofdata, but provides many more kinds of data including Numbers of Claims Closed, Number of Claims Settled at Nil, and Payments for Claims Finalised. It is for a line of business, Employers Liability, with which many actuaries may be already quite experienced.

Summaries of the data for sets A and C.

The opening paragraphs of the Insureware report are reproduced below, click on the link to download the complete PDF.

The Insureware report was co-authored by David Odell (Insureware), Ben Zehnwirth (Professorial visiting Fellow University of New South Wales and Insureware), Henk Jan Prins (Chief Actuary Non Life ASR Nederlands), Paul Kedney (Chief Actuary, AEGIS London)

Introduction and Summary

We provide results on two dataset sequences A and C to illustrate the way we consider loss reserving issues. In our view sound loss reserving implies that the actuary does not select a method (model) a priori, but analyses the data and then creates models that work for the data. A good model is characterised by four essential and integral components of interest, namely, each of the trends in the development period, accident period and calendar period direction as well as the quality of the volatility about the trend structure. Calendar year trends project in the other two directions. The triangle is regarded as a sample path from the fitted probability distributions in each cell. All model assumptions are tested and validated by the data.

The analysis of calendar year trends, superimposed or total, in the historic claims performance is crucial to properly carry out the forecasting process. This is because assumed level of future inflation usually has a major impact on the forecast ultimate claim costs. One of the main drawbacks of the classical link ratio techniques is the fact that they do not involve analysis and modelling of inflationary trends. In practise the claims performance of most Non Life portfolios is characterised by fluctuating inflationary trends. The inability of the ratio techniques to cope with this phenomenon often results in incorrect and misleading results.

In ICRFS-Plus the forecasting assumptions are explicit; they may be based on data types other than paid losses and must be argued for in terms of the salient features in the past experience. The identified model forecasts lognormal distributions for each cell and their correlations. To determine the distribution of aggregates (of lognormals) by accident year, calendar year and total reserve we simulate from the correlated lognormals as there is no analytical distribution for aggregates of lognormals.

For our analysis we use the so-called Probabilistic Trend Family (PTF) modelling framework. PTF type analyses are compared to two models, often used in the actuarial world, which (among others) are part of the Extended Link Ratio Family (ELRF) modelling framework. The initial (default) models in the ELRF modelling framework formulate average link ratio methods as regressions (average trends) through the origin. These are extended to include an intercept (Murphy) and a constant trend in each development period across the accident periods. The only models we consider in the ELRF modelling framework is Mack, that is, the regression formulation of volume weighted average (chain ladder) link ratios, and the regression equivalent of arithmetic average link ratios.

Both modelling frameworks are incorporated in the ICRFS-Plus software and are described in the paper “Best Estimates for Reserves” (Barnett and Zehnwirth (2000)).

With traditional techniques the parameters (e.g. average link ratios) are functions of the data. By contrast, in the PTF modelling framework the model design as well as the parameters are a function of the data.

Set A

For this sequence of datasets we illustrate the PTF modelling framework on the incremental paid loss development array paying particular attention to the issues listed below:

We track the estimated mean ultimate for each accident year as we proceed through the successive datasets and observe a high degree of consistency in these figures. This phenomenon is a feature of the data, as the only change in structure of the model (and hence the data) on updating is the development trends in recent development periods. The calendar year and accident year trend structure do not change.

By contrast the Mack and arithmetic average link ratio (regression) methods applied to the paid and incurred (cumulative) triangles have no predictive power, give wildly differing means, and often give very different answers on updating. The CVs are inconsistent by accident year and do not reduce on updating the triangles-indeed they increase. These are features of the methods, not the data.

We also model the Case Reserve Estimates (CREs) in the PTF modelling framework and find that calendar year trend is negative until the last two years when it becomes positive. Yet, calendar year trend in the incremental paid triangle is always zero. The recent positive trend in the CREs is probably due to catch-up. Perhaps the company was on the selling block earlier on! The negative trend in CREs partially explains low reserves given by Mack applied to the incurred triangle in the first few datasets in the sequence.

Set C

In this example we illustrate a different kind of analysis. The LOB is Employers Liability and so we use broad assumptions about the probable shape of the development decay for this line to forecast at each stage out to an ultimate at twenty-five development years. The forecast assumptions then form the basis for year by year monitoring of the results including the Case Reserve Estimates (or Outstandings). Since there is a lot of information with this dataset relating to claim numbers and closures we consider the possibility of modelling in a frequency/severity framework.

This methodology seems to be behind some of the counterintuitive CREs. We find it of limited promise due in part to the higher volatilities and correlations of Closed Claim Counts and Average Severities. In particular we see a very nice illustration of Greg Taylor’s see saw hypothesis which says that average severity is inversely proportional to closure rate. The closed claim count triangle has unstable calendar year trends and therefore is unpredictable in terms of future trends, whereas the paid losses possess a stable calendar year trend.