Published in


Approaches to risk: uncertainties in USS’s Test 1 and stochastic modelling

Number 76: #USSbriefs76

Andrew Chitty, University of Sussex
Tim Wilson, University of Dundee

Click for printable PDF

This is a USSbrief, published on 28 July 2019, that belongs to the OpenUPP (Open USS Pension Panel) series. It has been submitted to the UCU-UUK JEP (Joint Expert Panel) by the authors on 14 June 2019.

The first report of the Joint Expert Panel (JEP) was an excellent document, informative, balanced in presentation and diplomatic in language. We anticipate that the forthcoming report will be equally informative, but we believe the time for balance and diplomacy is passed. We hope the panel’s judgements and recommendations will be presented in language that leaves no possibility of misinterpretation or obfuscation. We are conscious of the magnitude of the task the panel has assumed and have endeavoured to restrict our submission to recent developments and aspects we believe deserve greater examination. We first consider specifics of the current methodology.

Test 1

1. Accompanying this submission is USSbrief68 (Why ‘Test 1’ must be dropped: a critique of its design and implementation). This was written for a less expert audience; however, the panel may find some value in reading sections 6–9. It is clear from the first JEP report that the panel shares our view that Test 1 should not be integral to the valuation. In our brief we go further and argue that Test 1 is unfit for purpose and should be dropped altogether. USS is using Test 1 in an unchanged manner in the 2018 valuation, although it is noteworthy that it is no longer mentioned. For this reason, we believe it is necessary that the panel repeat its criticisms of Test 1 and reject its continuing use in direct and unambiguous language.


2. The panel has previously said that Test 1 was poorly understood. Part of the reason for this is likely to be the lax manner in which the word ‘reliance’ has been used by USS. It has changed meaning since 2014 (see USSbrief68, p.7), and has been used to express the related but separate concepts of self-sufficiency deficit and the difference between the self-sufficiency value of liabilities and the technical provisions.

The concept of maximum acceptable reliance also appears to have drifted in meaning. Here is the definition of Test 1 in 2014:

The difference between the liabilities assessed on a self-sufficiency approach (for this purpose a discount rate of gilts plus 0.5% is used) and the actual technical provisions basis should generally not exceed what we refer to as the amount of contributions payable in extremis, which we will indicatively measure as the difference between (i) the maximum contribution of 18% of salaries stated by the employers as being desirable and (ii) the maximum identified as being affordable by employers […] of 25% of salaries, over a long period such as 15 to 20 years.

As constructed, the test was based on the assessed ability of employers to pay an additional 7% of salary in extremis. This was independently assessed as £13bn; however this was later lowered to £10bn, with the justification for the change being the employers’ desire to accept less risk.

On re-setting the maximum acceptable reliance at £10bn, the USS board confused the employers’ ability to pay with their willingness to pay. In our opinion, in making this change the board unreasonably favoured employers over the best interest of members. The decision increased the technical provisions at year 20 and so lowered the discount rate, which created a greater deficit and higher future contributions that helped justify the employers’ push for scheme closure.

3. Salary is assumed to grow at CPI+2% in the 2017 (and 2018) valuation. This is justified on the basis that historical data shows that salaries tend to increase with GDP over the long term (many decades), albeit with significant shorter-term deviations. This assumption is also used in calculating the self-sufficiency value of liabilities at year 20. In stark contrast, the maximal acceptable reliance at year 20, which is measured as a proportion of salaries, is inflated only by CPI in Test 1. Therefore, these two assumptions are internally inconsistent. First Actuarial has advocated matching the two assumptions, and USS recognised the inconsistency in its employer consultation of February 2017:

Allowing reliance to increase in line with increases in salaries would be consistent with other elements of the valuation and reflect the economics of the sector.

The panel has identified this disparity as a form of prudence; however, we believe the panel should go further in criticising these assumptions. Salary growth affects the level of self-sufficiency, and so inflating the self-sufficiency value of liabilities at a higher rate than the maximum acceptable reliance means that in implementing Test 1 the technical provisions figure will be higher than it would have been if the same assumption had been made for both calculations, leading to a lower discount rate from year 20 onwards and a greater required rate of de-risking from years 1 to 20.

The effect of mismatched assumptions is substantial. Figure 9 of the first JEP report shows that if the maximum acceptable reliance was inflated at CPI+2% in line with USS’s assumption of salary growth used in calculating the self-sufficiency value of liabilities at year 20, then it would have grown from £13bn to £19bn in 20 years. USS has stated that a £4bn increase in the maximum acceptable reliance would decrease the deficit by approximately £2bn and the future contribution rate by about 1.5%. So the 2017 valuation would have looked very different if more appropriate assumptions had been used. It appears that employers wanted the figure for maximum acceptable reliance to grow only by CPI and USS accepted this. Yet USS did not simultaneously revise its salary growth assumption for calculating the self-sufficiency value of liabilities at year 20. Again, choices were made that substantially impacted the valuation to the detriment of members.

We ask the panel to make recommendations about how the reliance on covenant should be assessed in the future.

Stochastic modelling

4. An example of the output from the stochastic modelling is given below (Figure 1), taken from a presentation by USS to Imperial College London in November 2017. The actual numbers are not that important; the most striking feature is the extraordinary spread of the distribution. With a 20-year horizon the spread would be even wider. With a bit of practice we could generate a similar level of precision with a blindfold and a set of darts! Yet despite this level of uncertainty USS consistently portrays the technical provisions deficit (which, via the Test 1 mechanism, is predicated on predictions of the self-sufficiency deficit at year 20) with absolute certainty. Even if many of the inputs to the valuation were not in dispute it is surely misleading to state the deficit as, for example, -£7.5bn when the figure is based on data of this precision. The JEP should recommend that valuations based on stochastic modelling should be accompanied by appropriate estimates of uncertainty.

Figure 1: From ‘Meeting with Imperial College on aspects of the 2017 Actuarial Valuation of USS’, p.8 of 14 of PDF document.

5. USS decided to use the 67th percentile for a prudent valuation in 2017, whereas it used the 65th percentile in 2014. The value selected as prudent is entirely arbitrary and the change may appear to be of little importance, but because of the spread of the distribution of expected investment returns the 2% change increased the technical provisions deficit by £1.3bn. No explanation was offered for the change. The question of what constitutes prudence is highly subjective and there should be no reason to change this from one valuation to the next.

6. The tail of the distribution in Figure 1 represents an extraordinary loss of value to the scheme and, unlike the short-term risks that USS obsess about in the 2018 consultation, would threaten the viability of the scheme. Are these scenarios genuine possibilities or artefacts of an imperfect model? If genuine, are they driven predominantly by a loss in the predicted value of assets or an increase in the predicted value of self-sufficiency liabilities? How do the different classes of assets contribute to the outcome of these scenarios? One possible explanation for dramatic increases in the self-sufficiency deficit could be that the exceptionally low prevailing interest rates means that investments in bonds are highly leveraged. Increases in gilt yields could see very significant losses in the value of that portion of the fund held in gilts. It would be ironic if the assets USS promotes as a safe harbour actually represent a severe risk.

We believe the panel should investigate the risks underlying the most negative scenarios of the stochastic modelling and make the information public.

Intergenerational fairness

7. Any consideration of intergenerational fairness should start by acknowledging two injustices:

  • an enormous decrease in pension benefits and a significant increase in contributions has been imposed on members this decade, and
  • the expansion of the precariat in the HE sector means many employees have limited access to decent pensions.

The second JEP report may to a small extent ameliorate the first and will do nothing to rectify the second.

8. Because of the imposition of cost-sharing, the 2017 schedule of contributions and all options for the 2018 valuation require members to pay deficit recovery contributions for the first time. This is most obviously and ludicrously unfair to a member who joins in 2019 but is just as unfair to a member who joins in March 2018. And is it fair that a member of long standing pays the same amount towards deficit recovery contributions as one who joined in, say, 2014? A fair apportionment would be extremely challenging. To avoid such obvious injustices the panel should recommend that all deficit recovery contributions are paid by employers.

A better valuation methodology

9. When Sam Marsh published his October 2018 calculations assuming that the current portfolio was retained, USS’s obsession with ‘de-risking’ was made clear. USS hadn’t done any calculation based on this assumption. Only by knowing the current status of the scheme can the worth of any change be assessed. The panel should recommend a valuation methodology that, like Marsh’s calculations, can predict how the scheme will perform in the long term if it is assumed that there is no change to benefits, contribution rates or investment portfolio, and can also predict the long-term implications of any proposed change.

10. Although we recognise that the 2004 Pensions Act and the 2005 Defined Benefit (DB) scheme regulations mandate an ‘accrued benefits’ funding method, in our view the JEP should consider other approaches to scheme funding based on cash flow forecasting. Indeed, we believe the panel should recommend to the Trustee an accrued benefits valuation methodology that satisfies the legal requirements and an alternative methodology based on cash flow, as a more complete picture based on multiple approaches will help to restore confidence in the scheme.


11. The great injustice of Test 1 is that it brings forward some of the costs of scheme closure, which are then passed on to members through reduced benefits and higher contributions. Why should this be so when any perceived or actual benefits accrue to employers? All of the problems of Test 1 were identified in 2014, yet five years later the costs to members are ratcheted ever higher. The JEP should give thought as to how restitution to members of unwarranted impositions might be implemented.

This is a USSbrief, published on 28 July 2019, that belongs to the OpenUPP (Open USS Pension Panel) series. It has been submitted to the UCU-UUK JEP (Joint Expert Panel) by the authors on 14 June 2019. This paper represents the views of the authors only. The authors believe all information to be reliable and accurate; if any errors are found please contact us so that we can correct them. We welcome discussion of the points raised and suggest that discussants use Twitter with the hashtags #USSbriefs76 and #OpenUPP2018; the authors will try to respond as appropriate. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store


A set of papers written by University Staff and Students, on University Staff and Students, for University Staff and Students.