The USS analysis of reliance is seriously flawed and biased against the scheme
Evidence to the Joint Expert Panel
Dennis Leech, Emeritus Professor of Economics, University of Warwick, 15 August 2018
The assessment of reliance on the employers’ covenant is fundamental to the valuation and funding of the scheme. If the likelihood of the employers being asked to make additional payments above the maximum they can afford is too high, then that spells too much reliance and there will have to be benefit cuts. The details of their approach to this calculation are set out in various USS documents, including the Draft 2017 Actuarial Valuation, 1 September 2017, and Proposed approach to the methodology for the 2017 actuarial valuation: response to the Valuation Discussion Forum, 22 November 2016, and formalized in their various Tests.
The reliance measure is the difference between the scheme’s assets and what is termed the ‘self sufficiency’ liability. Self sufficiency is the level of assets that would be required for a low-risk investment strategy with a low probability of ever needing further contributions to provide benefits. The self sufficiency liability is described by USS as a ‘safe haven’ because it could be covered by investing in government bonds that are completely safe. Self sufficiency liability is calculated using a ‘gilts plus’ discount rate (gilts plus 0.5 or 0.75%) regardless of the actual investment strategy of the scheme. Because current gilt rates are so very low due to government monetary policy, this gives an extremely large figure of £82.4 billion for the liabilities. The conclusion reached by the USS from this is that the reliance is near the maximum that the employers can support.
There are however, serious problems with this approach to reliance. The methodology is flawed in two important ways.
- Essentially what is at issue is a choice between two statistical hypotheses that have far reaching consequences that are very different: on the one hand, the scheme remains open indefinitely, and on the other, there is a high likelihood of it having to close at some stage in the not-too-distant future. An even-handed treatment that tested the two hypotheses fairly would use the method of scientific testing of statistical hypotheses. This has not been done and instead the USS approach has, in effect, been biased in favour of supporting the second hypothesis.
- The treatment of risk by the USS is not consistent. In particular no allowance has been made for the fact that risk is not an absolute factor like it would be in a gambling situation, but is conditioned by circumstances. It is different for each of the two hypothesis under consideration. Two types of risk need to be considered: that which depends on the particular hypothesis being tested and that which is independent of it. We can call these respectively endogenous and exogenous risk.
Mistaken testing methodology
The USS tests of the reliance of the scheme on the covenant amount to essentially testing two hypotheses against one another using statistical reasoning. Either the scheme remains open indefinitely, and continues as it has been until now, or it eventually changes fundamentally and makes the journey to self sufficiency or closure. However, in their analysis, the USS have not treated both hypotheses in the same way, as a truly scientific approach would warrant. The hypothesis that the scheme remains open without undue reliance has not been thoroughly investigated and has been rejected nonetheless.
The scientific approach to testing a statistical hypothesis proceeds by first deriving the probability distribution of the relevant test statistic on the assumption that the hypothesis is true. The decision about whether to accept or reject the hypothesis is made by partitioning the theoretically possible values of the test statistic into two sets, the acceptance region and the rejection region. The test statistic is then computed and a decision made depending on where its value falls.
The test statistic in this application is the liability and the decision rule used is to compare the liability with the assets plus the amount of reliance that the institutions can afford. The affordable maximum reliance has been fixed at around 7 percent of payroll over 40 years.
The key point that the USS has ignored is that the liability depends on the particular hypothesis being assumed true and tested. If the scheme is regarded as continuing indefinitely then it can invest in high volatility assets to gain the higher returns that such assets will bring. High volatility is not a source of risk in this case. And if the scheme is cash positive, as the USS is and is forecast to continue to be indefinitely (see the paper by Salt and Benstead) it need not match the risk of the assets held with the liabilities. As Salt and Benstead put it:
“Being an open scheme brings significant investment advantages, which can be exploited to the benefit of the employers and members. The investment time horizon is infinitely long. An open scheme pays its benefits from contribution and asset income without any need to sell investments. If the asset income is sufficient, fluctuations of their market value is relatively unimportant.”(p8)
It follows that the liability for an open scheme will be appropriately calculated using a – higher - discount rate reflecting the expected return on the assets held in the investment portfolio. That will give a low figure for the liability.
On the other hand, if the scheme is seen as being in danger of closure, the volatility of the asset prices assumes central importance, and becomes risk. Under this ‘weak covenant’ hypothesis there is a danger that the scheme will have to sell off assets to pay benefits at some time in the future when prices are low and will be unable to pay the benefits in full. Therefore on this hypothesis the scheme needs to invest in – low volatility - assets that match the liabilities in terms of risk and return. The appropriate discount rate is therefore the - low – bond rate. The USS ‘self sufficiency’ portfolio is calculated on this basis.
The proper assessment of reliance therefore requires two different calculations of the liability: one assuming a strong covenant, that uses a liability based on best estimate returns from an income-seeking investment portfolio (such as the current one); and one for a weak covenant, that uses the self-sufficiency gilts-plus approach, assuming a low risk portfolio. Both calculations should be done. If they both give a reliance figure that is probably above 7 percent of salary, then the result is clear that the employers are unable to continue to support the scheme.
It seems likely however that if the two approaches are applied correctly by the USS they will give different results. The assets on the valuation date of 31 March 2017 were £59.9 billion and the best-estimate liability figure was £51.7 billion, giving a surplus of £8.2 billion, and therefore no further reliance on the employers is indicated. On the other hand, the self-sufficiency-gilts-plus liability was £82.6 billion, a reliance of £22.6 billion, larger than the target reliance of £10 billion.
The conclusion is that the result of the tests depends on what is assumed. On the hypothesis that the scheme remains open the inference is that it does not place particularly great reliance on the employers, and therefore we should accept the hypothesis and keep the scheme open. On the other hand, on the hypothesis that the scheme does not remain open indefinitely, there is likely to be high reliance on the employers.
Keeping the scheme open indefinitely would seem to be a perfectly reasonable and prudent course of action that would require neither increases in contributions nor cuts in benefits.
Inconsistent idea of risk
When valuing pension schemes it is important to separate the risks they face into two types: those that exist independently of the valuation, exogenous risks; and those which are consequent on it, what might be called endogenous risks.
A scheme in deficit is threatened by an existential risk due to the deficit itself, whatever other risks there may be. The need for the employer sponsor to make additional payments into the scheme threatens the company’s solvency and weakens the covenant. Many schemes have closed as a result of the actuarial valuation showing a deficit requiring deficit repair contributions over and above the employer’s regular contributions.
A scheme in surplus does not face that risk but is still subject to exogenous risks from other sources. A company, or association of employers such as the USS, can become insolvent for reasons unconnected with its pension scheme. For example large numbers of students might start deciding that a university degree is not worth the cost and refuse to incur the level of debt required leading to universities becoming insolvent.
The assessment of the reliance on employers should separate out these two types of risk and treat them fundamentally differently in the analysis. The exogenous risks can be allowed for by adjusting discount rates for prudence according to the usual actuarial principles.
However the endogenous risks require recognition of the simultaneity between the liability valuation and the covenant. If the covenant is strong the scheme can remain open and invest in high return assets and ignore the short term volatility in their prices. Therefore the risk of not being able to pay the benefits is low. But if the covenant is regarded as weak there is a non-negligible risk of not being able to pay the benefits.
The reliance calculation that has been done by the USS is deeply flawed because it ignores this endogeneity of the liability with respect to the strength of the covenant. It uses a liability value calculation based on a low-risk investment portfolio, with a low – gilts-plus - discount rate, appropriate to a situation of weak covenant. This leads to a greatly exaggerated idea of the scale of risk and therefore biases the analysis against finding that employers can afford the scheme - provided it remains open indefinitely. The USS is in effect assuming what it is setting out to test: by using the ‘safe haven’ valuation for the self-sufficiency liability it is assuming the existence of risk that is a consequence of an assumption of a weak covenant, and then claiming it shows there is too much reliance. It is circular reasoning.
The calculation by the USS is incoherent because it fails to recognize that the liability depends on the assumed strength of the covenant. The self-sufficiency-as-safe-haven definition is appropriate for an assumption of a weak covenant where prudence requires the lowest risk investment strategy. But if the covenant is assumed strong, so that short run market volatility does not pose a serious threat, the scheme can remain open to new joiners indefinitely and implement an investment strategy accordingly. It can invest in assets like equities that have high expected returns since their greater price volatility does not pose risk to its survival.
The analysis of reliance by the USS does not establish what is being claimed for it.
Salt, Hilary and Derek Benstead, Progressing the valuation of the USS, Report for UCU, First Actuarial, 8 August 2017
USS, Proposed approach to the methodology for the 2017 actuarial valuation: response to the Valuation Discussion Forum, 22 November 2016.
USS, Draft 2017 Actuarial Valuation, 1 September 2017.
 Strictly the two hypotheses are non-nested with respect to each other. Neither is a special case of the other. The result of the tests may be indeterminate in that both are accepted or both rejected.(See for example https://core.ac.uk/download/pdf/7092823.pdf)
 This figure includes no allowance for prudence as required. However it is hard to believe it would not still be very large after such allowance was made.