the ecb survey of professional forecasters (spf)

70 Pages·2007·1.16 MB·English

OCCASIONAL PAPER SERIES
NO 59 / APRIL 2007
THE ECB SURVEY OF
PROFESSIONAL
FORECASTERS (SPF)
A REVIEW AFTER EIGHT
YEARS’ EXPERIENCE
ISSN 1607148-4
by Carlos Bowles, Roberta Friz,
Veronique Genre, Geoff Kenny,
9 771607 148006 Aidan Meyler and Tuomas Rautanen
OCCASIONAL PAPER SERIES
NO 59 / APRIL 2007
THE ECB SURVEY OF
PROFESSIONAL
FORECASTERS (SPF)
A REVIEW AFTER EIGHT
YEARS’ EXPERIENCE *
by Carlos Bowles, Roberta Friz,
Veronique Genre, Geoff Kenny,
Aidan Meyler and Tuomas Rautanen
In 2007 all ECB This paper can be downloaded without charge from
publications
feature a motif http://www.ecb.int or from the Social Science Research Network
taken from the
€20 banknote. electronic library at http://ssrn.com/abstract_id=967604.
* This work was carried out while all the authors were working in the Euro Area Macroeconomic Developments Division of the
ECB. The views expressed in this paper are those of the authors and do not necessarily reflect those of the European Central
Bank (ECB). We would like to thank Juan Angel Garcia, Hans-Joachim Klöckers, Gerard Korteweg, Bettina Landau, Klaus Masuch,
participants at a Eurosystem Monetary Policy Committee seminar and an anonymous ECB referee for their useful comments.
Any errors are of course the sole responsibility of the authors. Corresponding author: Aidan Meyler ([email protected]).
Individual anonymised SPF data are available upon request, for further information please contact [email protected]
© European Central Bank, 2007
Address
Kaiserstrasse 29
60311 Frankfurt am Main
Germany
Postal address
Postfach 16 03 19
60066 Frankfurt am Main
Germany
Telephone
[phone]
Website
http://www.ecb.int
Fax
[phone]
Telex
411 144 ecb d
All rights reserved. Any reproduction,
publication or reprint in the form of a
different publication, whether printed or
produced electronically, in whole or in
part, is permitted only with the explicit
written authorisation of the ECB or the
author(s).
The views expressed in this paper do
not necessarily reflect those of the
European Central Bank.
ISSN 1607-1484 (print)
ISSN 1725-6534 (online)
CONTENTS
CONTENTS
ABSTRACT 4 5 CONCLUDING REMARKS AND FUTURE SPF
RESEARCH 47
EXECUTIVE SUMMARY 5
ANNEXES
1 INTRODUCTION 9 1 Is there bias in the SPF forecast
results? Evidence from a panel
2 SHORT- TO MORE MEDIUM-TERM data analysis 49
FORECASTING PERFORMANCE 13 2 A closer look at individual
2.1 Forecast performance statistics forecaster quality 52
for the aggregate SPF 13 3 The probability forecasts from
2.2 Testing for bias in the SPF 18 the ECB SPF: A preliminary
2.3 Comparing SPF and Consensus assessment 56
Economics forecasts 20
2.4 Heterogeneity and individual REFERENCES 61
forecast performance 21
EUROPEAN CENTRAL BANK
3 ASSESSING FORECAST UNCERTAINTY OCCASIONAL PAPER SERIES 65
USING THE SPF 28
3.1 Alternative dispersion-based
uncertainty indicators from
the SPF 29
3.2 Comparison of alternative
uncertainty indicators from
the SPF 32
3.3 Other characteristics of the
forecast probability distribution 34
3.4 Overall assessment of uncertainty
measures 36
4 SPF LONG-TERM EXPECTATIONS 37
4.1 Long-term inflation expectations:
an indicator of the ECB’s
credibility 38
4.2 Long-term real GDP growth
expectations: a view on trend
potential output growth 42
4.3 Long-term unemployment
expectations 45
ECB
Occasional Paper No 59 3
April 2007
ABSTRACT Consistent with this, respondents have generally
reported the balance of risks to their long-term
Eight years have passed since the European unemployment expectations to be to the upside
Central Bank (ECB) launched its Survey of citing concerns as to whether required structural
Professional Forecasters (SPF). The SPF asks a reforms in the labour market will be undertaken
panel of approximately 75 forecasters located as the main factor behind this risk assessment.
in the European Union (EU) for their short- to
longer-term expectations for macroeconomic
variables such as euro area inflation, growth
and unemployment. This paper provides an
initial assessment of the information content
of this survey. First, we consider shorter-term
(i.e., one- and two-year ahead rolling horizon)
forecasts. The analysis suggests that, over the
sample period, in common with other private
and institutional forecasters, the SPF
systematically under-forecast inflation but that
there is less evidence of such systematic errors
for GDP and unemployment forecasts. However,
these findings, which generally hold regardless
of whether one considers the aggregate SPF
panel or individual responses, should be
interpreted with caution given the relatively
short sample period available for the analysis.
Second, we consider SPF respondents’
assessment of forecast uncertainty using
information from their probability distributions.
The results suggest that, particularly at the
individual level, SPF respondents do not seem
to fully capture the overall level of
macroeconomic uncertainty. Moreover, even at
the aggregate level, a more sophisticated
evaluation of the SPF density forecasts using
the probability integral transform largely
confirms this assessment. Lastly, we consider
longer-term macroeconomic expectations from
the SPF, where, as expectations cannot yet be
assessed against so few actual realisations, we
provide a mainly qualitative assessment. With
regard to inflation, the study suggests that the
ECB has been successful at anchoring long-
term expectations at rates consistent with its
primary objective to ensure price stability over
the medium term. Long-term GDP expectations
– which should provide an indication of the
private sector’s assessment of potential growth
– have declined over the sample period and the
balance of risks reported by respondents has
generally been skewed to the downside.
ECB
4 Occasional Paper No 59
April 2007
EXECUTIVE
SUMMARY
EXECUTIVE SUMMARY forecasters form their inflation expectations,
the SPF also collects information on expectations
According to economic theory, expectations for GDP growth and the unemployment rate
play a crucial role in how the economy functions, over both long and short horizons, as well as
as well as in how, and the extent to which, probability distributions surrounding the
policy agents, including central banks, can forecast point estimates for all three
influence macroeconomic outcomes.1 At the macroeconomic variables.
most fundamental level, this reflects the idea
that economic agents, both firms and households, This paper constitutes an initial assessment of
have strong incentives to make their key the information content of the SPF by providing
economic decisions (e.g. consumption, a detailed analysis of the survey data collected
investment or price-setting) in a way that over the period between 1999 and 2006. Given
optimally takes into account the likely future that the available sample period is still relatively
developments of the economic variables short, the results of this first comprehensive
relevant to such decisions. In line with the assessment of SPF data must be treated with
importance of expectations in understanding some caution. Nonetheless, a number of
the functioning of the economy, the European important insights can be obtained based on the
Central Bank (ECB), in particular, has a clear experience over the last eight years.
interest in being able to assess the private
sector’s short to medium-term inflation SHORT TO MORE MEDIUM-TERM FORECASTING
expectations when conducting its regular broad- PERFORMANCE
based analysis and assessment of the risks to
price stability. Information about such A key result of the analysis presented in this
expectations can, for example, provide evidence paper is that SPF panel members, like most
on the extent to which shocks affecting the other forecasters, have tended to underpredict
inflation process are perceived by agents as euro area inflation for most of the period since
likely to persist or be more short-lived. In 1999. However, this result warrants careful
addition, it is clearly useful to collect interpretation. On the one hand, the clear
information on private sector forecasts for tendency for SPF participants to underestimate
inflation, as this can serve as a point of the inflation outcome over one and two-year
comparison with – though no replacement for horizons points to the need to exercise a high
– the Eurosystem’s own assessment of the degree of caution when using such expectations
inflation outlook. Finally, a key ECB concern is as a predictor of future inflation. On the other
also to obtain information that can help assess hand, our analysis shows that a large part of this
the extent to which private agents hold “systematic” error can be explained by the
expectations that are in line with the ECB’s sequence of asymmetric and largely
quantitative definition of its price stability unpredictable shocks that hit euro area inflation
objective. Such expectations can help in over the period; oil prices, weather-related food
assessing the credibility of the single currency’s shocks, unforeseen changes in administered
monetary policy, as perceived by economic prices and indirect taxes. Adjusting for the
experts located throughout the European Union effects of such shocks, there is far less evidence
(EU). To this end, the Survey of Professional of a systematic underprediction or bias in the
Forecasters (SPF) also seeks to gather SPF inflation expectations.
information about longer-term expectations for
the euro area Harmonised Index of Consumer We found much less of a one-sided pattern in
Prices (HICP), the price index on which the GDP growth forecast errors. In the early stages
ECB announced its performance would be
1 Carroll (2001) cites Keynes (1936) as among the first to
assessed. In order to provide the broader
recognise the importance of expectations within the economic
macroeconomic context in which professional system.
ECB
Occasional Paper No 59 5
April 2007
of the survey, respondents underestimated the reflect a high degree of commonality in the
strength of GDP growth. However, following information available (and not available) to
the slowdown in 2001, GDP growth was then panel members, thus leading them to “get it
consistently overpredicted by respondents; an wrong” not only in the aggregate, but also
error common to many other forecasters. The individually. A similar pattern is observed for
tendency towards overprediction is most clearly both GDP growth and the unemployment rate,
evident for the two-year ahead horizon. Finally, in the sense that the overall pattern of forecast
the pattern of errors in the unemployment rate errors is broadly and consistently shared among
forecasts is somewhat at odds with the errors the vast majority of panel members.
related to GDP. Respondents have tended to
expect a higher unemployment rate than the Another cross-sectional feature of the SPF
actual outcome, but if they were to have forecast panel that we examined was whether there was
the real side of the economy according to any perceptible difference between the
Okun’s Law, one would have expected an error performance of panel members from the
in the opposite direction. However, once financial sector and the performance of those
account is taken of statistical and methodological from the non-financial sector. In general, for all
revisions in the unemployment rate, the three macroeconomic variables and all horizons,
inconsistency between the unemployment rate the evidence does not point to any significant
and GDP growth expectations is substantially differences between these two sectors.
reduced. Additionally, the effects of labour Additionally, there is no evidence that economic
market reforms should be taken into account. conditions in the country of the individual panel
members have had any significant explanatory
More formal regression tests also provide power for their euro area expectations. As a
additional evidence of bias in the aggregate final component to our analysis of short to more
SPF results. This evidence is strongest for medium-term forecasting performance, we
inflation expectations and weaker for investigated whether there are forecasters who
unemployment and GDP growth expectations. are systematically better or worse than others.
Importantly, some of these findings may be The analysis suggests that a small number of
subject to a small sample bias. Using a longer forecasters consistently perform above average
sample period, in which the shocks driving for some variables and horizons (although not
inflation might be less asymmetric and one- systematically for all variables and all
sided, could reverse this result. Whilst these horizons).
results confirm the need to exercise caution
when considering the information content of FORECAST UNCERTAINTY
the SPF for future macroeconomic outcomes,
SPF forecasts generally appear to be slightly The SPF provides several dimensions for
superior to naïve and purely backward-looking measuring forecast uncertainty. One may rely
benchmarks. Lastly, in terms of bias and overall on information provided by individual point
predictive accuracy, the SPF performance over estimates only and consider disagreement
the period analysed is broadly similar to the among the forecasters’ views as an indication
results of other similar surveys, such as of uncertainty, or one may also make use of the
Consensus Economics. probability distributions that each panel member
assigns to his/her forecast, which is one of the
Our analysis of the individual SPF replies SPF’s most original features. Disagreement
suggests that the broad pattern of the individual among panel members, simply measured by the
forecasts is essentially the same as that of the standard deviation of individual point estimates,
aggregate SPF results. For example, at least does not appear to be a good proxy for overall
90% of the panel consistently forecast inflation macroeconomic uncertainty. Across all
to be below the actual outcome. This may macroeconomic variables and horizons, the
ECB
6 Occasional Paper No 59
April 2007
EXECUTIVE
SUMMARY
spread of the cross-sectional distribution is failed to accurately assess the risks of lower
rarely sufficiently wide to encompass the actual growth and higher unemployment associated
outcome. This suggests some limitations to the with the economic downturn. These conclusions
usefulness of measures of macroeconomic suggest a need for caution when using the SPF
uncertainty that are based on the cross-sectional distributions as an indicator of the overall
distribution of point estimates alone, i.e. a high uncertainty about the macroeconomic outlook.
degree of consensus is not necessarily an However, it would seem that more definitive
indication of a low level of forecast uncertainty. and robust conclusions must await the
For example, over the period 2001-04, the accumulation of further data.
short-term uncertainty about GDP growth,
measured as the sum of the average variance of LONG-TERM EXPECTATIONS AND UNCERTAINTY
individual distributions and the variance of
point estimates, tended to rise and remain high, Our analysis of the longer-term expectations in
while the level of disagreement among panel the SPF dataset is more descriptive, since such
members actually declined. expectations cannot be assessed against actual
realisations. There are, nonetheless, a number
More generally, since the start of the SPF, the of findings worth noting. First, a key feature
pattern of short-term forecast uncertainty, as that emerges is the clear anchoring of long-term
perceived by SPF respondents, has been broadly inflation expectations at a level in line with
stable for both inflation and the unemployment the definition of price stability adopted by the
rate. For GDP growth, however, the SPF has ECB Governing Council. Throughout each of
indicated a higher degree of variability, as the eight years analysed, average long-term
reflected in a considerable increase in perceived expectations have remained “below, but close
short-term uncertainty over the period 2002-03. to, 2%”, and therefore fully in line with the
For both inflation and the unemployment rate, ECB’s quantitative definition of its price
risks over the one-year horizon have generally stability objective. In particular, average long-
been assessed by SPF respondents to be skewed, term expectations have exhibited considerable
on average, to the upside over the period resilience notwithstanding the persistently
1999-2006. This contrasts with GDP growth higher-than-expected inflation outcomes over
for which downside risks have tended to shorter horizons. Rather than revising upward
predominate. their long-term inflation expectations in
response to the numerous short-run inflation
There are also some indications that panel shocks, respondents have instead reflected such
members may not fully internalise the overall developments in the balance of risks. In line
level of macroeconomic uncertainty. For with these findings, while there has been a
example, compared with the level of uncertainty significant decline in disagreement about long-
indicated by the historical volatility of the term inflation forecasts among SPF participants,
actual inflation rate, the perceptions of uncertainty surrounding each individual
individual panel members indicate a lower level forecast has tended to increase slightly over
of uncertainty. In particular, individual panel time.
members tend to give a relatively low weight to
outcomes further away from the average. While At the level of individual panel members, a
this possible underestimation of overall number of other important features also emerge
inflation uncertainty is much less severe at the from our analysis. First, those individual panel
aggregate level, our evaluation of the aggregate members with expectations towards the lower
SPF density forecasts suggests some evidence end of the cross-sectional distribution (around
that respondents failed to accurately assess the 1.5%) have gradually revised upward their
risk of higher inflation outcomes over this long-term inflation expectations but nevertheless
period. In addition, SPF respondents may have they are still in line with price stability (e.g.
ECB
Occasional Paper No 59 7
April 2007
around 1.8/1.9%). This process has resulted in
a gradual “narrowing” of the cross-sectional
distribution of point estimates, implying higher
consensus that outcomes will be below, but
close to, 2%. However, another feature of the
cross-sectional distribution of long-term
inflation expectations was its “bi-modal”
nature, giving rise to a relatively high proportion
of respondents reporting long-term expectations
of 1.8% and 2.0%. Only a small proportion
quoted longer-term inflation at 1.9%, although
this has changed in 2006. The relatively high
proportion of respondents reporting exactly
2.0% or slightly above cautions against any
complacency on the part of the ECB in taking
its credibility for granted.
Finally, over the longer horizons, our analysis
of real variables also reveals some interesting
findings. In particular, long-term GDP
expectations – which should provide an
indication of the private sector’s assessment of
potential growth – have declined over the
sample period. Moreover, the overall level of
uncertainty surrounding long-term expectations
for GDP growth has been assessed by SPF
respondents to have been quite stable over the
period 1999-2006, although the balance of risk
has generally been skewed to the downside.
The uncertainty surrounding the longer-term
outlook for the unemployment rate is generally
assessed to be considerably higher than for
either inflation or GDP growth. In addition, the
risks to the longer-term unemployment rate
expectations have been persistently assessed by
SPF participants to be on the upside.
Respondents have linked this to concerns as to
whether required structural reforms in the
labour market will be undertaken.
ECB
8 Occasional Paper No 59
April 2007
1 INTRODUCTION
1 INTRODUCTION of the risks to price stability.3 Information about
such expectations could, for example, provide
Around the time of its inception in June 1998, evidence on the extent to which shocks affecting
the ECB began preparations – in collaboration the inflation process are perceived by agents as
with the national central banks (NCBs) in the likely to persist or to remain more short-lived.
EU – to establish a survey of the private sector’s Indeed, at the start of Stage III of European
expectations of future euro area macroeconomic Monetary Union (EMU), the whole topic of
developments, in particular inflation. And thus, building forecasts for the euro area
the first SPF took place in the first quarter of macroeconomy represented new ground, with
1999, immediately following the establishment much uncertainty existing about which
of the single currency. Since then, up to the underlying economic models to adopt and their
fourth quarter of 2006, 32 surveys have been associated parameters. In this context, it was
carried out on a quarterly basis, thereby deemed useful to collect information on private
contributing to the growth of a rich source of sector forecasts for inflation, as it could serve
information on euro area macroeconomic as a point of comparison with – though no
forecasts and expectations.2 For a more detailed replacement for – the Eurosystem’s own
overview of the ECB SPF and its main features, assessment of the inflation outlook.
see Garcia (2003), which provided a preliminary
assessment of the SPF. In this paper, based on a In addition, at the start of Stage III of EMU, a
longer time span of data, we attempt to provide key concern for the new central bank was to
a more comprehensive review of the performance obtain information that could help in the
of the SPF to date. assessment of the extent to which private agents
held expectations that were in line with the
HISTORY AND MOTIVATION FOR THE SPF ECB’s quantitative definition of its price
stability objective. Information on such
A number of factors underpinned the ECB’s expectations would help in the assessment of
wish to initiate the SPF and collect information the credibility of the new single currency’s
on the macroeconomic expectations and monetary policy, as perceived by economic
forecasts of the private sector. In particular, experts located throughout the EU. In this
according to mainstream economic theory, respect, the SPF sought to gather information
expectations play a crucial role in influencing about medium to long-term expectations for the
how the economy functions, as well as how and euro area HICP, the price index on which the
to what extent policy agents, including central ECB announced that its performance would be
banks, can influence macroeconomic outcomes. assessed. At that time, no other survey measures
At the most fundamental level, this reflects the of long-term inflation expectations for the euro
idea that economic agents, both firms and area HICP were available.4 Lastly, in order to
households, have strong incentives to make
their key economic decisions (e.g. consumption,
2 In this paper, we use the terms “expectations” and “forecasts”
investment or price-setting decisions) in a way interchangeably.
that optimally takes into account the likely 3 Issing (2003) provides an overview of the two-pillar approach
used by the ECB, in the context of the ECB’s 2003 review of its
future developments of the economic variables
monetary policy strategy. See also ECB (2004a).
relevant to such decisions. 4 Since then a number of surveys, such as those conducted by
Consensus Economics and the Eurozone Barometer, have
published information on long-term inflation expectations in
In line with the importance of expectations in the euro area, defined in terms of the HICP. In the United States,
understanding the functioning of the economy, the Federal Reserve Bank of Philadelphia has conducted a
Survey of Professional Forecasters since 1990 – although this
the ECB, in particular, had a clear interest in
survey actually dates back to 1968 when it was originally known
being able to assess the private sector’s short to as the ASA/NBER Economic Outlook Survey and was conducted
by the American Statistical Association (ASA) and the National
medium-term inflation outlook when conducting
Bureau of Economic Research (NBER). See Croushore (1993)
its regular broad-based analysis and assessment for further details.
ECB
Occasional Paper No 59 9
April 2007