OPINION: The business confidence surveys are being talked about a lot at the moment. But what do these surveys actually mean for the New Zealand economy?
- Business confidence slumps to a seven-year low - report
- New Zealand economy isn't stalling - Prime Minister Jacinda Ardern
Do they even have a relationship with economic wellbeing? Do any of these types of measures really matter at all? And if they don't matter then what measures should when we are talking about economic wellbeing?
Let's deal with the meaning of the business confidence surveys first. We got curious about these surveys, and especially the ANZ one. We wanted to know who it included, who it excluded, and what questions they were asked. We dug around a bit and found what we can only describe as a very confusing document about the method. Sharon Zollner, ANZ's Chief Economist, admitted it made no sense when asked about it on Twitter.
There is now an updated paper describing the survey that is much clearer.
That is, it is now clearer that it is not a very good survey. As one statistics lecturer put it: "My Year Three survey design students would fail if they handed in that design."
There are two main problems. The first is the likelihood that the survey is biased and unrepresentative. The second is that its usefulness is being over-emphasised.
The survey is a self-selected opt-in survey. ANZ describes approaching 1500-2000 businesses, of which 300-400 take part each month. To start with, ANZ do not describe how they choose those 1500-2000 businesses. Their followers on Twitter? A random selection from their customer base? Or those people who fill in the form at the end of the document to say they want to take part? That is the first place bias may have been introduced.
The next layer of selection bias is even more obvious. Even if the people contacted were randomly sampled, those responding are self-selected. ANZ itself admits in its updated paper that there is a high probability that only those people motivated to respond will opt-in to the survey. This likely explains the political bias in results. That is not unusual in opt-in surveys - like Facebook polls for example. We would never use a Facebook poll to predict anything.
It's hard to save a survey from the selection bias we have described, but we can use some techniques to help. One of these is called weighting, though, from what we can tell, ANZ has not used weighting in the survey.
Weighting is something statisticians can do to help make a survey more representative of a wider population - which is important if that is how people use it. And that is how people do use this survey. People in media and politics and the business sector use the survey tell us how "business owners in New Zealand feel about the economy". What they survey really tells us is how John, Jane and Manu who follow ANZ on Twitter feel about the economy.
The second problem is that a self-selected measure of feelings (and there are about four in the survey) is not useful in this context. That is, it does not have real meaning for economic wellbeing.
Data analyst David Hood took the ANZ data and has been doing some interesting analysis on it. He found that the less leisure time a CEO has in a month, the lower their business confidence. And that both this 'weekends off' effect and ANZ business confidence had a weak relationship with future GDP growth. In other words neither measure was a strong predictor of economic wellbeing.
The ANZ data is not the only business confidence measure. The OECD uses a survey carried out by NZIER. They provided even less public information on their methods. Another analyst, Peter Ellis, found that food prices, last quarters' growth and possibly even visitor arrivals were more useful predictors of economic growth than NZIER business confidence.
When food prices are lower, we do better economically. Food prices are an objective measure. They are also something that is actually meaningful to most New Zealanders. Food prices are a real experience of the health of that thing we have constructed and called "the economy".
And this is an important issue. Do the measures that we use to assess national economic wellbeing, like business confidence or GDP, mean anything to people's lives? We would suggest in the main they don't. Yet New Zealanders are talking about economic wellbeing all the time. We talk about wages, and sick leave. They talk about being able to afford rent and food, and finding a job that is stable and secure. We discuss whether the environment will be in good shape in 20 years' time. And why work caring for a child is not counted and seen as proper work despite it being an economic activity.
We can keep exploring whether business confidence means anything to economic wellbeing. What we should be exploring is what other measures of economic activity and wellbeing mean something to all New Zealanders' lives. And how can we accurately measure and report on those?
Asking people in business across New Zealand how they feel is an important metric. And if we are going to do that we should do it properly, using a strong process to do so.
We should not rely on that measure to predict how our economy is serving the majority of the people in the country.
Dr Jess Berentson-Shaw is Co-Director of The Workshop, a not-for profit research and policy collaborative.
Tze Ming Mok is an associate of The Workshop.
Newshub Nation.
Watch Newshub Nation 9:30am Saturday/10am Sunday on TV3, and follow us on Facebook and Twitter.
