The results of the NSO survey 2017-18 are truly bizarre

Surjit S Bhalla
Some results of the National Statistical Office’s consumer expenditure survey (CES) for 2017-18 have been leaked. It is hoped that the official release (not endorsement for the reasons enunciated below) of the unit-level data will follow soon, so that researchers, analysts, politicians and even former prime ministers can evaluate for themselves how bad the NSO data really are.

The previous NSO survey on employment (PLFS) estimated the population in 2017-18 to be 1,074 million, when even mathematically challenged individuals estimated it to be upwards of 1,300 million (actually 1,339 million). That error of a 265 million under-estimation is a national record for the highest under-estimation of such a basic number — most NSO surveys have under-estimated population by around 5-10 per cent. The underestimation in 2017-18 at 20 per cent is a record.

This under-estimation has consequences for a major policy variable of interest — jobs and job growth. The unemployment rate is not affected by the estimate of aggregate population; but the number of jobs is affected. Most scholars have estimated employment generated by the PLFS data to decline by around 18 million in the number of jobs in 2017-18 relative to 2011-12. This is according to the usual status of employment, a measure which counts both half-time work (employed for 30-182 days) and full-time work (employed for more than 182 days) as full-time employment. In a detailed (forthcoming) paper on employment, Tirtha Das and I find that the desired full-time jobs (defined as principal status) increased by eight million between 2011-12 and 2017-18 — an increase not that different from what was obtained in the high growth years of 2004-5 to 2011-12 (a 14 million increase, but over seven years).

It is much easier to count people as employed or not employed, than to ask about their monthly per capita expenditure. This is where the world record is on the way to being established. The CES survey for 2017-18 shows that the per person real monthly expenditure (mpce in NSO parlance and not income as mistakenly assumed by some) declined from Rs 1,573 in 2011-12 to Rs 1,514 in 2017-18 (data converted from 2009-10 prices to 2011-12 prices to make it consistent with other data)

In my book, Imagine There’s No Country, I had documented how there was a declining trend in the amount captured by the surveys over time. Household surveys (S) were capturing less and less of consumption as revealed by an alternative calculation — the national accounts (NA). While the two definitions (survey and national accounts) are not identical, they are broadly comparable.

The average S/NA ratio, around the world, was in the mid 80s in the 1980s, that is, if the NA estimate of per-capita consumption was 100, then the household survey would estimate it to be 85. It is worth remembering that the S/NA ratio in India in the 1950’s and 1960’s was upwards of 95 per cent. Too high to be true? In a manner of speaking, yes. For then, the household survey provided the estimate of consumption for national accounts.

But, with time, economies became complicated, and the national accounts data moved with the times, became more sophisticated and captured the trends in the economy much better than the surveys. Survey organisations like the NSO refused to move. In 1983, the S/NA ratio in India collapsed to 63 per cent from the high 70s level just a decade earlier. It was to be 30 years later (in 2012) when the world reached the low 60’s average.

That year (2011-12), India recorded a 55 per cent ratio for S/NA. Just six years later (2017-18), the S/NA ratio in India has collapsed to just 33 per cent — the second lowest ever recorded around the world for economies without hyper-inflation (when S/NA ratio really gets distorted) and with populations above 10 million. The worst ever was Nigeria in 2009 with a S/NA ratio of 27.2 per cent.

There is yet another comparison one can make. The two most recent consumption surveys in India, just six years apart, yield a decline of 22 percentage points. This is the second worst sequential decline in the world. The worst was Pakistan in 2001 when the S/NA ratio was 46.9 per cent, down 26.9 percentage points from the 73.8 per cent estimate recorded in 1998.

The secular decline in NSO has now persisted for some 50 years and marks a sad occasion for an institution that was a trend setting statistical institution in not only the emerging economies, but in the world as well. In the early 1950s, the world famous statistician P C Mahalanobis was its head.

I was privileged to be a member of the first National Statistical Commission of India headed by an internationally renowned economist Suresh Tendulkar. I was sent to Calcutta by Tendulkar to interact with the NSSO and to find out why the Indian S/NA ratio had sharply declined and what could be done to improve survey response. I met with little success and came back frustrated with the ancient techniques being followed by them.

The most recent statistical commission chairman, P C Mohanan, was a colleague. He has been quoted as not being surprised with the decision of the government to not accept the findings of the latest record-low NSO survey. His view is that the government is suppressing reports that are not “favourable”.

If I thought that the NSO consumption surveys were misleading and not acceptable in 2002 and 2006, I can be forgiven for thinking that the surveys are even less acceptable today. The results of the NSO survey 2017-18 are truly bizarre — a decline in average real consumption of 0.6 percentage per year between 2011-12 and 2017-18, when the NA consumption estimate is of a positive 5.8 per cent annual growth. As discussed above, the NSO estimate for 2017-18 is so out of the box that it is actually out of any (reasonable) ball-park. If the government does not accept the findings of the survey (as has been suggested by a recent press release) then a genuine reform of the NSO can actually begin. Even if it does not return to its previous glory, a reformed NSO can become a respectable institution. That will not be easy, but it is a path worth embarking upon.

I have been surprised by how many respected analysts have pointed to the “findings” of the NSO 2017-18 and are relating it to the slowdown in the economy in 2019-20. Some of these very same “analysts” were cheering the RBI/MPC a year ago when it raised the repo rate to 6.5 per cent in June 2018, the very last month of the 2017-18 survey. Their reason for cheering the MPC — growth was too high, so high that it was leading to high and accelerating inflation. Both views cannot be right, and it is worse than disingenuous to hold both views simultaneously. The so-called experts have to make up their mind — if growth was disturbingly high in June 2018, then it cannot under any stretch of the imagination be argued that the CES 2017-18 survey is even close to being right.

Not every government report should be accepted. Just like individuals fail exams, and editors reject papers (and columns), sometimes, institutions fail to produce a credible report. But, I do believe that the unit-level data should be released. Let the world, and experts, find out for themselves how truly informative and credible the NSO CES data really are.

This article first appeared in the print edition on November 27, 2019 under the title ‘Rebuilding credibility’. The writer is executive director IMF, representing India, Sri Lanka, Bangladesh and Bhutan.