Needed: A data sensitisation blitz

What changed in the Household Consumption Expenditure Survey 2022-23 - and did it deliver?

The exit polls got their predictions of the results of the Lok Sabha 2024 elections wrong. One can only imagine the discomfort of the pollsters who were on live television while the results were being announced. Among the reasons proffered for them being off the mark is respondent behaviour, a challenge in all surveys. More importantly, they talked about the difficulties of predicting the winner in a closely contested constituency. I give full credit to the pollsters since they did not hide, were present on live television, and answered the uncomfortable questions.

Lessons for statistical system

There is a lesson here for the Indian statistical system, which has faced considerable flak. The statistical system needs to be at the front and centre of conversations. It should communicate directly with the heterogeneous group of stakeholders. Educate the media. Make it difficult for armchair criticism, based on an incomplete understanding of the issues, to dominate the news cycle.

Timing of data sensitisation blitz

There will never be a better time than now to start this conversation. The timing is important since data from the household consumption expenditure survey (HCES) 2022-23 has just been released. Hopefully, data from HCES 2023-24, which is slated to be completed by June 2024, will be released without delays.

Two points to be communicated

The first point to be communicated is the challenge of conducting surveys. Worldwide, the nonresponse rate to surveys is on the rise. In May 2018, The Economist ran a piece “Plunging response rates to household surveys worry policymakers”. An article published in the Journal of Economic Perspectives in 2015 was titled “Household Surveys in Crisis”. Both pieces focused on surveys in developed countries and the respective titles convey the gist of the message.

The second point is that individuals are less inclined to answer a long questionnaire, and even if they do, the quality of responses decline with the time taken for the survey. The issue of respondent fatigue and data quality was flagged in an article in Sarvekshana, the in-house journal of the Ministry of Statistics and Programme Implementation (MoSPI). In a recent article published in the Journal of Development Economics, Dahyeon Jeong and co-authors found that in a long survey of consumption expenditure, there is a reduction in item nonresponse by 10 to 64 per cent. However, they also found that “an extra hour of survey time lowers (reported) food expenditures by 25 per cent”.

What changed and why in HCES 2022-23

How these two challenges were addressed while planning for the HCES 2022-23 has to be part of the sensitisation. This will also provide a basis for justifying why a survey identical to the methodology used in HCES 2011-12 was not conducted.

To begin with, the National Statistical Commission (NSC) had recommended that a household survey should not last more than 45 minutes. Reducing the size of the questionnaire was not an option since detailed information is required for constructing the basket for consumer price index.  So a decision was made to visit a household three times. The HCES 2022-23 schedule was split into four components — HCQ (household characteristics), FDQ (food items), CSQ (consumables & services) and DGQ (durable items). A household was visited in each month of a quarter. In the first month, information was canvassed based on the HCQ and one of the other three components. In the next two months, the other two schedules were administered. In order to ensure there is no bias, all possible six sequences FDQ, CSQ and DGQ were randomised across different households.

What were the concerns?

There were concerns with this approach. First, was the issue of household casualty, i.e. whether a household that was visited the first time will respond to the questions in the second and third visit. Second, whether the respondent in the three visits will be the same person. Third, whether there are any differences across households depending on the sequence in which the modules —FDQ, CSQ and DGQ — were administered. Fourth, with the staggered visits to households, will the item nonresponse rate decline?

Did the changes deliver on expected lines?

I am confident that a statistical exercise must have been conducted on these issues. These should have been addressed headlong by MoSPI and the findings can be released in the public domain as a short note. Another point that needs to be clarified is that, whether it is sanitation practices or the utilisation of government programmes, survey estimates will not match with that of administrative records.

Sampling from affluent blocks and households

In a significant departure, an attempt was made to include affluent rural and urban households by sampling those who possessed land and owned a four-wheeler, respectively. Also, households from villages within 5 kilometres of a city were sampled. Did it work? The ratio of average consumption of the top 5 per cent to the bottom 5 per cent is 7.6 in rural India and 10.4 in urban India, which seems plausible. But the ratio of urban top 5 per cent to rural top 5 per cent is barely 2, which appears low. Using unit level data to compare consumption patterns of households in affluent urban blocks may throw more light.

Divergence between survey & national accounts estimates is the norm, not the exception

Next, the elephant in the room needs to be acknowledged. Whether the estimates of average consumption from a survey match the estimates from national accounts is a matter not specific to India. In a paper published in the journal Review of Income and Wealth, Espen Beer Prydz and co-authors compiled a dataset of 2,095 household survey means from 166 countries, which they then matched with the means from national accounts aggregates. They found that across countries, the estimate of average per consumption from household surveys is 20 per cent lower than that in national accounts. The gross domestic product per capita was higher by 50 per cent. In the Indian context, a comparison between 2011-12 and 2022-23 does indicate that the gap between the survey estimate of consumption and that from national accounts has narrowed, but not sufficiently. 

Everyone knew what was happening but no one pieced it together

The changes effected in the survey design do find mention in action taken reports of the NSC and the annual report of MoSPI. Nothing was confidential. The changes were reported in the media. But the information was scattered. Now, there is an opportunity to kick-start a larger conversation on survey design in India.

The writer, a professor at the Indira Gandhi Institute of Development Research, Mumbai, was a member of the Working Group of HCES 2022-23. The views expressed are personal and do not reflect those of the Working Group