How politics and economics have their own versions of the Shepherd Problem

Some problems are unsolvable even though they have data, because the information is inadequate

These days everyone is trying to solve two problems: which party will lead the next coalition and when will the economy recover. This reminds me of a problem in mathematics, which is actually not a problem at all.

It is called the Shepherd Problem. It goes as follows: “A shepherd is tending 93 goats and seven sheep. How old is the shepherd?” There are, of course, several variations of the problem.

For example: Yesterday 67 planes landed and 43 took off. At 6 pm there were noon there were 32 planes on the ground. How many planes were still there at midnight?

Serious mathematicians have refused to try and solve these problems, for a very good reason: they are unsolvable. This is because although the question contains data, it contains inadequate information for solving it

Children, on the other hand, when asked this question have tried very hard to solve it. They have used the numbers that the question contains.

Faced with the problem of inadequate information they have applied their intuition to test various permutations and combinations of 93 and 7. They have even tried square roots.

Importantly, they have rejected the impossible answers, like the shepherd being 120 0r 150 years old. But whenever the answer has seemed to within the probable age limit – 10 to 100 – they have said that is the correct answer. 

Sadly, they have not known that their method was complete nonsense mathematically. All that has mattered to them is that they get a probable answer, never mind how absurd the reasoning was. 

But a very small minority of students of less than one per cent has recognised that the problem is not solvable. The rest have not paused to wonder if their reasoning is correct.

Politics

This problem exemplifies Indian elections. Those who seek to analyse them, the so-called psephologists, exemplify the behaviour of the children.

Is it any wonder that they mostly get the wrong answers, or can only predict a broad range within which the prediction is accurate? Unlike the children, however, they get paid fat sums for being wrong.

The same thing happens in economics also. Economists have been trying for decades to make accurate predictions, not realising that (a) they don’t, and never will, have adequate information and (b) that their reasoning is wrong.

Consider politics first. An underlying assumption of psephology is that past voting trends are a reliable guide to the future. 

But this is not the case for a very good reason: the age of voters changes in any five year period and for a very large number of them, the first time voters, there simply isn’t enough information.

This would not matter if elections were not so closely fought. But very often they are -- and in enough constituencies to make a huge difference to the final outcome. 

The 2004 election was a clear indicator of this when the BJP lost over 25 seats by very small margins. The opposite happened to the Congress in 2009 when it won over 40 seats by very thin margins.

Economics

Exactly the same thing is true of economics also – there just isn’t enough information to solve a problem once and for all. This is why there is the big scramble for Big Data. 

But if anyone thinks this is going to help, they are mistaken. The data might be big but its composition is going to vary hugely across time and geographies. 

Even with Big Data, the information is inadequate.

And this problem gets magnified in macroeconomics which is why governments make such a hash of it. There simply isn’t enough information, even when there is a huge amount of data.

Indeed this is the crux of the matter: data is not the same thing as information. That is what the Shepherd Problem is about.

As long as psephologists and economists don’t grasp this, they will be tilting at the windmills.

Twitter: @tca_tca