Subir Shukla: Does measuring outcomes improve accountability in education?

In the last few years, the clamour for measuring learning outcomes and using that as a means to ensure accountability has grown louder. In fact the current Five-Year Plan insists that learning outcomes be measurable and be measured. Corporate houses funding various foundations and NGOs are big on learning assessment and look to it as a means of bringing about improvement. Many sensible people are voicing views to the effect that if a teacher is unable to generate learning outcomes, he/she should be shoved aside and replaced by someone better. And, of course, the feeling persists that we are not measuring the quality of learning enough.This is unfortunate. Not because measuring outcomes is not important or somehow wrong but because the present formulations of the issue are simplistic to the extent that they prevent underlying issues to be addressed. Here is how.

First, it is not as if the quality of learning is not being measured, or has not been measured in the last twenty years. The first all-India survey of learning levels was conducted by the NCERT in 1995, and there have been many since. Several large-scale independent studies of students’ learning levels have been run, including ASER and surveys of Education Initiatives. Small-scale learning assessments have been conducted for innumerable research studies (e.g. of 1 lakh children in Tamil Nadu to assess the state’s Activity-Based Learning Programme) or pilot projects (for instance, several states have piloted their textbooks and used learning achievement as a benchmark). And of course at least hundreds (if not thousands) of NGOs/NGO-run programmes (often in government schools) have incorporated assessment as an effectiveness measure.

There are, thus, any number of assessments available –and they've been telling us for the last twenty years that our children are not learning. Only, this doesn’t seem to have resulted in improved learning, thus questioning the assumption behind the clamour for measurement.

This is a little like weighing a child to assess the level of nutrition – unfortunately, merely weighing the child will not lead to better nutrition... Something else is clearly required, and that doesn’t seem to be happening.

Second, insisting on having 'measurable' outcomes is hugely misleading – the mere fact that you can measure something doesn't make it more worthwhile (e.g. we do want students to be creative or considerate or civic, though there are no easy measures for these). Several of the assessments mentioned suffer from this. Thus an Adivasi child who displays great resourcefulness, knowledge of the environment and concern for others would be called poorly educated since the ‘tests’ measure only basic literacy and numeracy.

Measuring outcomes would be useful only when we measure what matters most to us. Not whether a child can read something aloud but whether he can form an opinion on it and give the reasons behind them. Not whether a child can do calculations but whether she can apply it in real-world contexts to solve problems or take a decision. Some of these may be hard to measure, but it would be useful to remember that it is not the purpose of education to be assessable, but, rather, the purpose of assessment is to measure what is considered most worth learning.

ALSO READ: Rahul Jacob: India's failing grades

Third, measuring outcomes does not account for contexts and tends to disadvantage (and label) those facing adverse conditions,  which then makes it even more difficult for them to improve. There are many teachers who work very hard in difficult conditions – but don't attain the kind of outcomes expected because the curriculum assumes children will be able to attend daily or speak the school language at home (and several other such notions), which don't apply to the children they work with (some 60-70% in India). We'll end up shoving these teachers out if we take the advice to replace them – instead of overhauling the system which has designed itself in such a way that marginalized children WILL fail.

Fourth, there is a danger that the present focus on outcomes is actually obfuscating – instead of increasing – accountability. India's challenges now arise from its success in rapidly expanding the school system to bring in so many children. The consequence is that we now have students (at all levels) who traditionally never attended schools - working children, migrant groups, girls from various communities, children with disabilities, socially excluded communities…. the list is endless. What this means is that while the nature of our students has changed, the curriculum, pedagogy and assessment remain as they used to be and so, the DESIGN ITSELF leaves these learners out.

At a second level, when it comes to implementation, there is a tendency in those responsible to ignore laxity on the assumed ground that it is only happening to those who do not matter. (Just as it is easier to ask a poor person to push a stalled car rather than a well-dressed one, similar prejudices operate in all walks of our society, including government officials.) Even now, therefore, it is mainly those from better-resourced families who continue to succeed, and we continue to have poor education for the poor. So the accountability really needs to be demanded at the level of the system (NCERT, MHRD, Departments of Education) and state / district / block officials.

As long as people keep pointing fingers at teachers as the main villains, those  really responsible will continue to escape accountability. For instance, when the NCERT's own national survey shows low levels of learning, why does nothing happen to anyone at any level, including the NCERT itself (whose curriculum has been taken by many states now performing poorly)? How come officials at various levels continue exactly as they have been for decades with impunity when every measure  brings out dismal levels of learning  under their watch? Recently, when our group, IgnusERG assessed class 9 students in a district we found 68% of them to be at class 4-6 levels, 7% below class 3 level, and only 4% at the class 9 level where they were expected to be. When this finding is shared, everyone finds a way to blame someone  else!

Finally, let me leave you with this – in the current form, knowledge of outcomes attained does not help bring about improvement. Most states will be implementing SLAS (State Learning Assessment Survey) in the coming months. But once a state finds out it is performing poorly, say, in mathematics, that finding will not inform it of the reasons why this is so. It could be the poor curriculum (e.g. overambitious expectations) or weak syllabus (less time allocated than required), or inappropriate pedagogy (no use of concrete materials at an early age) or bad textbooks (poorly sequenced or giving discrete rather than contextual examples) or de-motivated teachers or insufficient teaching time (because the state continues using teachers for non-teaching tasks even after RTE and court orders to this effect) or home vs school language issues or at least 10 other problems that can be named, each of which can seriously lead to poor outcomes. So where will the improvement begin?

The point, as mentioned earlier, is: do ask for outcomes, but don't keep it simplistic, or we'll continue to get the poor outcomes we've been documenting over the last 20 years.

SubirShukla is with IgnusERG, a guild of resource persons working to support teachers and enhance the quality of education, particularly in government schools. Contact: subirshukla@gmail.com