It is difficult to talk of value when you cannot measure the value you are delivering: Rohan Murty

The IT services industry in India faces two key challenges today. One, creating differentiation in an increasingly commoditising market. Two, managing scale and growth while improving quality and productivity. Against this backdrop, "measuring the output produced by software engineers must become the core organising principle for how this industry delivers value to clients and how companies compete in the marketplace," said Rohan Murty, a junior fellow at the Harvard Society of Fellows and, until June 2014, executive assistant to his father, Infosys co-founder NR Narayana Murthy. He was speaking at the 50th anniversary symposium of the computer science department at Cornell University. Here is an excerpt from his speech where Murty addressed some key questions relating to employee productivity: How does one measure how much output a software engineer produces? How does one scientifically determine how many software engineers one needs to solve a problem? How then does one pick people to form a team to solve a given problem?

My primary area of research has been in systems and networking. However, today I have decided to speak on another area that has not received much attention from researchers in computer science - industrialising IT services.

Today I wish to talk about a key theme across work over the past year - how do I measure how much output a software engineer produces? How do I scientifically determine how many software engineers do I need to solve a problem? How then do I pick people to form a team to solve the problem? I seek to find a systematic measurement methodology and scientific basis to address these questions.

Therefore, I will primarily speak on the problem of estimating individual productivity of a software engineer and briefly mention its potential to reorganise the entire labour force in this industry.

This idea of measuring individual productivity is a topic that I have got to know fairly well in the recent past when I have been on leave from Harvard to spent time in India working at Infosys. When I refer to IT services in this talk, I will restrict myself to Indian IT services since this is a geography, culture, and industry I understand reasonably well. My goal is to highlight some key technical challenges in this industry and the potential ramifications of addressing these challenges.

The core value of the Indian IT industry is providing the best value for money in bespoke software development, software maintenance, infrastructure management, large-scale software testing, big data analytics, and business process management (BPM).

The industry was born in the late sixties in India but gained global recognition only after the Indian economic reforms of 1991. Indian IT services and the BPM Business Process Management (BPM) companies generated around $100 billion in 2013. That is roughly 8-10 per cent of the estimated global IT services market. The industry employs about 3.1 million people and has around $30,000 of per capita revenue. This is roughly 20 times India's per-capita GDP of $ 1,500. These 3.1 million jobs are high disposable income jobs, the kind that India had not seen before its economic reforms of 1991 in such large numbers. Subsequently, the growth of this industry has played a key role in the rise of the Indian middle class.

  This year alone, the industry is expected to hire around two lakh engineers, which is approximately 20 per cent of the graduating engineering population in the country.

This industry forms a key part of the technology strategy of most global companies. Consequently, almost every one of you in this room has benefitted from the value addition of the Indian IT services industry. You have received better value for money in the clothes you wear, the food you eat, the cars you drive, your bank transactions, airplanes that you fly, drugs that you take and the insurance cover you get, thanks to the Indian IT services industry.

The IT services industry faces two key challenges today. First is in creating differentiation in an increasingly competitive and commoditising market. Second is in managing scale and growth while retaining culture, customer focus, employee aspiration, quality and productivity of its diverse, global talent base. For example, IT companies in India add between 20,000 and 55,000 employees each annually to a base of 100,000 to 300,000 employees. This is like adding a new Google or six Facebooks every year.

The closest analogy we have to these two challenges comes from the manufacturing world, which has managed to achieve scale, differentiation, and quality at the same time. This was predominantly fuelled by a series of innovations in industrialisation, which included creating the assembly line, division of labour, rigorous processes, decomposing work into discrete tasks that were measurable and a focus on both quality and cost control, among several others.

When the likes of Ettore Bugatti crafted beautiful cars, car design and manufacturing were treated as a work of art. But at the turn of the 20th century, Henry Ford, turned car manufacturing from an art-form to a mass production industrial activity. In the process he lowered costs, made cars more accessible, and enabled a mass production and adoption of cars. Thus the term - Fordism was born.

There are lessons here for the IT industry, which is today at the cusp of a similar transformation. Software architecture and design are considered to be highly creative, imaginative and an art form much like the first set of hand-crafted cars. There are also no established formal methods available in practice for 'architecting' and designing large-scale software applications. Thanks to the work of people like Bob Constable at Cornell and his students of the NuPRL group, in time that art form is likely to become science in the years to come.

On the other hand, I believe activities such as programming, testing, maintenance, infrastructure management, and some aspects of business process management are becoming increasingly mundane, repetitive, and are therefore ripe for industrialisation. These mundane activities represent a significant fraction of the industry's revenues.

At the scale that this industry has begun to operate at, I believe we have no choice but to start treating the metaphor of 'software factories' as a reality. But to do so we must first fully embrace the idea of industrialisation, much like several other industries have done. Industrialisation in this context has five facets: (1) standardisation of processes, (2) a clear division of work or componentisation of the final product, (3) measurement of output produced and productivity, (4) optimisation of processes, and (5) augmenting human effort with machines (software, in this case), where possible.

Of these, I believe the industry has so far implemented the first two facets reasonably well. They have adopted the Capability Maturity Model (CMM) enunciated by the Software Engineering Institute at CMU for process orientation.

They have introduced componentisation for enhancing work quality, productivity for specific software systems, built knowledge management systems to enable component reuse. However, there appears to be very little headway in the area of measuring the output produced by software engineers and, therefore, the productivity of an individual or a team.

To put it another way, though human effort is the key ingredient in delivering value to clients, today, we are as yet unable to clearly quantify the output this very same human effort delivers. Our best bet so far has been to quantify effort merely in terms of time (the input). This is insufficient.

What do I mean by measuring individual productivity? In the abstract it is the output produced per unit time.

But why is individual productivity necessary and important from the perspective of industrialisation? Measuring the output produced by software engineers must become the core organising principle for how this industry delivers value to clients and how companies compete in the marketplace. Measuring individual productivity forms the basis for ensuring 'mass production' of software. After all, without measuring productivity, it is impossible to ascertain inefficiency lurking in a team or in an organisation. For example, it may be the case that you really do not need 100 people to solve a problem but you only need 80 people. But to realise this truth you must first instrument and measure individual productivity. As Lord Kelvin said, you cannot improve unless you measure.

Moreover, this industry's edge - the best value for money argument - will disappear if you don't find inefficiencies in your own system and improve on them.

Today, there is hardly any serious mention of productivity in discussions between clients and the software industry. This has to change. There is zero discussion or effort being made to measure individual productivity.

Here is a sample format of the data we are able to measure based on the individual productivity measurements performed at Infosys over the last year.

This is the typical format of the data that we are able to generate for several teams in the organisation. It is a table that displays the individual productivity of each member of a team, in comparison to the overall team's performance. This is perhaps for the first time in this industry that anybody is able to generate and view productivity with such fine granularity. It is impossible to expect every individual to operate at the top end of the productivity curve. There will always exist a variation of productivities in any team. However, we took steps to improve the productivity of individuals by targeting specific training, enhancing the process via which work was distributed to members of the team, or even construct an all-star team (much like the NBA) consisting of high productivity individuals across the organisation to solve specific problems for urgent needs for clients.

But this measurement of individual productivity is far more powerful since it has the potential to affect several important aspects of this industry. Here are a few examples of the changes we were actively pursuing:

• Human resources management: Such measurements provide mechanisms to appraise software engineers on an objective basis rather than on a subjective basis as is done by most companies today.
• Proactive interventions: Such measurement systems help a project manager to proactively step in and figure out how to improve the productivity of individuals with low productivity. For example, in certain cases we can target training towards individuals with lower productivity.
• Team composition and better work allocation: Managers start taking into account the productivity of individuals when allocating work to the team members based on their productivity. Invariably, the most productive people appear to get the most complex task.
• Better supply chain management: The supply chain function in a software company must take into account the productivity of individuals when forming project teams. The complexity of the project, the criticality of the applications to the client, and the elapsed time of the project must form the input for optimal composition of the project team. Such optimal composition helps in eliminating resource underutilisation and waste.
• Sales differentiation: The ability to measure individual productivity can become an important differentiation in a highly competitive marketplace. Such a measurement system allows a company to focus more on fixed-price projects and enhance margins. Today, most of the projects undertaken by the Indian IT services industry are on a time-and-material (T and M) basis. In T and M projects, most of the productivity gains go to the client. However, even in such projects, higher individual productivity methods translate to higher quality and reduced total cost of ownership. Such high productivity also means companies can take up complex projects, which can provide premium pricing to the IT services company.

Today, individual productivity is not perceived a big deal yet because people don't seem to worry about what input goes into meeting customer SLAs (service-level agreements). The industry predominantly appears to sell on price rather than value. But it is difficult to talk of value when you cannot measure the value you are delivering. The client CIO is somewhat happy because the industry continues to find some way to give them semi-regular price cuts and still maintain healthy margins. However, this cannot continue forever.

As late as June 2014, we had a plan to roll out these measurements to cover 10,000 software engineers at Infosys (about 7 per cent of the population). I do not know of its current status. We built models to measure productivity of engineers focused on ticket-oriented work. We faced challenges in scaling this up from measurements, better modelling of the work done by the humans, to effectively changing the culture in the organisation to focus more on productivity.

But this problem of measuring individual productivity is non-trivial. I believe it will require addressing some hard computer science problems. Several crucial issues come into discussion.

Let us consider programming as an example. Is a person who writes 100 lines of code in unit time more productive than a person who writes 50 lines of code in the same unit time? We can make the argument go both ways. Similarly, if Engineer-1 produces 100 lines of Java code in unit time, how do you compare him/her to Engineer-2 who produces 100 lines of Python code in unit time?

Should we think about lines of code at all? Can we then infer that, in general, Python programmers are more productive than Java programmers? And does this arise due to the inherent design and syntax of the respective languages? Imagine dealing with this problem in the enterprise where we see a cavalcade of languages in the code base - from COBOL to Ruby. There are similar such questions when we think of software testing, maintenance, and business process management tasks.

But this then raises several other interesting questions. When building software applications should we think of the productivity of the end user who uses the application? Would we then build and optimise software applications in a different way if there was a heavy focus on the end-user's productivity? And how about teamwork? How do you compare the productivity of an individual in taking up a stand-alone task versus working as part of a team where the productivity depends on the complexity and latency of interactions? Professors Eva Tardos and Jon Kleinberg would worry about creating appropriate incentives to balance the individual's interest with those of the team. On the other hand, Fred Schneider would worry about the possible relationship between productivity and security- whether a highly productive programmer is necessarily producing more secure code than a programmer who is far less productive?

Ultimately, I believe, beginning to address individual productivity will require a serious research agenda that is at the confluence of programming languages, systems, software engineering, mechanism design, and security.

There is no single silver bullet for individual productivity. We will need different models for different kinds of work in the IT services industry. But any model for productivity will need to have one important property: it must permit us to add up the productivities of individuals and give us a composite measure of the team's productivity. This will ensure that such models form the very basis for re-organising the labour force in this industry.

It is true that function point analysis introduced in the late seventies is still used today to measure software development productivity. The truth, however, is that this is rather antiquated work that very few people on the ground understand or are willing to accept. It does not answer the kinds of questions that I have raised thus far in my talk.

Finally, I believe that quality and productivity are duals of each other. Our measurements thus far have shown that those individuals at the top of the productivity curve tend to produce work that has fewer bugs or client escalations. Customers appear to be happier with the work done by these individuals. Therefore, it is necessary to first recognise, measure, and improve productivity at an individual's level if an IT services company wants to produce high quality work.

There are several other important problems that come up when attempting to industrialise IT services but in the interest of time I shall end here. There are, of course, sociological issues that we must grapple with. Such measurements must form the beginning of truth, not the entirety.

I have often found a bias that 'innovative' work can only be done in product-oriented software companies. However, I hope I have given you a sense of why this is merely a bias as opposed to a fact. The problems I have mentioned in this talk demonstrate that there do exist several deep and hard problems in computer science in the software services industry in India and solving these can have a big impact on society. They present a tremendous opportunity for researchers in systems and theory to transform how technology is delivered and consumed.