 "Better innovation")
Most businesses lack sufficient data to inform their decisions pertaining to innovation. Big Data can provide clues about the past behaviour of customers, but not about how they will react to bold changes. As a result, managers often tend to rely on their experience or intuition in making decisions. But ideas that are truly innovative - the kind that transform industries - typically go against the grain of executive experience and conventional wisdom.
There is, however, a simple way to determine whether or not a new product or business programme will succeed, and that is to subject the concept to rigorous tests. A pharmaceutical company, for instance, would never introduce a drug without first conducting a round of experiments based on established scientific protocols. Yet many companies miss this essential step when they roll out new business models or novel concepts.
The simple reason why companies choose not to test out their risky overhauls is that most of them are reluctant to fund proper business experiments and, even if they do so, face considerable difficulty in executing them. Although the process of experimentation seems straightforward, it is surprisingly hard in practice. To ensure that business experimentation is worth the effort, companies need to ask themselves the right questions: Does the experiment have a clear purpose? Are stakeholders committed to abide by the results? Is the experiment doable? How can one ensure reliable results?
Defining the purpose
Experiments should be conducted if they are the only practical way to answer specific questions about proposed management actions. In determining whether an experiment is needed, managers must first figure out exactly what they want to learn. Only then can they decide if testing is the best approach and, if it is found to be such, the scope of the experiment.
All too often, though, companies lack the discipline to hone their hypotheses, leading to tests that are unnecessarily costly or, worse, ineffective in answering the question at hand. In some situations, executives may need to go beyond the direct effects of an initiative and investigate its ancillary effects.
Securing stakeholder commitment
All stakeholders must be agreed beforehand on how they will proceed once the results are in. They should be willing to weigh all the findings instead of cherry-picking data that supports a particular point of view. Most importantly, they should be willing to walk away from a project if it is not supported by the data. A process should be instituted to ensure that test results are not ignored even if they contradict the assumptions or intuition of top executives.
There might, at times, be good reasons for rolling out an initiative even when the anticipated benefits are not supported by the data. For instance, a programme, which experiments have shown will not substantially boost sales, might still be necessary to build customer loyalty. In such cases, when proceeding with the initiative is a foregone conclusion, time and expense can be spared that would otherwise be lost on tests.
Is the experiment is doable?
Experiments must have testable predictions. But the causal density of the business environment 'the complexity of the variables and their interactions' can make it extremely difficult to determine cause-and-effect relationships. Learning from a business experiment is not necessarily as easy as isolating an independent variable, manipulating it, and observing changes in the dependent variable. Environments are constantly changing, the potential causes of business outcomes are often uncertain and so linkages between them are poorly understood.
To deal with environments of high causal density, companies need to consider whether it is feasible to use a sample large enough to average out the effects of all variables except those being studied. Unfortunately, this type of experiment is not always doable. The cost of a test involving an adequate sample size might be prohibitive, or the change in operations could be too disruptive. In such instances, executives may employ sophisticated analytical techniques to increase the statistical validity of their results.
Ensuring reliable results
More often than not, companies have to make trade-offs between reliability, cost, time and other practical considerations. Randomised field trials, blind tests and Big Data can help reduce such trade-offs, thereby increasing the reliability of the results.
The search for new business ideas and new business models is perceived as a hit-or-miss scenario by most corporations despite extraordinary pressure on executives to grow their businesses. It is important, in this context, to view experimentation as a prerequisite to innovation and an integral part of an organisation's learning culture.
Stefan Thomke
William Barclay Harding Professor of Business Administration & Faculty Chair for Driving Growth through Innovation - India, Harvard Business School
There is, however, a simple way to determine whether or not a new product or business programme will succeed, and that is to subject the concept to rigorous tests. A pharmaceutical company, for instance, would never introduce a drug without first conducting a round of experiments based on established scientific protocols. Yet many companies miss this essential step when they roll out new business models or novel concepts.
The simple reason why companies choose not to test out their risky overhauls is that most of them are reluctant to fund proper business experiments and, even if they do so, face considerable difficulty in executing them. Although the process of experimentation seems straightforward, it is surprisingly hard in practice. To ensure that business experimentation is worth the effort, companies need to ask themselves the right questions: Does the experiment have a clear purpose? Are stakeholders committed to abide by the results? Is the experiment doable? How can one ensure reliable results?
Defining the purpose
Experiments should be conducted if they are the only practical way to answer specific questions about proposed management actions. In determining whether an experiment is needed, managers must first figure out exactly what they want to learn. Only then can they decide if testing is the best approach and, if it is found to be such, the scope of the experiment.
All too often, though, companies lack the discipline to hone their hypotheses, leading to tests that are unnecessarily costly or, worse, ineffective in answering the question at hand. In some situations, executives may need to go beyond the direct effects of an initiative and investigate its ancillary effects.
Securing stakeholder commitment
All stakeholders must be agreed beforehand on how they will proceed once the results are in. They should be willing to weigh all the findings instead of cherry-picking data that supports a particular point of view. Most importantly, they should be willing to walk away from a project if it is not supported by the data. A process should be instituted to ensure that test results are not ignored even if they contradict the assumptions or intuition of top executives.
There might, at times, be good reasons for rolling out an initiative even when the anticipated benefits are not supported by the data. For instance, a programme, which experiments have shown will not substantially boost sales, might still be necessary to build customer loyalty. In such cases, when proceeding with the initiative is a foregone conclusion, time and expense can be spared that would otherwise be lost on tests.
Is the experiment is doable?
Experiments must have testable predictions. But the causal density of the business environment 'the complexity of the variables and their interactions' can make it extremely difficult to determine cause-and-effect relationships. Learning from a business experiment is not necessarily as easy as isolating an independent variable, manipulating it, and observing changes in the dependent variable. Environments are constantly changing, the potential causes of business outcomes are often uncertain and so linkages between them are poorly understood.
To deal with environments of high causal density, companies need to consider whether it is feasible to use a sample large enough to average out the effects of all variables except those being studied. Unfortunately, this type of experiment is not always doable. The cost of a test involving an adequate sample size might be prohibitive, or the change in operations could be too disruptive. In such instances, executives may employ sophisticated analytical techniques to increase the statistical validity of their results.
Ensuring reliable results
More often than not, companies have to make trade-offs between reliability, cost, time and other practical considerations. Randomised field trials, blind tests and Big Data can help reduce such trade-offs, thereby increasing the reliability of the results.
The search for new business ideas and new business models is perceived as a hit-or-miss scenario by most corporations despite extraordinary pressure on executives to grow their businesses. It is important, in this context, to view experimentation as a prerequisite to innovation and an integral part of an organisation's learning culture.
Stefan Thomke
William Barclay Harding Professor of Business Administration & Faculty Chair for Driving Growth through Innovation - India, Harvard Business School