Innovation as Search

How can a company set out to develop some new technology or capability (that is, to commit resources to it)? Or, what level of surety must they have that they can do it? The answer is that they believe the search space is sufficiently small. Domain experts exist, and can be asked how feasible this is, how close is it to what they or others have done.

More generally, innovation is search. Edison's version, at least as popularly told, relied on some intuition plus a lot of experimentation - at the extreme this is the 'hard' search, or exhaustive search - try everything, every combination until you find what you're looking for.

A company, which needs to be able to show something for its efforts at some point in time, needs to have some level of surety that an attempt at innovation will bear fruit. The level of surety depends on the size of the search space, which in turn depends on three things:

First is how far the targeted innovation is from the current state of the art, whether within the company or within the world. In other words, this is the size of the brute force or 'hard' search space, where you try every combination/option.

Second is how much the search space can be thinned by expertise or intuition in determining which combinations/options to pursue.

The third factor is the number of different versions of success - i.e. the number of points in the search space that are valuable, that is that they can represent the end of the search. More than one point probably has a payoff, and in general the payoffs may not be equal.

Most generally, we can think of this in terms of expected value, or as a histogram of payoff over the basis of the total number of points in the search space that must be evaluated.