Thinking about Ideas as a System: Evaluation of Ideas
How great ideas can be bad ideas, and a guideline for going beyond
Background
In Thinking about Ideas as a System: Ideas as Cost we concluded that ideas are not value but cost. Therefore, a system is required to evaluate, explore, and exploit ideas. This article will refer to that system as Ideas as a System.
Ideas as a System has three sub-systems:
- A sub-system for evaluation of ideas
- A sub-system for exploration of ideas
- A sub-system for exploitation of ideas
Evaluation of ideas aims toward removing as much noise as possible. At this point, the number of people exposed to the idea is the lowest among the three sub-systems. The sub-system boundary is from being exposed to the idea to the point when the idea’s value has been found to meet the criteria for being a valid candidate for exploration.
Exploration of ideas is focused on making ideas as buildable as possible. At this point, the number of people actively engaged with the idea is the highest among the three sub-systems. The sub-system boundary is from an idea having successfully passed an evaluation to the point where a commitment is made to exploit it.
Exploitation of ideas is focused on building things. At this point, the number of people actively engaged with the idea mostly consists of those who are building things, such as software engineers. The sub-system boundary is from an idea having become a work order — a confirmed resource allocation — up to the point where the work order has been fulfilled and the built thing can be released.
The sub-system for evaluating ideas is introduced next.
The Process
There are in total eight stages between the conception of an idea, and the idea becoming a thing that can be released and monetized.
- Idea (Evaluation)
- Hypothesis (Evaluation)
- Business Case (Evaluation)
- Concept (Exploration)
- Proof-of-Concept (Exploration)
- Request-for-Work (Exploration)
- Request-for-Comments (Exploitation)
- Work Order (Exploitation)
Idea is the actual idea. At this point, no formal evaluation has taken place. The idea may have been casually discussed and otherwise informally evaluated.
This marks the outer boundary of the Evaluation sub-system.
Hypothesis is a statement such as “This particular features is not the right idea to exploit further” followed by at least some basic testing of that hypothesis. The purpose of this stage is to ensure that the ideas that move to the subsequent phases are the right ideas. The primary means by which evaluation is performed is the Three Rs introduced in the following section of this article.
Business case is a proposal, which illustrates why a particular idea is a right idea, what the implications are for exploiting the idea, what resources are required to exploit it, and how much value can be generated through exploiting it. The primary means by which evaluation is performed are financial modeling and business planning.
This marks the inner boundary of the Evaluation sub-system and the outer boundary of the Exploration sub-system.
Concept is a refined and detailed description of how the idea will manifest as it is exploited, in terms of features, data, user experience, and other such factors. The primary means by which evaluation is performed are the so-called conceptual design frameworks and similar approaches.
Proof-of-Concept is an actual — albeit rough, limited, and hacked together — demo which somehow resembles the thing to be built. Sometimes it’s not feasible to create such a demo, in which case words are used to clearly explain the user experience of an imagined demo. The primary means by which evaluation is performed are software development or “industry fiction” where narrative in prose is created instead of software.
This marks the inner boundary of the Evaluation sub-system and the outer boundary of the Exploitation sub-system.
It is important to note that the artifacts produced in each of the stages become the basis for the artifacts produced in the following stage. Each progression in stages stands as a refinement of what had already been done in the previous stage.
Request-for-Work, Request-for-Comment, and Work Order belong to the exploitation sub-system, and will not be covered in this article further. They are anyhow explained in some detail in The Pyramid of Goodness for Research and Development Organizations [link] and will not be further explained here.
The Actors
In a typical scenario, the primary actors playing either an owning or a contributory role in the process laid out in the above section, are four:
- Sales department
- Marketing department
- Product department
- R&D department
The below graphic illustrates the roles that different departments play in the various stages. There are exactly two roles; contributor and owner.
The basic rule is that contributors contribute based on receiving prompts from the owner of the stage. So for example, during the Request-for-Comments phase, Product department actors can no longer proactively make changes or suggestions, but do so solely based on receiving prompts from R&D department actors.
The Three Rs
During the Hypothesis stage — the stage with the primary function of conserving available resources and reducing noise — the primary means by which ideas are evaluated is the Three Rs method. Using the analogue of a mountain stream, the Hypothesis stage is like the glacier. If the ice of the glacier is polluted, then the water of the whole stream will also be polluted. It is therefore critically important that most ideas never make it past the Hypothesis phase.
While using the Three Rs, the assumption is always — regardless of the idea — that it’s not the right idea. The assumption is that it must not be allowed into to pollute the stream (of ideas flowing through the organization). This is regardless of how great or amazing the idea may seem or feel.
The aspects through which ideas are tested are:
- Resource
- Risk
- Reward
Resource here means how much money, time, and other things are required for exploiting the idea. Important questions include:
- Why this idea?
- Why the timing is right for this idea?
- Why not another idea instead?
- Why not a simpler version of this idea?
Risk here means how much there is economic and other risk associated with exploiting the idea. Important questions include:
- How complex it is to build?
- How many unknowns building it involves?
- How aligned it is with current engineering resources?
- How clearly understood total cost of ownership is?
Reward here means how much money and other benefits are associated with exploiting the idea. Important questions include:
- How does this unlock new financial value (e.g. revenue)?
- How does this unlock other customer value (e.g. reduce the time required for critical workflows)?
- How does this reduce the cost of ownership of the system?
- How does this reduce the energy consumption of the system?
Conclusion
Ideas can and must be evaluated in the same way and with the same rigor — following a formalized system — as other important business decisions are evaluated. In this article, we have provided a high-level overview of a system to do just that. We call it Ideas as a System.
