For the sake of argument, I've assumed a project team that works over ten units of effort (the horizontal axis: you can think of this either as time units or as money units spent), consumes a cost of 10 monetary units and produces 20 monetary units worth of value. These numbers are pulled out of my hat and don't matter much: only the numbers on the scales will change but the graphs themselves will be identical. The ROI is of course value minus cost compared to cost: with these numbers the ROI is 100% which is fairly low as ROIs go but serves our illustrative purpose.
I will use this same team and project to draw up and project the ROI for four different approaches:
- A traditional plan-driven project delivering near the end
- An unfocused project with continuous delivery
- A focused project with an 80/20 Pareto distribution of value
- A focused project with an 80/50 Pareto distribution of value
First, let's consider the traditional plan-driven IT project. The team implements requirements in any old order (easiest first? most interesting first? software stack from bottom up?) and makes a 1.0 delivery quite late in the project followed by 1.0.1 and 1.1 deliveries. In this case, the return on investment looks like this from the customer's viewpoint:
Doesn't it look realistic? :-) Please note that this is a successful plan-driven IT project that actually delivers on budget! The costs are accumulating all the time, but the customer receives their first dose of value quite late. The last deliveries add some random functions and fix a number annoying bugs, but the ROI (the dotted line) doesn't increase much anymore but hovers around 100%.
From the start up until the first delivery, the customer doesn't really know what's happening. A lot of reports have dropped in, but no working code. The customer would not know if the project was in trouble.
Now consider the same team doing continuous delivery. They're still implementing requirements — errr, backlog items! — in any order that seems reasonable, but deliver to the customer on a weekly basis. From the customer's perspective this is a game-changer: after the initial product has been delivered, the value just racks up!
This does require some basic enablers such as a system for continuous integration and automated testing. Plus a strongly disciplined team that doesn't tolerate bugs.
However, while all requirements are of equal value, some just might be more equal than others. In fact, some researchers [1, 2] think that the value distribution should be a Pareto curve, also known as the "20/80 rule". In plain language this means that 20% of the work brings 80% of the value. If the team could somehow (hint hint: ask the customer!) determine which requirements are the most important, the situation would instead look like this:
What's happening here? Instead of going up linearly, the return on investment rises sharply before turning into a slow decline! Indeed, there seems to be a point of maximum ROI somewhere around 3 effort units, where the return on investment is almost 150%: way more than the projected 100%.
Now if you were the business owner and were looking at the ROI only, when would you terminate the project? Most likely at some point between 2 and 4 effort units, because the ROI curve is quite flat at the top and there's a quite large span of effort that would bring you almost the maximum ROI. So let it run to 4 E.U., then terminate. It really doesn't make economic sense to continue after that.
Is it possible to terminate the project at 4 E.U.? Yes of course. Since the team is delivering regularly on a weekly or perhaps even daily basis, the customer always has a working system and there are no technical objections to terminating the development project. Taking a leaf from Jeff Sutherland's "Money for Nothing and (Your) Change for Free", the contract could specify that the customer can terminate the project with a one-sprint notice period, by paying a certain percentage of the extant work.
This approach also requires discipline, but this time it's from the customer. The customer must work with the team early and often, maintaining and prioritizing the backlogs. The customer must also be prepared to "cut out" a large swatch of the initial "requirements". (It helps if you're the kind of person who sees a half-empty glass as half full.)
And here's a more moderate 50/80 Pareto curve, taking 50% of the effort to reach 80% of the value. In this case the maximum ROI is reached at around 6 effort units, but anything between 3 and 9 E.U. will bring in more than the projected original ROI. Or to put it another way, the supplier could double or triple their hourly fees and still meet the cost expectations of the customer.
So what's the point of this Excel exercise? As I stated at the very beginning, relentless focus and rapid feedback (in the form of continuous delivery) can be a game-changer.
These models are of course severely simplified, e.g. they don't account for the fact that it takes a while to whip up the first usable version of the product. But the point is still valid, I think. What do you think?
References:
[1] B. Boehm. Value-based software engineering: reinventing. SIGSOFT Softw. Eng. Notes, 28(2):3–, 2003. ISSN 0163-5948. doi: 10.1145/638750.638775.
[2] J. Bullock. Calculating the value of testing. Software Testing and Quality Engineering, pages 56–62, May/June 2000.
