No Dice
What could be simpler than dice?
If everyone has a fair un-weighted die, all have the same chance to win or lose.
P(A wins over B) = P(B wins over C) = P(C wins over A) = 0.5
You win half the time, you lose half the time.
We can change the dice to make the battle less equal. We can give some multiple sides marked 6, and others multiple sides marked 1. We can shift the balance to give A an edge over B, B an edge over C… and then overall C is the big loser.
Mathematically we call this the transitive property.
If A > B and B > C, then A > C.
Obviously and intuitively this must be true, right?
You’d be wrong in thinking so.
It is possible to design a set of dice where A on average beats B, B on average beats C and C on average beats A. We call these non-transitive dice.
An example of such dice could have sides marked (if you don’t want to click the link):
- Sides: 3, 3, 3, 3, 3, 6
- Sides: 2, 2, 2, 5, 5, 5
- Sides: 1, 4, 4, 4, 4, 4
Die 1 beats die 2 in 21 out of the 36 possible combinations.
Die 2 beats die 3 in 21 out of the 36 possible combinations.
Die 3 beats die 1 in 25 out of the 36 possible combinations.
Obviously, intuitively, this isn’t possible, yet there it is.
Best Man for the Job
Which is where a recent management dilemma comes into the picture.
I somewhat flippantly summarised my problem as related to explaining a potential budgeting issue due to non-transitive aspects to skill-to-project mappings. That was really a slightly hand-wavy allusion to the dice above.
Let’s make it a little more concrete.
Let’s say we have 3 IT projects:
- Web project; no complicated coding, but it does need to look good
- Banking system; routine processing, but it does need to be fail-proof
- Page-rank; a very mathy problem, but once understood easy to code
Let’s say we have 3 Developers (all parallels to real people is coincidental):
- One genius: Steve Yegge
Great at almost everything, except graphical design - One average developer: Jeff Atwood
A flair for design, and knows enough transaction-safety, but no maths-whizz - One manager who shouldn’t be coding: Bill Gates
Passable design-sense, passed maths, but don’t trust him with your money
Let’s say we have to estimate how long it will take to complete the 3 IT Projects with the Developers we have available. We could specifically allocate names to the projects, but the projects are due to start at different points in the future that we do not know yet.
Well… how about we just estimate everything for a general average developer and then work out the names later? Intuition says that some will be over and some will be under, but on average we’ll get our estimates right… right?
Web | Banking | Page-rank | |
Yegge | 50 days | 10 days | 10 days |
Atwood | 10 days | 30 days | 50 days |
Gates | 30 days | 50 days | 30 days |
On average, each of the projects will take 30 days.
On average, Yegge is faster than anyone else.
On average, Atwood is average.
On average, Gates is slower than anyone else.
But let’s just step away from the averages and have a look at what could happen once reality unfolds in three different ways:
- Yegge does Banking, Atwood does Page-rank, Gates does Web
Result: 90 days total, 30 average per project
A truly average result - Yegge does Banking, Atwood does Web, Gates does Page-rank
Result: 50 days total, 16.7 average per project
The best possible outcome - Yegge does Web, Atwood does Page-rank, Gates does Banking
Result: 150 days total, 50 average per project
A learning experience for everyone; also: OH GOD, WHY? IT BURNS!
So, depending on who is available when the projects kick off, either through chance or because we planned it to be that way, it could take as little as 50 days or as many as 150 days.
It could be 40 days under (almost 50%), or 60 days over (more than 60%).
And note that this is with a relatively mild skills gap.
Some developers are order-of-magnitude faster than others.
Also note that in this example I’ve glossed over the other project work that would make it impossible to assign Yegge to both Banking and Page-rank, which would leave Bill no coding to do (arguably an even better outcome for everyone).
Again, our intuition is wrong.
And for very similar reasons to the dice.
What this means is that it’s almost always a mistake to try to estimate project work based on an “average” developer. It is better to always have a specific developer in mind, and if that changes, adjust the effort estimate accordingly. You will end up with far fewer counter-intuitive results to explain after the fact.
Try Science!
And that brings me back to an earlier point.
If possible, always try science first. Even on problems that you cannot completely “solve” mathematically. An approximation will at least warn you if There Be Dragons before you get et.
Pingback: Why are software development estimates regularly off by a factor of 2-3 times? | JerryJvL