In my previous post I talked about the problem of poor estimating. This piece is about solutions. That’s possibly too neat and precise a word. The point is that there aren’t neat and precise techniques you can use to get the right answer, and if you think that there are then you’ll just be perpetuating the problem.
I think that the right attitude is more important than any technique. If you accept and understand that then you’re less likely to get into damaging situations than if you think it’s simply a matter of acquiring greater technical expertise in estimating.
There are four elements to the “right” attitude; clarity, realism, being comfortable with uncertainty and honesty.
Clarity about what we’re trying to do
What are we really trying to do when we’re asked for an estimate?
Often we get confused about the question we’ve been asked. We hear a question, then instinctively answer either the question we’d be more comfortable with, or change the question to reflect our assumptions about the questioner, or the business. It’s not a simple matter of providing the answer we think is expected. That happens, especially in dysfunctional organisations where questions are merely a way of relaying orders, rather than trying to elicit information, where ”can you do this? means ”do it!”
No, I’m more interested in cases when we genuinely don’t realise what we are being asked.
So a user asks us; ”can you deliver a new interface for travel insurance to the XYZ application by the end of November?”
And we hear; ”I’m assuming there’s an absence of hard evidence that you can’t deliver … do you agree?”. And we simply say ”yes, we’ll do our best”, without considering the question carefully.
Or we think; ”another crazy project with an impossible schedule on its way” and we dig in our heels or stall.
Either way we’re assuming that the question represents a fixed, final position or an attempt to dictate one. It’s not necessarily so. Perhaps the real question that needs to be answered is; ”how much functionality can you deliver by the end of November (or for a given sum of money)?”, or “when can you be reasonably confident you can deliver to the standard we require if we’re to keep the customers happy?”
In that travel insurance example we should really be treating the question as the start of a discussion, rather than a final demand. Are our users looking for a firm commitment that they can communicate to external clients or the outside world? Or are they simply looking for the best information about when implementation is likely to take place? In reality these are different questions, but the same words might be used in each case.
What is the significance of November? Maybe it is so that the application will be in place in time for Christmas, so that when people turn their minds to summer holidays everything will be in place. Perhaps, for that reason, the travel firms, who sell insurance on our behalf, say it must be ready by then.
Does the whole solution need to be in place for that date? Would it be acceptable for the underwriting to be delivered first, then the claims functionality and then the management information, pricing and reserving interfaces?
The only time in my career when I was in a role where I felt we were competent at estimating development work was when we had built up a relationship with the same set of users over a couple of years, so that we understood them and their business concerns.
We were able to ask the right questions, and negotiate from an ideal but impossible request down to a feasible and acceptable one.
Realism – resist the optimism bias
Obviously, we have to resist the optimism bias and the danger of excessive detail, as I discussed in my last piece “it always take longer (part 1)”.
If it always took x months in the past it will do so in the future, unless there is a very good reason. “We were unlucky”, or “that team screwed up, we’re better than that” do not count as good reasons.
The lesson we have to learn is to use our experience to understand what has happened to previous projects. i.e. what is the distribution of outcomes? We should then sternly tell ourselves that we need very good reasons indeed to believe that we can buck the trend.
I should have mentioned this excellent piece by Eliezer Yudkowsky about ”the planning fallacy” in my last article.
”Just ask how long similar projects have taken in the past, without considering any of the special properties of this project. Better yet, ask an experienced outsider how long similar projects have taken.
You’ll get back an answer that sounds hideously long, and clearly reflects no understanding of the special reasons why this particular task will take less time. This answer is true. Deal with it.”
I mentioned earlier the time I was in a team that was fairly competent at estimating. An important reason was that we had a good base of experience on which to base our estimates. We were were building a very large suite of management information applications, and each development was a relatively small addition of new functionality to a technical architecture we understood, in order to satisfy business needs that we understood.
Of course the practice of breaking a huge project down into smaller, more manageable and more predictable components massively reduced the risk and gave us greater confidence in our estimates. Producing better estimates isn’t a matter of doing one thing perfectly; it’s about doing many things more sensibly.
Remember that we are better at estimating the work of others, so try to get some objective independent assessment of your estimate. As Yudkowsky suggests, an outside perspective can be valuable.
Even within the team it is important to canvass views. The Planning Poker technique is interesting. Of course it’s specifically designed for an Agile project, and therefore assumes that the development is already broken down into manageable chunks (stories). However, I like the way that it involves everyone. People have to state their expectations and justify their assumptions. In my experience some people are more comfortable keeping quiet and not contributing to the estimation process even when they can contribute usefully. They prefer someone else to take the responsibility for producing an estimate that might well be inaccurate.
I used to find estimating stressful because I couldn’t be confident about how long work would take. Then it dawned on me that my lack of confidence and my lack of knowledge about the amount of work weren’t signs of incompetence or diffidence. They were signs that I knew the reality. The uncertainty was real. Certainty was an illusion.
Be clear about what we can’t know. Don’t pretend that certainty is available when it isn’t. Don’t allow your audience to believe they can be very confident about happy outcomes in conditions of uncertainty.
You need to be able to point to relevant experience that justifies high levels of confidence or you are kidding yourself and the stakeholders. If that base of evidence is not available then say so. Explain how that must affect the confidence you can attach to any figures.
Dan North came up with a great insight a couple of year ago in a piece he wrote, ”the perils of estimation”.
“… the business wants accuracy and we’re giving them precision. If you tell me it will take 4.632 months and it takes 8 months, that’s worse than useless. If you tell me it takes “about six months” and it takes seven months, I should still be onto a winner.“
The accurate answer is usually an approximation with a certain degree of confidence. Precision is spurious and misleading. You need to be clear about that.
This takes us into territory I discussed last year when I was looking at the implications of using external suppliers. I recommend that you check out Mary Poppendieck’s work, in particular about the Predictability Paradox, which states that striving for predictability pushes back the point when you truly can find out what is required and possible. The relevant links are in my blog post, “usability and external suppliers – part 2”
Honesty – it’s an ethical issue
Clarity, realism and an acceptance of uncertainty are all essential if you’re going to have a chance of producing useful estimates. They’re not enough if you’re not honest about what’s going on and what you’re going to do. In turn, honesty is essential but it really depends on the other three.
There are some outright rogues whose business model depends on producing unrealistically low estimates to secure work, with the conscious intention of screwing the client through change requests. Perhaps I’m being naïve, but I think we more often see the same result because it’s convenient to ignore the truth, not because we are consciously lying.
IT suppliers and project managers work in a murky world where it’s easy to lose sight of the need for honesty, and it is quietly left behind at the station as the project express chugs blindly on into the gathering darkness. Nobody notices, or owns up till it’s far too late. Yes, I know I’ve rather overworked the metaphor, but it does appeal to me.
These two articles are worth reading, and make the point well. This one, by Alan Kelly, ”dear customer: the truth about IT projects”, makes the general point that we can’t really know what we’re going to do, and we don’t own up to that fact.
The second, by Shim Marom states bluntly that this is an ethical issue; “the impact of ethical project management on project success”.
If the corporate culture is to encourage blind optimism then you have a difficult personal decision to make. Do you play the corporate game or do you expose the uncomfortable truth? Life is simpler if you’re not sufficiently astute and experienced to recognise the problem. You make the implausibly optimistic estimates that are welcomed and get on with the job, integrity intact. If you do know the score and you play along anyway then it does become a question of conscience and integrity.
There’s no point telling people that doing the “right thing” is all that matters, even at the expense of their family. It’s a decision everyone has to take for themselves. All I can offer is the advice not to get into that situation in the first place. It’s better not to work for that sort of damaged, and damaging company, or to find a new job once you realise what’s going on.
I’ll happily admit I’ve provided no neat fixes for anyone wanting to estimate better. But I know that these four points are crucial if you’re ever going to deal with the problem better.
Dale Emery gave a very good five minute talk at STAREAST 2011 when he addressed some of these points, “it’s not an estimating problem”.
Dale framed the problem in three ways; rather than being a genuine problem with estimation it’s usually either a definition problem, an honesty problem or a negotiation problem.
As a final word of advice; if in doubt, remember Hofstader’s Law!.“It always takes longer than you expect, even when you take into account Hofstadter’s Law”.