Why is estimating an important skill?

How many sweets are in the jar?

As well as playing an important role in our daily lives, estimates are crucial to an organisation’s success. Planning budgets, bidding on future projects, resource allocation, personnel recruitment - all require us to stick a finger in the air and venture a (hopefully) educated guess.

In most instances we have data to help us, but there are occasions, particularly in risk assessment, when we can’t rely on past experience to guide our estimates.

The goal is to reduce uncertainty

There’s been lots of research done on probability estimation, with two broad conclusions:

• Most people are bad at estimating probabilities
• Most people can be trained to be very good at estimating probabilities.

Most of us don’t like putting forward a number without solid evidence to back it up, particularly in a work environment, for fear of being held accountable to it. Terms like “We can’t measure that”, “That depends” or even “There are too many variables” are used a lot in these situations. But remember: the objective of estimating is not to determine something with absolute certainty, but rather to reduce uncertainty allowing for better decision making.

Try expressing your estimate as a range

Rather than estimating an exact value, it’s a good idea to use a range within which we have a 90% confidence that the correct answer will be. That is to say, nine-out-of-ten times we are confident that the answer will be between the lower and upper bounds of our range, or confidence interval (CI).

It’s worth noting at this point that we are not using a 100% confidence interval, so we accept that there is still a small, but non-negligible chance that the answer could fall outside of our chosen range. This is important because we are openly admitting that once every ten times the answer will not be in our estimated range, and allows this to be taken into account.

Test and calibrate your 90% CI

This technique was developed by Douglas Hubbard and Richard Seierson in their insightful book “How to Measure Anything in Cybersecurity Risk”. It’s an effective way of getting a baseline of your 90% confidence interval, and is done by performing a simple test.

Answer the following question giving the answer as a 90% confidence range: How many days per year does it rain in the UK?

After you’ve answered the question, but before checking the answer, perform the following check: If money was riding on this question, which of these deals would you prefer?

• A) You get £1000 if the answer is within your estimated range, and nothing if it outside of your range
• B) You spin a wheel of which nine tenths would mean you win £1000 and 1 tenth means you receive nothing

If, like most people, you prefer the sound of option B), then you are under-confident in your range and should expand it. If you prefer option A), then you might be over-confident in your range and could reduce it.

Ideally you want to be indifferent to options A) and B), as they should offer the exact same probability of winning the money.

Photo by Clem Onojeghuo on Unsplash.