The 5 Laws of Software Estimates

The 5 Laws of Software Estimates

Estimates are typically a necessary evil in software development. Unfortunately, people tend to assume that writing new software is like building a house or fixing a car, and that as such the contractor or mechanic involved should be perfectly capable of providing a reliable estimate for the work to be done in advance of the customer approving the work. This is pretty attainable by building contractors and auto mechanics, who generally are working with known materials building known things in known ways. Your auto insurance company already knows how long it should take and how much the parts should cost for just about anything you might need to fix on your car (not to mention everything about your model of car). With custom software, however, a great deal of the system is being built from scratch, and usually how it’s put together, how it ultimately works, and what exactly it’s supposed to do when it’s done are all moving targets. It’s hard to know when you’ll finish when usually the path you’ll take and the destination are both unknown at the start of the journey.

one-does-not-simply-estimate-task-duration

I realize that estimates are a hard problem in custom software development, and I am certainly not claiming to be the best at producing accurate estimates. However, I have identified certain aspects of estimates that I believe to be universally (or nearly) true. In typical Internet clickbait style, I’ve dubbed these “laws” of (software) estimates (“and you won’t believe what happens next!”):

1st Law of Estimates: Estimates are Waste

Time spent on estimates is time that isn’t spent delivering value. It’s a zero-sum game when it comes to how much time developers have to get work done – worse if estimates are being requested urgently and interrupting developers who would otherwise be “in the zone” getting things done. If your average developer is spending 2-4 hours per 40-hour week on estimates, that’s a 5-10% loss in productivity, assuming they were otherwise able to be productive the entire time. It’s even worse if the developer in question is part-time, or is only able to spend part of their work week writing software.

A few years ago, a Microsoft department was able to increase team productivity by over 150% without any new resources or changes to how the team performed software engineering tasks (design, coding, testing, etc). The primary change was in when and how tasks were estimated. Ironically, much of this estimating was at the request of management, who, seeking greater transparency and hoping for insight into how the team’s productivity could be improved, put in place policies that required frequent and timely estimates (new requests needed to be estimated within 48 hours). Even though these estimates were only ROMs (Rough Orders of Magnitude), the effort they required and the interruptions they created destroyed the team’s overall productivity. Learn more about this project in this Microsoft white paper or David Anderson’s book, Kanban (learn more about kanban in my Pluralsight course, Kanban Fundamentals).

2nd Law of Estimates: Estimates are Non-Transferable

Software estimates are not fungible, mainly as a corollary to the fact that team members are not fungible. This means one individual’s estimate can’t be used to predict how long it might take another individual to complete a task.

estimating-things-you-know-nothing-about

What’s even worse than having to complete an estimate written by another developer on your team is being held to a deadline based on an estimate produced by a salesperson whose incentive is to win the sale, not deliver a realistic estimate.

estimates-deadlines

The transferability of estimates is obviously improved when the estimator and the implementer have similar experience levels, and even moreso when they work together on the same team. Some techniques, like planning poker, will try to bring in the entire team’s experience when estimating tasks, ensuring estimates don’t miss key considerations known to only some team members or that they’re written as if the fastest coder would be assigned every task. This can help produce estimates, or estimate ranges (uncertainty is a part of estimating – see the 3rd Law below), that are more likely to be accurate, but it does so by multiplying the time spent on estimating by the entire team’s size.

3rd Law of Estimates: Estimates are Wrong

Estimates aren’t promises. They’re guesses, and generally the larger the scope and the further in the future the activity being estimated is, the greater the potential error. This is known as the Cone of Uncertainty.

Possible paths expand the further into future one tries to predict

Possible paths expand the further into future one tries to predict

Nobody should be surprised when estimates are wrong; they should be surprised when they are right. If estimates were always accurate, they’d be called exactimates.

estimates-way-off

Since smaller and more immediate tasks can be estimated more accurately than larger or more future tasks, it makes sense to break tasks down into small pieces. Ideally, individual sub-features that a user can interact with and test should be the unit of measuring progress, and when these are built as vertical slices, it is possible to get rapid feedback on newly developed functionality from the client or product owner. Queueing theory also suggests that throughput increases when the work in the system is small and uniform in size, which further argues in favor of breaking things down into reasonably small and consistent work items.

Estimates of individual work items and projects tend to get more accurate the closer the work is to being completed. The most accurate estimate, like the most accurate weather prediction, tells you about what happened yesterday, not what will happen in the future.

4th Law of Estimates: Estimates are Temporary

Estimates are perishable. They have a relatively short shelf-life. A developer might initially estimate that a certain feature will take a week to develop, before the project has started. Three months into the project, a lot has been learned and decided, and that same feature might now take a few hours, or a month, or it might have been dropped from the project altogether due to changes in priorities or direction. In any case, the estimate is of little or perhaps even negative value since so much has potentially changed since it was created.

To address this issue, some teams and development methodologies recommend re-estimating all of the items in the product backlog on a regular basis. However, while this does address the perishable nature of estimates, it tends to exacerbate the waste identified by the 1st Law of Estimates. Would you rather have your team estimate the same backlog item, half a dozen times, while never actually starting work on it, or would you rather they deliver another feature every week? Again, see the Microsoft white paper referenced above for empirical evidence on the effect repeated estimation can have on overall team productivity.

We know from the 3rd Law of Estimates that estimates tend to grow more accurate the later they’re made (and the closer they are to the work actually being done). Thus, the longer an estimate can be reasonably delayed, the more accurate it is likely to be when it is made. This ties in closely with Lean Software Development’s principle of delaying decisions until the last responsible moment. Estimates, too, should be performed at the last responsible moment, to ensure the highest accuracy and the least need to repeat them. In some cases, the “estimate” can in fact be done after the work is done, when it can be 100% accurate (at virtually zero cost!).

5th Law of Estimates: Estimates are Necessary

Despite the first four Laws of Estimates, estimates are often necessary. Businesses cannot make decisions about whether or not to build software without having some idea of the cost and time involved. Service companies frequently must provide an estimate as part of any proposal they make to build an application or win a project. Just because the above laws are true doesn’t magically mean estimates can go away (#NoEstimates). However, one can better manage expectations and time spent on estimating if everybody involved, from the customer to the project manager to the sales team to the developer, understands these truths when it comes to custom software estimates.

It’s worth remembering Goodhart’s Law: “When a measure becomes  a target, it ceases to be a good measure.” If accurate estimates are desired, they should not be used as commitments or deadlines. If they’re going to be used as targets, they’re going to be modified (read the comments below or on reddit for examples).

Summary

These are my 5 Laws of Software Estimates. They apply to just about every custom software project I’ve been involved with, in roles ranging from developer to architect to salesperson to customer. There’s no such thing as a free lunch, and estimates have real real costs associated with them that should be considered before making them too central to your software development process. Once sufficient high-level estimation and ROI analysis has been done for a project to be approved, additional work estimating may not deliver as much value as more rapid delivery of the actual work to be done.

As usual, Dilbert has some great comics on the topic of estimates:

And finally, if you’ve read this far and are interested in software development, you may want to check out the 2016 Software Craftsmanship Wall Calendar, available now on Amazon.

Updated and published on Medium, with fewer meme gifs.

If you’re in a position where you want a reliable estimate for a software project, and you’re having a hard time getting one from your developer/team, remember this quote (source unknown – I heard it from Ron Jeffries but he didn’t attribute it): “You can’t find someone who knows how long this will take, but you can probably find someone who will lie to you.”

Essentially: The more difficult it is for you to get an estimate, the more likely it is that when you finally do, it’s not terribly accurate.

  • peterjohn936

    IT Departments have to work to stay in business. So they will underestimate so that the project get funded. Telling the true means they will have little to do.

    • WaGonLi

      Yes but is that really a problem ?
      Getting job to do at all cost, deceiving the very people you’re working with is a pretty common but toxic behavior.
      You give them wrong insights on the cost, you stress (a lot) the team in charge of accomplishing the task. Doesn’t ring like a win-win situation (except for some little chief that want to show off)

      • peterjohn936

        It is never a win-win, its a win. IT departments exist to provide work for IT department heads. If telling the true puts you out of business then you won’t.

  • JL

    Here’s the deal, you’re comparing building software to buidling a pyramid,…it’ll get done when it’s done. And though I agree to some degree that for anything to be well built, quality over time is best, even a pyramid has to be ready when the pharaoh dies, so at some point effort (via resources, time on task, etc.) has to be adjusted to met the requirement(s). Estimates are just that “estimates” and should be based on historical work of a similar nature. Yes, each IT project is “unique”, but often the tech, personnel, and requirements are similar enough to give estimates for planning based on past projects. Things must be planned. You’re not creating paintings and even paintings (by very famous artists) often had to meet deadlines. So whether it’s the best thing or not, businesses will need estimates to plan (so just give then one…over estimating is best to give room for quality and risks), and face the facts that estimates and planning is a part of life (and the IT app building project).

    • ardalis

      Yep, hence the 5th Law of Estimates. Thanks for the feedback!

      • JL

        Yes, only any insight in more detail than the 5th law on “how” they “customer to the project manager to the sales team to the developer” should or could do this is MUCH appreciated. 😉

    • “it’ll get done when it’s done” unless it actually never gets done. The longer a project runs the higher probability it will never succeed, ever.

      • ardalis

        An approach to address this is to deliver frequently, with small amounts of scope that is still usable by end-users or at least project stakeholders. This maximizes feedback while minimizing the amount of scope (minimizing what must be estimated and also smaller amounts of scope tend to be estimated more accurately).

  • From the business owner / product road map planner perspective, all laws are acknowledged and cooperation with developers around the reality of development, adoption of agile processes, etc. can lead to a comfortable symbiosis — if communication is stellar. A few things we do that help make estimates valuable to business planning and management of priorities (and the Cone of Uncertainty graph in Law 3 is perfect to explain the compromises that have to be acknowledged in a more fluid estimation process):
    1. The developer who will do the work is the developer that provides the meaningful estimate and they do so after a time-boxed period (e.g. first day of a new sprint) reviewing what is assigned to them and doing initial investigations so they understand what they are likely to need to do
    2. If in the process of executing, they see their original estimate being blown by more than 50% (e.g. estimated 3 hrs, now at 4.5 hours and not even close yet), then they update the estimate and alert the PM so that decisions can be made about priorities – based on revised understanding of size of task, what needs to be sacrificed from this sprint, from the next milestone/release, from customer expectations? Is this task bigger than something we should tackle right now and we should park it? etc…

    The whole idea of agile development from a project management perspective (managing external stakeholders expectations in balance with the realities of the development process) is to allow scope to be fluid but to have fantastic communication around its fluidity so that these expectations can be managed

  • Jakub Leśniak

    All of the above is true, but the reality is that most project managers do not actually want estimates when they ask for them. Instead they want the deadline or that we have a good chance of actually delivering .They just need to know what to promise the customer. No-one is going to have a problem with delivering ahead of time, unless we’re on a highly competitive market and price is really important to success. This makes it a lot easier to produce the number, because simple rule of multiplying by two gives you enough certainty without spending too much time. Everyone is happy, because you deliver, most of the time exceed expectations – you just start working on new stuff if you finish earlier or improve quality.

    • JL

      I agree. What most stakeholders wants to know is “when will it be ready” (call this a deadline or whatever), but you want to know when will it be “deliverable” at a certain level of capability (operationally) and quality that the key stakeholders have agreed upon. So the “estimates” are only needed to help determine this (and they don’t need to take hours or days to figure out, they shouldn’t if you’re organized and have been monitoring how you work and documenting it). I don’t understand what’s so hard about doing this.

      • The two most important questions in business: “when will it be done?” and “how much will it cost?”.

        You should always be able to answer those questions. The only time it is fair to not be able to answer those questions is when you’re doing R&D for something you’ve never remotely done before or has never occurred ever (exceedingly rare)

  • This article is really disappointing Steve. Just because software is hard to estimate properly doesn’t mean it can’t be done. The problem is the lack of engineering applied to the creation of the estimate. Blindly pulling a number from out of nowhere of course will be wrong. But with a well designed system architecture and project plan you can estimate software.

    • ardalis

      If you’re building the same kind of thing, again, using the same tools and targeting the same platform, then yes, you can probably estimate it with some degree of accuracy. Not 100%, most likely, but the cone of uncertainty will be smaller. But if you’re building something that’s using a new framework, or is built on an entirely new library, or you’re using a different kind of architecture (cloud instead of on-premises, for example) or data access (whether your ORM or maybe you’ve switched from relational to no-SQL), or front-end (web, web-SPA, wpf, mobile (many kinds)) all of these will have a tendency to increase the likelihood that your estimates will be way off. Things you thought would be difficult will be easy, occasionally, and more often things you thought would take an hour will take days to figure out and/or debug. Once you gain experience, you get faster at delivering software and also better at estimating your pace. However, with experience you also learn that most new projects will involve some new component(s) in today’s current pace of delivery (unless you’re very specialized and/or you only work on one platform that never changes) and you learn that when this happens, your estimates aren’t going to be very good, at least up front (see the 4th law – the initial estimates are worth less than later ones).

      Estimates are necessary (see the 5th law) in a great many cases. They’re not impossible, but they’re not easy unless you’re doing easy, repeatable work. If you’re building new stuff on top of new stuff targeting new platforms, there’s a lot more that can vary and thus can send the actual time taken way off of what the estimate says.

      • Which is why you don’t estimate a project. You estimate manageable discrete activities. By creating a dependency network of these activities you analyze the float in the project and derive the risk based upon the analysis of available float (or lack of). You use this information to… project the completion date.

        Sure, nearly every new software solution contains some amount of new technologies but there’s a deep vein of lunacy if you think you can change the entire world. The risk of taking a .NET/windows shop to be a python/linux shop would be off the charts to attempt to do it in the course of exactly 1 project.

        Your mentioning of using a different architecture, like cloud. You should only do this either in a prototype / small pilot project or bringing on someone who has deep experience in the cloud. You shouldn’t plan an entire project about some amorphous concept like “going to the cloud” and execute the project blindly stabbing in the dark for the entire time with the entire team. Of course this will yet again lead to absolutely meaningless estimates, because no one knows what they’re doing. A software project should never be lead where no one knows what they’re doing. There needs to be atleast several with a vested stake and understanding of a paradigm.

        • ardalis

          We agree on all of these things, but I think you’re being optimistic if you think it’s rare that real projects are being built in real-time (not as prototypes) by developers who are learning new things as they go. That’s the norm, not the exception, even if it’s not the ideal.

    • Imigo

      “Just because software is hard to estimate properly doesn’t mean it can’t be done.”

      I’m not sure that we read the same post if that was the takeaway that you got from it, or even if you interpreted anything in the post that suggested that estimating software can’t be done. My takeaway was that estimations are something that are necessary, inaccurate (although less so over time), and take time away from getting stuff done. Even with the best practices for estimation, a pragmatist will take any estimate with a grain of salt and make decisions knowing what they don’t know.

  • Sanjay

    forget customers, forget stakeholders.. The question is, should a developer still estimate? Should an architect still estimate? Is there a need for these 2 to estimate other than the management/sales need? I think such thinking can make us better developers or achitects. And also feels that if we are determined(as developers and architects) then it is possible to estimate correctly to the minute. Imagine an architect making a statement. “With my framework I can finish this project in x days with people from Africa. If we use people from Bulgaria then in y days. If we use another framework then these many days changes… etc.” What would you do to make such a bold statement? Goodhart’s Law: is the favourite part in this beautiful article. And feels that such discussion should be had between developers more often. And yes we cannot ignore estimation

    • ardalis

      If we forget customers and stakeholders, then what value is produced by the estimates of developers/architects, in your scenario? Who, other than the stakeholders, needs the estimate?

      “if we are determined then it is possible to estimate correctly to the minute” – really? The problem is just that people aren’t trying hard enough, not that the future is uncertain and current knowledge is imperfect? That doesn’t map to my own experience, but I’d be interested in hearing more about this (and while we’re at it, can you estimate accurately to the dollar a few stocks for me, too?).

      • Sanjay

        Ofcourse estimation is done for the business. The reason why I told about forgetting it is to point in other directions. such as the benefit of estimates for developers are architects(this is a bonus benefit in addition to the business benefit). So if we forget the business need( just for 5 mins 🙂 ) and move in the above mentioned direction where the architect making that bold statement.. we would see that such statements becomes a necessity if the architect himself owns the company. There I would link estimation to ‘resposibility’ and there comes the link to business need as well.

  • Very nice article, thank you. Having my own small business and being the primary developer, I can say that in my experience this is all true. Most, if not all, of the estimates I have done compared to when the project was finished ended up being very different, hence rendering the original estimate practically meaningless. The ONLY time my estimates were near truth was when they were SMALL projects, say a day or two of work. Larger projects changed 100% of the time, most of the time for the better. I.e. customer realizes a new potential or idea and wants it added. There were some references in the comments about using past experiences for estimation. I found that this does not work much. Most of the projects were always using some new technologies, methods, versions, etc. that we did not have direct past experience with. Thanks again.

  • Manish Kumar

    Great article. I have actually built a tool to help with software estimation.

    – quickfpa.com

  • Andrew Siemer

    Certainly a topic of passion/religion. Akin to testing I would guess. Some folks are still against testing or think devs shouldn’t be required to do it. While I agree with all your points about estimating. This article doesn’t describe when it is appropriate to do testing – and there are times where estimates add value. If someone wants to plan which of the 5 features to do – and only one can be done – it might be important for the business to know how expensive or how complex a feature might be. Some level of design and estimation will provide value. Or in the case of a large product with a large engineering team, building a product road map might be of use. But even that would require some level of estimation/design/planning. Again knowing that all of your points are accurate – estimation shouldn’t be a random or frequent interruption to the dev team. Instead it can be a planned activity (do it once a sprint) or it can be a queued activity (do it when you get a chance). And an estimate shouldn’t be done around notions of time – but rather complexity (tshirt sizing or story points are useful). This helps get the developer away from the evils of “you said it would take 5 days…its day 6 WTF?”. Velocity is a fun concept in this realm. Done right – estimates can be helpful and useful…if done appropriately. And not all dev activities have to be valuable to the devs. Estimates can be valuable to the business. If we are all focused on delivering business value then some evils like estimates might be necessary from time to time.

  • Dave Van den Eynde

    So when is ASP.NET Core RTM?