The Challenge of Effective Estimation: A Process for Quantifying the Impossible

Originally published in the PMI Government SIG Newsletter

If it were possible to define the single greatest challenge that faces a project manager on any project, the answer for most would likely be the difficulty of defining estimates that are actually in the ballpark of what a project will really cost. And a close runner up for second place is the challenge of convincing our customers to accept that number.

Part of the challenge with estimating projects is that there is no one answer of what it will take to conduct a project. On a recent project that I assisted a customer with estimating, for example, we modelled the likely cost of the project using a software estimation tool. Assuming a fixed scope of work, the range of project costs was modelled based upon a variation in project duration – from approximately 35 weeks in duration to nearly two years. While the scope of the project was a constant, the cost of the project increased almost 20% to accommodate the compressed schedule, the quality of the final deliverable declined by a similar factor, with a significantly larger number of resulting defects being projected to be delivered and the size of the team required to do the work of the project increased four-fold.

This example clearly illustrates the reality of the situation many project managers face: while they have an estimate which they feel to be the most realistic and effective based upon the approach that they have defined, they often come under pressure to do it in less time or for less money. In reality, it is altogether likely that it is possible to do the project in less time or less money. The problem is understanding and being able to quantify the impact of these choices.

Taking the example I cited earlier, the number of defects increased approximately 20%. The projected impact of these increases, however, was an almost doubling of the support costs for the resulting information system over the first two years of its life. A number of major or critical defects were likely to make it into the production release as a result of crashing the schedule. The impact on support cost of removing those defects increases significantly, with considerably greater effort required in post-implementation maintenance as a result. Not only does the project cost increase, so does the overall lifecycle cost.

Which leaves us with a significant question: if your customer understood the longer-term impact of doing a project for less time or less money, would they still ask for the reductions? Would the increased costs and reduced quality be justified by the increased time to deliver?

For many project managers, the reason we acquiesce to customer requests to reduce our estimates and schedules is because we lack the information to be able to effectively articulate the consequences of these choices. Lacking this information, we lack sufficient confidence in our estimates to be able to defend them from challenge. As a result, we often feel forced to accept decisions that we believe in our gut to be bad ones, but that we feel unable to logically oppose.

The following steps outline a strategy that will both help increase both the quality of the information that is available to us in estimating projects, and the confidence that we have in our resulting estimates:

• Understand what the customer really wants. At the end of the day, one of the greatest challenges in estimating projects is understanding what is truly expected of the project, and how project success will be evaluated. Taking the time to fully define and ensure agreement on requirements becomes an essential first step in ensuring that all of the required work is planned in order to be able to deliver the requirements.
• Know the customer’s driving priorities. The triangle of constraints isn’t just a communication tool to explain to the customer why they can’t control all aspects of the project. It is in fact an essential basis of defining a customer’s expectations and priorities. Looking at the three points of the triangle – cost, schedule and scope/quality – we can control one, attempt to manage a second and the third one must be allowed to shift. What is essential to understand up front is what these priorities are in the eyes of the customer: Is cost mandatory? Is time to market the second most important factor? Is scope or quality therefore the area that will be sacrificed where required? Knowing this information up front provides critical input into the actual strategy that will be applied in managing the project. The activities that are scheduled, the rigour with which they are performed, and the level of quality that will be assessed will be very different depending upon the priorities that the customer defines.
• Estimate the size of the team. It seems counter-intuitive to estimate the size of the team before actually estimating the effort of the project and scheduling it. The reality, however, is that the effort required to perform an activity in a 50-person project team is very different than for a 5-person project team. By having a rough understanding of team size, the additional effort associated with communications, review, feedback and management of expectations can be incorporated into the estimate.
• Estimate bottom-up effort in hours. Once the work of the project is defined using a Work Breakdown Structure (WBS), the most reliable estimate is one that considers the effort associated with each activity in turn. At this stage, the objective is to estimate the pure effort associated with the activity, in hours, assuming a person of average skill to be able to perform the required job (an average developer, an average analyst, an average brick-layer). We don’t estimate based upon skill factors, the specific person assigned, or contingency – just the pure work effort, which gives us an assumption against which we can later make adjustments.
• Adjust effort for specific resources, skills and other soft factors. Once the bottom-up estimate is defined, and the activity is resourced, we can then make the adjustments for skill and other soft factors. By making this a discrete step, we retain an understanding of our original assumptions about the individual activity. If we need to make changes to assigned resource, and assign someone with a greater or lesser skill level, it becomes much easier to evaluate the required change in estimate to accommodate the new resource. We are not comparing them against each other, but comparing each to our own standardized perception.
• Estimate duration using the standard day. With the effort estimates established, it is possible to develop an effective estimate of the length of time that it will take to complete each activity. If we assume a standard day of 5 hours, for example (which is not an unreasonable assumption where any specific information is unavailable) then a 50 hour estimate would have a duration of 10 days. By defining the duration of each activity, and understanding the underlying dependency logic of the project, we are able to estimate the overall duration of the project.
• Level resources to eliminate over-allocations. The first time we schedule a project, the answer is one we generally do not want to see – it takes too long, assumes heroic amounts of overtime, or requires five times as many resources as we have available. Levelling resources allows us to make choices – to lengthen activities, redefine deliverables, reassign resources or revise the expected quality – in order to meet the required envelopes of the project. And by knowing our driving priorities, we are able to select the most appropriate strategy for the project.
• Now, and only now, estimate cost. Once you have been through each of these steps, you are finally able to estimate cost. This is the essential final activity, as each previous step has the potential to seriously revise the required costs. By changing schedules, contemplating overtime, shifting resource assignments and delaying or extending activities, we can significantly impact the potential final cost of the project.

By adhering to this process, it is possible to both establish a significantly greater understanding of the work that needs to be conducted, and a much more refined estimate of what it will take to accomplish the work. We are also in a far better position to understand the remaining unknowns and the contingency required to manage them, and to defend the estimation decisions that we have made. While top-down estimates and the use of estimation tools can help to support this process, they are still only a basis of validation – they do not replace the process of logically defining and understanding what needs to happen.

The more information we have about our projects, the better we are able to define and manage the work. The greater our confidence in our definitions, the better we are able to defend our estimates and schedules. And the better our estimates and schedules, the greater the confidence of our customers and management in our ability to produce results.