Practical Thoughts on Forecasting, Planning and Budgeting
Copyright Alistair Davidson, 2009. All rights reserved. Alistair Davidson is a strategic consultant with extensive experience in developing budgeting, strategic planning and business intelligence systems. He is a contributing editor to Strategy and Leadership magazine, author of three books on strategy and technology. His career includes developing numerous software planning systems and tools, in addition to facilitating business and IT strategies.
Contact information: alistair@eclicktick.com, +1-650-450-9011

Executive Summary
Developing a forecast or budget sounds like it should be a simple task.
It rarely is. A planning and budgeting process is often used for multiple purposes and has many stakeholders, each with his/her own interests, constraints, risk profiles, deadlines and commitment to the process. Often budgeting processes become a minimalist exercise that reflects the power of the CFO rather than a process that is useful to strategic thinking and strategic management in the organization. There is nothing worse than a budget developed six months prior to the beginning of the fiscal year that is out of date before the year has begun.

Developing a forecast or a budget is typically an iterative process. The number of iterations within a particular year will affect the perception of the value of the process (more iterations typically annoys people). Just as importantly, forecasts and budgets are exercises that will repeat typically at least quarterly and annually. They will always evolve over time.


Developing a successful forecast or budget requires five elements for success:

1. Senior sponsorship is required to ensure that cooperation is obtained from all participants.

2. The level of work imposed or solicited needs to be perceived as reasonable and appropriate, often a difficult task in a changing environment without good supporting technology.

3. Senior management should avoid using the forecasting process to educate themselves and causing significant workload on the numerous participants. This is best achieved by spending time up front to understand the workload caused by a forecast exercise.

4. A longer term perspective on the forecast should be addressed up front by analyzing the workload and number of pieces of data being developed for each forecast in the current and future periods. Perfect systems should be avoided and iteration anticipated.

5. The forecast should be strategic and focus upon critical issues rather than letting the apparent simplicity of spreadsheets drive linear extrapolations of past performance. Forecasts should explicitly address uncertainty and risk, capacity utilization and the process of forecast revision over the forecasting period.

Introduction: Why Is Forecasting Difficult?
Planning and budgeting (P&B)are basic tools in managing a business. Budgets are often used to control spending and set expectations. Forecasts are often thought of as revisions to budgets done on a periodic basis to make sure that the budget is compared based on more recent data. Forecasts often extend beyond the budget period as well. If you are doing a quarterly update, there is little point in a cycle of 9 month, 6 month and 3 month projections, it's often equally as easy to do a rolling 12 month forecast.


Forecasting is often a problem for a companies because it serves multiple purposes and the importance of these multiple purposes looks different to different stakeholders throughout the organization. In an ideal world, the designer of a P&B process would identify the key stakeholders (i.e. people or groups that make the project a success or block it) and identify their minimum requirements for the P&B project's success. Generally speaking a stakeholder will have a threshold that must be delivered for satisfaction and other elements which they want emphasized a great deal.


Let's examine the stakeholders for the process:

1. The Board of Directors typically expects a forecast so that they can determine the future performance of the company. Their bias is typically towards financial results and to a lesser extent risk management. Strategic issues are often downplayed particularly in diversified companies with many different business.

2. Senior management in the company often uses the budgeting and forecasting process to force the organization to make choices about allocation of capital budgets, operating budgets, new product development processes, sales and marketing efforts. Annoyingly for the developer of forecasts and budgets, the projections are often used as a way of educating senior management about the business leading to multiple iterations.

3. Middle management and financial functions in the organization often use performance against the budget and forecast to revise their activities to meet their goals and performance metrics. Just as importantly, they use revisions to the budgets to prepare other stakeholders for surprises. Failure to deliver a budget is frequently less of a problem than the problem of surprising a boss with a failure to meet a budget. The budgeting and forecasting process is often seen as a chore rather than a productive use of time.


A very common problem for managers is the design of and successful implementation of a planning and budgeting process. There are several key constraints that make the design process difficult.


First, most managers undertaking a P&B process have little experience in their design and consistently underestimate the effort involved.


Second, P&B issues change constantly. If the CEO or profit center owner decides something is important, it is likely to be dropped into a P&B process at the last moment causing significant work load to the implementers.


Third, planning systems often draw upon information in other systems and formal migration processes for designing, testing, rolling out and training users is weak. Integration issues and business intelligence issues are surprisingly rarely addressed even though the information used in a good P&B system is very valuable to many users.


Fourth, planning tools are often dictated by functional areas for their own benefit and don't consider other users.


Fifth, budgets for P&B systems are generally small. Custom work has to be shoehorned into short time periods without regard to the actual work required.


Sixth, year over year comparisons are often extremely difficult because the planning system varies from year to year. The consistent information is often not the most important information for operating managers.


Seventh, disagreement about the scope of planning is very common. Planning is a bit like the elephant in the apocryphal story about the six blind men and the elephant. It feels very different depending upon where you are relative to the elephant.


Diagnosing Your Forecasting Problems
Before you can improve your forecasting (and often before you can obtain cooperation from stakeholders), it's typically a good idea to seek to understand from stakeholders what past problems have occurred and what needs are going unmet.



Typical problems might include:

1. Poor accuracy in the budget or forecast
2. Time wasting revisions to the P&B data as stakeholder requests evolve.
3. Excessive manual work.
4. Irrelevance after the process has been completed due to the length of time taken, iterative approvals or changes in the environment subsequent to the development and approval.
5. Inappropriate levels of granularity i.e. budgeting at too detailed or insufficient detail (which we might refer to as the Goldilocks problem)
For each of these five problems, different solutions are available. The resulting P&B efforts and focus are likely to look different.


The Multi- Period Problem
Perhaps the most important problem with P&B systems is that they tend to be focused upon financial requirements and not upon customers. Consider the situation of a customer that buys a pilot project that if successful could lead to 10-100X more volume of purchases and significant profitability. However, if the subsequent profitable purchase falls outside the budget or forecast period, resource allocation decisions may cause underinvestment in developing pilot projects.


Customer profitability is rarely well represented by a single period view of a customer. P&B systems that only look at single year profitability may be missing the most important drivers of overall profitability. A software company that only looks at financial results can show revenue growth while in decline. If a healthy number of new customers are not being acquired, maintenance revenues may continue to increase but the competitive position may be deteriorating.


If the purpose of forecasting is to be useful, then the information base for the forecasting should be useful, relevant and flexibly capable of being modeled and changed.


The Multi-Forecast Problem
A common problem with P&B systems is that budgets and forecasts get revised. In theory, it should be easy to compare a budget with a revised forecast that is updated every three months. However, it becomes exponentially hard if the data is driven off detailed data and the detailed data does not exist on a quarterly basis e.g. a total market size number that is only revised annually by a tech consulting firm or if the forecasts have been inconsistent in their use of bottoms-up and top-down forecasting.


There are no easy solutions to this particular problem.


The Implicit Strategic Decision Problem
A common problem in forecasting processes is the hidden assumption that the forecasting process should force managers or enable divisional or head office managers to make resource allocation decisions while developing a forecast.
The challenge here is that budgeting and forecasting systems tend to be linear extrapolations of the past. Innovations, in contrast, are often harder to forecast and have higher uncertainty. As a result, plans and forecasts presented in a spreadsheet often seem to imply that each line of the forecast is equally certain and reliable.



The truth is typically otherwise.


And the more detailed the forecast, the more likely it is that each line will produce a variance in subsequent quarters. The solution is typically thought to be focusing at a high level of forecast that irrelevant or immaterial variances don't show up.


Infrastructure Issues

P&B systems typically cause the creation of spreadsheets and PowerPoint slideshows. More sophisticated systems have been developed for consolidating hierarchical budgets and forecasts. These typically fall into two categories - multidimensional spreadsheet or OLAP tools and relational databases. Because these tools are typically difficult to use, the reality is that most work is done in spreadsheets and then uploaded to consolidation tools.


However, it is not clear that these consolidation tools are helpful in the process of developing a forecast. Without going into the mechanics of building up a consolidation or reconciling a top down goal setting with a bottom up forecast, let's just say that it is messy and complicated. The result is that forecasts are often painful to develop and inaccurate.


Dealing with High Uncertainty - Scenarios vs. Business Success
In highly uncertain environments, there are four basic approaches to forecasting.


Multiple Forecasts
The approach used most often is develop multiple forecasts. A high, medium and low forecast is the most common version seen. A more sophisticated approach is to develop a probability weighted set of forecasts where each forecast is assigned a probability. Each forecast then results in an expected value that adjusts the revenue forecast by the probability of revenues. Probabilities add to up to 100% so that the sum of the expected values is the best estimate of revenues. The unattractive side of this approach is that while it is useful for revenue forecasting, it is less useful for many expenses as they will be adjusted to reflect different revenue cases. Another weakness is that research suggests that most managers underestimate variability and risk.


Monte Carlo or Stochastic Modeling
Spreadsheet based forecasts are typically characterized as deterministic. Monte Carlo modeling involves a more detailed approach to uncertainty. Each line in a forecast is mapped to a potential probability distribution (e.g. a normal or power curve distribution). A simulation is run hundreds or thousands of times to see the overall outcome distribution with each line item varying according to its probability distribution each simulation run. Because of the number of runs made and the need to map a distribution to each item in the forecast, more detailed consolidations are typically avoided in smaller companies and forecasts are done at a fairly high level of summary.


Scenarios
Multiple forecasts are often erroneously called scenarios. But more properly, a scenario is a term reserved for naming and describing a future in which the organization might have to operate. Scenarios represents ways of stretching the thinking of the organization so that the organizations anticipates the impact of a proposed strategy across a more diverse set of potential environments.


Resource Allocation Frameworks
Many organizations address resource allocation decisions separately from P&B processes. They may make resource allocation decisions within the P&B process, but they separate out earlier stages of resource allocation. Often these systems are characterized as new product processes, productizing processes (in service organizations where services migrate from custom to standard) and capital budgeting processes, where a limited pool of capital is allocatable to major investment projects via a formal approval process.


Thinking Outside the Box - What are you not measuring and not forecasting
It may seem strange to think about what you are not forecasting but well managed companies should look at what they are not addressing and the larger issue of how well an organization is doing in the overall environment.


There are several ways of thinking about this problem.


Some companies look at share of wallet or share of expenditures. They measure their forecasts as a percent of total spending by a group of customers. When forecasts are looked at in this light, companies can understand to what extent they are not obtaining revenues and profits from areas of the market where they have blind spots.


Another key measure that is important to look at is what is your addressable market vs. the total market. A forecast can often look good in the context of your addressable market. However, if competitors are making in-roads into the total market, you business may be in decline without you realizing it.


Benchmarked performance. International companies often have difficulty comparing markets. Having a benchmark allows for comparisons of performance that are independent of pricing, cost inputs and currency fluctuations. The publisher, Harlequin, used to look at books sold per thousand women over 18 per year as a measure of market penetration and maturity.


Capacity Management and Forecasting
Many organizations do a good job of forecasting revenues and expenses, but do a bad job of forecasting capacity and utilization. There are many reasons for this bias. It's difficult to forecast lagged variables such as hiring a person, training them and make them effective or processes whose cycle times are longer than the cycles times of marketing and sales. And developing people and capacity often needs to be done in advance of obtaining sales leading to a perception of risk, both in terms of internal evaluations of performance and financial outcomes.


The key task here is to make sure that assumptions are explicit rather than implicit and that the organization is committed to the marketing and sales objectives. By having clarity and consensus better decision are likely to be made and delivery failures are less likely.


Explicit decisions on training part time people can also be pursued to manage peak loading and capacity problems.

Summary
Planning and budgeting systems are more complex to design, specify and obtain compliance with than most managers anticipate. Making sure that sponsorship, scoping, evolution and links to the rest of the organization are considered early in the process will increase the likelihood of success.