Evaluation – What percentage of your budget should you spend?

“The math is simple: if an evaluation whose cost is equal to 5 per cent of the grants under review delivers results that are 10 per cent better as a result of the evaluative information, it is a worthwhile expenditure” Hewlett Foundation (2014)

Throughout the different roles and projects I have worked on, the question of how much an organisation should be spending on measuring their impact or on evaluation often comes up. The answer is simple but at the same time complex.

The simple version can be expressed in two ways, depending on the situation. Scenario one is where you tell the client or colleague it is a fixed percentage of their project cost, generally ranging in the 5-10% of their total program cost. In this case, they will do the maths in their head, and if it’s a small budget they will think it sounds reasonable but unaffordable. If it’s a large-scale program with high costs, they will rapidly start losing colour in their face.

The complicated version is to explain that measuring or valuing your impact in any sector should be driving continuous improvement in the delivery of what you do, therefore, you should be heavily investing in it. I say “should”, because the reality is that getting internal stakeholders or external clients to go from “should” invest, to “do” invest, isn’t very successful.

So what do organisations spend on evaluation?

A report by the Hewlett Foundation into their own evaluation spending surveyed similar foundations to see what they were spending on evaluation. They found similar peer organisations spent between 1.5% and 7.5% of program budget on evaluation.

In Australia, Government of Western Australia states that many organisations set a certain percentage of total program budget for evaluation and the rule of thumb ranges from 5 per cent to 20 per cent of program costs. It recommends high investment programs to consider quarantining an evaluation budget between 5 and 20 per cent of total program costs.

According to Americorps, through a meta-analysis of evaluations carried out from a federal competitive grant program in the US, they looked at the rule of thumb of 5-10% and concluded that it is too simplistic.

In fact, on average they found programs to be spending on average 19% and a median of 15% of their annual program budget on evaluation. Even in the case of non-experimental designs, they spent on average 16% and a median of 14% of their annual program budget.

Americorps concluded the rule of thumb leads to under-budgeting, and the cost of evaluation rises with the level of evidence you aim for, with RCTs being the most expensive. It’s a mind-blowing analysis with many take away messages. One of the most striking insights is so obvious but also very rarely provided in such simple terms – “Determine what you want to learn from your evaluation and build a budget that can answer those research questions”

How to get the right level of investment in evaluation budgets?

If we are to get consistent levels of investment in impact measurement happening across different sectors in Australia, we need established organisations to look long-term, with 5 to 10 year measurement plans. Smaller organisations should look to start small and scale their impact measurement if, and when, they see the returns from using the results.

The ASVB can assist organisations with measuring their impact in a cost-effective way. We support our subscribers to carry out robust measurement and reporting of their social value. Please get in touch if you would like to know more about the ASVB or would like to attend our next introductory webinar.