ASVB Value Calculator – Applying Multiple Outcomes

Choosing which outcomes to measure can be complex and one part of this decision process is whether you should be applying multiple outcomes. We provide subscribers to the ASVB Value Calculator with guidance on how to navigate this decision process. This guidance can be used by anyone struggling with the same dilemma in their impact measurement or evaluation approach.

Applying Mulitple Outcomes

It is possible to select up to three outcomes per program in the ASVB Value Calculator when your program directly affects three different parts of someone’s life. It is important that you apply multiple outcomes with care and attention, as one outcome might also be capturing the effect of another outcome and counting both would lead to “double-counting” and an overstatement of the program’s social impact. This is because the Wellbeing Value captures all of the associated effects on life satisfaction, positive and negative, created by a particular outcome.

In order to avoid double counting, we recommend that you think through the following principles. 

Principle 1: Which outcome is most relevant to your program?

You should choose the most relevant outcome by focusing on the aim of your project and should not let the relative sizes of values influence your choice. For an employment program, the most likely outcome is gaining employment. You should apply a value to an individual only once e.g. if someone gains a couple of casual jobs, you only apply one “obtained casual employment” value.

Principle 2: Can I add multiple values for the same participant?

To answer this, ask yourself: does the value of my most relevant outcome also capture the benefit of the outcome I would like to add?

The answer to this second question of whether the value is captured will be “yes” when a second or third outcome is a result of the first outcome. For example, if finding employment also leads to improved confidence, in this case the second outcome (improved confidence) is just a result of the first outcome (employment) and so you cannot add the value of confidence for individuals who achieve
employment.

The answer to this second question of whether the value is captured will be “no” when the different outcomes are as a result of a separate activity within the program, for example, when the creation of a social group for support creates long-lasting friendships whereby some participants “Meet friends regularly”. In this example of the job readiness program, the resulting outcome is “obtaining full-time employment”. In this case, it is possible to add this second value of “Meets friends regularly”.

Principle 3: Can I add multiple values for the program i.e. for different participants?

The answer will be “yes” where the participant did not achieve the most relevant outcome but did achieve another outcome. Staying with the job readiness example, it is possible to apply the confidence value for the participants who did not secure a job but who are more confident.

The answer will be “no” where the participant did not achieve another outcome. If the participants do not secure a job or become more confident, then you should not apply either the value for confidence or employment. You may consider a third outcome for the program (e.g. improved computer skills) and apply the value to those who improved their computer skills but did not increase their confidence or attain employment.

Principle 4: Can I add more than three outcomes to a program?

The short answer is “no”. The total number of outcomes claimed for a program should be no more than three. If you consider that the aim of the program incorporates more than three outcomes, then it may be that you’re thinking of the program too broadly. We consider a program to be a structured set of activities aimed at a particular goal, not the entire work of your organisation. If the program can be separated out, for example, running the social group and support for jobs readiness in the above example, then it may be worth doing so in order to compare these programs. If you do separate out the program, you will need to have a clear idea of separating out the costs. For example, the room hire and the entertainment for the social group can be considered quite separately to the tuition for job training.

Conclusion

If you follow the decision chart and work through the principles outlined above, you will have considered the major factors in deciding whether to apply multiple outcomes. You will have removed two of the major mistakes in impact measurement which are double-counting and over-claiming.

Double-counting: Double-counting of outcomes by including the same social impact more than once, in what erroneously seem to be different outcomes

Over-claiming:  Ensuring your identifying the difference that is made, and how much can be attributed back to your activity.

Learn more about how the ASVB can help?