12.1 Conceptualising outcome measurement

As outlined in Chapter 8, outcome measurement for suicide prevention is complex and multifaceted. In part, this reflects the range of factors that need to be considered in measuring outcomes, as illustrated by the following definition:

Outcomes are changes, results, and impacts that may be short or long term; proximal or distal; primary or secondary; intended or unintended; positive or negative; and singular, multiple, or hierarchical. Outcomes are enduring changes, in contrast to outputs, which are more specific.¹⁰⁶
In the context of evaluating effectiveness, outcome measurement plays a central role in addressing the following core questions posited earlier in Section 8:

Question 1: Did interventions or programs deliver what they said they would?
Question 2: What were the outcomes of these programs?
Figure 12-1 provides a causal mechanism model to explain how NSPP-funded activities are intended to lead to a reduction in completed suicides and demonstrates how these two questions fit into the evaluation continuum. Question 1 is primarily addressed by measuring inputs and outputs while Question 2 is addressed by measuring outcomes (proximal and distal). Note: This causal mechanism model is based on the conceptual framework that underpins the overarching program logic developed as part of the evaluation framework (see Appendix F).

In the causal mechanism model (Figure 12-1), Question 1 focuses on program/project inputs and outputs and thus falls primarily into the domain of process evaluation.¹⁰⁷ Question 2 specifically relates to outcome measurement. To date, evaluations of NSPP-funded projects have largely addressed Question 1. For most projects, therefore, outcome measurement represents a further and unfamiliar step in the evaluation continuum.

In the context of NSPP-funded projects, it is also worth noting that issues related to outcome measurement at project level are overlaid by the broader questions of what works and what is likely to work according to the evidence base. As outlined in Section 7.2, the evidence base for effective suicide prevention activities that answer these broad questions is small and growing. This is coupled with the fact that:

• Suicide prevention is a dynamic, constantly changing field that generates new questions that need to be addressed.
• There is often a time lag between innovation and evidence which contributes to significant gaps within the evidence base. The impact of social media on suicide is one such example.

Therefore, while NSPP-funded projects currently engage in a diverse range of activities that, overall, are consistent with current best practice as indicated by the literature review and the LIFE Framework (see chapter 5 and chapter 6), the evidence base on which these best practices have been developed is incomplete. Consequently, outcome measurement is not only an issue for individual projects but also for the broader suicide prevention research community.

Outcome measurement at project level has the potential to:

• Contribute to the evidence base by highlighting innovative initiatives that show promise and thus warrant further investigation by the research community
• Provide better understanding of the chain of effects related to short- and medium-term outcomes being achieved by projects
• Provide a translational research environment where outcome measurement tools and evidence-based findings can be explored in specific settings and contexts in Australia. This could include involvement in case-control studies, quasi-experimental evaluation designs and randomised control trials being undertaken by the research community.
• Provide ongoing quality control in service provision
• Identify initiatives that could potentially be expanded in terms of reach

To achieve this potential and bridge the evidence practice gaps, further collaboration and partnership is required between projects and the research community is required. Top of page

Figure 12-1: Overarching causal mechanism model

Text version of figure 12-1

The Overarching Causal Mechanism Model has two sequential parts: a process evaluation and an outcome evaluation. The process evaluation shows inputs and output domains. The outcome evaluation shows program outcomes over time.
The process evaluation addresses the question: "Did interventions or programs delivery what they said they would?" and involves reviewing inputs and outputs.

Inputs refer to project logistics such as funding, staff, and supportive mechanisms such as policies. Output domains refer to activities that stem from inputs, such as: provision and consolidation of suicide prevention resources, information and resources; direct client services to high risk individuals; community focused services/activities for groups at increased risk of suicide; and training in recognising and responding to risk for front line workers, families, community members and workforce development.

The outcome evaluation addresses the question: "What were the outcomes of these programs?" Program outcomes are shown on a continuum to reflect progression over time, starting with proximal outcomes and ending with distal outcomes.

Initial proximal outcomes lead to improved individual resilience and wellbeing, increased social connection and participation, increased community awareness, and improved community strength and resilience.

Ultimately, this leads to two key distal outcomes, namely reduced incidence of suicidal behaviour in terms of self-harm or suicide attempts, and reduced incidence of completed suicide.

¹⁰⁶ S Mathison, 'Outcomes'.
¹⁰⁷ PH Rossi, MW Lipsey & HE Freeman, Evaluation: A Systematic Approach, 7th edn., Sage Publications, London, 2004.