The causal inference model presented in Figure 12-1 identifies four domains (types) of project outputs:
- Provision and consolidation of suicide prevention information resources and research
- Direct individual client services to high-risk groups
- Community focused services/activities for people at risk of suicide
- Training in recognising and responding to risk for front line workers, families and community members, and workforce development
- Information resources, research and information provision (Figure 12-2)
- Individual client services (Figure 12-3)
- Community focused services (Figure 12-4)
- Training (Figure 12-5)
Output domains rather than LIFE Action Areas are used as the basis of these causal inference models because, from a project perspective, these output domains are more applied and less conceptual. As a result, projects can more easily identify with the output domains that relate to the modes of activities they provide.
Consistent with best practice advocated by the realist evaluation approach, a range of timeframes and tools/data are suggested in the following four models (Figure 12-2 to Figure 12-5). These include a mix of quantitative and qualitative measures that are designed to unpack 'the long sequences of steps before the outcome' 112 while at the same time ensuring the validity and reliability of the data collected.113 The qualitative component is essential to understanding lived experiences and the context-specific elements that serve as mediating factors in achieving outcomes.
Figure 12-2: Causal mechanism model: Information resources, research and information provision
Text version of figure 12-2
This diagram of the Causal Mechanism Model applies to Information resources, research and information provision. The model has two sequential parts: monitoring and evaluation. Monitoring shows inputs and outputs, and evaluation shows outcomes over time. Examples of possible data sources for monitoring and evaluating each part are provided.Monitoring addresses the question: "Did interventions or program deliver what they said they would?" and involves reviewing the inputs and outputs.
Inputs refer to project logistics such as funding, staff and supportive mechanisms. Possible data sources include funding agreements, progress reports, and the MDS.
Outputs refer to activities that stem from inputs, such as research papers and presentations, information resources, and policy/strategy development. Possible data sources include the number of publications, number of presentations, and number of information resources developed.
Evaluation addresses the question: "What were the outcomes of these programs?" Program outcomes are shown as a continuum to reflect progression over time, starting with proximal outcomes and ending with distal outcomes.
Initial proximal outcomes include: increased access to information, improved/expanded evidence base. Possible data sources include citation indexes, number of people attending presentations, and number of information resources distributed.
These initial proximal outcomes lead to improved understanding of epidemiology of suicide, improved understanding of what works in practice, and increased community understanding of suicide. Possible data sources include key informant interviews, and survey of information resource users.
These consequent outcomes lead to increased evidence based practice, and improved capacity to respond at tipping points and points of imminent risk.
Ultimately, this leads to two distal outcomes, namely reduced incidence of suicidal behaviour in terms of self-harm or suicide attempts, and reduced incidence of completed suicide. Possible data sources include ABS Data and state-based suicide registers where they exist. Top of page
Figure 12-3: Causal mechanism model: Individual client services
Text version of figure 12-3
This diagram of the Causal Mechanism Model applies to Individual Client Services. The model has two sequential parts: a process evaluation and an outcome evaluation. The process evaluation shows inputs and outputs. The Outcome Evaluation shows program outcomes over time. Examples of possible data sources for evaluating each part are provided.The process evaluation addresses the question: "Did the interventions or programs deliver what they said they would?" and involves reviewing the inputs and outputs.
Inputs refer to project logistics such as funding, staff, and supportive mechanisms. Possible data sources include funding agreements, progress reports, and the MDS.
Outputs refer to activities that stem from inputs, such as strong referral pathways and coordinated care, provision of an appropriate range of direct support and care services, engagement of high risk groups, and targeted suicide prevention activities in high risk groups. Possible data sources include client records (to establish the number of clients and compare client demographics to target groups), client satisfaction surveys, and the MDS.
The outcome evaluation addresses the question: "What were the outcomes of these programs?" Program outcomes are shown as a continuum to reflect progression over time, starting with proximal outcomes and ending with distal outcomes.
Initial proximal outcomes include: increased help seeking, decreased suicidal ideation, and improved quality of life or social and emotional well-being. Possible data sources include pre- and post-K10, pre- and post-quality of life measures, pre- and post-social and emotional well-being measures, and exit interviews.
These initial proximal outcomes lead to increased social connection and participation, as well as improved capacity to respond at tipping points and points of imminent risk. A possible data source is client follow-up surveys 12 months after the intervention, online or over the phone.
These consequent outcomes lead to improved individual resilience and well-being. Possible data sources include focus groups, network analysis, surveys, and interviews.
Ultimately, this leads to two key distal outcomes, namely reduced incidence of suicidal behaviour in terms of self-harm or suicide attempts, and reduced incidence of completed suicide. Possible data sources include ABS data and state-based suicide registers where they exist. Top of page
Figure 12-4: Causal mechanism model: Community focused activities
Text version of figure 12-4
This diagram of the Casual Mechanism Model applies to Community Focused Activities. In this diagram, "community" refers to either: the general community (whole of Australia) or a more specific geographic community; or a community of shared interest or culture.This model has two sequential parts: monitoring and evaluation. The process of monitoring shows inputs and outputs, and the process of evaluation shows program outcomes over time. Examples of possible data sources for evaluating each part are also provided.
The process monitoring addresses the question: "Did interventions or program deliver what they said they would?" and involves reviewing the inputs and outputs.
Inputs refer to project logistics such as funding, staff, and supportive mechanisms. Possible data sources include funding agreements, progress reports, and the MDS.
Outputs refer to activities that stem from inputs, such as community events/ activities. Possible data sources include event records (number of attendees and demographics compared to target), satisfaction surveys, and the MDS.
The outcome evaluation addresses the question: "What were the outcomes of these programs?" Program outcomes are shown as a continuum to reflect progression over time, starting with proximal outcomes and ending with distal outcomes.
Initial proximal outcomes include increased community awareness of what is needed to prevent suicide. Possible data sources include community survey (pre and post), and interviews with key informants/stakeholders.
These initial proximal outcomes lead to improved community strength and resilience, increased understanding of whole of community risk and protective factors and how best to build resilience of communities and individuals. A possible data source is a community survey (pre and post), for example a community readiness question interview.
These consequent outcomes lead to an environment that encourages and supports individual help seeking. A possible data source is interviews with key informants/stakeholders.
Ultimately, this leads to two key distal outcomes, namely reduced incidence of suicidal behaviour in terms of self-harm or suicide attempts, and reduced incidence of completed suicide. Possible data sources include ABS data and state-based suicide registers where they exist. Top of page
Figure 12-5: Causal mechanism model: training
Text version of figure 12-5
This diagram of the Causal Mechanism Model applies to Training. The model has two sequential parts: a process evaluation and an outcome evaluation. The process evaluation shows inputs and outputs. The outcome evaluation shows program outcomes over time. Examples of possible data sources for evaluating each part are also provided.The process evaluation addresses the question: "Did interventions or program deliver what they said they would?" and involves reviewing the inputs and outputs.
Inputs refer to project logistics such as funding, staff, and supportive mechanisms. Possible data sources include funding agreements, progress reports, and the MDS.
Outputs refer to activities that stem from inputs, such as provision and consolidation of suicide prevention resources and information, provision of evidence based training, and people completing training. Possible data sources include training content/curriculum, trainee records such as number of trainees and demographics compared to target, trainee satisfaction surveys, and the MDS.
The outcome evaluation addresses the question: "What were the outcomes of these programs?" Program outcomes are shown as a continuum to reflect progression over time, starting with proximal outcomes and ending with distal outcomes.
Initial proximal outcomes include changes in understanding, and changes in confidence. Possible data sources include pre and post training knowledge, and pre and post training confidence.
These initial proximal outcomes lead to the application of knowledge, staff roles post training, staff retention, and performance change. A possible data source is a survey of trainees 12 months after training.
These consequent outcomes lead to increased recognition of signs/symptoms and increased referrals, which then lead to increased help seeking, increased resilience, and increased awareness. Possible data sources in a survey of trainees 12 months after training, and a survey of beneficiaries or original trainees such as teachers, students, and workmates.
Ultimately, this leads to two key distal outcomes, namely reduced incidence of suicidal behaviour in terms of self-harm or suicide attempts, and reduced incidence of completed suicide. Possible data sources include ABS data and state based suicide registers where they exist.
112 Pawson & Tilley, Realistic Evaluation. Top of page
113 JW Cresswell & VL Plano Clark, Designing and Conducting Mixed Methods Research, 2nd edn, Sage Publications, Thousand Oaks, 2011.