3.11 Scope, findings and learnings from previous NSPS/ NSPP evaluations

Elements of the NSPS and the NSPP have been evaluated in recent years. An overview of these evaluations is provided below and the implications of these findings for this Evaluation are discussed in later chapters.

3.11.1 Learnings from Suicide Prevention Initiatives Project evaluation

The Learnings from Suicide Prevention Initiatives Project evaluated the first wave of NSPP-funded suicide prevention projects (to December 2005).²⁵ The project involved a desktop review of the 156 funded projects.²⁶ The evaluators found that the projects reached a broad range of target groups in a range of settings, and employed a range of approaches. The projects achieved improvements in knowledge about risk and protective factors for suicide, social connectedness and mental health literacy, and reductions in depressive symptomatology.

Factors considered important in project success included: understanding contextual factors, investigating participants' needs, drawing on sound evidence, developing multi-faceted strategies, garnering stakeholder support and employing capable staff. Projects' sustainability was found to be constrained by their short-term funding.²⁷ A key recommendation was that processes to promote project evaluation should be strengthened, ideally through a common evaluation framework, to enable more rigorous assessment of the effectiveness of the NSPP. The authors also called for improved communication between projects to provide a forum for applying learnings.

The Learnings from Suicide Prevention Initiatives Project authors highlighted a number of key challenges to their evaluation that meant their findings needed to be interpreted with caution. These included:

• Variability in report quality, sub-optimally designed evaluations and reliance on largely qualitative data sources restricted the range of analysis possible, which made comparisons between projects impossible, and limited the development of conclusions
• Consultations with project representatives and state/territory-based DoHA personnel were outside the scope of their evaluation. The authors reported that such consultations would have provided insights to barriers and enablers to project implementation at project level and would have clarified the overarching approaches to implementation and evaluation in each jurisdiction. Important contextual information was absent from their evaluation as a result.

3.11.2 Evaluation of the National Suicide Prevention Strategy

The Evaluation of the National Suicide Prevention Strategy – Final Report²⁸ was also produced in 2006. This report provided a high-level evaluation of the NSPS as a whole, including its governance structures and administration, as well as a review of the appropriateness, effectiveness and efficiency of the funded projects. The evaluation report concluded that while the NSPS is widely supported and perceived as an appropriate and necessary strategy that addresses an ongoing community need, there was scope to further refine its governance structures and processes.

In relation to the effectiveness of specific projects, the report concluded that while some gains were made in terms of capacity building (at an individual and service level), help-seeking, referral, and risk and protective factor profiles, there was little evidence available to indicate whether any NSPS-funded project had led to reductions in suicide or self-harming behaviour. Consistent with the Learnings from Suicide Prevention Initiatives Project, the authors highlighted the need for a stronger evaluation framework to enable more rigorous evaluation of the effectiveness of the NSPS and its funded projects.

Although the methodology for this evaluation included stakeholder consultations and written submissions in addition to a review of project documentation, the authors identified a number of methodological challenges including:

• The small number of completed national and 'cluster' evaluations of NSPS projects available for review ('cluster' evaluations in this context refers to groups of community-based projects in a particular state or territory)
• The limited corporate knowledge regarding the life of the NSPS among certain stakeholders
• The limited availability of outcome data concerning the community-based projects.

²⁵ A Headey, J Pirkis & B Merner, The 'Learnings from Suicide Prevention Initiatives' Project Final Report, University of Melbourne Program Evaluation Unit, Melbourne, 2006.
²⁶ Some of these 156 projects have continued to operate during the timeframes for the current evaluation; others had been completed by 2005; and others were completed subsequently.
²⁷ A Headey, J Pirkis, B Merner et al, 'A review of 156 local projects funded under Australia's National Suicide Prevention Strategy: Overview and lessons learned', Australian e-Journal for the Advancement of Mental Health, vol 5, no 3, December 2006, pp.247-261.
²⁸ Department of Health and Ageing, Evaluation of the National Suicide Prevention Strategy – Summary Report, prepared by Urbis Keys Young, Sydney, 2006.