13.3 Effectiveness

13.3.1 Outcomes and achievements
13.3.2 Effectiveness: enablers and barriers

13.3.1 Outcomes and achievements

Assessing the effectiveness of NSPP activities was hampered by a general absence of quantifiable outcome measurement by the projects. Routine progress reports submitted by funded organisations were largely based on quantitative output and financial data, with narrative self-report used to describe the effects of activities. Outcome measurement involving validated tools has been rare among NSPP-funded activities. Even in cases where independent external evaluations had been undertaken; most reported on the achievement of project objectives rather than on short, medium or long-term outcomes. This issue is not unique to the NSPP and has been a challenge for suicide prevention activities throughout Australia and internationally.

The dearth of validated and standardised tools limited the extent of comparison that could be made between NSPP-funded projects engaged in similar activities.

Based on self-reported assessments, most projects, including those in their infancy, reported having achieved their objectives. A wide range of project achievements were cited as a result of a diverse range of activities. The MDS data identified that 16,222 individual client contacts/activities and 2,428 group activities occurred, over the six months to March 2013.

The LIFE Framework lists a number of LIFE Action Areas that describe the intended effect of the NSPS. Achievements related to these LIFE Action Areas were assessed using documentation/reports and survey responses from the funded organisations. Based on this data, self-reported achievements were demonstrated across the full range of LIFE Action Areas, particularly in relation to:

• Improving understanding of imminent risk and how best to intervene, particularly through gatekeeper training and community awareness approaches
• Improving access to support for people at risk of suicide and, in some cases, improving the knowledge, attitudes and help-seeking behaviours of those at high risk
• Improving community strength through capacity-building approaches, particularly for some well-defined target populations
• Providing information about suicide prevention
• Improving the profile of risk and protective factors at the individual level.

Positive unintended outcomes of the projects included positive reciprocity and expansion of project reach or goals. A negative unintended outcome was that some individuals felt that accessing services led them to be stigmatised in the community.

Although significant achievements have been identified, it should be noted that it is not possible to determine the extent to which the NSPP-funded activities have impacted on rates of suicide.

The DoHA staff who administer the NSPP-funded projects spoke positively about the achievements of the projects; however they also expressed concern that the existing reporting mechanisms (progress reports, final reports) did not adequately capture information about project outcomes and impacts. Furthermore, it was noted that data generated through the NSPP has not been made available in the public domain or to funded projects.

The documentation, reports and survey responses submitted by funded organisations indicate areas with scope for improvement. These areas include:

• Limited opportunities exist for funded organisations to share strategies and best practice.
• There was little evidence that regionally integrated approaches were operating.
• The ability to achieve long-term, structural change was beyond the scope of many projects. Many projects reported that this partly due to the short-term nature of NSPP funding.
• Many project representatives expressed a desire for greater support to evaluate their activities.
• There was limited access to information and data about suicide prevention activities. Top of page

13.3.2 Effectiveness: Enablers and barriers

Project representatives identified a number of enablers that contributed to the success of projects. These included:

• Strong, effective relationships between service providers and a range of other community stakeholders
• Strong relationships within project teams
• Recruiting the right staff and providing them with adequate support.

Key barriers to project effectiveness included:

• Funding limitations (amount and duration)
• Difficulties recruiting and retaining staff
• Sub-optimal partnerships and relationships (and the amount of time invested in these)
• Difficulties reaching and engaging the project's target populations, including a number of specific challenges for Aboriginal and Torres Strait Islander populations and rural and remote populations
• Difficulties collecting data

In addition, projects reported that a number of project-specific design issues, and suboptimal data collection and evaluation had limited their ability to measure effectiveness.

Project representatives made a number of suggestions for improving effectiveness, including:

• Increasing funding amounts and periods
• Improving collaboration with, and coordination between, funded organisations
• Providing support for organisations to improve capabilities in project development and evaluation