1.5 Summary of findings

Key findings are outlined below, under the headings of appropriateness, effectiveness and efficiency. Findings are also presented regarding the position of the NSPP in Australia's suicide prevention efforts.

1.5.1 Appropriateness
1.5.2 Effectiveness: outcomes and achievements
1.5.3 Effectiveness: enablers and barriers
1.5.4 Efficiency
1.5.5 Positioning the NSPP in Australia's suicide prevention efforts

1.5.1 Appropriateness

This report demonstrates that NSPP-funded projects provide a range of activities across the LIFE Action Areas, using a mix of approaches and targeting a broad range of groups known to be at higher risk, as advocated in the LIFE Framework. Importantly, this mix not only occurred at state/territory level but also within individual projects.

Overall, project activities address most of the recognised target groups. Some gaps are evident at state/territory level in terms of the number of projects and the reported coverage of higher risk groups (Section 6.6.2). However, other non-NSPP-funded initiatives (which are not part of this Evaluation) may be filling these gaps.

A mix of universal, selective and indicated approaches was evident in project activities. A number of NSPP-funded projects used universal approaches to address media reporting of suicide and mental illness, awareness-raising and promotion of help-seeking.

Gatekeeper training and community capacity-building activities were among the selective approaches reported by the projects; however, there was considerable variation in the way these services were delivered across target groups and settings. While only one project targeted the knowledge and awareness of medical practitioners, there are a number of other initiatives that support GPs to better identify and refer suicidal patients to appropriate care. These include the ATAPS Suicide Prevention service initiative (see Chapter 11) and the Better Access to Psychiatrists, Psychologists and General Practitioners through the Medicare Benefits Schedule (Better Access) Program.

Several projects used indicated approaches which aimed to improve access to care and support pathways for people following suicide attempts, for example by improving transition from the emergency department to primary care or community mental health services.

Importantly, none of the NSPP-funded projects reported using activities or approaches that were identified in the peer review literature as potentially harmful. Survey responses from funded organisations indicated that research and evidence were used in project design and implementation for the majority of projects. The range of activities reported included a mix of innovative and established evidence-based activities in terms of target groups, settings and approaches.

NSPP project funding per capita varied considerably between jurisdictions. However, in general, jurisdictions with the lowest funding per capita were those with the lowest age-standardised suicide rate and those with the highest funding per capita were those with the highest age-standardised suicide rate. Jurisdictions with the greatest need (ie, highest age-standardised suicide rate) were therefore recipients of the highest funding per capita. Top of page

1.5.2 Effectiveness: outcomes and achievements

Effectiveness is defined as the extent to which an intervention or program produces desired or intended outcomes.¹ Assessing the effectiveness of NSPP activities was hampered by a general absence of quantifiable outcome measurement by NSPP-funded organisations.² As addressed in Chapter 10, outcome measurement is not something that funded organisations have engaged in to any great extent to date. This issue is not unique to the NSPP and has been a challenge for suicide prevention activities throughout Australia and internationally. Routine progress reports submitted by funded organisations were largely based on quantitative output and financial data, with narrative self-report used to describe the effects of activities. Outcome measurement involving validated tools has been rare among NSPP-funded activities. Even in cases where independent external evaluations had been undertaken, most reported on the achievement of project objectives rather than on short, medium or long-term outcomes.

The dearth of validated and standardised tools limited the extent of comparison that could be made between projects engaged in similar activities across the program.

Most projects reported having achieved their objectives. While a lack of outcome data made it difficult for projects to demonstrate their effectiveness, a diverse range of activities and a wide range of project achievements were cited. The MDS identified that 16,222 individual client contacts/activities and 2,428 group activities occurred over the six months to March 2013.

The LIFE Framework lists a number of LIFE Action Areas that describe the intended effect of the NSPS. Projects' achievements related to these LIFE Action Areas were assessed using documentation/reports and survey responses from funded organisations. Based on this data, self-reported achievements were demonstrated across the full range of LIFE Action Areas, particularly in relation to:

• Improved understanding of imminent risk and how best to intervene (particularly through gatekeeper training and community awareness approaches)
• Improved access to support for people at risk of suicide and, in some cases, improved knowledge, attitudes and help-seeking behaviours of those at high risk
• Improved community strength through capacity-building approaches, particularly for some well-defined target populations
• Provision of information about suicide prevention
• Improving the profile of risk and protective factors at the individual level

Although significant achievements have been identified, it should be noted that it is not possible to determine the extent to which the NSPP-funded activities have impacted on rates of suicide.

The documentation/reports and survey responses submitted by funded organisations indicate areas with scope for improvement. These areas include:

• Limited opportunities exist for funded organisations to share strategies/best practice
• There was little evidence that regionally integrated approaches were operating
• The ability to achieve long-term, structural change was beyond the scope of many projects. Many projects reported that this was partly due to the short-term nature of NSPP funding
• Many project representatives expressed a desire for greater support to evaluate their activities
• Limited access to information and data about suicide prevention activities. Top of page

1.5.3 Effectiveness: enablers and barriers

Project representatives identified several enablers that contributed to the success of projects. Relationship building between service providers and other stakeholders was reported to be vital but time-consuming and sometimes complicated by a lack of clarity around roles and responsibilities.

Recruiting and retaining appropriate staff was identified as important. Maintaining strong relationships within the project staff team and providing staff with adequate support was a high priority for many projects. Where difficulty was encountered with staff recruitment or retention, this presented a significant barrier. Most problems were reportedly due to the short-term nature of NSPP funding or insufficient funding to deliver the number, range, intensity or geographical coverage of services needed.

A further barrier to program effectiveness was difficulties experienced in engaging with some target groups due to:

• Competing priorities within settings such as schools and workplaces
• Social stigma relating to suicide which resulted in a reluctance to talk about suicide or seek help
• Time and distances required to attend meetings/appointments.

Project representatives reported that a number of project-specific design issues had impacted effectiveness, and that sub-optimal data collection and evaluation had limited their ability to measure effectiveness.

Project representatives were asked to provide suggestions for improving project effectiveness. The most commonly cited responses were:

• Improved collaboration with, and coordination between, funded organisations
• Providing support for organisations to improve capabilities in project development and evaluation
• Larger funding amounts and longer funding periods. Top of page

1.5.4 Efficiency

Efficiency was examined from an operational perspective, as follows:

• Analysis of the apparent cost efficiency of projects, by relating costs to outputs (ie, hours of service delivered) to calculate and compare average cost per hour.
• Sustainability of projects.
• Potential efficiency improvements, based on consultations with project representatives and the Department.

This analysis found that:

• The average cost per hour of service provision varies across projects
• Projects that provide relatively more hours of direct service provision (to individuals or groups) tend to have a lower cost per hour and hence appear to be more efficiently delivering services
• Projects that spend relatively more time on travel and event/activity planning, appear to have higher costs, ie, travel and event/activity planning appear to be key drivers of costs.

In relation to sustainability, more than half of the projects indicated that they receive no funding other than through the NSPP (Section 10.3.1). The vast majority of project representatives do not believe their project would be sustainable without continued NSPP funding.
Project representatives reported high levels of satisfaction with the level of communication and responsiveness of the DoHA officers responsible for the administration of their project. There were some suggestions for improving DoHA administration of projects, related primarily to improvements in contract management and data collection and storage.

STO and CO staff believed that the DoHA National Alignment (DNA) changes will lead to more efficient administration of the NSPP projects, however these benefits have not yet been realised.

The absence of quantifiable outcome data restricted not only the extent to which the effectiveness of the NSPP could be evaluated in this current report, but also the range of economic analysis that could be conducted. This highlights the need for a detailed independent economic assessment of the cost of suicide and attempted suicide in Australia in order to determine the economic benefit of prevention, to help inform future investment decisions. Top of page

1.5.5 Positioning the NSPP in Australia's suicide prevention efforts

The NSPP represents one component within a complex range of suicide prevention activities in Australia. While it was outside the scope of this Evaluation to map these in detail, a desktop review of two initiatives was undertaken; namely the ATAPS Suicide Prevention service initiative and the MindMatters initiative. MindMatters was a national mental health promotion program for secondary schools that addressed some of the risk and protective factors for suicide. Review of previous evaluation reports for these two initiatives indicated that:

• The ATAPS Suicide Prevention service initiative is an appropriate and effective suicide prevention intervention. The efficiency of the program has not been established due to a lack of data.
• MindMatters has had high levels of uptake and acceptance across Australian schools and appears to be an appropriate intervention. The evaluation reports produced to date (from 2006 to 2012) do not address the effectiveness or efficiency of the program.

Through interviews, stakeholders expressed a range of views regarding the positioning of the NSPP in Australia’s suicide prevention efforts. The following findings emerged:

• People working in the suicide prevention sector held mixed and sometimes confused views of what the NSPP is. Many did not see the NSPP as a distinct component of the Australian Government’s activity around suicide prevention, and several confused the NSPP with the NSPS or the LIFE Framework.
• Communication and leadership between DoHA, the jurisdictions, states and territories and the sector was seen as an area for improvement, to ensure the NSPP is integrated with other suicide prevention activities in Australia.
• Some stakeholders argued that suicide prevention is too strongly linked to a mental health agenda, at the expense of a broader social determinants approach.
• Stakeholders felt that most of the funded NSPP projects would not be able to continue in the absence of NSPP funding, and that the impact of this would be felt by service users at the local level.
• Stakeholders stressed the importance of a strong and continuing Australian Government commitment to suicide prevention.
• The concept of setting a national suicide reduction target was raised by several stakeholders; however details of what this target should be or how it should be set were not specified.

¹ J Davidson, 'Effectiveness', in S Mathison (ed), The Encyclopedia of Evaluation, Sage Publications, London, 2007, p.123.
² Outcomes include 'changes, results, and impacts that may be short or long term; proximal or distal; primary or secondary; intended or unintended; positive or negative; and singular, multiple, or hierarchical. Outcomes are enduring changes, in contrast to outputs, which are more specific.' S Mathison, 'Outcomes', in S Mathison (ed), The Encyclopedia of Evaluation, Sage Publications, London, 2007, p.288.