2.2.1.Survey Score

Each participant response (10 cases per survey) is classified on the basis to which the response is considered to match a "target diagnosis". The assessments for each case are classified in descending order into the following categories where “Concordant” is the optimal result followed by “Minor discordance”, “Differential diagnosis”, “Discordant” and “Unable to be assessed” or “No submission received”. Within each of these five categories the median value is determined and termed midpoint value.

The purpose of adopting this approach was to reduce to a minimum any subjective element in assessment and making each midpoint value a consequence of the performance in relation to all members of the peer group enrolled for the relevant survey.

Figure 1, for a hypothetical case, graphically shows the distribution of responses for each assessment category and the resulting mid‐point values. For example, for results classified as "a minor discordance", responses occupied in the region from 60% to 40% of the total proportion, received the midpoint value of 50%.


Figure 1. Mid‐point values
For each survey the sum of the midpoint values for each case is determined (e.g. for each of the 10 cases in one survey in the General and Breast modules). This sum of mid‐point values for each participant, is then expressed as a percentage of the maximum attainable score (i.e. the sum of the midpoint values for the cases where the assessment is "concordant"), in that survey.

The mid‐point value in each assessment category, for each case reflects the relative difficulty of each case. As the degree of difficulty increases fewer participants are assessed as concordant and the midpoint value adjusts upwardly in all categories. Discordant assessments in more difficult cases will obtain a higher mid‐point value than those in straightforward cases. The size of the gap between the 15 mid‐point value for a concordant assessment and for a discordant assessment could be quite variable and will be influenced by the number of participants assigned to the other assessment categories. Also of note is that, partial submissions including “Unable to be assessed” and “No submission received” assessments (i.e. case nonparticipation) are assessed below "discordant", will adversely affect the survey score and can be minimised with complete participation.