Equal merit and the margin of error
26 June 2013
The Judicial Appointments Commission is consulting about using their new powers to choose between candidates of equal merit by preferring the candidate who will increase diversity in the office for which they are selecting.1
What’s interesting about this is the way they propose to do this.
The JAC selects on five ‘qualities and abilities’: intellectual capacity; personal qualities; ability to understand and deal fairly; authority and communication skills and efficiency. These are normally evaluated over three assessments: a written test; a role-play and an interview.
This process will produce a numerical score, which allows the JAC to rank the candidates. They then recommend the appointment of the candidates with the highest scores until the open posts are filled.
What they propose to do is to identify a zone, close to the cut off point, where the candidates are of ‘equal merit’ despite the difference in scores. Some of these candidates will be above the cut off point – and therefore would normally be appointed. But some would be below – and so normally would not be appointed. Having decided that this group is of equal merit, the JAC will select candidates on the basis of improving diversity.
This is interesting because it is based on the idea that the JAC’s scoring isn’t perfectly accurate. A candidate scored 65 may be better than a candidate scored 68.2 There is, the JAC is suggesting, a margin of error. Or at least a margin where the difference, if any, between the candidates is immaterial.
I think this is a welcome bit of common sense.
Properly scored recruitments, where a good person specification is drawn up and then candidates are objectively assessed against it, are very good things. Done properly, they give you the best chance of selecting the right candidate and the best chance of avoiding subconscious discrimination.
But there is a risk of falling into spurious accuracy. Selection methods are imperfect. Even if they weren’t, they can only measure a thin slice of an individual; their performance on a particular day. And, even if neither of these things were true, no personal specification can fully list and weigh the characteristics needed for a post.
So I’m sure that that JAC is right that there is a zone in many of their recruitment exercises where the candidates’ merit is indistinguishable, notwithstanding differences in their scores.
That being the case, even as someone profoundly sceptical about the merits of any form of positive discrimination or affirmative action, I can see the merits of using this zone to shade judicial appointments towards achieving a more diverse judiciary.
But, somewhat counterintuitively, I also think the JAC must strive to limit the effect of this proposal. They must, as they no doubt do now, try to ensure that their selection methods are as accurate as possible. This will inevitably reduce the size of the relevant zone.