IS THERE A SELECTION EFFECT IN CANADIAN ADMINISTRATIVE LAW?

• Author: Schafer, Devan
• Position: Response to article George L. Priest and Benjamin Klein, Journal of Legal Studies, vol. 13, p. 1, 1984

INTRODUCTION

In their seminal 1984 article "The Selection of Disputes for Litigation," George Priest and Benjamin Klein developed an economic model of civil litigation which led them to conclude that because of selection effects amongst litigants, no inferences could be drawn about the character of the law based merely on success rates at trial.(1) While their model has been subject to criticism (principally on the basis that the conclusion that no inferences could be drawn is too strong), even critics concede that their "central insight that litigated cases are a selected, non-random sample of all disputes" marks an important contribution.(2)

Priest and Klein's selection effect model has proven influential in American law and economics scholarship, leading to an increasing recognition of the need to account for selection effects when interpreting success rates in litigated cases.(3) The selection effect model has been tested and applied in the context of civil litigation,(4) and its central insights have been recognized in other areas of law,(5) including by American administrative law scholars.(6) Priest and Klein's work has also received some passing attention from Canadian legal scholars studying civil litigation.(7)

The question this article seeks to answer is whether and to what extent the selection model might have some explanatory power in the context of Canadian administrative law. I will argue that the selection model offers two interrelated insights in the context of Canadian administrative law. First, applications which proceed to a hearing in a given administrative regime cannot be presumed to be representative of the larger pool of filed applications. Second, because applications which proceed to a hearing cannot be presumed to be representative, inferences about the effectiveness of an administrative regime cannot automatically be drawn on the basis of applicant success rates alone.

Appreciating the existence of a possible selection effect is important for two reasons. The first reason is that success rates alone can and have been used to draw conclusions about the viability of specific administrative regimes. The second reason is that ignoring a possible selection effect encourages explanations of success rates which focus on either adjudicators or the relevant legal standard. This, in turn, can have policy consequences. The importance of appreciating a selection effect as offering a partial explanation of success rates might be seen as especially urgent because government responses and media coverage of various administrative regimes are often dominated by headlines about appeal success rates.(8) In many, if not all, of these discussions, the implicit inference is that the success rate is somehow representative of the larger body of cases.

For example, a 2017 article reported that nearly half of denied applicants for Canada Pension Plan (CPP) disability benefits were successful on appeal to the Social Security Tribunal (SST).(9) The article quoted Allison Schmidt, a pension disability case manager, who suggested that this was unacceptable, but that a success rate of 10-20% would be understandable.(10) No explanation for why the lower rate might be preferable was offered, but it is not difficult to reconstruct the most plausible line of reasoning here. The criticism about the high success rate at the SST seems to be premised on the assumption that it is indicative of a high error rate amongst all initial determinations of eligibility, meaning that around half of CPP disability benefit applicants are wrongfully denied. If this inference could be drawn, then it would suggest that the system for initial determinations was severely dysfunctional (one could also imagine how these same statistics could be used to argue that the SST is overly permissive in granting eligibility). The corollary then would be that a lower success rate on appeal would indicate a more accurate system of initial determinations. However, if there are even partial selection effects in play, then we may need to be more cautious about what the success rates on appeal can tell us about the administrative regime as a whole, and by extension, what a normatively desirable success rate might look like. Appreciating a partial selection effect does not rule out that there may be problems either with the initial eligibility determinations or at the SST. Instead, it highlights the possibility that the cases which proceed to the SST may not be representative of the larger body of applications. Applications to the SST may tend to have certain characteristics that make success at the SST more likely than those cases which are abandoned after the initial determination of ineligibility. In other words, there may be such a high success rate at the SST in part because of characteristics of the appealed cases themselves, not necessarily (or entirely) because of institutional dysfunction at any level of decision-making.

In Part I of this article, I will examine the Priest-Klein selection model and some of its criticisms and clarifications. In Part II, I will argue that the selection model may extend to administrative law, even though the model was originally developed to explain trial selection in civil litigation. In Part III, I will demonstrate how selection effects might prove insightful via a case study of the appeal success rates of the Ontario Disability Support Program (ODSP).

I THE PRIEST-KLEIN SELECTION MODEL

1. THE SELECTION MODEL

   The initial selection model set out by Priest and Klein in 1984 has undergone subsequent clarification and modification by other scholars, but it is necessary to start with the original model.

   The selection model is based entirely on economic concerns and, as the name implies, seeks to explain why it is that parties will pursue a costly trial rather than settle. The relevant factors in the parties' decisions include the expected costs, the information the parties possess about the likelihood of success, and the direct costs of litigation and settlement.(11) As Priest and Klein emphasize, the "most important assumption of the model is that potential litigants form rational estimates of the likely decision."(12) For the sake of simplicity, the issue of damages is ignored in the original model--damages are presumed to be stipulated such that there is no disagreement between the parties about them.(13) Additionally, in the simplest form of the model, the parties have symmetrical stakes in the litigation, since all that is at stake for either party is the potential award of pecuniary damages.(14)

   The model revolves around the concept of a 'decision standard,' which refers to a consistent standard for resolving legal disputes. Ideally, the decision standard would be the relevant legal standard, but Priest and Klein allow that it may be something more nefarious such as racial prejudice.(15) The decision standard divides all filed cases along a continuum based on case quality. Cases that would result in plaintiff victories if tried lie to the right of the decision standard, and those that would result in defendant victories lie to the left.(16) Cases which are far to the right of the decision standard are those in which the plaintiff is very likely to be victorious. Cases which are far to the left of the decision standard would almost certainly result in a defendant victory at trial. Accordingly, cases which are distributed close to the decision standard are those where it is much less clear which party might be victorious.(17)

   Parties form expectations of the likelihood of victory based on the application of the decision standard to the facts of their case. These expectations may diverge because of the parties' error in estimating case quality, or because of uncertainty about how exactly the decision standard will be applied(18) (the equivalent, in other words, of a vague legal standard).(19) Since the model as originally conceived applies to civil litigation, settlement filters out cases where both parties' expectations of the likelihood of victory are more closely aligned. For example, if the defendant's behaviour is thought by both parties to fall so far to the right of the decision standard that the plaintiff's victory at trial is almost assured, then settlement is extremely likely as the parties will agree on the outcome of the litigation and therefore settle to save the costs of going to trial.(20) If the parties' expectations of a plaintiff victory vary significantly, then the case will likely go to trial.(21) Although Priest and Klein never used the phrase in the original article, the model has become known as a 'divergent expectations' theory of litigation.(22)

   In the simplest form of the model, where there is little uncertainty about the decision standard, the cases which go to trial will be clustered around the decision standard. This is because the expectations of the parties are likely to diverge most significantly for those cases: the facts of the case will be so close to the decision standard that it is unclear whether a defendant will be found liable.(23) Accordingly, the cases most likely to go to trial are those which are most susceptible to being decided either way, and as a result, the model predicts that the plaintiff victory rate at trial will be driven towards 50% regardless of how plaintiff-or defendant-friendly the decision standard actually is.(24) The move towards 50% plaintiff victory will also involve a decrease in the number of cases taken to trial as parties' experience with a given legal standard increases and they are correspondingly better able to estimate the likelihood of plaintiff victory.(25)

   Priest and Klein acknowledge that the 50% plaintiff victory rate is a limiting case,(26) and that the decision standard's favourability to plaintiffs or defendants will have some effect on the rate of plaintiff victories. That said, they maintain that "the proportion of...