ADMINISTRATIVE LAW AND THE GOVERNANCE OF AUTOMATED DECISION MAKING: A CRITICAL LOOK AT CANADA'S DIRECTIVE ON AUTOMATED DECISION MAKING.

• Author: Scassa, Teresa

INTRODUCTION

The adoption and use of automated decision making (ADM) in government is a growing trend, and one that raises particular concerns about fairness, transparency, and accountability. In 2019, the Canadian government adopted the Directive on Automated Decision-Making (DADM) (1) and its accompanying Algorithmic Impact Assessment (AIA) tool. The DADM and AIA are designed to guide the adoption and use of automated decision making at the federal level. This paper assesses these tools as a means of achieving fairness, transparency and accountability in public sector administrative decision making. It does so using administrative law as the normative frame. As Engstrom and Ho observe, "[a]dministrative law's approach to the issues of transparency and reason-giving ... is multi-faceted and tailored to particular governance tasks, providing a richer and as yet unexplored set of frames for assessing and resolving the accountability dilemmas in an increasingly digitized government." (2)

The DADM and AIA are built upon norms for administrative decision making that have their roots in common law principles of procedural fairness. These rules reflect an important societal preoccupation with decision making by governments. Concerns about procedural fairness in government decision making are thus not unique to the age of intelligent machines and have been developed over a long history in relation to human decision making. The incorporation of administrative law principles into the DADM and AIA attempts to adapt the existing normative context to automated decision making. Questions for consideration in this paper include, therefore: (i) how the principles of fairness in decision making translate to the context of automated decision making in the DADM; (ii) what differences emerge in the course of this translation; (iii) what this might tell us about the particular nature of automated decision making; and (iv) what gaps emerge that remain to be addressed.

This paper begins with a discussion of ADM in government and raises some core questions about the nature of ADM and its relationship to the administrative state. Part II provides an account of the history and context for the development of the DADM and AIA in Canada. Part III assesses the DADM and AIA according to administrative principles of procedural fairness.

1. AUTOMATED DECISION MAKING IN GOVERNMENT

   Artificial intelligence (AI) technologies have attracted considerable attention in both public and private sectors alike. In the public sector context, they are attractive to governments who see their potential to streamline service delivery, cut costs in the making of simple and routine decisions, and render other decision-making processes more efficient. Some have suggested that AI in the public sector "has the potential to make governments--and even whole democratic systems--more accurate, more efficient, and more fair." (3) The Organisation for Economic Co-operation and Development (OECD), in studying the potential for AI in government, observes that AI "can play a pivotal role in supporting the public sector in its perpetual fight to simplify processes or increase their efficiency." (4) It also suggests that citizens expect their governments to begin to deliver similar levels of "responsiveness and personalization" as are available in private sector services. (5) Interestingly, the OECD appears to see AI as a means of shifting to more effective digital open government. (6) Karlin, Casovan and Corriveau also emphasize the efficiencies of ADM. They note the potential to offer round-the-clock service through virtual assistants, (7) as well as the ability of AI systems to perform mundane and routine tasks without fatigue--and to do so 24/7. (8) They argue that "[t]hese technologies have the potential to guide the public service towards a future of greater effectiveness and responsiveness to the needs of society than was ever possible before." (9)

   Excitement about the promise of AI is necessarily tempered with caution. The OECD observes that governments that move towards the incorporation of AI and ADM face similar challenges to the private sector, including access to a sufficient quantity of high quality, interoperable data. This in turn requires the adoption of standards, and a careful consideration of privacy principles. In addition, it is necessary to work through the ethical issues associated with the use of AI. The OECD observes:

   Considering the growing role of AI within public sectors, it is key for governments and regulators to review the role they should play to ensure a balance between encouraging AI to foster public sector innovation and improved public services, while protecting the public and service users' interests from potential unintended negative consequences of the use of these disruptive technologies. (10) There has been considerable interest in the development of codes of ethics (11) or even algorithmic assessment tools for the adoption of AI systems. For example, in a report for the AI Now Institute at New York University, Reisman et al proposed the adoption of AIAs for all public agencies. (12)

   While the asserted benefits are many, critics of the adoption of AI in government have been less sanguine about the potential of these technologies. Virginia Eubanks, for example, sees little of the transformative potential of AI. She notes that "... automated decision making in our current welfare system acts a lot like older, atavistic forms of punishment and containment. It filters and diverts. It is a gatekeeper, not a facilitator." (13) Others have raised concerns about the impacts of ADM on fairness, (14) privacy, (15) equality, (16) and transparency. (17) For the most part, these concerns have not been exclusive to the public sector, but within the public sector these issues present significant justice and rule of law issues. (18)

   Eubanks also cautions that automated decision making tends to be implemented first on the poor or those with little power. (19) Not only do these segments of the population lack the ability or resources to push back against unfair decision making, she also notes that "the uptake of these tools is occurring at a time when programs that serve the poor are as unpopular as they have ever been." (20) Thus there may be little broader public interest in pushing back against their deployment. (21) Eubank's scathing critique comes with a warning: "[A]nd while the most sweeping digital decision-making tools are tested in what could be called 'low rights environments' where there are few expectations of political accountability and transparency, systems first designed for the poor will eventually be used on everyone." (22)

   1. AUTOMATED DECISION MAKING AS A SUBSET OF AI IN GOVERNMENT

      Automated decision making is a subset of AI. Governments will adopt AI systems that include ADM, but some systems will perform functions different from ADM. In thinking about the governance of ADM in government, it is worth considering what is or is not included. The diversity and complexity of AI systems make them hard to define, and most definitions acknowledge this difficulty. One of the challenges of discussing AI is that the term can mean different things in different contexts. As Karlin, Casovan and Corriveau note,

      AI has grown to become a term that includes a broad spectrum of related technologies that seek to imitate and enhance aspects of human intelligence, such as vision, identifying patterns in information, or understanding language. In a sense, AI is when computers do what only humans could before. The term is used to describe applications as innocuous as a system that recommends books to read, to fictional advanced human-like intelligence capable of everything a human is. As such, there is no single, internationally-recognized definition for AI, and the term may mean different things to different people. (23) The diversity of AI applications clearly contributes to the definitional challenges. The OECD has defined AI as "a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments." (24) The means by which it does so can vary. According to the OECD, AI can use "machine and/or human-based inputs to: i) perceive real and/or virtual environments; ii) abstract such perceptions into models through analysis in an automated manner--for example, with machine learning--or manually; and iii) use model inference to formulate options for information or action." (25)

      There is no single paradigm for the use of AI systems in government. A recent study on the use of AI in Canada's immigration system noted that Canadian governments have used the terms artificial intelligence, machine learning, and predictive analytics to talk about different activities that fit under the umbrella of AI. (26) The federal government has also referred to automated decision systems, "automated systems, decision-support systems, machine learning, and machine intelligence." (27) The 2018 Toronto Declaration on Protecting the Right to Equality and Non-Discrimination in Machine Learning Systems (28,) refers to AI more broadly, while the 2018 Montreal Declarationfor Responsible AI Development (29) focuses on artificial intelligence, but refers to autonomous and intelligent agents, intelligent machines, and artificial agents. (30)

      There is already a plethora of examples of public sector adoption of AI tools. Some uses may relate to service delivery. For example, the use of intelligent agents such as chatbots is frequently referenced as an option for governments to enhance service delivery. (31) AI can be used to operate infrastructure--for example, to maintain optimal temperatures or to conserve energy resources. (32) AI systems may be used to perform functions such as translation, either alone or in conjunction with human translators. (33) AI can be used to...