Deployment Concerns in Machine Learning Systems: Unintended Interactions and Accountability

Abstract

Machine learning (ML) models are increasingly being deployed for client-facing services (e.g., chatbots, search engines, and browsers), high-stakes decision-making applications (e.g., healthcare and criminal justice), and as part of larger systems (e.g., autonomous vehicles and operating systems). However, to deploy ML models for a particular application, practitioners need to address various deployment concerns including (i) infrastructure issues (e.g., latency, throughput, interoperability, scalability), (ii) model design (e.g., high utility and generalization, minimal overfitting, hyperparameter tuning, data processing), (iii) environmental impact (e.g., reducing carbon emissions, water and power consumption by data centers), (iv) adversarial and societal risks (e.g., security, privacy, safety, unfairness, poor transparency, misalignment, misinformation, and cyberattacks), and (v) enabling governance (e.g., verifying claims by practitioners, and regulatory compliance). I focus on two deployment concerns: adversarial and societal risks, and enabling governance, and address unintended interactions and accountability within these respective concerns. I present them as two parts of the thesis.

(Part-1) Unintended Interactions in ML: Substantial prior work explores the design of defenses against individual risks to security, privacy, fairness, transparency, and safety. I argue that this is not sufficient for real-world ML models that must protect against multiple risks simultaneously. Practitioners need to address additional challenges that emerge when doing so, including unintended interactions. A systematic understanding of such interactions is lacking, and I study three unintended interactions: (a) a defense against one risk may increase or decrease other unrelated risks; (b) conflicts among defenses can decrease their effectiveness when combined; and (c) potential for collusion among adversaries can enable executing an attack to amplify others. I propose frameworks to identify factors underlying such interactions, and present guidelines to conjecture about unexplored ones.

(Part-2) Accountability in ML Pipelines: Practitioners' claims about executing various ML operations needs verification by a verifier (e.g., regulator). This includes demonstrating ML properties covering the model, its training process, its training data, as well as deploying defenses and accounting for unintended interactions from Part-1. Such claims are currently communicated via ML property cards (e.g., model, data, and inference cards). I propose ML property attestation mechanisms that allow provers (e.g., model trainers) to demonstrate ML properties to verifiers, while ensuring model and data confidentiality. I show that existing software-based mechanisms are either inefficient (e.g., cryptographic mechanism), or ineffective and easily evaded (e.g., ML-based mechanism). I then identify hardware-based mechanisms using trusted execution environments as an efficient and effective alternative for providing ML property attestations. These attestations can then be used for verifiable ML property cards, to ensure accountability for practitioners' claims.