Analyzing Threats of Large-Scale Machine Learning Systems

Abstract

Large-scale machine learning systems such as ChatGPT rapidly transform how we interact with and trust digital media. However, the emergence of such a powerful technology faces a dual-use dilemma. While it can have many positive societal impacts in providing equitable access to information, ML systems can also be misused by untrustworthy entities to cause intentional harm. For example, a system could unintentionally disclose private information about its training data and jeopardize the privacy of individuals in the training data. The system's generated content could also be misused for unethical purposes, such as eroding trust in digital media by misrepresenting generating content as authentic. Providing untrustworthy users with these new capabilities could amplify potential negative consequences emerging through this technology, such as a proliferation of deep fakes or disinformation. I analyze these threats from two perspectives: (i) Data leakage, when the model cannot be trusted because it has memorized private information during training, and (ii) Misuse when users cannot be trusted to use the system for its intended purposes. This thesis presents five projects to assess these risks to the privacy and security of ML systems and evaluates the reliability of known countermeasures. To do so, I assess the privacy risks of extracting Personally Identifiable Information from language models trained with differential privacy. As a method of controlling unintended use, I study the effectiveness and robustness of fingerprinting and watermarking methods to detect the provenance of models and their generated content.