Class Based Strategies for Understanding Neural Networks
Loading...
Date
2020-02-07
Authors
Kumar, Devinder
Journal Title
Journal ISSN
Volume Title
Publisher
University of Waterloo
Abstract
One of the main challenges for broad adoption of deep learning based models such as Convolutional Neural Networks (CNN), is the lack of understanding of their decisions. In many applications, a simpler, less capable model that can be easily understood is favorable to a black-box model that has superior performance. Hence, it is paramount to have a mechanism for deep learning models such as deep neural networks to explain their decisions.
To resolve this explainability issue, in this thesis the main goal is to explore and develop new
class-enhanced support strategies for visualizing and understanding the decision-making process of deep neural networks. In particular, we take a three level approach to provide a holistic framework for explaining deep neural networks predictions.
In the first stage (Chapter 3), we first try to answer the question: based on what information neural networks make their decision and how it relates to a human expert's domain knowledge? To this end, we propose to introduce attentive response maps. The attentive response maps are able to show: 1) The locations in the input image that are contributing to decision-making and 2) the level of dominance of such locations. Through various experiments we elaborate how through attention response maps, we are able to visualize the decision-making process of deep neural networks and show where the neural networks were able to or failed to use landmark features similar to a human expert's domain knowledge.
In second stage (Chapter 4), we propose a novel end-to-end design architecture for obtaining end-to-end explanations through attentive response maps. Towards the end of this stage, we explore some of the shortcomings of the attentive response maps in failing to explain some of the complex scenarios.
In the last stage, (Chapter 5), we try to overcome the shortcomings of the binary attention maps introduced in the first stage. Towards this goal, a CLass-Enhanced Attentive Response (CLEAR) approach was introduced to visualize and understand the decisions made by deep neural networks (DNNs) given a specific input based on spatial support. CLEAR facilitates the visualization of attentive regions and levels of interest of DNNs during the decision-making process. It also enables the visualization of the most dominant classes associated with these attentive regions of interest. As such, CLEAR can mitigate some of the shortcomings of attention response maps-based methods associated with decision ambiguity, and allows for better insights into the decision-making process of DNNs.
In the last Chapter of this thesis (Chapter 6), we draw conclusions about the introduced class based explanation strategies and discuss some interesting future directions, including a formulation for class based global explanation that can be used for discovering and explaining the concepts identified by trained deep neural networks using human attribute priors.
Description
Keywords
explainable AI, interpretability, CNN, fairness, XAI