POEM: Pattern-Oriented Explanations of CNN Models

Abstract

While Convolutional Neural Networks (CNN) achieve state-of-the-art predictive performance in applications such as computer vision, their predictions are difficult to explain, similar to other types of deep learning models. Different solutions have been proposed to explain CNNs, from explanations of individual image predictions, to interpretable models that approximate the predictions of the CNN model. A recent line of research focuses on explaining CNNs using semantic concepts in images, such as objects, shapes, or colors, which are easier to understand. We contribute to this line of research by proposing POEM, a framework that produces patterns of concepts to explain image classifier CNNs. POEM identifies patterns such as “If bed, then bedroom”, meaning that if an image contains a bed and the model pays attention to the bed, then the model classifies the image as a bedroom.

We first introduce the general pipelined framework used in POEM, which we also use to describe the current related solutions. Then we propose improvements in each of the pipeline steps for more accurate explanation of CNNs. We also create a web-based tool for interactive visual analysis of the patterns. Finally, we demonstrate the effectiveness of our solution using multiple use cases involving different CNN models and datasets.