Deep SELECTOR-JPEG: ADAPTIVE JPEG IMAGE COMPRESSION FOR COMPUTER VISION IN IMAGE CLASSIFICATION AND HUMAN VISION

Abstract

Deep Neural Networks (DNNs) demonstrate excellent performance in many Computer Vision (CV) applications such as image classification. To meet storage/bandwidth requirements, the input images to these CV applications are compressed using lossy image compression standards, among which JPEG is the most common. Classical JPEG is designed to consider Human Vision (HV) and pays a little attention to CV, resulting in classification accuracy drop of DNNs, especially at high Compression Ratios (CRs). This work presents Deep Selector-JPEG, an adaptive JPEG compression method that simultaneously targets both image classification and HV. For each image, Deep Selector-JPEG selects a Quality Factor (QF) adaptively to compress the image so that a good trade-off between the Compression Ratio (CR) and DNN classifier Accuracy (Rate-Accuracy performance) can be achieved over a set of images for a variety of DNN classifiers while the PSNR of such compressed image is greater than a threshold value predetermined by HV with a high probability. Towards this end, Deep Selector-JPEG first defines a set of feasible QFs such that an image compressed at any QF within this set has PSNR greater than a predetermined threshold value with a high probability. For some images, multiple QFs within this set are suitable (ON) for compressing for a DNN classifier, which means compressing at these QFs at least maintains the ground truth rank of the original input for the DNN classifier. For a given image, Deep Selector-JPEG first determines the QFs that are ON among the set of feasible QFs. This problem is represented as a Multi-label Classification (MLC) problem since each image has multiple corresponding suitable QFs. We solve MLC using a binary relevance procedure, which involves training an independent binary DNN classifier for each QF within the feasible set to predict the ON/OFF labeling for each input image. Given a target CR, we empirically derive a subset of feasible QFs for this target CR and select the least QF that is ON in this set. Experimental results show that in comparison with the default JPEG, Deep Selector-JPEG indeed achieves better Rate-Accuracy performance over the entire ImageNet validation set for all tested DNN classifiers with gains in classification accuracy up to 1% at the same CRs, while satisfying HV constraints and keeping complexity under control.