2025-01-14-le25a.md

title	Towards Robust Saliency Maps
booktitle	Proceedings of the 16th Asian Conference on Machine Learning
year	2025
volume	260
series	Proceedings of Machine Learning Research
month	0
publisher	PMLR
pdf	https://raw.githubusercontent.com/mlresearch/v260/main/assets/le25a/le25a.pdf
url	https://proceedings.mlr.press/v260/le25a.html
openreview	2tv0Ubg3o7
abstract	Saliency maps are one of the most popular tools to interpret the operation of a neural network: they compute input features deemed relevant to the final prediction, which are often subsets of pixels that are easily understandable by a human being. However, it is known that relying solely on human assessment to judge a saliency map method can be misleading. In this work, we propose a new neural network verification specification called saliency-robustness, which aims to use formal methods to prove a relationship between Vanilla Gradient (VG) – a simple yet surprisingly effective saliency map method – and the network’s prediction: given a network, if an input $x$ emits a certain VG saliency map, it is mathematically proven (or disproven) that the network must classify $x$ in a certain way. We then introduce a novel method that combines both Marabou and Crown – two state-of-the-art neural network verifiers, to solve the proposed specification. Experiments on our synthetic dataset and MNIST show that Vanilla Gradient is surprisingly effective as a certification for the predicted output.
layout	inproceedings
issn	2640-3498
id	le25a
tex_title	Towards Robust Saliency Maps
firstpage	351
lastpage	366
page	351-366
order	351
cycles	false
bibtex_editor	Nguyen, Vu and Lin, Hsuan-Tien
editor	given family Vu Nguyen given family Hsuan-Tien Lin
bibtex_author	Le, Nham and Gurfinkel, Arie and Si, Xujie and Geng, Chuqin
author	given family Nham Le given family Arie Gurfinkel given family Xujie Si given family Chuqin Geng
date	2025-01-14
address
container-title	Proceedings of the 16th Asian Conference on Machine Learning
genre	inproceedings
issued	date-parts 2025 1 14
extras