Suggestions for using your code base for circuit finding

[ashwath98]

Hi, congrats on the cool work, and thanks for open-sourcing your code base
I am working on a method to find input-dependent circuits in visual models, and I think your work would make for a fair comparison

I was wondering if you could help me with where I should look if I had to modify your code to find a circuit for just a single example.

For example, if I have a single tiger image from the imagenet dataset, I would like to find the channels that Spade marks as important after the optimization process

Thanks
-Ashwath

[ohaijen]

Hi Ashwath, thanks for reaching out!

SPADE generally operates on a single-example level. In our paper, custom subnetworks (what we call a 'trace' in the paper and what I believe you're calling a 'circuit') is computed for each example; for our use-case they are simply intermediate steps for the creation of, e.g., a saliency map and are never stored on disk. Thus, our code does not need to be modified at all to work on the single-example level; the only thing you might need to do is to save the model weights. To be perfectly clear, the critical lines in the sample_notebook.ipynb file are:

pruning_dataset = SingleImageDataset(im, 1024,  normalize=True)
sparsity_function=lambda x: [s for l,s in tuned.items() if l in x]
layer_filter=lambda k: True if ("conv"  in k or "fc"  in k or "downsample.0" in k) and k!="conv1" else False
pruned_model=FOBC_core(pruning_dataset, Dense_model, layer_filter, sparsity_function=sparsity_function)

There might be one point of confusion that I'd like to clarify. Even though the dataset is really based on a single image, e.g. of a tiger, it actually consists of 1024 images, which are the original image with some standard transformations (crops, small color changes, etc) applied to it. This is necessary for a technical reason - otherwise the pruner doesn't work - and also, we believe, makes the interpretations a bit more general, since the transformations are small enough that the same feature and final class detectors should still be activated.

Cheers,

Jen