Formulating neuroimaging & CV problem in SBI

[Aceticia]

Hi all. I'm a neuroimaging researcher. I recently discovered the idea and package of sbi. It's so interesting to me and I'd like to give it a try to analyze my data. My specific research problems involve training deep, modular models of visual perception, with each module representing a specific "function" such as recall or edge detection, and then using fmri to figure out which part of the brain is implementing which module.

This is how I imagine sbi can help: we can assign one (or more) function to a location in the brain, and this is the parameter of interest. Depending on what model function that a location has, it produces different fmri signals in response to a set of inputs. Sampling a posterior of these parameters would give you an estimate of what each location is doing in the brain.

Does this sound about right? If so, I have 2 questions about this:

1. Does SBI handle large amount of parameters efficiently? We can use embedding nets and GPU, but how about the parameters themselves? This setup of the problem demands the magnitude of O(number of locations) parameters.
2. How to estimate reliability of results? Are there any multiple comparison correction equivalents in this context? I'm mainly familiar with the frequentist toolbox so unfortunately I'm not too sure how to handle this outside of permutation tests and stuff :<

[michaeldeistler]

Hi there,

yup this sounds reasonable to me.

1. Number of parameters is challenging. It depends on a few factors: (a) how good is your prior? If your prior is very uninformative then you will be able to handle fewer parameters. (b) how fast is the simulator? You will need more simulations if you have more parameters and, therefore, need a very fast simulator at some point. In general, for simulators that have a uninformative prior and that run around 3seconds it is very challenging to have more than 30 parameters. Most successful applications of sbi have less than 20 parameters.
2. See here and especially here.

Hope this helps!
Michael

[Aceticia]

Thanks. This is very helpful. To run the simulator fast, I might need access to GPUs since I'm generating features from a DNN. I don't think this is supported yet?

Another question I have is how to handle conditionals in posterior sampling. We'd like to keep the parameters conditioned on a sequence of test images, because as you can imagine the functions change depending on the images viewed. But I haven't seen anything on the tutorials that discusses conditionals during sampling (for example I is a sequence of images, P(theta|I), P(X|theta, I))

[michaeldeistler]

Yes GPU is supported, see here.

I don't think I really understand the issue of conditionals. In particular, why would you want to sample p(X|theta, l)? Isn't that the likelihood? What is your notation here?

[Aceticia]

But that is for training density estimators, not running the simulation? I guess alternatively the simulator can be run in a separate server with multiple GPUs and just send results back for sbi to consume.

Sorry my language was very confusing. All the data that I have and the simulator has to be conditioned on some sequence of images. That means I'm ultimately only interested in P(theta|X, I). One way to handle this is to merge I with X altogether and generate both I and X, but this is not ideal.

[michaeldeistler]

Yes, that is true, this is only for training. And indeed, for simulation, I would recommend to run the simulations offline (i.e., on a separate server) and then just load the results for training.

Hm, thanks for the explanation. I think that the way to go really is to merge l and X altogether.