Pathfinder Variational Inference

[aphc14]

Notebook proposal

Title: Pathfinder Variational Inference

Why should this notebook be added to pymc-examples?

As the existing Pathfinder example notebook is really out-of-date, I propose replacement notebook to demo the new features from Pathfinder pymc-extras>=0.2.2 so that the user has a good understanding on:

• How the algorithm works
• Function argument recommendations
• Compute speed and posterior comparisons between NUTS and ADVI
• Interpreting the Pathfinder stdout results
• (Maybe) How to use estimates from Pathfinder as initial values for NUTS
• Pros and Cons of Pathfinder

Unlike ADVI, which in most cases, doesn't require users to think that deeply about setting the argument values and there being only a few arguments to consider, Pathfinder requires some extra more thought for many arguments. It can be highly sensitive to the argument values. And knowing how to set the argument values based on feedback from the stdout results and posterior visuals is going to help users understand adjustments they need to make on the argument values (before thinking this feature is broken or algorithm isn't effective).

Examples of function argument to discuss:
num_paths maxcor maxiter ftol gtol jitter importance_sampling compile_kwargs (if mode="JAX" is fixed), and inference_backend (if blackjax pathfinder is fixed)

Suggested categories:

• Level: Intermediate
• Diataxis type: Explanation & How-To Guide

Related notebooks

https://www.pymc.io/projects/examples/en/latest/variational_inference/pathfinder.html

References

Zhang, L., Carpenter, B., Gelman, A., & Vehtari, A. (2022). Pathfinder: Parallel quasi-Newton variational inference. Journal of Machine Learning Research, 23(306), 1–49.

[AlexAndorra]

Love it @aphc14 ! Feel free to ping me for reviews