Skip to content

Commit b239132

Browse files
authored
update bart examples to reflect changes in last version (#759)
1 parent 0067c6e commit b239132

8 files changed

+455
-522
lines changed

examples/bart/bart_categorical_hawks.ipynb

+113-86
Large diffs are not rendered by default.

examples/bart/bart_categorical_hawks.myst.md

+11-7
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ myst:
1414
pip_dependencies: pymc-bart
1515
---
1616

17-
+++ {"editable": true, "slideshow": {"slide_type": ""}}
17+
+++ {"slideshow": {"slide_type": ""}}
1818

1919
(bart_categorical)=
2020
# Categorical regression
@@ -136,11 +136,11 @@ It may be that some of the input variables are not informative for classifying b
136136

137137
```{code-cell} ipython3
138138
---
139-
editable: true
140139
slideshow:
141140
slide_type: ''
142141
---
143-
pmb.plot_variable_importance(idata, μ, x_0, method="VI", random_seed=RANDOM_SEED);
142+
vi_results = pmb.compute_variable_importance(idata, μ, x_0, method="VI", random_seed=RANDOM_SEED)
143+
pmb.plot_variable_importance(vi_results);
144144
```
145145

146146
It can be observed that with the covariables `Hallux`, `Culmen`, and `Wing` we achieve the same R$^2$ value that we obtained with all the covariables, this is that the last two covariables contribute less than the other three to the classification. One thing we have to take into account in this is that the HDI is quite wide, which gives us less precision on the results, later we are going to see a way to reduce this.
@@ -152,7 +152,7 @@ It can be observed that with the covariables `Hallux`, `Culmen`, and `Wing` we a
152152
Let's check the behavior of each covariable for each species with `pmb.plot_pdp()`, which shows the marginal effect a covariate has on the predicted variable, while we average over all the other covariates.
153153

154154
```{code-cell} ipython3
155-
pmb.plot_pdp(μ, X=x_0, Y=y_0, grid=(5, 3), figsize=(6, 9));
155+
pmb.plot_pdp(μ, X=x_0, Y=y_0, grid=(5, 3), figsize=(12, 7));
156156
```
157157

158158
The pdp plot, together with the Variable Importance plot, confirms that `Tail` is the covariable with the smaller effect over the predicted variable. In the Variable Importance plot `Tail` is the last covariable to be added and does not improve the result, in the pdp plot `Tail` has the flattest response. For the rest of the covariables in this plot, it's hard to see which of them have more effect over the predicted variable, because they have great variability, showed in the HDI wide, same as before later we are going to see a way to reduce this variability. Finally, some variability depends on the amount of data for each species, which we can see in the `counts` from one of the covariables using Pandas `.describe()` and grouping the data from "Species" with `.groupby("Species")`.
@@ -215,11 +215,14 @@ with pm.Model(coords=coords) as model_t:
215215
Now we are going to reproduce the same analyses as before.
216216

217217
```{code-cell} ipython3
218-
pmb.plot_variable_importance(idata_t, μ_t, x_0, method="VI", random_seed=RANDOM_SEED);
218+
vi_results = pmb.compute_variable_importance(
219+
idata_t, μ_t, x_0, method="VI", random_seed=RANDOM_SEED
220+
)
221+
pmb.plot_variable_importance(vi_results);
219222
```
220223

221224
```{code-cell} ipython3
222-
pmb.plot_pdp(μ_t, X=x_0, Y=y_0, grid=(5, 3), figsize=(6, 9));
225+
pmb.plot_pdp(μ_t, X=x_0, Y=y_0, grid=(5, 3), figsize=(12, 7));
223226
```
224227

225228
Comparing these two plots with the previous ones shows a marked reduction in the variance for each one. In the case of `pmb.plot_variable_importance()` there are smallers error bands with an R$^{2}$ value more close to 1. And for `pm.plot_pdp()` we can see thinner bands and a reduction in the limits on the y-axis, this is a representation of the reduction of the uncertainty due to adjusting the trees separately. Another benefit of this is that is more visible the behavior of each covariable for each one of the species.
@@ -254,7 +257,8 @@ all
254257
```
255258

256259
## Authors
257-
- Authored by [Pablo Garay](https://github.com/PabloGGaray) and [Osvaldo Martin](https://aloctavodia.github.io/) in May, 2024
260+
- Authored by [Pablo Garay](https://github.com/PabloGGaray) and [Osvaldo Martin](https://aloctavodia.github.io/) in May, 2024
261+
- Updated by Osvaldo Martin in Dec, 2024
258262

259263
+++
260264

examples/bart/bart_heteroscedasticity.ipynb

+52-65
Large diffs are not rendered by default.

examples/bart/bart_heteroscedasticity.myst.md

+1
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,7 @@ The fit looks good! In fact, we see that the mean and variance increase as a fun
147147
- Authored by [Juan Orduz](https://juanitorduz.github.io/) in Feb, 2023
148148
- Rerun by Osvaldo Martin in Mar, 2023
149149
- Rerun by Osvaldo Martin in Nov, 2023
150+
- Rerun by Osvaldo Martin in Dec, 2024
150151

151152
+++
152153

examples/bart/bart_introduction.ipynb

+198-252
Large diffs are not rendered by default.

examples/bart/bart_introduction.myst.md

+4-2
Original file line numberDiff line numberDiff line change
@@ -199,14 +199,15 @@ Finally, like with other regression methods, we should be careful that the effec
199199

200200
### Variable importance
201201

202-
As we saw in the previous section a partial dependence plot can visualize and give us an idea of how much each covariable contributes to the predicted outcome. Moreover, PyMC-BART provides a novel method to assess the importance of each variable in the model. You can see an example in the following figure.
202+
As we saw in the previous section a partial dependence plot can visualize give us an idea of how much each covariable contributes to the predicted outcome. Moreover, PyMC-BART provides a novel method to assess the importance of each variable in the model. You can see an example in the following figure.
203203

204204
On the x-axis we have the number of covariables and on the y-axis R² (the the square of the Pearson correlation coefficient) between the predictions made for the full model (all variables included) and the restricted models, those with only a subset of the variables.
205205

206206
In this example, the most important variable is `hour`, then `temperature`, `humidity`, and finally `workingday`. Notice that the first value of R², is the value of a model that only includes the variable `hour`, the second R² is for a model with two variables, `hour` and `temperature`, and so on. Besides this ranking, we can see that even a model with a single component, `hour`, is very close to the full model. Even more, the model with two components `hour`, and `temperature` is on average indistinguishable from the full model. The error bars represent the 94 \% HDI from the posterior predictive distribution. This means that we should expect a model with only `hour` and `temperature` to have a similar predictice performance than a model with the four variables, `hour`, `temperature`, `humidity`, and `workingday`.
207207

208208
```{code-cell} ipython3
209-
pmb.plot_variable_importance(idata_bikes, μ, X);
209+
vi_results = pmb.compute_variable_importance(idata_bikes, μ, X)
210+
pmb.plot_variable_importance(vi_results);
210211
```
211212

212213
`plot_variable_importance` is fast because it makes two assumptions:
@@ -405,6 +406,7 @@ This plot helps us understand the reason behind the bad performance on the test
405406
* Juan Orduz added out-of-sample section in Jan, 2023
406407
* Updated by Osvaldo Martin in Mar, 2023
407408
* Updated by Osvaldo Martin in Nov, 2023
409+
* Updated by Osvaldo Martin in Dec, 2024
408410

409411
+++
410412

examples/bart/bart_quantile_regression.ipynb

+75-110
Large diffs are not rendered by default.

examples/bart/bart_quantile_regression.myst.md

+1
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,7 @@ We can see that when we use a Normal likelihood, and from that fit we compute th
149149
* Authored by Osvaldo Martin in Jan, 2023
150150
* Rerun by Osvaldo Martin in Mar, 2023
151151
* Rerun by Osvaldo Martin in Nov, 2023
152+
* Rerun by Osvaldo Martin in Dec, 2024
152153

153154
+++
154155

0 commit comments

Comments
 (0)