Merge pull request #67 from pymc-devs/main

Sync Fork from Upstream Repo

Author: sthagen

.github/ISSUE_TEMPLATE.md

@@ -18,7 +18,7 @@ If you have questions about a specific use case, or you are not sure whether thi
 ## Versions and main components
 
 * PyMC3 Version:
-* Aesara Version:
+* Aesara/Theano Version:
 * Python Version:
 * Operating system:
 * How did you install PyMC3: (conda/pip)

.github/workflows/arviz_compat.yml

@@ -3,7 +3,7 @@ name: arviz-compatibility
 on:
   pull_request:
   push:
-    branches: [master]
+    branches: [main]
 
 jobs:
   pytest:

.github/workflows/jaxtests.yml

@@ -3,7 +3,7 @@ name: jax-sampling
 on:
   pull_request:
   push:
-    branches: [master]
+    branches: [main]
 
 jobs:
   pytest:

.github/workflows/pre-commit.yml

@@ -3,7 +3,7 @@ name: pre-commit
 on:
   pull_request:
   push:
-    branches: [master]
+    branches: [main]
 
 jobs:
   pre-commit:

.github/workflows/pytest.yml

@@ -3,7 +3,7 @@ name: pytest
 on:
   pull_request:
   push:
-    branches: [master]
+    branches: [main]
 
 jobs:
   pytest:
@@ -12,52 +12,78 @@ jobs:
         os: [ubuntu-18.04]
         floatx: [float32, float64]
         test-subset:
+        # Tests are split into multiple jobs to accelerate the CI.
+        # The first job (starting in the next block) shouldn't run any tests, but
+        # just ignores tests because that don't work at all, or run in other jobs.'
+        # Any test that was not ignored runs in the first job.
+        # A pre-commit hook (scripts/check_all_tests_are_covered.py) enforces that
+        # test run just once.
+
+        # Because YAML doesn't allow comments in the blocks below, here they are..
+        # 1st block: These tests are temporarily disabled, because they are _very_ broken
+        # 2nd block: The JAX tests run through their own workflow: jaxtests.yml
+        # 3nd & 4rd: These tests are covered by other matrix jobs
+        # 5th block: These tests PASS without a single XFAIL
+        # 6th block: These have some XFAILs
           - |
-            --ignore=pymc3/tests/test_dist_math.py
-            --ignore=pymc3/tests/test_distribution_defaults.py
-            --ignore=pymc3/tests/test_distributions.py
-            --ignore=pymc3/tests/test_distributions_random.py
             --ignore=pymc3/tests/test_distributions_timeseries.py
-            --ignore=pymc3/tests/test_examples.py
-            --ignore=pymc3/tests/test_gp.py
+            --ignore=pymc3/tests/test_missing.py
             --ignore=pymc3/tests/test_mixture.py
-            --ignore=pymc3/tests/test_ode.py
+            --ignore=pymc3/tests/test_model_graph.py
+            --ignore=pymc3/tests/test_modelcontext.py
             --ignore=pymc3/tests/test_parallel_sampling.py
-            --ignore=pymc3/tests/test_posteriors.py
-            --ignore=pymc3/tests/test_quadpotential.py
+            --ignore=pymc3/tests/test_profile.py
             --ignore=pymc3/tests/test_random.py
-            --ignore=pymc3/tests/test_sampling.py
-            --ignore=pymc3/tests/test_sampling_jax.py
-            --ignore=pymc3/tests/test_shape_handling.py
             --ignore=pymc3/tests/test_shared.py
             --ignore=pymc3/tests/test_smc.py
+            --ignore=pymc3/tests/test_starting.py
             --ignore=pymc3/tests/test_step.py
-            --ignore=pymc3/tests/test_updates.py
+            --ignore=pymc3/tests/test_tracetab.py
+            --ignore=pymc3/tests/test_tuning.py
+            --ignore=pymc3/tests/test_types.py
             --ignore=pymc3/tests/test_variational_inference.py
+            --ignore=pymc3/tests/test_sampling_jax.py
+            --ignore=pymc3/tests/test_dist_math.py
+            --ignore=pymc3/tests/test_minibatches.py
+            --ignore=pymc3/tests/test_pickling.py
+            --ignore=pymc3/tests/test_plots.py
+            --ignore=pymc3/tests/test_special_functions.py
+            --ignore=pymc3/tests/test_updates.py
+            --ignore=pymc3/tests/test_examples.py
+            --ignore=pymc3/tests/test_gp.py
+            --ignore=pymc3/tests/test_model.py
+            --ignore=pymc3/tests/test_model_func.py
+            --ignore=pymc3/tests/test_ode.py
+            --ignore=pymc3/tests/test_posdef_sym.py
+            --ignore=pymc3/tests/test_quadpotential.py
+            --ignore=pymc3/tests/test_shape_handling.py
+            --ignore=pymc3/tests/test_distributions.py
+            --ignore=pymc3/tests/test_distributions_random.py
+            --ignore=pymc3/tests/test_idata_conversion.py
+
           - |
+            pymc3/tests/test_modelcontext.py
             pymc3/tests/test_dist_math.py
-            pymc3/tests/test_distribution_defaults.py
-            pymc3/tests/test_distributions_random.py
-            pymc3/tests/test_parallel_sampling.py
-            pymc3/tests/test_random.py
-            pymc3/tests/test_shared.py
-            pymc3/tests/test_smc.py
+            pymc3/tests/test_minibatches.py
+            pymc3/tests/test_pickling.py
+            pymc3/tests/test_plots.py
+            pymc3/tests/test_special_functions.py
+            pymc3/tests/test_updates.py
+
           - |
+            pymc3/tests/test_idata_conversion.py
+            pymc3/tests/test_distributions.py
+            pymc3/tests/test_distributions_random.py
             pymc3/tests/test_examples.py
-            pymc3/tests/test_mixture.py
+            pymc3/tests/test_gp.py
+            pymc3/tests/test_model.py
+            pymc3/tests/test_model_func.py
             pymc3/tests/test_ode.py
-            pymc3/tests/test_posteriors.py
+            pymc3/tests/test_posdef_sym.py
             pymc3/tests/test_quadpotential.py
-          - |
-            pymc3/tests/test_distributions_timeseries.py
             pymc3/tests/test_shape_handling.py
             pymc3/tests/test_step.py
-            pymc3/tests/test_updates.py
-            pymc3/tests/test_variational_inference.py
-          - |
-            pymc3/tests/test_distributions.py
-            pymc3/tests/test_gp.py
-            pymc3/tests/test_sampling.py
+
       fail-fast: false
     runs-on: ${{ matrix.os }}
     env:

.github/workflows/windows.yml

@@ -3,10 +3,11 @@ name: windows
 on:
   pull_request:
   push:
-    branches: [master]
+    branches: [main]
 
 jobs:
   pytest:
+    if: false
     strategy:
       matrix:
         os: [windows-latest]

CONTRIBUTING.md

@@ -43,7 +43,7 @@ The preferred workflow for contributing to PyMC3 is to fork the [GitHub reposito
    $ git checkout -b my-feature
    ```
 
-   Always use a ``feature`` branch. It's good practice to never routinely work on the ``master`` branch of any repository.
+   Always use a ``feature`` branch. It's good practice to never routinely work on the ``main`` branch of any repository.
 
 4. Project requirements are in ``requirements.txt``, and libraries used for development are in ``requirements-dev.txt``. The easiest (and recommended) way to set up a development environment is via [miniconda](https://docs.conda.io/en/latest/miniconda.html):
 
@@ -73,7 +73,7 @@ The preferred workflow for contributing to PyMC3 is to fork the [GitHub reposito
    After committing, it is a good idea to sync with the base repository in case there have been any changes:
    ```bash
    $ git fetch upstream
-   $ git rebase upstream/master
+   $ git rebase upstream/main
    ```
 
    Then push the changes to your GitHub account with:

README.rst

@@ -1,4 +1,4 @@
-.. image:: https://cdn.rawgit.com/pymc-devs/pymc3/master/docs/logos/svg/PyMC3_banner.svg
+.. image:: https://cdn.rawgit.com/pymc-devs/pymc3/main/docs/logos/svg/PyMC3_banner.svg
     :height: 100px
     :alt: PyMC3 logo
     :align: center
@@ -11,7 +11,7 @@ algorithms. Its flexibility and extensibility make it applicable to a
 large suite of problems.
 
 Check out the `getting started guide <http://docs.pymc.io/notebooks/getting_started>`__,  or
-`interact with live examples <https://mybinder.org/v2/gh/pymc-devs/pymc3/master?filepath=%2Fdocs%2Fsource%2Fnotebooks>`__
+`interact with live examples <https://mybinder.org/v2/gh/pymc-devs/pymc3/main?filepath=%2Fdocs%2Fsource%2Fnotebooks>`__
 using Binder!
 For questions on PyMC3, head on over to our `PyMC Discourse <https://discourse.pymc.io/>`__ forum.
 
@@ -100,7 +100,7 @@ License
 =======
 
 `Apache License, Version
-2.0 <https://github.com/pymc-devs/pymc3/blob/master/LICENSE>`__
+2.0 <https://github.com/pymc-devs/pymc3/blob/main/LICENSE>`__
 
 
 Software using PyMC3
@@ -129,7 +129,7 @@ Contributors
 ============
 
 See the `GitHub contributor
-page <https://github.com/pymc-devs/pymc3/graphs/contributors>`__. Also read our `Code of Conduct <https://github.com/pymc-devs/pymc3/blob/master/CODE_OF_CONDUCT.md>`__ guidelines for a better contributing experience.
+page <https://github.com/pymc-devs/pymc3/graphs/contributors>`__. Also read our `Code of Conduct <https://github.com/pymc-devs/pymc3/blob/main/CODE_OF_CONDUCT.md>`__ guidelines for a better contributing experience.
 
 Support
 =======
@@ -158,20 +158,20 @@ Sponsors
 |ODSC|
 
 .. |Binder| image:: https://mybinder.org/badge_logo.svg
-   :target: https://mybinder.org/v2/gh/pymc-devs/pymc3/master?filepath=%2Fdocs%2Fsource%2Fnotebooks
+   :target: https://mybinder.org/v2/gh/pymc-devs/pymc3/main?filepath=%2Fdocs%2Fsource%2Fnotebooks
 .. |Build Status| image:: https://github.com/pymc-devs/pymc3/workflows/pytest/badge.svg
    :target: https://github.com/pymc-devs/pymc3/actions
-.. |Coverage| image:: https://codecov.io/gh/pymc-devs/pymc3/branch/master/graph/badge.svg
+.. |Coverage| image:: https://codecov.io/gh/pymc-devs/pymc3/branch/main/graph/badge.svg
   :target: https://codecov.io/gh/pymc-devs/pymc3
 .. |Dockerhub| image:: https://img.shields.io/docker/automated/pymc/pymc3.svg
   :target: https://hub.docker.com/r/pymc/pymc3
 .. |NumFOCUS| image:: https://www.numfocus.org/wp-content/uploads/2017/03/1457562110.png
    :target: http://www.numfocus.org/
-.. |Quantopian| image:: https://raw.githubusercontent.com/pymc-devs/pymc3/master/docs/quantopianlogo.jpg
+.. |Quantopian| image:: https://raw.githubusercontent.com/pymc-devs/pymc3/main/docs/quantopianlogo.jpg
    :target: https://quantopian.com
 .. |NumFOCUS_badge| image:: https://img.shields.io/badge/powered%20by-NumFOCUS-orange.svg?style=flat&colorA=E1523D&colorB=007D8A
    :target: http://www.numfocus.org/
-.. |ODSC| image:: https://raw.githubusercontent.com/pymc-devs/pymc3/master/docs/odsc_logo.png
+.. |ODSC| image:: https://raw.githubusercontent.com/pymc-devs/pymc3/main/docs/odsc_logo.png
    :target: https://odsc.com
 .. |tidelift| image:: https://img.shields.io/badge/-lifted!-2dd160.svg?colorA=58595b&style=flat&logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABEAAAAOCAYAAADJ7fe0AAAAAXNSR0IArs4c6QAAAAlwSFlzAAAWJQAAFiUBSVIk8AAAAVlpVFh0WE1MOmNvbS5hZG9iZS54bXAAAAAAADx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iIHg6eG1wdGs9IlhNUCBDb3JlIDUuNC4wIj4KICAgPHJkZjpSREYgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj4KICAgICAgPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9IiIKICAgICAgICAgICAgeG1sbnM6dGlmZj0iaHR0cDovL25zLmFkb2JlLmNvbS90aWZmLzEuMC8iPgogICAgICAgICA8dGlmZjpPcmllbnRhdGlvbj4xPC90aWZmOk9yaWVudGF0aW9uPgogICAgICA8L3JkZjpEZXNjcmlwdGlvbj4KICAgPC9yZGY6UkRGPgo8L3g6eG1wbWV0YT4KTMInWQAAAVhJREFUKBV1kj0vBFEUhmd2sdZHh2IlGhKFQuOviEYiNlFodCqtUqPxA%2FwCjUTnDygkGoVERFQaZFlE9nreO%2BdM5u5wkifvuee892Pu3CyEcA0DeIc%2B9IwftJsR6Cko3uCjguZdjuBZhhwmYDjGrOC96WED41UtsgEdGEAPlmAfpuAbFF%2BFZLfoMfRBGzThDtLgePPwBIpdddGzOArhPHUXowbNptE2www6a%2Fm96Y3pHN7oQ1s%2B13pxt1ENaKzBFWyWzaJ%2BRO0C9Jny6VPSoKjLVbMDC5bn5OPuJF%2BBSe95PVEMuugY5AegS9fCh7BedP45hRnj8TC34QQUe9bTZyh2KgvFk2vc8GIlXyTfsvqr6bPpNgv52ynnlomZJNpB70Xhl%2Bf6Sa02p1bApEfnETwxVa%2Faj%2BW%2FFtHltmxS%2FO3krvpTtTnVgu%2F6gvHRFvG78Ef3kOe5PimJXycY74blT5R%2BAAAAAElFTkSuQmCC
    :target: https://tidelift.com/subscription/pkg/pypi-pymc3?utm_source=pypi-pymc3&utm_medium=referral&utm_campaign=enterprise

RELEASE-NOTES.md

@@ -4,15 +4,26 @@
 ### Breaking Changes
 - ⚠ Theano-PyMC has been replaced with Aesara, so all external references to `theano`, `tt`, and `pymc3.theanof` need to be replaced with `aesara`, `at`, and `pymc3.aesaraf` (see [4471](https://github.com/pymc-devs/pymc3/pull/4471)).
 - ArviZ `plots` and `stats` *wrappers* were removed. The functions are now just available by their original names (see [#4549](https://github.com/pymc-devs/pymc3/pull/4471) and `3.11.2` release notes).
+- The GLM submodule has been removed, please use [Bambi](https://bambinos.github.io/bambi/) instead.
+- The `Distribution` keyword argument `testval` has been deprecated in favor of `initval`.
+- `pm.sample` now returns results as `InferenceData` instead of `MultiTrace` by default (see [#4744](https://github.com/pymc-devs/pymc3/pull/4744)).
 - ...
 
 ### New Features
 - The `CAR` distribution has been added to allow for use of conditional autoregressions which often are used in spatial and network models.
+- The dimensionality of model variables can now be parametrized through either of `shape`, `dims` or `size` (see [#4696](https://github.com/pymc-devs/pymc3/pull/4696)):
+  - With `shape` the length of dimensions must be given numerically or as scalar Aesara `Variables`. Numeric entries in `shape` restrict the model variable to the exact length and re-sizing is no longer possible.
+  - `dims` keeps model variables re-sizeable (for example through `pm.Data`) and leads to well defined coordinates in `InferenceData` objects.
+  - The `size` kwarg behaves like it does in Aesara/NumPy. For univariate RVs it is the same as `shape`, but for multivariate RVs it depends on how the RV implements broadcasting to dimensionality greater than `RVOp.ndim_supp`.
+  - An `Ellipsis` (`...`) in the last position of `shape` or `dims` can be used as short-hand notation for implied dimensions.
+- Add `logcdf` method to Kumaraswamy distribution (see [#4706](https://github.com/pymc-devs/pymc3/pull/4706)).
 - ...
 
 ### Maintenance
 - Remove float128 dtype support (see [#4514](https://github.com/pymc-devs/pymc3/pull/4514)).
 - Logp method of `Uniform` and `DiscreteUniform` no longer depends on `pymc3.distributions.dist_math.bound` for proper evaluation (see [#4541](https://github.com/pymc-devs/pymc3/pull/4541)).
+- `Model.RV_dims` and `Model.coords` are now read-only properties. To modify the `coords` dictionary use `Model.add_coord`. Also `dims` or coordinate values that are `None` will be auto-completed (see [#4625](https://github.com/pymc-devs/pymc3/pull/4625)).
+- The length of `dims` in the model is now tracked symbolically through `Model.dim_lengths` (see [#4625](https://github.com/pymc-devs/pymc3/pull/4625)).
 - ...
 
 ## PyMC3 3.11.2 (14 March 2021)

benchmarks/asv.conf.json

@@ -13,9 +13,9 @@
     // project being benchmarked
     "repo": "..",
 
-    // List of branches to benchmark. If not provided, defaults to "master"
+    // List of branches to benchmark. If not provided, defaults to "main"
     // (for git) or "tip" (for mercurial).
-    "branches": ["master"],
+    "branches": ["main"],
 
     // The DVCS being used.  If not set, it will be automatically
     // determined from "repo" by looking at the protocol in the URL