Enhance ids generated by parametrization. (#21)

Author: tobiasraabe

docs/changes.rst

@@ -17,6 +17,8 @@ all releases are available on `Anaconda.org <https://anaconda.org/pytask/pytask>
 - :gh:`18` changes the documentation theme to alabaster.
 - :gh:`19` adds some changes related to ignored folders.
 - :gh:`20` fixes copying code examples in the documentation.
+- :gh:`21` enhances the ids generated by parametrization, allows to change them via the
+  ``ids`` argument, and adds tutorials.
 - :gh:`23` allows to specify paths via the configuration file, documents the cli and
   configuration options.
 

docs/how_to_guides/how_to_extend_parametrizations.rst

@@ -0,0 +1,69 @@
+How to extend parametrizations
+==============================
+
+Parametrization helps you to reuse code and quickly scale from one to a multitude of
+tasks. Sometimes, these tasks are expensive because they take long or require a lot of
+resources. Thus, you only want to run them if really necessary.
+
+
+The problem
+-----------
+
+There are two problems when extending parametrizations which might trigger accidental
+reruns of tasks.
+
+
+IDs
+~~~
+
+If you do not know how ids for parametrized tasks are produced, read the following
+:ref:`section in the tutorial about parametrization <how_to_parametrize_a_task_the_id>`.
+
+The problem is that argument values which are not booleans, numbers or strings produce
+positionally dependent ids. The position might change if you extend the parametrization
+which re-executes a task.
+
+To resolve the problem, you can choose one of the two solutions in the tutorial. Either
+pass a function to convert non-standard objects to suitable representations or pass your
+own ids.
+
+
+Modification of the task module
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To extend your parametrization, you would normally change the module in which the task
+is defined. By default, this triggers a re-run of the task.
+
+
+Solution: side-effect
+---------------------
+
+The problem can be resolved by introducing a side-effect. Add another module with the
+following content.
+
+.. code-block:: python
+
+    # Content of side_effect.py
+
+    ARG_VALUES = [(0,), (1,)]
+    IDS = ["first_tuple", "second_tuple"]
+
+And change the task module to
+
+.. code-block:: python
+
+    import pytask
+    from side_effect import ARG_VALUES, IDS
+
+
+    @pytask.mark.parametrize("i", ARG_VALUES, ids=IDS)
+    def task_example(i):
+        pass
+
+The key idea is to not reference the ``side_effect.py`` module as a dependency of the
+task. Now, you can extend the parametrization without re-executing former tasks.
+
+**Caveat**: Be careful, because pytask does not care about which object is passed to the
+parametrized function. Thus, it would be better to replace ``IDS`` with a function which
+hashes the tuples to recognize changes as shown in the :ref:`tutorial
+<how_to_parametrize_a_task_convert_other_objects>`.

docs/how_to_guides/index.rst

@@ -9,3 +9,4 @@ pytask.
    :maxdepth: 1
 
    how_to_write_a_plugin
+   how_to_extend_parametrizations

docs/tutorials/how_to_parametrize_a_task.rst

@@ -140,3 +140,114 @@ The signature can be passed in three different formats.
    .. code-block:: python
 
        ["first_argument", "second_argument"]
+
+
+.. _how_to_parametrize_a_task_the_id:
+
+The id
+------
+
+Every task has a unique id which can be used to :doc:`select it <how_to_select_tasks>`.
+The normal id combines the path to the module where the task is defined, a double colon,
+and the name of the task function. Here is an example.
+
+.. code-block::
+
+    ../task_example.py::task_example
+
+This behavior would produce duplicate ids for parametrized tasks. Therefore, there exist
+multiple mechanisms to produce unique ids.
+
+
+Auto-generated ids
+~~~~~~~~~~~~~~~~~~
+
+To avoid duplicate task ids, the ids of parametrized tasks are extended with
+descriptions of the values they are parametrized with. Booleans, floats, integers and
+strings enter the task id directly. For example, a task function which receives four
+arguments, ``True``, ``1.0``, ``2``, and ``"hello"``, one of each dtype, has the
+following id.
+
+.. code-block::
+
+    task_example.py::task_example[True-1.0-2-hello]
+
+Arguments with other dtypes cannot be easily converted to strings and, thus, are
+replaced with a combination of the argument name and the iteration counter.
+
+For example, the following function is parametrized with tuples.
+
+.. code-block:: python
+
+    @pytask.mark.parametrized("i", [(0,), (1,)])
+    def task_example(i):
+        pass
+
+Since the tuples are not converted to strings, the ids of the two tasks are
+
+.. code-block::
+
+    task_example.py::task_example[i0]
+    task_example.py::task_example[i1]
+
+
+.. _how_to_parametrize_a_task_convert_other_objects:
+
+Convert other objects
+~~~~~~~~~~~~~~~~~~~~~
+
+To change the representation of tuples and other objects, you can pass a function to the
+``ids`` argument of the :func:`~_pytask.parametrize.parametrize` decorator. The function
+is called for every argument and may return a boolean, number, or string which will be
+integrated into the id. For every other return, the auto-generated value is used.
+
+To get a unique representation of a tuple, we can use the hash value.
+
+.. code-block:: python
+
+    def tuple_to_hash(value):
+        if isinstance(value, tuple):
+            return hash(a)
+
+
+    @pytask.mark.parametrized("i", [(0,), (1,)], ids=tuple_to_hash)
+    def task_example(i):
+        pass
+
+This produces the following ids:
+
+.. code-block::
+
+    task_example.py::task_example[3430018387555]  # (0,)
+    task_example.py::task_example[3430019387558]  # (1,)
+
+
+User-defined ids
+~~~~~~~~~~~~~~~~
+
+Instead of a function, you can also pass a list or another iterable of id values via
+``ids``.
+
+This code
+
+.. code-block:: python
+
+    @pytask.mark.parametrized("i", [(0,), (1,)], ids=["first", "second"])
+    def task_example(i):
+        pass
+
+produces these ids
+
+.. code-block::
+
+    task_example.py::task_example[first]  # (0,)
+    task_example.py::task_example[second]  # (1,)
+
+This is arguably the easiest way to change the representation of many objects at once
+while also producing ids which are easy to remember and type.
+
+
+Further reading
+---------------
+
+- :doc:`../how_to_guides/how_to_extend_parametrizations`.

docs/tutorials/how_to_select_tasks.rst

@@ -50,6 +50,9 @@ for the analysis.
 Expressions
 -----------
 
+General
+~~~~~~~
+
 Expressions are similar to markers and offer the same syntax but target the task ids.
 Assume you have the following tasks.
 
@@ -85,3 +88,27 @@ To execute a single task, say ``task_run_this_one`` in ``task_example.py``, use
 .. code-block:: console
 
     $ pytask -k task_example.py::task_run_this_one
+
+
+.. _how_to_select_tasks_parametrization:
+
+Parametrization
+~~~~~~~~~~~~~~~
+
+If you have a task which is parametrized, you can select individual parametrizations.
+
+.. code-block:: python
+
+    @pytask.mark.parametrize("i", range(2))
+    def task_parametrized(i):
+        pass
+
+To run the task where ``i = 1``, type
+
+.. code-block:: bash
+
+    $ pytask -k task_parametrized[1]
+
+Booleans, floats, integers, and strings are used in the task id as they are, but all
+other Python objects like tuples are replaced with a combination of the argument name
+and an iteration counter. Multiple arguments are separated via dashes.

src/_pytask/debugging.py

@@ -48,8 +48,14 @@ def pytask_parse_config(config, config_from_cli, config_from_file):
     )
 
 
-@hookimpl
+@hookimpl(trylast=True)
 def pytask_post_parse(config):
+    """Post parse the configuration.
+
+    Register the plugins in this step to let other plugins influence the pdb or trace
+    option and may be disable it. Especially thinking about pytask-parallel.
+
+    """
     if config["pdb"]:
         config["pm"].register(PdbDebugger)
 
@@ -69,6 +75,8 @@ def pytask_execute_task(task):
 
 
 def wrap_function_for_post_mortem_debugging(function):
+    """Wrap the function for post-mortem debugging."""
+
     @functools.wraps(function)
     def wrapper(*args, **kwargs):
         try:
@@ -93,6 +101,8 @@ def pytask_execute_task(task):
 
 
 def wrap_function_for_tracing(function):
+    """Wrap the function for tracing."""
+
     @functools.wraps(function)
     def wrapper(*args, **kwargs):
         pdb.runcall(function, *args, **kwargs)