Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 5 additions & 7 deletions docs/backend_authors.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ entries, load each entry's `provider`, and call its hooks at the right point in
the [PEP 517][] build. Because the entries are an explicit ordered list, there
is no dependency graph to solve.

The easiest way to do this is to take a build-time dependency on this package
and call the reference loader in {mod}`dynamic_metadata.loader`, which is what
this page shows. **You do not have to depend on us**, though: you can vendor the
loader or reimplement it from a precise description of its behaviour — see
The easiest way is a build-time dependency on this package, calling the
reference loader in {mod}`dynamic_metadata.loader`what this page shows. **You
do not have to depend on us**, though: you can vendor the loader or reimplement
it from a precise description of its behaviour — see
[Reimplementing the loader](backend_authors_reimplement.md).

## Where plugins plug into a build
Expand Down Expand Up @@ -93,9 +93,7 @@ project = process_dynamic_metadata(project, entries, build_state="wheel")

After it returns, anything left in `project["dynamic"]` was declared but never
produced — surface that as an error if your backend requires every dynamic field
to be filled. You still have access to the original dynamic list, of course, if
you need it. The exact ordering, validation, and merge rules the loader applies
are documented in
to be filled. The exact ordering, validation, and merge rules are documented in
[Reimplementing the loader](backend_authors_reimplement.md#resolving-the-metadata);
you only need them if you are replacing this call.

Expand Down
131 changes: 68 additions & 63 deletions docs/backend_authors_reimplement.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,14 @@

**You do not need to depend on dynamic-metadata to support plugins.** You can
vendor {mod}`dynamic_metadata.loader` or reimplement it from the description
below — the behaviour is what matters, not the code. This page describes that
behaviour precisely enough to replace the loader, and every example here is
self-contained: none of it imports from `dynamic_metadata`.
below — the behaviour is what matters. Every example here is self-contained:
nothing imports from `dynamic_metadata`.

It assumes you have read [For backend authors](backend_authors.md) for where the
hooks plug into a PEP 517 build, how to read the configuration, and what
`build_state` is. Reimplementing means replacing four things that page gets from
the package: loading a provider, detecting its optional hooks, the field
taxonomy, and the resolution loop (with its merge rules).
This page assumes you have read [For backend authors](backend_authors.md) for
how the hooks plug into a PEP 517 build, how to read the configuration, and what
`build_state` is. Reimplementing replaces four things that page gets from the
package: loading a provider, detecting its optional hooks, the field taxonomy,
and the resolution loop with its merge rules.

## Loading a provider

Expand All @@ -27,7 +26,7 @@ are bound methods and can share state through `self`); an already-instantiated
object (a `module:instance` entry point) is used directly.

```python
from importlib.metadata import entry_points # importlib_metadata on <3.10
from importlib.metadata import entry_points


def load_provider(spec):
Expand All @@ -36,30 +35,31 @@ def load_provider(spec):
matches = [
ep for ep in entry_points(group="dynamic_metadata.provider") if ep.name == spec
]
if not matches:
raise ModuleNotFoundError(f"Unknown provider {spec!r}")
if len(matches) > 1: # two distributions registered the same name
raise RuntimeError(f"Provider {spec!r} is ambiguous: {matches}")
obj = matches[0].load()
return obj() if isinstance(obj, type) else obj
match matches:
case []:
raise ModuleNotFoundError(f"Unknown provider {spec!r}")
case [ep]:
obj = ep.load()
return obj() if isinstance(obj, type) else obj
case _: # two distributions registered the same name
raise RuntimeError(f"Provider {spec!r} is ambiguous: {matches}")
```

Names are conventionally prefixed with the providing package (the bundled
plugins are `dynamic_metadata.regex`, …); a name registered by more than one
distribution is a hard error rather than a non-deterministic pick.

The inline-table `path` lets a plugin live inside the project being built (a
local directory not installed as a package). To import it, install a
`sys.meta_path` finder scoped to that directory for the single import —
mirroring how `pyproject_hooks` handles PEP 517's `backend-path`. Scoping
matters: the in-tree provider must win over any same-named installed module, and
a missing provider must raise rather than silently importing the wrong one.
distribution is a hard error, not a non-deterministic pick.

The inline-table `path` lets a plugin live inside the project being built. To
import it, install a `sys.meta_path` finder scoped to that directory for the
single import, mirroring how `pyproject_hooks` handles PEP 517's `backend-path`.
Scoping matters: the in-tree provider must win over any same-named installed
module, and a missing provider must raise rather than import the wrong one.
`load_provider` and `_ProviderPathFinder` in {mod}`dynamic_metadata.loader` are
a worked implementation you can copy.

You will load each entry's provider several times (for requirements, metadata,
and the METADATA 2.2 pass), so a small generator that loads the provider and
splits off the plugin-specific keys as `settings` is handy:
You load each entry's provider several times (for requirements, metadata, and
the METADATA 2.2 pass), so a small generator that loads the provider and splits
off the plugin-specific keys as `settings` is handy:

```python
def load_dynamic_metadata(entries):
Expand All @@ -80,7 +80,7 @@ detected by their presence — a plain `hasattr` check:
| `get_requires_for_dynamic_metadata` | `get_requires_for_dynamic_metadata` | during `get_requires_for_build_*` |
| `dynamic_wheel(settings)` | `dynamic_wheel` | after metadata, for METADATA 2.2 |

The requirements and METADATA 2.2 passes are then the loops shown on the
The requirements and METADATA 2.2 passes are the loops shown on the
[backend authors](backend_authors.md#collecting-build-requirements) page, with
each `isinstance(provider, SomeProtocol)` replaced by `hasattr(provider, ...)`:

Expand All @@ -97,8 +97,8 @@ def collect_requires(entries):

You need to know which `[project]` fields a provider may set and what _shape_
each value has, because the shape decides how a fragment merges. `name` and
`dynamic` are intentionally excluded — a provider can set neither. This mirrors
`dynamic_metadata.info`; copy it if you would rather not maintain your own:
`dynamic` are intentionally excluded. This mirrors `dynamic_metadata.info`; copy
it if you would rather not maintain your own:

```python
STR_FIELDS = {"version", "description", "requires-python", "license"}
Expand Down Expand Up @@ -129,8 +129,6 @@ snapshot of the project as resolved so far**, returns a `dict` fragment of
one's output with `project[field]`; a forward reference is just a `KeyError`,
and cycles are impossible because nothing can read ahead.

Here is the loop with the validation and merge details inlined:

```python
from types import MappingProxyType

Expand Down Expand Up @@ -170,43 +168,50 @@ def process_dynamic_metadata(project, entries, build_state):

Key points a reimplementation must preserve:

- **`snapshot` is a live read-only view** of `result`. Because
`MappingProxyType` wraps the same dict, every provider sees fields written by
earlier entries without you rebuilding the snapshot. It is read-only so a
provider cannot mutate the project out from under the loader.
- **`snapshot` is a live read-only view** of `result`. `MappingProxyType` wraps
the same dict, so every provider sees fields written by earlier entries with
no rebuild; being read-only stops a provider mutating the project under the
loader.
- **Validate against the field taxonomy.** A returned field must be in
`ALL_FIELDS` (`name` and `dynamic` are intentionally excluded) and must be
listed in `project.dynamic`.
- **A field is resolved once written:** remove it from `dynamic`. After the
loop, anything left in `dynamic` was declared but never produced — surface
that as an error if your backend requires every dynamic field to be filled.
`ALL_FIELDS` and listed in `project.dynamic`.
- **A field is resolved once written:** remove it from `dynamic`. Anything left
after the loop was declared but never produced — surface that as an error if
your backend requires every dynamic field to be filled.

## Merge semantics (PEP 808)

`merge(field, current, addition)` combines a field's current value with a
provider's fragment. The rule depends on the field's shape (from the taxonomy
above) and on whether `current` is a _static_ value or an _earlier entry's_
output:

- **List fields** (`dependencies`, `classifiers`, `authors`, …): concatenate,
existing entries first. A provider returns only its additions.
- **Table fields** (`urls`, `scripts`, `entry-points`, `optional-dependencies`,
…): add keys only ([PEP 808][] is add-only). A provider may **not** change the
value of a key that already exists — raise if it tries. `entry-points` and
`optional-dependencies` nest one level (a table of groups/extras, each holding
a table or list), so merge them recursively with the same add-only rule.
- **Scalar fields** (`version`, `description`, `requires-python`, `license`,
`readme`): cannot be extended.
- Against a **static** value this is the illegal "static _and_ dynamic" case —
raise.
- Against an **earlier entry's** output (tracked by `produced`) a later entry
**replaces** it, enabling a transform pipeline (one plugin extracts a
version, another normalizes it). This is the
`field in produced and field in SCALAR_FIELDS` branch above.

The `produced` set is what distinguishes "a user wrote this in `[project]`" from
"an earlier plugin computed this", which is the only thing the static-vs-replace
decision turns on. `_merge_metadata` and `_merge_dict` in
provider's fragment, dispatching on the field's shape:

```python
def merge(field, current, addition):
match field:
case _ if field in LIST_STR_FIELDS | LIST_DICT_FIELDS:
return current + addition # concatenate, existing entries first
case "entry-points" | "optional-dependencies":
return merge_table(current, addition, nested=True) # table of tables/lists
case _ if field in DICT_STR_FIELDS:
return merge_table(current, addition, nested=False)
case _: # scalar — cannot be extended
raise ValueError(f"{field!r} is static and cannot also be dynamic")
```

- **List fields** concatenate, existing entries first; a provider returns only
its additions.
- **Table fields** are add-only ([PEP 808][]): a provider adds keys but may
**not** change an existing key's value — `merge_table` raises if it tries.
`entry-points` and `optional-dependencies` nest one level (a table of
groups/extras), so merge recursively with the same rule.
- **Scalar fields** cannot be extended. Reaching this branch means merging
against a **static** value — the illegal "static _and_ dynamic" case. Merging
against an **earlier entry's** output never gets here: the loop's
`field in produced and field in SCALAR_FIELDS` branch **replaces** instead,
enabling a transform pipeline (one plugin extracts a version, another
normalizes it).

The `produced` set is the only thing the static-vs-replace decision turns on: it
distinguishes "a user wrote this in `[project]`" from "an earlier plugin
computed this". `_merge_metadata` and `_merge_dict` in
{mod}`dynamic_metadata.loader` are a per-shape implementation you can copy.

[PEP 808]: https://peps.python.org/pep-0808/
2 changes: 0 additions & 2 deletions docs/plugin_authors.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,6 @@ project as resolved so far; read another field's value with

## Optional hooks

There are three optional hooks.

### Receiving the build state

```python
Expand Down
2 changes: 1 addition & 1 deletion docs/users.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ backends known to support this currently include:

## Example: regex

An example regex plugin is provided in this package. It is used like this:
The bundled regex plugin is used like this:

```toml
[build-system]
Expand Down
Loading