Skip to content

Commit 8ae7ea4

Browse files
asmeurerasmeurer
committed
Split implementation notes into separate pages
1 parent c68389e commit 8ae7ea4

File tree

4 files changed

+114
-121
lines changed

4 files changed

+114
-121
lines changed

‎docs/dev/implementation-notes.md‎

Lines changed: 0 additions & 121 deletions
Original file line numberDiff line numberDiff line change
@@ -1,96 +1,5 @@
11
# Implementation Notes
22

3-
This page outlines some notes on the implementation of array-api-compat. These
4-
details are not important for users of the package, but they may be useful to
5-
contributors.
6-
7-
## Special Considerations
8-
9-
array-api-compat requires some special development considerations that are
10-
different from most other Python libraries. The goal of array-api-compat is to
11-
be a small library that packages can either vendor or add as a dependency to
12-
implement array API support. Consequently, certain design considerations
13-
should be taken into account:
14-
15-
- *No Hard Dependencies.* Although array-api-compat "depends" on NumPy, CuPy,
16-
PyTorch, etc., it does not hard depend on them. These libraries are not
17-
imported unless either an array object is passed to
18-
{func}`~.array_namespace()`, or the specific `array_api_compat.<namespace>`
19-
sub-namespace is explicitly imported.
20-
21-
- *Vendorability.* array-api-compat should be [vendorable](vendoring). This
22-
means that, for instance, all imports in the library are relative imports.
23-
No code in the package specifically references the name `array_api_compat`
24-
(we also support renaming the package to something else).
25-
Vendorability support is tested in `tests/test_vendoring.py`.
26-
27-
- *Pure Python.* To make array-api-compat as easy as possible to add as a
28-
dependency, the code is all pure Python.
29-
30-
- *Minimal Wrapping Only.* The wrapping functionality is minimal. This means
31-
that if something is difficult to wrap using pure Python, or if trying to
32-
support some array API behavior would require a significant amount of code,
33-
we prefer to leave the behavior as an upstream issue for the array library,
34-
and [document it as a known difference](../supported-array-libraries.md).
35-
36-
This also means that we do not at this point in time implement anything
37-
other than wrappers for functions in the standard, and basic [helper
38-
functions](../helper-functions.rst) that would be useful for most users of
39-
array-api-compat. The addition of functions that are not part of the array
40-
API standard is currently out-of-scope for this package (see the
41-
[Scope](scope) section of the documentation).
42-
43-
- *No Side-Effects*. array-api-compat behavior should be localized to only the
44-
specific code that imports and uses it. It should be invisible to end-users
45-
or users of dependent codes. This in particular implies to the next two
46-
points.
47-
48-
- *No Monkey Patching.* `array-api-compat` should not attempt to modify
49-
anything about the underlying library. It is a *wrapper* library only.
50-
51-
- *No Modifying the Array Object.* The array (or tensor) object of the array
52-
library cannot be modified. This also precludes the creation of array
53-
subclasses or wrapper classes.
54-
55-
Any non-standard behavior that is built-in to the array object, such as the
56-
behavior of [array
57-
methods](https://data-apis.org/array-api/latest/API_specification/array_object.html),
58-
is therefore left unwrapped. Users can workaround issues by using
59-
corresponding [elementwise
60-
functions](https://data-apis.org/array-api/latest/API_specification/elementwise_functions.html)
61-
instead of
62-
[operators](https://data-apis.org/array-api/latest/API_specification/array_object.html#operators),
63-
and by using the [helper functions](../helper-functions.rst) provided by
64-
array-api-compat instead of attributes or methods like `x.to_device()`.
65-
66-
- *Avoid Restricting Behavior that is Outside the Scope of the Standard.* All
67-
array libraries have functions and behaviors that are outside of the scope
68-
of what is specified by the standard. These behaviors should be left intact
69-
whenever possible, unless the standard explicitly disallows something. This
70-
means
71-
72-
- All namespaces are *extended* with wrapper functions. You may notice the
73-
extensive use of `import *` in various files in `array_api_compat`. While
74-
this would normally be questionable, this is the [one actual legitimate
75-
use-case for `import *`](https://peps.python.org/pep-0008/#imports), to
76-
re-export names from an external namespace.
77-
78-
- All wrapper functions pass `**kwargs` through to the wrapped function.
79-
80-
- Input types not supported by the standard should work if they work in the
81-
underlying wrapped function (for instance, Python scalars or `np.ndarray`
82-
subclasses).
83-
84-
By keeping underlying behaviors intact, it is easier for libraries to swap
85-
out NumPy or other array libraries for array-api-compat, and it is easier
86-
for libraries to write array library-specific code paths.
87-
88-
The onus is on users of array-api-compat to ensure their array API code is
89-
portable, e.g., by testing against [array-api-strict](array-api-strict).
90-
91-
92-
## Avoiding Code Duplication
93-
943
Since NumPy, CuPy, and to a degree, Dask, are nearly identical in behavior,
954
most wrapping logic can be shared between them. Wrapped functions that have
965
the same logic between multiple libraries are in `array_api_compat/common/`.
@@ -138,33 +47,3 @@ identical PyTorch uses a similar layout in `array_api_compat/torch/`, but it
13847
differs enough from NumPy/CuPy that very few common wrappers for those
13948
libraries are reused. Dask is close to NumPy in behavior and so most Dask
14049
functions also reuse the NumPy/CuPy common wrappers.
141-
142-
## Tests
143-
144-
The majority of the behavior for array-api-compat is tested by the
145-
[array-api-tests](https://github.com/data-apis/array-api-tests) test suite for
146-
the array API standard. There are also array-api-compat specific tests in
147-
[`tests/`](https://github.com/data-apis/array-api-compat/tree/main/tests).
148-
These tests should be limited to things that are not tested by the test suite,
149-
e.g., tests for [helper functions](../helper-functions.rst) or for behavior
150-
that is not strictly required by the standard.
151-
152-
array-api-tests is run against all supported libraries are tested on CI
153-
([except for JAX](jax-support)). This is achieved by a [reusable GitHub Actions
154-
Workflow](https://github.com/data-apis/array-api-compat/blob/main/.github/workflows/array-api-tests.yml).
155-
Most libraries have tests that must be xfailed or skipped for various reasons.
156-
These are defined in specific `<library>-xfails.txt` files and are
157-
automatically forwarded to array-api-tests.
158-
159-
You may often need to update these xfail files, either to add new xfails
160-
(e.g., because of new test suite features, or because a test that was
161-
previously thought to be passing actually flaky fails). Try to keep the xfails
162-
files organized, with comments pointing to upstream issues whenever possible.
163-
164-
From time to time, xpass tests should be removed from the xfail files, but be
165-
aware that many xfail tests are flaky, so an xpass should only be removed if
166-
you know that the underlying issue has been fixed.
167-
168-
Array libraries that require a GPU to run (currently only CuPy) cannot be
169-
tested on CI. There is a helper script `test_cupy.sh` that can be used to
170-
manually test CuPy on a machine with a CUDA GPU.

‎docs/dev/index.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@ It is recommended that contributors read through this documentation.
66
```{toctree}
77
:titlesonly:
88
9+
special-considerations.md
910
implementation-notes.md
11+
tests.md
1012
releasing.md
1113
```

‎docs/dev/special-considerations.md‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# Special Considerations
2+
3+
array-api-compat requires some special development considerations that are
4+
different from most other Python libraries. The goal of array-api-compat is to
5+
be a small library that packages can either vendor or add as a dependency to
6+
implement array API support. Consequently, certain design considerations
7+
should be taken into account:
8+
9+
- *No Hard Dependencies.* Although array-api-compat "depends" on NumPy, CuPy,
10+
PyTorch, etc., it does not hard depend on them. These libraries are not
11+
imported unless either an array object is passed to
12+
{func}`~.array_namespace()`, or the specific `array_api_compat.<namespace>`
13+
sub-namespace is explicitly imported.
14+
15+
- *Vendorability.* array-api-compat should be [vendorable](vendoring). This
16+
means that, for instance, all imports in the library are relative imports.
17+
No code in the package specifically references the name `array_api_compat`
18+
(we also support renaming the package to something else).
19+
Vendorability support is tested in `tests/test_vendoring.py`.
20+
21+
- *Pure Python.* To make array-api-compat as easy as possible to add as a
22+
dependency, the code is all pure Python.
23+
24+
- *Minimal Wrapping Only.* The wrapping functionality is minimal. This means
25+
that if something is difficult to wrap using pure Python, or if trying to
26+
support some array API behavior would require a significant amount of code,
27+
we prefer to leave the behavior as an upstream issue for the array library,
28+
and [document it as a known difference](../supported-array-libraries.md).
29+
30+
This also means that we do not at this point in time implement anything
31+
other than wrappers for functions in the standard, and basic [helper
32+
functions](../helper-functions.rst) that would be useful for most users of
33+
array-api-compat. The addition of functions that are not part of the array
34+
API standard is currently out-of-scope for this package (see the
35+
[Scope](scope) section of the documentation).
36+
37+
- *No Side-Effects*. array-api-compat behavior should be localized to only the
38+
specific code that imports and uses it. It should be invisible to end-users
39+
or users of dependent codes. This in particular implies to the next two
40+
points.
41+
42+
- *No Monkey Patching.* `array-api-compat` should not attempt to modify
43+
anything about the underlying library. It is a *wrapper* library only.
44+
45+
- *No Modifying the Array Object.* The array (or tensor) object of the array
46+
library cannot be modified. This also precludes the creation of array
47+
subclasses or wrapper classes.
48+
49+
Any non-standard behavior that is built-in to the array object, such as the
50+
behavior of [array
51+
methods](https://data-apis.org/array-api/latest/API_specification/array_object.html),
52+
is therefore left unwrapped. Users can workaround issues by using
53+
corresponding [elementwise
54+
functions](https://data-apis.org/array-api/latest/API_specification/elementwise_functions.html)
55+
instead of
56+
[operators](https://data-apis.org/array-api/latest/API_specification/array_object.html#operators),
57+
and by using the [helper functions](../helper-functions.rst) provided by
58+
array-api-compat instead of attributes or methods like `x.to_device()`.
59+
60+
- *Avoid Restricting Behavior that is Outside the Scope of the Standard.* All
61+
array libraries have functions and behaviors that are outside of the scope
62+
of what is specified by the standard. These behaviors should be left intact
63+
whenever possible, unless the standard explicitly disallows something. This
64+
means
65+
66+
- All namespaces are *extended* with wrapper functions. You may notice the
67+
extensive use of `import *` in various files in `array_api_compat`. While
68+
this would normally be questionable, this is the [one actual legitimate
69+
use-case for `import *`](https://peps.python.org/pep-0008/#imports), to
70+
re-export names from an external namespace.
71+
72+
- All wrapper functions pass `**kwargs` through to the wrapped function.
73+
74+
- Input types not supported by the standard should work if they work in the
75+
underlying wrapped function (for instance, Python scalars or `np.ndarray`
76+
subclasses).
77+
78+
By keeping underlying behaviors intact, it is easier for libraries to swap
79+
out NumPy or other array libraries for array-api-compat, and it is easier
80+
for libraries to write array library-specific code paths.
81+
82+
The onus is on users of array-api-compat to ensure their array API code is
83+
portable, e.g., by testing against [array-api-strict](array-api-strict).

‎docs/dev/tests.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Tests
2+
3+
The majority of the behavior for array-api-compat is tested by the
4+
[array-api-tests](https://github.com/data-apis/array-api-tests) test suite for
5+
the array API standard. There are also array-api-compat specific tests in
6+
[`tests/`](https://github.com/data-apis/array-api-compat/tree/main/tests).
7+
These tests should be limited to things that are not tested by the test suite,
8+
e.g., tests for [helper functions](../helper-functions.rst) or for behavior
9+
that is not strictly required by the standard.
10+
11+
array-api-tests is run against all supported libraries are tested on CI
12+
([except for JAX](jax-support)). This is achieved by a [reusable GitHub Actions
13+
Workflow](https://github.com/data-apis/array-api-compat/blob/main/.github/workflows/array-api-tests.yml).
14+
Most libraries have tests that must be xfailed or skipped for various reasons.
15+
These are defined in specific `<library>-xfails.txt` files and are
16+
automatically forwarded to array-api-tests.
17+
18+
You may often need to update these xfail files, either to add new xfails
19+
(e.g., because of new test suite features, or because a test that was
20+
previously thought to be passing actually flaky fails). Try to keep the xfails
21+
files organized, with comments pointing to upstream issues whenever possible.
22+
23+
From time to time, xpass tests should be removed from the xfail files, but be
24+
aware that many xfail tests are flaky, so an xpass should only be removed if
25+
you know that the underlying issue has been fixed.
26+
27+
Array libraries that require a GPU to run (currently only CuPy) cannot be
28+
tested on CI. There is a helper script `test_cupy.sh` that can be used to
29+
manually test CuPy on a machine with a CUDA GPU.

0 commit comments

Comments
 (0)