Skip to content

Commit 373b5bc

Browse files
Ralph UrlusRUrlus
authored andcommitted
DOC: Update documentation to new version
1 parent e357a0b commit 373b5bc

File tree

10 files changed

+823
-473
lines changed

10 files changed

+823
-473
lines changed

carma_config.cmake

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ CMAKE_MINIMUM_REQUIRED(VERSION 3.12)
1515
# -- ENABLE_CARMA_EXTRA_DEBUG --
1616
# This option enables additional debuggin statements
1717
OPTION(ENABLE_CARMA_EXTRA_DEBUG "Enable CARMA_EXTRA_DEBUG" OFF)
18+
OPTION(ENABLE_ARMA_EXTRA_DEBUG "Enable ARMA_EXTRA_DEBUG" OFF)
1819

1920
# -- ENABLE_CARMA_SOFT_STEAL --
2021
# When stealing the data of an array replace it with
@@ -35,5 +36,6 @@ OPTION(ENABLE_CARMA_HARD_STEAL "Enable CARMA_HARD_STEAL" OFF)
3536
OPTION(ENABLE_CARMA_DONT_REQUIRE_OWNDATA "Enable CARMA_DONT_REQUIRE_OWNDATA" OFF)
3637

3738
# -- REQUIRE_F_CONTIGUOUS --
38-
# Do NOT copy c-style arrays, default behaviour is to copy c-style arrays
39+
# Do NOT copy C-contiguous arrays, default behaviour is to copy C-contiguous arrays to Fortran order as this is what Armadillo is optimised for.
40+
# Note that on the conversion back it is assumed that the memory of a Armadillo object has Fortran order layout.
3941
OPTION(ENABLE_CARMA_DONT_REQUIRE_F_CONTIGUOUS "Enable CARMA_DONT_REQUIRE_F_CONTIGUOUS" OFF)

docs/source/basic_usage.rst

Lines changed: 38 additions & 250 deletions
Original file line numberDiff line numberDiff line change
@@ -5,50 +5,28 @@ First steps
55
###########
66

77
Carma relies on Pybind11 for the generation of the bindings and casting of the arguments from Python to C++.
8-
Make sure you are familiar with `Pybind11 <https://pybind11.readthedocs.io/en/stable/intro.html>`__ before continuing on.
8+
Make sure you are familiar with `Pybind11 <https://pybind11.readthedocs.io/en/stable/intro.html>`_ before continuing on.
99

10-
Requirements
11-
************
10+
You can embed CARMA in a Pybind11 project using CMake command
1211

13-
Before starting you should check that your environment is set up properly.
12+
.. code-block:: cmake
1413
15-
Carma has two requirements:
14+
ADD_SUBDIRECTORY(extern/carma)
15+
TARGET_LINK_LIBRARIES(<your_target> PRIVATE carma)
1616
17-
* `Pybind11 <https://github.com/pybind/pybind11>`__
18-
* `Armadillo <http://arma.sourceforge.net/download.html>`__
17+
See `Pybind11's CMake build system documentation <https://pybind11.readthedocs.io/en/stable/compiling.html#building-with-cmake>`_ or `CARMA's examples <https://github.com/RUrlus/carma/blob/stable/examples/CMakeLists.txt>`_ for a start.
1918

20-
Carma provides both tests and examples that can be compiled without any additional libraries, although you will need additional libraries to use Armadillo in practice.
21-
22-
.. note:: The Pybind11 and Armadillo libraries are linked with `carma` as submodule in `third_party` directory. To get them using :bash:`git clone`, don't forget to use :bash:`--recursive` option.
23-
24-
Manual compilation
25-
******************
26-
27-
Although using a build system is suggested, the :bash:`examples/example.cpp` can be compiled with:
28-
29-
**on Linux**
30-
31-
.. code-block:: bash
32-
33-
c++ -O3 -Wall -shared -std=c++14 -fPIC -larmadillo `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix`
34-
35-
**on MacOS**
36-
37-
.. code-block:: bash
19+
.. note::
3820

39-
c++ -O3 -Wall -shared -std=c++14 -larmadillo -undefined dynamic_lookup `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix`
40-
41-
Manual compilation requires that Pybind11 and Armadillo are discoverable.
42-
43-
Build systems
44-
*************
21+
At the time of writing CARMA requires a forked version of Armadillo that
22+
uses Numpy's allocator and deallocator.
23+
The forked version is shipped with library and provided at build time, see :ref:`Build Configuration` for details.
4524

4625
The tests and examples can be compiled using CMake.
4726
CMake can be installed with :bash:`pip install cmake`, your package manager or directly from `cmake <http://cmake.org/download/>`__.
4827

4928
.. code-block:: bash
5029
51-
git submodule update --init
5230
mkdir build
5331
cd build
5432
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=. -DBUILD_EXAMPLES=true -DBUILD_TESTS=true .. && make install
@@ -77,237 +55,47 @@ Installation directory contains
7755

7856
.. code-block::
7957
80-
include # carma headers
81-
tests # carma python tests with python module (if enabled using -DBUILD_TESTS=on)
82-
examples # carma python examples with python module (if enabled using -DBUILD_EXAMPLES=on)
83-
84-
Advanced build system configuration
85-
***********************************
86-
87-
`Carma` requirements can be provided out of :bash:`third_party` directory.
88-
89-
To do so, you have to define locations of `armadillo` or/and `pybind11` by setting:
90-
91-
.. code-block:: bash
92-
93-
-DARMADILLO_ROOT_DIR=/path/to/armadillo-code/root/directory
94-
95-
.. code-block:: bash
96-
97-
-DPYBIND11_ROOT_DIR=/path/to/pybind11/root/directory
98-
99-
Sometimes, if you have multiple python interpret available in your system,
100-
you may want to specify the one you want. Python detection is delegated to pybind11 dependency
101-
and you can drive it using
102-
103-
.. code-block:: bash
104-
105-
-DPYTHON_PREFIX_PATH=/path/to/directory/containing/your/favorite/python/interpret
106-
-DPYBIND11_PYTHON_VERSION=/version/of/your/favorite/python/interpret
107-
108-
e.g.:
109-
110-
.. code-block:: bash
111-
112-
-DPYTHON_PREFIX_PATH=/usr/bin
113-
-DPYBIND11_PYTHON_VERSION=3.7
114-
115-
Carma as an embedded CMake project
116-
++++++++++++++++++++++++++++++++++
117-
118-
You can embed `Carma` using CMake command
119-
120-
.. code-block::
121-
122-
add_subdirectory(/path/to/carma/root/directory)
123-
124-
If you do so, you can use :code:`ARMADILLO_ROOT_DIR` and :code:`PYBIND11_ROOT_DIR` to define requirements
125-
(as CMake variables in main project).
126-
127-
Nevertheless, if :code:`armadillo` or/and :code:`pybind11` CMake targets already exist, `carma` will use them
128-
(to avoid conflict with already existing targets in your main project).
129-
130-
Moreover, it could be useful to define
131-
132-
.. code-block::
133-
134-
set(CARMA_DEV_TARGET false)
135-
136-
to disable carma development targets (e.g. ``clang-format``).
137-
138-
Examples
139-
########
140-
141-
On a high level `carma` provides four ways to work with Numpy arrays and Armadillo:
142-
See the :doc:`Function specifications <carma>` section for details about the available functions and the examples directory for runnable examples.
58+
include # carma headers
59+
extern/armadillo-code # fork of armadillo required by carma
60+
tests # carma tests with python module (if enabled using -DBUILD_TESTS=on)
61+
examples # carma python examples with python module (if enabled using -DBUILD_EXAMPLES=on)
14362
144-
Manual conversion
145-
*****************
63+
Design Patterns
64+
###############
14665

147-
The easiest way to use `carma` is manual conversion, it gives you the most control over when to copy or not.
148-
You pass a Numpy array as an argument and/or as the return type and call the respective conversion function.
66+
CARMA was designed with three patterns in mind: borrow, transfer ownership and view.
14967

150-
.. warning:: Carma will avoid copying by default so make sure not to return the memory of the input array without copying or use `update_array`.
68+
Borrow
69+
------
15170

152-
.. code-block:: c++
153-
154-
#include <armadillo>
155-
#include <carma/carma.h>
156-
#include <pybind11/pybind11.h>
157-
#include <pybind11/numpy.h>
158-
159-
py::array_t<double> manual_example(py::array_t<double> & arr) {
160-
// convert to armadillo matrix without copying.
161-
arma::Mat<double> mat = carma::arr_to_mat<double>(arr);
162-
163-
// normally you do something useful here ...
164-
arma::Mat<double> result = arma::Mat<double>(arr.shape(0), arr.shape(1), arma::fill::randu);
165-
166-
// convert to Numpy array and return
167-
return carma::mat_to_arr(result);
168-
}
71+
You can borrow the underlying memory of a Numpy array using the ``arr_to_*(py::array_t<T>, copy=false)``. The Armadillo object should not be returned without a copy out. Use this when you want to modify the memory in-place. If the array is not well behaved, see :doc:`Memory Management -- well behaved <well_behaved>`, the data is copied to a well-behaved memory and swapped in place of the input array. If ``copy=true`` is equivalent to the copy approach below.
16972

170-
Update array
171-
************
172-
173-
.. code-block:: c++
174-
175-
#include <armadillo>
176-
#include <carma/carma.h>
177-
#include <pybind11/pybind11.h>
178-
#include <pybind11/numpy.h>
179-
180-
void update_example(py::array_t<double> & arr) {
181-
// convert to armadillo matrix without copying.
182-
arma::Mat<double> mat = carma::arr_to_mat<double>(arr);
183-
184-
// normally you do something useful here with mat ...
185-
mat += arma::Mat<double>(arr.shape(0), arr.shape(1), arma::fill::randu);
186-
187-
// update Numpy array buffer
188-
carma::update_array(mat, arr);
189-
}
73+
.. note:: the size of the Armadillo object is not allowed change when you borrow.
19074

19175
Transfer ownership
192-
******************
76+
------------------
19377

194-
If you want to transfer ownership to the C++ side you can use:
78+
You can transfer ownership to Armadillo using steal or copy.
19579

196-
.. code-block:: c++
80+
Steal
81+
*****
19782

198-
#include <armadillo>
199-
#include <carma/carma.h>
200-
#include <pybind11/pybind11.h>
201-
#include <pybind11/numpy.h>
202-
203-
arma::Mat<double> steal_array(py::array_t<double> & arr) {
204-
// convert to armadillo matrix
205-
arma::Mat<double> mat = carma::arr_to_mat<double>(arr);
206-
// inform numpy it no longer owns the data
207-
carma::set_not_owndata<double>(arr);
208-
return mat;
209-
}
210-
211-
py::array_t<double> numpy_view(arma::Mat<double> & mat) {
212-
/* Return view on the buffer */
213-
py::array_t<double> arr = carma::mat_to_arr<double>(mat);
214-
// inform numpy it that it doesn't own the data
215-
carma::set_not_owndata<double>(arr)
216-
return arr;
217-
}
218-
219-
py::array_t<double> numpy_view(const arma::Mat<double> & mat) {
220-
/* Return read only view on the buffer */
221-
py::array_t<double> arr = carma::mat_to_arr<double>(mat);
222-
carma::set_not_owndata<double>(arr)
223-
carma::test_set_not_writeable<double>(arr)
224-
return arr;
225-
}
226-
227-
Automatic conversion
228-
********************
229-
230-
For automatic conversion you specify the desired Armadillo type for either or both the return type and the function parameter.
231-
When calling the function from Python, Pybind11 will call `carma`'s type caster when a Numpy array is passed or returned.
232-
233-
.. warning:: Make sure to include `carma` in every compilation unit that makes use of the type caster, not including it results in undefined behaviour.
234-
235-
.. code-block:: c++
236-
237-
#include <armadillo>
238-
#include <carma/carma.h>
239-
#include <pybind11/pybind11.h>
240-
#include <pybind11/numpy.h>
241-
242-
arma::Mat<double> automatic_example(arma::Mat<double> & mat) {
243-
// normally you do something useful here with mat ...
244-
arma::Mat<double> rand = arma::Mat<double>(mat.n_rows, mat.n_cols, arma::fill::randu);
245-
246-
arma::Mat<double> result = mat + rand;
247-
// type caster will take care of casting `result` to a Numpy array.
248-
return result;
249-
}
83+
If you want to take ownership of the underlying memory but don't want to copy the
84+
data you can steal the array. The Armadillo object can be safely returned out without a copy. There are multiple compile time definitions on how the memory is stolen, see :doc:`Configuration <configuration>` for details. If the memory of the
85+
array is not well-behaved a copy of the memory is stolen.
25086

251-
.. warning::
252-
253-
The automatic conversion will **not** copy the Numpy array's memory when converting to Armadillo objects.
254-
When converting back to Numpy arrays the memory will **not** be copied when converting back from matrices but **will be** copied from a vector or cube.
255-
See :doc:`Memory Management <memory_management>` for details.
87+
After stealing the Armadillo behaves as if has allocated the memory itself, hence it will also clean the memory upon destruction.
25688

257-
ArrayStore
258-
**********
89+
.. note:: the size of the Armadillo object is allowed change after stealing.
25990

260-
There are use-cases where you would want to keep the data in C++ and only return when requested.
261-
For example, you write an Ordinary Least Squares (OLS) class and you want to store the residuals, covariance matrix, ... in C++ for when additional tests need to be run on the values without converting back and forth.
91+
Copy
92+
****
26293

263-
ArrayStore is a convenience class that provides conversion methods back and forth.
264-
It is intended to used as an attribute such as below:
94+
If you want to give Armadillo full control of underlying memory but also want to keep Numpy as owner you should copy. The Armadillo object can be safely returned out without a copy. If the memory of the array is not well-behaved a copy of the memory is used instead.
26595

266-
.. code-block:: c++
96+
.. note:: the size of the Armadillo object is allowed change after copying.
26797

268-
#include <armadillo>
269-
#include <carma/carma.h>
270-
#include <pybind11/pybind11.h>
271-
#include <pybind11/numpy.h>
272-
273-
class ExampleClass {
274-
private:
275-
carma::ArrayStore<double> _x;
276-
carma::ArrayStore<double> _y;
277-
278-
public:
279-
ExampleClass(py::array_t<double> & x, py::array_t<double> & y) :
280-
// steal the arrayand store it as an Armadillo matrix
281-
_x{carma::ArrayStore<double>(x, true)},
282-
// copy the arrayand store it as an Armadillo matrix
283-
_y{carma::ArrayStore<double>(y, false)} {}
284-
285-
py::array_t<double> member_func() {
286-
// normallly you would something useful here
287-
_x.mat += _y.mat;
288-
// return mutable view off arma matrix
289-
return _x.get_view(true);
290-
}
291-
};
292-
293-
void bind_exampleclass(py::module &m) {
294-
py::class_<ExampleClass>(m, "ExampleClass")
295-
.def(py::init<py::array_t<double> &, py::array_t<double> &>(), R"pbdoc(
296-
Initialise ExampleClass.
297-
298-
Parameters
299-
----------
300-
arr1: np.ndarray
301-
array to be stored in armadillo matrix
302-
arr2: np.ndarray
303-
array to be stored in armadillo matrix
304-
)pbdoc")
305-
.def("member_func", &ExampleClass::member_func, R"pbdoc(
306-
Compute ....
307-
)pbdoc");
308-
}
309-
310-
.. warning::
311-
312-
The ArrayStore owns the data, the returned numpy arrays are views that
313-
are tied to the lifetime of ArrayStore.
98+
View
99+
----
100+
101+
If you want to have a read-only view on the underlying memory you can use ``arr_to_*_view``. If the underlying memory is not well-behaved it will be copied.

0 commit comments

Comments
 (0)