You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: docs/components.md
+4-4
Original file line number
Diff line number
Diff line change
@@ -2,10 +2,10 @@
2
2
3
3
This toolkit consists of three components.
4
4
5
-
The first component ([](/src/rpft/parsers/common)) is RapidPro-agnostic and takes care of reader spreadsheets and converting them into internal data models and other output formats, see [](sheets.md)
5
+
The first component ([/src/rpft/parsers/common]) is RapidPro-agnostic and takes care of reader spreadsheets and converting them into internal data models and other output formats, see [sheets.md]
6
6
7
-
The second component ([](/src/rpft/parsers/creation)) defines data models for a spreadsheet format for RapidPro flows, and process spreadsheets into RapidPro flows (and back) using the first component.
7
+
The second component ([/src/rpft/parsers/creation]) defines data models for a spreadsheet format for RapidPro flows, and process spreadsheets into RapidPro flows (and back) using the first component.
8
8
9
-
The third component ([](/src/rpft/rapidpro)) defines internal representations of RapidPro flows and to read and write to a JSON format that can be import to/exported from RapidPro. It is partially entangled with the second component, as it needs to be aware of the data models of the second component to convert RapidPro flows into the spreadsheet format.
9
+
The third component ([/src/rpft/rapidpro]) defines internal representations of RapidPro flows and to read and write to a JSON format that can be import to/exported from RapidPro. It is partially entangled with the second component, as it needs to be aware of the data models of the second component to convert RapidPro flows into the spreadsheet format.
10
10
11
-
The latter two components are (poorly) documented here: [](rapidpro.md)
11
+
The latter two components are (poorly) documented here: [rapidpro.md]
Copy file name to clipboardexpand all lines: docs/models.md
+6-8
Original file line number
Diff line number
Diff line change
@@ -1,13 +1,12 @@
1
1
# Models
2
2
3
-
`RowModel`s are subclasses of [`pydantic.BaseModel`]
4
-
(https://docs.pydantic.dev/latest/concepts/models/#basic-model-usage), and may
3
+
`RowModel`s are subclasses of [`pydantic.BaseModel`](https://docs.pydantic.dev/latest/concepts/models/#basic-model-usage), and may
5
4
contain basic types, lists and other models as attributes, nested arbirarily
6
5
deep. Every `Sheet` can only be parsed in the context of a given `RowModel`
7
6
(which can, however, be automatically inferred from the sheet headers, if desired).
8
7
9
8
Technically, there is no `RowModel` class, but instead it is called `ParserModel`
10
-
and is defined in [](/src/rpft/parsers/common/rowparser.py). `ParserModel` attributes have to be
9
+
and is defined in [/src/rpft/parsers/common/rowparser.py]. `ParserModel` attributes have to be
11
10
basic types, lists or `ParserModel`s.
12
11
The only addition to `pydantic.BaseModel` are the (optional) methods `header_name_to_field_name`, `field_name_to_header_name` and (for full row models) `header_name_to_field_name_with_context` that allow remapping
13
12
column header names to different model attributes.
@@ -20,8 +19,7 @@ class SubModel(ParserModel):
20
19
number: int = 0
21
20
22
21
class MyModel(ParserModel):
23
-
numbers: List[int] = []
24
-
sub: SubModel = SubModel()
22
+
numbers: List[int] = [ sub: SubModel = SubModel()
25
23
```
26
24
27
25
The headers of a sheet and its content that can be parsed into `MyModel` could for example be:
@@ -46,9 +44,9 @@ How sheets and their column headers correspond to `RowModel`s is specified in
Copy file name to clipboardexpand all lines: docs/rapidpro.md
+2-2
Original file line number
Diff line number
Diff line change
@@ -18,13 +18,13 @@ There are two sheet types of particular interest
18
18
-[`DataSheet`](/src/rpft/parsers/creation/contentindexparser.py): Similar to a [`RowDataSheet`](sheets.md), however, assumed that the `RowModel` has an `ID` field, and rather than storing a list of rows, it stores an ordered dict of rows, indexed by their ID.
19
19
-[`TemplateSheet`](/src/rpft/parsers/creation/contentindexparser.py): Wrapper around `tablib.Dataset`, with template arguments.
20
20
21
-
Note: It may be worthwhile unifying the data structures used here, to be consistent with `Sheet` and `RowDataSheet` documented in [](sheets.md). Also see the discussion there why `DataSheet`s can be exported to nested JSON, while `TemplateSheet`s can only be exported to flat JSON.
21
+
Note: It may be worthwhile unifying the data structures used here, to be consistent with `Sheet` and `RowDataSheet` documented in [sheets.md]. Also see the discussion there why `DataSheet`s can be exported to nested JSON, while `TemplateSheet`s can only be exported to flat JSON.
22
22
23
23
`DataSheet`s are often used to instantiate `TemplateSheet`s, and the ContentIndexParser has mechanisms for this, see [New features documentation]. Furthermore, `DataSheet`s can also be concatenated, filtered and sorted via the `operation` column, see [here](https://docs.google.com/document/d/1Onx2RhNoWKW9BQvFrgTc5R5hcwDy1OMsLKnNB7YxQH0/edit#heading=h.c93jouk7sqq)
24
24
25
25
26
26
27
-
Relevant code: `parse_all_flows` in [](/src/rpft/parsers/creation/contentindexparser.py).
27
+
Relevant code: `parse_all_flows` in [/src/rpft/parsers/creation/contentindexparser.py].
Copy file name to clipboardexpand all lines: docs/sheets.md
+8-6
Original file line number
Diff line number
Diff line change
@@ -47,7 +47,7 @@ details of the three conversion steps below.
47
47
48
48
## Conversion between spreadsheet files and `Sheet`s
49
49
50
-
SheetReaders and `Sheet`s are defined in [](/src/rpft/parsers/sheets.py)
50
+
SheetReaders and `Sheet`s are defined in [/src/rpft/parsers/sheets.py]
51
51
52
52
The `Sheet` class wraps [`tablib.Dataset`]
53
53
(https://tablib.readthedocs.io/en/stable/api.html#dataset-object) (which is
@@ -95,7 +95,7 @@ instances. `RowModel`s are subclasses of [`pydantic.BaseModel`]
95
95
(https://docs.pydantic.dev/latest/concepts/models/#basic-model-usage), and may
96
96
contain basic types, lists and other models as attributes, nested arbirarily
97
97
deep. How sheets and their column headers correspond to `RowModel`s is
98
-
documented in more detail in [](models.md).
98
+
documented in more detail in [models.md].
99
99
100
100
101
101
### Forward direction
@@ -127,7 +127,7 @@ row into a `RowModel` instance containing the provided data.
127
127
corresponding entry of the spreadsheet in this row, and is provided
128
128
by the `SheetParser`. Column headers determine which field of the
129
129
model the column contains data for, and different ways to address fields
130
-
in the data models are supported, see [](models.md).
130
+
in the data models are supported, see [models.md].
131
131
132
132
The `RowParser` interprets the column headers and if the column contains
133
133
a non-basic type (e.g. a list or a submodel), it invokes the `CellParser`
@@ -148,7 +148,7 @@ Examples:
148
148
-`a\;b` --> 'a;b'
149
149
-`a,b|1,2` --> [['a','b'],['1','2']]
150
150
151
-
More examples can be found in [](/tests/test_cellparser.py).
151
+
More examples can be found in [/tests/test_cellparser.py].
152
152
153
153
154
154
#### Templating
@@ -160,7 +160,7 @@ With a given templating context mapping variable names to values
160
160
(e.g. {"user_name": "Chris"), such a string can be evaluated,
161
161
e.g. to `Hello Chris!`.
162
162
163
-
More examples can be found in [](/tests/test_cellparser.py).
163
+
More examples can be found in [/tests/test_cellparser.py].
164
164
165
165
166
166
##### Instantiating templated sheets
@@ -253,7 +253,7 @@ or a reference to an already defined model).
253
253
254
254
255
255
The CLI command `save_data_sheets` implements exporting all data sheets referenced in
256
-
a content index as (a single) nested JSON. This is implemented in `save_data_sheets` in [](/src/rpft/converters.py), using the [`ContentIndexParser`](rapidpro.md). However, it its
256
+
a content index as (a single) nested JSON. This is implemented in `save_data_sheets` in [/src/rpft/converters.py], using the [`ContentIndexParser`](rapidpro.md). However, it its
257
257
own `DataSheet` class via its `to_dict` method. It would be good to unify `DataSheet`
258
258
and `RowDataSheet`, and provide this as standalone functionality, once it's decided which meta-data describing the underlying model needs to be stored.
259
259
@@ -262,6 +262,8 @@ Below is some (untested) code outlining roughly how this could look like:
262
262
263
263
```python
264
264
from converters import create_sheet_reader
265
+
from rpft.parsers.common.model_inference import model_from_headers
0 commit comments