Skip to content

Commit 557d1d6

Browse files
vtushar06pre-commit-ci[bot]niksirbi
authored
fix: update default individual name from 'individual_0' to 'id_0' acr… (#618)
* fix: update default individual name from 'individual_0' to 'id_0' across codebase and tests * docs: update input_output guide and tests/tests_units to reflect id_0 default naming * Fix: Add fallback for missing individual in plot function; minor doc improvements * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * removed printing error message casuing error * removed error message provided * fix: change default individual name to 'id_0' for missing metadata * chore: clean trailing whitespace from Makefile * reset -W for warnings * finishing touches --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: niksirbi <[email protected]>
1 parent bfbe1fe commit 557d1d6

File tree

8 files changed

+93
-53
lines changed

8 files changed

+93
-53
lines changed

docs/source/user_guide/input_output.md

Lines changed: 66 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
(target-io)=
2+
23
# Input/Output
34

45
## Overview
@@ -17,32 +18,33 @@ It may be useful to think of `movement` supporting two types of data loading/sav
1718
You are also welcome to try `movement` by loading some [sample data](target-sample-data) included with the package.
1819

1920
(target-supported-formats)=
21+
2022
## Supported third-party formats
2123

2224
`movement` supports the analysis of trajectories of keypoints (_pose tracks_) and of bounding box centroids (_bounding box tracks_),
2325
which are represented as [movement datasets](target-poses-and-bboxes-dataset)
2426
and can be loaded from and saved to various third-party formats.
2527

26-
| Source Software | Abbreviation | Source Format | Dataset Type | Supported Operations |
27-
|-----------------|--------------|-------------|--------------|-------------------|
28-
| [DeepLabCut](dlc:) | DLC | DLC-style .h5 or .csv file, or corresponding pandas DataFrame | Pose | Load & Save |
29-
| [SLEAP](sleap:) | SLEAP | [analysis](sleap:tutorials/analysis) .h5 or .slp file | Pose | Load & Save |
30-
| [LightningPose](lp:) | LP | DLC-style .csv file, or corresponding pandas DataFrame | Pose | Load & Save |
31-
| [Anipose](anipose:) | | triangulation .csv file, or corresponding pandas DataFrame | Pose | Load |
32-
| [VGG Image Annotator](via:) | VIA | .csv file for [tracks annotation](via:docs/face_track_annotation.html) | Bounding box | Load |
33-
| [Neurodata Without Borders](https://nwb-overview.readthedocs.io/en/latest/) | NWB | .nwb file or NWBFile object with the [ndx-pose extension](https://github.com/rly/ndx-pose) | Pose | Load & Save |
34-
| Any | | Numpy arrays | Pose or Bounding box | Load & Save* |
28+
| Source Software | Abbreviation | Source Format | Dataset Type | Supported Operations |
29+
| --------------------------------------------------------------------------- | ------------ | ------------------------------------------------------------------------------------------ | -------------------- | -------------------- |
30+
| [DeepLabCut](dlc:) | DLC | DLC-style .h5 or .csv file, or corresponding pandas DataFrame | Pose | Load & Save |
31+
| [SLEAP](sleap:) | SLEAP | [analysis](sleap:tutorials/analysis) .h5 or .slp file | Pose | Load & Save |
32+
| [LightningPose](lp:) | LP | DLC-style .csv file, or corresponding pandas DataFrame | Pose | Load & Save |
33+
| [Anipose](anipose:) | | triangulation .csv file, or corresponding pandas DataFrame | Pose | Load |
34+
| [VGG Image Annotator](via:) | VIA | .csv file for [tracks annotation](via:docs/face_track_annotation.html) | Bounding box | Load |
35+
| [Neurodata Without Borders](https://nwb-overview.readthedocs.io/en/latest/) | NWB | .nwb file or NWBFile object with the [ndx-pose extension](https://github.com/rly/ndx-pose) | Pose | Load & Save |
36+
| Any | | Numpy arrays | Pose or Bounding box | Load & Save\* |
3537

36-
*Exporting any `movement` DataArray to a NumPy array is as simple as calling xarray's built-in {meth}`xarray.DataArray.to_numpy()` method, so no specialised "Export/Save As" function is needed, see [xarray's documentation](xarray:user-guide/duckarrays.html) for more details.
38+
\*Exporting any `movement` DataArray to a NumPy array is as simple as calling xarray's built-in {meth}`xarray.DataArray.to_numpy()` method, so no specialised "Export/Save As" function is needed, see [xarray's documentation](xarray:user-guide/duckarrays.html) for more details.
3739

3840
:::{note}
3941
Currently, `movement` only works with tracked data: either keypoints or bounding boxes whose identities are known from one frame to the next, across consecutive frames. For pose estimation, this means it only supports the predictions output by the supported software packages listed above. Loading manually labelled data—often defined over a non-continuous set of frames—is not currently supported.
4042
:::
4143

4244
Below, we explain how to load pose and bounding box tracks from these supported formats, as well as how to save pose tracks back to some of them.
4345

44-
4546
(target-loading-pose-tracks)=
47+
4648
### Loading pose tracks
4749

4850
The pose tracks loading functionalities are provided by the
@@ -58,6 +60,7 @@ To read a pose tracks file into a [movement poses dataset](target-poses-and-bbox
5860

5961
:::{tab-item} DeepLabCut
6062
To load DeepLabCut files in .h5 format:
63+
6164
```python
6265
ds = load_poses.from_dlc_file("/path/to/file.h5", fps=30)
6366

@@ -66,13 +69,16 @@ ds = load_poses.from_file(
6669
"/path/to/file.h5", source_software="DeepLabCut", fps=30
6770
)
6871
```
72+
6973
To load DeepLabCut files in .csv format:
74+
7075
```python
7176
ds = load_poses.from_dlc_file("/path/to/file.csv", fps=30)
7277
```
7378

7479
You can also directly load any pandas DataFrame `df` that's
7580
formatted in the DeepLabCut style:
81+
7682
```python
7783
ds = load_poses.from_dlc_style_df(df, fps=30)
7884
```
@@ -81,6 +87,7 @@ ds = load_poses.from_dlc_style_df(df, fps=30)
8187

8288
:::{tab-item} SLEAP
8389
To load [SLEAP analysis files](sleap:tutorials/analysis) in .h5 format (recommended):
90+
8491
```python
8592
ds = load_poses.from_sleap_file("/path/to/file.analysis.h5", fps=30)
8693

@@ -89,14 +96,18 @@ ds = load_poses.from_file(
8996
"/path/to/file.analysis.h5", source_software="SLEAP", fps=30
9097
)
9198
```
99+
92100
To load SLEAP files in .slp format (experimental, see notes in {func}`movement.io.load_poses.from_sleap_file`):
101+
93102
```python
94103
ds = load_poses.from_sleap_file("/path/to/file.predictions.slp", fps=30)
95104
```
105+
96106
:::
97107

98108
:::{tab-item} LightningPose
99109
To load LightningPose files in .csv format:
110+
100111
```python
101112
ds = load_poses.from_lp_file("/path/to/file.analysis.csv", fps=30)
102113

@@ -108,38 +119,44 @@ ds = load_poses.from_file(
108119

109120
Because LightningPose follows the DeepLabCut dataframe format, you can also
110121
directly load an appropriately formatted pandas DataFrame `df`:
122+
111123
```python
112124
ds = load_poses.from_dlc_style_df(df, fps=30, source_software="LightningPose")
113125
```
126+
114127
:::
115128

116129
:::{tab-item} Anipose
117130
To load Anipose files in .csv format:
131+
118132
```python
119133
ds = load_poses.from_anipose_file(
120-
"/path/to/file.analysis.csv", fps=30, individual_name="individual_0"
121-
) # Optionally specify the individual name; defaults to "individual_0"
134+
"/path/to/file.analysis.csv", fps=30, individual_name="id_0"
135+
) # Optionally specify the individual name; defaults to "id_0"
122136

123137
# or equivalently
124138
ds = load_poses.from_file(
125139
"/path/to/file.analysis.csv",
126140
source_software="Anipose",
127141
fps=30,
128-
individual_name="individual_0",
142+
individual_name="id_0",
129143
)
130144
```
131145

132146
You can also directly load any pandas DataFrame `df` that's
133147
formatted in the Anipose triangulation style:
148+
134149
```python
135150
ds = load_poses.from_anipose_style_df(
136-
df, fps=30, individual_name="individual_0"
151+
df, fps=30, individual_name="id_0"
137152
)
138153
```
154+
139155
:::
140156

141157
:::{tab-item} NWB
142158
To load NWB files in .nwb format:
159+
143160
```python
144161
ds = load_poses.from_nwb_file(
145162
"path/to/file.nwb",
@@ -157,19 +174,23 @@ ds = load_poses.from_file(
157174
pose_estimation_key="PoseEstimation",
158175
)
159176
```
177+
160178
The above functions also accept an {class}`NWBFile<pynwb.file.NWBFile>` object as input:
179+
161180
```python
162181
with pynwb.NWBHDF5IO("path/to/file.nwb", mode="r") as io:
163182
nwb_file = io.read()
164183
ds = load_poses.from_nwb_file(
165184
nwb_file, pose_estimation_key="PoseEstimation"
166185
)
167186
```
187+
168188
:::
169189

170190
:::{tab-item} From NumPy
171-
In the example below, we create random position data for two individuals, ``Alice`` and ``Bob``,
172-
with three keypoints each: ``snout``, ``centre``, and ``tail_base``. These keypoints are tracked in 2D space for 100 frames, at 30 fps. The confidence scores are set to 1 for all points.
191+
In the example below, we create random position data for two individuals, `Alice` and `Bob`,
192+
with three keypoints each: `snout`, `centre`, and `tail_base`. These keypoints are tracked in 2D space for 100 frames, at 30 fps. The confidence scores are set to 1 for all points.
193+
173194
```python
174195
import numpy as np
175196

@@ -182,6 +203,7 @@ ds = load_poses.from_numpy(
182203
fps=30,
183204
)
184205
```
206+
185207
:::
186208

187209
::::
@@ -192,9 +214,10 @@ the pose estimation software.
192214

193215
For more information on the poses data structure, see the [movement datasets](target-poses-and-bboxes-dataset) page.
194216

195-
196217
(target-loading-bbox-tracks)=
218+
197219
### Loading bounding box tracks
220+
198221
To load bounding box tracks into a [movement bounding boxes dataset](target-poses-and-bboxes-dataset), we need the functions from the
199222
{mod}`movement.io.load_bboxes` module, which can be imported as follows:
200223

@@ -207,6 +230,7 @@ We currently support loading bounding box tracks in the VGG Image Annotator (VIA
207230
::::{tab-set}
208231
:::{tab-item} VGG Image Annotator
209232
To load a VIA tracks .csv file:
233+
210234
```python
211235
ds = load_bboxes.from_via_tracks_file("path/to/file.csv", fps=30)
212236

@@ -217,12 +241,14 @@ ds = load_bboxes.from_file(
217241
fps=30,
218242
)
219243
```
220-
Note that the x,y coordinates in the input VIA tracks .csv file represent the the top-left corner of each bounding box. Instead the corresponding ``movement`` dataset `ds` will hold in its `position` array the centroid of each bounding box.
244+
245+
Note that the x,y coordinates in the input VIA tracks .csv file represent the the top-left corner of each bounding box. Instead the corresponding `movement` dataset `ds` will hold in its `position` array the centroid of each bounding box.
221246
:::
222247

223248
:::{tab-item} From NumPy
224-
In the example below, we create random position data for two bounding boxes, ``id_0`` and ``id_1``,
249+
In the example below, we create random position data for two bounding boxes, `id_0` and `id_1`,
225250
both with the same width (40 pixels) and height (30 pixels). These are tracked in 2D space for 100 frames, which will be numbered in the resulting dataset from 0 to 99. The confidence score for all bounding boxes is set to 0.5.
251+
226252
```python
227253
import numpy as np
228254

@@ -234,6 +260,7 @@ ds = load_bboxes.from_numpy(
234260
individual_names=["id_0", "id_1"]
235261
)
236262
```
263+
237264
:::
238265

239266
::::
@@ -242,8 +269,8 @@ The resulting data structure `ds` will include the centroid trajectories for eac
242269

243270
For more information on the bounding boxes data structure, see the [movement datasets](target-poses-and-bboxes-dataset) page.
244271

245-
246272
(target-saving-pose-tracks)=
273+
247274
### Saving pose tracks
248275

249276
To export [movement poses datasets](target-poses-and-bboxes-dataset) to any of the supported third-party formats,
@@ -259,20 +286,24 @@ Depending on the desired format, use one of the following functions:
259286

260287
::::{tab-item} DeepLabCut
261288
To save as a DeepLabCut file, in .h5 or .csv format:
289+
262290
```python
263291
save_poses.to_dlc_file(ds, "/path/to/file.h5") # preferred format
264292
save_poses.to_dlc_file(ds, "/path/to/file.csv")
265293
```
294+
266295
The {func}`movement.io.save_poses.to_dlc_file` function also accepts
267296
a `split_individuals` boolean argument. If set to `True`, the function will
268297
save the data as separate single-animal DeepLabCut-style files.
269298
::::
270299

271300
::::{tab-item} SLEAP
272301
To save as a SLEAP analysis file in .h5 format:
302+
273303
```python
274304
save_poses.to_sleap_analysis_file(ds, "/path/to/file.h5")
275305
```
306+
276307
When saving to SLEAP-style files, only `track_names`, `node_names`, `tracks`, `track_occupancy`,
277308
and `point_scores` are saved. `labels_path` will only be saved if the source
278309
file of the dataset is a SLEAP .slp file. Otherwise, it will be an empty string.
@@ -285,44 +316,51 @@ each attribute and data variable represents, see the
285316

286317
::::{tab-item} LightningPose
287318
To save as a LightningPose file in .csv format:
319+
288320
```python
289321
save_poses.to_lp_file(ds, "/path/to/file.csv")
290322
```
291323

292324
Because LightningPose follows the single-animal
293325
DeepLabCut .csv format, the above command is equivalent to:
326+
294327
```python
295328
save_poses.to_dlc_file(ds, "/path/to/file.csv", split_individuals=True)
296329
```
330+
297331
::::
298332

299333
::::{tab-item} NWB
300334
To convert a `movement` poses dataset to {class}`NWBFile<pynwb.file.NWBFile>` objects:
335+
301336
```python
302337
nwb_files = save_poses.to_nwb_file(ds)
303338
```
339+
304340
To allow adding additional data to NWB files before saving, {func}`to_nwb_file<movement.io.save_poses.to_nwb_file>` does not write to disk directly.
305341
Instead, it returns a list of {class}`NWBFile<pynwb.file.NWBFile>` objects---one per individual in the dataset---since NWB files are designed to represent data from a single individual.
306342

307343
The {func}`to_nwb_file<movement.io.save_poses.to_nwb_file>` function also accepts
308-
a {class}`NWBFileSaveConfig<movement.io.nwb.NWBFileSaveConfig>` object as its ``config`` argument
344+
a {class}`NWBFileSaveConfig<movement.io.nwb.NWBFileSaveConfig>` object as its `config` argument
309345
for customising metadata such as session or subject information in the resulting NWBFiles
310346
(see {func}`the API reference<movement.io.save_poses.to_nwb_file>` for examples).
311347

312348
These {class}`NWBFile<pynwb.file.NWBFile>` objects can then be saved to disk as .nwb files using {class}`pynwb.NWBHDF5IO`:
349+
313350
```python
314351
from pynwb import NWBHDF5IO
315352

316353
for file in nwb_files:
317354
with NWBHDF5IO(f"{file.identifier}.nwb", "w") as io:
318355
io.write(file)
319356
```
357+
320358
::::
321359

322360
:::::
323361

324-
325362
(target-saving-bboxes-tracks)=
363+
326364
### Saving bounding box tracks
327365

328366
We currently do not provide explicit methods to export a movement bounding boxes dataset in a specific format. However, you can save the bounding box tracks to a .csv file using the standard Python library `csv`.
@@ -349,9 +387,11 @@ with open(filepath, mode="w", newline="") as file:
349387
writer.writerow([frame, individual, x, y, width, height, confidence])
350388

351389
```
390+
352391
Alternatively, we can convert the `movement` bounding boxes dataset to a pandas DataFrame with the {meth}`xarray.DataArray.to_dataframe` method, wrangle the dataframe as required, and then apply the {meth}`pandas.DataFrame.to_csv` method to save the data as a .csv file.
353392

354393
(target-netcdf)=
394+
355395
## Native saving and loading with netCDF
356396

357397
Because `movement` datasets are {class}`xarray.Dataset` objects, we can rely on
@@ -367,7 +407,7 @@ To save any xarray dataset `ds` to a netCDF file:
367407

368408
```python
369409
ds.to_netcdf("/path/to/my_data.nc")
370-
````
410+
```
371411

372412
To load the dataset back:
373413

@@ -416,8 +456,8 @@ ds["speed"] = compute_speed(ds["position_smooth"])
416456
ds.to_netcdf("my_data_processed.nc")
417457
```
418458

419-
420459
(target-sample-data)=
460+
421461
## Sample data
422462

423463
`movement` includes some sample data files that you can use to
@@ -444,6 +484,7 @@ To load one of the sample files as a
444484
filename = "SLEAP_three-mice_Aeon_proofread.analysis.h5"
445485
ds = sample_data.fetch_dataset(filename)
446486
```
487+
447488
Some sample datasets also have an associated video file
448489
(the video for which the data was predicted). You can request
449490
to download the sample video by setting `with_video=True`:

examples/smooth.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,11 +49,14 @@
4949
def plot_raw_and_smooth_timeseries_and_psd(
5050
ds_raw,
5151
ds_smooth,
52-
individual="individual_0",
52+
individual=None,
5353
keypoint="stinger",
5454
space="x",
5555
time_range=None,
5656
):
57+
# If no individual is specified, use the first one
58+
if individual is None:
59+
individual = ds_raw.individuals[0]
5760
# If no time range is specified, plot the entire time series
5861
if time_range is None:
5962
time_range = slice(0, ds_raw.time[-1])

0 commit comments

Comments
 (0)