Skip to content

Commit 3671b44

Browse files
authored
Merge pull request #47 from nipreps/enh/revise-docker-docs
ENH: Centralize fMRIPrep's and MRIQC's guidelines for Docker & DataLad
2 parents 9af55c7 + b3b52d8 commit 3671b44

File tree

2 files changed

+304
-81
lines changed

2 files changed

+304
-81
lines changed

docs/apps/datalad.md

Lines changed: 35 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,4 @@
1-
Apps may be able to identify if the input dataset is handled with
2-
*DataLad* or *Git-Annex*, and pull down linked data that has not
3-
been fetched yet.
4-
One example of one such application is *MRIQC*, and all the examples
5-
on this documentation page will refer to it.
6-
7-
!!! important "Summary"
1+
!!! abstract "Summary"
82

93
Executing *BIDS-Apps* leveraging *DataLad*-controlled datasets
104
within containers can be tricky.
@@ -18,6 +12,12 @@ on this documentation page will refer to it.
1812

1913
## *DataLad* and *Docker*
2014

15+
Apps may be able to identify if the input dataset is handled with
16+
[*DataLad*](https://www.datalad.org/) or [*git-annex*](https://git-annex.branchable.com), and pull down linked data that has not
17+
been fetched yet.
18+
One example of one such application is *MRIQC*, and all the examples
19+
on this documentation page will refer to it.
20+
2121
When executing *MRIQC* within *Docker* on a *DataLad* dataset
2222
(for instance, installed from [*OpenNeuro*](https://openneuro.org)),
2323
we will need to ensure the following settings are observed:
@@ -27,9 +27,29 @@ we will need to ensure the following settings are observed:
2727
* the uid who is *executing MRIQC* within the container must
2828
have sufficient permissions to write in the tree.
2929

30-
### Setting execution uid
30+
!!! tip "Check *ReproNim* if the suggestions here did not work"
31+
32+
The actions suggested here are expected to work in most circumstances,
33+
but your system may have specific circumstances that require additional
34+
or alternative approaches.
35+
For instance, [the *ReproNim* project](https://www.repronim.org/) maintains
36+
[ReproNim/containers](https://github.com/ReproNim/containers), a
37+
*DataLad* dataset with ready-to-use Singularity images for released *BIDS Apps*, *NeuroDesktop* applications,
38+
and other containers.
39+
Its [`README.md`](https://github.com/ReproNim/containers?tab=readme-ov-file#runnable-script) guides through an approach via that dataset with *built-in* execution helper taking care about bind-mounts,
40+
proxying critical *Git* configuration and potentially executing *Singularity* images via *Docker* (e.g., under OSX).
41+
42+
In the particular case of *MRIQC*, please consider updating (if necessary)
43+
and fetching the required data before execution and then
44+
add the `--no-datalad-get` argument to workaround issues with
45+
*DataLad*.
46+
47+
### Setting a regular user's execution uid
3148

32-
If the uid is not correct, we will likely encounter the following error:
49+
If the execution uid does not match the uid of the user who installed
50+
the *DataLad* dataset, we will likely encounter the following error
51+
with relatively recent
52+
[*Git* versions (+2.35.2)](https://github.blog/open-source/git/git-security-vulnerability-announced/#):
3353

3454
```
3555
datalad.runner.exception.CommandError: CommandError: 'git -c diff.ignoreSubmodules=none -c core.quotepath=false -c annex.merge-annex-branches=false annex find --not --in . --json --json-error-messages -c annex.dotfiles=true -- sub-0001/func/sub-0001_task-restingstate_acq-mb3_bold.nii.gz sub-0002/func/sub-0002_task-emomatching_acq-seq_bold.nii.gz sub-0002/func/sub-0002_task-restingstate_acq-mb3_bold.nii.gz sub-0001/func/sub-0001_task-emomatching_acq-seq_bold.nii.gz sub-0001/func/sub-0001_task-faces_acq-mb3_bold.nii.gz sub-0001/dwi/sub-0001_dwi.nii.gz sub-0002/func/sub-0002_task-workingmemory_acq-seq_bold.nii.gz sub-0001/anat/sub-0001_T1w.nii.gz sub-0002/anat/sub-0002_T1w.nii.gz sub-0001/func/sub-0001_task-gstroop_acq-seq_bold.nii.gz sub-0002/func/sub-0002_task-faces_acq-mb3_bold.nii.gz sub-0002/func/sub-0002_task-anticipation_acq-seq_bold.nii.gz sub-0002/dwi/sub-0002_dwi.nii.gz sub-0001/func/sub-0001_task-anticipation_acq-seq_bold.nii.gz sub-0001/func/sub-0001_task-workingmemory_acq-seq_bold.nii.gz sub-0002/func/sub-0002_task-gstroop_acq-seq_bold.nii.gz' failed with exitcode 1 under /data [info keys: stdout_json] [err: 'git-annex: Git refuses to operate in this repository, probably because it is owned by someone else.
@@ -40,20 +60,16 @@ git config --global --add safe.directory /data
4060
git-annex: automatic initialization failed due to above problems']
4161
```
4262

43-
Confusingly, following the suggestion from *DataLad* directly on the host
44-
(`git config --global --add safe.directory /data`) will not work in this
63+
Confusingly, following the suggestion from *DataLad*
64+
(just propagated from *Git*) of executing
65+
`git config --global --add safe.directory /data` will not work in this
4566
case, because this line must be executed within the container.
67+
However, containers are *transient* and the setting this configuration
68+
on *Git* will not be propagated between executions unless advanced
69+
actions are taken (such as mounting a *HOME* folder with the necessary settings).
4670

4771
Instead, we can override the default user executing within the container
4872
(which is `root`, or uid = 0).
49-
This can be achieved with
50-
[*Docker*'s `-u`/`--user` option](https://docs.docker.com/engine/containers/run/#user):
51-
52-
```
53-
--user=[ user | user:group | uid | uid:gid | user:gid | uid:group ]
54-
```
55-
56-
We can combine this option with *Bash*'s `id` command to ensure the current user's uid and group id (gid) are being set.
5773
Let's update the last example in the previous
5874
[*Docker* execution section](docker.md#running-a-niprep-directly-interacting-with-the-docker-engine):
5975

0 commit comments

Comments
 (0)