Skip to content

Commit

Permalink
Merge pull request datalad#440 from datalad/knitting-threads
Browse files Browse the repository at this point in the history
Knitting threads: bringing changes from all over into upstream
  • Loading branch information
jsheunis authored Apr 16, 2024
2 parents ab87477 + 3b9d423 commit d8cc3ec
Show file tree
Hide file tree
Showing 13 changed files with 164 additions and 97 deletions.
8 changes: 4 additions & 4 deletions datalad_catalog/catalog/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,16 +20,16 @@ The `artwork` and `assets` directories contain images and web assets (such as Ja

## Serving the content

Since this site is self-contained and static, no further build processes, server-side implementations, or access to content delivery networks (CDNs) are necessary in order to serve the content. All that is needed is a simple HTTP server.
Since this site is self-contained and static, no further build processes or access to content delivery networks (CDNs) are necessary in order to serve the content. All that is needed is a simple HTTP server with one specific addition - a custom redirect. This is required because the application makes use of [Vue Router's history mode](https://v3.router.vuejs.org/guide/essentials/history-mode.html#html5-history-mode), which requires a server-side redirect configuration to deal with the fact that a VueJS application is actually a single-page app.

This can be achieved locally, for example using Python:
For serving the content locally, this is already taken care of in `datalad-catalog`, and you can simply run the following:

```bash
cd path/to/catalog/directory
python3 -m http.server
datalad catalog-serve -c .
```

The content can also be hosted and served online. A straightforward and free way to achieve this is via GitHub and [GitHub Pages](https://pages.github.com/). After publishing this content as a GitHub repository, you can activate GitHub Pages in the repository's settings. See detailed instructions [here](https://docs.github.com/en/pages/getting-started-with-github-pages/creating-a-github-pages-site).
The content can also be hosted and served online. A straightforward way to achieve this, and one which has a free tier, is via [Netlify](https://www.netlify.com/) (the common alternative, [GitHub Pages](https://pages.github.com/), does not currently support server-side redirects). After publishing this content, e.g. as a GitHub repository, you can link that repository to a site on Netlify. [See here](https://docs.netlify.com/routing/redirects/) how to set up redirects with Netlify. Of course, the site can also be served from your preferred server setup, as long as page redirects can be supported.

## Maintaining content

Expand Down
82 changes: 46 additions & 36 deletions datalad_catalog/catalog/assets/app_component_dataset.js
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@ const datasetView = () =>
files_ready: false,
tags_ready: false,
description_ready: false,
description_display: "",
citation_busy: false,
citation_text: "",
invalid_doi: false,
Expand All @@ -43,14 +42,13 @@ const datasetView = () =>
watch: {
subdatasets_ready: function (newVal, oldVal) {
if (newVal) {
console.log("subdatasets fetched!");
console.debug("Watched property: subdatasets_ready = true")
console.debug("Subdatasets have been fetched:");
this.subdatasets = this.selectedDataset.subdatasets;
console.log("from watcher");
console.log(this.subdatasets);
console.debug(this.subdatasets);
tags = this.tag_options;
if (this.subdatasets) {
this.subdatasets.forEach((subds, index) => {
console.log(index + "; " + subds);
if (subds.available == "true" && subds.keywords) {
tags = tags.concat(
subds.keywords.filter((item) => tags.indexOf(item) < 0)
Expand All @@ -67,8 +65,10 @@ const datasetView = () =>
dataset_ready: function (newVal, oldVal) {
// TODO: some of these methods/steps should be moved to the generatpr tool. e.g. shortname
if (newVal) {
console.debug("Watched property: dataset_ready = true")
console.debug("Active dataset:");
dataset = this.selectedDataset;
console.log(this.selectedDataset);
console.debug(this.selectedDataset);
disp_dataset = {};
// Set name to unknown if not available
if (!dataset.hasOwnProperty("name") || !dataset["name"]) {
Expand Down Expand Up @@ -156,6 +156,13 @@ const datasetView = () =>
}
} else {
disp_dataset.url = dataset.url;
if (disp_dataset.url && dataset.url.toLowerCase().indexOf("gin.g-node") >= 0) {
disp_dataset.is_gin = true;
disp_dataset.url = disp_dataset.url.replace('ssh://', '');
disp_dataset.url = disp_dataset.url.replace('[email protected]:', 'https://gin.g-node.org');
disp_dataset.url = disp_dataset.url.replace('[email protected]', 'https://gin.g-node.org');
disp_dataset.url = disp_dataset.url.replace('.git', '');
}
}
// Description
if (
Expand Down Expand Up @@ -200,6 +207,19 @@ const datasetView = () =>
else {
disp_dataset.show_export = false
}
// Determine show/hide confirg for "Request access" button
if (dataset.config?.hasOwnProperty("dataset_options") && dataset.config.dataset_options.hasOwnProperty("include_access_request")) {
disp_dataset.show_access_request = dataset.config.dataset_options.include_access_request
}
else {
// default should be to display the access request button, if access request contact/url are included
disp_dataset.show_access_request = true
}
// Show / hide binder button: if disp_dataset.url exists OR if dataset has a notebook specified in metadata
disp_dataset.show_binder_button = false
if ( disp_dataset.url || dataset.hasOwnProperty("notebooks") && current_dataset.notebooks.length > 0 ) {
disp_dataset.show_binder_button = true
}
// Write main derived variable and set to ready
this.displayData = disp_dataset;
this.display_ready = true;
Expand Down Expand Up @@ -338,6 +358,8 @@ const datasetView = () =>
}, 1000);
},
async selectDataset(event, obj, objId, objVersion, objPath) {
console.debug("Inside selectDataset")
console.debug(event)
event.preventDefault()
var newBrowserTab = event.ctrlKey || event.metaKey || (event.button == 1)
if (obj != null) {
Expand Down Expand Up @@ -375,29 +397,6 @@ const datasetView = () =>
this.search_tags = []
this.clearSearchTagText()
},
selectDescription(desc) {
if (desc.content.startsWith("path:")) {
this.description_ready = false;
filepath = desc.content.split(":")[1];
extension = "." + filepath.split(".")[1];
desc_file = getFilePath(
this.selectedDataset.dataset_id,
this.selectedDataset.dataset_version,
desc.path,
extension
);
fetch(desc_file)
.then((response) => response.blob())
.then((blob) => blob.text())
.then((markdown) => {
this.description_display = marked.parse(markdown);
this.description_ready = true;
});
} else {
this.description_display = desc.content;
this.description_ready = true;
}
},
gotoHome() {
// if there is NO home page set:
// - if there is a tab name in the URL, navigate to current
Expand Down Expand Up @@ -464,12 +463,19 @@ const datasetView = () =>
gotoURL(url) {
window.open(url);
},
openWithBinder(dataset_url) {
openWithBinder(dataset_url, current_dataset) {
const environment_url =
"https://mybinder.org/v2/gh/datalad/datalad-binder/main";
const content_url = "https://github.com/jsheunis/datalad-notebooks";
const content_repo_name = "datalad-notebooks";
const notebook_name = "download_data_with_datalad_python.ipynb";
if (current_dataset.hasOwnProperty("notebooks") && current_dataset.notebooks.length > 0) {
// until including the functionality to select from multiple notebooks in a dropdown, just select the first one
notebook = current_dataset.notebooks[0]
content_url = notebook.git_repo_url.replace(".git", "")
content_repo_name = content_url.substring(content_url.lastIndexOf('/') + 1)
notebook_name = notebook.notebook_path
}
binder_url =
environment_url +
"?urlpath=git-pull%3Frepo%3D" +
Expand Down Expand Up @@ -558,8 +564,9 @@ const datasetView = () =>
fetch(doi, { headers })
.then((response) => response.text())
.then((data) => {
this.selectedDataset.citation_text = data;
console.log(data);
// strip html tags from response text
let doc = new DOMParser().parseFromString(data, 'text/html');
this.selectedDataset.citation_text = doc.body.textContent || "";
this.citation_busy = false;
});
} else {
Expand All @@ -582,7 +589,7 @@ const datasetView = () =>
// If a tab parameter is supplied via the router, navigate to that tab if
// part of available tabs, otherwise default tab
else {
selectTab = tabs.indexOf(tab_param.toLowerCase())
selectTab = available_tabs.indexOf(tab_param.toLowerCase())
if (selectTab >= 0) {
this.tabIndex = selectTab;
} else {
Expand All @@ -598,6 +605,7 @@ const datasetView = () =>
},
},
async beforeRouteUpdate(to, from, next) {
console.debug("Executing navigation guard: beforeRouteUpdate")
this.subdatasets_ready = false;
this.dataset_ready = false;

Expand All @@ -624,7 +632,6 @@ const datasetView = () =>
this.$root.selectedDataset.keywords = this.$root.selectedDataset.keywords
? this.$root.selectedDataset.keywords
: [];
this.dataset_ready = true;

if (
this.$root.selectedDataset.hasOwnProperty("subdatasets") &&
Expand Down Expand Up @@ -706,7 +713,7 @@ const datasetView = () =>
this.$root.selectedDataset.has_files = false;
}
// Now list all tabs and set the correct one
// order in DOM: subdatasets, content, publications, funding, provenance,
// order in DOM: content, subdatasets, publications, funding, provenance,
sDs = this.$root.selectedDataset
available_tabs = ['content', 'subdatasets']
standard_tabs = ['publications', 'funding', 'provenance']
Expand Down Expand Up @@ -741,9 +748,11 @@ const datasetView = () =>
available_tabs_lower,
this.$root.selectedDataset.config?.dataset_options?.default_tab
)
this.dataset_ready = true;
next();
},
async created() {
console.debug("Executing lifecycle hook: created")
// fetch superfile in order to set id and version on $root
homefile = metadata_dir + "/super.json";
homeresponse = await fetch(homefile);
Expand Down Expand Up @@ -772,7 +781,6 @@ const datasetView = () =>
}
text = await response.text();
app.selectedDataset = JSON.parse(text);
this.dataset_ready = true;
if (
this.$root.selectedDataset.hasOwnProperty("subdatasets") &&
this.$root.selectedDataset.subdatasets instanceof Array &&
Expand Down Expand Up @@ -872,8 +880,10 @@ const datasetView = () =>
available_tabs_lower,
this.$root.selectedDataset.config?.dataset_options?.default_tab
)
this.dataset_ready = true;
},
mounted() {
console.debug("Executing lifecycle hook: mounted")
this.tag_options_filtered = this.tag_options;
this.tag_options_available = this.tag_options;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,12 +47,7 @@
]
},
"name": "Studyforrest Structural MRI scans",
"description": [
{
"source": "datacite_gin",
"content": "\"This dataset contains T1 and T2 weighted MRI scans, susceptibility weighted\n images, and diffusion imaging scans. These data are suitable for analyses\n of brain structure.\n\n For more information about the project visit: http://studyforrest.org\""
}
],
"description": "\"This dataset contains T1 and T2 weighted MRI scans, susceptibility weighted\n images, and diffusion imaging scans. These data are suitable for analyses\n of brain structure.\n\n For more information about the project visit: http://studyforrest.org\"",
"license": {
"name": "Open Data Commons Public Domain Dedication and License (PDDL)",
"url": "https://opendatacommons.org/licenses/pddl/"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -165,12 +165,7 @@
"value": 11
}
],
"description": [
{
"source": "datacite_gin",
"content": "\"Extension of the dataset published in Hanke et al. (2014; doi:10.1038/sdata.2014.3) with\n additional acquisitions for 6 of the original 20 participants (and one additional participant).\n Participants performed a central fixation task while being stimulated with oriented gratings.\n The experiment was repeated in four different sessions with 7T BOLD fMRI acquisition, each\n with a different scan resolution (0.8, 1.4, 2.0, and 3.0 mm).\n \n This research was supported by a grant from the German Research Foundation (DFG) awarded to\n S. Pollmann and O. Speck (DFG PO 548/15-1). This research was, in part, also supported by the\n German Federal Ministry of Education and Research (BMBF) as part of a US-German collaboration\n in computational neuroscience (CRCNS), co-funded by the BMBF and the US National Science\n Foundation (BMBF 01GQ1112; NSF 1129855). Work on the data-sharing technology employed for this\n research was supported by US-German CRCNS project, co-funded by the BMBF and the US National\n Science Foundation (BMBF 01GQ1411; NSF 1429999).\""
}
],
"description": "\"Extension of the dataset published in Hanke et al. (2014; doi:10.1038/sdata.2014.3) with\n additional acquisitions for 6 of the original 20 participants (and one additional participant).\n Participants performed a central fixation task while being stimulated with oriented gratings.\n The experiment was repeated in four different sessions with 7T BOLD fMRI acquisition, each\n with a different scan resolution (0.8, 1.4, 2.0, and 3.0 mm).\n \n This research was supported by a grant from the German Research Foundation (DFG) awarded to\n S. Pollmann and O. Speck (DFG PO 548/15-1). This research was, in part, also supported by the\n German Federal Ministry of Education and Research (BMBF) as part of a US-German collaboration\n in computational neuroscience (CRCNS), co-funded by the BMBF and the US National Science\n Foundation (BMBF 01GQ1112; NSF 1129855). Work on the data-sharing technology employed for this\n research was supported by US-German CRCNS project, co-funded by the BMBF and the US National\n Science Foundation (BMBF 01GQ1411; NSF 1429999).\"",
"publications": [
{
"type": "",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -172,12 +172,7 @@
"value": 10
}
],
"description": [
{
"source": "datacite_gin",
"content": "\"Extension of a matching fMRI dataset (Sengupta, et al., 2017; OpenFMRI ds000113c) on participants performing a\n central fixation task while being stimulated with oriented visual gratings. This dataset extends the previous one\n with acquisitions for 3 matching spatial resolutions (1.4, 2.0, and 3.0 mm) at 3T (complementing the previous 7T\n acquisitions at 0.8, 1.4, 2.0, and 3.0 mm). Five of the total of seven participants are identical in both datasets.\n All participants are part of the studyforrest.org project.\n\n- This research was enabled by a grant from the German Research Foundation (DFG) awarded to S. Pollmann and\n O. Speck (DFG PO 548/15-1).\n\n- This research was, in part, also supported by the German Federal Ministry of Education and Research (BMBF) as\n part of a US-German collaboration in computational neuroscience (CRCNS), co-funded by the BMBF and the US\n National Science Foundation (BMBF 01GQ1112; NSF 1129855).\n\n- Data management technology (DataLad) employed for this research was supported by US-German CRCNS project,\n co-funded by the BMBF and the US National Science Foundation (BMBF 01GQ1411; NSF 1429999).\""
}
],
"description": "\"Extension of a matching fMRI dataset (Sengupta, et al., 2017; OpenFMRI ds000113c) on participants performing a\n central fixation task while being stimulated with oriented visual gratings. This dataset extends the previous one\n with acquisitions for 3 matching spatial resolutions (1.4, 2.0, and 3.0 mm) at 3T (complementing the previous 7T\n acquisitions at 0.8, 1.4, 2.0, and 3.0 mm). Five of the total of seven participants are identical in both datasets.\n All participants are part of the studyforrest.org project.\n\n- This research was enabled by a grant from the German Research Foundation (DFG) awarded to S. Pollmann and\n O. Speck (DFG PO 548/15-1).\n\n- This research was, in part, also supported by the German Federal Ministry of Education and Research (BMBF) as\n part of a US-German collaboration in computational neuroscience (CRCNS), co-funded by the BMBF and the US\n National Science Foundation (BMBF 01GQ1112; NSF 1129855).\n\n- Data management technology (DataLad) employed for this research was supported by US-German CRCNS project,\n co-funded by the BMBF and the US National Science Foundation (BMBF 01GQ1411; NSF 1429999).\"",
"publications": [
{
"type": "",
Expand Down
Loading

0 comments on commit d8cc3ec

Please sign in to comment.