Sync of causatives 37 -> managed 38 - 2026.07.12

[northwestwitch]

To do

• Do a backup of managed variants from scout instance 38 (export the collection in json format using compass)
• On hasta CLI prod, export the causative variants from last month (SNVs and cancer SNVs, build 37):

up
scout export causatives -b 37 --category cancer --category snv --as-managed --managed-link-base-url https://scout.scilifelab.se --within-days 31

• Empty managed variants collection on STAGE instance build 37
• Load the managed variants exported using the CLI before
• Install Export and re-import managed variants with bcftools liftover #6225 on hasta stage:

down
us
bash /home/proj/production/servers/resources/hasta.scilifelab.se/update-tool-stage.sh -e S_scout -t scout -b managed_liftover

• Lift over and export the managed variants 37 to 38:

scout export managed --liftover-from 37

• Empty managed variants stage 38 collection
• Add the lines exported to a file and import in the managed variants stage, build 38
• Check formats, and if fine load the file into managed variants prod, build 38

[dnil]

You may want to add steps to save the STAGE 37 collection as well, and restore it after export. But it is stage, so.. 🤷‍♂️

[northwestwitch]

You may want to add steps to save the STAGE 37 collection as well, and restore it after export. But it is stage, so.. 🤷‍♂️

I would skip this, but I'dd add a step where I load the lifted variants to scout 38 stage. To see that they work and that the format is fine. Done

[northwestwitch]

Sync results:

• The last month has produced: 341 new causatives in build 38 from any institute.
• Loading them into managed variants 37 stage triggered one error, specifically for the causative variant of this case: F0078865

A number of them are multiple times the same variant, because there are 299 variants actually inserted in the collection. I haven't counted them all but in the list of managed variants I can actually see that also from the links to multiple variants. This applies especially to cancer SNVs, see for instance those in JAK2 or SF3B1

• Lift them over to build 38 ->

• Loaded lifted variants to scout 38 stage:

[northwestwitch]

All good until the last step of my previous comment. Good that I decided to give a test to the lifted variants on stage first. The links created are like this, with an extra / included towards the beginning. I will fix it in that PR branch! Easy but annoying

[northwestwitch]

All good until the last step of my previous comment. Good that I decided to give a test to the lifted variants on stage first. The links created are like this, with an extra / included towards the beginning. I will fix it in that PR branch! Easy but annoying

Ah no, the error is not in that branch. It's actually the URL I provide in the first step. It added an extra /. I'll fix directly in that file

[northwestwitch]

There is still something wrong in the exported causatives, possibly a bug. Check for instance this link:

https://scout.scilifelab.se/cust002/F0079025/a5baae11f01acfa48c0be56d2dcd7367

But this variant belongs to cust002.

All links are broken like this except for those that are really from cust003

[northwestwitch]

This is blocked by a bug: #6395

[northwestwitch]

The bug has been fixed, but we have to wait for the next release of the software to be deployed on hasta prod in order to do this sync