First version

[gavin-peng]

No description provided.

[gavin-peng]

see ticket https://jira.oicr.on.ca/browse/GRD-833

[pruzanov]

Since this is a first implementation I would strongly recommend to avoid using separate wdl files. It will add additional maintenance headaches, especially given that the files are almost identical. I see that in this case we could introduce a mode variable which would route the execution two one of two tasks - one for PERL and another for R mode. We had a similar approach implemented in bclconvert wf which allows choosing between hpc and dragen modes.

[lheisler]

I'm also inclined to choose and implement just one of the versions, perl or java. I'm working with CHUM now to get their vardict output and we can run on a test sample to see what each version generates. I'll also investigate a bit more to better understand if it is clear that a specific version is being used in the genpipes pipelines that CHUM is using

[gavin-peng]

Yes we are in the process of choosing one of perl or java version. For a small inputs they generate indentical output, the problem is for large inputs they both have outof memory issue. I have tested java multiple times increasing memory, last attempt used 512G job memory with > 300G inputs from CHUM, still failed because of memory after 60h 32m.
The vardict github page claims java version is 10X faster, for now I haven't observed huge difference in speed.

[gavin-peng]

Since this is a first implementation I would strongly recommend to avoid using separate wdl files. It will add additional maintenance headaches, especially given that the files are almost identical. I see that in this case we could introduce a mode variable which would route the execution two one of two tasks - one for PERL and another for R mode. We had a similar approach implemented in bclconvert wf which allows choosing between hpc and dragen modes.

The perl version now removed

[pruzanov]

I approved, but recommend modifying calculate.sh test script

[lheisler]

include the parameter
-th 8
which will give opportunity for the software to use multiple slots on the node

i tested this on one small dataset and it reduced the processing time from
166m to 33m

[lheisler]

even with threading, this runs very long.

for vardict, and for the neoantigen pipeline in particular, we should be okay to exclude intergenic regions

I tested this out with a bedfile from UCSC knowngenes, splitting the bed file by chromosome, and a simplified command.
jobs completed in the range of 3 m to 1258 m (20 h)

test command was
varDict -G REF -N MoHQ-CM-1-180.DT -b 'Tbam|Nbam' -c 1 -S 2 -E 3 -g 4 -th 8 -D -y knownGene.chr.bed
(the above is extracted from the current workflow command, which pipes to additional filtering commands. i removed the pipes for testing purposes)

my tests were with knownGene , but i think we can use the same bed file that is being used for TMB assessment

https://bitbucket.oicr.on.ca/projects/GSI/repos/interval-files/browse/accredited/MANE_Select_v1.3.bed

[gavin-peng]

Trying to run vardict with -th 8 and uses UCSC knowngene bed file, seeing speed much faster, chrY completed in 28 minutes, versus 33h 22m last time. Still running though.