Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reference files are not downloading #12

Open
ankitasks1 opened this issue Nov 27, 2018 · 9 comments
Open

Reference files are not downloading #12

ankitasks1 opened this issue Nov 27, 2018 · 9 comments

Comments

@ankitasks1
Copy link

Hi,
I am using MEA for my analysis.
The reference file are not downloading.

Please see.
Which type of analysis do you want to do?
[1] ChIPseq
[2] RNAseq
[3] BSseq
Press 1, 2 or 3 [current: 2] : 1

Which type of alignment tool do you want to use?
[1] BWA
[2] Bowtie
[3] Bowtie2 (recommended)
Press 1, 2 or 3 [current: 1] : 3

How would you like to specify the reference genome location?
[1] Keep current settings
[2] Automatically download genome, or change between pre-exsiting builds (recommended)
[3] Manually set file paths to local files
Press 1, 2 or 3 [current: 2] : 2

Which genome would you like to download?
Recent builds: mm10 ; hg19 ; ce11 ; dm6 ;rn6 ; sacCer3
Older builds: mm9 ; h18 ; ce10 ; dm5 ; rn5 ; sacCer2 ; etc.
Please specify build code [current: undefined] : mm10
File .fa does not exist. Downloading it now...
Downloading mm10.chrom.sizes from UCSC...
--2018-11-27 11:37:50-- http://hgdownload-test.cse.ucsc.edu/goldenPath/mm10/bigZips/mm10.chrom.sizes
Resolving hgdownload-test.cse.ucsc.edu (hgdownload-test.cse.ucsc.edu)... 128.114.198.32
Connecting to hgdownload-test.cse.ucsc.edu (hgdownload-test.cse.ucsc.edu)|128.114.198.32|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://hgdownload-test.cse.ucsc.edu/goldenPath/mm10/bigZips/mm10.chrom.sizes [following]
--2018-11-27 11:37:51-- https://hgdownload-test.cse.ucsc.edu/goldenPath/mm10/bigZips/mm10.chrom.sizes
Connecting to hgdownload-test.cse.ucsc.edu (hgdownload-test.cse.ucsc.edu)|128.114.198.32|:443... connected.
The certificate's owner does not match hostname ‘hgdownload-test.cse.ucsc.edu’
Downloading mm10.fa from UCSC
--2018-11-27 11:37:52-- http://hgdownload-test.cse.ucsc.edu/goldenPath/mm10/bigZips/mm10.2bit
Resolving hgdownload-test.cse.ucsc.edu (hgdownload-test.cse.ucsc.edu)... 128.114.198.32
Connecting to hgdownload-test.cse.ucsc.edu (hgdownload-test.cse.ucsc.edu)|128.114.198.32|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://hgdownload-test.cse.ucsc.edu/goldenPath/mm10/bigZips/mm10.2bit [following]
--2018-11-27 11:37:53-- https://hgdownload-test.cse.ucsc.edu/goldenPath/mm10/bigZips/mm10.2bit
Connecting to hgdownload-test.cse.ucsc.edu (hgdownload-test.cse.ucsc.edu)|128.114.198.32|:443... connected.
The certificate's owner does not match hostname ‘hgdownload-test.cse.ucsc.edu’
mm10.2bit is not a twoBit file
sed: can't read mm10.chrom.sizes: No such file or directory
Download complete. Setting new path variables

Thanks
@julienrichardalbert
Copy link
Owner

Thank you for bringing this to our attention, this was a bug caused by UCSC changing their FTP link addresses. MEA has been updated to fix the automatic download step. Appreciated!

@ankitasks1
Copy link
Author

Hi,
I tested MEA again. The error still persist, even after new release files.
It also asks for sudo permit without which it says permission denied. However in MEA manual sudo is not mentioned.
Please see my commands:
sudo docker build -t taskkoike/mea:1.0 /home/ankitv/tools_av/MEA-master/docker/
export MEA='docker run -v /media//MEA/test/mm10:/mea-data -i taskkoike/mea:1.0'
sudo $MEA setting

Thanks

@biscuit13161
Copy link
Collaborator

@ankitasks1
docker does not require sudo to build the container, unless your specific environment is set up to enforce it. This would also explain why you require sudo to run MEA.

@biscuit13161 biscuit13161 reopened this Dec 2, 2018
@ankitasks1
Copy link
Author

Hi,
The error in downloading files is still there. I downloaded the newer version uploaded by you 2 days back. The domain name of UCSC download is still different in MEA.
I set the docker install path as in my previous command.
Error:
Please specify build code [current: mm10] : mm10
File .fa does not exist. Downloading it now...
Downloading mm10.chrom.sizes from UCSC...
--2018-12-02 11:33:51-- http://hgdownload-test.cse.ucsc.edu/goldenPath/mm10/bigZips/mm10.chrom.sizes
Resolving hgdownload-test.cse.ucsc.edu (hgdownload-test.cse.ucsc.edu)... 128.114.198.32
Connecting to hgdownload-test.cse.ucsc.edu (hgdownload-test.cse.ucsc.edu)|128.114.198.32|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: **https://hgdownload-test.cse.ucsc.edu**/goldenPath/mm10/bigZips/mm10.chrom.sizes [following]
--2018-12-02 11:33:52-- https://hgdownload-test.cse.ucsc.edu/goldenPath/mm10/bigZips/mm10.chrom.sizes
Connecting to hgdownload-test.cse.ucsc.edu (hgdownload-test.cse.ucsc.edu)|128.114.198.32|:443... connected.
The certificate's owner does not match hostname ‘hgdownload-test.cse.ucsc.edu’
Downloading mm10.fa from UCSC
--2018-12-02 11:33:53-- http://hgdownload-test.cse.ucsc.edu/goldenPath/mm10/bigZips/mm10.2bit
Resolving hgdownload-test.cse.ucsc.edu (hgdownload-test.cse.ucsc.edu)... 128.114.198.32
Connecting to hgdownload-test.cse.ucsc.edu (hgdownload-test.cse.ucsc.edu)|128.114.198.32|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://hgdownload-test.cse.ucsc.edu/goldenPath/mm10/bigZips/mm10.2bit [following]
--2018-12-02 11:33:54-- https://hgdownload-test.cse.ucsc.edu/goldenPath/mm10/bigZips/mm10.2bit
Connecting to hgdownload-test.cse.ucsc.edu (hgdownload-test.cse.ucsc.edu)|128.114.198.32|:443... connected.
The certificate's owner does not match hostname ‘hgdownload-test.cse.ucsc.edu’
mm10.2bit is not a twoBit file
sed: can't read mm10.chrom.sizes: No such file or directory
Download complete. Setting new path variables

As you will notice -test is extra word in http address. I dont know how to remove it..

Also if I correctly understood MEA manual, we do not need to install any software in /opt directory and rather just give docker installation commands and all dependencies will be set.
Well it is not working this way.

Any suggestion and help will be appreciated

Thanks

@biscuit13161
Copy link
Collaborator

@ankitasks1
I'm afraid it appears you haven't managed to download the updated version of MEA, I can confirm that the version currently on github does contain the correct URLs.

@biscuit13161
Copy link
Collaborator

@ankitasks1
"Also if I correctly understood MEA manual, we do not need to install any software in /opt directory and rather just give docker installation commands and all dependencies will be set.
Well it is not working this way."
in what way is it not working... are the dependencies missing from the /opt directory within the container? (you can check using docker run -ti taskkoike/mea:1.0 then ls /opt)

@ankitasks1
Copy link
Author

Hi thanks for suggestion.
It was not working with new docker installation also. So I edited the settings.sh file in container manually (which still contain wrong http address of ucsc ). Basically I removed -test.
So Downloading worked after that

Next createGenome command has given the error that VCF file does not exist.
sudo $MEA createGenome -snps-indels-separately /media/Arc1/MEA/test/mm10/mgp.v5.merged.snps_all.dbSNP142.vcf.gz /media/Arc1/MEA/test/mm10/mgp.v5.merged.indels.dbSNP142.normed.vcf.gz PWK_PhJ C57BL6J /media/Arc1/MEA/test/mm10/mousechipseq

[mea] [createGenome] Started at [Sun Dec 2 15:09:49 UTC 2018]
Error: File /media/Arc1/MEA/test/mm10/mgp.v5.merged.snps_all.dbSNP142.vcf.gz does not exist

@julienrichardalbert
Copy link
Owner

Hi again,
re: downloading a new version of MEA, you have to remove the Docker image/container on your system before installing the new one or else Docker will attempt to save time by NOT installing the new version.

re: createGenome, please see the user manual about setting the "mea-data" path. The command will then read something more similar to:

$MEA createGenome -snps-indels-separately
mea-data/mgp.v5.merged.snps_all.dbSNP142.vcf.gz
mea-data/mgp.v5.merged.indels.dbSNP142.normed.vcf.gz PWK_PhJ C57BL6J
mea-data/mousechipseq

lastly, please use VCF v4.1 files as we have not empirically tested allele-specific analyses using 4.2 files.

@ankitasks1
Copy link
Author

ankitasks1 commented Dec 4, 2018 •
edited
Loading

Thank you for the suggestion.
The commands worked Ok with V5 version of VCF, but not with the recently released version V6 mgp.v6.merged.norm.snp.indels.sfiltered.vcf.gz

If possible please update MEA for the same also.

Thank you very much

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@biscuit13161 @julienrichardalbert @ankitasks1