[go: up one dir, main page]
Menu ▾ ▴

Crossbow errors and Cancer Genome Atlas data

Crossbow Help
Nobody/Anonymous
2013-04-13
2013-06-05

  • Nobody/Anonymous

    Nobody/Anonymous - 2013-04-13

    Hello,

    My name is Dr. Shyam Sarkar and I am located at Fremont, California.  I am working on Cancer

    genomics and Big Data Analytics.  I downloaded Crossbow and other tools and tried to run first

    e-coli example on my linux box as a single computer without Hadoop.  I got errors for getting data

    files from ftp locations.  Please see the error messages at the end.

    I am also curious about large data sets from Cancer Genome Atlas website for downloading and

    running with Crossbow on Amazon EMR.  My questions are as follows :

    (1) Data sets at Cancer Genome Atlas web site are mostly in text format.  I did not see .sra files there.

    How can I make Crossbow to work with text files uploaded at that site from universities like Stanford, MIT

    and Harvard ?

    (2) Is there an example Crossbow run with data downloaded from Cancer Genome Atlas ? Do I need a JAR

    file along with such data sets to download ? I want to get details of all steps for such data sets to run with

    Crossbow on Amazon EMR.

    Please help me on this issue.

    Best Regards,

    Dr. Shyam Sarkar

    Fremont, California

    E-mail: ssarkar@ayushnet.com

    Crossbow job

    Local commands in: /tmp/crossbow-Qx9746Bnqn/invoke.scripts/cb.7345.sh
    Running…
    === Directory checker ===
    Time: 11:55:14 12-Apr-2013
    Input: /home/ssarkar/crossbow/crossbow-1.2.0/example/e_coli/small.manifest
    Output: /home/ssarkar/crossbow/crossbow-1.2.0/example/e_coli/output_small
    Intermediate: /tmp/crossbow-Qx9746Bnqn/crossbow/intermediate/7345
    Counters: /home/ssarkar/crossbow/crossbow-1.2.0/example/e_coli/output_small_counters
    Options:
    Removing directory /home/ssarkar/crossbow/crossbow-1.2.0/example/e_coli/output_small due to -force
    Removing directory /home/ssarkar/crossbow/crossbow-1.2.0/example/e_coli/output_small_counters due to -force
    ==========================
    Stage 1 of 4. Preprocess
    ==========================
    Time: 11:55:14 12-Apr-2013
    === Map ===
    # parallel mappers: 1
    Input: /home/ssarkar/crossbow/crossbow-1.2.0/example/e_coli/small.manifest
    Output: /tmp/crossbow-Qx9746Bnqn/crossbow/intermediate/7345/preproc
    Intermediate: /tmp/crossbow-Qx9746Bnqn/crossbow/intermediate/7345/preproc.map.pre
    Retries / delay: 3 / 5
    Options:
    Starting 1 mappers with command:
    perl /home/ssarkar/crossbow/crossbow-1.2.0/Copy.pl -compress=gzip -stop=0 -maxperfile 500000 -fastq-dump /home/ssarkar/crossbow/sratoolkit.2.3.1-centos_linux64/bin/fastq-dump -push /tmp/crossbow-Qx9746Bnqn/crossbow/intermediate/7345/preproc -counters /home/ssarkar/crossbow/crossbow-1.2.0/example/e_coli/output_small_counters/counters.txt
    Pid 7372 processing input ftp://ftp-trace.ncbi.nih.gov/sra/sra-instant/reads/ByRun/litesra/SRR/SRR014/SRR014475/SRR014475.lite.sra        0 …
    Non-zero return (512) after closing pipe '| perl /home/ssarkar/crossbow/crossbow-1.2.0/Copy.pl -compress=gzip -stop=0 -maxperfile 500000 -fastq-dump /home/ssarkar/crossbow/sratoolkit.2.3.1-centos_linux64/bin/fastq-dump -push /tmp/crossbow-Qx9746Bnqn/crossbow/intermediate/7345/preproc -counters /home/ssarkar/crossbow/crossbow-1.2.0/example/e_coli/output_small_counters/counters.txt >/tmp/crossbow-Qx9746Bnqn/crossbow/intermediate/7345/preproc/map-07372 2>/tmp/crossbow-Qx9746Bnqn/crossbow/intermediate/7345/preproc.map.pre/map.err/map-07372'
    Retrying in 5 seconds…
    Non-zero return (2304) after closing pipe '| perl /home/ssarkar/crossbow/crossbow-1.2.0/Copy.pl -compress=gzip -stop=0 -maxperfile 500000 -fastq-dump /home/ssarkar/crossbow/sratoolkit.2.3.1-centos_linux64/bin/fastq-dump -push /tmp/crossbow-Qx9746Bnqn/crossbow/intermediate/7345/preproc -counters /home/ssarkar/crossbow/crossbow-1.2.0/example/e_coli/output_small_counters/counters.txt >/tmp/crossbow-Qx9746Bnqn/crossbow/intermediate/7345/preproc/map-07372 2>/tmp/crossbow-Qx9746Bnqn/crossbow/intermediate/7345/preproc.map.pre/map.err/map-07372'
    Retrying in 5 seconds…
    Non-zero return (2304) after closing pipe '| perl /home/ssarkar/crossbow/crossbow-1.2.0/Copy.pl -compress=gzip -stop=0 -maxperfile 500000 -fastq-dump /home/ssarkar/crossbow/sratoolkit.2.3.1-centos_linux64/bin/fastq-dump -push /tmp/crossbow-Qx9746Bnqn/crossbow/intermediate/7345/preproc -counters /home/ssarkar/crossbow/crossbow-1.2.0/example/e_coli/output_small_counters/counters.txt >/tmp/crossbow-Qx9746Bnqn/crossbow/intermediate/7345/preproc/map-07372 2>/tmp/crossbow-Qx9746Bnqn/crossbow/intermediate/7345/preproc.map.pre/map.err/map-07372'
    Retrying in 5 seconds…
    Non-zero return (2304) after closing pipe '| perl /home/ssarkar/crossbow/crossbow-1.2.0/Copy.pl -compress=gzip -stop=0 -maxperfile 500000 -fastq-dump /home/ssarkar/crossbow/sratoolkit.2.3.1-centos_linux64/bin/fastq-dump -push /tmp/crossbow-Qx9746Bnqn/crossbow/intermediate/7345/preproc -counters /home/ssarkar/crossbow/crossbow-1.2.0/example/e_coli/output_small_counters/counters.txt >/tmp/crossbow-Qx9746Bnqn/crossbow/intermediate/7345/preproc/map-07372 2>/tmp/crossbow-Qx9746Bnqn/crossbow/intermediate/7345/preproc.map.pre/map.err/map-07372'
    Retrying in 5 seconds…
    Fatal error 1.2.0:M130: Out of retries; aborting…

    When requesting support, please include the full output printed here.
    If a child process was the cause of the error, the output should
    include the relevant error message from the child's error log.  You may
    be asked to provide additional files as well.
    ******
    * Aborting master loop because child 7372 failed
    * (other children may also have failed)
    * Input file or string was:
    *   ftp://ftp-trace.ncbi.nih.gov/sra/sra-instant/reads/ByRun/litesra/SRR/SRR014/SRR014475/SRR014475.lite.sra        0
    * Error message is in file: /tmp/crossbow-Qx9746Bnqn/crossbow/intermediate/7345/preproc.map.pre/map.err/map-07372, also printed below
    ******
    * Warning: No TOOLNAME file in tool directory: Bin
    * Copy.pl: Line: ftp://ftp-trace.ncbi.nih.gov/sra/sra-instant/reads/ByRun/litesra/SRR/SRR014/SRR014475/SRR014475.lite.sra        0
    * Copy.pl: Not a comment line
    * Copy.pl: Doing unpaired entry SRR014475.lite.sra
    * Copy.pl: Fetching ftp://ftp-trace.ncbi.nih.gov/sra/sra-instant/reads/ByRun/litesra/SRR/SRR014/SRR014475/SRR014475.lite.sra SRR014475.lite.sra 0
    * reporter:counter:Short read preprocessor,Read data fetched,0
    * -fastq-dump argument "/home/ssarkar/crossbow/sratoolkit.2.3.1-centos_linux64/bin/fastq-dump" doesn't exist or isn't executable
    ******
    Fatal error 1.2.0:M140: Aborting because child with PID 7372 exited abnormally

    When requesting support, please include the full output printed here.
    If a child process was the cause of the error, the output should
    include the relevant error message from the child's error log.  You may
    be asked to provide additional files as well.
    Non-zero exitlevel from Preprocess stage
    elastic-mapreduce script completed with exitlevel 256

     

Log in to post a comment.