institute of biotechnology >> brc >> bioinformatics >> internal >> biohpc cloud: user guide
 

BioHPC Cloud:
: User Guide

Support e-mail is now support@biohpc.cornell.edu
Upgrade to Rocky 9
2 active announcements posted - click here to read full text

 


BioHPC Cloud Software

There is 944 software titles installed in BioHPC Cloud. The sofware is available on all machines (unless stated otherwise in notes), complete list of programs is below, please click on a title to see details and instructions. Tabular list of software is available here

Please read details and instructions before running any program, it may contain important information on how to properly use the software in BioHPC Cloud.

3d-dna, 454 gsAssembler or gsMapper, a5, ABRicate, ABruijn, ABySS, AdapterRemoval, adephylo, Admixtools, Admixture, agrep, albacore, Alder, AliTV-Perl interface, AlleleSeq, ALLMAPS, ALLPATHS-LG, Alphafold, AMOS, AMPHORA, amplicon.py, AMRFinder, analysis, ANGSD, Annovar, ant, antiSMASH, anvio, apollo, arcs, ARGweaver, Arlequin, ART, aspera, assembly-stats, ASTRAL, atac-seq-pipeline, ataqv, athena_meta, ATLAS, Atlas-Link, ATLAS_GapFill, atom, ATSAS, Augustus, AWS command line interface, AWS v2 Command Line Interface, axe, axel, BactSNP, bakta, bam2fastx, bamtools, bamUtil, barcode_splitter, BarNone, Basset, BayeScan, Bayescenv, baypass, bazel, BBMap/BBTools, BCFtools, bcl2fastq, BCP, Beagle, Beast2, bedops, BEDtools, bfc, bgc, bgen, bigQF, bigWig, bioawk, biobakery, biobambam, Bioconductor, biom-format, BioPerl, BioPython, Birdsuite, Bismark, Blackbird, blasr, BLAST, BLAST_to_BED, blast2go, BLAT, BlobToolKit, BLUPF90, BMGE, bmtagger, bonito, Boost, Bowtie, Bowtie2, BPGA, Bracken, BRAKER, BRAT-NextGen, BRBseqTools, BreedingSchemeLanguage, breseq, brocc, BSseeker2, BUSCO, BWA, bwa-mem2, bwa-meth, bwtool, cactus, CAFE, caffe, cagee, canu, CAP3, caper, CarveMe, catch, cBar, CBSU RNAseq, CCMetagen, CCTpack, cd-hit, cdbfasta, CEGMA, CellRanger, cellranger-arc, cellranger-atac, cellranger-dna, centrifuge, centroFlye, CFM-ID, CFSAN SNP pipeline, CheckM, chimera, chip-seq-pipeline, chromosomer, Circlator, Circos, Circuitscape, CITE-seq-Count, ClermonTyping, clues, CLUMPP, clust, Clustal Omega, CLUSTALW, Cluster, cmake, CMSeq, CNVnator, colabfold, compat, CONCOCT, Conda, Cooler, copyNumberDiff, cortex_var, CoverM, CRISPRCasFinder, CRISPResso, CrossMap, CRT, cuda, Cufflinks, cutadapt, cuteSV, dadi, dadi-1.6.3_modif, danpos, DBSCAN-SWA, dDocent, DeconSeq, Deepbinner, DeepTE, deepTools, defusion, delly, DESMAN, destruct, DETONATE, diamond, diploSHIC, discoal, Discovar, Discovar de novo, distruct, DiTASiC, DIYABC, Docker, dREG, dREG.HD, drep, Drop-seq, dropEst, dropSeqPipe, dsk, dssat, Dsuite, dTOX, duphold, dynare, ea-utils, ecopcr, ecoPrimers, ectyper, EDGE, edirect, EDTA, eems, EgaCryptor, EGAD, EIGENSOFT, elai, EMBLmyGFF3, EMBOSS, EMIRGE, Empress, entropy, epa-ng, ephem, epic2, ermineJ, ete3, EVM, exabayes, exonerate, ExpansionHunterDenovo-v0.8.0, eXpress, FALCON, FALCON_unzip, Fast-GBS, fasta, FastANI, fastcluster, FastME, FastML, fastp, FastQ Screen, fastq_pair, fastq_species_detector, FastQC, fastqsplitter, fastsimcoal2, fastspar, fastStructure, FastTree, FASTX, feems, feh, FFmpeg, fgbio, fineRADstructure, fineSTRUCTURE, FIt-SNE, flash, flash2, flexbar, Flexible Adapter Remover, Flye, FMAP, FragGeneScan, FragGeneScan, freebayes, FSA, funannotate, FunGene Pipeline, G-PhoCS, GADMA, GAEMR, Galaxy in Docker, Galaxy Server, GATK, gatk4, gatk4amplicon.py, gblastn, Gblocks, GBRS, gcc, GCTA, GDAL, gdc-client, GEM library, GEMMA, GENECONV, geneid, GeneMark, GeneMarker, Genome STRiP, Genome Workbench, GenomeMapper, GenomeStudio (Illumina), GenomeThreader, genometools, GenomicConsensus, genozip, gensim, GEOS, germline, gerp++, GET_PHYLOMARKERS, GffCompare, gffread, giggle, git, glactools, GlimmerHMM, GMAP/GSNAP, GNU Compilers, GNU parallel, go-perl, GO2MSIG, GoShifter, gradle-4.4, graftM, grammy, GraPhlAn, graphtyper, graphviz, GRiD, Grinder, GROMACS, GroopM, GSEA, gsort, GTDB-Tk, GTFtools, Gubbins, GUPPY, hail, HapCompass, HAPCUT, HAPCUT2, hapflk, HaploMerger, Haplomerger2, haplostrips, HaploSync, HapSeq2, HarvestTools, haslr, hdf5, hget, hh-suite, HiC-Pro, HiCExplorer, hifiasm, HISAT2, HMMER, Homer, HOTSPOT, HTSeq, htslib, https://github.com/CVUA-RRW/RRW-PrimerBLAST, humann, HUMAnN2, hyperopt, HyPhy, hyphy-analyses, iAssembler, IBDLD, idba, IDBA-UD, IDP-denovo, idr, idseq, IgBLAST, IGoR, IGV, IMa2, IMa2p, IMAGE, ImageJ, ImageMagick, Immcantation, impute2, IMSA-A, INDELseek, infernal, Infomap, inStrain, InStruct, Intel MKL, InteMAP, InterProScan, ipyrad, IQ-TREE, iRep, jags, Jane, java, jbrowse, JCVI, jellyfish, JoinMap, juicer, julia, jupyter, kallisto, Kent Utilities, keras, khmer, kinfin, king, KmerFinder, KmerGenie, kraken, KrakenTools, KronaTools, kSNP, kWIP, LACHESIS, lammps, LAPACK, LAST, lastz, lcMLkin, LDAK, LeafCutter, leeHom, lep-anchor, Lep-MAP3, lftp, Liftoff, Lighter, LinkedSV, LINKS, localcolabfold, LocARNA, LocusZoom, lofreq, longranger, LS-GKM, LTR_retriever, LUCY, LUCY2, LUMPY, lyve-SET, MACE, MACS, MaCS simulator, MACS2, maffilter, MAFFT, mafTools, MAGeCK, MAGeCK-VISPR, Magic-BLAST, magick, MAKER, mapDamage, MAQ, MARS, MASH, mashtree, Mashtree, MaSuRCA, MATLAB, Matlab_runtime, Mauve, MaxBin, MaxQuant, McClintock, mccortex, mcl, MCscan, MCScanX, medusa, megahit, MeGAMerge, MEGAN, MELT, MEME Suite, MERLIN, MetaBAT, MetaCache, MetaCRAST, metaCRISPR, MetAMOS, MetaPathways, MetaPhlAn, metaron, MetaVelvet, MetaVelvet-SL, methpipe, mfeprimer, MGmapper, Migrate-n, mikado, MinCED, Minimac3, Minimac4, minimap2, mira, miRDeep2, MISO (misopy), MITObim, MiXCR, MixMapper, MKTest, mlift, mlst, MMAP, MMSEQ, MMseqs2, MMTK, modeltest, MODIStsp-2.0.5, module, moments, mono, monocle3, mosdepth, mothur, MrBayes, mrsFAST, msld, MSMC, msprime, MSR-CA Genome Assembler, msstats, MSTMap, mugsy, MultiQC, multiz-tba, MUMandCo, MUMmer, mummer2circos, muscle, MUSIC, Mutation-Simulator, muTect, MZmine, nag-compiler, nanofilt, NanoPlot, Nanopolish, nanovar, ncftp, ncl, NECAT, Nemo, Netbeans, NEURON, new_fugue, Nextflow, NextGenMap, nf-core/rnaseq, ngmlr, NGS_data_processing, NGSadmix, ngsDist, ngsF, ngsLD, NgsRelate, ngsTools, NGSUtils, NINJA, NLR-Annotator, NLR-Parser, Novoalign, NovoalignCS, nQuire, NRSA, NuDup, numactl, nvidia-docker, nvtop, Oases, OBITools, Octave, OMA, Oneflux, OpenBLAS, openmpi, openssl, OrthoFinder, orthologr, Orthomcl, pacbio, PacBioTestData, PAGIT, pal2nal, paleomix, PAML, panaroo, pandas, pandaseq, pandoc, PanPhlAn, Panseq, Parsnp, PASA, PASTEC, PAUP*, pb-assembly, pbalign, pbbam, pbh5tools, PBJelly, pblat, pbmm2, PBSuite, pbsv, PCAngsd, pcre, pcre2, PeakRanger, PeakSplitter, PEAR, PEER, PennCNV, peppro, PERL, PfamScan, pgap, PGDSpider, ph5tools, Phage_Finder, PHAST, phenopath, Phobius, PHRAPL, PHYLIP, PhyloCSF, phyloFlash, phylophlan, PhyloPhlAn2, phylophlan3, PhyML, Picard, PICRUSt2, pigz, Pilon, Pindel, piPipes, PIQ, PlasFlow, platanus, Platypus, plink, plink2, Plotly, Point Cloud Library, popbam, PopCOGenT, PopLDdecay, Porechop, poretools, portcullis, POUTINE, pplacer, PRANK, preseq, primalscheme, primer3, PrimerBLAST, PrimerPooler, prinseq, prodigal, progenomics, progressiveCactus, PROJ, prokka, Proseq2, ProtExcluder, protolite, PSASS, psmc, psutil, purge_dups, pyani, PyCogent, pycoQC, pyfaidx, pyGenomeTracks, PyMC, pymol-open-source, pyopencl, pypy, pyRAD, Pyro4, pyseer, PySnpTools, python, PyTorch, PyVCF, QIIME, QIIME2, QTCAT, Quake, Qualimap, QuantiSNP2, QUAST, quickmerge, QUMA, R, RACA, racon, rad_haplotyper, RADIS, RadSex, RagTag, rapt, RAPTR-SV, RATT, RAxML, raxml-ng, Ray, rclone, Rcorrector, RDP Classifier, REAGO, REAPR, Red, ReferenceSeeker, regenie, Relate, RelocaTE2, Repbase, RepeatMasker, RepeatModeler, RERconverge, RFMix, RGAAT, rgdal, RGI, Rgtsvm, Ribotaper, ripgrep, rJava, rMATS, RNAMMER, rnaQUAST, Rnightlights, Roary, Rockhopper, rohan, rphast, Rqtl, Rqtl2, RSEM, RSeQC, RStudio, rtfbs_db, ruby, sabre, SaguaroGW, salmon, Sambamba, samblaster, sample, SampleTracker, samplot, samtabix, Samtools, Satsuma, Satsuma2, SCALE, scanorama, scikit-learn, Scoary, scythe, seaborn, SEACR, SecretomeP, selscan, Sentieon, seqkit, SeqPrep, seqtk, Seurat, sf, sgrep, sgrep sorted_grep, SHAPEIT, SHAPEIT4, shasta, Shiny, shore, SHOREmap, shortBRED, SHRiMP, sickle, sift4g, SignalP, SimPhy, simuPOP, singularity, sinto, sistr_cmd, SKESA, skewer, SLiM, SLURM, smcpp, smoove, SMRT Analysis, SMRT LINK, snakemake, snap, SnapATAC, SNAPP, snATAC, SNeP, Sniffles, snippy, snp-sites, SnpEff, SNPgenie, SNPhylo, SNPsplit, SNVPhyl, SOAP2, SOAPdenovo, SOAPdenovo-Trans, SOAPdenovo2, SomaticSniper, sorted_grep, spaceranger, SPAdes, SPALN, SparCC, SPARTA, split-fasta, sqlite, SRA Toolkit, srst2, stacks, Stacks 2, stairway-plot, stampy, STAR, Starcode, statmodels, STITCH, STPGA, StrainPhlAn, strawberry, Strelka, stringMLST, StringTie, STRUCTURE, Structure_threader, Struo2, stylegan2-ada-pytorch, subread, sumatra, supernova, SURPI, sutta, SV-plaudit, SVDetect, SVseq2, svtools, svtyper, SWAMP, sweed, SweepFinder, SweepFinder2, sweepsims, tabix, tagdust, Taiji, Tandem Repeats Finder (TRF), tardis, TargetP, TASSEL 3, TASSEL 4, TASSEL 5, tbl2asn, tcoffee, TensorFlow, TEToolkit, TEtranscripts, texlive, TFEA, tfTarget, thermonucleotideBLAST, ThermoRawFileParser, TMHMM, tmux, Tomahawk, TopHat, Torch, traitRate, Trans-Proteomic Pipeline (TPP), TransComb, TransDecoder, TRANSIT, transrate, TRAP, treeCl, treemix, Trim Galore!, trimal, trimmomatic, Trinity, Trinotate, tRNAscan-SE, UCSC Kent utilities, UMAP, UMI-tools, Unicycler, UniRep, unitig-caller, unrar, usearch, vamb, Variant Effect Predictor, VarScan, VCF-kit, vcf2diploid, vcfCooker, vcflib, vcftools, vdjtools, Velvet, vep, VESPA, vg, ViennaRNA, VIP, viral-ngs, virmap, VirSorter, VirusDetect, VirusFinder 2, vispr, VizBin, vmatch, vsearch, vt, WASP, wget, wgs-assembler (Celera), windowmasker, Wise2 (Genewise), wombat, Xander_assembler, xpclr, yaha

Details for jbrowse (If the copy-pasted commands do not work, use this tool to remove unwanted characters)

Name:jbrowse
Version:1.16.11
OS:Linux
About:Genome browser - local instance in BioHPC Lab using Docker
Added:3/29/2017 3:17:39 PM
Updated:8/7/2022 10:53:25 AM
Link:https://github.com/GMOD/jbrowse
Manual:http://gmod.org/wiki/JBrowse_Configuration_Guide
Notes:

JBrowse is a web based genome browser.  The following instructions show how to start a JBrowse server instance through docker, and how to access your data. (The docker image was built based on https://github.com/GMOD/jbrowse-docker, using nginx as web server)

In the text below labid should be replaced with your Lab ID.

Prepare data files

  • Copy your genome fasta file and any other files (gff, bed, bam) to the directory /workdir/$USER/data to your server. Example below uses  Volvox genome.
    #create a data directory, it could be any name but must be under /workdir/$USER
    mkdir /workdir/$USER/jbrowse_data

    #put genome fasta, gff, and bam files under /workdir/$USER/data, for testing, you can use these example files. 

    cp /programs/jbrowse-docker-1.16.11/example/volvox/*fa /workdir/$USER/jbrowse_data/
    cp /programs/jbrowse-docker-1.16.11/example/volvox/*gff3 /workdir/$USER/jbrowse_data/
    cp /programs/jbrowse-docker-1.16.11/example/volvox/volvox-sorted.bam* /workdir/$USER/jbrowse_data
    
  • If you have JBrowse fromatted data from previous JBrowse runs you can copy it to /workdir/$USER/jbrowse_data/

Start JBrowse instance

  • First, load docker image from the file (pick a verison, either 1.12.1 or 1.16.11)
    # to load version 1.12.1
    docker1 import /programs/docker/images/jbrowse.1.12.1.tar
    
    # to load version v1.16.11
    docker1 load -i /programs/docker/images/jbrowse.1.16.11.tar
    
    #verify the imported docker image, which should be biohpc_$USER/jbrowse 
    docker1 images
    
  • Start JBrowse container using image name from last step, and mount the data directory inside container as /jbrowse/data. "8029" is the external port number.  On BioHPC system, you can use a number between 8009 and 8039.
    #command to start v1.12.1
    docker1 run -dit -v /workdir/$USER/jbrowse_data:/jbrowse/data -p 8029:80 biohpc_$USER/jbrowse /docker-entrypoint.sh
    #command to start v1.16.11
    docker1 run -dit -v /workdir/$USER/jbrowse_data:/jbrowse/data -p 8029:80 biohpc_$USER/jbrowse:1.16.11 /docker-entrypoint.sh

Format your data for JBrowse

  • Format data (skip this step if you are using a previously formatted data directory)
    docker1 exec d3d74e7869a6 /jbrowse/bin/prepare-refseqs.pl --fasta /jbrowse/data/volvox.fa -out /jbrowse/data
    
    docker1 exec d3d74e7869a6 /jbrowse/bin/flatfile-to-json.pl --gff /jbrowse/data/volvox.gff3 --trackType CanvasFeatures --trackLabel gene --out /jbrowse/data
    
    #register bam files
    docker1 exec d3d74e7869a6 /jbrowse/bin/add-bam-track.pl --label mybam --bam_url /data/volvox-sorted.bam --in /jbrowse/data/trackList.json
    
    #make the data readable to web users
    docker1 exec d3d74e7869a6 chmod -R a+rX /jbrowse/data/

For more formatting options go to JBrowse documentation

Access JBrowse

  • There are two options here:
  • If you are on campus, you can access the web site directly. If your are off campus, you need to use Cornell VPN, or through SSH tunneling (see note below). 
  1. From your laptop, open a web browser, navigate to the URL   http://cbsuxxxxx.biohpc.cornell.edu:8029/

     
  2. If you are off campus , Tunnel website access via your ssh client and use web browser on your client machine.
    You need to map port 8080 on your reserved machine to port 8080 on your client machine using ssh.
    Windows.
    Open PuTTY and go to Connection->SSH->Tunnels in the left panel. Type 8080 in the Source port box and localhost:8080 in the Destination box, click Add. Now click on Session and proceed with normal session, i.e. connect to the rented machine. Once connected, start a Web browser on your client machine and type the following URL http://localhost:8029/
    Mac.
    Connect to the rented machine with ssh using "-D 8029:localhost:8029", e.g.
    ssh -D 8029:localhost:8029 labid@cbsuXXX.tc.cornell.edu
    then start a Web browser on your local client machine and type the following URL http://localhost:8029/

Stop JBrowse and (optionally) save formatted genome files for future use

Once you are finished please stop the container.
$ docker1 stop d3d74e7869a6

You can now save the formatted files for future use.
$ cd /workdir/$USER
$ tar -cf /home/labid/myjbrowsedata.tar data

Remove the container
$ docker1 rm d3d74e7869a6

 

Instructions to setup https:

You will need to email support@biohpc.cornell.edu for assistance with firewall setup. The ver1.16.11 image includes the certbot and certbox-nginx. Follow instructions this page to setup https. https://www.nginx.com/blog/using-free-ssltls-certificates-from-lets-encrypt-with-nginx/


Notify me if this software is upgraded or changed [You need to be logged in to use this feature]

 

Website credentials: login  Web Accessibility Help