Conda Environments in Docker/Singularity Containers

The Conda environment can be built either on a BioHPC server, or on your own Windows/Mac/Linux computer. If on your own computer, after the installation, the Docker or Singularity image file can be uploaded into any BioHPC server to run the software.

1. To do the installation on your own computer, you need to install Docker Desktop on your Windows (https://docs.docker.com/desktop/install/windows-install/) or Mac (https://docs.docker.com/desktop/install/mac-install/). Once installed, you can run the "docker run" or "conda install" commands using the Windows Command or Mac Terminal tools.

    

2. Select a start Docker image to build the Conda environment. We recommend two base images: mambaorg/micromamba or continuumio/miniconda3. The mambaorg/micromamba is much smaller and faster and supported by the Mamba team, while continuumio/miniconda3 is supported by the Anaconda team.

    

3. Create a text file named "Dockerfile" (no extension in file name) using the template below. Replace python, samtools and bwa with Conda package names that you want to install.

   FROM mambaorg/micromamba
   RUN micromamba install --yes --name base -c bioconda -c conda-forge \
         python=3.9.1 \
         samtools \
         bwa
   RUN micromamba clean --all --yes
   
   #if you need to run pip install in the same environment, uncomment the following lines
   
   #ARG MAMBA_DOCKERFILE_ACTIVATE=1
   
   #RUN pip install --no-cache myPythonPackage
      
   

    

4. Run the following commands to build the Docker image based on the Dockerfile. The image name should all be in lower case.

   On you Mac or Windows computer:

   cd directory_with_the_Dockerfile
   docker build -t myimage .
   

    

   On BioHPC

   cd directory_with_the_Dockerfile
   docker1 build -t myimage /workdir/$USER/directory_with_the_Dockerfile
   

   After done, you can see the newly created image by command (use docker1 on BioHPC).

   docker images
   

    

5. Save the docker image into a portable .tar file. Replace myimage with any name you want.

   docker save -o myimage.tar myimage
   

    

6. Convert the Docker image .tar file into a Singularity .sif file.

   Run the following command from the directory where the myimage.tar file is located. Replace myimage with your image name.

On Linux or Mac:

docker run --rm -v $(pwd):/data biohpc/sindocker singularity build --force myimage.sif docker-archive://myimage.tar

On Windows Command (double quotes are used in case there are spaces in directory names):

docker run --rm -v "%cd%":/data biohpc/sindocker singularity build --force myimage.sif docker-archive://myimage.tar

7. Keep the .tar file. As Docker image is easier to modify if needed later. Upload the ".sif" file to BioHPC server. You are ready to run the software

   #make the .sif file executable
   chmod a+x myimage.sif
   
   #run the software in container.
   #in this example, the ref.fa and reads.fq files should be located in the current directory where the command is launched. 
   #replace bwa with your own software command.
   
   #In newer version of singularity. To mount current directory, SINGULARITY_BINDPATH needs to be set to either $PWD or any directories above $PWD
   export SINGULARITY_BINDPATH=/workdir
   
   ./myimage.sif bwa mem ref.fa reads.fq > aln-se.sam
   

   • myimage.sif run as a Sigularity container. By default, the current directory and your home directory on the host server is mounted inside container as the same directory name. The container has access to 1) your home directory; 2) the current directory, and any directories under the home and current directory.