Installing conda packages locally
June 21 2018
Conda package manager
Conda is an open source package manager to make installation of packages and their dependencies relatively easier (like pip). Package managers are especially really helpful in an HPC setting, allowing the users to install packages and their many dependencies locally with just one command. We all know how painful software installation can be.
This blog post walks through the steps to install packages using
- Download and build from source
- using "pip install" in a python virtual environment - goto here
- using "
condainstall" in a condaenvironment
You can install programs using these options as well, we are working on these
Installing packages using
Step1 – Load the anaconda module
The above command will list the modules that are already loaded to your environment. The note to make here is python2 is already loaded.
[bhnala@h2 maker-2]$ module list
Currently Loaded Modulefiles:
1) gcc/6.3.0 3) python/2.7.13 5) quota/1.0 7) core
2) intel/18.0.2 4) perl/5.24.1 6) git/2.13.0
Anaconda also uses python but likes its own python installation, which means you need to unload this default python and load an anaconda environment.
module unload python
module load anaconda/python2.7/4.3.1 # anaconda module with python2
module load anaconda/python2.7/4.3.1 # anaconda module with python3
Step 2- Create a
Let's first take a look at why you need to create environments.
Many of you may have tried to run "
conda install package-to-install
Don't forget to replace "package-to-install" with an actual
What happened is that the command
Now, lets actually create a
conda create -y -n
Replace the name
Now, within this environment you can download as many
Step 3- Activate the environment you just created
Don't forget to replace
Step 4- Download and install the
conda package in the environment
This command may vary based on the program you are installing- lookup the documentation of the program or script. For example, to download
conda install -c
conda install -c
"-c" flag is telling
To confirm that the program installed type
conda list #the package you installed should be listed here now and its dependencies.
Step 5- Run your analysisRun your commands for the program here within the environment. To make sure you are within the environment, look for the
If you don’t see the
Step 6- Deactivating environment
Once you are done with your analysis- you can deactivate the
The command prompt will now go back to be
conda environment already installed
The next time you would like to run the program you installed in this
source activate conda-env
Run the commands.
Deactivate the environment
You don’t have to install the package every time you activate the environment, it should already be installed.
Submitting these commands as a job (new to writing job scripts- go here), just add the above lines to your job script. For example
#PBS -m abe
#PBS -M <your email>
#PBS -N <name of
#PBS -l nodes=1:ppn=1,vmem=16gb,walltime=2:00:00 #See note below
#The –l flag requires you to set how long and what resources you are requesting. Please see https://kb.iu.edu/d/bdkd for information on this setting.
#Load modules required
module unload python
module load anaconda/python2.7/4.3.1 #or anaconda/python2.7/4.3.1
#you must enter the directory where your data is, with an absolute path
#enter your commands here
#end of the job script
- To take a look at the packages available in anaconda module you can run the command
conda list #lists of all the packages
- To list all the
condaenvironments you built
conda info --
- To delete a
conda env remove --name conda-env #replace the conda-env to your env name.
- Sharing your
#activate the env and run the command
conda env export > environment.yml #exports the list of dependencies in your
#Email this environment.yml file to your friend and then ask them to run
conda env create -f enviroment.yml
This will download the