Introduction#

Many programs are installed locally under user’s space. Some of these programs are: R packages, Python packages, Perl packages, Julia packages. There is already a minimal installation of R, Python, Perl and Julia as modules that can be used to install the packages a user may need. Other than that, there are programs hat can be installed under user’s account or as modules. We do not use sudo and package managers, like sudo apt-get install <package_name>. Most of the programs are installed from source or from pre-compiled binaries. The installation is based on one of the following procedures:

• Precompiled programs and binaries: unpack and adjust the parameters where needed.

• Compile source codes using Make: in this case, usally a Makefile is provides with the source files: make or make -j4 or make [some options].

• Compile the codes using: ./configure && make && make test {or check} && make install The ./configure command takes options like --prefix=[path to the installation directory] and generate the Makefile used to build the program.

• Compile the codes using: cmake && make && make test {or check} && make install The cmake command takes options like --DCMAKE_INSTALL_PREFIX=[path to the installation directory] and generate the Makefile used to build the program.

• For other programs, predefined bash scripts or python programs are provided to configure and build the software.

This section describes how to install:

• R packages

• Python packages

• Perl packages

• Julia packages

• Pre-compiled Java applications

• Compile a code from source using make: STAR as example.

• Configure and ompile a code from source using ./configure && make && make install: Treemix as example.

• Configure and ompile a code from source using cmake && make && make install: DIAMOND as example.

How to install R packages?#

First, use module spider to see how to load R module:

To load R module, use:

module load StdEnv/2023 gcc/12.3 r

Then launch R:

R

Then, install the packages using the following commands:

> Sys.setenv("DISPLAY"=":0.0")
> install.packages("sp")
> install.packages("dplyr")

To see the list of all the packages installed, run the command:

> installed.packages()

To load the packages, use:

> library("sp")
> library("dplyr")

Install other packages as needed.

Once the installation is over, close R session:

> quit()

What to include in your job script:

• All the modules used to install the packages: module load …
• The command line to run R program: Rscript your-r-program.R

How to install python packages?#

How to proceed with the installation of python packages?

• Python module (use module spider to see if the module is available).
• Load python module: module load …
• Load other dependencies if needed.
• Use module list to see if all the modules are correctly loaded.
• Create a virtual environment: virtualenv ~/my_venv
• Activate the virtual environment: source ~/my_venv/bin/activate
• Update pip if needed: pip install –upgrade pip
• Install the packages using pip install [package name].
• Generate a requirements.txt file for future reference: pip freeze > ~/requirements.txt
• Log all the steps and modules in a separate README file for future reference

Here is an example for installing cutadapt on national systems

• Load the modules:

module load python
module list

• Create a virtual environment:

virtualenv ~/my_venv

• Activate the virtual environment:

source ~/my_venv/bin/activate

• Upgrade pip if needed and install the packages:

pip install --upgrade pip
pip install cutadapt

• Run a quick test:

cutadapt --help

• Generate the requirements.txt file for future reference:

pip freeze > ~/requirements-cutadapt.txt

• Close the virtual environment:

deactivate

What to include in your job script:

• All the modules used to install the packages: module load …
• Activate the virtual environment: source ~/my_venv/bin/activate
• Command line to run the python program: python my-program.py or program [+options if any]

How to install perl packages?#

How to proceed with the installation of perl packages?

• Load perl module
• Install the packages using cpan or cpanm
• Test if the module is installed.

Here is an example for installing Hash::Merge package:

• First load perl module:

module load perl
cpan install Hash::Merge

Run a quick test to see if the package is installed:

echo "use Hash::Merge;" > test.pl && perl test.pl
echo "use HTTP::Tiny;" >> test.pl && perl test.pl

How to install Julia packages?#

How to proceed with the installation of Julia packages?

• Load julia module
• Load any other dependency or external module
• Start or lauch julia: julia
• Install the packages using: Pkg.add(“name of the package”)
• Test if the package is installed: using [name of the package

Note: This example was taken from this page

• First load julia and HDF5 modules: this package requires HDF5.

module load gcc hdf5 julia
export HDF5_DIR=$EBROOTHDF5

The command export HDF5_DIR=$EBROOTHDF5 is used to set the environment variable HDF5_DIR that points to the installation path of HDF5.

• Now, start julia and install the packages:

julia

julia> using Libdl
julia> push!(Libdl.DL_LOAD_PATH, ENV["HDF5_DIR"] * "/lib")
julia> using Pkg
julia> Pkg.add("JLD")
julia> using JLD

[~@login1 ~]$ module load gcc hdf5 julia
[~@login1 ~]$  export HDF5_DIR=$EBROOTHDF5
[~@login1 ~]$  julia       

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.12.5 (2026-02-09)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org release
|__/                   |
               _
julia> using Libdl

julia> push!(Libdl.DL_LOAD_PATH, ENV["HDF5_DIR"] * "/lib")
1-element Vector{String}:
 "/cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v4/Compiler/gcc12/hdf5/1.14.6/lib"

julia> using Pkg

julia> Pkg.add("JLD")

julia> using JLD

How to install Trimmomatic?#

How to proceed with pre-compiled java applications?

• Download the jar file using wget, curl …
• Load java module, use module spider java to see how to load a module.
• Run the application

Here is an example of Trimmomatic

• First, download Trimmomatic archive using wget:

wget http://www.usadellab.org/cms/uploads/supplementary/Trimmomatic/Trimmomatic-0.39.zip

• Unpack the archive: Trimmomatic-0.39.zip

unzip Trimmomatic-0.39.zip

• Load java modules:

module load java

• Run the code using:

java -jar <path to>/trimmomatic-0.39.jar
or
java -jar Trimmomatic-0.39/trimmomatic-0.39.jar --help

How to install STAR?#

**How to compile and install STAR on national clusters?

Home page: https://github.com/alexdobin/STAR

• First, download the version 2.7.11b from GitHub:

wget https://github.com/alexdobin/STAR/archive/refs/tags/2.7.11b.tar.gz

• Then, unpack the archive:

tar -xvf 2.7.11b.tar.gz

• Then, load the required modules (gcc):

module load gcc

• Then, go to the directory STAR-2.7.11b/source

cd STAR-2.7.11b/source/
ls
ls Makefile
make

• Look for the binary “STAR”:

ls -lrt STAR
realpath STAR

• Run STAR command with –help option:

./STAR --help

• Set the path and test the code:

Run the command ‘realpath STAR’ from the same directory from where you compiled the program to get the path to the program:

• Set the path:

export PATH=${PATH}:<path-to-STAR-2.7.11b/source>

What to include in a job script:

• load the same modules used to compile the code.
• set path to the location of the binary (STAR in this case).
• command line to run the code

module load gcc 
export PATH=$PATH:<path-to-STAR-2.7.11b/source>
STAR --help

How to install Treemix?#

How to configure and build Treemix on national clusters?

Note: the program requires boost and gcc Home page: https://bitbucket.org/nygcresearch/treemix/wiki/Home

• First, download the version 1.13:

wget https://bitbucket.org/nygcresearch/treemix/downloads/treemix-1.13.tar.gz

• Then, unpack the archive:

tar -xvf treemix-1.13.tar.gz

• Load the required modules and dependencies if any (Treemix requires boost):

module load gcc boost

• Go to the directory treemix-1.13, then configure and compile the program:

 
cd treemix-1.13
ls 
./configure --prefix=${HOME}/software/treemix/1.13
make
make install

The option –prefix=${HOME}/software/treemix/1.13 is used to specify the path to the installation directory. Otherwise, t he command make install will try to install the program under a directory where you do not have write access (directories owned by root) and leads to Permission denied error message.

What to include in a job script:

• load the same modules used to compile the code.
• set the path to the location of the binary.
• command line to run the code.

module load gcc boost
export PATH=$PATH:${HOME}/software/treemix/1.13/bin
treemix --help
treemix [+options if any]

How to install DIAMOND?#

How to configure and compile DIAMOND on CC clusters?

Home page: https://github.com/bbuchfink/diamond Reuires: gcc, cmake

• First, download the version 2.1.24:

wget https://github.com/bbuchfink/diamond/archive/refs/tags/v2.1.24.tar.gz

• Then, unpack the archive:

tar -xvf v2.1.24.tar.gz

• Load the required modules and dependencies if any:

module load gcc cmake

• Then, go to the directory diamond-2.1.24 to configure and compile the program:

cd diamond-2.1.24

• Create a build directory, go to build directory:

mkdir build
cd build

• Now, configure and compile the code:

cmake .. -DCMAKE_INSTALL_PREFIX=${HOME}/software/diamond/2.1.24
make
make test
make install

The option -DCMAKE_INSTALL_PREFIX=${HOME}/software/diamond/2.1.24 is used to specify the path to the installation directory. Otherwise, the command make install will try to install the program under a directory where you do not have write access (directories owned by root) and leads to Permission denied error message.

What to include in a job script:

• load the same modules used during the compilation.
• set the path to the binaries {installation directory}
• command line to run the code.

Note: cmake is not required at run time {only used during the configuration and compilation of the code}. Therefore, it is not needed to add it to the list of modules inside a job script.

module load gcc
export PATH=$PATH:${HOME}/software/diamond/2.1.24
diamond {+options if any}

How to install Geant4?#

How to configure and compile Geant4 on national clusters?

• First, download the version: 11.3.1

wget https://gitlab.cern.ch/geant4/geant4/-/archive/v11.3.1/geant4-v11.3.1.tar.gz

• Then, unpack the archive:

tar -xvf geant4-v11.3.1.tar.gz

• Load the required modules:

module load gcc clhep tbb qt cmake

• Change the directory, create a build directory, configure and compile the code:

cd geant4-v11.3.1
mkdir build
cd build

cmake .. -DCMAKE_INSTALL_PREFIX=${HOME}/software/geant4/11.3.1 -DCMAKE_BUILD_TYPE="Release" -DGEANT4_BUILD_MULTITHREADED="ON" -DBUILD_STATIC_LIBS:BOOL="ON" -DGEANT4_USE_GDML:BOOL="ON" -DXercesC_LIBRARY_RELEASE="$EBROOTGENTOO/lib64/libxerces-c.so" -DEXPAT_LIBRARY="$EBROOTGENTOO/lib64/libexpat.so" -DEXPAT_INCLUDE_DIR="$EBROOTGENTOO/include" -DCLHEP_DIR="$EBROOTCLHEP/lib/CLHEP-${CLHEPVERSION}" -DPKG_CONFIG_EXECUTABLE="/usr/bin/pkg-config" -DGEANT4_BUILD_STORE_TRAJECTORY:BOOL="ON" -DGEANT4_BUILD_TESTS:BOOL="ON" -DGEANT4_USE_RAYTRACER_X11="ON" -DGEANT4_USE_SYSTEM_ZLIB="ON" -DGEANT4_USE_OPENGL_X11="ON" -DGEANT4_ENABLE_TESTING="ON" -DGEANT4_USE_TBB="ON" -DGEANT4_USE_QT:BOOL="OFF" -DQT_QMAKE_EXECUTABLE="$EBROOTQT/bin/qmake" -DGEANT4_USE_QT_QT6="ON" -DGEANT4_INSTALL_EXAMPLES:BOOL="ON" -DGEANT4_INSTALL_DATA:BOOL="OFF" -DGEANT4_INSTALL_DATASETS_NUDEXLIB="ON" -DGEANT4_INSTALL_DATASETS_TENDL="ON" -DGEANT4_INSTALL_DATASETS_URRPT="ON" -DGEANT4_USE_HDF5="OFF" -DGEANT4_USE_VTK="OFF" -DGEANT4_USE_FREETYPE="ON" -DGEANT4_USE_G3TOG4="ON" -DGEANT4_USE_GDML="ON" -DGEANT4_USE_SMARTSTACK="ON" -DGEANT4_USE_INVENTOR="OFF" -DGEANT4_USE_PTL_LOCKS="ON" -DGEANT4_USE_SMARTSTACK="ON" -DGEANT4_USE_SYSTEM_PTL="OFF" -DGEANT4_USE_USOLIDS="OFF"  -DGEANT4_USE_XM="ON"
make -j8
make test
make install

Notes:

• To see all available options for cmake, use the GUI mode by running the command: ccmake instead of cmake.
• To avoid downloading data, use: -DGEANT4_INSTALL_DATA:BOOL=“OFF”
• To enable downloading data, use: -DGEANT4_INSTALL_DATA:BOOL=“ON”
• The configuration with cmake require many options, like setting the instalation path, the location of some external libraries (QT, CLHEP, EXPAT, …). The external libraries could be installed as external modules or pat of the operating system.
• If the compilation fails, change the options for cmake and start over.