Working with modules

Introduction#

How to use modules on Grex? As examples, we show how to load gromacs and lammps.

Modules#

Here are the most used commands for manipulating modules:

  • module list
  • module avail
  • module spider soft/version
  • module load soft/version
  • module unload {rm} soft/version
  • module show soft
  • module help soft
  • module whatis soft
  • module purge; module –force purge
  • module use ~/modulefiles; module unuse ~/modulefiles

To see if a given software or program, called foo, is available on the software stach, run the command:

module spider foo

How to load gromacs on Grex?#

First, run the command module spider gromacs to search if gromacs is already available as a module:

module spider gromacs

The above shows that many versions are available:

  • gromacs/2021.6
  • gromacs/2022
  • gromacs/2023.3
  • gromacs/2024.1
  • gromacs/2025.2
  • gromacs/2025.3

Now, try to load a particular version 2025.3:

To see how to load this particular version, run the command:

module spider gromacs/2025.3

and read the instructions.

Two modules are available:

arch/avx512  gcc/13.2.0  openmpi/5.0.6
cuda/12.4.1  arch/avx2  gcc/13.2.0

Load one of the following modules:

  • For CPU version, use:
module load arch/avx512  gcc/13.2.0 openmpi/5.0.6 gromacs/2025.3
  • for GPU version, use:
module load cuda/12.4.1  arch/avx2  gcc/13.2.0 gromacs/2025.3 gromacs/2025.3

Load one module and experiment with the commands:

module list
module show gromacs
ls ${MODULE_GROMACS_PREFIX}/bin
which gmx_mpi
module rm gromacs
module show gromacs
which gmx_mpi
While running the command module show gromacs, have a look to the environment variables, like GROMACS_PREFIX that points to t he installation directory.
Note that gmx_mpi is a binary used to run gromacs. For another program, you should know the binaries to use.

How to load LAMMPS on Grex?#

First, run the command module spider lammps to search if lammps is already available as a module:

module spider lammps

The above shows that many versions are available:

  • lammps/2021-09-29
  • lammps/2024-08-29p1-nep
  • lammps/2024-08-29p1

Now, try to load a particular version 2024-08-29p1:

To see how to load this particular version, run the command:

module spider lammps/2024-08-29p1

and read the instructions.

Three modules are available:

  • arch/avx512 gcc/13.2.0 openmpi/4.1.6
  • arch/avx512 intel-one/2024.1 openmpi/4.1.6
  • cuda/12.4.1 arch/avx2 gcc/13.2.0 openmpi/4.1.6

Load one of the following modules:

  • CPU version built with GCC:
module load arch/avx512  gcc/13.2.0  openmpi/4.1.6 lammps/2024-08-29p1
  • CPU version built with Intel vompiler:
module load arch/avx512  intel-one/2024.1  openmpi/4.1.6 lammps/2024-08-29p1
  • GPU version:
module load cuda/12.4.1  arch/avx2  gcc/13.2.0  openmpi/4.1.6 lammps/2024-08-29p1

Load one module and experiment with the commands:

module list  
module show lammps
ls ${MODULE_LAMMPS_PREFIX}/bin
which lmp
module rm lammps
module show lammps
which lmp
While running the command module show lammps, have a look to the environment variables, like MODULE_LAMMPS_PREFIX that points to the installation directory.
Note that lmp is a binary used to run lammps. For another program, you should know the binaries to use.

Experiment with other programs#

For more examples, experiment with other programs, like boost, netcdf, python, openmm, geant4 or any other program you have in mind.

CVMFS on Grex#

How to switch to CCEnv which provides the same software stack installed on national clusters?

For the first time, you may need to run the commands:

ls /cvmfs/
ls -1 /cvmfs/

to see what directories are available.

[~@bison ~]$ ls -1 /cvmfs/
cvmfs-config.computecanada.ca
neurodesk.ardc.edu.au
oasis.opensciencegrid.org
singularity.opensciencegrid.org
soft.computecanada.ca

**To switch to CCEnv, use the following:

module load CCEnv
module load arch/avx512
module load StdEnv/2023

The above will ensure that the same software installed on national systems under the environment StdEnv/2023 is mounted on Grex.

Fo example, to load geant4/11.3.0 on national systems, one should use:

module load StdEnv/2023  gcc/12.3 geant4/11.3.0

To load the same module on Grex, one should use:

module load CCEnv
module load arch/avx512
module load StdEnv/2023
module load gcc/12.3 geant4/11.3.0

How to mount neurodesk.ardc.edu.au on Grex?

From the login node, run the commands:

[~@yak ~]$ ls -1 /cvmfs/
cvmfs-config.computecanada.ca
restricted.computecanada.ca
soft.computecanada.ca

The first timje, we run ls -1 /cvmfs/, the repo neurodesk.ardc.edu.au did not show up. The directories under cvmfs are auto-mounted. It means, they should be first accessed using ls for example:

[~@yak ~]$ ls /cvmfs/neurodesk.ardc.edu.au
[~@yak ~]$ ls -1 /cvmfs
cvmfs-config.computecanada.ca  
neurodesk.ardc.edu.au   
restricted.computecanada.ca  
soft.computecanada.ca

Now, the repo shows up under /cvmfs

To use the modules from this repo, one should source the modulefiles directory:

[~@yak ~]$ ls /cvmfs/neurodesk.ardc.edu.au/neurodesk-modules
[~@yak ~]$ module use /cvmfs/neurodesk.ardc.edu.au/neurodesk-modules

Now, let’s search for a module called FSL that is availbale on this repo:

[~@yak ~]$ module spider fsl
[~@yak ~]$ module spider functional_imaging/fsl/6.0.7.18
[~@yak ~]$ module load functional_imaging/fsl/6.0.7.18

This module relies on singularity:

[~@yak ~]$ module load singularity
[~@yak ~]$ module list
[~@yak ~]$ which fsl
/cvmfs/neurodesk.ardc.edu.au/containers/fsl_6.0.7.18_20250928/fsl
https://neurodesk.org/getting-started/neurocontainers/cvmfs/
There are other repositories available under cvmfs but they need to be listed first to be able to see them. They are auto-mounted.

Quick test using LAMMPS#

Here, we show how to run a quick test of lammps on the login nodes in different modes: serial, OpenMP, MPI and Hybrid.

Here are the instructions to run a quick test using LAMMPS on Grex:

Requireements:

  • access to login or compute node via salloc.
  • access to a module (lammps in this case)
  • Input file: lammps-input.in
Example of input file used to run LAMMPS.
# 3d Lennard-Jones melt
  
units           lj
atom_style      atomic

lattice         fcc 0.8442
region          box block 0 10 0 10 0 10
create_box      1 box
create_atoms    1 box
mass            1 1.0

velocity        all create 3.0 87287

pair_style      lj/cut 2.5
pair_coeff      1 1 1.0 1.0 2.5

neighbor        0.3 bin
neigh_modify    every 20 delay 0 check no

fix             1 all nve

thermo          50

run             10000

#write_data     config.end_melt

# End of the Input file.

First, connect to a cluster {Grex in this case}

Search for lammps modules:

  • Module “lammps/2024-08-29p1”.
  • Use module spider to see how to load this module.
module purge
module spider lammps/2024-08-29p1

Then load the module “lammps/2024-08-29p1

module load arch/avx512  gcc/13.2.0  openmpi/4.1.6 lammps/2024-08-29p1

Make sure that the module is loaded and available in your environment by running

  • The binary for this program is called lmp
  • See if the binary is in your path by running the command:
module list
which lmp

Now try to run the test interactively by invoking the command:

lmp -in lammps-input.in

Before running any script, use cat command to see the content of the script

cat <your script>

Run the following commands from your terminal

sh ./grex-runlmp-1cpu-serial.sh
sh ./grex-runlmp-2cpu-openmp.sh
sh ./grex-runlmp-2cpu-mpi.sh
sh ./grex-runlmp-4cpu-openmp.sh
sh ./grex-runlmp-4cpu-mpi.sh
sh ./grex-runlmp-8cpu-2mpi-4openmp.sh
sh ./grex-runlmp-8cpu-4mpi-2openmp.sh

The above will run lammps using:

  • 1 CPU ==> Serial program
  • 2 CPUs ==> OpenMP program
  • 2 CPUs ==> MPI program
  • 4 CPUs ==> OpenMP program
  • 4 CPUs ==> MPI program
  • 8 CPUs ==> Hybrid mode (2 MPI + 4 OpenMP)
  • 8 CPUs ==> Hybrid mode (4 MPI + 2 OpenMP)

The above will generate the following output:

  • output_lammps-serial.txt
  • output_lammps-openmp-2cpus.txt
  • output_lammps-mpi-2cpus.txt
  • output_lammps-openmp-4cpus.txt
  • output_lammps-mpi-4cpus.txt
  • output_lammps-8cpu-2mpi-4openmp.txt
  • output_lammps-8cpu-4mpi-2openmp.txt
Note that login nodes are not meant for intensive tests. For any memory or CPU intensive test, please user interactive jobs via salloc.