From Beocat
Jump to: navigation, search
(Add OpenMP and fix some formatting)
(33 intermediate revisions by 6 users not shown)
Line 1: Line 1:
== Drinking from the Firehose ==
== Drinking from the Firehose ==
For a complete list of all installed software, see [[NodePackageList]]
For a complete list of all installed modules, see [[ModuleList]]


== Toolchains ==
A toolchain is a set of compilers, libraries and applications that are needed to build software. Some software functions better when using specific toolchains.
We provide a good number of toolchains and versions of toolchains make sure your applications will compile and/or run correctly.
These toolchains include (you can run 'module keyword keychain compiler'):
; GCC:    The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Java, and Ada, as well as libraries for these languages (libstdc++, libgcj,...).
; GCCcore:    The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Java, and Ada, as well as libraries for these languages (libstdc++, libgcj,...).
; foss:    GNU Compiler Collection (GCC) based compiler toolchain, including OpenMPI for MPI support, OpenBLAS (BLAS and LAPACK support), FFTW and ScaLAPACK.
; gcccuda:    GNU Compiler Collection (GCC) based compiler toolchain, along with CUDA toolkit.
; gmvapich2:    GNU Compiler Collection (GCC) based compiler toolchain, including MVAPICH2 for MPI support.
; gompi:    GNU Compiler Collection (GCC) based compiler toolchain, including OpenMPI for MPI support.
; gompic:    GNU Compiler Collection (GCC) based compiler toolchain along with CUDA toolkit, including OpenMPI for MPI support with CUDA features enabled.
; goolfc:    GCC based compiler toolchain __with CUDA support__, and including OpenMPI for MPI support, OpenBLAS (BLAS and LAPACK support), FFTW and ScaLAPACK.
; icc:    C and C++ compiler from Intel
; iccifort:    Intel Cluster Toolkit Compiler Edition provides Intel C,C++ and fortran compilers, Intel MPI and Intel MKL
; ifort:    Fortran compiler from Intel
; iomkl:    Intel Cluster Toolchain Compiler Edition provides Intel C/C++ and Fortran compilers, Intel MKL & OpenMPI.
; iompi:    Intel C/C++ and Fortran compilers, alongside Open MPI.
You can run 'module spider $toolchain' to see the versions we have:
$ module spider iomkl
* iomkl/2017a
* iomkl/2017b
* iomkl/2017beocatb
If you load one of those (module load iomkl/2017b), you can see the other modules and versions of software that it loaded with the 'module list':
$ module list
Currently Loaded Modules:
  1) icc/2017.4.196-GCC-6.4.0-2.28
  2) binutils/2.28-GCCcore-6.4.0
  3) ifort/2017.4.196-GCC-6.4.0-2.28
  4) iccifort/2017.4.196-GCC-6.4.0-2.28
  5) GCCcore/6.4.0
  6) numactl/2.0.11-GCCcore-6.4.0
  7) hwloc/1.11.7-GCCcore-6.4.0
  8) OpenMPI/2.1.1-iccifort-2017.4.196-GCC-6.4.0-2.28
  9) iompi/2017b
  10) imkl/2017.3.196-iompi-2017b
  11) iomkl/2017b
As you can see, toolchains can depend on each other. For instance, the iomkl toolchain, depends on iompi, which depends on iccifort, which depend on icc and ifort, which depend on GCCcore which depend on GCC. Hence it is very important that the correct versions of all related software are loaded.
With software we provide, the toolchain used to compile is always specified in the "version" of the software that you want to load.
== Most Commonly Used Software ==
== Most Commonly Used Software ==
=== [http://www.open-mpi.org/ OpenMPI] ===
=== [http://www.open-mpi.org/ OpenMPI] ===
Version 1.4.3
We provide lots of versions, you are most likely better off directly loading a toolchain or application to make sure you get the right version, but you can see the versions we have with 'module spider OpenMPI':
 
=== [http://openmp.org/wp/ OpenMP] ===
OpenMP isn't really a software package itself. It is a set of directives for C, C++, and Fortran which greatly simplifies parallelizing applications on a single node. There is a good tutorial for OpenMP at [https://computing.llnl.gov/tutorials/openMP/ https://computing.llnl.gov/tutorials/openMP/]


=== [http://www.scilab.org Scilab] ===
* OpenMPI/2.0.2-GCC-6.3.0-2.27
Version 5.4.0
* OpenMPI/2.0.2-iccifort-2017.1.132-GCC-6.3.0-2.27
* OpenMPI/2.1.1-GCC-6.4.0-2.28
* OpenMPI/2.1.1-GCC-7.2.0-2.29
* OpenMPI/2.1.1-gcccuda-2017b
* OpenMPI/2.1.1-iccifort-2017.4.196-GCC-6.4.0-2.28
* OpenMPI/2.1.1-iccifort-2018.0.128-GCC-7.2.0-2.29


=== [http://www.r-project.org/ R] ===
=== [http://www.r-project.org/ R] ===
Version 3.0.3
We currently provide (module -r spider '^R$'):
* R/3.4.0-foss-2017beocatb-X11-20170314


==== Modules ====
==== Packages ====
We provide a small number of R modules installed by default, these are generally modules that are needed by more than one person.
We provide a small number of R modules installed by default, these are generally modules that are needed by more than one person.


==== Installing your own modules ====
==== Installing your own R Packages ====
To install your own module, login to Beocat and start R interactively
To install your own module, login to Beocat and start R interactively
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
module load R
R
R
</syntaxhighlight>
</syntaxhighlight>
Line 35: Line 83:
==== Running R Jobs ====
==== Running R Jobs ====


You cannot submit an R script directly. 'qsub myscript.R' will result in an error. Instead, you need to make a bash script that will call R appropriately. Here is a minimal example. We'll save this as submit-R.qsub
You cannot submit an R script directly. '<tt>sbatch myscript.R</tt>' will result in an error. Instead, you need to make a bash [[AdvancedSGE#Running_from_a_qsub_Submit_Script|script]] that will call R appropriately. Here is a minimal example. We'll save this as submit-R.sbatch


<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
#!/bin/bash
#!/bin/bash
# First, lets tell the qsub command which resources we need
#SBATCH --mem-per-cpu=1G
# lets start with memory (in this case I ask for 1 gigabyte).
# Now we tell qsub how long we expect our work to take: 15 minutes (D-H:MM:SS)
# For help on these, see [[SGEBasics]]
#SBATCH --time=0-0:15:00
 
#$ -l mem=1G
# Now lets do some actual work. This starts R and loads the file myscript.R
# Now we tell qsub how long we expect our work to take: 15 minutes (H:MM:SS)
module load R
R --no-save -q < myscript.R
#$ -l h_rt=0:15:00
# Lets output a little useful information This will put something like "Starting the job at: Thu Jan 26 10:43:26 CST 2012" in your output file
echo -n "Starting the job at: "
date
# Now lets do some actual work. A lot of our users use R, so we'll go over that
# This starts R and loads the file myscript.R
R --no-save -q < myscript.R
# like before, this is just useful information
echo -n "Ending the job at: "
date
</syntaxhighlight>
</syntaxhighlight>


Now, to submit your R job, you would type
Now, to submit your R job, you would type
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
qsub submit-R.qsub
sbatch submit-R.sbatch
</syntaxhighlight>
</syntaxhighlight>
=== [http://www.java.com/ Java] ===
=== [http://www.java.com/ Java] ===
Versions 1.6 and 1.7
We currently provide (module spider Java):
 
* Java/1.8.0_131
We support 4 versions of the Java VM on Beocat. [[wikipedia:IcedTea|IcedTea]] 6 and 7 (based on [[wikipedia:OpenJDK|OpenJDK]]), Sun JDK 1.6 (Java 6), and Oracle JDK 1.7 (Java 7).
* Java/1.8.0_144
 
We allow each user to select his or her Java version individually. If you do not select one, we default to Sun JDK 1.7.


==== Selecting your Java version ====
First, lets list the available versions. This can be done with the command <code>eselect java-vm list</code>
<pre>
% eselect java-vm list
Available Java Virtual Machines:
  [1]  icedtea-bin-6
  [2]  icedtea-bin-7
  [3]  oracle-jdk-bin-1.7  system-vm
  [4]  sun-jdk-1.6
</pre>
If you'll note,  oracle-jdk-bin-1.7 (marked "system-vm") is the default for all users. If you have a custom version set, it will be marked with "user-vm". Now if you wanted to use icedtea-6, you could run the following:
<syntaxhighlight lang="bash">
eselect java-vm set user 1
</syntaxhighlight>
Now, we see the difference when running the above command
<pre>
% eselect java-vm list
Available Java Virtual Machines:
  [1]  icedtea-bin-6  user-vm
  [2]  icedtea-bin-7
  [3]  oracle-jdk-bin-1.7  system-vm
  [4]  sun-jdk-1.6
</pre>
To verify you are seeing the correct java, you can run <code>java -version</code>
<pre>
% java -version
java version "1.6.0_27"
OpenJDK Runtime Environment (IcedTea6 1.12.7) (Gentoo build 1.6.0_27-b27)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)
</pre>
=== [http://www.python.org/about/ Python] ===
=== [http://www.python.org/about/ Python] ===
We currently provide (module spider Python)
* Python/2.7.13-foss-2017beocatb
* Python/2.7.13-GCCcore-7.2.0-bare
* Python/2.7.13-iomkl-2017a
* Python/2.7.13-iomkl-2017beocatb
* Python/3.6.3-foss-2017b
* Python/3.6.3-foss-2017beocatb
* Python/3.6.3-iomkl-2017beocatb


We have several versions of Python available:
If you need modules that we do not have installed, you should use [https://virtualenv.pypa.io/en/stable/userguide/ virtualenv] to setup a virtual python environment in your home directory. This will let you install python modules as you please.
* [http://docs.python.org/2.7/ CPython 2.7]
* [http://docs.python.org/3.2/ CPython 3.2]
* [http://pypy.org/ PyPy] versions 1.9 (Python 2.7.2) and 2.0.2 (Python 2.7.3)
 
For the uninitiated PyPy provides [[wikipedia:Just-in-time_compilation|just-in-time compilation]] for python code. While it doesn't support all modules, code which does run under PyPy can see a significant performance increase.
 
If you just need python and its default modules, you can use python2 python3 pypy-c1.9 or pypy-c2.0 as you would any other application.
 
If, however, you need modules that we do not have installed, you should use [http://www.doughellmann.com/projects/virtualenvwrapper/ virtualenvwrapper] to setup a virtual python environment in your home directory. This will let you install python modules as you please.


==== Setting up your virtual environment ====
==== Setting up your virtual environment ====
* [[LinuxBasics#Shells|Change your shell]] to bash
* Make sure ~/.bash_profile exists
<syntaxhighlight lang="bash">
if [ ! -f ~/.bash_profile ]; then cp /etc/skel/.bash_profile ~/.bash_profile; fi
</syntaxhighlight>
* Add a line like <code>source /usr/bin/virtualenvwrapper.sh</code> to your .bash_profile.
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
echo "source /usr/bin/virtualenvwrapper.sh" >> ~/.bash_profile
# Load Python
module load Python/3.6.3-iomkl-2017beocatb
</syntaxhighlight>
</syntaxhighlight>
* Show your existing environments
(After running this command Python is loaded.  After you logoff and then logon again Python will not be loaded so you must rerun this command every time you logon.)
* Create a location for your virtual environments (optional, but helps keep things organized)
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
workon
mkdir ~/virtualenvs
cd ~/virtualenvs
</syntaxhighlight>
</syntaxhighlight>
* Create a virtual environment. Here I will create a default virtual environment called 'test', a python2 virtual environment called 'testp2', a python3 virtual environment called 'testp3', and a pypy environment called testpypy. Note that <code>mkvirtualenv --help</code> has many more useful options.
* Create a virtual environment. Here I will create a default virtual environment called 'test'. Note that <code>virtualenv --help</code> has many more useful options.
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
mkvirtualenv -p $(which python2) testp2
virtualenv test
mkvirtualenv -p $(which python3) testp3
mkvirtualenv -p $(which pypy-c2.0) testpypy
</syntaxhighlight>
</syntaxhighlight>
* Lets look at our virtual environments
* Lets look at our virtual environments
<pre>
<pre>
%workon
% ls ~/virtualenvs
testp2
test
testp3
testpypy
</pre>
</pre>
* Activate one of these
* Activate one of these
<pre>
<pre>
%workon testp2
%source ~/virtualenvs/test/bin/activate
</pre>
</pre>
(After running this command your virtual environment is activated.  After you logoff and then logon again your virtual environment will not be loaded so you must rerun this command every time you logon.)
* You can now install the python modules you want. This can be done using <tt>pip</tt>.
* You can now install the python modules you want. This can be done using <tt>pip</tt>.
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
pip install numpy biopython
pip install numpy biopython
</syntaxhighlight>
</syntaxhighlight>
==== Using your virtual environment within a job ====
==== Using your virtual environment within a job ====
Here is a simple job script using the virtual environment testp2
Here is a simple job script using the virtual environment testp2
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
#!/bin/bash
#!/bin/bash
source /usr/bin/virtualenvwrapper.sh
module load Python/3.6.3-iomkl-2017beocatb
workon testp2
source ~/virtualenvs/test/bin/activate
~/path/to/your/python/script.py
python ~/path/to/your/python/script.py
</syntaxhighlight>
</syntaxhighlight>
==== A note on [http://www.numpy.org/ NumPy] ====
NumPy is a commonly-used Python package.


Make sure the following is executed before running <code>pip install numpy</code>
=== [http://www.perl.org/ Perl] ===
<syntaxhighlight lang="bash">
The system-wide version of perl is tracking the stable releases of perl. Unfortunately there are some features that we do not include in the system distribution of perl, namely threads.
cp /opt/beocat/numpy/.numpy-site.cfg ~/.numpy-site.cfg
 
</syntaxhighlight>
If you need a newer version (or threads), just load one we provide in our modules (module spider Perl):
==== A note on [http://mpi4py.scipy.org/docs/usrman/index.html mpi4py] ====
* Perl/5.26.0-foss-2017beocatb
If you are wanting to use mpi with your python script and are using a virtual environment, you will need to send the correct environment variables to all of the mpi processes to make the virtual environment work.
* Perl/5.26.0-iompi-2017beocatb
 
==== Submitting a job with Perl ====
Much like R (above), you cannot simply '<tt>sbatch myProgram.pl</tt>', but you must create a [[AdvancedSlurm#Running_from_a_sbatch_Submit_Script|submit script]] which will call perl. Here is an example:
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
#!/bin/bash
#!/bin/bash
# sample mpi4py submit script
#SBATCH --mem-per-cpu=1G
source /usr/bin/virtualenvwrapper.sh
# Now we tell qsub how long we expect our work to take: 15 minutes (H:MM:SS)
workon testp2
#SBATCH --time=0-0:15:00
# figure out the location of the python interpreter in the virtual environment
# Now lets do some actual work.  
PYTHON_BINARY=$(which python)
module load Perl
# mpirun the python interpreter within the virtual environment
perl /path/to/myProgram.pl
# if you don't use the interpreter within the virtual environment, i.e. just using 'python'
# the system python interpreter (without access to your other modules) will be used.
mpirun ${PYTHON_BINARY} ~/path/to/your/mpi-enabled/python/script.py
</syntaxhighlight>
</syntaxhighlight>


=== [http://www.perl.org/ Perl] ===
=== Octave for MatLab codes ===
The system-wide version of perl is tracking the stable releases of perl. Unfortunately there are some features that we do not include in the system distribution of perl, namely threads.
 
module load Octave/4.2.1-foss-2017beocatb-enable64
 
The 64-bit version of Octave can be loaded using the command above.  Octave can then be used
to work with MatLab codes on the head node and to submit jobs to the compute nodes through the
sbatch scheduler.  Octave is made to run MatLab code, but it does have limitations and does not support
everything that MatLab itself does.
 
#!/bin/bash -l
#SBATCH --job-name=octave
#SBATCH --output=octave.o%j
#SBATCH --time=1:00:00
#SBATCH --mem=4G
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
module purge
module load Octave/4.2.1-foss-2017beocatb-enable64
octave < matlab_code.m
 
=== MatLab compiler ===
 
Beocat also has a <B>single-user license</B> for the MatLab compiler and the most common toolboxes
including the Parallel Computing Toolbox, Optimization Toolbox, Statistics and Machine Learning Toolbox,
Image Processing Toolbox, Curve Fitting Toolbox, Neural Network Toolbox, Sumbolic Math Toolbox,
Global Optimization Toolbox, and the Bioinformatics Toolbox.
 
Since we only have a <B>single-user license</B>, this means that you will be expected to develop your MatLab code
with Octave or elsewhere on a laptop or departmental server. Once you're ready to do large runs, then you
move your code to Beocat, compile the MatLab code into an executable, and you can submit as many jobs as
you want to the scheduler.  To use the MatLab compiler, you need to load the MATLAB module to compile code and
load the mcr module to run the resulting MatLab executable.
 
module load MATLAB<BR>
mcc -m matlab_main_code.m -o matlab_executable_name
 
#!/bin/bash -l
#SBATCH --job-name=matlab
#SBATCH --output=matlab.o%j
#SBATCH --time=1:00:00
#SBATCH --mem=4G
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
module purge
module load mcr
./matlab_executable_name
 
Again, we only have a <B>single-user license</B> for MatLab so the model is to develop and debug your MatLab code
elsewhere or using Octave on Beocat, then you can compile the MatLab code into an executable and run it without
limits on Beocat.


==== Getting Perl with threads ====
For more info on the mcc compiler see:  https://www.mathworks.com/help/compiler/mcc.html
* Setup perlbrew
** [[LinuxBasics#Shells|Change your shell]] to bash
** Install perlbrew
<syntaxhighlight lang="bash">
curl -L http://install.perlbrew.pl | bash
</syntaxhighlight>
** Make sure that ~/.bash_profile exists
<syntaxhighlight lang="bash">
if [ ! -f ~/.bash_profile ]; then cp /etc/skel/.bash_profile ~/.bash_profile; fi
</syntaxhighlight>
** Add <code>source ~/perl5/perlbrew/etc/bashrc</code> to ~/.bash_profile
<syntaxhighlight lang="bash">
echo "source ~/perl5/perlbrew/etc/bashrc" >> ~/.bash_profile
</syntaxhighlight>
** Then source your bash profile
<syntaxhighlight lang="bash">
source ~/.bash_profile
</syntaxhighlight>
* Now, install perl with threads within perlbrew
** Find the current Perl version.
<pre>
% perl -version


This is perl 5, version 16, subversion 3 (v5.16.3) built for x86_64-linux
(with 22 registered patches, see perl -V for more detail)
(...several more lines deleted)
</pre>
** In this case the version is 5.16.3, so we run
<syntaxhighlight lang="bash">
perlbrew install -f -n -D usethreads perl-5.16.3
</syntaxhighlight>
** To temporarily use the new version of perl in the current shell, we now run
<syntaxhighlight lang="bash">
perlbrew use perl-5.16.3
</syntaxhighlight>
** To switch versions of perl for every new login or job, run
<syntaxhighlight lang="bash">
perlbrew switch perl-5.16.3
</syntaxhighlight>
** You can reverse this switch with
<syntaxhighlight lang="bash">
perlbrew switch-off
</syntaxhighlight>
== Installing my own software ==
== Installing my own software ==
Installing and maintaining software for the many different users of Beocat would be very difficult, if not impossible. For this reason, we don't generally install user-run software on our cluster. Instead, we ask that you install it into your home directories.
Installing and maintaining software for the many different users of Beocat would be very difficult, if not impossible. For this reason, we don't generally install user-run software on our cluster. Instead, we ask that you install it into your home directories.

Revision as of 17:37, 27 February 2018

Drinking from the Firehose

For a complete list of all installed modules, see ModuleList

Toolchains

A toolchain is a set of compilers, libraries and applications that are needed to build software. Some software functions better when using specific toolchains.

We provide a good number of toolchains and versions of toolchains make sure your applications will compile and/or run correctly.

These toolchains include (you can run 'module keyword keychain compiler'):

GCC
The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Java, and Ada, as well as libraries for these languages (libstdc++, libgcj,...).
GCCcore
The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Java, and Ada, as well as libraries for these languages (libstdc++, libgcj,...).
foss
GNU Compiler Collection (GCC) based compiler toolchain, including OpenMPI for MPI support, OpenBLAS (BLAS and LAPACK support), FFTW and ScaLAPACK.
gcccuda
GNU Compiler Collection (GCC) based compiler toolchain, along with CUDA toolkit.
gmvapich2
GNU Compiler Collection (GCC) based compiler toolchain, including MVAPICH2 for MPI support.
gompi
GNU Compiler Collection (GCC) based compiler toolchain, including OpenMPI for MPI support.
gompic
GNU Compiler Collection (GCC) based compiler toolchain along with CUDA toolkit, including OpenMPI for MPI support with CUDA features enabled.
goolfc
GCC based compiler toolchain __with CUDA support__, and including OpenMPI for MPI support, OpenBLAS (BLAS and LAPACK support), FFTW and ScaLAPACK.
icc
C and C++ compiler from Intel
iccifort
Intel Cluster Toolkit Compiler Edition provides Intel C,C++ and fortran compilers, Intel MPI and Intel MKL
ifort
Fortran compiler from Intel
iomkl
Intel Cluster Toolchain Compiler Edition provides Intel C/C++ and Fortran compilers, Intel MKL & OpenMPI.
iompi
Intel C/C++ and Fortran compilers, alongside Open MPI.

You can run 'module spider $toolchain' to see the versions we have:

$ module spider iomkl
  • iomkl/2017a
  • iomkl/2017b
  • iomkl/2017beocatb

If you load one of those (module load iomkl/2017b), you can see the other modules and versions of software that it loaded with the 'module list':

$ module list
Currently Loaded Modules:
  1) icc/2017.4.196-GCC-6.4.0-2.28
  2) binutils/2.28-GCCcore-6.4.0
  3) ifort/2017.4.196-GCC-6.4.0-2.28
  4) iccifort/2017.4.196-GCC-6.4.0-2.28
  5) GCCcore/6.4.0
  6) numactl/2.0.11-GCCcore-6.4.0
  7) hwloc/1.11.7-GCCcore-6.4.0
  8) OpenMPI/2.1.1-iccifort-2017.4.196-GCC-6.4.0-2.28
  9) iompi/2017b
 10) imkl/2017.3.196-iompi-2017b
 11) iomkl/2017b

As you can see, toolchains can depend on each other. For instance, the iomkl toolchain, depends on iompi, which depends on iccifort, which depend on icc and ifort, which depend on GCCcore which depend on GCC. Hence it is very important that the correct versions of all related software are loaded.

With software we provide, the toolchain used to compile is always specified in the "version" of the software that you want to load.

Most Commonly Used Software

OpenMPI

We provide lots of versions, you are most likely better off directly loading a toolchain or application to make sure you get the right version, but you can see the versions we have with 'module spider OpenMPI':

  • OpenMPI/2.0.2-GCC-6.3.0-2.27
  • OpenMPI/2.0.2-iccifort-2017.1.132-GCC-6.3.0-2.27
  • OpenMPI/2.1.1-GCC-6.4.0-2.28
  • OpenMPI/2.1.1-GCC-7.2.0-2.29
  • OpenMPI/2.1.1-gcccuda-2017b
  • OpenMPI/2.1.1-iccifort-2017.4.196-GCC-6.4.0-2.28
  • OpenMPI/2.1.1-iccifort-2018.0.128-GCC-7.2.0-2.29

R

We currently provide (module -r spider '^R$'):

  • R/3.4.0-foss-2017beocatb-X11-20170314

Packages

We provide a small number of R modules installed by default, these are generally modules that are needed by more than one person.

Installing your own R Packages

To install your own module, login to Beocat and start R interactively

module load R
R

Then install the package using

install.packages("PACKAGENAME")

Follow the prompts. Note that there is a CRAN mirror at KU - it will be listed as "USA (KS)".

After installing you can test before leaving interactive mode by issuing the command

library("PACKAGENAME")

Running R Jobs

You cannot submit an R script directly. 'sbatch myscript.R' will result in an error. Instead, you need to make a bash script that will call R appropriately. Here is a minimal example. We'll save this as submit-R.sbatch

#!/bin/bash
#SBATCH --mem-per-cpu=1G
# Now we tell qsub how long we expect our work to take: 15 minutes (D-H:MM:SS)
#SBATCH --time=0-0:15:00

# Now lets do some actual work. This starts R and loads the file myscript.R
module load R
R --no-save -q < myscript.R

Now, to submit your R job, you would type

sbatch submit-R.sbatch

Java

We currently provide (module spider Java):

  • Java/1.8.0_131
  • Java/1.8.0_144

Python

We currently provide (module spider Python)

  • Python/2.7.13-foss-2017beocatb
  • Python/2.7.13-GCCcore-7.2.0-bare
  • Python/2.7.13-iomkl-2017a
  • Python/2.7.13-iomkl-2017beocatb
  • Python/3.6.3-foss-2017b
  • Python/3.6.3-foss-2017beocatb
  • Python/3.6.3-iomkl-2017beocatb

If you need modules that we do not have installed, you should use virtualenv to setup a virtual python environment in your home directory. This will let you install python modules as you please.

Setting up your virtual environment

# Load Python
module load Python/3.6.3-iomkl-2017beocatb

(After running this command Python is loaded. After you logoff and then logon again Python will not be loaded so you must rerun this command every time you logon.)

  • Create a location for your virtual environments (optional, but helps keep things organized)
mkdir ~/virtualenvs
cd ~/virtualenvs
  • Create a virtual environment. Here I will create a default virtual environment called 'test'. Note that virtualenv --help has many more useful options.
virtualenv test
  • Lets look at our virtual environments
% ls ~/virtualenvs
test
  • Activate one of these
%source ~/virtualenvs/test/bin/activate

(After running this command your virtual environment is activated. After you logoff and then logon again your virtual environment will not be loaded so you must rerun this command every time you logon.)

  • You can now install the python modules you want. This can be done using pip.
pip install numpy biopython

Using your virtual environment within a job

Here is a simple job script using the virtual environment testp2

#!/bin/bash
module load Python/3.6.3-iomkl-2017beocatb
source ~/virtualenvs/test/bin/activate
python ~/path/to/your/python/script.py

Perl

The system-wide version of perl is tracking the stable releases of perl. Unfortunately there are some features that we do not include in the system distribution of perl, namely threads.

If you need a newer version (or threads), just load one we provide in our modules (module spider Perl):

  • Perl/5.26.0-foss-2017beocatb
  • Perl/5.26.0-iompi-2017beocatb

Submitting a job with Perl

Much like R (above), you cannot simply 'sbatch myProgram.pl', but you must create a submit script which will call perl. Here is an example:

#!/bin/bash
#SBATCH --mem-per-cpu=1G
# Now we tell qsub how long we expect our work to take: 15 minutes (H:MM:SS)
#SBATCH --time=0-0:15:00
# Now lets do some actual work. 
module load Perl
perl /path/to/myProgram.pl

Octave for MatLab codes

module load Octave/4.2.1-foss-2017beocatb-enable64

The 64-bit version of Octave can be loaded using the command above. Octave can then be used to work with MatLab codes on the head node and to submit jobs to the compute nodes through the sbatch scheduler. Octave is made to run MatLab code, but it does have limitations and does not support everything that MatLab itself does.

#!/bin/bash -l
#SBATCH --job-name=octave
#SBATCH --output=octave.o%j
#SBATCH --time=1:00:00
#SBATCH --mem=4G
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1

module purge
module load Octave/4.2.1-foss-2017beocatb-enable64

octave < matlab_code.m

MatLab compiler

Beocat also has a single-user license for the MatLab compiler and the most common toolboxes including the Parallel Computing Toolbox, Optimization Toolbox, Statistics and Machine Learning Toolbox, Image Processing Toolbox, Curve Fitting Toolbox, Neural Network Toolbox, Sumbolic Math Toolbox, Global Optimization Toolbox, and the Bioinformatics Toolbox.

Since we only have a single-user license, this means that you will be expected to develop your MatLab code with Octave or elsewhere on a laptop or departmental server. Once you're ready to do large runs, then you move your code to Beocat, compile the MatLab code into an executable, and you can submit as many jobs as you want to the scheduler. To use the MatLab compiler, you need to load the MATLAB module to compile code and load the mcr module to run the resulting MatLab executable.

module load MATLAB
mcc -m matlab_main_code.m -o matlab_executable_name

#!/bin/bash -l
#SBATCH --job-name=matlab
#SBATCH --output=matlab.o%j
#SBATCH --time=1:00:00
#SBATCH --mem=4G
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1

module purge
module load mcr

./matlab_executable_name

Again, we only have a single-user license for MatLab so the model is to develop and debug your MatLab code elsewhere or using Octave on Beocat, then you can compile the MatLab code into an executable and run it without limits on Beocat.

For more info on the mcc compiler see: https://www.mathworks.com/help/compiler/mcc.html

Installing my own software

Installing and maintaining software for the many different users of Beocat would be very difficult, if not impossible. For this reason, we don't generally install user-run software on our cluster. Instead, we ask that you install it into your home directories.

In many cases, the software vendor or support site will incorrectly assume that you are installing the software system-wide or that you need 'sudo' access.

As a quick example of installing software in your home directory, we have a sample video on our Training Videos page. If you're still having problems or questions, please contact support as mentioned on our Main Page.