Installing software

From imdb
Jump to: navigation, search

Introduction

Normally, new software should be installed through the system package handler: yum. In some circumstances one might have to install by hand by using the module system. Some of those circumstances are

  • Conflicts with already installed software
  • Introduces non standard flags to build
  • Is not in the package system

yum

To install package through yum you call: sudo yum install $PACKAGE

If the package is not in the repository one can contact the IT department so that they can add it.

The module system

We are using the Environment Modules project to handle the software environment on the system. It provides the module command. With the module command a user can load and unload different packages and even use different versions of the same software in different settings depending on the requirements the user has at that time.

More information about the module command can be found in the module manpage.

It is also possible to install new software to the module system if required.

Installing software to the module system

To install new software to the module system it is required to write a modulefile. A detailed description of how to do that can be found in the modulefile manpage, but the general way of doing it is as follows:

Download the software

One would download the package, f.ex. python-3.5, from the provider to your $HOME-folder or a folder in /scratch/private (preferably /scratch/private/$USER) and, if packaged in an archive, extracted there.

mkdir -p /scratch/private/$USER
cd /scratch/private/$USER
wget https://www.python.org/ftp/python/3.5.0/Python-3.5.0.tar.xz
tar xf Python-3.5.0.tar.xz
cd Python-3.5.0

Configure and compile

As a convention, we use /data_bck/sw/$APPNAME/$RELEASE-$COMPILER as the default destination folder for installation. All folders should, as a convention, be lower case. How the sofware is configured and compiled depends on what build system the software uses, but in the case of python it would be to run the following in the Python folder:

./configure --prefix=/data_bck/sw/python/3.5.0-gnu
make
make test

To run make test is strictly not necessary, but it is advisable to run any tests the package provides to check if there are some problems with the package, especially since we are compiling most software our self and setting the environment. There is a possibility that a modulefile is misconfigured and running tests could catch any such errors.

Installing the software

Installing the software and making it ready for use requires first to move the compiled binaries or other files into the destination folder, for python and most other projects using make, it is as simple as running the following command in the python folder:

make install

Creating a module file

Information on creating module files can be found in the modulefile manpages. To be able to use software package after installation we need to set some system variables such as PATH, if the package contains any programs, or LD_LIBRARY_PATH if the package contains any shared libraries that you are interested in using.

The module file for python would look like this:

#%Module

proc ModulesHelp { } {
    puts stderr "The modulefile defines the system paths and"
    puts stderr "environment variables needed to use Python 3.5.0"
}

set appname python
set release 3.5.0

set LCcompiler gnu

set app_path            /data_bck/sw/$appname/$release-${LCcompiler}

prepend-path PATH               $app_path/bin
prepend-path MANPATH            $app_path/share/man
prepend-path LD_LIBRARY_PATH    $app_path/lib
prepend-path PKG_CONFIG_PATH    $app_path/lib/pkgconfig

system logger -t module -p info USER=\$USER\, APP=$appname/$release

The language that is used inside the module file is tcl. It is a general purpose programming language. The module file needs to be saved to /data_bck/modulefile/$APPNAME/$RELEASE, where $RELEASE is the name of the module file. It will then show up in the module system if you run module avail.

Loading packages on login

It is possible to load specific packages on login by entering them in $HOME/.modulerc without trying to load modules when logging into systems that do not support modules.

User specific module files

If you want to create private module files you can add $HOME/.modulefiles to the environment variable MODULEPATH in your .bashrc (or similar) file.

Language conventions

Python

Using pip with pyvenv isolates modules for your project. Create a virtual environment with pyvenv and use pip as you wish.

Using pyvenv

cd into the folder where you want to work in.

[est040@nelson ~]$ cd kategorier

Create a virtual environment with

pyvenv nyhetskategorier

Enter your environment with

source nyhetskategorier/bin/activate.csh 
#or 
source nyhetskategorier/bin/activate

Notice that the environment name now is prepended to your command prompt:

[nyhetskategorier] [est040@nelson ~/kategorier]$
[nyhetskategorier] [est040@nelson ~/kategorier]$ pip install <stuff>
[nyhetskategorier] [est040@nelson ~/kategorier]$ pip --help

Use deactivate to exit your environment

deactivate


R

To make reproducible and isolated experiments in R you can use packrat

Other languages