Difference between revisions of "Installing software"

From imdb
Jump to: navigation, search
Line 107: Line 107:
'''Using pyenv'''
'''Using pyenv'''
cd into the folder where you want to work in.  
cd into the folder where you want to work in.  
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">

Revision as of 14:12, 7 October 2015


Normally, new software should be installed through the module system. In some circumstances it can be OK to install through the package manager, but it should be generally avoided if it can be helped.

Circumstances that allows one to install software through the package manager includes, but is not restricted to, software that is:

  • Very hard to install
  • Large dependency trees
  • Generally useful to for users
  • Critical to security of the system

Any software that is installed through the package manager must be approved by a second user.

The module system

We are using the Environment Modules project to handle the software environment on the system. It provides the module command. With the module command a user can load and unload different packages and even use different versions of the same software in different settings depending on the requirements the user has at that time.

More information about the module command can be found in the module manpage.

It is also possible to install new software to the module system if required.

Installing software to the module system

To install new software to the module system it is required to write a modulefile. A detailed description of how to do that can be found in the modulefile manpage, but the general way of doing it is as follows:

Download the software

One would download the package, f.ex. python-3.5, from the provider to your $HOME-folder or a folder in /scratch/private (preferably /scratch/private/$USER) and, if packaged in an archive, extracted there.

mkdir -p /scratch/private/$USER
cd /scratch/private/$USER
wget https://www.python.org/ftp/python/3.5.0/Python-3.5.0.tar.xz
tar xf Python-3.5.0.tar.xz
cd Python-3.5.0

Configure and compile

As a convention, we use /data_bck/sw/$APPNAME/$RELEASE-$COMPILER as the default destination folder for installation. All folders should, as a convention, be lower case. How the sofware is configured and compiled depends on what build system the software uses, but in the case of python it would be to run the following in the Python folder:

./configure --prefix=/data_bck/sw/python/3.5.0-gnu
make test

To run make test is strictly not necessary, but it is advisable to run any tests the package provides to check if there are some problems with the package, especially since we are compiling most software our self and setting the environment. There is a possibility that a modulefile is misconfigured and running tests could catch any such errors.

Installing the software

Installing the software and making it ready for use requires first to move the compiled binaries or other files into the destination folder, for python and most other projects using make, it is as simple as running the following command in the python folder:

make install

Creating a module file

Information on creating module files can be found in the modulefile manpages. To be able to use software package after installation we need to set some system variables such as PATH, if the package contains any programs, or LD_LIBRARY_PATH if the package contains any shared libraries that you are interested in using.

The module file for python would look like this:


proc ModulesHelp { } {
    puts stderr "The modulefile defines the system paths and"
    puts stderr "environment variables needed to use Python 3.5.0"

set appname python
set release 3.5.0

set LCcompiler gnu

set app_path            /data_bck/sw/$appname/$release-${LCcompiler}

prepend-path PATH               $app_path/bin
prepend-path MANPATH            $app_path/share/man
prepend-path LD_LIBRARY_PATH    $app_path/lib
prepend-path PKG_CONFIG_PATH    $app_path/lib/pkgconfig

system logger -t module -p info USER=\$USER\, APP=$appname/$release

The language that is used inside the module file is tcl. It is a general purpose programming language. The module file needs to be saved to /data_bck/modulefile/$APPNAME/$RELEASE, where $RELEASE is the name of the module file. It will then show up in the module system if you run module avail.

Loading packages on login

It is possible to load specific packages on login by entering them in $HOME/.modulerc without trying to load modules when logging into systems that do not support modules.

User specific module files

If you want to create private module files you can add $HOME/.modulefiles to the environment variable MODULEPATH in your .bashrc (or similar) file.


To install package through yum you call: sudo yum install $PACKAGE

Any installation of packages through yum must be approved by at least 1 other user. Any package that is installed this way will be removed on a reinstall of the system and will need to be installed again. If it is desirable that the package should be immediately available after a reinstall, we need to tell the IT department to add the package in the puppet configuration script for our server.

If we want it to be impossible to install certain packages through yum we can also contact the IT department to add a configuration to puppet that will remove certain packages.

Language conventions


Using pip with pyvenv isolates modules for your project. Create a virtual environment with pyvenv and use pip as you wish.

Using pyenv

cd into the folder where you want to work in.

[est040@nelson ~]$ cd kategorier

Create a virtual environment with

pyvenv nyhetskategorier

Enter your environment with

source nyhetskategorier/bin/activate.csh 
source nyhetskategorier/bin/activate

Notice that the environment name now is prepended to your command promt:

[nyhetskategorier] [est040@nelson ~/kategorier]$
[nyhetskategorier] [est040@nelson ~/kategorier]$ pip install <stuff>
[nyhetskategorier] [est040@nelson ~/kategorier]$ pip --help

Use deactivate to exit your environment


Other languages