User:Est040

From imdb
Revision as of 13:38, 7 October 2015 by Est040 (talk | contribs)
Jump to: navigation, search

Hei.

Eirik dokumenterer første bruker av pyvenv

module load python # får python 3.5

cd ~
mkdir kategorier
cd kategorier

pyvenv nyhetskategorier
source nyhetskategorier/bin/activate.csh 
# NB prosjekt/bin/activate uten .sch funket ikke, formodelig fordi jeg får tildelt -tcsh og ikke -bash av neldon. (echo $0) - vet ikke hvorfor jeg får -tcsh.

pip install numpy

pip list
#numpy (1.10.0)
#pip (7.1.2)
#setuptools (18.2)

pip install sklearn

sklearn trenger scipy, og

>>> from sklearn import metrics

krasjer. Skulle ønske pip tok med dependencies...

 pip install scipy

..funker ikke.

pip install scipy
Collecting scipy
  Downloading scipy-0.16.0.tar.gz (18.1MB)
    100% |████████████████████████████████| 18.1MB 32kB/s 
Installing collected packages: scipy
  Running setup.py install for scipy
    Complete output from command /home/est040/kategorier/nyhetskategorier/bin/python3.5 -c "import setuptools, tokenize;__file__='/tmp/pip-build-ptqy9f_1/scipy/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-22ol6nmp-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/est040/kategorier/nyhetskategorier/include/site/python3.5/scipy:
    lapack_opt_info:
    openblas_lapack_info:
      libraries openblas not found in ['/home/est040/kategorier/nyhetskategorier/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/']
      NOT AVAILABLE
    
    lapack_mkl_info:
    mkl_info:
      libraries mkl,vml,guide not found in ['/home/est040/kategorier/nyhetskategorier/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/']
      NOT AVAILABLE
    
      NOT AVAILABLE
    
    atlas_3_10_threads_info:
    Setting PTATLAS=ATLAS
      libraries tatlas,tatlas not found in /home/est040/kategorier/nyhetskategorier/lib
      libraries lapack_atlas not found in /home/est040/kategorier/nyhetskategorier/lib
      libraries tatlas,tatlas not found in /usr/local/lib64
      libraries lapack_atlas not found in /usr/local/lib64
      libraries tatlas,tatlas not found in /usr/local/lib
      libraries lapack_atlas not found in /usr/local/lib
      libraries tatlas,tatlas not found in /usr/lib64/sse2
      libraries lapack_atlas not found in /usr/lib64/sse2
      libraries tatlas,tatlas not found in /usr/lib64
      libraries lapack_atlas not found in /usr/lib64
      libraries tatlas,tatlas not found in /usr/lib/sse2
      libraries lapack_atlas not found in /usr/lib/sse2
      libraries tatlas,tatlas not found in /usr/lib
      libraries lapack_atlas not found in /usr/lib
      libraries tatlas,tatlas not found in /usr/lib/sse2
      libraries lapack_atlas not found in /usr/lib/sse2
      libraries tatlas,tatlas not found in /usr/lib/
      libraries lapack_atlas not found in /usr/lib/
    <class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
      NOT AVAILABLE
    
    atlas_3_10_info:
      libraries satlas,satlas not found in /home/est040/kategorier/nyhetskategorier/lib
      libraries lapack_atlas not found in /home/est040/kategorier/nyhetskategorier/lib
      libraries satlas,satlas not found in /usr/local/lib64
      libraries lapack_atlas not found in /usr/local/lib64
      libraries satlas,satlas not found in /usr/local/lib
      libraries lapack_atlas not found in /usr/local/lib
      libraries satlas,satlas not found in /usr/lib64/sse2
      libraries lapack_atlas not found in /usr/lib64/sse2
      libraries satlas,satlas not found in /usr/lib64
      libraries lapack_atlas not found in /usr/lib64
      libraries satlas,satlas not found in /usr/lib/sse2
      libraries lapack_atlas not found in /usr/lib/sse2
      libraries satlas,satlas not found in /usr/lib
      libraries lapack_atlas not found in /usr/lib
      libraries satlas,satlas not found in /usr/lib/sse2
      libraries lapack_atlas not found in /usr/lib/sse2
      libraries satlas,satlas not found in /usr/lib/
      libraries lapack_atlas not found in /usr/lib/
    <class 'numpy.distutils.system_info.atlas_3_10_info'>
      NOT AVAILABLE
    
    atlas_threads_info:
    Setting PTATLAS=ATLAS
      libraries ptf77blas,ptcblas,atlas not found in /home/est040/kategorier/nyhetskategorier/lib
      libraries lapack_atlas not found in /home/est040/kategorier/nyhetskategorier/lib
      libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib64
      libraries lapack_atlas not found in /usr/local/lib64
      libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
      libraries lapack_atlas not found in /usr/local/lib
      libraries ptf77blas,ptcblas,atlas not found in /usr/lib64/sse2
      libraries lapack_atlas not found in /usr/lib64/sse2
      libraries ptf77blas,ptcblas,atlas not found in /usr/lib64
      libraries lapack_atlas not found in /usr/lib64
      libraries ptf77blas,ptcblas,atlas not found in /usr/lib/sse2
      libraries lapack_atlas not found in /usr/lib/sse2
      libraries ptf77blas,ptcblas,atlas not found in /usr/lib
      libraries lapack_atlas not found in /usr/lib
      libraries ptf77blas,ptcblas,atlas not found in /usr/lib/sse2
      libraries lapack_atlas not found in /usr/lib/sse2
      libraries ptf77blas,ptcblas,atlas not found in /usr/lib/
      libraries lapack_atlas not found in /usr/lib/
    <class 'numpy.distutils.system_info.atlas_threads_info'>
      NOT AVAILABLE
    
    atlas_info:
      libraries f77blas,cblas,atlas not found in /home/est040/kategorier/nyhetskategorier/lib
      libraries lapack_atlas not found in /home/est040/kategorier/nyhetskategorier/lib
      libraries f77blas,cblas,atlas not found in /usr/local/lib64
      libraries lapack_atlas not found in /usr/local/lib64
      libraries f77blas,cblas,atlas not found in /usr/local/lib
      libraries lapack_atlas not found in /usr/local/lib
      libraries f77blas,cblas,atlas not found in /usr/lib64/sse2
      libraries lapack_atlas not found in /usr/lib64/sse2
      libraries f77blas,cblas,atlas not found in /usr/lib64
      libraries lapack_atlas not found in /usr/lib64
      libraries f77blas,cblas,atlas not found in /usr/lib/sse2
      libraries lapack_atlas not found in /usr/lib/sse2
      libraries f77blas,cblas,atlas not found in /usr/lib
      libraries lapack_atlas not found in /usr/lib
      libraries f77blas,cblas,atlas not found in /usr/lib/sse2
      libraries lapack_atlas not found in /usr/lib/sse2
      libraries f77blas,cblas,atlas not found in /usr/lib/
      libraries lapack_atlas not found in /usr/lib/
    <class 'numpy.distutils.system_info.atlas_info'>
      NOT AVAILABLE
    
    lapack_info:
      libraries lapack not found in ['/home/est040/kategorier/nyhetskategorier/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/']
      NOT AVAILABLE
    
    lapack_src_info:
      NOT AVAILABLE
    
      NOT AVAILABLE
    
    /home/est040/kategorier/nyhetskategorier/lib/python3.5/site-packages/numpy/distutils/system_info.py:1552: UserWarning:
        Atlas (http://math-atlas.sourceforge.net/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [atlas]) or by setting
        the ATLAS environment variable.
      warnings.warn(AtlasNotFoundError.__doc__)
    /home/est040/kategorier/nyhetskategorier/lib/python3.5/site-packages/numpy/distutils/system_info.py:1563: UserWarning:
        Lapack (http://www.netlib.org/lapack/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [lapack]) or by setting
        the LAPACK environment variable.
      warnings.warn(LapackNotFoundError.__doc__)
    /home/est040/kategorier/nyhetskategorier/lib/python3.5/site-packages/numpy/distutils/system_info.py:1566: UserWarning:
        Lapack (http://www.netlib.org/lapack/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [lapack_src]) or by setting
        the LAPACK_SRC environment variable.
      warnings.warn(LapackSrcNotFoundError.__doc__)
    Running from scipy source directory.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-ptqy9f_1/scipy/setup.py", line 253, in <module>
        setup_package()
      File "/tmp/pip-build-ptqy9f_1/scipy/setup.py", line 250, in setup_package
        setup(**metadata)
      File "/home/est040/kategorier/nyhetskategorier/lib/python3.5/site-packages/numpy/distutils/core.py", line 135, in setup
        config = configuration()
      File "/tmp/pip-build-ptqy9f_1/scipy/setup.py", line 175, in configuration
        config.add_subpackage('scipy')
      File "/home/est040/kategorier/nyhetskategorier/lib/python3.5/site-packages/numpy/distutils/misc_util.py", line 1001, in add_subpackage
        caller_level = 2)
      File "/home/est040/kategorier/nyhetskategorier/lib/python3.5/site-packages/numpy/distutils/misc_util.py", line 970, in get_subpackage
        caller_level = caller_level + 1)
      File "/home/est040/kategorier/nyhetskategorier/lib/python3.5/site-packages/numpy/distutils/misc_util.py", line 907, in _get_configuration_from_setup_py
        config = setup_module.configuration(*args)
      File "scipy/setup.py", line 15, in configuration
        config.add_subpackage('linalg')
      File "/home/est040/kategorier/nyhetskategorier/lib/python3.5/site-packages/numpy/distutils/misc_util.py", line 1001, in add_subpackage
        caller_level = 2)
      File "/home/est040/kategorier/nyhetskategorier/lib/python3.5/site-packages/numpy/distutils/misc_util.py", line 970, in get_subpackage
        caller_level = caller_level + 1)
      File "/home/est040/kategorier/nyhetskategorier/lib/python3.5/site-packages/numpy/distutils/misc_util.py", line 907, in _get_configuration_from_setup_py
        config = setup_module.configuration(*args)
      File "scipy/linalg/setup.py", line 19, in configuration
        raise NotFoundError('no lapack/blas resources found')
    numpy.distutils.system_info.NotFoundError: no lapack/blas resources found
    
    ----------------------------------------
Command "/home/est040/kategorier/nyhetskategorier/bin/python3.5 -c "import setuptools, tokenize;__file__='/tmp/pip-build-ptqy9f_1/scipy/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-22ol6nmp-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/est040/kategorier/nyhetskategorier/include/site/python3.5/scipy" failed with error code 1 in /tmp/pip-build-ptqy9f_1/scipy
[nyhetskategorier] [est040@nelson ~/kategorier]$

Bjarte mener jeg bør forsøke å instyallere openblas http://www.openblas.net/ i modulsystemet..

Prøver det. Feiler med å lage folder, drista meg til å SUDO lage folder

[est040@nelson private]$ sudo mkdir -p $USER

We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:

    #1) Respect the privacy of others.
    #2) Think before you type.
    #3) With great power comes great responsibility.

[sudo] password for est040: 
[est040@nelson private]$ ls
bjarte  est040  truls

[est040@nelson private]$ sudo chown est040:ansatt est040

ok, tror det nå skal funke. Følger videre guiden til bjarte.

# https://github.com/xianyi/OpenBLAS/wiki/Installation-Guide
wget, tar, cd
make FC=gfortran
make PREFIX=/data_bck/sw/openblas/0.2.14-gnu install

# Lage modulefile
data_bck/sw/modulefiles/openblas/0.2.14

noe feiler..

Jeg må fortelle pip hvor openblas ligger

module load python
module load openblas
pyvenv test
source test/bin/activate
mkdir test/build
pip install -d test/build numpy
pip install -d test/build scipy
cd test/build
tar xf numpy*.tar
cd numpy*
touch site.cfg
cat <<EOF
[DEFAULT]
library_dirs = /data_bck/sw/openblas/0.2.14-gnu/lib
include_dirs = /data_bck/sw/openblas/0.2.14-gnu/include

[atlas]
atlas_libs = openblas
libraries = openblas

[openblas]
libraries = openblas
library_dirs =  /data_bck/sw/openblas/0.2.14-gnu/lib
include_dirs = /data_bck/sw/openblas/0.2.14-gnu/include
EOF > site.cfg
python setup.py install
cd ..
tar xf scipy*.tar
cd scipy*
cp ../numpy*/site.cfg .
python setup.py install

Så krangla jeg en del med pip uten å bli klok, men endelig:

>>> import numpy
>>> import scipy
>>> import sklearn


Dump av mine notater fra første dag vi fikk serveren:

  1. Stuff we want
  • mosh https://mosh.mit.edu/ (needs a port opened )
  • git (gogs)
  • mysql/postgress
  • emacs (Truls sudo yum installs emacs..)

Installed software

  • python3 (we compile and make a module of for the modules system)
  • mongoDB (added to module system)

Notater, instalere python på IMDB.

Bjarte lasta ned Python-3.5.0.tgz til /scratch installerte til /data_bck/sw/python

Modulefile for python 3 laget, bruk

module load python

(dette adder python3 til path)