Install the platform » Historique » Révision 24
« Précédent |
Révision 24/62
(diff)
| Suivant »
pierre kestener, 20/01/2014 11:35
fox typo about fermi arch; add comments on ATLAS
Install the platform without MAGMA¶
The COMPASS platform is distributed as a single bundle of CArMA and SuTrA libraries and YoGA and its AO extension for Yorick.
Hardware requirements¶
The system must contain at least an x86 CPU and a CUDA capable GPU. list of compatible GPUs can be found here http://www.nvidia.com/object/cuda_gpus.html. Specific requirements apply to clusters (to be updated).
Environment requirements¶
The system must be running a 64 bit distribution of Linux or Mac OS with the latest NVIDIA drivers and CUDA toolkit. The installation of the corresponding version of the CULA tools is also required. The following installation instructions are valid if the default installation paths have been selected for these components.
Additionally, to benefit from the user-oriented features of the platform, Yorick should be installed as well as the latest version of Python and the associated pygtk module.
To install Yorick, download the latest version from the github repository:
git clone https://github.com/dhmunro/yorick.git yorick.git
then cd onto the created directory and install:
./configure && make && make install
once Yorick is locally installed, you will have to add this directory : yorick.git/relocate/bin to your PATH to have an easy access to the yorick executable. You may want to add support for command history by using rlwrap and alias the yorick executable as :
alias yorick='rlwrap path_to_yorick_executable/yorick'
Installation process¶
First check out the latest version from the svn repository :
svn co https://version-lesia.obspm.fr/repos/compass compass
then go in the newly created directory and then trunk:
cd compass/trunk
once there, you need to modify system variables in the define_var.sh executable :
emacs define_var.sh
in this file define properly CUDA_ROOT, CULA_ROOT and YoGA path. Note that for the latter, as YoGA is distributed with SUTrA you should just point to the newly created trunk directory. On a Linux system you should normally have:
export CUDA_ROOT=/usr/local/cuda export CULA_ROOT=/usr/local/cula export YOGA_DIR=/home/MyUserName/path2compass/trunk
in this file, you also have to indicate the proper architecture of your GPU so as the compiler will generate the appropriate code. Modify the following line:
export GENCODE="arch=compute_12,code=sm_12"
and change both 12 to your architecture : for instance a Tesla Fermi will have 2.0 computing capabilities so change 12 to 20, a Kepler GPU will have 3.0 or 3.5 (K20) computing capabilities, change 12 to 30 (or 35).
Once this is done, you're ready to compile the whole library. First run define_var.sh to define the system variables that will be used during the compilation process:
./define_var.sh
then identify the absolute path to your Yorick executable using:
which yorick
and run the compilation script:
./reinstall absolute_path_to_yorick
If you did not get any error, CArMA, SuTrA and YoGA are now installed on your machine. You can check that everything is working by launching a GUI to test a simulation:
yorick -i yoga_ao/ywidgets/widget_ao.i
Install the platform with MAGMA¶
Why MAGMA ?¶
The MAGMA project aims to develop a dense linear algebra library similar to LAPACK but for heterogeneous/hybrid architectures, starting with current "Multicore+GPU" systems.
Unlike CULA, MAGMA propose a dense linear algebra library handling double for free.
But MAGMA needs a LAPACK and a BLAS implementation. Actually, we try two options : ATLAS BLAS (free, easy to install) and MKL (free, need a registration but more powerful)
Dependencies : gfortran¶
Use your package manager to install dependencies:- on scientific linux : yum install gcc-gfortran libgfortran
- on debian : apt-get install gfortran gfortran-multilib
Configure MAGMA with ATLAS¶
Dependencies : blas, lapack, atlas¶
Use your package manager to install dependencies:- on scientific linux : yum install blas-devel lapack-devel atlas-devel
- on debian : apt-get install libblas-dev liblapack-dev libatlas-base-dev libatlas-dev
The binary packages of ATLAS (and also OpenBLAS / GotoBLAS2) distributed by your Linux distribution (SL, Fedora, Debian,...) are generic packages, which are not optimized for a specific machine.
It is strongly advised to recompile ATLAS on your local machine to get best performances.
extraction¶
MAGMA is available here : http://icl.cs.utk.edu/magma/software/index.html
extract the tgz file and go into the new directory
$ tar xf magma-1.4.1-beta.tar.gz
$ cd magma-1.4.1
configuration¶
You have to create your own make.inc :
- example on a scientific linux : please verify GPU_TARGET, LAPACKDIR, ATLASDIR, CUDADIR
#//////////////////////////////////////////////////////////////////////////////
# -- MAGMA (version 1.4.1) --
# Univ. of Tennessee, Knoxville
# Univ. of California, Berkeley
# Univ. of Colorado, Denver
# November 2013
#//////////////////////////////////////////////////////////////////////////////
# GPU_TARGET specifies for which GPU you want to compile MAGMA:
# "Tesla" (NVIDIA compute capability 1.x cards)
# "Fermi" (NVIDIA compute capability 2.x cards)
# "Kepler" (NVIDIA compute capability 3.x cards)
# See http://developer.nvidia.com/cuda-gpus
GPU_TARGET ?= Fermi
CC = gcc
NVCC = nvcc
FORT = gfortran
ARCH = ar
ARCHFLAGS = cr
RANLIB = ranlib
OPTS = -fPIC -O3 -DADD_ -fopenmp -DMAGMA_SETAFFINITY
F77OPTS = -fPIC -O3 -DADD_
FOPTS = -fPIC -O3 -DADD_ -x f95-cpp-input
NVOPTS = -O3 -DADD_ -Xcompiler "-fno-strict-aliasing -fPIC"
LDOPTS = -fPIC -fopenmp
# Depending on how ATLAS and LAPACK were compiled, you may need one or more of:
LIB = -llapack -lf77blas -latlas -lcblas -lcublas -lcudart -lstdc++ -lm -lgfortran
# define library directories here or in your environment
LAPACKDIR = /usr/lib64
ATLASDIR = /usr/lib64/atlas
CUDADIR = /usr/local/cuda
LIBDIR = -L$(LAPACKDIR) \
-L$(ATLASDIR) \
-L$(CUDADIR)/lib64
INC = -I$(CUDADIR)/include
- example on debian : please verify GPU_TARGET, LAPACKDIR, ATLASDIR, CUDADIR
#////////////////////////////////////////////////////////////////////////////// # -- MAGMA (version 1.4.1) -- # Univ. of Tennessee, Knoxville # Univ. of California, Berkeley # Univ. of Colorado, Denver # November 2013 #////////////////////////////////////////////////////////////////////////////// # GPU_TARGET specifies for which GPU you want to compile MAGMA: # "Tesla" (NVIDIA compute capability 1.x cards) # "Fermi" (NVIDIA compute capability 2.x cards) # "Kepler" (NVIDIA compute capability 3.x cards) # See http://developer.nvidia.com/cuda-gpus GPU_TARGET ?= Fermi CC = gcc NVCC = nvcc FORT = gfortran ARCH = ar ARCHFLAGS = cr RANLIB = ranlib OPTS = -fPIC -O3 -DADD_ -fopenmp -DMAGMA_SETAFFINITY F77OPTS = -fPIC -O3 -DADD_ FOPTS = -fPIC -O3 -DADD_ -x f95-cpp-input NVOPTS = -O3 -DADD_ -Xcompiler "-fno-strict-aliasing -fPIC" LDOPTS = -fPIC -fopenmp # Depending on how ATLAS and LAPACK were compiled, you may need one or more of: LIB = -llapack -lf77blas -latlas -lcblas -lcublas -lcudart -lstdc++ -lm -lgfortran # define library directories here or in your environment LAPACKDIR = /usr/lib ATLASDIR = /usr/lib CUDADIR = /usr/local/cuda LIBDIR = -L$(LAPACKDIR) \ -L$(ATLASDIR) \ -L$(CUDADIR)/lib64 \ -L/usr/lib/x86_64-linux-gnu INC = -I$(CUDADIR)/include
Configure MAGMA with MKL¶
extraction¶
To download MKL, you have to create a account here : https://registrationcenter.intel.com/RegCenter/NComForm.aspx?ProductID=1517
extract l_ccompxe_2013_sp1.1.106.tgz and go into l_ccompxe_2013_sp1.1.106
install it with ./install_GUI.sh and add IPP stuff to default choices
configuration¶
- example on debian : please verify GPU_TARGET, MKLROOT, CUDADIR
#////////////////////////////////////////////////////////////////////////////// # -- MAGMA (version 1.4.1-beta2) -- # Univ. of Tennessee, Knoxville # Univ. of California, Berkeley # Univ. of Colorado, Denver # December 2013 #////////////////////////////////////////////////////////////////////////////// # GPU_TARGET contains one or more of Tesla, Fermi, or Kepler, # to specify for which GPUs you want to compile MAGMA: # Tesla - NVIDIA compute capability 1.x cards # Fermi - NVIDIA compute capability 2.x cards # Kepler - NVIDIA compute capability 3.x cards # The default is all, "Tesla Fermi Kepler". # See http://developer.nvidia.com/cuda-gpus # GPU_TARGET ?= Fermi CC = gcc NVCC = nvcc FORT = gfortran ARCH = ar ARCHFLAGS = cr RANLIB = ranlib OPTS = -fPIC -O3 -DADD_ -Wall -fno-strict-aliasing -fopenmp -DMAGMA_WITH_MKL -DMAGMA_SETAFFINITY F77OPTS = -fPIC -O3 -DADD_ -Wall FOPTS = -fPIC -O3 -DADD_ -Wall -x f95-cpp-input NVOPTS = -O3 -DADD_ -Xcompiler "-fno-strict-aliasing -fPIC" LDOPTS = -fPIC -fopenmp # gcc with MKL 10.3, Intel threads LIB = -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lpthread -lcublas -lcudart -lstdc++ -lm -liomp5 -lgfortran # define library directories preferably in your environment, or here. # for MKL run, e.g.: source /opt/intel/composerxe/mkl/bin/mklvars.sh intel64 MKLROOT ?= /opt/intel/composerxe/mkl CUDADIR ?= /usr/local/cuda -include make.check-mkl -include make.check-cuda LIBDIR = -L$(MKLROOT)/lib/intel64 \ -L$(CUDADIR)/lib64 INC = -I$(CUDADIR)/include -I$(MKLROOT)/include
In this example, I use gcc but with MKL, you can use icc instead of gcc. In this case, you have to compile yorick with icc. For this, you have to change the CC flag in Make.cfg
compilation and installation¶
compilation¶
just compile the shared target (and test if you want)
~$ make -j 8 shared
installation¶
To install libraries and include files in a given prefix, run:
~$ make install prefix=/usr/local/magma
The default prefix is /usr/local/magma. You can also set prefix in make.inc.
tune (not tested)¶
For multi-GPU functions, set $MAGMA_NUM_GPUS to set the number of GPUs to use.
For multi-core BLAS libraries, set $OMP_NUM_THREADS or $MKL_NUM_THREADS or $VECLIB_MAXIMUM_THREADS to set the number of CPU threads, depending on your BLAS library.
Platform installation¶
Just just define $MAGMA_PATH and use the standard procedure
Mis à jour par pierre kestener il y a plus de 10 ans · 24 révisions