Projet

Général

Profil

Install the platform » Historique » Version 60

Arnaud Sevin, 15/02/2017 11:20

1 14 Arnaud Sevin
{{toc}}
2 1 Damien Gratadour
3 48 Arnaud Sevin
h1. Install Anaconda with python2
4 1 Damien Gratadour
5 49 Arnaud Sevin
more info: https://www.continuum.io/downloads#linux
6
7 1 Damien Gratadour
h2. Download and installation
8 49 Arnaud Sevin
9 48 Arnaud Sevin
* wget https://repo.continuum.io/archive/Anaconda2-4.2.0-Linux-x86_64.sh
10
* bash ./Anaconda2-4.2.0-Linux-x86_64.sh
11
* update your .batchrc
12
  add anaconda2/bin into the $PATH
13
14 1 Damien Gratadour
h2. add more packets
15 49 Arnaud Sevin
16 59 Arnaud Sevin
To avoid any incompatibility this python modules, it's highly recommended to use the gcc provided with anaconda:
17 52 Arnaud Sevin
18 59 Arnaud Sevin
* conda install -c compass compass
19 48 Arnaud Sevin
20 1 Damien Gratadour
h1. Install MAGMA
21 24 pierre kestener
22 36 Damien Gratadour
h2. Why MAGMA ?
23 1 Damien Gratadour
24
The MAGMA project aims to develop a dense linear algebra library similar to LAPACK but for heterogeneous/hybrid architectures, starting with current "Multicore+GPU" systems.
25
26 26 Arnaud Sevin
Unlike CULA, MAGMA propose a dense linear algebra library handling double for free.
27
28 1 Damien Gratadour
But MAGMA needs a LAPACK and a BLAS implementation. Actually, we try two options : openBLAS (free, easy to install) and MKL (free, need a registration but better optimized on Intel processors)
29
30 28 Arnaud Sevin
h2. Configure MAGMA with openBLAS
31 1 Damien Gratadour
32 28 Arnaud Sevin
h3. Dependencies : openblas (http://www.openblas.net)
33 1 Damien Gratadour
34 28 Arnaud Sevin
First, clone the GIT repository:
35
<pre>
36
git clone https://github.com/xianyi/OpenBLAS.git
37 1 Damien Gratadour
</pre>
38 28 Arnaud Sevin
39
compile it:
40 1 Damien Gratadour
<pre>
41 28 Arnaud Sevin
cd OpenBLAS/
42
make
43 14 Arnaud Sevin
</pre>
44 1 Damien Gratadour
45 28 Arnaud Sevin
install it:
46
<pre>
47 56 Arnaud Sevin
make install PREFIX=$HOME/local/openblas
48 28 Arnaud Sevin
</pre>
49
50 1 Damien Gratadour
add to you .bashrc:
51 51 Arnaud Sevin
<pre><code class="PHP">
52 48 Arnaud Sevin
export OPENBLAS_ROOT=$HOME/local/openblas
53 51 Arnaud Sevin
</code></pre>
54 1 Damien Gratadour
55
h3. extraction
56 14 Arnaud Sevin
57 1 Damien Gratadour
MAGMA is available here : http://icl.cs.utk.edu/magma/software/index.html
58 14 Arnaud Sevin
59
extract the tgz file and go into the new directory
60 57 Arnaud Sevin
> ~$ wget http://icl.cs.utk.edu/projectsfiles/magma/downloads/magma-2.2.0.tar.gz
61
> ~$ tar xf magma-2.2.0.tar.gz
62
> ~$ cd magma-2.2.0
63 14 Arnaud Sevin
64
h3. configuration
65 1 Damien Gratadour
66
You have to create your own make.inc based on make.inc.openblas:
67 57 Arnaud Sevin
> ~$ cp make.inc-examples/make.inc.openblas make.inc
68 1 Damien Gratadour
69
example : *please verify GPU_TARGET, OPENBLASDIR, CUDADIR*
70
71 50 Arnaud Sevin
<pre><code class="PHP">
72 54 Arnaud Sevin
#//////////////////////////////////////////////////////////////////////////////
73 57 Arnaud Sevin
#   -- MAGMA (version 2.2.0) --
74 1 Damien Gratadour
#      Univ. of Tennessee, Knoxville
75
#      Univ. of California, Berkeley
76
#      Univ. of Colorado, Denver
77 57 Arnaud Sevin
#      @date November 2016
78 1 Damien Gratadour
#//////////////////////////////////////////////////////////////////////////////
79
80 54 Arnaud Sevin
# GPU_TARGET contains one or more of Fermi, Kepler, or Maxwell,
81 1 Damien Gratadour
# to specify for which GPUs you want to compile MAGMA:
82 54 Arnaud Sevin
#     Fermi   - NVIDIA compute capability 2.x cards
83
#     Kepler  - NVIDIA compute capability 3.x cards
84
#     Maxwell - NVIDIA compute capability 5.x cards
85 1 Damien Gratadour
#     Pascal  - NVIDIA compute capability 6.x cards
86 28 Arnaud Sevin
# The default is "Fermi Kepler".
87 54 Arnaud Sevin
# Note that NVIDIA no longer supports 1.x cards, as of CUDA 6.5.
88 1 Damien Gratadour
# See http://developer.nvidia.com/cuda-gpus
89 28 Arnaud Sevin
#
90 57 Arnaud Sevin
GPU_TARGET ?= Pascal
91 1 Damien Gratadour
92 28 Arnaud Sevin
# --------------------
93
# programs
94 1 Damien Gratadour
95
CC        = gcc
96 28 Arnaud Sevin
CXX       = g++
97 1 Damien Gratadour
NVCC      = nvcc
98
FORT      = gfortran
99
100
ARCH      = ar
101
ARCHFLAGS = cr
102
RANLIB    = ranlib
103
104
105
# --------------------
106
# flags
107
108 28 Arnaud Sevin
# Use -fPIC to make shared (.so) and static (.a) library;
109
# can be commented out if making only static library.
110 14 Arnaud Sevin
FPIC      = -fPIC
111 28 Arnaud Sevin
112 57 Arnaud Sevin
CFLAGS    = -O3 $(FPIC) -DNDEBUG -DADD_ -Wall -fopenmp
113
FFLAGS    = -O3 $(FPIC) -DNDEBUG -DADD_ -Wall -Wno-unused-dummy-argument
114
F90FLAGS  = -O3 $(FPIC) -DNDEBUG -DADD_ -Wall -Wno-unused-dummy-argument -x f95-cpp-input
115
NVCCFLAGS = -O3         -DNDEBUG -DADD_       -Xcompiler "$(FPIC)"
116
LDFLAGS   =     $(FPIC)                       -fopenmp
117 53 Arnaud Sevin
118 1 Damien Gratadour
# C++11 (gcc >= 4.7) is not required, but has benefits like atomic operations
119
CXXFLAGS := $(CFLAGS) -std=c++11
120 53 Arnaud Sevin
CFLAGS   += -std=c99
121
122 54 Arnaud Sevin
123 1 Damien Gratadour
# --------------------
124
# libraries
125
126 28 Arnaud Sevin
# gcc with OpenBLAS (includes LAPACK)
127
LIB       = -lopenblas
128 17 Arnaud Sevin
129 57 Arnaud Sevin
LIB      += -lcublas -lcusparse -lcudart -lcudadevrt
130 17 Arnaud Sevin
131
132 28 Arnaud Sevin
# --------------------
133
# directories
134 17 Arnaud Sevin
135 28 Arnaud Sevin
# define library directories preferably in your environment, or here.
136 57 Arnaud Sevin
OPENBLASDIR ?= $(HOME)/local/openblas
137 54 Arnaud Sevin
CUDADIR ?= /usr/local/cuda
138 28 Arnaud Sevin
-include make.check-openblas
139
-include make.check-cuda
140 17 Arnaud Sevin
141 28 Arnaud Sevin
LIBDIR    = -L$(CUDADIR)/lib64 \
142
            -L$(OPENBLASDIR)/lib
143 17 Arnaud Sevin
144 55 Arnaud Sevin
INC       = -I$(CUDADIR)/include \
145
            -I$(OPENBLASDIR)/include
146 17 Arnaud Sevin
</code></pre>
147
148
h2. Configure MAGMA with MKL
149
150
h3. extraction
151
152
To download MKL, you have to create a account here : https://registrationcenter.intel.com/RegCenter/NComForm.aspx?ProductID=1517
153
154
extract l_ccompxe_2013_sp1.1.106.tgz and go into l_ccompxe_2013_sp1.1.106
155
156
install it with ./install_GUI.sh and add IPP stuff to default choices
157
158
h3. configuration
159
160 28 Arnaud Sevin
You have to create your own make.inc based on make.inc.mkl-gcc-ilp64:
161
162 50 Arnaud Sevin
example: *please verify GPU_TARGET, MKLROOT, CUDADIR*
163 17 Arnaud Sevin
<pre><code class="PHP">
164 1 Damien Gratadour
#//////////////////////////////////////////////////////////////////////////////
165 53 Arnaud Sevin
#   -- MAGMA (version 2.1.0) --
166 17 Arnaud Sevin
#      Univ. of Tennessee, Knoxville
167 1 Damien Gratadour
#      Univ. of California, Berkeley
168
#      Univ. of Colorado, Denver
169 53 Arnaud Sevin
#      @date August 2016
170 1 Damien Gratadour
#//////////////////////////////////////////////////////////////////////////////
171
172 53 Arnaud Sevin
# GPU_TARGET contains one or more of Fermi, Kepler, or Maxwell,
173 1 Damien Gratadour
# to specify for which GPUs you want to compile MAGMA:
174 53 Arnaud Sevin
#     Fermi   - NVIDIA compute capability 2.x cards
175
#     Kepler  - NVIDIA compute capability 3.x cards
176
#     Maxwell - NVIDIA compute capability 5.x cards
177
#     Pascal  - NVIDIA compute capability 6.x cards
178 28 Arnaud Sevin
# The default is "Fermi Kepler".
179 53 Arnaud Sevin
# Note that NVIDIA no longer supports 1.x cards, as of CUDA 6.5.
180 1 Damien Gratadour
# See http://developer.nvidia.com/cuda-gpus
181
#
182 14 Arnaud Sevin
#GPU_TARGET ?= Fermi Kepler
183 28 Arnaud Sevin
184
# --------------------
185
# programs
186 20 Arnaud Sevin
187 53 Arnaud Sevin
CC        = icc
188
CXX       = icpc
189 20 Arnaud Sevin
NVCC      = nvcc
190 53 Arnaud Sevin
FORT      = ifort
191 1 Damien Gratadour
192
ARCH      = ar
193
ARCHFLAGS = cr
194
RANLIB    = ranlib
195
196
197
# --------------------
198
# flags
199
200
# Use -fPIC to make shared (.so) and static (.a) library;
201
# can be commented out if making only static library.
202
FPIC      = -fPIC
203
204 53 Arnaud Sevin
CFLAGS    = -O3 $(FPIC) -openmp -DADD_ -Wall -Wshadow -DMAGMA_WITH_MKL
205
FFLAGS    = -O3 $(FPIC)         -DADD_ -warn all -warn nounused -nogen-interfaces
206
F90FLAGS  = -O3 $(FPIC)         -DADD_ -warn all -warn nounused
207
NVCCFLAGS = -O3                 -DADD_ -Xcompiler "$(FPIC) -Wall -Wno-unused-function"
208
LDFLAGS   =     $(FPIC) -openmp
209 1 Damien Gratadour
210
# Defining MAGMA_ILP64 or MKL_ILP64 changes magma_int_t to int64_t in include/magma_types.h
211
CFLAGS    += -DMKL_ILP64
212 53 Arnaud Sevin
FFLAGS    += -integer-size 64
213
F90FLAGS  += -integer-size 64
214 1 Damien Gratadour
NVCCFLAGS += -DMKL_ILP64
215
216
# Options to do extra checks for non-standard things like variable length arrays;
217
# it is safe to disable all these
218
CFLAGS   += -pedantic -Wno-long-long
219
#CFLAGS   += -Werror  # uncomment to ensure all warnings are dealt with
220 53 Arnaud Sevin
221
# C++11 (icc >= 13) is not required, but has benefits like atomic operations
222
CXXFLAGS := $(CFLAGS) -std=c++11
223 1 Damien Gratadour
CFLAGS   += -std=c99
224
225
226
# --------------------
227
# libraries
228
229 53 Arnaud Sevin
# IMPORTANT: these link lines are for 64-bit int !!!!
230 1 Damien Gratadour
# For regular 64-bit builds using 64-bit pointers and 32-bit int,
231
# use the lp64 library, not the ilp64 library. See make.inc.mkl-gcc or make.inc.mkl-icc.
232 53 Arnaud Sevin
233 1 Damien Gratadour
# see MKL Link Advisor at http://software.intel.com/sites/products/mkl/
234 53 Arnaud Sevin
# icc with MKL 10.3, Intel OpenMP threads, 64-bit int
235
# note -DMAGMA_ILP64 or -DMKL_ILP64, and -integer-size 64 in FLAGS above
236
LIB       = -lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_core -lpthread -lstdc++ -lm
237 1 Damien Gratadour
238 53 Arnaud Sevin
LIB      += -lcublas -lcusparse -lcudart
239 1 Damien Gratadour
240
241
# --------------------
242
# directories
243
244
# define library directories preferably in your environment, or here.
245
# for MKL run, e.g.: source /opt/intel/composerxe/mkl/bin/mklvars.sh intel64
246
#MKLROOT ?= /opt/intel/composerxe/mkl
247
#CUDADIR ?= /usr/local/cuda
248
-include make.check-mkl
249
-include make.check-cuda
250
251
LIBDIR    = -L$(CUDADIR)/lib64 \
252
            -L$(MKLROOT)/lib/intel64
253
254
INC       = -I$(CUDADIR)/include \
255
            -I$(MKLROOT)/include
256
</code></pre>
257
258
In this example, I use gcc but with MKL, you can use icc instead of gcc. In this case, you have to compile yorick with icc. For this, you have to change the CC flag in Make.cfg  
259
260
h2. compilation and installation
261
262
h3. compilation
263
264
just compile the shared target (and test if you want)
265 58 Arnaud Sevin
> ~$ make -j 8 shared sparse-shared
266 1 Damien Gratadour
267
h3. installation
268 48 Arnaud Sevin
269 1 Damien Gratadour
To install libraries and include files in a given prefix, run:
270
> ~$ make install prefix=$HOME/local/magma
271
  
272 38 Damien Gratadour
The default prefix is /usr/local/magma. You can also set prefix in make.inc.
273 1 Damien Gratadour
274 38 Damien Gratadour
h3. tuning (not tested)
275 29 Arnaud Sevin
276
For multi-GPU functions, set $MAGMA_NUM_GPUS to set the number of GPUs to use.
277
For multi-core BLAS libraries, set $OMP_NUM_THREADS or $MKL_NUM_THREADS or $VECLIB_MAXIMUM_THREADS to set the number of CPU threads, depending on your BLAS library.
278
279
h1. Install the platform
280
281
The COMPASS platform is distributed as a single bundle of CArMA and SuTrA C++ / Cuda libraries and their Python extensions NAGA & SHESHA. 
282
283
h2. Hardware requirements
284 41 Arnaud Sevin
285 29 Arnaud Sevin
The system must contain at least an x86 CPU and a CUDA capable GPU. list of compatible GPUs can be found here http://www.nvidia.com/object/cuda_gpus.html. Specific requirements apply to clusters (to be updated).
286
287
h2. Environment requirements
288
289 47 Arnaud Sevin
The system must be running a 64 bit distribution of Linux with the latest NVIDIA drivers and "CUDA toolkit":https://developer.nvidia.com/cuda-downloads. The following installation instructions are valid if the default installation paths have been selected for these components.
290 29 Arnaud Sevin
291
Additionally, to benefit from the user-oriented features of the platform, Anaconda2 should be installed (https://www.continuum.io/downloads#_unix).
292
In the last versions of compass (r608+), Yorick is no more supported.
293
294
h2. Installation process
295
296 39 Arnaud Sevin
First check out the latest version from the svn repository :
297 1 Damien Gratadour
<pre>
298 29 Arnaud Sevin
svn co https://version-lesia.obspm.fr/repos/compass/trunk compass
299
</pre>
300 39 Arnaud Sevin
then go in the newly created directory and then trunk:
301 29 Arnaud Sevin
<pre>
302
cd compass
303 50 Arnaud Sevin
</pre>
304 29 Arnaud Sevin
once there, you need to modify system variables in our .bashrc :
305
<pre><code class="PHP">
306
# CUDA default definitions
307
export CUDA_ROOT=$CUDA_ROOT #/usr/local/cuda
308 1 Damien Gratadour
export CUDA_INC_PATH=$CUDA_ROOT/include
309 29 Arnaud Sevin
export CUDA_LIB_PATH=$CUDA_ROOT/lib
310 51 Arnaud Sevin
export CUDA_LIB_PATH_64=$CUDA_ROOT/lib64
311 50 Arnaud Sevin
export PATH=$CUDA_ROOT/bin:$PATH
312 1 Damien Gratadour
export LD_LIBRARY_PATH=$CUDA_LIB_PATH_64:$CUDA_LIB_PATH:$LD_LIBRARY_PATH
313 50 Arnaud Sevin
</code></pre>
314 33 Arnaud Sevin
in this file, you also have to indicate the proper architecture of your GPU so as the compiler will generate the appropriate code.
315 50 Arnaud Sevin
<pre><code class="PHP">
316 48 Arnaud Sevin
export GENCODE="arch=compute_52,code=sm_52"
317 43 Arnaud Sevin
</code></pre>
318 29 Arnaud Sevin
and change both 52 to your architecture : for instance a Tesla Fermi will have 2.0 computing capabilities so change 52 to 20, a Kepler GPU will have 3.0 or 3.5 (K20) computing capabilities, change 52 to 30 (or 35), a Maxwell GPU have 5.2 (M6000), a Pascal have 6.0 (P100). 
319
(more informations here: https://developer.nvidia.com/cuda-gpus)
320 50 Arnaud Sevin
321 29 Arnaud Sevin
If you are using CULA, you have to specify it:
322
<pre><code class="PHP">
323
# CULA default definitions
324 1 Damien Gratadour
export CULA_ROOT= /usr/local/cula
325 29 Arnaud Sevin
export CULA_INC_PATH= $CULA_ROOT/include
326 51 Arnaud Sevin
export CULA_LIB_PATH= $CULA_ROOT/lib
327 50 Arnaud Sevin
export CULA_LIB_PATH_64= $CULA_ROOT/lib64
328 29 Arnaud Sevin
export LD_LIBRARY_PATH=$CULA_LIB_PATH_64:$CULA_LIB_PATH:$LD_LIBRARY_PATH
329
</code></pre>
330 50 Arnaud Sevin
331 29 Arnaud Sevin
If you are using MAGMA, you have to specify it:
332 1 Damien Gratadour
<pre><code class="PHP">
333
# MAGMA definitions (uncomment this line if MAGMA is installed)
334
export MAGMA_ROOT=$HOME/local/magma
335 50 Arnaud Sevin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$MAGMA_ROOT/lib
336 1 Damien Gratadour
export PKG_CONFIG_PATH=$MAGMA_ROOT/lib/pkgconfig
337
</code></pre>
338
339
Last variables to define:
340
<pre><code class="PHP">
341
export COMPASS_ROOT=/path/to/compass/trunk
342 51 Arnaud Sevin
export NAGA_ROOT=$COMPASS_ROOT/naga
343 50 Arnaud Sevin
export SHESHA_ROOT=$COMPASS_ROOT/shesha
344
export LD_LIBRARY_PATH=$COMPASS_ROOT/libcarma:$COMPASS_ROOT/libsutra:$LD_LIBRARY_PATH
345
</code></pre>
346
347
At the end, you .bashrc shoud containts all those informations:
348
<pre><code class="PHP">
349
# conda default definitions
350
export CONDA_ROOT=/your/path/anaconda2
351
export PATH=$CONDA_ROOT/bin:$PATH
352
353 1 Damien Gratadour
# CUDA default definitions
354 50 Arnaud Sevin
export CUDA_INC_PATH=$CUDA_ROOT/include
355
export CUDA_LIB_PATH=$CUDA_ROOT/lib
356 51 Arnaud Sevin
export CUDA_LIB_PATH_64=$CUDA_ROOT/lib64
357 50 Arnaud Sevin
export PATH=$CUDA_ROOT/bin:$PATH
358
export LD_LIBRARY_PATH=$CUDA_LIB_PATH_64:$CUDA_LIB_PATH:$LD_LIBRARY_PATH
359
export GENCODE="arch=compute_52,code=sm_52"
360
361 60 Arnaud Sevin
# OPENBLAS definitions
362
export OPENBLAS_ROOT=$HOME/local/openblas
363
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$OPENBLAS_ROOT/lib
364
export PKG_CONFIG_PATH=$OPENBLAS_ROOT/lib/pkgconfig
365
366 50 Arnaud Sevin
# MAGMA definitions
367
export MAGMA_ROOT=$HOME/local/magma
368
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$MAGMA_ROOT/lib
369 1 Damien Gratadour
export PKG_CONFIG_PATH=$MAGMA_ROOT/lib/pkgconfig
370 50 Arnaud Sevin
371
# COMPASS default definitions
372
export COMPASS_ROOT=/your/path/compass
373 51 Arnaud Sevin
export NAGA_ROOT=$COMPASS_ROOT/naga
374 50 Arnaud Sevin
export SHESHA_ROOT=$COMPASS_ROOT/shesha
375 29 Arnaud Sevin
export LD_LIBRARY_PATH=$COMPASS_ROOT/libcarma:$COMPASS_ROOT/libsutra:$LD_LIBRARY_PATH
376
</code></pre>
377
378 44 Arnaud Sevin
Once this is done, you're ready to compile the whole library:
379 29 Arnaud Sevin
<pre>
380
make clean all
381
</pre>
382
383 45 Arnaud Sevin
If you did not get any error, CArMA, SuTrA, NAGA and SHESHA are now installed on your machine. You can check that everything is working by launching a GUI to test a simulation:
384 29 Arnaud Sevin
<pre>
385 1 Damien Gratadour
ipython -i $SHESHA_ROOT/widgets/widget_ao.py
386
</pre>