Projet

Général

Profil

Install the platform » Historique » Version 55

Arnaud Sevin, 13/02/2017 14:52

1 14 Arnaud Sevin
{{toc}}
2 1 Damien Gratadour
3 48 Arnaud Sevin
h1. Install Anaconda with python2
4 1 Damien Gratadour
5 49 Arnaud Sevin
more info: https://www.continuum.io/downloads#linux
6
7 1 Damien Gratadour
h2. Download and installation
8 49 Arnaud Sevin
9 48 Arnaud Sevin
* wget https://repo.continuum.io/archive/Anaconda2-4.2.0-Linux-x86_64.sh
10
* bash ./Anaconda2-4.2.0-Linux-x86_64.sh
11
* update your .batchrc
12
  add anaconda2/bin into the $PATH
13
14 1 Damien Gratadour
h2. add more packets
15 49 Arnaud Sevin
16 52 Arnaud Sevin
To avoid any incompatibility this python modules, it's highly recommended to use the one provided with anaconda:
17
18 48 Arnaud Sevin
* conda install gcc pyqtgraph
19
20 1 Damien Gratadour
h1. Install MAGMA
21 24 pierre kestener
22 36 Damien Gratadour
h2. Why MAGMA ?
23 1 Damien Gratadour
24
The MAGMA project aims to develop a dense linear algebra library similar to LAPACK but for heterogeneous/hybrid architectures, starting with current "Multicore+GPU" systems.
25
26 26 Arnaud Sevin
Unlike CULA, MAGMA propose a dense linear algebra library handling double for free.
27
28 1 Damien Gratadour
But MAGMA needs a LAPACK and a BLAS implementation. Actually, we try two options : openBLAS (free, easy to install) and MKL (free, need a registration but better optimized on Intel processors)
29
30 28 Arnaud Sevin
h2. Configure MAGMA with openBLAS
31 1 Damien Gratadour
32 28 Arnaud Sevin
h3. Dependencies : openblas (http://www.openblas.net)
33 1 Damien Gratadour
34 28 Arnaud Sevin
First, clone the GIT repository:
35
<pre>
36
git clone https://github.com/xianyi/OpenBLAS.git
37 1 Damien Gratadour
</pre>
38 28 Arnaud Sevin
39
compile it:
40 1 Damien Gratadour
<pre>
41 28 Arnaud Sevin
cd OpenBLAS/
42
make
43 14 Arnaud Sevin
</pre>
44 1 Damien Gratadour
45 28 Arnaud Sevin
install it:
46
<pre>
47 48 Arnaud Sevin
sudo make install PREFIX=$HOME/local/openblas
48 28 Arnaud Sevin
</pre>
49
50 1 Damien Gratadour
add to you .bashrc:
51 51 Arnaud Sevin
<pre><code class="PHP">
52 48 Arnaud Sevin
export OPENBLAS_ROOT=$HOME/local/openblas
53 51 Arnaud Sevin
</code></pre>
54 1 Damien Gratadour
55
h3. extraction
56 14 Arnaud Sevin
57 1 Damien Gratadour
MAGMA is available here : http://icl.cs.utk.edu/magma/software/index.html
58 14 Arnaud Sevin
59
extract the tgz file and go into the new directory
60 48 Arnaud Sevin
> ~$ tar xf magma-2.1.0.tar.gz
61
> ~$ cd magma-2.1.0
62 14 Arnaud Sevin
63
h3. configuration
64 1 Damien Gratadour
65 28 Arnaud Sevin
You have to create your own make.inc based on make.inc.openblas:
66 1 Damien Gratadour
67 37 Damien Gratadour
example : *please verify GPU_TARGET, OPENBLASDIR, CUDADIR*
68 1 Damien Gratadour
69 50 Arnaud Sevin
<pre><code class="PHP">
70 1 Damien Gratadour
#//////////////////////////////////////////////////////////////////////////////
71 54 Arnaud Sevin
#   -- MAGMA (version 2.1.0) --
72 1 Damien Gratadour
#      Univ. of Tennessee, Knoxville
73
#      Univ. of California, Berkeley
74
#      Univ. of Colorado, Denver
75 54 Arnaud Sevin
#      @date August 2016
76 1 Damien Gratadour
#//////////////////////////////////////////////////////////////////////////////
77
78 54 Arnaud Sevin
# GPU_TARGET contains one or more of Fermi, Kepler, or Maxwell,
79 1 Damien Gratadour
# to specify for which GPUs you want to compile MAGMA:
80 54 Arnaud Sevin
#     Fermi   - NVIDIA compute capability 2.x cards
81
#     Kepler  - NVIDIA compute capability 3.x cards
82
#     Maxwell - NVIDIA compute capability 5.x cards
83
#     Pascal  - NVIDIA compute capability 6.x cards
84 28 Arnaud Sevin
# The default is "Fermi Kepler".
85 54 Arnaud Sevin
# Note that NVIDIA no longer supports 1.x cards, as of CUDA 6.5.
86 1 Damien Gratadour
# See http://developer.nvidia.com/cuda-gpus
87 28 Arnaud Sevin
#
88 54 Arnaud Sevin
GPU_TARGET ?=  Kepler
89 1 Damien Gratadour
90 28 Arnaud Sevin
# --------------------
91
# programs
92 1 Damien Gratadour
93
CC        = gcc
94 28 Arnaud Sevin
CXX       = g++
95 1 Damien Gratadour
NVCC      = nvcc
96
FORT      = gfortran
97
98
ARCH      = ar
99
ARCHFLAGS = cr
100
RANLIB    = ranlib
101
102
103 28 Arnaud Sevin
# --------------------
104
# flags
105 1 Damien Gratadour
106 28 Arnaud Sevin
# Use -fPIC to make shared (.so) and static (.a) library;
107
# can be commented out if making only static library.
108 14 Arnaud Sevin
FPIC      = -fPIC
109 28 Arnaud Sevin
110 54 Arnaud Sevin
CFLAGS    = -O3 $(FPIC) -DADD_ -Wall -fopenmp
111 1 Damien Gratadour
FFLAGS    = -O3 $(FPIC) -DADD_ -Wall -Wno-unused-dummy-argument
112 28 Arnaud Sevin
F90FLAGS  = -O3 $(FPIC) -DADD_ -Wall -Wno-unused-dummy-argument -x f95-cpp-input
113
NVCCFLAGS = -O3         -DADD_       -Xcompiler "$(FPIC)"
114 14 Arnaud Sevin
LDFLAGS   =     $(FPIC)              -fopenmp
115 1 Damien Gratadour
116 53 Arnaud Sevin
# C++11 (gcc >= 4.7) is not required, but has benefits like atomic operations
117 1 Damien Gratadour
CXXFLAGS := $(CFLAGS) -std=c++11
118 53 Arnaud Sevin
CFLAGS   += -std=c99
119
120 54 Arnaud Sevin
121 28 Arnaud Sevin
# --------------------
122 1 Damien Gratadour
# libraries
123
124 28 Arnaud Sevin
# gcc with OpenBLAS (includes LAPACK)
125
LIB       = -lopenblas
126 17 Arnaud Sevin
127 54 Arnaud Sevin
LIB      += -lcublas -lcusparse -lcudart
128 17 Arnaud Sevin
129
130 28 Arnaud Sevin
# --------------------
131
# directories
132 17 Arnaud Sevin
133 28 Arnaud Sevin
# define library directories preferably in your environment, or here.
134 54 Arnaud Sevin
OPENBLASDIR ?= $HOME/local/openblas
135
CUDADIR ?= /usr/local/cuda
136 28 Arnaud Sevin
-include make.check-openblas
137
-include make.check-cuda
138 17 Arnaud Sevin
139 28 Arnaud Sevin
LIBDIR    = -L$(CUDADIR)/lib64 \
140
            -L$(OPENBLASDIR)/lib
141 17 Arnaud Sevin
142 55 Arnaud Sevin
INC       = -I$(CUDADIR)/include \
143
            -I$(OPENBLASDIR)/include
144 17 Arnaud Sevin
</code></pre>
145
146
h2. Configure MAGMA with MKL
147
148
h3. extraction
149
150
To download MKL, you have to create a account here : https://registrationcenter.intel.com/RegCenter/NComForm.aspx?ProductID=1517
151
152
extract l_ccompxe_2013_sp1.1.106.tgz and go into l_ccompxe_2013_sp1.1.106
153
154
install it with ./install_GUI.sh and add IPP stuff to default choices
155
156
h3. configuration
157
158 28 Arnaud Sevin
You have to create your own make.inc based on make.inc.mkl-gcc-ilp64:
159
160 50 Arnaud Sevin
example: *please verify GPU_TARGET, MKLROOT, CUDADIR*
161 17 Arnaud Sevin
<pre><code class="PHP">
162 1 Damien Gratadour
#//////////////////////////////////////////////////////////////////////////////
163 53 Arnaud Sevin
#   -- MAGMA (version 2.1.0) --
164 17 Arnaud Sevin
#      Univ. of Tennessee, Knoxville
165 1 Damien Gratadour
#      Univ. of California, Berkeley
166
#      Univ. of Colorado, Denver
167 53 Arnaud Sevin
#      @date August 2016
168 1 Damien Gratadour
#//////////////////////////////////////////////////////////////////////////////
169
170 53 Arnaud Sevin
# GPU_TARGET contains one or more of Fermi, Kepler, or Maxwell,
171 1 Damien Gratadour
# to specify for which GPUs you want to compile MAGMA:
172 53 Arnaud Sevin
#     Fermi   - NVIDIA compute capability 2.x cards
173
#     Kepler  - NVIDIA compute capability 3.x cards
174
#     Maxwell - NVIDIA compute capability 5.x cards
175
#     Pascal  - NVIDIA compute capability 6.x cards
176 28 Arnaud Sevin
# The default is "Fermi Kepler".
177 53 Arnaud Sevin
# Note that NVIDIA no longer supports 1.x cards, as of CUDA 6.5.
178 1 Damien Gratadour
# See http://developer.nvidia.com/cuda-gpus
179
#
180 14 Arnaud Sevin
#GPU_TARGET ?= Fermi Kepler
181 28 Arnaud Sevin
182
# --------------------
183
# programs
184 20 Arnaud Sevin
185 53 Arnaud Sevin
CC        = icc
186
CXX       = icpc
187 20 Arnaud Sevin
NVCC      = nvcc
188 53 Arnaud Sevin
FORT      = ifort
189 1 Damien Gratadour
190
ARCH      = ar
191
ARCHFLAGS = cr
192
RANLIB    = ranlib
193
194
195
# --------------------
196
# flags
197
198
# Use -fPIC to make shared (.so) and static (.a) library;
199
# can be commented out if making only static library.
200
FPIC      = -fPIC
201
202 53 Arnaud Sevin
CFLAGS    = -O3 $(FPIC) -openmp -DADD_ -Wall -Wshadow -DMAGMA_WITH_MKL
203
FFLAGS    = -O3 $(FPIC)         -DADD_ -warn all -warn nounused -nogen-interfaces
204
F90FLAGS  = -O3 $(FPIC)         -DADD_ -warn all -warn nounused
205
NVCCFLAGS = -O3                 -DADD_ -Xcompiler "$(FPIC) -Wall -Wno-unused-function"
206
LDFLAGS   =     $(FPIC) -openmp
207 1 Damien Gratadour
208
# Defining MAGMA_ILP64 or MKL_ILP64 changes magma_int_t to int64_t in include/magma_types.h
209
CFLAGS    += -DMKL_ILP64
210 53 Arnaud Sevin
FFLAGS    += -integer-size 64
211
F90FLAGS  += -integer-size 64
212 1 Damien Gratadour
NVCCFLAGS += -DMKL_ILP64
213
214
# Options to do extra checks for non-standard things like variable length arrays;
215
# it is safe to disable all these
216
CFLAGS   += -pedantic -Wno-long-long
217
#CFLAGS   += -Werror  # uncomment to ensure all warnings are dealt with
218 53 Arnaud Sevin
219
# C++11 (icc >= 13) is not required, but has benefits like atomic operations
220
CXXFLAGS := $(CFLAGS) -std=c++11
221 1 Damien Gratadour
CFLAGS   += -std=c99
222
223
224
# --------------------
225
# libraries
226
227 53 Arnaud Sevin
# IMPORTANT: these link lines are for 64-bit int !!!!
228 1 Damien Gratadour
# For regular 64-bit builds using 64-bit pointers and 32-bit int,
229
# use the lp64 library, not the ilp64 library. See make.inc.mkl-gcc or make.inc.mkl-icc.
230 53 Arnaud Sevin
231 1 Damien Gratadour
# see MKL Link Advisor at http://software.intel.com/sites/products/mkl/
232 53 Arnaud Sevin
# icc with MKL 10.3, Intel OpenMP threads, 64-bit int
233
# note -DMAGMA_ILP64 or -DMKL_ILP64, and -integer-size 64 in FLAGS above
234
LIB       = -lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_core -lpthread -lstdc++ -lm
235 1 Damien Gratadour
236 53 Arnaud Sevin
LIB      += -lcublas -lcusparse -lcudart
237 1 Damien Gratadour
238
239
# --------------------
240
# directories
241
242
# define library directories preferably in your environment, or here.
243
# for MKL run, e.g.: source /opt/intel/composerxe/mkl/bin/mklvars.sh intel64
244
#MKLROOT ?= /opt/intel/composerxe/mkl
245
#CUDADIR ?= /usr/local/cuda
246
-include make.check-mkl
247
-include make.check-cuda
248
249
LIBDIR    = -L$(CUDADIR)/lib64 \
250
            -L$(MKLROOT)/lib/intel64
251
252
INC       = -I$(CUDADIR)/include \
253
            -I$(MKLROOT)/include
254
</code></pre>
255
256
In this example, I use gcc but with MKL, you can use icc instead of gcc. In this case, you have to compile yorick with icc. For this, you have to change the CC flag in Make.cfg  
257
258
h2. compilation and installation
259
260
h3. compilation
261
262
just compile the shared target (and test if you want)
263
> ~$ make -j 8 shared sparse
264
265
h3. installation
266 48 Arnaud Sevin
267 1 Damien Gratadour
To install libraries and include files in a given prefix, run:
268
> ~$ make install prefix=$HOME/local/magma
269
  
270 38 Damien Gratadour
The default prefix is /usr/local/magma. You can also set prefix in make.inc.
271 1 Damien Gratadour
272 38 Damien Gratadour
h3. tuning (not tested)
273 29 Arnaud Sevin
274
For multi-GPU functions, set $MAGMA_NUM_GPUS to set the number of GPUs to use.
275
For multi-core BLAS libraries, set $OMP_NUM_THREADS or $MKL_NUM_THREADS or $VECLIB_MAXIMUM_THREADS to set the number of CPU threads, depending on your BLAS library.
276
277
h1. Install the platform
278
279
The COMPASS platform is distributed as a single bundle of CArMA and SuTrA C++ / Cuda libraries and their Python extensions NAGA & SHESHA. 
280
281
h2. Hardware requirements
282 41 Arnaud Sevin
283 29 Arnaud Sevin
The system must contain at least an x86 CPU and a CUDA capable GPU. list of compatible GPUs can be found here http://www.nvidia.com/object/cuda_gpus.html. Specific requirements apply to clusters (to be updated).
284
285
h2. Environment requirements
286
287 47 Arnaud Sevin
The system must be running a 64 bit distribution of Linux with the latest NVIDIA drivers and "CUDA toolkit":https://developer.nvidia.com/cuda-downloads. The following installation instructions are valid if the default installation paths have been selected for these components.
288 29 Arnaud Sevin
289
Additionally, to benefit from the user-oriented features of the platform, Anaconda2 should be installed (https://www.continuum.io/downloads#_unix).
290
In the last versions of compass (r608+), Yorick is no more supported.
291
292
h2. Installation process
293
294 39 Arnaud Sevin
First check out the latest version from the svn repository :
295 1 Damien Gratadour
<pre>
296 29 Arnaud Sevin
svn co https://version-lesia.obspm.fr/repos/compass/trunk compass
297
</pre>
298 39 Arnaud Sevin
then go in the newly created directory and then trunk:
299 29 Arnaud Sevin
<pre>
300
cd compass
301 50 Arnaud Sevin
</pre>
302 29 Arnaud Sevin
once there, you need to modify system variables in our .bashrc :
303
<pre><code class="PHP">
304
# CUDA default definitions
305
export CUDA_ROOT=$CUDA_ROOT #/usr/local/cuda
306 1 Damien Gratadour
export CUDA_INC_PATH=$CUDA_ROOT/include
307 29 Arnaud Sevin
export CUDA_LIB_PATH=$CUDA_ROOT/lib
308 51 Arnaud Sevin
export CUDA_LIB_PATH_64=$CUDA_ROOT/lib64
309 50 Arnaud Sevin
export PATH=$CUDA_ROOT/bin:$PATH
310 1 Damien Gratadour
export LD_LIBRARY_PATH=$CUDA_LIB_PATH_64:$CUDA_LIB_PATH:$LD_LIBRARY_PATH
311 50 Arnaud Sevin
</code></pre>
312 33 Arnaud Sevin
in this file, you also have to indicate the proper architecture of your GPU so as the compiler will generate the appropriate code.
313 50 Arnaud Sevin
<pre><code class="PHP">
314 48 Arnaud Sevin
export GENCODE="arch=compute_52,code=sm_52"
315 43 Arnaud Sevin
</code></pre>
316 29 Arnaud Sevin
and change both 52 to your architecture : for instance a Tesla Fermi will have 2.0 computing capabilities so change 52 to 20, a Kepler GPU will have 3.0 or 3.5 (K20) computing capabilities, change 52 to 30 (or 35), a Maxwell GPU have 5.2 (M6000), a Pascal have 6.0 (P100). 
317
(more informations here: https://developer.nvidia.com/cuda-gpus)
318 50 Arnaud Sevin
319 29 Arnaud Sevin
If you are using CULA, you have to specify it:
320
<pre><code class="PHP">
321
# CULA default definitions
322 1 Damien Gratadour
export CULA_ROOT= /usr/local/cula
323 29 Arnaud Sevin
export CULA_INC_PATH= $CULA_ROOT/include
324 51 Arnaud Sevin
export CULA_LIB_PATH= $CULA_ROOT/lib
325 50 Arnaud Sevin
export CULA_LIB_PATH_64= $CULA_ROOT/lib64
326 29 Arnaud Sevin
export LD_LIBRARY_PATH=$CULA_LIB_PATH_64:$CULA_LIB_PATH:$LD_LIBRARY_PATH
327
</code></pre>
328 50 Arnaud Sevin
329 29 Arnaud Sevin
If you are using MAGMA, you have to specify it:
330 1 Damien Gratadour
<pre><code class="PHP">
331
# MAGMA definitions (uncomment this line if MAGMA is installed)
332
export MAGMA_ROOT=$HOME/local/magma
333 50 Arnaud Sevin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$MAGMA_ROOT/lib
334 1 Damien Gratadour
export PKG_CONFIG_PATH=$MAGMA_ROOT/lib/pkgconfig
335
</code></pre>
336
337
Last variables to define:
338
<pre><code class="PHP">
339
export COMPASS_ROOT=/path/to/compass/trunk
340 51 Arnaud Sevin
export NAGA_ROOT=$COMPASS_ROOT/naga
341 50 Arnaud Sevin
export SHESHA_ROOT=$COMPASS_ROOT/shesha
342
export LD_LIBRARY_PATH=$COMPASS_ROOT/libcarma:$COMPASS_ROOT/libsutra:$LD_LIBRARY_PATH
343
</code></pre>
344
345
At the end, you .bashrc shoud containts all those informations:
346
<pre><code class="PHP">
347
# conda default definitions
348
export CONDA_ROOT=/your/path/anaconda2
349
export PATH=$CONDA_ROOT/bin:$PATH
350
351 1 Damien Gratadour
# CUDA default definitions
352 50 Arnaud Sevin
export CUDA_INC_PATH=$CUDA_ROOT/include
353
export CUDA_LIB_PATH=$CUDA_ROOT/lib
354 51 Arnaud Sevin
export CUDA_LIB_PATH_64=$CUDA_ROOT/lib64
355 50 Arnaud Sevin
export PATH=$CUDA_ROOT/bin:$PATH
356
export LD_LIBRARY_PATH=$CUDA_LIB_PATH_64:$CUDA_LIB_PATH:$LD_LIBRARY_PATH
357
export GENCODE="arch=compute_52,code=sm_52"
358
359
# MAGMA definitions
360
export MAGMA_ROOT=$HOME/local/magma
361
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$MAGMA_ROOT/lib
362 1 Damien Gratadour
export PKG_CONFIG_PATH=$MAGMA_ROOT/lib/pkgconfig
363 50 Arnaud Sevin
364
# COMPASS default definitions
365
export COMPASS_ROOT=/your/path/compass
366 51 Arnaud Sevin
export NAGA_ROOT=$COMPASS_ROOT/naga
367 50 Arnaud Sevin
export SHESHA_ROOT=$COMPASS_ROOT/shesha
368 29 Arnaud Sevin
export LD_LIBRARY_PATH=$COMPASS_ROOT/libcarma:$COMPASS_ROOT/libsutra:$LD_LIBRARY_PATH
369
</code></pre>
370
371 44 Arnaud Sevin
Once this is done, you're ready to compile the whole library:
372 29 Arnaud Sevin
<pre>
373
make clean all
374
</pre>
375
376 45 Arnaud Sevin
If you did not get any error, CArMA, SuTrA, NAGA and SHESHA are now installed on your machine. You can check that everything is working by launching a GUI to test a simulation:
377 29 Arnaud Sevin
<pre>
378 1 Damien Gratadour
ipython -i $SHESHA_ROOT/widgets/widget_ao.py
379
</pre>