Sunday, February 20, 2011

Installing BLAS/Lapack/ITPP on Amazon EC2/Ubuntu Linux

BLAS/Lapack are efficient matrix math libraries. The following instructions explains how to install them for Amazon EC2 (Ubuntu maverick version, and Amazon Linux). It++ (itpp) is a popular c++ wrapper for blas/lapack.

DISLAIMER: The below instructions are for 64 bit machines. For 32 bit machines follow other instructions: http://bickson.blogspot.com/2011/06/graphlab-pmf-on-32-bit-linux.html

FOR LAZY READERS:
Just use Amazon EC2 public image ami-c21eedab (Ubuntu)

INSTALLATION VIA YUM/APT-GET
Try to install itpp using the following command:
sudo yum install libitpp-dev
Or
sudo apt-get install libitpp-dev
TIP: You may also want to install libitpp7-dbg using the yum/apt-get command.
It is not mandatory, but it helps debugging when you link against libitpp_debug.so
(instead of libitpp.so).

If the above worked then we are done. If not, you will need to follow
instructions below. Thanks to Udi Weinsberg for this tip.

FOR ADVANCED USERS:


0) Start with an Ubuntu image like ami-641eed0d, or an Amazon AMI image like:

1) Install required packages. For Ubuntu:
sudo apt-get install --yes --force-yes automake autoconf libtool* gfortran
For Amazon Linux:
sudo yum install -y automake autoconf libtool* gcc-gfortran

2) Install lapack.
Here again their are two options:

The easy way is to simply (On Ububtu)
sudo apt-get install --yes --force-yes liblapack-dev
On Amazon Linux:
sudo yum install -y lapack-devel blas-devel

Thanks Akshay Bhat from Cornell for this tip!
If the liblapack setup was successful, go to step 3.

If the above command DOES NOT work for you (depends on your OS and setup) you will need to install lapack manually. The procedure is explained in steps a-c below.

a) Download and prepare the code
wget http://www.netlib.org/lapack/lapack.tgz
tar xvzf lapack.tgz
cd lapack-3.3.0  //if version number changes, change here to the right directory
mv make.inc.example make.inc

b) edit make.inc and add -m64 -fPIC flag to fortran compiler options:
# FORTRAN, OPTS, NOOPT, LOADER

c) compile
make blaslib
make
If everthing went OK, test will be run for a couple of minutes
and the files blas_LINUX.a and lapack_LINUX.a will be created at the main directory

3) setup LDFLAGS
export LDFLAGS="-L/usr/lib -lgfortran"

4) Download and install itpp from
wget http://sourceforge.net/projects/itpp/files/itpp/4.2.0/itpp-4.2.tar.gz
tar xvzf itpp-4.2.tar.gz
cd itpp-4.2
./autogen.sh 
If you installed Lapack from yum/apt-get, you should use the following command:
./configure --without-fft --with-blas=/usr/lib64/libblas.so --with-lapack=/usr/lib64/liblapack.so --enable-debug CFLAGS=-fPIC CXXFLAGS=-fPIC CPPFLAGS=-fPIC
Where /usr/lib64/ is the place where lapack was installed.

If you installed lapack from source, use the following command
./configure --without-fft --with-blas=/home/ubuntu/lapack-3.3.0/blas_LINUX.a --with-lapack=/home/ubuntu/lapack-3.3.0/lapack_LINUX.a CFLAGS=-fPIC CXXFLAGS=-fPIC CPPFLAGS=-fPIC

make
sudo make install
Note: If you installed lapack from yum/apt-get, don't forget to add the -lblas -llapack linker flag when you compile against lapack/blas.

Verifying installation
To verify that installation went Ok, run the following commands:
itpp-config --cflags
itpp-config --libs
1) The command itpp-config should be available from shell.
2) The right installation path should appear as output.

Known issues you may encounter:
Problem:
*** Warning: Linking the shared library libitpp.la against the^M
*** static library /usr/lib64/libblas.a is not portable!^M
libtool: link: g++ -shared -nostdlib /usr/lib/gcc/x86_64-amazon-linux/4.4.4/../../../../lib64/crti.o /usr/lib/gcc/x86_64-amazon-linux/4.4.4/crtbeginS.o  -Wl,--whole-archive ../itpp/base/.libs/libbase.a ../itpp/stat/.libs/libstat.a ../itpp/comm/.libs/libcomm.a ../itpp/fixed/.libs/libfixed.a ../itpp/optim/.libs/liboptim.a ../itpp/protocol/.libs/libprotocol.a ../itpp/signal/.libs/libsignal.a ../itpp/srccode/.libs/libsrccode.a -Wl,--no-whole-archive  -L/usr/lib64/ /usr/lib64/liblapack.a /usr/lib64/libblas.a -lgfortranbegin -lgfortran -L/usr/lib/gcc/x86_64-amazon-linux/4.4.4 -L/usr/lib/gcc/x86_64-amazon-linux/4.4.4/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-amazon-linux/4.4.4/../../.. -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/x86_64-amazon-linux/4.4.4/crtendS.o /usr/lib/gcc/x86_64-amazon-linux/4.4.4/../../../../lib64/crtn.o    -Wl,-soname -Wl,libitpp.so.7 -o .libs/libitpp.so.7.0.0^M
/usr/bin/ld: /usr/lib64/liblapack.a(dgees.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC^M
/usr/lib64/liblapack.a: could not read symbols: Bad value^M
collect2: ld returned 1 exit status^M
make[2]: *** [libitpp.la] Error 1^M
Solution: it seems that lapack was statically compiled without the -fPIC option and thus itpp refuses to link against it. Follow step 2a to install lapack manaually with the -fPIC option.

Problem:
make[1]: *** Waiting for unfinished jobs....
[ 83%] Building CXX object src/graphlab/CMakeFiles/
graphlab_pic.dir/distributed2/distributed_scheduler_list.o /usr/local/lib/libitpp.so: undefined reference to `zgesv_' /usr/local/lib/libitpp.so: undefined reference to `dorgqr_' /usr/local/lib/libitpp.so: undefined reference to `dswap_' /usr/local/lib/libitpp.so: undefined reference to `dgeqp3_' /usr/local/lib/libitpp.so: undefined reference to `dpotrf_' /usr/local/lib/libitpp.so: undefined reference to `dgemm_' /usr/local/lib/libitpp.so: undefined reference to `zungqr_' /usr/local/lib/libitpp.so: undefined reference to `zscal_' /usr/local/lib/libitpp.so: undefined reference to `dscal_' /usr/local/lib/libitpp.so: undefined reference to `dgesv_' /usr/local/lib/libitpp.so: undefined reference to `dgetri_' /usr/local/lib/libitpp.so: undefined reference to `zgemm_' /usr/local/lib/libitpp.so: undefined reference to `zposv_' /usr/local/lib/libitpp.so: undefined reference to `zgetri_' /usr/local/lib/libitpp.so: undefined reference to `dgeev_' /usr/local/lib/libitpp.so: undefined reference to `zgemv_' /usr/local/lib/libitpp.so: undefined reference to `zgeqrf_' /usr/local/lib/libitpp.so: undefined reference to `zgerc_' /usr/local/lib/libitpp.so: undefined reference to `zswap_' /usr/local/lib/libitpp.so: undefined reference to `zgeev_' /usr/local/lib/libitpp.so: undefined reference to `daxpy_' /usr/local/lib/libitpp.so: undefined reference to `dgetrf_' /usr/local/lib/libitpp.so: undefined reference to `zgels_' /usr/local/lib/libitpp.so: undefined reference to `zgetrf_' /usr/local/lib/libitpp.so: undefined reference to `dgees_' /usr/local/lib/libitpp.so: undefined reference to `dcopy_' /usr/local/lib/libitpp.so: undefined reference to `dger_' /usr/local/lib/libitpp.so: undefined reference to `dgels_' /usr/local/lib/libitpp.so: undefined reference to `dgeqrf_' /usr/local/lib/libitpp.so: undefined reference to `zpotrf_' /usr/local/lib/libitpp.so: undefined reference to `zgees_' /usr/local/lib/libitpp.so: undefined reference to `dgesvd_' /usr/local/lib/libitpp.so: undefined reference to `zgeru_' /usr/local/lib/libitpp.so: undefined reference to `dsyev_' /usr/local/lib/libitpp.so: undefined reference to `zaxpy_' /usr/local/lib/libitpp.so: undefined reference to `ddot_' /usr/local/lib/libitpp.so: undefined reference to `zgesvd_' /usr/local/lib/libitpp.so: undefined reference to `zgeqp3_' /usr/local/lib/libitpp.so: undefined reference to `zcopy_' /usr/local/lib/libitpp.so: undefined reference to `dgemv_' /usr/local/lib/libitpp.so: undefined reference to `dposv_' /usr/local/lib/libitpp.so: undefined reference to `zheev_' collect2: ld returned 1 exit status make[2]: *** [tests/anytests] Error 1 make[1]: *** [tests/CMakeFiles/anytests.dir/all] Error 2
Solution: itpp was compiled using dynamic libraries, but your application did not include the -lblas and -llapack link flags.

Problem:
*** Error: You must have "autoconf" installed to compile IT++ SVN sources
*** Error: You must have "automake" installed to compile IT++ SVN sources
*** Error: You must have "libtoolize" installed to compile IT++ SVN sources
Solution:
Need to install the packages autoconf, automake and libtoolize. See yum/apt-get documentation.

Problem:
/usr/bin/ld: /home/bickson/lapack-3.3.1/lapack_LINUX.a(dgees.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/home/bickson/lapack-3.3.1/lapack_LINUX.a: could not read symbols: Bad value
collect2: ld returned 1 exit status
Solution:
It seem you forgot to follow section 2b.

TIP: It is useful to enable also itpp_debug library which is very useful when debugging your code. This is done by adding the flag --enable-debug to the configure script.

Problem:
*** Error in ../../../itpp/base/algebra/ls_solve.cpp on line 271:
LAPACK library is needed to use ls_solve() function
Solution:
It seems that itpp is not installed properly -it did not link to lapack.

6 comments:

  1. i try to install itpp but i receive this error when i make $make
    ./../itpp/base/copy_vector.h:61: error: ‘memcpy’ was not declared in this scope
    In file included from ../../itpp/base/vec.h:459,
    from ../../itpp/base/bessel.h:36,
    from ../../itpp/base/bessel.cpp:33:
    ../../itpp/base/mat.h: In member function ‘bool itpp::Mat::set(const char*)’:
    ../../itpp/base/mat.h:597: error: ‘EOF’ was not declared in this scope
    In file included from ../../itpp/base/bessel.h:36,
    from ../../itpp/base/bessel.cpp:33:
    ../../itpp/base/vec.h: In member function ‘bool itpp::Vec::set(const char*)’:
    ../../itpp/base/vec.h:536: error: ‘EOF’ was not declared in this scope
    make[4]: *** [libbase_la-bessel.lo] Error 1
    make[4]: Leaving directory `/home/mai/ns-allinone-2.30/ns-2.30/itpp-3.10.7/itpp/base'
    make[3]: *** [all-recursive] Error 1
    make[3]: Leaving directory `/home/mai/ns-allinone-2.30/ns-2.30/itpp-3.10.7/itpp/base'
    make[2]: *** [all-recursive] Error 1
    make[2]: Leaving directory `/home/mai/ns-allinone-2.30/ns-2.30/itpp-3.10.7/itpp'
    make[1]: *** [all] Error 2
    make[1]: Leaving directory `/home/mai/ns-allinone-2.30/ns-2.30/itpp-3.10.7/itpp'
    make: *** [all-recursive] Error 1


    please tell me what shall i do to solve this error

    ReplyDelete
    Replies
    1. Something is wrong with your machine setup - those system calls should be automatically recognized. Maybe you updated the compiler without updating c libraries?

      Delete
  2. Hi, thank you for your notes. I tried to install lapack following the steps above. Every step went ok, or at least there where no error messages. I installed lapack by using apt-get. The last step, to verify that installation went ok, I get the following:
    user@local:~/itpp-4.2$ itpp-config --cflags
    user@local:~/itpp-4.2$ -I/usr/local/include -fPIC
    user@local:~/itpp-4.2$ itpp-config --libs
    user@local:~/itpp-4.2$ -L/usr/local/lib -litpp

    Then I went to the directory where I have the file (in fortran77) to be compiled:
    user@local:~/fortran-files$ gfortran -llapack -O4 file.f
    I got the following message
    user@local:~/fortran-files$ /tmp/ccb8uFsZ.o:file.f:(.text+0x7773): more undefined references to `zheev_' follow
    collect2: ld returned 1 exit status


    I will appreciate if you could give me a hint how to fix this.

    Thank you very much for your attention.
    p.s. I have ubuntu 12.04 64bit.
    best regards
    Ronald

    ReplyDelete
    Replies
    1. Hi Ronald,
      The answer is written above ;-)
      Solution: itpp was compiled using dynamic libraries, but your application did not include the -lblas and -llapack link flags.
      You need to also verify that LD_LIBRARY_PATH is defined and pointing to your lapack libraries.

      Best,

      Delete
  3. Hi, Danny
    My lalack is installed in /usr/lib (instead of /usr/lib64). So in step 4), I changed the path to /usr/lib. After running the ./configure..., I did not get the link flags -lblas and -llapack. In stead, I got LIBS : /usr/lib/liblapck.so /usr/lib/libblas.so.
    Is this the correct one?
    How should I include the library in the makefile to be able to use routines in lapack?
    I appreciate it.
    Best

    ReplyDelete
  4. Hi,
    When you configure lapack, you should point to the correct location of libblas and liblapack:
    ./configure --without-fft --with-blas=/usr/lib/libblas.so --with-lapack=/usr/lib/liblapack.so --enable-debug CFLAGS=-fPIC CXXFLAGS=-fPIC CPPFLAGS=-fPIC

    Best,

    ReplyDelete