The prerequisites needed to build this repository can be found here.

Build Instructions

Dependencies

External dependencies

cmake >= 3.26
MPI
C++17 compiler (information on supported compilers here here.)
CUDA >= 11.7 (Required only for CUDA builds)
ROCM >= 5.5 (Required only for ROCM builds)

The remaining dependencies are automatically built and do not need to be installed explicitly:

GlobalArrays
HPTT, Librett
HDF5
BLAS/LAPACK (BLIS and netlib-lapack are automatically built if no vendor BLAS libraries are provided)
BLAS++ and LAPACK++
Eigen3, doctest

Choose Build Options

CUDA Options

-DTAMM_ENABLE_CUDA=ON (OFF by default)
One of -DGPU_ARCH=X (OR) -DCMAKE_CUDA_ARCHITECTURES=X is required. Set the arch value X to 70 for Volta, 80 for Ampere, 90 for Hopper and 95 for Blackwell.

HIP Options

-DTAMM_ENABLE_HIP=ON (OFF by default)
-DROCM_ROOT=$ROCM_PATH
One of -DGPU_ARCH=gfx90a (OR) -DCMAKE_HIP_ARCHITECTURES=gfx90a is required.

DPCPP options

-DTAMM_ENABLE_DPCPP=ON (OFF by default)

CMake options for developers (optional)

-DUSE_GA_PROFILER=ON #Enable GA's profiling feature (GCC Only).

-DMODULES="CC" (empty by default)

Building TAMM

export REPO_ROOT_PATH=$HOME/TAMM
export REPO_INSTALL_PATH=$HOME/tamm_install

git clone <repo-url> $REPO_ROOT_PATH
cd $REPO_ROOT_PATH
# Checkout the branch you want to build
mkdir build && cd build

In addition to the build options chosen, there are various build configurations depending on the BLAS library one wants to use.

Default build using BLIS and NETLIB LAPACK
Default build on MACOS
Build using Intel MKL
Build instructions for Frontier
Build instructions for Perlmutter and Polaris
Build instructions for Aurora

Default build using BLIS and NETLIB LAPACK

To enable CUDA build, add -DTAMM_ENABLE_CUDA=ON and -DGPU_ARCH=<value>

cd $REPO_ROOT_PATH/build
CC=gcc CXX=g++ FC=gfortran cmake -DCMAKE_INSTALL_PREFIX=$REPO_INSTALL_PATH ..

make -j3
make install

Default build on MACOS

Note

The prerequisites for MACOS can be installed using brew as detailed here.

cd $REPO_ROOT_PATH/build
FC=gfortran cmake -DCMAKE_INSTALL_PREFIX=$REPO_INSTALL_PATH -DLINALG_VENDOR=OpenBLAS ..

make -j3
make install

Build using Intel MKL

To enable CUDA build, add -DTAMM_ENABLE_CUDA=ON and -DGPU_ARCH=<value>

cd $REPO_ROOT_PATH/build

CC=gcc CXX=g++ FC=gfortran cmake -DLINALG_VENDOR=IntelMKL \
-DLINALG_PREFIX=/opt/intel/mkl \
-DCMAKE_INSTALL_PREFIX=$REPO_INSTALL_PATH ..

make -j3
make install

Build instructions for Frontier

module load cpe
module load cray-python cmake cray-hdf5-parallel
module load cce
module load cray-mpich
module load rocm
export CRAYPE_LINK_TYPE=dynamic

cd $REPO_ROOT_PATH/build

CC=cc CXX=CC FC=ftn cmake \
-DCMAKE_INSTALL_PREFIX=$REPO_INSTALL_PATH \
-DGPU_ARCH=gfx90a \
-DTAMM_ENABLE_HIP=ON -DROCM_ROOT=$ROCM_PATH \
-DHDF5_ROOT=$HDF5_ROOT ..

make -j3
make install

Build instructions for Perlmutter and Polaris

Perlmutter modules and env

module load PrgEnv-gnu
module load cmake
module load cpe-cuda
module load cudatoolkit
module unload craype-accel-nvidia80

export CRAYPE_LINK_TYPE=dynamic
export MPICH_GPU_SUPPORT_ENABLED=0

Polaris modules and env

module use /soft/modulefiles/
module load PrgEnv-gnu
module load cudatoolkit-standalone/12.6.1 spack-pe-base cmake
module unload craype-accel-nvidia80

export CRAYPE_LINK_TYPE=dynamic
export MPICH_GPU_SUPPORT_ENABLED=0

Common build steps

cd $REPO_ROOT_PATH/build

cmake -DTAMM_ENABLE_CUDA=ON -DGPU_ARCH=80 -DBLIS_CONFIG=generic \
-DCMAKE_INSTALL_PREFIX=$REPO_INSTALL_PATH ..

make -j3
make install

Build instructions for Aurora

module restore
module load cmake python

cd $REPO_ROOT_PATH/build

CC=icx CXX=icpx FC=ifx cmake \
-DCMAKE_INSTALL_PREFIX=$REPO_INSTALL_PATH \
-DLINALG_VENDOR=IntelMKL -DLINALG_PREFIX=$MKLROOT \
-DTAMM_ENABLE_DPCPP=ON \
-DTAMM_CXX_FLAGS="-march=sapphirerapids -mtune=sapphirerapids -ffast-math -fsycl -fsycl-device-code-split=per_kernel -fsycl-targets=intel_gpu_pvc -sycl-std=2020"

make -j12
make install