Installation on Frontera

Note: this page is under construction...

Frontera is the new machine at TACC. Installation is very similar to Stampede2.

For details see the Frontera User Guide

Some notes:

• currently only $SCRATCH works, not $WORK.
• running binaries with MPI does not work (hang) unless you use ibrun, for exampleibrun -n X ./aspect
• to connect: ssh [email protected]
• interactive job to compile: idev -m 120
• intel 19.0.4 compilers cause problems on large (1000+ ranks), so we use 18 for now.

Installation steps:

modules:

module reset
module load intel/18.0.5

echo "module load intel/18.0.5" >$SCRATCH/enable.sh

1. candi

cd $SCRATCH
git clone https://github.com/dealii/candi.git
cd candi
echo "MKL=ON" >>candi.cfg
echo "MKL_DIR=$MKLROOT/lib/intel64" >>candi.cfg
./candi.sh -j 50 --packages="once:p4est once:trilinos" -p $SCRATCH/libs
echo ". $SCRATCH/libs/configuration/enable.sh" >>$SCRATCH/enable.sh

2. deal.II

. $SCRATCH/libs/configuration/enable.sh
cd $SCRATCH
git clone https://github.com/dealii/dealii.git
mkdir deal-build
cd deal-build
CC=mpicc CXX=mpicxx cmake \
      -D DEAL_II_WITH_MPI=ON \
      -D DEAL_II_CXX_FLAGS="-march=native" \
      -D DEAL_II_WITH_64BIT_INDICES=ON \
      -D DEAL_II_COMPONENT_EXAMPLES=OFF \
      -D CMAKE_INSTALL_PREFIX=`pwd`/install  \
      ../dealii
make -j 50 install
echo "export DEAL_II_DIR=`pwd`/install" >>$SCRATCH/enable.sh

3. ASPECT

. $SCRATCH/enable.sh
git clone https://github.com/geodynamics/aspect.git
mkdir aspect-build
cd aspect-build
cmake ../aspect
make release
make -j 50

Running:

ibrun -n 32 ./aspect bla.prm