This page contains some notes on running Cactus on the IA64 Linux Cluster at the NCSA.
The web pages for Titan are at
http://www.ncsa.uiuc.edu/UserInfo/Resources/Hardware/IA64LinuxCluster/
If you have an account on Titan you will be able to logon with your usual NCSA (that is modi4) username and password. If you find you don't have an account, fill out the form linked here.
ssh titan.ncsa.uiuc.edu -l <user_name>
Note: If your logon node is really busy you should try the other ones.
There are three interactive nodes in total
(user0[1-3]@.ncsa.uiuc.edu).
I like to use this node for interactive benchmarks...
The standard compilation options for Titan can be found on our Machine Configurations page.
Single processor (non-MPI) jobs can be run as usual interactively.
For MPI jobs you have to first request processors, using
qsub -I -V -l walltime=<job runtime in
HH:MM:SS>,nodes=<#nodes>:ppn=<#procs per node>:prod
Then if you get some free nodes you can then run interactive jobs using
vmirun [-np <total #procs>] cactus_<config> <parameter file>
You can use the qs2 if you like script to submit batch jobs. The only queue for standard use is the standard queue, which has in principle 160 nodes. Typically other queues are not available for use.
Titan has a web based cluster monitor http://padmin1.ncsa.uiuc.edu/ which is handy for checking the status of the queue.
You may run into some problems with the standard Intel compilers,
e.g. they cannot compile snprintf.c without the -ip
option. It is suggested to user the beta 7.0 compilers instead.
To use the beta 7.0 compilers instead of the standard ones, simply
follow the instructions found in /usr/news/intel7b.
The following thorns have been tested with the beta 7.0 compilers, using MPICH on two processors:
| Thorn | last checked | failing |
|---|---|---|
| CactusBase/Boundary | 22 Nov 2002 | |
| CactusBase/CartGrid3D | 22 Nov 2002 | |
| CactusBase/IOASCII | 22 Nov 2002 | |
| CactusBase/IOBasic | 22 Nov 2002 | |
| CactusBase/IOUtil | 22 Nov 2002 | |
| CactusBase/Time | 22 Nov 2002 | |
| CactusBase/LocalInterp | 29 Oct 2002 | |
| CactusConnect/HTTPD | 22 Nov 2002 | |
| CactusConnect/HTTPDExtra | 29 Oct 2002 | |
| CactusConnect/Socket | 22 Nov 2002 | |
| CactusEinstein/ADM | 22 Nov 2002 | test_ADMadj |
| CactusEinstein/ADMAnalysis | 22 Nov 2002 | 4BL_holes |
| CactusEinstein/ADMBase | 22 Nov 2002 | |
| CactusEinstein/ADMConstraints | 22 Nov 2002 | |
| CactusEinstein/ADMMacros | 22 Nov 2002 | |
| CactusEinstein/AHFinder | 22 Nov 2002 | |
| CactusEinstein/CoordGauge | 22 Nov 2002 | |
| CactusEinstein/Extract | 22 Nov 2002 | |
| CactusEinstein/IDAnalyticBH | 22 Nov 2002 | |
| CactusEinstein/IDAxiBrillBH | 29 Oct 2002 | |
| CactusEinstein/IDBrillData | 29 Oct 2002 | |
| CactusEinstein/IDSimple | 29 Oct 2002 | |
| CactusEinstein/Maximal | 22 Nov 2002 | test_maximal_1 |
| CactusEinstein/PsiKadelia | 29 Oct 2002 | |
| CactusEinstein/SpaceMask | 22 Nov 2002 | |
| CactusEinstein/StaticConformal | 22 Nov 2002 | |
| CactusEinstein/ADMCoupling | 22 Nov 2002 | |
| CactusElliptic/EllBase | 22 Nov 2002 | |
| CactusElliptic/EllSOR | 22 Nov 2002 | |
| CactusExternal/FlexIO | 22 Nov 2002 | |
| CactusExternal/jpeg6b | 22 Nov 2002 | |
| CactusIO/IOJpeg | 29 Oct 2002 | |
| CactusPUGH/PUGH | 22 Nov 2002 | |
| CactusPUGH/PUGHInterp | 22 Nov 2002 | |
| CactusPUGH/PUGHReduce | 22 Nov 2002 | |
| CactusPUGH/PUGHSlab | 22 Nov 2002 | |
| CactusPUGHIO/IOFlexIO | 22 Nov 2002 | |
| CactusPUGHIO/IOHDF5 | 22 Nov 2002 | |
| CactusPUGHIO/IOHDF5Util | 22 Nov 2002 | |
| CactusPUGHIO/IsoSurfacer | 29 Oct 2002 | |
| CactusPUGHIO/IOStreamedHDF5 | 29 Oct 2002 | |
| CactusUtils/NaNChecker | 22 Nov 2002 | |
| CactusWave/IDScalarWaveC | 22 Nov 2002 | |
| CactusWave/WaveBinarySource | 18 Nov 2002 | |
| CactusWave/WaveToyC | 22 Nov 2002 | |
| CactusEOS/EOS_Base | 22 Nov 2002 | |
| CactusEOS/EOS_Ideal_Fluid | 22 Nov 2002 | |
| AlphaThorns/FileAdvertiser | 29 Oct 2002 | |
| AlphaThorns/Nice | 22 Nov 2002 | |
| BetaThorns/Cartoon2D | 29 Oct 2002 | |
| Lazarus/Zorro | 29 Oct 2002 | test_zorro_1 test_zorro_2 |
| Excision/SimpleExcision | 29 Oct 2002 | test_SimpleExcision_1 |
| Excision/LegoExcision | 29 Oct 2002 | |
| AEIThorns/ADM_BSSN | 29 Oct 2002 | |
| AEIThorns/BAM_Elliptic | 22 Nov 2002 | |
| AEIThorns/BAM_VecLap | 22 Nov 2002 | test_bam_veclap_minshift |
| AEIThorns/IOAHFinderHDF5 | 29 Oct 2002 | |
| AEIThorns/Exact | 22 Nov 2002 | |
| AEIThorns/IDConstraintViolate | 29 Oct 2002 | ConstraintViolate |
| AEIDevelopment/Embedding | 29 Oct 2002 | |
| AEIDevelopment/FishEye | 29 Oct 2002 | |
| AEIDevelopment/Announce | 22 Nov 2002 | |
| AEIDevelopment/EHFinder | 29 Oct 2002 | eh_test1 |
| AEIDevelopment/BSSN_MoL | 22 Nov 2002 | |
| AEIDevelopment/AHFSetDCCentroid | 22 Nov 2002 | |
| AEIDevelopment/DriftCorrect | 22 Nov 2002 | |
| CactusMoL/MoL | 22 Nov 2002 | |
| EUHydro/EOS_Polytrope | 22 Nov 2002 | |
| EUHydro/Whisky | 22 Nov 2002 | whisky_test_RotNS whisky_test_TOV |
| EUHydro/Whisky_Init_Data | 22 Nov 2002 | |
| EUHydro/Whisky_IVP | 22 Nov 2002 | |
| EUHydro/Whisky_TOVSolver | 22 Nov 2002 | |
| EUHydro/Whisky_TOVSolverC | 22 Nov 2002 |
If you don't see your favorite thorn there, just let me (David) know and I'll try to test it out.
This page last modified: $Date: 2004/03/02 14:14:38 $