The following is a list of the messages received in the run*.out file and the cause(s) of the problem when running WACCM:
1.) The following two error messages occurred on a regular hybrid mpi/openmp run and the cause was turning off hybrid running and trying to do a hybrid run. The following were in the run script and should have been deleted:
#BSUB -a poe  # use LSF openmp elim
# For totalview
setenv MP_EUILIB ip
setenv MP_PROCS 1
60000 19510101
Total Mass= NaNQ (mb), Dry Mass= NaNQ (mb)
Total Precipitable Water = NaNQ (kg/m**2)
PS max = 1036.24957044650932 min = 553.681181310953889
U max = 228.345004934455886 min = -241.346468117400974
V max = 250.938216790207093 min = -273.790397003314467
T max = 573.628492520743293 min = 97.2009664599987104
W (mb/day) max = 707.629452019978771 min = -1908.42122173505254
Average Height (geopotential units) = NaNQ
PRECC max = 57.1353198264621369 min = 0.000000000000000000E+00
PRECL max = 128.444813059888219 min = 0.000000000000000000E+00
Total precp= 2.59255605465378824 CON= 0.724049951348041843 LS= 1.86850610330574640
ERROR: 0031-250 task 11 Segmentation fault
ERROR: 0031-250 task 15 Segmentation fault
ERROR: 0031-250 task 7 Segmentation fault
ERROR: 0031-250 task 3 Segmentation fault
ERROR: 0031-250 task 2 Terminated
ERROR: 0031-250 task 4 Terminated
ERROR: 0031-250 task 13 Terminated
ERROR: 0031-250 task 5 Terminated
ERROR: 0031-250 task 14 Terminated
ERROR: 0031-250 task 12 Terminated
ERROR: 0031-250 task 8 Terminated
ERROR: 0031-250 task 1 Terminated
ERROR: 0031-250 task 6 Terminated
ERROR: 0031-250 task 10 Terminated
ERROR: 0031-250 task 0 Terminated
ERROR: 0031-250 task 9 Terminated
ATTENTION: 0031-386 MP_INSTANCES setting ignored when LoadLeveler is not being used.
Job /usr/local/lsf/7.0/aix5-64/bin/poejob /blhome/joemci/wa3/model/2x/bld wa3548T01L66Dev_2x
TID HOST_NAME COMMAND_LINE STATUS TERMINATION_TIME
===== ========== ================ ======================= ===================
00000 be0902en blhome/joemci/w Exit (143) 09/16/2008 15/11/23