WRFv4.1.1Compiler Test
We observed differences in answers for WRF-CMAQ when running different domain decompositions. To isolate the issue to CMAQ, I ran a stand-alone WRFv4.1.1 test to determine if answers in wrf output file depend on # processors used.
module load openmpi_4.0.1/gcc_9.1.0
Followed this tutorial to download and compile WRFv4.1.1
For the QA, I will use a tool provided by David Wong at EPA from the scienceftp site. It is currently installed on dogwood under: /proj/ie/proj/CMAS/WRF-CMAQ/from_fahim/dpp.tar.gz
The tool is dpp (you can type dpp -h to find out how to use it). Modify Makefile according and modify the dpp script to point to the location of the executable. In your case, you can use it in this way:
dpp -dif file1 file2 > diff.out
I had to use the intel compiler to build dpp. Location of code, and readme contains tips on how to run. /proj/ie/proj/CMAS/WRFv4.1.1/openmpi_4.0.1_gcc_9.1.0/from_fahim/dpp The gcc compile was not successful.
readme:
need to compile with intel
use openmpi_3.1.4/intel_18.2
edit .cshrc to use
setenv NCF_COMBO /proj/ie/proj/CMAS/WRF-CMAQ/openmpi_3.1.4_intel_18.2/Build_WRF/LIBRARIES/netcdf_combined
setenv LD_LIBRARY_PATH ${NCF_COMBO}/lib:${LD_LIBRARY_PATH}
The report showed that the difference between the two wrf output files was zero for all variables.
I repeated this test to confirm there is no variation output as you change the number of processors used for the WRFv4.1.1-CMAQv5.3.2 code (to account for any modifications created when running twoway code to couple WRF with CMAQ). I used the following runtime option.
setenv RUN_CMAQ_DRIVER F # [F]
The diff.out results look like this:
ncd_64bit_offset | T
option_type | 2
num_file | 1
spc | F
mynsteps | -99
mytinc | 1 (default)
convert | 0
ctime | 15:16:55
writeout | F
num_append_step | 0
rscale | 1 (default)
arm_time_interva | 60 (default)
frestart | F
mysdate | -99
ofile | ./output_12km_sf_rrtmg_20_5_1_v411532.nocmaq.32pe/wrfout_d01_2016-07-01_00:00:00
time: 2016-07-01_00:00:00
XLAT total diff: 0
XLONG total diff: 0
LU_INDEX total diff: 0
ZNU total diff: 0
ZNW total diff: 0
ZS total diff: 0
DZS total diff: 0
VAR_SSO total diff: 0
U total diff: 0
V total diff: 0
W total diff: 0
PH total diff: 0
PHB total diff: 0
T total diff: 0
THM total diff: 0
HFX_FORCE total diff: 0
LH_FORCE total diff: 0
TSK_FORCE total diff: 0
HFX_FORCE_TEND total diff: 0
LH_FORCE_TEND total diff: 0
TSK_FORCE_TEND total diff: 0
MU total diff: 0
MUB total diff: 0
NEST_POS total diff: 0
P total diff: 0
