CMAQ version 5.2 DDM Technical Documentation
Release Notes for CMAQv5.2_DDM - Target: September 2017
The Community Multiscale Air Quality (CMAQ) modeling system Version 5.2 DDM is being tested prior to release
The CMAQv5.2 DDM official release repository will be available from the U.S. EPA GitHub site.
Markdown draft version of the documentation for CMAQv5.2 DDM, including build instructions and technical features is currently available from [1].
Release Testing
The CMAS Center will test the CMAQv5.2 DDM release package with the Portland Group, Intel, and GNU Fortran compilers. The CMAQv5.2 Technical Notes contains timing information for the base release. The results of the CMAS Center tests for the CMAQv5.2 DDM release, including run times for different compilers and configurations will be provided below. The CMAQv5.2 DDM tests were conducted with the U.S. EPA benchmark domain testing dataset for a single day: July 1, 2011.
DDM changes to bldit_cctm.csh:
set Mechanism = cb6r3_ae6nvPOA_aq set Ddm3d = ddm3d set sens = ( -Dsens )
DDM changes to run_cctm.csh:
set MECH = cb6r3_ae6nvPOA_aq setenv CTM_ABFLUX N setenv CTM_GRAV_SETL N
sensinput.cmas.dat contains:
ENX EMIS GRID 1 EMIS_1 SPECIES NO2 NO ESX EMIS GRID 1 EMIS_1 SPECIES SO2 END
See example run script for all of the settings under this heading:
- # ===================================================================
- #> CMAQ-DDM-3D
- # ===================================================================
set NPMAX = 2 setenv SEN_INPUT <path>/sensinput.cmas.dat setenv DDM3D_HIGH N # allow higher-order sensitivity parameters setenv DDM3D_RGN N # allows for a file specifying regions setenv REGIONS_1 setenv DDM3D_BCRGN N # allows for a file specifying boundary regions setenv BCREGIONS_1 setenv DDM3D_ES Y # emissions split into categories setenv DDM3D_BCS F # use sensitivity bc file for nested runs
Compiler flags:
PGI: -Mfixed -O3 -Mextend GCC: -ffixed-form -ffixed-line-length-132 -O3 -funroll-loops -finit-character=32 Intel: -fixed -132 -O3 -override-limits -fno-alias -mp1 -fp-model precise -fp-model source -shared-intel -openmp In the Intel Basic Test: -fixed -132 -O3 -openmp In the NoOpt Tests: -O0 with extend source and fixed line length flags
Scatter plots
Concentration at the location of the domain-wide max of the absolute value of the difference for each timestep between the sensitivity cases and the base case for CMAQ v5.2 DDM which is using mvapich and the intel compiler version 15.0 (mvapich2_intel15.0).
Sensitivity cases include:
Github release version of the CMAQ v5.2 code with matching MECH and flags to the DDM case on 16 pe.
CCTM_v52_gcc_SE52BENCH_3
CMAQ v5.2 code + DDM code, and matching MECH and flags to the DDM case, but with the compile done with the noDDM option on 16 pe.
CCTM_v52_gcc_SE52BENCH_ddm_noDDM
CMAQ v5.2 code + DDM code, compiled with DDM option turned on
mvapich2_intel15.0_base (16 pe) mvapich2_gcc4.8.1_16pe mvapich2_gcc4.8.1_32pe openmpi_gcc_4.8.1_16pe mvapich2_pgi15.7_16pe serial_mvapich2_pgi15.7_1pe
Google Drive location of DDM Testing Plots
UNC module pgi/16.1| Scenario | Compiler | netCDF | I/O API | MPI_YN (#P) | MPI | CMAQv5.2 Timing (HH:MM:SS) |
CMAQv5.2 DDM Timing (HH:MM:SS) |
Notes |
|---|---|---|---|---|---|---|---|---|
| Gfortran openmpi | Gfortran version 4.8.1 | 4.1.1 | 3.1 (Nov 2015) | Y (16) | openmpi | 34:13.15 | 1:45:33 | UNC module openmpi_gcc/4.8.1 |
| Gfortran MVAPICH2 | Gfortran version 4.8.1 | 4.1.1 | 3.1 (Nov 2015) | Y (16) | mvapich2-1.7 | 1:07:23 | UNC module mvapich2_gcc/4.8.1 | |
| Intel MVAPICH2 | Intel Fortran version 15.0 | 4.1.1 | 3.1 (Nov 2015) | Y (16) | mvapich2-1.7 | 1:33:10 | UNC module mvapich2_intel/15.0 | |
| Portland Serial | PG Fortran version 16.1 | 4.1.1 | 3.1 (Nov 2015) | N | N/A | 31:24:04 | UNC module pgi/16.1 | |
| Portland MVAPICH2 | PGI Fortran version 15.7 | 4.1.1 | 3.1 (Nov 2015) | Y (16) | mvapich2-1.7 | 2:11:40 | UNC module mvapich2_pgi/15.7 | |
| Portland MVAPICH2 | PGI Fortran version 15.7 | 4.1.1 | 3.1 (Nov 2015) | Y (32) | mvapich2-1.7 | 1:07:22 | UNC module mvapich2_pgi/15.7 |
