- PLEASE READ THIS BEFORE RUNNING SMOKE. ***
- 1 Introduction
- 2 Requirements for processing emissions for air quality modeling
- 3 Description of packages
- 4 Installation of data files and scripts
- 5 Directory Structure
- 6 Case description and instructions for each sector
- 7 Run instructions for each sector
- 8 Description of ancillary file packages
- 9 Description of script packages
- 10 Source apportionment applications
- 11 CAMx conversion
- 12 Other miscellaneous programs
These packages contain scripts, inventories, and ancillary files related to the 2011ek_cb6v2_v6_11g, 2017ek_cb6v2_v6_11g, 2017ek_ussa_cb6v2_v6_11g, and 2017ek_cntl_cb6v2_v6_11g cases, which serve as the basis for EPA's 2011v6.3 platform for air quality modeling. This platform is being used for assessing ozone transport within the continential U.S. and potentially for other studies.
CMAQ and CAMx model-ready emissions generated using these packages with SMOKE v3.7 should be identical to those used in EPA's 2011v6.3 platform. However, there may be emissions differences resulting from differences in the Linux operating system, hardware platform, and other system-specific differences.
Additional information on the 2011v6.3 platform is available here: http://www.epa.gov/ttn/chief/emch/index.html#2011
The data files are divided into several directories: - 2011emissions contains emission inventories for the year 2011 including national CEMS emissions, nonroad inventories and onroad activity data, and point and nonpoint inventories - 2017emissions contains emissions inventories for the year 2017 including the 2017 control case - ancillary_inputs contains general ancillary files (ge_dat), including those related to speciation, spatial allocation (gridding), temporalization, and a gridded ocean chlorine emissions file which is included in the final model-ready emissions - smoke contains scripts to run SMOKE and utility programs - spatial_surrogates contains 12km, 36km, and 4km spatial surrogates
Section 4 of this README includes information about the modeling sectors used in the platform. Section 5 of this README includes information about the inventories provided. Section 6 of this README includes information about the ancillary (non-inventory) files included. Section 8 of this README includes information about running SMOKE for source apportionment applications in CMAQ or CAMx.
Requirements for processing emissions for air quality modeling
If you are only reviewing inventories and not developing emissions for air quality modeling, you do not need to install SMOKE or to follow the instructions below. Instead, unzip the files with the data of interest and examine those and the corresponding reports that are provided.
If you plan to develop emissions for air quality modeling, SMOKE v3.7 is REQUIRED to process this case. SMOKE v3.7 includes additional updates and bug fixes as compared to versions 3.6 and 3.6.5, and so SMOKE v3.7 should be used when processing the 2011v6.3 set of emissions cases.
Python: We also recommend (if not require) python version 2.6 or later, along with select python libraries. Many of the helper scripts included in this package use python. The python scripts within this package reference '#!/usr/bin/env python' or '#!/usr/bin/env python2.6'; you may need to change this on your computing platform.
Description of packages
Base Year Inventory Packages
Inventories for 2011ek are included in the following files, all of which should be unpackaged in INSTALL_DIR:
The "biogenics" package includes gridded land use, BIOSEASON, and biogenic emission factor files for input to BEIS 3.6.1. The latest version of BEIS uses different land use and emission factor inputs compared to prior versions, such as BEIS 3.14. The 2011v6.3 platform includes updated land use - BELD 4.1 - compared to 2011v6.2. This package also includes a b3grd file for 12US2. This file is output by the Normbeis program, and is included in the package for those people who do not wish to run Normbeis and generate their own b3grd using the provided land use and emission factor files. This b3grd file is not speciation-specific and can be used with both CB05 and CB6.
The "cem" package includes the hourly CEM (Continuous Emissions Monitoring) emissions used by the ptegu sector. This is the same data as is available on EPA's Air Markets Program Data website (ampd.epa.gov), except that we've split the data into months and days as needed for our scripts. For 2017ek, the "cem" package includes two separate sets of CEMs: one for summer, and one for winter.
The "inputs_nonpoint" package includes inventories for the following sectors: afdust, ag, agfire, cmv, nonpt, np_oilgas, rail, rwc.
The two "inputs_nonroad" packages include the inventories for the nonroad sector. It was split into two .zips in order to reduce the size of each individual .zip. The "part1" zip includes all states from Alabama to Missouri, alphabetically, except for California (FIPS 01 through 29, except 06). The "part2" zip includes all states from Montana to Wyoming, alphabetically, and also California (FIPS 30 through 56, plus 06). Part 2 also includes Puerto Rico and U.S. Virgin Islands.
The "inputs_onroad" package includes the activity data for the onroad and onroad_catx_adj sectors. It does not include the emission factor tables also required to run SMOKE-MOVES; these are not stored on the FTP site due to their large size and instead are furnished upon request. CB05 and CB6 each have a separate set of emission factor tables.
The "inputs_oth" package includes the inventories for Canada, Mexico, and offshore emissions. These are used by the othafdust, othar, othon, and othpt sectors.
The "inputs_point" package includes the inventories for the following sectors: ptnonipm, ptegu, pt_oilgas.
The "inputs_ptfire" package includes the inventories for the ptfire and ptfire3D sectors.
The "inputs" packages also include sample list (.lst) files, which list the inventories used by each sector. The run scripts automatically create new .lst files with the correct paths, so there is no need to edit these unless you are running SMOKE with your own scripts.
Future Year Inventory Packages
The 2017emissions directories contain the corresponding inventories for the 2017ek_cb6v2_v6_11g case. As with the scripts, inventories for sectors that are carried over from 2011 for the future year cases - agfire, beis, othafdust, and all ptfire - are not provided separately in the 2017 inventory packages; they must be downloaded from 2011emissions. There are separate .zip files for the 2017ek control case emissions - an hourly CEM package and a point inventory (ptegu) package.
See section 4 for a description of each modeling sector.
Installation of data files and scripts
This readme covers the installation of the SMOKE inventories, scripts, and ancillary files used for the 2011v6.3 platform.
Choose an install directory on your system; we will refer to this directory as "INSTALL_DIR". To review/reproduce emissions for all sectors, unzip all the .zip files into INSTALL_DIR. The packages have subdirectories embedded within them, so it is important that all files be unpacked in the same place in order for the scripts know where to find the inputs. If you are only interested in reproducing or examining emissions for specific sectors, you may download and unzip only the data for those sectors from the 2011emissions directory, but for SMOKE processing you should also include the files in the directories: ancillary_inputs, smoke/smoke_core_scripts, and spatial_surrogates for the grid(s) you plan to run. Additionally, precompiled SMOKE executables and I/O API utilities are available in smoke/smoke_precompiled_execs. The files in smoke/smoke_preproc_scripts and smoke/smoke_postproc_scripts are not necessary to run SMOKE; these are only for developing SMOKE inputs or processing SMOKE outputs.
All SMOKE inventories, scripts, and ancillary files used for the 2011v6.3 platform are provided, except emission factor tables for onroad processing via SMOKE-MOVES. The full set of emission factor tables is too large to permanently store on the FTP server and instead are furnished upon request.
MCIP meteorology data is not included in the package. This is used for the afdust, onroad, onroad_catx_adj, biogenics (beis), and ptfire3D sectors. See MET_ROOT instructions below. The sea salt portion of the CAMx conversion (see section 9) requires CAMx-formatted WRF output, which is also not included in the package.
Prior to running SMOKE, you will need to edit the INSTALL_DIR (your install directory) and MET_ROOT (location of MCIP meteorology data) environment variable definitions in the "directory_definitions.csh" script located in each CASE/scripts directory. This script is sourced by each of the individual run scripts for each sector. Regarding the MCIP data, SMOKE only uses the GRIDCRO2D, METCRO2D, and METCRO3D files.
In the ptegu run script, you will also need to edit NAMEBREAK_HOURLY so that it equals the number of underscores in your INSTALL_DIR (the full path), plus 5 (for 2011ek) or 6 (for 2017ek) or 7 (for 2017ek_cntl). Example for 2011ek: if INSTALL_DIR = '/disk/your/home_directory/foo_foo/install_dir', then set NAMEBREAK_HOURLY = 8 (3+5).
List of Directories (bold) and associated Environment Variables (red font)
- All zips are unzipped here.
- All of the following environment variables are set equal to INSTALL_DIR: $SMK_HOME, $OUT_ROOT, $PROJECT_ROOT, $DAT_ROOT, $SMK_SUBSYS
- Helper scripts which define some environment variables.
- Helper scripts which define some environment variables.
- Compiled I/O API utilities (m3xtract, juldate, etc).
- $IOAPI_LOCATION if defined by user, otherwise $IOAPIDIR as defined by assigns/sysflags
- This scripts directory only includes a script called “platform” which defines some environment variables. SMOKE scripts are elsewhere in the structure.
- SMOKE ancillary files. The root ge_dat directory itself includes some files, such as the COSTCY and INVTABLE. There are several subdirectories within ge_dat:
- gridding/: Includes AGREF/MGREF, GRIDDESC, and SRGDESC.
- nhapexclude/: Includes NHAPEXCLUDE files.
- point/: Includes ancillary files related to point source processing, such as the PELVCONFIG.
- smk_dates/: Includes files that map calendar dats to representative dates. SMOKE does not actually use these; only our scripts do.
- smkreport/: Includes SCC descriptions, NAICS descriptions, etc.
- repconfig/: REPCONFIGs are kept in another level below smkreport/.
- speciation/: GSPROs, GSREFs, GSCNVs, GSPRO_COMBOs.
- [case]/: See spec_parts.
- spec_parts/: We keep GSPROs and GSREFs in separate components that are concatentated together by our run scripts. The separate components are in speciation/spec_parts, and the concatenated verisons are in [case]/ (one separate directory per emissions case). The root speciation/ directory typically only includes complete GSPROs/GSREFs, such as those we use in SMOKE-MOVES, and also GSPRO_COMBOs.
- temporal/: Temporal profiles and cross-references.
- Home of SMOKE executables;
- assigns/: Includes ASSIGNS.emf and other helper scripts, which define SMOKE environment variables, directory structures, and so forth. :::$ASSIGNS_FILE = $INSTALL_DIR/$SMK_VERSION/assigns/ASSIGNS.emf
- Linux2_x86_64ifort/: Compiled SMOKE executables.
- $SMOKE_LOCATION if defined by user, otherwise $SMK_BIN as defined by assigns/sysflags
- scripts/: SMOKE run scripts.
- afdust_adj/: Scripts for applying transportable fraction and meteorological adjustments to afdust emissions
- annual_report/: Scripts for creating annual total reports from daily Smkmerge reports
- emf/: EMF helper scripts. These, such as smk_ar_annual_emf.csh, are called by the scripts that are created by the EMF. These scripts themselves call the scripts in run/.
- log_analyzer/: The script that scans SMOKE logs for errors or non-routine warnings.
- run/: SMOKE run scripts, called by the scripts in scripts/emf. These are the scripts that actually execute SMOKE.
- Each emissions case has its own directory within $INSTALL_DIR.
- inputs/[sector]/: Each inputs/[sector] directory includes the emissions inventories for that sector.
- $INSTALL_DIR/$CASE/inputs/$SECTOR, also $INVDIR
- intermed/[sector]/: All SMOKE outputs are located in intermed/[sector]/, including intermediate files (e.g. Temporal matrices), and sector-specific model-ready emissions (Smkmerge outputs). Does NOT include inline model-ready emissions or stack_groups files.
- $IMD_ROOT/$SECTOR, also $INTERMED
- logs/: SMOKE logs are kept here. Mrggrid logs are in intermed/mrggrid/logs/.
- [RPX]/: For SMOKE-MOVES, there is another directory level corresponding to rate-per-distance (RPD), rate-per-hour (RPH), and so on. For example, onroad RPD SMOKE outputs are in intermed/onroad/RPD/.
- reports/: SMOKE reports.
- annual_report/: Annual post-SMOKE totals derived from Smkmerge reports. These reports are created by the script in scripts/annual_report/.
- inv/: All Smkreport output.
- log_analyzer/: Reports generated by the log analyzer in scripts/log_analyzer.
- mrggrid/: Reports generated by Mrggrid. Also includes post-Mrggrid domain totals, which our scripts generate.
- programs/: Includes reports generated by Elevpoint. Also includes post-adjustment afdust reports, generated by the scripts in scripts/afdust_adj/.
- smkmerge/[sector]/: Daily Smkmerge reports, with separate directories for each sector. As with intermed/onroad, reports for RPD, RPH, etc are kept in separate directories beneath this level, e.g. smkmerge/onroad/RPD/.
- temporal/: Smkreport output for Temporal reports, although we almost never run these reports.
- scripts/: Includes the scripts created by the EMF that define all SMOKE inputs and parameters from the EMF case. These scripts each call one EMF helper script in scripts/emf/.
- smoke_out/: SMOKE outputs get a separate directory from CMAQ and CAMx outputs.
- [case]/: Emissions case name.
- [grid]/: Grid name, e.g. 12US2.
- [speciation]/: Speciation name, e.g. cmaq_cb6. $OUTPUT = the full path up through [speciation]
- This directory level includes the merged 2-D emissions and stack_groups files.
- [sector]/: Inline emissions files are in separate subdirectories by sector.
- [grid]/: Grid name, e.g. 12US2.
- [case]/: Emissions case name.
Image of Directory Structure
Case description and instructions for each sector
Base Year Cases
2011ek_cb6v2_v6_11g emissions were processed for a 12km national grid (12US2) and uses CB6 speciation. The 'v2' part of 'cb6v2' refers to a newer version of the CB6 speciation. The speciation is mostly unchanged from the 2011v6.2 platform, except for the onroad speciation that is internal to MOVES.
Future Year Cases
Emissions were also processed for the future year scenario 2017ek_cb6v2_v6_11g, and a future year control scenario, 2017ek_cntl_cb6v2_v6_11g, both of which were also only run with CB6 speciation. Source apportionment emissions for the 2017ek_ussa_cb6v2_v6_11g scenario were developed concurrently with 2017ek_cb6v2_v6_11g in the same runs.
To run any of these cases with CB05 or SAPRC speciation, packages including speciation profiles for those mechanisms are posted on the FTP server under the 2011v6.2 platform (v2platform/ancillary_data). Onroad emission factor tables for CB05 speciation are also available for both 2011ek and 2017ek and are furnished upon request.
Supported Air Quality Models
Furthermore, emissions were processed for two air quality models: CMAQ and CAMx. SMOKE outputs are "CMAQ-ready" as-is. After all CMAQ-ready emissions are processed, they are converted to CAMx by a standalone converter. This converter is included in the package and is described in section 9. Creating CMAQ-ready emissions is an intermediate step in creating CAMx-ready emissions; however, the current version of CMAQ (as of 16 Jul 2015) only supports CB05 speciation, not the CB6 speciation that was used in this modeling. See the previous paragraph for information on processing these cases with CB05 speciation. Future CMAQ versions will support CB6.
Emissions processing is split into "sectors". Each sector has its own run scripts for processing, with one (or more) run scripts per case. (See section 7 for information about the run script zips.)
Run instructions for each sector
How to run individual SMOKE Sectors
All sectors are US-only unless otherwise listed as other or "OTH" in the sector name.
All of the run scripts are found under the $INSTALL_DIR/$CASE/$SCRIPTS directory.
The naming convention for the run scripts is the following:
To list all run scripts that are that are available for a sector use the unix command:
For example the following command will list all of the scripts associated with the ptfire sector.
All sectors are US-only unless otherwise noted. The sectors are:
The sectors are described here:
For Dust sectors, run afdust/othafdust first, then afdust_adj/othafdust_adj.
AFDUST: Particulate emissions from fugitive dust sources. This sector is processed in two steps. The first (Annual_afdust_12US2_*) processes the annual inventory, and the second (Annual_afdust_adj_12US2*) applies adjustments - transportable fraction and meteorologically-based - and outputs the adjusted emissions under the sector name "afdust_adj". The afdust scripts must be run in that order.
AG: Agricultural ammonia emissions.
AGFIRE: Agricultural burning emissions. In the 2011v6.1 platform, these emissions were part of the nonpt sector.
- Marine and Rail
CMV: Emissions from C1, C2, and C3 commercial marine sources, including ports and navigable waterways. Includes offshore C1/C2 marine emissions in the Atlantic and Pacific Oceans, and the Gulf of Mexico. Does NOT include offshore C3 marine emissions; those are in the othpt sector. In the 2011v6.2 platform, the C1 and C2 emissions were combined with railway emissions in a sector called "c1c2rail", and the C3 emissions were in a sector called "c3marine". In the 2011v6.3 platform, the c1c2rail and c3marine sectors have been replaced by new sectors called cmv and rail.
RAIL: Area source railway emissions. In the 2011v6.2 platform, these emissions were part of the c1c2rail sector. In the 2011v6.3 platform, railway emissions have their own sector.
NONPT: Area source emissions not included in other sectors. NONROAD: Off highway mobile source emissions. NP_OILGAS: Area source oil and gas emissions. RWC: Area source residential wood combustion emissions.
BEIS: Biogenic emissions generated using the BEIS model, version 3.6.1.
For onroad sectors, run RPV/RPD/RPH/RPP first, then the merge.
ONROAD: On highway mobile source emissions, excluding California and Texas. This sector is rocessed using SMOKE-MOVES with multiple scripts as described in section 4B.
ONROAD_CATX_ADJ: On highway mobile source emissions, California and Texas only. This sector is processed using SMOKE-MOVES with multiple scripts as described in section 4B.
OTHAFDUST: Particulate emissions from fugitive dust sources in Canada. Just like with afdust, this sector is processed in two steps. The first (Annual_othafdust_12US2_*) processes the annual inventory, and the second (Annual_othafdust_adj_12US2*) applies adjustments - transportable fraction and meteorologically-based - and outputs the adjusted emissions under the sector name "othafdust_adj". The othafdust scripts must be run in that order. Fugitive dust emissions in Mexico are included in the othar sector and do not need the same transportable fraction and meteorological adjustments that the Canada fugitive dust emissions in othafdust do.
OTHAR: Area source emissions from Canada and Mexico.
OTHON: Mobile source emissions from Canada and Mexico.
All 'Point' sectors are processed via two scripts: the 'onetime' script, and the 'daily' script. The point sector run script names will have the sector name and the process (daily or onetime) included in the name. For example for the PTNONIPM Sector there are two scripts: Annual_ptnonipm_onetime_12US2_*.csh and Annual_ptnonipm_daily_12US2_*.csh. The order of how the process scripts are run is important, the 'onetime' script must be run prior to the daily script(s).
OTHPT: Point source emissions from Canada, Mexico, and offshore areas. Includes all offshore C3 commercial marine emissions. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. All emissions in this sector are elevated (no low-level contribution).
PTEGU: Electric generating unit emissions. This sector incorporates CEM (Continuous Emissions Monitoring) hourly emissions for a majority of sources. In the 2011v6.2 platform, this sector was configured to separate emissions into low-level (emis_mole files) and inline elevated (inln_mole files). In the 2011v6.3 platform, considering how few ptegu sources end up being classified as "low-level", ptegu is now configured to be a 100% elevated sector. This means all emissions for the sector are included in the inline (inln_mole) files, and emis_mole files are no longer generated. This is a 'point' sector, and like all 'point' sectors, is processed via a 'onetime' script first, followed by a 'daily' script. For ptegu there are two 'daily' scripts for different months of the year: 'summer' (May through September), and 'winter' (October through April). For sources without hourly CEM emissions, summer and winter use different hourly temporalization, and so they are run with separate inputs. You may need to change the NAMEBREAK_HOURLY setting in the run scripts, as described in the run script comments, and in section 3 of this readme.
PTEGU_SUMMER, PTEGU_WINTER: For 2017ek future year modeling, the ptegu sector has been split into two separate sectors - one for the summer season, and one for the winter season. The separate sectors use separate inventories with seasonal emissions totals rather than annual, and also separate temporalization. This is a different setup from 2011ek, where summer and winter processing have different run scripts, but all under the same sector "ptegu". For 2017ek, ptegu summer and winter processing are handled in completely separate sectors, with separate "onetime" and "daily" run scripts.
PTNONIPM: Point source emissions from industrial activities. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first.
PTFIRE: Point source emissions from year specific controlled burning and wild fires. For this sector (as opposed to ptfire3D), fires are processed in the 'inline' format for CMAQ, and are all elevated (no low-level contribution). These inline fire emission files are only needed by CMAQ, not CAMx, so if you are not running CMAQ, you can omit this sector and only run ptfire3D. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. PTFIRE3D: Same as ptfire, except instead of processing emissions as 'inline', 3-D layered emissions files are produced, using meteorology data and the SMOKE Laypoint program to calculate plume rise. Fire emissions are output into one of Z vertical layers, where Z = the number of layers in your MCIP data. The number of layers is defined in the run script by the parameter SMK_EMLAYS; for us, this was equal to 25. These 3-D ptfire emissions files are only needed by CAMx, not CMAQ, so if you are not running CAMx, you can omit this sector and only run ptfire. In the 2011v6.2 platform, 3-D fire processing was split into separate sectors ptprescfire3D (prescribed burning) and ptwildfire3D (wildfires) to support future source apportionment modeling. In the 2011v6.3 platform, fires are no longer split into separate sectors for prescribed burning and wildfires. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first.
PT_OILGAS: Point source oil and gas emissions. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first.
Notes regarding onroad
Onroad emissions are processed using SMOKE-MOVES. The processing is split into multiple run scripts. In the 2011v6.3 platform, SMOKE-MOVES inputs were prepared using the latest version of MOVES: MOVES2014a. The 2011v6.2 platform used MOVES2014.
As described in the SMOKE online documentation, SMOKE-MOVES handles onroad emissions separately for four types of processes: - On-network emissions (RatePerDistance, or RPD) - Off-network emissions, fuel vapor venting (RatePerProfile, or RPP) - Off-network emissions, extended idling (RatePerHour, or RPH) - Off-network emissions, non-venting, non-extended idle (RatePerVehicle, or RPV)
For each of the two onroad sectors (onroad, onroad_catx_adj), there are separate run scripts for RPD, RPP, RPH, and RPV, plus a merge script that combines emissions from RPD, RPP, RPH, and RPV into a single emissions file per day.
These scripts may take a particularly long time to run, especially RPD. Therefore, consideration should be given to running multiple RPD jobs in parallel, such as one job per quarter.
The reason onroad has been split into two sectors - onroad and onroad_catx_adj - is in order to match SMOKE-MOVES annual emission totals with those provided by California and Texas for non-refueling sources. To do this, we split California and Texas into a separate sector, and ran SMOKE-MOVES with a control factor file (CFPRO) which nudges the emissions so that the annual totals post-SMOKE-MOVES equal those provided by CARB and TCEQ. Both CARB and TCEQ provided updated onroad emissions inventories for use in the 2011v6.3 platform, and we match their provided emissions at the county/SCC6/process level for California/CARB, and the county/SCC level for Texas/TCEQ.
For the onroad sector, a CFPRO is provided in the ancillary_inputs/ge_dat_for_2011ek_other.zip package which zeroes out gasoline refueling emissions in 53 Colorado counties. We zero out these emissions in order to prevent a double count with the ptnonipm sector, since that sector includes gasoline refueling emissions in these counties. Another CFPRO is provided for extended idle (RPH) emissions which cuts all extended idle emissions in Maricopa County, Arizona, by 50%. This is in response to feedback received from the 2011v6.2 platform.
SMOKE-MOVES can be run more efficiently if running more than one day at a time. For example, Movesmrg can create one 7-day emissions file more quickly than it can create seven individual 1-day emissions files. To turn on this feature, use the DAYS_PER_RUN variable, set to the number of days you wish to run in a single Movesmrg instance. The recommended value for DAYS_PER_RUN is 7. The onroad scripts include a setting called "DAYS_PER_RUN", set to 1 as the default.
If DAYS_PER_RUN > 1, after Movesmrg is run, the run scripts will use the I/O API utility m3xtract to split up the multi-day emissions file into single day (25-hour) emissions files.
Multi-day Movesmrg runs will never cross months. For example, if DAYS_PER_RUN = 7, then the last Movesmrg run of January will start on January 29th and end on January 31st (3 days), and the first Movesmrg run of February will start on February 1st and end on February 7th.
Using the multi-day Movesmrg functionality requires multi-day MCIP files. For example, if DAYS_PER_RUN = 7, your METCRO2D files must also be 7 days (169 hours) long. These multi-day MCIP files should be stored in MET_ROOT/[grid]/mcip_out/Xday/, where X = DAYS_PER_RUN (i.e. /7day for DAYS_PER_RUN = 7).
The primary drawback to using this multi-day Movesmrg functionality is an increase in the memory usage.
After all sectors have been processed, the Sector_merge script merges the low-level emissions from all sectors into a single CMAQ-ready emissions file per day.
Merged model-ready emissions will be output to: INSTALL_DIR/$CASE/smoke_out/$CASE/12US2/$SPC/
Inline emissions files will be output to the same directory, except in subdirectories by sector name (e.g. .../$SPC/ptnonipm/).
For the CAMx conversion, see section 9.
Future year scenarios
The future year scenario, 2017ek_cb6v2_v6_11g, reuses emissions from the 2011ek_cb6v2_v6_11g case for the following sectors: agfire, beis, othafdust, and ptfire/ptfire3D. Scripts for these sectors are identical to those in the 2011ek_cb6v2_v6_11g package.
The future year control scenario, 2017ek_cntl_cb6v2_v6_11g, only differs from the 2017ek base case in the ptegu sectors (ptegu_summer, ptegu_winter). For the control case, only ptegu scripts are provided. All other sectors should be run under the normal 2017ek_cb6v2_v6_11g case. Additionally, a sector merge script is not provided for 2017ek_cntl_cb6v2_v6_11g, because ptegu emissions (as of this platform) are entirely contained in the elevated "inline" emissions files. In other words, the 2-D merged emissions for control case are the same as for the base 2017ek case.
Description of ancillary file packages
The following packages should be unpacked in INSTALL_DIR:
ancillary_inputs/ge_dat_for_2011v3_gridding.zip ancillary_inputs/ge_dat_for_2011v3_other.zip ancillary_inputs/ge_dat_for_2011v3_speciation.zip ancillary_inputs/ge_dat_for_2011v3_temporal.zip ancillary_inputs/ocean_chlorine.zip spatial_surrogates/surrogates_CONUS12_2010_CAN_MEX.zip spatial_surrogates/surrogates_CONUS12_2010_v5_20141015.zip
The "surrogates" packages contain the gridding surrogates for 12US2, for US and Canada+Mexico. Surrogates for 36km (36US1) and 4km are also provided for the US only in the spatial_surrogates directory.
The "gridding" package includes all SMOKE inputs related to gridding other than the spatial surrogates, including cross-references, surrogate descriptions, and gridded transportable fractions used in afdust_adj and othafdust_adj.
The "speciation" package includes speciation profiles, cross-references, and VOC-to-TOG conversion factors. This .zip includes files for multiple speciation mechamisms, even though only CB6 is used in this platform. Files for other mechanisms, in particular the SAPRC07TB .zip, are provided on an as-is basis and do not include all of the speciation profiles needed in order to process the entire 2011v6.3 platform. In addition, we do not have onroad (SMOKE-MOVES) emission factor tables for SAPRC07TB, or for additional HAPs such as mercury, for the 2011ek or 2017ek cases. The onroad emission factor tables do include unspeciated NONHAPTOG by mode (e.g. EXH_NHTOG), however, in support of processing onroad emissions with alternate speciation mechanisms such as SAPRC.
The "temporal" package includes temporal profiles and cross-references, including daily and hourly temporal profiles developed by the SMOKE program Gentpro for use in the rwc and ag sectors, respectively.
The "other" ge_dat package includes all other SMOKE ancillary files not included in the above packages, including:
- Inventory tables (INVTABLE)
- Lists of sources to exclude from CAP/HAP integration (NHAPEXCLUDE)
- SMOKE-MOVES ancillary files, including the reference county (MCXREF) and fuel month (MFMREF) cross-references, pollutant (MEPROC) and emission factor table (MRCLIST) lists, activity SCC to full SCC cross-references (SCCXREF), hourly speed profiles (SPDPRO), daily temperature data (METMOVES), and control factors for onroad_ca_adj and Colorado gasoline refueling (CFPRO)
- Other miscellaneous SMOKE inputs, such as the ARTOPNT, COSTCY, HOLIDAYS, MACTDESC, NAICSDESC, ORISDESC, PELVCONFIG, PSTK, SCCDESC
- Smkreport configuration files (REPCONFIG, all in ge_dat/repconfig/default)
- SOURCE_GROUPS files in support of source apportionment applications (see Section 8)
This ocean_chlorine.zip package contains gridded ocean chlorine emissions, which are included in the sector merge. Unzip in INSTALL_DIR.
The run scripts (see section 7) are already set up to use the proper ancillary files and inventories for each sector and case.
Description of script packages
For 2011, these two run script packages should be unpacked in INSTALL_DIR:
For 2017, there are two additional script package: smoke/smoke_core_scripts/2017ek_cb6v2_v6_11g_scripts.zip (2017 base case) smoke/smoke_core_scripts/2017ek_cntl_cb6v2_v6_11g_scripts.zip (2017 control case)
The helper_scripts package includes scripts which are called by the primary run scripts. You should not have to run any of these scripts directly.
The 2011ek scripts package includes the scripts that you run directly in order to replicate our emissions. Separate script(s) are provided for each sector. The scripts are unpacked to INSTALL_DIR/$CASE/scripts/.
See section 4 for information pertinent to each sector. In general, you edit the directory_definitions.csh file, in particular INSTALL_DIR and MET_ROOT, and then run each sector. Sector scripts are organized into subdirectories within CASE/scripts by sector category: biogenics, nonpoint, onroad, point, and merge. For afdust sectors, run afdust/othafdust first, then afdust_adj/othafdust_adj. For point sectors, run "onetime" first, and then "daily". For onroad sectors, run RPV/RPD/RPH/RPP first, then the merge.
For CAMx conversion scripts, see section 9.
Source apportionment applications
In addition to the 2011ek and 2017ek emissions cases, EPA also prepared 2017ek emissions for source apportionment modeling in CAMx. The source apportionment version of 2017ek is sometimes called "2017ek_ussa_cb6v2_v6_11g", but emissions files in support of source apportionment can be generated concurrently with the "normal" 2017ek emissions. Brief instructions on how to run SMOKE for source apportionment applications are included here.
The general procedure for creating model-ready emissions files for source apportionment is as follows, with further detail below: - Run SMOKE for each sector in source apportionment mode, creating model-ready emissions in CMAQ inline format - Convert the CMAQ-ready emissions files to CAMx format
To run SMOKE in source apportionment mode, we need SOURCE_GROUPS files that assign certain sources or regions to a "tag". The SOURCE_GROUPS files for 2017ek_ussa are included in the ancillary_data/ge_dat_for_2011v3_other.zip package. Tags are assigned by state, with each US state receiving its own tag for all anthropogenic emissions excluding wild and prescribed fires, biogenics, and particulate dust. Biogenic and particulate dust (afdust) emissions received a separate tag; all Canadian and Mexican emissions received another separate tag; offshore marine emissions received their own tag; and wild and prescribed fires received their own tag as well.
The 2017ek_ussa case used two SOURCE_GROUPS files: one for all US sectors including the state-specific tags, and one for Canada and Mexico including the Canada+Mexico tag. The filenames for these files are smkmerge_groups_ussa_US_02apr2015_v1 and smkmerge_groups_ussa_nonUS_02apr2015_v0, respectively. The package also includes two additional SOURCE_GROUPS files for onroad, which are the same as smkmerge_groups_ussa_US_02apr2015_v1 except for one without California and Texas (for the onroad sector), and one with California and Texas only (for the onroad_catx_adj sector). Splitting up the SOURCE_GROUPS files for onroad helps reduce the memory requirements when running SMOKE-MOVES with source apportionment. To turn on source apportionment, the environment variable SMK_SRCGROUP_OUTPUT_YN should be set to "Y". The 2017ek run scripts all include this variable, set to "N" by default. The appropriate SOURCE_GROUPS file is also defined in each run script.
When source apportionment is turned on, the usual emissions files (emis_mole and/or inln_mole) are still generated, but additional inline format emissions files are also created, even area and mobile source sectors, that include the tag information. Because of this, emissions files for source apportionment and for non-source apportionment modeling can be generated at the same time.
These additional inline emissions files, typically named sginln_mole_*, include all emissions in the sector, independent of whether the emissions are considered "low-level" or "elevated". In other words, for point sectors that normally have emissions split into a low-level file and an elevated file, the source apportionment "sginln" file includes both low-level and elevated emissions. These sginln files, along with the associated stack_groups_out files, can be used as inputs to CMAQ, but we convert them to CAMx format later on for our modeling. The usual emis_mole and inln_mole files are not needed if running CMAQ or CAMx for source apportionment.
When running SMOKE, special considerations must be taken for the following sectors:
AFDUST: The scripts which apply transportable fraction and meteorological adjustments to post-SMOKE afdust emissions files do not work with the "sginln" files that SMOKE creates for source apportionment. This sector includes PM emissions only, and the focus of the 2017ek_ussa model run was ozone production, for which PM emissions are not vital. Therefore, it was decided to not apply state-specific tags to afdust, and instead apply a sector-wide tag during the CAMx conversion step of the process. So for afdust, we run SMOKE in "normal" 2-D mode, and then apply a sector-wide tag to the 2-D emissions during the CAMx conversion step.
OTHAFDUST: Same as afdust.
PTFIRE: If we were modeling with CMAQ, we could run ptfire through SMOKE with source apportionment just like the other sectors. However, because we are modeling with CAMx, we must have SMOKE create 3-D layered ptfire emissions. This is a problem because SMOKE only supports the creation of inline format emissions for source apportionment, not 3-D layered emissions for source apportionment. So, just like with afdust, we do not run ptfire through SMOKE with source apportionment. We run it in normal 3-D mode, and then apply a sector-wide tag during the CAMx conversion step.
BEIS: Similar to afdust and ptfire, since this sector is only receiving a sector-wide tag, this sector was run normally through SMOKE, with a sector-wide tag applied during the conversion to CAMx format.
ONROAD: Two things to consider with source apportionment for onroad: - Running SMOKE-MOVES with source apportionment requires a very large amount of memory, particularly for RPD processing. It may not be possible to run SMOKE-MOVES with source apportionment for the entire US on all systems. - The onroad "merge" step, which combines emissions from RPD, RPP, RPV, and RPH, is omitted when running onroad with source apportionment. The individual files from RPD, RPP, etc are input directly to CMAQ (if running CMAQ) or to the CAMx converter (if running CAMx).
For source apportionment cases, the sector merge (Mrggrid) should only merge sectors that are NOT getting source apportionment tags applied. If all sectors are getting tags - including sectors like afdust that are getting tagged outside of SMOKE - then Mrggrid does not need to be run at all. This was true in the 2017ek_ussa case. The source appportionment inline-format emissions (sginln_mole_*) themselves are not merged. The sector emissions are all input directly to CMAQ or the CAMx converter.
Details on converting SMOKE outputs (CMAQ-ready inline files) to CAMx format for source apportionment are in Section 9C.
The package smoke/smoke_postproc_scripts/2011v3_camx_conversion_scripts.zip contains all scripts and programs needed in order to convert the CMAQ-ready emissions (SMOKE outputs) into emissions inputs for the CAMx air quality model. Scripts are provided for 2011ek, 2017ek, 2017ek_cntl, and 2017ek_ussa.
The primary run scripts are unpacked to INSTALL_DIR/$CASE/scripts/camx/. The 2011ek ptfire script converts the 3-D gridded SMOKE emissions from 2011ek (ptfire3D) to CAMx format and should be run first, regardless of whether you are running the 2011 case or one of the 2017 cases. Then, the primary camx_convert script(s) should be run, which convert the remaining SMOKE emissions to CAMx cormat, including the outputs from the ptfire script. The ptfire conversion is included in separate scripts because this step normally takes longer than the remaining steps. 2017 CAMx conversion scripts are separated by season because in the 2017 cases, there are separate ptegu sectors for summer and winter. The "winter1" script converts January through April, and "winter2" converts October through December.
For the 2017ek control case, the CAMx conversion script is configured to skip the conversion of the 2-D merged emissions, since the 2-D merged emissions from 2017ek control equal the 2-D merged emissions from 2017ek base.
Sea salt emissions are added to the emis2d file in the primary camx_convert script, rather than in an additional script as in prior platforms.
All CAMx-ready files will go to: INSTALL_DIR/$CASE/smoke_out/$CASE/12US2/camx/
FORTRAN programs used by the CAMx converter
The CAMx converter uses several FORTRAN programs. The camx_conversion.zip contains both our compiled FORTRAN executables (which may or may not work on your system), and the original FORTRAN source code and Makefiles, so that you may recompile these helper programs yourself.
The source code is in INSTALL_DIR/camx/src/, and our executables are in INSTALL_DIR/camx/bin/. If you have to recompile the code, this is where the recompiled executables should ultimately go. There are multiple versions of the cmaq2uam program, each of which is used.
Other files used by the CAMx converter
The subdirectories 'dates', 'land', 'seasalt', and 'Species_Mapping_Tables' in INSTALL_DIR/camx/ include other files used by the converter.
The addition of sea salt emissions requires CAMx-formatted meteorological data from WRF. Specifically, this requires a 'met3d' file and a 'met2d' file. Like the MCIP meteorology, these files are not included in the package.
CAMx conversion for source apportionment
Converting emissions to CAMx format for source apportionment (SA) requires several additional steps. In this type of run, all sectors are tagged and written as point or "ptsr" type emissions files. This is a multi-step process involving the conversion and tagging of certain low-level sources (beis, othafdust, and afdust) along with the conversion of tagged inline files from the remaining sectors.
The scripts for the SA CAMx conversion are in the same package (2011v3_camx_conversion_scripts.zip) and are extracted to 2017ek_ussa_cb6v2_v6_11g/scripts/camx/.
First, when running SMOKE, you must create SGINLN and STACK_GROUPS_OUT files for all sectors except afdust, othafdust, beis, and ptfire3D (see Section 8).
Next, the gridded SMOKE emissions outputs for those four sectors must be converted to CAMx-ready ptsr format. For ptfire3D, this is done with the same script as for other cases (2011ek_cb6v2_v6_11g/scripts/camx/camx_convert_2011ek_cb6v2_12US2_ptfire.csh). For afdust, othafdust, and beis, scripts are provided in 2017ek_ussa_cb6v2_v6_11g/scripts/camx/. Note that othafdust and beis emissions are assumed by these scripts to come from the 2011ek case.
The next step is to convert SGINLNs to CAMx-ready ptsr format, and then merge all ptsr files together - including the afdust, othafdust, beis, and ptfire ptsrs developed in prior steps - into a single mrgpt file per day. This is all done by the camx_convert_2017ek_ussa_cb6v2_12US2* series of scripts (two scripts for winter and one for summer, as with the non-SA 2017ek cases). This script applies a sector-wide tag to afdust, othafdust, beis, and ptfire3D, and preserves the tags contained within each of the SGINLN files as applied by SMOKE.
Finally, sea salt emissions must be generated. In a non-SA CAMx conversion, sea salt is created within the CAMx conversion script and is merged with the 2-D emissions. In a full SA case, there are no standalone 2-D emissions - all emissions are included in the mrgpt files. So, we must create standalone sea salt emissions files. The seasalt_2017ek_12US2.csh script does this.
Other miscellaneous programs
The scripts in the following packages are not intended to be "portable"; they are only provided as examples.
AFDUST XPORTFRAC: Afdust emissions are adjusted by applying a transportable fraction file (XPORTFRAC) to reduce the emissions after SMOKE is run. The package includes a XPORTFRAC file for 12US2. To generate XPORTFRAC files for other grids, inputs and sample scripts are provided in this package: smoke/smoke_preproc_scripts/afdust_xportfrac_regrid_scripts.zip
BEIS LAND USE: Land use files for processing biogenic emissions with BEIS are provided for 12US2. To generate land use files for other grids, inputs and sample scripts are provided in this package: smoke/smoke_preproc_scripts/beld4.1_land_use_create.zip
CEM CORRECT: A program is provided which corrects anomalous values in the continuous emissions monitoring (CEM) hourly emissions data available on the CAMD website. This program has already been run on the hourly CEM data provided in these packages. The program is available in: smoke/smoke_preproc_scripts/cem_correct_v2.1.zip
CREATE MERGE DATES: Our emissions platforms process sectors with "representative days" - for example, one day per week for a month, rather than every day of the month, to reduce computational requirements. Our Mrggrid sector merge script uses a set of "merge dates" files when running Mrggrid for each calendar day, which tells the run script which representative day maps to each calendar day. Merge dates files for 2011 are provided in the package. To create merge dates files for other years, a script is provided in this package: smoke/smoke_preproc_scripts/create_merge_dates.zip
GENTPRO: For those wishing to generate their own meteorology-dependent temporal profiles for residential wood combustion (RWC) or ag livestock ammonia emissions using the SMOKE program Gentpro, sample Gentpro scripts and inputs have been provided in: smoke/smoke_preproc_scripts/smoke_gentpro_sample_scripts.zip
MET4MOVES: The downloadable package includes met4moves outputs used by SMOKE based on our meteorological data. For alternate sets of meteorological, met4moves should be run separately. SMOKE uses the "SMOKE_DAILY" gridded files output by met4moves. Sample scripts for running met4moves are here: smoke/smoke_preproc_scripts/met4moves_sample_scripts.zip
MOVES POST-PROCESSING: For processing onroad emissions, we run MOVES to create a set of emission factor tables to be used by SMOKE. MOVES does not actually create the emission factor tables directly; instead, MOVES creates a series of databases, which are converted to emission factor tables using a MOVES post-processor script called "moves2smk". The moves2smk script and its inputs are provided in this package: smoke/smoke_preproc_scripts/onroad_MOVES2014_postprocessing_v1.5_Oct2015.zip
MOVESMRG REPORT POST-PROCESSING: Onroad processing with SMOKE-MOVES does not use an input emissions inventory with county/SCC-level emissions already defined. Instead, we need SMOKE-MOVES outputs to serve as the emissions "inventory" for report and summary purposes. Therefore, Movesmrg is set up to create full county/SCC-level daily emissions reports. A script to aggregate the Movesmrg daily reports to more digestable annual and monthly reports by state, state/SCC, county, and county/SCC, are provided in this package: smoke/smoke_postproc_scripts/onroad_movesmrg_report_postprocessor.zip