ARTICLE IN PRESS

Atmospheric Environment 37 (2003) 3481–3489

A case study for assessing uncertainty in local-scale

regulatory air quality modeling applications
Todd Sax*, Vlad Isakov
California Air Resources Board, 1001 I Street, P.O. Box 2815, Sacramento, CA 95812, USA
Received 5 October 2002; accepted 16 May 2003

Abstract

In this paper, we expand upon established uncertainty analysis techniques to demonstrate a general method for
assessing variability and uncertainty in Gaussian air pollutant dispersion modeling systems. To illustrate this method,
we estimated variability and uncertainty in predicted hexavalent chromium concentrations generated by welding
operations at a shipbuilding and repair facility in California. Using Monte Carlo statistical techniques, we propagated
uncertainty across both ISCST3 and AERMOD, and estimated the contribution of variability and uncertainty from
four model components: emissions, spatial and temporal allocation of emissions, model parameters, and meteorology.
Our results indicated the 95% confidence interval uncertainty at each receptor spanned an order of magnitude. From a
practical perspective uncertainty is most important at receptors with highest predicted concentrations. In this case
study, emissions were the primary source of uncertainty. However, Gaussian models are also sensitive to location of
emission releases, meteorology, and model parameters. Simplified modeling approaches may lead to errors in pollutant
concentration estimates, especially in close proximity to emissions sources where predicted concentrations are highest.
r 2003 Elsevier Ltd. All rights reserved.

Keywords: AERMOD; Air quality modeling; Air toxics; Emissions inventory; Hazardous air pollutants; Hexavalent chromium;
ISCST3; Uncertainty analysis; Welding

1. Introduction performance is to analyze sources of variability and

In 1999, the California Air Resources Board (CARB)
initiated its Neighborhood Assessment Program to
develop source-receptor-based, cumulative-impact/risk
assessment methodologies suitable for evaluating neigh-
borhood-scale air pollution impacts and for comparing
neighborhood-scale exposures within a region (CARB,
2000). The objective of this program is to provide risk
managers the tools for evaluating local scale air
pollution impacts and targeting risk reduction strategies.
To ensure modeling results are appropriate for
regulatory decisions, model performance should be
evaluated (Venkatram, 1981). One way to assess model
*Corresponding author. Tel.: +1-916-323-4865;
fax: +1-916-323-1075.
E-mail address: tsax@arb.ca.gov (T. Sax).
uncertainty in air quality models and to determine how
those various sources affect confidence in model results.
Variability refers to the temporal, spatial, or inter-
individual differences in the value of an input (Cullen
and Frey, 1999). Uncertainty refers to the lack of
knowledge or information about an unknown quantity
whose true value could be established if a perfect
measurement device were available (Cullen and Frey,
1999).
Both variability and uncertainty affect Gaussian
modeling results. Variability in Gaussian model results
is present primarily through meteorology and emission
rates, because weather conditions and processing rates
vary over time. We believe uncertainty in Gaussian
models can be categorized into three components: input
uncertainty, parameter uncertainty, and conceptual
uncertainty. Model input uncertainty arises where inputs

1352-2310/03/$ - see front matter r 2003 Elsevier Ltd. All rights reserved.
doi:10.1016/S1352-2310(03)00411-4
 ARTICLE IN PRESS
3482 T. Sax, V. Isakov / Atmospheric Environment 37 (2003) 3481–3489
to an air quality model, such as meteorological data or
emissions, are themselves uncertain due to measurement
error, estimation error, and inherent variability. Para-
meter uncertainty is present because a single model
parameter value can never completely characterize a
modeling domain. Conceptual uncertainty occurs be-
cause simple mathematical and numerical programming
code used to predict dispersion cannot completely
characterize complex physical processes and associated
inherent naturally occurring variability (Hanna et al.,
1998). Because data are limited, further research is
necessary to characterize conceptual uncertainty in
Gaussian model results.
This paper describes the application of uncertainty
analysis techniques to assess variability and uncertainty
in predicted annual average concentrations of hexava-
lent chromium (CrVI) generated by welding processes at
a shipbuilding and repair facility in California. Our
objective was to demonstrate a general methodology for
the quantitative assessment of variability and uncer-
tainty in Gaussian air quality models through a case
study, and to compare case study results to simplified
modeling approaches.
2. Application of uncertainty analysis methods to case
study
In Gaussian modeling systems, users specify emissions
from each release location, meteorological data files,
qand model parameters. Over the past two decades,
uncertainty analysis techniques have been applied both
to air quality models and their related inputs. Meth-
odologies have been developed to assess uncertainty in
meteorology (Irwin et al., 1987; Dabberdt and Miller,
2000) and emissions (Frey and Rhodes, 1996; Frey and
Li, 2002; Frey and Zheng, 2002). Studies have been
conducted to assess uncertainty in Gaussian air quality
models (Freeman et al., 1986; Seigneur et al., 1992),
photochemical air quality models (Hanna et al., 1998,
2000; Bergin et al., 1999) and health risk assessment
(Seigneur et al., 1992; Frey, 1993; Cullen and Frey,
1999).
To assess variability and uncertainty in model results,
we integrate and expand upon methodologies for
estimating uncertainties in model results to demonstrate
a generalized approach for use in Gaussian regulatory
modeling applications. We assess variability and un-
certainty first by separating model inputs and para-
meters into four components: emissions, spatial and
temporal allocation of emissions, model parameters,
and meteorology. We then assess relevant sources of
variability and uncertainty in each component, and
apply Monte Carlo techniques to propagate un-
certainty in each component to pollutant concentration
estimates.
3. Uncertainty in emissions characterization
During the welding process filler metal alloys, which
often contain chromium, are melted to join metal parts
together. Under certain conditions the welding process
produces airborne CrVI from filler metals containing
chromium. Both the content of chromium in filler
metals, and the type of welding process applied, are
known to affect CrVI emissions and must be analyzed
separately. Four general types of alloys can be identified
and classified by their chromium or nickel content: mild
steels (o0.5% chromium; MS), low alloy steels (o2.5%
chromium; LA), stainless steels (4–32% chromium; SS),
and nickel alloys (>60% nickel, 2.5%–>17% chro-
mium) (AWS, 1969). Welding processes are classified
according to the type of solid or gaseous shield used to
ensure oxygen cannot contaminate welds. Solid shields
are used for shielded metal arc welding (SMAW), where
the consumable electrode is coated with a shield, and
flux core arc welding (FCAW), where the consumable
electrode contains a shield core surrounded by alloy.
Gas shields are used for gas metal arc welding
(GMAW), where the consumable electrode is a bare
wire, and gas tungsten arc welding (GTAW), where the
electrode is a non-consumable solid tungsten electrode
and welding wire is bare. FCAW processes may also be
conducted in the presence of a shielding gas where weld
quality is a major concern (GFCAW). Emissions in this
case study are calculated for each process-alloy combi-
nation by multiplying an average annual process activity
rate by an average emission factor consisting of a fume
generation rate for particulate matter and an estimate
for CrVI content in fume.
Emissions in this case study are subject to both
variability and uncertainty. Process rates will vary over
time depending on the specific types of welding processes
and size of individual projects at the facility. At the same
time, assessing true variability in process rates given
limited data is uncertain. Emission factors are uncertain
because they are subject to uncertain measurement and
estimation techniques. Emission factors may also exhibit
variability to the extent that emission rates may vary
over time or under variable conditions from the same
welding equipment. The limited availability of welding
emission factor data limits our ability to distinguish
variability from uncertainty in emission factors, and so
we consider emission factors to be uncertain. In this
study, we assessed inter-annual variability and uncer-
tainty in process rates and uncertainty in emission
factors. We then used Monte Carlo techniques to
quantify a range of possible annual average emission
estimates.
Process activity rates are measured by the mass of
specific filler metal alloys consumed in each welding
process over time. We obtained purchase records for the
years 1997, 1998, and 1999 to estimate variability in
 ARTICLE IN PRESS
T. Sax, V. Isakov / Atmospheric Environment 37 (2003) 3481–3489 3483

annual process activity rates. We quantified annual
process rates by process-alloy combination using filler
metal codes specified by the American Welding Society
(AWS) (AWS, 1969). To determine whether these three
years represented longer-term trends at the facility we
reviewed data with facility engineers. Based on these
discussions we determined nearly all filler metals are
used in the year they are purchased. We also estimated
that because of uncertainty about future on-site activity
with respect to each process-alloy combination, alloy
usage by process might vary by as much as one-third
higher than the highest observed value or one-third
lower than the lowest observed value.
To assess uncertainty in fume generation rates we
assessed peer-reviewed studies, many of which were
summarized for the US Environmental Protection
Agency (MRI, 1994). We extracted fume generation
rate data from each study and categorized them by
process-alloy combination. Several data sources devel-
oped by the AWS indicated process factors like
voltage and current affect emission rates. Other data in
the literature did not confirm these results, probably
because source testing and analytical methods
varied between studies, and so we did not consider
dependence between these factors and emission rates.
Only limited data representing the GFCAW process and
all processes involving nickel alloys were available. We
assumed the GFCAW process is similar to either the
GMAW or SMAW process and processes involving
nickel alloys behave similarly to processes involving
stainless steels.
Three types of data are available to estimate the
amount of CrVI in fume: direct emission factor studies,
industrial hygiene studies (Castner and Null, 1998), and
AWS standards for consumable electrode composition.
All studies provided information about CrVI generated
as a function of alloy type, but fewer studies provided
electrode classifications sufficient for determining initial
chromium content. Because industrial hygiene and
emission factor data are not directly comparable, we
used several different techniques to estimate fume CrVI
content, including estimating a range of the ratio of
CrVI in total generated fume, and a range of the ratio of
CrVI to total chromium in fume. We applied the same
assumptions to both CrVI fume content estimates and
fume generation rates.
Efron and Tibshirani (1993) describe how to estimate
the bootstrap standard error of a population statistic,
which we applied to estimate the distribution of annual
average emissions estimates from the facility. We used
polynomial regression techniques to fit process rate and
emission factor data to cumulative probability
density functions (PDFs). The use of polynomial
regression is analogous to using a continuous empirical
distribution as described by Law and Kelton (2000). To
ensure proper fit, we accepted regressions with R2
values exceeding 0.95 whose behavior did not result in
negative process rate or emission factor values. Emis-
sions were propagated individually by process-alloy
combination.
Table 1 demonstrates the distribution of possible
average annual CrVI emission rates by process-alloy
combination. Our analysis suggested the 95% con-
fidence interval of predicted CrVI emissions reached an
order of magnitude in many cases. Emissions from the
GFCAW process and SMAW-low alloy combination
exceeded one order of magnitude. This uncertainty arose
from the lack of sufficient data available to analyze CrVI
content in fume for these processes. Over 98% of all
alloys processed at the facility were mild and low alloy
steels, and we estimated the use of these alloys
accounted for 60% of CrVI emissions. Emission factors
for mild and low alloy steels are generally two orders of
magnitude lower than for stainless steels.
4. Uncertainty due to spatial and temporal allocation of
emissions
The case study facility generally operates 16 h per day,
6 days per week excluding Sundays. Because welding
activity is constant during operating hours, we did not

Table 1
Distribution of CrVI emissions by process/alloy combination

Process Alloy Emissions (g yr
1) by percentile

2.5 25 50 75 97.5

Gas metal arc welding Mild steel 1 9 20 25 45

Gas metal arc welding Stainless steel 10 15 25 30 35
Gas metal arc welding Nickel alloy 14 35 130 180 410
Shielded metal arc welding Mild steel 1 10 45 90 230
Shielded metal arc welding Stainless steel 90 320 410 550 680
Shielded metal arc welding Low alloy steel 20 230 820 950 3500
Flux core arc welding using a shielding gas Mild steel 90 270 770 1045 4090
Flux core arc welding using a shielding gas Stainless steel 45 130 550 770 1850
 ARTICLE IN PRESS
3484 T. Sax, V. Isakov / Atmospheric Environment 37 (2003) 3481–3489
estimate a range of uncertainty or variability in the
temporal allocation of emissions. In this case study the
spatial location of emissions was uncertain because
welding equipment is mobile and locations cannot be
precisely identified, and variable because activities in any
given day are based upon the type of work scheduled for
that day at various locations.
To estimate the spatial allocation of emissions, we
toured the facility and discussed the location of welding
operations with facility engineers and managers.
Through our discussions with facility personnel we
determined certain types of welding process-alloy
combinations can be identified to occur only in specified
areas. These judgements, while potentially subject to
bias and error (Morgan and Henrion, 1990), are based
upon the best information available regarding the
subject facility. We identified 13 generalized locations
where welding can occur, as shown in Fig. 1. We used six
area sources to represent emissions generated on
automated manufacturing lines where most GFCAW
welding is conducted, a volume source to estimate
emissions from a machine shop where most stainless
steel welding was conducted, and six groups of volume
sources to estimate the location of welding emissions
generated from outdoor machining areas, dry docks,
and wet docks.
Based on discussions with facility personnel we
assigned a range of the percent of total emissions for
each process-alloy combination to each source. Lacking
Fig. 1. Schematic map of source locations for three different approaches: database, health risk assessment (HRA), and Monte Carlo
(MC).
any information to estimate the characteristics of this
variability and uncertainty, we applied uniform distribu-
tions (Law and Kelton, 2000). In our simulation the
location of emissions fluctuated between multiple
volume sources in each of the six volume source groups,
and between each of the 13 generalized locations. When
conducting Monte Carlo analysis, we selected a single
emission value by process alloy combination for each
iteration. We then allocated emissions by sampling from
uniform distributions representing the ratio of emissions
from each process-alloy combination attributable to
each release location. When allocating emissions from
each process-alloy combination to each location we
ensured the total mass of emissions was conserved. Our
analysis indicated emissions at most locations fluctuated
20–30% in either direction of median emission rates.
Predicted variability and uncertainty in the allocation of
emissions to the machine shop was minimal (o5% in
either direction of median emission rates) because nearly
all stainless steel emissions were released from this
location.
5. Uncertainty due to model parameters
Both ISCST3 and AERMOD can be run with some
degree of variation in model setup consisting of
differences in model parameters (US EPA, 1995, 1998).
Our approach is to assign a single model parameter
 ARTICLE IN PRESS
T. Sax, V. Isakov / Atmospheric Environment 37 (2003) 3481–3489
where one value is appropriate, and to assign a range of
values where one would not be appropriate. When a
range is assigned, we conduct a sensitivity analysis in
each model to determine the range of predicted
concentrations at each receptor that can be generated
from the range of model parameters. The distribution of
predicted concentrations represents parameter uncer-
tainty.
For this case study, all modeling runs were conducted
using the urban option in ISCST3 and AERMOD. We
did not consider atmospheric reduction or deposition of
CrVI due to the proximity of receptors to sources.
Model parameter ranges were assigned for release
parameters at each location, terrain, building down-
wash, albedo, Bowen ratio, and surface roughness.
To run each model, we specified a base case consisting
of median emission rates from each release location, a
single meteorology data file, and a single choice for each
model parameter. To estimate uncertainty caused by
parameter configurations, we ran the base case multiple
times for each model, replacing the base case model
parameters with different parameter configurations. For
this case study, we analyzed four receptors located
approximately 100 (receptor 1), 500 (receptor 2), 1000
(receptor 3), and 1500 (receptor 4) meters downwind of
emission sources, as shown in Fig. 2. Modeling results
suggested predicted concentrations ranged about 30% in
either direction of median values due to parameter
uncertainty.
Fig. 2. Map of the modeling domain showing source and receptor locations. Contour lines indicate median annual average
concentrations (ng m
3) of CrVI as predicted by ISCST3 using on-site meteorological data.
3485
6. Uncertainty due to meteorology
Weather conditions and meteorological data sets used
to model them exhibit variability over time. The
application of meteorological data sets from a single
location to a modeling domain is also uncertain. In this
study, we used on-site meteorology and considered how
inter-annual variability in meteorology affects uncer-
tainty in long-term annual average CrVI concentration
estimates. Two data sources were available for use: a
proximate National Weather Service site and a tempor-
ary monitoring site located in the modeling domain.
Observations at the temporary monitoring site were
measured about 1 km downwind of emission sources, as
shown in Fig. 2. To develop a meteorological data set
suitable for Gaussian models we combined on-site
surface measurements with upper air conditions from
the nearest NWS site, located about 3 miles northwest of
the case study domain. Meteorological characteristics
between the case study site and NWS site were generally
similar, but some differences between the two sites were
apparent, and were caused by differences in local
geographic and topographic conditions. We used the
on-site data set as our base case for modeling, and
assumed that the data set adequately characterized
meteorological conditions in the case study domain.
To analyze inter-annual variability in meteorology, we
first obtained 10 years of surface and upper air
observations at the nearest NWS site. We then ran both
 ARTICLE IN PRESS
3486 T. Sax, V. Isakov / Atmospheric Environment 37 (2003) 3481–3489
models in the base case, replacing the on-site meteor-
ological data set with each of ten data sets based upon
NWS data, and analyzed concentration results at each
of the four analyzed receptors. Results indicated up to
25% variability in pollutant concentrations at each
receptor might be expected due to inter-annual varia-
bility in meteorology.
7. Monte Carlo analysis to estimate model uncertainty
To estimate a range of predicted pollutant concentra-
tions we ran both ISC and AERMOD in base case
configurations. Running Gaussian models with unit
emission rates for each of the source-receptor combina-
tions is equivalent to running each model with specified
emission rates. From model results, we developed a
matrix of predicted unit concentrations generated by
each source at each of the four analyzed receptors.
To propagate variability and uncertainty, data repre-
senting each model component were fit to PDFs using
polynomial regression unless otherwise indicated. We
compiled emissions distributions by process-alloy com-
bination, and spatial allocation distributions by process-
alloy combination and release location. Uncertainty in
model parameters was normalized to predicted concen-
trations at each receptor using the base case. To assess
variability due to meteorology, we analyzed 10 years of
Fig. 3. Distributions of uncertainty for each of the four receptors simulated by Monte Carlo runs. Distributions are represented by box
and whisker plot: 212 percentile, 10th percentile, 25th percentile, mean, 75th percentile, 90th percentile, and 9712 percentile.
NWS data and assumed variability at the NWS site
would be equivalent to variability at the on-site location.
We analyzed this variability by normalizing these results
to predicted concentrations at each receptor and
calculating a standard deviation. We then applied this
standard deviation to the base case model results at each
receptor.
After expressing variability and uncertainty in model
components to PDFs, we defined an equation to post-
process these distributions and estimate a range of
pollutant concentration results.
Pc ¼ EM  SA  Cr  Met  MP; ð1Þ
where, Pc is the annual average pollutant concentration
estimate (ng m
3), EM the annual average emissions by
process/alloy combination (kg yr
1), SA the Spatial
allocation by process/alloy combination and release
location (percent deviation from base case), Cr the Base
case unit concentration at each receptor (ng m
3), based
upon unit emission rate from each location, base case
model parameters, and on-site meteorology, Met the
inter-annual variability in meteorology (percent devia-
tion from base case) and MP the Uncertainty in release
parameters (percent deviation from base case).
To propagate uncertainty using Eq. (1), we applied a
simple random sampling Monte Carlo technique de-
scribed by Efron and Tibshirani (1993). It is important
to note that any equation used to propagate variability
 ARTICLE IN PRESS
T. Sax, V. Isakov / Atmospheric Environment 37 (2003) 3481–3489 3487

and uncertainty in a Monte Carlo approach requires
independence between each of the components (Frey
and Rhodes, 1999). We believe the assumption of
independence is appropriate for Gaussian models, but
may not be appropriate for more complex modeling
systems that incorporate photochemistry.
8. Results
to the predicted base case pollutant concentration.
Emissions were the dominant source of uncertainty in
this case. Both the spatial allocation of emissions and
meteorology would have been more important factors in
this case if information specific to the facility and
modeling domain had not been obtained. Fig. 4 also
demonstrates AERMOD is capable of producing a
greater range of results than ISCST3, and this range is
inversely proportional to source-receptor distance for
low-level sources.

Our results indicated the 95% confidence interval of

predicted pollutant concentrations spanned roughly an
order of magnitude at each receptor, as shown in Fig. 3.
From a practical perspective, the magnitude of uncer-
tainty will be most important where concentrations are
highest. Results also demonstrated the predicted relative
range of pollutant concentrations at receptor 1 was
smaller than for receptors 2–4. Predicted concentrations
at receptor 1 were primarily affected by emissions from
stainless steel at the machine shop, while other receptors
were primarily affected by GFCAW and SMAW
processes involving low alloy and mild steels at
manufacturing lines.
Fig. 4 shows the contribution of each model
component to uncertainty at each receptor, normalized
9. Comparison to simplified modeling approaches
For regulatory purposes it is often useful to use
simplified, ISCST3 based point-estimate modeling ap-
proaches. We compared two scenarios to our case study
to identify whether these approaches would be consis-
tent with predicted model uncertainty. In our first
approach, we allocated emissions in ARB’s statewide
emissions inventory database (database approach) to a
volume source located at the recorded location of the
facility. In our second approach, we reproduced model-
ing runs conducted by the facility for a regulatory
risk assessment (regulatory health risk assessment

Fig. 4. Distributions of uncertainty at each of the modeling receptors for five model components: emissions inventory (Emis), spatial
allocation of emissions (SA), dispersion models (ISCST3 and AERMOD), and Meteorology (Met). (A) Receptor 1, (B) Receptor 2, (C)
Receptor 3, (D) Receptor 4. Distributions are represented by box and whisker plot: 212 percentile, 10th percentile, 25th percentile, mean,
75th percentile, 90th percentile, and 9712 percentile. All distributions are normalized to the predicted base case concentration at each
receptor.
 ARTICLE IN PRESS
3488 T. Sax, V. Isakov / Atmospheric Environment 37 (2003) 3481–3489
Table 2
Comparison of Monte Carlo analysis results to simplified modeling approaches
CrVI concentration (ng m
3) by modeling approach
Receptor Median Monte 95% confidence interval
Carlo result Monte Carlo results
1 0.7 0.3–1.5
2 0.1 0.03–0.3
3 0.03 0.01–0.08
4 0.01 0.004–0.03
approach). For each scenario model runs were created
using on-site meteorology and results were compared to
the four receptors, as shown in Table 2. Results were
consistent with estimated variability and uncertainty
except at receptor 1 using available information in
ARB’s statewide emissions database.
10. Conclusions
In this case study, we used established uncertainty
analysis techniques to demonstrate a generalized method
for assessing variability and uncertainty in Gaussian air
pollutant dispersion modeling systems. Our results in
this case study indicated the range of uncertainty at each
receptor spanned an order of magnitude. However,
uncertainty is most important at receptors with highest
predicted concentrations. In this case emissions were the
dominant source of uncertainty. Gaussian models are
sensitive to emissions, the location of emission releases,
and meteorology, so these inputs must be well-char-
acterized to minimize uncertainty in model results.
Simplified modeling approaches may lead to errors in
pollutant concentration estimates, especially in close
proximity to emissions sources where concentrations are
likely to be highest.
AERMOD predicted a greater range of pollutant
concentrations than ISCST3 for low-level sources in this
case. This range increased as source-receptor distance
decreased. If AERMOD is to be applied on a regulatory
basis in the future, it will be important to minimize
uncertainty in model parameters by using consistently
applied modeling approaches. Additional research and
guidance may be necessary to ensure AERMOD
applications are sufficiently credible to ensure consistent
risk management decisions.
11. Disclaimer
The contents of this paper and the findings of its
authors do not necessarily reflect the opinions, views,
and policies of the California Air Resources Board.
Regulatory health risk Database approach
assessment approach
1.2 2.5
0.09 0.08
0.03 0.03
0.01 0.01
References
American Welding Society (AWS), 1969. Standards AWS A5.1-
69, A5.2-69, A5.3-69, A5.4-69, A5.5-69, A5.6-69, A5.8-69,
A5.9-69, A5.10-69, A5.11-69, A5.12-69, A5.14-69, A5.15-69,
A5.18-69, A5.19-69, A5.2-69. American Welding Society,
Miami, FL.
Bergin, M.S., Noblet, G.S., Petrini, K., Dhieux, J.R., Milford,
J.B., Harley, R.A., 1999. Formal Uncertainty Analysis of a
Lagrangian Photochemical Air Pollution Model. Environ-
mental Science and Technology 33, 1116–1126.
California Air Resources Board (CARB), 2000. Neighborhood
Assessment Program Work Plan, June 30, Available online
at http://www.arb.ca.gov/ch/programs/nap/napworkplan.
pdf.
Castner, H.R., Null, C.L., 1998. Chromium, Nickel and
Manganese in Shipyard Welding Fumes. Welding Journal
June Supplement, S223–S231.
Cullen, A.C., Frey, H.C., 1999. Probabilistic Techniques in
Exposure Assessment: a Handbook for Dealing with
Variability and Uncertainty in Risk Analysis. Plenum Press,
New York.
Dabberdt, W.F., Miller, E., 2000. Uncertainty, ensembles, and
air quality dispersion modeling: applications and challenges.
Atmospheric Environment 34, 4667–4673.
Efron, B., Tibshirani, R.J., 1993. An Introduction to the
Bootstrap. Chapman & Hall/CRC, New York, NY.
Freeman, D.L., Egami, R.T., Robinson, N.F., Watson, J.G.,
1986. A method for propagating measurement uncertainties
through dispersion models. Journal of the Air Pollution
Control Association 36, 246–253.
Frey, H.C., 1993. Separating variability and uncertainty in
exposure assessment: motivations and method. Proceedings
of the 86th Annual Meeting Air and Waste Management
Association, Paper No. 93-79.01, June.
Frey, H.C., Li, S., 2002. Methods for quantifying variability
and uncertainty in AP-42 emission factors: case studies for
natural gas-fueled engines. Emissions Inventories—Partner-
ing for the Future. Proceedings of the EPA 11th Interna-
tional Emission Inventory Conference, April 15–18, 2002,
available at http://www.epa.gov/ttn/chief/conference/ei11.
Frey, H.C., Rhodes, D.S., 1996. Characterizing, simulating,
and analyzing variability and uncertainty: an illustration of
methods using an air toxics example. Journal of Human and
Ecological Risk Assessment 2, 762–797.
Frey, H.C., Rhodes, D.S., 1999. Quantitative Analysis of
Variability and Uncertainty in Environmental Data and
 ARTICLE IN PRESS
T. Sax, V. Isakov / Atmospheric Environment 37 (2003) 3481–3489
Models, Vol. 1. Theory and Methodology Based Upon
Bootstrap Simulation. US Department of Energy DOE/ER/
30250—Vol. 1.
Frey, H.C., Zheng, J., 2002. Method for Development of
Probabilistic Emission Inventories: Example Case Study for
Utility NOx Emissions. Emissions Inventories—Partnering
for the Future. Proceedings of the EPA 11th International
Emission Inventory Conference, April 15–18, 2002. Avail-
able at http://www.epa.gov/ttn/chief/conference/ei11.
Hanna, S.R., Chang, J.C., Fernau, M.E., 1998. Monte Carlo
estimates of uncertainties in predictions by a photochemical
grid model (UAM-IV) due to uncertainties in input
variables. Atmospheric Environment 32, 3619–3628.
Hanna, S.R., Zhigang, L., Frey, H.C., Wheeler, N., Vukovich,
J., Arunachalam, S., Fernau, M., Hansen, D.A., 2000.
Uncertainties in predicted ozone concentrations due to
input uncertainties for the UAM-V photochemical grid
model applied to the July 1995 OTAG domain. Atmo-
spheric Environment 35, 891–903.
Irwin, J.S., Rao, S.T., Petersen, W.B., Turner, D.B., 1987.
Relating error bounds for maximum concentration esti-
mates to diffusion meteorology uncertainty. Atmospheric
Environment 21, 1927–1937.
3489
Law, A.M., Kelton, D.W., 2000. Simulation Modeling and
Analysis, 3rd Edition. McGraw-Hill, San Francisco, CA.
Midwest Research Institute (MRI), 1994. Development of
Particulate and Hazardous Emission Factors for Electric
Arc Welding (AP-42 Section 12.19) Final Report. EPA
Contract No. 68-D2-0159, US Environmental Protection
Agency.
Morgan, M.G., Henrion, M., 1990. Uncertainty. Cambridge
University Press, New York.
Seigneur, C., Constantinou, E., Permutt, T., 1992. Uncertainty
Analysis of Health Risk Estimates. Document No. 2460-
009-510, Electric Power Research Institute, Palo Alto, CA.
US Environmental Protection Agency (USEPA), 1995. User’s
Guide for the Industrial Source Complex (ISC3) Dispersion
Model. Volume I. EPA-454/B-95-003a, USEPA, Research
Triangle Park, NC 27711.
US Environmental Protection Agency (USEPA), 1998. User’s
Guide for the AMS/EPA Regulatory Model—AERMOD.
A-99-05, II-A-2, OAQPS, Research Triangle Park, NC
27711.
Venkatram, A., 1981. Model predictability with reference to
concentrations associated with point sources. Atmospheric
Environment 15, 1517–1522.