IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 32, NO.
2, MARCH 2017
Evaluating the Performance of the IEEE Standard
1366 Method for Identifying Major Event Days
Joseph H. Eto, Member, IEEE, Kristina Hamachi LaCommare, Michael D. Sohn,
and Heidemarie C. Caswell, Member, IEEE
Abstract—IEEE Standard 1366 offers a method for segmenting
reliability performance data to isolate the effects of major events
from the underlying year-to-year trends in reliability. Recent anal-
ysis by the IEEE Distribution Reliability Working Group (DRWG)
has found that reliability performance of some utilities differs from
the expectations that helped guide the development of the Standard
1366 method. This paper proposes quantitative metrics to evaluate
the performance of the Standard 1366 method in identifying major
events and in reducing year-to-year variability in utility reliability.
The metrics are applied to a large sample of utility-reported reli-
ability data to assess performance of the method with alternative
specifications that have been considered by the DRWG. We find
that none of the alternatives perform uniformly “better” than the
current Standard 1366 method. That is, none of the modifications
uniformly lowers the year-to-year variability in System Average
Interruption Duration Index without major events. Instead, for
any given alternative, while it may lower the value of this metric
for some utilities, it also increases it for other utilities (sometimes
dramatically). Thus, we illustrate some of the trade-offs that must
be considered in using the Standard 1366 method and highlight
the usefulness of the metrics we have proposed in conducting these
evaluations.
Index Terms—Electricity reliability, IEEE Standard 1366, major
events, power system reliability, reliability, system average inter-
ruption duration index.
I. INTRODUCTION
EVERE, yet infrequent, events (often the result of weather)
S can dramatically affect the year-to-year reliability perfor-
mance of a utility. Such unforeseen events can cause widespread
and prolonged interruptions of electric service on only a handful
of days, which can easily exceed the cumulative interruptions
experienced by customers over the remainder of a year. If factors
such as these are not taken into account, it is not only difficult
to assess a utility’s reliability performance over time, but likely
to lead to flawed conclusions because the number and severity
of these events varies greatly from year to year.
Manuscript received November 23, 2015; revised February 29, 2016 and
April 28, 2016; accepted June 24, 2016. Date of publication July 18, 2016; date days.” Now, almost 20 years later, the DRWG has amassed a
of current version February 16, 2017. The work described in this report was
funded by the National Electricity Delivery Division of the U.S. Department of
Energy’s Office of Electricity Delivery and Energy Reliability under Lawrence
Berkeley National Laboratory Contract No. DE-AC02-05CH11231. Paper no.
TPWRS-01665-2015.
J. H. Eto, K. H. LaCommare, and M. D. Sohn are with Lawrence Berke-
ley National Laboratory, Berkeley, CA 94720 USA (e-mail: jheto@lbl.gov;
kshamachi@lbl.gov; mdsohn@lbl.gov).
H. C. Caswell is with PacifiCorp, Portland, OR 97232 USA (e-mail:
heide.caswell@pacificorp.com).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TPWRS.2016.2585978
0885-8950 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications standards/publications/rights/index.html for more information.
1327
To address these concerns, utilities and their regulators have
developed methods for segmenting the reporting of reliability
performance using a concept known as “major events.” Major
events are defined periods during the year when the electricity
delivery system is exposed to stresses that are beyond normal
expectations, typically as a result of severe weather. Identifying
them explicitly provides a means for treating them separately
in reviewing annual performance metrics. The development of
these methods is considered foundational in providing a cred-
ible, historical assessment of utility reliability. This paper ex-
amines the method used by industry to identify major events; it
does not explore the causes of major events.
Early on, the definitions for major events were based on the
local experiences of utilities, their regulators, or an oversight
board. As a result, there was considerable variation in how each
utility defined a major event [1], [2]. In the late 1990s, the Insti-
tute of Electrical and Electronics Engineers (IEEE) Distribution
Reliability Working Group (DRWG) pioneered the development
of a statistically-based, standard method for identifying major
events. The method relies on identifying what are called “ma-
jor event days” or MEDs. The method was developed based on
an expectation that it would identify, on average, 2 to 5 MEDs
per year, a notable limitation to using this method, as not all utili-
ties necessarily fall within this expected range. For the most part,
however, this approach represents the vast majority of utilities.
The method is now embodied in the IEEE voluntary Standard
1366 [3]–[5]. (For brevity, we shall refer to this as the Standard
1366 method in this paper.)
Once the development of a transparent major event screening
method was developed, the IEEE DRWG began ongoing col-
lection of detailed daily utility-reported reliability information
from members of the working group. This work was undertaken
so engineers could assess their reliability performance and the
impact of various engineering practices on electricity reliability
as well as establish a credible screening process for “outlier
much larger historical record of information on utility reliability
performance than was available to the original developers of the
Standard 1366 method. The DRWG has conducted analysis us-
ing this larger record of information and has found that, for some
utilities, application of the Standard 1366 method consistently
leads to the identification of significantly more MEDs per year
than was expected [6]. This has led to discussion of possible
alternatives or modifications to the Standard 1366 method.
This paper develops two metrics that quantify the perfor-
mance of such alternatives compared to the performance of the
1328
Standard 1366 method. The metrics formalize two fundamental
objectives of the Standard 1366 method; namely, identification
of a reasonable number of MEDs per year and, through removal
of these MEDs, reduction in the year-to-year variability of the
System Average Interruption Duration Index (SAIDI) and the
System Average Interruption Frequency Index (SAIFI). We use
the metrics to evaluate the performance of alternative formula-
tions of the Standard 1366 method, which have been suggested
by the DRWG and others. The analysis is conducted using a
large set of utility reliability performance information collected
by the DRWG.
We organize this paper as follows:
1) In Section II, we provide background on the Standard 1366
method. The background emphasizes that the purpose of
the method is to provide a reliable basis for separating the
highly varying influences of major events from year-to-
year assessments of reliability performance.
2) In Section III, we describe the information from the IEEE
DRWG Benchmark study that serves as the basis for our
evaluation.
3) In Section IV, we describe alternatives to the basic for-
mulation of the Standard 1366 method that have been
suggested by the DRWG and others. We then compare
the performance of these alternatives to the performance
of the Standard 1366 method using two metrics, which
measure different aspects of performance.
4) In Section V, we summarize our findings and offer
thoughts on possible next steps.
II. IEEE STANDARD 1366 METHOD FOR IDENTIFYING
MAJOR EVENTS
This paper focuses on sustained interruptions1 and the method
articulated in Standard 1366 to partition reporting using the con-
cept of the MED. The metric for assessing sustained interrup-
tions is called the SAIDI. It can be thought of as the total number
of minutes a customer, on average, is without electric service
over a specified time period. It is defined as follows:
Σ Customer Interruption Durations
SAIDI = (1)
Σ Customers Served
SAIDI is generally reported on an annual basis. However,
it can be calculated over any time period. For example, the
annual SAIDI represents the number of minutes customers on
average are without power over an entire year, while the daily
SAIDI represents the number of minutes customers on average
are without power on any given day within that year. Although
annual SAIDI are more commonly reported by utilities, daily
values of SAIDI form the basis for identifying major events
using IEEE Standard 1366.
A. Definition of a Major Event Day
IEEE Standard 1366-2012 defines MEDs as follows [5]:
1 IEEE Std. 1366-2012 defines a momentary interruption as an interruption
of five minutes or less and a sustained interruption as any interruption that
is not a momentary interruption; effectively, a sustained interruption refers to
interruptions lasting longer than five minutes.
IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 32, NO. 2, MARCH 2017
A major event day is a day in which the daily SAIDI exceeds a
Major Event Day threshold value [called TMED]. For the purposes of
calculating daily system SAIDI, any interruption that spans multiple
calendar days is accrued to the day on which the interruption began.
Statistically, days having a daily system SAIDI greater than TMED
are days on which the energy delivery system experienced stresses
beyond that normally expected (such as during severe weather).
Activities that occur on Major Event Days should be separately
analyzed and reported.
The method for identifying MEDs is based on calculating a
threshold value for daily SAIDI. Daily values that exceed the
threshold are removed from the calculation of annual SAIDI.
Annual SAIDI calculated in this way is commonly referred to
as “SAIDI without major events.”
Standard 1366 requires the previous five years of daily SAIDI
to identify the threshold value. Equation (2) represents the
threshold value, which is known as TM ED , and is as follows:
TM ED = e(α +2.5β ) (2)
where for each utility:
α is the log-normal average of the previous five years of daily
SAIDI;
β is the log-normal standard deviation of the previous five years
of daily SAIDI.
B. Origin and Objective of the Standard 1366 Method
Investigations of reliability data available in the mid-1990s
led industry experts to observe that daily SAIDI values, when
plotted from smallest to largest, followed a nearly log-normal
distribution [7]–[9]. They observed that this pattern held fairly
consistent until the very largest daily SAIDI values were plotted.
That is, these values were observed to be much larger than would
be expected if the distribution were log-normal. For the purpose
of assessing performance without major events, these very large
daily SAIDI values underscored the need for segmentation and
supported the use of log-normal as an appropriate distribution
to use in determining threshold values.
These observations led to the formulation of the Standard
1366 method. The developers expected that, on average, the
method would lead to the identification of 2 to 5 MEDs per
year. This, of course, was no more than an expectation regarding
the average performance of the method over time. It was fully
understood that in any given year the number of MEDs could
and should fluctuate higher or lower than this expected average.
See, for example, [8].
Still, it was expected that the log-normal distribution would
be generally appropriate for most utilities, independent of size
or other utility-specific characteristics [1], particularly since that
was one of the goals of the method.
It is important to emphasize that the purpose of the Stan-
dard 1366 method is not specifically to identify MEDs. In-
stead, the purpose is to establish a reasonable, transparent, and
easily reproducible means for segmenting a utility’s reliability
performance so that routine reliability events can be assessed
separately from major reliability events, which occur only a few
days each year, yet have large impacts on annual SAIDI. Consis-
tent segmentation enables independent assessment of trends in
ETO et al.: EVALUATING THE PERFORMANCE OF THE IEEE STANDARD 1366 METHOD FOR IDENTIFYING
performance during major events versus routine events, includ-
ing identification of variations over time.
III. THE IEEE DRWG BENCHMARK STUDY
The DRWG conducts an annual benchmarking exercise in
which utilities voluntarily submit their daily SAIDI (and SAIFI)
information for analysis. The DRWG calculates a variety of reli-
ability metrics following the Standard 1366 method and presents
their findings at the annual meeting. The meeting materials are
posted on the IEEE Power Engineering Society website, which
documents the work of the committee over the years.2 Informa-
tion on each utility is made anonymous by assigning a unique
identifier. Utilities that submit their data are able to assess their
performance relative to peers both within their region and across
the country.
The DRWG has been performing this benchmarking service
for its membership for many years. As a result, it has amassed a
large data set of daily SAIDI values from roughly 100 utilities.
For some of these utilities, the records span nearly 20 years.3
The DRWG received permission from a large number of the
utilities that participate in the survey to allow their data to be
used in this analysis. This work focuses on years 2005–2012
because it provides the largest pool of utilities to analyze.4
In addition, discussions with members of the DRWG indicated
that more recent reliability performance data are likely more
accurate than the more historic data. This is due to the adoption
of automated outage management systems by utilities to record
customer power interruptions. Our analysis data set consists of
complete records for 80 of the roughly 100 utilities that have
provided data to the IEEE DRWG Benchmark Study.
IV. EVALUATING PERFORMANCE OF THE STANDARD
1366 METHOD
In this section, we evaluate the Standard 1366 method by
comparing its performance to that of alternative specifications.
We use two metrics to compare performance. The metrics for-
malize the two fundamental objectives of the Standard 1366
method; namely, identification of a reasonable number of MEDs
per year and, through removal of these MEDs, reduction in the
year-to-year variability of SAIDI (and SAIFI). We use the met-
rics to evaluate three classes of alternative specifications to the
Standard 1366 method for calculating TM ED .
The first class of alternatives considers different multipliers
for β, the log-normal standard deviation of the previous five
years of historic daily SAIDI. Standard 1366 specifies the use
2 For more information please see: http://grouper.ieee.org/groups/td/dist/sd/
doc/2014-08-Benchmarking-Results-2013.pdf.
3 One challenge that we cannot address in our analysis is inconsistencies that
may exist in the data originally provided by the utilities to the IEEE Benchmark
Study. Industry sources have made us aware that, especially, in the early years
of the survey when early versions of IEEE Standard 1366 were also under
development, utility reporting practices sometimes were (understandably) not
all consistent with the final guidelines articulated in IEEE Standard 1366.
4 Note that for our analysis to begin with year 2005, IEEE Standard 1366
requires daily SAIDI values for the five years prior to 2005 (2000–2004). So,
in fact, our analysis is based on data from the IEEE Benchmarking Study that
dates to 2000.
1329
Fig. 1. The effect of changing the multiplier on beta from 2.5 to 2.0, 3.0,
and 4.0 on the average number of MEDs and year-to-year variability in SAIDI
without major events.
of a multiplier of 2.5. Based on suggestions by the DRWG, we
examine using multipliers of two, three, and four. The second
class of alternatives considers greater and fewer historical years
to estimate the log-normal distribution, which is used to deter-
mine β. Standard 1366 specifies the use of five historical years.
Based on suggestions by the DRWG, we explore using six, four,
three, and two historical years. The third class of alternatives
considers more than one distribution to fit the log-transformed
data. Standard 1366 uses a single distribution; we examine a
bimodal distribution.
We use two metrics to evaluate the performance of alterna-
tives to the Standard 1366 method. The first metric is the average
number of MEDs per year identified by each alternative, a met-
ric the DRWG has been evaluating since the development of the
standard. The second metric is the annual variability in SAIDI
without major events, as measured by the standard deviation.
The performance of the alternatives is expressed by examining
the percentage change in the value of each of these metrics com-
pared to its value when applied to the Standard 1366 method.
Using both metrics highlights the important trade-offs that need
to be considered when exploring alternative approaches. For ex-
ample, a percentage decrease in average MEDs per year means
that the alternative yields fewer MEDs per year compared to
the Standard 1366 method. Similarly, a percentage decrease in
annual variability in SAIDI without major events means that the
alternative reduces year-to-year variability in SAIDI without
major events compared to the Standard 1366 method.
A. Effect of Using Different Multipliers for β to Calculate
TM ED
Fig. 1 shows how both the year-to-year variability in SAIDI
without major events and the annual average number of MEDs
change if the 2.5 multiplier for β is replaced by multipliers of
two, three, and four. Looking at the results for the alternative
involving a multiplier of four (blue dots in Fig. 1) dramatizes
the interplay between the two performance metrics. A multiplier
1330
Fig. 2. The effect of using varying historical years to calculate TM E D on the
average number of MEDs per year over time and year-to-year variability in
SAIDI without major events.
of four, by definition, will lower the average number of MEDs
per year. Yet, in lowering the average number of MEDs per
year, year-to-year variability in SAIDI without major events is
increased significantly. Thus, for the majority of utilities, using
a multiplier of four defeats the purpose of the method because
it increases the year-to-year variability in SAIDI without major
events compared to the Standard 1366 method.
The results using a multiplier of two are mixed. Doing this
(green diamonds in Fig. 1) reduces year-to-year variability in
SAIDI without major events for a slight majority of utilities, but
increases it for the remaining utilities. In almost all cases, using
a multiplier of two also increases the average number of MEDs
per year, sometimes dramatically.
B. Effect of Using Fewer Historical Years to Calculate TM ED
Fig. 2 shows how both the year-to-year variability in SAIDI
without major events and the annual average number of MEDs
change, if the number of historical years of data used to fit
a log-normal distribution is changed from five years to four
or six years.5 In contrast to changing the multiplier for β,
the percentage changes in both performance metrics are much
smaller. All percentage changes are less than 100%.
Of greater significance is the fact that the effect of changing
the number of historical years used to calculate TM ED also
does not affect the reliability metrics uniformly. Increasing and
decreasing the number of historical years both decreases and
increases the standard deviation of SAIDI without major events.
It also increases and decreases the average annual number of
MEDs over time. By and large, the effect of using four historical
years is more modest than the effect of using six historical years.
5 For example, in order to calculate a T
M E D to apply to the year 2010,
Standard 1366 requires daily SAIDI values for the five prior years, 2005-2009.
Following this example, we modified the Standard 1366 method, as follows: We
developed a “four-year TM E D ” based on daily SAIDI for the years 2006-2009
and a “six-year TM E D ” based on daily SAIDI for the years 2004-2009.
IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 32, NO. 2, MARCH 2017
Fig. 3. The effect of using a two-part model to calculate TM E D on average
MEDs/year over time and year-to-year variability in SAIDI without major events
(using data from 2005 to 2012).
C. Effect of Using a Two-Part Mixture Model to Calculate
TM ED
In this subsection, we examine an alternative to the assump-
tion implicit in the Standard 1366 method that the daily SAIDI
values are best fit by a single log-normal distribution. Instead, we
fit two complementary “log-normal” distributions to the data:
one for the smaller values and a separate one targeted for the
larger values. In effect, we consider whether it is better to rep-
resent daily SAIDI values as a bimodal population. Technically,
this is known as a “mixture model.” More information about the
mixture model is presented in the Appendix.
Fig. 3 shows how both the year-to-year variability in SAIDI
without major events and the annual average number of MEDs
change, if a two-part mixture model is used to calculate TM ED .
Results are shown for two-part mixture models using both five
and two historical years of data.
Fig. 3 shows that the effect of using two statistical distri-
butions instead of one to fit log-transformed daily SAIDI is
similar to the effect of increasing the multiplier for β. Gener-
ally speaking, the average MEDs/year over time are reduced,
but year-to-year variability in SAIDI without major events is in-
creased. There are minimal differences between the use of five
and two historical years of data.
D. Assessment of the Performance of the Standard 1366
Method
Table I summarizes the results presented in this section. In
addition to percentage changes in standard deviation of SAIDI
without major events and average MEDs per year over time, the
table also presents percentage changes in the slope of MEDs per
year over time. To provide some perspective on the magnitude of
the differences among the alternatives, two ranges of percentage
change are shown: greater or less than 10% change (for standard
deviation of SAIDI without major events and average MEDs per
year over time) and greater or less than 50% change (for slope
of MEDs per year over time).
ETO et al.: EVALUATING THE PERFORMANCE OF THE IEEE STANDARD 1366 METHOD FOR IDENTIFYING 1331

TABLE I
SUMMARY OF IMPACTS FROM MODIFICATIONS TO THE STANDARD 1366 METHOD

Description % Change in Std Dev of SAIDI w/o MEDs % Change in Average # MEDs % Change in Slope of MEDs

++ + Zero - – ++ + Zero - – ++ + Zero - –

2.0 times multiple of 21 2 0 10 44 77 0 0 0 0 10 5 0 15 47
Beta
3.0 times multiple of 67 4 0 3 3 2 0 0 1 74 3 3 0 7 64
Beta
4.0 times multiple of 70 3 0 4 0 0 0 0 0 77 7 1 1 11 57
Beta
Using 6 historical 22 18 0 10 25 35 20 0 11 11 13 10 3 17 32
years to calculate
Tm e d
Using 4 historical 14 21 19 18 5 2 21 27 21 6 6 15 23 25 8
years to calculate
Tm e d
Using 3 historical 20 20 11 16 10 10 18 20 16 13 6 16 12 32 11
years to calculate
Tm e d
Using 2 historical 30 16 8 6 17 18 18 13 19 9 8 11 9 28 21
years to calculate
Tm e d
Two-part model 57 6 1 4 9 4 0 1 6 66 11 4 4 27 31
using 5 historical
years to calculate
Tm e d
Two-part model 61 4 0 6 6 4 1 1 4 67 10 1 6 27 33
using 4 historical
years to calculate
Tm e d
Two-part model 62 4 0 1 10 5 2 1 2 67 6 6 4 16 45
using 3 historical
years to calculate
Tm e d
Two-part model 63 3 1 5 5 6 0 2 3 66 14 4 3 14 42
using 2 historical
years to calculate
Tm e d
Two-part model 10 1 0 3 63 77 0 0 0 0 29 14 2 2 30
using 5 hist. years to
calculate Tm e d and
2 times multiple of
beta
Two-part model 68 3 0 2 4 1 0 0 0 76 10 3 2 17 45
using 5 hist. years to
calculate Tm e d and
3 times multiple of
beta

++ Denotes change greater than 10% for standard deviation of SAIDI without major events and average MEDs/year, and greater than 50% for slope of MEDs/year.
+ Denotes a change between 0-10% for standard deviation of SAIDI without major events and average MEDs/year, 0-50% for slope of MEDs/year.
- Denotes a change between −10-0% for standard deviation of SAIDI without major events and average MEDs/year, −50%-0% for slope of MEDs/year.
– Denotes a change less than −10% for standard deviation of SAIDI without major events and average MEDs/year, less than 50% for slope of MEDs/year.

The table highlights—for the utilities we examined—that
none of the alternatives performs uniformly “better” than the
Standard 1366 across one or more of the performance at-
tributes (metrics) we considered. That is, none of the alter-
natives uniformly lowers the year-to-year variability in SAIDI
without major events for all utilities. Instead, for any given
alternative, while it may lower the value of this metric for
some utilities, it also increases it for other utilities (sometimes
dramatically).
The table also illustrates the trade-offs that exist among per-
formance metrics. The example presented involving a much
larger multiplier for β shows how decreasing the number of
MEDs per year directly increases the year-to-year variabil-
ity in SAIDI without major events. Thus, trade-offs among
performance metrics must also be considered.
With respect to the objective of reducing year-to-year
variability in SAIDI without major events, in our opinion, the
alternatives that vary the number of historical years used to
calculate TM ED show the greatest promise for further explo-
ration. First, in comparison to the other alternatives considered,
this alternative led to much more modest percentage changes
in this performance metric. Second, this alternative is relatively
straightforward to implement.
It should be noted that Standard 1366 does not discuss
whether applicability of the method depends on specific utility
characteristics. It is conceivable that one or more characteris-
tics of a utility, such as size or geographic location, affect how
well the Standard 1366 controls for year-to-year variability in
reliability performance. These are factors we hope to explore in
future research.
1332 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 32, NO. 2, MARCH 2017

V. CONCLUSION
IEEE Standard 1366 prescribes a method for removing the
effects of major events from annual reliability performance
metrics by segmenting out days of extreme reliability per-
formance in support of benchmarking analysis across utili-
ties of various sizes, environments and other parameters. The
purpose is to establish a reasonable, transparent, and easily
reproducible means for segmenting a utility’s reliability per- Fig. A-1. Examples of mixture model candidates.
formance so that routine reliability events can be assessed sepa-
rately from major reliability events, which occur only a handful APPENDIX
of days each year. These major events can have a dispropor-
tionate effect on the annual reliability performance of a utility. A mixture model represents a combination of multiple distri-
Removing their effects enables better evaluations of year-to-year butions to represent a single data set. As such, mixture models
changes in utility performance in response to routine reliability are well-suited for representing data that exhibit subpopulations.
events. A mixture model is mathematically represented by a distribu-
This paper was motivated by concerns that for a large num- tion f and is a mixture of K component normal distributions as
ber of utilities, the Standard 1366 method was not performing follows:
according to the expectations that were used to develop it. For K

some utilities, application of the method leads to the identifi- f (x) = λk fk (x) (3)
cation of significantly more major events in a year than was k =1
expected and does so consistently over many years. To try and
In (3), λ represents the mixing weights, such that lambda is
understand this issue, members of the IEEE DRWG began ex-
greater than zero and the sum of the weights across K com-
perimenting with alternative formulations of the Standard 1366
ponents equals 1. Although any number of components or
method in an effort to improve its performance. Initially, the
distributions can be used, for simplicity, we considered a two-
group focused on how alternatives affected the number of major
component mixture model. The model we ran uses probabilistic
events identified.
clustering and a weighted expectation maximum algorithm to
As an extension of the efforts made by the DRWG, this paper
determine the weights that best fit each component. One advan-
provides the Working Group and those using Standard 1366 with
tage to using this formulation is that if the data for a given utility
two distinct metrics that can be used to evaluate and compare the
are, in fact, best represented with a single distribution, the mix-
performance of the Standard 1366 method with any proposed
ture model would represent this by placing all the weighting on
alternative. The first metric assesses the number of major events
the first component; that is, λ[SUB1] = 1. The Standard 1366
identified in a given year over time. The second metric assesses
method is then a one-part mixture model.
the resulting year-to-year variability in SAIDI, once the effects
Fig. A-1 below shows two sample utilities in select years
of major events have been removed.
where the five-year daily log-transformed SAIDI shows a
This paper demonstrates how the metrics can be used to il-
skewed distribution with a higher frequency of larger daily
lustrate the trade-offs that arise when considering alternative
SAIDI values than smaller values. For data that are distributed
approaches to using the current Standard 1366 method. As an
like this, a two-part mixture model is able to better represent the
example, increasing the multiplier of β in the TM ED equation
distribution with the use of multiple components. Each compo-
results in the identification of fewer major events, however this
nent, or curve, represents a proportion of the overall distribution
change increases the year-to-year variability of the resulting
using the expectation maximum as the probabilistic technique
SAIDI without major events, which defeats the purpose of the
for representing the likelihood each data point falls within either
method. Hence, increasing the multiplier is not a useful alterna-
component. The result from the fit will be estimates of λ1 , λ2 ,
tive to the current Standard 1366 method.
and the mean and variance parameters for the two individual
In exploring various sets of alternatives to the Standard 1366,
distributions.
we conclude that none perform consistently “better” than the
In order to calculate TM ED using the two-part mixture model,
current method in reducing the year-to-year variability of the
we simply use the 2.5 beta logic in Standard 1366 to identify the
SAIDI without major events. This paper demonstrates quantita-
area under the two curves that represents a confidence interval
tively the “reasonableness” of the current Standard 1366 method
of 99.45% to the total area under their combined area.
compared to the alternatives that have been considered to date.
It is not the intent of this study to suggest that changes to the
Standard 1366 method are warranted. Instead, the purpose has ACKNOWLEDGMENT
been to propose and demonstrate, using a wide cross-section The authors acknowledge the membership of the IEEE Dis-
of information on utility reliability experiences collected over tribution Reliability Working Group both for allowing us to
many years, the use of metrics that can be used to assess the analyze information they have provided to the benchmarking
performance of the Standard 1366 method compared to any study and for their expert review of early presentations of the
alternative. findings presented in this paper.
ETO et al.: EVALUATING THE PERFORMANCE OF THE IEEE STANDARD 1366 METHOD FOR IDENTIFYING
REFERENCES
[1] J. H. Eto and K. H. LaCommare, “Tracking the reliability of the U.S.
Electric Power System: An assessment of publicly available information
reported to state public utility commissions,” Lawrence Berkeley Nat. Lab.,
Berkeley, CA, USA, Rep. LBNL-1092E, Oct. 2008. [Online]. Available:
http://certs.lbl.gov/certs-rtina-pubs.html
[2] J. H. Eto and K. H. LaCommare. “A quantitative assessment of utility
reporting practices for reporting electric power distribution events,” in Proc.
2012 IEEE Power & Energy Soc. General Meeting, San Diego, CA, USA,
Jul. 22–26, 2012.
[3] IEEE Trial-Use Guide for Electric Power Distribution Reliability Indices,
IEEE Std. 1366-1998, Dec. 1998.
[4] IEEE Guide For Electric Power Distribution Reliability Indices, IEEE Std.
1366-2003, May 2004.
[5] IEEE Guide for Electric Power Distribution Reliability Indices, IEEE Std.
1366-2012, May 2012.
[6] I. Hoogendam, “Kolmogorov–Smirnov normality test for distribution reli-
ability data analysis,” in Proc. 2014 IEEE Joint Tech. Committee Meeting,
New Orleans, LA, USA, Jan. 12–16, 2014.
[7] J. Bouford, IEEE DRWG General Meeting personal communication,
Aug. 1, 2014.
[8] R. D. Christie. “Statistical classification of major event days in distribution
system reliability,” IEEE Trans. Power Del., vol. 18, no. 4, pp. 1336–1341,
Oct. 2003.
[9] H. Caswell, V. Forte, J. Fraser, A. Pahwa, T. Short, M. Thatcher, and V.
Werner, “Weather normalization of reliability indices: IEEE Task Force on
Weather Normalization of Reliability Indices.” IEEE Trans. Power Del.,
vol. 26, no. 2, pp. 1273–1279, Apr. 2011.
Joseph H. Eto (M’87) received the A.B. degree in philosophy and the M.S.
degree in energy and resources from the University of California, Berkeley,
CA, USA. He is a Staff Scientist in the Environmental Energy Technologies
Division, Lawrence Berkeley National Laboratory. He has authored more than
150 publications on electricity policy, electricity reliability, transmission plan-
ning, cost-allocation, demand response, distributed energy resources, utility
integrated resource planning, demand-side management, and building energy-
efficiency technologies and markets. He is a registered professional Mechanical
Engineer in the State of California.
1333
Kristina Hamachi LaCommare received the B.S. and M.S. degrees in atmo-
spheric science from the University of California, Davis, CA, USA. She is a
Program Manager in the Electricity Markets and Policy Group. On the research
side, she analyzes efforts to track distribution electricity reliability and improve
the methods used to calculate and report reliability metrics and has also helped
DOE support the preparation of the National Transmission Grid Study. Since
1999, she has been working at the Lawrence Berkeley National Laboratory. She
also manages various project, group finances, subcontracts, and budgets.
Michael D. Sohn received the B.S. and M.S. degrees in mechanical engineering
from the University of California, Los Angeles, CA, USA, and also the M.S.
degree in engineering and public policy and the Ph.D. degree in civil and
environmental engineering from the Carnegie Mellon University, Pittsburgh,
PA, USA. He is the Deputy Leader of the Sustainable Energy Systems Group
and the former Leader of the Airflow and Pollutant Transport Group (Indoor
Environment Department). Since 1998, he has been with the Lawrence Berkeley
National Laboratory. He is a California-licensed Professional Engineer (Civil),
and has worked at an environmental engineering firm where he conducted
environmental health risk assessments.
Heidemarie C. Caswell (M’02) received the B.Sc. degree in civil engineering
from the University of Washington, Seattle, WA, USA, in 1987. She worked
for Puget Sound Energy and its predecessor Washington Natural Gas where she
held numerous positions, including managing, engineering, planning and special
assignments. She is currently the Director of Transmission & Distribution Asset
Performance for PacifiCorp, which is a Berkshire Hathaway Energy Company,
based in Portland, OR, USA. She is a member of the WG on System Design,
for which she is secretary. She Chairs the Data Analysis & Benchmarking
Task Force. She participates in the development of the Major Event definition
for IEEE 1366. She is also the Vice-Chair of the North American Electric
Reliability Corporation’s Performance Analysis Subcommittee.