ORIGINAL STUDY

Teaching Therapeutic Seizure Criteria to Psychiatrists

David Michael Semple, MBChB, MRCPsych, William Gunn, MBChB, MRCPsych, Zoe Davidson, MBChB,
and Filippo Queirazza, MD, MRCPsych

Objectives: Following on from our previous work looking at the

interrater reliability of assessing seizure adequacy for electroconvulsive
therapy (ECT), we sought to examine whether a specic teaching module
could improve the reliability of visual inspection of electroencephalography (EEG) recordings for specic features of seizure length, presence of
polyspike, wave activity, and post-ictal suppression.
Methods: Twelve medical practitioners at varying levels of training and
ECT experience rated 15 EEG traces after minimal training and a further
set of 15 EEG traces after a more detailed training. Results were analyzed
to examine the interrater reliability of the EEG features and the overall assignment of traces as meeting old (1995) or new (2005) Royal College
of Psychiatrists criteria for therapeutic seizures compared with the agreed
ratings of 2 experienced ECT practitioners (standard ratings).
Results: There was evidence for a specic training effect for the old
criteria with an improvement in the interrater reliability (generalized ,
0.590 vs 0.813) associated with a signicantly better estimation of seizure length as assessed by comparison of the root mean square difference from the standard ratings (mean, 7.83 vs 4.49; P < 0.003). The
interrater reliability for the new criteria did not improve (generalized
, 0.599 vs 0.581) but was already at quite a good standard. Examination of individual features did demonstrate improvement in the rating
of activity (generalized , 0.564 vs 0.655) and post-ictal suppression
(generalized , 0.553 vs 0.611) after the training. When these 2 criteria
were grouped together ( suppression), interrater reliability was shown
to be signicantly improved after the training (generalized , 0.568 vs
0.659). Although not statistically signicant, the participants reported
that the training improved their condence in using both criteria for therapeutic seizures (old: 49% vs 67%; new: 27% vs 48%).
Conclusions: The reliability of assessments of seizure length, presence of
activity, and post-ictal suppression can be measurably improved with a specic teaching module. Using the suppression criteria together with the accurate estimation of seizure length on EEG may have greater clinical utility
when it comes to instructing trainees in ECT administration, assessment of
therapeutic seizures, and developing protocols for dose adjustment.
Key Words: electroconvulsive therapy, seizure duration, seizure
monitoring, electroencephalography
(J ECT 2014;30: 220223)

General Adult Psychiatry, Hairmyres Hospital, Glasgow, Scotland.

Received for publication September 20, 2013; accepted October 9, 2013.
Reprints: David Michael Semple, MBChB, MRCPsych, Hairmyres Hospital,
Eaglesham Rd, East Kilbride, Glasgow, G75 8RG Scotland
(email: d.semple@btinternet.com).
The authors have no conicts of interest or nancial disclosures to report.
Note: Specialist training in psychiatry in the UK is 6 years, divided into 3
years of core training (CT1-3) and 3 years of specialty training (ST4-6).
Trainees are called core trainees and specialty registrars, respectively.
An associate specialist (or specialty doctor) is a senior grade doctor
with more than 10 years experience who has, for one reason or
another, chosen not to complete higher specialist training or, having
completed higher specialist training, has not taken up a consultant
appointment (for more information, see
http://www.rcpsych.ac.uk/training/careersinpsychiatry/careersinfoforugs.aspx).
Copyright 2014 by Lippincott Williams & Wilkins
DOI: 10.1097/YCT.0000000000000087

220

www.ectjournal.com

he recent 2013 Royal College of Psychiatrists (RCPsych)

College Report CR176 (published as The ECT Handbook,
third edition)1 states that clinical efcacy of ECT depends on
the induction of generalized cerebral seizure activity (page
62). This is essentially a restatement of the long-established
view that a clear evidence of a generalized seizure is an essential
component of a therapeutic electroconvulsive therapy (ECT)
session. What is less well established is the best way to determine that a therapeutic seizure has occurred. Electroencephalogram (EEG) monitoring is regarded as the simplest means of
assessing cerebral seizure activity,2 and all modern ECT
machines provide this essential facility. The same College Report (2013) describes typical EEG features as widespread
high-frequency spike waves (polyspike activity) followed by
slower spike and wave complexes, typically around 3 Hz
followed by a phase of relative or complete suppression of electrical activity (post-ictal suppression) (page 62) but counsels
against relying solely on EEG monitoring because it is vulnerable to both technical problems in obtaining a good quality trace
as well as difculties in interpretation. Common problems are
as follows: the presence of signicant artifact obscuring features
of the EEG trace, difculties in dening a clear end point, and a
lack of clear post-ictal suppression. Previous studies, including
our own, have provided evidence that experienced clinicians
can make reliable assessments of criteria for therapeutic seizures.3,4 There is also evidence in the literature that teaching
about seizure morphology does improve reliability of ratings.5
We sought to establish whether a specic educational module
on EEG interpretation could improve interrater reliability for criteria of therapeutic seizures. Older criteria, such as those in the
rst edition of The ECT Handbook by the RCPsych, published
in 1995,6 suggested that effective ECT seizures should consist
of a convulsion lasting 15 seconds or more or the EEG recording showing seizure activity lasting 25 seconds or more (old
RCPsych [1995] criteria). The second edition of The ECT Handbook in 2005 did not specify a particular seizure length but
did advise that there should be evidence of typical features on
EEG: polyspike activity, /spike and wave complexes, and
post-ictal suppression (new RCPsych [2005] criteria).7 For
comparison, we decided to use both the old and new RCPsych
criteria for therapeutic seizures. At the time of the study, the
third edition of the RCPsych's ECT Handbook had not been
published, but as quoted earlier, the suggested EEG criteria for
a therapeutic seizure remain unchanged from the second edition.

MATERIALS AND METHODS

Electroencephalogram Traces
Traces from a bank of more than 100 EEG printouts were
graded as being of low, moderate, or high difculty to interpret
according to the clinical consensus of 2 experienced ECT
practitioners. These traces were obtained during routine ECT
sessions at Hairmyres Hospital, East Kilbride, using the
Thymatron System IV ECT machine, manufactured by Somatics,
The Journal of ECT Volume 30, Number 3, September 2014

Copyright 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

The Journal of ECT Volume 30, Number 3, September 2014

LLC. All patients received bilateral ECT using a standarddose titration method. The machine starting settings were
0.5-millisecond pulse width and 10-Hz frequency. Treatment
dose was 1.5 to 2 times estimated seizure threshold. The anesthetic agent used was low-dose propofol, and neuromuscular
blockade was achieved with suxamethonium chloride. The printouts comprised a baseline EEG trace of 10 seconds or more,
followed by the treatment trace. Cerebral activity was recorded
on 2 channels, left and right, using prefrontal-mastoid positioning
of EEG leads. The traces were further subdivided into groups
meeting or not meeting old and new RCPsych criteria for therapeutic seizures induced by ECT. Two balanced groups of 15
traces were then selected for pretraining and posttraining testing.
Only traces for which there was a complete agreement between
the 2 independent ECT practitioners were included. Length of
seizure, presence of polyspike, wave activity, and post-ictal
suppression were assessed. These answers and whether the traces
were judged as meeting old or new RCPsych criteria were
recorded on the standard rating answer sheets, which were
used for marking and statistical analysis.

Teaching Therapeutic Seizure Criteria

activity, presence or absence of post-ictal suppression, and

meeting old or new criteria.
Subsequently, the participants were given a longer talk on
EEG interpretation. This included the typical EEG rhythms:
, , , , , and . There was a discussion of waves in sleep
EEGs, on ECT traces, and on magnetoconvulsive therapy/
magnetic seizure therapy EEG traces. The old RCPsych (1995)
criteria and the new RCPsych (2005) criteria were considered
in the light of recent studies on interrater reliability and the 3
key features (polyspike, /spike, and wave) and post-ictal suppression were looked at in more detail. Further examples of
difculties in interpreting EEG traces were given, such as determining end points, common artifacts, and variations of relative
post-ictal suppression (based on the study of Scott2 and our
own examples). This longer presentation lasted approximately
20 minutes, with 10 minutes for discussion. Afterward, the
participants were taken to another room where the second set
of 15 EEG traces were laid out on tables for them to examine and rate in the same way as before over an additional
15 minutes. At the end of the session, the participants were
asked to rate how condent they felt in using the 2 sets of criteria before and after the training sessions on a Likert scale.

Participants
The 12 participants were a varied group of 5 medical
trainees (2 foundation year and 3 core trainees), 4 specialty
doctors, and 3 consultants, with a male-female ratio of 1:2.
They all were attending the Lanarkshire ECT Training Day on
May 8, 2013. A third of the participants had little or no previous
experience in ECT, a third had some experience, and a third
regarded themselves as competent in ECT. Only 1 participant
stated that they were competent in interpreting EEGs. Of the
others, 7 had little or no previous experience and 4 had some experience in EEGs.

Training Sessions and Test Administration

After a brief presentation of the old and new criteria
denitions and a quick talk about a typical EEG trace (taken
from the Royal College's ECT Handbook [2005]) lasting approximately 5 minutes, the participants were given 15 minutes
to rate the rst 15 traces laid out for visual inspection on tables
around the room. They were given an example of the baseline
trace, comprising a 5- to 10-second EEG recording immediately
before electrical stimulation, along with the recorded length of
the visualized convulsive seizure, and were asked to rate the associated EEG trace for seizure length, presence or absence of
polyspike activity, presence or absence of /spike and wave

Statistical Analysis
Demographic information of those taking part in the study
was tabulated, along with the pretraining and posttraining
ratings of the EEG traces. The generalized statistic and its derived variance8,9 were used using a Microsoft Excel spreadsheet
that calculates the generalized , values for each rating category (along with associated standard error estimates), overall
standard error estimates, associated probability values, and condence intervals (developed by King10 and available for download at http://www.ccit.bcm.tmc.edu/jking/homepage/). This
statistical test determines interrater reliability on the basis of
the number of agreements and disagreements among the judges,
allowing calculation of values for multiple raters. This was
used to calculate values for pretraining and posttraining testing of EEG rating (Table 1). The participants were also asked
to estimate the length of seizure, and the root-mean-square deviation of these values from the standard ratings was calculated. Pretraining and posttraining values were compared using
the paired independent samples t test, with equal variances not assumed. Likert scale values for condence with the old and
new RCPsych criteria pretraining and posttraining were similarly compared (Table 2). Correlations of demographic data
and ratings of seizure characteristics were explored using the

TABLE 1. Generalized Statistic and Its Derived Variance for Pretraining and Posttraining Testing of EEG Ratings (n = 12)
Generalized (Confidence Interval)

Variable
Criteria
Old RCPsych (1995) criteria
New RCPsych (2005) criteria
Suppression criteria
EEG Features
Polyspike
/spike and wave
Post-ictal suppression

2014 Lippincott Williams & Wilkins

Pretraining

Posttraining

0.590 (0.555, 0.625)

0.599 (0.555, 0.645)
0.568 (0.524, 0.612)

0.813 (0.771, 0.856)

0.581 (0.542, 0.620)
0.659 (0.618, 0.699)

0.541 (0.492, 0.589)

0.564 (0.521, 0.606)
0.553 (0.509, 0.597)

0.447 (0.407, 0.488)

0.655 (0.611, 0.701)
0.611 (0.570, 0.651)

www.ectjournal.com

Copyright 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

221

The Journal of ECT Volume 30, Number 3, September 2014

Semple et al

TABLE 2. Independent Samples t Test, Equal Variances not Assumed, for Pretraining and Posttraining Ratings of Seizure Length
and Condence in Using RCPsych Criteria (n = 12)
Mean (Standard Deviation)

t Test

Variable

Pretraining

Posttraining

Root-mean-square difference for seizure length

Percentage condence for old RCPsych (1995) criteria
Percentage condence for new RCPsych (2005) criteria

7.83 (2.91)
48.96 (36.85)
27.07 (27.11)

4.50 (1.62)
67.32 (21.84)
47.77 (29.80)

0.003
0.193
0.143

Pearson correlation coefcient within the SPSS statistical package version 17.0.

ceased. This has long been a reason for routine EEG monitoring13 and is recommended in the latest RCPsych handbook
(page 73).1
The lack of improvement in the reliability of the new
RCPsych (2005) criteria reected a reduction in interrater reliability for ratings of the presence of polyspike activity. This
could be explained by training leading to an increased awareness of the possible occurrence of other rhythms such as subthreshold or thalamic recruiting activity on EEG resulting in
more uncertainty in identifying true polyspike activity. Previous
research has shown that clinicians can be trained to visually inspect the EEG strips during ECT and determine the adequacy of
the seizure by evaluating the amplitude of the ictal EEG relative
to baseline, symmetry of the right and left hemispheric EEGs,
distinct spike and wave pattern, and degree of post-ictal suppression.5 We were able to demonstrate a denite training effect
on the reliability of /spike and wave assessment. This improvement may have been caused by the fact that the training module
focused on activity and highlighted issues of dominant frequency, symmetry, and variations of the typical spike and wave
pattern. Generalized scores for the post-ictal suppression did
rise with training, but the difference did not reach statistical
signicance.
We had thought that previous experience in ECT and/or
EEGs might have inuenced the accuracy of assessing therapeutic seizures. Our data did not support this idea, and people
who claimed to have little experience in EEGs or ECT did not
seem to do any worse on rating EEG criteria, although they
rated themselves as less condent after the training. In this
study, EEG training had particular benecial effects (on activity ratings and seizure length) that were independent of previous
experience probably because previous experience did not include such clear guidance on specic EEG features as was given
in the longer training session.
In both Rattehalli et als3 and our own study4 on interrater
reliability, using the presence of 2 or more of the 3 characteristic
EEG features produced greater reliability. In this study, it seems
that the assessment of /spike and wave activity as well as postictal suppression can be easily improved with training. There
may be utility in using only these 2 features when determining
a therapeutic seizure because they are less subjective than the
presence or absence of polyspike activity is. When grouped together, the interrater reliability of these suppression criteria
does signicantly improve with training (see Table 1 for condence intervals) with generalized values increasing from
0.568 to 0.659. Pragmatic guides, such as the excellent Clinical
Manual of Electroconvulsive Therapy14, suggest that looking
for the evolution of the dominant frequency (ie, activity) is
more useful than looking for characteristic waveforms when
trying to distinguish EEG seizure activity from the EEG activity
induced by anaesthesia where no seizure has been elicited
(page 122). The subsequent sustained slowing of dominant frequency is thought to represent onset of inhibitory mechanisms

RESULTS
A specic training effect was found for the old RCPsych
(1995) criteria with an improvement in interrater reliability
(generalized , 0.590 vs 0.813). This was accounted for by a
signicantly better estimation of seizure length on EEG as
assessed by comparison of the root-mean-square difference
from the standard ratings (mean, 7.83 vs 4.49; P < 0.003).
Interrater reliability for the new criteria did not improve (generalized , 0.599 vs 0.581), although it was already at a good standard. This was caused by a signicant worsening of reliability in
rating polyspike activity (generalized , 0.541 vs 0.447) despite
a signicant improvement in the rating of activity (generalized
, 0.564 vs 0.655) and some improvement in the post-ictal suppression estimation (generalized , 0.553 vs 0.611) after the
training (see Table 1 for condence intervals). Although not statistically signicant, the participants reported that the training
improved their condence in using both criteria for therapeutic seizures (old: 48.96% vs 67.32%; new: 27.07% vs 47.77%;
Table 2). Correlations between demographic data and the ratings
of seizure characteristics revealed one interesting association:
there was a signicant correlation (P = 0.032) with self-rated
EEG experience and condence in using the new but not the
old criteria after the training. However, there was no signicant
correlation between self-rated EEG experience and any of the
ratings of EEG features.

DISCUSSION
A specic teaching module does improve the reliability
of ratings for some aspects of visual inspection of EEG traces
from ECT-induced seizures. Specically, estimations of seizure
length and identication of /spike and wave activity signicantly improve. We found a clear signicant improvement in
the reliability of using the old RCPsych (1995) criteria. This
is perhaps not surprising because the subjectivity of these criteria is related solely to the estimation of seizure length on EEG,
which showed a denite training effect. Unfortunately, the duration of the seizure is no longer regarded as a clear indicator of a
likely benet from ECT. Sackeim et al11 were the rst to demonstrate that the therapeutic benet of the ECT seizure was actually dependent on the amount above the individual patient's
seizure threshold that the stimulus was administered. With some
important caveats relating to individual variation and the effects
of medication, changes in seizure length during a course of ECT
can be an important indicator of changes in seizure threshold.12
Taking the clinical context into account, an accurate assessment
of seizure length on EEG is crucial to ensure that appropriate
adjustments in ECT dose are made. Greater ability in the identication of the seizure end point would also help in excluding
prolonged seizures by better determining that the seizure has

222

www.ectjournal.com

2014 Lippincott Williams & Wilkins

Copyright 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

The Journal of ECT Volume 30, Number 3, September 2014

that ultimately produce the characteristic post-ictal suppression

because of reduced activity in the refractory phase of neuronal
depolarization. There is also evidence that the degree of postictal suppression may be correlated with ECT efcacy.15,16
The results of this study suggest that the detection of these
2 features, suppression together with the accurate estimation
of seizure length on EEG, may have greater clinical utility when
it comes to initially instructing trainees in ECT administration,
assessment of therapeutic seizures, and developing protocols
for dose adjustment. More subtle interpretations of the EEG
trace can then be built into training programs with further
hands-on practical experience and the supervision of an experienced ECT practitioner. Future research should address whether
doing so ultimately enhances the therapeutic outcome for
patients receiving a course of ECT.
ACKNOWLEDGMENTS
The authors thank the ECT team at Hairmyres Hospital
and the psychiatrists who took part in the teaching exercise
for their enthusiasm and support in helping to make this study
possible.
REFERENCES

Teaching Therapeutic Seizure Criteria

5. McCall WV, Farah A, Reboussin DM. Can we teach psychiatric

residents to rate seizure regularity? Convuls Ther.
1995;11:248252.
6. Scott AI, Lock T. Monitoring seizure activity. In: Freeman CP, ed. The
ECT Handbook. London, UK: Gaskell/The Royal College of
Psychiatrists; 1995.
7. Whitehouse AM, Scott AI. Monitoring seizure activity. In: Scott AI, ed.
The ECT Handbook. London, UK: The Royal College of Psychiatrists;
2005.
8. Fleiss JL. Measuring nominal scale agreement among many raters.
Psychol Bull. 1971;76:378382.
9. Fleiss JL, Nee JCM, Landis JR. Large sample variance of kappa in the
case of different sets of raters. Psychol Bull. 1979;86:974977.
10. King JE. Software solutions for obtaining a kappa-type statistic for use
with multiple raters. Annual Meeting of the Southwest Educational
Research Association; 2004; Dallas, TX.
11. Sackeim HA, Decina P, Kanzler M, et al. Effects of electrode placement
on the efcacy of titrated, low-dose ECT. Am J Psychiatry.
1987;144:14491455.
12. Lalla FR, Milroy T. The current status of seizure duration in the practice
of electroconvulsive therapy. Can J Psychiatry. 1996;41:299304.

1. Waite J, Easton A, eds. The ECT Handbook. 3rd ed. London: Royal
College of Psychiatrists; 2013.

13. Weiner RD, Krystal AD. EEG monitoring of ECT seizures. In: Coffey
CE, ed. The Clinical Science of Electroconvulsive Therapy. Washington, DC:
American Psychiatric Press; 1993:93109.

2. Scott AIF. Monitoring electroconvulsive therapy by

electroencephalogram: and update for ECT practitioners. Adv Psychiatr
Treat. 2007;13:298304.

14. Mankad MV, Beyer JL, Weiner RD, et al. Clinical Manual of
Electroconvulsive Therapy. Arlington, VA: American Psychiatric
Publishing, Inc; 2010.

3. Rattehalli RD, Thirthalli J, Rawat V, et al. Measuring

electroencephalographic seizure adequacy during electroconvulsive
therapy: a comparison of 2 denitions. J ECT. 2009;25:243245.

15. Mayur P. Ictal electroencephalographic characteristics during

electroconvulsive therapy: a review of determination and clinical
relevance. J ECT. 2006;22:213217.

4. Semple DM, Gunn W, Ellahi R, et al. Interrater reliability of 2

denitions of seizure adequacy in electroconvulsive therapy. J ECT.
2013;29:2124.

16. Azuma H, Yamada A, Shinagawa Y, et al. Ictal physiological

characteristics of remitters during bilateral electroconvulsive therapy.
Psychiatry Res. 2011;185:462464.

2014 Lippincott Williams & Wilkins

www.ectjournal.com

Copyright 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

223