Human Factors in Intelligence Failures

By Moin Rahman

The intelligence failure of 9-11 was blamed on the inability to connect the dots. The prewar
intelligence failure that resulted in the war against Iraq is now being blamed on missing dots (WMDs).

Congress and two independent commissions (9-11 & prewar intelligence, respectively) appointed by
the president are investigating these massive intelligence failures. The most recent episode in this saga
are the public hearings held by the 9-11 commission and the testimony of George Tenet, director of
CIA, regarding the prewar intelligence to the Senate Armed Services Committee on March 9.

To correctly diagnose the reasons for the prewar intelligence failure, it behooves that the investigators
understand the fallibilities and fallacies of humans, when they process information and make decisions
in ominous and uncertain circumstances.

Mr. Tenet pointed out in a speech at Georgetown University that intelligence is “almost never
completely wrong or completely right.” Due to its nebulous nature, it gives the appearance of not
being an exact science; however, there is a science called Signal Detection Theory (SDT) that deals
with how humans process information – and make decisions – in a world that is inexact.

SDT in its essence explains how humans’ process information in an uncertain world and react to it
when there is a need to separate wheat (referred as “signal”) from the chaff (“noise”). Human Factors
engineers utilize SDT to understand human information processing and decision-making capabilities
when designing critical systems, such as nuclear power plant control rooms, where there is no room for
error.

Let us consider the example of a radiologist to understand SDT in detail. When a radiologist looks at
an X-ray and correctly identifies a tumor (signal) it is called a “hit”; if he fails it is called a “miss”.
However, if he falsely concludes that there is a tumor when in reality there was none – say due to the
fuzziness of the picture – it will be a “false alarm”. Alternatively, if he correctly rejects a mass that is
an integral part of a healthy organ, but gives a false appearance of a tumor, it will be called a “correct
rejection”.

The accuracy of the two correct decisions (hit and correct rejection) could be affected by prior
knowledge of the radiologist. For example, if the radiologist was aware that the patient was referred
because of possible cancer, the radiologist is likely to lower his threshold – or “decision criterion” in
SDT jargon – for detecting a tumor. This may result in the radiologist adopting a conservative
approach and wrongly concluding a normal mass (noise) as a tumor (signal).

As seen with the above example, SDT provides a scientific framework to dispassionately analyze
problems relating to human information analysis and decision making on uncertain terrain. Thus SDT
provides a bipartisan approach, in a scientific sense, to analyze the intelligence failures pertaining to
the Iraqi WMDs in a politically charged atmosphere.

“Intelligence” as Mr. Tenet explained in his speech, “deals with the unclear, the unknown, [and] the
deliberately hidden”. When intelligence is seen under the rubric of SDT involves identifying a signal
that could be disguised as noise: for example, Saddam Hussein’s deliberate deception when he
constructed dual purpose facilities that produced both arms and industrial goods in the early 80s and
90s. Furthermore, a smoke and mirror effect could be willfully applied to mislead observers that could
result in mistaking noise for a signal: for instance, even though Saddam fully disarmed in recent years
he kept the world guessing by not showing it to the inspectors, and was even willing to bear hardships
caused by international sanctions and lost oil revenues.

Thus Intelligence analysts have to work in a very complex, confusing, and demanding environment,
where overlooking one thing or falsely claiming something that did not exist can have severe
repercussions (9-11 attack and the Iraq war, respectively). The human factor of intelligence analysis
can benefit by the application of SDT to maximize correct decisions and minimize error.

The human factor in the decision making phase involves the consumers of intelligence information,
which include the higher echelons of our government. They are charged with an enormous
responsibility of correctly comprehending the information and making the right decisions to protect the
American people.

So the next question is where (and why) did the Bush administration set its decision criterion, which
resulted in a conclusion that war was necessary? But to give this a fair hearing, it is important to see
the decision-making in the light of events following the 9-11 terrorist attacks.

Did the terrible consequences of 9-11 and prior knowledge of Saddam’s WMD capabilities in the 80s
and 90s – even if unrelated – result in a jittery Bush administration lowering its decision criterion? In
other words, did they develop a conservative response to wage a war abroad rather than take American
casualties on the homeland? Based on SDT this scenario is plausible, but it should not be
misconstrued as an excuse for the administration’s decision to go to war.

Or as the critics of the Bush administration claim, did the neo conservative ideology within the
administration result in the cherry picking of intelligence information to make the case for war? If
true, this would have biased the decision makers and lowered their decision criterion even further
resulting in a shoot first ask questions later response

The independent commissions appointed by President Bush to investigate intelligence failures should
utilize SDT to ensure that the human factor part of the investigation is done in a fair and balanced
manner. This will help the investigators determine if the actual performance of intelligence analysts,
key decision makers, including the commander-in-chief, was a normal human response given the
circumstances. Or did intelligence analysts commit genuine mistakes trying to tease apart probable
proof for WMDs from Saddam’s bluff? Or whether ideology driven decision makers overruled
probable proof that there were no WMDs and rushed to war?

Moin Rahman is a human factors engineer and writes on science and current affairs.