Lesson Bayesian Networks

In this Lesson
• Beliefs and evidence.

3
• Types of reasoning..
• Reasoning in practice.
• Reasoning using specialized software.

Duration : 100 minutes

Beliefs and evidence.

In the previous lessons we learned how to represent a domain and its uncertainty using a Bayesian
network. In this lesson we will learn how we can produce reasoning about a domain using the
Bayesian network. We can make observations about the values of some variables and then to
condition upon the new information. The process of conditioning is known as probability
propagation, inference or belief updating. The idea is to provide with new evidence the network and
analyze current beliefs to predict a result or to diagnose a cause. This is done via a flow of
information through the network, without being limited to the directions of the arcs. In a
probabilistic system such as Bayesian networks this can be done by computing the posterior
probability distribution for a set of query nodes, given values for some evidence (observation) nodes.

Before we continue any further talking about beliefs and evidences, let’s first make clear what we
mean when we refer to those two terms.

Beliefs are the probability that a variable will be in a certain state based on the addition of evidence
in a current situation. A special case of beliefs are the a-priory beliefs that are based only on prior
information. A-priory beliefs are stored only in the belief network’s conditional probability tables.

Evidence is information about a current situation. In general we can consider as evidence a definite
finding that a node X has a particular value x, which we write as X=x. This kind of evidence often is
referred to as specific evidence or hard evidence. For example, suppose that the patient has flu, then
it would be Flu=True, which is specific evidence. On the other hand the evidence might be simply any
probability distribution over the node X. For example suppose that the doctor is not completely sure
whether the patient has allergy or not. He thinks that the patient has allergy but he is only 70% sure.
This kind of evidence in Bayesian networks is known as virtual evidence, likelihood evidence or soft
evidence.

International Hellenic University 1

Paraskevopoulos Konstantinos
Bayesian Networks

Types of reasoning.
In the previous paragraph we said that someone can condition a Bayesian network without being
limited to the directions of the arcs connecting the nodes i.e. Bayesian networks can support any
direction of reasoning.

For example, someone can perform reasoning from symptoms to cause. This reasoning in Bayesian
networks is known as diagnostic reasoning and occurs in the opposite direction to the network arcs.
Think again of the example provided in the previous lesson, if the doctor observes headache then his
belief about the patient having sinus problems will be updated as well as whether the patient has a
flu. In Figure 2 there is a graphical representation of this example.

Diagnostic Reasoning Predictive Reasoning

Query Evidence

Fl A Fl A
Reasoning Direction

Reasoning Direction
S Query S Query

Ti H Ti H
Evidence Query Query

Figure 2 Figure 2

On the other hand, someone can perform reasoning from cause to symptoms following the direction
of the arcs. This kind of reasoning is known in Bayesian networks as predictive reasoning. For
example, the patient may tell the doctor that he has an allergy even before any symptoms have been
assessed. Now the doctor has to increase the chances of the patient having sinus problems and of
course to increase the chances that the patient will exhibit other symptoms, such as tiredness and
headache. In Figure 2 there is a graphical representation of this example.

One other type of reasoning is called intercausal reasoning. This type of reasoning involves the
mutual causes of a common effect. An example of this type of reasoning is shown in Figure 4 with
causes of flu and allergy having a common effect, the sinus problems. Our model shows that the two
causes are independent from each other. For example a patient that has allergy does not change the
probability of the patient having flu. However if we know that the patient has sinus problems then
the chances that he has flu and allergy are increasing. Suppose now that we learn that the patient
has an allergy. This new evidence explains the sinus problems, and in the same time lowers the

International Hellenic University 2

Paraskevopoulos Konstantinos
Bayesian Networks

probability that he has flu. So even though the two causes seem to be independent, the one cause
can render the other cause less likely.

Sometimes we have to combine the different types of reasoning as the reasoning can’t fit in any of
the three types described above. In Bayesian networks any node can be a query node and any may
be evidence node as it shown in Figure 4.

Intercausal Reasoning Combined Reasoning

Query Evidence Evidence

Fl A Fl A

S Evidence S Query

Ti H Ti H

Query Evidence

Figure 4 Figure 4

Reasoning in practice.
Till now in this lesson we described the types of reasoning in Bayesian networks theoretically, in this
paragraph it’s time for some action. We are doing reasoning in Bayesian networks, by calculating
probability distributions.

Let’s try to answer some questions from our example sinus network. For example, suppose we want
to know what the probability of sinus problems is. In a Bayesian network if you wish to know the
probability that node N, say, is true, you have to look at its parent nodes (i.e. its causes). You have to
list all possible combinations of the values of the parent nodes and then consider the probability that
N is true for each combination. In sinus problem we have to consider all the possible combinations of
the two parent nodes Fl and A. In this case there are four of them, so there are four ways the patient
can have sinus problems:

He has flu and has allergy and has sinus problems.

He does not have flu and has allergy and has sinus problems.
He has flu and does not have allergy and has sinus problems.
He does not have flu and does not have allergy and has sinus problems.

International Hellenic University 3

Paraskevopoulos Konstantinos
Bayesian Networks

We have to work out the probabilities of all four situations and add them together. The probability of
each of these cases is

P(A^F^S) = P(S|A^F)P(A)P(F)

ഥ ^F^S) = P(S|A
P(A ഥ ^F)P(A
ഥ )P(F)

P(A^Fത^S) = P(S|A^Fത)P(A)P(Fത)

ഥ ^Fത^S) = P(S|A
P(A ഥ ^Fത)P(A
ഥ )P(Fത)

The values of all quantities on the right-hand side are given in Lesson 2. So we can work out the
probabilities of all for cases. P(S) is just the sum of all four.

P(S) = 0.8×0.3×0.4 + 0.5×0.7×0.4 + 0.4×0.3×0.6 + 0.1×0.7×0.6 = 0.35 (1)

Suppose we wish to know the probability of sinus problems and allergy, i.e. you are told that a
patient has allergy and has sinus problems but you don’t know whether he has flu or not. There are
two ways in which a patient could have sinus problems and allergy. They are given bellow :

He has allergy and flu and has sinus problems.

He has allergy and does not have flu and has sinus problems.

We have already worked out the probabilities as these are just the first and third case from the
previous list. All we have to do is add them together.

P(S^A) = 0.8×0.3×0.4 + 0.4×0.3×0.6 = 0.168

From this we can work out the probability that a patient has sinus problems given that he has allergy.

୔(ୗ^୅) ଴.ଵ଺଼
P(S|A)= = =0.56 (2)
୔(୅) ଴.ଷ

Suppose that we wish to know the probability of having the feeling of tiredness given that a patient
has allergy, i.e. you are told that the patient has allergy but you don’t know whether he has flu or
whether he has sinus problems.

The feeling of tiredness Ti is caused by sinus problems S which is in turn caused by an allergy A. So we
have a chain of cause and effect. But we don’t know the value of the middle link S in this chain. So we
have to consider each possible value of S as follows

He has allergy and has sinus problems and has a feeling of tiredness.

He has allergy and does not have sinus problems and has a feeling of tiredness.

We have to work out the probabilities of each case

P(Ti|A)=P(Ti|S)P(S|A)+ P(Ti|Sത)P(Sത|A)
International Hellenic University 4
Paraskevopoulos Konstantinos
Bayesian Networks

Now we are given P(Ti|S) in the original data in Lesson 2. And we have just worked out P(S|A)
immediately above. P(Sത|A) is just 1- P(S|A). So we have all the quantities on the right-hand side.
So we can work out P(Ti|A)

P(Ti|A)= 0.8×0.56 + 0.1×0.44 = 0.492 (3)

This shows that you can use the Bayesian networks to find the influence of one node on another. A
influences Ti via S. So we first calculate the influence of A on S and then S on Ti. Because we don’t
know whether S is true or not in this case we must first calculate the probabilities of each value of S
given A and then calculate the probability of Ti in each of those cases. That’s why the Bayesian
networks are also called Influence Diagrams.

Some example cases of reasoning with the exact numbers of the updated beliefs calculated using the
methodology shown above is shown in Table 1. This table shows all the priors and conditional
probabilities originally specified in Figure 2 in Lesson 2.

Node No Reasoning Case

P(A)=0.3 Evidence Diagnostic Predictive Intercausal Combined
H=T A=T S=T S=T H=T
A=T A=T
Fl=T 0.400 0.586 0.400 0.674 0.571 0.532
A=T 0.300 0.422 1 0.480 1 1
S=T 0.350 0.790 0.560 1 1 0.899
Ti=T 0.345 0.653 0.492 0.800 0.800 0.729
H=T 0.310 1 0.436 0.700 0.700 1
Table 1

In Table 2 is shown what happens if allergy rate among population increases from 30% to 60%, as
represented by a change in the prior probability of A, Allergy node. Since the two cases differ only in
the prior probability of allergy P(A=T)=0.3 versus P(A=T)=0.6, when the evidence itself is about the
patient having an allergy, then the prior becomes irrelevant and both networks give the same
numbers.

Node No Reasoning Case

P(A)=0.6 Evidence Diagnostic Predictive Intercausal Combined
H=T A=T S=T S=T H=T
A=T A=T
Fl=T 0.400 0.558 0.400 0.618 0.571 0.532
A=T 0.600 0.719 1 0.764 1 1
S=T 0.440 0.846 0.560 1 1 0.899
Ti=T 0.408 0.692 0.492 0.800 0.800 0.729
H=T 0.364 1 0.436 0.700 0.700 1
Table 2

International Hellenic University 5

Paraskevopoulos Konstantinos
Bayesian Networks

Reasoning using specialized software (Part I).

In this paragraph we will show how Belief and Decision Networks tool from AIspace can be used to
make observations and perform queries. For the demonstration you will need to load the file
Exercise_2.xml as it was updated in Lesson 2.

1. Start the application.

2. Click on File menu and load the file Exercise_2.xml that you created in Lesson 2.
The main window should look like the one shown in Figure 5.

Figure 5

3. Click on the Solve tab to switch to Solve Mode, where you can start making observations,
among other things. The Solve Toolbar, for Belief Networks, looks like the one shown in
Figure 6.

Figure 6

4. Let’s try to find out what is the probability of node S (sinus problem) that we calculated in
Formula 1 in the previous paragraph. You can do that with two ways, either by monitoring a
node either by queering it.
To monitor a node, click on the Toggle Monitoring button in the toolbar.
5. Left click on node S. The node is now transformed as you can see in Figure 7.

International Hellenic University 6

Paraskevopoulos Konstantinos
Bayesian Networks

Figure 7

Information about node’s probability is being displayed below its name, and that information
will be updated as observations are made about the probabilities of other nodes that affect
it. Notice that the probability of node S to become true is 0.35 as we calculated in the
previous paragraph in Formula 1.
6. Left click on node S to hide the information about the probabilities of the node.
7. The second way to gain information about a node's probability is querying. On the toolbar
click on button Query.
8. Left click on node S, to open the dialog frame Query Mode that is shown in Figure 8.

Figure 8

There are three different query modes for the application: Brief, Verbose, and Prompt. The
default setting is Prompt mode. In this mode, each time you attempt to query a node, a
dialog will open asking whether you would like a Brief or Verbose query. This dialog looks like
the one in Figure 8. You can change the mode from menu Network Options > Query Modes
and selecting the radio button next to the desired mode. You can also change the mode in
the prompt dialog by selecting the appropriate radio button at the bottom of the window.

International Hellenic University 7

Paraskevopoulos Konstantinos
Bayesian Networks

9. Click on the Brief button. A dialog frame will open showing the node's probabilities, and it
will list any observations you have made.
As you can see in Figure 9, if node S is observed to be True, then the prior probability for
node S, is 0.35.

Figure 9

10. Let’s try now to find out what is the probability of sinus problems and allergy. As we said in
the previous paragraph suppose that we know that a patient has allergy and has sinus
problems but we don’t know whether he has flu or not.
To find out what is the probability P(S|A) click on the button Make Observation in the
toolbar.
11. Click on the node A. The frame dialog Observation for A will open as is shown in Figure 10.
12. Select the value T (for True) as this is the value you want to observe (the patient has allergy)
and click OK. The value you select will be displayed beneath the node name as you can see in
Figure 11.

Figure 10

13. Click on the button Toggle Monitoring in the toolbar to select it.
14. Left click on node S, to show the information for the probabilities of the node as it is shown
in Figure 12. As we calculated in Formula 2 in the previous paragraph, the probability P(S|A)
is 0.56.

International Hellenic University 8

Paraskevopoulos Konstantinos
Bayesian Networks

Figure 11

Figure 12

15. Having the toolbar button Toggle Monitoring selected, click on node S to hide the
information.
16. Click the button Make Observation and then left click on node A, to open the frame dialog
Observation for A.
17. In the frame dialog Observation for A, select the lists item None, and click OK to close the
dialog and to cancel the observation mode for node A.
18. Suppose now that we want to find the probability P(Ti|A).
Having the button Toggle Monitoring selected, click on node Ti to show the information for
the probabilities of the node.
The value 0.49 is the probability P(Ti|A) as we have already calculated in Formula 3 in the
previous paragraph.
19. Try by yourself to test the findings of the Table 1 and Table 2 by making Observations and
queering the nodes of the model.

International Hellenic University 9

Paraskevopoulos Konstantinos