Comparison of Perceptual Voice Quality of VoIP from

G.711 and G.729 Using Conversation-Opinion Tests

Comparison of Perceptual Voice Quality of VoIP Provided by

G.711 and G.729 Using Conversation-Opinion Tests
Therdpong Daengsi
Faculty of Information Technology, KMUTNB, Thailand
E-mail: therdpong1@yahoo.com

Saowanit Sukparungsee
Dept. of Applied Statistics, Faculty of Applied Science, KMUTNB, Thailand
E-mail: saowanits@hotmail.com

Chai Wutiwiwatchai
Human Language Technology Laboratory, NECTEC, Thailand
E-mail: chai.wutiwiwatchai@nectec.or.th

Apiruck Preechayasomboon
Network Planning Department, TOT, Thailand
E-mail: apiruck@tot.co.th

Abstract

1.

This paper reports the results from the

study about perception of Thai users to the
voice quality of VoIP provided by two
codecs, G.711 and G.729, referring to packet
loss. The conversation-opinion tests have
been conducted with 250 participants to
obtain the Mean Opinion Score. This study
found that the user perception to G.711 and
G.729 is not different significantly referring
to the packet loss rates of 0%, 2%, 6%, 10%
and 20% approximately. However, the trend
of the difference is increased according to
the increment of packet loss rates of 10%
and 20% in the network. Therefore, it can be
recommended to use G.729, which requires
only 8 kbps for its voice payload, compared
to G.711 that requires 64 kbps, particularly
with packet loss of 6% or below, to reduces
traffic in the IP network.

Internet is a powerful technology that

becomes the backbone of many emerging
services/applications. It is estimated that now
there are around one billion computers
connected to the Internet, as in Figure 1.
The Internet has become a part of daily
living for a lot of people. Besides, it has
been applied to many sectors, such as, ecommerce, eLearning and telecommunications.

Keyword:
VoIP,
voice
quality
evaluation/assessment/measurement, packet
loss, conversation-opinion tests.

Introduction

Figure 1 Internet Hosts by September 2011,

adopted from [1]
For e-commerce, the combination of
traditional commerce and information and
communication
technologies,
allows

International Journal of the Computer, the Internet and Management Vol. 20. No.1 (January-April, 2012) pp 21 -26

21

Therdpong Daengsi, Saowanit Sukparungsee, Chai Wutiwiwatchai, and Apiruck Preechayasomboon

businesses/organizations to exchange information to support sale of products and/or

services electronically, based on the Internet.
Amazon is a well-known example of this
which has been referred to as the number 1
online bookstore [2]. Whereas, eBay has
become the world leader in the online
auction market [3]. The success of ecommerce also requires support from ebanking/Internet banking. The security issues
of e-payments have also been improved and
have become more trustable with such
methods as Paypal [4].
For eLearning, a kind of distance learning, educational materials can be delivered to
remote students, based on Internet technology. eLearning can be real-time with an
instructor and/or self learning using eLearning materials and tools, online and/or offline,
analog and/or digital, and narrow cast and/or
broadcast, for students who have difficulties
attending traditional classes [5-7]. Therefore,
they can watch online lectures/presentations
and practice with interactive learning
material which can respond automatically
[8]. However, eLearning students may need
to discuss with classmates and/or the
instructor using e-mail, chatting and
telephone [9].
For telecommunications, with Internet
protocol (IP), Voice over IP (VoIP) becomes
the modern telecommunication. It has many
advantages for businesses, consumers and
services providers [10]. The biggest advantage of VoIP for general users/consumers is
extra-low rates for international calls.
Therefore, e-commerce customers could use
VoIP to contact suppliers and enquire about
delayed goods or any other issues from the
merchant directly if he/she did not get a
reply from e-mailing. Another example
would be eLearning students using Voice
over IP (VoIP) to talk and discuss with
classmates and/or the instructors who live in
a different city/country easily.
Focusing on telecommunications that
are necessary to support a lot of activities,
this paper presents the study of VoIP, and
covers the voice quality perception of Thai

users to G.711 and G.729, referring to packet

loss effects, which has been extended from
[11].
2.

Background

A. VoIP Overview [12]

VoIP is based on IP network, packetbased technology, as in Figure 2. Basically,
after call set-up using IP signaling such as
Session Initiation Protocol (SIP), voice
signals are converted into voice packets
using a codec such as G.711. Then, voice
packets are transported to the destination
before converting back into voice signals
eventually.
B. Call Types of VoIP Services
VoIP services, provided by operators
can be classified based on the call types as
follows [13-14]:
1) PC-to-PC: normally, this call type
can be used free-of-charge. One of the most
popular VoIP applications used widely is
Skype [15]. Skype can also support video
call if each PC has a webcam. Moreover, in
Thailand, all major PSTN operators also
provide this kind of services/application such
as TOT netcall by TOT, True NetTalk by
True [13-14].
2) PC-to-phone: this call type is not
free, for example, Skype users of this service
must buy credit using Paypal and/or a
credit card [15]. Similar to TOT netcall, True
Nettalk and CAT2call, users have to register
and pay in advance, before using this kind of
services [13-14, 16].

Figure 2. VoIP system overview, adopted

from [14]

22

Comparison of Perceptual Voice Quality of VoIP from

G.711 and G.729 Using Conversation-Opinion Tests

C. VoIP Limitation: Voice Quality Issue

One major limitation of VoIP
applications/services is voice quality when
compared to traditional telephone calls
provided by PSTNs. Thus, due to Internet
Protocol
being
based
on
data
communication, VoIP which is a real time
application can be affected by factors in the
IP network [12], such as:
1) Packet loss: this can be the cause of
voice clipping/chopping. Therefore, it is
recommended to control packet loss rates in
an IP network at 3-5% maximum [17]
2) Packet delay: it is recommended that
packet delay in an IP network for VoIP
applications is 400 ms maximum [18].

whereas, G.711 is used over LAN.

Therefore, only these two codecs are
described: G.729: its MOS is 3.92. It is an 8
Kbps coding technique called Conjugate
Structure - Algebraic code-excited linear
prediction (CS-ACELP). Therefore, for this
reason it is recommended to be used over
WAN.
2) G.711: its MOS is 4.1. It is a 64 Kbps
coding technique called Pulse Code
Modulation (PCM). Subtypes of G.711 are
G.711-law and G.711A-law. G.711-law is
used in North America and Japan but
G.711A-law is used in the rest of the world,
including Thailand and Southeast Asia

D. Voice Quality Metric: MOS

In the same situation with the same
network factors, level of satisfaction to
perceptual voice quality from one person
may be different to another person.
Therefore, to judge voice quality with
accuracy and reliability, ITU-T, which is the
major organization for telecommunication
standardization, recommends using MOS. It
stands for Mean Opinion Score. Basically, it
is obtained from the judgment by a group of
subjects, typically 24-32 subjects [19], using
a 5-point grading scale where 5 to 1
represents excellent, good, fair, poor and bad
voice quality respectively [18].
However, there are limitations to
measuring voice quality using subjective
methods, such as time consumption, high
cost and collaboration. Therefore, many
objective methods for voice quality measurement/evaluation have been developed to
obtain the MOS easily [12, 20-22], for
example, PESQ and E-model. The overview
of measurement methods is presented in
Figure 3.
E. Codecs [11]
1) Codec selection can also affect voice
quality. Codec is the part which changes
voice signals into voice packets before
transportation at the source and vice versa.
Normally, G.729 is used over WAN,

Figure 3. Voice quality measurement

methods, adopted from [14, 22-24]

International Journal of the Computer, the Internet and Management Vol. 20. No.1 (January-April, 2012) pp 21 -26

23

Therdpong Daengsi, Saowanit Sukparungsee, Chai Wutiwiwatchai, and Apiruck Preechayasomboon

3.

System,
Environment,
Experimental Design

and

and Figure 4. However, most MOSs from

G.711 and G.729 are almost the same,
particularly at the packet loss rates of 0-6%,
whereas, most values of the standard
deviation (SD) are about 0.5, except at the
packet loss rates of 10% and 20%. Thus,
these hypotheses have been investigated with
the raw data from opinion scores using Ttest, a statistic method, then compared to
95% Confidence Interval (CI), as H, and then
analyzation of results are presented in Table
2:
H: The user perception to G.711 and
G.729 is the same

The system and laboratory for this study

were in the studio area at the Central Library
of KMUTNB, consisting of two low background noise rooms. Conversation-opinion
tests by each pair of subjects/participants per
round were conducted there. The main
system consists of the VoIP system using
Asterisk 1.6.2, network emulator using
Dummynet for packet loss generation and 2
SIP phones [11].
This study was designed to evaluate the
voice quality of VoIP referring to packet loss
rates of 0%, 2%, 6%, 10% and 20%
approximately, with G.729 codec and G.711
(A-law) codec. Therefore, there were ten
conditions to test. Following [25], each
condition has been conducted with at least 24
inexperienced subjects/participants or at least
240
inexperience
subjects/participants
totally, who were science and technology
students, Thai native speakers, in KMUTNB.
They had few criteria of homogeneity which
only consisted of the range of age,
nationality, and the background of education
(based on science and technology in the
same university).
The method for voice quality evaluation
is conversation-opinion tests, using paper
based Richards task, following ITU-T
recommendations. The advantages of
conversation-opinion tests are reaching the
realism and obtaining two sets of data per
round of tests [13].
However, each condition has been tested
by about 30 subjects/participants due to
some data classified as outliers (e.g.
distinguishable high or low scores comparing
to the majority), to validate the gathered
data.

5. Discussion
From Table 1 and Figure 4, at 10% and
20% packet loss, MOS results of G.711 are
higher than G.729 obviously, compared to
the packet loss of 0%, 2% and 6% which
shows the MOS results of G.711 and G.729
are slightly different. Moreover, after the
investigation using T-test, this study found
that all p-values are higher than 0.05,
although the p-value at 10% packet loss is
0.63, which is the lowest. That means there
is no significance about Thai subjects/users
perception to G.711 and G.729, within 95%
CI, referring to packet loss rate of 20%
maximum.
6. Conclusion and Future Work
From this study, it has been found that
although there is packet loss rates in the
network up to 20%, the user perception of
G.711 and G.729 are not significantly
different. This is consistent with the previous
works based on Thai users [11, 18] and this
evidence confirms that G.729 can be used
instead of G.711 without significant
difference of voice quality perception,
whereas, G.729 only requires about one third
of the bandwidth used by G.711, including
the packet header.
However, packet loss rates are more
than 20% (e.g. 30%-40%). This will be the
focus of future work.

4. Result and Analysis

After the conversation-opinion tests, a
subjective voice quality evaluation was
conducted with 24-28 subjects/participants
per codecs, the result is presented in Table I

24

Comparison of Perceptual Voice Quality of VoIP from

G.711 and G.729 Using Conversation-Opinion Tests

Acknowledgement

References

Thank you to the students and lecturers

of KMUTNB who participated and
supported this research. Special thanks to the
Central Library of KMUTNB and the studio
staff, Mr. Wiwat Suwanuntawong, for his
kind support. Lastly, the first author would
like to dedicate this paper to his original
advisor Dr. Gareth Clayton, who sadly
passed away.

[1]

Table 1. Comparison of MOS provided by

G.711 and G.729 from Subjective Tests with
Packet Loss Conditions, where Ns is the
number of subject and SD is the standard
deviation.
Ns

MOS

G.729 G.711 G.729

4.13
3.78
3.50
3.04
2.42

0.54
0.51
0.50
0.56
0.72

0.54
0.42
0.58
0.62
1.02

MOS

Packet
G.711 G.729 G.711
Loss
0%
26
24
4.15
2%
24
24
3.79
6%
24
26
3.58
10 %
28
24
3.34
20 %
24
26
2.79

SD

Packet Loss (%)

Figure 4 The Chart of the MOS Results from

Table 1.
Table 2 The Analyzed Result of Hypothesis
Test
Hypotheses
H: 0 % packet loss
H: 2 % packet loss
H: 6 % packet loss
H: 10 % packet loss
H: 20 % packet loss
Remark: p-value < 0.05 is significant

p-value
0.851
0.950
0.590
0.063
0.147

(2012) The Zakon website. [Online].

Available:
http://www.zakon.org/robert/internet/ti
meline/
[2] (2012) The 20 Best Books Sites Online
Today on Net Top 20 [Online].
Available: http://books.nettop20.com/
[3] (2012) The 20 Best Auction Sites
Online Today on Net Top 20 [Online].
Available:
http://auctions.nettop20.com/
[4] (2012) The PayPal website. [Online].
Available:
https://www.paypal-apac.
com/th/getting-started/about-paypal.
aspx?-outside.
[5] F. Anaraki (2004), Developing an
Effective and Efficient eLearning
Platform, IJCIM, vol. 12(2), pp. 5763, May.
[6] C. Brahmawong (2005), eLerning
Courseware
Production
System:
Underlying
Principles,
Major
Components, and Evaluation Criteria,
in Proc. 2nd Int. Conf. on eLearning for
Knowledge-Based Society, paper. 7.1.
[7] A. Kamsin (2005), Is E-Learning the
Solution
and
Substitute
for
Conventional Learning?, IJCIM, vol.
13(3), pp. 79-89, Sep.
[8] A. Kok (2010), An Activity System
Perspective of E-Learning and the
reframing of Knowledge, IJCIM, vol.
18(2), pp. 17-25, May.
[9] U. Jamornmann (2004), Techniques
for
Assessing StudentseLearning
Achivement, IJCIM, vol. 12(2), pp.
26-31, May.
[10] (2012)
The
International
Telecommunication Union website.
[Online]. Available:
http://www.itu.int/itunews/manager/dis
play.asp?lang=en&year=2007&issue=0
2&ipage=futureVoice2&ext=html.
[11] T. Daengsi, C. Wutiwiwatchai, A.
Preechayasomboon
and
S.
Sukparungsee (2012), A Study of
VoIP Quality Evaluation: User

International Journal of the Computer, the Internet and Management Vol. 20. No.1 (January-April, 2012) pp 21 -26

25

Therdpong Daengsi, Saowanit Sukparungsee, Chai Wutiwiwatchai, and Apiruck Preechayasomboon

Perception of Voice Quality from

G.729, G.711 and G.722, in Proc. 9th
IEEE-CCNC2012SS QoE, paper
QoE.5, p. 358.
[12] F. D. Rango, M. Tropea, P. Fazio, and
S. Marano (2006), Overview on VoIP:
Subjective and Objective Measurement
Methods, IJCSNS, Vol. 6(1B), pp.
140-153, Jan.
[13] (2012) The TOT Netcall website.
[Online]. Available:
http://www.totnetcall.com/VoIP/tdetinf
o.aspx.
[14] (2012) The True NetTalk website.
[Online]. Available:
http://www.truenettalk.com/en/about_tr
uenettalk/what_is_truenettalk.html
[15] (2012) The Skype website. [Online].
Available:
http://www.skype.com/intl/en/home.
[16] (2012) The CAT2call website. [Online].
Available:
http://www.cat2call.com/index.php?opt
ion=com_content&task=view&id=1&It
emid=3
[17] Quality
Management:
Troubleshooting Techniques for Voice over
IP, Fluke networks, NJ, 2005
[18] T. Daengsi, C. Wutiwiwatchai, A.
Preechayasomboon
and
S.
Sukparungsee,
VoIP
Quality
Measurement: Insignificant Voice
Quality of G.711 and G.729 Codecs in
Listening-Opinion Tests by Thai
Users, Inform Technol J., In Press.
[19] A. W. Rix (2003), Comparison
between subjective listening quality
and P.862 PESQ score, Psytechnics,
Sep.
[20] M. Goudarzi (2008), Evaluation of
Voice Quality in 3G Mobile
Networks, M.S. Thesis, University of
Plymouth, Plymouth, UK, Sep.
[21] (2012) The ITU website. [Online].
Available:
http://www.itu.int/ITUD/tech/events/2011/Moscow_ZNIIS_A
pril11/Presentations/09-PomyPOLQA.pdf.

[22] S. Karapantazis and F.-N. Pavlidou

(2009), VoIP: A comprehensive
survey on a promising Technology,
Comput. Netw., vol. 53(12), pp. 20502090. March.

26