Designing low-cost VoIP videophones
By Michael Ward
Director of Product Line
Management
Trinity Convergence Inc.
E-mail: mward
@trinityconvergence.com
Video telephony has faced chal-
lenges in quality and the cost of
its equipment and associated
services. These factors, along
with video frame rates in the
single to 10 frames per second,
resulted in a videophone that
consumers left sitting on the
shelves.
One by one, the technologi-
cal challenges facing video te-
lephony are being overcome
with increasingly sophisticated
VoIP solutions. Broadband
Internet access has become
pervasive, creating sufficient
bandwidth to the home. A com-
bination of startup and estab-
lished carriers is providing
packet-based telephony ser-
vices that can serve as the foun-
dation for video telephony.
Most important, Moore’s Law
has worked its magic to deliver
processing devices capable of
cost-efficient full-motion
video. By using soft-VoIP de-
vice design techniques in
conjunction with hardware-
accelerated video processing,
phones can now be created that
meet the aggressive cost-points
needed for video telephony to
finally reach its mass-market
potential.
Price is right
For the widespread adoption of
videophones, providers are tar-
geting a consumer price point
of $99 for the residential video
telephony market. While this is
challenging for the current gen-
eration of technology, many
VoIP service providers are sub-
sidizing videophones in ex-
change for a year or more of sub-
scription. With current video-
phone models retailing for
about $800 before subsidies, a
heavy burden is placed on the
service provider that can reduce
the drive to offer such services.
Current VoIP-enabled
videophone designs are expen-
sive because they require a high
number of specialized compo-
nents. Traditional video and
VoIP (V2IP) designs often con-
tain as many as three indepen-
dent processors—a DSP, video
encode and decode processor,
and an applications processor.
These processors combined
with necessary electronic com-
ponents, such as the camera,
LCD and audio codec, quickly
push the overall electronics
BOM (eBOM) to the $500 range
before any costs for software,
packaging or manufacturing
are added.
Mic Speaker
Audio
codec
DSP
- VoIP audio
processing
Applications
processor
- VoIP call control
- GUI
- System mgt
Keypad
Figure 1: DSP handles the packet voice processing, while an applications
processor manages the VoIP call control protocol and user interface.
Fortunately, emerging tech-
nology allows the eBOM for a
videophone to be cut by at least
50 percent. By using integrated
applications processors and
soft-DSP techniques, such as
the VeriCall EdgeTM, the mul-
tiple processors previously re-
quired can be combined into a
single device, yielding signifi-
cant savings in cost, power and
device size.
In traditional videophone
devices, a DSP handles the
packet voice processing (voice
encode/decode, tone genera-
tion and detection, echo cancel-
lation and noise reduction). A
separate DSP or dedicated co-
processor handles the video en-
code and decode, while an ap-
plications processor manages
the VoIP call control protocol
and user interface (Figure 1).
In this multidevice architec-
ture, tasks are relegated to dif-
ferent components of the sys-
tem for processing. Hence, the
task of coordinating and man-
aging the overall system is in-
creased, extending the size of
the design and overall complex-
ity of the circuit board. Given
the multiple devices and their
DSP or Display
co-processor
- Video encode
and decode
Camera
Ethernet
interface
varying power requirements,
the power-supply design within
the system may become very
complex, requiring the use of
multiple voltage converters to
generate the various power
rails needed.
Traditional architecture us-
ing separate processors for
voice, video and system control
also requires multiple pro-
gramming models and develop-
ment tool chains. Thus, larger
development teams, increased
training requirements and ad-
ditional costs will be needed.
VoIP codecs (G.711, G.729AB,
G.723.1 and iLBC), audio pro-
cessing (DTMF and call
progress tone detection/gen-
eration), voice-quality en-
hancement (line and acoustic
echo cancellation, and jitter
buffers) and other similar func-
tions can now be executed on
the applications processor if
carefully implemented with as-
sembly-coded and hand-opti-
mized software, while using
some form of hardware accel-
eration for the video encode
and decode (Figure 2).
In a typical videophone de-
sign, a hardware audio codec
(e.g. an AC97 codec) provides
the physical interface between
the microphone/speaker and
the general-purpose processor.
Software running on general-
purpose applications proces-
sors sends and receives audio
data and performs the neces-
sary VoIP processing, such as
taking incoming audio data
from the microphone and pro-
cessing it via a voice codec and
sending it out as a packet
stream, or perhaps playing a
generated DTMF tone to the
speaker (via the hardware audio
codec).
Detailed knowledge in the
processor architecture and
VoIP audio processing algo-
rithms is needed to effectively
implement these processor-in-
tensive algorithms. While a sig-
nificant effort is required to
implement these voice-process-
ing modules for the appropriate
applications processor archi-
tecture, the result is well worth
the investment.
Due to the real-time nature
of VoIP traffic and the need to
provide a wide range of VoIP
codec support to ensure inter-
operability across devices, a
flexible framework is needed.
This allows runtime selection
and configuration of the appro-
priate VoIP codec and dynamic
configuration of the media-pro-
cessing elements to be used
within a given media channel.
The framework and its associ-
ated scheduler component
must ensure that all algorithms
required for a given channel
definition are executed in the
time period allowed.
While in a single-channel
system the task of scheduling
algorithms is little more than a
series of consecutive calls to the
appropriate algorithms in or-
der, multichannel systems offer

Speaker
Mic
Audio
codec
Applications
processor
- VoIP audio
processing
- VoIP call control
- GUI
- System mgt
Keypad Ethernet
interface
Figure 2: Software running on general-purpose applications processors
sends and receives audio data and performs the necessary VoIP processing.
a more complex scenario in
which different VoIP codecs
may be required for each chan-
nel, as well as certain channels
requiring echo cancellation,
while others do not. Video-
phones are typically “single-
channel” systems, although
they will often provide the ca-
pability for three-way audio
calling with the audio mixed lo-
cally on the videophone, so the
need for multichannel support
arises.
Simplifying life for device
designers, software platforms
such as VeriCall Edge provide
a highly optimized and inte-
grated solution in a form that
can be quickly integrated into
the end-product design. Provid-
ing all the requisite media pro-
cessing algorithms and SIP- or
H.323-based VoIP call control
within a flexible framework in-
tegrated for ARM9, ARM9E
and MIPS32-based devices al-
lows the videophone developer
to focus on support for value-
added services on the device.
By merging the packet voice
processing onto a GPP, only
one device is required to handle
the VoIP-related tasks of the
videophone. The VoIP call con-
trol, user interface and general
telephony control application
execute on this general-pur-
pose applications processor. As
these tasks do not exhibit the
same time-critical attributes,
they are implemented in C
without the need for optimized
assembly code. Merging the
media processing and call con-
dia applications processor fall
into this category. The i.MX21
device contains an ARM9E-
based GPP along with a dedi-
cated hardware acceleration
engine for H.263 & MPEG-4
encode/decode for up to 30fps
video at CIF resolution. The
VeriCall Edge solution takes
advantage of this architecture
by running all the VoIP media
processing on the ARM9E GPP,
DSP or Display while the VeriCall Edge soft-
co-processor ware framework configures and
- Video encode
and decode manages the hardware-based
video-acceleration block to
Camera control the H.263 or MPEG-4
video stream.
Getting the picture
The most processing-intensive
task in a videophone is the pro-
cessing of the video itself. The
specific processing required
may vary considerably, depend-
ing on the size of the transmit-
trol/system management onto ted and received image, as well
a single applications processor as on the particular encoding
enables a simplified design, scheme used. H.263, MPEG-4
which reduces the device and H.264 are among the most
count, cost and size. It also commonly used codecs for video
removes the need for separate telephony. H.264 has the ben-
development efforts and tool- efit of requiring less bandwidth
chains for the VoIP media pro- to transmit a video image of
cessing and system control. quality comparable to that of
To further increase the level H.263, but requires signifi-
of system integration and gain cantly more processing power to
additional benefits of lower achieve this higher level of com-
power, smaller footprint and pression. The specific video
overall lower system costs, an compression scheme, desired
SoC that contains the GPP maximum frame rate and range
along with a dedicated video co- of supported resolutions will
processor or DSP can be used drive the “right” solution for
(Figure 3). Devices such as the video processing in a device.
Freescale TM i.MX21 multime- Options available to the sys-
Speaker
Microphone
Audio
codec
Applications
processor DSP or Display
- VoIP audio co-processor
processing - Video encode
- VoIP call control and decode
-GUI
-System mgt
Camera
Keypad Ethernet
interface
Figure 3: To further increase system level integration, an SoC that contains
the GPP along with a dedicated video co-processor or DSP can be used.
tem designer range from soft-
ware implementations running
on a GPP or a DSP to silicon
devices dedicated to the encod-
ing and decoding of specific
video streams. Full software
encoding and decoding on a
GPP is really only viable for
very-low-frame rate and low-
resolution video. The encoding
of the video stream requires
significantly more processing
power than the decoding opera-
tion, so a reasonable alternative
is to perform the video decode
on a GPP while handling the
encode via some form of hard-
ware-based acceleration. With
the additional processing
power in emerging general-pur-
pose applications processors
(such as those based on ARM11
and MIPS24K cores), software-
based decoding combined with
hardware-assisted encoding is a
suitable solution for many next-
generation videophones.
To achieve full-motion
(30fps) video at CIF or greater
resolutions, current-genera-
tion VoIP videophones will re-
quire some form of hardware-
based acceleration. This could
be in the form of an SoC with a
GPP to handle voice, and either
a dedicated video encode device
or a more general-purpose DSP
to handle video. Advantages of
dedicated video encode devices
include a simplified program-
ming model and implementa-
tions that are often more effi-
cient in terms of power and sili-
con gate-count than a DSP
when embedding the device in
an SoC. However, a DSP pro-
vides more f lexibility for the
system designer, allowing DSP
software updates to deliver new
video-codec formats (assuming
the utilized DSP has sufficient
processing power for the new
video codec). Ultimately, the
system designer must deter-
mine the desired operating
characteristics, taking into
consideration the video for-
mats to be supported, available
processing capacity, power
budget and footprint require-
ments for the device.
Bringing it all together
With the design points for the
A/V systems identified, perhaps
the most critical aspect of the
design is the integration of
these two, disparate sub-
systems. In a VoIP-based
videophone, A/V signals are
transmitted and received as two
independent packet streams
that must be correlated and syn-
chronized on the receiving side.
Failure to properly synchronize
the A/V stream results in a user
experience akin to that of
watching a late night Godzilla
movie, where the actors’ words
never match the video on the
screen.
A jitter buffer designed with
consideration for video tele-
phony is needed and careful
attention must be paid to this
critical system element to cre-
ate a viable product. While the
A/V packet streams contain
timestamps that can be used to
correlate data, the system must
also contend with packet loss
and network jitter that can oc-
cur in the network. The design
must take into account the need
to buffer and synchronize the
two separate streams while do-
ing so in a manner that does not
introduce unnecessary delay to
the system.
Certain videophones provide
basic synchronization by allow-
ing the user to modify a delay or
offset between the A/V streams.
While this solution enables
synchronization of the A/V
streams, it must be modified
each time the user places a call.
If network conditions change
during a call, it is possible for
these streams to go out of sync.
The VeriCall Edge V2IP solu-
tion incorporates an advanced
technology that provides auto-
matic dynamic synchronization
of the A/V streams, removing
the need for the user to manually
synchronize the streams while
also adapting to changing net-
work conditions. By actively
monitoring the incoming A/V
streams and managing these
streams to accommodate de-
layed or missing packets in one
stream or the other, VeriCall
Edge can ensure a high-quality
experience to the user in a wide
range of network environments.
Low-cost videophone
While video telephony has
faced many false starts over the
past 40 years, elements are now
in place to allow cost-effective
video telephony to reach the
mass market using the fast-
growing VoIP infrastructure.
By coupling soft-DSP VoIP
techniques with hardware-
accelerated video encoding,
videophones can be developed
at a savings of as much as 50
percent in the eBOM of the de-
vice. This savings in product
cost will allow videophones to
be marketed at price points
that finally enable mass-market
adoption. The use of integra-
ted software frameworks like
VeriCall Edge allows system de-
velopers to deliver innovative
products with less risk and
faster time-to-market.
As the demand for low-cost
video telephony grows, it will
inevitably find its way into
other devices beyond the tradi-
tional desk phone. Wi-Fi-based
personal video terminals are al-
ready in development that will
allow friends to share experi-
ences in real-time, wherever
Wi-Fi access exists. Remotely
monitored V2IP door cameras
can become commonplace, al-
lowing a homeowner to answer
a door and converse with the
visitor from any place in the
world that has a broadband
Internet connection. These and
many other innovative applica-
tions for video telephony will
surely arise, all driven by the
ability to deliver low-cost video
terminals that are enabled by
DSP-free VoIP techniques.