APPLICATION LAYER OPTIMIZATION FOR EFFICIENT VIDEO
STREAMING OVER IEEE 802.11 BASED WIRELESS NETWORKS
Azfar Moid and Abraham O. Fapojuwo
Department of Electrical and Computer Engineering
University of Calgary, AB, Canada, T2N 1N4
{amoid@ucalgary.ca, fapojuwo@ucalgary.ca}
Abstract — Most of the existing video streaming systems
employ the worst case analysis in application layer buffer
size dimensioning. Even though the worst case buffer size
dimensioning provides deterministic quality of service
(QoS) guarantees that are desirable in multimedia
transmission, however, this also over-provisions the scarce
memory resources. In this paper, we propose a dynamic
technique for buffer and rate allocation under two
scenarios: 1) when the channel conditions are known a-
priori, and 2) when the channel conditions are unknown.
Simulation results show up to an order of magnitude
savings in the application layer buffer requirements for the
two scenarios considered. Furthermore, a-priori knowledge
of the channel conditions at the application layer yields an
improved video quality.
Index Terms — Buffer dimensioning, Video streaming,
Wireless networks, Rate-control.
I. INTRODUCTION
The H.264 video format is the latest state-of-the-art
international video coding standard developed by the Joint
Video Team (JVT) of ITU-T and ISO/IEC [1], which is also
useful for video streaming purposes. For video streaming
over wireless networks, high efficiency can be achieved by
making the mobile devices adapt to the network conditions
(e.g., the real-time channel conditions, available
transmission bandwidth, traffic load, desired spatial or
temporal resolution, delay allowance, error resilience, and
so forth).
The dynamic content of video frames makes the bit-rate
of the encoded video variable in nature, necessitating buffer
management at the application layers of the transcoder and
decoder. To avoid long delays in real-time video streaming,
the transcoder and decoder buffer sizes are usually limited.
However, with smaller buffer sizes, there is an inherent risk
of packet-dropping at the application layer. Moreover, on
client devices, the memory is an important contributor to the
overall power budget [2]. Hence, an optimized solution is
required at the application layer to balance the tradeoff
between packet-delay and packet-dropping. The key to
application layer buffer management is the rate-control
schemes employed. A rate control scheme determines the
optimum encoding rate, which is used during the video
compression process for adjusting the coding parameters,
978-1-4244-3508-1/09/$25.00 ©2009 IEEE
e.g., the quantization point (QP), to prevent the application
layer buffers from overflow or underflow.
The motivation for this work comes from the fact that the
dynamic buffer management for video coding is not very
well studied in the literature, mainly because of the variable
sizes of video frames. The fixed group of pictures (GOP)
size causes the periodic inclusion of intra (I-) frames for
MPEG-4 video sequences, thus making the video frames
variable in nature. In the case of H.264 video encoding
scheme, where only a single or a very few I-frames are used
to encode the video sequence [1], the generated bits per
frame are relatively identical for successive frames, unlike
those for the MPEG-4 scheme. Therefore, there is a
requirement of revisiting the transcoder and decoder buffer
dynamics, to get the optimized sizes of the application layer
buffers, under the constraints of avoiding buffer overflow
and underflow. For example, Reference [3] employed the
transcoding ratio constraints to avoid overflow and
underflow of the transcoder and decoder buffers, regardless
of the video encoding scheme but assuming fixed buffer
sizes. It is shown in this paper that dynamic buffer sizes,
used in conjunction with controlling the encoding rate help
prevent buffer overflow and underflow. The advantage of
dynamic buffer sizes over fixed buffer sizes is the
application layer memory savings.
The main contribution of this paper is the proposal of a
technique for dynamic buffer and rate control management,
studied with and without a-priori knowledge of channel
information at the application layer. Aside from the memory
savings, it is also shown that the a-priori knowledge of
channel information at the application layer enhances the
video quality. The problem is formulated as an optimization
problem, where the goal is to minimize the distortion
without violating the buffer constraints. Throughout the
paper, a packet refers to an IEEE 802.11 data-link layer
protocol data unit, whereas a frame denotes a video frame at
the application layer.
The paper is organized as follows. In section II, the
preliminaries of the analysis are first discussed followed by
a formal definition of the problem. Section III contains the
proposed solution scenarios for dynamically controlling the
transcoder and decoder buffers. In Section IV, simulation
results are presented and the paper concludes in section V.
789
 II. PROBLEM FORMULATION yT

A. Preliminaries
Rbt( y ) (T ) = ∫
( y −1)T
Rc (t )dt . (3)

1) Model Assumptions By assumption A4, the instantaneous transcoder buffer

The system model assumptions are as follows: occupancy at any time t can be calculated as:
A1. The maximum size of transcoder and decoder buffers t
is limited and denoted by Btmax and Bdmax (in bits), Bt (t ) = ∫ ( r' (h) − Rc (h) )dh . (4)
respectively. 0

A2. The decoder waits for D video frames in its buffer Specifically, the transcoder buffer occupancy after
before starting the decoding process. It is necessary to keep transcoding y frames is given as:
a certain minimum threshold number of frames in the yT
decoder buffer to provide a cushion against any blackout
periods, in the case of buffer underflow.
Bt ( yT ) = ∫ ( r' (h) − R (h) )dh .
0
c (5)

A3. The decoder buffer is considered empty when there are This can also be written in discrete form as:
only D frames in the buffer. The deadline time, during y
which the next frame should arrive, is assumed to meet the
criteria of maintaining the threshold of D frames in the
Bt ( yT ) = ∑ ⎡⎣ R
j =1
( j) ( j) ⎤
bg (T ) − Rbt (T ) ⎦ , (6)

decoder buffer. where j is the frame index.

A4. Transcoder and decoder buffers are empty at the The expression in (6) shows that the buffer occupancy
startup time t=0, i.e., Bt(t=0) = 0 and Bd(t=0) = 0, after transcoding the yth frame is just the summation of all
respectively. Here, Bt(t) and Bd(t) are respectively the the accumulated bits during the interval 0 to yT at the
transcoder and decoder buffer size at time t. transcoder buffer. Equation (6) can also be written in a
2) Video Distortion recursive manner, where the current buffer occupancy after
Video distortion is a measure of the pixel quality of the transcoding the yth frame is written in the form of buffer
received video as compared to the transmitted video. For a occupancy after transcoding the (y-1)th frame.
given frame y, usually it is estimated as the mean-square
error (MSE) value of the difference between pixel value y −1
( f ) of the transmitted frame and pixel value ( f ) of the Bt ( yT ) = ∑ ⎡⎣ R
j =1
( j) ( j) ⎤ ⎡ ( y) ( y) ⎤
bg (T ) − Rbt (T ) ⎦ + ⎣ Rbg (T ) − Rbt (T ) ⎦ ,
(7)
received frame, as given in (1):
N SL N MB N PX
= Bt ( ( y − 1)T ) + ⎡ Rbg
( y)
(T ) − Rbt( y ) (T ) ⎤ .
∑∑ ∑ E ⎡⎢⎣{ f } ⎤⎥⎦ .
2
MSE( y ) = ( y)
− fz(,ys ,)g (1) ⎣ ⎦
z,s, g
z =1 s =1 g =1 4) Decoder Buffer
In (1), NSL is the number of slices in frame y, NMB is the Let r''(t) denote the rate (in bits/sec) of rendering the
number of macro-blocks in a slice, and NPX represents the video sequence to the user terminal. The number of bits
( y)
number of pixels in a macro-block. rendered Rbr (T ) to the video terminal during the interval
3) Transcoder Buffer ( y − 1)T to yT, is given as:
Let r(t) denote the incoming video bit-rate (in bits/sec) at
yT
the transcoder input, r'(t) denotes the bit-rate (bits/sec) of
∫
( y)
the transcoded video, and Rc(t) is the channel bit-rate Rbr (T ) = r'' (t )dt . (8)
(bits/sec). The transcoded video bit-rate can be written as: ( y −1)T

r' (t ) = β (t )r (t ) , where β(t) is a scaling function. After a According to assumption A2, the decoder waits for D
video frame y is processed at the transcoder, the total frames before starting the decoding process, this
( y) corresponds to a delay of DT seconds. The initial decoder
number of bits generated Rbg (T ) at the buffer, during a buffer occupancy at t = DT is calculated by:
video frame interval time T, is calculated by: D
yT Bd ( DT ) = ∑R ( j)
bt (T ) , (9)
∫
( y)
Rbg (T ) = r' (t )dt , (2) j =1

( y −1)T In general, the decoder buffer occupancy after decoding the

yth frame is given by:
y
where y is the video frame index and T is the frame inter-
arrival time. Bd ( yT ) = Bd ( DT ) + ∑ ⎡⎣ R
j =1
(D+ j)
bt
( j)
(T ) − Rbr (T ) ⎤
⎦
. (10)
Similarly, the transmitted bits Rbt( y ) (T ) from the
The expression given in (10) shows that the instantaneous
transcoder buffer, during the interval (y-1)T to yT, is:

790
decoder buffer occupancy is a function of the initial buffer
occupancy and accumulated bits at the decoder buffer.
5) Channel Estimation
As given in [4] and [5], for an IEEE 802.11 wireless
channel, the channel information can be estimated at the
data-link layer using the number of transmission attempts.
Each transmission attempt at the data-link layer costs a
round-trip time (RTT), which is a measure of the delay on
the network. Because of the RTT cost, the maximum
number of transmission attempts (Rmax) is limited for time-
sensitive applications, such as video streaming. According
to [5], if the number of transmission attempts reaches Rmax,
this indicates a bad network condition. The typical Rmax
value for IEEE 802.11 based wireless network is 4 [6]. In
this paper, we introduce three thresholds L1, L2 and L3
packet transmission attempts for defining the state of the
channel. We assume the threshold L1 = 1 transmission
attempt indicates a good channel. The threshold L2 = 2
packet transmission attempts indicates a moderate channel
condition. Finally, the threshold L3 = 3 or 4 packet
transmission attempts denotes a bad channel, this setting is
consistent with [4] and [5]. The channel information (i.e.,
good, moderate or bad channel condition) is available after a
successful transmission of each data-link layer packet and
this information is used for encoding the next video frame.
B. The Optimization Problem
Define a vector G, which denotes the application layer
parameters:
{
G = Bt ( yT ), Bd ( yT ), ( y)
Rbg }.
(T ) (11)
y
where Bt(yT), Bd(yT) and Rbg (T ) are given by eqns. (7),
(10) and (2), respectively.
Problem P1:
arg min {MSE( y )} ,
(G )
subject to:
(12)
1 0 < Bd ( yT ) ≤ Bdmax ,
2 0 < Bt ( yT ) ≤ Btmax .
where MSE(y) is given by eqn. (1). According to (12), the
goal is to find the application layer parameters vector G, for
which the video distortion is minimized without violating
the buffer constraints.
III. SOLUTION SCENARIOS
The problem P1 is solved by considering two scenarios.
A. Scenario 1:Without Knowledge of Channel
Information
When the channel information is not known at the
application layer, the transcoding rate cannot be adapted to
the channel. The problem P1 is then solved to determine the
791
optimum values for the transcoder and decoder buffer,
subject to non-occurrence of buffer underflow and overflow.
For the decoder buffer, it is important to note that both the
underflow and overflow are critical, as the former will lead
to terminal screen blackout due to packet starvation, while
the latter would cause the packet-dropping eventually
leading to video jerks. In case of the transcoder, buffer
overflow is more critical than the underflow because
overflow leads to packet-dropping, hence resulting in
quality loss. Conversely, transcoder buffer underflow would
not cause much harm as the decoder still carries a cushion of
packets (assumption A2) to be displayed at the terminal.
The decoder buffer underflow can be avoided if:
0 < Bd ( yT ) . Applying (10) and, after rearranging the terms,
the buffer underflow constraint becomes:
y
∑ ⎡⎣ R
j =1
( j) ( D+ j)
br (T ) − Rbt (T ) ⎤
⎦
< Bd ( DT ) . (13)
The expression given in (13) is the key in finding the
threshold number of video frames (D) that the decoder must
keep in the buffer to avoid any underflow. From knowledge
of the fixed video rates and, by combining equations (9) and
(13), the minimum number of D packets can be determined.
Similarly, from (10) the decoder buffer overflow can be
avoided if Bd ( yT ) ≤ Bdmax . This can also be written as:
y
Bd ( DT ) + ∑ ⎡⎣ R
j =1
(D+ j)
bt
( j)
(T ) − Rbr (T ) ⎤
⎦
≤ Bdmax . (14)
Once D is determined from equations (9) and (13), the value
of Bdmax can be found using equation (14).
At the transcoder side, the overflow constraint is given
by: Bt ( yT ) ≤ Btmax , which can also be written as:
Bt ( ( y − 1)T ) + Rbg
( y)
(T ) − Rbt( y ) (T ) ≤ Btmax . (15)
From knowledge of transcoding rate, channel rate and
previous buffer occupancy conditions, the limit on the
transcoder buffer can be determined using (15).
B. Scenario 2: With Known Channel Information
When the channel information is known, the optimization
problem P1 is solved by controlling the transcoding rate in
the vector G, in conjunction with varying the buffer sizes. In
the following, we propose a three-step strategy to solve the
problem P1.
1) Step 1:
Regardless of the video coding standard used, the source
encoding parameters (e.g., target bit-rate and quantization
point) are pre-estimated for the required bit-budget
allocation. For example, in the H.264 standard, an estimate
of bit-budget is made to distribute the available bits to each
frame [1], considering the empty buffers at the application
layer.
 2) Step 2: should be avoided. Also, a lower value (<0.8) of the
Find the optimal buffer sizes for which the distortion can multiplication factor for the bad channel condition would
be minimized, as given in section III.A. This sets an upper distort the video quality.
bound on the size of transcoder and decoder buffer for
which the optimization is achieved. IV. SIMULATION RESULTS
3) Step 3:
We used the JM reference software [8] to transcode the
For the given frame, after capping the transcoder and
video sequence, while NS2 [9] based simulation is used for
decoder buffer sizes to a fixed value determined in step 2,
video streaming. The first frame of the video sequence is
the new transcoding rates are calculated to ensure that the
encoded as an I-frame, while subsequent frames are P-
constraints are not violated. It is proposed here to further
frames (i.e., predicted frames). The rate-distortion (RD)
reduce the video transcoding rate if the channel condition is
optimization was enabled and context-adaptive binary
bad. This will not only help improve the loading on the
arithmetic coding (CABAC) was used for the entropy
network, but also smooth-out the transcoded video stream.
coding. The video frame rate is set to 30 frames per second.
For the moderate channel, it is proposed to use the
Without loss of generality, an IEEE 802.11b link is selected
calculated video bit-rate as is to take the full advantage of
between the access point (AP) and the client device, as
the current channel state. When the channel condition is
shown in Fig. 1. A video frame is divided into the data-link
good, the target bit-rate is increased to exploit the good
layer packets of equal sizes, such that a packet size would
channel condition for higher video quality.
not exceed 1000 bytes. A joint FEC/ARQ scheme, as
When the error correction mechanisms, e.g., joint forward
described in our previous work [7], is implemented at the
error correction (FEC) and automatic repeat request (ARQ)
data-link layer for error correction. Three different video
[7] are used for video streaming over wireless networks, the
sequences, i.e., Foreman, Container and Akiyo are selected
packet transmission information is readily available at the
each from high, medium and low motion categories,
data-link layer. In this paper, we use the cross-layer
respectively. Three test cases are considered, where in 1) no
signaling strategy to convey the transmission and hence
buffer and rate control scheme is applied, 2) buffer control
channel condition information to the application layer,
is applied when no channel information is available, and 3)
where the transcoder utilizes this information for video
buffer and rate control schemes are applied when the
transcoding. An algorithm for refining the calculated target
channel information is available. The maximum size of the
transcoding rate is given as follows:
decoder buffer (Bdmax) is set as equal to the five times the
Algorithm I: Refining the Calculated Target Transcoding
size of the largest frame in the video sequence, i.e., the I-
Rate
( y)
frame. The cushion value of 3 video frames in the decoder
Input: number of transmission attempts=L, Rbg (T ) buffer gives sufficient protection against any blackout
( y)
Output: Rbg (T ) period.
Begin A. Improvement in Video Quality
if ( L = L1 ) Peak-signal-to-noise-ratio (PSNR) is used as the measure
{ /* c h a n n e l s t a t e = G o o d * / of objective video quality. The quality of wireless channel is
R b( gy ) ( T ) = 1 .2 R b( gy ) ( T ) determined from the data-link layer packet error probability
} (µ), which is varied from 10-4 to 10-1, i.e., going from good
e ls e i f (L = L2 ) to bad channel condition. As shown in Fig. 2, when the
{ /* c h a n n e l s t a t e = M o d e r a te * / channel condition deteriorates, there is a general trend of
R b( gy ) ( T ) = R b( gy ) ( T ) lower PSNR, as expected. For all the video sequences,
} under bad channel condition (µ=10-1) a 0.5 dB to 1 dB
e ls e i f (L = L3 ) increase in PSNR is observed, when both the buffer and rate
{ /* c h a n n e l s t a t e = B a d * / control schemes are used. This is attributed to the fact that,
R b( gy ) ( T ) = 0 .8 R b( gy ) ( T ) for the bad channel condition, there is a high probability of
}

End
where L1, L2 and L3 are given in section II.A.5. Note that the
multiplication factors in Algorithm I are empirically
determined values that best suit the channel conditions. For
the good channel condition, a larger value (>1.2) of the
multiplication factor would lead to the disturbance in pre-
calculated bit-budget allocation in H.264 encoder [1], hence Figure 1: Simulation model

792
 20000
No Control

Decoder Buffer Size (bits)

Buffer Control (w/o channel info)
15000
Buffer+Rate Control (w/ channel info)

10000

5000

0
Foreman Container Akiyo
Video Sequence

Figure 3: Comparison of decoder buffer sizes

Figure 2: Comparison of the average PSNR values
packet-dropping at the data-link layer, but the rate reduction
mechanism proposed in this paper reduces the video bit-rate,
thereby lowering the probability of packet-dropping and
hence an increase in PSNR. For the good channel condition
(i.e., µ=10-4), a slight increase of about 0.1 dB to 0.2 dB can
be seen in all the three video sequences, when rate control
mechanism is used, due to the increase in video encoding
bit-rate as given by Algorithm I.
B. Dynamic Buffer Stabilization
From Fig. 3, it is seen that, under moderate channel
condition (µ=10-2), the average buffer requirement drops by
almost an order of magnitude when both buffer and buffer
plus rate control schemes are employed. The reduction in
buffer size for the adaptive schemes is due to the fact that
buffer sizes are now calculated in real-time for each video
frame instead of being at a fixed value, as is the case when
there is no control. Comparing only the two adaptive
schemes, there is a small increase of about 20 bits for the
case where the buffer plus rate control scheme is employed.
This is attributed to the fact that in the case of availability of
channel information, the transcoder gets another chance of
increasing or decreasing the encoding rate, based on good or
bad channel conditions, respectively. Under bad channel
condition, the encoding rate drops thus giving less number
of bits per frame, however the packet error probability
increases, hence negating the effect of lower bit-rate for the
decoder buffer. Under good channel condition, the increase
in encoding bit-rate translates to a higher buffer
requirement, but this is not a very significant increase when
compared to the fixed buffer case.
V. CONCLUSION
In this paper, we have presented a technique for
dynamically optimizing the application layer parameters and
compared it against the case where no such scheme is
implemented. It is shown in this paper that when the channel
information is available at the application layer, the video
quality improves by up to 1 dB, which translates to a better
viewing experience. Additional saving of the application
layer buffer by approximately an order of magnitude is also
achieved, thereby decreasing the memory requirement.
ACKNOWLEDGMENT
The authors acknowledge the support of the University of
Calgary, TRLabs and National Sciences and Engineering
Research Council (NSERC) Canada for this research.
REFERENCES
[1] ITU-T and ISO/IEC JTC1, “Advanced video coding for generic
audiovisual services,” ITU-T Recommendation H.264 – ISO/IEC
14496 AVC, 2003.
[2] M. Yokotsuka, “Memory motivates cell-phone growth,” Wireless
Systems Design, vol. 9, no. 3, 2004, pp. 27–30.
[3] Z. Lei and N. D. Georganas, “Adaptive video transcoding and
streaming over wireless channels,” Journal of System and Software,
March 2004, pp. 253 – 270.
[4] M. van der Schaar and D. S. Turaga, “Cross-layer packetization and
retransmission strategies for delay-sensitive wireless multimedia
transmission,” IEEE Transactions on Multimedia, vol.9, no.1, Jan.
2007, pp.185-197.
[5] J. Lee and M. Kang, “Design of a dynamic bandwidth reallocation
scheme for hot-spot video stream transmission over the IEEE 802.11
WLAN,” 2006 TENCON, IEEE Region 10 Conference, Nov. 2006.
[6] V. Sgardoni, P. Ferre, A. Doufexi, A. Nix and D. Bull, “Frame delay
and loss analysis for video transmission over time-correlated
802.11a/g channels,” IEEE 18th International Symposium on
Personal, Indoor and Mobile Radio Communications, PIMRC 2007,
3-7 Sept. 2007.
[7] A. Moid and A. O. Fapojuwo, “An analytical model for optimum
byte-level and packet-level FEC assignment using buffer dynamics,”
Research Letters in Communications, Article ID 546184, 2008.
[8] JM Reference Software, Available at:
http://iphome.hhi.de/suehring/tml, Accessed on June 20, 2008.
[9] Network Simulator (NS2), Available at: http://www.isi.edu/nsnam/ns,
Accessed on May 03, 2007.

793