Sie sind auf Seite 1von 6

Adaptive Video Streaming for UAV Networks

Severin Kacianka Hermann Hellwagner


Institute of Information Technology (ITEC) Institute of Information Technology (ITEC)
Alpen-Adria-Universität Klagenfurt Alpen-Adria-Universität Klagenfurt
Austria Austria
severin.kacianka@aau.at hermann.hellwagner@aau.at

ABSTRACT mobility of UAVs leads to ever-changing antenna orienta-


The core problem for any adaptive video streaming solution, tions and network topologies. The induced problems can be
particularly over wireless networks, is the detection (or even so severe that some authors [2, 14] suggest that a new wire-
prediction) of congestion. IEEE 802.11 is especially vulner- less LAN standard might be needed. Asadpour et al. [2]
able to fast movement and change of antenna orientation. write that the “special ecosystem [of UAV networking] un-
When used in UAV networks (Unmanned Aerial Vehicles), doubtedly requires a rethinking of wireless communications
the network throughput can vary widely and is almost im- and calls for novel networking approaches.”
possible to predict. This paper evaluates an approach orig- Streaming a video over such an unreliable channel causes
inally developed by Kofler for home networks, in a single- additional problems:
hop UAV wireless network setting: the delay between the • Video streaming requires a lot of bandwidth,
sending of an IEEE 802.11 packet and the receipt of its cor-
responding acknowledgment is used as an early indicator of • reacts strongly to link degradation, and
the link quality and as a trigger to adapt (reduce or increase)
the video stream’s bitrate. Our real-world flight-tests indi- • a single missing packet can cause the loss of several
cate, that this avoids congestion and can frequently avoid seconds of video data.
the complete loss of video pictures which happens without
Yet the quality of video streams can be adapted to fit (al-
adaptation.
most) any available bandwidth. If a link degradation can be
detected (or even predicted) in time, it is possible to reduce
Categories and Subject Descriptors the stream’s bitrate and avoid congestion and a complete
loss of the video picture.
C.2.1 [Computer-Communication Networks]: Network
The goal of our research was to address these problems
Architecture and Design—Wireless communication
and prevent the video signal’s complete loss by using a method
described by Kofler [8, 9] to indicate near-term congestion
Keywords and to adapt the UAV’s video stream bitrate to the current
Video Streaming, Adaptive Streaming, UAVs, UAV Com- network conditions.
munication
2. RELATED WORK
1. INTRODUCTION 2.1 Wireless Networks
In most application scenarios, UAVs will require a perma- In a survey paper, Bekmezci et al. [4] identify communica-
nent link to some base or control station. Especially in civil- tion as the biggest challenge for multi-UAV systems and coin
ian and scientific systems, this link will often use the IEEE the term “Flying Ad-Hoc Networks (FANETs).” This term
802.11 wireless LAN technology. 802.11’s greatest advan- is inspired by previous work on “Mobile Ad-Hoc Networks”
tages are that it is cheap, readily available, well understood (MANETs) and “Vehicle Ad-Hoc Networks” (VANETs) and
by developers, and widely supported by software and op- can be considered a special form thereof. MANETs gener-
erating systems. However, 802.11 was never intended to be ally consider people moving in an environment using mobile
used for fast moving and direction changing devices. The in- devices and VANETs generally consider cars moving on a
tended use cases were static devices like laptops. The high road. Bekmezci et al. differentiate FANETs from MANETs
Permission to make digital or hard copies of all or part of this work for and VANETs for several reasons:
personal or classroom use is granted without fee provided that copies are not
made or distributed for profit or commercial advantage and that copies bear • UAVs move much faster and have more degrees of free-
this notice and the full citation on the first page. Copyrights for components dom than people or cars.
of this work owned by others than ACM must be honored. Abstracting with
credit is permitted. To copy otherwise, or republish, to post on servers or to • Because of UAVs’ higher mobility the network topol-
redistribute to lists, requires prior specific permission and/or a fee. Request
permissions from Permissions@acm.org. ogy will change much more frequently.
MoVid ’15, March 18-20, 2015, Portland, OR, USA
Copyright 2015 ACM 978-1-4503-3353-5/15/03 ...$15.00 • FANETs deliver control commands as well as sensor
http://dx.doi.org/10.1145/2727040.2727043 data.
• The distance in FANETs is usually much greater than tion (like sync or probe messages) needs to be trans-
in MANETs or VANETs. Thus communication hard- mitted.
ware needs to be different.
2. Being a local approach, it does not need any sup-
Andre et al. [1] focus on the design choices of aerial com- port from clients. Any client that supports RTP and
munication networks and cover all common wireless com- H.264/SVC can be used.
munication technologies, namely IEEE 802.15.4 (XBee Pro),
3. An increase in queueing delays precedes packet loss,
IEEE 802.11 (WiFi), 3G/LTE (mobile phone network), and
thus allowing fast reaction to upcoming link degrada-
infrared. For video streaming from UAVs, Andre et al. rec-
tion.
ommend IEEE 802.11 technology. They find that WLAN
can deliver the required data rates even over longer dis- The adaptation logic then works as follows: If the delay
tances and that this distance can easily be extended by using exceeds a certain threshold, the proxy will reduce the video’s
multi-hop networks. They conclude that “(...) the mea- quality. If, however, the proxy measures a delay which is
sured throughput confirms the practicality of WLAN. Nev- lower than the threshold, the proxy will evaluate past delays
ertheless, achievable throughput heavily depends on antenna and if it finds them low as well, it will try to increase the
placement and the mobility of the MAV [micro aerial vehi- stream’s quality. Previous packets serve as “packet pairs”
cle], and technological improvements are required to guar- and their delay is used to estimate available link capacity.
antee necessary throughput.” [1] This works because video stream packets are usually the
same size and sent back-to-back.
2.2 Video Streaming
Despite the prominence of the use case, there is scant lit- 3. HYPOTHESES
erature that focuses on UAV networks (or FANETs) and
Kofler [8, 9] could link the delay between the sending of an
video transmission. Most papers seem to focus either on the
IEEE 802.11 packet and the receipt of its acknowledgment
task of recording a video (quality, coding, object tracking,
to the link quality and adapt a video stream accordingly.
etc.) or on the UAV network and its properties. The au-
This paper tries to apply this idea to UAV video streaming
thors are not aware of any paper describing a system that
and to evaluate three hypotheses:
combines networking and adaptive video streaming or other
video transmission techniques. For MANETs, however, Lin- 1. The delay is related to video quality.
deberg et al.’s [10] survey paper is a good resource to get an
overview of the “challenges and techniques” for video stream- 2. The delay is a good indicator of impending problems
ing in MANETs. They stress the fact that meeting band- with the video stream.
width, delay and jitter constraints in wireless networks is far
3. The delay can be the input for an adaptation proxy
more challenging than in wired networks. They state that
that improves the overall video quality.
“(...) a significant amount of overall packet drops in wireless
networks are perceived as being random, rather than caused Kofler could show that this delay is a good indicator for
by congestion (...).” impending link degradation in a home-use setup. This setup
Although technologies like Scalable Video Coding (SVC) consisted of a common-off-the-shelf router and a laptop.
[13] or lightweight encoders [5] will become interesting in the Because the scenario was home use, the movement evalu-
future, there are currently no common off-the-shelf solutions ated was only at walking speed (a person walking through
that make them readily available (especially for real-time his/her flat). For this slow movement, the approach worked
applications). So in real-world settings, one is reduced to remarkably well. However, as touched upon in the introduc-
tweaking the H.264 encoder’s parameters. tion, UAV networks change much faster and more violently.
This paper provides the first evaluation of an adaptive video
2.3 Congestion Detection streaming approach in such an environment.
Kofler [8, 9] combines the idea of packet pairing and pas-
sive sensing from the field of bandwidth estimation (e.g., 4. SYSTEM DESIGN
[6, 12]) with the fact that video streams (using H.264 and
RTP) will have many packets of equal size. He presents a 4.1 Overview
proxy server that can adapt SVC video streams according
to current network conditions. However, “instead of reacting The experiments used a Logitech C920 HD webcam [11]
on packet loss, [the] approach uses an increase in queueing mounted on an Ascending Technologies (AscTec) Firefly [3]
delay at the router to detect phases of throughput degrada- UAV (shown in Figure 1). It carried a “Mastermind board”
tion.” [9] Kofler measures the queueing delay indirectly. He that was outfitted with a 1.86GHz processor and 4GB of
noticed that the Linux kernel’s wireless monitoring interface RAM. The wireless card was a common off-the-shelf Atheros
will only show frames after they were acknowledged by the AR93xx chipset with three external antennas and IEEE 802.11
recipient. So the worse a link is, the longer the time between a/b/g/n support. The system used AscTec’s Ubuntu 12.04
sending of the frame and the receipt of the corresponding (LTS) image with Linux kernel 3.5.
acknowledgment will be (because, for example, a frame has Figure 2 shows an overview of the system architecture.
to be retransmitted). It could be shown that this queue- The camera delivers an H.264 video stream and is controlled
ing delay corresponds with the link quality and the packet by gstreamer [7], an open-source multimedia framework.
loss. Increasing delay usually precedes actual packet loss by gstreamer can request (almost) any bit rate from the cam-
almost two seconds. This approach has three advantages: era and passes the H.264 encoded video stream on to the
proxy as a normal RTP stream. gstreamer (or more ex-
1. The information is available locally – no extra informa- actly, the shell script controlling it) then offers the proxy
3. When the connection is good (for example the UAV is
next to the laptop), the delay does not increase and
the video stream stays stable.

The proxy consists of two threads (Figure 3): The thread


proxy just accepts RTP packets from gstreamer and for-
wards them to the base station while noting the time of
sending. The sniffer thread listens on the Linux kernel’s
monitoring interface (in our case always called mon0) and
whenever it sees an RTP packet it will log the time (which
is the time of the reception of the MAC layer acknowledg-
ment) and compare it to the send time. It matches the
packets by their RTP sequence number.

Figure 1: The UAV in flight

an interface to request a rate change. In one thread, the


proxy forwards the RTP stream unaltered to the base sta-
tion (that is running the video streaming client software),
but keeps track of when each RTP packet is sent. At the
same time, another thread listens on the wireless monitoring
interface of the Linux kernel for the MAC layer ACK of the
sent frame. Depending on the delay between the sending
of an RTP packet and its acknowledgment, the proxy will
adapt the stream.
Figure 3: Overview of the proxy architecture

The decision when to adapt is directly influenced by the


three observations mentioned above. The algorithm tries to
avoid delays higher than 1.0 seconds by reducing the video
stream’s bit rate as soon as the delay crosses the threshold
of 0.3 seconds. As it usually takes some time for the delay
to react to the lower bitrate, i.e., to decrease, the algorithm
will wait for a period of 25 consecutive packets (“cooldown
period”) before reducing the quality further. Without this
cooldown period, the proxy would reduce the quality down
to the lowest level most of the time.
Figure 2: Overview of the system architecture
When there is an extended period (250 packets) of packets
with a low delay (<0.1sec), the proxy will increase the bit
4.2 Adaptation Proxy rate by one level. Please note that all these values were
gathered empirically. After the initial experiments, these
The Adaptation Proxy forwards the RTP stream sent from values were determined to give the best results. However,
gstreamer, measures the delay between the sent packets and tuning these values was not a priority – so there are probably
the reception of their ACKs and makes adaptation decisions. combinations that would provide better results.
From Kofler’s work the following three assumptions were
drawn:
5. RESULTS
1. A frame only shows up on the monitoring interface The final data set consists of 12 evaluation flights (9 with
after it was transmitted and the ACK received. adaptation, 3 without adaptation) taken on two consecutive
2. The lower the delay, the better the connection. days (May 5–6, 2014)1 . The flights were conducted in an
area close to the University and the UAV was controlled by
3. The delay is mainly affected by the connection quality one of the authors. During the flights, the UAV was flown
(no random changes). towards a small wood, where three white hay bales were
“hidden.” The goal of the flights was to find the “hidden”
By means of extensive testing, the following observations hay bales. Around 50 flights were conducted, however, only
to confirm these assumptions were made: 12 used the same software version and settings and are thus
comparable. Each test flight produced two artifacts: (1) a
1. When the delay is high (above ∼1.0 sec), there will be log file that contains information about the packets and (2)
strong artifacts in the video stream.
1
Example videos can be found on http://kacianka.at/ma/
2. When the delay climbs above ∼0.3 sec, it usually con- demo_no_adapt_2014_05_06.mp4 and http://kacianka.
tinues to climb further. at/ma/demo_adapt_2014_05_06.mp4.
the recording of the video stream as it was received at the
base station. The log file was stored on the UAV and con-
tains the following variables2 : distance in meters between
the UAV and its take off position; UNIX timestamp in sec-
onds; time delay between the sending of a packet and the re-
ceipt of the ACK in seconds; RTP sequence number. Flights
that performed adaptation logged an integer representation
of the quality level that was being used and the number of
packets with a “low” delay (less than 0.1 seconds).
In contrast to the log files, the videos are impossible to an-
alyze objectively as there is no “perfect” video, i.e., a refer-
ence, to compare them to. There are no clear-cut indicators
when a video signal is “good” and at what point it becomes Figure 6: “Bad” frame, with hay bales lost due to
“bad.” In the course of this work, “good” means that sig- artifacts
nificant features on the ground can still be made out (even
though the resolution is low) and “bad” means that artifacts
blur the picture beyond recognition. Figure 5, for example,
shows a low resolution image of the hay bales. Although this
image is of significantly lower resolution than the image in
Figure 4, the hay bales can still be made out and the image
is thus considered “good.” Figure 6 on the other hand is
blurred beyond recognition and is thus considered “bad.”

Figure 7: Development of delay without adaptation

as the delay crosses the threshold of 0.3 seconds. It does


not fall immediately, because the cooldown period keeps the
bit rate too high. As soon as quality level 3 is reached, the
delay immediately falls.

Figure 4: High quality frame showing the hay bales

Figure 8: Development of delay with adaptation

The following figures were created by aggregating the data


from all 12 flights. The first three bars (in blue) in each fig-
Figure 5: Low quality frame showing the hay bales ure are the aggregate value of all flights (both with and with-
out adaptation). The next nine bars (in red) show only the
flights with adaptation and the final three bars (in yellow)
5.1 Characteristics of the Delay show only the flights without adaptation.
Figure 7 shows how quickly the delay can rise from low
levels to high peaks. Without a mechanism to counter the 5.2 Delay
effect, the delay will rise until the UAV stops moving and/or Figures 9 and 10 show the maximum and mean delay,
the antenna orientation improves. respectively. It is interesting to see that the flights with
When active bit rate adaptation takes place, it can counter adaptation had a significantly lower mean delay than the
the rise of the delay and keep the periods of high delay flights without adaptation. While the data set is small, these
shorter. Figure 8 shows how the quality level is reduced results confirm Kofler’s findings and suggest that the delay
2
is indeed a very good indicator of the link quality.
The raw data files can be found on
http://kacianka.at/ma/raw_flight_data.zip.
Figure 9: Maximum delay between sending a packet Figure 12: Mean number of packets between the
and receiving the ACK packet first packet with a long delay (>0.3s) until the delay
is below 0.3s again

Kofler [8] could confirm these hypotheses in the context of


(indoor) wireless home networks. The work at hand is based
on these previous results and evaluated them in a single-hop
UAV network. Kofler’s idea was implemented with different
tools, but the basic results can be confirmed. This paper
found a clear correlation between high delays (higher than
1 second) and bad video quality (up to the complete loss
of the video stream). All experiments in the lab as well as
around 50 real-world flight tests exhibited this behavior. A
Figure 10: Mean delay between sending a packet connection breakdown without a period of high delays was
and receiving the ACK packet never observed.
However, sometimes artifacts that do not seem to be re-
lated to any measured factors still arise. They might be in
5.3 Number of Packets to Low Delay the camera’s H.264 encoder or in the decoder. So the delay
Figures 11 and 12 show the number of consecutive packets will not account for all possible problems in a video stream.
that have a delay greater than 0.3s. 0.3s is the threshold used The results strictly assume that a good connection is char-
to reduce the bit rate of the video. The fewer packets have a acterized by a low delay (less than 0.1 seconds) and that a
high delay, the better the video quality is. The flights with bad connection is indicated by an increase of the delay.
adaptation have far shorter periods of “bad connection” than The results of the real-world flight tests indicate that the
the flights without adaptation. As longer periods of lower described solution works indeed better than just using a
delays mean a better video quality, these results suggest that static video bit rate. The authors’ (biased) observation is
applying Kofler’s congestion detection method and adaptive that flying with video adaptation leads to shorter “break
video streaming to UAV networks will indeed lead to a better downs” of the video pictures and that one can still get a
video picture quality. good overview of the area of flight when streaming in low
quality. This is backed up by the data displayed in Figure 12
that shows that flights with adaptation have far fewer de-
layed packets than flights without adaptation.
In summary, this paper cautiously confirms all three hy-
potheses in the context of UAV networks. However, the
number of test flights was low and thus the results should be
considered preliminary. They are encouraging and beckon
further research.

Figure 11: Maximum number of packets between


7. LIMITATIONS OF THE EXPERIMENTAL
the first packet with a long delay (>0.3s) until the DESIGN
delay is below 0.3s again The experiments’ greatest weakness is the low number of
flights. Due to new laws and problems with Austria’s flight
control agency Austro Control, the authors were only able to
6. DISCUSSION conduct twelve comparable flights of which three were done
without video adaptation. “Comparable,” however, does not
In this paper three hypotheses were presented: mean that all influencing factors were actively controlled.
On the contrary, the experimental design did not control the
1. The delay is related to video quality.
UAV’s flight path and the antenna orientation. As laid out
2. The delay is a good indicator of impending problems in the introduction, Wi-Fi connectivity is highly impacted
with the video stream. by antenna orientation, movement speed, and height. All
these factors were not logged and not fixed. It is possi-
3. The delay can be the input for an adaptation proxy ble that the flights without adaptation were (unconsciously)
that improves the overall video quality. conducted with a worse UAV positioning. As with the ori-
entation of the UAV, the orientation of the base station was 11. REFERENCES
also not controlled. [1] T. Andre, K. A. Hummel, A. P. Schoellig, E. Yanmaz,
Another significant weakness is that the Wi-Fi equipment M. Asadpour, C. Bettstetter, P. Grippa,
was used with default settings. It is likely that the settings H. Hellwagner, S. Sand, and S. Zhang.
can be tweaked to improve performance and reliability of the Application-driven design of aerial communication
connection. Fixing the channel’s bit rate or disabling block networks. IEEE Communications Magazine,
acknowledgments could change the video stream’s behavior. 52(5):129–137, 2014.
Another weak point is that the described solution requires [2] M. Asadpour, B. Van den Bergh, D. Giustiniano,
acknowledgment packets, which are not sent for multicast K. Hummel, S. Pollin, and B. Plattner. Micro aerial
transmissions. vehicle networks: An experimental analysis of
challenges and opportunities. IEEE Communications
8. REPRODUCIBILITY Magazine, 52(7):141–149, 2014.
These shortcomings in the experimental design violate a [3] Ascending Technologies. http://www.asctec.de/uav-
cornerstone of the scientific method: reproducibility. While applications/research/products/asctec-firefly/. Online,
the configuration and setup of the UAVs is easy to repro- no date. Accessed 2014/05/17.
duce, the evaluation of Kofler’s congestion detection algo- [4] İ. Bekmezci, O. K. Sahingoz, and Ş. Temel. Flying
rithm relies on preliminary “ad-hoc” tests. The aforemen- ad-hoc networks (FANETs): A survey. Ad Hoc
tioned problems with Austro Control prevented the authors Networks, 11(3):1254–1270, 2013.
from designing the flight tests more carefully and forced [5] M. Bhaskaranand and J. Gibson. Low-complexity
them to rely on data gathered during “proof-of-concept” test video encoding for UAV reconnaissance and
flights. surveillance. In Military Communications Conference,
Future work should therefore be mindful of these influenc- 2011 - MILCOM 2011, pages 1633–1638, 2011.
ing factors: [6] A. Cabellos-Aparicio, F. Garcia, and
• The position of the base station and the UAV take-off J. Domingo-Pascual. A novel available bandwidth
spot should be kept stable. estimation and tracking algorithm. In IEEE Network
Operations and Management Symposium Workshops,
• The UAV’s flight path should not be manually con- pages 87–94, 2008.
trolled. Flying to a predefined set of GPS coordinates [7] GStreamer. http://gstreamer.freedesktop.org. Online,
should be preferred. no date. Accessed 2014/06/11.
[8] I. Kofler. In-Network Adaptation of Scalable Video
• The UAV’s altitude and cruise speed should be prede- Content. PhD thesis, Alpen-Adria-Univerität
fined. Klagenfurt, 2010.
• The UAV’s yaw (and thus the antenna’s orientation) [9] I. Kofler, R. Kuschnig, and H. Hellwagner. In-network
should be predefined. adaptation of H.264/SVC for HD video streaming over
802.11 g networks. In Proceedings of the 21st
• The interferences of the fuselage and the mainboard International Workshop on Network and Operating
on the antennas should be reduced by placing them on Systems Support for Digital Audio and Video, pages
a base that extends below the UAV. 9–14. ACM, 2011.
[10] M. Lindeberg, S. Kristiansen, T. Plagemann, and
• All wireless traffic could be recorded by a third lis-
V. Goebel. Challenges and techniques for video
tening station that is positioned along the flight path.
streaming over mobile ad hoc networks. Multimedia
Later analysis of this data might yield useful informa-
Systems, 17(1):51–82, 2011.
tion.
[11] Logitech. http://www.logitech.com/en-
However, as touched upon by [1, 2, 14] reproducibility for us/product/hd-pro-webcam-c920. Online, no date.
UAV networks and the design of testbeds are still open and Accessed 2014/05/17.
difficult problems. [12] R. Prasad, C. Dovrolis, M. Murray, and K. Claffy.
Bandwidth estimation: metrics, measurement
9. FUTURE WORK techniques, and tools. IEEE Network, 17(6):27–35,
2003.
The results of these experiments suggest that the delay
[13] T. Schierl, T. Stockhammer, and T. Wiegand. Mobile
is a good indicator for the link quality. It can be used to
video transmission using scalable video coding. IEEE
adapt the quality of the video stream to fit the available
Transactions on Circuits and Systems for Video
bandwidth. Further research should first establish whether
Technology, 17(9):1204–1217, 2007.
the effect indeed exists and how (statistically) significant the
effect really is. A follow-up study could repeat the experi- [14] E. Yanmaz, R. Kuschnig, and C. Bettstetter. Channel
ment described in this paper, but take great care to conduct measurements over 802.11 a-based uav-to-ground
the experiments in a reproducible way. links. In IEEE GLOBECOM Workshops (GC
Wkshps), 2011, pages 1280–1284. IEEE, 2011.
10. ACKNOWLEDGMENTS
The work was supported by the ERDF, KWF, and BABEG
under grant KWF-20214/24272/36084 (SINUS). It has been
performed in the research cluster Lakeside Labs.

Das könnte Ihnen auch gefallen