Real-Time Transmission of Video Using WLAN

Engineering Thesis Progress Seminar by Alexander Stevens ---------------------------Supervisor Konstanty Bialkowski

Motivation for Project

Need a cost effective way to stream high quality video from experimental UAV's (Unmanned Aerial Vehicles) What if the UAV travels out of eye sight? Immediate feedback on performance of UAV Computer vision systems are processor intensive; need video transmission to offboard computer

Previous Work into UAV Traffic Surveillance

Carried out by University of Florida Investigated the use of UAV's instead of CCTV Found it cheaper to use UAV's on microwave tower infrastructure (TV, Radio, etc) Regulations dictate licensing and bandwidth

Previous Work into UAV Traffic Surveillance

Model relied on high data throughputs Because of this, raw video lacks quality of compressed video

The Task

Real-time video streaming from portable device Rate control algorithm to adhere to <200ms of lag Optimise stream to keep high frame-rate Control quality of video to acceptable levels

200ms

The Platform

Needs to be portable, low power Needs to be able to encode video in real time High processing speed... Low power?! D: Will need an ARM based platform with Digital Signal Processors (Hardware Acceleration) PandaBoard vs BeagleBoard-xM

720p Video 30 FPS Encode/Decode

Pandaboard by Texas Instruments

30 FPS Encode/Decode

BeagleBoard-xM

802.11 b/g/n

Pandaboard (<5W Power) BeagleBoard (~2W Power) If optimisations don't kick in, need extra processing power!

The Operating System

Since platforms are both ARM based, need ARM based Operating System Linux vs Android (Essentially, C/C++ vs Java) Linux wins for PandaBoard development Choice of compiler Pre-built libraries (x264, ioctl, libv4l2, etc) Therefore use Ubuntu, as headless images are provided (and supported by Texas Instruments)

Capture

Need library to interface with camera Video4Linux2 (V4L2) built into Linux Kernel libV4L2 as an abstraction layer to V4L2 Enables us to use essentially any camera Open source!
Camera Linux Kernel Logitech Video4Linux2 Camera Driver libV4L2 Program/Encoder

V4L2 not cross platform, but that's okay

The Encoder

Need an encoder with adequate compression Has to be open source or have libraries, so we can freely modify the code Google's VP8 and H.264 (x264), both open source VP8 has no hardware acceleration on the Pandaboard, where as H.264 does Therefore, the x264 open source implementation of H.264 is most appropriate encoder

The Encoder H.264

Achieves compression through different frame types; I, P and B-frames

Intra-Frames: Minimal compression, Quality Predictive-Frames: Medium compression Bi-directional-Frames: Maximum compression

Transmission

Need library to interface with encoder and packet into a standardised protocol RTCP (Real-time Transport Control Protocol) ideal for conference calls and in this case, transmission of portable video RTCP gives feedback on transmission stats Need these stats to determine dropped frames (need to know when intra-frame dropped), and to have feedback control

Transmission continued...

RTCP standardised within many media players (VLC, MPlayer, etc) Live555 establishes RTCP connections while interfacing on the back end of an encoder
Camera Screen Client
RTCP Compliant Player

libV4L2

Control Algorithm

Video Wireless RTCP Wireless Stats

UAV Application x264 Encoder Live555

Open source! Can freely modify code

Current Progress

First Step

Capturing images from Camera (V4L2) Setting images up into a Buffer (mmap) Save images (for experimentations sake) Demonstration Writing x264 calls to pull buffered images Encode buffered images into video frames Save encoded stream (experimentation) Demonstration

Second Step Currently at (almost finished)

Future Progress

Third Step About to Begin

Write H.264 interface for Live555, so Live555 understands how to packet H.264 stream Connect encoded stream to Live555 Experiment with wireless connection to VLC Write control algorithm that reads data from RTCP statistics and controls encode rates (kb/s, frame-rate, resolution, etc) Optimise program and control

Fourth Step

Future Potential

Potential for extensions to software... Examples:

Bunny hopping data over wireless routers Pan and Zoom functionality (for camera) Control of UAV remotely Open the video stream for computer vision projects

Questions?

http://code.launchpad.net/~alex-stevens/+junk/spyPanda