Setting up Multiple USB3 Cameras

Revised 1/31/2017

Applicable Products 2
Application Note Description 2
FLIR USB 3.1 Technology 2
Configure Drivers 3
Bandwidth Allocation 3
Device Link Throughput Limit—GenICam 3
Format 7 Packet Size—IIDC 3
Calculating Required Bandwidth 4
Multiple Camera System Configurations 5
Multiple Cameras Connected Directly to a USB 3.1 Host Controller 5
Multiple Cameras Connected via a USB 3.1 Hub 6
Multiple Cameras Connected to Multiple Host Controllers 8
Troubleshooting 9
Enumeration 9
Link Recovery Errors 9
Skipped Image Frames 9
Low Frame Rate 10
Downloads and Support 10
Finding Information 10
Contacting Technical Support 11

Names and marks appearing on the products herein are either registered trademarks or trademarks of FLIR Systems, Inc. and/or its subsidiaries.
© 2012-2017 FLIR Integrated Imaging Solutions Inc. All rights reserved.
TAN2012005—Setting up Multiple USB3 Cameras

Applicable Products
FLIR USB 3.1 machine vision cameras

Application Note Description

This application note explores multiple USB 3.1 camera configurations. The purpose of this document is to provide
a reference and a starting point for USB 3.1 camera users who wish to use multiple USB 3.1 cameras in their
application.

FLIR USB 3.1 Technology

The type of data transfer that our USB 3.1 cameras use is bulk transfer. When factoring in the related overhead,
the effective bandwidth is approximately 450 MB/s.  Our USB 3.1 implementation uses image buffering to
improve multiple camera system reliability and ensure data integrity.

Packet Retransmission

Our USB 3.1 cameras use a type of data transfer called bulk transfer. This type of transfer guarantees delivery and
retransmits corrupt data packets, recovering from data errors when they occur.

Data Buffer

Our USB 3.1 cameras have a data buffer that allows images to be temporarily queued in the camera after readout. 
The main advantage of the data buffer is that it allows the system to handle momentary slowdowns on the bus
without dropping any data. If the overall data throughput drops below requested data rate for an extended period of
time, the camera can send entire uncorrupted frames out slower, and drop entire frames cleanly, rather than drop
partial frames and cause image data errors.

USB 3.1 link layer IP

We have implemented our own USB 3.1 link layer technology in the camera FPGA.  This allows us to introduce
new functionality and ensure compatibility with third party hardware.

USB3 Vision Standard

Our USB 3.1 cameras comply with the USB3 Vision standard. USB3 Vision is a communication interface for
vision applications based on the USB 3.1 technology. All cameras supporting USB3 Vision interact the same way
®
with software also supporting USB3 Vision. The Spinnaker SDK uses GenICam to support the USB3 Vision
standard.

For more information on the USB3 Vision standard, see visiononline.org.

1/31/2017
©2012-2017 FLIR
Integrated Imaging Setting up Multiple USB3 Cameras 2
Solutions Inc.
All rights reserved.
TAN2012005—Setting up Multiple USB3 Cameras

Configure Drivers
FLIR sells three USB 3.1 host controller cards: two based on the FL1100 chipset from Fresco and the µPD720200
chipset from Renesas. In addition to the manufacturer drivers, we recommend using the PGRUSBCam driver.  For
a list of compatible chipsets, see TAN2011005 Recommended USB 3.1 System Components.
®
With SpinView (for Blackfly S) or our driver control utility, you are able to identify the driver you are using and
easily switch between them.  Select the Point Grey Driver from the driver selection window. 

Bandwidth Allocation
USB 3.1 does not automatically manage bandwidth allocation. To help users accommodate multiple cameras on
a single bus, we use a camera attribute which limits peak data bandwidth.  This can be used to reduce the amount
of data coming from each camera, preventing skipped frames or connectivity issues.

Device Link Throughput Limit—GenICam

Use the device throughput limit feature to verify your system’s bandwidth limit, ensure your total bandwidth use is
within that limit, and adjust your camera’s bandwidth accordingly. Consider the total of all the cameras on the
system to calculate the required bandwidth.

We recommend that the bandwidth requirements be divided equally among the number of cameras used, for
example—two cameras, each requiring half of the maximum bandwidth; three cameras, each requiring one third
of the maximum bandwidth.

In the figure below, we see that the device max throughput is 38768000 bytes per second (384 MB/s). The current
device link throughput limit is 18320000 bytes per second (184 MB/s).

Format 7 Packet Size—IIDC

Format 7 packet size provides the user with a mechanism to allocate bandwidth for a particular camera. Reducing
the packet size lowers the camera’s maximum frame rate and limits the amount of bandwidth the camera can
®
use. The packet size can be controlled in FlyCapture 2 or the IIDC registers.

FlyCap2 can be used to determine the bandwidth allocated for a given packet size.  Reduce or increase the packet
size to ensure the total amount of bandwidth output by the camera(s) does not exceed system limits.

In the figure below, we see that reducing the packet size to 22624 bytes limits the maximum frame rate to 55 FPS,
assuming image size and pixel format stay the same.

1/31/2017
©2012-2017 FLIR
Integrated Imaging Setting up Multiple USB3 Cameras 3
Solutions Inc.
All rights reserved.
TAN2012005—Setting up Multiple USB3 Cameras

Calculating Required Bandwidth

To calculate your bandwidth requirements, use your required resolution, frame rate, and pixel format (bytes per
pixel) in the following equation.

Height x Width x Frame Rate x Bytes per Pixel = Bandwidth in MB

For example:

Camera model: FL3-U3-13S2M-CS

Resolution: 1328 x 1048

Frame rate: 60 FPS

Pixel format: Mono16

Bandwidth = 1328 x 1048 x 60 x 2 = 167 MB

Pixel format Mono8/Raw8 Mono12/Raw12 Mono16/Raw16 RGB8 YUV411 YUV422 YUV444

Bytes per pixel 1 1.5 2 3 1.5 2 3

1/31/2017
©2012-2017 FLIR
Integrated Imaging Setting up Multiple USB3 Cameras 4
Solutions Inc.
All rights reserved.
TAN2012005—Setting up Multiple USB3 Cameras

Multiple Camera System Configurations

This section documents various configurations and scenarios that have been tested with USB 3.1 cameras. The
hardware and software setup with each configuration is also documented.

Multiple Cameras Connected Directly to a USB 3.1 Host Controller

The figure below illustrates 4 cameras running on a 4 port USB3 card with Fresco FL1100 host controller.

Test System
Component Description
CPU Intel Core i-7 4790
Motherboard ASUS Z97-PRO
RAM 8 GB
Host adapter FLIR 4 Port USB 3.1 PCI Host Adapter (ACC-01-1202)
Cabling FLIR 3-meter USB 3.1 Cable Type-A to Micro-B
Operating system Windows 7 x64
Driver PGRUSBCam Driver 2.7.3.18
Software Test application based on FlyCapture2 Release 2.7.3.13 x64
Camera Grasshopper®3 GS3-U3-23S6M
Camera firmware 2.11.3.0
We used a console test application to obtain the test result below. The test application configures each camera to
run at high performance mode and verifies that there are no dropped images.

Test Results
Frame Packet Size  Bandwidth
Camera Model Pixel Format Resolution
Rate (Bytes) (MB/s)
GS3-U3-23S6M-C Raw8 1920 x 1200 40 33120 92
GS3-U3-23S6M-C Raw8 1920 x 1200 40 33120 92
GS3-U3-23S6C-C Raw8 1920 x 1200 40 33120 92
GS3-U3-23S6C-C Raw8 1920 x 1200 40 33120 92
Total bandwidth 368 MB/s

1/31/2017
©2012-2017 FLIR
Integrated Imaging Setting up Multiple USB3 Cameras 5
Solutions Inc.
All rights reserved.
TAN2012005—Setting up Multiple USB3 Cameras

Using the configuration above, a total of 368 MB/s is transferred to the host from multiple cameras.  Although the
effective USB 3.1 bandwidth is approximately 450 MB/s, peak performance can vary, depending on how the
USB3 host adapter card manages bandwidth.

Multiple Cameras Connected via a USB 3.1 Hub

This configuration demonstrates free-running 2 and 3 cameras using a single USB 3.1 hub. A table is provided to
demonstrate the frame rates achieved in each configuration. As shown from the results, the USB 3.1 hub used
could not sustain a data rate of greater than 345 MB/s.

Test System
Component Description
CPU Intel Core i-7 4770
Motherboard ASUS Z87-EXPERT
RAM 8 GB
Host Adaptor Onboard Intel USB 3.1 port
Hub 4-Port USB 3.1 Hub with Screw Locks and External Power Adapter (ACC-01-6000)
Cabling FLIR 3-meter USB 3.1 Cable Type-A to Micro-B (ACC-01-2300)
Operating system Windows 7 x64
Driver PGRUSBCam Driver 2.7.3.18
Software Test application based on FlyCapture2 Release 2.7.3.13 x64
Camera Grasshopper®3 GS3-U3-23S6M
Camera firmware 2.11.3.0
Note: For further recommendations, please see TAN 2011005 Recommended USB 3.1 System Components.

Test Results—One camera connected to a hub

Frame Packet Size  Bandwidth
Camera Model Pixel Format Resolution
Rate (Bytes) (MB/s)
GS3-U3-23S6M-C Raw8 1920 x 1200 162 48096 373
Total bandwidth 373 MB/s

1/31/2017
©2012-2017 FLIR
Integrated Imaging Setting up Multiple USB3 Cameras 6
Solutions Inc.
All rights reserved.
TAN2012005—Setting up Multiple USB3 Cameras

Test Results—Two cameras connected to a hub

Frame Packet Size  Bandwidth
Camera Model Pixel Format Resolution
Rate (Bytes) (MB/s)
GS3-U3-23S6M-C Raw8 1920 x 1200 70 48096 161
GS3-U3-23S6M-C Raw8 1920 x 1200 70 48096 161
Total bandwidth 322 MB/s

Test Results—Three cameras connected to a hub

Frame Packet Size  Bandwidth
Camera Model Pixel Format Resolution
Rate (Bytes) (MB/s)
GS3-U3-23S6M-C Raw8 1920 x 1200 50 48096 115
GS3-U3-23S6M-C Raw8 1920 x 1200 50 48096 115
GS3-U3-23S6M-C Raw8 1920 x 1200 50 48096 115
Total bandwidth 345 MB/s

Using the configurations above, the total throughput ranges from 322 MB/s to 373 MB/s.  Although the effective
USB 3.1 bandwidth is approximately 450 MB/s, peak performance can vary depending on how the USB 3.1 host
controller manages bandwidth.

1/31/2017
©2012-2017 FLIR
Integrated Imaging Setting up Multiple USB3 Cameras 7
Solutions Inc.
All rights reserved.
TAN2012005—Setting up Multiple USB3 Cameras

Multiple Cameras Connected to Multiple Host Controllers

The following configuration uses multiple host controllers to share the bandwidth for 15 cameras.

Test System
Component Description
CPU Intel i7 4790 @ 3.8 GHz
ASUS Z97 Pro
Motherboard
Z97-PRO BIOS 2205
RAM 8 GB 1333 MHz non-ECC memory
Hard Drive 500 GB Segate - 7200 RPM
Power Supply 500 W Thermtake PSU
2 x onboard Intel 9 series USB3 ports
Host Adaptor 1 x onboard ASMedia USB3 port
3 x quad channel quad port Fresco FL1100 USB 3.0 card
Intel USB3.0 eXtensible Host Controller version 4.0.0.36
Host Adaptor
ASMedia XHCI 1.0 Controller version 1.16.23.0
Driver
Fresco USB3 Extensible Host Controller version 2.7.3.79
Cabling 15 x FLIR 3-meter USB 3.1 Cable Type-A to Micro-B (ACC-01-2300)
Operating system Windows 7 x64
Driver PGRUSBCam Driver 2.7.3.18
Software Test application based on FlyCapture2 Release 2.8.3.1 x64
Camera 15 x Grasshopper®3 USB3
Camera firmware See test results

Test Results—15 cameras connected to multple host controllers

Frame
Camera Model Pixel Format Resolution Firmware Bandwidth (MB/s)
Rate
GS3-U3-120S6M Mono8 640 x 480 30 1.17.2.0 9
GS3-U3-120S6M Mono8 1288 x 968 17 2.22.3.0 21
GS3-U3-120S6M Mono8 1288 x 968 21 2.22.3.0 64
GS3-U3-120S6M Mono8 1280 x 968 25 2.22.3.0 17
GS3-U3-91S6C Raw8 1288 x 968 17 2.22.3.0 21
GS3-U3-91S6C Raw8 1288 x 968 17 2.22.3.0 21
GS3-U3-91S6C Raw8 1288 x 968 17 2.22.3.0 21
GS3-U3-91S6C Raw8 1384 x 1032 45 2.22.3.0 64
GS3-U3-60QS6C Raw8 1288 x 968 27 2.22.3.0 34

1/31/2017
©2012-2017 FLIR
Integrated Imaging Setting up Multiple USB3 Cameras 8
Solutions Inc.
All rights reserved.
TAN2012005—Setting up Multiple USB3 Cameras

Frame
Camera Model Pixel Format Resolution Firmware Bandwidth (MB/s)
Rate
GS3-U3-41C6M Mono8 640 x 480 188 2.22.3.0 233
GS3-U3-41C6C Raw8 1280 x 968 188 2.22.3.0 233
GS3-U3-41C6M Mono8 1280 x 968 188 2.22.3.0 233
GS3-U3-41S4M Mono8 1288 x 968 29 2.22.3.0 36
GS3-U3-41S4M Mono8 1288 x 968 29 2.7.3.0 36
GS3-U3-15S5C Raw8 1384 x1032 45 2.22.3.0 64
Total bandwidth 1137 MB/s*
*Bandwidth shared across multiple host controllers

Troubleshooting
Enumeration
n When using a USB 3.1 camera, ensure it is adequately powered by an external adapter.

n Ensure the latest firmware and drivers are being used.

Link Recovery Errors

n The host controller enters a recovery state to recover from bit errors that it has detected. This is often
caused by a long cable or a bad connection.

n Use the same cables to ensure the performance of each camera is consistent. Shorter cables are also
preferred for better reliability.

n For more information, see TAN2011006 Diagnosing USB 3.1 Camera and Bus Errors.

Skipped Image Frames

n This happens when the host is too slow in receiving and acknowledging the data coming from the
camera. It can be caused by a host system that is too busy or by having too much data on the USB 3.1
bus for the host to handle.

n To view the number of skipped frames:

o In SpinView, see the Log Viewer.

o In FlyCapture2, see the Camera Information panel.

n To alleviate the issue, manually reduce the Format 7 packet size until the max frame rate is reduced and
images are no longer being skipped.

n Use a higher performance PC to ensure fewer frames are skipped.

1/31/2017
©2012-2017 FLIR
Integrated Imaging Setting up Multiple USB3 Cameras 9
Solutions Inc.
All rights reserved.
TAN2012005—Setting up Multiple USB3 Cameras

Low Frame Rate

n To increase the frame rate:

o In SpinView, increase the Device Link Throughput Limit.

o In FlyCapture 2, increase the Format 7 packet size.

n Ensure the PCI Express bus is 2.0 compliant. If a PCI Express 1.0 bus is used, the effective bandwidth
(and hence frame rate) is reduced by half.

n Ensure images frames are not being skipped. Please refer to the Skipped Image Frames section above for
more information.

n For more information, see KB 10108 My USB 3.1 camera does not achieve full frame rate.

Downloads and Support

FLIR endeavors to provide the highest level of technical support possible to our customers. Most support
resources can be accessed through the Support section of our website.

The first step in accessing our technical support resources is to obtain a Customer Login Account. This requires a
valid name and email address. To apply for a Customer Login Account go to our Downloads page.

Customers with a Customer Login Account can access the latest software and firmware for their cameras from
our website. We encourage our customers to keep their software and firmware up-to-date by downloading and
installing the latest versions.

Finding Information
FlyCapture SDK—The FlyCapture SDK provides API examples and the FlyCap camera evaluation application.
Available from our Downloads page.

API Documentation—The installation of the FlyCapture SDK comes with API references for C++, C#, and C
code. Available from Start Menu→All Programs→Point Grey FlyCapture2 SDK→Documentation

Product Documentation—The camera's Getting Started Manual provides information on installing components
and software needed to run the camera. The Technical Reference provides information on the camera’s
specifications, features and operations, as well as imaging and acquisition controls. They are available from the
Downloads page.

Knowledge Base—A database of articles and application notes with answers to common questions as well as
articles and tutorials about hardware and software systems. Available from our Knowledge Base.

Learning Center—Our Learning Center contains links to many resources including videos, case studies, popular
topics, other application notes, and information on sensor technology.

1/31/2017
©2012-2017 FLIR
Integrated Imaging Setting up Multiple USB3 Cameras 10
Solutions Inc.
All rights reserved.
TAN2012005—Setting up Multiple USB3 Cameras

Contacting Technical Support

Before contacting Technical Support, have you:

1. Read the product documentation?

2. Searched the Knowledge Base?

3. Downloaded and installed the latest version of software and/or firmware?

If you have done all the above and still can’t find an answer to your question, contact our Technical Support team.

1/31/2017
©2012-2017 FLIR
Integrated Imaging Setting up Multiple USB3 Cameras 11
Solutions Inc.
All rights reserved.