EyesWeb 5.0.3 Gesture Processing User Manual

EyesWeb XMI 5.0.3.
0 – Expressive Gesture processing

Library
July 30, 2009

Part I
Description
1
The EyesWeb Expressive Gesture Processing Library includes a collection of software
modules and patches (interconnections of modules) contained ....
2
Part II
Reference
3
Chapter 1
Gesture Processing Catalog
1.1 Blocks
1.1.1 Add point
bitmap
class name Add point

catalog name Gesture Processing
catalog id EywGP
class id AddPoint
authors Gualtiero Volpe
This block appends the input point to the input trajectory.
Inputs
Input trajectory
id InputTrajectory
type Kernel, Generic datatype
type id kernel, generic datatype
required for initialization
required
required for execution
read only/read write read write
referred as inplace Output trajectory
referred as inherited *no*
The trajectory to which the input point has to be added.
4
Input point
id InputPoint
required
read only/read write read only
referred as inplace *no*
The point to be added to the input trajectory.
Outputs
Output trajectory
id OutputTrajectory
inplace id Input trajectory
inherited id *no*
The output trajectory including the added point.
5
1.1.2 Blob ellipse
bitmap
class name Blob ellipse

catalog id EywGP
class id BlobEllipse
This block generates an elliptical approximation of a blob. From such ellipse it is possible
to obtain an estimate of parameters such as blob orientation and contraction/expansion.
Inputs
Input image
id InputImage
type Base, Blob 2D
type id base, blob2d
required
The input image containing the blob to be approximated by an ellipse. It must be a
black & white image.
Outputs
Output ellipse
id OutputEllipse
type Base, Ellipse 2D double
type id base, ellipse 2d double
inplace id *no*
inherited id *no*
The ellipse approximating the input block.
Output ellipse
id OutputCenter
type Base, Graphic Point 2D double
type id base, graphic point 2d double
inplace id *no*
inherited id *no*
6
The ellipse approximating the input block.
7
1.1.3 Contraction Index
bitmap
class name Contraction Index

catalog id EywGP
class id ContractionIndex
This block computes the Contraction Index (CI) of a blob.
Inputs
Input blob
id InputBlob
type Base, Blob 2D
required
The input blob whose Contraction Index has to be computed.
Outputs
Contraction Index
id OutputIndex
type Kernel, Double datatype (Kernel Catalog).
type id kernel, double
inplace id *no*
inherited id *no*
The computed Contraction Index.
Parameters
Contraction Index
id ComputeBlobCI
type Kernel, Bool datatype (Kernel Catalog).
type id kernel, bool
Contraction Index
8
MinHeightRect
id ComputeMinHeightRectRatio
MinWidthRect
id ComputeMinWidthRectRatio
Compute Eccentricity
id ComputeEllipseEccentricity
Reset
id Reset
type Kernel, Trigger datatype (Kernel Catalog).
type id kernel, trigger
9
1.1.4 ConvexHull
bitmap
class name ConvexHull

catalog id EywGP
class id convexhullcomputation
Computes the convex hull of the input contour.
Inputs
Input contour
id ConvexHullInContour
type Gesture Processing, Integer 2D Graphic Contour
type id EywGP, IntGraphicContour
required
A contour. This block computes the convex hull of the input contour
Outputs
The convex hull
id ConvexHullOutConvexHull
type Base, Graphic Polygon 2D int
type id base, graphic polygon 2d int
inplace id *no*
inherited id *no*
This is the convex hull computed by the block
10
1.1.5 Empty trajectory
bitmap
class name Empty trajectory

catalog id EywGP
class id EmptyTrajectoryGenerator
This block generates an empty trajectory.
Outputs
Output trajectory
id OutputTrajectory
type Gesture Processing, Integer 2D Geometric Trajectory
type id EywGP, IntGeometricTrajectory2D
inplace id *no*
inherited id *no*
The generated trajectory. It will be an empty trajectory.
Parameters
Trajectory type
id TrajectoryType
type Kernel, Int datatype (Kernel Catalog).
type id kernel, int
Combo Box:
Geometric Integer 2D
Geometric Real 2D
layout Graphic Integer 2D
Graphic Real 2D
Geometric Real 3D
domain [ 0, 6 )
The type of trajectory to be generated. It may be 2D, 3D or nD, geometric or graphical,
interger or real.
Trajectory label
id Label
type Kernel, String datatype (Kernel Catalog).
type id kernel, string
11
The label of the generated 2D trajectory.
Sampling rate
id SamplingRate
The sampling rate in Hz at which the trajectory is generated and its points are sampled.
Is maximum of points fixed?

id IsSizeFixed
If set to true, the Initialization matrix parameter is added to the block and displayed.
The potential function will be initialized according to the values contained in the matrix
provided for the Initialization matrix parameter. Otherwise the potential function is ini-
tialized with zeros. The default value is false.
Maximum number of points

id MaxSize
type id kernel, int
domain [ 2, +infinity )
The maximum number of points in the generated trajectory. This parameter is available
only if the maximum number of points is fixed.
Reset
id Reset
Reset the generated trajectory by removing all its points.
Color
id Color
type Kernel, RGBColor datatype (Kernel Catalog).
type id kernel, rgbcolor
The color of the generated trajectory. The value of this parameter is taken into account
only if the generated trajectory is a graphic trajectory.
Thickness
id Thickness
The thickness of the generated trajectory. The value of this parameter is taken into
account only if the generated trajectory is a graphic trajectory.
12
1.1.6 Extract contours
bitmap
class name Extract contours

catalog id EywGP
class id ExtractContours
This block extracts the contours of the blobs detected in the input image. It returns
the (x,y) coordinates of all the points in the extracted contours.
Inputs
Input image
id InputBlob
type Base, Blob 2D
required
The input image containing the blobs whose contours have to be extracted.
Outputs
Extracted contours
id ExtractedContours
type Kernel, Labelled set
type id kernel, labeled set
inplace id *no*
inherited id *no*
The extracted contours of the blobs detected in the input image.
Parameters
Extract contours
id ExtractContours
Extract contours
13
Extract outer contour
id ExtractOuterContours
Extract outer contour
Contour type
id ContourType
type id kernel, int
Combo Box:
layout Geometric Real 2D
Graphic Integer 2D
Graphic Real 2D
domain [ 0, 4 )
Specifies the type of data used for the output contour
Minimum number of points

id MinNumOfPoints
type id kernel, int
The minimum number of points for a contour in order to be considered.
Approximation method
id ApproxMethod
type id kernel, int
Combo Box:
None
layout Simple
TC89 L1
TC89 KCOS
domain [ 0, 4 )
The approximation method to be used for extracting contours.
Color
id Color
14
Thickness
id Thickness
IsFilled
id IsFilled
Specifies whether the contour has to be filled
FillColor
id FillColor
Specifies the color to be used to fill the contour
15
1.1.7 GeneralisedAutoCorrelation
bitmap
class name GeneralisedAutoCorrelation

catalog id EywGP
class id generalised auto correlation
authors Paolo Coletta
Computes the Generalised Auto-Correlation function from a static time-series. For de-
tails about the Generalised Auto-Correlation function see ”Recurrence plots for the analysis
of complex systems” by Norbert Marwan, M. Carmen Romano, Marco Thiel, and Jürgen
Kurths; in Physics Reports 438 (2007), pages 237-329. This block implements equation
(50) pag. 265 or equivalently (111) pag. 289. The value of epsilon is user-specified, N is
the buffer-size of the input time-series. The user can specify both a specific value for tau
or a range. In the first case the output is a scalar number which gives the value of the
auto-correlation for the given tau and epsilon; in the second case the output is the function
RR(tau).
Inputs
Input signal
id input
required
Input signal on which to compute the generalised auto-correlation; the input signal
should implement the IStaticTimeSeries interface.
Required interfaces
Kernel, StaticTimeSeries
16
Outputs
RR function of Tau
id output function of tau
type Kernel, Double matrix.
type id kernel, double-matrix
inplace id *no*
inherited id *no*
This output represents the value of the function RR for each possible Tau in the user-
specified domain
Parameters
Epsilon
id epsilon
Specifies the upper bound of the distance of two near points. I.e., two point are consid-
ered to be one near the other one if their distance is belowe this threashold. The value of
R in the sum is 1 for near points.
Output type
id parameter output type
type id kernel, int
Combo Box:
layout Scalar
Function Of Tau
domain [ 0, 2 )
Specifies whether the output is a Scalar value, which represents the value of RR(epsilon,tau)
for for given values of epsilon and tau, or a Function, which represent the value of RR(epsilon,*)
for a given value of epsilon and all possible values of tau
Tau Min
id parameter tau min
type id kernel, int
This parameter represents the minimum value of tau for which the output function
RR(tau) is computed
Tau Spread
id parameter tau spread
type id kernel, int
17
This parameter represents the spread of tau, i.e., tau is asusmed to be in the range [Tau
Min, Tau Min + Tau Spread]. This determines the size of the output function of tau.
18
1.1.8 Geometric Trajectory from points
bitmap
class name Geometric Trajectory from points

catalog id EywGP
class id GeometricTrajectoryFromPoints
This block generates a trajectory conisting of the temporal sequence of the points pro-
vided as input.
Inputs
Input 2D point.
id InputPoint
type Base, Graphic Labelled Set 2D int
type id base, graphic labeled set 2d int
required
The input 2D point to be added to the generated trajectory.
Outputs
Output 2D trajectory
id OutputTrajectory
type Base, Graphic Labelled Set 2D int
type id base, graphic labeled set 2d int
inplace id *no*
inherited id *no*
19
Parameters
Trajectory type
id TrajectoryType
type id kernel, int
Combo Box:
Geometric Real 2D
Graphic Real 2D
Geometric Real 3D
domain [ 0, 6 )
interger or real.
Trajectory label
id Label
Sampling rate
id SamplingRate
The sampling rate in Hz at which the input points are sampled.

id IsSizeFixed
Set this parameter to true if the maximum number of points belonging to the trajectory
is fixed. When such number is reached the oldest points are removed from the trajectory.

id MaxSize
type id kernel, int
20
Reset
id Reset
Color
id Color
Thickness
id Thickness
21
1.1.9 Get kinematical features
bitmap
class name Get kinematical features

catalog id EywGP
class id GetKinematicalFeatures
This block computes kinematical features (e.g., velocity, acceleration) on the input tra-
jectory.
Inputs
Input trajectory
id InputTrajectory
required
The trajectory to which the input point has to be added.
Outputs
Output trajectory
id OutputFeature
inplace id *no*
inherited id *no*
The output trajectory including the added point.
22
Parameters
Feature
id FeatureToExtract
type id kernel, int
Combo Box:
Velocity
layout
Acceleration
Direction
domain [ 0, 3 )
The kinematical feature to be extracted.
Numeric derivative
id NumericDerivative
type id kernel, int
Combo Box:
layout Backward difference
Central difference
domain [ 0, 2 )
The numeric derivative algorithm to be applied.
23
1.1.10 Get point
bitmap
class name Get point

catalog id EywGP
class id GetPoint
This block retrieves the required point from the input trajectory. Note that the point
is not removed from the trajectory.
Inputs
Input trajectory
id InputTrajectory
required
The input trajectory from where the indicated point will be retrieved.
Outputs
Output point
id OutputPoint
inplace id *no*
inherited id *no*
The retrieved output point.
24
Parameters
Point to extract
id GetPointMode
type id kernel, int
Combo Box:
Get latest point
layout
Get oldest point
Get point with index
domain [ 0, 3 )
Indicate here which point you want to extract: the latest, the oldest, or a point to be
identified by its numerical index.
Index
id Index
type id kernel, int
The index of the point to be retrieved. It ranges from 0 to the current size of the
trajectory - 1. This parameter is enabled only if the ”Point to extract” parameter is set to
”Get point with index”.
25
1.1.11 Get properties
bitmap
class name Get properties

catalog id EywGP
class id GetTrajectoryProperties
This block retrives and returns the properties (e.g., label, sampling rate, number of
points) of the input trajectory.
Inputs
Input trajectory
id InputTrajectory
required
The input trajectory whose propeties have to be retrieved and returned.
Outputs
Number of points
id Size
type id kernel, int
inplace id *no*
inherited id *no*
The current number of points in the input trajectory.
Parameters
Return potential matrix
id OutputType
26
If set to true the Potential function output becomes available and the whole potential
matrix is returned by the block. The default value is true..
Return hit regions

id OutputLabel
If set to true, the Hit regions output becomes available and a vector is returned, where
each item is set to 0 if the corresponding region is missed or to 1 if the corresponding region
is hit. The default value is false.
Return indexes of hit regions

id OutputSamplingRate
If set to true the Hit regions indexes output becomes available and a list of indexes of
hit regions is returned by the block. The default value is false.
Return region values

id OutputIsSizeFixed
If set to true, the Region values output becomes available and a vector is returned,
where each item is set to the current potential value for the corresponding region. The
default value is false.

id OutputSize
27
1.1.12 Get trajectory features
bitmap
class name Get trajectory features

catalog id EywGP
class id GetTrajectoryFeatures
This block retrives and returns features (e.g., lenght, directness index) of the input
trajectory.
Inputs
Input trajectory
id InputTrajectory
required
The input trajectory whose features have to be computed.
Outputs
Output feature
id OutputFeature
inplace id *no*
inherited id *no*
The computed output feature.
28
Parameters
Feature to extract
id FeatureToExtract
type id kernel, int
Combo Box:
layout Length
Directness index
domain [ 0, 2 )
The feature to be extracted.
29
1.1.13 Hit Cell Detector
bitmap
class name Hit Cell Detector

catalog id EywGP
class id HitCellDetector
Given a model in which a 2D space is divided in cells, this block computes the cells
occupied (hit) by the current input points.
Inputs
Input positions
id InputPositions
type Kernel, List
type id kernel, list
required
The (x, y) coordinates of the input positions.
Outputs
Hit cells
id HitCells
type Kernel, List
inplace id *no*
inherited id *no*
The (i, j) indexes of the hit cells corresponding to the current input positions.
Parameters
Cells along x
id CellsX
type id kernel, int
The number of cells in which the input space has to be divided along the x dimension.
30
Cells along y
id CellsY
type id kernel, int
The number of cells in which the input space has to be divided along the y dimension.
Minimum along x
id XMin
The minimum input value along the x dimension.
Maximum along x
id XMax
The maximum input value along the x dimension.
Minimum along y
id YMin
The minimum input value along the y dimension.
Maximum along y
id YMax
The maximum input value along the y dimension.
Clip values
id ClipValues
If set, out of range values are clipped to the provided maximum and minimum values.
Otherwise a warning message is displayed and the hit cell is not computed.
Clear input list

id ClearList
If set, the input list is cleared after the computation of the hit cells.
31
Quantization algorithm
id QuantAlg
type id kernel, int
Combo Box:
layout
Linear uniform
domain [ 0, 1 )
The quantization algorithm to be employed for computing the hit cell.
32
1.1.14 Hit Region Detector
bitmap
class name Hit Region Detector

catalog id EywGP
class id HitRegionDetector
Given a model in which a 2D space is divided in cells, this block computes which regions
(groups of cells) are occupied (hit) by the silhouette of a blob provided as input. Regions
are defined through a mask provided as parameter. Cells correspond to pixels in the image
from which the blob silhouette is extracted.
Inputs
Input image
id InputImage
type Base, Image
type id base, image
required
An image containing the silhouette of a blob. The block will compute which regions in
the 2D space the input blob occupies (hits). The input image must be black and white,
single channel, and have 8-bit depth.
Outputs
Hit regions
id OutputList
type Kernel, List
inplace id *no*
inherited id *no*
A list of integer numbers. Such numbers are the indexes of regions the input blob
occupies (hits). The indexes are defined in the mask provided as parameter.
33
Parameters
Mask
id Mask
type Kernel, Integer matrix.
type id kernel, int-matrix
An integer matrix containing the indexes (identifiers) of the regions defined in the space.
Items labeled with the same index (integer value) belong to the same region. The range
of such indexes is [0, MaxNumOfRegions -1] where MaxNumOfRegions is the maximum
number of regions that can be defined. The block considers as maximum number of regions
the size of the Thresholds vector (see below). The mask matrix should have the same
dimensions of the image from which the silhouette of the input blob has been extracted.
The default dimensions are 288 rows and 352 columns; the default value is a matrix of
zeros.
Thresholds
id Thresholds
An integer vector containing the thresholds to be applied to the occupation values of
each region. Each item is the threshold for the corresponding region. A region is considered
to be hit if the number of its cells the input blob occupies exceeds this threshold. The block
considers the size of this vector as the maximum number of regions that can be defined.
The default dimensions are 1 row and 256 columns (corresponding to the definition of 256
cells). The default value is a vector of zeros.
Return occupation values

id OutOccValues
If set to true the block returns both the list of the indexes (identifiers) of the hit regions
and, separately, the list of the indexes and of the occupation values of the hit regions.
Otherwise the list of the indexes of the hit regions only is returned. The default value is
false.
34
1.1.15 InternalQuantityOfMotion
bitmap
class name InternalQuantityOfMotion

catalog id EywGP
class id quantity of motion
Calculate internal quantity of motion. The measuring reguards internal modifications of

the silhouette. The algortihm needs both image and blob input.ERROR AND WARNING
MESSAGE. - During initialization phase: - Matrix and image must be the same dimension:
if the dimension of the matrix of weight and the dimension of the image are different. -
Block Internal QOM : The mask or the anchor size are invalid: if the dimensions of median
filter are not odd. - During execution phase: - Ipp library info: ippiv81.lib 5.2 5.2.108.410-
Ipp Error description : Invalid mask size.
Inputs
input blob
id input blob
type Base, Blob 2D
required
This is the blob extracted from the input image. Actually the block works only with a
single blob.
input image
id input image
type Base, Image
type id base, image
required
35
Input image.
Outputs
output qom
id output qom
inplace id *no*
inherited id *no*
Quantity of internal motion. The output represents the number of nonzero pixels of
QOM image. If the parameter ’type’ is set to int, then the output is the number of pixels.
If it is set to double the output is normalized according to ’Normalization Type’ parameter
output image
id output image
type Base, Image
type id base, image
inplace id *no*
inherited id *no*
The image represents differents beetween the first and the last image of a dynamic
queue. The lenght of the queue is expressed by the parameter ’Window lenght’
Parameters
Weight
id output qom
type id kernel, int
Combo Box:
Euclidea
Euclidea Inversa
layout
From Matrix
From Matrix Inverse
None
domain [ 0, 5 )
The parameter represent the type of weight. Euclidean: the weight of pixels is the
distance from the baricenter. Inverse Euclidean: the weight of pixels is the inverse of the
distance from the baricenter. From Matrix: weights of pixels are stored in the matrix .
Inverse From Matrix: weight of pixels are the inverse of the matix. None: There is no
weight for each pixel.
36
WindowLenght
id window lenght
type id kernel, int
The size of a dynamic queue. The queue stores images and the algorithm process the
first and the last image in the buffer.
Threshold
id threshold
The threshold used in the binarization of image.
Reset
id reset
Reset the method and clear the queue of images. When reset button is pressed the
algorithmneeds to read ’Window lenght’ images.
MedianOnOff
id median
Active median filter.
MedianX
id median x
type id kernel, int
The width of median filter mask used in the method
MedianY
id median y
type id kernel, int
The height of median filter mask used in the method
37
Type
id output qom type
type id kernel, int
Combo Box:
layout Int
Double
domain [ 0, 2 )
Type of QOM. If it’s set to int value then the output is the number of non zero pixels in
the output image. If it’s set to double value the output is the normalization of the number
of non zero pixels according to the parameter Normalization Type.
Normalization Type
id norm qom
type id kernel, int
Combo Box:
AreaofImage
layout
AreaofBoundingRectangle
AreaofBlob
domain [ 0, 3 )
It specifies if the numeric output is normalized by the area of entire image, by the area
of the bounding rectangle or by the area of the blob.
38
1.1.16 Motion duration
bitmap
class name Motion duration

catalog id EywGP
class id MotionDuration
This block computes the time duration of pause and motion phases starting from a
segmentation signal provided as input.
Inputs
Input segmentation signal
id SegmentationSignal
required
The input segmentation signal. It should be 0 during pause phases and different from
0 during motion phases.
Outputs
Motion duration
id MotionDuration
inplace id *no*
inherited id *no*
The duration of the last motion phase.
Pause duration
id PauseDuration
inplace id *no*
inherited id *no*
39
The duration of the last pause phase.
Parameters
Measure unit
id MeasureUnit
type id kernel, int
Combo Box:
layout Frames
Milliseconds
domain [ 0, 2 )
The measure unit for the output durations. It can be frames or milliseconds.
Sampling rate
id SamplingRate
The sampling rate of the input segmentation signal. This parameter is enabled only if
the output measure unit is milliseconds.
Continuous output
id ContinuousOutput
If this parameter is set to false the output is returned only at the end of a motion or
pause phase. Otherwise, the last value is continuously returned.
40
1.1.17 PixelWeightQuantityOfMotion
bitmap
class name PixelWeightQuantityOfMotion

catalog id EywGP
class id pixel weight quantity of motion
Calculate quantity of motion. The block can weight pixels according to parameter
weight. — ERROR AND WARNING — - During Initilalization phase. - Matrix and image
must be the same dimension: if the dimension of the matrix of weight jand the dimension
of the image are different. - Block Internal QOM : The mask or the anchor size are invalid:
if the dimensions of median filter are not odd. - During execution phase: - Ipp library info:
ippiv81.lib 5.2 5.2.108.410- Ipp Error description : Invalid mask size.
Inputs
input blob
id input blob
type Base, Blob 2D
required
This is the blob extracted from the input image. Actually the block works only with a
single blob.
Outputs
output qom
id output qom
inplace id *no*
inherited id *no*
Value of qom
41
output image
id output image
type Base, Image
type id base, image
inplace id *no*
inherited id *no*
The image represents differents beetween the first and the last image of a dynamic
queue. The lenght of the queue is expressed by the parameter ’Window lenght’
Parameters
Weight
id output qom type
type id kernel, int
Combo Box:
Euclidean
Inverse Euclidean
layout
From Matrix
Inverse From Matrix
None
domain [ 0, 5 )
The parameter represent the type of weight. Euclidean: the weight of pixels is the
distance from the baricenter. Inverse Euclidean: the weight of pixels is the inverse of the
distance from the baricenter. From Matrix: weights of pixels are stored in the matrix .
Inverse From Matrix: weight of pixels are the inverse of the matix. None: There is no
weight for each pixel.
WindowLenght
id window lenght
type id kernel, int
The size of a dynamic queue. The queue stores images and the algorithm process the
first and the last image in the buffer.
Threshold
id threshold
The threshold used in the binarization of image.
Reset
id reset
42
Reset the method and clear the queue of images. When reset button is pressed the
algorithmneeds to read ’Window lenght’ images.
Normalization Type
id normalization;
type id kernel, int
Combo Box:
AreaofImage
layout
AreaofBoundingRectangle
AreaofBlob
domain [ 0, 3 )
It specifies if the numeric output is normalized by the area of entire image, by the area
of the bounding rectangle or by the area of the blob.
MedianOnOff
id median
Active median filter.
MedianX
id median x
type id kernel, int
The width of median filter mask used in the method.
MedianY
id median y
type id kernel, int
The height of median filter mask used in the method.
43
1.1.18 Position Depending Potentials
bitmap
class name Position Depending Potentials

catalog id EywGP
class id PositionDependingPotentials
Given a model in which a 2D space is divided in cells, this block computes a 2D potential
function depending on the indexes (h, k) of the cells received as input. The kind of potential
function can be provided as parameter.
Inputs
Input cells
id InputPositions
type Kernel, List
required
A list of input cells used for computing the potential function. This list contains a
collection of 1x2 or 2x1 vectors whose items are the (h, k) indexes of a cell in the space.
Usually, this input comes from an Hit Cell Detector and these are the cells occupied by a
collection of input points.
Outputs
id OutputPotential
inplace id *no*
inherited id *no*
44
Parameters
Cells along X
id CellsAlongX
type id kernel, int
The number of cells in which the input space has been divided along the x dimension. It
corresponds to the number of columns in the output potential matrix. If the block receives
its input from a Hit Cell Detector, this parameter should have the same value than the
corresponding parameter in the Hit Cell Detector. The default value is 10.
Cells along Y
id CellsAlongY
type id kernel, int
The number of cells in which the input space has been divided along the y dimension.
It corresponds to the number of rows in the output potential matrix. If the block receives
Potential type
id PotentialType
type id kernel, int
Combo Box:
Linear increment/decrement
Polynomial
layout
Occupation rates
Gaussian
Logarithmic
The kind of potential to be computed. The following options are available at the mo-
ment: linear increment /decrement, polynomial, occupation rates, Gaussian, logarithmic.
Use initialization matrix

id UseInitMatrix
If set to true, the Initialization matrix parameter is added as parameter to the block
and displayed. The potential function will be initialized according to the values contained
in the matrix provided for the Initialization matrix parameter. Otherwise the potential
function is initialized with zeros. The default value is false.
45
Reset
id Reset
Each time a reset command is triggered, the potential function is reinitialized: the new
initial values are those contained in the Initialization matrix or zeros, depending on the Use
initialization matrix parameter.
Increment factor
id IncrementFactor
The increment factor to be applied to the hit cell. For each cell in the input list this
increment factor is applied. The default value is 1.
Decrement factor
id DecrementFactor
The decrement factor to be applied to missed cells. For each cell which is not in the
input list this decrement factor is applied. The default value is 1.
46
1.1.19 Region Depending Potentials
bitmap
class name Region Depending Potentials

catalog id EywGP
class id RegionDependingPotentials
Given a model in which a 2D space is divided in cells, this block computes a 2D potential
function depending on regions defined in such a space and, in particular, on the regions
that are occupied by points or to which the cells received as input belong. The kind of
potential function can be provided as parameter.
Inputs
Input list
id InputPositions
type Kernel, List
required
A list of either input cells or indexes of hit (occupied) regions. In the former case, this
list contains a collection of 1x2 or 2x1 vectors whose items are the (h, k) indexes of a cell
in the space. In the latter, this is a list of integer numbers (region indexes). This input
usually comes from a Hit Cell Detector (and in such a case it is a list of input cells) or from
a Hit Region Detector (and in such a case it is list of region indexes). If the input is a list
of cells, hit regions indexes are automatically computed.
Outputs
Potential function
id Potential
inplace id *no*
inherited id *no*
A matrix representing the computed potential function. The number of items in this
matrix is equal to the number of cells the space is divided in. Therefore, the number of rows
47
is equal to the number of cells along y and the number of columns is equal to the number
of cells along x. The output is available only if the Return potential matrix parameter is
set to true.
Parameters
Cells along X
id CellsAlongX
type id kernel, int
The number of cells in which the input space has been divided along the x dimension. It
corresponds to the number of columns in the output potential matrix. If the block receives
Cells along Y
id CellsAlongY
type id kernel, int
The number of cells in which the input space has been divided along the y dimension.
It corresponds to the number of rows in the output potential matrix. If the block receives
Potential type
id PotentialType
type id kernel, int
Combo Box:
Linear increment/decrement
layout
Linear occupation rates
Exponential occupation rates
The kind of potential to be computed. The following options are available at the mo-
ment: linear increment /decrement, polynomial, occupation rates, Gaussian, logarithmic.
Use initialization matrix

id UseInitMatrix
If set to true, the Initialization matrix parameter is added as parameter to the block
and displayed. The potential function will be initialized according to the values contained
in the matrix provided for the Initialization matrix parameter. Otherwise the potential
function is initialized with zeros. The default value is false.
48
Reset
id Reset
Each time a reset command is triggered, the potential function is reinitialized: the new
initial values are those contained in the Initialization matrix or zeros, depending on the Use
initialization matrix parameter.
Regions mask
id Mask
An integer matrix containing the indexes (identifiers) of the regions defined in the space.
Items labeled with the same index (integer value) belong to the same region. The range
is [0, MaxNumOfRegions -1] where MaxNumOfRegions is the maximum number of regions
that can be defined. Such value is provided through the (Maximum) number of regions
parameter. The size of the mask matrix is automatically computed and it is the same
size of the matrix of the potential function, i.e., the number of rows is the number of cells
along y and the number of columns is the number of cells along y. In case the input comes
from a Hit Region Detector, this mask should be the same mask provided to the Hit region
detector and the number of cells and regions to be set accordingly. The default value is a
matrix of zeros.
(Maximum) Number of regions

id NumOfRegions
type id kernel, int
The maximum number of regions that can be defined. Regions will be labeled with
progressive integer numbers from 0 up to this value minus one. In case the input of the
block comes from a Hit Region Detector, this parameter should be set equal to the size of
Thresholds vector in the Hit Region Detector. The default value is 10.
Use initialization values for regions

id UseInitValForReg
If set to true, the Initialization values for regions parameter is added to the block and
displayed. The potential function will be initialized according to the values contained in the
vector provided for the Initialization values for regions parameter. Otherwise the potential
function is initialized with zeros. Since just one way of initializing the potential function is
allowed, if the Use initialization matrix parameter is set to true, this one is automatically
set to false. The default value is false.
49
Return potential matrix
id OutputPotentialMatrix
If set to true the Potential function output becomes available and the whole potential
matrix is returned by the block. The default value is true..
Return hit regions

id OutputHitRegions
If set to true, the Hit regions output becomes available and a vector is returned, where
each item is set to 0 if the corresponding region is missed or to 1 if the corresponding region
is hit. The default value is false.
Return indexes of hit regions

id OutputHitRegionIndexes
If set to true the Hit regions indexes output becomes available and a list of indexes of
hit regions is returned by the block. The default value is false.

id OutputRegionValues
Increment factor
id IncrementFactor
The increment factor to be applied to hit regions. In case of list of cells as input, for
each cell in the input list this increment factor is applied to the region it belongs to. The
default value is 1.
Decrement factor
id DecrementFactor
The decrement factor to be applied to missed regions. In case of list of cells as input,
for each cell which is not in the input list this decrement factor is applied to the region it
belongs to. The default value is 1.
50
1.1.20 Remove point
bitmap
class name Remove point

catalog id EywGP
class id RemovePoint
This block retrieves the oldest point from the input trajectory and removes it.
Inputs
Input trajectory
id InputTrajectory
required
referred as inplace Output point
The input trajectory from where the indicated point will be retrieved.
Outputs
Output point
id OutputTrajectory
inherited id *no*
Output point
id OutputPoint
inplace id *no*
inherited id *no*
51
1.1.21 Set properties
bitmap
class name Set properties

catalog id EywGP
class id SetTrajectoryProperties
This block modifies the properties (label, sampling rate, etc.) of the input trajectory.
Inputs
Input trajectory
id InputTrajectory
required
referred as inherited Output trajectory
The input trajectory whose properties have to be changed.
Outputs
Output trajectory
id OutputTrajectory2
inherited id *no*
The output trajectory with the modified properties.
Parameters
Modify label
id ModifyLabel
Set this parameter to true if you want to change the trajectory label.
52
Modify size
id ModifySize
Set this parameter to true if you want to change the trajectory sizel.
Trajectory label
id Label
The label of the trajectory.

id IsSizeFixed

id MaxSize
type id kernel, int
53
1.1.22 Trajectory from points
bitmap
class name Trajectory from points

catalog id EywGP
class id TrajectoryFromPoints
This block generates a trajectory conisting of the temporal sequence of the points pro-
vided as input.
Inputs
Input 2D point.
id InputPoint
type Base, Point 2D int
type id base, point 2d int
required
The input 2D point to be added to the generated trajectory.
Outputs
Output 2D trajectory
id OutputTrajectory
type Gesture Processing, Integer 2D Geometric Trajectory
type id EywGP, IntGeometricTrajectory2D
inplace id *no*
inherited id *no*
The generated 2D trajectory
54
Parameters
Trajectory type
id TrajectoryType
type id kernel, int
Combo Box:
Geometric Real 2D
Graphic Real 2D
Geometric Real 3D
domain [ 0, 6 )
interger or real.
Trajectory label
id Label
Sampling rate
id SamplingRate
The sampling rate in Hz at which the input points are sampled.

id IsSizeFixed

id MaxSize
type id kernel, int
55
Reset
id Reset
Color
id Color
Thickness
id Thickness
56
1.2 Datatypes
1.2.1 Double 2D Geometric Contour
class name Double 2D Geometric Contour
catalog id EywGP
class id DoubleGeometricContour
This datatype stores 2D contours and provides information about them.
Supported interfaces
• kernel, trasformation 2d
• EywGP, ContourBase
• EywGP, Contour2DBase
• EywGP, GeometricContour2DBase
• EywGP, ContourAttributes
1.2.2 Double 2D Geometric Trajectory

class name Double 2D Geometric Trajectory
catalog id EywGP
class id DoubleGeometricTrajectory2D
This datatype stores 2D trajectories and provides information about them (e.g., kine-
matical features, expressive features).
• Base, Transformations2D
• EywGP, TrajectoryBase
• EywGP, Trajectory2DBase
• EywGP, GeometricTrajectory2DBase
• EywGP, TrajectoryAttributes
• EywGP, TrajectoryFeatures
57
1.2.3 Double 2D Graphic Contour
class name Double 2D Graphic Contour
catalog id EywGP
class id DoubleGraphicContour
• Base, Drawing
• EywGP, GraphicContour2DBase
1.2.4 Double 2D Graphic Trajectory

class name Double 2D Graphic Trajectory
catalog id EywGP
class id DoubleGraphicTrajectory2D
• Base, Drawing
• EywGP, GraphicTrajectory2DBase
58
1.2.5 Double 3D Geometric Trajectory
class name Double 3D Geometric Trajectory
catalog id EywGP
class id DoubleGeometricTrajectory3D
• kernel, trasformation 3D
1.2.6 Integer 2D Geometric Contour

class name Integer 2D Geometric Contour
catalog id EywGP
class id IntGeometricContour
• EywGP, GeometricContour2DBase
59
1.2.7 Integer 2D Geometric Trajectory
class name Integer 2D Geometric Trajectory
catalog id EywGP
class id IntGeometricTrajectory2D
1.2.8 Integer 2D Graphic Contour

class name Integer 2D Graphic Contour
catalog id EywGP
class id IntGraphicContour
• Base, Drawing
• EywGP, GraphicContour2DBase
60
1.2.9 Integer 2D Graphic Trajectory
class name Integer 2D Graphic Trajectory
catalog id EywGP
class id IntGraphicTrajectory2D
• Base, Drawing
• EywGP, GraphicTrajectory2DBase
1.2.10 Integer 3D Geometric Trajectory

class name Integer 3D Geometric Trajectory
catalog id EywGP
class id IntGeometricTrajectory3D
61
1.3 Datatype converters
1.3.1 Geometric To Graphic Trajectory Converter
class name Geometric To Graphic Trajectory Converter
catalog id EywGP
class id IntGeometricToGraphicTrajectory2DConverter
This converter converts a geometric trajectory to a graphic trajectory, i.e., a trajectory

that can be displayed.By default, the output graphic trajectory will be black and will have
a thickness of one pixel.
1.3.2 Geometric To Graphic Trajectory Converter

class name Geometric To Graphic Trajectory Converter
catalog id EywGP
class id DoubleGeometricToGraphicTrajectory2DConverter
This converter converts a geometric trajectory to a graphic trajectory, i.e., a trajectory

that can be displayed.By default, the output graphic trajectory will be black and will have
a thickness of one pixel.
1.3.3 Graphic To Geometric Trajectory Converter

class name Graphic To Geometric Trajectory Converter
catalog id EywGP
class id IntGraphicToGeometricTrajectory2DConverter
This converter converts a graphic trajectory to a geometric trajectory i.e., it removes

graphic information such as color and thickness.
1.3.4 Graphic To Geometric Trajectory Converter

class name Graphic To Geometric Trajectory Converter
catalog id EywGP
class id DoubleGraphicToGeometricTrajectory2DConverter
This converter converts a graphic trajectory to a geometric trajectory i.e., it removes

graphic information such as color and thickness.
62
1.3.5 Trajectory To Dynamic Temporal Data Converter
class name Trajectory To Dynamic Temporal Data Converter
catalog id EywGP
class id DoubleGeomTraj2DToDynTempDataConverter
This converter converts a trajectory to a time series having as many channels as the
number of componenets of the trajectory points and as many samples as the number of
points of the trajectory.

catalog id EywGP
class id DoubleGraphicTraj2DToDynTempDataConverter

catalog id EywGP
class id DoubleGeomTraj3DToDynTempDataConverter
1.4 Authors
1.4.1 Gualtiero Volpe
class name Gualtiero Volpe
catalog id EywGP
class id gualtiero
Software Engineer mailto:Gualtiero.Volpe@unige.it
63
1.4.2 Paolo Coletta
class name Paolo Coletta
catalog id EywGP
class id paolo
paolo.coletta@gmail.com
1.5 Companies
1.5.1 InfoMus Lab
class name InfoMus Lab
catalog id EywGP
class id InfoMus
Laboratory of Musical Informatics http://www.infomus.dist.unige.it news://infomus.dist.unige.it

mailto:info@infomus.dist.unige.it
64
Chapter 2
Machine Learning Catalog
2.1 Blocks
2.1.1 LibSVM Save
bitmap
class name LibSVM Save

catalog name Machine Learning
catalog id EywML
class id LibSVMSave
This block allows saving a matrix in the format required by the libsvm tools)
Inputs
Input Matrix
id InputMatrix
required
The input matrix to be saved in LibSVM format
65
Parameters
File Name
id FileName
Filename,
MustExist=true,
layout SaveMode=true,
OverwritePrompt=true,
Filter=”Text files (*.txt)—*.txt—All files (*.*)—*.*——”
Name of the output file that will be formatted in LibSVM style
Save Mode
id SaveMode
type id kernel, int
Combo Box:
layout Append
Overwrite
domain [ 0, 1 ]
Specify whether new data should overwrite the older ones or if the new data are ap-
pended after the older ones
66
2.1.2 SVM Predict
bitmap
class name SVM Predict

catalog id EywML
class id SVM
This block provides support for performing prediction using Support Vector Machines
(SVMs)
Inputs
Input Matrix
id InputMatrix
required
Input data to be analysed by the SVM
Outputs
Output Vector
id OutputVector
inplace id *no*
inherited id *no*
Output from the SVM model
67
Parameters
Model File Name
id ModelFileName
Filename,
MustExist=true,
layout SaveMode=false,
Name of the file including the SVM model
Refresh model
id RefreshModel
Refresh the SVM model
Show probabilities
id Probabilities
If set, an additional output is generated, providing probabilties values for classes. Note
that the model should be suitably trained in order to provide probabilities values.
68
2.1.3 SVM Train
bitmap
class name SVM Train

catalog id EywML
class id SVMTrain
This block provides support for training a Support Vector Machine (SVM)
Inputs
Input Matrix
id InputMatrix
required
The input training set
Outputs
Trained
id TrainedFlag
type id kernel, int
inplace id *no*
inherited id *no*
The block returns a non-zero value if an SVM model has been successfully generated.
Ranges
id Ranges
inplace id *no*
inherited id *no*
Ranges of the training set attributes
69
Parameters
SVM Type
id SVMType
type id kernel, int
Combo Box:
C-SVC
Nu-SVC
layout
One-class SVM
Epsilon-SVR
Nu-SVR
domain [ 0, 4 ]
Specify the type of SVM
Kernel Type
id KernelType
type id kernel, int
Combo Box:
Linear
layout Polynomial
Radial Basis Function
Sigmoid
domain [ 0, 3 ]
Specify the type of the SVM kernel
Degree
id Degree
Degree of the kernel polynome (for polynomial SVMs only)
Gamma
id Gamma
Gamma value for the SVM model
Offset
id Coeff0
Offset value for the SVM model
70
Epsilon
id Epsilon
Stopping criterium
Cache Size
id CacheSize
Cache size in MB
C
id C
C value for the SVM model
Nu
id Nu
Nu value for the SVM model
p
id p
p value for the SVM model
Shrinking
id Shrinking
Use shrinking heuristics
Probability
id Probability
Do probability estimates
71
Model File Name
id ModelFileName
Filename,
MustExist=true,
Name of the file containing the model
Train Model
id TrainModel
Trains the Support Vector Machine
Training Mode
id TrainingMode
type id kernel, int
Combo Box:
layout Incremental
One shot
domain [ 0, 1 ]
Select whether to incremenatally update the model or to perform a one-shot training.
Enable Maximum Size

id EnableMaxSize
If set to true, check for a maximum size of the training set. In case such maximum size
is reached, training is forced.
Maximum size of training set

id MaxSize
type id kernel, int
The maximum allowed size for the training set.
Enable Logging
id EnableLog
If set to true, enable writing of a log file containing information on the training
72
Log File Name
id LogFileName
Filename,
MustExist=true,
Name of the log file
Scale Values
id ScaleValues
If set to true, the block will scale the values in the training set
Lower Bound
id LowerBound
Lower bound of the rescaled training set
Upper Bound
id UpperBound
Upper bound of the rescaled training set
73
2.1.4 Stabilizer
bitmap
class name Stabilizer

catalog id EywML
class id Stabilizer
This block stabilizes the output of a machine learning classifier.
Inputs
Input value
id InputValue
required
Input value to the stabilizer
Outputs
Recognition
id Recognized
type id kernel, int
inplace id *no*
inherited id *no*
True if the current value is recognized
Output value
id OutputValue
inplace id *no*
inherited id *no*
74
Output value from the stabilizer
Parameters
Number of values
id NValues
type id kernel, int
Number of previous values the stabilizer takes into account
Threshold
id Threshold
Threshold for the stabilizer.
Not recognized value

id NotRecValue
Value returned as output when the input is not recognized
75
2.1.5 Vector Rescaler
bitmap
class name Vector Rescaler

catalog id EywML
class id VectorRescaler
This block rescales the input vector according to the given ranges.
Inputs
Input Vector
id InputVector
required
Input vector
Input Ranges
id InputRanges
required
Ranges of each item in the input vector
76
Outputs
Output Vector
id OutputVector
inplace id *no*
inherited id *no*
Rescaled vector
Parameters
Lower bound
id OutputLowBound
Lower bound of the rescaled vector
Upper bound
id OutputHighBound
Upper bound of the rescaled vector
77
2.2 Datatypes
2.3 Authors
2.3.1 Gualtiero Volpe
class name Gualtiero Volpe
catalog id EywML
class id gualtiero
Software Engineer mailto:Gualtiero.Volpe@unige.it
2.4 Companies
2.4.1 InfoMus Lab
class name InfoMus Lab
catalog id EywML
class id InfoMus
InfoMus Lab Laboratory of Musical Informatics http://infomus.dist.unige.it news://infomus.dist.unige

mailto:staff@infomus.dist.unige.it
78
Part III
Appendices
79
Appendix A
Release notes
A.1 Notes on EyesWeb XMI Expressive Gesture pro-

cessing Library 5.0.3.0
Released on Xxxx XX, 2009.
• TODO Finally released
• Added block GeneralisedAutoCorrelation
80
Appendix B
License
Use of the EyesWeb Gesture Processing Library (hereinafter ’SOFTWARE’) is contingent

on your agreement to the following terms:
WARRANTY & USE: DIST - University of Genoa grants you a limited, non-exclusive
license to use the SOFTWARE free of charge only for educational purposes and research
at universities and government research laboratories. Companies and private research lab-
oratories are required to obtain a separate license. For any other (not educational) use,
commercial or private, it is also required to obtain a license. DIST - University of Genoa
makes no representations about the suitability of this software for any purpose. It is pro-
vided ”as is” without express or implied warranty. DIST - University of Genoa is not
obligated to provide maintenance or updates for the SOFTWARE.
DISTRIBUTION: universities and government research laboratories may freely dis-
tribute the SOFTWARE, in original version, only to other universities and government
research laboratories provided that this copyright and permission notice appear on all copies
and supporting documentation, and that the DIST - University of Genoa copyright notices
are referred in the following ways:
• DIST - University of Genoa’s copyright notice should be included in the documenta-

tion, regardless of the media used to supply the documentation;
• The DIST - University of Genoa and EyesWeb Logos have to appear on packages and
promotional material (DIST - University of Genoa makes the logos available on the
EyesWeb’s ftp site ftp://ftp.infomus.org;
• In the ’about box’ of the product, in the case that it is not the EyesWeb about box,
DIST - University of Genoa and EyesWeb must be cited in the following manner:
”EyesWeb is copyright (c) Laboratorio di Infomatica Musicale - DIST - University
of Genoa (www.infomus.dist.unige.it)”, and the EyesWeb Gesture Processing Library
must be cited in the following manner: ”The EyesWeb Gesture Processing Library
is copyright (c) Laboratorio di Infomatica Musicale - DIST - University of Genoa
(www.infomus.dist.unige.it)”.
• Any public event or product/applications must be communicated in advance to info@infomus.dist.un
Distribution to companies, private research laboratories, or any other kind of subject

must be preliminarly agreed with DIST.
81

EyesWeb 5.0.3 Gesture Processing User Manual

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

EyesWeb 5.0.3 Gesture Processing User Manual

Hochgeladen von

Copyright:

Verfügbare Formate

EyesWeb XMI 5.0.3.

0 – Expressive Gesture processing

July 30, 2009

Gesture Processing Catalog

class name Add point

This block appends the input point to the input trajectory.

class name Blob ellipse

class name Contraction Index

This block computes the Contraction Index (CI) of a blob.

class name ConvexHull

Computes the convex hull of the input contour.

class name Empty trajectory

This block generates an empty trajectory.

Is maximum of points fixed?

Maximum number of points

class name Extract contours

Minimum number of points

class name GeneralisedAutoCorrelation

class name Geometric Trajectory from points

Is maximum of points fixed?

Maximum number of points

class name Get kinematical features

class name Get point

class name Get properties

Return hit regions

Return indexes of hit regions

Return region values

Return region values

class name Get trajectory features

class name Hit Cell Detector

Clear input list

class name Hit Region Detector

Return occupation values

class name InternalQuantityOfMotion

Calculate internal quantity of motion. The measuring reguards internal modifications of

class name Motion duration

class name PixelWeightQuantityOfMotion

class name Position Depending Potentials

Use initialization matrix

class name Region Depending Potentials

Use initialization matrix

(Maximum) Number of regions

Use initialization values for regions

Return hit regions

Return indexes of hit regions

Return region values

class name Remove point

class name Set properties

Is maximum of points fixed?

Maximum number of points

class name Trajectory from points

Is maximum of points fixed?

Maximum number of points

This datatype stores 2D contours and provides information about them.

1.2.2 Double 2D Geometric Trajectory

This datatype stores 2D contours and provides information about them.

1.2.4 Double 2D Graphic Trajectory

1.2.6 Integer 2D Geometric Contour

This datatype stores 2D contours and provides information about them.

1.2.8 Integer 2D Graphic Contour

This datatype stores 2D contours and provides information about them.

1.2.10 Integer 3D Geometric Trajectory

This converter converts a geometric trajectory to a graphic trajectory, i.e., a trajectory

1.3.2 Geometric To Graphic Trajectory Converter