An Approach of Automatic Motion Transverse Extension

IPASJ International Journal of Information Technology (IIJIT)
Web Site: http://www.ipasj.org/IIJIT/IIJIT.htm

A Publisher for Research Motivation ........ Email:editoriijit@ipasj.org
Volume 7, Issue 1, January 2019 ISSN 2321-5976
An Approach of Automatic Motion Transverse

Extension
QU Shi1, CAI Yi-chao2, LIU Tai-yang3 and LIU Jian-xun4
1
Department of Space and Air Early Warning, Air Force Early Warning Academy, Wuhan, 430019, China
2
3
4
ABSTRACT
We present a concept and technique of human motion reuse based on motion extension, which contains lengthways extension
and transverse extension. Motion extension can generate more abundant new motion from finite motion data, which provides
uniform solution for motion forecast, motion repair and group motion. Lengthways extension extends human motion along
time axis and generates new motion containing more frames, and transverse extension extends human motion along
characters, which generates multiple character group motion from single character motion. Present an approach of automatic
motion transverse extension, which can generate more characters motion from single character motion. Experiments show that
motion transverse extension can advance the reuse of sample motion.
Keywords: virtual human, motion, transverse extension, reuse
1. INTRODUCTION
Recently, virtual human motion synthesis has been applied widely to movie, advert, game, military affairs and so on.
Because of the complexity of the human skeleton, generating realistic human motion in the computer is very difficult.
The key problem of virtual human motion synthesis is how to generate vivid and reasonable posture. There are two
sorts of method to solve this problem, which is physical simulation and data driven. Generally, physical simulation
simulates human motion based on kinematics and dynamics. But, kinematics and dynamics have inborn limitations, for
example, the reality of kinematics is lacking because of its unsubstantial motion physics modeling, and the efficiency of
dynamics is low because of its complicated model. These limitations restrict the development and application of
physical simulation.
With the development of motion capture technique, the motion synthesis method based on data driven has obtained
unprecedented development. With respect to physical simulation, data driven is a fire-new motion synthesis mode,
which is the essential change of methodology and thought mode. Data driven is a more direct method, which directly
drives human skeleton to implement corresponding motion with motion capture data. Thus, data driven can avoid
cockamamie computation to confirm virtual human joint angles and has many strongpoints, such as easily to
implement, high computation efficiency, strong fidelity, and so on. But, with the application of virtual human
becoming wider and wider, new problems are gradually emerged in data driven method, which are mainly shown in
three aspects. Firstly, given motion capture data is only fit to generate corresponding motion because the specificity of
motion capture data is very strong. It is need to capture whole motion afresh even if there is only a little modification
request. Secondly, the contradiction between finite motion capture data and diversiform application requirement still
exits, finite motion capture data cannot satisfy the ever increasing application requirement. Finally, it is different,
dangerous even unable to capture some special motion.
Motion reuse is an efficacious approach to solve those problems. In this paper, the concept of motion extension is
Volume 7, Issue 1, January 2019 Page 5

presented, which can generate more abundant motion from finite motion data. Further, automatic motion lengthways
extension technique has been presented in our other paper[1]. In this paper, we present automatic motion transverse
extension technique to automatic and intelligent motion reuse.
2. RELATED WORKS
Because of the complexity of the human skeleton and motion, human motion data are high dimensional. It is often
required to reduce the dimension of motion data before analysis and research on them. According to the mapping
relation between high dimensional data and low dimensional data, dimensionality reduction techniques can be
classified to linear and nonlinear. Principal component analysis (PCA)[2] is a linear dimensionality reduction
technique in common use, which project the high dimensional data along principal axis chosen by computing variance.
Many correlative techniques are proposed based on PCA, which have better capability than PCA in specific application
field. The probabilistic principal component analysis (PPCA)[3], introducing probability model into PCA, can complete
dimensionality reduction and density estimate by likelihood. In PPCA, which is linear too, high dimensional data
(before dimensionality reduction) and low dimensional data (after dimensionality reduction) obey respective
probabilistic distribution with noise. Combining PCA and kernel method [4], the kernel principal component analysis
(KPCA) [5] is proposed. The basic idea of KPCA is to map the high dimensional data to a feature space by a nonlinear
method, and do principal component analysis in it. KPCA, as a nonlinear method, has the excellence of a kernel
method that establishes a connection between high dimensional data and low dimensional data with a kernel function,
without needing to determine the specific relation between them. Gaussian process latent variable models
(GPLVM)[6][7] is another effective nonlinear dimensionality reduction technique, which completes dimensionality
reduction and density estimate using Gaussian process regression. The GPLVM, different from PPCA, does not
estimate the parameters of mapping but margin them. From the viewpoint of Bayesian, GPLVM maximizes the
posterior probability to reduce dimensionality given prior probability.
As human motion has character of a time series, it is required to model dynamics of motion data sequence.
Dimensionality reduction deals with single motion frame and dynamics researches the correlation among frames and
the evolvement rule of motion frame. Because of dimensionality reduction to motion data, it is feasible to model the
dynamics of low dimensional data to simulate dynamics of high dimensional data. Modelling dynamics can be achieved
with two classes of techniques, linear dynamics and nonlinear dynamics. In linear dynamics, the current system state is
considered to be linearly related to the last state. This is the simplest case, and dynamics is often nonlinear in fact.
Nonlinear dynamics is much more complex than linear dynamics and, establishing a common adaptive model for it is
very difficult. Kalman filtering [8] is an early technique to estimate the state in a dynamic system, which estimates the
state based on feedback control. In Kalman filtering, measurements with noise of the state are used to improve the
estimation of the state to make the estimation approach the true state more. In general, Kalman filtering contains two
parts that is time update equation and measure update equation. The time update equation estimates the next state as
the prior estimation for it, which is improved in the measure update equation as the posterior estimation. Monte Carlo
sampling method [9][10] is also used to the estimate state in a dynamic system, whose kernel is to represent
probabilistic distribution with a number of discrete samples. Markov chain [11] models the aftereffectless stochastic
process, and a 2-order Markov chain can simulate human motion well because human motion can be described with
velocity and acceleration.
3. MOTION DESCRIPTION AND MOTION DATA PRETREATMENT

The skeleton determines body posture in human motion. The skeleton includes more than 200 bones, which need to be
predigested to research motion. By predigesting, we establish the skeleton model as shown in figure 1, which includes
52 degrees of free.

Figure 1 skeleton model

The degree of free shows the motion direction of the bone. In this skeleton, the root includes 3 translation direction
and 3 rotation direction, the other bones include 0 to 3 rotation direction. In motion, the posture and location of
skeleton is determined by these degrees of free. A vector can be established by these degrees of free, which goes by the
name of feature vector. In this way, a N frames motion can be descripted as feature vector sequence { y1 , y2 , , y N } . In
order to insure out algorithm independent of measurement of motion data, the motion data needs to be standardized.
Given a N frames motion Y '  [ y1' , y '2 , , y 'N ]T , compute the mean and standard deviation with computing formulas:
N
1 （1）
y 'j  '
 Y , ( j  1, 2, , D )
ij
N i 1
1 N （2）
S 'j   (Yij'  y 'j ) 2 , ( j  1, 2, , D)
N  1 i 1
Here, Yij' is the element at the i-th row and the j-th line of matrix Y ' . Then, standardize the motion data with mean
and standard deviation:
Y '  y'
Yij  ij ' j , (i  1, 2, , N ; j  1, 2, , D) （ 3）
Sj
The motion data is denoted by Y  [ y1 , y2 , , y N ]T after standardized, whose variance does equal 1:
' '
1 N 1 N Yij  y j 1 N Yij'  y 'j
S 2j   ij j N  1 
(Y  y ) 2
 (   )2
N  1 i 1 i 1 S 'j N i 1 S 'j
1 N '
' ' Yij'   Yij 2 （ 4）
1 N Y y 1 ij j
N
N i 1
  ( S'  N
N  1 i 1

i 1 S 'j
)
j
' ' '2

1 N Yij  y j 2 S j
 
N  1 i 1
(
S 'j
) 
S 'j 2
1
4. AUTOMATIC MOTION TRANSVERSE EXTENSION

Gaussian process dynamical models (GPDM)[12] is a specific latent variable model, which implements dimensionality
reduction for a time sequence with Gaussian process regress and models the dynamics for the time sequence in latent
space. Given the time sequence Y  [ y1 , y 2 , , y N ]T in observation space where y i is D-dimensional column vector,
i  1, 2, , N , and X  [x1 , x2 ,, x N ]T is the corresponding representation in the low dimensional latent space of Y where

x j is q-dimensional column vector, j  1, 2, , N , GPDM comprises two mappings as shown in Figure 2a. One is the
observation mapping from latent space to observation space parameterized by B; The other is the dynamics mapping in
latent space parameterized by A. From the viewpoint of Bayesian, we can marginalize out the parameters A and B by
integrating to obtain the joint distribution of latent variable and observation variable as shown in Figure 2b.
(a) Mappings in latent variable model (b) GPDM

Figure 2 Gaussian process dynamical models
The observation mapping, based on scaled Gaussian process latent variable models (SGPLVM)[13], completes
dimensionality reduction for a time sequence in high dimensional observation space. The dynamics mapping in latent
space is based on 2-order Markov chain. The two mappings can be described as
g : xt   ai i (xt 1 , xt  2 )  n x ,t （ 5）
i
h : Wy t   b j j (xt )  n y ,t （ 6）
j
where g is the dynamics mapping from the last two latent states xt 1 , xt  2 to the next latent state xt , parameterized by
A  [ a1 , a2 , ]T ; h is the observation mapping from latent space to observation space, parameterized by B  [b1 , b2 ,]T . The
yt is weighted by W  diag[1 , 2 ,, ,  D ] to make a difference among each dimensionality,  i and  j are nonlinear
basic function, n x ,t and n y ,t are isotropic Gaussian noise. g and h are implemented with Gaussian process regress,
which describes the probabilistic distribution of function. From the viewpoint of Bayesian, we do not estimate the
precise value of parameter A and B like traditional fitting but lay emphasis on the uncertainty about them. Gaussian
process is determined by a mean function and a kernel function (covariance function) similar to the way that Gaussian
stochastic variable is determined by mean and covariance. Training data has been pretreated minus mean, so mean
function is set to zero. In order to implement Gaussian process, maximize the likelihood of training data by tuning
parameters in the kernel function.
The mapping g is based on 2-order Markov chain, give the prior
N
p( X)  p (x1 , x 2 ) p (x n | x n 1 , x n  2 ) （ 7）
n 3
Given the prior Gaussian distribution for parameters A , ai ~ N (ai | 0, I) , and marginalize out it to obtain the joint
probability of latent variable X as
p( X α)   p (X | A,α ) p( A | α )dA
N
 p(x1 ) p (x 2 | x1 )  p(xt | xt 1 , xt  2 , A,α ) p ( A | α)dA （ 8）
t 3
p (x1 ) p(x 2 | x1 )
 q ( N 1) q exp( 12 tr(K X1 X3:N , XT3:N )
(2 ) 2
|KX | 2
where the vector α is the hyper parameters [1 ,  2 ,  3 ,  4 ,  5 ,  6 ] .

For the mapping h， given the prior Gaussian distribution for parameters B , bi ~ N (bi | 0, I ) , and marginalize

out it to obtain the probability of time sequence Y in observation space as

N
p(Y X,β , W)    p (y n | x n , B,β, W) p (B)dB
n 1
（ 9）
1 1 1 T
 DN D exp( tr(K YWY )
2 Y
(2 ) 2 | K | 2
where the vector β is the hyper parameter [ 1 , 2 ,  3 ] .
Given the prior for hyper parameters p (αβ)  
 i   j , the joint probability of all variable is defined as
i j
p( X, Y,α, β)  p(Y | X,β, W) p( X | α) p(αβ) （ 10）

The technique of automatic motion transverse extension can generate multiple character group motion from single
character motion automatically. We map the motion data from observation space to latent space based on GPDM:
Y  [y1 , y 2 ,  , y N ]T  X  [x1 , x 2 ,  , x N ]T （ 11）
T
Given the training data Y  [y1 , y 2 , , y N ] , train the GPDM by iteration to learn parameters of the model and
corresponding latent states X  [x1 , x 2 , , x N ]T . This can be implemented with maximizing the posterior probability
of X represented by
LGPDM   ln p ( X, α, β, W | Y)
D 1 q （ 12）
 ln | K Y |  tr (K Y1YW 2 Y T )  ln | K X |
2 2 2
1 1
 tr (K X1X3:N XT3: N )  x1,2
T
x1,2   ln β j   ln αi  N ln | W |
2 2 j i
The training algorithm can be described as algorithm 1.

Algorithm 1. training GPDM using existing motion data
Input: Y
Initialize   [0.9,1,1, 0.1, 0.1, e] ,   [1,1, e] , {k }  1 , d  3 , I  100 , J  20 .
Initialize X with PCA on Y with d dimensions.
for i=1 to I do
for j=1 to D do
Y j  [Y1 j , Y2 j , , YNj ]T .
 j  N / YTj K Y1Y j . //update W according to literature Error! Reference source not found.;
end for
{X, α, β}  minimize LGPDM using SCG for J iterations. // refer to the literature Error! Reference
source not found.;
end for
Output: X, α, β, W .
Transverse motion extension extends human motion along characters, and it generates multiple character group
motion from single character motion automatically. Multiple character group motion is that many characters perform
the same motion together, in which every character motion is similar to each other and there is difference among them.
To implement transverse motion extension, train GPDM with single motion to obtain corresponding latent states which
form a smooth latent trajectory described as algorithm 1, then, sample near it in latent space to obtain more latent
trajectories with hybrid Monte Carlo (HMC)[17]. After optimizing the new latent trajectories, map them from latent
space to observation space to generate new motions which form group motion together. Optimize latent trajectory X
according to
X  Max{ p( X α )} （ 13）
The new latent trajectories sampled by HMC lie near the latent trajectory of training data, and the motion poses are
similar if their latent states are nearby, so the new motions reconstructed by new latent trajectories are similar to the

training motion, but there is difference among them. This is suitable for group motion.
In the end, we reconstruct the pose y t in observation space from the latent state xt . The new pose y t in observation
space is Gaussian about the latent state xt .
y t ~ N  Y (xt ),  Y2  xt  I  （ 14）
Here,
Y (x t )  YT K Y1kY (xt ) （ 15）
2 T 1
 (xt )  kY (xt , xt )  kY (xt ) K k (xt )
Y Y Y （ 16）
According as the mean prediction[18], which predicts with probability mean, y t can be generated by
y t  YT K Y1kY (xt ) （ 17）
5. RESULTS AND ANALYSIS

In our experiment, the training motions come from the Carnegie Mellon University motion capture database[19], which
is 50 dimensional, and the latent space is 3 dimensional.
The training motion is the football motion which consists of 362 frames as shown in the top line of Figure 4. Train
GPDM with it to obtain corresponding latent states, and extend the latent trajectory to 5 ones by hybrid Monte Carlo
sampling in latent space as shown in Figure 3, in which the sign ‘*’ denotes training motion and ‘o’ denotes extended
latent trajectories.
training motion
extended motion
Figure 3 Dimensionality reduction of training motion and transverse extension of latent trajectory
After extending in latent space, map the extended latent trajectories to observation space to generate new motions as
shown in Figure 4, in which the extended motions are similar to the training motion, but there is difference among
them. This is suitable for multiple character group motion.

1 17 33 49 65 81 97 113 129 145 161 177 193 209 225 241 257 273 289 305 321 337 353
frame
train
character
Figure 4 Transverse motion extension
6. CONCLUSION
The human motion transverse extension technique, proposed in this paper, can generate multiple character group
motion from single character motion automatically. Motion transverse extension does not repeat existing human
motion from one character to another by rote but generate new motions following the motion rule of training data,
which is depicted by probabilistic distribution of motion pose. The extended motion is that many characters perform the
same motion together, in which every character motion is similar to each other and there is difference among them.
Motion transverse extension can extend the applicable field of training motion.
ACKNOWLEDGEMENTS
Thanks to the anonymous reviewers for their helpful comments, Neil Lawrence for his on-line source code, and
Carnegie Mellon University for human motion capture data.
This work is supported by the National Nature Science Funding No. 61602506, No. 11805278 and the Hu Bei
Province Nature Science Funding No. 2016CFB307.
References
[1] QU Shi, CAI Yi-chao, LIU Jian-xun, REN Ming-qiu, LI Shi-lei, “Automatic Motion Lengthways Extension for
Data Driven Virtual Human”, International Journal of Emerging Technology and Advanced Engineering, 8(11),
pp. 99-103, 2018.
[2] Sun Hongwei, Gu Ming and Sun Jiaguang, “A Coding Algorithm Using PCA-Based Correlation Vector
Quantization”, Journal of Computer-aided Design & Computer Graphics, 17(8), pp.1662-1666, 2005.
[3] Michael E. Tipping, Chreistopher M. Bishop. “Probabilistic Principal Component Analysis”, Journal of the Royal
Statistical Society, 61(3), pp.611-622, 1999.
[4] Christopher M. Bishop, “Pattern Recognition and Machine Learning”, Singapore, 2006.
[5] R.Rosipal, M.Girolami, L.J. Trcjo, A. Cichochi, “Kernel PCA for Feature Extraction and De- Noising in Non-
linear Regression”. Neural Computing & Applications, 10(3), pp. 231- 243, 2001.
[6] Neil D. Lawrence. “The Gaussian process latent variable model”. Tech. Report. 2006.

[7] QU Shi, WEI Ying-mei, KANG Lai, WU Ling-da. “Pose Synthesis of Virtual Character Based on Statistical
Learning”, The International Symposium on Computer Network and Multimedia, Wuhan, China, pp. 36-39,
2009.
[8] QU Shi, WU Ling-da, WEI Ying-mei, YU Rong-huan. “Dimensionality Reduction and Generation of Human
Motion”, International Journal of Computers, Communications & Control, 8(6), pp. 869-877, 2013.
[9] Jacobs, O. L. R. “Introduction to Control Theory”, 2nd Edition, Oxford University Press, 1993.
[10] J. Carpenter, P. Clifford, and P. Fernhead. “An improved particle filter for non-linear problems”. In IEE
Proceedings on Radar, Sonar and Navigation, 1999.
[11] Kiam Choo and David J. Fleet. “People tracking using hybrid Monte Carlo filtering”. In ICCV, pp. 321-328,
2001.
[12] Sheldon M. Ross. “Introduction to Probability Models”. POSTS & TELECOM PRESS, China, 2007.
[13] Wang, J. M., Fleet, D. J., Hertzmann, A. “Gaussian Process Dynamical Models”. In Proc. NIPS 2005. Vancouver,
Canada. pp. 1441-1448, 2005.
[14] Keith Grochow, Steven L. Martin, Aaron Hertzmann, Zoran Popović. “Style-based Inverse Kinematics”. ACM
Transactions on Graphics (Proceedings of SIGGRAPH 2004), 23(3), pp.522-531, 2004.
[15] Y. Yacoob and M.J. Black, “Parameterized Modeling and Recognition of Activities”, Computer Vision and Image
Understanding, 73(2), pp. 232-247, 1999.
[16] M. Møller. “A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning”. Neural Networks, pp. 525-
533, 1993.
[17] Radford M. Neal, “Probabilistic Inference Using Markov Chain Monte Carlo Methods”, 1993.
[18] D. MacKay.” Information Theory, Inference, and Learning Algorithms”. 2003.
[19] http://mocap.cs.cmu.edu/.
AUTHOR
QU Shi received his PHD in Systems Engineering from National University of Defense Technology,
Changsha, China, in 2011. Now, he works in Air Force Early Warning Academy, Wuhan, China. His
research interests include computer graphics and character animation.

An Approach of Automatic Motion Transverse Extension

Hochgeladen von

Dokumentinformationen

Copyright

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

An Approach of Automatic Motion Transverse Extension

Hochgeladen von

Copyright:

IPASJ International Journal of Information Technology (IIJIT)

Web Site: http://www.ipasj.org/IIJIT/IIJIT.htm

An Approach of Automatic Motion Transverse

Volume 7, Issue 1, January 2019 Page 5

3. MOTION DESCRIPTION AND MOTION DATA PRETREATMENT

Volume 7, Issue 1, January 2019 Page 6

Figure 1 skeleton model

' ' '2

4. AUTOMATIC MOTION TRANSVERSE EXTENSION

Volume 7, Issue 1, January 2019 Page 7

(a) Mappings in latent variable model (b) GPDM

where the vector α is the hyper parameters [1 ,  2 ,  3 ,  4 ,  5 ,  6 ] .

Volume 7, Issue 1, January 2019 Page 8

out it to obtain the probability of time sequence Y in observation space as

p( X, Y,α, β)  p(Y | X,β, W) p( X | α) p(αβ) （ 10）

The training algorithm can be described as algorithm 1.

Volume 7, Issue 1, January 2019 Page 9

5. RESULTS AND ANALYSIS

Volume 7, Issue 1, January 2019 Page 10

Volume 7, Issue 1, January 2019 Page 11

Volume 7, Issue 1, January 2019 Page 12

Das könnte Ihnen auch gefallen