2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Daejeon Convention Center
October 9-14, 2016, Daejeon, Korea
A Reactive Stepping Algorithm Based on Preview Controller with Observer
for Biped Robots
Oliver Urbann, Matthias Hofmann
Abstract Reactive stepping is an important utility to regain
balance when bipedal walking motions are disturbed. This
paper sheds light on the reasons for humanoid robots to
fall down. It presents a method to calculate modifications
of predefined foot placements with the objective to minimize
deviations of the Zero Moment Point from a reference without
interrupting the walk. The calculation is in closed-form, and
is embedded into a well-evaluated preview controller with
observer based on the 3D Linear Inverted Pendulum Mode
(3D-LIPM). Experiments in simulation and on a physical robot
prove the benefit of the proposed system.
I. I NTRODUCTION
Research in the area of biped robots has become increas-
ingly important over the last years. Especially in environ-
ments that contain obstacles and barriers such as stairs,
or small objects lying on the ground, biped robots are
advantageous to wheeled robots. Bipedal locomotion in such
surroundings is naturally susceptible to disturbances that
occur in case of collisions with those objects.
A. Related work
The Zero Moment Point (ZMP) is a widely used criterion
to determine the stability of humanoid robots. If it is inside
the support polygon of the feet, the robot is stable.
There are many approaches to derive a CoM trajectory
from the desired ZMP based on the 3D-LIPM, for instance
a convolution sum [1], or an analytical solution [2]. Kajita
et al. [3] proposed a method for an arbitrary ZMP trajectory
based on a linear-quadratic regulator. This approach is suc-
cessfully applied to real world robots [4], [5] and therefore
an adequate base for this work. It is possible to incorporate
measurements from different sensors by integrating an ob-
server [6], [7], [8]. While a stabilizing effect of this approach
can be shown1 [8], it is limited due to the fact that major
disturbances cannot be compensated without a lunge2 .
There are various approaches to determine a step that fits
best to the desired motion and dynamical state of the robot.
Utilizing the orbital energy leads to an analytical solution
of balance recovery [9]. While this is in fact not a lunge
where the desired foot position is modified, Urata et al. [10]
proposed a technique based on a preview controller that
optimizes the foot placement numerically. Also Nishiwaki
et al. [11] generate the walking patterns utilizing a preview
1 A video of the closed-loop performance can be watched here
as a comparable set-up will be utilized here in the experiments:
https://www.youtube.com/watch?v=ZuebspajdZU
2 The term for a step that is needed to compensate stronger disturbances.
A lunge is a modification of a planned foot position, and also called reactive
stepping or just stepping.
978-1-5090-3761-2/16/$31.00 2016 IEEE
controller and modify the foot placement within the preview
phase. The basic idea is therefore comparable to this work,
however it is a simplified version with the goal to limit
the divergence and can lead to instabilities. Here we prefer
a provable zero-error strategy. Other approaches are based
on the Capture Point (CP) [12]. This is the point where the
robot may step on to stop with the CoM directly above.
The CP calculation uses an extension of the 3D-LIPM to
generate a walking motion [13]. While the calculation of
the steps are in closed form, the CP varies over time [14].
Moreover, the CP approach does not consider the point in
time when the stance foot is changed. Hence, Zutven et
al. [15] invented the foot placement estimator (FPE) and
the foot placement indicator (FPI). However, all points (CP,
FPE and FPI) share the drawback of generating a walking
motion counter-intuitively as they were originally introduced
to immediately stop the robot after disturbances occur.
Morisawa et al. [16] utilize a quadratic program to find
appropriate foot placements. This solution might reveal large
CoM and ZMP deviations. In latter cases, the disturbances
should be balanced by other means than stepping [17]. Herdt
et al. [18] propose Model Predictive Control (MPC): A
quadratic programming problem is formulated such that the
position of the feet is automatically modified by the optimizer
if the robot cannot be balanced by the ZMP inside the
support polygon. The main disadvantage of the preceding
two approaches is the necessity for numeric solvers.In the
evaluation we compare the computation time of an MPC
scheme with reactive stepping with our approach.
In this paper we present an approach that fulfils require-
ments as explained in the next section. We presented a
previous version for simulated robots in [19] and extended
this algorithm to be applicable on a physical robot. This
paper also contributes algorithms needed for real hardware
as explained in chapter V. The evaluation in this paper proves
the advantage of the newly proposed algorithms.
B. Objectives
This paper proposes a method to modify the predefined
foot placement in order to regain balance after disturbances
were observed. To this end, the following requirements on
the system are defined:
1) A reference ZMP of an arbitrary shape to design a
pattern that mitigates oscillations of the body (see
section V-A for a detailed explanation).
2) Lunge execution must be optional as it might be
disadvantageous in situations where the target position
is next to an object.
5324
 3) The system must guarantee a solution for the modifi-
cation computation of the foot placement.
4) Calculation in closed-form is preferred as the controller
is intended to run on platforms with limited computing
power. Moreover we demand a controller frequency of
at least 100 Hz.
5) As kinematic constraints must be considered to avoid
self-collision, the system must be able to defer the
lunge execution.
Since a reference ZMP of an arbitrary shape is desired, the
preview controller with observer approach [6], [7], [8] is the
basis for this work.
C. Structure
The remainder of the paper is structured as follows:
Different aspects of the dynamic state of the robot are
shown in section II. Section III briefly summarizes the main
concepts of the used preview controller and observer, and
reflects the output of the system in case of disturbances.
Section IV presents a method to cope with these disturbances
and outlines how the modification of the foot placement
is processed. Section V gives an overview of the walking
algorithm used in this paper. Our approach is proven to
be beneficial in section VI by showing experiments in
simulation and on the Nao robot by Aldebaran Robotics3 .
The paper concludes in section VII including future work.
II. DYNAMIC S TATE OF THE ROBOT
This section describes a number of design choices on the
modelling of the dynamic state of the robot that have to
be taken before implementing a ZMP-based walking engine.
Furthermore, appropriate sensor sources for measuring dis-
turbances are discussed.
A. Model
The 3D-LIPM abstracts the robot to a single point of mass
and is used in our work to construct the controller. This mass
is supported by a single pole, and can only move in a plane
with a constant height over the ground. To keep the mass at
the height cz , the ZMP p must be at the x position of the
pivot joint. The acceleration of the CoM in x direction cx
must then satisfy:
cz
p = cx cx , (1)
g
where g is the gravity.
Note that the ZMP is the perpendicular projection of the
CoM on the ground if the CoM is not accelerated. If it is
accelerated, the ZMP deviates proportionally to the opposite
direction.
The state of the robot is described as a vector x =
T
(cx , c x , p) which consists of the position and speed of the
CoM, and the position of the ZMP. The paper presents all
equations exemplary for the x axis. It is important to note
that the system works for the y axis in the same way.
The state x must be updated by sensor measurements. The
next section discusses appropriate sources for this task.
3 http://www.aldebaran-robotics.com
5325
crl
csmpl cz
pr pr
Fig. 1. Approximate calculation of the measured position of the CoM.
B. Sensor Sources
In this section we discuss different types of disturbances
and appropriate measurement methods. As reactive stepping
is appropriate to deal with greater disturbances causing
feet to partly leave the ground, the inertial measurement
unit (IMU) is utilized to sense the posture of the body. It
consists of an accelerometer (max. 2g) and a gyroscope
(max. 500 /s). Its output, the angle around the lateral
and transversal axis, can be utilized to calculate the actual
position of the CoM with respect to the ground. Magnetic
Rotary Encoders are utilized to measure the joint angles for
forward kinematics with a precision of 0.1 .
Figure 1 depicts the approximate calculation of the CoM
in case the robot is tipping with an angle of . On the left,
a robot is tipping around the back of its foot. It is simplified
according to the 3D-LIPM, and represented by a single CoM
crl , as can be seen on the right. For simplicity, we assume
that the robot rotates around the coordinate system of the
foot, which coincides with pr in this figure. This results in
csmpl where due to the rotation also the height differs. Since
our model assumes a CoM with constant height cz , a small
error occurs.
III. P REVIEW C ONTROLLER AND O BSERVER
This section describes the concepts used in the preview
controller. Afterwards an observer is presented which inte-
grates the previously mentioned sensor sources.
A fast way to determine the target position of the CoM
is a preview controller as proposed by [3]. It is a linear-
quadratic regulator (LQR) that requires a preview of the
reference ZMP, denoted here as a vector pref of all reference
points for the walk. A detailed description is given by [7],
and proofs are given by [20].
1
A discrete-time system (with a sampling frequency of t )
describes the physical behaviour, and can be formulated as:
xk+1 = A0 xk + buk (2)
pk = Cxk (3)
where uk denotes the controller output which is the time
T
xk = (cx,k , c x,k , pk ) is the state
derivative of the ZMP p,
T
of the system and b = (0, 0, t) integrates the controller
output over time. Matrix C = (0, 0, 1) selects the reference
 0.1
IV. I NTEGRATION OF FOOT PLACEMENT MODIFICATIONS
Position y [m]
0 Disturbances change the kinematic and dynamical state of
the system with an influence on the actual ZMP. Falling is a
pref result of a condition where the ZMP is outside of the support
0.1
cx
p polygon. Hence, balancing implies a continuation of the walk
1.5 1.6 1.7 1.8 1.9 2 2.1 2.2 2.3 2.4 2.5 after a disturbance with as small as possible ZMP deviations.
Time [s]
In this sense, figure 2 reveals an inadequate compensation of
Fig. 2. Lateral plot of disturbed walk balanced by the controller.
the disturbance. We conclude that mitigating the deviation of
the ZMP from the reference is a requirement for a balancing
algorithm.
ZMP as the system output. The matrix The open-loop case where no disturbance is measured

1 t 0
leads to a well tracked reference. We thus want the closed-
A0 = cgz t 1 cgz t (4) loop system to behave like in the open-loop case, with an
actual ZMP that tracks the reference well. This should avoid
0 0 1
the three peaks appearing in Fig. 2. It is therefore reasonable
incorporates the 3D-LIPM equation 1. to postulate that the closed-loop ZMP of a disturbed walk
The controller minimizing a given cost function as de- must be equal to the open-loop case:
scribed in [8] needs N + 1 future ZMP reference points in
order to work properly and is given by: !
k h N
xT +2 = CxT +2 ,
Cb (9)
X i X
uk = GI Cxi pref
i Gx xk Gd,j pref
k+j , (5) where T is the time when the error is measured. Please note
i=0 j=1 that T + 1 is the time when the error appears in the state
where GI and Gx are gain matrices for the integral and vector. The controller u
bT +1 induces the ZMP deviation into
proportional part of the controller respectively and Gd,j is xT +2 .
b
an array of scalars utilized to incorporate the ZMP preview. In the following we assume that only at time T an error is
As current ZMP and CoM positions can be measured (see measured. Then equation 9 holds until T +1. The closed loop
section II-B) [8], it is common standard to implement an state and the open loop state differ at this time. Therefore
observer that computes an estimated state vector x. the controller output is different at T + 2 as can be seen in
Hence, a Matrix Cm is defined as equation 5. To achieve uT +2 = u T +2 , we define the closed-
  loop controller with a modified reference ZMP pref k+j :
1 0 0
Cm = (6)
0 0 1
k h i N
which is used to select the measurable part y of the state
X X
k = GI
u Cxi pref
i Gx xk Gd,j pref
k+j , (10)
vector
x: i=0 j=1
 snr 
cx,k
yk = Cm xk = , (7)
psnr
k pref
k = pref
k + pm
k , (11)
where csnr
x,k and pk
snr
denote the measured CoM and ZMP k < T + t0 : pm
k = 0, (12)
respectively. k T + t0 : pm
k = m, (13)
The discrete time system with sensor feedback is defined
slightly different from the open-loop case: where t0 is the starting time of the modification, and m is
the modifier.
b xk + L [yk Cmb
xk+1 = A0b xk ] +bb
uk . (8) Theorem 1: Given an impulse-like error et with et = 0
| {z }
=ek for t 6= T , a map S : {2, . . . , N + 1} R3 R,
(t0 , e) 7 m exists that can be calculated in closed-form
Figure 2 illustrates an example walk where a distur- !
such that Cb xT +2 = CxT +2 holds.
bance is balanced by the controller. It shows the input
Proof: S can be derived by examining the influence of
(pref ) and output (cx and p) of a walk that is perturbed
eT on the state vector xT +2 by applying equation 8 and 10
by impulse-like errors. The following errors are measured:
recursively:
(1.7 s) = (0.005 m, 0 m)T , (2 s) = (0.003 m, 0 m)T and
(2.05 s) = (0.003 m, 0 m)T . One frame after an error was Cb xT + eT + bb
xT +2 = C (A0 (A0b uT ) + eT +1 + bb
uT +1 )
measured, the observed ZMP discloses a large deviation. This
is the reaction of the controller that accelerates the disturbed (14)
CoM trajectory cx towards the reference CoM cx that is u
bT +1 = IT +1 + XT +1 + DT +1
b b b (15)
optimal in the undisturbed case. Starting with the next frame, T +1 h i
the CoM is then decelerated until the CoM position and
X
IbT +1 = GI ui1 ) pref
xi1 + ei1 + bb
C (A0b i
velocity regain optimal values. This deceleration yields to i=0
a ZMP error in the opposite direction. (16)

5326
 bT +1 = Gx (A0b
X xT + eT + bb uT ) (17) (vx , vy , )

N
X h i Pattern
b T +1 =
D Gd,i pref
T +1+i (18) Generator
i=1
N h N
FW CS
X i X
Gd,i pref Gd,i pm
 
= T +1+i T +1+i (19) 2

i=1 i=1 ZMP Swinging Leg

XN h i N
X Generation pm Controller
= Gd,i pref
T +1+i m Gd,i (20)
i=1 i=t0 1 pref FW
3
CS

Equations 12 and 13 are applied in equations 18 to 20. Since

the modification is constant starting at t0 , the sum of the ZMP/IP Orientation
mapping pm can be replaced by the corresponding scalar Controller Controller
value m. Furthermore equation 20 clarifies the first possible FW
3,r
CS

time where the modification can begin. Since i 1 the + Inverse

first possible non-zero modificator is pm m
T +t0 = pT +1+1 and Kinematics
bcx
therefore t0 2.
Equations 14-20 are simplified by exploiting the following FRCS
3,r
x
b
statements: Arm
Movement
k T : b
xk = xk , u
bk = uk (21)
k 6= T : ek = 0 (22) psnr
k T + t0 , 2 t0 : pref = pref ZMP/IP
k k , (23) Robot
Observer
Equation 21 holds since e is an impulse-like error at time
e
T . It is important to note that there exists no difference
between the open-loop case and the closed-loop case before Fig. 3. Overview of the simplified structure of the walking algorithm
T . Hence, equations 14-20 are similar in the open-loop case where F are the foot positions in world coordinate system (WCS) or robot
except for the error terms e and the ZMP modification pm . coordinate system (RCS). Subscripts 2 and 3 denote the dimension of the
position variables and subscript r the orientation handling as explained
If we reformulate equation 9, this cancels any term that does in section V-C. The measured body orientation is denoted by and the
not contain e or m : measured ZMP by psnr .

xT +2 CxT +2
0 =Cb (24)
N
!!
X mathematical results can be used by any ZMP-based walking
=C A0 eT + b GI CeT Gx eT m Gd,i algorithm.
i=t0
(25)
!1 A. Basic Walking Algorithm Structure
N
X Input of the algorithm is a three-dimensional speed vector
m = Cb Gd,i C (A0 bGI C bGx ) eT .
(vx , vy , )
for x and y direction as well as the rotation.
i=t0 1
| {z } The functionality of the walking engine is encapsulated in
=:Ge,t0 several modules (see figure 3). The first module Pattern
(26) Generator calculates foot positions on the ground that lead
Please note that the inverse exists as it is a scalar. Function to the desired speed. These positions are two dimensional,
S can now be defined as: i.e. no height over ground is given.
The ZMP Generator plans the reference ZMP within the
S(t0 , e) = Ge,t0 e, t0 {2, . . . , N + 1}. (27) support polygon of the generated steps with special focus
Therefore the modification of the reference ZMP and the foot on the avoidance of undesired oscillations of the body. To
steps can be calculated by a simple multiplication. achieve this, we minimize the distance between the desired
The gains Ge can be calculated in closed-form together with ZMP and the expected CoM trajectory: In x direction we
the gains described in section III offline before walking. assume a constant reference ZMP speed which yields a
constant CoM speed. In y direction we utilize a Bezier curve
V. WALKING A LGORITHM that is similar to the CoM movement. The ZMP generation
This section shows how the foot placement modifications is described in more detail by [8].
are included into our walking engine. In simplified terms, The main module of the system is the ZMP/IP-Controller
the foot positions are modified about the same value the (see section III, equation 5) which computes the output of
ZMP is modified such that the reference ZMP position the controller cx and u as well as the state x (equation 8).
within the support polygon remains the same. However, the This is contrary to the ordinary controller/observer design as

5327
the observed state is also determined in this module.
Hence, the module Observer calculates exclusively the
error e which is used by the ZMP/IP-Controller. It is also
the input for the module Swinging Leg Controller where
the modification of the steps are handled.
B. Swinging Leg Controller and Conditions for Lunge Exe-
cution
The Swinging Leg Controller adds the third dimension
(i.e. height) to the foot positions from the Pattern Generator
and interpolates from the start to the target position: A B-
Spline calculates points such that the foot lifts off slowly
and stays at its maximum height as long as possible to avoid
undesired collisions with the ground in case of tilting. Details
are given by [8].
Algorithm 1 Decision procedure for the current time frame
if e is balanced by a step modification m at step number k,
where 0 is the step that begins when the current swinging
foot touches the ground. The maximum step number kmax
ensures that small errors do not lead to large modifications.
To ensure step modifications are smaller than mmax , a sum
sk is calculated including all modifications applied to step k
before.
1: k := 0, T := (Time stamps of start of future steps)
2: if k < kmax then
3: m = Ge,Tk e
4: if m + sk > mmax then
5: k := k + 1 // Try to modify next step
6: GOTO 2
7: end if
8: Generate pm from m (Eq. 13) and apply (Eq. 12)
9: sk + = m
10: end if
The modification of the feet is computed as shown in
equation 27. A modification might lead to a step that is
physically not executable (e.g. in case of self-collisions, and
not-reachable positions). Hence, we implemented a decision
procedure which is shown in Alg. 1. For this decision
we introduce lower bounds that guarantees small distances
between the feet, and limit the maximum modification of one
step. It is noteworthy that adjustments within one frame are
small so that the maximum is possibly reached after multiple
adjustments. Hence, we integrate all modifications that are
applied to one step until the bound is reached.
C. Handling of Erroneous Orientation
The Orientation Controller prevents undesired contacts
of the feet with the ground which result in an erroneous
orientation of the body.
The module consists of two phases: First, the foot po-
sitions are adapted as shown in figure 4. The origin of the
swinging foot Orf is rotated around the origin of the foot Olf
by . Additionally, the swinging foot is rotated by such
that it is parallel to the ground. This adaptation is performed
exclusively when the foot is in the air. In the second phase,
5328
0
Orf
Olf
Orf
Fig. 4. The swinging foot with the position Orf is rotated around Olf to
avoid undesired contacts with the ground. The foot is additionally rotated
around the angle to be in parallel with the ground.
OR
cx cbx
fR
OW
fW
Fig. 5. To calculate the position of the feet in the robot coordinate system
OR only simple vector calculations are needed if the positions of the feet
are given in the world coordinate system OW .
the robot must be erected to its desired posture when the foot
touches the ground. We set f to the last measured angle
before the swinging foot touches the ground. Afterwards, the
desired orientation df of the foot on the ground is adapted as
described previously using the angle df = f t1s , where 1
is a constant factor < 1 and ts N0 is the frame within the
support phase. Thus, df becomes smaller over time erecting
the body to the upright position.
As can be seen in Fig. 4, this implies a lift of the CoM
in upright direction that leads to high torques on the joints.
To assist this procedure we therefore allow a lower CoM
by changing the demanded height according to the currently
measured body angle: c0z = cz 2 with an arbitrary
factor 2 . This height adaption is applied at all times which
results in a smooth body erection and CoM lifting until it is
upright even if it takes more than one step.
Reasonable values for 1 and 2 are elaborated experi-
mentally. Higher values lead to a faster stabilization after a
disturbance but may overexert the joints. We use 1 = 0.7
and 2 = 0.1 in this paper.
D. Coordinate System Conversion and Inverse Kinematics
The CoM cxb of the robot is given in the robot coordinate
system OR while the desired position cx is represented
in the world coordinate system OW (see figure 5). The
corresponding foot position f R such that cx = cxb holds is
calculated by:
f R = cxb cx + f W ,
 0.1 0.2 1.6
0.15 flW td 1.4
Position y [m]
0 0.1 frW 1.2

Position x [m]

Time [ms]
pref 5 102 cx 1
cx
0.1 p 0 0.8
pref 5 102 0.6
1.5 1.6 1.7 1.8 1.9 2 2.1 2.2 2.3 2.4 2.5 0.1 0.4
Time [s] 0.15 0.2
0.2 0
Fig. 6. Example walk with the same measured errors as in figure 2 that 0 2 4 6 8 10
are balanced by reactive stepping. Time [s]

Fig. 7. Timing (computation time td of one control loop) of the relevant

where f W is the foot position in the world coordinate parts of the ZMP/IP walking algorithm during a disturbed walking in a
system. simulation. Here the lateral axis of the robot is depicted.
The positions of the feet are translated to joint angles
by inverse kinematic, completed by angles for the arm
movement and finally sent to the robot. Details about the 0.2
fopt td
arm movement are given by [8]. 0.1 copt
popt 150
VI. E VALUATION

Position y [m]
0

In this section we evaluate the proposed reactive stepping
algorithm in conjunction with the walking engine. The ef-
fectiveness of the 3D-LIPM and preview controller is shown
by Kajita et al. [3]. The stabilising effect of the observer
is proven by [8]. Therefore, we examine the benefits of
the reactive stepping algorithm in five different experiments.
This includes set-ups evaluating the reaction to disturbances
with various speeds and different abstraction levels (3D-
LIPM, rigid body simulation and real robot). Additionally,
we compare some of the results with the MPC scheme by
Diedam et al. [21] and the approach by Nishiwaki et al. [11].
A. 3D-LIPM Example
This section starts with an example that presents the stabi-
lizing effect of the ZMP/IP algorithm on a robot modeled as
a single mass (3D-LIPM). To this end, we follow up on the
simulation of the walk that is shown in figure 2, and assume
the same disturbances for our experiment.
Figure 6 depicts an example4 where modifications of
the foot positions are planned for the beginning of double
support phases at 1.95 s and 2.3 s. Utilizing the algorithm
for reactive stepping as presented in this paper, all errors
depicted in 2 can be balanced while the observed (generated)
ZMP p is equal to the desired one.
This allows a comparison with the foot placement modifi-
cation approach by Nishiwaki et al. [11]. In [11, Fig. 11] the
generated ZMP px (in our paper called the observed ZMP p)
does not track the reference ZMP even if balanced using step
modification, e.g. at time 20 s-25 s, whereas in our approach
the reference is always tracked as shown above if the error
is balanced by a step modification.
B. Timing Analysis and Comparison with MPC
We conduct a runtime analysis of the proposed algorithm
in comparison to the MPC-based solution. For the latter we
utilize qpOasis5 as suggested by [21] and [18]. All runtime
4 Matlab scripts calculating this example are available at https://
github.com/OliverUrbann/ReactiveStepping.git.
5 http://www.kuleuven.be/optec/software/qpOASES
Time [ms]
0.1 100
50
0
0 0.5 1 1.5 2 2.5 3 3.5 4
Time [s]
Fig. 8. Timing of the optimizer of the MPC algorithm.
experiments are performed on an Intel Core i5 1.8 Ghz PC.
We can confirm the results of Wieber et al. [21] that the
system is solvable in less than 1ms in average as long as
we adopt the controller frequency of 10 Hz as proposed.
However, our goal is a frequency of at least 100 Hz. As long
as the solution of the current frame is similar to the last one,
the quadratic program is solved in 2ms in average which is
still real-time capable according to our definition. However,
when the stance foot changes, solving takes 50 100ms
(Fig. 8) which is a result of the fact that the hot-start strategy
of the solver now fails. In contrast, the preview controller
requires between 0.2ms and 0.4ms for computation, see
Fig. 7. For a limited platform like the Nao (ATOM Z530
1.6 GHz CPU) this would mean that MPC is not real-time
capable, while preview control is real-time capable even in
conjunction with other algorithms like vision systems.
However, the requirement of a numeric iterative solver is
a consequence of the automatic ZMP and step planning that
can be seen in Fig. 8. The ZMP is automatically chosen
to minimize energy consumption first till 1.6 s where the
walk is disturbed. In the following the ZMP is planned
at the edge of the polygon. The steps are therefore only
minimally modified if the ZMP would otherwise leave the
support polygon. This is clearly an advantage of the MPC
scheme. Thus it depends on the area of application if a fast
and reliable or an automatic planning is preferred.

5329
C. Rigid Body Simulation
This experiment examines the performance and capabili-
ties of the algorithm in a simplified, reliable environment as
a precedent step for the application on the physical robot.
Hence, we utilize SimRobot [22] which contains a model of
the Nao. The simulated robot provides 11 degrees of freedom
in the legs, FSR sensors, an accelerometer, a gyroscope, and
the ability to measure the angle of the joints. The simulated
robot is controlled and measured at a frequency of 100 Hz.
The distance of the ankle to the back end of the foot is
smaller than to the front. Thus, we choose a disturbance that
is most problematic for the robot: a push during a walk at
vx = 0.1 ms from the front such that the robot tilts around
the back end of the feet. We apply a moment of 2Ns to
the chest body at time 6.5s which immediately leads to a
fall down if no corrective action is taken. Figure 7 reveals
that reactive stepping compensates the disturbance four steps
after the impact by stepping back.
D. Pendulum Experiment
This section describes the pendulum experiment which is
conducted on the physical robot. The robot is pushed from
the front by a pendulum mass with a momentum of 1.13Ns
(see figure 10). This is sufficient to cause the physical robot
to fall down if he is properly hit. To increase the probability
of a correct push, the robot walks in place. Since a proper
hit can not be guaranteed, the experiment is conducted 60
times, 30 times with activated reactive stepping, and 30 in
ordinary walking configuration. As heating may cause the
exerted joint torques to decrease, we used both configurations
in an alternating manner.
We first count the immediate fall-downs. We define this
as a fall-down that happens directly after a hit without
changing the tilting direction. Thus, in this setup as tilting
backwards only. In 11 of the 30 iterations without reactive
stepping the robot fell down immediately and in none of
the 30 iterations with reactive stepping. Figure 9 depicts the
moment of the impact and thereafter. It is cognizable that
modifications are applied at every frame due to sensor noise.
Large modifications to the back are performed shortly after
the impact which stops the deviation of the measured CoM
csnr
x from the observed one (cx ).
Due to inaccuracies of the real robot, the system tilts to the
front after the capture step in Fig. 9. This overshooting effect
of the system is well-known from controller theory. It may
be caused by the modified steps itself or by the P controller
described in Sec. V-C. In 5 of the 30 trials with reactive
stepping a fall-down happened after these oscillations.
This leads to the conclusion of this experiment that im-
mediate fall-downs are prevented if the proposed reactive
stepping algorithm is applied. However, the overshooting
effect may destabilize the system after a successful capture
step which must be further investigated in future work.
E. Beam Experiment
In this section, we compare the reactive stepping approach
with the ZMP/IP-Controller and observer by conducting the
5330
very same experiment as proposed by [8] which we will
describe first: A beam of 5mm width and height is fixed
on ground. The intention is to induce an instability in the
system without modeling the unlevel ground. In [8], the robot
moves at 50 mm s such that one foot walks over the beam.
This induces an oscillation that leads to a fall down without
sensor feedback in all of the 10 trials. The observer damps
this oscillation such that the robot falls down in none of the
10 cases.
In this paper we alter the setup slightly which is the second
scenario: The robot goes at the same speed but with both feet
perpendicular on the beam, see figure 11. As a consequence,
the tilt of the body increases until it falls down. As our
approach does not include the beam into the step planning
to avoid stepping onto it, it is not possible to pass the beam
and the robot falls down in any case.
In a third scenario, reactive stepping is activated. When
the robot hits the beam with its feet, it tilts rearwards and
walks consequently backward to avoid the fall down. With
lower body inclination it walks forward again. Due to the
reason explained above, it is still not possible to pass the
beam resulting in an endless loop of a forward walk and
backward reactive stepping.
To conclude this experiment, in a situation with slow
raising body inclination our reactive stepping approach can
avoid a fall-down reliably without overshooting.
VII. C ONCLUSION AND F UTURE W ORKS
This paper presents a reactive stepping algorithm based
on the generation of walking motions with sensor feedback.
It points out that the reason for unsuccessful stabilization in
bipedal walking is mainly an inappropriate reaction to distur-
bances. A technique is described that enables the closed-form
calculation of foot placement modifications such that distur-
bances of the CoM position can be balanced with negligible
ZMP deviations. The benefits of our approach are proven in
different experiments, and a runtime analysis shows that our
algorithm is suitable for real-time applications.
Further investigations may focus on the avoidance of the
overshooting effect of the body orientation that was observed
in the pendulum and beam experiments while retaining
the reactivity to large disturbances. Moreover, existing ap-
proaches like the MPC system modify the ZMP considering
certain validity constraints. We plan to extend our ZMP/IP
approach with this feature.
R EFERENCES
[1] J.-H. Kim, Walking pattern generation of a biped walking robot
using convolution sum, in Humanoid Robots, 2007 7th IEEE-RAS
International Conference on, 29 2007-dec. 1 2007, pp. 539 544.
[2] K. Harada, S. Kajita, K. Kaneko, and H. Hirukawa, An analytical
method for real-time gait planning for humanoid robots, International
Journal of Humanoid Robotics, vol. 3, no. 01, pp. 119, 2006.
[3] S. Kajita, F. Kanehiro, K. Kaneko, K. Fujiwara, K. Harada, K. Yokoi,
and H. Hirukawa, Biped walking pattern generation by using preview
control of zero-moment point. in ICRA. IEEE, 2003, pp. 16201626.
[4] F. Xue, X. Chen, J. Liu, and D. Nardi, Real Time Biped Walking
Gait Pattern Generator for a Real Robot, in RoboCup 2011: Robot
Soccer World Cup XV Preproceedings. RoboCup Federation, 2011.
 0.45
0.5
Position x [m]
0.55
0.6
0.65
0.7
1.5 2 2.5 3 3.5 4 4.5
Time [s]
Fig. 9. The impact and the following oscillation along the x axis of the robot during the pendulum experiment. On the left, the feet positions in the world
coordinate system are denoted by flW and frW (left and right foot respectively). The measured position of the CoM is denoted by csnr x . On the right the
measured and the reference ZMP are visualized. The grey background indicates the support polygon. As can be seen, the measured ZMP stays within the
support polygon even after the immediate replanning for example at 2.5 s. This indicates that the modification was correctly chosen.
1.13Ns
26cm
(a) (b)
Fig. 10. The pendulum experiment. The robot is hit by a pendulum during
the walk.
(a)
Fig. 11. The robot walks slowly over a beam of 5 mm width and height
with both feet. The walk can be regarded as stable as long as cx is between
a and b. Walking further on beam leads to raising angles and a fall down.
[5] C. Mericli and M. Veloso, Improving biped walk stability using real-
time corrective human feedback, in RoboCup 2010: Robot Soccer
World Cup XIV. Springer, 2011, pp. 194205.
[6] S. Czarnetzki, S. Kerner, and O. Urbann, Applying dynamic walking
control for biped robots, in RoboCup 2009: Robot Soccer World Cup
XIII. Springer Berlin Heidelberg, 2010, pp. 6980.
[7] , Observer-based dynamic walking control for biped robots,
Robotics and Autonomous Systems, vol. 57, no. 8, pp. 839845, 2009.
[8] O. Urbann and S. Tasse, Observer based biped walking control, a
sensor fusion approach, Autonomous Robots, vol. 35, no. 1, pp. 37
49, 2013.
[9] S. Kajita, F. Kanehiro, K. Kaneko, K. Yokoi, and H. Hirukawa, The
3d linear inverted pendulum mode: a simple modeling for a biped
walking pattern generation, in Intelligent Robots and Systems, 2001.
Proceedings. 2001 IEEE/RSJ International Conference on, vol. 1,
2001, pp. 239 246 vol.1.
[10] J. Urata, K. Nshiwaki, Y. Nakanishi, K. Okada, S. Kagami, and
M. Inaba, Online decision of foot placement using singular lq preview
regulation, in Humanoid Robots (Humanoids), 2011 11th IEEE-RAS
International Conference on, oct. 2011, pp. 13 18.
0.45
flW psnr
frW pref
0.5
cx
csnr
Position x [m]
x
0.55
0.6
0.65
0.7
5 5.5 6 6.5 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5
Time [s]
[11] K. Nishiwaki and S. Kagami, Strategies for adjusting the zmp
reference trajectory for maintaining balance in humanoid walking, in
Robotics and Automation (ICRA), 2010 IEEE International Conference
on. IEEE, 2010, pp. 42304236.
[12] J. E. Pratt, J. Carff, S. V. Drakunov, and A. Goswami, Capture point:
27.5cm A step toward humanoid push recovery. in Humanoids. IEEE, 2006,
pp. 200207.
[13] J. Englsberger, C. Ott, M. Roa, A. Albu-Schaffer, and G. Hirzinger,
Bipedal walking control based on capture point dynamics, in In-
telligent Robots and Systems (IROS), 2011 IEEE/RSJ International
22cm
Conference on, sept. 2011, pp. 4420 4427.
[14] S.-K. Yun and A. Goswami, Momentum-based reactive stepping
controller on level and non-level ground for humanoid robot push
recovery, in Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ
International Conference on, sept. 2011, pp. 3943 3950.
[15] P. van Zutven, D. Kostic, and H. Nijmeijer, Foot placement for planar
bipeds with point feet, in Robotics and Automation (ICRA), 2012
IEEE International Conference on, may 2012, pp. 983 988.
[16] M. Morisawa, K. Harada, S. Kajita, K. Kaneko, J. Sola, E. Yoshida,
N. Mansard, K. Yokoi, and J.-P. Laumond, Reactive stepping to pre-
vent falling for humanoids, in Humanoid Robots, 2009. Humanoids
2009. 9th IEEE-RAS International Conference on, dec. 2009, pp. 528
534.
[17] M. Morisawa, F. Kanehiro, K. Kaneko, N. Mansard, J. Sola,
E. Yoshida, K. Yokoi, and J. Laumond, Combining suppression of the
cx a disturbance and reactive stepping for recovering balance, in Intelligent
b Robots and Systems (IROS), 2010 IEEE/RSJ International Conference
(b) on, oct. 2010, pp. 3150 3156.
[18] A. Herdt, H. Diedam, P.-B. Wieber, D. Dimitrov, K. Mombaur, and
M. Diehl, Online walking motion generation with automatic footstep
placement, Advanced Robotics, vol. 24, no. 5-6, pp. 719737, 2010.
[19] O. Urbann and M. Hofmann, Modification of foot placement for bal-
ancing using a preview controller based humanoid walking algorithm,
in Proceedings RoboCup 2014 International Symposium, Eindhoven,
Netherlands, 2014.
[20] T. Katayama, T. Ohki, T. Inoue, and T. Kato, Design of an optimal
controller for a discrete-time system subject to previewable demand,
International Journal of Control, vol. 41, no. 3, pp. 677 699, 1985.
[21] H. Diedam, D. Dimitrov, P.-B. Wieber, K. Mombaur, and M. Diehl,
Online walking gait generation with adaptive foot positioning through
Linear Model Predictive control. IEEE, 2008, vol. 24, no. 5, pp.
11211126.
[22] T. Laue, K. Spiess, and T. Rofer, SimRobot - A General Physical
Robot Simulator and Its Application in RoboCup, in RoboCup
2005: Robot Soccer World Cup IX, ser. Lecture Notes in Artificial
Intelligence, A. Bredenfeld, A. Jacoff, I. Noda, and Y. Takahashi, Eds.,
no. 4020. Springer, 2006, pp. 173183.
5331