ROTATION SETS FOR MAPS OF TORI

MICHAL MISIUREWICZ AND KRYSTYNA ZIEMIAN

Introduction
The notion of the rotation number of an orientation preserving homeomorphism
of a circle was introduced by Poincare in [8], and since then it has proved to be very
useful. It was generalized to the case of continuous maps of a circle of degree one by
Newhouse, Palis and Takens in 1979 [7]. In this case one gets a rotation interval. This
concept also is very useful. Therefore it seems natural to try to generalize this notion
to many-dimensional cases. This idea appears in the papers of Kim, MacKay and
Guckenheimer [5], Llibre and MacKay [6] and Herman [3]. Here we shall try to
proceed more systematically.
In Section 1 we present several possible definitions of the rotation set (and choose
the one we like best). In Section 2 we prove some properties of the rotation set which
hold for all continuous maps of tori homotopic to the identity in all dimensions (for
example the connectedness of the rotation set). In Section 3 we concentrate on
homeomorphisms in two dimensions and prove more properties of the rotation set
(for example its convexity). In Section 4 we present several examples and pose some
conjectures.
Progress in this area is very rapid. Soon after this paper was submitted for
publication, new results were obtained by Llibre and MacKay (a new version of [6])
and Franks [2]. We discuss these more closely in Section 4.

1. Definitions
The natural class of maps for our purpose will be continuous maps of an
AH-dimensional torus
T m = Um/Zm

into itself, homotopic to the identity. Since often the nice properties of continuous
maps in one dimension are shared by homeomorphisms in two dimensions but not all
continuous maps in two dimensions, we shall study also the class of homeomorphisms
of Jm, and especially of T 2 , homotopic to the identity.
As in the one-dimensional case, the rotation numbers and rotation sets depend
really not on the maps of the torus but on their liftings to Um. We shall denote the
class of liftings of all continuous maps of the torus T m homotopic to the identity by
# m and the class of liftings of all homeomorphisms of T m homotopic to the identity
by 3Vm. The elements of the class c€m are precisely these continuous maps F: Um -» Um
for which F(x + k) = F{x) + k for all keZ™. The class 3^m consists of all
homeomorphisms belonging to # m . We consider these spaces with the topology of
uniform convergence.

Received 20 July 1988; revised 17 January 1989

1980 Mathematics Subject Classification (1985 Revision) 54H20.
J. London Math. Soc. (2) 40 (1989) 490-506
 ROTATION SETS FOR MAPS OF TORI 491

We shall denote by n:Um -> T m the natural projection. If Fe<$m t h e n / : T m -> T m

will be the map of which F is a lifting (that is,fon = noF).
The straightforward generalization of the definition of the rotation set would be
the following. Let Fe^m, xeUm. Let p(F,x) be the set of all limits of convergent
subsequences of the sequence ((Fn(x)-x)/n)™_v Then let pp(F) = {JxeRmp(F,x)
(p stands for 'pointwise'). For ^ we get in such a way the whole rotation interval
(see [4]).
However, we shall mainly use another definition. The motivation is as follows.
The aim of the rotation set is to measure the average movement of any point. This
average movement is measured by finite parts of orbits, then passing with the lengths
of the parts to infinity. However, there is no special reason to fix a point before this
last step. Therefore we shall not do it and we shall take the limits of all convergent
sequences

The set obtained in such a way we shall call the rotation set of F and denote it by p(F).
Clearly, pv{F) c= p(F). If we want to have a more compact form of the definition of
p(F) we may notice that
= n u**(n o-o
where

*( ) - j £ *e j - (• )
Notice that since Fk(x)-x - Fk(x + l)-(x + l) for all leZm, we have also

where / = [0,1]. Therefore Kk(F) is compact.

For Fe%, by the definition of the rotation interval (written here [a, b]), we have
[a, b] a pp{F) c= p(F). It is known that a rational number l/k is larger than a
(respectively smaller than b) if and only if F\x) — x < /(respectively Fk{x)—x > t) for
all xeU. This is in turn equivalent to the inequality l/k > sup Kk(F) (respectively
l/k > infKk(F)). For every k there exists / such that b < l/k ^ b + (I/A:). Hence,
supKk(F) < b + (\/k). Ifk^n then supKk < b + (\/n). Therefore

sup |J Kk^b + -

and consequently supp(F) ^ b. Analogously infp(F) ^ a. Hence p(F) c= [a,b].

Consequently, for Fe^, p(F) is equal to the rotation interval in the usual meaning.
This shows that our definition indeed generalizes the one-dimensional notion of the
rotation set.
The set p(F) has better properties than the set pp(F). From (1.1) it follows
immediately that p(F) is closed. As we shall see, it is connected, and in the case of
Fe J^2, it is convex. A simple example [6] shows that pp(F) need not be connected even
if FeJff2.
We shall also define another notion related to p(F) and pp(F).
Let Fe<$n. We define a continuous map (f>:lm -> IRm by cj>(x) = F(y)-y, where
492 MICHAL MISIUREWICZ AND KRYSTYNA ZIEMIAN

yen~\x) (since F(y + k) = F(y) + fc for all keZm, F(y)—y does not depend on the
choice of yen~l(x)).
Denote the space of all/-invariant probabilistic measures on T m by Jt{f) and its
subspace consisting of ergodic measures by JfE(f). The space M{f) with the weak*-
topology is compact. If p,eJ(E{f) then, by the ergodic theorem,
1 n~1 f
-n E ^ o / * ~ * \<t>dfi /z-almost everywhere. (1.3)
Jfc-0 n-*oo J
On the other hand, if n{y) = x then
1 n~1 1n-1 Fn( v)-v
- E 0°/ fc w = L E (^fc+1(^)-n^)) =
W W
„ . (i.4)
*-0 fc-0 "
By (1.3) and (1.4), for //-almost every xelm and every yen'^x), the sequence
((Fn(y)-y)/n)n-i converges to j<j>dji. Therefore J</>d^iepp(F). Write

We have
(1.5)
Notice that if x is/-periodic and n(y) = x then /?(F,3;) = J <pdfix, where //x is the
probabilistic measure equidistributed on the orbit of x.

2. General properties of p(F)

m
For any subset A of U we shall denote by Conv (A) the convex hull of A (that
is, the smallest convex set containing A). If A a Um, e > 0, then we write
B(A, e) = {x e Um: dist (x, A) < e}
and
B(A, e) = {xeUm: dist (x, A) ^ e},
where dist(x,/4) = infV6/4 ||JC-J>|| and || • || is the Euclidean norm.
By /*(//) we denote the image of a measure p., that is, the measure such that
f f
\¥dfM= Wofdp.
J J
for all continuous functions y/: Jm -> R.
PROPOSITION 2.1. IfFe<gm,peZm andqeM then
(al) p(F°-p) = qp(F)-p,
(a2) p(F 9 -/>, x) = ^ 7)(F, x) -/> /or a// x £ Um,
(a3) Pp(F«-p) = qpp(F)-p,
(a4) pmes(F«-p) = qpmes(F)-p.
IfFetfm then
(bl) /KF-1) = -/>(i0,
(b2) ^ ( r 1 ) = -/> mes (F).

Proo/. If vepiF^—p) then there exist sequences («<)*! and (JC,)*! such that
nt -»• oo and ,„„ .„ , N
(fpr.W*
 ROTATION SETS FOR MAPS OF TORI 493
Then

so (v+p)/qep(F). This proves that p(F9-p) <= qp(F)-p.

Now let vep(F). Then there exist sequences (n^ and (x^ such that nt -> GO and

hm = v.

Setfc<= E{njq), where £(•) denotes the integer part. There exists a constant M
such that for all zeUm and O ^ r ^ g - l , || F"(z) - z || ^ M. Write nt-q'kt = r,
F^ix,) = z(. Then
FnjXi)-Xi -Xj zi-xi

qK qkt

Since IKF" r/qkt^0, , and

we get

<-»0O
Therefore

so qv—pep(Fg—p). Hence qp(F)—p a p{FQ—p). This completes the proof of (al).

Replacing x4 in the above proof by JC, we get the proof of (a2). By the definition
of /? p (), (a3) follows immediately from (a2).
Now we prove (a4). If we replace Fby Fq —p then the function (f> has to be replaced
by

Therefore, if vepmes(FQ—p) then there exists a measure neJtE{fQ) such that

9-1 r

j-0 J
The measure
19-1

v=- y
belongs to JfE(f). We have
v+p

so (v+p)/qepmes(F). This proves that pnJF'-p) c qpmes(F)-p.

Now let vep^^i 7 ). Then there exists a measure veJtE(f) such that j<f>dv = v.
The measure v is invariant for/ 9 , but not necessarily ergodic. However, it is known
that for some r dividing q and fi€J£B(f), we have

j-0
494 MICHAL MISIUREWICZ AND KRYSTYNA ZIEMIAN

Hence v = (\/q) YX-lft^1) anc* therefore

Thus qv — pGpmes(P' — p). This shows that q-pmes(F)—p c p^iF^—p) and completes
the proof of (a4).
Now assume that FeJ^m. The statement (bl) follows immediately from the
equality
Fni) (F-

where y{ = Fni(xt). The statement (b2) follows immediately from the equality
-^EH'1) = -^E(/) a n d t n e observation that if we replace F by F'1 then 0 will be
replaced by — <f>of~l.

LEMMA 2.2. //Fe<&m then p(F) c Conv(Kn(F)) for all neN.

Proof. Fix n e N. Let O: Um -> U be a linear functional. Take ke N and z e Rm.

We have
<D(F*(z) - z) = £ 0>(F(i+1) n(z) - /^ n (z)) + O ^ C z ) - Fln(z)),
i-0

where I = E(k/n). For each y = 0,1, ...,n-\ the set { ^ ( J C ) - x : x e R " 1 } is bounded
] i i
since F (x) — x=^ fzl^(n(F (x))) and <p is continuous. Therefore there exists a
constant M such that 0>(F'(;t)-*) ^ M for all x e R m , y = 0 , 1 , . . . , « - 1 . We obtain

Consequently

Since l i m ^ ^ In/k = 1 (because / = E{k/n)) and M, n, sup © ( ^ ( F ) ) do not depend on

k, we obtain
limsup(sup<D(/(:t(F)) ^ sup ®(Kn(F)).
k-*oo

Hence, by the definition of p(F),

sup O(/>(F)) ^ sup (<t>(Kn(F))). (2.1)

The set Conv (Kn(F)) is convex and closed. Therefore for every point ye lRm which
does not belong to Conv (Kn(F)), one can find a linear functional <J> such that
<D(y) > sup<X>(Conv(#n(F))).

Since for any subset A of Um

sup <D(Conv (A)) = sup <b(A),
we obtain by (2.1) that

and hence v$p(F). This proves that />(.F) <= Conv (Kn(F)).
 ROTATION SETS FOR MAPS OF TORI 495
THEOREM 2.3. If Fe^m then the rotation set p(F) is connected.

Proof. By Lemma 2.2, p(F) is contained in a compact set K^F). Since it is closed,
it is also compact.
Suppose that p{F) is not connnected. Then there exist disjoint sets A,B c p(F)
such that p(F) = A [) B and both A and B are closed in p(F). Hence they are also
compact. Therefore there exist open neighbourhoods Uand VofA and B respectively
such that U and V are compact and disjoint. Set S = dist (C/, V) > 0. Take veA,
weB. By the definition of p(F), there exist sequences of integers nt -*• oo, kt -*• oo
and sequences of points of IRm, (x<)£0, {y$.o such that

lim - , and li

The set /^(F) is compact and hence there exists M such that \\F(t) —1\\ ^ M for all
felRm. Therefore if we take zelRTO and write / = Fn(z) then

\F»+\z)-z F«(z)-z II \\F(t)-z t-z

n+\ n+l
F(t)-z F(t)-z ||F(Q-z t-z\\
n n
n+l n II II
+
1 M l(\\F» \z)-z
n n+l) n n\\\ n+l
n+1
Since ||F (z)-z|| + ... + ||F(z)-z|| ^(n+\)M, we get
n+1
F (z)-z F\z)-z II 2M
w n n
II +11 IIII ^
Since p(F) is contained in an open set U [} V, there exists ./V such that

for all n ^ JV and zelR7". Take i such that 2M/nt <d, n^ N, (Fn<(x() - xt)/n{ e U.
Then takey such that kt ^ nv and {Fki{y^—y^/k}e V. Since

n+l n II
n
for all n ^ n{, we get {F {x^-x^/neUfor all n ^ n<. In particular, {F^ix^—x^/k^e U.
Take a curve joining x4 with yy Its image under (FfcJ — id)/fcy joins (F^x^ — x^/kje U
with (F^y^—y^/kje V. This contradicts the assumption that (Fn(z) — z)/neU U F"
for all « ^ Af and zelR m . Hence p(F) is connected.

THEOREM 2.4. If F e #m Conv (p(F)) = Conv (pmes(F)).

Proof. Since p mes (F) c= /?(F), it is enough to show that the extremal points of the
set Conv (p(F)) belong to pmes(F). Let v be an extremal point of Conf (p(F)). Then
vep(F) and hence there exist sequences nt -*• oo and yteUm such that

as oo.
496 MICHAL MISIUREWICZ AND KRYSTYNA ZIEMIAN

Let xt = n{y}). Set

l *
L
"i fc-0
where Sz denotes the probabilistic measure concentrated at z. From the sequence
(/0£i w e c a n choose a subsequence weakly*-convergent to some probabilistic
measure n on T m . For every continuous function ^ : T m -+ U, since

\vd(JM)- lyd/ii
'<
we have JV^(/*(/*)) = j y/d/i. Therefore f+{p) = fi; that is, y, is/-invariant. By the
continuity of 0 we have

Udfi = lim Ud/it = l i m ^ ^

n
J i-*aoJ i->ao i

We can decompose the measure pi into ergodic components. This means that there
exists a probabilistic measure v on JtE(J) such that for every continuous function

((/dfi=

v = JV dii = J( J^rf^)^(^ B ). (2.2)

Notice that for every ^ £ e ME{f) we have

</>dfiE€Pmes(F) cz p(F) c Conv(p(F)). (2.3)

/•
From (2.2), (2.3) and the assumption that u is an extremal point of Conv(p(.F)) it
follows that for v-almost every measure nE e ME(f) we have J 0 dpiE = v. Therefore
vepmea(F). This completes the proof.

REMARK 2.5. Since ergodic measures are the extremal points of M(J~), we have

Conv(p mes (F)) =

The right-hand side of the above equality is Herman's definition of the rotation
set, see [3].
In view of (1.5) we obtain the following corollary to Theorem 2.4.

COROLLARY 2.6. IfFe<gm then Conv(p(F)) = Conv(pp(F)).

Since for every n e M{f) almost every point is non-wandering, we obtain the next
corollary.

COROLLARY 2.7. If Fe <&m then Conv (p(F)) = Conv ((J p(F, x)), where the union
is taken over all xen~\Cl(f)).

LEMMA 2.8. For every Fe<tfm and e > 0 there exists n0 such that for all n^ n0 we
have Kn(F) c B(Conw(p(F)),e).
 ROTATION SETS FOR MAPS OF TORI 497

Proof. Suppose that this is false. Then there exist Fe# m , e > 0 and sequences
nt -*• oo, vt e Kn (F) such that vt $ /?(Conv (p(F)), e). By choosing a suitable subsequence
one can assume that the sequence («<)*! converges. Then its limit belongs to p(F) (by
the definition of p(F)) but does not belong the B(Conv (p(F)), e); a contradiction.
This completes the proof.

PROPOSITION 2.9. //Fe<&m then Conv(p(F)) = f]n>1 Conv(Kn(F)).

Proof. By Lemma 2.2.

Conv 0>(F))c= D Conv (*„(/•)).

By Lemma 2.8, for every e > 0, if n is sufficiently large then

Conv^Jc^Conv^F))^).
Therefore
D C o n v ^ F ) ) a Conv(/>(F)).

THEOREM 2.10. 7Vi6? function Conv (/>( •)) from <$m into the space of subsets of Um
is upper semicontinuous; that is, for every Fe# m ande > 0 there exists a neighbourhood
V of F in (€m such that for every GeV we have
Conv (/>((?)) c B(Conw(p(F)),e).

Proof. For given Fe<$m and e > 0, by Lemma 2.8 there exists an integer n such
that ConvCKn(F)) c £(Conv (p(F)), |e). There exists a neighbourhood V of F in tfm
such that, for every GeV,
Conv (Kn(G))
Then, in view of Lemma 2.2,
p{G) c= Conv (Kn(G)) c B(Conv(p(F)),e).
As was noticed by M. Herman, Theorem 2.10 follows also from Theorem 2.4,
Remark 2.5 and the upper semicontinuity of the map/V-+./#(/).

REMARK 2.11. It is easy to see that from Theorem 2.10 it follows that for each
jceRm, the set {Fe%>m:xe Conv (p(F))} is closed. As was noticed by M. Herman, this
fact can be also deduced directly from Theorem 2.4 and Remark 2.5.

3. Properties of p(F) for Fe3f2

In this section we assume that Fe 34?2 and prove stronger properties of p(F), in
particular that p(F) is convex (Theorem 3.4(b)).
The following lemma is basic to our further considerations. If a,beU2, we shall
denote the open straight line segment joining a with b by ab (and the closed segment
by HE).

LEMMA 3.1. Let a,beU2, a^b. Let T be a translation by a vector v. Let y be a

simple curve joining a with b, not intersecting ab. Assume that y does not intersect T(y).
Then T(a) does not belong to the open region D bounded by ab U y.
498 MICHAL MISIUREWICZ AND KRYSTYNA ZIEMIAN

Proof (due to A. Douady). We regard y as a continuous map from / into U2.

Without any loss of generality we may assume that a = 0. Since y does not intersect
T(y) = y + v, the equation y(s) = y(t) + v has no solutions (s, t)eP. Consider the map
V:P^U2 given by
V(t) ( ) ( t )
Then . Define a closed curve a by

a(0 = (l,3f—1) for/e[|,§],

(3-3/, 3 - 3 0 for*e[§,l]
(see Figure 3.1). The curve is the concatenation of the curves y and b — y (see
Figure 3.2).
0,1)

(0,0) (1,0)
FIG. 3.1

-7
FIG. 3.2

Set pit) = tb for tel. Then ¥(<x) is the concatenation of two closed curves
S = y*fi~x and e = ft* (b — y). By the assumptions, the curve S is simple. If veD then
the index indv(<5) of S with respect to v is either +1 or — 1. Since e is symmetric to 3
with respect to the point \b, e is also simple and either indv(e) = indv(S) (if b — veD)
or ind^e) = 0 (if b-v$D). Therefore
 ROTATION SETS FOR MAPS OF TORI 499

Consequently, v is the image of some point of the triangle bounded by a, a

contradiction. Hence, T(a) = v$D.

LEMMA 3.2. Let a,beU2, a # b. Let y be a simple curve joining a with b, not
intersecting ab. Assume that n(y) is a simple curve. Then
(a) n(a)$7t(D), where D is as in Lemma 3.1,

Proof. Suppose that n(a)e7i(D). Then there exists ceD such that the vector c — a
has integer coordinates. Call T the translation by c — a. Then the assumptions of
Lemma 3.1 are satisfied and we obtain a contradiction. This proves (a).
Next we prove (b). Again, by making a small rotation if necessary, we may assume
that ab is neither horizontal nor vertical. To show (b) it is enough to prove for every
open square with a side of length 1 and the vertices belonging to the lattice a + Z 2 , that
if it intersects the segment ab then it intersects y. If such a square intersects ab, then
it intersects D, but its vertices do not belong to D (by (a)). Since a straight line cannot
separate an interior point of a square from all its vertices, at least one of these vertices
has to be separated by y. Hence, y has to intersect the closure of this square. This
completes the proof.

LEMMA 3.3. IfGe^ then Conv(G(/ 2 )) c B(G(P), y/2).

Proof. Let x e Conv (G(P))\G(P). Look at the set A of all elements 0 of the unit
circle S1 such that the ray {x + tO: t ^ 0} intersects G(P). Since G(P) is connected and
compact, so is A. Hence, since JCe Conv (G(P)), A has to contain a closed semicircle.
This implies that there exist a,beG(P) such that xeab and that ab is disjoint from
G(P). We have a = G{y) and b = G(z) for some y,zeP. The curve y = G(yz) satisfies
the assumptions of Lemma 3.2, unless n(y) = n{z). In this exceptional case, in any
neighbourhood of b there exists b' = G(z') # b such that ab' is disjoint from G(P)
(because G(P) = int (G(P))). Then we replace b by b', z by z' and x by x', where x'
is the point of ab' closest to x. From Lemma 3.2(b), x (or x') belongs to B(G{P), y/2).
Since b' could be chosen arbitrarily close to x, x' is arbitrarily close to x. Hence in
both cases we get xe B(G(A), -\/2).

THEOREM 3.4. Let FeJ^2. Then

(a)
(b) p(F) is convex,
(c) for every vep(F) there exists a sequence (xn)*_1 such that

hm — ^ - ^ = v,

(d) Kn(F) converges to p(F) in the Hausdorff metric as n -> oo.

Proof. From Lemma 2.2 we get

fl Conv (Kn(F)). (3.0)

From the definition of Kn(F) we have

500 MICHAL MISIUREWICZ AND KRYSTYNA ZIEMIAN

Therefore
5 /Conv (*"(/•)) V2\ ( 3 1}

n n
From Lemma 3.3,
(3.2)
n \ n
Again from the definition of Kn(F) we have

(3.3)

From (3.1), (3.2) and (3.3) we obtain

3
Conv (Kn(F)) c B\ Kn(F), -^\ (3.4)

n(F)) then by (3.4) there exists a sequence (JCB)"_I such that

3V2
—v
Then
|jm F {xj-x. = s (3 5)

n-»oo '

and hence vep(F). This proves that P|nS5lConv(£nCF)) cr p(F) and together with
(3.0) proves (a). Moreover, we have (3.5) for all vep(F), and this proves (c). Then (b)
is an immediate corollary to (a).
Let e > 0. From (3.4) and Lemma 2.2 it follows that if n is sufficiently large then
p{F) cr B(Kn(F), e). From (b) and Lemma 2.8 it follows that if n is sufficiently large
then Kn(F) c B(p(F),e). This proves (d).

In view of Theorem 3.4(b), for FeJf2 we can get corollaries to Theorem 2.4,
similar to Corollaries 2.6 and 2.7.

COROLLARY 3.5. If FE tf2 then p(F) = Conv (pmes(F)) = Conv (pp(F)).

COROLLARY 3.6. IfFe Jf?2 then p(F) = Conv ((J p(F, x)), where the union is taken
over all xen-\Q.(J)).

From Theorem 3.4(b), Theorem 2.10 and Remark 2.11 we obtain the following
corollaries.

COROLLARY 3.7. The function pfrom Jf2 into the space of subsets ofU2 is upper
semicontinuous.

COROLLARY 3.8. For each XEU2, the set {FsJ^2:xep(F)} is closed.

Now we consider the problem of existence of periodic orbits with various rotation
points (we avoid the name 'rotation vector' since this may lead to misunderstanding,
for instance when speaking about collinearity).
 ROTATION SETS FOR MAPS OF TORI 501

For Fe<£m, if xeJm is a periodic point of/and yen'^x), then the set p(F,y)
consists of one point.
The following theorem was proved by Franks. We state it using our notation.

THEOREM 3.9 [1, Theorem 3.5]. IfFe Jf2 and 0epmes(F) then F has afixedpoint.

As a corollary to this theorem, we obtain the following.

THEOREM 3.10. If FeJf 2 and (r/q,s/q)epmes(F), where r,seZ, qeN and the
greatest common divisor of r,s,q is 1, then there exists an f-periodic point xeJ2 of
period q such that p(F,y) = {(r/q,s/q)}for y£n~x{x).

Proof. By Proposition 2.1(a4), 0epmes(FQ-(r,s)). Therefore FQ-(r,s) has a

fixed point z. Clearly, p(FQ - (r, s), z) = {0}. By Proposition 2.1(a2),

and if x = n(z), yen^ix) then also p(F,y) = {(r/q,s/q)}. We have/ 9 (x) = x, so x is

periodic for / o f some period q' dividing q. Then Fq\z) = z + (r',s') for some r', i'eZ
and hence

Therefore r = qr'/q', s = qs'/q' and q/q' divides r, s and q. Hence q/q' = 1; that is,

From Theorem 3.10 we obtain the following immediately.

COROLLARY 3.11. If Fe3V2 then pmes(F) n O2 = \Jp(F,y), where the union is

taken over all y such that n{y) is periodic for f

4. Examples and conjectures

In this section we shall state some conjectures and construct examples showing
that some obvious generalizations of these conjectures (and other results) are false.
The recent results of Franks, Llibre and MacKay confirmed two of our original
conjectures, so we renamed them as theorems (4.3 and 4.6). The result of Franks [2]
is exactly Theorem 4.6. Llibre and MacKay proved, among other results, the
following theorem [6, Theorem l(iv)].

THEOREM 4.0 [6]. IfFe Jf2, JC15 x2, xz e U2 are the points such that nix^ is periodic,
p(F,xt) = {v^ for i= 1,2,3 and the points v1,v2,v3 are not collinear, then for every
y^yg} there exists xeM2 with p{F,x) = {v}.

From this and Theorem 4.6, Theorem 4.3 follows immediately. Let us mention
also that one can easily deduce Theorem 3.4(b) from Theorems 4.0 and 4.6 in the case
when p(F) is not contained in a straight line. However, our proof is much more
elementary and straightforward.
Let us show now some counterexamples. Notice first that [6, Example 1] shows
 502 MICHAL MISIUREWICZ AND KRYSTYNA ZIEMIAN

that for Fe3V2, the set pp(F) can consist of two points and therefore we cannot hope
that p(F) = pp(F).
Another example, [6, Example 4] shows that there exists Fe Jf3 for which p(F) is
not convex. In particular, we cannot then have p(F) = Conv (pp(F)). However, in this
example p(F) has empty interior. We shall build two examples, one for %, another for
Jf3 (it will be obvious that similar examples can be constructed for all J^m, m ^ 3),
where p{F) is not convex, but has non-empty interior. Moreover, even the closure of
the interior of p(F) is not convex.

EXAMPLE 4.1. Fix two disjoint balls of radius ±, say U and V in T2. In each of
them choose a closed straight-line segment, called J and AT respectively. Then it is clear
that by a continuous deformation of the identity we can obtain a continuous map /
of T2 such that (see Figure 4.1.)
(a) / i s equal to the identity outside U [) V,
(b) the lifting F of/does not move any point of n~\U) more than by \ to the
left or down and more than by f to the right or up, and does not move any
point of n~\V) more than by f to the left or down and more than by \ to the
right or up,
(c)/(t/)n v=f(V)o u=0,
(d) Fstretches a component / o f n~\J) and Kof n~\K) mapping them through
/, /+(0,1), / + ( l , 0 ) and K, K-(0,1) and K-(\,0) respectively.

A component of n '(V)

FIG. 4.1

By (a) and (c),/"(£/) n V=fn(V) n U = 0 for all n. Therefore by (b),

n(F\
P\r
) ca i\ —4» £12 uII LF—
i d £ 14J
— 45 12

(see Figure 4.2). On the other hand, by (d) a horseshoe effect is created and we can find
trajectories of F which start at /, then stay in / for time q, move to /+(l,0),
/ + (2,0),...,/+ (r, 0) in the next r steps and to / + (r, 1),..., / + (r, s) in the next s steps.
They can even be made inverse images under n of periodic/-orbits. In such a way we
see that />P(.F) (and therefore p(F)) contains the triangle A with vertices (0,0), (1,0) and
(0,1). Analogously, replacing JbyK and + by - , we see that pp(F) contains - A .
Therefore pp(F) (and hence p(F)) has non-empty interior, but p(F) is not convex (see
Figure 4.2).
 ROTATION SETS FOR MAPS OF TORI 503

An example Set containing p(F)

of a point which
belongs to
Conv(p(F))V(F)

Set contained in p(F)

FIG. 4.2

EXAMPLE 4.2. One can make a construction analogous to that of Example 4.1 to
get an element of Jf3 with similar properties. The segments J and K have to be
replaced by cubes. Then / h a s to be stretched by Fin one dimension and contracted
in the other two dimensions and mapped through /, / + (1,0,0), /+(0,1,0) and
/+(0,0,1), in a horseshoe-like manner (see Figure 4.3).

FIG. 4.3

The sets K and F(K) can be obtained from / and F(J) respectively, by a central
symmetry, as in Example 4.1. Since our map has to be a homeomorphism, we cannot
make it in such a way that it is the identity outside small neighbourhoods of K and
J. However, we can make it the identity outside some disjoint neighbourhoods U and
V of J and K respectively, which look like 3-dimensional grids (see Figure 4.4 for the
inverse image under n of one of these neighbourhoods). Obviously, two such grids can
be made disjoint in three dimensions. We can make our construction in such a way
that a property analogous to (b) of Example 4.1 also holds.
Hence, in our example, pp(F) (and hence p(F)) contains the union of two
tetrahedrons with vertices (0,0,0), (1,0,0), (0,1,0), (0,0,1) and (0,0,0), (-1,0,0),
(0, —1,0), (0,0, — 1) respectively. On the other hand, p{F) is contained in the union
of cubes [ - i f ] 3 and [-f,|] 3 . Therefore, as in Example 4.1, p(F) is not convex.
504 MICHAL MISIUREWICZ AND KRYSTYNA ZIEMIAN

FIG. 4.4

THEOREM 4.3 [6, 2]. IfFeJf2 then int (p(F)) c pp(F).

The next two examples show that in this theorem one cannot replace JV2 by
either % or JVZ.

EXAMPLE 4.4. We make a small modification of Example 4.1. Instead of (c), we

require only that f(V) n U = 0, and we require in addition that F maps / also
through some component of n~\K) (see Figure 4.5). We get now Fe%. In this new
situation we shall not get any new asymptotic behaviour of trajectories (there is a
transition from n~\U) to n~\V) but not back). Therefore pp(F) remains contained in
[ - i f ] 2 U [—1,|]2. On the other hand, for s,teN there arefinitepieces of trajectories
travelling for some time t inside n~\U) and then switching to n~\V) and travelling
there for time s. For these trajectories we can first get close to any point aeA with
(P(x)—x)/t and then close to any point be —A with (F'(y)-y)/s, where y = F+1(x).
Therefore we get the point (ta + sb)/(t + s) in p(F). Hence Conv(A U (-A)) c: p(F),
and consequently int (p(F)) is not contained in pp(F) (see Figure 4.6).

A component of iC' (V)

A component
of7Tl(U)

FIG. 4.5
 ROTATION SETS FOR MAPS OF TORI 505

Set containing pJF)

I Set contained in p(F)

FIG. 4.6

EXAMPLE 4.5. One can modify Example 4.2 in the same way as we modified
Example 4.1 to get Example 4.4. Then for the obtained F e ^ , int (p(F)) is not
contained in pp(F), by the same reasons as in Example 4.4.

THEOREM 4.6 [2]. IfFe J^2, (p/q, r/q) e int (p(F)) and the greatest common divisor
ofp, q, r is 1 then there exists xe U2 such that the point n(x) is periodic of period q and

From this theorem it follows that

int (p(F))0 Q2 cz(j p(F,x), (4.1)
where the union is taken over all x such that n(x) is periodic for / .

CONJECTURE 4.7. If Fs JV2 then int (p(F)) c pmes(F).

This conjecture is stronger than Theorem 4.3, and by Theorem 3.10, stronger than
Theorem 4.6.
Examples 4.4 and 4.5 show that Theorem 4.6, Conjecture 4.7 and formula (4.1) are
false when j(?2 is replaced by % or ^ 3 . It is easy to see that the analogous examples
can be constructed also in higher dimensions. They can also be made smooth, even
of class C°°.
We conclude with an example showing that p as a function from jf?2 to the space
of subsets of U2 is not lower semicontinuous. This example is a small modification of
[6, Example 1].

EXAMPLE 4.8. The map F is the time-1-map of a vector field which is (a,0) on
horizontal lines with the second coordinate from Z and (b, 0) on horizontal lines with
the second coordinate from f + Z {a # b) and outside these lines the second coordinate
of this vector field is positive (see Figure 4.7). Since Q ( / ) consists of two circles and
on one of them we have the rotation by (a, 0) and on the other one the rotation by
(6,0), by Corollary 3.6 we obtain p(F) = [a, b] x {0}. By an arbitrary small
perturbation (even in the C°°-topology, if F was of class C00) we can destroy one
of these circles. For the new map G we will have Q(g) consisting of one circle and
on it g is the rotation by (a,0) (or (b,0)). Therefore by Corollary 3.6, p{G) = {(a,0)}
(or {(6,0)}).
506 ROTATION SETS FOR MAPS OF TORI

FIG. 4.7

Acknowledgements. This paper was written during the visit of the authors to the
University of Warwick and Durham Symposia on Dynamical Systems 1988 (including
the travel from Coventry to Durham when Theorem 2.4 was proved). The authors
gratefully acknowledge the hospitality and support of the Nonlinear Systems
Laboratory and the Durham conference.
We are also grateful to Jaume Llibre for posing a problem from which our
research started, to Robert MacKay, John Franks and Michael Herman for helpful
discussions, and to Adrien Douady for a short proof of Lemma 3.1 which replaced
our long one.
References
1. J. FRANKS, 'Recurrence and fixed points of surface homeomorphisms', Ergod. Theory Dynam.
Systems 8* (1988) 99-107.
2. J. FRANKS, 'Realizing rotation vectors for torus homeomorphisms', Trans. Amer. Math. Soc. 311
(1989) 107-116.
3. M.R.HERMAN, 'Existence et non existence de tores invariants par des diffeomorphismes
symplectiques', preprint, Palaiseau 1988.
4. R. ITO, 'Rotation sets are closed', Math. Proc. Cambridge Philos. Soc. 89 (1981) 107-111.
5. S. KIM, R. S. MACKAY and J. GUCKENHEIMER, ' Resonance regions for families of torus maps',
Nonlinearity 2 (1989) 391^K)4.
6. J. LLIBRE and R. S. MACKAY, ' Rotation vectors and entropy for diffeomorphisms of the torus
isotopic to the identity', preprint, Warwick 1988.
7. S. NEWHOUSE, J. PALIS and F. TAKENS, ' Bifurcations and stability of families of diffeomorphisms',
Publications Mathematiques 57 (Institut des Hautes Etudes Scientifiques, Paris, 1983) 5-71.
8. H. POINCARE, Oeuvres completes, tome 1 (Gauthier-Villars, Paris, 1952) 137-158.

Institute of Mathematics
Warsaw University
PKiN IX p.
00-901 Warszawa
Poland