Compact Connected Lie Transformation Groups on Spheres With Low Cohomogeneity - II

Selected Titles in This Series 595 Eldar Straume, Compact connected Lie transformation groups on spheres with low

cohomogeneity, II, 1997 594 Solomon Friedberg and Herve Jacquet, The fundamental lemma for the Shalika subgroup

of GL(4), 1996 593 Ajit Iqbal Singh, Completely positive hypergroup actions, 1996 592 P. Kirk and E. Klassen, Analytic deformations of the spectrum of a family of Dirac

operators on an odd-dimensional manifold with boundary, 1996 591 Edward Cline, Brian Parshall, and Leonard Scott, Stratifying endomorphism algebras,

1996 590 Chris Jantzen, Degenerate principal series for symplectic and odd-orthogonal groups,

1996 589 James Damon, Higher multiplicities and almost free divisors and complete intersections,

1996 588 Dihua Jiang, Degree 16 Standard L-function of GSp(2) x GSp(2), 1996 587 Stephane Jaffard and Yves Meyer, Wavelet methods for pointwise regularity and local

oscillations of functions, 1996 586 Siegfried Echterhoff, Crossed products with continuous trace, 1996 585 Gilles Pisier, The operator Hilbert space OH, complex interpolation and tensor norms,

1996 584 Wayne W. Barrett, Charles R. Johnson, and Raphael Loewy, The real positive definite

completion problem: Cycle completability, 1996 583 Jin Nakagawa, Orders of a quartic field, 1996 582 Darryl McCollough and Andy Miller, Symmetric automorphisms of free products, 1996 581 Martin U. Schmidt, Integrable systems and Riemann surfaces of infinite genus, 1996 580 Martin W. Liebeck and Gary M. Seitz, Reductive subgroups of exceptional algebraic

groups, 1996 579 Samuel Kaplan, Lebesgue theory in the bidual of C(X) , 1996 578 Ale Jan Homburg, Global aspects of homoclinic bifurcations of vector fields, 1996 577 Freddy Dumortier and Robert Roussarie, Canard cycles and center manifolds, 1996 576 Grahame Bennett, Factorizing the classical inequalities, 1996 575 Dieter Heppel, Idun Reiten, and Sverre O. Smal0, Tilting in Abelian categories and

quasitilted algebras, 1996 574 Michael Field, Symmetry breaking for compact Lie groups, 1996 573 Wayne Aitken, An arithmetic Riemann-Roch theorem for singular arithmetic surfaces,

1996 572 Ole H. Hald and Joyce R. McLaughlin, Inverse nodal problems: Finding the potential

from nodal lines, 1996 571 Henry L. Kurland, Intersection pairings on Conley indices, 1996 570 Bernold Fiedler and Jiirgen Scheurle, Discretization of homoclinic orbits, rapid forcing

and "invisible" chaos, 1996 569 Eldar Straume, Compact connected Lie transformation groups on spheres with low

cohomogeneity, I, 1996 568 Raul E. Curto and Lawrence A. Fialkow, Solution of the truncated complex moment

problem for flat data, 1996 567 Ran Levi, On finite groups and homotopy theory, 1995 566 Neil Robertson, Paul Seymour, and Robin Thomas, Excluding infinite clique minors, 1995 565 Huaxin Lin and N. Christopher Phillips, Classification of direct limits of even Cuntz-circle

algebras, 1995 (Continued in the back of this publication)

Compact Connected Lie Transformation Groups

on Spheres with Low Cohomogeneity, II

This page intentionally left blank

MEMOIRS -LIT A 0f the

American Mathematical Society

Number 595

Compact Connected Lie Transformation Groups

on Spheres with Low Cohomogeneity, II

Eldar Straume

January 1997 • Volume 125 • Number 595 (first of 5 numbers) • ISSN 0065-9266

American Mathematical Society Providence, Rhode Island

1991 Mathematics Subject Classification. Primary 57S15; Secondary 57R60, 22E47.

Library of Congress Cataloging-in-Publication D a t a Straume, Eldar.

Compact connected Lie transformation groups on spheres with low cohomogeneity, II / Eldar Straume.

p. cm.—(Memoirs of the American Mathematical Society, ISSN 0065-9266 ; no. 595) "January 1997, volume 125, number 595 (first of 5 numbers)." Includes bibliographical references. ISBN 0-8218-0483-9 (alk. paper) 1. Topological transformation groups. 2. Homology theory. I. Title. II. Series.

QA3.A57 no. 595 [QA613.7] 510 s—dc20 [514] 96-44760

CIP

Memoirs of the American Mathematical Society This journal is devoted entirely to research in pure and applied mathematics.

Subscription information. The 1997 subscription begins with number 595 and consists of six mailings, each containing one or more numbers. Subscription prices for 1997 are $414 list, $331 institutional member. A late charge of 10% of the subscription price will be imposed on orders received from nonmembers after January 1 of the subscription year. Subscribers outside the United States and India must pay a postage surcharge of $30; subscribers in India must pay a postage surcharge of $43. Expedited delivery to destinations in North America $35; elsewhere $110. Each number may be ordered separately; please specify number when ordering an individual number. For prices and titles of recently released numbers, see the New Publications sections of the Notices of the American Mathematical Society.

Back number information. For back issues see the A MS Catalog of Publications. Subscriptions and orders should be addressed to the American Mathematical Society, P. O. Box

5904, Boston, MA 02206-5904. All orders must be accompanied by payment. Other correspondence should be addressed to Box 6248, Providence, RI 02940-6248.

Copying and reprinting. Individual readers of this publication, and nonprofit libraries acting for them, are permitted to make fair use of the material, such as to copy a chapter for use in teaching or research. Permission is granted to quote brief passages from this publication in reviews, provided the customary acknowledgment of the source is given.

Republication, systematic copying, or multiple reproduction of any material in this publication (including abstracts) is permitted only under license from the American Mathematical Society. Requests for such permission should be addressed to the Assistant to the Publisher, American Mathematical Society, P. O. Box 6248, Providence, Rhode Island 02940-6248. Requests can also be made by e-mail to [email protected].

Memoirs of the American Mathematical Society is published bimonthly (each volume consisting usually of more than one number) by the American Mathematical Society at 201 Charles Street, Providence, RI 02904-2294. Periodicals postage paid at Providence, RI. Postmaster: Send address changes to Memoirs, American Mathematical Society, P. O. Box 6248, Providence, RI 02940-6248.

© 1997 by the American Mathematical Society. All rights reserved. This publication is indexed in Science Citation Index®, SciSearch®, Research Alert®,

CompuMath Citation Index®, Current Contents®/Physical, Chemical & Earth Sciences. Printed in the United States of America.

@ The paper used in this book is acid-free and falls within the guidelines established to ensure permanence and durability.

10 9 8 7 6 5 4 3 2 1 02 01 00 99 98 97

Table of Contents

Introduction 1

Chapter I : Organization of orthogonal models and orbit structures 5 § 1 : A review of basic notions and results 5 §2 : Orbit structures of linear groups of cohomogeneity < 3 6

Chapter II : Orbit structures for G-spheres of cohomogeneity two 15 §1 : Weight patterns and calculation of orbit types 15 §2 : Simple weight patterns and the proof of Theorem D 18

Chapter III : The Reconstruction Problem 22 § 1 : G-diffeomorphisms of manifolds of cohomogeneity one 23 §2 : G-disk bundles of cohomogeneity two and equivariant attaching . . 31

Chapter IV : G-spheres of cohomogeneity two with at most two isolated orbits . 34 § 1 : Equivariant twisting of the orthogonal model 34 §2 : The basic lens spaces L^ as 3-dimensional models 38 §3 : Construction of G-spheres with lens spaces L^ as reduction . . . . 44

Chapter V : G-spheres of cohomogeneity two with three isolated orbits . . . . 56 §1 : Bad and good vertices in the orthogonal model 56 §2 : More examples of exotic G-spheres 64

Figures 72

References 74

vii

Abstract

The cohomogeneity of a transformation group (G, X) is, by definition, the dimension of its orbit space, c = dim X/G. We are concerned with the classification of differentiable compact connected Lie transformation groups on (homology) spheres, with c < 2, and the main results are summarized in five theorems, A, B, C, D and E. This paper is part II of the project, and it is devoted to the last two theorems. The first three theorems are proved in part I, which appeared as volume 119, number 569, in the January 1996 Memoirs.

The subfamily of orthogonal transformation groups on standard spheres constitute the "orthogonal models", which already exhibit a great amount of variation of orbit structures. However, non-orthogonal or "exotic" G-spheres also exist for c = 1 and c = 2. In part I there is a complete and new proof of the classification for the case c = 1, using the geometric weight system as a unifying tool. The geometric weight system is also determined for all G-spheres with c = 2.

A G-sphere X n with c = 2 has a unique orthogonal model (G, Sn), by Theorem A. Namely, the geometric weight system of (G, Xn) coincides with the (classical) weight system of (G, Sn). As a consequence of this they must also have the same orbit structure (Theorem D). Finally, we are left with the Reconstruction Problem, which amounts to the construction of all G-spheres having the orbit structure of a given orthogonal model. The basic technique is equivariant "twisting" of the orthogonal model, which is reminiscent of the exotic sphere construction used by Milnor in the late 1950's, combined with a reduction principle valid for compact Lie transformation groups in general. The existence of "exotic" G-spheres is summarized by Theorem E. The classification results are exhaustive, with very few exceptions.

Key words and phrases. Compact Lie transformation groups, low cohomogeneity, homotopy spheres, equivariant twisting, reduction principle, geometric weight system.

1991 Mathematics Subject Classification. Primary 57S15; Secondary 57R60, 22E47.

vm

Introduction

Let G be a compact connected Lie group acting on a manifold M. The cohomogeneity (or the degree of intransitivity) of (G, M) is, by definition, the codimension of the principal orbits, or equivalently, the dimension of the orbit space M/G. In this paper we are concerned with the case where M is a (homotopy) sphere and dim M/G = 2, and working in the differentiable category, we are going to classify all such pairs (G, M), up to equivariant diffeomorphism.

This project was initiated in [S6], as Part I, and the present paper should be regarded as Part II of the program. These two papers have altogether five main theorems, namely Theorem A, B, C, D and E ; the first three are stated and proved in Part I, whereas Theorem D and E are stated below. The present paper is mainly concerned with the proof of the last two theorems.

First, we shall give a brief review of previous results along these lines. The study of low cohomogeneity symmetry groups of spheres started with the work of Montgomery, Samelson and Borel in 1940-50, namely the case of compact transitive transformation groups (i.e. zero cohomogeneity) on spheres, cf. [MS], [Bo2]. In 1960 H. C. Wang published his work on the classification of G-spheres of cohomogeneity one, cf. [W]. Wang had some dimension restrictions, and came to the conclusion that all the actions were of orthogonal type. However, he overlooked the existence of a rather interesting family of non-orthogonal actions, first pointed out by the Hsiang brothers. Following the ideas of Wang, Asoh [A] completed in 1983 the remaining cases in [W]. As far as classification of cohomogeneity two transformation groups on spheres are concerned, substantial results were first obtained by G. Bredon in two papers of 1965, cf. [Brl, Br2]. He settled the special case where there are only two types of orbits, demonstrating the existence of non-orthogonal actions on certain homotopy spheres En, n = 2k-1 and k odd. In fact, G = SO(k) in these cases, and shortly afterwards it turned out that these actions could be extended to a larger group G = SO(2) x SO(k) so that (G, Sn) is just one of those missing non-orthogonal examples of cohomogeneity one mentioned above. Recently, there are also a few more studies of cohomogeneity two actions on spheres with narrowly specified groups and orbit structures, cf. Uchida-Watabe [UW] and Nakanishi [N].

Due to the rich variety of orthogonal transformation groups of cohomogeneity two on spheres, it seems clear that Wang's approach would be rather "hopeless" for an exhaustive investigation of cohomogeneity two actions. Furhermore, since non-orthogonal examples are already known to exist, one cannot expect uniform results along these lines without knowing what are the possible groups and corresponding orbit structures. The situation is quite different from the analogous study of low cohomogeneity actions on euclidean spaces [Rn. The reason is the existence of fixed point in the latter case. For example, dim [Rn/G < 4 implies F(G, (Rn) * 0 , see [HS1]. In fact, in the paper [MSY] of 1956, Montgomery,

Received by the editor September 29, 1994

1

2 ELDAR STRAUME

Samelson and Yang proved that a differentiable action of a compact connected Lie group on [Rn with dim [Rn/G = 2 must be G-equivalent to a linear one. A major step in their proof was to show the existence of a fixed point. Similarly, in the case of cohomogeneity two G-spheres with fixed points, we can now show rather easily that the action is globally G-diffeomorphic to an orthogonal action.

Now, what is the principal tool that enables us to handle G-spheres without fixed point as well ? The answer is the geometric weight system. Recall that linear G-representations (and orthogonal G-spheres) are uniquely determined by their weight system (or character). The geometric weight system for G-spheres is an invariant which generalizes the classical notion of weights in the linear case. Although it is generally not a complete invariant, this is nearly so for low cohomogeneity actions. Moreover, unlike the approach of Wang, we can now make more effectively use of the fact that the G-space is a (homology) sphere.

A new and complete proof of the cohomogeneity one classification initiated by Wang is given in Part I (cf. [S6]). The geometric weight system, in fact, completely distinguishes all G-spheres of cohomogeneity one. Therefore, it is interesting, but not so surprising, that it cannot detect non-orthogonal G-spheres of cohomogeneity two. This is a consequence of Theorem A in Part I; a simpler version of this theorem is restated for convenience :

Theorem A' Let G be a compact connected Lie group, X n ~ S n a compact and differentiable G-manifold which is a Z-homology n-sphere, and assume dim Xn/G = 2 and G ^ SO(2). Then there is a unique orthogonal representation O : G —> SO(n+l) such that £2(0) = £2(Xn). In other words, the geometric weight system of (G, Xn) equals the weight system of the orthogonal G-sphere (G, O, Sn).

We shall refer to the above orthogonal G-sphere as the orthogonal model of (G, Xn). Next let's introduce our notion of G-orbit structure, as follows. The G-orbit structure of a smooth G-space X is characterized by

a) specifying the orbit space X/G (with the induced functional smooth structure) and its orbit type stratification, and

b) specifying what are the orbit types corresponding to the various strata.

The geometric weight system is a complete invariant for G-spheres of cohomogeneity < 1. For low cohomogeneities in general, it is not far-fetched to assert that the geometric weight system still exerts a rather strong "control" on the orbit structure. This "control" tends to decrease as the weight system increases in complexity. Therefore, in the case of cohomogeneity two, the following result is very satisfactory from our point of view.

Theorem D Let (G, Xn) be as in Theorem A'. Then (G, Xn) has the same orbit structure as its orthogonal model (G, O, Sn). Moreover, the corresponding slice representations are identical. In particular, Xn/G is a disk with at most 3 vertices.

LOW COHOMOGENEITY ACTIONS 3

Remark This theorem confirms a conjecture made by Montgomery and Yang (cf. [Br3, p.214]), saying that the orbit space disk should have at most 3 vertices.

We have avoided the special case of a circle group G = SO(2) since a complete classification of closed 3-dimensional SO(2)-manifolds is available in the literature, cf.[R] and [O]. In fact, the Poincare space S^ = SO(3)/I (with the obvious SO(2)-action) gives the only example of a non-orthogonal action on a zZ-homology 3-sphere. Here the orbit space is topologically a 2-sphere. In the topological category, Montgomery, Zippin and Jacoby (cf.[MZ], [Ja]) have shown that all continuous actions of SO(2) on S 3 are, indeed, equivalent to orthogonal actions.

Next, the Reconstruction Problem asks what are the possible G-manifolds X n having a given G-orbit structure. Here the interesting G-orbit structures are those of the appropriate orthogonal models (G, O, Sn), and we want X n to be a (homology) sphere. The main observation is that X n can be constructed from its orthogonal model by the technique of "equivariant twisting". More precisely, we "twist" a tubular neighborhood of some isolated and singular orbit, namely an orbit which is a corner of the 2-disk Sn/G. This involves equivariant glueing of disk bundles along its boundary, which is typically a product of two spheres. We shall refer to the above construction as equivariant twisting of the orthogonal model.

The solution of our Reconstruction Problem can be stated briefly as follows.

Theorem E Let (G, Xn) be a differentiable (integral homology) G-sphere and assume dim Xn/G = 2 , G * SO(2), and let (G, O, Sn) be its orthogonal model. Then X n is a homotopy sphere, and moreover :

(i) If (G, O) does not belong to the following exceptional list of linear groups :

(a) G = SO(2k+l),k>2, (b) G = Sp(2)xSp(l),

(c) G = Spin(9),

(d) G = Sp(l)xSp(m)xSp(l) ,m>l,

(e) G = Sp(l)xSp(m), m> 1,

(0 G = U(2) x Sp(m), m> 1,

(g) G = SO(2) x Spin(9),

d> = 2p 2 k + 1

o = v2(8)[Hv1 + p 5

O = Ag + p 9

° = v l®H v m + vm®H v l O = S 3 v 1 0 H v m

0 = ji2®([vm

O = p 2 ® A9

then (G, Xn) is equivariantly diffeomorphic to its orthogonal model.

4 ELDAR STRAUME

(ii) There are countably infinitely many non-orthogonal G-spheres for each of the models of type (a) - (e). On the other hand, for each of the models of type (f) - (g) there are at most one non-orthogonal G-sphere.

(iii) Xn is diffeomorphic to the standard sphere Sn in the cases of type (b) - (f).

Remark This theorem disproves a conjecture made by Bredon (cf. [Br3, p.214]), namely that the only non-orthogonal examples should be those of type (a).

Note that the exceptional list contains only non-polar linear groups, namely linear groups having no "section", cf. Chapter I, §2. In case (a) there are also exotic spheres, and the G-spheres of this type were first discovered by Bredon in 1965, cf. [Brl, Br2]. In cases (a) - (d) the orbit space is a disk with at most 2 vertices; for a complete classification of the non-orthogonal examples we refer to Chapter IV and its Theorem 3.9.

In the cases (e), (f) and (g) there are 3 vertices, and the construction of non-orthogonal examples is discussed in §2 of Chapter V. Here our classification is not complete; some additional work remains to be done, and this will be left as an open problem. In this connection we also mention that it would be nice to have an alternative description, say, in terms of explicit (algebraic) equations, of the non-orthogonal actions in the cases (b) - (g). Indeed, such a description is well known in case (a), where the G-spheres Xn are realizable as codimension two G-invariant subvarieties of orthogonal G-spheres Sn , see e.g., Part I, p. 4, for more information. This is also left as a challenging open problem.

Finally, for the sake of comparison with Theorem E, we shall recall some facts about topological actions on Sn. First, note that the above "exotic" actions are also topologically distinct. In fact, it is known that all actions of compact connected (Lie) groups G on S4

with dim S /G < 2 are equivalent to orthogonal actions, see Richardson [Ril, Ri2]. On the other hand, for n > 5 there are also non-smoothable actions of G SO(2) on spheres Sn

with dim Sn/G < 2. Here are some examples. Consider the join En = Z3* Sn~4 , where I? = SO(3)/I. Then, by the solution of the double suspension conjecture, cf. [C], for n > 5 it follows that Zn - Sn. Hence, G = SO(3) x SO(n-3) acts on Sn with dim Sn/G = 1; in particular, SO(3) has a non-smoothable action on S^ with dim S5/SO(3) = 2 and S1 as fixed point set.

Notation and terminology As a rule, the notation and terminology from Part I (cf. [S6]) will be continued. However, in each chapter formulas or statements are numbered consecutively as (1), (2), etc. For the sake of easy cross reference, a footer on each page will indicate the chapter number.

Chapter I. Organization of orthogonal models and orbit structures

In this chapter we shall give a description of the orbit structure of all orthogonal G-spheres (G, O, Sn) of cohomogeneity two, where G is compact and connected. We shall also collect data and explicit results about isotropy types in the various cases, since this information is needed in later chapters and is only partially available in the literature.

§1. A review of basic notions and results

We shall refer to an (almost faithful) orthogonal representation O : G —» 0(n+l) as a linear group , and we write either (G, O), (G,V) or (G, O, V); V = [Rn+1, SV = S n = unit sphere. In general we shall not distinguish between G and its image in O(V) = 0(n+l) . In particular, representations differing only by an outer automorphism2 of G are identified, and groups belonging to the same conjugacy class (G) in O(V) are regarded as equivalent linear groups.

The cohomogeneity of G c O(V) is the integer c(O) = c(G) = dim V/G. We say G and G' are C-equivalent if, modulo conjugation of G, both groups have precisely the same orbits in V. Since the groups are connected it is not difficult to see, assuming G c G ' , that c(G) = c(G') if and only if the groups are C-equivalent. In each C-equivalence class there is a unique maximal element, called a maximal linear group. We say (G, O) is splitting if it can be decomposed as an outer direct direct sum

(1) (G, O) = (G],Oi) 0 (G 2 , 0 2 ) , dim Oj > 1.

For a reducible representation O = Oi + <X>2 in general, we have c(®\ + 0 2 ) > c(Oj) +

c(2), but in (1) c(O) = c(i) + c(2) clearly holds. In (1) we also allow G 2 = 1 and

0 2 = x^ , namely (G, O) is splitting if has a trivial summand T^. Those aspects of linear

groups most interesting to us reduce easily to the case of non-splitting groups.

Assume G cz O(V) has nontrivial principal isotropy type (H). Define G = N(H)/H and V = V " = the fixed point set of H, with the induced linear action of G. The linear group (G, V) is called the reduction of (G, V); its principal isotropy type is trivial, but its orbit structure is generally much easier to handle. The important fact is that the induced map at orbit space level

V -> V (2) i i

V/G -» V/G

This convention also applies to our classification of actions.

5

6 ELDAR STRAUME

is an isometry (in the orbital distance metric) as well as an isomorphism between the orbit type stratifications. In particular, the reduction of linear groups preserves the number of orbit types. In some cases reduction is possible even when H is trivial, namely by first extending G to a group G' (possibly disconnected) in the same C-equivalence class and then applying reduction to (G',V). This leads to the notion of the minimal reduction, where no further reduction is possible (i.e. when dim G is minimal). The minimal reduction of

(G, O) will also be denoted (0,3>). We refer to [S4].

Remarks 1.1 (a) It is easy to see that the number of orbit types cannot decrease by an extension G c G ' within the same C-equivalence class. In fact, it seems to be generally true that the minimal reduction also preserves the number of orbit types. To our knowledge, a counter example (G, V) would need c(G) > 4.

(b) Let G c G ' b e C-equivalent groups in O(V), let K = K'n G and K' be the corresponding isotropy groups at some XG V, and let V x be a linear slice at x. Then, as a consequence of the slice theorem, the slice representations of K and K' give C-equivalent linear groups in 0(VX). Moreover, there is a canonical inclusion G c: G, cf. also [S4].

§2. Orbit structures of linear groups of cohomogeneity < 3

In this section we are specializing to connected groups G cz O(V) with c(G) = 3. Of course, since splitting groups must also be considered, all groups of cohomogeneity < 3 are actually involved. Nonsplitting groups of cohomogeneity < 3 are listed in Tables I-III of Part I (see [S6]). So, for example, a complete list of all groups with c(G) = 3 are obtained by taking outer direct sums of groups whose cohomogeneities add up to 3. The geometry of SV/G for all the associated orthogonal G-spheres SV is described in [S4] ; let's first recall some of the basic results. The following lemma is stated for convenience.

Lemma 2.1 (cf. [S4, Theorem 3.1]) Let G cz O(V) be a linear group (possibly disconnected) with dim G > 0 and c(G) = 2. Then its minimal reduction is a dihedral reflection group D k c 0(2), k = 1, 2, 3, 4, 6. In particular, the orbit space SV/G = S 1 ^ is a circular arc of angle 7i/k . Moreover, the number of orbit types is 2 or 3 for k odd or even, respectively.

Recall that all real 4-dimensional representations of the circle group U(l), with no trivial summand, are of type

(3) (U(l), [(j^)™ + 01!)*%), 1 < m < k, (m, k) = 1.

The image of U(l) in SO(2) x SO(2) <= 0(4) is denoted Umk. The orbit space S 3 /U m k

is a surface of revolution (homeomorphic to S2), whose poles correspond to the

exceptional isotropy groups isomorphic to Z m and Z^.. In particular, SJ/Uj j = <LPl=

S^(l/2) is the round sphere of radius 1/2. In all cases of (G, V) different from (3), the topological type of SV/G = SV/G is

a 2-disk D^. The boundary circle is the union of the singular strata; there are no exceptional orbits. The possible metrics are described by the following theorem. .

Theorem 2.2 (cf. [S4, Theorem A, B and 5.1]) Let (G,V) be a compact connected linear

group of cohomogeneity 3, and assume it is not of type (3). Then its minimal reduction

(G, V) satisfies dim G < 1, and W = G /G° is a Weyl group. More precisely, either

(i) G = W c 0(3) is a crystallographic reflection group, or

(ii) G c 0(4) and G° = Uj j or U | 2- W acts effectively on the "sphere"

S^/G° ~ S^ as an isometric reflection group (Dj) , k < 3, or Dj x D3 , and moreover,

W = Dj i f C° = U1 > 2 .

Corollary 2.3 (i) Assume G° * U j 2- The disk SV/G is isometric to a fundamental

domain of (W, S^). Here S^ is a round sphere, of radius 1 or 1/2 for G° = 1 or G° = U | j , respectively. The geodesic region SV/G has 0, 2 or 3 vertices, whose angles are determined by the Coxeter group W (in the usual way).

(ii) Assume G° = Uj 2 • Then S^/G° is an ovaloid of revolution with a conical

singularity of total angle K at one pole, whereas the other pole is actually smooth. The

fundamental domain of W = Dj is a "curved" disk S^/G = SV/G with only one vertex

(of angle TC/2).

Remarks 2.4 (i) Irrespective of cohomogeneity, orthogonal groups G c O(V) with dim G = 0 turn out (cf.[S4]) to be the same as polar groups, in the terminology of [Da], [PT]. The latter type was characterized by the existence of a section Z, namely a c(G)-dimensional linear subspace intersecting each orbit orthogonally. To this situation is associated a finite group W c 0(E), called the generalized Weyl group, so that the inclusion X -> V induces V/G = E/W. Then it is not surprising that (G ,V) = (W, Z) holds.

If we assume G is connected, then another characterization of the above type of (G,V) is that the C-equivalence class of (G,V) contains a linear group (G',V) which is the isotropy representation of a compact symmetric space K/G'; the Weyl group of K/G' is just the above W. In particular, W is a reflection group and therefore a fundamental chamber in S is a cross section for the orbit map.

It seems to be true that merely the existence of a global cross section characterizes polar linear groups among all connected linear groups (G,V). However, we shall not pursue this topic here.

8 ELDARSTRAUME

(ii) If G° = Uj 1, then (W, S2) and (0, S3) may not induce the same orbit type

stratification. This happens when W has order 4 or 12. Then two vertices of S2/W have the

same W-orbit type, whereas their G -orbit types (or G-orbit types) are distinct. (iii) The possible angles at a vertex are 7t/k, 2 < k < 6 , k ^ 5 . However, k = 6 is only

— 9

possible when G = W = D^. In any case, the angles are determined by (W, Sz) as usual. (iv) If some vertex of SV/G has angle 7C/3, then the two arcs meeting at the vertex have

the same G-orbit type. In fact, the reduction (in the slice representation) of the isotropy group K at the vertex must be K = D 3 cz 0(2), and isotropy groups of the meeting arcs are even conjugate in K, cf. also Lemma 2.1.

We shall describe in more detail the various orbit structures of the groups (G,V) covered by Theorem 2.2. First, we divide the groups into four main types labelled 0,1, II or III, according to the number of vertices of SV/G. The various strata components are labelled by the corresponding isotropy types (Kj), (Lp, (H); for simplicity, a conjugacy class (Q) is written Q. The four main types are illustrated in Figure 1.

Further subdivision of type II or III goes as follows :

II0 : K 1 = K 2 = G [=> d i m V G = 1, G = Wandrk W = 2 ]

II! : Kj * K 2 [=> Lj ^ L 2 , G ° = U 1 j , W = D 2 ]

(4) III! : Kx * K 2 = K3 , L 1 = L 2 = L 3 [ = > G = W = A 3 ]

III2 : Kj * K2 = K3 , L 1 = L 2 ^ L 3 [ => G = W = D{ x D 3 ]

I I I ^ K j ^ K j , L ! = L 2 * L 3 [=> G = W = B 3 , o r G * W = D 1 x D 3 ]

I I I 4 : K j ^ K y Lj*Lj [=> W = Dj x D k , k = 2 , 4 , 6 ]

Note The above notation also reflects the fact that Type III^ has (3 + k) orbit types!

An explicit description of the orbit types for all the above (G, <X>) will be crucial, but this task is rather laborious. The calculations involve the rich subgroup structure of compact Lie groups combined with the representation theory of these groups. Technically, there is little difference between the linear and the nonlinear case, thanks to the geometric weight system. We refer to Chapter II for further information on the explicit calculation of isotropy groups.

Let T c G be a fixed maximal torus. Representative isotropy groups Q can and will be chosen to be T-adapted in the sense that (T n Q)° is a maximal torus of Q. Moreover, each of these Q will, in fact, contain a common principal isotropy group H. Note that inclusions like H c Lj or H e Kj are well defined up to simultaneous conjugacy in G, see [S6, p.77]. But some caution is needed since the simultaneous conjugacy class of triples like H <z Lj c Kj may not be unique. However, our "standard" choice of T and our notation for subgroups and various inclusions among them should not lead to ambiguities.

Clearly, for each C-equivalence class it suffices to know the isotropy types for the maximal group, also denotedby (GQ,O0), since this reduces the remaining technical problem to the calculation of intersections of G and isotropy subgroups of GQ.

Consider first the case where (G0,O0) is splitting. If O0 has three summands, then the orbit types follow from Table I of [S6]. Next, assume €>0 = Oj 0 3>2, 0(^2) = 2 and O2 is irreducible. The latter is among the 12 types of isotropy representations (G\ O2) of irreducible symmetric spaces of rank 2; these are listed in Table II (loc. cit.) together with a principal isotropy group. The missing information is the singular isotropy types (Kj), i = 1,

2, of <J>2. Table I in [S2] covers all cases except when (G', O2) is one of the groups

(F 4 ,K 2 6) , (G 2 ,Ad) , ( S O ^ v ^ S ^ ) , (U(l)xSpin(lO), [ J I J ^ A ^ I R ) .

In F4 both groups Kj are conjugate to Spin(9), in G2 (resp. SO(4)) the Kj are non-conjugate embeddings of U(2) (resp. 0(2), cf. [S6, p.33]). Finally, in the last case

K j = Spin(7) x Sl, H' = SU(4) x S l, K2 = U(5)' - (SU(5) x S l )/AZ5 ,

where AZ5 ~ 7L$ sits diagonally, but such that U(5)' is not isomorphic to U(5) !

Henceforth, we assume (G, O) is non-splitting, in particular Kj ^ G. With reference to Table III of [S6], we shall divide into four cases according to the type of the linear groups :

(5) Type O : (G, $) = (0(2), 2p2).

a) (G0, O0) = (SO(m), 2pm), m > 2; G^ = 0 for m > 3, H = SO(m-2), K = SO(m-l).

b) G = G2 (exceptional group, m = 7), G = SO(3).

c) G = Spin(7), m = 8, G = 0 .

(6) Type I : (0, 5) = (0(2), p 2 + p'2), ker p'2 = Z2 .

1) (G, O) = (U(2), p 3 + [[i2\l ( H c L c K ) = ( l c U(l) x 1 c U(l)2), G = G (since H is trivial).

2a) (G0, O0) = (Sp(l) x Sp(2), p 5 + v ^ ^ v ^ , G^ = 0, H = {(a, a, a)} c {(a, a, c)} = L c K,

b 0

0 c K={(a,b,c)} = {(a

2b) G = U(l) x Sp(2), G = U(2). 2c) G = Sp(2), G = G (since H is trivial).

)} = Sp(l)3.

10 ELDAR STRAUME

(7)

3) (G0 , O 0 ) = (Spin(9), A9 + p 9 ) ; G 0 = G , H = G2 ,

L = S p i n a l , K = Spin(8).

Type IIX : (0 , 5 ) = (0(1) x 0(2), p 2 + Pi®P2)-

l a ) (G0 ,O 0) = (U(l) x U(m), [ i i m + M-l<S>Mm W ' m ^ 2 '

( G ^ , O ^ ) - ( G 0 , O 0 ) w i t h m = 2.

We may assume G 0 = U(l) x SU(m) x U(l), O 0 = [ | i i®| im + MV^M-lt

and then

(8) aO OC

Kj = {(a

L 2 = {(a,

b)}, K 2 = { ( a , b o o c

,b)} , Lx = K i n K 2

a 0 0 0 b 0 0 OD

, b )} , H c L 2 : a = b .

(a, b e U ( l ) , C eU(m-l) , D eU(m-2))

lb ) G = U(l) x SU(m), m > 2, U(l) c U( l ) 2 . Then G = G 0 , or G = G (ifm = 3).

2a) (G 0 ,O 0 ) = ( Sp(l) x Sp(m) x Sp(l), v ^ ^ + v ^ V ! ) , m > 1,

G Q = G . Isotropy groups are similar to (8), with a , b e Sp(l).

2b) G = U( l )xSp (m)xSp( l ) , G = U ( l ) x U ( 2 ) .

2c) G = Sp(l) x Sp(m) x 1, G = Sp(l) x Sp(2).

Remark 2.5 In (8), Lj c K| , L j c K2 but L2 <Z K2. Although K2 contains some

nl^n" , neN(H), one cannot choose L2 so that L2 c: K2 is also achieved. This reflects

the non-polar nature of the above linear groups (cf. also Definition 1.2 of Chapter V).

Type III : Consider first polar groups (G0, O0 ) ; these are isotropy representations of irreducible symmetric spaces of rank 3 (cf.Table III in [S6], #8 - #21, except #11 and #20). The isotropy types are listed in [S2, Table I], except in the case #21, where (U( l )xE 6 , [R 5 4 ) .

In the above exceptional case, let G = (U(l) x E^/AZ^ c SO(54) be the effective

group. This is a linear group of type III3, whose minimal reduction is given by the

standard action of the Weyl group W = B3 = 0(1) J X2>^ on [R , whereas G is the

"thickening" U(l)-~> X^3 which acts on d . 3^ denotes the permutation group on k letters.

Isotropy groups in G can be chosen as follows :

(9) Kj = (S! x Spin(10)/AZ4 , K2 = S 2 x Spin(9), K3 = F 4 (exceptional group) Lj = S 2 x Spin(8) , L 2 = Spin(9) ~ Spin(9)' = L 3 , H = Spin(8) ,

where F 4 = < Spin(9), Spin(9)' >, Spin(9)' c Spin(lO), Spin(9) c2 Spin(lO) and Spin(8)

= Spin(9) n Spin(9)'. The Sj are circle groups not inside E^, and a precise description of

the embeddings Kj a G can be given in terms of weights (which we omit here).

As far as (non-splitting) non-maximal groups (G, O) of type III are concerned, it is generally true that G ^ W, that is, dim G > 0. The exceptions are #24, #35a , #36a in Table III (loc. cit.), where G is a product of Sp-factors and O is reducible. We omit the cumbersome description of their isotropy types; this is straightforward and they are obtained by intersecting G with the isotropy groups of the corresponding maximal linear group.

Finally, non-polar linear groups (G, O) of type III constitute the following short family, with the minimal reduction as indicated3 :

(e) (Sp(l)xSp(m), S^Cg^Vjn ), m > 1, 0 ° = Uj j , W = D l X D 3

(10) (f) (U(2) x Sp(m), [\i2 % v m ] K , m > 1, 0 ° = U u , W = ( D ^ 3

(g) (SO(2 )xSp in (9 ) ,p 2 ®A 9 ) , 0 ° = U l f l , W = (D 2 ) 3

In (e) the reduction (G, O) coincides with the minimal reduction, more precisely the 1-dimensional linear group (D3 x 0(2), (p 2 I D3)<8)p2 ). The cases (f) and (g) have the same reduction, namely the semi-direct product of (G~ ,0^) in case la) of (7) with the group 2>i (of order 2), where the latter group interchanges the two summands of the former representation. (See also Chap.V, §1, or [S4, §5].)

Our "canonical" representatives of isotropy groups in all three cases of (10) are explained below :

Case (e) Let Sp(l) = {a ; aeSp(l)} be the image of Sp(l) in Sp(2) via S3v1? and

note that the image of Sp(l) by p 5 : Sp(2) -> SO(5) is the linear group (SO(3), S 2 p 3 - x ^ .

We fix a covering Sp(l) -> SO(3) so that N = < e i 0 , j > and N' = < eJ0, k > are the inverse images of 0(2) = S(0(2) x 0(1)) and 0(2)' = S(0(1) x 0(2)), respectively. Then

Q = N n N' = {±1, ±i, ±j, ±k } (quaternion group of order 16),

( N D Q C N') is an associated triple for (Sp(l), S 2 p 3 - x1? S4) , cf. [S6, p.75].

We may choose Sp(l) so that a = (a i , a ^ e Sp(l) x Sp(l) for all aeN; in fact, ai = a. Also

a = e10 implies a2 = e"310, and ^2 = a for ae Q. We shall regard (G, O) as a subgroup of

(Sp(2) x Sp(m), V2 ®[Hvm)' w n o s e isotropy types are well known (cf. e.g. [S2]).

3 Here the labelling (e), (f), (g) is consistent with that of Theorem E in the Introduction.

12 ELDARSTRAUME

This leads to the following maximal isotropy groups Kj in G and corresponding slice

representations <|)j for the orthogonal G-action on the unit sphere :

(11) K1 = {(a, a 0 0 Sp(m-2) ) } » Sp(l)xSp(m-2), ( ^ = S 2 P 3 - T i

K2 = { (a, aj 0 0 Sp(m-l) ) ; a e N } - N x S p ( m - l ) , $2 = p ' 2 + ^2®[Hvm-l,

K3 = { (a, a2 0 0 Sp(m-l) ) ; a e N } « N x S p ( m - l ) , (t>3 = p ' 2 + ^ l®[H vm-l ,

where s \ i | N = Xi + A^ » ^i(a) = 3aj , and p ' 2 is the 2-dimensional 0(2)-representation

with kernel Z 2 . Choose L] to be the subgroup of K j with ae N. Here a = (a i , a2) lies

diagonally in Sp(2), and replacing a by (a2 , ai) we get L3 c: K3. Note that L3 <Z Kq, but

L3 ~ L j . Now L 2 is the subgroup of K2 with ae Q, and H is the subgroup of Kj with ae Q.

Case (f) As above, we shall regard (G, <f>) as a subgroup of (Sp(2) x Sp(m),

v 2 ®Q-|Vm). Sp(l) is identified with the subgroup SU(2) c U(2), say a <-> A.

(12) K1 = {(A, a 0 0 Sp(m-l) ) } « Sp( l )xSp(m-l) ,

K2 = Ci 0 0 C2

Ci 0

0 Sp(m-l) )} ~ U ( l ) 2 x S p ( m - l ) ,

K3 = { (B, B 0 0 Sp(m-2)

); BeU(2)} - U(2) x Sp(m-2) .

The corresponding slice representations in the sphere are 01 = P3 + Vj ®[j-|vm-l'

^2 = tMl®([M',lllR + [M-'l^^m-l^R ( w n e r e M-l» r e sP- Hi *s m e u m t representation of the

first, resp. second factor of U(l) 2) and ( 3 = P3 + [det(B)]K. Let L | « U(l) x Sp(m-l) be the

subgroup of K 2 defined by ^ 2 = ^ , let L 2 be the subgroup of K3 (and K2) with Be U(l) ,

and choose the groups H c L3 to be the subgroups U(l) x Sp(m-2) c SU(2) x Sp(m-2) of K3.

Case (g) Let (01, . ., 64) be "standard" coordinates for the maximal torus T c: SO(9),

covered by T c Spin(9). We can choose coordinates ( t i , . ., t^) for T so that the projection

n : Spin(9) -> SO(9) restricted to f looks like

(e27Tit1? ? e27iit4) _> ( e 2mt l 5 e27ci(t2 - t i ) ? e27ci(t3 -12)9 e27Ti(2t4 - t3) ) f

that is, $i = t | , 0 2 = t 2 - t j , 63 = t3 - 1 2 , 0 4 = 2t4 - t 3 . In particular, t 4 = 1?<QV

The tj's are the basic weights for Spin(9). Let SU(4)+ 3 SU(3) be the simple groups in

Spin(9) generated by the root spaces of (0j - 6j), with i < j < 4 in the SU(3) case. These

subgroups are in fact determined by their maximal tori Tj z> T 2 defined by the condition

t 4 = 0, resp. (t4 = 0, 9 4 = 0 ), and Ti does not contain the center of Spin(9). SU(4)+ is

contained in a (not unique!) group Spin(7)+e Spin(8); the latter inclusion is mapped by n

to the inclusion A7 : Spin(7) -> SO(8). Recall that Spin(9)/Spin(7)+ = S 1 5 . As usual,

Spin(7) also denotes the subgroup of Spin(8) lying above SO(7) e SO(8). From

representation theory it follows that Spin(7)+ n Spin(7) ~ G 2 for any choice of

representatives in Spin(8). Hence, we shall keep in mind the following diagram of

subgroups and inclusions

^ S U ( 4 ) ^ Spin(7) (13) SU(3) -> Go ^ 3 Spin(8)-> Spin(9)

N»SU(4) + -> S p i n ( 7 ) + / ^

and the corresponding diagram for SO(9).

Now, let (0, t j , . ., t4) be coordinates for the torus f = SO(2) x f c SO(2) x Spin(9),

and let's first describe the torus types of O = P 2 ® A Q , namely the maximal tori of isotropy

groups of the action O | S 3 1 . By the Torus Algorithm [HS2, 2.9] or [S6, p. 19]), these types are given by the following four subtori of T:

f x : 0 + t4 = 0 , f2 : 0 + t4 = 0 ^ 0 2 + 0 3 = 0 , f3 : t4 = 0 = 0

f4 = f2 n f3 : 0 = 0 4 = Q{ + 0 2 + 0 3 = 0 .

Define a circle subgroup

§! = {(e"2Tci2t, e2 7 l i t , e2 7 t i 2 t , e2 7 c i 3 t , t2ni2t)} c f l f

and observe that < f 1 , S 1 > = T 2 , fY n Sj = {(1,8,1,8, l ) ;e = ±l }= Z 2 e 2 4 =

Z(SU(4)+). The group

K2 = < SU(4)+, Tj > = < SU(4)+, Sj > « [SU(4) x U(1)]/AZ2

14 ELDAR STRAUME

is a maximal isotropy group for O = p2®Ag, and it is the only type of corank 1. One

checks that the intersection of SU(4) and the image of S in K^/ker O equals the center of

SU(4), and hence K2/ker O - U(4). The (connected) centralizer of Spin(7) in Spin(9) is the circle group S : t | = t2 = t^ = 0.

This is also the (connected) centralizer of G2- We shall describe K3, having maximal torus T2 = T4 x S, by introducing the circle group S , as follows :

S = { (e27cie, 1,1,1, e-27ci9)} c (SO(2) x S ) n f 2

K3 = < G2, S > = G2 x S .

Note that S contains ker O ~ Z^, and T3 is a maximal torus of Kj = < Spin(7)+, ker O >. Finally, we shall list all the groups Kj, as subgroups of the effective group

[SO(2) x Spin(9)]/kerO, together with their slice representations :

K1 = Spin(7)+, $i = A7 + p7 ,

(14) K2 = U(4) = (SU(4)+x U(1))/AZ4 , ^ = td e% + ^ l ) ~ 3 % M R ' K3 = G 2 x S / Z 2 = G2xSO(2), (t>3 = p 7 + p 2 ,

and consequently L] = SU(4)+, L2 = SU(3) x SO(2), L3 = G2 and H = SU(3). In fact, Lj_i and Lj are isotropy groups of (|)j, where LQ is conjugate to (but different from) L3.

Chapter II. Orbit structures for G-spheres of cohomogeneity two

The main purpose of this chapter is to explain the basic concepts and techniques leading to a proof of Theorem D (see Introduction). A major problem is the lack of a systematic and effective procedure for the calculation of all orbit types. (In the linear case there is a "brute force" algorithm which is exhaustive, at least in principle). Nevertheless, knowledge of a few (maximal) isotropy groups may sometimes lead us to all orbit types, by combining various kinds of information. Although the subgroup lattice structure of a compact Lie group G can be rather complicated, the construction of a G-sphere with a reasonably simple orbit structure imposes very strong constraints on the possible orbit types which can be combined. From this viewpoint, the orthogonal models provide us with actual solutions of the problem, which in turn will serve as our guiding beacons in the study of general G-spheres.

In § 1 we focus attention on the connection between isotropy groups and the geometric weight system. This is a key fact which has also been utilized in Chapter I, §2. A natural conjecture associated with low cohomogeneity representations is discussed in §2, and our solution in a special case gives the proof of Theorem D.

§1. Weight patterns and calculation of orbit types

Let X be a -homology sphere and G a compact connected Lie group acting smoothly on G. The pair (G, X) is simply referred to as a G-sphere, and the cohomogeneity of (G, X) is the number c(X) = dim(X/G). The (integral) geometric weight system is denoted Q(X|G) or simply Q(X). As in the linear case, this weight system will be regarded as a multiset of the weight lattice of a fixed maximal torus T c G . The weight pattern of (G, X) consists of the nonzero weights, and is denoted Q!(X). We refer to [S3, S6] and [HS2] for a review of the geometric weight system, including the notion of a p-weight system for each prime number p, as well as some basic properties and techniques.

The connection between Q(X), the orbit structure and the homological structure of fixed point sets, leading to various fixed point theorems of P. A. Smith type, has been demonstrated at various occations, e.g. [HS1, HS2], [S2, S3, S5, S6]. This relationship is particularly strong if the weight pattern is reasonably "simple"; in fact, there is the general expectation that the orbit types are completely determined by Q(X) in these cases. Namely, if the same G-weight pattern, with rk G > 1, is realized on two G-spheres, then they ought to have the same orbit types (apart from fixed points). However, the notion of a "simple" G-weight pattern is somewhat vague. For example, we may choose to measure the complexity of Cl'(X) by the size of its Weyl group orbits and the number of orbits, but it is also roughly measured by a single numerical invariant such as the "density" #Q'(X)/dim G, cf. [S5]. The definition of "simple" given below (cf. 2.1) is convenient for us; it is accompanied by our "simple weight pattern conjecture" (cf. 2.4) which is a natural generalization of Theorem D.

15

16 ELDARSTRAUME

Next, we shall recall some ideas useful for the calculation of isotropy groups in the linear as well as the nonlinear case. The set of conjugacy classes of isotropy groups, called orbit or isotropy types , is denoted by d(X). Similarly, let #°(X) be the set of conjugacy classes of the identity components of isotropy groups, and finally, ^ ( X ) is the set of conjugacy classes of maximal tori of isotropy groups, called torus orbit types. The calculation of orbit types may be viewed as a three-step procedure

^ ( X ) -> d°(X) -> fl(X).

Concerning the first step, we recall that # (X) can be calculated explicitly from the weight system by the socalled Torus algorithm , cf. [HS2] or [S6, p. 19]. But the last two steps are generally less mechanical; their partial dependence on Q(X) can be described by equation (1) below. To this end, let A(G) denote the root system of G and let (K, (j)) be the slice representation of some isotropy group K. The following general equation, easily proved in the linear as well as the nonlinear case, also relates weights and roots in the more general setting of G-spheres :

(1 ) fl(X | K) s A(G) | K - A(K) + Q(<|>) (mod zero weights)

More precisely, let T' c T be a maximal torus of K. The left side of (1) can be calculated as the restriction Q(X) IT' , and (1) is an identity between multisets in the weight lattice of T'. In particular, let H be a principal isotropy group with known maximal torus TQ. Its root system is determined by the equation

(2) A(H) = A(G) | T 0 - Q(X) ITQ ( mod zero weights )

Equation (1) is used to show that A(K) must contain certain roots, whose corresponding root spaces generate a " minimal" subalgebra, namely a common Lie subalgebra of all isotropy groups K having T' as maximal torus. In our applications this simple idea drastically reduces the possibilities for the connected group K°; in many cases K° is shown to be unique since its Lie algebra must coincide with the above "minimal" algebra. Finally, the p-weight version of (1) can be applied to check the possibility of p-torsion in K/K°. Similarly, the p-weight version of equation (2) helps us to determine H completely.

The following procedure may reduce the calculations to a simpler situation. Suppose G can be decomposed as a product G = G jx G 2 of nontrivial groups. In analogy with splitting

linear groups we say the weight pattern Q'(X) is splitting (with respect to Gjx G 2 ) if

(3) Q'(X) = Q'(X | G j) + Q . ( X | G2) (cf. [S3] or [S6, p.21 ]) .

Proposition 1.1 Let (G, X) be a G-sphere, G = Gjx G2 and assume the weight

pattern Q'(X) splits, cf. (3). Then each isotropy group Gx is a splitting subgroup, namely

Gx = (Gx n Gj) x (Gx n G2).

Proof A somewhat weaker version is proved in [S3, 5.10], from which it follows that (Gx)° is splitting, and moreover, it proves the proposition in the special case F(T) ^ 0 . Our new proof works in full generality, but (for brevity) let's assume all connected groups (Gx)° are already known to be splitting and rather complete the last step which settles the case of disconnected isotropy groups.

Suppose there is a nonsplitting isotropy group Gx. The idea is to produce a contradiction by studying some induced action (S,Y), where Y is a p-homology sphere for some prime p, xe Y, and S = S *x S * is a torus of rank 2 acting on Y, such that the following conditions hold:

(4) 1) The (rational) weight system Q0(Y) is splitting. 2) The isotropy group Sx is non-splitting; it does not contain 7L~ x 7L~ but

contains a diagonal subgroup AZL = <(kj, k2)>, where < kj > = 2L.

The action of 7L^ x 7L~ defines the p-weight system ( Y ) . On the other hand, we also know £\,(Y) e Q0(Y) | Z p x Z p (modulo zero weights), cf. [S3, Lemma 1.1]. Hence, Qp(Y) is actually splitting, and this implies F(AZp,Y) = F(Zp x Zp,Y). But the latter identity clearly contradicts the fact Zp x ZL ct Sx.

In order to arrive at a pair (S,Y) as above, choose first some k = (kj, k2) in ^x = (^ l x ^ 2 ) x , w*tn ^ ^x* ^° ^ e g m with, let K = cl(< k > ) c G be the closed group generated by k, say K = K° x 7Lm where K° c (Gx)° is a torus. Clearly, 7Zm contains some non-splitting subgroup Zpm> for some prime p and m > 1. Therefore we may as well assume

2 p m « K = <k>d(Zpmx 71^) c S 1 x S 2 c T 1 x T 2 = T , l < m , < m ,

where Sj c Gj is a circle group containing kj and < k±> = 2Lm, < k2> = 2Lm \ Let S = S jx S2 and Y = X. If m = 1 then the conditions in (4) already hold. Next, if

m > 1 then we first reduce to m = m' by replacing (S, Y) by (S/K', F(K',Y)), where

K' = < (k|)P , 1 ) > C K . Now Sx contains a diagonal group A(2Lm) of 2Lm x ZLm, but not the whole product. Thus, we can reduce to m = 1 by successively dividing out by 2L x Zp until Sx no longer contains 2L x 2L. This completes the proof.

18 ELDARSTRAUME

Finally, let's invoke the reduction principle for (differentiable) G-manifolds M in general. Orbit spaces have the smooth functional structure induced by the orbit map. We may assume M is endowed with a G-invariant Riemannian metric, so that M/G with the induced orbital distance metric becomes a stratified Riemannian structure. We shall state some general facts without proofs, which are more or less known in the literature; see for example [SS].

As usual, let H denote a principal isotropy group and assume (for simplicity) that M " is connected. As in Chap.I, §2, the reduction of (G, M) is the pair (G, M), where G = N(H)/H and M = M*\ Then the induced map i : M /G —» M/G is a diffeomorphism as well as an isometry, which induces a bijection between strata components. However, since an orbit type stratum may be disconnected, we can only claim that i ~ * maps a stratum into a stratum. In particular, ##(M) < #d(M) always holds.

Isotropy groups of the reduced action (G, M) are of type K = [K n NQ(H)] /H = Nj^(H)/K, where K is a G-isotropy group at some point in M. We shall also refer to K as the H-reduction of K (in G).

Naturally, one expects that the generally simpler transformation group (G, X) provides crucial information about the orbit structure data of (G, X). This may work well if the structure of (G, X), say in terms of Q(X) or its rational or p-weight versions, can be determined from our preliminary and more easily accessible information of (G, X). Useful data will be the dimensions of various fixed point sets F(K, X), K c G , calculated via local linearity and (generalized) Smith theory.

§2. Simple weight patterns and the proof of Theorem D

Definition 2.1 A G-weight pattern, G ^ S , is simple if it coincides with the weight pattern Q!(Q>) of a G-representation of cohomogeneity c(O) < 3. The isotropy groups of (G,0) will be referred to as O-subgroups (See examples in Chapter I with c(O) = 3.)

Remark 2.2 All weight patterns encountered in this paper are actually simple. We may assume the "model" representation O has no trivial summand, and put m +1 = dim O. By Theorem B\ in [S6, p.71], with this kind of weight pattern a G-sphere Xn has an orthogonal model (G, O + xd, Sn) in the sense that Q(X) = &(<£ + Td), where d = (n - m) > 0. Moreover, F(G, X) is a ^-homology sphere of dimension d-1.

Observation 2.3 Let K * G be a maximal O-subgroup, with <3) as above. Then N(K)/K is one of the groups 1, 0(1), U(l) or Sp(l). Indeed, each orbit ~ G/K in Sm is isolated and F(K, G/K) = N(K)/K. But the sphere F(K, Sm) is the union of the fixed point sets of K in these orbits. In particular, N(K) = K if and only if there are exactly two orbits of type (K).

Otherwise, there is only one orbit of type (K) in S m . Conjecture 2.4 Let X = X n be a G-sphere (any cohomogeneity !) and assume the weight

pattern is simple, say Q(X) = £2(0 + T^), n = m + d. Then the following hold :

(i) The isotropy groups of (G, X) are the O-subgroups of G (except G, if d = 0). (ii) For each isotropy group K, the fixed point set F(K, X) is a Z^-homology sphere.

(iii) The orbit space is a Z^-homology disk whose orbit type stratification is

homologically modelled after the d-fold suspension of the orbit space of (G, O, Sm) .

The case c(O) = 1 is actually settled; the actions are mono-axial and well understood, see [S5, §4]. The conjecture will be verified below in the cases c(O) = 2 or 3. The first step is to extend Theorem Bj in [S6] by including all O-subgroups as well.

Theorem B2 Assume (G, X) has a simple weight pattern Q'(^) a n d let K be a

O-subgroup. Then the fixed point set F(K, X) is a Z^-homology sphere.

Note In most cases (but not all!) the fixed point sets are actually 2Z-homology spheres. This is definitely so when c(O) = 1. Moreover, F(K, X) has the "correct" dimension, namely the dimension of F(K, Sn) in the orthogonal model (G, O + x^, Sn).

Proof This relies on the explicit structure of the groups in question. K/K° is most often

trivial, occasionally it is a 2-group, or of type K = < K°, ZL> where ZL is a cyclic group

commuting with K°. In the latter case one can check from the weight system that

dim F(K°, X) = F(K\ X) for each K' = K ° x Z p c K , p odd prime, hence F(K) = F(K°).

Here F(K°) will be a 2-homology sphere.

Let Ks be the semisimple component of K°. We claim that F(KS) is a homology sphere

(mod 7L or 7Lj). This amounts to calculate the subset of Q ( 0 | Ks) lying in the root lattice

of Ks. Then the rest of the proof is completely similar to the case K = G in [S6, Chap. III].

Lemma 2.5 Assume (G, X) has a simple weight pattern, as above. Then (i) all O-subgroups ^ G are isotropy groups, and (ii) the maximal O-subgroups ^ G are also maximal isotropy groups ^ G.

Proof By local linearity, (i) is easily verified unless d = 0 and c(O) = 3. In any case, the maximal O-subgroups K have nonempty fixed point set in X by Theorem B2, and if we know they are isotropy groups, then their slice representations are seen to be the same as in the orthogonal model. In particular, the remaining O-subgroups will also be isotropy groups. Hence, we need only check that K c K ' and K * K' together imply F(K') = F(G).

By Proposition 1.1 we may assume (G, O) is nonsplitting. The analysis is similar to the

20 ELDAR STRAUME

proof of Theorem B2, the possibilities for K' being very limited. From the weight system (or the p-weight system) we calculate dimensions of fixed point sets of tori (resp. p-tori). In most cases we find that the rank (or p-rank, for some p) of K' cannot be larger than that of K unless F(K') = F(G). So all these cases are settled.

But there are also cases where one cannot "distinguish" between K and K' by maximal

tori or p-tori. We choose one example to illustrate what we can do to circumvent the problem

in a typical situation. Consider (G, O) = (U(2), [|i2lR + P3). w i th K = U( l ) 2 . The only

group between K and G is K' = N(K) = < K, 7L^ >• Now, D2 = 0(1 ) 2 is a maximal 2-torus

of G and F(K) = F(D2). However, K' also contains some conjugate D'2 = gE>2& * ^ 2 ' s o

clearly F(K') c F(D2) n F(D'2) = F(< K, gKg_1>) = F(G).

Lemma 2.6 Assume the weight pattern of (G, X) is simple, as above, and the complement of the principal stratum in the orbit space is connected. Then all isotropy groups are 3>-subgroups.

Proof Let Y be the union of all non-principal orbits in X/G, let Yj be the closure of all strata

whose isotropy groups are O-subgroups, and let Y2 be the closure of the remaining strata in

Y. Now, Yj * 0 by Lemma 2.5, so Y 2 * 0 would imply Y j n Y 2 ^ 0 since Y is

connected. But this is impossible by the slice theorem, since all isotropy groups of the slice

representations of O-subgroups are still O-subgroups.

Finally, we turn to the special case c(X) = 2 and complete the proof of Theorem D. The following result is well known, but we shall sketch a proof for convenience.

Lemma 2.7 Let X _ 2 S n be a G-sphere, G compact connected and ^ S , and assume dim(X/G) = 2. Then X/G is homeomorphic to a disk; the interior consists of principal orbits and the boundary consists of singular orbits.

Proof By the Torus algorithm (cf.[6, p. 19]), the ranks of the torus orbit types occur in a consecutive "string", starting from rank T (or possibly one less). Ignoring a few simple and easy cases, ^ ( X ) has more than one class. Therefore singular orbits must occur, and moreover, at least one effective slice representation is not a finite rotation. Now, by the slice theorem and known properties of representations of cohomogeneity two (cf. [S4, §3]), it follows that X/G is a manifold with nonempty boundary. The manifold must be the 2-disk D 2 since it is simply connected, using the fact that the map Hi (X; <Q) —> H](X/G; (D) is onto (see [Br3, II, 6.5]). Finally, applying [Br3, IV, 8.6] it also follows that there are no exceptional orbits.

Corollary 2.8 If c(X) = 2 and G * S1, then (G, X) has the same orbit structure as its orthogonal model.

This follows from lemmas 2.5, 2.6, 2.7, except that there may not be the correct number of vertices of a given type (K). But this is settled by considering N(K)/K and dim F(K, X), just as in Observation 2.3.

The above corollary also completes the proof of Theorem D. Concerning the equality of slice representations, we have quite often relied on case by case considerations rather than uniform arguments, since we cannot see, a priori and without knowing the isotropy groups, why there should be only one possibility.

As a further support of the above conjecture we also include the following result.

Proposition 2.9 If G is finite, then Conjecture 2.4 holds for (G, X).

Proof In the orthogonal model, G = W is a Weyl group of rank < 3 acting on F(H) = X as a group generated by reflections, and it is easily checked (by comparing dimensions) that the same generators of W also act on X by reflections. X is a homology sphere (at least modulo ILrj) and from the theory of topological reflection groups on (homology) spheres it follows that all strata components of the orbit space have the "correct" homology (cf. [SI]). In particular, Lemma 2.6 applies, and it is not difficult to verify that each stratum in (G, X) of type (K) corresponds to a unique stratum in (W, X) of type (K), where K = Nj^(H)/H.

Remarks 2.10 (i) The circle group S * is the only torus with (effective) actions of cohomogeneity 2, namely S -manifolds of dimension 3. These are well understood (cf. Raymond [R] or Orlik [O]). In particular, smooth actions on X ~ 2 S^ are either the orthogonal ones or the obvious action on the Poincare homology sphere SO(3)/I. Therefore, in this paper we shall assume G is a nonabelian compact connected Lie group.

(ii) The reduction (G , X) was not really utilized in the proof of Theorem D. However, various results such as Proposition 2.9 indicate that reduction will also be a useful tool in higher cohomogeneities and for the complete proof of Conjecture 2.4 when dim G > 0. But we leave this problem here.

The reduced models (G, <D), namely the reductions of all the linear models (G, O), have been calculated since they are needed in later chapters. Here are some summarizing data concerning non-splitting groups (G, <D) with c(O) = 3 (cf. §2 of Chapter I):

G° has no simple factor of rank > 1, except when G = G = Sp(2), cf. (6). G° is trivial (that is, G is finite), resp. a torus (of rank at most 3) in about 17% , resp. 60% of all cases.

(iii) In Table III of [S6] we have listed H, but not G = N(H)/H. Unfortunately, in some cases the notation does not specify the conjugacy class (H) uniquely. With the proper interpretation of H, the reader may find it amusing and challenging to calculate G. But he will also find relevant information on G in various cases in the later chapters.

Chapter III. The Reconstruction Problem

Consider the problem of constructing all G-manifolds X n having a given orthogonal model (G, Sn), i.e., the orbit structure is one of four types illustrated by Figure 1. Let Y = Sn/G be the given orbit space. The way we proceed is by cutting Y into smaller disks Yj which only intersect along arcs of their boundary, and no disk contains more than one vertex of Y. Above each disk there will be a unique G-manifold Xj (with boundary), and therefore equivariant "glueing" of these pieces along their boundary components leads to all possible (G, Xn) with the given orbit structure. However, at this point it is likely that some space Xn constructed in this way must be discarded since it does not have the desired topology, namely a (homology) sphere. Actually this happens only for some specific orthogonal models (cf. Chapter IV).

Let H c L c K c G be compact Lie groups. We shall be concerned with some algebraic as well as topological constructions involving these groups. The topological setup involves a smooth compact G-manifold M whose orbit space is homeomorphic to a closed interval I = [0, 1]. The group Homeo(M), with the compact-open topology, has several interesting subgroups, such as Homeoj(M), Homeo^(M) and their intersection Homeo j (M). Homeo^(M) consists of the G-equivariant homemorphisms, and Homeoj(M) is the subgroup of maps taking each G-orbit to itself. Similarly, in the

differentiable category there are corresponding groups of diffeomorphisms, e.g. Diff j (M).

In our constructions M will be a homogeneous G- sphere bundle 3(G xx^D(l+^) = G xi^S^.

Typically, Xj will be a closed tubular neighborhood of a singular orbit ~ G/K, namely a disk bundle G xKD<i+1 . In fact, two such pieces Xj will suffice if Y has less than 3 vertices, cf. Chapter IV. The result of attaching Xj to a G-manifold X' depends on the choice of glueing map (|)e Diff^M). Therefore, we shall need a closer look at the group Diff°(M) in order to control the glueing construction. The purpose of this chapter is to provide the necessary information of this kind, which will be applied in the more concrete situations of Chapter IV and V.

Actually, we are primarily interested in the subgroup Diff j (M) of the less tractable group

Diff^M). This is so because we may regard the (subdivided) orbit space Y = uYj as fixed,

namely it is invariant under the equivariant twisting procedure. The latter construction takes

two G-submanifolds X j and X' of Sn , with a common boundary component M, say X]/G =

Yj , X7G = Y1 = Y2 u . . , and "glues" them together along M. The invariance of Y | U Y '

means that the "glueing" map <|)eDiff^(M) belongs to the subgroup Diffj (M) of orbit preserving maps.

22


Our notation and terminology in this chapter is close to that of Bredon [Br3], which also has some useful constructions along these lines. In addition, we have found some relevant ideas in Nakanishi [N], where the reconstruction problem is investigated for some of the simplest orthogonal models with G a classical simple group. For the specific orbit structures assumed in [N], the results are in agreement with ours.

§1. G-diffeomorphisms of manifolds of cohomogeneity one

Let Kj , K 2 . . be subgroups of G. Their simultaneous normalizer in G is denoted

N G ( K j , K 2 . . ) = N Q C K J ) n N Q ( K 2 ) n . . . In the case that H c K j and H is a principal

isotropy group of some specified G-action, we shall write

G = NG(H)/H , NG(H, K) = N G (H, K)/H , NG(H, K1, K2) = N G (H, K1, K2)/H

Similarly, Aut(Ki, K2 . .; G) is the group of automorphisms of G leaving invariant the

indicated subgroups Kj. Finally, our convention is that all groups of homeomorphisms

M —> M are regarded as left transformation groups on M. Let M be any G-space whose orbit space is a closed interval I. Then it can be analyzed in

terms of the double mapping sylinder construction (cf. e.g. [S6, Chap.IV]). Let's first recall the basic facts. To each canonical projection n : G/H —> G/L one constructs a G-space, namely the mapping sylinder

(1) M(TC) = (G/H x [0, 1] u G/L ) / ~ , (gH, 0) ~ gL .

Given a triple (LQ D H c L j ) i n G , there are canonical projections 7ij: G/H —» G/Lj, i = 0,1, and one constructs the union of their mapping sylinders by identifying their common "boundary" orbit ~ G/H :

(2) M = M(TC0, 7ti) = ( G/H x [0, 1] u G/L0 u G/L1) / - ,

( g H , 0 ) ~ g L o , ( g H , l ) ~ g L i .

There is a natural identification M/G = I = [0, 1], so the orbit structure looks like

(3) M/G = I : L 0 - Lx

In (3) we shall actually consider "individual" groups rather than conjugacy classes, namely

(3) indicates a cross-section C c M with specified isotropy groups. Clearly, (G, M) can now be reconstructed from the data in (3) via (2). Conversely, any G-space M satisfying M/G = I

24 ELDARSTRAUME

has a cross-section with constant isotropy group H on the image of (0, 1). Let L0 and L] be the "end point" isotropy groups, as indicated in (3). We shall refer to (L0 z> H cz Lj) as an associated triple for (G, M). However, we also demand that M shall have the topology of a manifold ; the appropriate condition on the groups is that (H cz Lj) is a spherical pair , in the sense that Lj/H « Smi is a sphere, mj > 0 (cf. also [W]).

Suppose Lj cz K cz G, and therefore (L0 z> H cz Lj) is a triple in both K and G. Then the construction (2) gives a K-space M' as well as a G-space M naturally containing M' with a common (canonical) cross-section, (3), so that there is a natural identification M/G = MVK = I. In fact, M may be expressed as a G- homogeneous fibre bundle over G/K with fibre (K, M') :

(4) M' -» M = G xK M* -> G/K

Given a pair H cz G of compact Lie groups, there are natural homomorphisms

N(H) ->N(H)/H R - > Diff°(G/H) , n -» R_ : gH -» gn^H

(5) I

Aut(H; G) P-> Diff(G/H) , cp -> p^ : gH -» <p(g)H

where R : N(H)/H -> Diff°(G/H) is an isomorphism. Note however, the natural homomorphism, "conjugation by n": N(H) —> Aut(H; G), will not make the diagram (5) commutative since its composition with p gives the map p^ = L^ oR_, where Lj| is left

translation in G/H. We shall find an extended version of (5) which is valid if G/H is replaced by M in (2). The

smoothness of the homeomorphisms involved will depend crucially on the next lemma. Let (H c L) be a spherical pair. It is well known that there is a unique representation *F : L —> 0(m+l) of cohomogeneity cQ¥) = 1, and with H as a principal isotropy group. Via this action of L we shall identify L/H with the unit sphere Sm.

Lemma 1.1 Let Sm = L/H be as above and let G = L in (5). Then the corresponding homomorphism p in (5) maps each automorphism to an isometry, namely

p : Aut(H; L) -> 0(m+l) c Diff(Sm).

Proof The homomorphism p factorizes as

Aut(H; L) -> Aut(H/ker; L/ker) -> Diff(Sm),

where ker = ker *F is the ineffective kernel of the spherical pair (HcL) . Therefore we may assume (H <z L ) is an "effective" pair, namely L <z 0(m+l).

Let L have the Riemannian metric induced from the bi-invariant metric of 0(m+l). In particular, conjugation is an isometry and L —> L/H = 0(m+l)/0(m) = S m is a Riemannian submersion, where the metric on Sm is standard. Hence, if (pe Aut(H; L) is an inner automorphism of L, then the induced map Sm —> Sm is an isometry. So, it remains to check those cases where cp is an outer automorphism of L preserving H. In particular, cp preserves both L° and H° = H n L°, and since L°/H° = L/H we may as well assume L = L°.

Connected spherical pairs are well known, e.g. (L, H) can be read off from [S6, Table I]. We may also assume L is simple, since otherwise L = L j x L2 (up to a finite covering), where L| = U(l) or Sp(l), and L2 is still transitive on Sm. Then we are left with two cases having an outer automorphism a, namely m odd and L = SO(m+l) or SU((m+l)/2). However, the effect of a is just conjugation by some element in the normalizer N(L) c: 0(m+l), and it is straightforward to check that a induces an orthogonal transformation on Sm.

Proposition 1.2 Let M be the G-space in (2), associated with the triple (L0 D H C L J ) ,

where both (Lj, H) are spherical pairs. Then M has a (unique) differentiable structure, making (G, M) a smooth G-manifold with orbit structure as in (3). Moreover, there are natural homomorphisms R (injective) and p into Diff j(M):

NG(H, LQ, LX) R-> Diff?(M), n -> R_ : [(gH, t)] -> [(gn^H, t)]

(6) 1

Aut(H, L0, L i ; G) P-* Diffj(M), P(p : [(gH, t)] -> [((p(g)H, t)]

Proof M is the union of two mapping sylinders M(7Cj), cf. (1), (2). Let M(TT) denote one of these, defined by the triple ( H c L c G ) . Since L/H = S m can be regarded as the boundary of the unit disk D m , there is a natural G-equivariant homeomorphism

(7) M(TC) - G x L D m + 1 , [(gH, t)] <-> [g, fv0] ,

where v0e S m is a point with isotropy group Lv = H. By (7) we shall regard M(TC) as a

smooth homogeneous disk bundle, L acting orthogonally on the fiber via a representation *F of cohomogeneity one. There are also actions of NQ(H, L) and Aut(H, L; G) on M(7C), defined exactly as in (6). Hence, NQ(H, L0, L J ) and Aut(H, LQ, Lj; G) act on both spaces M(7ij), in a way compatible with the union, (2). Therefore it suffices to check that the groups act via smooth transformations on M(7C).

26 ELDAR STRAUME

To this end, any point in D m + 1 can be written as X(VvQ), where It I < 1, Xe L, and vQ as

above. Now, the induced action of tye Aut(H, L; G) on the total space of the bundle satisfies

[g, t-v0] —> [9(g), t-v0], and consequently

(8) [g, Uvw0)] = [gk, fv 0 ] -> [y(gl), fv 0 ] = [cp(g), q>(*)t-v0] ,

and we observe that the map S m —> S m , X(v0) "^ (P(^)vo' *s m e s a m e a s Pep m (^).

By Lemma 1.1 this map is orthogonal, so the map in (8) is smooth. Finally, if ne N Q ( H , L), let cp denote the automorphism g —> ngn . Then the action of n

on the disk bundle in (7) is smooth since it is given by the map in (8) followed by left translation by n" * in G.

Remark 1.3 The above proposition helps clarify the notion of a smooth special G-manifold in the literature, cf. e.g., [Br3, Chap.VI], [J], [N]. There is some technical condition on the isotropy groups involved; this amounts to the explicit assumption that the above action of N Q ( H , L) on the disk bundle in (7) must be smooth. As is shown above, this explicit assumption is no longer necessary, so the problem left open in [Br3, p.368] is settled. Hence, a G-manifold is defined to be special if there are at most two orbit types near each orbit, and the slice representation of each non-principal isotropy group L splits as § - §\ + x^, where T^ is

trivial andc((|)i) = 1.

Let 7t0Homeoj (M) be the set (group) of equivariant homotopy classes over I, and

7i0Diffi (M) the set of equivariant smooth isotopy classes over I. In view of Remark 1.3, the following is now an immediate consequence of [Br3; 6.4, Chap.VI].

Corollary 1.4 Let M be a smooth G-manifold whose orbit space is a closed interval I. Then the natural map

7T0Diffp (M) -»7i0 Homeop (M) is a bijection.

Now, let's specialize to the case where (G, M) is a homogeneous bundle of type (4), whose fiber (K, M') = (K, S^) is an orthogonal K-sphere with S^/K = I. The remainder of this section is mainly devoted to a proof of the following theorem, which has an important consequence, namely Corollary 1.6.

Theorem 1.5 Let (K, S°l) be an orthogonal K-sphere whose orbit space is the interval I, and let

(L0 3 H cz Lj) be the triple in K associated with a cross section : I = S^/K —> S^, cf. (3).

Then the images of the corresponding homomorphisms R and p in (6) lie in 0(q+l), namely


A u t ( H , L 0 , L i ; K ) P ^ Diff I(S ( l)nO(q+l) = K ,

where K is the maximal linear group containing K and C- equivalent to K. The group NT^(H, L Q , L | ) is identified via R with a subgroup of K which commutes with K.

Corollary 1.6 Let K c G and O: K -^ 0(q+l) , c(O) = 2 and (L0 z) H cz Lj) as above.

Regard M = G xj^S^ as the sphere bundle of the G-homogeneous euclidean vector bundle

G x ^ 4 " 1 . Then the subgroup NG(H, LQ, Lj) cz Diff^(M), cf. (6), consists of orthogonal

sphere bundle maps.

In particular, each element of N G ( H , L0 , L^) can be extended to an element of

Diff^(G xj^D^"1"!), that is, to an equivariant diffeomorphism of the associated disk bundle.

Proof of 1.6 Let ne N G (H, L 0 , Lj) . Since L 0 and Lj generate K (cf. [S6; Chap. IV,

2.2(b)]), we have also ne N Q ( K ) . NOW regard both (G, M) and (K, S^) as a union of two

mapping sylinders, (2), using the same triple (L0 Z) H c Lj) in both cases. Applying (6) to

(G, M) we have Rn : [(gH, t)]-> [(gn^H, t)] .

On the other hand, let v = [(kH, t)]e S^ and observe that [g, v]e M = G xj^S^ corresponds to [(gkH, t)] in the notation of (6). Then

Rn : [g,v]^[gnKpn(v)]

where p n : S^ —> S^ is the image of n by the composition

N G(H, L 0 , L j ) -> Aut (H,L 0 ,L 1 ;K) -> Diff^S^) , cf. (6).

By Theorem 1.5, p n is an orthogonal transformation. Clearly, Rfi extends to a map in

DiffG(GxKD cl+ 1) which is orthogonal on fibers.

In the sequel we shall prove several lemmas leading to a proof of Theorem 1.5. Let K ' c K

be positive dimensional closed subgroups of 0(q+l) of cohomogeneity 2. We write S^/K' = I' ~

I = S^/K and (L0 D H C L J ) denotes a triple associated with a cross section I -» S^. K =

Np^(H)/H is the H-reduced group of K (cf. §1, Chap.I). Similar notation is used for the K'-

action on S^.

We say K is p -determined by K' if the homomorphism p : Aut(H, LQ, L j ; K) -» Diffj(S(l)

for (K, S^) is related to that of (K', S^) by a map JLL SO that

28 ELDAR STRAUME

(9) p : Aut(H,L0 , L ^ K ) ^ Aut(H', L'0 , L ' j ; K') P -» Diffr(S<l).

In particular, Theorem 1.5 holds for K if it holds for K'. The following is an easy consequence of the definitions, cf. (2), (6).

Observation 1.7 Assume K ' c K are C-equivalent groups, say H' = K' n H etc., and assume also Aut(H, L0 , L j ; K) e Aut(K'; K). Then K is p -determined by K\

Lemma 1.8 K is p -determined by its connected component K°.

Proof By the above observation we shall assume K and K° are not C-equivalent. Then K/K° acts by reflection on the arc I' = S^/K0 and one of the two halves may be identified with I. Consider the groups K° c K' c K, where K' is the kernel of the K-action on I\ Here K' is C-equivalent to K°, and clearly Observation 1.7 applies to this pair. Moreover, K' is a group of index 2 in K, say K = < K', y > where y2 e K'.

It remains to show that K is p -determined by K'. In terms of cross sections and associated

triples we can make the following choices, namely H = H', L 0 = L'0 , L j = < H, y > , L'j =

yLQy" . Since K' is generated by L'0 and L'^ (cf. proof of 1.6), it follows that K' =

< LQ, yLoy* >. Consequently, there is a map \i defined by restricting automorphisms

Aut(H, L 0 , L j ; K) = Aut(H, L 0 , Lv K'; K) ^ Aut(H\ L'0 , L\; K')

and it is not difficult to see that this is compatible with (9), namely K is p -determined by K'.

Lemma 1.9 If K is finite, then Difff (S^ ) c 0(q+l) .

Proof By [S4, p.3], K is some dihedral group D k c 0(2), 1 < k < 6, k * 5. Here 0(2) =

Iso(S*), where S* = F(H, S^), and an arc of length 7C/k on S^ is a cross section of S^/D^ =

S^/K. For simplicity, put K = Diff t (S^ ). Each map (|)GK leaves S^ invariant, and ((> is

determined by $ | S , so K injects to a subgroup of Diff t k (S1). Now, if k > 2 then the latter

group is trivial or {± Id}, consequently K is also trivial or {±Id} in this case.

Assume next k = 1 or 2, which implies Diff^ (S1) = D k cz 0(2). The splitting case (K,0)

= (K^Oj) © (K 2 ,0 2 ) is easy; here c(Oj) = 1 and clearly K = D k acts orthogonally on S^.

The reducible (non-splitting) case O = Oj + Q>2 follows easily from the splitting case, so we are

left with the case that (K, O) is irreducible. However, this implies k > 2, e.g. by [S4, p. 12], hence the last case is actually impossible.

Inn(-; K) c Aut(-; K) denotes the subgroup of inner automorphisms, and Out(-; K) is the quotient group.

Observation 1.10 p maps Inn(H, LQ, L j ; K) into 0(q+l) if and only if R maps

N K ( H , L0 , L]) into 0(q+l) . More precisely, let p n be the image of n by the composition

N K (H, L0 , Li ) -> Inn(H, L 0 , L j ; K) P -> Diff^S^),

where the first map is the natural surjective homomorphism. Then p n = <D(n)Rfi, where

O: K —> 0(q+l) is the given representation, cf. (6).

Lemma 1.11 If (K, O) is reducible, then p maps Aut(H, LQ, L j ; K) into 0(q+l) .

Proof By Lemma 1.1 and 1.8, we may assume O: K = K° c 0(q+l), O = O 0 + O^

and both summands are nontrivial. Consider first the splitting case, namely (K, O) =

(K0,<X>0) 0 (Ki ,0^) . Let Hj <z Kj be an isotropy group of Oj, so that LQ = H 0 x Kj ,

L | = K 0 x H j , H = H 0 x H j . It is not difficult to see that each (pe Aut(H, L 0 , L j ; K) splits,

that is, cp = (p0 x (p^e Aut(H0; K0) x Aut(Hi; K| ) . By Lemma 1.1, each (pj induces an

orthogonal map of Smi = Kj/Hj, and the induced action of (p on the join S°l = Smo * S m l <z

[R m o + 1 ©[R m l + 1 , where m 0 + rai+ 1 = q, is just the join of the two orthogonal maps. The

latter map is clearly orthogonal, too.

Next, suppose (K, O) does not split. Then we may regard (K, <f>) as the diagonal subgroup of the splitting extension (K, <DQ) 0 (K, Oi) , which is C-equivalent to (K, <£). But

each cpe Aut(H, L0 , L | ; K) is the restriction of (p x (p G Aut(H', L'0 , L ' j ; K'), where K' =

K x K etc., so (K, O) is p -determined (cf. (9)) by the splitting group, and the first part of the proof applies.

We shall complete the proof of Theorem 1.5. The above lemmas have reduced the proof to the case of a connected and irreducible linear group (K, O). Furthermore, for K finite we need only check the image of Out(H, L 0 , L j ; K). The discussion is divided into two cases.

Case (a) K is finite : Out(-; K) = 1, except that Out(-; K) has order 2 when (K, O) =

(SU(3), Ad) or (Sp(2) x Sp(2), v 2 ®[HV2)- L e t (t)G A u t ( " ' K ) represent the nontrivial outer

automorphism. We need to show that the image of § by p : Aut(-; K) —> Diff(S^) is an orthogonal transformation.

30 ELDAR STRAUME

K = SU(3): 0 can be chosen to be "complex conjugation" in 1R° = Lie algebra of

SU(3). We put K' = < K, c >, where c e 0(8) and conjugation by c corresponds to

0G Aut(K). Note that K = D3 and K' = D^. Now, (j) extends to an inner automorphism (j)' of

K', and the image of ((v by p ' : Aut(-; K') —> Diff(S') is an orthogonal transformation, by

1.9 and 1.10. However, this transformation is also the image of <|> by the map p.

Note The transformation arising from § is the identity on S - F(H, S'), and it can be checked that it is given by A —> - A , where A is the complex conjugate of the skew-Hermitean matrix A. This map is orthogonal for the Ad-invariant metric < A, B > = -Tr(AB).

K = Sp(2) x Sp(2) : Choose ty to be "flipping of factors". The same idea as in the previous case works. An explicit description of the induced orthogonal transformation on [Rl6 _ ^ 2 0 ^ 2 i s A (g)B _^ B0A , and this map is orthogonal.

Case (b) K is not finite : (K, O) is a complex linear group (cf. #8a - #1 lb in Table II of

[S6] ). K = 11(1)2 £ £2 is a 2-dimensional torus extended by the semi-direct factor 2>^ flipping

the two U(l) factors. Explicit calculation of K = N^(H, LQ, L^) shows that K is the diagonal

circle subgroup {(z, z); ze U(l)} of K, and the image of K in Difff (S^ ) turns out to be the centralizer of K in 0(q+l) .

There are also examples with Out(-; K) ^ 1, but it is not difficult to show the orthogonality of the induced map S^ —> S^. Since the calculations are similar in all cases, we shall choose the

linear group (K, O) = (U(5), A 2 | l 5 ) . Here H = Sp(l) x Sp(l) x U(l), L 0 = Sp(2) x U(l),

Lj = Sp(l) x U(3), and the central circle group of U(4) x 1 lies in Nj^(H, L0 , L | ) and

represents K.

In F(H, (E10) = (E (e] A e2) + (E (e3A e4) = (T2, K = {z = e i e} acts by complex multiplication v —> z2v, and since the action of K on S1^ commutes with K and is uniquely determined by its restriction to S^ = F(H, S1^), K acts also by complex multiplication on S , in particular, the K-action on S1^ is orthogonal. By Observation 1.10, the same conclusion holds for the group Inn(-; K).

Finally, Out(-; K) = H^ and the nontrivial element is represented by "complex conjugation" (|) of U(5) = K. The normalizer of K in O(20) is K' = < K, c >, where ce O(20) is "complex conjugation" in (C1^. Note that 0((|)(k)) v = c(0(k)v) for each VE (E1^. K' is finite ( = D4) and K' is C-equivalent to K. Similar to the case K = SU(3), the p -image of $ in Diff(S^) equals the P'-image of <|>, regarded as an inner automorphism of K\ But this image is an orthogonal transformation, by 1.9 and 1.10. In fact, it turns out that this transformation is the previous map by "complex conjugation", c : S ^ —> S ^ .


§2. G-disk bundles of cohomogeneity two and equivariant attaching

We start with a closer look at the group Homeo j (M), where M is a G-manifold whose orbit space is a closed interval I . As in §1, we shall regard M as a double mapping sylinder, (2), and we fix a cross section I —> M, t —> [(eH, t)], with associated triple (L0 D H C L J ) . Then G-homeomorphisms can be distinguished by how they transform the given cross section, cf. [Br3, Chap.V, §4 ]. Consequently,

(10) Homeo^(M) = {G : I -» G ; a cont., a(t) -» NQ(H, Lj) as t -> i, i = 0, 1}.

Actually we need only work with the technically more convenient subgroup

(11) Pj(G ; LQ, Lj) = {a : I -> G ; a(i)GL|, i = 0, 1}, where Lj = NG(H, Lj).

Remarks 2.1 (i) The group operation in (10) is "pointwise" multiplication (G1G2XO =

a2(t)a^(t); then the group acts on M from the left, cf. (6).

(ii) The group N^(H, L 0 , L | ) = NG(H, L0) n NG(H, Lj) is naturally identified with a

subgroup of the group in (11), namely the group of homeomorphisms defined by constant

paths G. This subgroup lies in Diffj (M) since it corresponds to the embedding R in (6).

The group 7iQPj(G ; L0 , L]) consists of homotopy classes of continuous paths a : I —> G>

relative to the condition that G(i)eNG(H, Lj) during a deformation of G, i = 1,2. Combining

Remark 2.1(h), Proposition 1.2 and (10) we obtain :

Corollary 2.2 Let (G, M) be as above. (i) The natural map

7C0Pj(G ; L 0 , 1 ^ ) -> 7C0Diff?(M) is a bijection .

(ii) If G is finite, then the embedding R in (6) is an isomorphism

NG(H, L ^ L ^ ~ - > Difff (M) = Homeo^(M).

In general, we shall measure the "complexity" of Diff?(M) by passing to G-isotopy classes modulo classes in the subgroup NG(H, LQ, Lj). Therefore we introduce the following coset space as a measure of "complexity".

32 ELDAR STRAUME

Definition 2.3 The twist space of (G, M) is the left coset space

r(G,M) = ^ D i f t f C M V ^ N ^ H , ^ ^ ! )

Observations 2.4 The twist space T(G, M) is trivial in the following two situations : (a) G is finite. (b) The conditions Q0 n Q\ ^ 0 and 7Cj(Q/Qi)=l hold for any choice of connected

components Qj of N^(H, Lj), i = 0, 1, whenever both Qj lie in the same component Q of G .

The above constructions and results will be applied to the following situation. Let Y be a given G-manifold of cohomogeneity 2, with a boundary component (dY)0 « G x^S^ = M, where (K, S^) is an orthogonal K-sphere and M/G = S /K = I. Our problem is to "determine" the different G-spaces obtained by equivariantly attaching the G-disk bundle G xj^Dcl+1 to Y along (9Y)n in various ways, depending on the attaching map (J) e Difrf(M).

We start by fixing an identification of (dY)0 with G xj^S^ , so that

(3Y)0 = Y n [ G x K D ^ + 1 ] = GxKS^ = a L G x ^ * 1 ] = M,

that is, Y and the disk bundle have a common boundary component M. The construction of the resulting spaces XA is illustrated by the diagram

(12) XA = [GxKD<l+1] U A Y

T t

G xKS^ = M *-» M

Remarks 2.5 (i) We are only considering attaching maps <|) which preserve G- orbits, namely ty e Diff^(M). This is technically convenient, and the justification is explained at the beginning of § 1.

(ii) It is well known that XA depends, up to equivariant diffeomorphism, only on the

equi variant isotopy class of (|>. Hence, XA in (12) depends only on [(()] G7i0Diffj (M).

Lemma 2.6 Suppose §Qe Diff<f(M) extends to some <jT0 in Diff^G xKD<l+1) or Diff°(Y).

Then XAA and XA are G-diffeomorphic for each <|> e Diffj (M).

<|)eDiff?(M)

Proof Referring to (12), define the map

¥ : X ^ X # Q ; x ^ $0-l(x) for x e G x ^ y ^ y f o r y e Y .

To check that *F is well defined, observe that xe G xj^S^ and <|>(x)e Y are identified in XA.

But their ^-images §0~ ' (x) and <|>(x) respectively, are identified in XAA since (|)(|)0(t)0~ (x) =

(|)(x). It is straightforward to check that *F is a G-diffeomorphism.

The proof also works if the roles of G x^D^"1"1 and Y, resp. the order of <j) and <|)0, are

interchanged. Recall from Corollary 1.6, each <j)0 in the subgroup N Q ( H , L Q , L J ) satisfies the

extension property in the above lemma. Therefore the following theorem, stated here for easy

reference, is an immediate consequence of 2.5(h) and the above lemma.

Theorem 2.7 Let Y be a differentiable G-manifold having a boundary component

(3Y)Q « M = G xj^SQ, where (K, S^) is an orthogonal K-sphere and S^/K = I an interval.

Consider all G-manifolds XA constructed by equivariantly attaching the G-disk bundle

G xKD c l + 1 along (3Y)0 , cf. (12). Then

(i) (G, XA) depends only on the (left) coset of [(j)] in the twist space T(G, M).

(ii) If (|)0 extends to some (j>0 in Diff^Y), then XA A and XA are G-diffeomorphic.

Chapter IV. G-spheres of cohomogeneity two with at most two isolated orbits

In the next three sections we shall complete the classification of all G-spheres X = Xn with 2-dimensional orbit space having at most 2 isolated orbits (or vertices, see Figure 1). The final result is expressed in Theorem 3.9. In Section §1 we reduce the problem to a construction used by Milnor in his construction of exotic spheres, cf. [Ml]. Certain 3-dimensional lens spaces (cf. [M2]) will play an important role in connection with the reduction principle. What we need about this family of spaces is worked out in §2. Finally, in §3 we complete the program by explicitly constructing G-spheres with one of the above lens spaces as reduction, namely as the G -invariant subspace X = F(H).

Our approach in Chapter IV and V has been motivated by the belief that the "genetic code" of (G, X), namely some "simple" data from which we can reconstruct (G, X), is very well represented by the reduction (G, X) (cf. also [SS]). In retrospect, we can say that this is precisely the situation.

§1. Description of the G-sphere as an equivariant twisting of the orthogonal model

Let (G, Xn) be a G-manifold, say a homology n- sphere mod 2, having an orthogonal model (G, O, Sn) of type 0,1 or II, see Figure 1. We fix an identification Xn/G = Sn/G = D , the isolated orbits are special points, called vertices or vertex orbits, on the boundary circle of D . However, in order to unify the arguments in this chapter we shall always work with two vertices, since nothing will prevent us from choosing an additional "vertex" (or two vertices ) of angle n if necessary. We think of the vertices as two "antipodal" points on the boundary, and Kj and K2 are specific isotropy groups corresponding to these orbits. Our notation is also illustrated by Figure 1.

Let us first introduce, in cases (a) and (b) below, common notation and recall some information about the above orthogonal models (G, O, Sn):

(a) Assume F(G) * 0 . Then K| = K2 = G and O = <f>' + x^, where Tj is a 1-dimensional trivial summand. Of course, the orbit structure is of Type 0 (resp. II) if O' has no (resp. one) summand Tj. In any case, the vertex orbits are two fixed points pj. Let Nj <z Sn denote the half spheres ~ Dn with pj as center, regarded as a tubular neighborhood of pj. Then (G, Nj) is equivalent to the orthogonal action (G, O', [Rn) restricted to the unit disk Dn.

34

(b) Assume F(G) = 0 . Then O = ®l + <$2 where c ^ ) = c(0>2) = 1 and G/Kj - S^i,

qj > 0, i = 1,2. The slice representation of Kj in the euclidean space is the restriction

(Kj, Oj, IR j ) and dim S j/Kj = 1, i j . Tubular neighborhoods of the vertex orbits are

(1) Nj= G x K D^2+ 1« S ^ l x D ^ + l , N2 = Gx K 2D ( H+ 1 =S ( l2xD c l l + 1 .

Here G acts diagonally on the product spaces (as subsets of , with the orthogonal action given by Oi and <X>2). The reason for the splitting is that the Kj -action on

the appropriate fiber D^j4"1 of the disk bundle Nj extends to a G-action on the same disk,

namely the linear action via 3>j. In particular, G acts diagonally on dNj ~ S^l x S^2, namely

orthogonally and via Oj on S^i.

Next, we claim that the tubular neighborhoods Nj can be chosen to intersect along their common boundary, denoted by M :

(2) s n = N 1 u N 2 = N 1u i ( iN2, Nj n N2 = 3Nj = M

In case (a) this is obvious, and here M = Sn . In case (b) we have M = S^lxScl2and(2) can be explained as follows. Recall the standard "straightening angle" procedure whereby the product of two closed unit disks in euclidean spaces is identified with a unit disk, say

, n = qj+ q2+ 1. This correspondence is in fact G-equivariant, and restriction to the boundary (G, Sn) renders (2) as the well known decomposition

(3) Sn = 3 p ^ l + 1 x D^2+1] = [3D^l+1x D^2+1] u [D^l+1x 3D^2+1]

= [S^l x D°12+1] u i d [D^l^x S^2] = N t u i d N2 .

Remark 1.1 We shall also regard (2) as a special case of the "equivariant attaching" construction which is depichted by diagram (12) of Chapter III, namely the Nj are glued together by the identity map M —» M :

Sn = N 1 u i d N 2

(4) t t M ldH>M

Figure 2 illustrates the decomposition of Sn at orbit space level. We note that M/G ~ I may be viewed as the common boundary arc of the two half-disks Nj/G in Figure 2.

36 ELDAR STRAUME

Consider one of the above orthogonal models (G, O, Sn) and let Xn be a G-manifold (say, a homology sphere) having the same orbit structure. First of all, we claim that the slice representation of Kj in X n must be the same as in the orthogonal model, essentially because

i) the group Kj belongs to a very specific family of subgroups of G, and ii) Kj -representations of cohomogeneity 2 are very special, and the weight system of

the slice representation of Kj is determined by a standard procedure (cf. Chapter II, §1).

In particular, the vertex orbits have the same tubular neirhborhoods Nj as in the orthogonal

model. Moreover, in this more general situation one can see from the rather simple orbit

structure that there is no obstruction to a suitable "extension" of each Nj, up to equivariant

diffeomorphism, so that Ni n N2 = 9Nj ~ M still holds. Therefore (G, Xn) and its

orthogonal model (G, Sn) can only differ by a "glueing" G -map cp : M -» M. We may as

well assume (p belongs to Diff T (M), cf. Chapter III, §2.

In other words, replacement of the identity map in (4) by (p leads to the "twisted sphere" Xn , and we shall refer to the above glueing construction as equivariant twisting. It is, in fact, an equivariant version of the "twisted sphere" construction in [Ml] which lead Milnor in the late 1950's to the discovery and construction of exotic homotopy spheres.

Thus, Theorem 2.7 of Chapter III together with the above observations immediately render the following theorem, which is the crucial step towards the solution of the Reconstruction Problem.

Theorem 1.2 Let (G, Xn) be a G-manifold having (G, O, Sn) as orthogonal model. Then (G, Xn) can be constructed by equivariant twisting of its orthogonal model :

(5) X n = X^ = N T U(pN2 , M / G « I = [0, 1]

T t M (p-> M , cp e Diff^(M)

Moreover, (G, X5) depends only on the class of [cp] in the twist space of (G, M), and there are the following two cases :

(a) F(G) ^ 0 , M = S n _ 1 e S n (twisting along the equator hypersphere),

(b) F(G) = 0 , M = S^l x S ^ (twisting along a "generalized torus" ).

Note Recall from Chapter III, §2, the twist space T(G, M) is calculated from an associated

triple ( L J D H C L2) of the G-space M. The triple can be chosen to be an associated triple

of (Kj, S^i), where S^i is the unit sphere in the slice representation of Kj. The groups Kj,

Lj can be chosen as indicated by Figure 2, where L 2 ^ L'2 if and only if F(G) = 0 .

To terminate this section we shall "complete" the simplest case of the above theorem, namely case (a). The final result goes as follows.

Theorem 1.3 Let (G, Xn) be a differentiable (Z-homology) G-sphere satisfying dim Xn/G = 2 and F(G) * 0 . Then (G, Xn) is equivariantly diffeomorphic to its orthogonal model.

Proof In the orthogonal model G acts on S n c: (Rn+1 by a representation O = O' + x\. The construction in (5) amounts to

(6) X n = X£ = D n u ( p D n , SnA ^ S 1 1 " 1 , cpe Diff^CS11-1),

where M = S n _ 1 is the unit sphere of (G, O', (Rn). We claim that the twist space T(G, M) is trivial, and this will complete the proof. The claim follows from Observation 2.4 of Chapter III by checking the orbit structures of linear groups (G, O ) of cohomogeneity two.

Consider an associated triple (Lj ID H <z L2) for (G, M), and the subgroups N Q ( H , LJ)

of G = N(H)/H. By Observation 2.4 of Chapter III, T(G, M) is trivial if G is finite or if

dim N Q ( H , LJ) = dim G for some i. This settles, for example, all cases where (G, O') is

splitting, in particular when Lj = G, that is, when O' has a trivial summand.

Assume next (G, <£') is nonsplitting and dim G > 0. (In [S6, Table II] these groups are

labelled #m, m > 7 (except #12, #14, #16a). Then either O' is complex irreducible, in which

case G ~ N T J ( 2 ) ( U ( 1 ) x U(l)), or O' is reducible. In the latter case G is one of the groups

0(2), U( l ) 2 , U(2), Sp(l) 2 or U(l) x SU(2). Direct calculation of H c: Lj shows that

dim N Q ( H , Lj) = dim G still holds for at least one i, except in the special case (G, <D) =

(SU(4), p 6 + [ji4]R ). But here G = U(2), NG(H, Lj) = SU(2), NG(H, L2) = U(l ) 2 ,

so Observation 2.4(b) of Chapter III still applies.

Remarks 1.4 (i) Theorem 1.3 also seems to hold more generally for G disconnected. Namely, one first needs to extend Theorem D (cf. Introduction) to disconnected groups acting with F(G) ^ 0 . Then the triviality of the twist spaces should be checked for disconnected linear groups (G, O') of cohomogeneity c(O') = 2, and these are well understood (cf. [S4]).

(ii) An alternative approach to prove Theorem 1.3 goes roughly as follows. By removing a fixed point one obtains a G-action on a space ~ \Rn , and this action must be equivalent to an orthogonal action on [Rn, using [MSY] or [Br3, p.208]. From this one should be able to conclude that (G, Xn) itself is equivalent to an orthogonal action.

38 ELDARSTRAUME

§2. The basic Lens spaces L^ as 3-dimensional models

In this section we cancel our "bar" notation for the reduction of a linear group, since we are solely working with the latter type of groups. So, (G, O, S3) denotes the minimal reduction of

an orthogonal model (G, 5>, Sn) of Type 0,1 or II, see Figure 1. Note that (G, O) is a linear group on euclidean 4-space, depending only on the type in question :

(7) Type 0 : G = 0(2) <D = 2p2

Type I : G = 0(2)

* = P2 + P'2

(kerp'2 = Z2)

Type II : G = 0(2)xO(l)

3> = P2 + P2®Pl

We shall write O = Oj + 0 2 , where O1 = p 2 . The purpose of this section is to construct all

possible G-manifolds X 3 having the same orbit structure as one of the above three reduced

models (G, O, S3).

We shall follow the notation from §1, such as in (3) and (5), but Sj c Dj denotes the unit

circle in the unit 2-disk of the representation (G, Oj, [R2). In particular, G/Kj = Sj and Nj =

Sj x Dj, i ^ j , in (3). The reduction commutes with equivariant twisting, so the reduced version

of (5) depicts the construction of a 3-dimensional G-manifold X 3 as the result of an appropriate

equivariant twisting of the 3-sphere S 3 along an embedded torus :

(8) X 3 = L^ = [ S 1 x D 2 ] u q , [ D 1 x S 2 ]

S j X S 2 (h—> S j X S 2

cpeDiff^ (S] x S 2 )

Here M = S j x S 2 is the 2-dimensional torus with the diagonal action of G via O^ and 0 2 ,

(9) (S! x S 2 )/G = I : Lj — L 2 , cf. Chapter III, § 1.

As noted in Theorem 2.7 of Chapter III, the G-space L^ in (8) depends only on the class

[cp]e T(G, M); Lm is a lens space, see below.

To fully understand the above family of 3-dimensional manifolds we shall have a closer look at their construction, (8), together with the twist space T(G, M). To calculate the twist space we need, first of all, an associated triple (L| z> H c L2) of (G, M).We start with a description of the appropriate subgroups of G.

Let Q be the (standard, diagonal) maximal 2-torus of G, namely Q = 0(1)2 or 0(1) 3 for G = 0(2) or 0(2) x O(l), respectively. Elements of Q are written as triples y = (e^, e2 , e),

where £j = ±1, e = ±1, but 8 = 1 if G = 0(2). All the isotropy groups Lj or Kj (cf. Figure 1) can be chosen to be subgroups of Q, namely

TypeO (10) Type I

Type II

Kj = Lj = 0 ( l ) x l

Kj = Q , L ! = K 2 = L ,2 = 0 ( l ) x l , L 2 = 1 xO( l )

Kj = (8J = 1), K 2 = (£ !=£) , Lj = Kj PI K2 , L 2 = (Sj = 1, 8 2 = 8),

L'2 = ( e 2 = l , e i = e ) .

In particular, N Q ( L J ) = N Q ( L 2 ) = Q holds in each case, and clearly H = 1 (cf.(9)).

Lemma 2.1 Let (G, M) = (G, SjxS2) be as above. Then the group n0 Diff^(M) is naturally

isomorphic to the semi-direct product Z x Q defined by the product rule

(k, y)(k', f ) = (k + sgn(y)k\ yy1), where sgn(y) = (8j82) = ±1.

Namely, the corresponding homomorphism Q —> Aut(Z) = {±1} is given by y—> sgn(y).

Proof We may replace 7C0Diffj (M) by the group 7iQPj(G; LQ, Lj), see Corollary 2.2 of

Chapter III, where in this case

(11) P I (G ;L 0 ,L 1 ) = { a : I - ^ G ; a ( i )eQ , i e 3 l } ,

that is, the set of continuous paths in G with end points in Q. The subgroup 7L represents

homotopy classes [a] of paths starting at l e G° = S0(2) = {e1^, 0 < 8< 2n}, SO that the upper

half-loop, i.e., 0 < 9 < n, corresponds to l e 7L. (The lower half-loop corresponds to -1.) On

the other hand, each element of Q c G is identified with the class of a constant path. Thus, 7L.

and Q are naturally embedded as subgroups of 7i0Pj(G; LQ, Lj).

Multiplication in (11) is defined pointwise by a^a2(t) = a2(t)a^(t), see also Remark 2.1 of

Chapter III. It is easily checked that Pj(G; LQ, L]) is generated by Zand Q, Zis a normal

subgroup, and moreover, y k y = sgn(y) k, for all y = (8j, 82, e)e Q and ke 2 . Hence, 7L and

Q generate a group whose structure is the semi-direct product in the lemma, and then the

product k y corresponds to the pair (k, y).

Definition 2.2 By Lemma 2.1 and Definition 2.3 of Chapter III, the twist space of (G, M) is naturally in 1-1 correspondence with the integers 2 ,

r(G,M) = (ZxQ)/Q « Z , (k,y)Q<->k.

40 ELDAR STRAUME

Define the twist number of a diffeomorphism ae Diff?(M) to be the integer k defined via

the composition Diff?(M) -* 7i0Diff?(M) -> T(G, M) - TL

The next task is to find a subgroup (or subset) of Diff^M) which realizes all twist numbers and is also tractable for computational purposes. To this end we shall utilize the toral structure of M, namely M = S j x S2 is regarded as the (flat) torus defined by Iz jl =

IZ2I = 1, where (zj, z^) are standard complex coordinates of the representation space ([©d

of O = 0 1 + Q?2- We define the torus automorphism group of M by

(12) Aut(S1xS2) = GL(2,2) = {A= [*|j] : (zv z2) -» (Z ]a z2

b , ZjC z2d) }

Clearly, it consists of the automorphisms of the torus as a Lie group.

Concerning the 0(2)-representations p2, resp. p ^ in (7), note that g = e ^ e U O ) =

SO(2) acts on (C by multiplication z —» gz, resp. z —» g2z. We may also assume that the element g 0 = diag(l, -1) of 0(2) acts on (E by complex conjugation in both p2 and p ^ .

Lemma 2.3 The subgroup of G-equivariant automorphisms in (12) is the infinite dihedral group

AutG(S] x S2) = {ak , a k 6 ; k e Z } c GL(2, Z ) ,

where 8 2 = 1, < a > « 7L and 8a8 = a - . In the three cases of (G, O) in (7) we have

a = 0Cj and 8 = 8j represented by the following integral matrices :

Type 0 : Type I : Type II :

M°iMo=[?i] «i-UJ]-«i-U-0.] -2-l^l-MJ-0,] Note a 2 = (ocQ)2 and 82 = ocQ80. Indeed, (G, O) in the first case is a subgroup of the linear

group in the third case, cf. (7).

Proof Since all three cases are similar, let's choose the second case, i.e. O = p2 + p'2-Let the matrix A in (12) be an automorphism commuting with G. The action of g0 leads to no condition on A since complex conjugation of the torus clearly commutes with all A in (12). On the other hand, since ge SO(2) acts by (Zj, z2) —» (gz1? g z2), A will commute with G if and only if

(13) A = 1 -2b b

2 - 2e - 4b 2b + e b e Z , e = det(A) = ±l

Now, 8 = 1 and b = 1 gives A = ocj, e = -1 and b = 0 gives A = 8^. It is easily seen that these

two matrices generate all matrices of type (13).

Lemma 2.4 The group Aut^(S] x S2) lies in Diff T (M). Moreover, the twist number of a^

is k, whereas otj^Sj has twist number k -1 or k for i = 0, resp. i = 1 or 2.

Proof Let h = 1/2 if G = 0(2) and h = 1/4 if G = 0(2) x O(l). A cross section of (Sj x S2)/G

= I ~ [0, h] is given by

C = { ( l , e 2 7 c i t ) ; 0 < t < h } cz S} x S 2 c (E2 .

A typical point on C is mapped by a = otjE Aut^(S j x S2) as follows :

0> = 2p2

(14) <D = p2 + p'2

0 = p2 + p 2®pj

(i s e27t i t) -> (e2 7 l i t , e27 l i2 t) = e27cit( 1, e27cit)

( 1 9 e27cit) -> (e27ci t , e27 l i3 t) = e27cit( 1, e27cit)

(1 e2rcit) we2rci2t e2m3t\ _ e2m2t( ^ e27iit)

Here the rightmost expression means a(t)(l, e27Ut)e G(C) c: M, where a is the upper-half loop

I —> SO(2) = G° and o(t)e G acts on the torus M. From this we see that a preserves G-orbits,

and moreover, a corresponds to the above path a, regarded as an element of the set in (11). But

this path represents the class (1, e)e 7L x Q = n0 Diff j (M), where e is the unit element of Q, so

the twist number of a^ equals k, by Definition 2.2.

Next, consider the other generator 8 = 8j of Aut^(S | x S2) in the above three cases. In the

last two cases (i.e. i = 1 or 2) it follows from the expressions in Lemma 2.3 that 8: sends

(1, e^711*) to (1, e - 2 7 n t) = g 0 ( l , e27l l t), where g 0e Q is the element defined earlier. So 8j also

leaves G-orbits invariant and 8j corresponds to the constant path at g 0 in (11). Then it is clear

that otjkSj has class (k, e)(0, g0) = (k, gQ) in 7L x Q, consequently its twist number is k.

Finally, 8Q maps (1, e2 7 n t) to (e2 7 n t , 1), which may be rewritten as G(t)(l, e2Tllt) where

a(t) = (e27 l i t)g0 = g0(e~27rit) e 0(2) = G. Again, 8 0 leaves G-orbits invariant. On the other

hand, the product rule of the group in (11) is the opposite of pointwise multiplication, so the

path a is a product (o')g0 where a' is the path t —> e"27Clt. Consequently, the class of ( a 0 ) ^8 0

in 7L x Q is the product (1, e)^(-l, e)(0, g0) = (k - 1 , g0), and its twist number is k -1. This

completes the proof of Lemma 2.4.

42 ELDARSTRAUME

For the sake of easy reference we list the following powers of matrices from Lemma 2.3 :

(15) ( a 0 ) k = • k + 1 k

-k k + 1 , ( a i ) k = -2k+l k

-4k 2k+l , ( a 2 ) k = ( a 0 ) 2 k

Recall that the 3-dimensional lens spaces L(p, q) can be constructed by glueing two solid tori S*x D^ and D2x S* along S'x S , as in (8), via the torus automorphism

(16) (p <-> s p E G L ( 2 , Z ) ,cf.(12),

and L(p, q) ~ L(lpl, ±qf) if q' = ±q (mod p) or qq' = ±1 (mod p), cf. Milnor [M2]. We also remind the reader that the G-manifold L^ in (8) depends only on the class [cp] in

T(G, M), which is equivalent to say that it depends only on the twist number k of cp, by Lemma 2.1 and Definition 2.2. Moreover, by Lemma 2.4, each twist number ke 7Z. can be realized by a suitable torus automorphism. So, L^ is some lens space L(p, q) and, in fact, all the different G-

manifolds L^ are achieved by choosing the automorphisms in (15). The following notation for

these G-manifolds is adopted, and their topological type is indicated :

(17)

(p = (oc0)k

(p = ( a 1 ) k

<p = ( a 2 ) k

L ( p = L (k

0 ) - L ( k + l , l )

L 9 = L(k

1}- L(2k+1, k) - L(2k+1, 2)

L(p = L (k

2 )-L(2k+l , l ) -L (2° k

)

The next problem is to decide which of the above G-manifolds are actually distinct, in each

of the three families {(G, L(k

}), k e 2 } , i = 0, 1, 2.

Lemma 2.5 The following equivalences hold among the G-manifolds in (17):

(o) _ T (o) Lk ~ L - k - 2 ' L k ~ L ( D ^ T ( ] )

-k-1 (2)_ T (2)

-k-1

Proof The idea is to use ''symmetries" in the construction of L^. To simplify the notation we

shall indicate the "diagrammatic" construction of L^ in (8) by writing

(18) L(P = [ ( S 1 D 2) ( P-^ ( D 1 S 2) ] • 9 ^ DiffV(S1xS2)

where (SjD:) represents the product Sj x D:, whose order of factors will be important. By

flipping the factors in one or both products in (8), and simultaneously replacing (p by a

composition with the "flipping" automorphism S | x S2 —> S2 x S j , we shall construct a chain of G-equivalences between spaces, using the corresponding notation (18) at each step. Another G-invariant operation is "inversion" :

[(SjD2) cp -> ( D ^ ) ] « [(PiS2) cp"1 -» ( S P 2 ) ] .

cp will be a torus automorphism, expressed as a product of matrices, cf. (12) and Lemma 2.3 for notation. Observe that 8 0 is the matrix defining the "flipping" automorphism ot the torus.

We shall describe the desired G-equivalences for each of the three types of orbit structures :

Type 0 ( 0 = 2 p 2 ) ;

(19) L £ \ = [ (SP2 ) ( a o ) k 8 o -> ( D l s 2 ) ] ~ t ( s l D 2 ) 8 o ( a o ) k 8 o "> ( S 2 D l ) l

« [(S2D!) 5 0 ( a 0 ) - k 5 0 -> ( S ^ ) ] - [ ( S p j ) ( a 0 ) - k 8 0 -> (D2Sj)] « L%

where the last equivalence is due to the fact that G acts on both disks Dj in the same way.

Type I (O = p2 + p'2 ) » In t n e proof of Lemma 2.4 we observed that the matrix 8j

represents the same class of 7C0Diff j (Sj x S2) as the "constant" g 0 G Q = N Q ( H , L J , L2). By

Corollary 1.6 of Chapter III, 8j extends to an element <Tj G Diff^(S j x D2) . (This can also be

checked directly). Therefore, Lemma 2.6 of Chapter III applies to show L^ ~ L^g for all

cp G Aut^J(S] x S2). On the other hand, since the summands of O are not equal, there is an

"asymmetry" in the representation g 0 —> 8 j . Thus, starting from S 2 x S] instead of S j x S 2

would lead to another matrix, namely 8']

third equivalence below,

1 0 1 -1

. The extension property of 8'] explains the

(20) L(k1} = [(SjD2) (ai)% -» (DjS2)] « [(D2Sj) S ^ ) 1 ^ -> ( S ^ ) ]

« [ ( S ^ ) 8 ^ ( 0 4 ) ^ 8 0 - ^ ( D 2 S 1 ) ] » [ ( S 2 D 1 ) 8 0 8 1 ( a 1 ) - k 8 0 8 ' 1 - ^ ( D 2 S 1 ) ]

- [ ( D ^ ) 8 1 ( a 1 ) - k 8 0 8 ' 1 8 0 ^ ( S 1 D 2 ) ] - [ ( S 1 D 2 ) q > ^ ( D 1 S 2 ) ] = L ^ . (1)

The last automorphism is cp = (8 0 8 ' j8 0 ) (a i ) k 8i = (aj) k ' . We started with the automorphism

( a ^ N ^ rather than (oc ) , both have twist number k by Lemma 2.4, but in the above

construction the latter choice does not lead to any new G-equivalence.

44 ELDAR STRAUME

Type II (O = p2 + p2®pj) • The same sequence of operations as above, (20), also works

here. In this case, however, the matrix 8'2 eGL(2, 2) = A u t ^ ^ xSj) corresponding to the

above 8'] above, is the same as §2 = 2 n * ^ e ^ s t t n e s e c l u e n c e °f automorphisms analogous

to those in (20):

(a 2 ) k , 80 (a 2) k8 0 , 80(a2)-k80 , 80(a2)-k8082 , (a2)-k808280 , 808280(a2)k

where the last product equals (0C2) 82, whose twist number is (-k-1), by Lemma 2.4.

As a direct consequence of the above lemmas and calculations, we now state the main result of §2 as the following theorem.

Theorem 2.6 Let G = 0(2) or 0(2) x O(l) and consider all G-manifolds whose orbit structure coincides with one of the three orthogonal models (G, O, S3) in (7). In each of the three cases there is precisely the following infinite family of distinct G-spaces (cf. (17)):

Type O : L$\ k > -1, Type I : L^1}, k > 0 , Type II : L^2), k > 0

Remark 2.7 For each type, the above lens G-spaces can be distinguished by their fundamental group, namely

7l1(4°)) = Zk+1>

7 t l(41)) = 7T1(42)) = Z2k+1.

In particular, we find that (G, LQ ) = (G, S3) is the orthogonal sphere model itself, and for example, Lr/ = S*x S^ and L\0)= P3 (projective 3-space).

§3. Construction of G-spheres with lens spaces L^ as reduction

In this section we shall "classify" all G-manifolds X^ = Xjl constructed in Theorem 1.2 by

equivariantly twisting an orthogonal G-sphere (G, O, Sn) of Type 0,1 or II, where G is

connected and F(G) = 0 . Recall that (G, X{I ) depends only on the equivariant isotopy class [cp]

of 9 in Diff j (M) modulo a certain subgroup, namely the coset of [cp] in the twist space

(21) T(G, M) = 7C0Pj(G; Lj, L2)/7t0NG(H, Lh L2), M = Sql x Sq2

(cf. Chap. Ill, §2). Therefore the calculation of this set will be our first concern.

The second task is to determine what quotient of the above twist space T actually is the "moduli space" for G-manifolds associated with the given G-orbit structure. We shall handle the latter problem (when T is nontrivial) by studying the possible reductions (G, X^), which turns

out to be the family of lens spaces L^ introduced in §2. The isotropy groups H, Lj, as well as information on G, are described in §1 of Chapter I, at

least for the maximal linear groups (G0,OQ). In fact, the groups listed there define associated

triples (L] z> H c L2) for the cohomogeneity one transformation group (G, M), and this is the

information needed for the calculation of the twist space, (21).

Proposition 3.1 Let (G, O) be a linear group, with no trivial summand, of Type 0,1 or II. Then either G is the minimal reduction G = 0(2) or 0(2) x O(l), or dim G > 3 and G is connected. The twist space in (21) is trivial if dim G > 3, and otherwise there are the following two cases :

i) G = SO(3): T(G, M) has 2 elements.

ii) dim G = 1 : T(G, M) is infinite. Here (G, O) is one of the linear groups in (7), and the twist space described in §2 (cf. 2.1 and 2.2) coincides with the twist space T(G, M), where

M = F(H, M) = F(H, S q l ) x F ( H , Sq2) = S] x S 2 ~ S ! x S 1 .

Furthermore, there is a natural identification T(G, M) ~ T(G, M) (cf. also Corollary 3.4).

Proof This involves of some case by case verification; we shall divide the discussion into five main cases :

(a) G = U(2) or Sp(2). (G, O) is of Type I, and G = U(2), Sp(2) or U(l) x Sp(2).

Write G = G(2) => G( l ) 2 . Now, Lj = L 2 = L and NG(H, L) = G(l ) 2 . By Observation 2.4(b) of

Chapter III, T(G, M) is trivial since 7i1(G(2)/G(l)2) = 1.

(b) G = SO(3). Either (G, O) = (SO(3), 2p3) or (G2, 2<|>1), both of Type 0.

Here N Q ( H , L J ) = NG(H, L2) = 0(2). Consequently, the twist space in (21) is

T = K0?I(SO(3)\ 0(2), 0(2))/7i00(2) - {±1} (two elements),

by standard homotopy arguments, cf. Observation 2.4(b) of Chapter III.

(c) (GQ, O0) = (U(l) x SU(m) x U(l) , [jLi10^m + u ^ u ^ ) (of Type II).

Either G ~U(1) x SU(m), m > 2, or G = GQ. The calculations are entirely similar in both cases,

so let's assume G = GQ. Then N Q ( H , LJ) = N Q ( H , L J , L2) = U( l ) 3 is the maximal torus of G =

U(l) xU(2). Again by Observation 2.4(b) of Chapter III, T(G, M) is trivial.

46 ELDAR STRAUME

(d) (G0 ,O0) = (Sp(l) x Sp(m) x Sp(l), v{®vm + v m ® V l ) (of Type II).

G is one of the three groups G" c G ' c G0 , where G" = Sp(m) x Sp(l), G' =

U(l) x Sp(m) x Sp(l). Write G(i) = O(i), U(i) or Sp(i) when G = G0 , G' or G" respectively.

There are natural inclusions G^ a CT a CP' and corresponding inclusions among related

subgroups, namely G 3 NG(H, Lj) 3 NG(H, L2) : G(2) x G(l) 3 G(l)2x G(l) 3 G(l)2x 0(1) ,

where the last inclusion is Q L J J X G ( I ) 3 G(l) 0

0 e ja :(e2a), £i = ± l , a E G ( l )

We claim that T(G, M) is trivial for G = G' or G". In fact, this follows immediately from

Observation 2.4 of Chapter III, since G/NG(H, Lx) « G(2)/G(l)2 - S 2 or S 4 is simply

connected. On the other hand, for G = G 0 it follows that NG(H, 1^) = NG(H, L2) = 0 (1 ) 3 = Q

is the maximal 2-torus of G = G = 0(2) x 0(1), Thus, we are left with the same twist space calculations involving Q as in §2, and the claim follows from this.

(e) G = 0(2). By §2, (G, <D) is of type O or I. Moreover, either G = GQ or (G, O) =

(Spin(7), 2A7). Straightforward calculations show NG(H, Lj) = % ( H , L2) = O( l ) 2 = Q, and

again T(G, M) is of the type described above (cf.also Definition 2.2).

Remark 3.2 In case (b), where G = S0(3), the nontrivial class of the twist space is represented by the "flipping" map (p : S^ x S^ —» S^ x S^, (x, y) —> (y, x). The corresponding G-manifold is a product of spheres, X^ = S^ x S "1" , with the standard orthogonal and diagonal action. Here

q = 2 or 6 for G = S0(3) or G2 , respectively. Hence, the orthogonal model (G, s 2q+l

) is the only homology sphere with a "bi-axial" action of SO(3) or G 2 (see page 66), in accordance with previous results obtained by Bredon, cf. [Brl, Br2].

Henceforth, we shall assume (G, O) is one of those linear groups with dim G = 1. As shown

above, only twisting of these orthogonal models (G, O, Sn) may possibly lead to non-orthogonal

G-spheres (G, Xn) . For convenience we shall make a list of these groups (G, O), whose

reductions (G, O) are listed in (7):

Type O : G = SO(m), m > 3, O = 2 p m

(22) G = Spin(7), O = 2A? (i.e. G cz SO(8), m = 8)

Type I : G = Sp(2) xSp( l ) , O = v 2 ® H v ] + p 5

G = Spin(9), O = A9 + p 9

Type II : G = Sp(l) x Sp(m) xSp(l) , O = v ^ ^ V j ^ + v m ® | H v 1 , m > 1

Lemma 3.3 Let (G, O j + O2) be one of the linear groups in (22), with reduction (G, Oj +

0 2 ) given by the corresponding group in (7). Write dim Oj = qj +1, M = S^l x S^2 and let (G,

M) be the reduction of (G, M). Then M = F(H, M) = Sj x S 2 is a torus, where Sj = F(H, Sqi)

is a circle with the orthogonal action of G via O j , and G acts diagonally on M.

We omit the the proof of the above lemma which is straightforward and similar for all cases. However, the special case G = Spin(9) is worked out in the proof of Lemma 3.6 below, where

an additional property of the inclusion Sj —> is needed.

Observe that (G, M) is exactly the pair denoted by (G, M) in §2, of the appropriate type.

Now, M/G = M/G = I, and we can choose a common cross section i n M c M with associated

triples (L ] z> 1 c L 2 ) and (Lj z) H a L2) in G and G, respectively, where Lj = (Lj n N(H))/H

is the group denoted Lj in (10). The important fact is that in all cases we have

NG(H, Lj) = (NG(H) n NG(Lj))/H = N Q (ETj) = Q , i = 1, 2,

where Q is the maximal 2-torus of G described in §2.

Corollary 3.4 The inclusion M —» M induces an isomorphism by restriction of diffeomorphisms

7C0Diff^(M) *-> 7C0Difff(M) = 2 5Q (cf. Lemma 2.1),

and hence the twist spaces of (G, M) and (G, M) are naturally identified

r(G, M) =-> r(G, M) = 7L x Q/Q <* 7L (cf. Definition 2.2)

Remark It follows that the twist number of an element cpe Diff j (M) is just the twist number of the restriction of cp to M. In §2 we constructed torus automorphisms of M realizing all twist numbers ke 7L, In the sequel we shall construct diffeomorphisms of M realizing all k, in fact, they will be extensions of our torus automorphisms.

The construction of the appropriate elements in Diff^(M) goes as follows. Let F denote one of the classical fields IR, (E, IH, or the Cayley numbers Cay, and let d = 1, 2, 4, 8 correspondingly. Let G(m) = O(m), U(m) or Sp(m) when d = 1, 2 or 4, and for m = 2, d = 8 we shall also write G(2) = Spin(9). In the special case m = 2, note that G(2) has a (d+1)-dimensional real representation

48 ELDAR STRAUME

(23) p' = p ' d + 1 : G(2) -> G(2)/Z = S0(d+1) (resp. 0(2), if d = 1),

where Z is the center of G(2), namely Z = 7L if d = 2 and Z = 2/> otherwise. This defines the

standard orthogonal G(2)-action on S . Note the special case d = 1 where p'2» indeed, is the

same as in (7). On the other hand, G(2) acts orthogonally and faithfully on S^d-lc F , in the

case of Spin(9) the representation in question is A9, of course. The case d = 2 is not really

needed later, but it is included here for the sake of comparison and completeness

Lemma 3.5 The projection 71 in the following Hopf fibration

Sd-1 _> s 2 d - l _^ s d ^ d = 1, 2, 4, 8,

is p'-equivariant, that is, 7c(gx) = p'(g)7i(x), for all geG(2) and xeS2c*-l.

Proof Since all cases are similar, let's choose the (mathematically) most interesting case d = 8. The Hopf fibration fits into the following diagram of group homomorphisms and induced maps between coset spaces :

Spin(8)/Spin(7)+ = S 7

i i

(24) Spin(7)+ -» Spin(9) -> Spin(9)/Spin(7)+= S 1 5

•I in i « i p' Spin(9)/Spin(8) = S 8

Spin(7) -> SO(8) -^ SO(9) -» SO(9)/SO(8)

We may assume the orthogonal actions of Spin(9) on S ^ and S 8 correspond to left translation

on the coset space representations of the spheres. Now the p'-equivariance of n is easy to check.

Next, let G(m) act on itself by conjugation. We shall construct a smooth G(m)-equivariant map 9 : S d r n _ 1 —» G(m), m > 2, which is constant on the fibres of the Hopf fibration :


(25) d= 1,2, 4 :

G(l) 1

sdm-i e_^ G ( m )

in S>>%

Fpm-1

d = 8:

i

S 1 5 0->Spin(9)

« i V* iP' S8 „ H>SO(9)

For d 8 the construction of 0 (or equivalently, the induced map '0 ) goes as follows. As indicated in (25), the fibration is a G(l)-principal fibration, where G(l) acts by "scalar" multiplication on the sphere. G(l) is the centralizer of G(m) when the groups are regarded as

subgroup of O(dm) = Iso(Sdm_1), and our map 0 will also be G(l)-equivariant.

0 is defined by sending xe S ~ to the F-reflection in the F-hyperplane of F m

perpendicular to x, more precisely, x -» 0X : y -» y - (x, y) 2x ,

where (x, y) = E 2XJ yx is the standard (Hermitean) inner product in Fm . (For F = OH we regard tHm as a right Fl-module and as a left Sp(m)-module.) It is not difficult to see that 0xe G(m) and 0gX = g0xg" for geG(m). Moreover, 0 a x = 0x = a0xa~* for aeG(l).

In the special case m = 2 one can construct the map 0 in a different way, by first constructing a map "0 : Sd -> SO(d+l), see (25). We lift this to a map '0 : Sd -» G(2) and finally define 0 by composition with 71. We shall follow this recipe in the case d = 8 (i.e. G(2) = Spin(9)). Then "0 is defined by letting H0X be the negative of the reflection in 0(9) which sends x to -x, namely

y —> (x, y) 2x - y. Clearly, "0 is Spin(9)-equivariant, but we also want the lifting '0 to be equivariant. Define a map

Spin(9) x S8 -> S0(9) x S0(9) , (g, x) -> ("ep. (g)x, p'(g)CQx)pXg~1)).

This lifts to a map Spin(9) x S 8 ^ Spin(9) x Spin(9), and we choose the lift so that at one point both components are equal. Now, both '0g x and g('0x)g~* are liftings of the map

Spin(9) x S8 -» SO(9) , (g, x) -> "e p . ( g ) x

and hence '0„x and g('0x)g"[ are equal everywhere, by uniqueness of lifting for covering spaces. Finally, we can also make sure that the property ('0x)^ = 1 holds for all x.

50 ELDAR STRAUME

Note In the case of G(2) = Sp(2), the lifting from SO(5) to Sp(2) of the negative of a reflection is a "symplectic reflection".

Lemma 3.6 Let (G, O j + 0 2 ) be of type I in (22), dim O^ = 2d, dim 0 2 = d+1, where

d = 4, 8 for G = Sp(2) x Sp(l) or Spin(9), respectively. Write S{ = F(H, S 2 ^ 1 ) and

S2 = F(H, Sd), cf. Lemma 3.3, with the orthogonal action of G = 0(2) via Oj and W2,

respectively. Then the map n : S | —> S2 induced from the G-equivariant Hopf fibration projection

71: S2(*"l —> S , is the G -equivariant Hopf fibration for d = 1, in the sense of Lemma 3.5.

Proof We choose the case G = Spin(9). Now, H = G2 and Ag \ Q^ ~ 2^1 + T 2' P9 ' ^ 2 =

(|)| + T2, where T2 is the trivial representation on IR2. S | and S2 are the unit circles in [R2 for the

two cases Ag and P9, respectively. The induced homomorphism

G = 0 ( 2 ) = N S p i n ( 9 ) ( G 2 ) / G 2 -> N s o ( 9 ) ( G 2 ) / G 2 = 0(2)

is a 2-fold covering, namely given by the representation 2 = p^- One may check that G acts

faithfully via p2 = ®\ on S | and acts via p'2 on S2.

On the other hand, in diagram (24), F(G2, Spin(8)/Spin(7)+) = S° is the "fiber" of the map

n : S] —> S2, so this is a 2-fold covering, as claimed.

Using the equivariant maps 0 and '0 defined above we shall define a G-equivariant diffeomorphism ft : M —> M with restriction a : M —> M, as follows :

S q l x S q 2 a - ^ S q l x S q 2 TypeO: ft(x, y) = (y, - 0yx )

(26) T T Type I : ft(x, y) = ( '0y0xx, p ' ( 'ey0x )y )

S l x S 2 a ~> S l x S 2 T y P e n : S ( x ' y) = ( e y 0 x x ' 9 y 9 x y ) (notation : 0xy = 0x(y))

To check that ft is equivariant amounts to using the fact that 0 g x = g0xg~ , and moreover,

0 a x = 0X if a belongs to a Sp(l )-factor of G. The inverse of ft, for example in theType II case,

is given by (x, y) —> (0x0yx, 0x0yy), and similarly in the other cases.

Proposition 3.7 The restriction a of ft in (26) coincides with the element a = 0Cj in

Aut(S]X S2) defined in Lemma 2.3. In particular, the power ft^ 6 Diff*f (S^lx Sq2) has twist number k.


Proof As in §2, we shall regard each Sj as the unit circle S* in (E, allowing multiplication of vectors lying in Sj. In particular, we can define real reflections Rx by

(27) Rxy = -x2y" l , for x, y in S ] , R x e 0(2) - SO(2) (Rx = 0X for d = 1)

We omit the simplest case, Type 0, where the calculations are reminiscent of those in [Br3, Chap. I, §7]. The other two cases are treated separately.

Type I: Assume G = Sp(2) x Sp(l). (The case G = Spin(9) is completely similar!). By (25) and Lemma 3.6 there is a commutative diagram

Sj - > S 7 9 -> Sp(2)

i n i ' e / * I P ' 7 t : x - > x 2 f o rx eS j

S 2 -> S 4 „ e -> SO(5) "6 y = - Ry

Let (x, y)GS]xS 2 . Now n maps '6y6xx to p'('9y6x)7i(x) = Ry R(x2)(x2) = Ry(-x2) =

y2x , using (27). So the identity '0y0xx = yx_1(or possibly -yx"1) holds in S | . Similarly,

in S 2 there is the identity p'('0yex)y = Ry R(x2)(y) = Ry(-x4y_ 1) = y2x"4y = x"4y3.

Then it follows from the formula for ft in (26) that

a(x, y) = (x_1y, x"4y3)

or equivalently, a is the torus automorphism defined by the matrix 0C| in Lemma 2.3.

Type II : We may regard both summands Oj of O as operating on the same underlying

space [H®njlHm ~ {Hm, and then the fixed point set [R2 of the principal isotropy group

'a 0 0* 0 a 0 0 0 A

H = {(a, 0 a 0 , a) ; a eSp( l ) , AeSp(m-2) } c Sp(l) x Sp(m) x Sp(l) [O 0 AJ

in both summands are naturally identified with the "real part" [R2cz DH2 c DH2 x (Hm~2. In this way both circles Sj are identified with S^e [R2.

Let ( x , y ) e S ' x S . One checks that 0X maps [R2 to itself, and moreover, 0X | [R2 is the

reflection Rx in 0(2). By (27) and the formula for ft in (26)

52 ELDAR STRAUME

a : (x, y) -> (9y9xx, 9y9xy) = (Ry(-x), RyRxy) = (x"1y2, x"2y3) ,

which we recognize as the torus automorphism defined by the matrix a 2 in Lemma 2.3.

Returning to the equivariant twisting construction, cf. Theorem 1.2, consider a glueing map

$ GDiff*f(Sqlx Sq2) and its restriction (or "reduction") 9 eDifff (Sjx S2). We may

combine diagram (5) and (8) as follows :

(28) X£ = X $ = [ S ^ l x D ^ + ^ u ^ [Dql + 1 xS q 2]

T T T

F(H, X<j>) = X* = L £ } = [Sjx D2] ucp [D ] X S2]

The vertical maps are inclusions. This diagram expresses the fact that the reduction of

(G, X x) is the lens G -space L(p = X^ constructed in §2. Moreover, the notation X£ is used

similar to L jj% since the G-space (resp. G -space) depends only on the twist number k of (p

(resp. (p). Therefore all possible pairs (G, X* ) are achieved by taking various powers $ = (X ,

k e Z However, we must also check which integers k actually give the same G-manifold (up to

equivalence). Although one expects the same kind of equivalences as for the lens spaces, so far we do not know whether the above G-manifolds are distinguished by their reduction. Fortunately, the proof of Lemma 2.5 applies also here with the following obvious modifications. Clearly, pairs like (S jD 2) = S^x D 2 are replaced by S^l x DC12+1. Then we note that each (pe Aut(S]X S2) involved in the proof has an extension (p : S q lx Sq2 with corresponding

properties, e.g. is equivariant or can be extended to S^lx D(12+1, say. This proves that there are equivariant diffeomorphisms among the above G-manifolds for the same pairs of twist numbers k as described by Lemma 2.5.

The following is a complete list of the distinct G-manifolds whose orbit structure coincides

with that of an orthogonal G-sphere (G, O, Sn), with (G, O) as in (22) :

(29) Type O : X^m_1, m > 3, k > -1 ; Type I : X^2 , Xjf, k > 0 ;

Type II : X J ^ 1 , m > l , k > 0

It remains to determine the diffeomorphism type of the above manifolds, and we return to the equivariant twisting construction, (5). By Van Kampen's theorem all of them are simply connected. Also, by the Mayer-Vietoris sequence, XjJ is an integral homology sphere if qj ^ q2,


namely in the case of Type I, so X£ is a homotopy sphere for n = 12 or 24. For these n the differentiable structure must be the standard one, cf. [KM].

Lemma 3.8 In the case of Type II, the manifolds Xkm~ are diffeomorphic to the standard

sphere S 8 m - ] .

Proof It suffices to show X km _ is an integral homology sphere. Indeed, from this it will

follow that the manifold is a homotopy sphere. On the other hand, the group in question is G = Sp(l) x Sp(m) x Sp(l), and by restricting the action to the subgroup Gj = Sp(m) x Sp(l), the

twist space T(G|, M) will be trivial since dim Gj > 3, see Proposition 3.1 and subcase 2c of (7)

in Chapter I. Therefore, equivariant twisting of the orthogonal model can only give the standard

sphere, and any ''exotic" G-action on X^111- must restrict to an orthogonal G\-action.

We turn to the calculation of the homology of X£, n = 8m - 1 , and write M = S^ x S^ in order to distinguish the two factors S^ of M, where q = 4m -1 . In terms of standard generators for the homology of M, we consider the isomorphism induced from a : M —> M as an integral matrix:

(30) (8)# = a b c d

: H Q (S?xS5)" -> HQ(S?xS^) = 2 x 2

Choose a base point K in each S^. Since 6 (x, y) = (-6yx, 0y0xy) , we infer

a = degree of : x —» -9Kx : S—> S^

c = degree of : x -> 9 K 9 X K : S?-» S^

b = degree of : y -»-9 yK : S%-+ S?

d = degree of : y -> 9y9Ky : S^-> S^

Now, 9X : S^ -> S^ is conjugate to diag(l, . . ,1 , - 1 , - 1 , -1 , - l)eO(4m), since 9X is a

"symplectic reflection", and consequently deg(9x) = deg(-9x) = a = 1, for all x. This also implies

b = c. Moreover, the map x —> 9XK is the composition

s 4m-l e _^sp(m) ^ S 4 ™ ' 1 , e = evaluation at K,

where the map 9 factorizes through [HPm , cf. (25). So, the composition cannot be surjective, hence b = c = 0. Then d = ad - be = ±1 (in fact d = 1). Since the isomorphism in (30) is the identity, it follows from the Mayer-Vietoris sequence that X£ is a homology sphere.

54 ELDARSTRAUME

Finally, consider the Type 0 manifolds Xkm~ , m > 3; G = SO(m) or if m = 8, the subgroup

Spin(7). The calculation of degrees is similar to above. The corresponding matrix (Q)# in (30) has in this case the following k-th power, used for the Mayer-Vietoris sequence calculations, namely

(a*)# = (60#K = ck d

where m even : ak = -k+1, b k = -ck = k, dk = k+1 ;

iH for i = 0or2m-l 2k + 1fori = m-l 0 otherwise

modd k even : ak = dk = 1, bk = ck = 0 ; H*(Xkm_1) = H*(S2m _1)

k odd : ak = dk = 0, bk = ck = 1 ; H*(Xkm_1) = H ^ S ^ x Sm)

In the special case m = 3, we see that this is in agreement with Remark 3.2, namely there are only two different SO(3)-manifolds Xk, S^ and S2x S , corresponding to k even or odd, respectively. However, for m > 3 the G-manifolds are different for different k > -1. In particular, the "standard" examples S2 m _ 1 and Sm~'x Sm correspond to k = 0 and k = -1, respectively. Clearly, Xk

m~ is a homotopy sphere if and only if m is odd and k is even.

By combining Theorems 1.2, 1.3, 2.6 with the results in this section we obtain the following theorem as a brief summary of Chapter IV.

Theorem 3.9 There are 3 types of infinite families of differentiable G-manifolds (G, Xn), where Xn is a homotopy n-sphere,

Type O : (SO(2h+l), X kh + 1 ) , h > 2, k > 0 and even

Type I : (Sp(2) x Sp(l), Xlk2 ) , (Spin(9), xf ) , k > 0

Type II : Sp( 1 )2 x Sp(m), X^m_1), m > 1, k > 0

with the following properties : (i) Each Xk is the standard sphere, except that Xk

+ is the Kervaire sphere for k = 2 or 4 (mod 8).

(ii) For each G the orbit structure is independent of k, and k = 0 gives an orthogonal transformation group on the standard sphere Sn.


(iii) The orbit space X[! /G is a 2-disk, with 0, 1 or 2 isolated singular orbits when the Type is 0,1 or II, respectively.

(iv) For each G the actions are distinguished by the integer k, and the reduction F(H, X{J), where H is a principal isotropy group, is a 3-dimensional lens space L(q, p) with fundamental group ZL, namely

L(k+1, 1) , L(2k+1, 2 ) , L(2k+1, 1)

when the Type is 0,1 or II, respectively.

(v) Let (G, Xn) be any compact differentiable G-manifold, where X n is a homology n-

sphere, G is compact connected and dim Xn/G = 2. Assume there are not 3 isolated singular

orbits. Then, either (G, Xn) is differentiably equivalent to an orthogonal transformation group

on Sn , or it is equivalent to one of the above (G, x £ ) for some k > 0.

Remarks 3.10 (i) The above SO(m)-manifolds, of Type 0, were first discovered by Bredon

in the early 1960's, cf. [Brl], [Br2]. Later he also used the notation E^m_1 (cf. [Br3, Chap. I,

§7]); this description is more close to ours. We find that Z k™~i = Xkm" .

It is also well known that these SO(m)-manifolds can be represented algebraically as

socalled Brieskorn varieties of type Z(2, 2, • • , 2, k+1), cf. [Bri], [Hi]. In particular,

assuming k even, Xk + is the standard sphere if k = 0 or 6 (mod 8) and is the Kervaire sphere

if k = 2 or 4 (mod 8). See also [HH].

(ii) The manifolds E ^ = X kh + 1 , k > 2 even, h > 2, also appear in [S6; Theorem C]

(in a different notation) as the only homology spheres admitting a non-orthogonal action of a compact connected Lie group with 1 -dimensional orbit space. Here the the group in question is SO(2) x SO(2h+l); the additional SO(2)-action which reduces the cohomogeneity from 2 to 1 is not easily seen from the viewpoint of equivariant twisting, but it is obvious from the Brieskorn variety equations, see [S6, p.4].

(iii) Besides the work of Bredon on the existence of non-orthogonal G-spheres of Type 0, Uchida and Watabe have attempted to classify (up to continuous equivalence) differentiable G-spheres of Type I, with G = U(2) or Sp(2) x S, where S = 1, U(l) or Sp(l)), cf. [UW]. They came to the conclusion that there are no non-orthogonal example. However, there are mistakes in their proofs, e.g., incorrect calculation of the normalizer N(H). Thus, in the case of Sp(2) x Sp(l) they failed to detect the "exotic" actions on S .

Chapter V. G-spheres of cohomogeneity two with three isolated orbits

In § 1 we first show that any "exotic" G-sphere can be obtained from its orthogonal model by equivariant twisting "around" isolated orbits, that is, vertices of the orbit space. To each vertex is associated a twist space whose elements "measure the twisting". The calculation of these twist invariants depends solely on the orbit structure of the orthogonal models, and this information is by now available to us.

It turns out that orthogonal groups of polar type do not lead to "exotic" (or nonstandard) actions on G-spheres. On the other hand, "exotic" actions modelled after nonpolar orthogonal groups with 3 isolated orbits do actually exist, as will be shown in §2. This result will also complete the proof of Theorem E stated in the Introduction.

§1. Bad and good vertices in the orthogonal model

Let (G, Xn) be a G-sphere with orthogonal model (G, O, Sn) of cohomogeneity 2 and orbit structure of Type III. We know that the orbit spaces of Xn and Sn are diffeomorphic, having the smooth functional structure induced by the orbit map, and both will be identified with a fixed (stratified) triangular region A, see Figure 1 and Figure 3. The complement of the three vertex orbits ~ G/Kj is a special G-manifold (cf. Remark 1.3 of Chapter III) lying

above A -{vertices}. The latter orbit space is a C°°-smooth manifold with three boundary components, and is obviously diffeomorphic to the (standard) unit disk with three boundary points removed. Isotropy types associated with strata are denoted as in Figure Id.

Let Nj ~ G xj .D i"1"*, 1 < i < 3, be a (small) tubular neighborhood of the vertex orbit

~ G/Kj, and let Mj = 9Nj = G x j^.S^i be the associated sphere bundle. Let Y be the

closure of the complement of the union of the Nj's in Xn, namely

(1) Xn = Y u N j u N 2 u N 3 , Y n N ^ M j , 3 Y = M 1 u M 2 u M 3

In Figure 3a, 3b and 3c we have shaded subregions of A, namely the image of Y, Y u N3 and Y u Nju N2, respectively.

We shall decompose Y into four G-invariant compact manifolds Yc and Yj, 1 < i < 3, lying above the subregions Ac and Aj of A, as illustrated in Figure 4,

(2) Y = Y c u Y j u Y 2 u Y 3 , Y/G = Ac u Aju A2 u A3.

56

Y c is a bundle of principal orbits over the disk Ac, so clearly Y c is G-diffeomorphic to the

product manifold G/H x Ac. (The corners of the last factor can be smoothed away so that

dY ~ G/H x S1). On the other hand, each Yj, i ^ c, is a special G-manifold with boundary

(and corners which can be smoothed away). To explain this, let (Lj, (Rmi+1) be the slice

representation for some non-principal orbit in Yj. Then there are equivalences

(3) G xL . D m i + 1 - G xL . (Dm i x D1) - (G xL . Dm i ) x D 1 - M(TCJ) x [0, 1] « Yj

where M(7Tj) is the mapping sylinder of 7tj : G/H —> G/Lj (cf. (7) in Chapter III).

Remarks 1.1 (i) The product decomposition of Yj in (3) is a special case of the smooth version of the "tube theorem", cf. [Br3, Chap.V, 4.2 and Chap.VI, §6].

(ii) The G-manifold M(7ij) depends only on the conjugacy classes (H) and (Lj), since

these classes give rise to exactly one simultaneous conjugacy class [H c Lj] in G. This

follows from a general criterion which in particular holds for isotropy groups H c Lj of a

representation, with H principal, cf. [S6, Chap.IV, Lemma 1.1 ]. (iii) Definition 1.2 below provides us with a suitable notion of fine orbit structure.

namely the orbit structure together with some "global" property of the family of isotropy groups. Previous versions of this notion can be found in [J] and [Br3, p. 254 ].

Let T c G be a fixed maximal torus. Recall from §2 of Chapter I, a family of subgroups of G is T-adapted if (Tn Q)° is a maximal torus of Q for each Q in the family.

Definition 1.2 Consider the isotropy types of an orthogonal model (G, O, Sn) of Type III, cf. Figure 1. A set 3 ={H, Lj, K;; 1 < i, j < 3} u {LQ} of isotropy groups is a cyclic family if the following conditions hold :

(i) The groups are T-adapted and H c Lj for all i.

(ii) The slice representation (Kj, (Rmi+1) has Lj _ j , Lj and H as isotropy groups.

(iii) (Lj _j z> H c Lj) is an associated triple for (Kj, Smi) , cf. §1 of Chapter III.

(iv) L 0 = nL3n_1, for some neN(H) (= N Q ( H ) ) whose class in the double coset

space N(H, Lj)\N(H)/N(H, L3) belongs to the connected component of the identity.

Recall that a G-manifold with a compact interval as orbit space, such as the sphere bundle Mj is uniquely determined by an associated triple of isotropy groups, namely isotropy groups

58 ELDARSTRAUME

along a cross section lying in the fixed point set of H. Condition iv) of 1.2 means that (L3 z> H e L | ) is also an associated triple for (G, Mj), although an associated cross section

may not exist as a subset of a (transversal) slice of the vertex orbit G/K j .

Proposition 1.3 Each orthogonal model (G, O, Sn) of Type III has a cyclic family of isotropy groups. Moreover, for polar groups (G, O) we can choose the family with L 0 = L3.

The above property of the orthogonal models is an observation based upon case by case calculations. We have no conceptual unifying proof of this, although we can argue that it suffices to check it for the maximal linear groups. We refer to §2 of Chapter I for more information about orbit structures and some of the explicit calculations. For example, the least familiar case of polar type is perhaps (U(l) x E5, [R^4 ); a cyclic family with L 0 = L3 is given in (9) of Chapter I.

For the nonpolar groups we have listed the slice representations of the three vertex orbits, from which the cyclic property can be verified, see (10) - (14) in Chapter I. In these cases L 0 ^ L3 seems unavoidable, but we shall not analyze further the basic reason for this.

Lemma 1.4 The differentiable G-submanifold Y of X, cf. (2) and Figure 4, is uniquely

determined by a cyclic family 3. In particular, Y depends only on the orthogonal model.

Proof It suffices to construct a (continuous) cross section a : Y/G —> Y, such that the isotropy groups are constant on each stratum, namely equal to H, L j , L2 or L3 . Then it is clear that (G, Y) is unique, up to topological G-equivalence at least. However, it also follows that (G, Y) is unique in the differentiable category, e.g. by using the fact that (G, Y) is a special G-manifold and applying the theory in [Br3, §6, Chap.VI].

Let Cj = Mj/G be the "circular" arc near the i-th vertex of the triangle A, see Figure 4. The

standard cross section of Yc ~ G/H x Ac , namely a : x —> (eH, x), can be extended along

each Cj so that the isotropy groups at the ends are Lj _ j , L j , where i -1 means 3 if i = 1. This

follows from the (double) mapping sylinder construction Mj ~ M(TIJ _\, 7Cj), cf. Chapter III,

if we arrange so that the natural cross section of M(KJ _J , Ttj) coincides with a over the arc

C j n A , .

It remains to extend the cross section a over the whole "rectangle" Aj, regarded as the orbit

space of M(7tj) x [0, 1], 1 < i < 3. Let's do this for i = 1. (The three cases are independent).

Our a is defined on those three edges E j , E c and E2 of Aj lying on C j , Ac and C2

respectively. We are going to extend a | (E |U Ec) along Aj, ignoring the previous values on

E2. First, observe that a cross section on Aj may be identified with some function :

A | -^ N(H)/H x [0, 1], and there is a retraction of Aj onto (EjU E c) which collapses the

"outer" (i.e., on the boundary of A) edge EQ of Aj to the corner E 0 n E | of Aj. By

composing with this retraction we obtain an extension to A} of the given function : (EjU Ec)

—> N(H)/H x [0, 1], so that the extension still represents a cross section, and moreover, with

constant isotropy group L^ along the edge E 0 of A^.

Thus, it follows that (G, Xn) is obtained from its orthogonal model (G, <D, Sn) by cutting

out a tubular neighborhood Nj of each vertex orbit G/Kj in Sn and glueing back Nj along its

boundary 3Nj = MJ by a "twisting" diffeomorphism <|>: Mj -> Mj. As before, the effect of the

"twist" depends only on the class of ty in the twist space Tj = T(G, Mj), see Definition 2.3 and

Theorem 2.7 of Chapter III.

Corollary 1.5 If all the three twist spaces Tj are trivial, then the given orbit structure can

only be realized by the orthogonal model (G, O, Sn). In particular, this holds if G is finite.

The following lemma is a slight improvement of Lemma 1.4.

Lemma 1.6 Let Nj c X n be a (small) tubular neighborhood of the i-th vertex orbit,

cf. (1). Then the G-manifold Y u Nj is uniquely determined by the orthogonal model.

Proof We shall apply Lemma 2.6 of Chapter III to prove that Y u N3 (see Figure 3b) is

uniquely determined, by showing that each ty e Diff j (M3) extends to some (j) e Diff°(Y). The

idea is to represent maps \j/e Homeo^(Y) over Y/G by cross sections o: Y/G -» Y, namely the

\j/ -image of some fixed cross section aQ. Having established the existence of a G-

homeomorphism (over Y/G) which extends $, we may as well obtain an extension which is a smooth G-diffeomorphism, by standard (approximation or isotopy) arguments.

To begin with, o is defined on the boundary arc C3 = M3/G of Y/G. As before, the

isotropy groups along the cross section are going to be constant on strata (and taken from a

cyclic family 3). First, and similar to the proof of Lemma 1.4, we can extend a over the

inner region Ac and the arcs C j and C2, and moreover, we can further extend o over A2 and

A3 by changing (if necessary) a along the arcs C] n A3 and C2 n A2. Finally, we can extend

a over Aj by possibly modifying the previous values on one of the arcs C j n A^ or C2 n Aj.

Henceforth, we shall assume dim G > 0. Regarding the twist space Tj as being associated

with the i-th vertex of the triangle A = Sn/G, let's say the vertex is good (resp. bad ) if Tj is

60 ELDAR STRAUME

trivial (resp. nontrivial). By Lemma 1.6, if two of the vertices are good, then we cannot produce a global nontrivial twisting of the orthogonal model.

The next task is to identify those orthogonal models (G, O, Sn) having at least two bad vertices. Indeed, only such models may lead to "exotic" or non-orthogonal G-spheres.

Lemma 1.7 If (G, <D) is of polar type, then the orbit structure of (G, O, Sn) has at most one bad vertex.

Proof Assume first (G, O) is splitting, and write (G, O) as an outer direct sum of

(Gj, <Dj), where c(Oj) = i, i = 1, 2. Let Hj <z Gj be a principal isotropy group for (Gj, Oj),

and put H = H j x H 2 , K3 = Hjx G 2 and Lj = Gjx H 2 ( = Kj n K2) . Then N(H, LT) =

N(H), and consequently r j and T 2 are trivial. In fact, T^ is also trivial unless (G2, 0 2 ) is

non-splitting. Henceforth, we assume (G, O) is non-splitting; these linear groups are listed in Table III of [S6].

The case of two irreducible summands, namely O = O j + 0 2 and c(Oj) = i, is similar to

the splitting case if we let K3 be an isotropy group of O^ (and L] = K j n K2 etc.). These

linear groups are listed as #25 - #34b in Table III, and are also characterized by having Weyl

group (or minimal reduction) W = Djx D4. In practically all cases G is the "thickening"

U ( l ) x [ U ( l ) 2 U 2 ] of W. We refer to Chapter I. Next, consider linear groups with three irreducible summands and dim G > 0; see end of

Table III, starting from #35 (except 35a and 36a where G = W = O( l ) 3 ). Either G is a "thickening" of W of type Sjx S 2 x S3, where Sj = U(l) or Sp(l), or (in #37 - #38) — 9 " 9

G = U ( l f x ^ 2 where 2 2 acts by inversion on U(l) . Direct calculations show that N(H, Lj) = N(H) holds for some i, hence at least two of the vertices are good. In fact, the third vertex is also good, except in the cases #35c, #35d and #38.

Finally, assume (G, O) is irreducible. Then the assumption dim G > 0 is equivalent to

saying that O is of complex type and W = B3. Here G is the obvious "thickening" U(\) x ^ 3

of W = O(l)-3 x ^ 3 . A common feature of these (G, O) is that two of the Lj are conjugate, say

L 2 and L3, and then L | satisfies (N(H, Lj))° = G°. This already implies Tj is trivial for i = 1, 2,

by Observation 2.4 of Chapter III.

In fact, 1 3 is also trivial, but this is less obvious. As an illustration, consider (G, <f>) =

(U(3), S2ji3), where H = O(l) 3 , Lj = U(l) x O( l ) 2 and L 2 = O(l) x 0(2) is conjugate to

L 3 = 0(2) x O(l). The proof of the triviality of T 3 goes as follows. Let S ( l , 2 ) c U ( l ) 3 =

G° = {(z], z2 , Z3)} be the subtorus defined by Z] = z2 , and define S(2, 3) similarly. Then

N(H, L3) = S(l , 2) x (22 x 1), N(H, L2) = S(2, 3) x (1 x ^ 2 ) , and standard homotopy

arguments show that any path in G° whose end points lie in the two subgroups respectively, can be deformed (relative to the subgroups) to a point in S(l, 2) n S(2, 3).

Calculations similar to the above ones also show that T^ is trivial when (G, O) = (SU(7),

A 2 ^ ) . We remark that this case has been studied in [N].

We are left with the following three cases of orthogonal models (G, O, Sn) :

(e) (Sp(l)xSp(m),<D, S8™"1), O = S 3 v 1 ® [ H v m , m > l ,

(4) (f) (U(2) x Sp(m), O, S8™-1), O = [ji2 % v m ] K , m > 1,

(g) (SO(2) x Spin(9), O, S3 1) , <E> = p 2 ®A9 ,

where (G, O) is one of the nonpolar linear groups listed in (10) of Chapter I. The various reductions in the above three cases consist of exactly two different orthogonal transformation groups (G, O, Sr), listed here for convenience :

(e) (D 3 x 0(2), p 2 ®p 2 , S3)

(5) (G, O, Sr) : (f) ([U(l) x SU(2) x U(l)] x A2, [ ^ O ^ + H 2 W 1 R 4> S7)

(g) Same as in case (f).

For each of the orthogonal models in (4) we shall choose a cyclic family 3 of isotropy groups,

according to Definition 1.2, so that the H-reductions Kj = [N(H) n Kj]/H and h: =

[N(H) n L;]/H of the groups in 3 define a cyclic family 3 for the reduction (G ,0 , Sr). Then

we shall calculate the twist spaces T(G, Mj) and T(G, Mj), where

(6) Mj = F(H, Mj) = F(H, G xK .S m i ) = G x j ^ S ^ i , 1 < i < 3,

S^i = F(H, Smi) and (Kj, S% is the reduction of (Kj, Smi) .

Let us, however, begin with (5) and describe a cyclic family 3 = {1, Kj, Lj } of isotropy

groups for (G , 0 , Sr), together with the normalizers N(L:) = NT^(LJ).

Case (i) (G, O) of type (e) in (5): We regard D3 as the subgroup < D j , Z3 > =

< D 1 , D ,1 > of 0(2), where D 1 = O(l) x 1 c O(l) xO(l ) = D 2 . Then

<5 restricted to U(l)2x SU(2) is splitting, but <£ itself is irreducible since 2>2 interchanges the summands.

62 ELDARSTRAUME

Ki=AE>3~D3 , and L0 = AD'j (A means "diagonally embedded")

The other groups in 3 are subgroups of the following group ~ (Z^r

D i x D 2 ^ ( o i J x ( o £ 2 j ^=±^=±n. Namely, define subgroups by appropriate conditions on 8 and 8j:

K 2 = (e2 = 1), K3 = (£ = e2 )> E3 = ( 8 = 82 > 81 = ! ) ' Lj = AD! = K] n K 2 , L 2 = K 2 n K 3 .

Finally, put D'2 = < D'j, ±Id > c 0(2), and we find that

(7) N(L1) = D1xD 2 = N(L3), N(L2) = D 3 xD 2 , N(E0) = D'jx D'2 .

Case (ii) (G, O) of type (f) in (5) : Write x32 = < y >, where y commutes with SU(2) and interchanges the U(l) factors. Note that the presentation in (5) is not effective, since H = ker 3> = L 1 n L 2 - %

Let U(l), resp. SO(2) ~ U(l) be the diagonal, resp. real circle subgroup of SU(2), and let AU(1) ~ U(l) be the diagonal embedding into the central torus U(l)^ of G . As a maximal torus of G we choose

T = U(l)2 xU(l) = {(a, L b , c) ; a, b, c eU(l)} c U(l)2 x SU(2)

and we define 2-dimensional tori T = AU(1) x U ( l ) c T and T" = AU(1) x SO(2) , and elements

. - « , ( i ? ) . o . ^ - , - , . ( i « ) . . , x -« . ( i ; i ) . „ . v . < . . . ( « i j . , , L|, L2 and K2 will be toral subgroups of T, so they are given by conditions on a, b, c. The subgroups of G which we need are as follows :

Lj = (a = b = c ) , L2 = (a = b = c), K2 = ( a = b ),

L3 = < Ky > , L0 = < Xy > , Kj = <Lj, Xy >, K3 = <L2, y > ,

(8) N(Lj) = < T, Xy>, N ^ ) = < T, y >, N(L3) = < T\ X\ y >,

N(L0) = < T", K\ y >.

Note that KG L2 , K'G T, Xe T", X& T, X'e T u T", and moreover

Kj/H - 0(2), K3/H - SO(2) x Z2 .

Lemma 1.8 Let (G, O) be one of the above nonpolar linear groups, of type (e), (f) or (g). Then the following naturally induced maps are bijective :

(i) N ( H , L j ) - » N ( E j ) , 0 < j < 3 ,

(ii) Homeo^ (Mj) -> Homeof (Mj) , 1 < j < 3,

(iii) T(G, Mj) -> T(G, Mj) , 1 < j < 3 .

Proof We refer to Chapter 1, where isotropy groups Kj and Lj in the three cases (e), (f), (g)

are listed in the last part of §2. This gives a cyclic family 3 with groups indexed so that Kj and

Lj, indeed, coincide with the subgroups of G given above.

The natural map

N(H, Lj) = (N(H) n N(Lj))/H -» N g ((Lj n N(H))/H) = N(Lj)

is always injective. To verify its surjectivity and hence prove (i), we can, for example, show by direct calculations that the image group has the same dimension and the same number of components as N(Lj). We omit these details.

Finally, by the results in §2 of Chapter III, both (ii) and (iii) are consequences of (i).

Proposition 1.9 The twist spaces Tj = T(G, Mj) = T(G, Mj) in the three cases of (4) or (5) are as follows :

case(e): r 1 = Z x Z 2 . r 2 « r 3 « Z

(9) case(f): Tj = 1 , r 2 - r 3 - Z 2

case (g): Same as in case (f) .

Proof The calculation of Tj involves deformations of paths in G whose two end points must

stay within the subgroups Qj = N(Lj), j = i — 1 and i, respectively. In particular, paths in

Qj_l n Qj define the trivial element of Tj. We discuss the two cases of (G, O) in (5) separately.

64 ELDARSTRAUME

Case (i) The groups Qj are described in (7). If i = 2 or 3, then Qj_j n Qj hits both

components of G, and all elements of Tj can be represented by paths in G ° = SO(2) starting at

1 and ending at ±1. Tj is actually the same as the twist space ~ 7L in Definition 2.2 of Chapter

IV. On the other hand, T\ ~ 7L u 7L where the second 7L corresponds to paths lying in 0(2)~

(so their order is 2 in 7in ). By identifying T] with a "group lift" in 7iQDiffj (Mi) we may

also write T] ~Zx Zr), where 7L^ acts by inversion on 7L

Case (ii) The groups Qj are described in (8). Qj and Q2 have two components,

G /Q] and G /Q2 are topologically ~ S^ and hence simply connected, whereas Q 0 and Q3 have

two components in each component of G. Now, both components of Q 0 in the same

component of G hit Q1, so T\ is trivial (cf. also 2.4 in Chapter III).

Next, Qj n Q2 = T and therefore a path between Qj and Q2 outside G° represents a

nontrivial class in I^ . But there is only one such class; it is represented by a path from Xy to y.

On the other hand, y belongs to Q2 n Q3 , so T^ can be represented by paths in G °. Since Q3

has one component in G ° and outside Q2, T^ is also nontrivial. The only nontrivial class is

represented by a path from 1 to X\ This completes the proof.

Note : Since at least two of the Tj are nontrivial, each of the above orthogonal models may possibly lead to non-orthogonal actions on G-spheres. The search for such "exotic" G-spheres is the topic in §2.

§2. More examples of exotic G-spheres

We shall have a closer look at the possibility of having homology G-spheres Xn , different from the orthogonal models, whose orbit space is a 2-disk with three vertices. In § 1 we came to the conclusion that the orthogonal model (G, O, Sn) of such a G-sphere must be among the ones listed in (4), namely (G, <D) is one of the nonpolar linear groups of Type III.

To begin with, recall from § 1 that (G, Xn) must be the result of equivariant twisting of the orthogonal model at two chosen vertex orbits, ignoring the third vertex Thus, in order to simplify the calculation of the various possibilites, the third vertex should be a "very bad" one, namely we should perform equivariant twisting at two vertices with the "smallest" twist spaces. Therefore, according to (9), in case (e) we shall select the two vertices labelled 2 and 3, whereas in case (f) and (g) we shall select the vertices labelled 1 and 2 (or 1 and 3).

LOW COHOM OGENEITY ACTIONS 65

But first of all, must Xn necessarily be a homotopy sphere ?

Lemma 2.1 If (G, X n) has the same orbit structure as one of the orthogonal transformation groups in (4), then X n is a homotopy sphere.

Proof Consider the G-spaces X n obtained from the above orthogonal models by equivariant twisting at the vertex labelled no. 3, namely the orbit of type G/K where K = K3 is listed in (11), (12) or (14) of Chapter I. Note that G/K - Sp(m)/Sp(m-2) or Spin(9)/G2 in case (f) or (g) respectively, whereas in case (e) G/K is a fibration over Qpm-1 w i t h f i b r e « Sp(l)2/AN(U(1)), where AN(U(1)) means N(U(1)) = N S p ( 1 ) (U( l ) ) embedded diagonally into Sp(l) 2 and Sp(l)/N(U(l)) - SO(3)/0(2).

We may write S n = Y u N, X n = Y U y N , where N = G x KD^+l and M = Y n N

= G x j^S^ , q > 2. In particular, a certain "initial" inclusion M —» Y gives Sn , whereas

some "modified" G-equivariant embedding \\f: M —> Y determines Xn . It follows from

Van Kampen's theorem that Y is simply connected, hence also X n is simply connected

irrespective of \\f. In case (e) new G-manifolds may possibly be constructed from the above

space X n by equivariant twisting at the vertex labelled no. 2. However, the same argument

applies to X n instead of Sn , showing that the resulting spaces are simply connected. Finally, by the Mayer-Vietoris sequence, the equivariant twisting operation applied to

Sn will not change its homology, but we omit these calculations which are analogous to those in §3 of Chapter IV.

Next, we would like to know whether the homotopy sphere X n actually must be the standard sphere Sn . This turns out to be true in the cases (e) and (f), by Lemma 2.2 below, whereas the case (g) will be left open. However, in the latter case X n can only be the standard sphere S^* if our Conjecture 2.4 below holds.

In the sequel we write G = Gj xG1, where Gj = Sp(l), U(2) or SO(2), depending on the three cases in (4).

Lemma 2.2 Let (G, Xn) have the orthogonal model (e) or (f) in (4), in particular G' = Sp(m). Then (G, Xn) is equivariantly diffeomorphic to the orthogonal action (Sp(m), 2v m , S° m _ 1 ) . Furthermore, the induced action of Gj on the orbit space X n /G is equivalent to the corresponding action in the orthogonal model, namely the following orthogonal action (Gj, ((), D5) on the 5-disk, where in the two cases

66 ELDARSTRAUME

(10) (e) : (G 1 , ^ ) = ( S p ( l ) , S 2 p 3 - T 1 ) ,

(f) : (G1,<|)) = (U(2),p3 + [det]IR).

Proof The restricted action of Sp(m) on X n is a bi-axial (or 2-regular) action, in the

sense that it has precisely the same isotropy groups ^ Sp(m) as in the representation 2vm ,

namely Sp(m-l) and Sp(m-2). The orbit space Xn/Sp(m) is known to be a compact

contractible manifold of dimension 5. We refer to (Sp(m), 2vm , S ° m - 1 ) as the orthogonal

model of (G', Xn ) . Bi-axial actions of Sp(m) on homotopy (8m-l)-spheres are known to be in 1-1

correpondence with smooth, compact, contractible 5-manifolds B , realized as the orbit space, and moreover, these actions are concordant to the orthogonal model; we refer to [DH]. In particular, X n = S 8 m _ 1 is the standard sphere. We also refer to [Br3], [D], [DH], [J], [S5] for various results on bi-axial actions and their properties.

We claim that the orbit space Xn/Sp(m) is diffeomorphic to the orbit space of the

orthogonal model, namely the disk D , and consequently (Sp(m), Xn) is itself equivalent to

its orthogonal model. To see this, consider the action of G j on B 5 and the homology 4-

sphere 3B5 . By the classification of (homology) Gj-spheres of cohomogeneity one

(cf.[S6]), it follows that (G,, 3B5) is equivariantly diffeomorphic to an orthogonal action

on S . As a consequence of this we have B 5 = D 5 and, moreover, the Gj -action on D^

must also be orthogonal. Finally, it is not difficult to verify that <J) is given by (10).

Next, we turn to the less familiar case (g) of (4), where G' = Spin(9). If we regard the basic Spin-representation A9 as the natural analogue of the standard representation of a

classical group, then (G, X 3 *) may be referred to as a bi-axial action of Spin(9) with no fixed point. The orthogonal model together with the isotropy types are as follows :

(11) (Spin(9), 2A9, S 3 l ) : Spin^)1=> SU(4) 3 SU(3) c G2 .

Problem 2.3 Classify bi-axial actions on homotopy spheres with (11) as the orthogonal

model. (More generally, replace (2A9, S3 1) by (2A9 + xd, S 3 1 + d ) ) .

Conjecture 2.4 If a bi-axial action of Spin(9) on a homotopy sphere X 3 ^ extends to an action of SO(2) x Spin(9) with a 2-dimensional orbit space, then (Spin(9), X3*) is equivalent to its orthogonal model, namely case (g) in (4).

To our knowledge, both 2.3 and 2.4 are open problems. As a suggestion in the case of Problem 2.3, it seems natural to try first the concordance approach of [DH].

We have some additional information on the orbit structure of the model (g). The orbit

space B 3 = S 3 i /G' is (topologically) the 3-disk D 3 whose boundary 2-sphere has an

equatorial circle as "edge", namely the stratum of type Spin(7)-. The upper and lower

hemispheres of are the strata of type SU(4) and G2, respectively. Moreover, B 3 has a

circular symmetry, corresponding to the induced action of G | = SO(2), and the "edge" of

dB3 is an SO(2)-orbit.

For a bi-axial action (Spin(9), X 3 1 ) which extends to G = SO(2) x Spin(9) such that (g) in (4) is the orthogonal model, SO(2) still acts on X3VSpin(9) with an arc as fixed point set and with the same orbit space as in the orthogonal case. From this we can at least establish an orbit strata preserving diffeomorphism X 3 VSpin(9) ~ S 3 VSpin(9) = B 3 and, moreover, the

induced action of SO(2) on B 3 will be equivalent to the action in the orthogonal case. Thus, in analogy with the case of bi-axial Sp(m)-actions, the above information provides

some support to Conjecture 2.4.

By Proposition 1.9, each of the orthogonal models of type (f) or (g) in (4) leads to at most one non-orthogonal transformation group (G, Xn). Moreover, in case (f) X n must be diffeomorphic to the standard sphere Sn . But the existence of a non-orthogonal G-sphere of type (f) or (g) is still an open question.

On the other hand, in analogy with the results of Chapter IV, we are inclined to believe that the reduction (G, X ' ) of a non-orthogonal example will distinguish it from the orthogonal model. Therefore, as a natural first step, let us rather inquire what transformation group can possibly be the reduction of such a non-orthogonal example ?

Recall that all the models of type (f) and (g) have the same reduction (G, O, S ') , where

G = G° x Z2 and

(12) (GO, O) = (U(l) x SU(2) x U(l) , [ ^ ( g ) ^ + j ^ l f o ) , cf. (5).

Let K be the image of G in 0(8). Then K = < K°, y > is the normalizer of K° in 0(8) and

(13) y = [? ol ' l = i d e n t i t y m a t r i x in 0(4).

By Lemma 1.8, the twist spaces Tj of (G, O, Sn) are naturally identified with the twist

68 ELDAR STRAUME

spaces of (G, O, S7), namely T] = 1, ^ ~ T3 « Z2, by Proposition 1.9. For a description of the nontrivial elements of T2 and T^ we refer to the last part of the proof of Proposition 1.9.

We observe that (K°, S7) is of Type II (cf. Figure 1) and moreover, there is no non-orthogonal compact connected transformation group with (K°, S7) as the orthogonal model, by the results of Chapter IV. Therefore, equivariant twisting of the orthogonal model (12) can only give the standard sphere S7 with the orthogonal action of G°. Hence, a possible non-orthogonal action of G on S7 must be the result of replacing y in (13) by some "exotic" involution Y|eDiff(S7) with the properties :

(14) (i) K = (K°, 72) is isomorphic to (K°, y) = K ,

(ii) Yj and y induce the same reflection on the 2-disk S7/K°.

Lemma 2.5 (i) If equivariant twisting of (G, <E, S7), defined by the nontrivial element of 12 (or 1 3), leads to a non-orthogonal G -action on S , then the corresponding equivariant twisting of (G, O, Sn) leads to a non-orthogonal G-sphere Xn.

(ii) Suppose the element y e K in (13) can be replaced by another involution Yj EDiff(S7) such that (14) holds and, moreover, the G -action on S7 defined by K is not equivalent to the orthogonal action. Then each of the orthogonal transformation groups of type (f) or (g) in (4) is the orthogonal model for some unique non-orthogonal G-sphere.

Proof By Lemma 1.8, the orthogonal transformation groups in (4) and their reductions have the same twist spaces Tj (at corresponding vertices), and moreover, "reduction commutes with equivariant twisting". The latter statement is, perhaps, too strong in general, but at least we know (from Chapter IV and § 1) that it holds in the cases of interest to us. In particular, (i) follows immediately from this.

In case (ii), the group K c Diff(S7) defines some non-orthogonal transformation group (G, S7) which is obtainable from the orthogonal model (G, <E, S7) by equivariant twisting. On the other hand, the corresponding equivariant twisting of (G, O, Sn) leads to some (G, Xn) with the above non-orthogonal transformation group (G, S7) as reduction, and therefore (G, Xn) itself is of non-orthogonal type.

In the remainder of this section we shall work solely with case (e) in (4). The main result is the following theorem, whose proof is based upon constructions and calculations similar to those in Chapter IV. This will also complete the proof of Theorem E stated in the Introduction


Theorem 2.6 There are (countably) infinitely many non-orthogonal G-spheres X n whose orthogonal model is given by (e) in (4). X n is always diffeomorphic to the standard sphere Sn . Furthermore, for each k > 1 there is a G-action on Sn whose reduction X^ = F(H, Sn) is a lens space L(k, 1) (cf. (17) of Chapter IV).

The idea of the proof is to show that equivariant twisting, say, at the second vertex, of the reduced model (G, 3>, S3) leads to the family of lens spaces of type L(k, 1). In particular, L(k, 1) ^ S^ for k > 1. As before, there is a corresponding equivariant twisting of the model (G, O, Sn). By lemma 2.2, the resulting space X n is still the standard sphere, but for different k the actions are different since they are distinguished by their reduced spaces, whose topological types are the lens spaces L(k, 1).

Henceforth, there is no need for the "bar" in our notation, since we shall be working solely with the reduced orthogonal model in question, namely the following orthogonal transformation group

(15) (G, O, S3) = (D3 x 0(2), p 2 ® p 2 , S3) > cf- ( e ) i n (5)-

We shall perform equivariant twisting at the vertex labelled 2, where the relevant orbit type data are as follows. The vertex orbit is ~ G/K, where K = K2 = Dj x Dj and K has slice

representation (|) = p 2 ® p j + p j ® l . A n associated triple for (K, (|>, S*) is (L | z> 1 c L2),

where Lj = ADj ~ D] (diagonally) and L 2 = 1 x D | . The twist space is (cf. Prop. 1.9)

(16) T2 = T(G, M) = 7 1 ^ ( 0 ; Qh Q2)/(Q1 n Q2) « Z, M = G x K S1 ,

where Qj = N(Lj) = Dj x D 2 , D3 x D 2 for i = 1, 2 respectively. The following lemma is easily verified.

Lemma 2.7 The 2-dimensional representation (K, ())) is the restriction of the G-

representation n ®p2 , where n : D3 —» 0( 1) is the 1 -dimensional nontrivial representation

ofD 3 .

By the above lemma, the tubular disk bundle around the orbit G/K is of type

(17) N = G x K D 2 = G/K x D 2 = [ ^ / D j ) x ( 0 ( 2 ) ^ ^ ] x D 2

= ( D 3 / D 1 ) x S 1 x D 2 = B 0 u B | U B 2 , Bj = ^ B

70 ELDARSTRAUME

which is a union of three solid tori permuted transitively by < tf> = 7L^ c D3, and G acts

diagonally on G/K x D 2 and via 71 ®p2 on D 2 . At this point we make the observation that

the actions on S^ defined by the representations (0(2), 2p2) and (Djx 0(2), pi ®p2 + P2)

are restrictions of the linear group in (15); they are, in fact, also the actions of Type 0 and

Type II studied in §2 of Chapter IV.

Write G' = D 1 xO(2) . Then

B = B 0 = G* x K D 2 = G7K x D 2 = 0(2)70! x D 2 = S !x D 2 ,

(18) Diff?(3N) - Diff'f (3B) « Diff?(2)(3B) = D i f f ^ ^ x S1),

where G' acts via p2 on the first factor of the torus and via Pi®p2 on the second factor.

By (18), 0(2)-diffeomorphisms of the torus 3B over I = 3N/G - (S ]x s t y o ^ ) are in 1-1-correspondence with G-diffeomorphisms of 3N = M over I, and this bijective correspondence still prevails at twist space level, namely

Z <-> T(G, 3N) ^ T(0(2), S !x S1) (cf. (16), and 2.1, 2.2 in Chap. IV)

where the rightmost twist space is defined by the familiar action (0(2), p2 x p2, S*x S*) in

§2 of Chapter IV.

Choose \j/e Diff j (3N) and let's consider the effect of equivariantly twisting (G, O, S^)

along the three tori 3N by means of \j>. The map \jir is uniquely determined by each of the

three restrictions \j/jG Diff J^ (3BJ), since they are mutually conjugate by elements of the

group D3 cz G -» 0(4) which permutes the tori transitively. On the other hand, by

regarding S^ as an 0(2)-manifold we actually perform "simultaneous" equivariant twisting

at three disjoint 0(2)-orbits ~ S1 (whose images in the disk S^/0(2) ~ D 2 are three points

on the boundary circle) with the above solid tori as tubular neighborhoods.

Lemma 2.8 Let xe D3 be the involution which leaves B 0 invariant (and flips B \ and B2,

so that \|/2 = Wl T ) - Then "simultaneous" equivariant twisting of (0(2), 2p2, S^) along the

tori dB\ and dB2, using the maps \{/j and \|/2 respectively, leads to an 0(2)-manifold which

is equivariantly diffeomorphic to (0(2), 2p2, S^).

Proof We shall sketch the idea of the proof. Up to equivariant diffeomorphism, we may regard S^ as decomposed into three pieces

S 3 = B 1 u C u B 2 = S * x D 2 u ( S ^ S ^ x f l ^ ] u D ^ S 1 .

The involution T flips B and B 2 and induces a reflection on the disk S^/0(2). In the above decomposition the attaching maps are

\\f{ :dB{ = SlxSl - ^ x S 1 = ( S 1 x S 1 ) x { i } c C .

We claim that the effects of the maps \J/J and \ j / 2 cancel each other (- imagine the interval [1, 2] is shrinking to a point), so the new 0(2) -space is equivalent to the original space Slx D 2 u i d D2x S 1 ^ S3 . This is illustrated by Figure 5.

Lemma 2.9 Let \|/e Diff? (2)(S lx S l) be the restriction of \j)e Diff?(3N), where 0(2)

acts diagonally via the standard representation p 2 on S*x S - 3B. Then the space X-*

obtained from (D3 x 0(2), p 2 ®p 2 , S. ) by equivariant twisting via \j/ at the vertex orbit

G/K is a lens space L p ~ L(k+1, 1). Moreover, all these lens spaces can be obtained by appropriate choices of \j/.

Proof By Lemma 2.8, the G-manifold X^ can be obtained by a two stage equivariant

twisting construction of 0(2)-manifolds. In the first stage we remove two of the three solid

torus components of the G-invariant set N <z S , see (17), and the resulting space is just S^

with the orthogonal 0(2)-action induced by 2p2 . The remaining solid torus, B, is a tubular

neighborhood of some 0(2)-orbit corresponding to a boundary point of the disk S3/0(2).

In the second stage we perform equivariant twisting of (0(2), 2p2 , S3), and we "twist"

dB according to \j/. By the results in §2 of Chapter IV, this construction leads to a lens

space L^ , where k is the twist number of [\}/]e T(0(2), dB). Finally, since we can choose

\j/ with any specified twist number k and \|/ determines a map \j/e Diff^(3N), all manifolds

L^ can be obtained in this way.

Remark 2.10 Equivariant twisting of (Sp(l) x Sp(m), O, S° m _ 1 ) is similar at the two vertex orbits of type G/K3 and G/K2. One may also perform "simultaneous" equivariant twisting at the two vertex orbits and possibly obtain new types of G -actions. However, we have not analyzed what combinations of "twist numbers" at G/K2 or G/K3 will lead to the same G-sphere or to distinct G-spheres. Hence, it remains a fascinating open problem to fully describe and distinguish, in some way or another, all the G-spheres with an orthogonal model of type (e) in (4).

Figures

+ * * N,/C w N2/C =Sn/G

Figure 2

3 3 3

(a) Y/G (b) (Y u N3)/G (c) ( Y u N , u N2)/G

Figure 3

72

LOW COHOMOGENETTY ACTIONS 73

Y - Y c u Y 1 u Y 2 u Y 3

Y/G « Ac u AjU A2 u A3

Figure 4

References

[A] T. AS OH : Compact transformation groups on Z2~cohomology spheres with orbit of codimension 1. Hiroshima Math. J. 11 (1981), 571-616 ; loc. cit. 13 (1983), 647- 652.

[Bol] A. BOREL et al. : Seminar on transformation groups. Ann. of Math. Studies, no. 46, Princeton Univ. Press, N. J., 1961.

[Bo2] : Le plan projectif des octaves et les spheres comme espaces homogenes. Comptes Rendue de TAcademie des Sciences, Paris 230 (1950), 1378-1383.

[Brl] G. BREDON: Transformation groups on spheres with two types of orbits. Topology 3(1965), 103-113.

[Br2] : Examples of differentiable group actions. Topology 3 (1965), 115-122.

[Br3] : Introduction to compact transformation groups. Pure and Applied Math. Vol. 46, Academic Press (1972).

[Bri] E. BRIESKORN: Beispiele zur Differentialtopologie von Singularitaten. Inventiones Math. 2 (1966), 1-14.

[C] J. W. CANNON : The recognition problem : What is a topological manifold ? Bulletin of the Amer. Math. Society, 84 No. 5 (1978), 832 - 866.

[D] M. DAVIS : Multiaxial Actions on manifolds. Lecture Notes in Math. no. 643, Springer- Verlag, 1978.

[Da] J. DADOK : Polar coordinates induced by actions of compact Lie groups. Trans. Amer. Math. Soc. 288 (1985) , 125-137.

[DH] M. DAVIS and W. C. HSIANG : Concordance classes of regular Un and Spn actions on homotopy spheres. Acta Mathematica 144 (1980).

[Hi] F. HIRZEBRUCH: Singularities and exotic spheres. Seminaire Bourbaki 19 (1966/67), No. 314.

[HH] W. C. HSIANG and W. Y. HSIANG: On compact subgroups of the diffeomorphism groups of Kervaire spheres. Annals of Math. 85 (1967), 359-369.

[HS1 ] W. Y. HSIANG and E. STRAUME: Actions of compact connected Lie groups on acyclic manifolds with low dimensional orbit space. J. reine und angew. Math. 369 (1982), 21-39.

[HS2] : On the orbit structures of SU(n)-actions on manifolds of the type of euclidean, spherical or projective spaces. Math. Ann. 278 (1987), 71-97.

[Ja] R. JACOBY : One-parameter transformation groups of the three-sphere. Proc. AMS.7(1956), 131-142.

[J] K. JANICH : Differenzierbare Mannigfaltigkeiten mit Rand als Orbitraume differenzierbare G-Mannigfaltigkeiten ohne Rand. Topology 5 (1966), 301-320.

[KM] M. KERVAIRE and J. MILNOR : Groups of homotopy spheres, I . Ann. of Math. 77 (1963), 504 - 537.

74

[Ml] J. MILNOR : Differentiable structures on spheres. Amer. J. Math. 81 (1959), 962-972.

[M2] : Whitehead torsion, Bull. Amer. Math.Soc.72(1966), 358-426.

[MS] D. MONTGOMERY and H. SAMELSON: Transformation groups on spheres. Ann. of Math. 44 (1943), 454-470.

[MSY] D. MONTGOMERY, H. SAMELSON, and C. T. YANG : Groups on E n with (n-2)-dimensional orbits. Proc. Amer. Math. Soc. 7 (1956), 719-728.

[MZ] D. MONTGOMERY, L. ZIPPIN : Topological Transformation Groups, Interscience Publishers, New York, 1955.

[N] A. NAKANISHI: SO(n), SU(n), Sp(n)-homology spheres with codimension two principal orbits.Tokyo J. Math. 7 (1984), 287-313.

[O] P. ORLIK : Seifert manifolds. Lecture Notes in Math. #291, Springer-Verlag 1972. [PT] R. S. PALAIS and C. L. TERNG. A general theory of canonical forms. Trans. Amer.

Math. Soc. 300 (1987) , 771-789. [R] F. RAYMOND : Classification of the actions of the circle on 3-manifolds. Trans.

Amer. Math. Soc. 132 (1968), 51-78. [Ril] R. W. RICHARDSON, JR. : Actions of the rotation group on the 5-sphere. Ann. of

Math. 74 (1961), 414-423. [Ri2] : Groups acting on the 4-sphere. Illinois J. Math. 5 (1961). [SI] E. STRAUME: The topological version of groups generated by reflections.

Math. Z. 176 (1981), 429- 446. [S2] : Weyl groups and the regularity properties of certain compact Lie group

actions. Trans. Amer. Math. Soc. 306 (1988), 165-190. [S3] : The integral weight system for torus actions on spheres with no fixed

point. Math. Scand. 66 (1990), 91-109. [S4] : On the invariant theory and geometry of compact linear groups of

cohomogeneity < 3. Diff. geom. and its Applications 4 (1994), 1-23. [S5] : Compact differentiable transformation groups on exotic spheres.

Math. Ann. 299 (1994), 355-389. [S6] : Compact connected Lie transformation groups on spheres with

low cohomogeneity, I. Memoirs of AMS, Vol. 119, No. 569, 1996. [SS] T. SKJELBRED and E. STRAUME : On the reduction principle for compact

transformation groups. Univ. of Oslo and Univ. of Troms0, 1993. [UW] F. UCHIDA and T. WAT ABE: A note on compact connected transformation groups

on spheres with codimension two principal orbits. Science Reports of Niigata Univ.16 (1979).

[W] H. C. WANG: Compact transformation groups of S n with an (n-l)-dimensional orbit. Amer. J. Math. 82 (1960), 698-748.

76 ELDAR STRAUME

Eldar Straume Institute of Mathematics and Statistics Norwegian University of Science and Technology Trondheim N-7055 Dragvoll Norway

Editorial Information

To be published in the Memoirs, a paper must be correct, new, nontrivial, and significant. Further, it must be well written and of interest to a substantial number of mathematicians. Piecemeal results, such as an inconclusive step toward an unproved major theorem or a minor variation on a known result, are in general not acceptable for publication. Transactions Editors shall solicit and encourage publication of worthy papers. Papers appearing in Memoirs are generally longer than those appearing in Transactions with which it shares an editorial committee.

As of September 30, 1996, the backlog for this journal was approximately 7 volumes. This estimate is the result of dividing the number of manuscripts for this journal in the Providence office that have not yet gone to the printer on the above date by the average number of monographs per volume over the previous twelve months, reduced by the number of issues published in four months (the time necessary for preparing an issue for the printer). (There are 6 volumes per year, each containing at least 4 numbers.)

A Copyright Transfer Agreement is required before a paper will be published in this journal. By submitting a paper to this journal, authors certify that the manuscript has not been submitted to nor is it under consideration for publication by another journal, conference proceedings, or similar publication.

Information for Authors and Editors Memoirs are printed by photo-offset from camera copy fully prepared by the

author. This means that the finished book will look exactly like the copy submitted.

The paper must contain a descriptive title and an abstract that summarizes the article in language suitable for workers in the general field (algebra, analysis, etc.). The descriptive title should be short, but informative; useless or vague phrases such as "some remarks about" or "concerning" should be avoided. The abstract should be at least one complete sentence, and at most 300 words. Included with the footnotes to the paper, there should be the 1991 Mathematics Subject Classification representing the primary and secondary subjects of the article. This may be followed by a list of key words and phrases describing the subject matter of the article and taken from it. A list of the numbers may be found in the annual index of Mathematical Reviews, published with the December issue starting in 1990, as well as from the electronic service e-MATH [telnet e-MATH.ams.org (or telnet 130.44.1.100). Login and password are e-math]. For journal abbreviations used in bibliographies, see the list of serials in the latest Mathematical Reviews annual index. When the manuscript is submitted, authors should supply the editor with electronic addresses if available. These will be printed after the postal address at the end of each article.

Electronically prepared papers. The AMS encourages submission of electronically prepared papers in AMS-T${ or A/i/fS-WT^i. The Society has prepared author packages for each AMS publication. Author packages include instructions for preparing electronic papers, the AMS Author Handbook, samples, and a style file that generates the particular design specifications of that publication series for both AMS-TE& and Aj^S-WT^.

Authors with FTP access may retrieve an author package from the Society's Internet node e-MATH.arns.org (130.44.1.100). For those without FTP

access, the author package can be obtained free of charge by sending e-mail to [email protected] (Internet) or from the Publication Division, American Mathematical Society, P.O. Box 6248, Providence, RI 02940-6248. When requesting an author package, please specify A\4S-T^. or A/^S-WF^i, Macintosh or IBM (3.5) format, and the publication in which your paper will appear. Please be sure to include your complete mailing address.

Submission of electronic files. At the time of submission, the source file(s) should be sent to the Providence office (this includes any T^X source file, any graphics files, and the DVI or PostScript file).

Before sending the source file, be sure you have proofread your paper carefully. The files you send must be the EXACT files used to generate the proof copy that was accepted for publication. For all publications, authors are required to send a printed copy of their paper, which exactly matches the copy approved for publication, along with any graphics that will appear in the paper.

T^X files may be submitted by email, FTP, or on diskette. The DVI file(s) and PostScript files should be submitted only by FTP or on diskette unless they are encoded properly to submit through e-mail. (DVI files are binary and PostScript files tend to be very large.)

Files sent by electronic mail should be addressed to the Internet address [email protected]. The subject line of the message should include the publication code to identify it as a Memoir. T^X source files, DVI files, and PostScript files can be transferred over the Internet by FTP to the Internet node e-math.ams.org (130.44.1.100).

Electronic graphics. Figures may be submitted to the AMS in an electronic format. The AMS recommends that graphics created electronically be saved in Encapsulated PostScript (EPS) format. This includes graphics originated via a graphics application as well as scanned photographs or other computer-generated images.

If the graphics package used does not support EPS output, the graphics file should be saved in one of the standard graphics formats—such as TIFF, PICT, GIF, etc.—rather than in an application-dependent format. Graphics files submitted in an application-dependent format are not likely to be used. No matter what method was used to produce the graphic, it is necessary to provide a paper copy to the AMS.

Authors using graphics packages for the creation of electronic art should also avoid the use of any lines thinner than 0.5 points in width. Many graphics packages allow the user to specify a "hairline" for a very thin line. Hairlines often look acceptable when proofed on a typical laser printer. However, when produced on a high-resolution laser imagesetter, hairlines become nearly invisible and will be lost entirely in the final printing process.

Screens should be set to values between 15% and 85%. Screens which fall outside of this range are too light or too dark to print correctly.

Any inquiries concerning a paper that has been accepted for publication should be sent directly to the Editorial Department, American Mathematical Society, P. O. Box 6248, Providence, RI 02940-6248.

Editors

This journal is designed particularly for long research papers (and groups of cognate papers) in pure and applied mathematics. Papers intended for publication in the Memoirs should be addressed to one of the following editors:

Ordinary differential equations, partial differential equations, and applied mathematics to JOHN MALLET-PARET, Division of Applied Mathematics, Brown University, Providence, RI 02912-9000; e-mail: am438000@ brownvm.brown.edu.

Harmonic analysis, representation theory, and Lie theory to ROBERT J. STANTON, Department of Mathematics, The Ohio State University, 231 West 18th Avenue, Columbus, OH 43210-1174; electronic mail: Stanton® f u n c t i o n . m p s . o h i o - s t a t e . e d u .

Ergodic theory, dynamical systems, and abstract analysis to DANIEL J. RUDOLPH, Department of Mathematics, University of Maryland, College Park, MD 20742; e-mail: [email protected].

Real and harmonic analysis and elliptic partial differential equations to JILL C. PIPHER, Department of Mathematics, Brown University, Providence, RI 02910-9000; e-mail: jpipherOgauss.math.brown.edu.

Algebra and algebraic geometry to EFIM ZELMANOV, Department of Mathematics, University of Wisconsin, 480 Lincoln Drive, Madison, WI 53706-1388; e-mail: zelmanov@math. w i se . edu

Algebraic topology and cohomology of groups to STEWART PRIDDY, Department of Mathematics, Northwestern University, 2033 Sheridan Road, Evanston, IL 60208-2730; e-mail: s_priddy@math. nwu. edu.

Global analysis and differential geometry to ROBERT L. BRYANT, Department of Mathematics, Duke University, Durham, NC 27706-7706; e-mail: bryantOmath.duke.edu.

Probability and statistics to RODRIGO BANUELOS, Department of Mathematics, Purdue University, West Lafayette, IN 47907-1968; e-mail: banuelos@ math.purdue.edu.

Combinatorics and Lie theory to PHILIP J. HANLON, Department of Mathematics, University of Michigan, Ann Arbor, MI 48109-1003; e-mail: phi l .hanlonOmath. lsa .umich.edu.

Logic and universal algebra to GREGORY L. CHERLIN, Department of Mathematics, Rutgers University, Hill Center, Busch Campus, New Brunswick, NJ 08903; e-mail: cherlin@math. r u t g e r s . edu .

Number theory and arithmetic algebraic geometry to ALICE SILVERBERG, Department of Mathematics, Ohio State University, Columbus, OH 43210-1174; e-mail: silver@math . o h i o - s t a t e . edu.

Complex analysis and complex geometry to DANIEL M. BURNS, Department of Mathematics, University of Michigan, Ann Arbor, MI 48109-1003; e-mail: [email protected].

Algebraic geometry and commutative algebra to LAWRENCE EIN, Department of Mathematics, University of Illinois, 851 S. Morgan (MIC 249), Chicago, IL 60607-7045; email: u224250uicvm.uic.edu.

All other communications to the editors should be addressed to the Managing Editor, PETER SHALEN, Department of Mathematics, Statistics, and Computer Science, University of Illinois at Chicago, Chicago, IL 60680; e-mail: [email protected].

Selected Titles in This Series (Continued from the front of this publication)

564 Wensheng Liu and Hector J. Sussmann, Shortest paths for sub-Riemannian metrics on rank-two distributions, 1995

563 Fritz Gesztesy and Roman Svirsky, (m)KdV solitons on the background of quasi-periodic finite-gap solutions, 1995

562 John Lindsay Orr, Triangular algebras and ideals of nest algebras, 1995 561 Jane Gilman, Two-generator discrete subgroups of PSL(2, R), 1995 560 F. Tomi and A. J. Tromba, The index theorem for minimal surfaces of higher genus, 1995 559 Paul S. Muhly and Baruch Solel, Hilbert modules over operator algebras, 1995 558 R. Gordon, A. J. Power, and Ross Street, Coherence for tricategories, 1995 557 Kenji Matsuki, Weyl groups and birational transformations among minimal models,

1995 556 G. Nebe and W. Plesken, Finite rational matrix groups, 1995 555 Tomas Feder, Stable networks and product graphs, 1995 554 Mauro C. Beltrametti, Michael Schneider, and Andrew J. Sommese, Some special

properties of the adjunction theory for 3-folds in P 5 , 1995 553 Carlos Andradas and Jesus M. Ruiz, Algebraic and analytic geometry of fans, 1995 552 C. Krattenthaler, The major counting of nonintersecting lattice paths and generating

functions for tableaux, 1995 551 Christian Ballot, Density of prime divisors of linear recurrences, 1995 550 Huaxin Lin, C*-algebra extensions of C(X), 1995 549 Edwin Perkins, On the martingale problem for interactive measure-valued branching

diffusions, 1995 548 I-Chiau Huang, Pseudofunctors on modules with zero dimensional support, 1995 547 Hongbing Su, On the classification of C*-algebras of real rank zero: Inductive limits of

matrix algebras over non-Hausdorff graphs, 1995 546 Masakazu Nasu, Textile systems for endomorphisms and automorphisms of the shift,

1995 545 John L. Lewis and Margaret A. M. Murray, The method of layer potentials for the heat

equation on time-varying domains, 1995 544 Hans-Otto Walther, The 2-dimensional attractor of x'(t) = -fix(t) + f(x(t - 1)), 1995 543 J. P. C. Greenlees and J. P. May, Generalized Tate cohomology, 1995 542 Alouf Jirari, Second-order Sturm-Liouville difference equations and orthogonal

polynomials, 1995 541 Peter Cholak, Automorphisms of the lattice of recursively enumerable sets, 1995 540 Vladimir Ya. Lin and Yehuda Pinchover, Manifolds with group actions and elliptic

operators, 1994 539 Lynne M. Butler, Subgroup lattices and symmetric functions, 1994 538 P. D. T. A. Elliott, On the correlation of multiplicative and the sum of additive

arithmetic functions, 1994 537 I. V. Evstigneev and P. E. Greenwood, Markov fields over countable partially ordered

sets: Extrema and splitting, 1994 536 George A. Hagedorn, Molecular propagation through electron energy level crossings,

1994 535 A. L. Levin and D. S. Lubinsky, Christoffel functions and orthogonal polynomials for

exponential weights on [-1,1], 1994 534 Svante Janson, Orthogonal decompositions and functional limit theorems for random

graph statistics, 1994 (See the AMS catalog for earlier titles)

Compact Connected Lie Transformation Groups on Spheres With Low Cohomogeneity - II

Documents

Transcript of Compact Connected Lie Transformation Groups on Spheres With Low Cohomogeneity - II