CONGRUENCE SUBGROUPS AND ENRIQUES SURFACE AUTOMORPHISMS 6 1 DANIEL ALLCOCK 0 2 Abstract. We give conceptual proofs of some results on the au- n tomorphism group of an Enriques surface X, for which only com- a J putationalproofshavebeenavailable. Namely,thereis anobvious 1 upper bound on the image of AutX in the isometry group of X’s numerical lattice, and we establish a lower bound for the image ] that is quite close to this upper bound. These results apply over G any algebraically closed field, provided that X lacks nodal curves, A or that all its nodal curves are (numerically) congruent to each . othermod2. Inthisgeneralitytheseresultswereoriginallyproven h t by Looijenga and Cossec–Dolgachev, developing earlier work of a Coble. m [ 1 v 1. Introduction 3 0 Our goal in this paper is to give conceptual proofs of some known 1 computer-based results on the group of automorphisms of an Enriques 0 surface X. These results are valid over any algebraically closed field. 0 . Ofcourse, AutX acts onPicX, hence onthe quotient Λ of PicX by its 1 0 torsion subgroup Z/2. This quotient Λ is called the numerical lattice, 6 and is a copy of the famous E lattice. One can describe it as E ⊕U 1 10 8 : where we take E8 to be negative definite and U = (cid:0)01 10(cid:1). v The main object of interest in this paper is the image Γ of AutX i X in O(Λ). This is “most” of AutX, because the kernel of AutX → r a Γ is finite, and in fact very tightly constrained [7, §7.2]. All of our arguments concern Λ and various Coxeter groups acting on it. For the underlying algebraic geometry we refer to [6], [7] and [9]. Because Λ has signature (1,9), the positive norm vectors in Λ ⊗R fall into two components. Just one of these contains ample classes; we call it the future cone and write O↑(Λ) for the subgroup of O(Λ) preserving it. Vinberg showed (theorem 2.2 below) that O↑(Λ) is the Date: January 1, 2016. 2010 Mathematics Subject Classification. Primary: 14J28; Secondary: 20F55. Supported by NSF grant DMS-1101566. 1 2 DANIEL ALLCOCK Coxeter group W with diagram 237 (1.1) Besides preserving Λ, the main constraint on Γ is that it must pre- serve the ample cone, hence its closure, the numerically effective cone nef(X). The nef cone is described in terms of X’s nodal curves (i.e., smoothrational curves), which have self-intersection −2 by the adjunc- tion formula. If X has nodal curves then nef(X) consists of the vectors in Λ ⊗ R having nonnegative inner product with all of them. In the special case that X lacks nodal curves, nef(X) is the closure of the future cone. The remaining constraint on Γ concerns the F vector space V := 2 Λ/2Λ. Dividing latticevectors’ normsby 2andthenreducing modulo 2 defines on V an F quadratic form of plus type. “Plus type” means 2 that V has totally isotropic spaces of largest possible dimension, in this case 5. Although we won’t use this property, it does explain the presence of some superscripts +. We write O(V) for the isometry group of this quadratic form. We will use ATLAS notation for group structures and finite groups throughout the paper; see [1], especially + §5.2. In this notation, O(V) has structure O (2) : 2. (Caution: the 10 ATLAS uses “O” for the simple composition factor of an orthogonal + group—in this case an index 2 subgroup. Some authors write O (2) 10 for O(V) itself.) Theorem 1.1 (Theunnodalcase). Suppose an Enriques surface X has no nodal curves. Then Γ contains the level two congruence subgroup W (2), meaning the kernel of the natural map O↑(Λ) → O(V). 237 So Γ must be one of the finitely many groups between W (2) and 237 W . Because O↑(Λ) → O(V) is a surjection, the possibilities corre- 237 + spond to subgroups of O (2) : 2. Different X can lead to different Γ, 10 so one cannot say much more without specifying X more closely. If X is unnodal then AutX acts faithfully on Λ, by [7, Thm. 7.3.6]. So one can identify Γ with AutX. Incharacteristic 0 onecandescribe Γinterms oftheperiodoftheK3 surface which covers X. In this way one can show that for a generic Enriques surface without nodal curves, Γ is exactly W (2); see [2]. 237 The positive characteristic analogue of this seems to be open. Given a nodal curve, regarded as an element of PicX, the corre- sponding nodal root means its image under PicX → Λ. It is called a root because it has norm −2 and so the reflection in it is an isometry CONGRUENCE SUBGROUPS AND ENRIQUES SURFACES 3 of Λ. Distinct nodal curves have intersection number ≥ 0, hence dif- ferent images in Λ. So the nodal curves and nodal roots are in natural bijection. Given a nodal root, its corresponding nodal class means its image in V, always an anisotropic vector. Theorem 1.2 below is the analogue of theorem 1.1 in the “1-nodal case”: when X has at least one nodal curve, and all nodal curves represent a single nodal class. We will use lowercase letters with bars to indicate elements of V, whether or not we have in mind particular lifts of them to Λ. By definition of the quadratic form on V, every nodal class ν¯ is anisotropic. So its transvectionx¯ 7→ x¯+(x¯·ν¯)ν¯isanisometryofV. Weindicatestabilizers using subscripts, for example O↑(Λ) in the next theorem. ν¯ Theorem 1.2 (The 1-nodal case). Suppose an Enriques surface X has a single nodal class ν¯ ∈ V. Then the O↑(Λ)-stabilizer O↑(Λ) of ν¯ is ν¯ the Coxeter group (1.2) Write W for the subgroup generated by the reflections corresponding 246 to the leftmost 10 nodes. Then (1) nef(X) is the union of the W -translates of the fundamental 246 chamber of the Coxeter group (1.2). (2) Γ lies in W , which is the full O↑(Λ)-stabilizer of nef(X). 246 (3) Γ contains the subgroup W (2) defined as the subgroup of W 246 246 that acts trivially on ν¯⊥ ⊆ V. (4) W (2) acts transitively on the facets of nef(X). 246 (5) AutX acts transitively on the nodal curves of X. Remarks. (a) The heavy edge in the diagram indicates parallelism of the corresponding hyperplanes in H9, or equivalently that the last pair of roots has intersection number 2. (b) Suppose X is a generic nodal Enriques surface. Then AutX acts faithfully on Λ, so it can be identified with Γ; see [7, Prop. 7.4.1]. Fur- thermore, in characteristic 6= 2,3,5,7 or 17, Γ coincides with W (2), 246 by [8, Thm. 1]. (c) W (2) is the group called W(2) by Cossec and Dolgachev [8]. 246 But, contrary to what the notation might suggest, the kernel of our W → O(V)isnotthesameastheirW(2). Thisisbecausetheydefine 246 their congruence subgroups withrespect totheReye lattice ratherthan the Enriques lattice Λ. The Reye lattice has index 2 in Λ: it is the preimage of ν¯⊥ ⊆ V. 4 DANIEL ALLCOCK Theorems 1.1 and 1.2 are modern forms of results of Coble [4, Thms. (4) and (30)]. But Cossec–Dolgachev [6, p. 162] state that his proofs were incorrect. They credit Looijenga with the first proof of theo- rem 1.1, never published, and give proofs of both theorems, following Looijenga’s ideas. See [6, Thms. 2.10.1 and 2.10.2]. Their proof of the first relied on a lengthy hand computation, and the second required computer assistance. The author is grateful to RIMS (Kyoto University) for its hospi- tality while working on this paper, to Igor Dolgachev for posing the problem of improving on the computer computations in [6], and to Shigeru Mukai for stimulating discussions. 2. The case of no nodal curves A root means a lattice vector of norm −2. In this section our model for Λ is the span of the rootsin figure 2.1. Two of them have inner product 1 or 0, according to whether they are joined or not. By the theory of reflection groups [3, V.4], these 10 vectors form a set of simple roots for the group W generated by their reflections. We will write W 237 235 resp. W for the subgroup generated by the reflections corresponding 236 to the top 8 resp. 9 nodes. Also, we will write Λ for the span of the 0 first 8 roots. This is a copy of the E lattice in the “odd” coordinate 8 system, namely (cid:8)(x1,...,x8) (cid:12)(cid:12) all xi in Z or all in Z+ 12, and Pxi ≡ x8 mod 2(cid:9) from [5, §8.1 of Ch. 4]. Its isometry group is the E Weyl group W , 8 235 + which has structure 2 · O (2) : 2. Sometimes we will write lattice 8 vectorsas(x;y,z)withx ∈ Λ andy,z ∈ Z, andinnerproduct(x;y,z)· 0 (x′;y′,z′) = x·x′ +yz′ +y′z. Lemma 2.1 (The stabilizer of a null vector). W is the full stabilizer 236 O(Λ) of the null vector ρ = (0;1,0). It has structure Λ : W , where ρ 0 235 Λ indicates the group of “translations” 0 (x;0,0) 7→ (x;−λ·x,0) (2.1) T : (0;1,0) 7→ (0;1,0) λ∈Λ0 2 (0;0,1) 7→ (λ;−λ /2,1) Proof. The T arecalled translations because of how they act onhyper- λ bolic space when ρ is placed at infinity in the upper halfspace model. One checks that they are isometries, that T = T T , and that λ+µ λ µ W = AutΛ acts on them in the same way it acts on Λ . Next, 235 0 0 W contains the reflection in λ = (−−−−−−−+;00), because this is 236 2 2 2 2 2 2 2 2 a root of Λ . Also, W contains the reflection in (−−−−−−−+;10), 0 236 2 2 2 2 2 2 2 2 CONGRUENCE SUBGROUPS AND ENRIQUES SURFACES 5 (+−000000;0,0) (0+−00000;0,0) (00+−0000;0,0) (−−−+++++;0,0) 2 2 2 2 2 2 2 2 (000+−000;0,0) (0000+−00;0,0) (00000+−0;0,0) (000000+−;0,0) (−−−−−−−+;1,0) 2 2 2 2 2 2 2 2 (00000000;−1,1) Figure 2.1. Simple roots for W = O↑(Λ), with re- 237 specttothenorm(x ,...,x ;y,z)2 = −x2−···−x2+2yz. 1 8 1 8 We have abbreviated ±1 to ± and hidden some commas. because this root is second from the bottom in figure 2.1. The product of these two reflections is T , the sign depending on the order of the ±λ factors. So W contains the translation by a root of Λ . Conjugating 236 0 by W shows that W contains the translations by all the roots of 235 236 Λ . Since Λ is spanned by its roots, W contains all translations. 0 0 236 Thetranslationsacttransitivelyon{(x;−x2/2,1) | x ∈ Λ }, whichis 0 the set of null vectors having inner product 1 with ρ. The simultaneous stabilizer of ρ and (0;0,1) is the orthogonal group of Λ , which is 0 W ⊆ W . Since O↑(Λ) and its subgroup W act transitively on 235 236 ρ 236 (cid:3) the same set, with the same stabilizer, they are the same group. The proof of the following theorem of Vinberg illustrates the tech- nique of cusp-counting, which we will use several times. To avoid repe- tition we take “null vector” to mean a future-directed primitive lattice vector of norm 0. “Cusp counting” means: when a Coxeter group acts on an integer quadratic form of signature (1,n) and has finite volume fundamental chamber in hyperbolic space, then its orbits on null vec- tors are in bijection with the ideal vertices of the chamber. And these in turn are in bijection with the maximal affine subdiagrams of the Coxeter diagram. Theorem 2.2 (Vinberg [10]). O↑(Λ) = W . 237 6 DANIEL ALLCOCK Proof. The image in hyperbolic 9-space of the fundamental chamber hasfinitevolume, withallvertices inH9 except foroneonitsboundary. This follows from the general theory of hyperbolic reflection groups: the vertices in H9 correspond to the rank 9 spherical subdiagrams of figure 2.1, and the last vertex corresponds to the affine subdiagram E . 8 e It follows that there is only one W -orbit of null vectors, i.e., W 237 237 acts transitively on them. Since W contains the full O↑(Λ)-stabilizer 237 of one of them (lemma 2.1), it is all of O↑(Λ). (cid:3) The most important ingredient in the proof of theorem 1.1 is the construction of automorphisms of X, for which we refer to the proof of theorem3in[9, §6]. Λhasmanydirectsumdecompositionsasacopyof E plus a copy of U. For every such decomposition, the transformation 8 whichnegates theE summand iscalledaBertini involution, andarises 8 from an automorphism of X. (Very briefly: consider the linear system |2E + 2E |, where E and E are the effective classes corresponding 1 2 1 2 to the null vectors in the U summand. This is a 2-to-1 map onto a 4- nodal quartic del Pezzo surface in P4, and the Bertini involution is the deck transformation of this covering.) Bertini involutions obviously lie in the level 2 congruence subgroup of O↑(Λ), hence in W (2). Also, 237 every conjugate of a Bertini involution is again an Bertini involution. So the group they generate is normal in O(Λ). Proof of theorem 1.1. The proof amounts to showing that the Bertini involutions generate W (2). We write S (“small”) for the group they 237 generate, and think of O↑(Λ) as the “large” group. To understand the relation between small and large, we will introduce a “medium” group M. Its relationships with S and O↑(Λ) are easy to work out. Then the relationship between S and O↑(Λ) will be visible. We define M as the group generated by S and W . The central 236 involution B of W ⊆ M is a Bertini involution. Also, its conjugacy 235 action on Λ ⊆ W is inversion. By the normality of S, M contains 0 236 T BT−1 ◦B−1 = T ◦BT−1B−1 = T ◦T−1 = T λ λ λ λ λ −λ 2λ for all λ ∈ Λ . It follows that M/S is a quotient of W /hB,all T i = 0 236 2λ 28 : O+(2) : 2. On the other hand, it is easy to see that W acts on 8 236 V as the full O(V)-stabilizer of ρ¯, which has structure 28 : O+(2) : 2. 8 (Repeat the proof of lemma 2.1, reduced mod 2.) This shows how S is related to M: since it has index ≤ |28 : O+(2) · 2| in M, and 8 lies in the kernel of the surjection M → O(V) ∼= 28 : O+(2) · 2, S ρ¯ 8 coincides with the kernel. That is, M → O(V) induces an isomorphism M/S → O(V) . ρ¯ CONGRUENCE SUBGROUPS AND ENRIQUES SURFACES 7 (+−000000;00) (0+−00000;00) (00+−0000;00) (−−−+++++;00) 2 2 2 2 2 2 2 2 (000+−000;00) (0000+−00;00) (00000+−0;00) (000000+−;00) (−−−−−−−+;02) 2 2 2 2 2 2 2 2 (−−−−−−−+;10) 2 2 2 2 2 2 2 2 Figure 2.2. SimplerootsforM = O↑(Λ) ; seetheproof ρ¯ of theorem 2.2. The advantage of working with M rather than S is that it contains the Coxeter group M whose simple roots are shown in figure 2.2. We 0 will see later that in fact M is all of M; for now we just prove M ⊆ 0 0 M. First, M contains W by definition. To see that M contains 236 the reflection in the last root (the lower right one), note that r = (−−−−−−−+;0,0)isarootofΛ ,soitsreflectionliesinW . Choose 2 2 2 2 2 2 2 2 0 235 an element λ of Λ having inner product −1 with it. Then T ∈ S 0 2λ sends r to (−−−−−−−+;2,0). Now consider the conjugate of T by 2 2 2 2 2 2 2 2 2λ the isometry of Λ which exchanges the last two coordinates. This lies in S by normality, and sends r to (−−−−−−−+;0,2). Therefore M 2 2 2 2 2 2 2 2 contains the reflection in this root. This finishes the proof that M contains M . 0 NextweclaimthatM isallofO↑(Λ) . ThefactthatO↑(Λ) contains 0 ρ¯ ρ¯ M (even M) is obvious. Now observe that M ’s chamber has finite 0 0 volume, with 3 cusps, corresponding to the D subdiagram and two 8 e E subdiagrams. Therefore M has 3 orbits on null vectors. On the 8 0 e other hand, O↑(Λ) has at least 3 orbits on null vectors, since it has ρ¯ three orbits on isotropic vectors in V. (Namely: ρ¯ itself, the other isotropic vectors, and the isotropic vectors not orthogonal to ρ¯.) So O↑(Λ) and its subgroup M have the same orbits on null vectors. The ρ¯ 0 stabilizer of ρ in either of them is W , proving M = O↑(Λ) . Since 236 0 ρ¯ M ⊆ M ⊆ O↑(Λ) , this also shows the equality of M with these 0 ρ¯ groups. 8 DANIEL ALLCOCK Finally, we have [O↑(Λ) : S] = [O↑(Λ) : M][M : S] = (cid:2)O↑(Λ) : O↑(Λ)ρ¯(cid:3)(cid:12)O(V)ρ¯(cid:12) (cid:12) (cid:12) = (cid:2)O(V) : O(V)ρ¯(cid:3)(cid:12)O(V)ρ¯(cid:12) (cid:12) (cid:12) = (cid:12)O(V)(cid:12) (cid:12) (cid:12) Since S lies in the kernel of the surjection O↑(Λ) → O(V), it must be (cid:3) the whole kernel, finishing the proof. 3. Preparation for the 1-nodal case This section can be summarized as “the same as section 2 with E ⊕U 7 in place of E ⊕U”. To tighten the analogy it is necessary to use the 8 “even”coordinatesystem fortheE latticeinplaceofthe“odd”onewe 8 used in the previous section. So now we take the E lattice to consist of 8 the vectors (x ,...,x ) with even coordinate sum and either all entries 1 8 in Z or all in Z+ 1. See [5, §8.1 of Ch. 4]; these coordinates differ from 2 those of section 2 by negating any coordinate. We take Λ to consist of the vectors (x ,...,x ;y,z) with (x ,...,x ) in the E lattice and 1 8 1 8 8 y,z ∈ Z. The normis still −x2−···−x2+2yz. Mimicking our notation 1 8 from section 2, we write Λ for the sublattice {(x ,...,x ;0,0)} of Λ. 0 1 8 We write ν for the root (++++++++;00) of Λ . It stands for 2 2 2 2 2 2 2 2 0 “nodal root”, although for this section it is just a root. Its orthogonal complement inΛ isacopyoftheE rootlattice, anditsfullorthogonal 0 7 complement ν⊥ in Λ is E ⊕U. It is easy to see that ν⊥ is spanned by 7 therootsinfigure3.1, andthattheirinnerproductsareindicatedinthe usual way by the edges of the diagram. In particular they form a set of simple roots for the Coxeter group W generated by their reflections. 245 We also write W for the subgroup generated by the reflections in the 244 top 8 roots, and regard both these groups as acting on all of Λ, not just ν⊥ = E ⊕U. The next two results are proven the same way lemma 2.1 7 and theorem 2.2 were. Lemma 3.1. W is the full stabilizer of the null vector ρ = (0;1,0) 244 in O(Λ) . (cid:3) ν Theorem 3.2 (Vinberg). W is all of O↑(Λ) . (cid:3) 245 ν Bertini involutions: from the presence of an E diagram in figure 3.1 8 we see that ν⊥ has sublattices isomorphic to E . Every E sublattice 8 8 is unimodular, hence a direct summand of Λ, so the involution that negates the E summand is an isometry. Since this summand was 8 CONGRUENCE SUBGROUPS AND ENRIQUES SURFACES 9 (+−000000;0,0) (0+−00000;0,0) (00+−0000;0,0) (000+−000;0,0) (−−−−++++;0,0) 2 2 2 2 2 2 2 2 (0000+−00;0,0) (00000+−0;0,0) (000000+−;1,0) (00000000;−1,1) Figure 3.1. Simple roots for O↑(Λ) , where ν is the ν root (++++++++;00) of Λ; see theorem 3.2. 2 2 2 2 2 2 2 2 chosen in ν⊥, we obtain an element of O↑(Λ) . These are called Bertini ν involutions, and act trivially on V. Kantor involutions: by construction, ν⊥ has direct sum decomposi- tions E ⊕ U. For any such decomposition, the central involution in 7 W(E ) acts by negation on the E summand and trivially on the U 7 7 summand. These are called Kantor involutions. Every one acts on V by the transvection in ν¯. (Proof: the complement in Λ of the U sum- mand is a copy of E containing hνi ⊕ E . The Kantor involution is 8 7 the product of the negation map of this E , which acts trivially on V, 8 with the reflection in ν.) Theorem 3.3. The Kantor and Bertini involutions generate the sub- group O↑(Λ)ν,ν¯⊥ of O↑(Λ) that fixes ν and acts trivially on ν¯⊥ ⊆ V. Proof. We reuse our strategy from theorem 1.1. That is, we write S for the subgroup of O↑(Λ) generated by the Kantor and Bertini ν involutions, and think of it as “small”. We think of O↑(Λ) as “large”. ν Obviously S is normal in O(Λ) . To relate these groups we define the ν “medium” group M to be generated by S and W . 244 Recall that W has structure E : W(E ) = E : (2×O (2)) where 244 7 7 7 7 theinitial E indicatestherootlatticeregardedasagroup. Thecentral 7 involution in 2 × O (2) is a Kantor involution. Mimicking the proof 7 of theorem 1.1 shows that M/(S ∩ M) is a quotient of 27 : O (2). 7 Continuing the mimicry, the image of M in O(V) has structure 27 : (2 × O (2)), which is the simultaneous stabilizer O(V) . (Note: 27 7 ν¯,ρ¯ and 2 × O (2) are subgroups of 28 and O+(2) : 2 from the proof of 7 8 10 DANIEL ALLCOCK e = (−3, 111111,−3;12) 10 2 222222 2 e = (+−000000;00) 1 e = (0+−00000;00) 2 e = (00+−0000;00) 3 e = (000+−000;00) (−−−−++++;00) = e 4 2 2 2 2 2 2 2 2 8 e = (0000+−00;00) 5 e = (00000+−0;00) (000000+−;02) = e 6 9 e = (000000+−;10) 7 Figure 3.2. Simple roots for M = O↑(Λ) where ρ = ν,ρ¯ (0;1,0) and ν = (++++++++;00); see the proof of 2 2 2 2 2 2 2 2 theorem 3.3. theorem 1.1. The 27 is the subgroup of O(V) that fixes ρ¯ and acts ν¯ trivially on ρ¯⊥/hρ¯i, and 2 × O (2) acts faithfully on ρ¯⊥/hρ¯i.) Every 7 Kantor involution acts trivially on ν¯⊥, and the image of M in O(ν¯⊥) ∼= O (2) has structure 27 : O (2). It follows that S is the kernel of the 9 7 action of M on ν¯⊥ ⊆ V. So we may identify M/S with the stabilizer of ρ¯ in O(ν¯⊥). Next we claim that M contains the Coxeter group M with sim- 0 ple roots pictured in figure 3.2. First, e ,...,e are the simple roots 1 8 of W , whose reflections lie in M by definition. The proof that M 244 contains the reflection in e is exactly the same as in the proof of the- 9 orem 1.1. (Only the Kantor involutions are needed.) For e , observe 10 that it and e ,...,e span a copy of the lattice A ⊕E . Furthermore, 2 8 1 7 (e +e +e +e )/2 lies in Λ, so the saturation of this A ⊕E is a copy 10 5 7 8 1 7 of E . The reflection in e is equal to the Bertini involution of this E , 8 10 8 times the central involution of the copy of W(E ) ⊆ W generated by 7 244 the reflections in e ,...,e . Therefore M contains this reflection. 2 8 The same argument as in the proof of theorem 1.1 shows that M = 0 M = O↑(Λ) . (This time the affine diagrams are D A and two E ’s.) ν,ρ¯ 6 1 7 e e e The final step of the proof is also conceptually the same as before. First, [O↑(Λ) : S] = [O↑(Λ) : M][M : S] ν ν = [O↑(Λ)ν : O↑(Λ)ν,ρ¯](cid:12)O(ν¯⊥)ρ¯(cid:12) (cid:12) (cid:12) = [O(V)ν¯ : O(V)ν¯,ρ¯](cid:12)O(ν¯⊥)ρ¯(cid:12) (cid:12) (cid:12)