Draft version January 6, 2017 PreprinttypesetusingLATEXstyleemulateapjv.5/2/11 STRONG CLUSTERING OF LYMAN BREAK GALAXIES AROUND LUMINOUS QUASARS AT Z ∼41,2 Cristina Garc´ıa-Vergara 3,4, Joseph F. Hennawi 4,5, L. Felipe Barrientos 3, and Hans-Walter Rix 4 Draft version January 6, 2017 ABSTRACT In the standard picture of structure formation, the first massive galaxies are expected to form at the highest peaks of the density field, which constitute the cores of massive proto-clusters. Luminous quasars (QSOs) at z ∼ 4 are the most strongly clustered population known, and should thus reside in massive dark matter halos surrounded by large overdensities of galaxies, implying a strong QSO- 7 galaxy cross-correlation function. We observed six z ∼ 4 QSO fields with VLT/FORS exploiting a 1 novel set of narrow band filters custom designed to select Lyman Break Galaxies (LBGs) in a thin 0 redshiftsliceof∆z ∼0.3,mitigatingtheprojectioneffectsthathavelimitedthesensitivityofprevious 2 searches for galaxies around z (cid:38) 4 QSOs. We find that LBGs are strongly clustered around QSOs, n and present the first measurement of the QSO-LBG cross-correlation function at z ∼ 4, on scales a of 0.1 (cid:46) R (cid:46) 9h−1Mpc (comoving). Assuming a power law form for the cross-correlation function J ξ = (r/rQG)γ, we measure rQG = 8.83+1.39h−1Mpc for a fixed slope of γ = 2.0. This result is in 4 0 0 −1.51 agreement with the expected cross-correlation length deduced from measurements of the QSO and LBG auto-correlation function, and assuming a linear bias model. We also measure a strong auto- ] A correlation of LBGs in our QSO fields finding rGG =21.59+1.72h−1Mpc for a fixed slope of γ =1.5, 0 −1.69 which is ∼ 4 times larger than the LBG auto-correlation length in random fields, providing further G evidencethatQSOsresideinoverdensitiesofLBGs. Ourresultsqualitativelysupportapicturewhere h. luminous QSOs inhabit exceptionally massive (Mhalo >1012M(cid:12)) dark matter halos at z ∼4. p Subject headings: cosmology: observations–earlyUniverse–large-scalestructureofuniverse–galax- - ies: clusters: general – galaxies: high-redshift – quasars: general o r t s 1. INTRODUCTION search for these so-called proto-clusters around known a massive galaxies at high redshift. [ Our understanding of structure formation suggests Oneveryfruitfultechniquetofindhigh-redshiftproto- that small inhomogeneities in the density field shortly 1 clusters has been to use the presence of an active su- after the Big Bang grew over cosmic time via gravi- v per massive black hole (BH) as a signpost for a massive tational instability (e.g. Dodelson 2003; Padmanabhan 4 galaxyandhencemassivedarkmatterhalointhedistant 2006; Schneider 2015) into massive dark matter halos at 1 Universe (e.g. Venemans et al. 2007; Kashikawa et al. z = 0. As clusters of galaxies are the most massive, 1 2007; Overzier et al. 2008; Morselli et al. 2014). This gravitationally bound structures in the Universe, they 1 technique is motivated by several considerations. First, musthaveformedfromthehighestdensitypeaksatearly 0 the masses of supermassive BHs (M ) are known to . times. This make them ideal laboratories for studying BH 1 the formation and evolution of cosmic structure. tightlycorrelatewiththebulgemassoftheirhostgalaxy 0 (Magorrian et al. 1998; Ferrarese & Merritt 2000; Geb- Because of the small areas of sky surveyed at high- 7 hardt et al. 2000), and possibly with the masses of their redshift, and the low comoving number density ∼ 1 10−7Mpc−3 oflocalclusters(Gioiaetal.2001;Vikhlinin host dark halos (Mhalo) (Ferrarese 2002, but see Kor- : mendy & Bender 2011). Intriguingly, the most luminous v et al. 2009), the evolutionary link between these low- i redshiftclustersandhigh-redshiftgalaxieshasbeenchal- quasars (QSOs) at z > 3 have MBH ∼ 1−6×109M(cid:12) X lenging to make. The progenitors of clusters are ex- (Shen et al. 2011), comparable to the most massive r tremelydifficulttoidentifywhenthedensitycontrastbe- knownlocalBHs. IfthepresentdayMBH−Mhalorelation a tween the forming cluster and its surroundings is small. holdsatearlytimes,suchBHsshouldresideinexception- For this reason, a commonly adopted approach is to ally massive halos. Second, some studies have suggested that the nuclear activity in active galactic nuclei (AGN) is triggered by processes related to the environment [email protected] where they reside. For example, galaxy mergers could 1Based on observations collected at the European Organi- trigger the AGN activity (Bahcall et al. 1997; Wyithe & zation for Astronomical Research in the Southern Hemisphere, Chile. Data obtained from the ESO Archive, Normal program, Loeb2002;Hennawietal.2015),andgalaxymergersoc- visitormode. ProgramID:079.A-0644. cur preferentially in dense environments (Lacey & Cole 2We dedicate this work to the memory of Josef Fried, who 1993). This would imply that the existence of an AGN originallyobtainedandanalyzedthedataonwhichthisworkis requiresadenseenvironmentaroundit. Finally,another based. 3Instituto de Astrof´ısica, Pontificia Universidad Cat´olica de lineofevidencethatQSOstracetherarestenvironments Chile,AvenidaVicun˜aMackenna4860,Santiago,Chile. at high redshift arises from their extremely strong clus- 4Max-PlanckInstitutfu¨rAstronomie(MPIA),K¨onigstuhl17, tering. Indeed,Shenetal.(2007)determinedthatQSOs D-69117Heidelberg,Germany. 5DepartmentofPhysics,UniversityofCalifornia,SantaBar- at z > 3.5 have a comoving auto-correlation length of bara,CA93106,USA r0 =24.3h−1Mpc (for a fixed correlation function slope 2 Garc´ıa-Vergara, C. et al. of γ = 2.0), making them the most strongly clustered galaxies around QSOs at z ∼ 6−7. Kim et al. (2009) population in the universe, and demanding that they re- studied five QSO fields at z ∼ 6 and reported a mix side in the most massive M > 1012M dark matter of overdensities and underdensities, and Husband et al. halo (cid:12) halos at this epoch. Additionally, the Shen et al. (2007) (2013) find galaxy overdensities in z ∼5 QSOs environ- correlationfunction,agreeswiththatrequiredtoexplain ments, but they note that even some randomly chosen the abundance of binary QSOs at z > 3.5 (Hennawi patches of sky without AGN signposts (‘blank fields’) et al. 2010; Shen et al. 2010), indicating that overdense at the same redshift contain similar galaxy overdensi- structures around QSOs extend down to scales as small ties. Indeed, surveys of a few deg2 for z ∼ 6 LBGs or as 100h−1kpc. Since in hierarchical clustering models, LAEs have identified comparable overdensities in blank QSOs and galaxies trace the same underlying dark mat- field pointings (e.g. Ouchi et al. 2005; Ota et al. 2008; ter density distribution, the generic prediction is that Toshikawa et al. 2012). These mixed results at z (cid:38)5 do galaxies should be very strongly clustered around QSOs not yet provide compelling evidence QSOs inhabit mas- at z (cid:38)3.5. Observationally this should be reflected as a sivedarkmatterhalosatthehighestredshifts,andmore strong QSO-galaxy cross-correlation function. work is clearly required. The QSO-galaxy cross-correlation function has been One complication of these studies is that the majority measured at z < 4 in the past. At z (cid:46) 1 it is found to of them are focused on dropout selection, which selects beingoodagreementwiththeauto-correlationofgalax- galaxiesoverabroadredshiftrangeof∆z ∼1(e.g.Ouchi ies and QSOs, and it has been shown to be independent etal.2004a),correspondingto∼520h−1cMpcatz =4. of the QSO luminosity, and weakly dependent on red- A large part of such a volume is unassociated with the shift (e.g. Padmanabhan et al. 2009; Coil et al. 2007). QSO, which introduces projection effects that dilute the Adelberger & Steidel (2005) measured the AGN-galaxy overdensity around the QSO making it much more dif- cross-correlation function at higher redshifts (2(cid:46)z (cid:46)3) ficult to detect. Furthermore, most work at the highest findingacross-correlationlengthofr ∼5h−1Mpcfora redshifts have focused their searches around a handful 0 slope of γ = 1.6 which is similar to the auto-correlation of individual QSOs, and given the poor statistics and of Lyman Break Galaxies (LBGs) at z ∼ 3 (Adelberger large cosmic variance (which is typically not taken into et al. 2003). They also claim an independence of the account), this could preclude the detection of an over- cross-correlationlengthwiththeAGNluminosity,imply- density. ing that both faint and bright AGNs should be found in In this paper we study the environs of QSOs at z ∼4. haloswithsimilarmasses. Thehighestredshiftmeasure- There are several advantages to working at this redshift. mentofQSOenvironmentsistheworkofTrainor&Stei- First, it is the highest redshift at which auto-correlation del(2012),whoquantifiedtheclusteringofLBGsaround measurements exist for QSOs (Shen et al. 2007), estab- 15 hyper-luminous QSOs at z = 2.7. They find a QSO- lishing that they reside in massive dark matter halos. LBG cross-correlation length of r = 7.3±1.3h−1Mpc Second, the luminosity function and clustering proper- 0 for a fixed slope of γ = 1.5 and claim that this mea- ties of z ∼ 4 galaxies are also well known (e.g. Shen surement is in agreement with the Adelberger & Steidel etal.2007;Ouchietal.2004a,2008). Thewell-measured (2005) results. Additionally, they compute a halo mass luminosity function allows us to accurately determine for those QSOs of log(M /M )=12.3±0.5, which is the background number density, essential for a robust halo (cid:12) in agreement with the halos masses inferred for fainter clustering analysis. Furthermore, the fact that the auto- QSOs at the same redshift (White et al. 2012). correlation of QSOs and galaxies are both known, gives Theoretical considerations suggest that high-redshift us an idea of what the cross-correlation should be on QSOs live in massive dark matter halos, but not nec- large scales where linear bias models apply. In practi- essarily the most massive ones (Fanidakis et al. 2013). cal terms, redshift z ∼ 4 also represents a compromise However,ahighsignaltonoiseclusteringanalysisisnec- since the dark matter halos hosting QSOs are still ex- essary to confirm this hypothesis. pected to be massive (Shen et al. 2007), while at the In addition to these statistical clustering analyses, same time the characteristic galaxy luminosity L can ∗ manystudiesofindividualAGNenvironmentshavebeen beimagedwithmuchshorterexposuretimesthangalax- conducted. The population of AGNs whose environ- ies at z (cid:38) 5, allowing us to observe a larger statistical ments have been studied most intensively are the high- sample of QSO fields. Note that at z ∼ 4 the universe redshiftradiogalaxies(HzRGs)atz ∼2−4, whichhave was only ∼ 1.5Gyr old, and only 0.5Gyr has elapsed beenshowntooftenresideinproto-clusterenvironments since the end of reionization. Thus, our QSO targets are (e.g. Venemans et al. 2007; Intema et al. 2006; Overzier definitelyyoungobjectsresidinginlargescalestructures etal.2008;Hennawietal.2015). Athigherredshiftsthe that are still forming. environments of other classes of AGN, such as optically- HerewepresentVLT/FORSimagingofsixz ∼4lumi- selected QSOs, are currently less well constrained. Most nousQSOsfields. Usinganovelnarrowband(NB)filter previous work focuses on searching for galaxies around technique designed to select LBGs in a narrow redshift the most distant z (cid:38) 5 QSOs, and these results paint range(∆z ∼0.3)aroundQSOs. Thisminimizestheline- a diverse and rather confusing picture: Stiavelli et al. of-sightcontamination,dramaticallyreducingtheprojec- (2005), Zheng et al. (2006), Kashikawa et al. (2007), tion effects which are inherent in broad-band selection. Utsumi et al. (2010), and Morselli et al. (2014) find a We measure the QSO-LBG cross-correlation function at quite strong enhancement of galaxies compared to con- z ∼ 4 for the first time, to determine whether luminous trol fields around z ∼5−6 QSOs, whereas Willott et al. QSOs at z ∼4 are surrounded by overdensities of galax- (2005) Ban˜ados et al. (2013),Simpson et al. (2014) and ies. The large sample of QSOs studied allows us to beat Mazzucchelli et al. (2016), find no significant excess of downthenoisefromlimitednumbersofgalaxiesandcos- Strong Clustering of Lyman Break Galaxies around Luminous Quasars at z ∼4 3 mic variance. Shen et al. 2009). A summary of the QSO properties Theoutlineofthispaperisasfollows. Insection§2we are listed in Table 1, where we show more recent M BH describe the QSO target selection, we explain the novel estimates taken from Shen et al. (2011). NB imaging technique used to select LBGs, and we give details of the imaging observations, data reduction, and 2.2. A Novel Method to Select LBGs photometry. We present the color criteria used to select The traditional Lyman break technique used to select LBGs and compute the redshift selection function of the high-redshift galaxies relies on the detection of the 912˚A sampleinsection§3. ThemeasurementoftheQSO-LBG flux break (the so-called Lyman limit break) observed in cross-correlation function and LBG auto-correlation in galaxiesduetotheabsorptionofphotonswithλ<912˚A QSO fields are presented in section § 4, where we also by neutral hydrogen in their interstellar and circum- estimate the power law correlation function ξ(r) = (r/ galactic media. For this selection method, two bands r )−γ parameters r and γ. We test the robustness of 0 0 are typically used bracketing the break, one located at our results in section § 5, and summarize and conclude λ<912(1+z)˚A,andtheotheratλ>912(1+z)˚A,where in section § 6. z is the redshift of the galaxies in question. Given this ThroughoutthispapermagnitudesaregivenintheAB configuration, a non-detection is expected in the band system (Oke 1974; Fukugita et al. 1995) and we adopt blueward of the break, whereas a clear detection is ex- a cosmology with H = 70kms−1Mpc−1, Ω = 0.26 0 m pected redward of it, such that a very red color will be and Ω = 0.74 which is consistent with the nine-year Λ measured. Additionally, a third band is added at longer Wilkinson Microwave Anisotropy Probe (WMAP) Ob- wavelengthsinordertoeliminatepossiblecontaminants. servations (Hinshaw et al. 2013). Comoving and proper This method was originally explored using the UGR fil- Mpc are denoted as “cMpc” and “pMpc”, respectively. tersystemtodetectgalaxiesatz ∼3(Steideletal.1995, 2. OBSERVATIONSANDDATAREDUCTION 1996, 2003), however, it was subsequently generalized to higher redshift (z ∼ 4−5) by using a filter set shifted The dataset presented in this section was obtained to longer wavelengths (Steidel et al. 1999; Ouchi et al. from the ESO Archive (Program ID: 079.A-0644, P.I: 2004a). Rix). This program was designed to search for LBGs in At higher redshifts (z (cid:38) 4), a second break in galaxy z ∼ 4 QSOs environments using a novel NB filter tech- spectra becomes important. The Lyα opacity of the in- nique. TheaimwastotestwhetherQSOswiththemost tergalactic medium (IGM) rapidly increases with red- massiveBHsatz ∼4liveinthemostmassivedarkmat- shift, such that a large fraction of photons emitted by terhalos. Inthissectionwesummarizethestrategyused toselectthetargetedQSOs,weexplaintheNBtechnique galaxies with λ<1216˚A are absorbed by neutral hydro- used to select LBGs, and we provide details of the imag- gen. This implies a break at λ = 1216˚A (the so-called ing observations, reduction process, and photometry. Lymanalphabreak),whichcanbeusedtoselectgalaxies analogous to the traditional Lyman break technique de- 2.1. QSO Target Selection scribed above. This Lyα break detection technique has ThePIofthisprogramdesignedacustomsetoffilters been used to successfully identify galaxies and QSOs at (see § 2.2 for details) to search for LBGs in QSO envi- z (cid:38)6 (Fan et al. 2000; Bouwens et al. 2007, 2010; Oesch ronments. Using experiments with mock-catalogs, they et al. 2010; Ban˜ados et al. 2016). showed that this filter set allowed one to select galaxies In order to achieve our goal of selecting galaxies phys- with z = 3.78±0.08. Given this small redshift interval, ically associated with high-redshift QSOs, we need to and with the goal of stacking the galaxy number counts select LBGs within a narrow redshift range centered on from several QSO fields, the QSO targets were selected the QSO. However, the Lyman break method (using ei- to span a narrow redshift range of ∆z = 0.04, centered ther the Lyman limit or Lyα breaks) efficiently selects at z =3.78. LBGs in a broad redshift slice of ∆z ∼ 1 (e.g. Ouchi Taking advantage of the large sample of QSOs from et al. 2004a; Bouwens et al. 2007, 2010), corresponding the Sloan Digital Sky Survey (SDSS; York et al. 2000), to ∼ 520h−1cMpc at z = 4. For such a broad redshift they first selected all QSOs in this redshift range. Given range,theoverdensitysignalaroundtheQSOwillbesig- the goal of studying the most massive dark matter ha- nificantly diluted by the projection of galaxies at much los at z ∼ 4, believed to be correlated with the most larger distances, hundreds of comoving Mpc away. massive BHs, only QSOs with M (cid:38) 109 M were se- In order to address this problem, the PI proposed a BH (cid:12) lected. As is typical, the M was estimated from the novel selection technique analogous to the Lyα break BH emission line widths and continuum luminosities using method, but with the difference that the selection of the so-called single-epoch reverberation mapping tech- LBGsisperformedusingtwoNBfilterslocatedveryclose nique (Vestergaard 2002). One of the targeted QSOs to each other, instead of using broad bands. These filter was not selected from SDSS, but it was added to the curves are compared to those used for traditional LBG samplebecauseitbelongstotheredshiftandM range selection in Fig. 1. The advantage of using NB filters is BH of interest. The final sample is comprised of six bright that they allow one to select LBGs in a much narrower QSOs with i<20.2 mag. redshiftrangeof∆z ∼0.3(∼167cMpcatz =3.78)(see We verified that none of the QSOs had a detected ra- section § 3.4), which is ∼3.3 times smaller than the red- dio emission counterpart at 20cm by checking the Faint shift range covered when broad bands are used, allowing Images of the Radio Sky at Twenty-centimeters (FIRST one to minimize line-of-sight projections from physically Becker et al. 1995) catalog, since it is known that radio unassociated galaxies. emissioncouldstronglyaffectthegalaxyclusteringprop- This method has never been used before to select erties in AGN environments (e.g. Venemans et al. 2007; LBGs, and the filters used to perform the observations 4 Garc´ıa-Vergara, C. et al. Table 1 TargetedQSOsproperties. Field RA(J2000) DEC(J2000) Redshift i log(MBH/M(cid:12))a SDSSJ0124+0044 01:24:03.78 00:44:32.67 3.834 17.99 10.15±0.03 SDSSJ0213–0904 02:13:18.98 -09:04:58.28 3.794 19.03 9.57±0.18 J2003–3300b 20:03:24.12 -32:51:45.02 3.773 17.04 9.7 SDSSJ2207+0043 22:07:30.48 00:43:29.37 3.767 19.47 9.13±0.16 SDSSJ2311–0844 23:11:37.05 -08:44:09.56 3.745 20.18 9.41±0.24 SDSSJ2301+0112 23:01:11.23 01:12:43.34 3.788 19.44 8.55±0.80 a VirialBHmassesfromShenetal.(2011). b ThisQSOwasnotselectedfromSDSS,butitwastargetedbecauseitbelongstotheredshiftrangeofinterest. The propertiesshownherearefrom(McLeod&Bechtold2009),whodonotreporttheerrorfortheBHmassmeasurement. 8000s, 4000s, and 1800s for NB , NB , and r 571 596 GUNN respectively. Observations were acquired in shorter indi- 1.0 Ly alpha z=3.78 vidualditheredexposures,inordertofillthegapbetween NB NB 0.8 571 596 the CCDs and to facilitate the data reduction process r 0.6 GUNN (cosmicrayandbadpixelrejection, buildingasuperflat, etc). A spectrophotometric standard star was observed 0.4 units) 00..02 Ly limit odnulryinigntthheethserceoenndigahntds wthaisrd0.6ni-gh0t.8. aTrchseect.ypical seeing bitrary 1.0 Ly alph a z=3.78 tasSkcsienancediomuargoews nwecruestroemduccoeddesuswinrgittsetnanidnartdheIRInAteFr6- F (arλ00..68 B R i aclcutdiveedDbaiatsasLuabntgraucatgioen(IaDnLd)fl.aTthfieerldedinugc.tiAons opurorciemssagines- Fshi ogwun000r...4e0240o01n0.aLUyL lpiBmpiGet5r00sp0iamnuella:teFd6i0lt0se0preλcc(toÅrnu)fimg7u0ar0a0ttizon=us38e0.d7080in(steheiss9se0tc0ut0idoyn, efiioasxbnetShlejdaeEribicvenxtdietgtsrerasadowcgcuieietittolnhslrauicgnesm(mudBfipranetae-arhmcrttelfliiinenopasntp.&cioinpFmmAgaoatrbratgnitleengohrseouna,drtstcis,,ttrhhwe1wmae9eet9.esfi6pcdr)ieseurwtnfsocamiernsmagfusrekstadehemddetehtausoelnlwrcflteirhtagehet-- ate a source catalog for each individual image and then § 3.1 for the simulated spectra details). The NBs were designed speciallyforthisprogram,inordertoidentifyLBGsatz∼3.78by SCAMP(Bertin2006)wasusedtocomputeanastromet- detectingtheLymanalphabreak. Thisfilterconfigurationselects ricsolution,usingtheSDSS-DR7r-bandstarcatalogsas galaxiesinaquietnarrowredshiftsliceof∆z∼0.3. Lower panel: the astrometric reference. Finally, the individual images Exampleofafiltersetusedtoidentifygalaxieswiththestandard were sky-subtracted, re-sampled, and median-combined LymanbreaktechniquewhichisbasedinthedetectionoftheLy- manlimitbreak. ThefiltercurvesshownarethoseusedbyOuchi using SWarp (Bertin et al. 2002), and then the noisy etal.(2004a)tofindLBGsatz∼4overaredshiftsliceof∆z∼1.0 edges of the combined images were trimmed. For the flux calibration, we only had observations of were custom designed to select LBGs at z ∼ 3.78 cen- the spectrophotometric standard star SA109-949 at the tered on the redshift of our six QSO targets. The two beginning of the last two nights. The tabulated spec- NB filters used in this study are NB571 (λeff = 5657˚A, trum of this star has a coarse sampling of 25˚A(Stone FWHM = 187˚A), and NB (λ = 5947˚A, FWHM = 1996) which is not suitable when NB filters are used. 596 eff 116˚A),whichwere designedto havea gapbetweenthem For the first night, spectrophotometric standard stars to exclude the Lyα emission line at z = 3.78. Then werenotobserved,butwetookadvantageoftwoexisting the galaxy selection is not influenced by the Lyα line- SDSS star spectra in one of the fields taken during that strength , but rather is sensitive to the Lyα break. Ad- night. The coordinates of the stars with available SDSS ditionallydatawasalsocollectedinthebroadbandfilter spectra are RAstar1 = 21.014, DECstar1 = 0.740872 and r (λ = 6490˚A) to help remove low-redshift inter- RAstar2 = 21.057, DECstar2 = 0.686577 and the median GUNN eff signal-to-noise ratio per angstrom of their spectra at the lopers. wavelengths of interest is 13.3 and 8.5 respectively. 2.3. VLT Imaging and Data Reduction The flux calibration process was as follows. For the first night calibration we convolved the SDSS star spec- Imaging observations were acquired on three con- tra with the three filters curves in order to obtain stan- secutive nights during 2007 September 9 - 11, using dardmagnitudes. Thesemagnitudeswerecomparedwith the FOcal Reducer and low dispersion Spectrograph 1 the stars instrumental magnitude (obtained using the (FORS1; Appenzeller & Rupprecht 1992) instrument MAG AUTO of SExtractor on the combined science im- on the Very Large Telescope (VLT). The field-of-view ages) to obtain the zero-points (ZPs) for each filter. A (FOV) of FORS1 is 6.8×6.8 arcmin2 which corresponds mean final ZP was computed from the two stars and the to ∼ 3.0×3.0pMpc2 at z = 3.78. The instrument pixel typical error for this ZP measurement was ∼ 0.08 mag. scale is 0.251 arcsec/pix for images binned 2×2. For the second and third night calibration, we used the EachQSOfieldwasobservedinthethreefiltersshown in Fig. 1. The total exposure time for the filters was 6 ImageReductionandAnalysisFacility Strong Clustering of Lyman Break Galaxies around Luminous Quasars at z ∼4 5 spectrumoftheobservedspectrophotometricstartocon- volve it only with the broad-band filter curve to obtain Table 2 4σ limitmagnitudesperfieldmeasuredina2(cid:48)(cid:48) diameteraperture ther ZP.Theerrorinthiscomputationwas∼0.02 GUNN andseeingmeasuredontherGUNN images. mag. After that, the differential ZPs from the first night wereusedtodeterminetheNBzero-pointsforthesecond Field NB571 NB596 rGUNN Seeing[(cid:48)(cid:48)] and third nights for which we obtained a typical error of ∼0.11 mag. SDSSJ0124+0044 26.04 25.51 25.86 0.83 SDSSJ0213–0904 26.18 25.71 25.92 0.89 2.4. Photometric Catalogs J2003–3300 26.05 25.44 25.62 0.45 SDSSJ2207+0043 26.03 25.38 25.78 0.53 Object detection and photometry were performed us- SDSSJ2311–0844 26.02 25.60 25.84 0.76 ing SExtractor in dual mode, with the r image as SDSSJ2301+0112 26.04 25.55 25.91 0.70 GUNN thedetectionimage. WesettheparametersBACK SIZE and BACKPHOTO THICK such that the background was calculated in regions of 64 pixels in size and then where the photometric catalogs are assumed to be 100% recomputed locally in an annulus area of 24 pixels of complete. We extrapolated the linear fit to fainter mag- thickness centered around the object. The parameters nitudes and we measured the completeness as a function DETECT MINAREA and DETECT THRESH were set ofmagnitudeastheratioofthehistogramrelativetothat such that every group of at least five contiguous pixels linearfit. Wefindthatatour4σ limitingmagnitudethe havingavalueabove1.5σ (withσ thebackgroundRMS) completeness is on average ∼12%. was considered as an object. 3. LBGSELECTIONATZ=3.78 In order to ensure an adequate color measurement we need to carry out photometry in the same object LBG candidates at z = 3.78 were selected using the area for the three different filters. Therefore, we con- Lyαbreaktechniqueadaptedtoourcustomfilters,which volved our images with a Gaussian kernel to degrade its target the Lyα break at λ =(1+z)1216˚A. Our rest−frame PSF to match it with the worst seeing image for each twoNBfilterswerechosentobracketthisbreak,andthus field. Then, the object magnitudes were estimated by we expect that LBGs at z =3.78 will have red colors in the MAG APER parameter of SExtractor using a fixed NB −NB . Butifweusedonlythiscolorcriteria,we 571 596 aperture of 2(cid:48)(cid:48) diameter. This magnitude is not neces- could be including some low-redshift galaxy interlopers sarily the total magnitude of the object, but is used to in the sample. In order to remove them, a third filter is computethecolorsofgalaxies. Withthischoice,ifgalax- usedtogiveameasurementoftheLBGcontinuumslope ies at z ∼4 are unresolved by the PSF, we are including using the NB −r color. 596 GUNN the flux out to ∼ 3σ of the object’s PSF (for a seeing Sincethefiltersusedinthisstudyarenotstandard,the of 0.8(cid:48)(cid:48)). This ensures that we measure the majority of colorcriteriatoselectLBGsisunknown. Wealsodonot the object’s flux, as well as avoid contamination from know what colorslow-redshift galaxy contaminants have other close sources. Magnitudes of objects not detected in this filter system. For this reason, we must explore or detected with a signal-to-noise ratio (S/N) <2 either howgalaxiespopulatethecolorspaceinordertoselecta in NB or NB were assigned the value of the corre- completeLBGssamplewhileavoidinglow-redshiftinter- 571 596 sponding 2σ limiting magnitude. lopers. Furthermore,inordertoperformaLBGscluster- Here, the S/N of each object is defined as the ratio ing analysis in QSO fields we need to know the number of counts in the 2(cid:48)(cid:48) aperture, given by SExtractor, to density of LBGs expected at random locations in the the rms sky noise in the aperture. This rms sky noise universe. When a standard filter set is used (e.g. LBG is calculated using an IDL procedure which performs 2(cid:48)(cid:48) selection using broad band filters), this number density aperture photometry in ∼ 5000 different random posi- canbecomputeddirectlyfromtheLBGluminosityfunc- tions in the image (avoiding the locations of objects) to tion measured from work using similar filters. However, compute a robust measurement of the mean sky noise. in our case if we compute the number density from this Thermsskynoiseiscalculatedasthestandarddeviation LBG luminosity function, we have to correct this quan- of the distribution of mean values. tity to take into account the fact that our filter system Magnitudes were corrected for extinction due to air- is mapping a different survey volume and does not nec- mass using the atmospheric extinction curve for Cerro essarily identify all of the LBGs selected by broad-band Paranal (Patat et al. 2011), and by galactic extinction selection. Specifically,weneedtoa)determinewhatfrac- calculatedusingtheSchlegeletal.(1998)dustmapsand tion of LBGs we are detecting at any redshift (i.e. the extinction laws of Cardelli et al. (1989) with R = 3.1. completeness) and b) determine the redshift range over V The error in the measured magnitude was computed by which we are selecting LBGs (∆z). Both of these goals errorpropagation,withtheobjectfluxerrorgivenbythe can be achieved by performing an accurate computation rms noise N in the aperture computed as we described of the redshift selection function φ (z), defined as the z above. LBG completeness as a function of redshift. The mean 4σ limiting magnitude of the reduced im- In order to perform the optimal LBG selection and ages was of 26.06 for NB , 25.53 for NB and 25.82 compute φ (z), we conducted detailed simulations to 571 596 z forr for2(cid:48)(cid:48)diameterapertures. Theselimitingmag- model the distribution of LBG colors in the color-space. GUNN nitudes are listed in Table 2 for each field. Inthissectionwedetailhowthecolormodelingwasper- For each field, we computed the completeness of the formed, we study what contaminants could be affecting photometriccatalogsfortheimagedetectionr . For our LBGs selection, and we define a color criteria to se- GUNN that, we linearly fitted the logarithmic magnitude distri- lect LBGs at z = 3.78. Finally, we present the redshift bution in the magnitude range 21.0 < r < 24.5 selection function providing the completeness as a func- GUNN 6 Garc´ıa-Vergara, C. et al. tion of redshift for the sample. 1.0 3.1. LBG Color Modeling We performed a Monte Carlo simulation of 1000 LBG 0.8 spectra at each redshift, that were created to have dif- ferent UV continuum slopes and Lyα equivalent widths L(EBWGLsypαe)c,tsruachintfhoramttehdeybyreopurrodkuncoewtlhedegsepaocfeLoBfGpospsriobple- W )αLy0.6 E erties. P ( 0.4 Eachsimulatedrest-framespectrumwascreatedinthe following way. As a starting point, we considered a tem- plate galaxy spectrum generated from Bruzual & Char- 0.2 lot (2003) population synthesis models7, corresponding to an instantaneous burst model with an age of 70Myr, 0.0 a Chabrier (2003) IMF, and a metallicity of 0.4Z , as −300 −200 −100 0 (cid:12) expected for LBGs at z ∼ 4 (Jones et al. 2012). We as- EWLyα sumedapowerlawUVcontinuumforthistemplatewith Figure 2. Normalized probability distribution function of amplitude A and a slope α , such that we modeled its flux as FBC(λ) = AλαBC. WBCe fit this model to the tem- cEoWrrLesypαonudsetdoefomristshioensliimneusl.atEeWd Lsypαecwtraas,cwhohseernefrnoemgataivGeauvassluiaens plate spectrum over the UV continuum range (here de- distribution with rest-frame mean µ = −25˚A and σ = 40˚A fined as 1300˚A< λ <2000˚A) by least-squares minimiza- (Shapley et al. 2003) plus an exponential tail of high EWLyα tion to obtain the best fit A and α parameters. valueswithscalelengthofW0=−64˚A(Ciardulloetal.2012). BC First, we modified the UV slope of this template by multiplyingitsfluxbyλα−αBC inordertoobtainaspec- amplitude B, as we described above, and Fcont is the trum with a power law UV continuum given by Aλα. flux of the continuum given by Aλα. Note that we de- The new slope α was chosen as a value taken randomly fined negative values of EWLyα for emission lines and fromaGaussiandistributionwithmeanµ=−1.676and positive for absorption lines. σ =0.39. These values are motivated by Bouwens et al. Once α and EWLyα were chosen for a given simulated (2009), whopresented the UVcontinuum slopedistribu- spectrum, we dust-attenuated it using the starburst red- tion of LBGs at z ∼ 4 for samples selected in different dening curve from Calzetti et al. (2000) and adopted a magnitude ranges. color excess value of E(B−V) = 0.16 according to the Second,weaddedaGaussianLyαlinewithrest-frame valuesestimatedforLBGsatz ∼3(Shapleyetal.2003). central wavelength λ = 1215.7˚A, standard deviation After the dust-attenuation is applied, we model the Lyα σ andamplitudeB whichadjuststheintensityofthe fact that only a small fraction of Lyman limit pho- Lyα line. For all the simulated spectrum we used a fixed tons escape LBGs with an escape fraction parameter σLyα =1˚A which agrees with the σLyα of the composite feλs<c912. Although this value is observationally poorly constrained, studies suggest it is in the range 0.04-0.14 spectrum of LBGs at z ∼ 4 (Jones et al. 2012). The B (Fern´andez-Soto et al. 2003; Shapley et al. 2006; Ouchi value was adjusted in order to model a Lyα line with a EWLyα value drawn randomly from a distribution cho- etal.2004a). Weassumedafixedvalueoffeλs<c912 =0.05, sen to agree with observations of LBGs. The EW and multiplied the spectrum at λ ≤ 912˚A by this Lyα distribution was given by a Gaussian core plus a tail value. Wealsotestedourresultsusingdifferentvaluesof to large negative equivalent widths to represent strong fλ<912, finding that the colors of simulated galaxies are esc line emitters. For the Gaussian core we adopted a mean relatively insensitive to the exact value of fλ<912 used, esc µ=−25˚A andstandarddeviationσ =40˚A(rest-frame), because these wavelengths are subsequently significantly basedonthemeasurementsofShapleyetal.(2003),who attenuated by the IGM transmission function (see be- studied the spectra of 811 LBGs at z ∼ 3. We thus as- low). sume that the Gaussian core of the LBG EW distri- Finally,weredshiftedeachmodelspectrumtodifferent Lyα bution does not evolve significantly from z ∼3 to z ∼4. redshifts on a grid with a grid spacing of 0.02 and rang- For the tail representing strong line-emitters, we modi- ing from z = 3.2 to z = 4.4. In the redshifting process fiedtheGaussianbyaddinganexponentialfunctionwith we used the IGM transmission model T (λ) for the cor- z rest-frame EW scale length of W = −64˚A, as pre- responding redshift z from Worseck & Prochaska (2011) Lyα 0 sentedinCiardulloetal.(2012). Inthiswayourmodelof to attenuate the flux blueward of the Lyα line8. Note line emission encompasses both LBG and LAE spectra. thatinprincipleweshouldattenuateboththecontinuum Fig. 2 shows the EW probability distribution func- bluewardoftheLyαlineandthelineitself,however,the Lyα tion used to simulate our spectral models. The EW EW valuesusedinthissimulationaretakenfromthe Lyα Lyα are defined as: literature, which are observed values that are not cor- (cid:90) F rected for IGM attenuation, such that this line emission EW =− Lyαdλ, (1) iseffectivelyalreadyattenuated. InFig.3weshowsome Lyα F cont examplesofourrest-framesimulatedspectra,whichhave where F is the flux of the Lyα line (with the con- been normalized to have the same flux at λ=1245˚A. Lyα tinuum subtracted), which is given by a Gaussian with Ateachredshift,weintegratedthespectraagainstour 7 Obtainedfromhttp://bruzual.org/ 8 KindlyprovidedtousbyG.Worseck. Strong Clustering of Lyman Break Galaxies around Luminous Quasars at z ∼4 7 0.16. However,ifweconsidertheintrinsicscatterinLBG 3.0 properties(continuumslopeandEW )andphotomet- 150 Lyα nits) 22..05 F (arbitrary units)λ15000 rp0staih.oc5ioinnuuatnlansdc)dhessrie−gptlhaea0icln.nyt6taciL(cid:46)eoBwsm,NGidtpBhseleer5it9ncze6ots(cid:39)−lhaoimr3rsG.rpb7aUl8renNo,gLNahedBo(cid:46)wGwseise0tlvhe.(8cein.tNri,doBIwinnc5ea7rp1teareg−lidsnioocNbninypBetlg5eoe9r,d6eowetb(cid:38)noe- u ry 10210 1215 1220 take into account the colors of low-redshift galaxies in ra 1.5 λ(Å) our filter system to define a final selection criteria. We bit perform this analysis in § 3.2, where we also test our r F (aλ1.0 LarByGtrcaoclkorpmreosednetliendgibnyprreepvriooduuscwinogrkthuesiLnBgGbreovaodlubtiaonnd- LBG selection. 0.5 3.2. Low-Redshift Galaxy Colors 0.0 500 1000 1500 2000 2500 3000 We use template galaxy spectra to develop a basic un- λ(Å) derstanding of how low-redshift galaxies populate the color-color diagram in our new filters. We used a set of Figure 3. Example of ten rest-frame simulated spectra using fivecommonlyusedtemplatesforestimatingphotometric ourMonteCarlosimulation. Thespectrahavebeennormalizedto redshifts, such that they span the range of galaxy spec- havethesamefluxvalueatλ=1245˚A.Thesubplotintheupper tralenergydistributions(SEDs). Thetemplatesarefrom rightcornershowsazoom-inoftheregionoftheLyαline. thephoto-zcodeEASY(Brammeretal.2008),whichare distilled from the PEGASE spectral synthesis models. three filter transmission curves to obtain the fluxes and We redshifted these template spectra from z = 0 to then the LBG colors. In order to model the impact of z = 3, and integrated them over our filter transmission noise,weaddedphotometricerrorstothesimulatedLBG curves to generate their evolutionary track. Note that photometry. Tothisendwefirstassignedanr mag- GUNN we need not attenuate these spectra by the IGM trans- nitude to each simulated object by randomly drawing a mission function T (λ), since our NB filters never cover value from the z ∼ 4 LBG luminosity function, inte- z rest-framewavelengthslowerthan1216˚A forthelowred- grated over the same magnitude range as our LBG sam- shifts considered. In Fig. 5 we show the evolutionary ple (24.0 ≤ r ≤ 25.6 or 0.76 ≤ L/L ≤ 3.5; see GUNN ∗ tracks for different galaxy types together with the me- §3.3)9. Wealsoweightedtheluminosityfunctionbythe dian LBG evolutionary track that we computed in § 3.1. completeness of the source detection at each apparent In order to test our Monte Carlo simulation as well as magnitude and for each field (computed in § 2.4), which theevolutionarytracksforlow-redshiftgalaxies,wehave takes into account the fact that the fraction of sources usedour1000simulatedspectraateachredshifttocom- detecteddependsontheirmagnitude,suchthatthepho- putethemedianLBGevolutionarytrackinthestandard tometric catalog is complete for bright sources but less BRifiltersetusedtoselectLBGsatz ∼4(seeFig.1)by complete at the faint end. In this way the incomplete- Ouchietal.(2004a). Wealsocomputedtheevolutionary ness of our photometry is also factored into our color track of these low-redshift galaxies in the standard LBG modeling. filtersinthesamewayasdescribedabove. Theseresults Based on the simulated LBG colors the chosen r GUNN are shown in Fig. 7, where we also overplot the selec- value, we then determined the magnitude in the other tion region used by Ouchi et al. (2004a) to select z ∼ 4 two filters NB and NB for each spectrum in each 571 596 LBGs. WefindthatthemedianLBGevolutionarytrack redshift bin. In order to construct a noise model , we from our Monte Carlo model lies within the Ouchi et al. selected a galaxy sample from our photometric catalogs, (2004a) selection region, and selects LBGs at z (cid:38) 3.5 andwecomputedthemedianmagnitudeerrorasafunc- as claimed. Note also that our LBG evolutionary track tionofthemagnitudeforeachfilter(withthemagnitude agrees well with the Ouchi et al. (2004a) evolutionary error computed as we explained in § 2.4). Finally, we track(seeFig.4oftheirpaper)determinedfromamuch assigned random Gaussian distributed magnitude errors simpler model of LBG spectra and IGM transmission. using our median relations, and then added this noise to In addition we see that the evolutionary tracks of low- the model photometry which defined the final photome- redshift galaxies lie comfortably outside the BRi LBG tryofthesimulatedspectra. Thecolorsforthe1000sim- selection region as claimed by Ouchi et al. (2004a). ulated spectra at each redshift are shown in Fig. 4. We HoweverFig.5showsthatinourNBfilterset,someof also computed the median of our 1000 rest-frame Monte the low-redshift galaxies have similar colors as z = 3.78 Carlo spectra, redshifted it, and obtained the colors at LBGs, which suggests that our new filter configuration each redshift to compute the median evolutionary track could make it challenging to select a sample of LBGs at of LBG colors, shown as the black solid line in Fig. 4. z = 3.78 with high completeness and at the same time Fig. 4 indicates that the median colors of LBGs at high purity. When we use NB filters the low-redshift z =3.78areNB −NB =1.05andNB −r = 571 596 596 GUNN galaxy colors are located in a wider region in the color- color plot in comparison with the location of the color 9 Giventhatforeachfieldwereachedslightlydifferentlimiting locus of contaminants when broad band filters are used. magnitudes, we simulated the LBG photometry field by field ac- cordingtotheirrespectiverGUNNlimitingmagnitude. Thisresults We attribute this to sensitivity of the NB filters to fea- inaslightlydifferentredshiftselectionfunctionforeachfield. turesinthegalaxyspectrasuchasemissionorabsorption 8 Garc´ıa-Vergara, C. et al. Figure 4. Color-color diagram showing the simulated colors for 1000 LBGs spectra, plotted as redshift color-coded points according to thecolorbar. ThemedianLBGevolutionarytrackisplottedasablackcurve. ThefilledpointsoverthiscurveindicatethemedianLBGs colors at different redshift ranging from 3.6 to 4.2. The largest circle shows the exact position of the median z =3.78 LBG colors. The dashedlineindicatestheselectionregionusedtoselectLBGsaccordingtotheeqn.(2). lines. In the case of broad bands these features are di- binationwiththeλ ∼2900˚A break. WhentheNB RF 571 luted by averaging over large regions of spectra, but for is located over this absorption the filter NB falls on 596 NB the features result in large excursions in color with the continuum, then, red colors are detected (points E changing redshift, making the low-redshift galaxy locus andGonthemagentaandbrowncurves,respectively,in extend over a larger region of color space that overlaps Fig. 5). Other interlopers are galaxies with strong flux with the colors of z =3.78 LBGs. breaks redshifted just between our NB filters. One ex- Given that LBG colors at z = 3.78 span the range amplearegalaxiesatz ∼1.23withalargeλ ∼2640˚A RF NB571−NB596 (cid:38) 0.5 and −0.6 (cid:46) NB596−rGUNN (cid:46) 0.8 break(pointsDandFonthemagentaandbrowncurves, (seeFig.4), thereareseveraltypesofcontaminantsthat respectively, in Fig. 5) and galaxies at z ∼ 1.83 with a could be affecting our LBG selection. Their colors are strong break at λ ∼ 2085˚A (point B on the brown RF indicatedbypointsontherespectivelow-redshiftgalaxy curve in Fig. 5). evolutionary tracks are labeled by with letters in Fig. 5, and some examples are shown in Fig. 6. The first type 3.3. Selection Region and LBG Sample are red galaxies at z ∼0.45 having a large λ ∼4000˚A RF As we are interested in measuring the clustering prop- Balmer break and strong Calcium H & K absorption. erties of LBGs at z = 3.78, we need to select a sample This break is located just between our two NBs, so they withhighcompletenessandpurity. Inordertoavoidlow- present red colors (point C in brown curve in Fig. 5). redshiftcontaminants,wewereforcedtochooseasmaller The second type of interlopers are star-forming galaxies selection region in the color-color diagram, which results atz ∼0.60withstrong[OII]3727˚A emissionlines. Ifthe in relatively low completeness, but it ensures that the NB is located just over this line, and NB over the 596 571 sample is not highly contaminated. continuum, we again detect red colors (point A on green First, we defined two vertical color cuts in Fig. 4, one curveinFig.5). Thethirdtypeofinterlopersaregalaxies to the left of the median LBG colors at z = 3.78 and at z ∼ 1.04 with strong MgI and MgII absorption lines one to the right. The first cut is meant to exclude LBGs atλ =2852˚A,andλ =2799˚A respectivelyincom- RF RF located in the upper left region of the diagram, which Strong Clustering of Lyman Break Galaxies around Luminous Quasars at z ∼4 9 22..55 44 LBG 3.9 Elliptical 22..00 SSba 33 3.8 1 Sc 3.7 Irregular 11..55 3.6 3 1 B596 G 22 33..54 3 2 B− N571 11..00 3.7 3B.8CDFE B - R 11 33..23 00032 2 1 N 00..55 A 3.9 2 1 3.6 30 3 LBG 00..00 00 0 2 1 EllipticSaal 4.0 Sb Sc Irregular −−00..55 --11 −−11..00 −−00..55 00..00 00..55 11..00 11..55 --00..55 00..00 00..55 11..00 11..55 22..00 NB − r R - i 596 GUNN Figure 5. Evolutionarytracksoflow-redshiftgalaxiesredshifted Figure 7. Same as Fig. 5 but using the filter system used by from z = 0 to z = 3. We plot as brown, magenta, orange, blue, Ouchi et al. (2004a) to select LBGs at z ∼ 4 (broad band filters and red curves the evolutionary track of elliptical, Sa, Sb, Sc, B, R, i). Filled circles over the black curve indicate colors of and Irregular galaxies, respectively. We overplotted the track of LBGsfromredshift3.2to3.9. Filledcirclesoverthecurveofthe LBGscomputedaswasexplainedinsection§3.1asablackcurve. low-redshift galaxies indicate colors from redshift 0.0 to 3.0. The Filled circles over the black curve indicate colors of LBGs from dashed line is indicating the region used Ouchi et al. (2004a) to redshift3.6to4.0,andthelargestblackpointindicatestheexact selectLBGsintheirwork. position of the color of LBGs at z = 3.78. Filled circles labeled with letters over the low-redshift galaxies evolutionary tracks are indicatingthecolorsofsomecontaminantsthatcouldbeaffecting maintaining the highest completeness possible. ourselection: galaxiesatz=0.60(A),z=1.83(B),z=0.45(C), We also tested several different color criteria to select z=1.23(DandF)andz=1.04(EandG). LBGs. In section § 5.2 we will further discuss our color selection,contaminationbylow-redshiftgalaxies,andthe z=0.45 z=0.60 impact that contamination can have on our clustering 1.0 1.0 measurements. There we argue that the choice of color 0.8 0.8 selection that we present here selects a reasonably com- F (arbitrary units)✁000...246 F (arbitrary units)✄000...246 pctiuloetntses:aLrBeGshsoawmnpilnew4i,thanhdigdhepfiunreidty.byOuthrefinfoallloswetinogfcroelloar- NB −NB >1.2 0.0 0.0 571 596 3000 4000 5000 6000 7000 8000 3000 4000 5000 6000 7000 8000 z=(cid:0)(1Å.0)4 z=✂(1Å.2)3 −0.6<NB596−rGUNN <0.8 1.0 1.0 NB −NB >0.7(NB −r )+0.9 (2) 571 596 596 GUNN 0.8 0.8 We selected LBGs based on our galaxy photometry, F (arbitrary units)✆000...246 F (arbitrary units)✞000...246 btbNhyuBetf5aL9rl6eBsqeaGudniredcetoedrncGttsUiionoNuunNrusc,mefiws.lteteIoornsn,hloyatrvodceeoernnastsioSud/rereNreedadu≥scooeb4lij.cde0ocndtinsteattmbehcoaitnttihoahntatiohvoneef 0.0 0.0 FLAGS= 0 in SExtractor, which excluded objects that 3000 4000 5000 6000 7000 8000 3000 4000 5000 6000 7000 8000 (Å) (Å) were blended, saturated, truncated (too close to an im- ☎ ✝ Figure 6. Examples of interlopers that could affect our LBGs age boundary), or affected by very bright neighboring selection. Weshowthegalaxyspectraandthepositionofourthree objects. Brightstarsinourimagesweremaskedinorder filters over it. Top left panel: The spectra of a elliptical galaxy to avoid spurious object detection due to contamination at z = 0.45, with strong Balmer break located at λ = 5840˚A obs from their stellar flux. This procedure results in a set of and intense Calcium H & K absorption. Top right panel: The spectra of a galaxy at z = 0.60, with intense OII emission line masks indicating where we were able to detect galaxies, at λobs = 5925˚A. Bottom left panel: The spectra of a galaxy at which we use later in our clustering analysis to compute z=1.04,withMgIandMgIIabsorptionatλobs∼5650˚A.Bottom the effective area of our survey. right panel: The spectra of a galaxy at z = 1.23, with a large We also imposed a lower limit on the magnitude in breakatλ =5887˚A. obs order to exclude bright low-redshift interloper galaxies fromourselection. Thusweonlyconsideredobjectswith mostly corresponds to LBGs at z ∼3.9 with strong Lyα magnitudes fainter than r = 23.97, corresponding GUNN lineemission. ThesecondcutavoidsLBGsatz >3.9. A to LBGs with L ∼ 3.5L . We chose this value by com- ∗ third color cut defines a lower limit for NB −NB , putingtheLBGluminosityfunctionatz ∼4,andfinding 571 596 which ensures we are detecting the Lyα break, while at the bright end cut at which we would lose no more than the same time avoiding LBGs at z (cid:46) 3.7. We used a 1% of the galaxies. In other words, 99% of the total diagonal color cut, to most effectively avoid the contam- number of LBGs have magnitudes between our bright ination of low-redshift galaxies (see Fig. 5), while at the end cut of r = 24.0 and the limiting magnitude GUNN same time including most of the LBGs at z =3.78, thus r = 25.82 (mean limit magnitude at 4σ for a 2(cid:48)(cid:48) GUNN 10 Garc´ıa-Vergara, C. et al. Figure 8. Color-color diagram for the six stacked QSO fields. Here the evolutionary track showed in Fig. 5 is plotted as redshift color-coded track according to the color bar. We have highlighted the selected LBGs as red points. The magenta points indicate the color of each QSO in the filters. Arrows indicate lower limits for NB571 −NB596 color. These are cases in which the object was not detected in NB571 filter at 2σ level andmagnitudewasreplacedbythecorrespondinglimitmagnitude. diameter aperture) of our images, which corresponds to L=0.76L . Inthiswaywecansafelyassumeweareex- ∗ cluding only extremely rare bright LBGs. For the LBG luminosity function we used the Schechter parameters from Ouchi et al. (2004a) who studied the photometric properties based on a large sample of ∼ 2200 LBGs at z ∼ 4. The values used are φ∗ = 2.8×10−3h3 Mpc−3, 70 M∗ =−20.6 mag, and α=−1.6. 1700 Given all of these selection criteria and the color cuts definedineqn.(2),weselectedLBGsineachofourfields. We compute the total area of our survey by adding the effective area of each individual field, which is defined by subtracting the masked area from the total area of the image. The the total area of our survey is 232.7 arcmin2 corresponding to an average area per image of 38.79 arcmin2 (recall the FOV of FORS1 is 6.8 × 6.8 arcmin2 or 46.24 arcmin2). We show color-color dia- gramsofobjectsdetectedinallsixofourfieldsinFig.8. Wefoundatotalof44LBGs(seeTable3)corresponding to a mean number density of 0.19 LBGs arcmin−2. Im- age cutouts in our three filters for several of our selected LBGsareshowninFig.9. InFig.10weshowthespatial distributionoftheLBGsrelativetotheQSO(reddotat zero)foroursixfields. Wealsoshowtheindividualcolor- colordiagramsandindicatethenumberofLBGsfoundin each individual field in Fig. 11. Note that the number of LBGs in the fields cannot be directly compared because each image has different limiting magnitude and differ- Figure 9. Images of some selected LBGs. From left to right we ent effective area (different reduced image size, masked show the NB571, NB596 and rGUNN images. Each panel is 7.5(cid:48)(cid:48) region, etc). In Fig. 12 we show a false color image of onaside. Theredcircleshowthepositionofthedetectedobject, the field around QSO SDSS J2301+0112 with the LBG anditssizecorrespondtotheregioninwhichthephotometrywas done(2(cid:48)(cid:48)indiameter). Themagnitudesareindicatedineachpanel. candidate positions indicated. 3.4. Redshift Selection Function nosityfunction,computemagnitudesandcolors,andadd WeusedtheLBGcolormodelingmachinerydescribed photometric errors. We then compute the completeness in § 3.1 to compute the redshift selection function φ (z) at each redshift by calculating the fraction of simulated z ofourLBGcolor-selectioncriteria. Ateachredshiftstep, LBGs that satisfy the selection criteria defined in § 3.3, we redshift the 1000 rest-frame simulated LBG spectra namely: fulfillthecolorcriteriaineqn.(2),andfulfillthe intotheobservedframe,drawluminositiesfromthelumi- magnitude constraints (given by the 4σ limiting magni-