SETIBURST: A Robotic, Commensal, Realtime Multi-Science Backend for the Arecibo Telescope

0.99 MB·

by  Jayanth Chennamangalam

Draft version January 18, 2017
PreprinttypesetusingLATEXstyleAASTeX6v.1.0
SETIBURST:
A ROBOTIC, COMMENSAL, REALTIME MULTI-SCIENCE BACKEND
FOR THE ARECIBO TELESCOPE
Jayanth Chennamangalam
Astrophysics,UniversityofOxford,DenysWilkinsonBuilding,KebleRoad,OxfordOX13RH,UK
7 David MacMahon
1
DepartmentofAstronomy,UniversityofCaliforniaBerkeley,Berkeley,CA94720,USA
0
2

n
Jeff Cobb
a
J DepartmentofAstronomy,UniversityofCaliforniaBerkeley,Berkeley,CA94720,USA

7
1
Aris Karastergiou1,2
]
M Astrophysics,UniversityofOxford,DenysWilkinsonBuilding,KebleRoad,OxfordOX13RH,UK
I
.
h
Andrew P. V. Siemion3,4
p
- DepartmentofAstronomy,UniversityofCaliforniaBerkeley,Berkeley,CA94720,USA
o
r
t
s
a Kaustubh Rajwade
[
DepartmentofPhysicsandAstronomy,WestVirginiaUniversity,POBox6315,Morgantown,WV26506,USA

1
v
8
Wes Armour
3
5 Oxforde-ResearchCentre,UniversityofOxford,KebleRoad,OxfordOX13QG,UK
4
0
.
1 Vishal Gajjar
0 DepartmentofAstronomy,UniversityofCaliforniaBerkeley,Berkeley,CA94720,USA
7
1
:
v Duncan R. Lorimer5
i
X DepartmentofPhysicsandAstronomy,WestVirginiaUniversity,POBox6315,Morgantown,WV26506,USA
r
a
Maura A. McLaughlin
DepartmentofPhysicsandAstronomy,WestVirginiaUniversity,POBox6315,Morgantown,WV26506,USA
Dan Werthimer
DepartmentofAstronomy,UniversityofCaliforniaBerkeley,Berkeley,CA94720,USA
Christopher Williams
Astrophysics,UniversityofOxford,DenysWilkinsonBuilding,KebleRoad,OxfordOX13RH,UK
2 Chennamangalam et al.
1DepartmentofPhysicsandElectronics,RhodesUniversity,POBox94,Grahamstown6140,SouthAfrica
2PhysicsDepartment,UniversityoftheWesternCape,CapeTown7535,SouthAfrica
3ASTRON,POBox2,7990AADwingeloo,TheNetherlands
4DepartmentofAstrophysics,RadboudUniversity,POBox9010,6500GLNijmegen,TheNetherlands
5NationalRadioAstronomyObservatory,POBox2,GreenBank,WV24944,USA
ABSTRACT
Radioastronomyhastraditionallydependedonobservatoriesallocatingtimetoobserversforexclusive
use of their telescopes. The disadvantage of this scheme is that the data thus collected is rarely used
for other astronomy applications, and in many cases, is unsuitable. For example, properly calibrated
pulsarsearchdatacan,withsomereduction,beusedforspectrallinesurveys. Abackendthatsupports
plugginginmultipleapplicationstoatelescopetoperformcommensaldataanalysiswillvastlyincrease
thesciencethroughputofthefacility. Inthispaper, wepresent‘SETIBURST’,arobotic,commensal,
realtimemulti-sciencebackendforthe305-mAreciboTelescope. Thesystemusesthe1.4GHz,seven-
beam Arecibo L-band Feed Array (ALFA) receiver whenever it is operated. SETIBURST currently
supports two applications: SERENDIP VI, a SETI spectrometer that is conducting a search for signs
of technological life, and ALFABURST, a fast transient search system that is conducting a survey of
fast radio bursts (FRBs). Based on the FRB event rate and the expected usage of ALFA, we expect
0–5 FRB detections over the coming year. SETIBURST also provides the option of plugging in more
applications. We outline the motivation for our instrumentation scheme and the scientific motivation
of the two surveys, along with their descriptions and related discussions.
Keywords: instrumentation: miscellaneous — extraterrestrial intelligence — pulsars: general
1. INTRODUCTION have access to data, vastly increasing the available sky
coverage.
Radioastronomyreliesonobservationsforwhichtele-
Commensal observing was pioneered by the early
scope time was obtained following a competitive pro-
searches for extraterrestrial intelligence (SETI) at the
posal review. This process is critical because telescope
HatCreekRadioObservatory(Bowyeretal.1983),and
timeislimited: Onlyonekindofobservationcanusually
later, at the Arecibo Observatory, home to the 305-m
be done at a given time. In addition to this exclusivity,
diameter Arecibo telescope. The original need for com-
the utility of the collected data is usually restricted to
mensalobservingwasduetothefactthatSETIrequires
the specific kind of experiment that it was obtained for.
searching a large parameter space for which a signifi-
Data reuse within a given field is standard practice –
cant amount of telescope time is required, and the in-
e.g., the original fast radio burst (FRB; see §1.2) was
ability of allocating dedicated time to such a large sur-
discovered in a reprocessing of data from a fast radio
vey that is speculative in nature. The Search for Radio
transient survey of the Magellanic Clouds using the 64-
Emissions from Nearby Developed Intelligent Popula-
m Parkes Radio Telescope (Lorimer et al. 2007) – but
tions (SERENDIP) project at the Arecibo Observatory
cross-disciplinary data reuse is a rarity. For example,
– of which the instrument described in this paper is
spectral line surveys, due to long integration times used
a part – has, throughout its existence, relied on com-
in its observations, result in data products that cannot
mensal data processing (see, for example, Bowyer et al.
be reused in a search for pulsars. On the other hand,
1993). Technologically, in recent times, relatively in-
properly calibrated pulsar search data can be used for
expensive networking hardware and high-performance
spectral line surveys as well, but it is rarely done. This
computing (HPC) machines have made it possible to
severely restricts the science throughput of a facility.
build multiple HPC-based backends that can easily dis-
Tooptimizedatacollectionandanalysis,commensalob-
tribute and process radio telescope data, enabling com-
servingisincreasinglybeingemployed,whereinmultiple
mensaldataprocessing. TheAllenTelescopeArraywas
data processing/recording processes run simultaneously
built with commensal observing as a design goal, such
on data from the telescope during observations. In such
that SETI and non-SETI observations could be done
a scheme, telescope pointing remains under the control
in parallel (Welch et al. 2009). In high time resolution
of the primary observer, but secondary observers also
astronomy, the need for commensal observing has been
madeapparentbythediscoveryofnewclassesoffastra-
diotransient,suchasrotatingradiotransients(RRATs;
[email protected]
McLaughlinetal.2006)andFRBs(Lorimeretal.2007;
SETIBURST 3
Thornton et al. 2013). The V-FASTR experiment at known, but setting up such beacons is the best possi-
the Very Large Baseline Array (Wayth et al. 2012) is ble way to advertise our presence in the Universe. Ra-
a commensal search for fast transients. VLITE1 is a dio emission is energetically, and hence, economically
ten-antenna system at the Very Large Array that per- inexpensive to generate. Radio waves can travel vast
forms ionospheric observations, transient searches, and distances with relatively less attenuation due to the in-
imaging in parallel with regular observations. Among terstellar medium (ISM) compared to electromagnetic
new facilties, the Australian Square Kilometre Array radiation at other wavelengths, ensuring a better like-
Pathfinder (ASKAP) is used for the Commensal Real- lihood of signal reception. Cleverly-designed beacons,
time ASKAP Fast Transients survey (Macquart et al. such as extremely narrow band signals near a natural
2010). In this paper, we describe a new instrument emission line commonly used in studying the Galaxy,
at the Arecibo Observatory that is centered around the such as that proposed by Cocconi & Morrison (1959),
idea of commensal observing, with a SETI experiment will increase the likelihood of the signal being noticed
andafasttransientsurveyasconsumersofthecollected by radio astronomers elsewhere. The rationale behind
data. searching for extremely narrow-band signals is that the
Theoutlineofthispaperisasfollows: Inthefollowing narrowest astrophysical lines are of the order of hun-
subsections,weintroducethetwosciencemotivationsof dreds of Hz wide. The narrowest line detected has a
the project, namely, SETI and fast radio transients. In width of 550 Hz (Cohen et al. 1987). Even if a civiliza-
§2, we describe the technical details of the system: the tiondoesnotsetupabeacon,radioemissioncreatedby
SERENDIP VI SETI backend and the ALFABURST their technology could leak out into space, at least dur-
fast transient backend. In §3, we describe the two com- ing the early stages of their technological development,
mensal surveys we are undertaking, and conclude in §4. in much the same way as coherent radio emission pro-
duced quite commonly by human technology routinely
leaksouttospace. Giventhathumanshavebeentrans-
1.1. The Search for Extraterrestrial Intelligence
mittingintheradioformorethanacentury,theearliest
The quest for life in the Universe has seen much
radio transmissions have traversed a distance > 30 pc
progress in recent years, with the exploration of the
awayfromus. Itisconceivablethatleakagesignalsfrom
Solar System and the detection of a large number of
othercivilizationsmaybepickeduponEarth, although
extrasolar planets. A whole new field – astrobiology –
giventhefactthattheyarenotintentionaltransmissions
has emerged to tackle the problem of whether life exists
to us, it is unlikely that they would be easy to detect
elsewhereintheGalaxy. SETIaimsonestepfurther,to
amid the noise background.
answer the more challenging question of the existence
of technological intelligent life. One of the first SETI 1.2. Fast Radio Bursts
attempts followed the suggestion by Cocconi & Mor-
Fast radio transients have been a staple of radio as-
rison (1959) that ETI may transmit narrow-band bea-
tronomy research for decades, starting with the discov-
consneartheradioemissionlineofneutralhydrogen,at
ery of the first pulsars (Hewish et al. 1968). In re-
1420 MHz, enabling radio astronomers in other civiliza-
cent years, new classes of such transient have emerged,
tions to detect them. Radio SETI observations started
namely, RRATs, which are thought to be highly inter-
with Drake (1961) who searched for narrow-band lines,
mittent pulsars (McLaughlin et al. 2006), and FRBs
and have continued to the present day with increasing
(Lorimeretal.2007;Thorntonetal.2013). However,in
levels of sophistication. For instance, Siemion et al.
spite of the long history and recent discoveries, robotic
(2013)recentlysearchedforinterplanetaryradarsignals
surveys in the radio are only beginning to be employed.
in multi-planet systems during conjunctions2, and es-
For example, the Survey for Pulsars and Extragalac-
tablishedthat(cid:46)1%oftransitingexoplanetsystemshost tic Radio Bursts3 (SUPERB) at the Parkes telescope
civilizations that emit narrow-band radiation in the 1–
usesthe‘Heimdall’realtimedataprocessingpipeline. A
2 GHz band with an equivalent isotropically radiated
robotic system in the radio, that operates as long as
power (EIRP) of ∼1.5×1013 W.
a supported receiver is available, can radically increase
Whether ETI would set up beacons for the benefit of
the time available on the sky and lead to discoveries of
curious radio astronomers in other civilizations is un-
fast transients such as RRATs and FRBs. Realtime de-
tection has the advantage of being able to trigger other
telescopes that are geographically separated and oper-
1 http://vlite.nrao.edu/
ate at other frequencies, as exemplified by Keane et al.
2 NotetheerrorinthesensitivitycalculationinSiemionetal.
(2013), and the values reported in their Table 3: Their charac-
teristic sensitivity should be ∼2×10−22 erg s−1 cm−2, and the
exponentinfootnote(d)intheirTable3shouldbe−22. 3 https://sites.google.com/site/publicsuperb/
4 Chennamangalam et al.
(2016), whereas offline analysis of data usually results sui et al. 2015). Since FRBs are non-repeating/highly
in a latency of days to months. The ability to follow up intermittent, increasing the amount of observing time
detected events within hours or days is critical to the availablewillallowmoretobedetected, enablingabet-
identification of potential afterglows or other indicators ter understanding of these events.
of the same event at different wavelengths, helping shed
1.3. SETIBURST
light on the location of, and the physics behind these
exotic sources. Keeping in mind the aforementioned scientific mo-
FRBs are broadband radio pulses with observed tivations, we have designed, developed, and de-
widthsoftheorderofafewmilliseconds. Duetothena- ployed SETIBURST, an automated, commensal, re-
ture of the dispersion caused by the ionized ISM, lower altime multi-science backend for the Arecibo tele-
frequency components of the pulse are delayed much cope. SETIBURST has two plug-in applications:
morethanthehigherfrequencycomponents. Thisdelay SERENDIP VI (referred to as ‘S6’ henceforth), the
is quantified in terms of the dispersion measure (DM), latest in the SERENDIP series of SETI spectrome-
which, for FRBs, is greater than that contributed by ters,andALFABURST,afasttransientsearchpipeline.
the ISM of the Galaxy, indicating an extragalactic ori- SETIBURST performs commensal processing of signals
gin. The millisecond time scale of the pulse implies a from the 1.4 GHz seven-beam Arecibo L-band Feed Ar-
compact object progenitor. Due to the fact that all re- ray6(ALFA)receiver. ALFAisArecibo’sworkhorsesur-
ported FRBs have been detected using single-dish ra- vey receiver, being used for such large-scale surveys as
dio telescopes that have wide beams, localization, and the Pulsar ALFA (PALFA) survey (Cordes et al. 2006),
hence, association with sources at other wavelengths, which has so far resulted in the discovery of 145 pul-
has proven to be a challenge. Therefore, a conclusive sars and one FRB (Cordes et al. 2006; Lazarus et al.
explanation of what FRBs are has remained elusive, 2015;Spitleretal.2014);theGalacticALFAContinuum
with various theories being proposed. Extragalactic- TransitSurvey(GALFACTS;see,forexample,Taylor&
origintheoriesthatpositcataclysmicexplosionsinclude Salter2010);andtheAreciboGalaxyEnvironmentSur-
the gravitational collapse of supramassive neutron stars vey(AGES;see,forexample,Auldetal.2006). ALFAis
(Falcke&Rezzolla2014)andbinaryneutronstarmerg- well-suited for an FRB survey at Arecibo given that all
ers (Totani 2013). Extragalactic-origin theories that except the Green Bank FRB were detected at 1.4 GHz.
predict repetition include giant-pulse-emitting pulsars Being a survey receiver with multiple beams, ALFA is
(see, for example, Lyutikov et al. 2016), flaring magne- suited for SETI surveys as well.
tars (Popov & Postnov 2013, for instance), and Alfv´en
2. SYSTEM DESCRIPTION
wings around planetary companions to pulsars (Mottez
&Zarka2014). ThereisatleastoneGalactic-originthe- SETIBURST is a heterogeneous instrument, i.e.,
ory (that also predicts pulse repetition) wherein these it uses Field Programmable Gate Arrays (FPGAs)
bursts originate in flare stars, and are dispersed by the and HPC machines equipped with Graphics Process-
stellar corona (Loeb et al. 2014). A resolution of the ing Units (GPUs). Heterogeneous instruments have in-
mystery has been made complicated by two recent dis- creased in popularity in astronomical signal processing
coveries, namely, that of FRB 150418 which has been applications in recent years (see, for example, DuPlain
claimed to be associated with a slow radio transient in etal.2008;Woods2010),astheycombinethehighband-
an elliptical galaxy interpreted to be an afterglow of a width capabilities of FPGAs with the features, flexibil-
cataclysmic, non-repeating event (Keane et al. 2016), ity, and ease of programming of GPUs. The network
and that of FRB 121102 which has been shown to re- switch has the potential to act as a datahubinto which
peat (Spitler et al. 2014, 2016)4. HPC backends may be plugged in, enabling multiple si-
Eighteen FRBs have been reported in published liter- multaneous experiments.
ature5 (Petroff et al. 2016). All reported FRBs except Figure 1 shows the high-level architecture of the sys-
two were discovered using the Parkes telescope. The tem. The digital system processes signals from the
exceptions were discovered using the Arecibo telescope ALFAreceiver. ALFAisaseven-beamsystemthatoper-
(Spitleretal.2014)andtheGreenBankTelescope(Ma- atesinthe1225–1525MHzrange,withthesevenbeams
arranged in a hexagonal pattern. Each beam is approx-
imately 3.5(cid:48) wide. The receiver has a cold sky system
4Indevelopmentspublishedduringthelatestagesofthereview temperature of ∼30 K. The central beam has a gain of
processofthismanuscript,therepeatingsourceFRB121102has ∼11KJy−1,withtheperipheralbeamshavingaslightly
been shown to be associated with an extragalactic source (Chat-
terjeeetal.2017).
5 http://www.astronomy.swin.edu.au/pulsar/frbcat/
6 http://www.naic.edu/alfa/
SETIBURST 5
lowergain. SomeofthesidelobesoftheALFAbeamsare and to reduce the output data rate, we pare the band
sensitiveaswell,withthepeakofthesesidelobeshaving down to 2560 channels (corresponding to a bandwidth
alossofonly−8.5dB,i.e.,thegainatthesidelobepeak of 280 MHz) and these are packetized. The complex
is only (1/7)th of that at boresight of the central beam. samples at the output of the PFB are packetized into
Theadvantageofthisisthatithastheeffectofincreas- 1296-byte-long User Datagram Protocol (UDP) pack-
ing the area on the sky, and the disadvantages include ets. Fig. 2 shows the S6 packet format. Each packet
increased uncertainty in localization and poor fidelity contains an 8-byte header that contains a 48-bit spec-
inthemeasurementofthespectralindicesofFRBsthat trum counter, a 12-bit field indicating the first channel
mayhavebeenpickedupinthesesidelobes(Spitleretal. inthepacket(denotedbyP inFig.2),anda4-bitbeam
2014). S6uses280MHzofALFA’s300MHzbandwidth, identifierthattakesonvaluesintherange0–6(denoted
while ALFABURST, in its current version, supports a by B in Fig. 2). The spectrum count is used at the re-
bandwidth of 56 MHz. ceiver (HPC) for packet loss checking. Each packet also
consistsofa64-bitfooterthatcontainsa32-bitcyclicre-
2.1. FPGA Gateware
dundancy check (CRC) for error detection on the HPC.
The 14 intermediate frequency (IF) signals from The bytes that make up these packets are transmitted
ALFA (one per polarization per beam) are split be- in network byte order.
fore being distributed to various backends at the ob- For ALFABURST, polyphase channelization is fol-
servatory. SETIBURST hardware taps into these split lowed by computation of pseudo-Stokes values XX∗,
signals, and digitizes them. The hardware consists YY∗,Re(XY∗)andIm(XY∗)(each16bitswide),where
of two ROACH27 FPGA boards, each equipped with X and Y are the Fourier representations of the two po-
two 1 Gsps ADC16x250-88 Analog-to-Digital Convert- larizations,andX∗ andY∗ aretheirrespectivecomplex
ers (ADCs). The first board (denoted by ‘ROACH2 conjugates. Note that full-Stokes values can be com-
A’ in Figure 1) processes beams 0 through 3, while puted from these values. The spectra are then time-
the second ROACH2 board (‘ROACH2 B’) processes integrated by a factor of 14, with a resulting time reso-
beams 4 through 6. The ADCs first sample the data lution of 128 µs. The spectra are then packetized into
at 896 MHz and digitizes it to 8 bits. The FPGA UDPpackets,andtransmittedover10GbE.Toconform
gateware uses a polyphase filter bank (PFB) to chan- to the Ethernet ‘jumbo frame’ standard, each packet
nelize the data to 4096 channels, with a resulting time needs to be not more than 9000 bytes long. Therefore,
resolution of 9.143 µs. The PFB is implemented us- for each beam, the 4096-channel spectrum is split into
ing the standard CASPER9 blocks pfb fir real10 and foursub-bands,eachcontaining1024channels,withone
fft wideband real11. The prefiltering uses 4taps, and sub-band per packet. Fig. 3 shows the ALFABURST
theco-efficientsaretheproductofasincfunctionanda packet format. In addition to the data, the UDP pay-
Hamming window. load also contains a 64-bit header that includes a 48-bit
The bandpass is split into eight sub-bands, which integration count, a one-byte sub-band identifier that
are packetized separately, each addressed to a differ- takesonvaluesintherange0–3(denotedbySinFig.3),
ent HPC pipeline, and transmitted over 10-Gigabit and a one-byte beam identifier that takes on values in
Ethernet (10GbE). The packetization mechanism as- therange0–6(denotedbyB inFig.3). Theintegration
signs a different IP address to packets sent to different count, along with the sub-band identifier allows us to
HPC nodes, using IP addresses stored in software regis- check for missing packets on the receiving (HPC) side.
ters on the ROACH2 boards, that are programmable The integration count, along with a timestamp of when
at run-time. Each HPC node runs two software in- that count was reset to zero – which is read from else-
stances/pipelines for all beams and polarizations (see wherebytheHPCsoftware–allowsustogetthetimes-
§2.2). Even though our digitization scheme results in tamp of each packet. Each packet also consists of a
a bandwidth of 448 MHz, ALFA has a bandwidth of footer similar to the one in S6. As in the case of S6,
only300MHz. Toremovechannelswithnoinformation bytes are transmitted in network byte order.
2.2. SERENDIP VI Software
7 ‘Reconfigurable Open Architecture Computing Hardware 2’; The UDP packets that are transmitted by the FPGA
http://casper.berkeley.edu/wiki/ROACH2
are forwarded to appropriate nodes in the HPC clus-
8 http://casper.berkeley.edu/wiki/ADC16x250-8 ter by a Juniper Networks EX4500-LB 10GbE switch.
9 CollaborationforAstronomySignalProcessingandElectron- The S6 HPC cluster consists of five server-class com-
icsResearch: https://casper.berkeley.edu
puters – one ‘head node’, and four ‘compute nodes’, as
10 https://casper.berkeley.edu/wiki/Pfb fir real showninFigure1. Eachcomputenodeisequippedwith
11 https://casper.berkeley.edu/wiki/Fft wideband real two Mellanox MCX312A-XCBT 10GbE network inter-
6 Chennamangalam et al.
Telescope status
Redis DB
Head Node

AONET Switch
s
e
d
No
BB22--XY ADCs B0-6 ute
BB11--YX B0-6 Comp
B3-X B0-6
B3-Y
2
B0-X B0-6
1 3 B0-Y ROACH2 A S6 Cluster
0
6 4

5 BB44--XY 10GbE Switch
BB66--YX B0,B1 Head Node
ALFA Receiver B2,B3
B5-X
B5-Y B4,B5
Cs B6
AD des
No
e
ut
mp
ROACH2 B Co
AB Cluster
Figure 1. Simplifiedhigh-levelarchitectureoftheSETIBURSTsystem. Bothpolarizations(denotedbyX andY)ofallALFA
beams(denotedbyB0−B6)areprocessedbytwoROACH2FPGAboardsanddistributedtocomputenodesthrougha10GbE
swith. ‘AB’ stands for ALFABURST in this diagram. The setup is described in detail in §2.
Figure 2. TheSERENDIPVIpacketformat,asdescribedin§2.1. EachUDPpayloadis1296byteslong,withan8-byteheader
and an 8-byte footer. The 12-bit field denoted by P indicates the first channel of this packet, and the 4-bit field denoted by B
contains the beam identifier. Bytes are transmitted in left-to-right, top-to-bottom order, i.e., in network byte order.
face cards (NICs) that receive data from the 10GbE and processing. HASHPIPE is a multi-threaded data
switch. Each compute node is a dual-socket, dual-GPU transport framework that moves high-bandwidth input
machine equipped with RAID data disks. We use com- data from the 10GbE NICs through a series of shared
mercial gaming GPUs, namely NVIDIA GeForce GTX memory ring buffers and signal-processing threads, all
780 Ti cards. the way to writing the output to disk. HASHPIPE
S6usestheHASHPIPE12softwarefordataacquisition is designed with a modular architecture so that user-
12 High Availability Shared Pipeline Engine; Available upon request.
SETIBURST 7
Figure 3. TheALFABURSTpacketformat,asdescribedin§2.1. EachUDPpayloadis8208byteslong,withan8-byteheader
and an 8-byte footer. The integration count and the sub-band identifier, denoted by S, together make it possible to check for
missing packets. B is the beam identifier. Bytes are transmitted in left-to-right, top-to-bottom order, i.e., in network byte
order.
supplied modules may be plugged in to perform certain events, where n is the number of channels in our range
ν
tasks. The first module that interfaces with the NIC of interest, n is the number of time samples, n is the
t b
is the ‘network thread’. The network thread reads data number of beams, and n is the number of polariza-
p
fromtheNIC,checksformissingpackets,andwritesthe tions, foraS/N of20, weexpecttodetect∼0.01events
data to the first shared memory ring buffer. The next in each block used in the coincidence rejection compar-
thread,the‘GPUthread’,readsthatdataandperforms ison. In reality, however, the statistics are dominated
fine channelization. Each of the 4096 channels that ar- by RFI, and the actual number of detected events is
rive from the FPGA are channelized into 131072 chan- much larger. Figure 4 shows the performance of this
nels by the GPU, with a resulting frequency resolution RFI rejection technique, applied to one polarization in
of ∼0.8 Hz and time resolution of 1.198 s. This data one pipeline instance. We note that these plots reflect
is then written to the next shared memory ring buffer a work in progress, and additional techniques necessary
where it is read by the next thread, the ‘CPU thread’. to identify bona fide candidates are discussed elsewhere
The CPU thread performs thresholding as follows: To (Gajjar et al., in preparation).
estimate the mean power level of spectra, this thread Inadditiontothesignalprocessingsoftwarethatruns
boxcar averages each spectrum with a window of length on the compute nodes, the S6 HPC system maintains
1024channels, andcomputesthemeanofthesmoothed a Redis key-value store on the head node. This is a
spectrum. Channelsthathavevalues20timesthemean database that is constantly updated with the status
are considered events of interest, which we term ‘hits’. of the telescope, read from the Arecibo observatory’s
We note that the power spectra follow a χ2 distribu- network, through a separate network switch (termed
tion with 2 degrees of freedom, such that the mean and ‘AONET switch’ in Figure 1). The information main-
rootmeansquare(RMS)areequal. Therefore,theratio tained in the database includes the receiver in use,
of the detected power in a given channel to the mean IF frequencies, and pointing and timing information,
power level is a measure of S/N in that channel. The among others.
hits are stored as a function of time, frequency, and sky As part of deployment and commissioning, we con-
coordinates in a FITS file on disk. ductedvariousteststoverifythefunctioningofthesys-
S6 alsoutilizes multi-beamcoincidence RFIrejection. tem. The primary end-to-end test involved injecting a
Individual hits from one beam are checked for coinci- testsignalwithbandwidth<0.8HzintotheIF,andre-
dence hits in the other beams. If hits are found within covering that signal at the expected level in the output
25000frequencychannels(correspondingto∼20.9kHz) data.
on either side of the event’s channel, and five samples
(corresponding to 5.99 s) on either side in time, across
2.3. ALFABURST Software
two or more beams, they are flagged as RFI. These fre-
TheALFABURSTHPCclusterisverysimilartothat
quency and time spans, and our S/N of 20, are chosen
ofS6. Itismadeupofoneheadnodeandfourcompute
empirically based on the prevailing average RFI condi-
nodes. Themaindifferencesarethateachcomputenode
tionsonsiteandourneedtoensurethatpotentialastro-
is equipped with a single Mellanox MCX312A-XCBT
physicalsignalsareretainedashits. Assuminganexpo-
10GbENICthatreceivesUDPpacketsfromthe10GbE
nential distribution of hit S/N giving us n n n n e−20
ν t b p
switch, and uses NVIDIA GeForce GTX TITAN GPU
8 Chennamangalam et al.
104 102
Beam 1 Beam 1
Beam 2 Beam 2
Beam 3 Beam 3
Beam 4 Beam 4
103 Beam 5 Beam 5
Beam 6 Beam 6
Beam 7 Beam 7
s s
Hit Hit
er of 102 er of 101
b b
m m
u u
N N
101
100 100
0 100 200 300 400 500 600 700 800 20 25 30 35 40
Relative Power (DETPOW/MEANPOW) Relative Power (DETPOW/MEANPOW)
Figure 4. Histograms of the ratio of detected power (‘DETPOW’) and mean power (‘MEANPOW’) in a coincidence rejection
comparison block (see §2.2) of a typical SERENDIP VI multi-beam data. This particular data comes from one of the system’s
34 MHz sub-bands processed by a single pipeline instance, centred at 1252 MHz, spanning ∼800 seconds. The left panel shows
thepowerdistributionbeforeRFIrejection,andtherightpanelshowsthepowerdistributionaftertheapplicationofmulti-beam
RFI rejection that removes events that are found in two or more beams that are similar in frequency and time.
cards that have a larger memory than those used in S6, thesoftwareservestheKarastergiouetal.(2015)survey
necessitatedbyALFABURSTsignalprocessingrequire- sufficiently,thespecificationsofALFABURSTaremuch
ments. more stringent, with a larger number of channels and a
The ALFABURST head node queries the S6 Redis muchlargerbandwidth. Thesoftwareinitscurrentform
database at a cadence of once per minute, checking isnotdesignedtoprocesstheentireALFAbandwidthat
which receiver is at the focus. When an observer selects the native time resolution provided by the FPGA gate-
the ALFA receiver, the corresponding value is updated ware. Wethereforeprocessonlyabandwidthof56MHz,
intheRedisdatabase. ThisisdetectedbyALFABURST andintegratetheincomingspectrawitharesultingtime
anddataacquisitionisinitiated. Whileobservationisin resolutionof256µs. Wenotethatthenarrowestknown
progress,theheadnodecontinuestoquerythedatabase FRB has a width of 350 µs (Petroff et al. 2016), so the
for changes to telescope state. Data acquisition is ter- choice of time resolution is reasonable in this regard.
minated when ALFA stops being the selected receiver. The first stage in the signal processing pipeline is
The compute nodes run the ALFABURST software the computation of full-Stokes values from the pseudo-
data acquisition pipeline instances. Three of the com- Stokes values in the packets following
pute nodes process data from two ALFA beams each,
I=XX∗+YY∗,
and therefore, run two instances of the software. The
Q=XX∗−YY∗,
remaining node processes the seventh ALFA beam, and
runs a single instance of the pipeline. The software ar- U=2Re(XY∗), and
chitecturefollowsaclient-servermodel,wheretheserver V =−2Im(XY∗).
receives incoming data from the 10GbE NIC, fills data
Thesearchprocessrequiresonlythetotalpower(Stokes
corresponding to missing packets with zeros, and for-
I), but it is worth saving the other Stokes parameters
wards the data to the client. The client is modular by
forpolarizationstudiesofanydetectedFRB.Inourcur-
design,witheachmodulehandlingonelogicalsignalpro-
rent implementation, we do not store the other Stokes
cessing stage. The ALFABURST data transport frame-
work and signal processing system13 are based on the parameters, but this will be supported in future ver-
sions of the software. Stokes computation is followed
ARTEMIS fast transient search software developed for
by the signal processing stages involved in searching for
arecently-concludedsurveyattheUKstationoftheLO-
FRBs. The following discussion briefly describes these
FAR telescope (Karastergiou et al. 2015). Even though
signal processing steps; for details, we refer the reader
to Karastergiou et al. (2015).
13 Thesoftwareisavailableuponrequest. Since searching for FRBs involves correcting for the
unknown dispersion delay introduced by the ISM, the
SETIBURST 9
major signal processing operation involves dedispersing center frequency in MHz, and ∆ν is the bandwidth in
the data over a range of trial DMs. We call this process MHz (Lorimer & Kramer 2005). For a nominal center
the dedispersion transform, converting dynamic spec- frequency of 1375 MHz, our experimental setup yields
tra (frequency versus time versus power) to a set of δDM ≈ 1.4 cm−3 pc. As our fixed step size, we have
dedispersed time series (DM versus time versus power). chosen1cm−3 pc,whichresultsinnolossinsensitivity.
Dedispersion involves summing the data over all fre- The dedispersion transform step is followed by
quencychannels,soitisimportanttoremovethedataof smoothing of the dedispersed time series. Each time
strongRFIthatwouldotherwiseresultinalargenumber series is decimated by factors of 2 to 16, in powers of 2.
of false positives. Accordingly, the data goes through This is a matched filtering operation meant to increase
the RFI clipper module. The RFI clipper implements the sensitivity of the search to pulses of varying widths.
an adaptive thresholding algorithm (Karastergiou et al. We note that the decimation is performed block-wise
2015) that takes into account the non-flat nature of the in the current implementation, as opposed to using a
bandpassandthetime-varyingbaseline,andnormalizes running window. Compared to doing true matched fil-
theoutputtohaveameanof0andastandarddeviation tering, i.e., using a running window, this has the effect
√
of 1. of reducing the net sensitivity by a factor of 2 (Keane
Following RFI removal and spectrum normalizaton, & Petroff 2015). All time series are then subject to a
the data undergoes the dedispersion transform. Being sensitivity threshold of 10 times the noise RMS. We do
the most compute-intensive operation, this is run on notexplicitlycomputetheRMSofthedata. Bydesign,
the GPU, and is implemented in Compute Unified De- the RFI clipper outputs spectra with a standard devia-
vice Architecture (CUDA)14. The dedispersion module tion of 1, so we take the RMS to be the square root of
is based on the Astro-Accelerate code developed by Ar- the number of channels that are summed during dedis-
mour et al. (2012). The dedispersion transform is per- persion. Events that cross the threshold are saved to
formed on data resident in buffers 32768 samples long. a candidates list that is written to disk, along with the
Atthecurrently-usedtimeresolutionof256µs,thiscor- RFI-removed filterbank data corresponding to the data
responds to a duration of ∼8.4 s. Temporal continuity buffer the event was found in. Fig. 5 is a schematic of
acrossbuffersismaintainedbyreusingthelastn the aforementioned operations.
maxshift
time samples from the previous buffer, where n Observations are followed by the generation of diag-
maxshift
is the number of time samples in the lowest frequency nostic plots of threshold-crossing event S/N as a func-
channel to be ‘shifted’, corresponding to the maximum tion of time and DM. In the current scheme, plots are
DM. If downstream processing results in a detection, automatically generated once a day, around noon local
data from the buffer which contains the pulse is saved time. This makes plots available within a few hours of
to disk for later inspection. recordingofthesignal. Webpagescontainingtheseplots
We perform an incoherent dedispersion search over a are automatically generated and made available using a
DM range of [0, 2560] cm−3 pc. The maximum DM webserver15. Beyondthisstage,dataanalysisismanual
among all known FRBs is ∼1629 cm−3 pc (Champion in nature. Plots are inspected visually, and interesting
et al. 2016), so our upper limit is a reasonable choice. events are followed up by examining the saved filter-
Even though the optimal DM step size is non-uniform bank data. For pulses that are seen in the filterbank
across the DM range of interest (Cordes & McLaugh- data, we compare the pointing information and DM to
lin 2003), it is simpler to implement a fixed step size as the entries in the ATNF Pulsar Catalogue16 (Manch-
is done in Armour et al. (2012), thereby oversampling ester et al. 2005) to check whether they correspond to
the DM space at larger values, and, depending upon known pulsars.
the step size, possibly undersampling the DM space at Commissioning tests of the system were conducted
smaller values. Undersampling the DM space implies from March to August 2015. Fig. 6 shows the results
the use of trial DMs that are offset from the true val- of one of our commissioning observations, wherein we
ues, and this leads to pulse broadening, resulting in a observed the pulsar B0611+22 in beam 1. We obtained
loss in sensitivity. The smallest step size in an incoher- detectionswhoseS/N peakedaround97cm−3 pc,asex-
entdedispersionsearchrequiredtominimizethislossin pected for the test pulsar. To verify the functionality of
sensitivity is the system further, we compared the number of events
wedetectedtothatobtainedbyapplyingthesameS/N
δDM=1.205×10−7 t ν3/∆ν cm−3 pc, (1)
samp
where t is the sampling interval in ms, ν is the
samp
15 http://www.naic.edu/~alfafrb/
16 http://www.atnf.csiro.au/people/pulsar/psrcat/
14 http://www.nvidia.com/object/cuda home new.html
10 Chennamangalam et al.
Pseudo-Stokes ple fields spread in right ascension across the northern
Data
sky(Auldetal.2006). Mostfieldsareabout5×4sq.deg.
in size. Each pointing in the survey has a dwell time of
Stokes
300 s. AGES is expected to observe for about 350 hr.
Computation
over the coming year (R. Minchin, private communica-
tion). AGESpointingsaremostlyawayfromtheGalac-
tic plane, and therefore, are conducive to a search for
extragalactic FRBs.
RFI Excision
3.2. SERENDIP VI Sensitivity
For a narrow band signal in a single polarization, the
minimumdetectableflux,inWm−2,ofasignalthathas
width less than the channel bandwidth ∆f Hz is given
Dedispersion by
Transform (cid:114)
∆f
F =σS , (2)
sys t
where σ is the threshold S/N, S is the system equiv-
sys
alent flux density (SEFD) in Jy, and t is the integration
Matched
Filtering time in s.
We use a detection threshold of 20-σ, which has been
determined empirically, based on the RFI environment
at the site. Given that the boresight SEFD of ALFA is
2.73Jy,forourthresholdS/N,foranintegrationtimeof
Thresholding
1.198 s, with a channel bandwidth of ∼0.8 Hz, the min-
imum detectable flux is ∼4.6×10−25 W m−2, or ∼55 Jy
across a channel.
Candidates
As an example of a transmitter, we consider the case
of the 2380 MHz transmitter of the Arecibo Planetary
Figure 5. SignalprocessingstagesoftheALFABURSTsoft- Radar, which is frequently used to determine the orbits
warepipeline. Thededispersiontransformisimplementedon ofnear-Earthasteroids. IthasanEIRPof∼2×1013 W.
GPUs.
Our sensitivity is high enough to detect similar trans-
mitters up to a distance of ∼60 pc. However, this en-
thresholdtoatimeseriesthatwasdedispersedusingthe
ergetics comparison is strictly for illustrative purposes.
SIGPROC17 software, which is a standard pulsar data
The detectability and decoding of terrestrial analogs at
processing package, yielding a match.
interstellar distances is a complex topic (see Sullivan
et al. 1978) and is not addressed here.
3. COMMENSAL SURVEYS
3.1. Sky Coverage 3.3. ALFABURST Sensitivity
TheS6andALFABURSTsurveyspiggybackonongo- Following the radiometer equation (see, for example,
ing Arecibo surveys, specifically the PALFA and AGES Lorimer & Kramer 2005), the threshold flux density of
surveys. PALFA is a survey for pulsars and fast tran- a single pulse search,
sients (Cordes et al. 2006), and has so far resulted in σS
S = sys , (3)
thediscoveryof145pulsarsandoneFRB(Lazarusetal. min (cid:112)
n ∆fW
p
2015;Spitleretal.2014). Beingapulsarsurvey,PALFA
where σ is the threshold S/N, S is the SEFD, n is
emphasizes coverage of the Galactic plane. PALFA sys p
pointings are towards the inner Galaxy (32◦ (cid:46) l (cid:46) 77◦; the number of polarizations, ∆f is the bandwidth in
|b| < 5◦) and the outer Galaxy (168◦ (cid:46) l (cid:46) 214◦; MHz, and W is the pulse width in ms. In the absence
of RFI, the choice of S/N threshold for a single pulse
|b| < 5◦), with dwell times of 268 s and 180 s, respec-
search is rather straightforward. If we assume Gaussian
tively. PALFAhasbeenallocated230hr. overthecom-
statistics, thenumberofeventscrossingthethresholdσ
ing year.
is
AGES is an extragalactic HI survey, observing multi-
N(>σ)≈2n θn , (4)
samp DM
where n is the number of time samples, n is the
samp DM
17 http://sigproc.sourceforge.net/ number of DM channels, and θ is the probability of