Table Of Content

Draftversion January17,2012 PreprinttypesetusingLATEXstyleemulateapjv.5/2/11 MULTIMOMENT RADIO TRANSIENT DETECTION L. G. Spitler, J. M. Cordes, S. Chatterjee AstronomyDepartmentandNAIC,CornellUniversity,Ithaca, NY,14853 and J. Stone BarnardCollege,NewYork,NY,10027 Draft version January 17, 2012 2 1 ABSTRACT 0 We present a multimoment technique for signal classification and apply it to the detection of fast 2 radiotransientsinincoherentlydedisperseddata. Specifically,wedefineaspectralmodulationindexin n termsofthefractionalvariationinintensityacrossaspectrum. Asignalwhoseintensityisdistributed a evenlyacrosstheentirebandhasalowermodulationindexthanaspectrumwhoseintensityislocalized J in a single channel. We are interested in broadband pulses and use the modulation index to excise 3 narrowbandradiofrequencyinterference(RFI)byapplyingamodulationindexthresholdabovewhich 1 candidate events are removed. The technique is tested both with simulations and using data from knownsourcesofradiopulses(RRAT J1928+15andgiantpulses fromtheCrabpulsar). Themethod ] is generalized to coherent dedispersion, image cubes, and astrophysical narrowband signals that are M steady in time. We suggest that the modulation index, along with other statistics using higher-order moments, should be incorporated into signal detection pipelines to characterize and classify signals. I . h p 1. INTRODUCTION We define fast radio bursts as having character- - o Surveys have always played an important role in as- istic widths less than about one second, so as- r tronomy and will play an increasingly important role in trophysical plasma delays, such as those encoun- t tered in pulsar signals and in the class of tran- s the future as observatories such as the Large Synoptic a sients known as rotating radio transients (RRATS, Sky Telescope (LSST) and the Square Kilometer Array [ McLaughlin et al. 2006), are important. When applied (SKA) come online. The huge volumes of data gener- to fast transients, our technique builds upon the meth- 2 ated by surveys require robust pipelines to identify and ods presented in Cordes and McLaughlin (2003) and v characterizepopulationswithmaximalcompletenessand McLaughlin and Cordes (2003) and that led to the dis- 7 minimal false positives. Regardless of the target source covery of RRATs. However, the basic idea applies to 7 class, all detection pipelines rely on signal-to-noise ra- 6 tio (SNR) to find sources and quantify their believabil- transients of any duration. 6 ity. This approach underutilizes the spectral informa- Actual signals will blur the distinction between RFI 9. tioncontainedin the data,because it onlyuses the total and signals of interest because some RFI will be broad- bandandsomebroadbandastrophysicalsignalswillshow 0 intensity, or first moment, of a spectrum. We propose significant frequency variation. Astrophysical sources 1 addingasecondstatistic,thespectralmodulationindex, mayhaveaspectraldependenceincludingasimplespec- 1 that uses both the first moment (signal mean) and sec- tral index or stronger modulations like those seen in so- : ond moment (signal variance) of a spectrum to classify v lar bursts. Compact sources of fast transients will typi- signals found through their SNR. i callyshowfrequencymodulationfrominterstellarscintil- X Althoughthe technique we presentinthis paper is ap- lation. We consider these effects in our implementation plicabletoanydatacollectedasintensityversustimeand r of the method. a frequency, we focus on surveys for fast radio bursts and The modulation index is not a new statistic; it has use the modulation index to identify and remove nar- been used to measure time variations in a variety of rowband radio frequency interference (RFI). Surveys at astronomical applications, such as variability of pul- radio frequencies must contend with RFI because, if not sars (Weisberg et al. 1986) and extragalactic sources mitigatedorexcised,itcanproduce manyfalsepositives (Kedziora-Chudczer et al. 2001) caused by interstellar in processing pipelines. One difficulty in removing RFI scintillations. It has also been used to study properties is that it arises from a wide variety of terrestrialsources of the solar wind (Spangler and Spitler 2004). with different signal characteristics. Moreover it can be Welayoutthemathematicalgroundworkformultimo- episodic or simply transientin nature along with the as- mentdedispersionandthe calculationofthe modulation trophysical signals that we are interested in. RFI can indexin 2. In 3wediscussthe implementationofthe be broad in time and narrow in frequency (e.g. Global § § modulation index in a fast transient detection pipeline Positioning System satellites) or broad in frequency and and present the results of a simulated single pulse de- narrowintime(e.g.lightning). Someradarsignalssweep tection pipeline. Also in 3 we apply the technique to in frequency and mimic the plasma dispersion of astro- § two known sources of single pulses: Crab giant pulses physical bursts. and RRAT J1928+15. The method is extended to other types of data sets in 4. We discuss how the spectral [email protected] § 2 modulation index can be used to classify bursts from a in time according to the ν−2 dispersion relation and av- varietyofastrophysicalsourcesin 5andmakeconclud- eraging to increase the signal-to-noise ratio (SNR). § ing remarks in 6. The standard incoherent or post-detection approach § for calculating a dedispersed time series is 2. METHOD 1 For specificity we consider broadband astrophysical I(t,DM)= I(t+t (ν),ν) (3) DM signals that are sampled as dynamic spectra; that is, Nν Xν a sequence of spectra separated in time by ∆t with s where I(t,ν) is the time-frequency intensity data, N is N frequency channels spanning a total bandwidth B. ν ν Most of the cases we discuss in this paper will have thenumberoffrequencychannels,andtDM(ν)isthede- layatfrequency ν fordispersionmeasureDM.Through- time-bandwidth products well in excess of unity, i.e., outwewilldenoteanaverageinfrequencywithabarover ∆t B/N 1,asisconsistentwithfast-dumpspectrom- s ν eters used≫in surveys for pulsars and radio transients. the variable. Survey data must be dedispersed using a setoftrialDMsbecausetheDMofthetargetsourcesare Ourdiscussionwillalsoconcentrateonincoherent(post- not known a priori, except in special applications, such detection) dedispersion, although we briefly discuss applications where coherent dedispersion is used. To illus- as searches of globular clusters with previously known pulsars. A list of candidate pulses is defined by apply- trate the basic method, we consider only a simple sum ing a minimum SNR threshold (SNR ) to the set of over frequency to yield an intensity time series; later we min willconsiderinterstellardispersiondelaysinthesumand dedispersed time series. Equation 3 implicitly weights all frequency channels usageofthedispersioneffectindiscriminatingastrophys- equally. In general though there will be effects, both ical signals from RFI. We define the modulation index m as the normalized astrophysical and instrumental, for which optimal de- I tection (i.e., maximum SNR) requires unequal weight- standard deviation of the intensity I across frequency ing. For example broadband astrophysical sources have (ν), spectra with slopes characterized by a spectral index α, I2 I2 and maximum SNR occurs when the frequency chan- m2 = − , (1) nels are given some weighting w that reflects α. Sim- I 2 ν I ilarly removing instrumental effects, like bandpass subtraction, may introduce channel-dependent root-mean- where the first and second moments, I and I2, respec- square noise. The generalized expressionfor a weighted, tively, are given by dedispersed, first moment time series is In =N−1 In(ν). (2) ν w (ν/ν )αI(t+t (ν),ν) Xν ν 0 DM The modulation index, m , characterizes the distribu- I(t,DM;α)= Xν . (4) I w tion of signal power in spectrum. If the power is dis- ν tributed evenly across the band, the variance is small, Xν andthe spectrum has alow modulationindex. A broad- AfulldiscussionoffrequencyweightsispresentedinSec- band pulse with a flat spectrum and infinite signal-to- tion 2.2. noise ratio (SNR) has mI = 0. In the opposite extreme Optimizationindetectioncanalsobe made byconsid- where the power is localized to single spectral channel, eringthedurationofthepulserelativetothetimeresolu- the modulation index is mI √Nν 1 with increas- tionofthedata. Theeffectivetime resolutioncanbede- → − ing SNR. These simple examples illustrate how broad- creased by smoothing the data, and the maximum SNR band astrophysicaltransients can be discriminated from occurswhentheeffectivetimeresolutionofthesmoothed RFI, which often consists of narrow spikes in frequency data matches that of the pulse (Cordes and McLaughlin accompanied by time variability, by requiring that the 2003). The simplest methodofsmoothing addsadjacent modulation index be less than some ceiling, mI,max. time samples until all of the signal’s power is in a single sample. In general any smoothing technique can be 2.1. Multimoment Dedispersion defined by applying smoothing weights w to the time- t The modulation index improves upon current detec- frequency data and averaging in time; tion schemes by including more information about the ′ signal through the calculation of both the first and sec- wtt′I(t,ν) ond moments. The signal processing algorithms used I (t,ν)= Xt′ , (5) in the detection of short-duration radio transients (i.e., s wtt′ dedispersionandsmoothing)mustthereforebeexpanded Xt′ to higherordermoments. Althoughweonly usethe first and second moments in this paper, one could consider where I (t,ν) is the smoothed time-frequency data and s ′ statistics that use higher order moments (e.g., skewness I(t,ν) is the original time-frequency data. A dedis- andkurtosis),so we introducegeneral, nth-orderexpres- persed, smoothed time series I (t,DM) is calculated ac- s sions. cording to Equation 3 substituting I for I. s Radiopulsestravelingthroughtheinterstellarmedium The calculation of the modulation index requires the aresubjecttofrequency-dependentdispersion. Standard secondmomentoftheintensitytimeseries. Wemakeone pulsarprocessingtechniquesremovetheeffectsofdisper- final generalization by expanding Equations 4 and 5 to sion by shifting the intensity in each frequency channel higher order moments. The weighted, dedispersed nth- 3 moment time series is spectralindex ofα=0. Pulsarshavetypicalspectralin- dices of α = 1.6 with significant variation (α = 0 α n o o,min w [(ν/ν ) I(t+t (ν),ν)] ν 0 DM and α = 3) (Lorimer et al. 1995). Processing with o,max In(t,DM;α)= Xν . (6) an implied spectral index of α = 0 is only ideal for the w minority of pulsars with the lowest observedspectral in- ν Xν dices. Instead, applying frequency weights correspond- ingto the meanpulsarspectralindex wouldincreasethe The smoothed, dedispersed nth-moment time series is SNR with little additional computational cost. ′ n Surveysfornew classesofsourceswhere αis unknown wtt′I(t +tDM(ν),ν) or surveys searching for extremely weak examples of a In(t,DM)= 1 Xt′  .(7) known population may warrant a search over spectral s Nν Xν  wtt′  index. Dedispersing data using a set of trial spectral in-  Xt′  dices would increase the required computation by a factor equal to the number of trial spectral indices. Such a Because the spectral modulation index is a measure of search may only be practical for offline post-processing. the frequencystructure,andnotthe temporalstructure, Frequency weights can also reflect frequency- thedataaresmoothedfirstintime,therebyconsolidating dependent noise variations caused by instrumental thesignalintoasinglespectrum,beforeitissquaredand effects. In general we can define a signal model averaged. The modulation index for smoothed data is then Iνt =bν(Tνt+gνtPνt), m2I = Is2−2Is2. (8) awthuerree,bgννtisisththeebagnaidnpa(es.sgs.,haKpJe,yT−ν1t)iasntdhePsνytsitsemthteepmuplsearr- Is or transient flux density. The quantity we are interested We have specified separate expressions for the dedis- in is Pνt, but the quantity we measure is Iνt. Isolat- persed intensity with frequency weights and smoothing ing Pν requires removing the three frequency-dependent weights solely for clarity; an optimal detection scheme instrumental effects, which may introduce frequency- would employ both. In the remainder of the paper we dependent rms noise. For example flattening the band- will explore the role of smoothing on the modulation in- pass (bν) will result in higher noise at the band edges. dex but keep the assumptions that w =1 and α=0. For systems operating at low frequencies ( 100 MHz) ν ∼ An alternative formulation to calculating the andlargetotalbandwidths,thesystemtemperaturemay smoothed, dedispersed time series is to square the varyacrossthe banddue to the strong frequency depen- data before smoothing: dence of the sky brightness temperature. Finally gain variations will be both time and frequency dependent wtt′In(t′+tDM(ν),ν) due to the particularities of the instrument. Again one In(t,DM)= 1 Xt′ . (9) chooses wν such that SNR is maximized, so if the ad- r Nν Xν wtt′ dwietiigvhetinnogistehecacnhabnenemlsobdyeletdheasinvGearsuessoiafnthweihritvearniaoniscee, Xt′ (i.e., w 1/σ2) results in the maximum SNR. ν ∝ ν This approach captures both the time and frequency To estimate the importance of considering frequency- variation of the data encompassed by the sumations. dependent noise variations, we adopted a simple model Throughoutthispaperwewillfocusonthespectralmod- for system temperature T =20K+10K(ν/ν )−2.7 and νt o ulationindexbutwilldiscussthis“timeresolved”modu- a signal with a non-zero spectralspectral index. The re- lationindexinSection2.6whenwediscussthesignature sulting SNR after correcting for the spectral index and of incorrect dedispersion. averaging over frequency was compared to the standard case that assumes a flat spectral index. At higher ob- 2.2. Frequency weights serving frequencies ( 1 GHz) the improvementis small ∼ Asdiscussedintheprevioussection,weightingthefre- ( 0.1%)forsmallorlargebandwidthsbecauseofthelow ∼ quencychannelsnon-uniformlywhencalculatingadedis- skybrightnesstemperature. Atlowerobservingfrequen- persedtimeseriescanimprovetheSNRofthedetection. cies( 100MHz),thereisanimprovementinSNRbyas ∼ In general an astrophysical signal will have a power law muchas 80%forwidebandwidths. Thisisparticularly ∼ spectrum relevant for observatories like the Low Frequency Array (LOFAR, de Vos et al. 2009) and Murchison Widefield ν −αo Array (MWA, Lonsdale et al. 2009). P =P (10) νt νot(cid:18)ν (cid:19) A non-zero spectral index will increase the variance of o a spectrum and therefore also increase the modulation where P is the pulse flux density attime t and channel νt index. This is undesirable because it would mistakenly ν,ν isareferencefrequency,andα isthesource’sspec- o o imply a lower filling factor and may result in a true sig- tral index. Frequency weights that yield the best SNR nal being flagged as RFI. The magnitude of the increase act to flatten the spectrum depends on the observation frequency (ν ), bandwidth o ν α (B), andspectralindex. For B/νo 0.1 the modulation wν,α = (11) indexincreaseisoforder 0.1forα∼ 3. Intheextreme (cid:18)νo(cid:19) caseofB/ν 1,themod∼ulationinc∼reaseisoforder 1 o ∼ ∼ where optimal detection occurs for α = α . Uniform for α 3. Current instruments have B/ν 0.1, so the o o ∼ ∼ spectralindexfrequencyweightsimplicitlyassumesaflat increase in modulation index due to a source’s spectral 4 index is negligible. The trend is for new instruments to havelargerbandwidths,soeventuallythemodulationin- crease will be significant enough that correcting for the 14 spectral index becomes necessary. Furthermore in surveys that employ a spectral index search, the modula- 12 tion index aids the analysis, because the trial spectral index closest to the true spectral index has the lowest 10 modulation index. Rt 8 2.3. Modulation Index N S Themodulationindexisaquantitativemeasureofthe 6 patchiness, or modulation, of a spectrum. It differen- tiates signals whose power is distributed evenly across 4 the band from those whose power is isolated to a few frequency channels. Broadband and narrowband sig- 2 nals have small and large modulation indices respectively. The level of modulation is parametrized by a 00 5 10 15 20 25 m frequency filling factor f = W /N where W is the I ν ν ν ν numberofchannelsinthespectrumthatcontainssignal. Broadband astrophysical signals have a high filling fac- Figure 1. TimeseriesSNRvs. modulationindexforasimulated tor(fν =1). RFIcanbebothnarrowbandorbroadband dataset with Nν = 256 containing noise (open circles), dispersed withafillingfactorrangingfromf =1/N to1. Asthis pulses(filledcircles),andtwosetsofone-channel-wideRFIspikes ν ν (open squares). The solid horizontal line represents the applied paper focuses on detecting broadband signals, the ulti- intensity threshold of SNRmin =3. The vertical dashed line rep- mategoalistodefine amodulationindexcutoff, mI,max, resentsmI,T =5.3as calculated forSNRt =3usingEquation14. that will enable us to flag signals with low filling factors ThedottedcurveshowsmI,bbasafunctionofSNRt(Equation13). andvastly reducethe numberofcandidatescreatedby a signal detection pipeline. To explore the behavior of the modulation index as a The analysis below assumes that the data have been function of SNRt and fν, we will look at the extrema searched for candidate signals by requiring a candidate of fν: fν = 1 and fν = 1/Nν. It is important to note tohaveanSNR largerthanaminimumSNR(SNR ), that Equation 12, as well as the limiting expressions de- t min and the modulation index is only calculated for these fined below, represents the ensemble average values of candidate signals. Because we assume that our data has the modulation index. Statistically they are the average zeromean,eitherthroughconstructioninthecaseofsim- valueexpectedforagivencombinationsofSNRt andfν, ulated data or throughbandpass subtraction in the case but noise in the data will cause variation in the actual of real data, this assumption assures that I > 0 and m values. Theexpressionsbelowarealsoidealizedinsofar I iswell-defined. Furthermore,ourinterpretationofmI as- as we have set mA =0. sumesSNR >1,whichisreasonableassumptionsince We also simulated a time-frequency data set contain- min such a low SNR would result in a deluge of events. ing broadband and ultra-narrowband signals added to min Toderiveasimple,analyticalexpressionforthedepen- Gaussian noise to accompany the discussion. It was denceofm onf ,weconsideraspectrumwithN chan- processed according to standard pulsar processing tech- I ν ν nelsthatcontainstwocomponents: noiseandsignal. The niques, and the results are shown in Figure 1. The data noiseisassumedtohaveameanofzeroandvarianceσ2. set has Nν = 256 and includes 100 broadband pulses x The signalfills Wν channels withintensity Ai in channel (fν = 1) with Wt = 1 and SNRt = 10, two sets of 100 i. While each Ai may be different, it will be useful to ultra-narrowband spikes (fν = 1/Nν) with Wt = 1 and define A, the average signal intensity over Wν channels. SNRt = 5 and SNRt = 10 respectively, and 106 noise- only spectra with zero mean and σ2 = 1. We ignored The averagesignalintensity overthe entire bandis fνA. x dispersion for simplicity, and the time series was calcu- Asweareusuallymoreinterestedinsignal-to-noiseratios lated using Equation 3 with t (ν) = 0. An intensity (SNR), we define a single-channel SNR, SNR =A/σ , DM νt x threshold was applied to the resulting time series with and time series SNR, SNR = √N f SNR . The latter t ν ν νt SNR = 3 (solid horizontal line), and the modulation expression assumes that the standard deviation of the min index was calculated for samples above threshold. The noise in the times series is σ /√N . x ν vertical dashed line and dashed curve are explained be- The modulation index for this simple signal model is low. N m2 1 f A broadband signal (fν = 1) with no inherent struc- m2 = ν + A + − ν (12) turedoesnotincreaseaspectrum’svariance,sothemod- I SNR2 f f t ν ν ulation index depends only on the number of frequency channels and the signal’s SNR where we have introduced a separate signal modulation t index,mA =σA/A. Thismodulationindexcharacterizes √Nν the inherent frequency structure of a signal, and for this mI,bb = . (13) SNR initialdiscussionweconsideronly signalswith negligible t structure (m 1). In Sections 2.4 and 2.5 we discuss Throughout we will see that the number of frequency A ≪ theeffectsofinterstellarscintillationandpulsarself-noise channelsscalesthemodulationindexbutdoesnotchange and will introduce a non-zero m . the relative magnitude (i.e., signals with larger f have A ν 5 lower m ). As the number of channels from a single in- I strument generally remains constant, it is unimportant 18 to the interpretation of a single data set but is impor- SNRt tant when comparing data from different instruments 16 3 and choosing an appropriate m . 5 I,max Equation13revealsadirectrelationshipbetweenSNR 14 10 t and m . The modulation index of a broadband pulse 100 I 12 is not arbitrary; rather, it must fall along a curve pro- portional to 1/SNR . Furthermore for a signal with a 10 t I givenSNR ,thebroadbandmodulationindexisthelow- m t 8 est m the signal may have on average, as any f < 1 I ν increasesthefrequencymodulationofasignalandcorre- 6 spondingly its modulation index. In Figure 1 the broadband pulses (closed circles) cluster in a stripe centered 4 at SNR = 10 and m = 1.6. While each pulse has t I,bb 2 aninherent SNR =10, the underlying noise in the data t spreadstheSNRt andmI ofindividualrealizationsabout 0 0.01 0.1 1 the ensemble average value (mI,bb) and along the curve f given by Equation 13 (dotted curve). (cid:0) In practice there is a special SNR : SNR , the min- t min imum SNR constraint applied to the time series in the Figure 2. Modulation index vs. filling factor calculated using “thresholding” operation. The corresponding modula- Equation 12with Nν =256. From top to bottom (thin to thick), curvesareshownforfourvaluesofSNRt: 3,5,10,100. tion index is given by Equation 13 centeredatSNR =10and5respectivelyandm =16. t I,s √N Like the broadbandsignals, the noise spreads the points ν mI,T = SNR . (14) about the ensemble average value, but clearly mI,s does min not depend on SNR for narrowbandsignals. t This modulation index represents the averagemaximum Themodulationindexforintermediatefillingfactorsat mI,bb that may exist in a set of thresholded candidates. constant SNRt must transition smoothly from mI,bb to In real data there will be spectra with SNR above the m asf goesfrom1to1/N . Theexactmannerofthe I,s ν ν SNRmin due to noise alone. The modulation index of transition is given by Equation 12. Figure 2 illustrates thresholdednoisefollowsthesamerelationasbroadband the analytical variation of modulation index with filling pulsesbutwithSNRt SNRmin. Mostoftheeventsdue factorforfourvaluesofthetimeseriesSNRrangingfrom ∼ to noise will cluster near mI,T and SNRmin, but the rare SNRt =3 (top) to SNRt =100 (bottom) and Nν =256. stronger noise events will pepper the broadband curve Atlowfillingfactorsallcurvestendtoward√N ,andat ν toward larger SNRt and lower mI. In Figure 1 thresh- high filling factors the stronger the signal, the lower the oldednoise is shownas opencircles,andonly 100points modulationindex. Mostofthedropinmodulationindex are plotted to reduce clutter. The weakestnoise clusters happens at f < 0.1, suggesting that this technique can ν nearSNRmin =3 andmI,T =5.3,andthe strongernoise easily classify signals with filling factors less than about climbs the mI 1/SNRt curve (dotted curve). 10%butislesssensitiveforsignalswithmoderatetohigh ∝ Because mI,T is on average the largest value of mod- filling factors. ulation index for a broadband signal in a candidate list, Inthe above analysiswe haveassumedthat a signalis it is an upper limit to the choice of mI,max. Choosing eithernarrowbandorbroadband. Realityismessier,and a mI,max > mI,T would only return events from thresh- wemighthaveoverlappingsignalsthatarebothnarrow- olded noise or RFI. The dashed vertical line in Figure 1 band and broadband. For example a pulse might occur showsmI,TforSNRmin =3. Ifamodulationindexupper at the same time as persistent narrowband RFI. To ex- limit is applied at mI,max = mI,T, events to the left of plore this case we adopt a simple model looking at the the line are kept (all of the broadband pulses and about modulationindex of a broadbandpulse with no intrinsic 85% of the thresholded noise), while all events to the frequency structure contaminated by an RFI spike that right are dropped (all the ultra-narrowband spikes and is one channel wide (i.e. the “ultra-narroband” case de- about15%ofnoiseevents). AlargerSNRmin wouldraise scribed above). Equation 16 estimates the modulation the solidline andmove the dotted line to the left, which index for this two-component case: reduces the false alarm rate due to noise but limits one to detecting stronger pulses that are presumably rarer. N (1+SNR2 ) ν t,s For the ultra-narrowband case where fν = 1/Nν and mI,bb+s = q , (16) SNRt ,thedependenceonSNRt dropsout,andthe SNRt,bb+SNRt,s →∞ average modulation reduces to where SNR and SNR are the time series SNR of t,bb t,s the broadband pulse and RFI spike respectively. Note m = N 1. (15) I,s p ν − when SNRt,s →0, the aboveequationreduces to the ex- Notethatthemodulationindexforspikysignalsdepends pression for broadband pulses (Equation 13), and when only on the number of channels. Equation 15 also gives SNR 0, the above equation reduces to the expres- t,bb → theupperlimitonm asanyf >1/N reducesthemod- sionforultra-narrowbandspikes(Equation15,upto the I ν ν ulation and decreases the modulation index. Spiky RFI “ 1”,whichisnegligibleforlargeN ). WhenSNR ν t,bb − ∼ is illustrated in Figure 1 as two stripes of open squares SNR , we see that m 0.5m . If we choose t,s I,bb+s I,s ≈ 6 the modulation index cutoff to be m , this implies a sulting averagemodulationindex ofa spectrumcontain- I,T SNR 2 to allow m to be above threshold, ing Gaussian noise and a broadband signal with expo- min I,bb+s ≤ whichisanunreasonablylowthresholdformostsurveys. nentially distributed amplitudes is givenby Equation12 Furthermore using Equation16, we can estimate that in with f = 1 and m = 1. For weak signals the leading ν A orderform m ,SNR /SNR >SNR 1. termofEquation12dominatesandthespectralmodula- I,bb+s I,T t,bb t,s min ≥ − For example if SNR =5, the SNR of the pulse must tionindexincreasesonlyslightlyoverthatforaperfectly min t be four times larger than the SNR of the narrowband flat signal (i.e. Equation 13). For example, the modula- t RFI. Using only the SNR and modulation index, we tionindexofaspectrumwithN =256andSNR =5 t ν min could easily miss a pulse if it occurs concurrent with increasesfromm =3.2tom =3.35. Forstrongsignals I I strong, narrowband RFI. This finding that the modula- (i.e.SNR √N )thefirstterminEquation12becomes t ν ≫ tion index is more sensitive to narrowband signals than negligible and m m 1 for all SNR √N . I A t ν broadband signals is consistent with Figure 2 and sug- ≈ ≈ ≫ gests our method is not a substitute for RFI excision 2.5. Self Noise in the Pulsar Signal techniques that identify persistent, strong, narrowband Broadbandpulsarsignalshavebeenmodeledasampli- RFI from raw data. tude modulated noise (Rickett et al. 1975) and as mod- 2.4. Interstellar scintillations ulated, polarized shot noise (Cordes 1976; Cordes et al. 2004). The noise in these models corresponds to the Small scale density irregularities in the ionized inter- emission over a broad range of radio frequencies while stellar medium (ISM) scatter and refract radio waves. themodulationaccountsforpulsestructure. Inthiscon- Diffractive interstellar scintillations (DISS) and refrac- text,themodulationindexofthesignalisnonzerobutis tive interstellar scintillations (RISS) are observational still smaller than the modulation indices expected from phenomena seen in compact radio sources due to these RFI.InthelimitwherethenoisehasGaussianstatistics, irregularities. Scintillations are characterized by inten- the intensity modulation index of polarized noise is sity variations with typical time (∆t ) and frequency DISS scales (∆νDISS), and for DISS in the strong scatter- m2I =m2ISS+(1+m2ISS)(1+d2p)/2, (17) ing regime, a time series is modulated as a random variable gDISS(t) with an exponential amplitude distri- where mISS is the modulation index of frequency struc- bution and fractional intensity variations on the order turefromDISSanddp isthe degreeofpolarization. The of unity. More generally, DISS varies with both time largest modulation is for m =d =1 when m =√3. ISS p I andfrequency(gDISS(t,ν))withthediffractiontimescale The observed modulation will be reduced if frequency scaling as ∆t ν1.2 and bandwidth scaling as structurefromDISSismuchbroaderthanthetotalband- DISS ∆ν ν4.4. Als∼o note that ∆ν and the pulse widthB orifmultiplepulse structuresareaveragedover DISS DISS ∼ broadening time (τ ) are Fourier transform pairs and in a single frequency channel of the spectrometer. d given by 2π∆ν τ = C (Cordes and Rickett 1998) DISS d 1 where C 1. For fast transients we can safely as- 2.6. Signature of Incorrect Dedispersion 1 ≈ sumethatthedurationofthepulseismuchlessthanthe A bright pulse in a survey dedispersed with a large diffraction time scale (at least for DM. a few hundreds number of trial dispersion measures will yield events at atν = 1 GHz), whereasthe scintillationbandwidth may many incorrect DMs in addition to the correctone. The be of the same order as the channel resolution or band- true DM will return the largest SNR and narrowest t width of a spectrometer. Frequency structure caused by pulse width on average. The residual pulse smearing DISS will increase the variance and thereby increase the from neighboring, incorrect DMs yields a lower SNR t modulation index of a signal. and wider pulse. The larger the DM error, the smaller Understanding the frequency structure of a spectrum the SNR is on average until the SNR drops below the duetostrongscatteringisclarifiedbydefiningthreelim- threshold. This SNR–DM signature is one of the tests its based on the relative sizes of ∆ν, B, and ∆νDISS that a signal is a true astrophysical pulse and not RFI. where ∆ν is the width of a single frequency channel The modulation index of a pulse also increases as the in a spectrum (Cordes et al. 2004). When the scin- DM error increases. tillation bandwidth is smaller than the channel band- Thetwo-dimensionalfillingfactorforadispersedpulse width (∆νDISS ∆ν), the instrument effectively aver- dedispersed with a DM error of δDM is ≪ agesoverseveral“scintles”andtheintensitymodulations are quenched. In this case m 1, and the modula- ∆t −1 tion index follows the expressiAon≪s in Section 2.3. Simi- fν,t = 1+ | δDM| , (18) (cid:20) ∆t (cid:21) p larlyinthe otherextreme wherethere isonly onescintle acrosstheband(0.2B <∆ν ),theintensityvariation where ∆t is the absolute value of the residual dis- DISS δDM | | is nearly flat over the entire bandwidth and m 1. persion smearing and ∆t is the intrinsic width of the A p ≪ For the intermediate case where there are several dis- pulse in seconds. For a pulse that is perfectly dedis- tinct scintles across the band (∆ν . ∆ν . 0.2B), persed, ∆t = 0 and f = 1, while for a large DM DISS δDM ν,t theamplitudeofeachscintleisexponentiallydistributed error,∆t and f 0. δDM ν,t →∞ → and m 1. As instruments become increasingly wide Ideal matched filtering of a dispersed pulse, either in A ≈ band and B/ν 1, this situation will become ever more the absence of dedispersion or from residual smearing ∼ relevant. Examples of these three cases can be seen in from incorrect dedispersion, smooths over the duration Cordes et al. (2004) for giant pulses from the Crab pul- of the pulse’s dispersion sweep, consolidating the signal sar. into a single time bin. The signal has a lower SNR t ConnectingthistothediscussioninSection2.3,there- thanthe pulse’s intrinsic SNR becausef <1, but the t ν,t 7 one-dimensional filling factor is still f = 1. The spec- m =1, as described in Section 2.4. ν I,max tral modulation index increases slightly due to the drop Real signals will not always cleanly divide into broad- in SNR (Equation 13). For example, a hypothetical, band or narrowband. For example RFI could be t unresolvedCrab giant pulse detected in the data set de- marginally broadband, a real pulse could have strong, scribed in Section 3.4.2 with an intrinsic SNR =30 has narrowband structure, or both narrowband and broad- t m = 0.75 when dedispersed using the true DM of the band signals could occur simultaneously. But for fast I Crab pulsar (DM = 56.71pc cm−3) and m 4.3 when transients we also have an additional parameter: disper- I dedispersed with a DM error of δDM=2pc≈cm−3. sion. Lookingatthepeakdispersionmeasureofacandi- The spectral modulation index does reflect the sig- date signal can break the degeneracy between RFI and nature of incorrect dedispersion but only indirectly. A anastrophysicalsignalwith the same modulation index. morepowerfulstatisticwoulddependonf ratherthan ν,t fν. This is accomplished by calculating an alternative 2.8. Correlation Bandwidth modulation index (m ) using the time resolved inten- I,r While the modulation index provides information sity moments given by Equation 9. For the same hypo- aboutthedegreeofmodulationinaspectrum,itdoesnot thetical Crab giant pulse described above, the resolved provideany informationabout the shape ofthe modula- modulation index is the same as the spectral modulation. A signal with amplitude A localized in N adjacent tionindex (m =0.75)whenthe pulseis correctlydedis- I bins has the same variance as a signal with the same persed, but it increases to m =24.7for a DM errorof δDM=2pc cm−3. I,r amplitude whose power is distributed in N isolated bins across the band. We therefore need a new measure that Combining the information provided by the spectral reflects the distribution of a signal in frequency. modulation index and resolved modulation index helps We define the characteristic bandwidth (B ) to be the to identify spurious events from incorrect dedispersion. c typical width of the frequency structure of signal. A By first applying a cutoff in spectral modulation in- broadband signal has B = B, and a narrowband sig- dex, events are classified as either broadbandor narrow- c nal has B ∆ν. We define the fractional correlation bandinfrequency. Theresolvedmodulationindexofthe c ∼ bandwidth as a measure of the typical correlation scale broadband signals distinguish those that are also broad compared to the total bandwidth in time from those that are narrow in time. B c FCB= . (19) 2.7. Modulation Index Cutoff B The role of the spectral modulation index in a source The B can be estimated by calculating the correlation c detection pipeline is analogous to the SNR threshold length from the autocorrelation (ACF) of a spectrum, (SNRmin). Asignalisfirstclassifiedasinterestingornot which we define to be the half width half maximum based on its SNR. For candidate signals that are strong (HWHM) of the first lobe of the ACF. enough, the application of a spectral modulation index Figure 3 illustrates the technique for two simulated cutoff (mI,max) classifies the candidate as interesting or spectra with the same total intensity and variance. The notbasedonhowbroadbanditis. JustasSNRmin isap- left spectrum contains a Gaussian-shaped signal with a plied to data automatically, the modulation index cutoff FWHM of 32 frequency channels. The right spectrum can be applied without human supervision. wasgeneratedbyrandomlyswappingthechannelsinthe ThechoiceofmI,maxisinfluencedbythecharacteristics leftspectrumingroupsoffour,sothatbothspectrahave of the instrumentation and pipeline, as well as the im- the same totalintensity, variance,and modulationindex portanceofastrophysicaleffectssuchasdiffractiveinter- (m 1.5). The bottom panels show the ACFs for the I stellar scintillations. The upper limit to mI,max is given two s≈pectra. To have the zero lag of the ACF equal to by modulation index at the SNR threshold (mI,T). As m2, the ACF was scaled and offset by ACF/(N I2) 1, explained in Section 2.3, mI,T is the largest modulation I ν − where I is the mean of the spectrum. For the Gaus- index on average for a broadband pulse with a perfectly sian spectrum, FCB = 0.18 and is consistent with the flat spectrum in a survey with an applied SNR mini- autocorrelation of a Gaussian with a FWHM=32 chan- mum SNR (Equation 14). Choosing m > m min I,max I,T nels. For the spiky spectrum, FCB = 0.03 is consistent onlyyieldscandidatesfromthresholdednoiseornarrow- with a signal with spikes approximately four channels band RFI. One exception is astrophysical signals with wide. Both spectra could be interesting astrophysical frequency structure, such as that caused by DISS. But signals. The left signalcould clearlybe causedby an as- as we showedin Section 2.4, the increase in the modula- trophysicalprocess. The right signal could be indicative tionindexforweaksignalsissmall( afewpercent),and ∼ of strong interstellar scintillations. In any case the frac- one couldincreasem to allowfor weak,scintillating I,max tionalcorrelationbandwidthprovidesanotherparameter signals with minimal increase in false positives. thatcanbe usedto automatically classifya signal. Note Choosing m < m will reduce the number of I,max I,T the values for the fractional correlation bandwidth were false positives caused by thresholded noise and incor- calculated automatically along with the ACF and other rectly dedispersed pulses, because it effectively applies statistics, suggesting the FCB could be implemented in alargerSNR (Equation13). Thecostisreducedsen- min an unsupervised pipeline. sitivity, as on average only pulses with SNR exceeding t than the larger effective SNR will be below m . ButforsurveyswhereinterstellmairnscintillationmaybIe,mimax- 3. APPLICATION portant,a hardlowerlimit to the modulationindex cut- This section applies the techniques described above to off is set by the nature of exponential statistics; namely, the detection of fast radio transients. First we discuss 8 pulses of different widths and profiles. A list of candidates is defined by applying a minimum SNR thresh- units) 6 oInldt(hSiNs Rpampine)rtwoethfoesceussmonoottwheods,pdeecdifiiscpeimrspedletmimenetasetrioienss. b. 4 of matched filtering: boxcar smoothing and clustering (ar 2 (friends-of-friends). y nsit 0 Boxcar smoothing convolves the data with a box- nte(cid:1)2 car function of length Wt samples. The correspond- I 0 32 64 96 1280 32 64 96 128 ing smoothing weights for Equation 5 are wtt′ = 1 for FrequencyChannel FrequencyChannel t′ =0,...,W . Inpractice boxcarmatchedfiltering is im- t plementedbyiterativelysummingadjacenttimesamples 2.0 FCB=0.18 FCB=0.03 so W = 2nsm where n is the number of smoothing t sm 1.0 iterations. The details of this technique are described F C in Cordes and McLaughlin (2003). A single signal will A 0.0 likely be detected at several values of n , so the event sm list should be sifted for the boxcar width that yields the 1.0 (cid:1) maximum SNR. 0 32 64 96 1280 32 64 96 128 FrequencyLag FrequencyLag The cluster, or friends-of-friends, algorithm looks for groupings of events in a time series. A cluster is defined as a set of events for which there is no gap in samples Figure 3. Fractionalcorrelationbandwidth(FCB)fortwoexam- larger than N . For a cluster with N samples, ple spectra. The top panels show two simulated spectra with the gap clu′ster sametotalintensityandvarianceandNν =128. Theleftspectrum the smoothing weights are wtt′ = 1 for t = t1,...,tN containsaGaussianwithaFWHMfrequencywidthof32channels, where t1 is the first sample in a cluster, tN is the last andtherightspectrumisaspikyspectrumgeneratedbyrandomly sample, and t′ t′ <N +1. Importantly, the clus- swapping groups of four channels in the left spectrum. The au- ter algorithmi+is1a−gniostic tgoapthe symmetry of the pulse, tocorrelation functions (ACF) of the samplespectra areshown in unlike the boxcar smoothing filter which is symmetric. the bottom panels. The ACF has been normalized by the square ofthemeanofthespectrumandthenumberofchannelsallminus Aspulsesfromhighlyscatteredsourceshaveexponential 1. Thecharacteristicbandwidth iscalculated asthehalfwidthat tails, this algorithm may be better suited to detecting halfmaximumofthefirstlobeoftheACF,andthecorresponding such astrophysicalobjects. FCBisgivenintheupperrightcornerofthelowerpanels. After a list of possible candidates is defined, one goes oneapproachofincorporatingthecalculationofthespec- backtotherawtime-frequencydatatocalculatethesec- tralmodulationindexintoatransientdetectionpipeline. ondmomentsandmodulationindices. Wegrabanarrow To illustrate the technique we simulate a transient de- range of raw data centered at the location of the can- tection pipeline and apply the detection scheme to real didate and reprocess it. The time-frequency snapshot data containing known transients. Our simulations and is dedispersed and smoothed at the DM and smoothing applications to real data show that a modulation index parameters determined from the first pass. A time se- cutoff efficiently flags RFI and significantly reduces the ries is calculated for both the first and second moments. number of candidates. Note our implementation starts The first moment time series is thresholded in intensity, with a list ofpulse candidates generatedin the standard and the modulation index is calculated for the samples manner and not on the raw data directly. A discussion above threshold. The modulation indices of the events of how the modulation index could be used to flag raw are compared to the modulation index cutoff, m , data in real time is discussed in Section 4.4. Also note I,max and flagged as either a signal of interest if m m our technique is independent of the number of beams I ≤ I,max or RFI if m >m . (e.g. Arecibo L-bandFeed Array1) or stations (e.g. Very I I,max Thetwo-passapproachisadoptedoutofpracticalcon- LongBaselineArray,Thompson et al.2011)usedtocol- siderations. While the first moment of the dedispersed lect the data. time series canbe smootheddirectly, the time-frequency 3.1. Implementation in a Detection Pipeline data must be smoothed before being squared and averaged in frequency (Equation 7). This would require a Thepipeline makestwopassesoverthedata. Thefirst differentdedispersed,smoothed,secondmomenttimese- pass defines a list of candidate events, and the second riesforeachmatchedfiltertypeandparameter. Further- passcalculatesthesecondmomentandmodulationindex moresomesmoothingapproaches,suchasthe clusteral- of these events. This two-pass approach is adopted for gorithm, determine the smoothing weights wtt′ from the practicalityandflexibilityandwillbeexplainedindetail thresholded first moment time series, making a parallel below. Also recall that the modulation index requires calculationofthesecondmomenttimeseriesimpractical. the first and second central moments, so the data must 3.2. Processing Requirements be bandpass subtracted before I and I2 are calculated. The data are first dedispersed as described by Equa- For most cases the two pass approach is more com- tion 3 to generate a first moment time series. Surveys putationally efficientthan the obviousalternativeof cal- use a range of trial dispersion measures, each generat- culating the second moment in parallel with the first. ing its own time series. These dedispersed time series We parameterize the processing required by the second are smoothed in time by applying a template bank of pass in terms of the processing required to do the dedis- matched filters with different properties to account for persion in the first pass. Generally dedispersion domi- nates the processing time in a transient survey, so this 1 http://www.naic.edu/alfa is a useful metric. The number ofoperationsrequiredto 9 dedisperse a block of time-frequency data with N time termediate data products (i.e., dedispersed time series) t samples and N frequency channels with N trial dis- can be kept in memory. ν DM persion measures is N = N N N . If our The fake data shown in panel (a) of Figure 4 set con- ops,1 t ν DM × × firstpassgeneratesN candidateevents,thenumber tains a single dispersed pulse, a Gaussian RFI spike, events ofoperationsrequiredto dedisperseanarrowtime range and a broadband RFI spike. The data properties are with N samples around each event at a single DM is N = 256, ∆t = 1 ms, N = 2000 time samples, s ν s t N =4N N N . Thefactorof4wasincluded ν = 1400 MHz, and B = 100 MHz. A single dispersed ops,2 events s ν o to consider the sq×uari×ng of the data, summing of both pulse was added at t = 0.25 s with DM = 500 pc cm−3, the originalandsquareddataandbandpasssubtraction. SNR = 1, and W = 1. A narrowband Gaussian νt t The processing required by the second pass normalized spike was added at t = 1.0 s and ν 1428 MHz with o ≈ by the dedispersion processing of the first pass is SNR =40, W =2, and W =2. Finally a broadband νt t ν RFI spike is represented by an undispersed pulse (i.e., N 4N N P = ops,2 = events s. (20) DM = 0 pc cm−3) at t = 1.5 s with SNR = 5 and 2 νt N N N ops,1 DM t W =1. The data were dedispersed over a range of trial t If N = 106, N = 103, N = 103, N = 104, P dispersionmeasuresDM=0 1000pc cm−3andDMin- t DM s events 2 terval ∆DM=6pc cm−3. A−list of candidate event was is a few percent of the original dedispersion processing. defined by applying anSNR thresholdof SNR =3 to Thisanalysisignorestheadditionaloverheadincurredby min the resulting time series. returningtothe rawdata,andinparticularonemustbe Figure 4 shows the results of this simulation. The top wary of excess file I/O. frame shows (a) the time-frequency data, (b) candidate MostRFIexcisiontechniquesoperateonrawdata,not events vs. DM and time, (c) 50/m for the candidate on the list of candidate pulses. Our technique is more I events, and (d) intensity vs. DM and time for candi- general, because it can be used at both the beginning datesbelowm =3.2. ForclaritytheSNRandpulse (see Section 4.4) and end of a source detection pipeline. I,max widthsofthethreesignalsinpanel(a)havebeenexagger- The most common RFI excision approach applied to a ated, andthe noise is not shown. Instead of plotting the listofcandidatesistoremovealleventsatlowdispersion intensity,panel(b)showsadotforeachsamplethatwas measureundertheassumptionthatterrestrialsignalsare abovetheintensitythresholdtoreduceclutter. Panel(c) not dispersed. This is a blunt instrument and does not shows that most of the events have similar modulation removeevents fromRFI athigher DMs. Calculating the indices; the one exception is for the broadband pulse, modulationindexofthecandidateeventsallowsformore which has a modulation index almost an order of mag- sophisticated RFI excision in a list of candidates. nitude lower than all other points due to its high SNR. Classifying signals with the modulation index should Panel (d) plots the SNR of the events with modulation be used in conjunction with other RFI excision algo- indices below m with the area of the circle propor- rithms. As described in Section 2.3, the modulation in- I,max tional to the SNR of the event. dex is more sensitive to narrowband signals, and weak The triangle-shaped group of events near t 1.5 s pulses may be missed if they occur simultaneously with ∼ in panels (b) and (c) are spurious hits caused by the strong, narrowband RFI. This suggests the modulation dedispersion path crossing the bright, broadband RFI indexalgorithmworksbesttogetherwithalgorithmsthat samples. Mostoftheseeventshavelowfillingfactorsbe- remove channels that contain persistent RFI. Similarly cause the RFI samples contribute only a few samples to broadband RFI has a low modulation index, so remov- the dedispersed spectrum and are not present in panel ing impulsive RFI through other means will reduce the (d). The exceptionis low DM andt=1.5 s where signal number of events from RFI that fall below the modula- from the RFI contributes to many frequency channels tion index cutoff. resulting in a low modulation index. This is the incor- 3.3. Simulations rect dedispersion effect described in Section 2.6. While this example of RFI does pass our modulation index fil- To assess the usefulness of the modulation index as a ter, the low DM of the event exposes it as RFI. Ap- signal diagnostic, we simulated a single-pulse event de- plying yet another filter that removes events at low DM tection pipeline using Python-based software. In our willremovesuchevents. ThenarrowbandGaussianspike simulations fake time-frequency data can be generated causesastripe ofspuriouseventsbetweent 0.9 1.0s with Gaussian-distributed noise, dispersed pulses, and ≈ − as the path of the each trial dispersion measure crosses a variety of RFI. A dispersed pulse is added with a the spike. But for reasons just described above, these specified dispersion measure and Gaussian pulse profile spurious candidates have modulation indices above our with a FWHM of W . Spiky RFI is modeled as a two- t threshold. Finallythetrueastrophysicaldispersedpulse, dimensionalGaussianwithaFWHMwidthintime(W ) t which is buried in the middle panels of Figure 4, stands and frequency (W ). Broadband RFI is modeled as an ν outin panel(d) at t=0.25s after applying the modula- undispersedpulse(i.e., DM=0 pc cm−3). The simulated tion index cutoff. data then undergo single pulse search processing. First ThebottomframeinFigure4plotsSNR vs.m anda the time-frequency data are dedispersed over a range of t I histogram of m to illustrate how the modulation index I trialdispersionmeasuresandbothI andI2 iscalculated groups signal types. The spurious events caused by the from the dedispersed time series. I is thresholded, and Gaussian spike (medium gray) are clumped together on forsamplesthatareabovethreshold,the modulationin- the far right of the plot between 8 < m < 20, consis- I dex is calculated. Note that because these simulations tent with the extreme narrowband case given by Equa- involve a small amount of data, the two-pass analysis as tion 15 (m = 16). The events due to the dispersed I,s described in Section 3.1 is not necessary, because all in- 10 pulseandbroadbandRFIfollowthelightgrayandblack tracksrespectively. The tracksare caused by incorrectly dedispersing the signal, and in both cases the lowest 1.45 a) modulation index corresponds to the correct dispersion GHz) 1.4 measure. The events from thresholded noise (black) are ( clumpednearm 5andSNR SNR aspredicted (cid:3) 1.35 I,T ∼ t ∼ min 1000 b) by Equation 14. Because the weaker events associated 750 with the broadband RFI overlap with the area contain- 500 250 ing thresholdednoise, we’veplotted them with the same 0 color. ThehistogramofmI inthelowerpanelshowsthat 3)1000 c) by applying a m =3.2, we have eliminated most of m(cid:2) 750 the events. I,max pcc 500 ( 250 M 3.4. Application to Data D10000 d) 750 In the next two subsections we apply our modulation 500 index classification technique to two known sources of 250 single pulses detectable by single-pulse search pipelines. 0 0.0 0.5 1.0 1.5 2.0 InSection3.4.1weapplythemethodtoRRATJ1928+15 Time(sec) andinSection3.4.2togiantpulsesfromtheCrabpulsar. Although in both cases we know the correct dispersion measureof the source,we re-analyzethe data ata range 102 of trial dispersion measures to recreate typical survey results. meTdhiaendbaatandwpearsesbananddspuabstsraccotrirnegctoeffdtbhyemdievaidni.nDgibvyidtinhge NRt 10 S bythemedianspectrumflattensthespectra,assuringthe bandpassshapeisnotcontributingtothevariancecalcu- lation. We choose the median because it is less sensitive 1 to extreme values caused by pulses or RFI. Subtracting the mean assures us that our spectra have zero mean. 103 3.4.1. RRATs ber102 m RRAT J1928+15 was discovered by Deneva et al. Nu 10 (2009) in the PALFA survey running at the Arecibo 1 Observatory. (See Deneva et al. 2009 for the details of the observation and PALFA parameters.) In brief, 10(cid:4)1 1 10 J1928+15 was discovered using the single-pulse search mI algorithms implemented by the Cornell pulsar search pipeline. Three pulses were detected with an interval Figure 4. Diagnostic plots for the simulated data described in of 0.403 s at DM = 242 pc cm−3. We reprocessed the Section 3.3. Top Frame: Panel (a) shows a grey scale of the bandpass corrected data with 642 trial dispersion mea- simulated time-frequency data where the SNRs and widths of the signals have been exaggerated for clarity. Panel (b) plots a sures roughly equally spaced over a DM range DM = 0 500 pc cm−3. The dedispersed time series had an point in the DM-time plane for each event above the intensity − threshold (SNRmin = 3). Panel (c) plots 50/mI for the candi- intensity thresholdapplied at SNRmin =4. The samples date events in panel (b). Panel (d) plots the SNR of candidates above the SNR threshold were run through the cluster overtheSNRthresholdandbelowthemodulationindexthreshold algorithm,and the sample in eachcluster with the max- (mI,max =3.2). The data werededispersed over aDMrange of0 to 1000 pccm−3 with ∆DM = 6 pccm−3. The dispersed pulse imum intensity was used to calculate the modulation in- is located at DM = 500 pccm−3 and at t = 0.5 s. A Gaussian dex. RFI spike is located at t=1.0 s and a frequency of ∼ 1428 MHz The results are shown in Figure 5. In all panels only andcausesthestripeofeventsbetweent≈0.9−1.0s. Broadband eventswithSNR>5areshown. Thetopframeplotsin- RFI is located at t = 1.5 s and causes the triangle-shaped patch of candidate events. Bottom Frame: The top panel plots SNRt tensityversusDMandtimeforalleventsabovethreshold vs. mI, and the bottom panel is a histogram of modulation in- (top) andthe events that also satisfy mI <1. (Compare dex. Events associatedwiththebroadband RFI(andthresholded to Figure 3 in Deneva et al. 2009). The strong RRAT noise) are plotted in black, events associated with the Gaussian pulse is clearly visible around t = 100 s, and a weak RFI spike are shown in medium gray, and the events associated withthedispersedpulseareplottedinlightgray. broadbandRFIspikeoccursneart=89s. Forthestrong RRAT pulsewemeasureSNR 20,andforthe preced- The bottom frame shows SNR vs. m (top) and a his- t t I ≈ ing weaker pulse, SNR 6. The difference between our togram of m (bottom). In the top panel the events t I ≈ values and those in Deneva et al. (2009) is likely due to associated with the strong RRAT pulse are localized thefactthatweusedtheclusteralgorithmandtheyused along the upper curve extending from SNR 20 and t ≈ the boxcar smoothing matched filter algorithm. We also m 0.8 to SNR 5 and m 3. The lower limit I t I ≈ ≈ ≈ donotdetecttheweaker,tailingpulse,likelyforthesame of m 0.8 is consistent with the value predicted by I ≈ reason. Applying a m = 1 eliminates 99% of the Equation 13 for SNR = 20. The curve itself comprises I,max t ≈ events, leavingonly the strongestevents associatedwith spurious events caused by the strong pulse being dedis- the bright RRAT pulse. persed at incorrect dispersion measures and account for