Table Of Content

PSDF: Particle Stream Data Format for N-Body Simulations Will M. Farra,∗, Jeff Amesb, Piet Hutb, Junichiro Makinoc, Steve McMilland, Takayuki Muranushie, Koichi Nakamuraf, Keigo Nitadorig, Simon Portegies Zwarth aNorthwestern UniversityCenterfor Interdisciplinary Research inAstrophysics, 2145 Sheridan Rd., Evanston IL 60208 USA bInstitute for Advanced Study, Princeton, NJ 08540, USA cInteractive Research Centerof Science, Graduate School of Science and Engineering TokyoInstitute of Technology, 2–12–1 Ookayama, Meguro, Tokyo 152-8551, Japan dDepartment of Physics, Drexel University,3141 Chestnut Street, Philadelphia, PA 19104 eThe Hakubi Center, KyotoUniversity, Kitashirakawa Oiwakecho, Sakyo-Ku, Kyoto, 606-8502, Japan fGraduate School of Information Science and Technology, The Universityof Tokyo. 7-3-1Hongo, Bunkyo-ku, Tokyo, Japan gCenterfor Computational Science, Universityof Tsukuba, 111, Tennodai, Tsukuba, Ibaraki 3058577, Japan hLeiden University,P.O. Box 9513, 2300 RA Leiden, The Netherlands 2 1 0 2 n Abstract a J We present a data format for the output of general N-body simulations, allowing the presence of individual time steps. 9 Byspecifyingastandard,differentN-bodyintegratorsanddifferentvisualizationandanalysisprogramscanallsharethe simulationdata,independentofthetypeofprogramsusedtoproducethedata. OurParticleStreamDataFormat,PSDF, ] Mis specified in YAML, based on the same approachas XML but with a simpler syntax. Together with a specification of PSDF, we provide backgroundand motivation, as well as specific examples in a variety of computer languages. We also I .offer a web site fromwhich these examples canbe retrieved,in orderto make it easy to augmentexisting codes in order h pto give them the option to produce PSDF output. - oKeywords: Stellar dynamics, Method: N-body simulation r t s a 1. Introduction as well as galactic nuclei, have relied on the use of in- [ dividual time steps, at least since the 1960s (Aarseth, 1 The simplest N-body calculations use a shared time 2009). The reason is that the presence of close binaries vstep length for all particles, implying a straightforward and triples in such systems would increase the computer 4structureoftheoutput. WithNparticlesandktimesteps, powerneededbyordersofmagnitudeincaseofsharedtime 9theoutputtakesontheformofanN∗kmatrixofparticle 6 steps,comparedtoindividualtimesteps. Inaddition,cos- data, wherethe latter typicallycontainthe mass,position 1 mologicalcodes,too,often useindividualtimesteps, given .and velocity of a single particle at a specific time, with the increasinglylargediscrepanciesofintrinsic time scales 1 possible additional information such as higher derivatives 0 that come with increasingly high spatial resolution (e.g. 2of the position (acceleration, jerk, etc.), the value of the Springel, 2005). 1potential at the position of that particle, and so on. The The simplest way to output data from individual time :output of this matrix can be done by ordering in time or v step codes would be to use shared time steps for the out- ibyorderingbytheidentityofparticles,inwhichcaseeach put. Indeed, typical legacy codes, such as NBODY6, do X particle’s history is output separately. just that by default. If all one wants to do is to make r Some complications may occur when particles are re- a a fixed movie of a simulation run, that approach suffices. moved, for example because they are escaping from the However, when we interactively inspect the results of a system, or because they represent a star that undergoes a simulation run, we want to be able to zoom in and out, destructive supernova leaving no remnant. However, the andspeed up and slowdownthe rate at whichwe run the basic I/O structure is simple enough that it is easy to graphics presentation of the run. With a fixed initial out- presentthesekindsofdatainoneofthestandarddatafor- put rate, it may not be possible to interpolate the motion mats,suchasFITS(Pence et al.,2010)orHDF(The HDF Group, of the particles that move at high speeds. Phrased dif- 2011), with a brief description. ferently, an output rate high enough to faithfully present The situation becomes vastly more complicated when the motion of all particles may be prohibitively expen- we allow for individual time steps. Simulations of dense sive in terms of memory. It would be much better to let stellar systems, such as open and globular star clusters, the graphics program itself decide how and where to ex- trapolate, given the original data it has received from a ∗Correspondingauthor simulations code. Email address: [email protected] (WillM.Farr) For example, when we display the dense center of a Preprint submittedto Elsevier January 10, 2012 star cluster, the graphics program can then use the full 2.1. Some basics of YAML information for the rapidly moving particles, while inter- The following is a simple example of data in YAML polating the data for the slower halo particles. Such an format. approach can easily save orders of magnitude of memory storage requirement. This approachwas implemented --- !Particle byMcMillanwithinthe“Starlab”simulationenvironment id: 0 (Portegies Zwart et al., 2001). The associated tdyn data r: format is described in Section 10 of Hut et al. (2003) and - 0.1 explored in more detail by Faber et al. (2010). However, - 0.2 this implementation was handcrafted for a specific code, - 0.3 reflecting the data structure used in that code. Clearly, v: it would be desirable to have a more universal data for- - -1 mat that allows different codes to share data in a more - -2 transparent way. - -3 Other concerns are to make a data format standard m: 1.0 machine independent, to make allowance for parallel pro- In the above example, the line cessing, and to avoid serious overhead penalties with re- spect to performance. --- !Particle Here we describe the “Particle Stream Data Format,” Isthe header,whichindicates thatitdescribesthedataof or PSDF, a machine-independent, algorithm-independent an object of type Particle. The line dataformatforstoringtheresultsofasimulationofpoint- mass gravitationaldynamics using individual timesteps. id: 0 defines a field with name “id”, and value 0. The text 2. Basic idea r: We wish to store the evolution of a gravitating system of N bodies throughout a simulation with individual - 0.1 - 0.2 timesteps foreachparticle. Conceptually,whatweneedis - 0.3 a stream of phase-space information of particles, updated each time the integration algorithm adjusts a particle’s means the field “r” is an array with three elements. The phasespaceinformation. Onepossibilityforsuchastream first“-” means this line is a data for anarray. By default, could be: numbers without “.” are regarded as integers, and with particle_id, time, mass, x, y, z, vx, vy, vz, ...“.” floatingpoint. Notethatindentationhasmeaninghere particle_id, time, mass, x, y, z, vx, vy, vz, ...and “-” must be indented the same level or deeper than particle_id, time, mass, x, y, z, vx, vy, vz, ...“r”andshouldbealigned. The“v”and“m”fieldsbehave similarly. The order of the fields is not important. The However, the data format should be flexible enough to be Particle object behaves as a map from names, like “r”, able to include more information, if available, such as tovalues,inthecaseabovethearray“[0.1, 0.2, 0.3]”. • hierarchicaldecompositions of the system into binaries, triples, etc. 3. Particle Stream Data Format • radius, and other info related to stellar evolution WiththeminimaldescriptionofYAMLintheprevious • merger history section, we can now define the Particle Stream Data For- mat: thedataformatisastreamofYAMLrepresentations • fluid properties if a particle is an SPH particle of particle objects. For example, a valid PSDF fragment • close encounter history is • stellar evolution history --- !Particle id: 0 • and so on. t: 0 One way to construct such a flexible data format is to r: use a self-describingdata format, suchas XML or YAML. - 0.1 Forsimplicity,weadoptYAML (YAML,2011)here;there - 0.2 are libraries for reading and writing YAML in many pop- - 0.3 ular programming languages, and the format is simple v: enough to be understood easily by humans, even if they - -1 are not already familiar with it. - -2 2 - -3 Name Meaning m: 1.0 id index (can be arbitrary text) --- !Particle m mass id: 1 t time t: 0 t max max time to which this record is valid r: r position, array with three elements - 0.2 v velocity, array with three elements - 0.3 pot potential - 0.4 acc acceleration, array with three elements v: jerk jerk, array with three elements - 0 snap snap, array with three elements - 0 crackle crackle, array with three elements - 0 pop pop, array with three elements m: 1.0 Table 1: The reserved names of PSDF and their meanings. Other This fragment describes two particles, at t=0, with ids 0 namesoccurringinthedatastreamaretobeignorediftheyarenot meaningfultoanapplicationorinterpretedinanapplication-specific and 1. wayiftheyare. Herethe jerk,snap,cracle, andpop areour names Particle objects behave in YAML as mappings from for the third, fourth, fifth, and sixth derivatives of position. We namestovalues. Therefore,aspecificationofthemeaning specifythateachvectormustcontainthreeelements;intheeventof of certain names and a procedure for handling unknown a two-dimensional simulation, one of the vector components should besettozero. names in the stream are sufficient to define the data format. In Table 1 we list the reserved names of our PSDF. Any particular particle in a PSDF stream need not in- to oneormorefiles tobe processedatalaterdate. We do clude a value in its map for any of these names, but if notimpose anyparticularorderingonthe particlerecords it does, the value must have the meaning in the table; in a PSDF stream. For some applications an ordering in similarly, if a particle object in a PSDF stream does con- time may be appropriate, while for others an ordering in tain a value with one of the meanings in the table, then particleidmaybebetter,orevenmorecomplexorderings; it should be identified by the corresponding name. Note inSection5weprovidereferencestocodethatcanconvert thattheserequirementsallowforeasyextensionofaPSDF between the time and particle-id orderings. stream with application-specific information by including any names and values not in Table 1 needed by the spe- 4. Rationale cialized application. Programs that understand this additional information can benefit, while those that do not Our goal is to describe a data format that is will still be able to function using the basic information from any included values of Table 1. In the future, we in- 1. Space-efficient for storing the data from individual- tend to provide extensions that are useful for hierarchical timestep N-body simulations. decompositionofanN-bodysystem,forexampleonecon- 2. Simple enoughto be humanreadable and writeable, taining tight binaries, triples, and higher multiples, and and safe for programsto read, even from un-trusted for description of fluid SPH particles. sources. We require that time, position, velocity and higher 3. Information-richtoallowforpost-processingandanal- derivatives areconsistent(for example, ifpositionis given ysis or even provide enough additional information in parsecs and time in years, velocity must be in par- for continuation or re-running of a simulation. sec/year). The name t max is rather special, in that it 4. Flexible enough to accommodate the special needs does not specify part of the state of a particle, but rather of programs that have more complex objects than givesthemaximumpossibletimethatthisrecordisusedto point-mass particles. predict the orbit of this particle; we expect that this may 5. Composable, so that fragments of the data can be prove useful to prevent invalid extrapolation in programs split off for separate analysis and recombined easily. that process the PSDF. If no t max is given, it should be In this section we describe how these design goals led to assumed that the particle recordis valid until an updated the specification in the previous section. record is encountered in the stream, or forever if no such updated record appears subsequently in the stream. 4.1. Space Efficient AcompletePSDFobjectisastreamofparticleobjects describing the states of individual particles in the system As outlined in the introduction, it is extremely waste- atparticulartimes. Sucha streammaybe consumedasit ful to produce a complete system snapshot in an N-body isproduced,asinthecaseofanintegratorprogramwhose simulationeverytimesomeofthebodiesupdatetheirposi- output is directedto agraphicsprogramthatdisplaysthe tionsorvelocities. ThePSDFformatallowsfortheoutput result of the integration;or sucha streammay we written of only the changed data—the new states of the updated 3 particles at the new time—to the stream. Thoughthe na- 4.5. Simple, Safe, and Composable tive format is text-based, for readability, we have found PSDF documents are composable, meaning that any that common compression algorithms such as gzip ap- two PSDF streams can be concatenated to form another plied to files containing PSDF data produce output that valid PSDF stream, and a single stream can be split into is within 10% of the size of equivalent compressed binary a number of valid sub-streams. It is quite simple to write data. ashortscriptinanynumberoflanguagesthatconsumesa PSDF stream and produces another recording the history 4.2. Simple and Human-Readable ofaparticularparticleintheoriginalstream,forexample. By using ageneralformat(YAML) in wide use forour Another example usefulinpractice is “thinning”a stream PSDF,weensurethattherearemature,debuggedlibraries by including only every nth particle update, when the in- availableforreadingandwritingourformat(YAML,2011). terpolationrequirementsofthe consumingapplicationare However, YAML is simple enough that it can we written looser than those of the producing application. Adding and modified easily “by hand” in a text editor using only header information, or instructions, or any other meta- ASCII characters. The idea of streaming updates to indi- data to the stream defeats this goal by requiring specifi- vidualparticlestatesalsomeshesnicelywiththeevolution cation of some way to split and combine the associated algorithmsinmostN-bodyintegrators,makingtheformat meta-data. easy to write from within such a code. PSDFobjectsaredescriptionsofdata,notinstructions 5. Repository for actions for an application. In other words PSDF does not contain any hidden “language” structures. For exam- We have examples ofcodes that generateandmanipu- ple, there is no instruction for “adding” or “deleting” a late PSDF streams at particle from the stream. Adding such instructions would require applications to implement interpreters for imple- https://github.com/jmakino/Particle-Stream-Data-Format mentingtheinstructionsinthestream,whichraisesissues The examples are in various different languages, and in- of security and language design that would significantly clude complicate the specification. • Programs to generate initial conditions for N-body 4.3. Information-Rich simulations in PSDF format. The fundamental state of a point-particle can be spec- • Code and references to several different individual- ified in 8 numbers: one mass, one time, three position co- time-stepintegratorsthat cantake PSDF input and ordinates and three velocity coordinates. However, some advance the corresponding system in time, produc- auxiliary information about the particle’s state can be ing PSDF output at each step. very helpful: accurate prediction of the particle’s position and velocity—for example, to display its track in a • Various post-processing tools that compute useful visualization—canbe facilitatedbyinformationaboutthe system properties from PSDF input. higher derivatives of its position in time. When available to the integrationroutine, these canbe easily providedby • Avisualizationprogramthattakes PSDFinput and our format (see Table 1). producesa3Drepresentationofthe systemthatcan be played forward and backward, zoomed, etc. 4.4. Flexible Wehopethattheexamplesweprovidewillmakeiteasyfor NotallparticlesinmanyN-bodysimulationsarepoint- thecommunitytousePSDFintheirsimulations,andthat masses! For example, simulations may attempt to model these users will, in turn, contribute their useful programs stellarevolution,andthereforestore“star”propertieslike back to the repository as examples for future users. radii and masses, or entropy profiles with their particles. Or,simulationsmayincludefluidparticlessubjecttonon- gravitational forces for SPH calculations. To attempt to Acknowledgments standardizenamesforeverypossibleparticlepropertywould Partoftheworkwasdonewhiletheauthorsvisitedthe result in a rigid and cumbersome format; instead, by al- Center for Planetary Science (CPS) in Kobe, Japan, dur- lowing arbitrary application-specific names in PSDF par- ingavisitthatwasfundedbytheHPCIStrategicProgram ticle mappings that can be ignored when not understood of MEXT. We are grateful for their hospitality. we permit complex applications to work with application- specific data while ensuring that simple applications can make use of the parts of the data they understand. 4 References Aarseth, S. J., 2009. Gravitational N-Body Simulations: Tools and Algorithms.CambridgeUniversityPress. Faber, N. T., Stibbe, D., Portegies Zwart, S., McMillan, S. L. W., Boily,C.M.,Jan.2010.Particle-basedsamplingofN-bodysimu- lations.MNRAS401,1898–1903. Hut,P.,Shara,M.M.,Aarseth,S.J.,Klessen,R.S.,Lombardi,Jr., J.C.,Makino,J.,McMillan,S.,Pols,O.R.,Teuben, P.J.,Web- bink,R.F.,May2003. MODEST-1: Integrating stellarevolution andstellardynamics.NewA8,337–370. Pence, W. D., Chiappetti, L., Page, C. G., Shaw, R. A., Stobie, E.,Dec.2010.DefinitionoftheFlexibleImageTransportSystem (FITS), version3.0.A&A524,A42. PortegiesZwart,S.F.,McMillan,S.L.W.,Hut,P.,Makino,J.,Feb. 2001.Starclusterecology-IV.Dissectionofanopenstarcluster: photometry. MNRAS321,199–226. Springel, V., Dec. 2005. The cosmological simulation code GADGET-2.MNRAS364,1105–1134. The HDF Group, 2011. Hierarchical data format version 5, 2000- 2010. http://www.hdfgroup.org/HDF5. YAML,2011.Theofficialyamlwebsite.Http://www.yaml.org/. 5