Table Of Content

Re(cid:29)ection in Pure Data IOhannes m zm(cid:246)lnig Institute of electronic Music and Acoustics, In(cid:27)eldgasse 10, 8010 Graz, Austria, [email protected] Abstract one can describe the entire environment as an agent-based system. One of the more prominent realtime computer music languages on linux is Pure Data. While Pure An agent as descibed in [2] is (cid:16)situated within data provides ample constructs for signal domain and a part of an environment that senses that speci(cid:28)c programming, it has only limited capabili- environmentandactsonit,overtime,inpursuit ties for metaprogramming. In this paper we present of its own agenda and so as to e(cid:27)ect what it iemguts, a collection of objects that add (amongst senses in the future.(cid:17) Usually Pd-objects do not other things) re(cid:29)ection capabilities to Pure Data. meet this criterion. Nevertheless, it does not take so much, to Keywords make a Pd-object aware of its environment in Metaprogramming, Pure data, agents order to be able to react on it. Unfortunately, plain Pd o(cid:27)ers only limited functionality to ac- 1 Introduction complish even this. What is needed to add (cid:16)self-awareness(cid:17) to a Pure Data is a programming language for re- programm (e.g. a Pd-patch) is called re(cid:29)ection altime signal processing and computer music. in computer sciences. According to Demers and While it can be regarded as a domain speci(cid:28)c Malenfant, computational re(cid:29)ection is (cid:16)the pro- language to accomplish the task of handling cessofreasoningaboutand/oractinguponone- and manipulating streams of numbers, an im- self(cid:17) [3], with this (cid:16)oneself(cid:17) being the program. portant aspect is the twodimensional diagram- This is closely related to what is called dy- matic representation of algorithms. As Mathieu namic patching within the Pd-community, gen- Bouchard has put it: (cid:16)The Diagram is the Pro- 1 erating (parts of) Pd-patches from within Pd. gram(cid:17). Metaprogramming/Re(cid:29)ection o(cid:27)ers ways to Due to the representational nature of Pd, it inspect the current state of a patch (from an is well suited for direct presentation to the au- interpreters point of view) as well as it can ease dience [1], e.g. by means of code projection the process of writing patches, e.g. by allowing during Live Coding performances: the graphi- to use (cid:16)macros(cid:17), that automatically create or in- cal environment o(cid:27)ers access for a lay audience sertsnippetsofcodewithouthavingtomanually by means of reading (or at least: something to patch them. look at). In section 2 of this paper we will have a closer If a graphical patch cannot be read as source look at what plain Pd o(cid:27)ers for this kind of pro- code (due to lack of programming knowledge), gramming, and what are the shortcomings of what remains is a set of labeled rectangles, the available mechanisms. We will then present which are interconnected by lines. (The in- iemguts, a library that expands the powers of terconnections becomes obvious in Live Cod- Re(cid:29)ections, in section 3. ing performances, when connected objects are movedaroundwiththeconnectinglinessticking 2 Metaprogramming in Pd totheinletsrespectiveoutletsoftherectangles). Re(cid:29)ection can be split into two di(cid:27)erent parts: Assuming that an audience can understand, self-examination and self-modi(cid:28)cation. While that the (cid:16)rectangles(cid:17) are supposed to represent Pd has some powerful abilities to modify the (cid:16)entities that do something(cid:17) (e.g. processes), running patch (with a focus on adding to the 1ThisisthemottoofthePd-addicted#data(cid:29)owIRC- patch rather than changing existing parts of channel at FreeNode it), there are only a few objects that allow a patch to examine its own runtime behaviour: [realtime] (and the related [cputime]). This object measures te time elapsed between two [bang(s, and can therefore be used for pro(cid:28)l- ing cpu-hungry objects (see Fig.1). Figure 2: Simple self-creating patch 2.2 The woes of metaprogramming in Pd Apart from being able to create patches on the (cid:29)y, dynamic patching can also simulate user-interaction, by sending special messages for mouse-movements and keyboard events to a canvas. However, this has some severe drawbacks: for one thing, such things can only be done on visible (that is: opened) canvases. Addition- ally, programmatically fuddling with the user- interaction interferes heavily with normal (non- programmatical) user-interaction, esp. when the former does not happen in zero logical time but in a scheduled way. E.g. the mouse-pointer will move eratically, which makes live-patching Figure 1: Pro(cid:28)ling the [hungryobject] which unseemly hard. takes approx. 0.1ms to execute Another major drawback is, that messages like connect work on patch-local object indices, Due to the realtime nature of Pd, the main rather than on object-labels. E.g. (cid:16)connect 0 st st application for this kind of self-examination lies 0 2 0(cid:17) means (cid:16)connect the 1 outlet of the 1 st rd probablywithintherealmsofdebuggingandop- object to the 1 inlet of the 3 object(cid:17). timizingthepatchmanually, ratherthanchang- This only works if the indices are known be- ingthebehaviourofthepatchduringexecution. forehand. As soon as the user starts creating their own objects in the canvas in concurrency 2.1 Dynamic patching to the dynamic patching system, the indices be- A more powerful kind of meta-programming is comeunknowntothesystem,thusbreakingcon- available via (cid:16)dynamic patching(cid:17). nections. Pd itself builds up patches (e.g. on loading), by sending messages to the current canvas. E.g. 3 iemguts: Design and conventions the code snippet in Listing 1 is taken from a Pd iemguts is a library designed to expose internal patch containing a [bang( message connected (ontheC-code)informationandmethodsatthe to a [print] object. patch-level. For instance it helps to keep track of dynamically changing object indices. Listing 1: Pd-patch excerpt In a (cid:28)rst implementation, abstractions are 1 #X msg 53 92 bang; given the possibility to (cid:28)nd out information 2 #X obj 53 113 print ; about themselves, rather than about other ob- 3 #X connect 0 0 1 0; jectslivinginthesameinstanceofPd, following On loading, Pd will send each line of the a $self-like convention. patch-(cid:28)le to the current patch (which is im- The reasoning behind this is, that if only ab- plictelyboundtothereceive-label#X).Itisthus stractions are considered, it is su(cid:30)cient to have equivalent to the patch shown in Fig.2, where some sort of self-awareness. Information about the messages get sent to the subpatch that is neighbouring abstractions can be acquired by implicitely bound to the receive-label pd-sub. implementing a query-response system that al- A good introduction to dynamic patching is lows absractions to exchange information about given in the pd-msg tutorial [4] themselves. The scope of re(cid:29)ection is therefore limited to theabstraction,orthecanvascontainingtheab- straction (the canvas being the (cid:16)environment(cid:17) of the abstraction, which might be of interest to the object). In order to be able to encapsu- late refelction-logic into (cid:16)sub-abstractions(cid:17) (ab- Figure 4: GOP-abstraction of Fig.3 in action stractions within the re(cid:29)ected abstraction), the scope can also be shifted up towards the top- level patch. The user can thus change the behaviour of an By canvas we mean what is represented vi- object by dragging it around the canvas. (For a sually as a (cid:16)window(cid:17), no matter whether this simple example see Fig.3 and 4) window is an abstraction, a sub-patch or either For systems that want to use more au- of them contained in one of the two. tonomous agents, it is not only possible to get To sum up, the scope of re(cid:29)ection is always thecurrentpositionbutalsotosetit, e(cid:27)ectively restrictedtoacertaindepthwithinthepatchhi- allowing the object to move around by itself. erarchy, and to the speci(cid:28)ed canvas (with other canvases at the same depth being out of scope). 3.2 Modes of interaction A few objects do not comply with this con- Objects can interact in several ways: either vention, having a global scope. This is due to by using explicit connections, or by connecting the global property of the content they access. to an (invisible) bus transporting messages via Forinstance, sincePdloadsclassesintoaglobal [send]/[receive]. scope, an object that queries the existance of a In Live Coding contexts, the send/receive classwithinPdoperatesonthesamescopelevel. method can be suboptimal, as it can easily With respect to (semi-)automatically interac- be overlooked, especially when considering ob- tionitwasimportanttoallowtowriteagent-like jects that can automatically start communicat- objects that do not need a supervisor in order ingwitheachotherifsomeconditionismet(e.g. to interact. thetwoobjectsarewithinacertainminimaldis- tance). In this case, the explicit communication 3.1 Space-awareness via connections is more readable. Since Pd is a graphical language, the most im- However, in order to allow connection man- portant property of an object is probably its agement by a patch itself, it has to gain some position within a canvas. This property is usu- knowledge about the objects to connect: their ally not used within Pd’s syntax, with position object indices and which iolets are actually having no meaning to the structure of a patch available. (with the noteable exception of [inlet]s and [canvasindex] will return the object’s [outlet]s). Nevertheless it is one of the most own index within its parent patch, while obvious things when merely looking at a patch. [canvasconnections] will give information In order to be able to use this property, the ob- about the available iolets. ject [canvasposition] will return the current Since we always need two objects in order position of the object within its containing can- to successfully connect, this information is not vas. fully su(cid:30)cient. The proposed solution is to use a send/receive based query-system in order to 2 (cid:28)nd out these properties of a partner object. Once connections have been made, [canvasconnections] can also be used to query which (other) object connects to a certain iolet. 3.3 Sharing a local namespace Pd does not have a local namespace for labels, Figure 3: querying the position of the abstrac- like send/receive-names. However, it provides tion within its canvas 10 times per second 2One could see this as an implicit subconscious communication between the two objects. a mechanism for creating pseudo-local names- 3.5 Self destruction paces, using the unique (per abstraction) vari- While Pd has built-in capabilities for creating able $0. Hierarchies of patches can share this patches, it completely lacks of programmatical unique value by passing it as an argument. In a waystodestroypartsofthesepatches. Theonly Live Coding situation this is sub-optimal since waytoremovesomethingfromapatchistocom- the performer has to remember to add the $0 pletely clear a canvas, which is rather coarse to all relevant objects, and (cid:21) more important (cid:21) in terms of granularity. An especially annoying the audience (that is probably not literate with bug within the implementation of Pd will make respect to Pure data) has to decode yet another the interpreter crash if an objects triggers the awkward language construct in order to under- clearance of a canvas that contains this very ob- stand better what is going on. Therefore sev- ject. eral people have implemented objects that can Toavoidsuchcrashesandtoallowanabstrac- access the value of $0 of the parent canvas with- tion to safely remove itself from the patch, the out having to explicitly pass as an argument. [canvasdelete] object can be used. Banging For the sake of simplicity (and to avoid de- the [canvasdelete] object will simply delete pendencies), iemguts includes its own object the containing object, allowing for objects that [canvasdollarzero] which accomplishes this have only a limited time of existance. task. 3.6 Anybody here? A more theoretical object is [classtest], an 3.4 Reproduction objectthattestswhetherPdknowsaboutacer- A simple form of cloning an object in Pd can be tain class. E.g. it can be used to test whether considered creating an object of the same class (cid:16)xyz(cid:17) names an objectclass in Pd, simply by (that is: name) and with the same state (that sending the symbol (cid:16)xyz(cid:17) to the [classtest]- is: arguments). object: it will output (cid:16)1(cid:17) if the given symbol However, more often than not the internal names a registered class and (cid:16)0(cid:17) otherwise. state of an object will be modi(cid:28)ed at runtime This information can then be used e.g. to (through external messages): in this case the dynamically build up a patch depending on givenargumentsdonotre(cid:29)ecttheinternalstate whether a certain (external) object is present anymore. Creatinganotherinstanceoftheclass or not. withthesamegivenargumentswillre-createthe This object is noteably di(cid:27)erent from the object in its initial state rather than its current ones introduced before, as it is not concerned state. with the state of the abstraction it is living in, but rather with the overall state of the Pd- In order to make cloning of objects use the interpreter. current state, one can utilize Pd’s way of object duplication. Whenever an object get’s dumped 4 Future works to a (cid:28)le (e.g. when the patch is saved), or The current implementation is focused on self- copied to the clipboard (when the user pressed awareness of objects, and does not give access Ctrl/Apple-C) or when an object get’s du- to other objects. While this is no problem plicated (by pressing Ctrl/Apple-D), Pd will when dealing only with abstractions (where re- query the object for a (cid:16)save-string(cid:17). If we can (cid:29)ection capabilities can easily be added using dynamically modify this save-string, we can use the iemguts-objects), it does not allow the same it to re(cid:29)ect the current state of the object, au- level of access to unmodi(cid:28)able objects (e.g. in- tomatically making copies of the object to be ternals, like [metro]. clones (at the time of duplication). It is thus planned to add a second class of ob- This functionality is provided by jectsthatworkexclusivelyonpropertiesofother [canvasargs]. objects within a canvas, referenced by their ob- A similar object is [canvasname] which can ject index. For this to work properly an ad- be used to both query and modify an abstrac- ditional object for (cid:28)nding an object within the tion’s name. In conjunction with patch-saving canvas by its class. this allows to build a simplistic version-control 5 Conclusions system for patches: whenever an abstraction is duplicatedanewbranchiscreatedfromthecur- We presented iemguts, a library for Pure data rent state. that adds the possibility of metaprogramming techniques like re(cid:29)ection to abstractions. This canbeusedtoenhancethepatchingwork(cid:29)owby implementingpatching-helperslikemacrosorto create self-aware intelligent agents like systems, that interact with each other in ways currently not possible. The main focus during the development of objects has been on Live Coding environments, wherein this library has been successfully used for various performances. References [1] IOhannes m zm(cid:246)lnig and Gerhard Eckel. Live coding: an overview. In Proc. of the ICMC, Kopenhagen, 2007. [2] Stan Franklin and Art Graesser. Is it an agent, or just a program?: A taxon- omy for autonomous agents. In Proceed- ings of the Third International Workshop on Agent Theories, Architectures, and Lan- guages. Springer, 1996. [3] Fran(cid:231)ois-Nicola Demers and Jacques Malen- fant. Re(cid:29)ection in logic, functional and object-oriented programming: a short com- parative study. In Proc. of the IJCAI’95 Workshop on Re(cid:29)ection and Metalevel Ar- chitectures and their Applications in AI, page 29(cid:21)38, 1995. [4] Damien Henry. pd-msg tutorial, 2002.