Table Of Content

Self-regulating Supply-Demand Systems EvangelosPournarasa,MarkYaob,DirkHelbinga aProfessorshipofComputationalSocialScience ETHZurich,Zurich,Switzerland {epournaras,dhelbing}@ethz.ch bSmartEnergy IBMT.JWatsonResearchCenter YorktownHeights,NYUSA [email protected] Abstract 7 1Supply-demand systems in Smart City sectors such as energy, transportation, telecommunication and others, are subject of rev- 0olutionary technological transformations with implications for the effectiveness of traditional regulatory practices. Can existing 2regulatory actions capture the new dynamics and opportunities that Internet of Things technologies bring to supply-demand sys- n tems? Regulatorydecision-makingbygovernmentalofficers,policymakersorsystemoperatorsisusuallylong-term,operationally aoffline and top-down. Such decisions may turn out to be ineffective or may even come in conflict with automated, online and J bottom-up decisions nowadays performed, on behalf of people, by software agents embedded in the physical assets of modern 6supply-demand systems, e.g. Smart Grids. This paper contributes a generic decentralized self-regulatory framework, which, in 1 contrast to related work, is shaped around standardized concepts and Internet of Things technologies for an easier adoption and applicability. An evaluation methodology, integrated within this framework, is introduced that allows the systematic assessment ] Yof optimality and system constraints, resulting in more informative and meaningful comparisons of different self-regulatory set- Stings. Evidence using real-world datasets of energy supply-demand systems confirms the effectiveness and applicability of the .self-regulatory framework. It is shown that a higher informational diversity in the options, from which agents make local selec- s ctions, results in a higher system-wide response and savings under different regulatory scenarios. Several strategies with which [agentsmakeselectionscomealongwithstrikingmeasurabletrade-offsbetweenresponseandsavingscreatingavastpotentialfor 1 radicalonlineadjustmentsincentivizedbyutilities,systemoperatorsandpolicymakers. v Keywords: self-regulation,supply,demand,SmartCity,InternetofThings,SmartGrid,optimization,agent,tree 9 7 6 41. Introduction supply and demand in different regulatory scenarios by steer- 0 ingthepriceelasticityofdemandinafullydecentralizedfash- . Regulating supply-demand systems in techno-socio- 1 ion? (ii) How to continuously measure and evaluate this self- 0economic sectors of Smart Cities, such as energy, trans- regulatorycapabilityindifferentregulatoryscenariosofsupply- 7portation, telecommunication, financial markets and others, demand systems? Regulatory scenarios refer to the mitigation 1is a complex, yet critical endeavor for societal development of imbalances in supply-demand originated from varying gen- v:and sustainability. Regulation usually implies government- eration and/or consumption, system failures, system reforms, iimposed controls on business activities and infrastructural X etc. Thispaperstudiesthebalanceofsupplyanddemandwith operations [1], in which decison-making by the involved a fundamental, yet highly applicable approach by introducing ractors is often centralized and operationally offline. However, a a minimal and standardized information exchange. This ap- the introduction of Internet of Things technologies [2] in proach does not require major system interventions and can supply-demand systems suggests a new required type of makeaself-regulationmechanismapplicableinvariousSmart participatory regulation, designed to operate in an automated, Citysectors. online and distributed fashion as a result of the inter-temporal nature of supply and demand [3]. For example, the regulation How to balance supply-demand or how to design efficient of electricity prices used to be a result of institutional and computational markets remains a fascinating challenge for a legalpolicydesignsstrictlyimposedinatop-downfashion. In very broad range of scientific communities. Most remarkably, contrast,SmartGridtechnologiesenableregimeswithinwhich game-theoretical approaches in combinatorial games are rele- bottom-up price formation can be automated via real-time vant in systems in which agents have a finite number of op- pricing schemes and tariffs that continuously balance supply tions to choose from and each choice influences the evolution anddemand[4,5,6,7]. and outcome of the game. Although mathematical tools ex- This new type of online bottom-up self-regulation imposes ist for solutions to particular problems [8], there is no uni- twograndscientificchallenges: (i)Howtoeffectivelybalance fied theory or framework addressing combinatorial elements PreprintsubmittedtoFutureGenerationComputerSystems January18,2017 inthesegames. Thisobservationusuallycharacterizesmatch- awell-establishedstandardizedconceptandimplementedtech- ing mechanisms [9] and learning techniques that have at least nology in real-world supply-demand systems of Smart Cities. the following limitations in applied contexts: (i) long con- Table A.1 in Appendix A outlines all mathematical symbols vergence times [10, 11] that can make them inapplicable in usedinthispaperintheordertheyappear. practice due to technological limitations, e.g. communication bandwidth or computational capacity. (ii) Solutions have of- 2.1. Overview tenalimitedscope,meaningthatequilibriumisnotguaranteed Figure 1 introduces a framework for the self-regulation of with a minor problem modification or when adding/removing supply-demandsystemsinvarioustechno-socio-economicsec- aconstraint[12]. Forexample, theliteratureoncertaingame- torsofSmartCities. Eachphysicalassetofsupplyanddemand theoreticalapproachesinSmartGridsrevealsthelimitedappli- isrepresentedandmanagedbyasoftwareagent. Theagentfa- cabilityofcooperativegamesanddynamicmodelstotacklethe cilitatestheself-regulationlogicandrepresentssocialoroper- pervasivepresenceoftime-varyingparameterssuchasgenera- ationalrequirementssuchasthetype/amountofresourcessup- tionanddemand[12]. Moreover,itisalsoshownthatnetwork pliedanddemanded. delays may lead to incorrect price inquiries. A similar con- clusionissharedwithinotherrelatedwork[13]claimingthata Provision Evalua.on highlyresponsiveandnon-disruptiveregulatorysystemforload (5) UB requiresdirectcentralizedcontrolbyutilities[13]. - CoMnsetarna ints in Demand --   RSaevsipnogns se -  Mean & vola2lity -  Mean & vola2lity error This paper contributes a novel and generic self-regulatory framework for supply-demand systems. A single-commodity (4) TIS (4) iTFS (5) iTFS (5) eTFS resource-oriented equilibrium market [9] resembles the opera- (6) Metrics Incen.viza.on tionsoftheproposedself-regulatedsupply-demandsystemthat - TIS genera2on interacts with a loop of (i) incentive signals that communicate (1) iTFS supply costs and (ii) feedback signals that communicate the Supply (3) eTFS demand acquired. Agent-based bilateral interactions are per- Feedback formedinthebackgroundofabottom-upcomputationalmech- Loop (2) TIS Demand anismadvancedtooperateinthisgenericcontextofdistributed Self-reg ula.on . . . supply-demand systems. This mechanism self-regulates the EPOS balanceofsupplyanddemandinasimilarfashionwiththein- --   SPelalenc g2eonne fruan2co2no n . . . tuitiveparadigmofAdamsSmith’s‘invisiblehand’[15]. Reg- Figure1:Aself-regulatoryframeworkforsupply-demandsystems. ulation becomes an emergent system property [16], originated fromreconfigurableinteractionsandadaptivedecision-making Self-regulation aims at balancing supply and demand under structured over a self-organized tree topology. Earlier work various operational scenarios and constraints, such as keep- motivatestheadoptionoftreestructuresincomputationalmar- ing demand within a certain range, distributing demand fairly ketsandsupply-demandsystems[17,14,18],however,thepro- among consumers, or supplying a minimum amount of re- posedframeworkdoesnotexcludetheadoptionofothermech- sources to each consumer. Self-regulation is based on two anismsandstructures[19]. typesofinformationsignals: (i)incentivesignalsand(ii)feed- Evidenceusingreal-worlddatasetsofenergysupply-demand backsignals. Theincentivesignalisthepriceforeachunitof systems confirms the effectiveness and applicability of the resource supplied. The feedback signal is the amount of de- framework in a broad spectrum of regulatory scenarios. The mand requested based on the incentive signal provided. Self- implications of informational diversity in consumers’ options regulation is governed by a feedback loop: agents coordinate but also the actual consumers’ selections are studied by quan- demand in a fully decentralized fashion to collectively form a tifyingtheregulatorycapabilitywithtwometricsreferredtoas feedback signal as a response to a given incentive signal. A ‘response’ and ‘savings’. A novel characteristic of these met- new incentive signal is potentially formed to further steer an rics is that they are relative to different upper bounds. This improvedfeedbacksignal. allows the systematic evaluation of optimality and constraints resulting in more informative and meaningful comparisons of different self-regulatory settings. Findings show striking mea- 2.2. Transactivecontrol surabletrade-offsbetweenresponseandsavingscreatingavast This framework is far beyond conceptual. Regulating sup- potentialforradicalonlineadjustmentsincentivizedbyutilities, plyanddemandwithincentiveandfeedbacksignalsistheba- systemoperatorsandpolicymakers. sisoftransactivecontrol,awell-establishedstandardizedcon- cept and implemented technology in the real world [20, 21]. Theterm‘transactive’referstotheuseofeconomicandcontrol 2. Self-regulatoryFramework techniquestoimprovesystemreliabilityandefficiency. Trans- active control defines the minimal interaction types and infor- This section introduces a self-regulatory framework for mation exchange in the domain of Smart Grids. However, it supply-demand systems. It is shown how this framework can does not define any computational logic or formal methodol- berealizedasadecentralizedtransactivecontrolsystemthatis ogyfortheregulationofsupplyanddemand,ortheevaluation 2 oftheregulatorycapability. Theproposedframeworkfillsthis the result of incentivization performed based on the outcome gap by introducing a decentralized regulation mechanism and of previous operational cycles and the current system require- anevaluationmethodologythatmotivatesthefurtheradoption ments.Incentivizationisperformedbysystemoperators,policy of transactive control in other Smart City applications such as makersorevenautomatedsoftwaresystemsthatchoosehowto gasnetworks,waternetworks,trafficsystems,supplychainsys- optimizeandregulateasupply-demandsystem[22]. Theprice tems,etc. informationintheTISsignaliscomputedbyusinginformation Atransactivesignalisasequence1S =(s )T ofnumeri- about the available supply or constraints in the characteristics TS TS,t t=1 calvalues,wheres ∈R .Thispaperassumesthatallsignals offuturedemand,e.g. themeanand/orvolatility. TS,t >0 aretimeseries,andtherefore,T isafuturetimehorizon. How- In the third signaling, depicted by Arrow 3 in Figure 1, an ever, there is no limitation to other signal types applicable to eTFSsignalisprovidedbydemandtosupplyasaresponsetothe severalcomputationalproblems[11,10]. Asequenceprovides receivedTISsignal. TheeTFSsignaliscomputedbytheadopted a computational degree of freedom in self-regulation: in the self-regulation mechanism. At this stage, supply has all in- caseoftimeseries,temporaladjustmentsscheduledinthetrans- formation to provision and evaluate the regulatory capability activesignalscanbeusedasregulatorytools,e.g. load-shifting achievedinthisoperationalcycleasoutlinesbelow. inpowergrids, load-balancingbetweenvehiclesintransporta- In the fourth signaling, depicted by Arrow 4 in Figure 1, a tionsystems,etc. hypotheticaloptimumTFSsignaliscomputed,theupperbound, Thetypesoftransactivesignalsaredefinedasfollows: giventheTISandiTFSsignals,alongwithconstraintsincharac- teristicsofdemand. Suchconstraintsmayconcernthecompu- • Transactive Incentive Signal (TIS): A time series with the tation of an upper bound signal with the same mean or, mean priceincurrencyperunitofresourcessupplied. andvolatility,withtheiTFSsignal. In the fifth signaling, depicted by Arrow 5 in Figure 1, the • TransactiveFeedbackSignal(TFS):Atimeserieswiththe evaluationoftheeTFSsignalistriggeredinthelightoftheiTFS unitsofresourcessuppliedordemanded. Thispaperaddi- signal and the UB signal. Evaluation measures the response, tionallydistinguishesthefollowingTFSsignals: savingsandthemean/volatilityerror. – InelasticTransactiveFeedbackSignal(iTFS):Anon- Finally, in the sixth signaling, depicted by Arrow 6 in Fig- regulatedhistoricorforecastTFSsignal. ure1,anoperationalcyclecompletesbyprovidingthevaluesof themeasuredmetricsasinputtoincentivization.Thenextoper- – Elastic Transactive Feedback Signal (eTFS): A regu- ationalcycleinitiatesthegenerationofanimprovedTISsignals latedTFSsignal. that can potentially achieve a higher response and savings but – UpperBoundSignal(UB):AreferenceTFSsignalfor lowermeanandvolatilityerror. theevaluationofself-regulation. The rest of this paper illustrates the self-regulation frame- workindetail. All interactions within the proposed framework are entirely basedontheabovetransactivesignalsasshowninFigure1. 3. Incentivization 2.3. Operationallifecycle Incentivization concerns the construction of TIS signals that Theself-regulatoryframeworkofFigure1performsthefol- accurately reflect the costs of supply, costs of infrastructure, lowing four operations: (i) incentivization, (ii) self-regulation, operationalconstraintsandcertainregulatoryactionsrequired. (iii) provision and (iv) evaluation. Each of these operations For instance, when a power generator fails, costs increase as may be performed at either supply or demand-side depending theavailablesupplyislower, infrastructuremayrequiremain- on how a supply-demand system is institutionalized. For con- tenance and complex operational constraints should be met, sistency and an intuitive understanding of the framework, the i.e. ramp up times for the activation of operational power re- rest of this paper positions the operation of self-regulation at serves[22]. Moreover,systemoperatorsandutilitiesmayiniti- demand-sideandallotheroperationsatsupply-side. Sixtrans- atecostlydemand-responseeventssothatconsumerslowerde- activesignalingsareperformedtocompleteanoperationalcy- mand. Thecostsofalltheseactionssumupandformtheprices cle. of a TIS signal. Forecasting is a critical component of incen- Inthefirstsignaling,depictedbyArrow1inFigure1,aniTFS tivization as cost information is not always available [23, 24]. signalisprovidedbydemandtosupply. Thissignalisusedasa For example, the availability of energy supply generated by historicorforecastbaseinformationbytheotheroperationsof renewables is highly stochastic. Therefore the accuracy of theframework. weatherforecastingtechniquesinfluencetheeffectivenessofa In the second signaling, depicted by Arrow 2 in Figure 1, a TISsignal. TISsignalisprovidedbysupplytodemand. ATISsignalmaybe These observations suggest that incentivization is a highly domain-dependent process. Implementations of the proposed self-regulatoryframeworkcanmakeuseofexistingtechniques 1Thispaperassumespositivesignalsforsimplifyingtheillustrationofthe adoptedbyvariousapplicationdomainsforconstructingTISsig- frameworkandresults. Negativesignalsarepossibleaswellandcorrespond nals. However, the framework introduces the option to con- tosupply-demandsystemsinwhichanagentmayactasboth,consumerand producer. structincentivesignalsinanevolutionaryfashion. Themetrics 3 measured at the stage of evaluation can be used as feedback UpperBound1. GivenaTISsignalSTISandaniTFSsignalSiTFS, informationtoconstructmoreeffectiveTISsignals,whereeffec- anupperboundS isgivenasfollows: UB1 tivenessismeasuredbytheproposedmetrics. Inotherwords, a weighting scheme of the costs or an elasticity function [25] µ(S ) based on the metrics of evaluation can be used to compute a s = iTFS n(t,S ),∀t∈[1,T] new incentive signal to improve the self-regulatory capability UB1,t µ(STIS) TIS (2) of a supply-demand system. This paper focuses on evaluating subjecttoµ(SUB1)=µ(SiTFS) asingleoperationalcyclebyusingreal-worldTISsignalscom- Proof. The average cost of one demand unit during the time putedviatoolkitfunctionsbasedonadvancedpredictiveanalyt- horizon T is given by µ(S ). Subject to µ(S ) = µ(S ), ics.Section7illustratestheTISsignalsstudiedinthispaper.The TIS UB1 iTFS the average demand acquired for one cost unit of demand is evaluation of an evolutionary operation of the self-regulatory µ(S )/µ(S ).Then,byproportionality,forn(t,S )costunits frameworkispartoffuturework. iTFS TIS TIS ofdemand, µ(SiTFS)n(t,S )unitsofdemandareacquiredforthe µ(STIS) TIS upperbounds . UB1,t 4. Provision UpperBound2. GivenaTISsignalSTISandaniTFSsignalSiTFS, anupperboundS isgivenasfollows: UB2 This section shows how to construct hypothetical eTFS signals representing an optimally regulated supply-demand given thhyepiontcheentitcivaelssiiggnnaallsSarTeISraenfderaresdettooafsduemppaenrdbcoounnsdtsra.iTnhtse.iTnhceense- sUB2,t =σ(SiTFS)r(t,SσTIS()S−µ)(STIS) +µ(SiTFS),∀t∈[1,T] (3) TIS tive signal is assumed to have a reversed linear2 relation with subjecttoµ(S )=µ(S )andσ(S )=σ(S ) UB2 iTFS UB2 iTFS thefeedbacksignal,meaningthatahigherpriceinsupplyshall result in lower demand. In other words, the incentive signal Proof. Let r(t,S ) − µ(S ) represent the price difference at TIS TIS seemsmoreappropriatetoconstructalowerboundsignal. One timetfromtheaveragepriceµ(S ). Thisdifference,measured TIS waytoovercomethismathematicalhurdleistocreateareflec- in units of price, can be measured in units of price volatility tionoftheincentivesignalS asfollows: oftheTISsignalas r(t,STIS)−µ(STIS). Thedemanddifferencefrom TIS σ(STIS) the average demand µ(S ) at time t can be constructed using TIS r(t,STIS)=2µ(STIS)−sTIS,t (1) tσh(eSdem)a=ndσv(Solati)l.itGyiσve(SnitThFSa)taµs(σS(Si)TF=S)rµ(t(,SSTσIS(S)−T)Iµ,S()tShTeIS)fisnuabljuepctpteor UB2 iTFS UB2 iTFS where the mean of the incentive signal µ(S ) is used as the bound signal is constructed by summing the average demand hyperplanefortheflip. TheisometryoftherTeISflectionr(t,STIS) µ(SiTFS)asσ(SiTFS)r(t,STσIS(S)−TIµS()STIS) +µ(SiTFS3). and the selection of the mean µ(S ) as hyperplane result in a TIS price signal with the same total cost for one unit of resources Althoughthesehypotheticalupperboundsareconstrainedto compared to the initial incentive signal S . An undesired ef- TIS fectofareflectionisthatitmayresultinasignalwithnegative aggregatedemandcharacteristicsoftheiTFSsignal,othermore complex operational constraints can be captured as well, de- values. AnormalizationisillustratedinAppendix Bthatguar- pending on the application domain considered. For example, anteesasignaln(t,S )withpositivevalues. Itisusedforthe TIS inSmartGrids,operationalconstraintsoftheindividualhouse- computationoftheUB1. hold devices or the availability of renewable energy resources Thispaperintroducestwoupperboundsdistinguishedbytwo canbeconsidered[26,27,28]. demandconstraints: • Themeanµ(SUB)oftheUBsignalequalsthemeanµ(SiTFS) 5. Evaluation of the iTFS signal. This constraint keeps the total amount of demand equal to the one of the iTFS signal, however This paper studies three factors that measure the effective- demand is distributed more effectively as defined by the ness of self-regulation: (i) response, (ii) savings and (iii) reflectediTFSsignalr(t,STIS). mean/volatilityerror. GivenaninitialfeedbacksignalSiTFS and afeedbacksignalS computedbyaregulationmechanismaf- eTFS • The mean µ(S ) and volatility σ(S ) of the UB signal ter applying an incentive signal S , response and savings are UB UB TIS equal the mean µ(SiTFS) and volatility σ(SiTFS) of the iTFS computedasfollows: signal. Thisconstraint keepsthetotalamount ofdemand equaltotheoneoftheiTFSsignaland,additionally,itlimits R= (cid:80)Tt=1|siTFS,t−seTFS,t| (4) thestandarddeviationofthevaluestotheleveloftheiTFS (cid:80)T |s −s | t=1 iTFS,t UB,t signal. (cid:80)T s ∗s −(cid:80)T s ∗s P= t=1 TIS,t iTFS,t t=1 TIS,t eTFS,t (5) (cid:80)T s ∗s −(cid:80)T s ∗s 2Thisstrongassumptioncanbeovercomewithempiricaldataordomain- t=1 TIS,t iTFS,t t=1 TIS,t UB,t specificmodelsofsupply-demandsystems. whereS isanupperboundsignal. UB 4 Response measures the distance |s − s | of the two Thispaperfillsthisgapbyintroducingtwonewselectionfunc- iTFS,t eTFS,t feedbacksignalsS andS inrelationtothemaximumdis- tions without any change in the algorithm and architecture of iTFS eTFS tancethatthesetwosignalscanhypotheticallyhave. Thismax- EPOS. imum distance is computed using the upper bound signal as TheplansofEPOSandthetransactivesignalsarebothmath- |s −s |. ematicallyequivalentsequences. Thismakesatransactivecon- iTFS,t UB,t Savingspayoffmeasuresthecostreductionbetweenthefeed- trolsystemfullyinteroperablewiththeEPOSmechanism. The back signals SiTFS and SeTFS in relation to the maximum hypo- supply-sideprovidestoEPOSaTISsignal3andreceivesbackan thetical cost reduction between the feedback signal SeTFS and eTFSsignalcomputedinEPOSasfollows: the upper bound signal S . Cost is computed by multiplying UB (cid:88)n theincentivesignal(price)withthefeedbacksignal(quantity), s = d (7) for example, s ∗ s provides the cost of s amount of eTFS,t i,s,t TIS,t eTFS,t eTFS,t i=1 resourceswithprices attimet. TIS,t wherenisthetotalnumberofagentsconnectedinthetreeover- An upper bound S is required to compute both response UB laynetworkofEPOSandd istheselecteddemandofagenti andsavings. Withthesetwometricsbasedonanupperbound, i,s,t attimet. different self-regulation strategies can be meaningfully com- Plan generation computes the possible plans. Each one can pared. Forexample,aself-regulationstrategy‘A’maydecrease beexecutedbytheagentlocallyifselected. Thepossibleplans thedemandcost5%morethananotherstrategy’B’.Thisfind- represent a degree of freedom in the operation of the physical ingisnotsoinformativecomparedtoknowingamaximumpos- assetsthattheagentregulates. Therefore,thephysicalcharac- sible cost reduction of demand that indicates the actual effec- teristics and the parameterization of these physical assets can tivenessofthesestrategiescomparedtoahypotheticaloptimal makethecomputationofpossibleplansfeasibleinvariousap- case. This evaluation methodology is especially useful in de- plication domains. For example, possible plans can be opera- centralized self-regulation mechanisms that capture objectives tionallyequivalent,andthereforealternatives,whenhousehold withoptimumsolutionsbeingcomputationallyintractable. appliancespermitthis[29,13]. Moreover, possibleplansmay Thecapabilityofaregulatorymechanismtosatisfythecon- represent social preferences, such as comfort levels [32? ] or straintsoftheupperboundscanbemeasuredbythemeanand even recommendations extracted from historic data [30]. This volatilityerrorsasfollows: paperstudiestheroleofinformationaldiversityinthepossible planswiththreegenerationschemesthatcanbeappliedinsev- µ(S ) σ(S ) (cid:15)µ =|1− µ(SeTFS)|,(cid:15)σ =|1− σ(SeTFS)|, (6) eral application domains and scenarios: (i) SHUFFLE, (ii) SHIFT iTFS iTFS and (iii) SWAP. In SHUFFLE, each possible plan is a random per- where(cid:15)µ istherelativeerrorofthemeanintheeTFSsignalrel- mutationoftheotherones. Thisschemerepresentsscenariosin ative to the mean of the iTFS signal, and (cid:15)σ is the respective whichagentshaveasignificantcontrollabilityintheoperation relativeerrorofdemandvolatility. oftheregulatedassets. InSHIFT,thevaluesofaplanarerotated bypushingvaluescertaintimestepsaheadintime.Thisscheme represents load-shifting scenarios or future resource realloca- 6. Self-regulation tion and rescheduling. In SWAP, a number of values in a plan interchange their positions. This scheme represents scenarios in which scheduled operations can be interchanged within the This paper advances the EPOS mechanism [29] to employ futurehorizon. it as a highly generic self-regulation mechanism for supply- Each scheme generates plans that contain the same values demand systems. EPOS is earlier studied in the Smart Grid and,therefore,theyhavethesamemean,volatilityandentropy domainasafullydecentralizedmechanismtoimprovethero- that is useful for the interpretation of the measurements per- bustnessofSmartgridsbystabilizingenergydemand,lowering formed. Inthisway,thegenerationschemesarenothighlydis- power peaks and other regulatory actions. Loads, represent- ruptiveforconsumers[13]anddonotinfluencethetotalamount ing electrical household devices or even the consumption of a ofsupplyscheduledwithintheperiodT. Moreover,theelastic- household as a whole [30], are regulated by software agents ity of volatility is exclusively governed by EPOS and not by self-organized [31] in a tree topology for structuring their interactions. A tree overlay network provides a cost-effective the generation schemes that would bias the measurements of responseandsavings. aggregation of the demand level required for coordinating the Theinformationaldiversityis definedbytheEuclideandis- decision-making: each agent generates p number of possible plansD =(d )p andcollectivelyselectsthepland thatfits tance between the positions of the demand values after apply- i i,j j=1 i,s ing a generation scheme. Let d being a seed possible plan besttoanobjective. Decision-makingisperformedwithselec- i,1 and d the diversified possible plan with one of the proposed tion functions [30] that evaluate characteristics of the demand i,2 generation schemes. The informational diversity is measured plan,e.g. howuniformlydistributeddemandisovertime,how largethedemandpeaksare,etc. ThemainlimitationofEPOSisthatitdoesnotinteractwith 3TheTISsignalcanbedistributedtoallagentsofthetreeoverlaynetwork supply and, therefore, it cannot be used for balancing supply viaapplication-levelbroadcastingsupportedinEPOSfordistributingtheglobal and demand under a broader range of operational scenarios. plan[29]. 5 as (cid:80)T |t−tˆ|, where t refers to the time point of demand d t=1 i,1,t and tˆ=d(t) refers to the new time point of demand di,2,tˆ after (cid:115) diversificationwithafunctiond(t)suchthatdi,1,t ≡di,2,tˆ. s=argpmk in (cid:80)Tt=1(ci,j,t−sUB,t)2 Figure 2 illustrates the probability density functions of in- j=1 T (9) formational diversity for the five generation schemes4. SHUF- subjecttos =c FLEhasthehighestinformationaldiversityfollowedbySHIFT(20). iTFS,t i,j,t SHIFT(10),SWAP(30)followverycloselyandSWAP(15)hasthelowest This relation gives the combinational plan, and therefore the diversity. selectedplanofeachchild,withtheminimumrootmeansquare error. Thisisthecombinationalplanwiththehighestmatching �� to the upper bound. The upper bound is recomputed for each �� combinational plan given that s = c . This condition is �� necessary as it makes collectiveiTdFSe,tcisioni-,jm,taking possible6. It actually means that when a number of agents have a selected �� pthlaenr,esmtoeafnthinegagtheentasgneenetdstionbthinedbtrhaenirchdeucnisdieornn-ematahkienagchtocthhielsde, existingchoices. �� ThethirdselectionfunctionMIN-COSTisdevisedtominimize thecostofdemand: �� pk (cid:88)T s=argmin s ∗c (10) Figure2: Probabilitydensityfunctionofinformationaldiversityforthefive TIS,t i,j,t generationschemes. j=1 t=1 where the price of supply sTIS,t in the TIS signal multiplied by Decision-makingisperformedbottom-upandlevel-by-level: the planned demand ci,j,t in the combinational plan at time t children interact with their parents and collectively decide providestherespectivecostofdemand. whichplantoexecutebasedon(i)theirowndemandplansand The generation schemes, selection functions and number of (ii) the selected plans of the agents connected to each of their possible plans are all configurations that influence the perfor- branchunderneath. Selectioniscomputationallyperformedby manceofself-regulations. Theycanbeincentivizedbysupply- theparentofthechildrenthatcomputesthecombinationalplans demandserviceprovidersviasomepayoff. asfollows: 7. ExperimentalFrameworkEvaluation Ci =(ci,j)pj=k1 =b(Au,...,Au+k) (8) Thissectionillustratestheapplicabilityandevaluationofthe where the parent agent i performs a brute force operation self-regulatoryframeworkinfourscenariosofbalancingenergy b(Au,...,Au+k) to compute all combinations between the se- supplyanddemand: (i)rampdown,(ii)generationfailure,(iii) quences of aggregates plans Au,...,Au+k received by its k chil- maximum entropy and (iv) minimum entropy. Each regulatory dren. Thechilduofagentiisassumedtobethefirstoneand scenario corresponds to a TIS signal of size 144 as standard- thechildu+kthelastone. Anaggregateplana ∈ A iscom- ized in transactive control for Smart Grids7 [33, 21]. Eval- u,j u putedbysummingthedemandpland andallselectedplans uation is based on both simulated and real-world Smart Grid u,j oftheagentsreceivedthroughthebranchunderneathchildu. datasources: (i)thesimulatedzonalpowertransmissioninthe Three selection functions are introduced with the following PacificNorthwest8 (SIM),(ii)theE´lectricitee´ deFrance9 utility self-regulation objective: match the actual demand to any de- (EDF)and(iii)thePacificNorthwestSmartGridDemonstration sireddemandcurve. Thisisanespeciallychallengingobjective Project8(PNW). tofulfillgiventhatdecision-makingisperformedinafullyde- A ramp down scenario concerns a situation in which power centralizedfashion. Thethreeselectionfunctionsusedifferent supply suddenly drops due to, for example, a low availability mathematical approaches to achieve the same self-regulation of renewable energy resources, e.g., wind generation. In this objective. ThefirsttwoselectionfunctionsMIN-RMSE-UB1andMIN-RMSE- 6Severalotherversionsofselectionfunctionsareexperimentallyevaluated, UB2aredevisedtomaximizeresponse5. Theycomputetheroot however,itisthisfunctionthatresultsineTFSsignalswithimprovedresponse meansquareerrorbetweenthecombinationalplansandoneof andsavings. theupperboundsillustratedinSection4: 7Thiscorrespondstoa72hoursforecastingofvariablegranularitywithina marketoperatingevery5minutes. 8Available upon request at http://www.pnwsmartgrid.org/ participants.asp(lastaccessed:November2015) 4ThetestisimplementedinJavausingrandomizationmethodsofJDK1.8. 9Available at https://archive.ics.uci.edu/ml/datasets/ 5Theyarealsoexpectedtodecreasethecostgiventhatareflectionofthe Individual+household+electric+power+consumption (last accessed: TISsignalisusedforthecomputationoftheupperbounds. November2015) 6 case, operating reserves with high operational costs are acti- Thethreefollowingsettingsvaryineachexperiment:(i)gener- vated such as backup diesel generators. Figure 3a shows that ationscheme,(ii)selectionfunctionand(iii)dataset. EacheTFS during the time period [40,80] the generation costs in the TIS signalisgeneratedbyaprocesspipelineoffourstagesasillus- signaloftheSIMdatasetdoublesreflectingthelowavailability tratedinFigure4. Thefirststageinthispipeline,thedisaggre- ofcheaprenewableenergyresourcesduringthisperiod. gation, is applied in the SIM and EDF datasets that only contain the aggregate base load of all residential consumers. In con- �� trast,thePNWdatasetcontainsthebaseloadforeachindividual �� household. �� 1( Planning( 1( 2( …( p( �� (1)$ 2( Planning( 1( 2( …( p( �� ...((( ...((( iTFS( �� n( Planning( 1( 2( …( p( (2)$ �� Evalua2on( Disaggregated( Possible(plans( �� base(load( (a) TIS signals from the SIM (b) TIS signals from the PNW eTFS( 2(1...((((p( (3)$ dataset. dataset. ...((( ...((( (4)$ 2(1(p( .(.(.( 2(1(p( Figure3: TheTISsignalscorrespondingtothefourregulatoryscenarios: (i) ...((( ...((( rampdown,(ii)generationfailure,(iii)minimumentropyand(iv)maximum 2(1(p( .(.(.( 2(1(p( entropy. Selected(plans(by(EPOS( The generation failure scenario concerns the unexpected Figure4: TheexperimentalsetupforgeneratinganeTFSsignal: (1)Theag- damageofanoperationalpowergeneratoratthesametimein gregatedbaseloadfromdifferentreal-worldSmartGriddemonstrationprojects which the ramp down scenario occurs as well. Further expen- isdisaggregatedtonindividualhouseholdbaseloads. (2)Anagentineach householdgeneratespnumberofalternativedemandplansbasedonagenera- siveoperatingreservesneedtobeactivatedfollowedupbyur- tionschemethatreceivesasinputthebaseloadofahousehold.(3)Theagents gent repair costs to meet demand. This scenario is illustrated areself-organizedinatreetopologyoverwhichthedecentralizedalgorithmof inFigure3awiththeTISsignalfromtheSIMdatasetthatshows EPOSrunstoselectthedemandplanforexecution. (4)Theagentsperforma the increase in the price of generation during the time period treeaggregationthatcomputestheeTFSsignal. [60,80]. Each regulatory scenario is applied in two project sites13 Theminimumentropyscenarioconcernsasituationinwhich thatbelonginasingletransmissionzone14. Theloadsineach the availability of power generation is highly dispersed over project site are regulated by the EPOS agents that are self- time. This dispersion is measured with the entropy of Shan- organized in an overlay network with a 3-ary balanced tree non’s information theory [34]. This scenario assumes that a topology. Self-organization is performed with the AETOS low entropy in the incentive signal corresponds to a high dis- overlay service [31]. Each EPOS agent generates 4 possible persion in power generation. Therefore, matching demand to the available supply is more challenging. The opposite holds plansusingthefollowinggenerationschemes: (i)SHUFFLE,(ii) for the maximum-entropy scenario that assumes a low disper- SHIFT(10), (iii) SHIFT(20), (iv) SWAP(15) and (v) SWAP(30). Plan se- sioninthepowergenerationovertime. Whendemandisregu- lection is performed with the MIN-RMSE-UB1, MIN-RMSE-UB2 and lated to have a maximum entropy, the required adjustments in MIN-COST selection functions of EPOS. The iTFS signal of the thegenerationareminimizedasdemandismoreuniformlydis- SIM dataset is disaggregated in 5600 and 2374 loads for each projectsiterespectively,emulatinginthiswaytheloadsofthe tributedcomparedtotheminimumentropyscenario. Figure3b ‘Flathead, Libby’and‘Flathead, Haskill’projectsites. Disag- illustratesaseriesofconsecutiveTISsignalsusedfromthePNW dataset10. Anexhaustiveheuristicalgorithmisappliedoverall gregation is performed as illustrated in Appendix C with pa- consecutiveTISsignalsofsize144todiscovertheoneswiththe rameter ε = 0.20. The iTFS signal of the EDF dataset is disaggregated, as illustrated in Appendix C, in 724 loads for each minimumandmaximumentropyasshowninthisfigure. Experimental evaluation is performed for each regulatory projectsiterespectively. ThePNWdatasetcontainstheresiden- tialbaseloadofapproximately1000residentialconsumerson scenariobymeasuringandcomparingtheresponse,savingsand volatilityerror11oftheeTFSsignalsusingtheupperboundUB212. the date 23.07.2014. It is split in two subsets referred to as PNW-MORNINGandPNW-EVENING.TheiTFSsignalforPNW-MORNING concerns the duration 01:00-13:00, whereas the iTFS for PNW- 10TheTISsignalsconcernthetransmissionzone12atprojectsite14forthe EVENINGconcernstheduration11:00-23:00. periodstartingon30/07/13at09:45to06/08/13at09:30. 11Themeanerrorisnotillustratedasthepossibleplansforeachagenthave thesamemeanandtherefore,foranyselectionperformed,µ(SeTFS)=µ(SiTFS) resultsarenotshowninthispapergiventhespacelimitationsandthefactthat is satisfied. Experimental evaluation confirms an average (cid:15)µ = 0.09 for all theyindicatesimilarfindings. performedexperiments. 13Theyareenumeratedasprojectsite12and14. 12ResponseandsavingsarealsomeasuredwiththeupperboundUB1.These 14Itisenumeratedastransmissionzone14 7 7.1. Response,savingsandvolatilityerror of14.75%inPNW-MORNING.Theinfluenceofadatasettotheper- Figure 5 illustrates the performance of the self-regulatory formanceconcernsthecomplexityoftheiTFSsignalincompar- framework in the light of four studied aspects: (i) generation isontoUB2. Forexample,whenbothsignalsaremonotonously schemes, (ii) selection functions, (iii) datasets and (iv) regu- increasing,theymayrequirealowerdegreeofadjustmentcom- latory scenarios. Performance is evaluated by averaging the paredtothecaseinwhichtheoneismonotonouslyincreasing response, savings and volatility error in each experiment that andthesecondmonotonouslydecreasing15. Thecomplexityof involvesthechoiceofagenerationscheme,selectionfunction, thesignalsineachdatasetisshowninthefiguresofSection7.2. dataset and regulatory scenario. Figure D.13, D.14 and D.15 Thehighestvolatilityerrorof40.37%isobservedinSIMandthe inAppendix Dillustratetheindividualresultsineachexperi- lowestoneof18.82%inPNW-EVENING. mentperformed. Figure5dillustratestheresponseandsavingsunderthefour differentregulatoryscenarios. Theminimumentropyscenario has the highest response of 44.14%, whereas generation fail- �� urehasthelowestresponseof31.05%. Themaximumentropy �� hasthehighestsavingsof40.22%,incontrasttothegeneration failurescenariowiththelowestsavingsof17.51%.Thehighest �� volatilityerroris31.61%formaximumentropyandthelowest �� oneis23.55%forgenerationfailure. Figure 6 visualizes the agent selections under the genera- �� tion failure scenario for the PNW-MORNING dataset. Figure 6a-c �� providequalitativecomparisonsabouttheselectionmadewith �� MFLIEN.-RAMgSeEn-UtBs2aadnodptMinINg-CthOeSTSuHsUiFnFgLEthgeegneenraetriaotniosncshcehmemeeunodfeSrHtUhFe- (a) GenerationSchemes (b) SelectionFunctions MIN-RMSE-UB2 and MIN-COST selections functions make 50.91% different selections. Figure 6d-f show selections made with �� MIN-RMSE-UB2,thoughusingthegenerationschemesSWAP(15)and �� SWAP(30). Agents adopting the MIN-RMSE-UB2 selection function under the SWAP(15) and SWAP30 generation schemes make 67% �� differentselections. Finally, Figure6g-ishowselectionsmade �� withMIN-COST,thoughusingthegenerationschemesSWAP(15)and SWAP(30). AgentsadoptingtheMIN-COSTselectionfunctionunder �� theSWAP(15)andSWAP30generationschemesmake64.59%differentselections. �� volGativileitny ethrreorp,earfpolramusainbcleeqrueessutlitosnotfharteasrpiosenssei,s isfatvhiensgesmaentd- rics are correlated in practice. Intuitively, a high response can (c) Datasets (d) RegulatoryScenarios be incentivized via reducing the demand cost. Similarly, high savingsrequireasignificantresponse. Finallyahighresponse Figure5: Response,savingsandvolatilityerrorineachstudiedaspectofthe self-regulatoryframework. can originate from adjustment in demand volatility. How correlated are these metrics in the empirical findings illustrated? Figure5ashowsthatSHUFFLEandSHIFT(20)havethehighestre- Which system parameter can be used to make trade-offs be- sponseof60.06%and38.02%respectively.Theseschemesalso tweenresponseandsavings,whileminimizingvolatilityerror? have the highest savings of 47.63% and 32.67%. In contrast, The total number of experiments performed are a popula- thelowestperformanceisobservedinSWAP(15)with18.01%re- tiononwhichthecorrelationbetweenresponse-savings,error- sponseand19.3%savings. SHUFFLEandSWAP(30)causethehigh- response and error-savings can be measured. In this case, the est volatility error of 53.72% and 28.15%, while SHIFT(10) the correlation coefficient is 0.3, 0.44 and -0.08. However, not all lowest one of 12.32%. These results confirm that a higher in- experimentsvarythesameparameter. Fourdifferentaspectsas formationaldiversityresultsinhigherperformance. studiedasshowninFigure5. Forthisreason,itismeaningful Figure5bshowsthattheMIN-RMSE-UB1selectionfunctionhas torestrictthecorrelationmeasurementsforeachstudiedaspect. thehighestresponseof38.12%,thoughMIN-RMSE-UB2alsohasa Figure 7 illustrates the correlation coefficient for each stud- significant response of 38.09%. In contrast, MIN-COST achieves iedaspectoftheself-regulatoryframework. Thethreemetrics thehighestsavingsof56.67%andMIN-RMSE-UB1thelowestsav- appearhighlycorrelatedwhenmeasurementsareperformedfor ings of 7.99%. The highest volatility error is 29.79% by MIN- the experiments that vary the generation scheme and regula- RMSE-UB1andthelowestoneis18.2%byMIN-COST. tory scenario. For example, the correlation range [0.69,0.93] Figure 5c illustrates that the highest response of 48.06% is indicatesthatresponse,savingsandvolatilityerrorarelikelyto observedinthePNW-EVENINGdatasetandthelowestresponseof 28.02%atthePNW-MORNINGdataset. Similarly,thehighestsav- 15Forexample, Figure11dshowsaniTFSandanUB2withaverysimilar ingsof52.31%areobservedinPNW-EVENINGandthelowestones complexity. 8 1 2 3 4 1 2 3 4 SelectedPlan SelectedPlan DifferentSelection SameSelection (a) SHUFFLE,MIN-RMSE-UB2 (b) SHUFFLE,MIN-COST (c) SHUFFLE,MIN-RMSE-UB2vs.MIN-COST 1 2 3 4 1 2 3 4 SelectedPlan SelectedPlan DifferentSelection SameSelection (d) SWAP(15),MIN-RMSE-UB2 (e) SWAP(30),MIN-RMSE-UB2 (f) SWAP(15)vs.SWAP(30),MIN-RMSE-UB2 1 2 3 4 1 2 3 4 SelectedPlan SelectedPlan DifferentSelection SameSelection (g) SWAP(15),MIN-COST (h) SWAP(30),MIN-COST (i) SWAP(15)vs.SWAP(30),MIN-COST Figure6:AgentselectionsunderthegenerationfailurescenarioforthePNW-MORNINGdataset. increaseordecreasesimultaneouslybyonlychangingthegen- error-responsehaveacorrelationof0.45anderror-savingshave eration scheme. This is a desired property of the framework asignificantnegativecorrelationof-0.995. asthechoiceofthegenerationschemeandregulatoryscenario The correlation results under different selection functions shouldnotgovernthedynamicsbetweenthethreemetrics. Un- der different datasets, response-savings have a correlation of have two striking implications. The first implication is that 0.56,error-responsehaveacorrelationof0.29anderror-savings thedesign ofselectionfunctions isempiricallyvalidated: MIN- haveanegativecorrelationof-0.33. Finally,underdifferentse- COST maximizes the savings while MIN-RMSE-UB1 and MIN-RMSE- lectionfunctions,response-savingshaveacorrelationof-0.44, UB2 maximize the response. Trade-offs between response and savingscanbeperformedbychoosingtherespectiveselection 9 �� Figure7:Correlationcoefficientbetweenresponse,savingsandvolatilityerror (a) SIM,SHIFT(20),MIN-COST (b) EDF, SHUFFLE, MIN-RMSE- foreachstudiedaspectoftheself-regulatoryframework. UB2 �� function. In practice, these trade-offs can be incentivized by �� systemoperatorsorpolicymakers[35,36,37]: eachconsumer �� of the supplied resources may change its selection function in �� ewxhcehnanthgeeovfolaatmiloitnyeetarrryorpsahyoouffl.dTrhemesaeincosntdricimtlyplmiciantiimonali,sitthaist �� more likely for the self-regulatory system to increase the sav- �� ingsratherthantheresponse. �� 7.2. Signalswithmaximumresponse �� (c) PNW-MORNING, SHUFFLE, (d) PNW-EVENING, SHIFT(20), This section illustrates the resulting eTFS signals with the MIN-RMSE-UB2 MIN-COST maximum response in each dataset and regulatory scenario. TheeTFSsignaliscomparedtoiTFSandUB2. Figure8:eTFSsignalswithmaximumresponseundertherampdownscenario. Forcomparison,theiTFSandUB2signalsareshown. Figure8illustratestherampdownscenario. Themaximum responseis96.61%,55.98%,59.38%and60.73%forSIM,EDF, PNW-MORNINGandPNW-EVENINGrespectively.Itisshownthatdur- 31.29%. Giventhatresponseandsavingsarerelativemeasure- ing the period [40,80] in which the cost generation is maxi- ments,lowdifferencesresultinsignificantchangesinthemag- mized,themajorityofthevaluesintheeTFSsignalsisdecreased nitudeofresponseandsavings. Thisexplainsthemoreextreme comparedtotherespectivevaluesoftheiTFSsignal. Figure8d valuescomputedwiththePNW-EVENINGdataset. shows a significant match of the eTFS signal with UB2. In the case of Figure 8a and 8d that maximum response is achieved 8. DiscussionandConclusion withSHIFT(20),asignificantlylowervolatilityerrorof9.14%and 3.3%isobserved. This paper illustrates extensive experimental findings using Figure9illustratesthegenerationfailurescenario. Themax- data from real-world operational supply-demand systems that imum response is 61.68%, 58.03%, 55.57% and 65.27% for confirmthefeasibilityofanewtypeofbottom-up,onlineself- SIM, EDF, PNW-MORNING and PNW-EVENING respectively. Similarly regulation. Incontrasttoearlierwork,thispapercontributesa totherampdownscenario,themajorityofthevaluesintheeTFS genericself-regulatoryframeworkshapedaroundstandardized signalsduringtheperiod[60,80]isdecreasedcomparedtothe conceptsandimplementedtechnologiesforaneasieradoption respectivevaluesoftheiTFSsignal. andapplicabilityinSmartCityapplications. Figure 10 illustrates the maximum entropy scenario. The An evaluation methodology, integrated within this frame- maximum response is 65.49%, 66.47%, 95.72% and 74.28% work, allows the systematic assessment of optimality and sys- for SIM, EDF, PNW-MORNING and PNW-EVENING respectively. A re- tem constraints, resulting in more informative and meaningful sponsewithahigherdemandatthebeginningandattheendof comparisons of different self-regulatory settings. For exam- theeTFSsignalisachievedduringwhichgenerationcostislow. ple, thecomparisonofdifferentagentselectionsbymeasuring Figure 11 illustrates the minimum entropy scenario. The response, savings and volatility error shows that purely cost- maximum response is 99.11%, 67.57%, 61.8% and 122.57% based metrics, such as savings, on which earlier work mainly for SIM, EDF, PNW-MORNING and PNW-EVENING respectively. The relies on, cannot adequately quantify the dynamics and capa- maximum response in SIM is achieved with MIN-COST that re- bilities of regulatory mechanisms to adjust demand. Instead, sults in a low volatility error of 16.58% confirming the sig- comparisons between response and savings suggest trade-offs nificant negative correlation between error-savings as shown thatopenupnewpathwaystoevaluatemultipledimensionsand in Figure 7. Note that in Figure 11d, the iTFS signal and the engineer multi-objective self-regulatory systems. The correla- UB2 signal have a very similar trend with a volatility error of tion results of the three metrics empirically validate the effec- 10