Table Of Content

Distributed Co-Simulation of Maritime Systems and Operations Severin Sadjina, Stian Skjong, Eilif Pedersen, Vilmar Æsøy Department of Marine Technology, Norwegian University of Science and Technology, NO-7491 Trondheim, Norway Lars Tandle Kyllingstad, Martin Rindarøy, Dariusz Eirik Fathi, Vahid Hassani, Trond Johnsen, Jørgen Bremnes Nielsen SINTEF Ocean, NO-7465 Trondheim, Norway Here,wepresenttheconceptofanopenvirtualprototypingframeworkformaritimesystemsand operationsthatenablesitsuserstodevelopre-usablecomponentorsubsystemmodels,andcombine them in full-system simulations for prototyping, verification, training, and performance studies. This framework consists of a set of guidelines for model coupling, high-level and low-level coupling interfaces toguaranteeinteroperability, a full-systemsimulationsoftware, and examplemodelsand demonstrators. We discuss the requirements for such a framework, address the challenges and the possibilitiesinfulfillingthem,andaimtogivealistofbestpracticesformodularandefficientvirtual 7 1 prototyping and full-system simulation. The context of our work is within maritime systems and 0 operations, but the issues and solutions we present here are general enough to be of interest to a 2 much broader audience, both industrial and scientific. n a I. INTRODUCTION ple,thepowerconsumptionoftheon-boardequipmentis J significant and will, to a large extend, determine the di- 4 mensionsoftheenergysystems. Atthesametime, there With operations that are becoming increasingly com- is an incentive to minimize these dimensions from eco- ] plex and demanding, volatile economic conditions, E nomic and environmental standpoints in order to keep stricter environmental and safety regulations, and de- C the entire vessel as small as possible. creasing project lead times, simulation methods have be- . If the optimization of the overall system performance s come a key indicator of merit in early design phases c within the maritime industry. They allow for a quick is the goal, ship designs can, generally23, not simply be [ optimized for the performance of individual components exploration of the design space and help to advance con- and subsystems, but need to be optimized with respect 1 cepts into certain directions.18 Simulation methods also to total operational performance. Only with the inter- v expose concept, interface, and safety flaws which in turn 7 helps reduce risks and enhance operational performance actions between components, the surroundings, control 9 systems and software, and operators properly accounted and efficiency. 9 for,isitpossibletochooseasystemdesignwithdesirable 0 Traditionalshipdesignisasequentialprocess:15 Devel- characteristics—such as fuel efficiency, maneuverability 0 opment is driven step-by-step and iteratively with each and safety. With more advanced operations requiring . engineering discipline using its own set of tools that are 1 morepower,interaction,andtiming,systemperformance 0 rarely inter-operational. This complicates system level simulationwillbecomeevenmoreimportantinthefuture. 7 analysisandverificationsignificantlyandobscureserrors Full-system simulation, however, remains a challenging 1 and issues until late in the design process when they are and elusive task for at least two reasons: : difficultandexpensivetofix. Becauseofthisthereisnow v i anincreasinginterestinbeingabletofullyintegratesim- 1. Typical maritime systems and operations are diffi- X ulation techniques into the ship design process for proto- cult and complex to model and simulate by nature: r typing, verification, training, and performance studies.19 a they are characterized by intricate interactions be- Throughthedevelopmentofvirtualprototypesitispossi- tween a wide range of physical and engineering do- bletotestthecharacteristicsanddynamicsofaproposed mains with dynamics taking place on vastly differ- concept early on, and expose possible issues long before ent time scales, see Fig. 1. Compare, for example, theintegrationandprototypingstagesduringwhichthey the slow dynamics of large mechanical systems to usually surface. This is especially important for the mar- the fast response of electronic components. itimeindustrywhich—unlikeotherindustriessuchasau- tomotive, aerospace, and railway—typically has to pro- 2. This complexity and diversity is reflected in a sim- vide one-of-a-kind solutions. Because of this, it is hard ulation landscape that is riddled with specialized tolearnlessonsandadvancewithatraditionalsequential toolsfordifferentphysicalandengineeringdomains, process, when designs are unique and different each time with different interfaces and incompatible model around. representations. Some existing simulation tools for Virtual prototyping also holds the promise of substan- maritimeapplicationsarehighlyadvancedinterms tially simplifying the search for an optimal design and of quality, functionality, and usability. But they helpingtokeepcosts, riskfactors, andenvironmentalim- are mainly developed with research and the opti- pacts low. For a typical offshore supply vessel, for exam- mization of components and subsystems in mind, 2 and lack interconnection capabilities. General soft- • guidelines for model coupling, ware solutions, on the other hand, are too inflexi- • high-level interfaces for coupling models from dif- ble,andoffermodeldevelopmentandconfiguration ferent engineering and physical domains, thatistootime-costlyorinaccurate. Moreover,the models themselves span a wide range of complex- • low-level interfaces for coupling models from differ- ities and accuracies, including continuous as well ent tools, as discrete behavior, and different focuses depend- • a full-system simulation software, ing on the analyzed phenomena. Consequently, un- derstandinghowdifferentsubsystemsinteractwith • and example models and demonstrators. each other and how they influence overall system It aims to make the communication between costumers, behavior becomes all the more challenging. designers,andproductdevelopersmoreefficientthrough- out the design process. It also facilitates the consistency and availability of objective Key Performance Indicators (KPIs)iftheyareintegratedintotheprototypingsystem. A. Outline Inthispaper,wediscussthedevelopmentofacommon technology platform and infrastructure supporting virtual prototyping and simulation-oriented work processes for maritime systems and operations. The ViProMa project serves as an exemplary framework that allows us tohaveacloserlookattheknowledgegapsandchallenges withrespecttothesimulationofentiremaritimesystems andoperations. Theaforementionedguidelineswhichare attheheartoftheVPFarecontinuouslysummarizedand Figure 1. Maritime systems and operations include a wide emphasized on throughout the text. While the context range of different engineering domains and physical systems of the present work is maritime systems and operations, with varying complexity and time scales. This naturally we would like to emphasize that the challenges discussed makes full-system simulation a challenging endeavor. and the solutions presented here are broadly applicable. Theyshould,thus,alsobeofinterestandvaluetopeople Traditionally, simulations of closely-coupled subsys- outside of the maritime sector. tems are constructed from the ground up, resulting in First,wegiveabriefoverviewoftherequirementsfora monolithicsimulationsforcustominterfacesthataretoo virtual prototyping framework for the maritime industry application specific, too customized, and too costly in in section II. In section III, we discuss the development terms of development time. The ability to assemble re- of a common architecture for system simulation (the usable and interchangeable subsystem models into vir- VPF). We review different simulation approaches and tual prototypes in a plug and play manner—regardless justify our choice of using co-simulation (simulator cou- of the environment in which they are developed—should pling) in ViProMa. We then touch upon the formidable cut down on development times significantly and enable task of defining standardized domain model interfaces in rapid innovation. Undoubtedly, this would be a big step section IV, in order to establish a modular framework forward for the maritime industry. with high interoperability and re-usability with a focus The integration of multi-physics simulations, human on maritime applications. Section V brings these con- behavior, and multiple parallel maritime operations has cepts together in order to elaborate on the construction already been successfully demonstrated in operational of full-system models and the underlying challenges. We training simulators.17 However, to date there are no uni- then discuss our decision to write our own in-house co- versally agreed upon methods or standards supporting simulation software in section VI, where we also present total systems integration and the analysis of operational the challenges faced and the progress made so far. Fi- performance. The project Virtual Prototyping of Mar- nally, weprovideaconclusionandshareourthoughtson itime Systems and Operations3,19 (ViProMa) was initi- future developments in section VII. ated within the Norwegian maritime industrial cluster by independent research organizations, universities, and industry partners24 with the goal of developing an open, II. VIRTUAL PROTOTYPING FRAMEWORK standardized framework and architecture for system sim- REQUIREMENTS ulation and virtual prototyping as a new platform for product development and cooperation: the Virtual Pro- The requirements for a framework for virtual proto- totyping Framework (VPF). This framework includes typing in the maritime industry are shaped by its most 3 important use cases. In terms of the ViProMa project, 6. The framework has to be sufficiently easy to use these are: and provide well-defined interfaces in order for the industry to actually adopt and use it. This is also Vessel design: Comparison and optimization of con- crucial in light of future development and mainte- cepts with respect to fuel efficiency, capabilities, nance of the framework. operabilities, availability, maintainability, and op- portunity for future expansion. This also includes 7. Itshouldhaveacompletecomponentdatabasecon- virtualseatrials,andthetestingofcontrolsystems taining at least generic domain models of varying and station keeping abilities. model complexity. Because final designs are often re-used to save money and time, the database Crew training: Extending the possibilities towards should also be easily searchable. higherrealismandtheinclusionofharsherenviron- Thefollowingsectionsaredevotedtoaddressingthechal- ments, and increasing the awareness of vessel and lenges and possibilities in realizing these core require- equipment limitations. ments. Due to limitations in funding and time, however, Decision support: Choice of vessel for a specific voy- some of the requirements listed here were only partially age, and aligning ship capabilities with weather or not at all fulfilled within the scope of the ViProMa windows and equipment capabilities with specific project. This will be commented on in Sec. VII. operations. Requirements are also dictated by the desired workflow III. DISTRIBUTED CO-SIMULATION that such a framework should facilitate: After a model ofasubsystemhasbeendeveloped,itisconnectedtothe In setting out to prototype and simulate a complex full-system simulator. A scenario is then selected, the maritime system (such as a modern offshore supply ves- full-systemsimulationrun,andbehaviorandcapabilities sel), and capture all its relevant dynamics and interac- evaluated. tions,onequicklyrealizesthatthetraditionalmonolithic WithintheViProMaproject,thisleadtothefollowing simulation approach is too inflexible, too costly, and too core requirements imposed on the VPF. Note that while inefficient. A model of an entire vessel usually takes a these are specific to the project and the maritime indus- longtimetogetreadyforsimulation,ifconstructedfrom try, most of them are highly relevant to a much broader thegroundup, andre-useisoftenprohibitedwhenfaced area of application. with similar problem sets due to the developed model being too customized and too application-specific. 1. There has to be support for distributed and cross- Itisclearthen,thatamodularapproachinwhichmod- platform operation due to heterogeneous technolo- elsoftherelevantsubsystemsareinterconnectedandsim- gies, tools, and platforms, and the possibility for ulatedtogetherisfavorabletocutdownoncostanddevel- workload distribution. opmenttime. Thisalsomeansthattheprocessofmaking changes to a subsystem should be effortless, in the sense 2. Human-in-the-Loop and Hardware-in-the-Loop, that it does not require modifications of any other parts and, thus, real-time operation, have to be sup- of the full-system model (i.e., the open-closed principle ported for integration with training simulators, dy- should be adhered to). There are, generally, three meth- namic positioning (DP) and other control systems, ods to combine several models: and various types of machinery and equipment. This also helps to save time in factory acceptance 1. Theuseofacommonmodelinglanguageintowhich tests. allmodelsaretranslatedforthepurposeofthesim- ulation, 3. Theabilitytousecomponentsasblackboxestopro- tectintellectualpropertyandsensitiveinformation 2. the exchange of models between tools to run a sim- has to be implemented. ulation in one of them (model exchange), 4. The framework must be license-free with no re- 3. and co-simulation (simulator coupling). strictions on commercial use to prevent vendor Considering the requirements discussed in Sec. II, the lock-in, lower the barrier of use, and guarantee a first two modeling approaches have several drawbacks: widespread commitment. • General software solutions typically are too inflexi- 5. Increasingly rigid time constraints in the industry ble and offer model development and configuration demand sufficient performance with regards to the that is too time-costly or inaccurate. overall prototyping process, as well as the simulations alone. In addition, strategies to achieve rea- • In general, they neglect the availability of matured sonable accuracy and stability of full-system simu- and specialized domain-specific analysis software lations need to be established and implemented. for maritime applications. 4 • Users are reluctant to change modeling languages tools and solvers for any given subsystem. This also ortools,asdoingsoisamajorundertakinginprac- extends to the possibility of separately taking care of tice,andmayrenderinvestmentsintoolchainsand initialization, pre-processing, time integration, and post- training worthless. processing with different specialized tools. This is very advantageous, because it allows for rapid and accurate • The choices for a suitable common modeling lan- model development that make efficient use of already guage or tool are strongly limited if the develop- availablemodelingsoftwareandlanguageswithoutmajor ment of an open framework for systems simulation investmentsinnewtoolchainsortraining. Co-simulation and virtual prototyping is the goal. furtherhasthepotentialbenefitofsignificantlyreducing simulation time by using model-specific solvers and in- In addition, the use of a common modeling language ternal step sizes, and by allowing for the distribution of sometimes means abandoning the black box approach computational loads onto different computers or proces- thatprotectsIntellectualPropertyRights(IPRs)andsen- sor cores. The possibility to conveniently hide internal sitive information. Model exchange, however, may play dynamicsandprotectsensitiveinformationisanotherat- a vital role in future developments for full-system simu- tractive trait of simulator coupling, especially from an lation, and we shall touch upon this briefly in Sec. VC. industrial perspective. Guideline 1 (Model coupling). Use co-simulation to construct full-system models from loosely coupled stand- alone models and modules. Mathematically,co-simulationcorrespondstothemod- ular time integration of subsystems that are assumed to be independent in between discrete communication points t ∈{t ,t ,...,t }. The interactions between the i 0 1 N subsystems are realized at these time points, and are ex- pressed in the form of interface constraint equations, u(t )=Ly(t ), (1) i i whereLisaconnectiongraphmatrixrelatingtheinputs uandtheoutputsy. Thishappensata ratecorrespond- Figure 2. In a co-simulation setting, different tools and mod- ing to a macro step size ∆ti, such that ti+1 = ti +∆ti. elsareinterconnectedandusedindependentlyandinparallel In general, there is no guarantee that the pieces play to- to form a full-system simulation. gether nicely, though. Time synchronization and data exchange are important tasks, consequently, and sound As mentioned previously, the fact that typical mar- and efficient communication between subsystems implies itime systems and operations comprise of a wide range an adequate understanding of the architecture. of physical and engineering domains naturally leads to a Remark 1.1. Becautiouswhenselectingcouplingmethod rather heterogeneous simulation landscape, with special- and co-simulation (macro) step size to avoid accuracy ized tools and proprietary model representations. While and stability issues. this may seem like a rather poor starting point for full- Because input variables are unknown to the subsys- systemsimulationandvirtualprototyping, itisprecisely tems during the time integration t → t and need to this modular structure of complex engineering systems, i i+1 be approximated in general (and often held constant), in conjunction with the availability of well-established co-simulationbringsitsownsetofstabilityandaccuracy domain- and application-tailored software, that lends it- issues. Most commonly, this is remedied by selecting a self quite well to co-simulation, see Fig. 2. The remain- sufficiently small macro step size. At the same time, de- der of the present section is devoted to a discussion of mands to keep the computational cost within reasonable theco-simulationapproachandabriefreviewofexisting bounds may, generally, require a lower limit, especially co-simulation software and standards. for real-time applications. Remark 1.2. For linear systems, the macro step size can A. Co-Simulation be chosen from the eigenfrequencies, but it may be very difficult to find a good choice for nonlinear problems. The basic idea behind co-simulation is the construc- In addition, there are several other subtleties and tion of systems from loosely coupled stand-alone mod- challenges to co-simulation accuracy and stability. For els and the simulation across different subsystems. Co- example, the presence of algebraic loops can result in simulationfacilitatestheindependentexchangeandmod- instability22, and the presence of different time integra- ification of components, and the use of the most suitable tion methods can actually decrease the overall accuracy 5 ofthefull-systemsimulationtobelowtheminimumaccu- High-Level Architecture (HLA). HLA is not one specific racyoftheindividualsubsystems31. Ingeneral,thedevel- softwarepackage;rather,itisastandardwhichdescribes opment of an efficient and robust co-simulation method a general-purpose co-simulation architecture. It was ini- thatiseasytouseandgenerallyapplicableisstillongoing tiallydevelopedbytheUSDepartmentofDefenseforuse research.5 Additionally, it is not always clear beforehand in wargaming and training simulations, and was eventu- where to draw subsystem boundaries and how to choose allymadeanIEEEstandard. Thelatestversionofthisis a set of ‘good’ interface constraint equations—both of IEEE 1516-2010, commonly called HLA Evolved.32 Sev- which can play a significant role in determining stability eral HLA implementations exist today, both commercial and accuracy. and free. The ViProMa project tried to address many of these Similar architectures include the Distributed Interac- issues through original research: tive Simulation4 (DIS), which is the precursor of HLA and is even more geared towards military applications, 1. A novel Energy-Conservation-based Co-Simulation andtheCommonSimulationInterface21(CSI)developed method28,29 (ECCO) was developed within by MARINTEK for the purpose of maritime vessel simu- ViProMa. It gives readily available feedback on lations. global simulation quality, and significantly im- These architectures are designed around the concept proves the accuracy and efficiency of non-iterative of a federation, which is a group of independent subsys- co-simulations. We shall discuss it briefly in tems (federates) that communicate through a common Sec. IVA2. Run-Time Infrastructure (RTI). The RTI is responsible for routing signals between the federates and for time 2. Additionally, ananalysistoolforglobalstabilityin synchronization. The federates may be numerical simu- linear distributed dynamical systems has been pro- lations, hardware interfaces, human interfaces, et cetera. posed by combining dynamic stability and solver Oft-statedadvantagesofHLAincludeinteroperability,in stability,34 both of which are intimately linked thatfederatesmayrunondifferentplatformsandusedif- through local and global time steps. Under certain ferent simulation methods, and re-use, in that federates conditions,analgebraicsolutionofthetotalsystem used for one simulation may be easily re-used in another. can be constructed, and probed for global stability. However,becausethewireprotocolbetweenthefederates However, this procedure can be very time consum- and the RTI is not standardised, a federate created for ing, andisonlyapplicableforlineardynamicalsys- one HLA implementation generally can’t be used with a tems. Extensions to nonlinear dynamical systems differentimplementation. This, alongwithotherreasons have been studied as well, but the corresponding which will be explored in later sections, is why HLA was work is still ongoing. deemed unsuitable for the VPF. The task of establishing an efficient and robust general- Another standard we will mention here is the Func- purpose co-simulation methodology is far from com- tional Mock-up Interface (FMI), which, unlike HLA, of- pleted, however. Especially numerical stability is a non- fers a way to make subsystems binary compatible with trivial subject to study due to the inherent complexities each other, thus, removing the need for recompilation (different solver methods of various orders, different cou- and facilitating model sharing and co-simulation. FMI pling schemes of various orders, the presence of direct has become a key component of the VPF, and we shall feed-through and algebraic loops, et cetera). therefore describe it in more detail in section IVB. There exist non-iterative (explicit) and iterative (im- plicit)schemestocouplesubsimulatorsinaco-simulation, and time steps can be performed in parallel (Jacobi) or IV. SIMULATOR INTERFACES in serial (Gauss-Seidel). The simplest and most straight- forward of all schemes is the explicit one with constant In order to ensure interoperability, modularity, and re- input approximation. It is easiest to realize, keeps the usebetweendifferentmodelsandsimulators,well-defined exchange of coupling data to a minimum, and does not interfacesareneeded. Suchsimulator interfaces areaset require the repetition of entire macro time steps (roll- ofconventionsthat,ifadheredto,allowasimulatortobe back). Because of this, it is frequently used in industrial coupled with other simulators using some co-simulation applications, and has also been the main focus for the middleware. Here, we distinguish between two levels of ViProMa project so far. It does, however, exacerbate interfaces, which we shall refer to as high-level and low- the aforementioned stability and accuracy issues that co- level interfaces. simulation brings about naturally. A high-level interface is concerned with the concepts whicharebeingmodeledandsimulated. Thatis,itdeals with what a simulator represents on a physical level and B. Existing Co-Simulation Software and Standards the physical interpretation of data exchanged between simulators. Forexample,itiscrucialthatthevalueofan Among existing solutions for performing distributed output variable which represents a force in units of kN co-simulations, the most prominent one is probably the in one simulator is not used for an input variable which 6 representsaforceinunitsofNinanother—oronewhich ables: a flow and an effort. Their product is always represents, say, a voltage. A high-level interface could a physical power—such as force and velocity, pressure prevent this by either mandating that certain quantities and flow rate, or voltage and current. The use of power must have specific units, or by defining some mechanism bonds provides a complete and universal, energy-flow- whereby the units can be communicated, so that the co- centered connectivity between mathematical models of simulation middleware can make the necessary value ad- different engineering and physical domains. As pointed justments and/or prevent invalid connections. On top of out recently28, they are thus perfectly suited for high- this, the interface can define groups of variables which level interfaces for co-simulation. together have some physical significance. An example of Guideline2(High-levelinterfaces). Properlydefineand these are power bonds, which are pairs of variables that use high-level interfaces to guarantee interoperability of represent different means of power transfer between enti- simulationmodels. Makeuseofpowerbondstomodelthe ties. Power bonds are discussed further in section IVA. flow of energy between subsimulators whenever possible. Finally, at the highest level, one can define interfaces The use of SI units is highly advised. If other units are that represent categories of components or subsystems used, explicitly and clearly document so. in the system being simulated. For example, one could define that an ‘engine’ has a power bond for rotational mechanical power and an output variable which repre- 1. Power Bonds for Co-Simulation sents fuel consumption. Then, any simulator which ad- heres to this interface could be used to represent an engine,andcouldbereplacedwithanyothersimulatorthat A power bond between two coupled simulators is real- has the same interface. This opens great possibilities for ized by connecting two input–output pairs. For example, ‘plug-and-play’ construction of complex system models a model of an electric generator could have a voltage as and, consequently, rapid evaluation and optimization of an output, which a connected electric consumer model different designs. would receive as an input. In turn, the consumer would The high-level interfaces are necessarily underpinned outputanelectriccurrent,whichthegeneratormodelac- by one or more low-level interfaces, which are concerned cepts as an input. These two exemplary models are then withthefinerdetailsofhowtheco-simulationmiddleware coupled via a power bond. interacts with the simulators. At the lowest level, the Demandingthatphysicalcouplingsbetweensimulators physics involved are completely disregarded, and there arerealizedthroughpowerbondshasamajoradvantage: is nothing preventing one from, for example, coupling a it allows to study the flow of energy between the subsys- force variable with a voltage variable; the interface deals tems directly using nothing but the simulator coupling in bits and bytes, not newton and volt. One example of values. In fact, it makes it possible to directly observe if a low-level interface is what is known as an application andwheretheconservationofenergyisviolatedthrough- binary interface (ABI). Among other things, an ABI de- out a co-simulation, which, in turn, helps identify po- fines how different types of data, such as integers, real tential simulation issues, and provides valuable feedback numbers and textual data, are represented in computer about the quality of the results. memory. Complementary to this is an application programming interface (API), which specifies the names of program functions, which data they receive and return, 2. Residual Energies and Energy Conservation and, to some extent, what the functions do. For the VPF, the choice fell on the Functional Mock-up Inter- In general, energy is incorrectly transferred between face, which defines a simulator API and more. This is two coupled simulators due to the fact that their states discussed further in section IVB. areevolvedindependentlyofeachotherbetweendiscrete communication time points, see Fig. 3. In effect, energy is either created or destroyed through the co-simulation A. High-Level Interfaces coupling during each macro time step.7 This residual energy directlyaltersthetotalenergyoftheoverallcoupled system.28 It thus changes its dynamics, deteriorates sim- Power and energy are the universal currencies of phys- ulation accuracy, and may pose a threat to numerical ical systems. Energy is conserved and continuous: en- stability. ergy flows out of, or into, a system are always accounted for by appropriate energy storage and dissipation. This Guideline 3 (Error estimation). Use an error estima- is the theoretical foundation of bond graph theory10,27, tion method to assess and control co-simulation coupling which balances energy flows for each subsystem sepa- errors, and guarantee the quality and validity of the sim- rately. This way, they can be connected together in ulation results. a modular fashion, while satisfying energy conservation and continuity for the entire system. The energetic cou- Remark 3.1. The use of the Energy-Conservation- plings between (sub)systems are realized with so-called based Co-Simulation method (ECCO) for reliable co- power bonds, which are defined by a pair of power vari- simulation error estimation is recommended. It is easily 7 implemented,computationallyinexpensive,anddoesnot substantially: depending on system reticulation and require the repetition of entire co-simulation step sizes. subsimulator-internalsolveraccuracies,reductionsinthe errorof93%ormorecanbeobservedinbenchmarks—at no additional computational cost.28,29 In practice, mod- δPk els may not support variable macro step sizes, however, generally leaving the potential for more accurate and S1 Pk1 Pk2 S2 more efficient co-simulations untouched. Remark 3.3. If power bonds are not applicable, other co-simulation methods for error estimation and adaptive step size control can be used, see Ref. 28 and references Figure 3. A residual power δP = −(P +P ) emerges therein. Almost all of these require the repetition of en- k k1 k2 and distorts the dynamics of the full system when energy is tire co-simulation step sizes and simulator-internal infor- exchanged between two subsimulators, S and S , in a co- mation, however.25 1 2 simulation If power bonds are used, such energy residuals are B. Functional Mock-up Interface conveniently calculated from the coupling variable values alone. These concepts are exploited by ECCO The Functional Mock-up Interface (FMI) is a tool in- to obtain global error estimates. Unlike virtually all dependent standard for the exchange of dynamic mod- other proposed co-simulation schemes, ECCO requires els and for co-simulation.8 The first version of the stan- neither rollback nor any specific information on model dard was published in 2010 as a result of the ITEA2 implementations.25 Consequently, it does not prohibit project MODELISAR. Since 2011, maintenance and de- the use of commercial or legacy software (which often velopment of the standard have been performed by the makes rollback inefficient or impossible), and it helps Modelica Association, and a second version of the stan- protect IPRs. Because of this, it is especially attractive dard was released in 2014.9 fromanindustrialperspective. Inaddition, itaccurately FMI specifies that models should be packaged as func- tracks coupling errors, even for relatively very large time tionalmock-upunits(FMUs),whicharearchivefilesthat steps,beyondwhichstabilityisalreadycompromised,see containmodelcodeforoneormoreplatforms,alongwith Fig. 4. metadata and documentation. The standard defines the format and structure of files and directories in an FMU, 105 as well as the APIs that must be implemented by the 1040 model code. These APIs are defined in terms of the C 104 programming language which, being the lingua franca of programming, allows FMUs to be written in, and used 103 1020 from, practically any other language. TheFMIstandardconsistsoftwomainparts: FMIfor (cid:68)P 102 Model Exchange and FMI for Co-Simulation. FMI for 100 0.05 0.075 0.1 Model Exchange specifies an interface for models that 101 represent differential, algebraic, and discrete equations, 100 whicharetypicallycoupledwithandsolvedtogetherwith other models in some simulation software. Important 10(cid:45)1 here is that the solver is supplied by the simulation soft- 10(cid:45)4 10(cid:45)3 10(cid:45)2 10(cid:45)1 ware and is not part of the FMU code. In contrast, FMI (cid:68)t for Co-Simulation, which we shall focus on here, defines aninterfaceformodelswhicharebundledwiththeirown Figure 4. Energy-conservation-based error estimation (red) solvers, and which can therefore be seen as separate sim- compared to the actual error in the power ∆P (gray) as a ulators in themselves. Several such ‘subsimulators’ will function of the co-simulation step size ∆t for the benchmark typically be coupled together in a co-simulation environ- model from Ref. 28. The critical step size is ∆t≈0.059s ment, a piece of software that enables data exchange between the subsimulators, and keeps them synchronized Remark 3.2. It is advisable to use methods which adap- in time. All data exchange takes place at communica- tively control the co-simulation step size. The use of tionpoints(sometimescalledsynchronizationpoints),be- ECCO for step size control is recommended to ensure tweenwhicheachmodelissolvedindependentlyfromthe approximatelyaccurateenergytransfersbetweensubsim- others by its own solver. Note that we use the terms ulators. ‘model’ and ‘simulator’ freely here; in practice, there is An adaptive control of the macro step size can eas- nothingpreventingtheseentitiesfrombeinginterfacesto ily be realized based on ECCO’s error estimate. This hardwaresuchassensors,actuators,ordevicesforhuman canimprovetheaccuracyandefficiencyofco-simulations input. 8 FMI for Co-Simulation is based on a master/slave so that the name mapping between federation data and model of communication and control, where subsimula- FMU variables can be specified in the FMU metadata. tors are slaves that are controlled by a master algorithm. One aspect of FMI which should be mentioned here, The subsimulators do not have any information about as it has an impact on the model structures discussed in eachother,noraboutthesimulationenvironment,except Sec.V,isthefactthatFMUsare, formostpracticalpur- forthevaluestheyreceivefortheirinputvariables. Thus, poses,closed for modification. Thatis,onceanFMUhas theyhavenoknowledgeaboutorcontroloverwhichother beencreated, thereisnosimplewaytomodifyitsbehav- subsimulators they are coupled to; the data is routed by iornoritsexternalinterface. Themodelcodeistypically the master algorithm. stored in compiled binary code form, and the numbers, The FMI standard is a fitting choice for maritime ap- types and names of input and output variables are fixed. plications for three main reasons: In some contexts, this puts severe limits on scalability. Forexample, imaginethatwehaveanFMUthatmodels 1. Itwascreatedincollaborationwiththeautomotive abodywhichisacteduponbyexternalforces. Then, we industry for many of the same reasons that we aim have to decide upfront, at model construction time, on to design the VPF for the maritime industry. a maximum number of force inputs. If more forces are needed, the model source code must be modified and a 2. The standard is already supported by a large num- new FMU compiled, or the force summation (and pos- beroftools,forexamplebyDymola,JModelica.org, sibly coordinate transformations) must take place exter- SIMPACK, SimulationX, and Simulink. nally. Also note that while FMI shows enormous potential 3. FMIiscompletelyopenandfreetouseforanypur- and is continuously and actively refined, it also has var- pose. ious deficiencies. Some aspects of co-simulation are al- Finally, a decisive reason to choose FMI was that the together poorly addressed by the standard, and because alternative seemed to be to define a new interface and, limitations are often rather subtle, it is well worth point- to a large extent, reinvent the wheel. ingthemout. Forexample,itispossibletodesignFMUs and master algorithms that are standard compliant but Guideline 4 (Low-level interfaces). Use the Functional exhibitnondeterministicandunexpectedbehavior.11Itis Mock-up Interface to ensure compatibility between differ- alsonotdirectlyclearhownon-continuousmodelscanbe ent simulation tools and languages. Packaging subsimu- encoded as FMUs.12,16,36 Furthermore, FMI 1.0 has no lators as functional mock-up units makes them tool inde- general rollback mechanism beyond the previous macro pendent and re-usable. time step, while FMI 2.0 makes rollback only optional, and error control is not addressed by FMI at all.11 It should be emphasized at this point that FMI only specifies how the (co-)simulation software interacts with the models; it is not in itself a simulation software, nor V. MODEL CONSTRUCTION does it specify or restrict any other parts of the architecture of such a software. More to the point, in a dis- Letusnowbringtogetherthetheoreticalconceptsand tributedco-simulationsetting, FMIdoesnotsayhow, or principles from the previous sections, and apply them to in what format, data are transported between the simu- the construction of full-system models from stand-alone lation nodes, nor how the nodes are time synchronized. components and submodels. We shall first elaborate on Assuch,FMIsupportcanbeimplementedinalmostany some general considerations and common challenges, be- type of simulation software, and, indeed, the number of fore we reap the fruits of our efforts and move on to tools that support this standard is large and growing discuss some model examples at the end of the present quickly.2 section. Remark 4.1. WhetherornotanygivenFMUiscompliant withtheFMIstandardcanbecheckedwiththefreeFMU Compliance Checker.2 A. Subsimulators Asaninterestingsidenote,Awaiset al.haveproposed Subsimulatorsrepresentthevariouscomponentswhich to use HLA as a co-simulation environment for FMI- based components.6 This is despite the fact that HLA’s areconnectedtogethertoformafull-systemsimulator. It is important to note that this concept does not only in- federation model stands somewhat in contrast to FMI’s clude time-dependent physical systems. The actual func- master/slave structure: federates are not passive slaves tionality of a subsimulator is at the complete discretion that simply wait to be given anonymous input data, but of its developer, and includes: instead actively request, by name, the data they require. The findings by Awais et al. appear to reflect this. Their • mathematical models of physical systems evolving conclusionisthatitistechnicallypossibletouseHLAas over time, an FMI master, but to be able to make properly generic ‘FMU federates’, the FMI standard must be extended • control system implementations, 9 • scenario controllers (units which control and steer Remark 5.1. The way a given system is split up for co- a simulation, e.g., by changing parameters during simulation generally affects accuracy and stability. An runtime), ill-chosen system reticulation can cause a significant de- terioration of the quality of the co-simulation results, or • hardware interfaces (e.g. to allow direct hardware even an entirely unstable simulation. input to the simulation), Asanexample,considerthetwodifferentsystemreticu- • and bridging to other software or co-simulations. lationsforthesimplelinearquartercarbenchmarkmodel studied in Ref. 28: the mean global error increases by al- This list is not exhaustive by any means, and several most a factor of ten when choosing the less favorable other use cases are possible. That some of these may reticulation, and relatively large (constant) macro step warrant the use of a different concept will be the subject sizes, which cause no issues with one reticulation, lead of Section VE. But first, let us discuss the boundaries to instability with the other. Another practical example between subsimulators. is given by the development of a Dynamic Positioning (DP) controller for ViProMa. The two main parts of such a controller are a high-level motion controller and a B. System Boundaries Thrust Allocation Algorithm (TAA). The initial DP controller design kept the high-level motion controller and When modeling a complex system composed from the TAA in two separate subsimulators. This seemed many different subsystems in a co-simulation environ- like a good idea from a modularity perspective, because ment, one of the many challenges is to determine where it allowed to easily swap out or modify either of the two to draw the boundaries between the separate modules. independently. However, experience showed that the ad- To exemplify this, consider simulating a ship which con- ditionalmacrotimestepdelayintroducedbysplittingthe sists of a power system, propulsion units, and a hull. A DP controller this way made the two parts much more few examples of how to divide such a system into dif- difficulttotune,andfragiletochangesinthemacrotime ferent subsystems are shown in Fig. 5: One option is step size. Putting both, the high-level motion controller toincludeeverythinginonesinglesubsimulator,butthis and the TAA, inside one subsimulator greatly improved makeschangesdifficultorimpossibletoperformforusers the tunability and robustness. not having access to the subsimulator implementation. By splitting the total system into several parts the mod- C. Tight Coupling ularity will increase, but so will the complexity. Wheretodrawthelineisverymuchuptothedesigner, butitshouldbemotivatedbywhatisunderinvestigation. Another important issue, which kept resurfacing when Typically, a higher level of modularity makes it easier to establishingtheVPF,ishowtosensiblydealwithtightly- include individual models of high fidelity. Therefore, a coupled subsystems. Consider, for example, the rigid seemingly good rule-of-thumb is to keep modularity and mechanical connection between a vessel’s hull and crane. fidelity high in parts of the system that are of interest, Ideally, suchaconnectioncallsforsolvingthehull–crane while lowering them for the rest of the system. This is system as one, and the straight-forward (explicit) co- exemplified by Fig. 6 which shows a model with high simulation of both as separate subsimulators with sepa- modularity in the power system. The entire remainder rate solvers is unfeasible. In other words, it may be best of the ship is captured inside only one subsimulator, by to simply refrain from splitting tightly-coupled systems contrast,anditsonlypurposeistoprovide‘good-enough’ for co-simulation altogether. dynamics to give studies of phenomena in the power sys- Guideline 6 (Tightcoupling). Avoid exposing tight cou- tem the necessary embedding. Keep in mind, however, plings on the system level whenever possible in order to that there needs to be a trade-off between this modeling minimizecouplingerrorsandavoidissueswithnumerical modularity on one hand, and accuracy and stability on stability. the other: as discussed in Sec. IIIA, higher modularity may, generally, decrease theoverallaccuracyforthepart This, however, challenges the general and modular of the system under investigation, and therefore, for the virtual prototyping and full-system simulation approach entire co-simulation. thatissodesirablefromapracticalpointofview. Itmay stillbepossibletoallowforsufficientflexibilityinalarge Guideline 5 (Systemreticulation). Trytoobtainagood numberofapplicationsifgenericmodelsareusedthatof- balance between modularity, complexity, accuracy, and fer sufficient parameterization. But in some cases it may numerical stability when splitting up a given system for simply be impractical to avoid exposing tight couplings co-simulation. Beware of time delays between subsimula- between subsimulators, and several strategies have been tors, and consider that accuracy may suffer with increas- proposed as solutions: ing modularity. At the same time, try to provide a sufficient level of modularity to facilitate interoperability and 1. The most common approach is to include dampers re-usability of models. and springs in one of the models. This works 10 Figure 5. Different levels of modeled modularity of systems on board a ship 3. Yet another way of dealing with rigid coupling is to use advanced iterative co-simulation approaches utilizing subsimulator outputs along with their re- spective Jacobians. An example of this is the In- terface Jacobian-based Co-Simulation Algorithm30, in which coupling conditions are solved iteratively with Newton’s method. Unfortunately, such approaches are presently mainly of academic interest: generallyrequiringJacobiansfromsubsimulators— along with their ability to redo macro time steps— is simply an unrealistic condition short-term and medium-term.26 4. Finally, one can neglect the effects of one of the subsystem on the other. For example, a hull would simply dictate the position of a crane attached to it, while the forces sent back to the hull are only due to the crane’s inertia. This approach is only Figure6. Exampleofsystemmodularizationforashipmodel sufficient, however, if an accurate representation of with a special focus on power-system dynamics the forces is not desired. As reflected by these choices, tight coupling is a com- fine in practice, but has some notable drawbacks: plex and sensitive issue for virtual prototyping and co- Firstly, suitable parameters need to be found to simulation, and requires further research. For the time configuretheseadditionalelements. This,secondly, being, the best course of action will be determined by means that the relatively stiff springs and strong the case at hand, and can hopefully be found with the dampers required introduce relatively small time examples given here. constants, which can easily become a challenge for co-simulation. 2. A possibly very fruitful approach is to implement D. Connections tightly-coupled subsystems using model exchange. Then, the models can be shared and modularly Connections define the interactions between subsimu- integrated, while they are being solved jointly by lators and are intimately interrelated with the system one solver. While this generally still satisfies the reticulation. Therefore, they need to be treated and im- black box criterion, it may mean abandoning well- plemented carefully. Mathematically, connections are ex- establishedsimulationtoolsforsomeusers. Itcould pressed via the connection graph matrix L of Eq. (1), alsoproofdifficulttofindappropriateopensoftware but in a co-simulation they are only enforced at the dis- solutions enabling the use of model exchange in a crete communication time instances. This naturally cre- co-simulation environment. ateschallengesforaccuracyandstability, asdiscussedin