Table Of Content

ObjectAgent for Robust Autonomous Control Derek M.Surka PrincetonSatelliteSystems 150S.WashingtonSt.,Suite201 FallsChurch,VA22046 [email protected] Abstract estimation of the cluster center-of-mass and the relative positionsofeachsatellite. The ObjectAgent system is being developed to create a robust software architecture for autonomous control of TheSpacecraftManagerisaflightsoftwarepackagethat complex systems. Agents are used to implement all of the provides an autonomous replacement for the ground software functionality and communicate through simplified operations team. It will control all aspects of spacecraft naturallanguagemessages.Theseagentshaveasetofbasic operation including fault detection and redundancy survival skills that monitor for internal software faults, management. The Spacecraft Manager provides an providing low-level fault detection and recovery. Higher- interface between the Cluster Manager and the rest of the level fault detection and recovery capabilities, including TechSat21flightsoftware. modern artificial intelligence techniques, can easily be Although ObjectAgent is being originally developed for incorporatedintoanObjectAgent-basedsystem. distributed satellite systems, its applicability extends to other distributed and complex systems. These include Introduction. systems of air, surface, and submersible vehicles, process control, and network control and administration, to name a Princeton Satellite Systems is developing the ObjectAgent few. (OA) systemto create anagent-basedsoftwarearchitecture Previous papers have addressed the basic Matlab that is designed for autonomous, distributed systems. The architecture of ObjectAgent and have described the system is being designed to deal with traditional real-time research into agent organizations for distributed satellite software issues, such as memory management and control (Schetter, Campbell and Surka 2000ab). Papers deadlock, and to provide a robust architecture that can be have also described the basic C++ architecture (Surka, usedtoimplementmodernfaultdetectiontechniques. Brito and Harvey 2001) as well as the application of During the first phase ofdevelopment, ObjectAgentwas ObjectAgent to the TechSat 21 program (Zetocha et al. prototypedinMatlab.Acomplete,GUI-basedenvironment 2000). This paper focuses on the built-in features of was developed for the creation, simulation, and analysis of ObjectAgentthatimprovesystemrobustnessandautonomy multi-agent, multi-satellite systems. Collision avoidance and describes some of the innovative fault detection and and reconfiguration simulations were performed for a recoverytechniquescurrentlyunderdevelopment. cluster of four satellites. ObjectAgent is now being ported The firstsectionprovidesageneraloverviewofthecore toC++andthepresentarchitecturerunsonaPowerPC750 ObjectAgentsystem.Thisincludesadiscussionofthebasic runningEnea’sOSEoperatingsystem. survival skills possessed by agents. The second section ObjectAgent is scheduled to fly on the Air Force’s describes the approach being taken to address traditional TechSat 21 satellite program in 2003. TechSat 21 is a real-time software issues. The third section describes the missionthatwillinvolvethreesatellitesflyinginformation recently added layer of background health monitoring and andactingasa“virtual”satellite.ObjectAgentwillbeused error handling. (See (Mueller, Surka and Lin 2001) for to build two elements of the flight software, the Cluster more detailed information.) The final section describes the ManagerandtheSpacecraftManager. new fault detection techniques currently under The Cluster Manager is a flight software package that development. controls all spacecraft operations that require the coordination of multiple spacecraft. It also provides Overview of ObjectAgent complete fault detection of all cluster operation related systems. One of the primary functions of the Cluster ObjectAgent is an agent-based, message-passing software Manager is to perform relative control of the satellites in architecture that uses agents to implement all software the cluster. This will include relative stationkeeping and functionality.Agentsarethebasisofthesystemratherthan just a top layer. This is a key feature that distinguishes Copyright©2001,AmericanAssociationforArtificialIntelligence ObjectAgentfromotheragentarchitectures.Eachagentisa (www.aaai.org).Allrightsreserved. Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE 3. DATES COVERED 2001 2. REPORT TYPE 00-00-2001 to 00-00-2001 4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER ObjectAgent for Robust Autonomous Control 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION Princeton Satellite Systems,33 Witherspoon Street,Princeton,NJ,08542 REPORT NUMBER 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S) 11. SPONSOR/MONITOR’S REPORT NUMBER(S) 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited 13. SUPPLEMENTARY NOTES 14. ABSTRACT see report 15. SUBJECT TERMS 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF ABSTRACT OF PAGES RESPONSIBLE PERSON a. REPORT b. ABSTRACT c. THIS PAGE 5 unclassified unclassified unclassified Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18 multi-threaded process and this architecture allows creation process. ObjectAgent is an integrated approach to decision-making, including fault detection and recovery agent and flight software design, making extensive use of capabilities, to be builtinatalllevelsofthesoftware.This simplified natural language and graphical user interfaces in turn alleviates the need for extremely intelligent high- (GUIs). This design environment not only simplifies the levelagentsandsimplifiesthesoftwareinterfaces. agent creation process but also provides a common AfundamentalcomponentofObjectAgentistheflexible interface to a number of advanced control and estimation messaging architecture that provides a reliable method for techniques. agent-to-agent communication both on a single processor and across networks. Each message has a content field written in natural language that is used to identify the Real-TimeSoftwareIssues purpose of the message and its contents. Natural language Since ObjectAgent is being implemented in real-time was selected so that users could easily send messages or control systems, the OA architecture must address commandstotheagentsaswellasunderstandthemessages traditional real-time software issues such as process and being sent between agents. The latter is important for memory management and deadlock. It is not possible to debugging purposes. Agent communication takes place address these issues in the single-threaded prototype solely through messages; there is no shared memory Matlab environment and work on these issues has just betweenagents. Thisensuresthatagentscanworktogether begun. This section describes how we propose to initially evenwhentheyarenotlocatedonthesameprocessor. address these issues in the C++ implementation of Two additional advantages of using agents for all ObjectAgent. software functionality are increased flexibility and The initial C++ version of ObjectAgent will rely on the robustness. Robustness is improved in ObjectAgent real-time operating system (RTOS) to address some of because allagentsare endowed witha setofbasic survival theseissues.Enea’sOSEwasselectedasthefirstoperating skills. Each agent has knowledge of its skills, inputs, and system for ObjectAgent because OSE is a message based outputs, and is capable of automatically configuring itself RTOS designed for distributed systems and has many upon launch. It will automatically seek out other agents features that lend itself to the ObjectAgent architecture. who canprovide itwiththe inputsitneedsaswellasother These features include multi-threading, very good process agents who need its outputs. In this sense, an ObjectAgent andmemorymanagement,anddynamicprocessloading. systemisself-organizing. Unlike traditional embedded operating systems, which These same survival skills enable agents to be utilize lightweight tasks to partition complex activity and dynamically added to a system to improve the system semaphores to establish communications, a messaging capabilities or recover from a failure. The flexible and RTOS uses memory-protected processes and message- reconfigurable messaging architecture provides a common based communications. This approach makes it easier to software interface thatisvitalto thisabilitytodynamically conceptualize complex applications and distribute add or change software. Since all software is implemented programming responsibilities across large development as agents with the common messaging interface, all teams. The messaging RTOS modelalso makesiteasier to softwarecanbeeasilyreplacedorupdated.Thismessaging compartmentalize critical operations and data, thereby architecturealsohelpsthesystemtorecoverfromfailures. enhancingreliabilityandsecurity. Another key feature of ObjectAgent is it allows the user TheOSEkernelprovidesverygoodprocessandmemory to specify the complexity of the agents and agent management. Processes may be grouped into blocks, each organizations and does not constrain users to a predefined with its own memory pool. While other kernels may notionofanagent.Theuserperformsthedecompositionof schedule tasks running in a shared memory environment, the system into agents. This allows greater flexibility, OSE knows what resources each owns, including such extensibility, upgradability, and compatibilitywithexisting things as file descriptors, sockets, as well as all memory systems. resources, and supervises to avoid conflicts. If tasks die, Althoughartificialintelligencetechniquesarenotbuiltin thekernelcanreclaimtheresourcesautomatically. totheObjectAgentcore,theOAsystemarchitectureallows OSE is a true message passing designed for distributed AI techniques to be incorporated at anyor all levels of the processing applications and features full central processor software. Many tools are available to create agent skills. unit (CPU) or destination transparent messaging. The For example, the system includes fuzzy logic, neural net, messaging schema naturally supports fault tolerant and/or system identification, learning control, expert system and highavailabilitydesigns,inthefollowingfashion: faultdetectiontoolsthattheusercanemploytosolvehisor her distributed control problems. These techniques can Processessendmessagestootherprocesses; evenbeaddedafterthesystemisinoperation,whichisnot Processesdynamicallybindtootherprocesses;and possible with today’s flight systems. In addition, tools for OSE supervises all communications between processes; enabling agent organizations are included. This permits if delivery fails, or a process dies (or is killed), all agentstocontrolthebehaviorofotheragents. connected processes are notified so they may take Finally, special attention has been paid to developing a corrective action. One corrective action could be to systemthatiseasy-to-use and simplifiesthe flightsoftware establish a connection with a backup process (or board) or messages may be dynamically re-routed to alternate to send the nexthealthreport,anynewerrorinformationis destinations. included. Once a network of agents is established, an easy-to-use Beyond the built-inprotectionprovidedbytheoperating HealthMonitor GUI enables the designer to specify the system, the ObjectAgent architecture will provide method of monitoring health among agents. It should be additional features to address traditionalreal-time software noted that both external parameters (e.g. temperature issues.OnesuchfeatureistheinclusionineveryOA-based measurements) and internal parameters (e.g. time to runan system of a “hall monitor” agent on each processor. This algorithm)maybemonitored.Whatthedesignerdecidesto agentwillmonitorallotheragentsrunningontheprocessor dowiththisinformationis,ofcourse,casespecific. to detect run-away processes and deadlock. The hall The fault detection architecture for ObjectAgent was monitor will then have the ability to shut down any such designed with two objectives in mind: 1) to provide a problem agent. The details of the hall monitor agent are flexible framework to detect, report, and recover from stillbeingworkedout. errors in a distributed agent environment; and 2) to minimize the amount of required user-intervention. The Health Monitoringand ErrorHandling user is required to supply the necessary algorithms for detection and recovery. The ObjectAgent software then To create a robust, autonomous system, it is not enough to usesthisinformationtoimplementtheerrorhandling. only address the issues associated with developing real- Themainelementofthefaultdetectionarchitectureisan time software. The final systemmustalso be able to detect error.InObjectAgent,anerrorisdefinedasfollows: and recover from other subsystem faults. A background layer of health monitoring and error handling has been A specific occurrence, with a unique name, that prototyped in the ObjectAgent Matlab environment to may be detected and recovered from in a enablesystemstodetectandrecoverfromthesefaults distinctmanner. One of the most important tasks involved with Therearethreeprimaryactionssurroundingtheoccurrence controlling a complex system is to provide information ofanerror:detection,recovery,andreporting.Inanagent- about the system’s health. Satellites are an excellent basedframework,onceanerrorisdetected,thatknowledge example—several important parameters, such as is initiallyisolated to the original agent. The recoverymay temperature, fuel, attitude, and battery charge, are be performed by the original agent or by an outside agent, constantlybeingstoredinmemoryandtelemetereddownto providing it has been informed. ObjectAgent uses its earth. In an agent-based system, where the computational messagepassingarchitecturetodistributeerrorinformation tasks are distributed, it is especially important to monitor to all appropriate agents, thereby allowing a distributed health. The ObjectAgent health monitoring architecture recoveryapproachtobeused. accomplishes two goals. First, it enables the user to easily The types of errors which may occur are divided into 2 define how agents conduct health monitoring. Second, it categories: InputErrorsandSkillErrors.AnInputErroris carries out health monitoring in the background for an any error that has to do with a specific input to a skill, agentnetworkofanysize. while a Skill Error is an error that occurs while a skill The information provided by the health monitoring updates. For example, “No Input” is a generic Input Error architecture can be used by agents to detect existing or that occurs when an input is expected, but is not received. potential problems in the system. Each agent has its own The default action is to seek for a new source. The health report, which is defined bythe user. Ahealthreport detection and recovery algorithms for this error have been consists of three types of information: an overall health writtenintoanFDIRfunction,whichmaybeappliedtoany number, a set of important parameters, and a descriptive inputofanyskillinanyagentofthesystem. listoferrors. Errors may be either environmental or software oriented The health number of an agent is an instantaneous in nature. An error such as “Bad Signal” or “High Temp” measure of “how well” the agent is doing. The current would be environmental, whereas “No Input” is software method of measuring the health number is to begin with a oriented because it refers to a skill not receiving an nominal value of 100. If the agentexperiencesanerror, its expectedinputfromanothersoftwareagent. health is reduced bythe severityassociated withthaterror. Finally, the user may select which agents are informed When the agentrecovers, itshealthisrestored bythe same when an error occurs. It is useful to report errors to other amount. agents for two reasons. First, other agents are likely to be A more detailed set of information is included in the affected byan error. Iftheyare informed, thentheyhave a important parameters list. All skill outputs of an agent are chance to minimize the negative impact that the error may available to be selected as important parameters. They are have on them. Second, it maybe useful for other agents to measured at a specified rate, and their name, value, and assistincorrectingtheerror. timeofmeasurementareincludedinthehealthreport. The following example, which is used in the The final element of the health report is a list of errors. ObjectAgent Tutorial, demonstrates the error handling Each time an agent detects an error, a packet of relevant features of ObjectAgent. A simple network of three agents information is stored in the agent memory. Whenitistime issimulated.ThestructureisillustratedinFigure1. the core ObjectAgent architecture. The detection and Level 2 PlottingAgent recovery scheme of this particular error may be applied to any input and more advanced schemes can easily be implemented. Advanced fault detection techniques currently under development at PSS are described in the Level 1 DataAgent1 DataAgent2 followingsection. Figure1: SimulatedNetwork Advanced FaultDetection Techniques All fault detection in ObjectAgent is performed by agents DataAgent1 and DataAgent2 are identical; each possesses and anagent’sfunctionalityisdetermined bytheskillsthat the skill “DataSkill”, which simply calculates the system it possesses. Therefore, fault detection is incorporated into time and sends it as an output called “data”. PlottingAgent ObjectAgentbycreatingspecialfaultdetectionskills.Both hastheskill“PlottingSkill”,whichplotstheinput“data”as traditional (e.g. if…then…else constructs) and advanced a function of time. Prior to running the simulation, fault detection techniques can easily be incorporated into DataAgent1isdefinedasPlottingAgent’ssourceof“data”. agent skills. The system currently includes fuzzy logic, The results ofthe simulationare showninFigure 2. The neural net, system identification, and expert system tools goal is to illustrate that the error handling architecture that the user can employ for fault detection. A number of works properly by using the “No Input” FDIR function. If additional advanced techniques are also under the“NoInput”errorisdetectedbyPlottingAgent,itshould development. seek for a newsource, and find thatsource inDataAgent2. Decision making is centralized using the Autonomous PlottingSkill is coded such that, when the source of “data” Dynamic Algorithmic Expert System (ADAXS). ADAXS changes,itplots“o”insteadof“x”. is an expert systemthat is designed to handle time-varying data and to apply algorithms (for example an FFT) as part of its decision making process. It can be thought of as a combination of expert system and mathematical language such as Mathematica or Matlab. ADAXS is designed to mimicthebehaviorofpeoplewhentheyaretryingtosolve problems. Data is loaded into ADAXS using standard OA messages. The datafield may contain a string object or MatrixobjectplusallstandardC/C++datatypes.Newdata can be appended to datafields using messages. Decision making involves processing the data usingfiltersand other transforms and then making a decision based on the transformed data. The decision making process uses productionrulesandtherulesmaketakeanyoneofseveral formsincludingif...then...elseandFuzzylogicconstructs. Additional tools exist for clarifying input data. For example,forfaultdetection,DetectionFiltersareavailable. Detectionfiltersarefixed-gainKalmanFilterswiththegain matrix designed to make the filter residuals sensitive to actuator and sensor failures and plant model changes. The Figure2: Healthand“data”ofPlottingAgent when residualsarenormallyzerounlessafailureoccurs. “NoInput”erroroccurs Another technique under development at Dartmouth University is Prof. Minh Phan’s Interaction Matrix Formulationforactuatorfailuredetection.Thisformulation At 10 seconds DataAgent1 is failed, preventing allows the software to detect actuator failure immediately, PlottingAgent fromreceiving “data”. PlottingAgentdetects based upon past and current input/output data and an the “No Input” error at 12 seconds (due to a 2 second efficient filter. This software is robust with respect to delay), and reduces its health by the severity of the error, measurement noise, outside disturbances, and errors in which was set to 50. After 4 seconds pass, the recovery system modeling. Furthermore, it uses a minimum of algorithmisshowntobesuccessfulwhen“data”isreceived computational resources and applies to single input, single fromDataAgent2. outputsystemsaswellasmulti-input,multi-outputsystems. Although this is an extremely simple example, it illustrates the ease with which error handling may be executed. It also demonstrates the dynamic reconfigurabilityofagentcommunicationsthatisbuilt-into Conclusions In summary, the ObjectAgent software architecture is designed to increase the autonomy of complex distributed systems. It does so by providing a robust, agent-based software architecture that enables advanced fault detection techniques to easily be added to real-time systems. The message-passing operating system addresses many of the traditional real-time software issues such as process and memory management. Each agent has a set of basic survival skills that creates a robust communications network. A basic fault detection architecture has been created for ObjectAgent that facilitates both health monitoring and error handling for an agent network of any size. Advanced fault detection techniques, such as the ADAXSandInteractionMatrixFormulationsystemsunder current development, can easily be incorporated into this architecture. Acknowledgments This work is supported by two United States Air Force SBIR Phase II contracts fromthe Surveillance and Control Division of the Air Force Research Laboratory's Space Vehicles Directorate. The contract numbers are F29601- 99-C-0029 and F29601-00-C-0025 and the program managerisPaulZetocha. References Mueller, J. B., D. M. Surka, and J. J. Lin. 2001. A Background Layer of Health Monitoring and Error Handling for ObjectAgent. To be Presented at FLAIRS 2001.KeyWest,Florida. Schetter, T. P., M. E. Campbell, and D. M. Surka. 2000a. Comparison of Multiple Agent-based Organizations for Satellite Constellations. In Proceedings of FLAIRS 2000. Orlando,Florida. Schetter, T. P., M. E. Campbell, and D. M. Surka. 2000b. Multiple Agent-Based Autonomy for Satellite Constellations. In Proceedings of the Second International Symposium on Agent Systems and Applications. Zurich, Switzerland. Surka, D. M., M. C. Brito, and C. G. Harvey. 2001. Development of the Real-Time ObjectAgent Flight Software ArchitectureforDistributedSatelliteSystems.To be Presented at IEEE Aerospace Conference 2001. Big Sky,Montana. Zetocha, P., L. Self, R. Wainwright, R. Burns, M. Brito, and D. Surka. 2000. Command and Controlofa Cluster of Satellites. IEEE Intelligent Systems. November/December 2000.