Co-Design of Anytime Computation and Robust Control

12 Pages·2016·0.69 MB·English

University of Pennsylvania
ScholarlyCommons
Real-Time and Embedded Systems Lab (mLAB) School of Engineering and Applied Science
2015
Co-Design of Anytime Computation and Robust
Control
Yash Vardhan Pant
University of Pennsylvania, [e-mail]
Kartik Mohta
University of Pennsylvania, [e-mail]
Houssam Abbas
University of Pennsylvania, [e-mail]
Truong Nghiem
University of Pennsylvania, [e-mail]
Joesph Deveitti
University of Pennsylvania, [e-mail]
See next page for additional authors
Follow this and additional works at:http://repository.upenn.edu/mlab_papers
Part of theComputer Engineering Commons, and theElectrical and Computer Engineering
Commons
Recommended Citation
Yash Vardhan Pant, Kartik Mohta, Houssam Abbas, Truong X Nghiem, Joesph Deveitti, and Rahul Mangharam, "Co-design of
Anytime Computation and Robust Control", Proceedings of the IEEE Real-Time Systems Symposium, San Antonio, Texas, 2015
This paper is posted at ScholarlyCommons.http://repository.upenn.edu/mlab_papers/91
For more information, please [e-mail].
Co-Design of Anytime Computation and Robust Control
Disciplines
Computer Engineering | Electrical and Computer Engineering
Author(s)
Yash Vardhan Pant, Kartik Mohta, Houssam Abbas, Truong Nghiem, Joesph Deveitti, and Rahul Mangharam
This conference paper is available at ScholarlyCommons:http://repository.upenn.edu/mlab_papers/91
2015 IEEE Real-Time Systems Symposium
Co-Design of Anytime Computation and Robust Control
Yash Vardhan Pant, Kartik Mohta, Houssam Abbas, Truong X. Nghiem, Joseph Devietti, Rahul Mangharam
Abstract—Controlsoftwareofautonomousrobotshasstrin-
gent real-time requirements that must be met to achieve the
Measurement from Sensors Perception-
controlobjectives.Onesourceofvariabilityintheperformance (e.g. Video Feed) based
of a control system is the execution time and accuracy of Estimator
the state estimator that provides the controller with state (Delay, Error) EsSttimataet e
information. This estimator is typically perception-based (e.g., Contract (e.g. Robot Position)
Computer Vision-based) and is computationally expensive.
When the computational resources of the hardware platform Controller
becomeoverloaded,theestimationdelaycancompromisecon- Control Action
to Physical System
trol performance and even stability. In this paper, we define (e.g. Motor Speed)
a framework for co-designing anytime estimation and control
Physical System Estimation and Control
algorithms, in a manner that accounts for implementation
(e.g. Autonomous Robot)
issues like delays and inaccuracies. We construct an anytime Fig.1. Contract-basedcontrollerandestimator.
perception-based estimator from standard off-the-shelf Com-
puter Vision algorithms, and show how to obtain a trade-off morestringent,thisseparationinthedesignandexecutionof
curve for its delay vs estimate error behavior. We use this controller and estimator can lead to degraded performance,
anytime estimator in a controller that can use this trade-
as will be shown in Example 1. The goal of this paper is
off curve at runtime to achieve its control objectives at a
to present a rigorous framework for the joint design of the
reduced energy cost. When the estimation delay is too large
forcorrectoperation,weprovideanoptimalmannerinwhich controller and estimator, in which the estimator explicitly
the controller can use this curve to reduce estimation delay presents a range of execution time/estimate error operating
at the cost of higher inaccuracy, all the while guaranteeing modes, and the controller switches between these modes in
basic objectives are met. We illustrate our approach on an
real-timetomaintaincontrolperformanceandreduceenergy
autonomous hexrotor and demonstrate its advantage over a
consumption.
system that does not exploit co-design.
Typical design practice determines the Worst-Case Exe-
I. INTRODUCTION cution Time (WCET) of the estimation task, and engineers
Real-time control of physical systems, like autonomous the system to satisfy deadlines under WCET conditions.
robots, raises a number of timing and control-related issues However, the actual execution time of such estimators is
at the interface between the controller that’s providing the heavily dependent on the actual data being processed. So
actuation and the estimator that’s providing periodic state WCET considerations, whether computed online or offline,
estimates to the controller. Some of these issues have to do produce a conservative design. Moreover, classical timing
with the inaccuracies introduced by the software implemen- analysis does not guarantee functional correctness of the
tation of both controller and estimator on a given hardware closed-loop system. In addition, the best estimate is not
platform. Specifically, controllers are typically designed to alwaysneeded:sometimesalowerqualityestimate,obtained
accomplish the functional goals of the system under simpli- withasmallerenergycost,issufficienttoachievethecontrol
fying assumptions on the quality of the state estimate (e.g., objectives. Finally, when obtaining better estimates requires
no or fixed error), the estimation delay (e.g., no or fixed longer runtimes of the estimation task, it may actually be
delay), and the actuation jitter (e.g., no jitter). Conversely, detrimental to ask for the best estimate. For example, when
estimation algorithms are typically designed without regard the computational resources are overloaded, there may be a
tohowtheirestimateswillbeusedandunderwhatoperating need to spend less time computing a state estimate.
conditions. In particular, an estimator will often run to Example 1: To illustrate the impact of estimation delay
completion: that is, its stopping criteria are designed to δ and estimate inaccuracy (cid:3) on control performance, we
provide the best estimate, regardless of runtime or energy show a simple PID controlling the motion of a point mass
consumption. The problem addressed here is that as the in the (x,y) plane. The position of the point mass must
real-time requirements on the closed-loop system become follow a reference constant trajectory, whose x dimension
is shown in Fig. 2 (the same plot can be obtained for
*ThisworkwassupportedbySTARnetaSemiconductorResearchCor- the y position). We simulate three cases of estimation (and
porationprogramsponsoredbyMARCOandDARPA,NSFMRI-0923518
therefore actuation) delay and error, where a larger delay
andtheUSDepartmentofTransportationUniversityTransportationCenter
Program valueδimpliesasmallerestimationerror(cid:3).Ascanbenoted
The Departments of Electrical and Systems Engineering and Com- inFig.2,theeffectofdelaycanbenon-negligible.Moreover,
puterandInformationSciences,UniversityofPennsylvania,Philadelphia,
U.S.A. {yashpant,kmohta,habbas,nghiem,rahulm}@seas.upenn.edu, decreasing delay doesn’t necessarily imply better tracking
[e-mail] performance: the effect of the concomitant estimation error
1052-8725/15 $31.00 © 2015 IEEE 43
DOI 10.1109/RTSS.2015.12
Fig.3. Autonomoushexrotorwithdownward-facingcameraflyingover
syntheticfeatures.
in the form of operating modes or contracts on the
accuracy and delay provided by the estimator (Section
III),
Fig.2. Effectofdelay,errorvaluesoncontrolperformance.
• acontrollerdesignthatcanvarytheaccuracyanddelay
must be taken into account. In this example, it can be seen oftheestimationtoachievecontrolobjectivesatalower
that the increasing error causes the tracking performance to energy cost (Sections IV, V), and
worsen. Running an estimation task with a fixed smaller • ageneralproceduretocomposerun-to-completionesti-
delay but larger estimation error does not necessarily solve mationalgorithmsintoacontract-basedestimator(Sec-
the problem of degraded performance, as can be seen in tion VI).
Fig. 2. Therefore, there is a need to rigorously quantify • We illustrate our approach on an autonomous flying
thetrade-offbetweencomputationtimeandestimationerror, robot (shown in Fig. 13) and demonstrate performance
then exploit that trade-off to achieve the best control per- and energy gains using our approach over a classical
formanceundertheproblemconstraints.Ratherthanalways controller (Section VII).
runningtheestimationtasktocompletion,itisusefultohave
several estimation tasks with varying utilities (i.e., varying II. RELATEDWORK
delay/errortrade-offs).Thesecanthenbeusedatruntimeto Anytime algorithms [1] are a class of algorithms that
satisfy the control objectives. (cid:2) can be interrupted at any point during their execution and
In this work, we develop the above remarks into a co- still return a usable solution, usually with a monotonically
designframework forareal-timecontrolsystems,wherethe improving quality with time. Contract algorithms [2] are
controller and estimator communicate via contracts. A con- one class of anytime algorithms where the interruption time
tract is a guarantee requested by the controller, and fulfilled is pre-determined for any given execution. Our approach,
bytheestimator,thatthelattercanprovideanestimatewith while similar to contract algorithms in the timing aspect,
a certain maximum error (cid:3), and within a certain deadline δ. differs significantly as the meaning of a contract expands to
Boththedeadlineandtheerrorboundarepartofthecontract. including both time and quality of the solution (estimation
Using these contracts, we show how the controller can error in our case).
throttletheexecutiontimeoftheestimationtasktopreserve Anytime algorithms have notably been studied for graph
good performance and to reduce energy consumption. Our search [3], evaluation of belief networks [4] and GPU
workfocusesonestimatorsthatincorporatecomputationally architectures [5].
intensive Computer Vision (CV) algorithms, such as those As overloaded real-time systems are becoming increas-
used in autonomous robot navigation. We refer to these as ingly common, anytime algorithms for control have become
perception-based estimators. Our experiments validate that a topic of research interest. Most notably, Quevedo and
the execution time of these algorithms is significant and far Gupta [6], Bhattacharya and Balas [7], and Fontanelli et
exceeds the computation time of the control software, and al. [8] have contributed to the topic. Our approach differs
can have an effect on control performance. significantly from these works as the anytime computation
Fig. 1 presents the proposed structure of contract-based assumption is on the perception-and-estimation algorithm
estimation and control. It shows a traditional feedback loop and our controller is a robust controller which can switch
incorporating estimator, controller and the physical system, betweendifferentoperatingmodesoftheanytimeestimator.
augmented with the (Delay, Error) contract between con- Also, while most of these works require either access to
troller and estimator. This contract forms the basis of the the full state of the system or have a fast estimator giving
proposed approach. them the state estimate [7], our algorithm accounts for
Summary of contributions. We present a contract-based the computation time/error of the perception-and-estimation
framework for the co-design of real-time controller and algorithms that are common in autonomous systems.
estimator algorithms, consisting of: In real-time systems, recent work [9] uses Typical Worst
• awell-definedinterfacebetweencontrolandestimation, Case Analysis of the software and Logical Execution Time
44
Contract-based Estimator
Timing and Accuracy
of Execution Paths PC1 PC2 PC3
Sensor Measurement
Offline Profiling CC1 CC2
Perception Toolchain
SC1 SC2
Pixel Classifier (δ,ε) (δ(cid:2),ε(cid:2))
State
Estimate (Delay, Error) Physical System
Connected Contract
Components
Controller
Shape Classifier ε Defloary -Eesrtriomr aCtourrve
xx
x
xxxxxxxxxxx
δ
uu
Control Action
Delay-error Curve
Fig.4. Contract-basedestimatorandcontroller
semantics to provide the controller with knowledge of the improveperformanceofreal-timeclosedloopsystemsusing
timing characteristics of the implementation. Our work, by computationallyandpowerlimitedplatforms,weproposethe
contrast, profiles the estimation software directly to obtain co-design of estimation and control. The co-design involves
timing and accuracy information. Whereas [9] is concerned using a contract-based framework for both estimator and
with formal verification of a given controller, we design controller. Namely, the controller requests the estimator to
controllers to take advantage of delay/accuracy trade-offs provide a state estimate within a certain deadline δ seconds
in real-time. The effect of increasing computation time of and with a certain error bound (cid:3). We refer to the tuple
a task on performance of a UAV has been explored in (δ,(cid:3)) as the contract between controller and estimator. The
[10] by using a resource allocation algorithm similar to estimatorthenprovidesanestimatethatrespectsthecontract.
QRAM [11]. Our work differs from this as we consider the Byrequestingestimateswithvaryingcontractsduringsystem
execution time of a task, the estimator, which is directly operation, the controller is able to adapt the closed-loop
related to the control performance of a closed loop system system performance in real-time according to the current
andalsoformulateacontrolproblemarounditthatprovides conditionofthephysicalsystem.Forexample,itcandecide
mathematical guarantees on the performance of the closed when an estimate is needed fast (but usually with higher
loop system. error), and when a more accurate estimate is needed (but
Also, in the field of computer architecture approximate with greater delay). Note, the (δ,(cid:3)) contract can also be
computing approaches [12], [13], [14] have been studied, thought of as setting an operating mode for the perception-
seekingtimeorenergysavingsbyperformingacomputation and-estimation algorithm. A high-level view of this setup is
approximatelyinsteadofprecisely.Whileanytimealgorithms shown in Fig. 1.
and approximate computing share a high-level goal, ap- To ensure that the estimator can respect the contract
proximate computing approaches are run-to-completion and (alternatively,thatthecontrollerisonlyrequestingcontracts
also lack a feedback mechanism to permit computation and thatcanbefulfilledbytheestimator),theestimatorisprofiled
resources to be balanced dynamically. Additionally the time off-line. Namely, the estimator’s parameters are varied and
and energy scale that our approach works at is much higher for each setting of the parameters, it is run on a profiling
than what approximate computing looks at. data set. This yields a finite set of (δ,(cid:3)) values, each one
corresponding to a setting of the parameters. These values
III. CO-DESIGNOFESTIMATIONANDCONTROL can be plotted on a curve, which we call the error-delay
In a traditional control system, the controller is unaware curve made up of discrete points, (δ,(cid:3)), represented by the
of the implementation details of the estimation module set Δ. Examples of such a curve are shown in Figs. 7 and
and the estimation module is unaware of the requirements 14. The detailed procedure for obtaining such a curve for a
of the controller. For example, the design of a feedback perception based algorithms is given in Section VI.
controllermightnottakeintoaccountthefactthatobtaining At run-time, when the estimator receives a (δ,(cid:3)) contract
astateestimatefromavideofeedwilltakeanon-negligible request from the controller, it can adapt its execution paths
amount of time, which we refer to as the estimation delay. to respect the contract, namely, to provide a state estimate
Conversely, the design of the perception and estimation in real-time within the requested error bound (cid:3), and within
might not in general take into account the varying real-time the requested deadline δ.
constraintsthatthecontrolledsystemmustsatisfy.Inorderto Inaddition,thecontrollerisdesignedwiththeknowledge
45
of the error-delay curve of the estimation algorithm, and returnsanoutputafterinternallydecidingthecompositionto
requests contracts from that curve. Thus, the error-delay bestmeetthedeadlineandtheexpectedqualityrequirement.
curve constitutes the interface between controller and es- Through extensive offline profiling, we guarantee with a
timator. This gives the controller the ability to leverage the high degree of confidence that the contract based estimator
flexiblenatureoftheestimationalgorithmtomaximizesome does not violate the contract. This helps in formulating
performance measure of control performance. a control algorithm that provides mathematical guarantees
Fig.4showstheclosedlooparchitectureinasystemwith on the feasibility of constraints for the safe operation and
co-designoftheestimatorandcontroller.Intheco-designed stability of the closed loop dynamic system as covered in
systemaspresentedinthispaper,thecontrollercanmakethe section V.
estimation algorithm switch to lower or higher time (and/or
C. RobustControlwithcontractbasedperceptionalgorithm
energy) consuming modes based on the control objective at
thecurrenttimestep.Themaincomponentsoftheco-design The control algorithm is designed to pick the best op-
architecture are a contract based perception-and-estimation erating point for the estimator, or the right (δ,(cid:3)) contract
algorithm,arobustcontrolalgorithmthatcomputesaninput to request from the perception and estimation algorithm.
tobe sentto thephysical systembeing controlleraswell as This is done based on the current state of the physical
the operating mode for the contract time estimator, and the system to maximize a performance measure while being
interface between them. More details on these components robust to the varying computation time and the varying
are in the following sections. estimation errors of the estimator with different contracts
as is provides estimates to the controller. In section V we
A. Contract based perception algorithms present a control algorithm that achieves this while also
Acontractbasedperception-and-estimationalgorithmcan guaranteeingfeasibilityofsystemconstraintsthestabilityof
operate at different deadlines and provide a usable solution the closed loop system.
for the control algorithm to operate on. This flexible opera-
IV. ROBUSTCONTROLWITHCONTRACT-BASED
tion is achieved by composing the algorithm of functional
ESTIMATOR
blocks that have different execution times and result in
different qualities of outputs. In this section we present the mathematical formulation
An example is a Computer Vision (CV) based Object to model the controller and physical system from Fig. 1,
recognition algorithm which is composed of different func- and demonstrate how the controller can, in real-time, use
tional blocks of varying execution time which result in knowledge of the estimator’s error-delay curve to decrease
a different accuracy when linked together to provide the computation delay and power in an error-aware fashion.
functionality of an object recognition algorithm. E.g. the
A. System Model
pixel classifier in the first stage of such a CV algorithm
could be a Gaussian Mixture Model with 2, 4, or 6 compo- Considerahexrotor,whichisanautonomousflyingrobot
nents, with more components providing better classification withsixrotors,showninFig.13.Thestatexofthehexrotor
performance (over-fitting is ruled out by cross-validation) at is made of its 3D position and 3D velocity. The input u
the cost of more computation time. Functions with similar to the robot consists of the desired pitch and roll angles,
characteristicslikeexampleabove,whenprofiledextensively and the desired thrust. The hexrotor’s mission is to fly a
offline and composed in the right order at run-time can be pre-defined pattern given by xref, where xref(t) gives the
used to compose a contract time anytime perception and desiredpositionateachtimet.Thedynamicsofthehexrotor,
estimation algorithm. More details follow in section VI. relatingthetime-evolutionofitsstatetothecurrentstateand
input, can be linearized and approximated by the following
B. Interface between contract based perception and robust Linear Time-Invariant (LTI) ODE:
control
x˙(t)=Acx(t)+Bcu(t)+wc(t) (1)
For the control algorithm to be able to leverage the
flexible nature of the contract based perception algorithm, wherex∈RnisthestateconstrainedtolieinasetX ⊂Rn,
it must have information about the computation time versus u∈Rm is the control input constrained to lie in a set U ⊂
output quality trade-off that the contract based perception Rm, and wc ∈ Rn is the bounded process noise assumed
algorithm offers. An interface that achieves this is obtained to lie in a set Wc ⊂ Rn. Ac ∈ Rn×n and Bc ∈ Rn×m are
by representing the profiled behaviour of the contract based matrices.LTIsmodelawiderangeofsystems,andourresults
algorithm to varying deadlines, as points on a perception applytoarbitraryLTIsoftheformgivenin(1)withcompact
quality versus deadline (δ,(cid:3)) curve, e.g. in Fig. 7. With andconvexconstraintsetsX,U andWc.ThesetsX andU
this profiled curve available to the controller at runtime, are part of the problem statement and are either chosen by
the exchange of information between the contract based the designer or determined by physical constraints.For the
perception-and-estimation algorithm and the control algo- hexrotor, X captures limits on the state such that the LTI
rithmconsistsofthecontrollerassigningadeadline(δ),ora dynamicsprovideagoodapproximationofthetruenonlinear
contracttotheperceptionalgorithmwhileexpectingabound dynamics. The set U restricts the inputs to values that can
on the error ((cid:3)) of its output. The perception algorithm then be supported by the rotors.
46
(cid:2)(cid:3)(cid:4)(cid:2)(cid:3) (cid:5)(cid:6)(cid:7)(cid:8)(cid:5)(cid:7)(cid:3) (cid:2)(cid:3)(cid:4)(cid:2)(cid:3) (cid:5)(cid:6)(cid:7)(cid:8)(cid:5)(cid:7)(cid:3) (cid:2)(cid:3)(cid:4)(cid:2)(cid:3) D. Discretized Dynamics
Because of time-triggered sensing and actuation, from
(cid:2)(cid:2)(cid:3)(cid:4) (cid:2)(cid:2)(cid:3)(cid:4)(cid:3)(cid:4)(cid:4)(cid:3)(cid:5)(cid:4) (cid:2)(cid:2)(cid:3)(cid:4)(cid:5)(cid:6) (cid:2)(cid:2)(cid:3)(cid:4)(cid:5)(cid:6)(cid:3)(cid:4)(cid:4)(cid:5)(cid:6)(cid:3)(cid:5)(cid:4)(cid:5)(cid:6) (cid:2) utismede.tTs,hkentoattat,ka,,kththeepnreevwioucsonctroonltrionlpuintpuukt uisk−c1omispustteildl
andappliedbythecontroller(seeFig.5).Forsimplification,
Fig.5. Time-triggeredsensingandactuation.Thefigureshowsthevarying we assume the computation time for the controller (τ) to
executiontimefortheestimatorandtheblueareashowstheexecutiontime
forthecontroller,whichissmall. be constant and lump it with the time for the estimator (δ).
Thisisjustifiedexperimentallyforourproblem(inSec.VII-
B)wherethetimeforthecontrollerisnegligiblecomparedto
B. Time-Triggered Sensing and Actuation the time taken by the estimation algorithm. The discretized
dynamics are given by
For flight the hexrotor needs to determine its current
position and speed, i.e., it needs to produce an estimate of xk+1=Axk+B1(δk)uk−1+B2(δk)uk+wk,k≥0 (2)
itscurrentstatex.Itdoessobytakingavideoduringflight
in which
through a downward facing camera, detecting and tracking
(cid:3)
ftToea>tthuer0essesaefcceroaontsudsrsef,sra.thmTuehsse,reacsnaudmltdienergaducicnainppgtueirrtiesosdoiawcnnmepweoassifutriaroemnmereenleatvsteivrayet B1(Aδ)==(cid:3)eA0δceTA,c(Tw−tk)B=cd0tT,eBA2c((δT)−=t)(cid:3)wδTc(etAs,ck(T+−tt))Bdtcdt.
instants ts,k =kT, where k∈N. Herewkistheaccumulatedprocessnoiseduringtheinterval,
The sampled measurement is fed to the estimator that andisconstrainedtolieinacompactconvexsetW because
computes the state estimate xˆk := xˆ(ts,k) with the desired wc(t) lies in the compact convex set Wc and T is finite.
accuracy(cid:3)k determinedbythecontrollerintheprevioustime Notethatboththecurrentcontrolukandthepreviouscontrol
step. The controller then uses this state estimate to compute uk−1 appear in (2). Furthermore, the input matrices B1(δk)
thecontrolinputuk aswellasdecideonthestateestimate’s andB2(δk)dependonthedelayδk.Theestimationaccuracy
delay and accuracy contract (δk+1,(cid:3)k+1) for the next step. (cid:3)k affects the state estimate xˆk used by the controller to
This control is applied to the physical system according to computeuk;therefore(cid:3)k indirectlyaffectsthedynamicsvia
(1) at instant ta,k =ts,k+δk+τk, where τk is the time it the control input.
takes to compute the input. See Fig. 5.
In our setting, the controller has access to the delay-error V. ROBUSTMODELPREDICTIVECONTROLSOLUTION
curveΔoftheestimator,andmakescontractselectionsfrom InthissectionwegiveanoverviewoftheRobustAdaptive
that curve. This curve is obtained offline as explained in Model Predictive Controller (RAMPC) that we use in the
Section III, and exemplified in Section VI. We remark that contract-basedsetupofFig.4.Themathematicaldetailsand
ineachstepk≥0,theestimationaccuracy(cid:3)k andhencethe derivations are available in the online technical report [15].
delayδk arealreadydecidedinthepreviousstepandknown Experimentsconfirmthatthefollowingcontrollercanberun
to the controller. In the first step k=0, the initial accuracy in real-time, and its computation uses a negligible amount
(cid:3)0, the initial delay δ0, and the initial control input u−1 are of time relative to the estimation delay.
chosen by the designer.
A. Solution overview
Recall the operation of the contract-based control and
C. Control Performance
estimationframeworkaspresentedinSectionIIIandFig.4.
The goal of the controller is twofold: it needs to ensure First,theestimatorisprofiledofflinetoobtainitsdelay-error
that the reference pattern is adhered to as closely as pos- curve,whichwedenotebyΔ.ThecurveΔrepresentsafinite
sible, and that the energy consumed to fly this pattern is number of (δ,(cid:3)) contracts that the estimator can satisfy. At
minimized. Thus we may define two (stage) cost functions: everytimestepk,thecontrollerreceivesastateestimatexˆk
first, (cid:6)(x,u)=(x−xref)TQ(x−xref)+uTRu defines a andusesittocomputetwothings:firstisthecontrolinputuk
weighted sum of the tracking error (first summand) and the tobeappliedtothephysicalsystemattimeta,k.Thesecond
inputpower(secondsummand).Here,QandR arepositive is the contract (δk+1,(cid:3)k+1) ∈ Δ that will be requested
semidefinite matrices. Second, π(δ) captures the average from the estimator at the next step. At k+1, the estimator
powerconsumedtoperformanestimationofdurationδ.This providesanestimatewitherrorat(cid:2)most(cid:3)k+1andwithindelay
power information is collected offline during the estimator δk+1. Finally, recall that J = Mk=0((cid:6)(xk,uk)+απ(δk))
profiling phase. The paper’s formulation holds for much combinestrackingerrorandinputpowerinthe(cid:6)terms,and
moregeneralstagecostfunctions.Thesestagecostfunctions estimation power consumption in the π terms. The scalar
are chosen by the designer to achieve a desired control α quantifies the importance of power consumption to the
performance. overall performance of the system.
The tota(cid:2)l cost function that the controller minimizes is The contract-based controller’s task is to find a sequence
then J = Mk=0((cid:6)(xk,uk)+απ(δk)), where M ≥0 is the of inputs uk ∈ U and of contracts (δk,(cid:3)k) ∈ Δ such that
duration of the system’s operation. the cost J is minimized, and the state xk is always in the
47
set X. The challenge in finding the control inputs is that Algorithm 1 summarizes the RAMPC algorithm.
the controller does not have access to the real state xk, but
onlytoanestimatexˆk.Thenormoftheerrorek =xˆk−xk Algorithm1RobustAdaptiveMPCalgorithmwithAnytime
is bounded by the contractual (cid:3)k, which varies at each time Estimation.
step. 1: (δ0,(cid:3)0) and u−1 specified by designer
Fix the prediction horizon N ≥ 1. Assume that the 2: Apply u−1
current contract (under which the current estimate xˆk was 3: for k=0,1,...,M do
obtained)is(δk,(cid:3)k),andthatthepreviouslyappliedinputis 4: Estimate xˆk with guarantee (δk,(cid:3)k)
uk−1.Tocomputethenewinputvalueuk andnextcontract 5: for each (δ,(cid:3))∈Δ do
(δk+1,(cid:3)k+1),theproposedRobustAdaptiveModelPredic- 6: J(∗δ,(cid:3))← Solve P(δ,(cid:3))(xˆk,δk,(cid:3)k,uk−1)
tive Control (RAMPC) seeks to solve the following opti- 7: end for
mizationproblemwhichwedenotebyPΔ(xˆk,δk,(cid:3)k,uk−1): 8: (δ∗,(cid:3)∗,u∗k)←argmin(δ,(cid:3))J(∗δ,(cid:3))
(cid:4)N 9: Apply control input uk = u∗k and estimation mode
J∗[0:N]= min ((cid:6)(xk+j,uk+j)+απ(δk)) (3) (δk+1,(cid:3)k+1)=(δ∗,(cid:3)∗)
u,x,δ,(cid:3)
j=0 10: end for
Here, RAMPC needs to find the optimal length-N input
sequence u∗ = (u∗,...,u∗ ) ∈ UN, corresponding state Weprovethefollowingresultinthetechnicalreport[15]:
k k+N
sequence x = (xk,...,xk+N) ∈ XN, delay sequence Theorem 5.1: If at the initial time step there exists a
δ =(δk,...,δk+N) and error sequence (cid:3)=((cid:3)k,...,(cid:3)k+N) contract value (δ,(cid:3)) ∈ Δ, an initial state estimate xˆ0 ∈ X,
such that (δk,(cid:3)k) ∈ Δ, which minimize the N-step cost andaninputvalueu−1∈U,suchthatP(δ,(cid:3))(xˆ0,δ0,(cid:3)0,u0−1)
J[0:N].Intheremainderofthissectionwediscusshowto is feasible then the system (2) controlled by Alg. 1 and
makethisproblemtractable.AsinregularMPC[16],oncea subjected to disturbances constrained by wk ∈ W robustly
solutionu∗ isfound,onlythefirst inputvalueu∗ isapplied satisfies the state constraint x ∈ X and the control input
k
to the physical system, thus yielding the next state xk+1 as constraint u ∈ U, and all subsequent iterations of the
per (2). At the next time step k+1, RAMPC sets up the algorithm are feasible.
new optimization PΔ(xˆk+1,δk+1,(cid:3)k+1,uk+1−1) and solves
VI. CONTRACTBASEDPERCEPTIONALGORITHMS
it again.
To make this problem tractable, we first assume that In Section III, we postulated the existence of an Estima-
the mode is fixed throughout the N-step horizon, i.e. tion Error vs Computation Delay curve Δ. This curve is
(δk+j,(cid:3)k+j)=(δ,(cid:3))forall1≤j ≤N.Thusforeveryvalue used at every time step by the controller to determine the
(δ,(cid:3))inΔ,wecansetupadifferentproblem(3)andsolveit. operating point (δ,(cid:3)) for the next time step. In this section
Let J∗ be the corresponding optimum. The solution with wedemonstrateindetailhowsuchacurvemaybeobtained
(δ,(cid:3))
the smallest objective function value yields the input value for particular applications and how points along the curve
u∗ to be applied and the next contract (δ∗,(cid:3)∗). are realized at runtime by the contract based perception
k
Because RAMPC only has access to the state estimate, algorithms.
we extend the RMPC approach in [17], [18]. Namely, the
A. Profiling And Creating an Anytime Contract Based
problem is solved for the nominal dynamics which assume
zero process and observation noise (wk+j = 0) and zero Perception-and-Estimation Algorithm
estimation error (xˆk+j =xk+j) over the prediction horizon. Thefirststeptowardsprofilingacontract-basedestimator
Let x be the state of the system under nominal conditions. is to identify the individual components (or algorithms) of
To compensate for the use of nominal dynamics, RMPC the perception tool chain. The second step is to identify
replaces the constraint (xk+j,uk−1+j) ∈ X × U := Z parameters of each component, such that modifying the
by (xk+j,uk+j) ∈ Zj((cid:3)k,(cid:3)), where Zj((cid:3)k,(cid:3)) ⊂ Z is Z valuesoftheseparametersleadstoachangeintheexecution
‘shrunk’byanamountcorrespondingto(cid:3),asexplainedinthe time and accuracy of the component’s output. This may
technical report [15]. Intuitively, by forcing (xk+j,uk−1+j) be as simple as changing the number of iterations in a
to lie in the reduced set Zj((cid:3)k,(cid:3)), the bounded estimation loop[12]orfindingalternateimplementationswithdifferent
error and process noise are guaranteed not to cause the resultant execution times δ and estimation error (cid:3). We call
true state and input to exit the constraint sets X and U. these parameters knobs of the component. We implement
The tractable optimization for a given (δ,(cid:3)), denoted by this procedure on a Computer Vision (CV)-based object
P(δ,(cid:3))(xˆk,δk,(cid:3)k,uk−1), is then recognition tool chain. An overview of the tool chain is
shown in Fig. 6.
(cid:4)N
J(∗δ,(cid:3)) =mu,ixn ((cid:6)(xk+j,uk+j)+απ(δk)) (4) ObTjehcetCOVf Itnotoelrecshta(iOnOtaIk)eascirnosasvthideeofrasmtreeasm. Tahnedfitrrsatcksstaagne
j=0
of the chain is a pixel classifier that assigns to each pixel
s.t. ∀j ∈{0,...,N}
of the image (after potential pre-processing) the probability
x =Ax +B (δ )u +B (δ )u
k+j+1 k+j 1 k k+j−1 2 k k+j of its being a pixel of interest, i.e., of belonging to an OOI
(xk+j,uk+j)∈Zj((cid:3)k,(cid:3)) or being a part of the background. A binary image is then
48
ClaPsixseifiler PC1 PC2 PC3 τ1 τ2 τ3 t pixels)1800000
or] (
n err 600
CCoomnpnoencetendts CC1 CC2 β1 β2 t erceptio 400
p
E[
200
CSlahsaspifieer SC1 SC2 γ1γ2 t 0 0.1 90th p0e.r2centile e0x.e3cution ti0m.4e (s) 0.5
(δ,ε) (δ(cid:2),ε(cid:2)) Fig.7. Profileddelay-errorcurvefortheobjectdetectiontoolchainrun
atdifferentparametersettings.
τ +β +γ τ +β +γ
1 1 1 3 1 2
t t reflected in Fig. 7 which shows the mean perception error1
δ δ(cid:2) andthe90th percentileexecutiontimeforthedifferentknob
settings.
Fig.6. Illustrationofthevariouscomponentsusedtocomposethecontract
The final step is to profile all the possible combinations
basedperceptionalgorithmandtheirrepresentationasreal-timetasks.For
agiven(δ,(cid:3))contract,knobsettingsarechosenatrun-timeresultingina of knobs by running the tool chain on a test data set. This
schedule to execute these sequential components, or tasks, to respect the profiling gives us: a) the output quality (or accuracy) of
contract.
theperception-and-estimationtoolchainunderconsideration,
and b) information about execution times for the stages of
theperceptiontoolchainunderdifferentknobsettings.This
information gathered offline is useful for making decisions
obtained which assigns the value1 to pixels of interest, and
atrun-time.Fig.7showstheprofiledperformanceoftheCV
0 to all others. Next, filtering and a Connected Components
tool chain.
(CC)algorithmisrunonthebinaryimagetogetridofnoise
in the classification process and segment its 1-valued pixels B. Run-time execution of the contract-based perception al-
into disconnected objects. A shape classifier is then run on gorithm
each object to determine whether it is of interest or not.
Having profiled the components of the contract-based
In our implementation, the pixel classifier is a Gaussian perception algorithm, we can make run-time decisions for
Mixture Model (GMM) classifier, whose knob is the num- knob settings in order to realize a given (δ,(cid:3)) contract.
ber of components in the GMM. Fewer Gaussians in the This is the equivalent of selecting different versions of
GMM yield a faster but less accurate classifier while more tasks (knobs for stages) and scheduling them in sequential
Gaussians will result in a higher execution time but provide order to best perform the object recognition task while
better classification performance. Knob values that cause meetingthegiventimecontractordeadline.Fig.6showsthe
data overfit are discarded by a cross-validation stage as is different task versions for each knob in the different stages
standard. and the resulting schedule based on the knob settings for
the stages. The offline profiling allows us to set the knobs
The filtering and Connected Components algorithm are such that we can achieve a feasible schedule for the given
lumped into one stage and have a two-valued knob to deadline, δ while maximizing the utility, or the expected
choose between a 4-connected and 8-connected component accuracy of the perception algorithm.
implementation.TheshapeclassifierisalsoaGMM,butthe
knobforitisthenumberofshapefeatures(likeeccentricity C. Visual Odometry
and lengths of major and minor axes). In our experiments Another algorithm we consider and later use in Section
the number of knob settings for the entire chain is K = VII is the Semi-Direct Monocular Visual Odometry (SVO)
(#Gaussiansforpixelclassifier,#neighborsforCC,#features [19]. The visual odometry algorithm detects corners in an
forshapeclassifier),andhasatotalof3×2×2=12values. image and tracks them across video frames to perform self-
localization of a moving robot. These estimates are used in
Note that for any given component in the chain, the
the closed loop control system that flies the hexrotor, hence
relation between knob value and quality of output is not
it is important for the visual odometry to run at or faster
necessarily monotonic. The pixel and shape classifiers are
than frame rate in order to provide a timely state estimate
machine learning algorithms that need to be trained on a
to the control algorithm The number #C and quality of
training set before being used and like all machine learning
cornersdetectedinaframedirectlyaffectstheruntimeofthe
algorithms,theiroutputqualityforagivenknobsettingwill
depend on the actual data set. The same is a fortiori true 1Erroristhedistancebetweenthetruecentroidandtheestimatedcentroid
of the quality of the output of the entire chain. This is also oftheOOI
49
1
0.028
#C=50 0.8
0.026 #C=100 e)
#C=150 m
0.024 #C=200 n ti0.6
(cid:3)or Bound, ()00..000.120822 ###CCC===235555000 cdf: F(executio00..24 #######CCCCCCC=======51122350050555000000
Err0.016
0.014 0
0 20 40 60 80
0.012 execution time (ms)
Fig. 9. Cumulative distribution of profiled execution times for visual
0.01 odometry running on the Odroid-U3 for varying maximum number of
20 30 40 50 60 70
cornersfromtheSVOalgorithm.
90th percentile execution time, ( (cid:2)) (ms)
Fig.8. Error-delaycurvefortheSVOalgorithmrunningontheOdroid-U3 VII. CASESTUDY:REAL-TIMEFEEDBACKCONTROLOF
withdifferentsettingsofmaximumnumberoffeatures(#C)todetectand AHEXROTORWITHCONTRACTBASEDESTIMATIONAND
track.Theverticallineshowsthecut-offformaximumdelayandtheSVO
ROBUSTCONTROL
settingsthatareallowableforclosedloopcontrolofahexrotorat20Hz.
A. Experimental setup
To evaluate our methodology on a real platform, we ap-
pliedittoahexrotortaskedwithrepeatedlyfollowingagiven
circular trajectory. We use SVO (Section VI) as estimator
cornerdetectorandtheresultingqualityofthestateestimate.
and RAMPC as the controller. The obtained execution time
Generallyspeaking,detectingmorecornersrequiresalonger
distributions for SVO are shown in Fig.15. Details of the
runtime, and results in better self-localization as long as we
experimental setup are in the online technical report [15].
are analysing a feature rich scene, i.e., assuming acceptable
quality of the detected corners. Thus the number #C of B. Experimental Evaluation
corners is a knob which can be varied to obtain an error-
After profiling the performance of the perception and
delay curve for self-localization with the visual odometry
estimation algorithm and formulating the Robust Adaptive
algorithm. If the scene is not rich enough in features, and
MPC controller for the hexrotor linearized around hover
a sizeable fraction of the #C corners are of poor quality
and modelled as an LTI system (Eq. 1), we experimentally
(i.e., unstable or hard to track across frames), then we can
evaluate the tracking performance and estimated energy
expect the self-localization error to actually increase as the
consumption based on actual flights around a pre-defined
poor quality of the unstable corners detected adds noise to
trajectory. For comparison, we use a Model Predictive Con-
the visual odometry estimates.
trollerwiththesamecostfunctionandinitialfeasiblesetsas
in our Robust MPC formulation. The MPC controller is an
Fig. 14 shows the error-delay curve of self-localization appropriatebaselineagainstwhichtomeasurethebenefitsof
errorusingtheSVO.ThecurvewasobtainedonanOdroid- ourco-designmethod,asitisasimilarcontrolalgorithmthat
U3[20],whichisthesameprocessorastheoneusedonthe does notleverage co-design andis unaware ofthe estimator
hexrotor for on-board computation. For each value of the algorithm that gives it a state estimate.
knob #C (i.e., each requested number of corners), we ran For the evaluation, we fly in a predefined circular trajec-
thevisualodometryalgorithmonavideosequencerecorded tory, repeating the experiment 10 times to gather enough
bythedownwardfacingcameraonthehexrotorwhileflying datatoconclusivelymeasuretheperformanceofRAMPCfor
certainpre-setpatterns.Groundtruthforcomputingtheself- different values of α and MPC with fixed modes of (δ,(cid:3)).
localizationerrorwasobtainedusingaViconmotioncapture Note that since the controller was a sampled discrete-time
system which provides position estimates with better than controllerworkingwithsimulated20Hzcameraupdates,this
millimeter level precision. As we repeat each flight several realisticallyrestrictsustousingmodesofestimatoroperation
times, this results in a distribution of (δ,(cid:3)) values for each withdelayδ lessthan1/20s,i.e.modescorrespondingto50,
value of #C. We retained the 90th percentile values for δ 100, 150 and 200 maximum corners (see Fig. 14). These
and (cid:3), since these can be used as the worst-case estimates modes and their estimated power consumption is in Table
and delays by the controller of Ssction IV. It can be seen I. Note, #C represents the maximum number of corners
thatalargernumberofrequestedcornersproducesasmaller requested, (cid:3) shows the worst case error bound on the state
estimationerrorandlongerruntime.Startingat250corners, estimate, δ is the 90th percentile execution time for that
the error increases, however. We hypothesize this is due to mode,and P representstheexpectedpower consumptionin
the decreasing quality of the corners being returned by the that mode as profiled offline. Note, the computation time
corner detection algorithm. for both the RAMPC and the MPC was less than 1ms,
50