RADIUS: A System for Detecting Anomalous Link Quality Degradation in Wireless Sensor Networks

1.3 MB·

by  Songwei Fu

RADIUS: A System for Detecting Anomalous Link Quality
Degradation in Wireless Sensor Networks
SongweiFu Chia-YenShih YumingJiang
NetworkedEmbeddedSystems, NetworkedEmbeddedSystems, DepartmentofTelematics,
UniversityofDuisburg-Essen, UniversityofDuisburg-Essen, NorwegianUniversityofScience
Germany Germany andTechnology(NTNU),Norway
MatteoCeriotti XintaoHuan PedroJose´ Marro´n
NetworkedEmbeddedSystems, NetworkedEmbeddedSystems, NetworkedEmbeddedSystems,
7
UniversityofDuisburg-Essen, UniversityofDuisburg-Essen, UniversityofDuisburg-Essen,
1
Germany Germany Germany
0
2

n
a Abstract tectingsuchanomalousdegradationinlinkqualityiscrucial
J To ensure proper functioning of a Wireless Sensor Net- for an operational WSN to decide possible remedy actions
4 work (WSN), it is crucial that the network is able to de- suchastuningstackparameters[3,8]. Insuchway,thenet-
tectanomaliesincommunicationquality(e.g.,RSSI),which workcancontinuouslymaintainitsperformanceandsatisfy

] maycauseperformancedegradation,sothatthenetworkcan theuser’srequirements.
I
N react accordingly. In this paper, we introduce RADIUS, a InresourceconstrainedWSNs,detectinganomalouslink
lightweightsystemforthepurpose. ThedesignofRADIUS qualitydegradationrequireslightweightsolutionswithlow
.
s is aimed at minimizing the detection error (caused by nor- overheads in using memory, computation and communica-
c
malrandomnessofRSSI)indiscriminatinggoodlinksfrom tionresources.Resource-hungrycentralizedmonitoringsys-
[
weak links and at reaching high detection accuracy under tems [4,13,21] and/or machine learning-based detection

1 diverse link conditions and dynamic environment changes. techniques[10,19,20]arehencehardlyapplicabletoWSNs,
v Centraltothedesignisathreshold-baseddecisionapproach due to large communication and/or computation overheads.
3
that has its foundation on the Bayes decision theory. In In addition, a solution should be accurate with a low error
6
RADIUS,varioustechniquesaredevelopedtoaddresschal- rate (false positive/negative rate) and be robust with con-
9
lenges inherent in applying this approach. In addition, sistent performance under diverse link conditions and dy-
0
through extensive experiments, proper configuration of the namicenvironmentchanges. However,inWSNs,duetothe
0
parameters involved in these techniques is identified for an stochastic nature of link quality metrics, e.g., received sig-
.
1 indoor environment. In a prototype implementation of the nalstrengthindicator(RSSI)[2], itischallengingtodistin-
0
RADIUS system deployed in an indoor testbed, the results guishbetweentruelinkqualitydegradationandnormalran-
7
show that RADIUS is accurate in detecting anomalous link domness. Data smoothing [5] may only mitigate the prob-
1
quality degradation for all links across the network, main- lem. CDF-based [5,24] and Chebyshev inequality-based
:
v tainingastableerrorrateof6.13%onaverage. [11,15,29] statistical techniques are lightweight and seem
i to be effective in making the distinguishing. However, our
X
1 Introduction
investigation,astobeshownlaterinthispaper,revealsthat
r
a The performance of a Wireless Sensor Network (WSN) it is difficult to optimize them to achieve both high detec-
often deteriorates after in-situ deployment of the network tionaccuracyandrobustnessforlinkswhichmayexperience
[9,13,23,26]. Linkqualitydegradation,dueto,e.g.,fading diverselinkconditionsanddynamicenvironmentchanges.
andinterference,isoneofthemajorreportedcausesbehind To meet these requirements, i.e., lightweight, accurate
such behavior, which may be significant enough to impact androbust,wehavedesignedasystemfordetectinganoma-
the link’s performance, e.g., the packet delivery ratio. De- lous link quality degradation, called RADIUS. In addition
tobeinglightweight,itsdesignhasalsobeenaimedatmin-
imizing the detection error (caused by normal randomness
of RSSI) in discriminating good links from weak links and
at being robust in maintaining the detection performance
fordifferentlinksandunderdynamicenvironmentchanges.
Centraltothedesignisathreshold-baseddecisionapproach
(for beinglightweight) that hasits foundation onthe Bayes
decisiontheory(forbeingaccurateandrobust).
Tothebestofourknowledge, nopriorworkhasinvesti-
gatedtheapplicabilityofBayesianthresholdingindetecting
anomalous link quality degradation in WSNs. A possible graphsattheback-endserverfromcollectedsystemmetrics
reason is perhaps due to the various challenges inherent in todetectlinkfailures. Otherapproaches,e.g.,self-diagnosis
applying the approach. To address these challenges, vari- [9],avoidsendingallinformationtothesinkbyencouraging
ous techniques have been developed to identify the number multiplesensornodestoexchangeinformationforcoopera-
ofRSSIsamplesneededtoachievea“good”approximation tivefailuredetection. Allthesesystemsarepowerfulatde-
of the mean and the standard deviation, to update the mean tectingvariousfailuretypes. However, theyintroducelarge
andstandarddeviationestimates,andtochooseandupdatea communicationoverheadtoenergy-constrainedWSNs. Be-
“proper”settingfortheaprioriprobability,wherethemean, sides, most of thesesystems use metrics like packetcounter
the standard deviation and the a priori probability are the orretransmissioncounter. Thoughsuchmetricsenableeasy
threefundamentalvariablesusedintheBayesformula. detection of packet losses, they can hardly be used to de-
A prototype of the RADIUS system has been imple- termine the cause, e.g., whether a loss is due to bad chan-
mentedanddeployedinanindoortestbed.Forproperconfig- nelconditionorpacketcollision. Instead, RADIUSutilizes
urationoftheparametersinvolvedinthevarioustechniques RSSI, a channel quality attribute resident within every re-
in RADIUS, suggestions on their settings are given based ceivedpacket,whichdoesnotrequireactiveinformationcol-
on extensive experiments. In addition, we found that high lection,allowingfullydistributedanomalydetection.
detection accuracy can be achieved by RADIUS under di- Theresearchonanomalydetectioninwiredandwireless
verselinkconditionsmorerobustlyascomparedtotheCDF ad hoc networks is quite mature, but only a few solutions
andChebyshevthresholdingtechniques.Moreover,theover- can be directly applied to WSNs due to the limited mem-
head analysis and the detection results show that RADIUS oryandcomputationalcapabilityofsensornodes. Datamin-
notonlyhaslowoverheadsinmemory,communicationand ingandcomputationalintelligence-basedtechniques,suchas
computation, but also is accurate in detecting link quality clustering[20],supportvectormachine[19]andneuralnet-
anomaliesforalllinksacrossthenetwork,maintainingasta- works[27],ownstrongdetectiongeneralityandaccuracyas
bleerrorrateof6.13%onaverage. Theseareanindication longasadequateattributesareinuse[28]. However,theyall
ofRADIUSinfulfillingtherequirements. come with high complexity. In addition, they often rely on
The rest of the paper is organized as follows. Section 2 acentralentitytocopewithheavytasks. PAD[10]deduces
discusses the related work. Section 3 presents the system link level errors with a probabilistic inference model main-
design and motivates the adoption of Bayesian threshold- tainedataserver. Statisticaltechniquessuchaskernelden-
ing. Section 4 introduces the key techniques used in RA- sity estimator also require high computational capability to
DIUS.Section5analyzestheeffectofthevariousinvolved generatethedensityestimator. Arecentworkofemploying
parameters in these RADIUS techniques on the detection suchtechniquesisRASID[5], implementingthesystemon
performance. Section6reportsthedetailsofourimplemen- morepowerfuldevices(WiFiaccesspoints)todetectintrud-
tation, the corresponding system overheads and the overall ers. Duetothelimitedresourcesofsensornodes,datamin-
performance evaluation in an operational system on an in- ingormachinelearningorientedapproachesarenormallyin-
doortestbed. Finally,Section7concludesthepaper. feasiblefordistributedanomalydetectionsystemsinWSNs.
The most widely used anomaly detection technique in
2 RelatedWork
WSNsisthestatisticalmeasure-basedtechnique(e.g. mean,
Anomalous link quality degradation is a major cause of variance,maximum,self-defined)duetoitslowcomplexity
high packet losses in WSNs, reported by previous deploy- and high effectiveness of finding detection boundaries (i.e.
ments[9,13,23,26]. Amongvariouslinkqualitymetrics[1], thresholds). For example, Fine-grained Analysis [25] de-
RSSI provides direct channel quality information at the re- tects security attacks when RSSI changes exceed the mea-
ceiver, which is typically a required input for remedy sys- suredmaximumfluctuationoccurredduringtheinitialtrain-
temsinordertotunestackparameterssuchastransmission ing phase. In the statistical measure category, there are
power [8] or other layer parameters [3,30]. Many studies twooftenusedtechniques: (1)CDF-based thresholding(or
[3,7]analyzedtherelationbetweenRSSIandpacketlossand percentile-based thresholding), and (2) Chebyshev inequal-
studiedthetemporalpropertiesofRSSI[2]. Onlyveryfew ity-based thresholding. In CDF-based schemes, the thresh-
workshaveinvestigatedhowtouseRSSIreadingstodetect old is defined as the x-th percentile of the underlying data
good links (with low packet losses) turning into weak links distribution of the monitored attribute. An example is the
(withhighpacketlosses)withoptimalperformance,i.e.,ro- Memento system [24] where an empirical CDF of consec-
bust detection with minimal detection error. Our work is utively missing heartbeat numbers is used to detect sensor
relatedtopreviousresearchontwotopics:(1)networkmon- failures.AnotherexampleisRASID[5]whichalsodefinesa
itoringapproachesfordetectinglink-relatedfailures,and(2) thresholdatagivenpercentileafterthedensityfunctionises-
anomalydetectiontechniquesdevelopedforWSNs. timated. Inothercases,whentheunderlyingprobabilitydis-
Existing approaches of network monitoring and diag- tributionofthemonitoredattributeisnotknownapriori,the
nosis generally rely on active collection of node and net- Chebyshev thresholding technique has often been applied.
workstatus. Someofthemarecentralizedapproaches,e.g., Forinstance,Chebyshevthresholdingisusedin[11]totrou-
Sympathy [21] and Emstar [4], in which a large amount bleshoot the network performance issues. In [29], a fusion
of status information from individual sensor nodes (e.g., threshold bound is derived using the Chebyshev inequality
packetcounter)isdeliveredtothesinktodeterminethefail- fortargetdetectioninWSNs.
ure causes. Agnostic Diagnosis [13] constructs correlation Despite their low complexity and easy adaptation to
WSNs, both CDF-based and Chebyshev inequality-based
methodsarenotdesignedforoptimizingdetectionaccuracy. Normal Detection Agent Visualizer and
Control Center
In addition, achieving robust performance in detection ac- Profile
curacy by them is also a challenge. Later in this paper, we Monitoring User
Interface
showthatemployingthesetwomethodstoachievebestde-
Training Set A Priori Probability
tectionaccuracyimplicitlyrequiresmanualfine-tuningofthe Update Module Refinement Module
threshold parameters for each monitored link, which is dif-
y
fideccuilstiotondthoeionrypr[a1c2t]icteo.mIninRimAiDzeIUthSe,dweeteecmtiopnloeyrrtohre, Bwahyicehs ThBrMeaysohedosuilaldenin g iroirP A tilibaborPretteS Alarms VCC
is also a thresholding technique and has been widely used Sink
in other fields, e.g., signal detection and image analysis. Training Set Size DA
Specifically, we apply the Bayesian thresholding technique Estimation Module Basic Detection DA
Module DA
toidentifygoodlinksandweaklinksbasedonthemonitored DA
RSSIvalues. ItscomplexityissimilartothatoftheCDFor
RSSI Readings Collector DA DA
Chebyshev thresholding technique. Additionally, we com-
bine the Bayesian thresholding technique with several sup-
Figure1:TheRADIUSsystemdesignanditsfunctionalmodules.
portingtechniquestobuildarobustandaccuratesystemfor
Bayes formula) and two statistical measures (i.e. the mean
detectinganomalouslinkqualitydegradationinWSNs.
andthestandarddeviation)ofthemeasuredRSSIvalues.
3 TheRADIUSSystemDesign
Duringthetrainingphase,theBayesian Thresholding
Inthissection,wegiveanoverviewofthesystemdesign Module in each DA constructs the RSSI profiles (mean
and the architecture of RADIUS, followed by the introduc- and standard deviation) of good links, and at the end of
tion of its major functional modules. In addition, we moti- this phase, the module derives a Bayes threshold for each
vatetheuseofBayesianthresholdinginRADIUStoachieve monitored link. Note that, although the detection error is
minimalerrorrateandhighrobustnessindetectinglinkqual- minimized by employing the Bayesian thresholding tech-
ityanomalies, basedonacomparisonwiththeCDFthresh- nique, the detection error rate may still be high when
oldingandtheChebyshevthresholdingtechniques. the mean and standard deviation estimates are not accu-
3.1 RADIUSSystemOverview rate due to insufficient RSSI training samples. To allevi-
atethisproblem,RADIUSemploysaTraining Set Size
To achieve its goal, RADIUS adopts a hybrid approach.
Estimation Module before the threshold computation. In
Itcomprisesasetofdistributedsoftwaremoduleslocatedat
this module, we use a confidence interval to estimate for
thesensornodes, whicharecalledDetectionAgents(DAs),
each link the number of minimal RSSI samples required to
to detect anomalous link quality degradation along routing
produceagoodapproximationofthetrueunderlyingdistri-
paths. In addition, a central server, called Visualizer and
butionoftheRSSIvaluesandhencethemeanandstandard
Control Center (VCC), is used to monitor the performance
deviation. Thismoduleensuresa“goodenough”qualityof
of both the network and the RADIUS system. The overall
the first Bayes threshold while keeping an acceptable train-
systemarchitectureofRADIUSisshowninFigure1.
ingsetsizetoavoidanoverlylongtrainingtime.
Two phases. Similar to other anomaly detection sys-
tems, RADIUSrunsintwophases: atrainingphaseandan After the above operations, the anomaly detection phase
anomalydetectionphase.Duringthetrainingphase,theuser starts. Recall that RADIUS is aimed to achieve high de-
firstobservesiftheperformanceofthenetwork,e.g.,packet tection accuracy for links under diverse link conditions and
deliveryrate,isabovetheuserrequirement.Inthiscase,each be robust to environment changes. This is realized mainly
DAmeasuresandcollectstheRSSIreadingsofthereceived by the Basic Detection Module and the Training Set
packets to construct a “normal profile” for each monitored Update Module. For the former, i.e. to achieve consistent
link, based on which a set of thresholds are generated. In highdetectionaccuracyfordifferentlinks,thesystemshould
the following anomaly detection phase, each DA compares avoidfine-tuningoftheparametersinvolvedinthethreshold-
the runtime RSSI readings with the generated thresholds to ing,whichspecificallymeansnotuningoftheaprioriprob-
detectifthereisananomalouslinkqualitydegradation. abilityforeachindividuallink. Forthispurpose,theBasic
System modules. Choosing an appropriate anomaly de- Detection Module compares the runtime RSSI measure-
tectiontechniqueisthekeytoachievehighdetectionperfor- ments after smoothing with the Bayes threshold to decide
mance, especially when using the highly varying RSSI val- aboutwhetherthereisananomalouslinkqualitydegradation
ues to distinguish the good links (with low packet losses) ornot. LaterinSection5.1,weshowthatanear-optimalde-
from weak links (with high packet losses) as accurately tection accuracy is possible for all links across the network
as possible. To tackle this problem, RADIUS employs a by even a coarse choice of the a priori probability setting.
thresholding technique based on the Bayes decision theory. Inaddition,tocopewithdynamicenvironmentchanges,the
Through the process of deciding a Bayes threshold, the de- Training Set Update Module is introduced, which up-
tection error rate can be minimized, as to be discussed in datestheRSSItrainingsetcontinuouslyduringthedetection
Section4.1. Ina(closeto)Gaussianchannel, thecomputa- phaseinamemory-efficientway.
tionoftheBayesthresholdforadesirederroronlyrelieson To further explore the potential of the a priori prob-
auser-definedparameter(i.e. theaprioriprobabilityinthe ability parameter in achieving or maintaining the high
detection accuracy, a feedback-based a priori probabil-
ity adaptation technique is introduced in RADIUS dur- −70 GWoeoadk lliinnkk ((PPDDRR == 5929%%))
ing the anomaly detection phase through the A Priori Threshold
Probability Refinement Module. If the module in the −75
m)
VCC observes that the error rate of RADIUS for a link in- B
d
actretahseeDsAabtoovetuanecethrteaisnettthinrgesohfotlhde, iat pthreionriinpfroorbmasbitlhitey.Setter SSI (−80 false negatives
R
In summary, the DA on each sensor node monitors the −85
link quality of the links used by the higher-layer network false positives
protocols and fire alarms to inform the VCC and the net- −90
work administrator about detected anomalous link quality 0 10 20 30 40 50 60 70 80 90 100
degradations. In the training phase, when the network per- Time (seconds)
formancesatisfiestheuserrequirements,eachDAgenerates Figure2:TheoverlappingRSSItracesofagoodlinkandaweaklink
implies that achieving high detection accuracy requires to
locallythebestRSSIthresholdusingBayesianthresholding
minimizeboththefalsepositiveandfalsenegativerate.
(Section4.1)foreachmonitoredlinkaftercollectingenough
chance that a good link is misidentified as a weak link and
samplesasdeterminedbytheminimaltrainingsetsizeesti-
reducing the chance that a weak link is falsely viewed as a
mation(Section4.2). Inthedetectionphase,usingthegen-
goodlink. Theformeriscalledfalsepositiverate(FPR)and
eratedthresholds,eachDAperformslocaldetectioninclud-
thelattercalledfalsenegativerate(FNR).
ing data smoothing (Section 4.3) and adaptively adjusts the
Mathematically, such a decision problem of finding the
thresholdeitherwithlocalinformationupdatingthetraining
best threshold minimizing the error rate has been compre-
set(Section4.4)orwiththerefinementoftheaprioriproba-
hensivelystudiedundertheBayesiandecisiontheory. Inad-
bilitybythefeedbackfromtheVCC(Section4.5)toachieve
dition,ifthemonitoredattributeisaGaussianrandomvari-
highaccuracyandrobustness.
able (e.g., RSSI in a Gaussian channel), the complexity of
Remarks. In RADIUS, we have focused on using the
Bayesianthresholdingdecreasessignificantly,fallingwithin
RSSI link attribute to detect anomalous link quality degra-
the limited capability of sensor nodes. This motivates our
dation. However,itsapproachisnotstrictlylimitedtousing
adoptionofBayesianthresholdinginRADIUS.
RSSI, which may be extended to use other communication
3.2.2 ComparisonamongThresholdingTechniques
attributesaspotentialindicatorsof(possiblyothertypesof)
network performance anomalies. For instance, packet CRC To illustrate the high accuracy and robustness achieved
error rate could be observed for identifying packet colli- byaBayesianthresholdingbaseddetectionsystem,wecom-
sions,packetoverflowrateforindicatingqueuinglosses,and pare the detection performance of Bayesian thresholding
packetinter-arrivaltimefordeterminingnodecrashes. withthatoftwoaforementionedpopularstatisticaldetection
techniquesinWSNs: CDF-basedandChebyshev-inequality
3.2 MotivationofBayesianThresholding
based thresholding techniques, both of which have similar
Before describing the details of the Bayesian threshold- complexityastheBayesianthresholdingtechnique.
ing technique, we first motivate and illustrate its need. Its In a CDF-based (or Percentile-based) detection scheme,
abilitytodealwiththechallengeofachievingminimizedde- thethresholdistypicallydefinedasthex-thpercentileofthe
tectionerrorinanoisychannelisthenevaluatedincompar- underlying data distribution of the monitored attribute. For
ison with the CDF-based (or Percentile-based) and Cheby- a Gaussian channel, the resulting threshold depends on the
shevinequality-basedthresholdingtechniques. meanandstandarddeviationofthecollectedRSSIsamples
3.2.1 AchievingMinimalDetectionError aswellasonaparametersettingofthepercentiledefinedby
Due to its stochastic nature, the quality of a WSN link theuser. Theotherthresholdingtechniqueoftenusedinthe
canvaryrandomly. Toillustratethis, Figure2presentstwo literature of WSNs is the Chebyshev-inequality based tech-
RSSItracescollectedfromarealWSNlink. Theupperone nique. Inthiscase,thethresholdisdefinedasfollows:
(cid:115)
is when the link operated in a normal state with a packet
1−P
target
deliveryratio(PDR)higherthan99%(i.e. goodlink),while Tcheby=m+σm∗ , (1)
P
theloweroneiswhenthelinkoperatedinanabnormalstate target
withPDRbelow52%(i.e.weaklink).Thefigurealsoshows where, in addition to the mean (m) and standard deviation
thatthetwoclustersofRSSIvalues,althoughmostlycentred (σ ) of the monitored attribute m, P is a user-defined
m target
around their respective means, partially overlap with each parameter for the desired false positive rate. Similarly, the
other and no threshold can clearly discriminate them. This Bayes threshold for Gaussian random RSSI depends on the
isduetothestochasticnatureandthewell-knowntemporal same statistical measures (mean and standard deviation) as
propertiesoflow-powerwirelesslinks[2]. the CDF and Chebyshev methods and hence incurs a simi-
AnimplicationofFigure2isthatnosingleRSSIthresh- larlylowcomputationandmemoryoverhead. Themaindif-
oldcan,basedonanRSSIvalue,leadtoadefiniteconclusion ference is the parameter involved in the computation of the
withouterrorifthelinkisinthegoodorweakstate. Ourob- Bayesthreshold: theaprioriprobabilityofalinkbeingina
jective is to find a threshold that minimizes the misidentifi- goodstate(P(H ),astobediscussedinSection4.1).
g
cationerror. Asshowninthefigure,findingsuchathreshold Forthiscomparison,weevaluatethesystemperformance
involvesatradeoffdecisiontobalancebetweenreducingthe in terms of accuracy and robustness, with a focus on how
Percentile threshold Chebyshev threshold Bayes threshold

Link 1 Link 1
1 Link 2 1 1 Link 2
Link 3 Link 3
NR0.8 Link 4 NR0.8 NR0.8 Link 4
F Link 5 F F Link 5
+ 0.6 + 0.6 Link 1 + 0.6
R R R
FP0.4 FP0.4 LLiinnkk 23 FP0.4
0.2 0.2 Link 4 0.2
Link 5
0 10−5 10−3 10−1 100 0 10−5 10−3 10−1 100 0 10−5 10−3 10−1 100
Percentile P P(H )
target g
Figure3:TheperformanceofCDF(Percentile)andChebyshevthresholdingvariesdramaticallydependingontheparametersetting. Instead,the
performanceofBayesianthresholdingisrobust,providingnear-optimalaccuracyunderdifferentlinkconditionswithacoarseP(Hg)setting.
the performance is influenced by the user-defined parame- useasinglesettingofthethresholdparameterP(H )forthe
g
ters,namelythex-thpercentileforCDFthresholding,P entirenetwork,thusavoidingparametertuningforeveryin-
target
forChebyshevthresholdingandP(H )forBayesianthresh- dividuallink. Thankstosuchfeatures,weemployBayesian
g
olding.Thepreferabletechniqueistheonewhosebestdetec- thresholdingasthecoredetectiontechniqueinRADIUS.De-
tionaccuracyperformanceisleastsensitivetoitsparameter tailsofBayesianthresholdingandthesupportingtechniques
choice.Ideally,suchatechniquedoesnotrequirefine-tuning neededtouseitaredescribedinthefollowingsection.
ofitsparametertoachieveconsistentlyoptimalaccuracy.
4 TheRADIUSTechniques
Weapplythesethreetechniquesindividuallytothesame
RADIUS aims to achieve robust and accurate detection
datatracescollectedfrom5differentlinksinanetwork. The
for maintaining the network performance by combining the
linksareselectedinsuchawaythatdiverselinkconditions
Bayesian thresholding with several supporting techniques.
(e.g., line-of-sight, non-line-of-sight, no human movements
In this section, we present the Bayesian thresholding tech-
or frequent movement etc.) are captured. We record the
nique, elaborate the supporting techniques and identify the
false positive rate (FPR) and false negative rate (FNR) for
involvedparameters.
each technique. A detection decision is considered as false
positive(falsenegative)whenthetechniquedeclaresthatan 4.1 BayesianThresholding
RSSIanomalyisdetected(notdetected)whilethepacketde-
A classical example that employs Bayesian thresholding
livery rate over the link is above (under) a minimum (e.g.,
is the binary detection problem, as known in the communi-
80%). The values of the above parameters of each method
cation literature [6]. The goal is to detect binary digits “0”
arevariedinthesamewiderangeof[10−5,1−10−5]. The
and“1”inanoisychannelbasedonthereceivedsignallevel
resultantoveralldetectionerrorrate(sumofFPRandFNR)
with the knowledge of the a priori probabilities of “0” and
forthethreemethodsarepresentedinFigure3.
“1”. Mapping such a problem to our problem of detecting
From Figure 3, we can clearly see that the system link quality degradation, our goal is to detect a link either
performance with the Bayes threshold is consistently ro- being a good link (with low packet losses) or a weak link
bust for different links and is insensitive to the value of (withhighpacketlosses)withaminimizederrorratebased
P(Hg) unless P(Hg) is approaching the extreme, i.e., 1. In ontheRSSIvaluesmeasuredatthereceiverofthelink.
contrast, the system performance of both Percentile-based
Mathematic Basis. Let H and H respectively denote a
g w
and Chebyshev-based approaches dramatically varies with
link being a good and a weak link. Let E denote a detec-
changingparametervalues. Furthermore,theoptimalP
target tionerror(eitherafalsepositive orafalsenegative). Then,
forChebyshevthresholdwithminimaldetectionerrorvaries
based on the Bayes decision theory, P(E), the probability
significantlyfromlinktolink. Instead, Bayesianthreshold-
of detection error, can be expressed in terms of conditional
ing with a global coarse setting of P(H ) (e.g., any value
g probabilitiesasfollows:
from0.1to0.9)fortheanalyzed5linksprovidesclosetothe
P(E)=P(E|H )P(H )+P(E|H )P(H ), (2)
minimal detection error achieved by the Chebyshev thresh- g g w w
olds. Finally,theaccuracywiththePercentilethresholdisin whereP(H )istheaprioriprobabilityofalinkbeingagood
g
general worse than that with the Bayes threshold for every link, and P(H )=1−P(H ). P(E|H ) is the probability
w g g
linkwithanyparametersetting. offalsepositives, i.e.,misidentifyingagoodlinkasaweak
An implication is that while CDF thresholding tends to link,whileP(E|Hw)istheprobabilityoffalsenegatives,i.e.,
provide tight RSSI bounds of good links for achieving a failingtodetectadegradationinlinkquality.
low FNR, it can easily cause a significantly high FPR in We assume that the RSSI follows a normal distribution
case of high randomness and temporal variations, making N(µ,σ), which has been experimentally validated for low
the technique difficult to achieve a low error rate. In addi- power communication in WSNs [3,22]. For simplicity, we
tion,whileChebyshevthresholdingcanreachhighdetection furtherassumethatthedistributionsofRSSIforaweaklink
accuracy, its performance highly depends on the choice of and for a good link, while with different means µ and µ
w g
P .Bayesianthresholding,ontheotherhand,isdesigned respectively, have the same standard deviation σ. In other
target
tominimizethedetectionerrorwhileatthesametimeitcan words,theprobabilitydensityfunctionsofRSSIforthegood
theVCCinformsallDAstostartthetrainingphase. During
Threshold τ this phase, the statistical measures µ and σ are computed
g
fromthecollectedRSSIsamplesforeachindividuallinkrel-
evant to the higher-layer routing protocols. For the value
of µ , we choose the border RSSI value of the “grey zone”
w
Weak link Good link (i.e.,µ =−88dBm)reportedinpreviousexperimentalstud-
w
ies[3,8],whichshowthatPDRdecreasessignificantlywhen
alinkentersthe“greyzone”.
False
False negative The a priori probability P(Hg) required in Equation 6
positive is typically computed empirically based on previous expe-
rate
rate rienceormeasurements. Asthisparameterhastobedefined
before the deployment, the setting of P(H ) impacts on the
g
µw µg detectionperformanceoftheoperationalsystem. InSection
3.2, we have presented its effect on the detection error in
general. InSection5.1,wewillquantifytheeffectofP(H )
Figure4:Illustrationoftheprobabilitiesoferrorforabinaryclassifi- g
cationproblemwithGaussiandistribution. ontheFPRandtheFNRinmoredetails.
link f (x)andfortheweaklink f (x)areasfollows: Bayesianthresholdingaloneisnotenough. Despitethat
g w
theBayesthresholdisdesignedinRADIUStominimizede-
f (x) = √1 exp(cid:8)−(x−µ )2/2σ2(cid:9) (3) tection error, it alone is not enough to build a robust and
g g
2πσ accurate system for the detection of anomalous link quality
f (x) = √1 exp(cid:8)−(x−µ )2/2σ2(cid:9) (4) degradationinWSNs. Thereareseveralinherentchallenges
w w thathavetobeaddressedinordertoapplythethresholding
2πσ
technique. Someofthemareintroducedbythetechniqueit-
AnexampleofthesedensityfunctionsisplottedinFigure4,
self (e.g., accurate estimation of µ and σ) while the others
where the false positive rate and the false negative rate are g
arecausedbythefactthatRSSIishighlyinfluencedbyenvi-
markedasshadedareaswithrespecttoanRSSIthresholdτ.
ronmentchanges(e.g.,smoothingandthresholdadaptation).
BasedonEquations3and4,theBayeserrorP(E)canbe
Intherestofthissection,weintroducethesechallengesand
expressedasafunctionofthethresholdτ:
thetechniquesthatweemployinRADIUStoaddressthem.
(cid:90) τ (cid:90) ∞
4.2 EstimatingtheMinimalTrainingSetSize
P (τ)= f (x)dx·P(H )+ f (x)dx·P(H ) (5)
E g g w w
−∞ τ AccordingtoEquation6,findingtheBayesthresholdre-
(cid:16) (cid:17) quirestocomputethemeanµ andthestandarddeviationσ
g
WeminimizeP(E)bylettingd P (τ) /dτ=0. Wecan
E fromacollectedRSSItrainingset. Thesizeofsuchatrain-
then obtain the optimal threshold TBayes that minimizes the ingsethasasignificantimpactontheestimationerrorofµg
detectionerrorandtheresultantBayeserrorPE asfollows: andσandhenceagreatimpactonthesystemdetectionac-
1 σ2ln(P(H )/P(H )) curacy. Deciding the training set size involves considering
w g
TBayes= (µg+µw)+ (6) a tradeoff between the detection accuracy and the training
2 µ −µ
g w latency. A larger training set can provide a more accurate
(cid:20) 1 (cid:16) (cid:17)(cid:21) estimationoftheunderlyingdatadistributionthushigherac-
P (α)=Q α− α−1ln P(H )/P(H ) P(H )
E 2 w g g curacyinestimatingµg andσ,whileitmaysignificantlyin-
(cid:20) 1 (cid:16) (cid:17)(cid:21) creasethetrainingtime. Inaddition,thetrainingsizeshould
+Q α+ α−1ln P(H )/P(H ) P(H ) (7) alsodifferfromlinktolinkforthespecificstatisticalcharac-
w g w
2
teristicsofindividuallinks: smalltrainingsetsizemaysuf-
whereQ(x)isaQ-function[18]andαisdefinedas: ficeforastablelinkwhilealargersizeisrequiredforlinks
withhighlyfluctuatingRSSIreadings.
α=(µ −µ )/2σ. (8)
g w To address this challenge, we analyze the confidence in-
Application to RADIUS. Applying the Bayesian thresh- tervaltoestimatetheminimaltrainingsetsizeofRSSIval-
oldingtechniquetoRADIUSessentiallyrequirestofindthe uesforeachindividuallink.Inthisway,RADIUSachievesa
Bayes threshold TBayes, as computed in Equation 6. The goodtradeoffbetweensystemdetectionaccuracyandtrain-
useofEquation6alsoindicatesthatthecomputationofthe inglatency.Afterthetrainingphasestarts,theminimaltrain-
Bayesthresholdonlydependsonafewstatisticalmeasures, ing set size is decided by the DAs for each individual link
significantly reducing the complexity in comparison to typ- afterthecollectionofafewRSSIsamples.
icalBayesiandecisionproblems. Thesestatisticalmeasures Inparticular,eachDAfirstcomputesthestandarddevia-
include the mean of the density distribution of RSSI for a tion σ for the first N samples of RSSI collected in a short
s s
good link and for a weak link (µg and µw, respectively), as time period. Then, the DA estimates the minimal training
wellasthestandarddeviationσofthedistribution. set size N for a given error E . According to the Central
ts µ
IntheRADIUSsystem,theVCCmonitorstheend-to-end LimitTheorem,foranattributexwithanytypeofunderlying
packetdeliveryratio(PDR)ofthenetworkafterthenetwork distribution, the margin error of the confidence interval for
√
is deployed. When the PDR satisfies the minimal user re- theattributemeanx¯ise =z·σ / n,wherezisthez-score
µ p
quirements,implyingthatalllinksbeingusedaregoodlinks, (z=2.58foraconfidencelevelof99%),nisthenumberof
samples and σ is the population standard deviation. With a iscalculatedbya =S¯/T,whereS¯istheslidingwindow
p t t
this,theminimaltrainingsizeN iscalculatedas: averageofRSSIvaluesandT istheBayesthreshold.
ts
(cid:18)z·σ (cid:19)2 Duringthedetectionphase,RADIUScollectsconsecutive
Nts= p (9) RSSIreadingsindisjointgroupsofsizelupdateandalsotheir
E
µ anomaly scores in a separate group to compute the average
whereE isauser-definedparameterforthemaximumerror anomaly score. The group of RSSI readings with an aver-
µ
of the estimated RSSI mean. In addition, σ can be substi- age anomaly score of less than one is added to the training
p
tuted by the standard deviation σ of the first N samples, set. Throughthis,thetrainingsetisupdated. Thistechnique
s s
whichhastobelargerthan30[17]. is similar to the silence profile updating scheme in [5] for
ApplyingEquation9allowseveryDAtofindappropriate intrusion detection. RADIUS, however, needs to keep sep-
RSSItrainingsetsizesforeachofitsobservedlinks,achiev- arate groups to store anomaly scores because the threshold
ingagoodtradeoffbetweenestimationaccuracyandtraining may vary in the middle of an update process due to the a
latency before computing the Bayes thresholds. Neverthe- prioriprobabilityrefinementtechniquediscussedinSection
less, we need to find appropriate settings of N and E to 4.5,whileitremainsconstantintheschemeusedin[5].
s µ
apply Equation 9. In Section 5.2, we will show the impact Tominimizethememoryoverheadduetotheupdatepro-
ofN andE onthetrainingsetsizeandsubsequentlyonthe cess,weemployamemory-efficientwaytoupdatetheRSSI
s µ
overalldetectionaccuracyofRADIUSandthensuggestthe meanandstandarddeviationwithoutincrementingthebuffer
parameterchoicesofN andE foranindoorenvironment. to store new RSSI samples. Details of its implementation
s µ
andmemoryoverheadaredescribedinSection6.2. Inaddi-
4.3 DataSmoothing
tion,updatingthetrainingsetrequiresapropersettingofthe
As illustrated by Figure 2, the RSSI signal is random in
parameter l . In Section 5.4, we quantify the effect of
nature. Inotherwords,anRSSIvaluelowerthantheBayes update
l on the performance of RADIUS and recommend the
threshold may actually be attributed to its normal random- update
appropriatel valueforanindoorenvironment.
ness while not due to anomalous channel quality degrada- update
4.5 RefinementoftheAPrioriProbability
tion. As a consequence, comparing each individual RSSI
valuewiththeBayesthresholdT andusingthecompar- InadditiontotheRSSImeanandstandarddeviationthat
Bayes
isonresulttodecideaboutananomalycanleadtoanoveror aremeasuredandupdatedaspreviouslydiscussed,Equation
underestimationofanomalies. 6usesanotherparameter,P(Hg).Thisparameterisanempir-
ToovercomethislimitationandmakeRADIUSmorero- ical,aprioriprobabilitydecidedbeforesystemdeployment.
bust,eachDAappliesaslidingwindowofsizel tocompute We showed earlier that the performance of RADIUS is not
a short-term average of RSSI and compares the l-averaged sensitivetothesettingofP(Hg). However,aninitialcoarse
RSSI with the Bayes threshold to trigger an anomaly de- settingofP(Hg),duetoenvironmentchanges,maynolonger
tection. Intuitively, the choice of l has an influence on the provide the best detection accuracy and hence become out-
detection accuracy. A smaller l makes the detection more dated,requiringarefinement.
responsive, but it may not be sufficient to clean the noise. To address this challenge, RADIUS refines the a priori
Ontheotherhand,alargerl maybeabetterchoicefordata probabilityP(Hg)andthenupdatestheBayesthresholddur-
cleaning,butoverlysmoothingmayfailtocaptureabnormal ing the detection phase when necessary. Specifically, when
events. Tounderstandtheimpactoftheslidingwindowsize, an anomaly is detected and an alarm is triggered, the VCC
weshowtheeffectofl onthesystemperformanceandsug- keeps recording the number of false alarms. An alarm is
gestapropersettingforindoorenvironmentsinSection5.3. considered as a false alarm if the PDR over the path of the
anomalouslink, whenthealarmisreceivedattheVCC,re-
4.4 UpdatingtheTrainingSet
mainsaboveagiventhreshold. RADIUScountsthenumber
After the Bayes threshold is determined, RADIUS per-
ofconsecutivefalsealarms.Whenthenumberexceedsapre-
forms anomaly detection on the monitored RSSI values by
defined value N , the VCC informs the specific DA and
comparing them against T . While T is designed to alarm
Bayes Bayes theDAadjustsP(H ),incrementingitbyδuntilreachinga
g
minimize the detection error, the underlying RSSI distribu-
predefinedmaximumP(H ). Here,wecallδtheadjustment
g
tionmayvaryduetoenvironmentalchanges. Consequently,
stepandmaximumP(H )theallowedupperlimitforthepa-
g
the mean and standard deviation estimated in the training
rametertoavoidover-adjustmentsthatmayleadtoasignifi-
phase may no longer be valid. This requires updating the
cantincreaseinFNR.EachtimewhenP(H )isupdated,the
estimatedRSSIdistribution,aswellasT accordingly,in g
Bayes Bayes threshold is updated accordingly. In Section 5.1, we
responsetoenvironmentalchanges.
willanalyzetheeffectoftheinitialandmaximumsettingof
To cope with dynamic changes in the environment, we
P(H ). In Section 5.5 we will discuss in details the effect
g
updatethemeanandstandarddeviationoftheRSSIdistribu-
ofN andδontheperformanceofRADIUSandrecom-
alarm
tion during the entire anomaly detection phase. The Bayes
mendtheappropriatesettingsforanindoorenvironment.
threshold is then updated using Equation 6. Specifically,
5 SettingtheRADIUSParameters
weupdatethemeanandstandarddeviationbyupdatingthe
trainingsetwiththenormalRSSIreadingsobservedduring In the previous section, we presented the details of the
theperiodofdetection. ToidentifywhetheranRSSIvalueis BayesianthresholdingandthesupportingtechniquesinRA-
normalornot,RADIUSassignsananomalyscorea foritat DIUS. We now study the impact of the aforementioned pa-
t
timet,indicatingthesignificanceoftheanomalousbehavior. rameters required in each individual technique on the sys-
0.4 Link 1 1 Link 2 1 Link 3 Link 4 0.4 Link 5
e FPR FPR FPR FPR FPR
or rat 0.2 FNR 0.5 FNR 0.5 FNR 00..12 FNR 0.2 FNR
Err 0 0 0.5 1 0 0 0.5 1 0 0 0.5 1 0 0 0.5 1 0 0 0.5 1
R 1 1 0.15 0.4
PR + FN00..24 0.5 0.5 0.12 00..23
F 0.1
0 0 0.08
0 0.5 1 0 0.5 1 0 0.5 1 0 0.5 1 0 0.5 1
P(H ) P(H ) P(H ) P(H ) P(H )
g g g g g
Figure5:EffectoftheaprioriprobabilityP(Hg)ontheerrorratesfor5representativelinks.P(Hg)variesintherange[10−5,1−10−5].
σ)s1.9 Ns = 250 Link 1 1.2 Link 1 Link 1
d deviation ( 11..36 LLLLLiiiiinnnnnkkkkk 23456 PR + FNR 00..168 Eµ =LLLL 1iiiinnnnkkkk 2345 ning set size 110,010000000 Eµ = 1 LLLLiiiinnnnkkkk 2345
Standar 1 LLiinnkk 78 F00..24 Trai 10
0 200 400 600 800 1000 00 1 2 3 4 5 0 1 2 3 4 5
The number of samples (N) to compute σ Error of the estimated mean E (dBm) Error of the estimated mean E (dBm)
s s µ µ
Figure6:EffectoftheparameterNs. Theresultant Figure7:EffectoftheparameterEµ. Eµ=1dBmprovidesagoodtradeoffbetweenthe
σsbecomesmorestableafterNs=250. detectionaccuracyandthetrainingsetsize(i.e.traininglatency).
temperformance. Specifically,weelaboratetheeffectofthe FNRtokeeptheBayeserrorratelowforalargerP(H ).
g
Bayes threshold parameter P(Hg) and then explore the pa- Moreover,wecanseeinFigure5thattheoverallerrorrate
rameter space of all the parameters involved in supporting mostly stays low regardless of the values of P(H ), except
g
techniques. Basedondetailedanalysis, wegiveinsightson forthecaseswhenthevalueofP(H )isextremelycloseto
g
thebestparametersettingforanindoorofficeenvironment. 0or1. Theresultsconfirmthatthesystemperformancewith
For this, we performed extensive experiments, collect- theBayesthresholdisinsensitivetothesettingofP(H )as
g
ing real data traces from an indoor testbed, whose details long as extreme values are not considered. The reason for
are reported in Section 6.3. To capture different link con- this is that the Bayesian thresholding approach always tries
ditions, we selected 8 sender-receiver pairs (either line-of- tobalancebetweenFPRandFNRforanyP(H )setting.
g
sightornon-line-of-sight)atdifferentlocationswithvarious From the figure, we can further see that a global P(H )
g
environment dynamics (e.g., human movements and obsta- settingfromawiderange(anyvaluenotcloseto0or1)may
cles). Ineachexperiment,wesimulatedagoodlinkturning notbethebestsettingforeachindividuallink. However, it
intoaweaklinkbydecreasingthetransmissionpowerofthe canprovideforalldifferentlinksnearoptimaldetectionac-
sendernodefromthemaximumlevelgraduallytothemini- curacyatthesametime. Inotherwords,withacoarseglobal
mumwithapacketsendingrateof5Hz. Thereceivernode settingof P(H )forall DAs, theBayesianthresholdingen-
g
recordstheRSSIandPDRtracesformorethan15minutes. suresRADIUStodelivernearoptimalaccuracyfordifferent
Werepeatedtheexperiment10timesforeachlink.Themin- linksunderdiverselinkconditionswithouttheneedoftuning
imum PDR that decides whether a link is a good link or a foreachofthem. Forourcase,weselecttheinitialsettingof
weaklinkissetto80%throughoutthewholeanalysis. P(H ) at 0.8 because we assume that the probability of the
g
linksbeinggoodisgenerallyhigherthanthatofbeingweak,
5.1 BayesianThresholding
in our deployment environment. Additionally, to avoid the
According to Equation 6, calculating the Bayes thresh-
significant increase of FNR caused by the over-adjustment
old requires a user-defined parameter: a priori probability
duetotheaprioriprobabilityrefinement,welimitthemax-
P(H ). DifferentfromthegeneralanalysisdepictedinSec-
g imumP(H )to0.99forourdeploymentenvironment.
tion3.2,wepresentherethedetailedanalysisoftheimpacts g
ofP(H )onthefalsepositiverate(FPR),falsenegativerate 5.2 EstimatingtheMinimalTrainingSetSize
g
(FNR)andthetotalerrorrate. AsdiscussedinSection4.2,thefirsttaskofaDAbefore
Figure5showsthechangeoftheerrorrateswithvarying generatingaBayesthresholdistoestimatetheminimaltrain-
P(H )rangingin[10−5,1−10−5]for5representativelinks ingsetsizeN foreachlink. AproperN needstoachieve
g ts ts
outofthe8analyzedlinks. WeobservethattheFPRalways agoodtradeoffbetweendetectionaccuracyandtrainingla-
decreaseswithP(H ),whiletheFNRincreaseswithP(H ). tency. ToapplyEquation9,thecomputationofN requires
g g ts
ThisisbecausealargerP(H )indicatesahigherweighton twoparameters: (1)thenumberoffirstN samplesofRSSI
g s
theFPRinthecomputationoftheBayeserror(seeEquation for computing the standard deviation σ , and (2) the maxi-
s
7). Hence, reducing FPR is more effective than reducing mumerroroftheestimatedmeanE . WhilethefirstN sam-
µ s
P(H ) = 0.6 P(H ) = 0.9 (a) RSSI trace (c) Threshold updates over time
0.25 g FPR 0.25 g FPR m)−60 Bm)−75
0.2 FFNPRR+FNR 0.2 FFNPRR+FNR dB−70 d (d
Error rate0.01.51 0.01.51 RSSI (−−98000 200Time4 0(s0econ6d00s) 800 Threshol−−88500 Wlupi2dth0a0oteTu =ti mT5S0e4 (u0s0pedcatoen6d0s0) 800
(b) PDR trace (d) Effect of update window size
0.05 0.05 100 0.2
80 R0.18
00 3Slidin6g wind9ow si1z2e 15 00 3Slidin6g wind9ow si1z2e 15 DR (%) 4600 R + FN00..1146
Figure8:Effectofdatasmoothingwithdifferentslidingwindowsizel P P0.12 Without TS update
fordifferentP(Hg).Errorrateisclosetolowestwhenl=3. 2000 200 400 600 800 F 0.10 50 With T1S00 update 150
Time (seconds) Update window size
ples of RSSI only give a quick indication, Nts is the resul- Figure9:Effectoftrainingset(TS)updatewithvaryingupdatewin-
tantminimaltrainingsetsize,fromwhichtheDAestimates dδo w= 0si.0ze01lupdate. δ = 0.003 δ = 0.005
theRSSImeanandstandarddeviationforcomputingBayes 0.2 0.2 0.2
thresholds. NtsisusuallylargeroratleastequaltoNs. R0.1 0.1 0.1
WefirststudytheimpactofNs.AsmallNsmayresultina FP
partialviewofthecompletechannelvariation,whileoverly
largeNsmayonlyincreasethetrainingdelay. Tounderstand 0 5 10 15 0 5 10 15 0 5 10 15
the impact of N , we plot in Figure 6 the resultant standard
s 0.5 0.5 0.5
deviationσsforvariousvaluesofNsbasedontheRSSItraces R0.3 0.3 0.3
ofall8links. Weobservethatthevaluesofσs initiallyhave FN0.1 0.1 0.1
alargervariationandbecomemorestablewhenN isclose
s
to 250. The reason for this is that a small set of samples 0 5 10 15 0 5 10 15 0 5 10 15
is insufficient to capture the overall temporal variations of 0.4
R 0.6 0.6
RSSI,especiallyinanindoorenvironmentwheremulti-path N
F 0.4 0.4
fadingandinterferenceareubiquitous. Basedontheresult, +
wechooseNs=250forourindoorenvironment. FPR 0.2 0.2 0.2
ThenwefocusontheimpactofEµ. AccordingtoEqua- 0 5 10 15 0 5 10 15 0 5 10 15
tion9, thechoiceofEµ hasatradeoff: smallerEµ indicates Nalarm Nalarm Nalarm
higherestimationaccuracyoftheRSSImeanandthushigher
Basic detection TS update only TS update & Param. update
detectionaccuracy;asmallerE ,however,mayalsoincrease
µ
the training set size significantly. By varying the estimated Figure10:EffectoftheapriorirefinementwithvaryingNalarmandδ.
errors E , we plot the resultant training set size and error point,therealRSSIanomalyeventsaresmoothedout,caus-
µ
ratesfor5representativelinksinFigure7. Thefigureshows ing a significant increase in FNR. The impact of l is also
thatthetotalerrorratedecreasessignificantlywithasmaller relatedtothesettingoftheminimalPDR(PDR )thatde-
min
E at the expense of a rapidly increasing training set size. fines a good link. In our case, as PDR is computed over a
µ
Tobalancebetweenthedetectionerrorandthetrainingtime, sliding window of 10 packets and PDR is set to 80%, a
min
we choose E =1 dBm for our indoor environment, which small sliding window l is preferred to avoid the significant
µ
causesonlyaslightincreaseofthedetectionerrorcompared increaseinFNR.Forourcase,wechoosel=3,atwhichthe
to that of E =0 dBm while at the same time keeping N totalerrorrateisclosetothelowestforbothP(H )settings.
µ ts g
withinthescaleofafewhundredsamples,achievingagood
5.4 UpdatingtheTrainingSet
tradeoffbetweenthetraininglatency(severalminuteswitha
To be adaptive to varying environment conditions, RA-
sendingfrequencyof5Hz)anddetectionaccuracy.
DIUS updates the training set as discussed in Section 4.4,
5.3 DataSmoothing dynamicallygeneratingnewthresholds.Forthis,therelevant
As mentioned in Section 4.3, smoothing the noisy data parameter is the update window size l . We first show
update
duringthedetectionphaserequiresaslidingwindowofsize that the detection performance is enhanced with this updat-
l to reduce the detection error caused by the normal RSSI ingtechnique,andthenwediscusstheimpactofl .
update
randomness. To see the impact of l, we demonstrate how InFigure9(a),wepresentanRSSItracewithavalleyof
theerrorratechangeswithdifferentvaluesofl(windowsize RSSI values (between 300 and 500 seconds) indicating an
from1to15)undertworepresentativeP(Hg)values. abnormalsituationthatcausesthemonitoredPDRtofallbe-
We observe from Figure 8 that, for both P(H ) settings, low the expected performance as shown in Figure 9(b). By
g
increasing l reduces FPR but increases FNR, which causes adaptingthetrainingsetandconsequentlythethreshold(see
thetotalerrorratetofirstdecreaseandthenincreasewitha Figure 9(c)), we can see from Figure 9(d) that in this ex-
larger l. The reason is that smoothing RSSI is effective to periment,thedetectionerrorcanbereducedof3%-4%with
reducefalsealarms. However,ifl keepsincreasing,atsome updatedthresholds,downfrom18%to14%approximately.
interface DetectionAgent {
Furthermore, we also observe from Figure 9(d) that the
command void configureDA(Struct_Param parameters);
impactofl isnotsignificantonthedetectionerror(we
update command error_t start_Training();
setlupdatetobe50forourexampledeployment). However,a command error_t stop_Training();
largerl mayrequirealongertimetofillupthewindow command error_t start_Detection();
update
making the threshold update less responsive in some cases. command error_t stop_Detection();
command void update_RSSI(uint8_t rssi, uint8_t childId);
With such setting of l , we observe the detection error
update }
canbereducedof3%to8%inallexperiments. Considering
thatthetotalerrorrateinmostofourexperimentsislessthan Figure11:TheprogramminginterfaceforDA.
20%,suchamountofreductionintheerrorrateissignificant.
5.5 RefinementoftheAPrioriProbability
In addition to updating the training data set, one other
situation that requires to generate a new threshold is when
the detection accuracy degrades with an increasing number se
of false alarms, indicating the need of updating the a pri- tu
b
oriprobability. AsdescribedinSection4.5,weconsiderthe irttA
maximumnumberofconsecutivefalsealarmsN andthe
alarm
adjustmentstepδofP(H ). Wequantifytheeffectsofthese
g
parametersinFigure10,wherewecomparethedetectionac- Message Center
curacywithandwithouttherefinementoftheaprioriproba-
bility. Specifically,weshowthedetectionperformancewith
varying N and δ. We use the suggested values in the
alarm
abovesectionsfortheotherparameters.
FromFigure10,wecanseethatasmallerN canre- Figure12:TheMonitorin0gUserInterfaceofVCC.
alarm
duceFPRbutitmayalsocauseasignificantincreaseinFNR figuration messages sent by the VCC. The interface also
due to over-adjustment. On the other hand, a larger Nalarm provides control commands such as start Training or
makes the system conservative on the a priori probability start DetectionforexecutingthedifferentphasesinRA-
refinementandhencetherefinementlesseffective. Theop- DIUS. The command update RSSI is used to update the
timal choice of Nalarm falls at the location where the total RSSI distribution of each link during both the training and
error rate is lowest. In addition, the choice of the parame- the detection phases. With this programming interface, an
terδneedstoconsideratradeoff: largerδindicatesamore application only needs to react to VCC’s control messages
effectiveadjustmentbutahigherriskofover-adjustment. In andcallthedifferentcommandsaccordingly. NotethatRA-
ourexample,wechooseNalarm=5andδ=0.003.Withsuch DIUSisnotrestrictedtoTelosBorTinyOSandcanbeeasily
parametersettings,theanalysisofalldatatracesshowsthat adaptedtootherembeddeddeviceswithlow-powerradios.
basedontheaccuracyimprovementachievedbythetraining
The component VCC, running on a standard PC, is im-
setupdatingtechnique,refiningP(H )canfurtherreducethe
g plementedinJava. Itprocessesthealarmsreceivedfromthe
errorrateinarangefrom2%to5%.
networkandproducesdiagnosisresultsreportingtherelevant
6 ImplementationandEvaluation anomalies and their locations for the nodes that are experi-
Intheprevioussections, wepresentedtheRADIUSsys- encinghighpacketlosses,whichassiststhesystemoperator
temdesignandanalyzedtheimpactofitsparametersonthe in identifying possible remedy actions. The VCC also in-
detectionperformance. Basedonthese,weimplementedthe cludes a Monitoring User Interface (see Figure 12), which
DAcomponentofRADIUSforTelosBsensorplatformsand providesavisualizationofthepacketdeliveryperformance,
theVCCforstandardPCs. Inthissection,wedetailourim- detectionstatusandthediagnosisresults. Viathisinterface,
plementation and discuss the system overhead. At last, we theoperatorcanmonitorandcontroltheRADIUSsystem.
showtheevaluationresultsonthedetectionperformanceof We have also extended the VCC with an Internet-of-
theoverallimplementedsysteminanindoortestbed. Things interface, which allows the VCC to connect with
6.1 SystemImplementation openHAB[16],anopensourcesmarthomeautomationsoft-
In this section, we first describe the implementation de- ware. MQTT[14],alightweightmessagingtransportproto-
tailsofthetwomajorRADIUScomponents: theDAandthe col, is used for the communication between openHAB and
VCC.WeintroducetheprogramminginterfaceoftheDAto theVCC.Withthisextension,theusercanremotelycontrol
showthatitiseasytouseforhigher-layerservicesandappli- the RADIUS system and monitor the detection results with
cations,followedbytheimplementationdetailsoftheVCC anAndroidsmartphone,asdepictedinFigure13.
andoftheRADIUSIoTextension. ThecurrentimplementationofRADIUSisabletodetect
To ease the integration of RADIUS into higher-layer anomalies in link quality. Nevertheless, the programming
applications, we implemented the DA component as a interface (Figure 11) and the DA module can be easily ex-
module on TinyOS 2.1.2, which provides an interface tendedtodetectanomaliesofotherattributes,e.g.,CRCerror
DetectionAgent(seeFigure11). TheconfigureDAcom- rateorpacketoverflowrate. Inaddition,RADIUScurrently
mand is used to configure DA with user-specified param- worksforstatictree-baseddatacollectionapplicationsbutit
eter settings (e.g., the initial P(H )) as provided in con- canbealsoappliedtootherroutingschemes.
g