Table Of ContentAFRT-SHAR-TR-G4-
REPORT DOCUMENTATION PAGE oe
SF
THEPORT HE T BHYEE ERED oT
Moved interim ot Nad 1S Apr at 34 Qe 03
Sharoton ateton Thing Data Mining Techies Fagopenievanrs
TATTOR APR WEEE
Eibeare J eye (FancpalTvetraie) [* _
bon R- Few :
FET GRE DAN REA TRH, Sa EEE oR TOTEFERT
Center for Computational Siti
MS4aT mgs
"40 nvesty Drive
Fairs, VA 22030-4444 |
SCEFRED RAT RAS TY EATER A ATOR “E-SFOWRNGNTORS AERONTAGT
Offer of Sponsored Pugrams APOSIURK,
[AF Office of Sciatic Research
44015 Wilson Bid, Room 713,
Salinglon, VA 22008
an 29040315 008
SEUSS FIST WOT TE VE pins andr ing ata akg ae thane he au id SOA Ee
Haea rican nearance sie At Ferce pul, ple oF 8coo, ones slesizated by afer eamentson
‘rncpa Tovemighor ue vstarch fellows conde ear in the tvelgation of elwori-bascd inst detection ns dm
Tiree cpuiques bet on advanced compotion sess and viualiatctdniqus. These new mathods wil provide
Feeney daca the yressooe uf waver chris aperlig Ouse stam, passively perform voinuous user
eases lavern gable netwere tacks ame informalic-geteerngsctivides, and provide nerfoce mpport fr “storm
Rance agen! Resey africa detection ales, daage msessinent un eet retort stot eal,
feoming data, evolutioanry graphice, rateentete vaxfore
Geassioner | OrrHGES™ | Eevard) Wepnan, PRD.
REPRE 1 RESTARET’ —[ e THS AGE HR, ELEPHONE WOURER Fc wa
Uneosstied —|linetessited | Untied ati as |
Hid Fss1691
‘angen Fg RE RS T
Tsay, 2004
Dr. Tata Evol Fegpans
Spacer Grentscontac OFeer
OfFe of Spannored Programs
Air Fores Off of Seealie Rsoarch
4015 Wilkow Bootevarg, Room 713,
‘alinglon, VA 22208
Dectemasoe Regent: April 15,200] - uae 14,2003
Invern and Final Regort on Tntusion Detecoa Using Date Mining Techniques
Pris Toveesigator. De, Edward 1 Wega
Sponsor F3§20-01-1.0278,
ABSTRACT
rineipal Ivesigacur ant research fellows conduct ear in de invesigtion of network-based ntotin
Soistion ving ia yiaiag techriqes sed on lsvest compoztions) statistics ood viswization
Iishsiquee: These nev metods yl provide the tan in tect presence of envert channel uprsling
lect yet, pasiely,perkown continuous ysee autho‘icaton, viscom mbes nelwele atch, and
informa gettenngackvhie, and provide ilesTace support fer “stm conte” situational display of
incon Actseion certs, demsige sossinent, vd eucent cetvrk enent-bealth
ovERVIEW
ecurch dig the sporting period as candied in he devetop-ne of enulivarnt dts viosbzafon and
peel courivae dngly of ester deta wl, compression methodotoges of massive snd supermassive
Chia see preserving Gute uearty Zar fll sais) snelyses of compressed dit, ond opine
(Sroutioneéy matens and dispiny of ighevolame sirnsiag deta. This research effort hns rested io the
Sablcrdan ur sabia for pet ation of IM aves and a sprcil issue ofthe counte) Compatttonal
States ad Dots Anabjis wih to therae of Pa Psnclizainn, ané the prevension nf woe exes
senuis at profeeronalvanftcnoce aod workshops (ste Eel). Te adéton, we yuetiipated 13 conferenee ot
hope ends in altn, sgonised Workshop ested Workshop on Swtsec! and Machine Feareg
stgaee for ConipterTerasion Debcton.
Te
‘We have lnnched u doen onTctin. elu ented Free! Keeatea wih he agiecment ofthe Univesty’s
{Oto ceprve al healer fos fo al Inte wate to and cut of the Unive, primarily focusing
fn Tee, UDP, SNMP, and 4CMP protoool pickets. The iste "siffer™ (deployed a the Universite
Sew) aed avaysi Satin ac eilectively capable of ocarding a0 analyzing up a terabyte of eal
(ius Eaperhnent ibn cursed ststessber nical bai of 5,000 to 150,000 packets per hour We
fave intumed sbtesof det afi ily woierey- ate or 18-40 megabytes of beader afc per minus,
olding epprexiuately 50-50 piubylee of beaderinfounation per dey in che larger Universiny center
Jndsatunt sre hat ve observe ak miGh a 24 erases of hecder woe per year daring cera 2003, Muck
of he prsiet uate ie edminievaive uae from rorters ex oreves “chater”, aod unernosa seme five
findomeivalchaastr ange desing weekdays, We havs tren inveested io matin for ease deteton
‘Ot auunion stacks ur rious network aciides, x0 sa. analysis methods eapable of concurrently
Shncling bot soaring dite and Wisteria smsrary dat ue cecesary.
Collaberuive cunacls lave been essed wih the Mavel Sufice Warne Comer competer sorry
Ietsrehgvung is Deron, Vin osstmblish uy elect. and analysis of intro TCP header athe
‘ellecied auido the Unsenlyfaewall ving afvancedvisuclzaio and statstcalwesheiguss with the tfeot
tw torke eimpatisons of vompsrer inmurion vetvtos and methodnloges aruyed aga 2 relative open
Univesity computing infcashoente az apposed (Hose aayed against a aceured mihiry sex,
DISTRIBUTION STATEMENT A.
Approved for Pubfic Reteass
Distribution Uetirited
CONCEPT
Rochground
“Ta be of practical us, ony in'idarAstaton eff within 2 wsivesty vironment must rot oly weeping
The enmter ystems orgaieios' agaten to minln goed csevenk Sneed, bul is ebore® 10 provide
emzutatanal support toe civrse population af sytem vers, Qs one hewd, cere exes 0 veiled ye real
feuse of urpenc} te achieve a ely high fevel of systen-sid iovtneraliry, eapedally with specs 0
frateiges atuck, On the ue, he network aur sccwnimods zecourch aedvies frat ross 2o-pexationst
sry tate lat — rd se Scenional havoc eas a arr of "ace sarisity" ic ef eellepiate
finticnges To proceed, it i mevesnary ta formulte the concep’ of data mining for knowledge of iwricer
fctecton tn tna that rote 10 expected Gnd emcapceted) weer activities, nad the various operstinnat
heatives of eytem adminis and menugerschayged wth sytem integrity
eta mining & University computing network fo nario ceoction ram be emached in fee of deta mining 2
rice” of sid wide web. Such sv sfistor ie faced wth fe perceplion end ctciation ofa plethora of
enced “information” nm mafite lst. Fer example, ee sul funeral eve ae: epoch of tine,
oogrenie cen and ecrrplenity of eesti.
"Epnch of toe mey vpptar 10 be a mime, a Kes fo tho scose Ubu am itesion sist of cou
eceeocy take pleve ever se dation of mt, “he ea Hers, “bough, is tha he Behavior of an itmder
sll et ect both the ineplicn of an inte 10 coven a paroxlar type of melicious at, ad the meament at
Thich aueb fae Lennes veanitated at rume Level of porsitence io a meseurahte inert with the
mienet This Tromeat,cqtces 10 he Meant im time ur epoch fom ovbieh te inmucer's exbsequent
segnasveSchovier ea be reckoned ail ‘ht i iher the elit acids 1 "up tbe ante” af reat wallor
Test Tiettove,sinee krowedgy of particular baruser. cps ois a er behave, sad such bebovor is
[breed in come of mes ant uations of ime, then ef ie betavio: and afomnstion glee Sor tit
behovior een be sefenced eo once base epoch a a dicsemsble change in fehavlor wo a neces moor
baee epoch, [Environmeva! ie, one of diy forte local nctwork bela mine? and analyze, ca sls be
reckons to apochs, hat in Ihe sense uf "runes of fndemectal pestis (vonsnally doa) changes in
‘eaalve bohavior thatthe University betwor presents che world wide web]
Geographic exont is indeed e minermer. The idea hove is to “valve” citer the ergin (sour) of en
Frmin(s), andr is desta ents nerded a eerniee, Tis cation pple just as exch to topo
sin geographic location! ts gengrape exten! can be Weasured both as» collecivn of dace sites ahd
fa gceqreiic ave or region, ia Biko monner eam epclogy be measured‘ teons of network let,
lenient ceatonships, price ~ end erpecily the stato heal (SOM) of sxe. Pebins i ight bs
rere ayprepaat to use a fern such 47 gonrotagy to rele this Wasa of eharscetizction. Thus,
‘rarricige pleaod Fon deta ming a he aezcled with efermce na given gotopoleg.
Complexity speaks vulunes. Data ring in ence ie hs extrection of knowege fom a “wean” of dats,
{pfurration, ard otter krowlge By “weal implied tape, huge we see amounts of dts, depending
oh one's avaiable conpriaionl resources, bt cervnly tge chauph to ovestude cacti sale
Diocecurss. Aether papular term fee da reining ia uce iv hesledge iccawery in databass. nd wis ell
frum het ofl exstng dtabaess, the taoat emp in ra of hesetegenely,stuce, sd zelibiky
ven neceesiiity «the word ade web, The idea here ca relat the scale aa sensu doy of he
doa mining 052110 th compleniy of tbe infoaaion sevconmes in which the sepa ate cinbedaed and
the nace ef how the ruling develuped honaledge isto be presented. With ineteate in sole comes
oticoeal ance tska~ cawineratien cleaning sad flu, cection, canafermation ~ and with
fenscy deplaemcnd comes sptifatnn of fete atomge stvtegy. and dang of tusester procedure. The
Jeong Knowledge can thus be ssteiced with the develeped methodology of data capture as well as
‘compton, exfowton ond peetation techoigue
[As an stido, it shoal be noted that ret ansampus sytem mangers and admiistator would So hard
P s ‘an epplies t ths University network, Ard the general wee ~ even caper sees —
Ur chile The geaort reaction m "we're under etki pl th plog. There sa “trmpant paucey” of
‘hat reeds to be protects and by how sub, Thus “ere ft he clear abjecive ea ws meas We exert oF
tn intusice. Tod. one Iypialygoeper for everthing. Patch cseything. lnc xtepice virus protection
reygubere Whe we pu our nage fe be wie", we ony detec he presence of malicious stil, an ether
fn no: Fae ow a egy igtote atrsping to deter nian Fos
‘Rod ths sation only azplis te what he University eotopucs eysleres ongsnictian has adnvniative
[ato over Whar about free drm mova? Laptape? "Witeces cero sig"? How serio heehee?
‘Te capexiy ate which date msing feneonliies te employ he kinds uf pater that 9) honey
Ge dbconsed ie fuser copanded by the Releugateons sete of che world wide vet, aod sein the
Coivesaty nonwork ie sper rine ihe collegiate secon: evvirnnent Lover, data icing i ys
‘Tay uae the approad tat indeed Sen any rel Tepe of achirvg this deunting 19k. Te ooly erin to
reli te usdona yptoachos swing ugh ke to adequately sess the station at George Mason
Tnierdiy, copctlty ihe results wre to be menriogfilly cee: ef wit sins tac analyses of wel
Tul al Ske niverties, win cmnsezcal aa gevstamnenal es, and ia be neighborhood of erry
Universty Network Environment:
‘inn of dy for he losl Univesity commciy gover the genre curses of waa a8 meh a it
‘Jenwaraee ramp eoge if ne oe aveibiiy. 24 feasts eiscereble “gtctlized™ epochs that can be
dented kr afr week oye dorng vonceing semcstee (23 rom abot BID am, or 3, the University
Aint Cri o saupupr8 sha and “uly ive work (2) eb Lec continues to Sw on roma, whe
TClovus offs tor al apt acy deze to func and ste Bil he lb; (2) peak Wet contri fram
rope 4-30 pane when ech athe sate to slow eflcting he er of the work doy (4) asic cortines
seat Rea 430 pa han wad ood G00 pm, a he remit Act (5) Trem 620 phe tafe bus
oan ac on aegpun tealnts tp toi (vied) down soome dn cts rex to the echee! eke ve do
ce eeh cai and ths eanlznes uxt [00 ast, or osha (6) tere ean abropt drop i eat 1:00 a.m and
frei a 3G a, und Te eye begins wt ae
tthe rei
Ade to this exvinnment obs ovedaying ‘fie aking flor Univetsny Centers of Fxeafieace, special
srogems, project and avisics of uvted dau, ytooked for ilrption of savio: us tester
cee Shtesmpar uafic, aud dicot viral retwork wate, though the resting pico that he
Tetsiy get: presen sate cd ie nb is ore or less usant i intensity for mt pasts oF the
Caen Pane cenealyesareeacd oe fighlyhorregencous 258 Shunating across extent and thas
Sic iaigte neha gure sexosscctior of bdr, not al meio nh bel ee, 1 any eve
on cf the Cotvctety network neu pind deal move discptinel wad thveZore secure am otber segment,
ene pork. se chitefarkly ano-zau's id.
“Thin sin cunts what yuu night logically Bilin a business eto i he reigheehond of siltay st,
inhore ily veuing (vl tia fably soaseat rae, dominsies. However, is abso exerted chat in a
sretanl doviomgent rece wily aie wit AE atcndane exelent 1 “draw fe” Give eeicione
rea ian ftom seipeboring auurces tray drfsed tke on a ehaacier nate comparable 10 the Uoiversy's
Tots andi hoped hoe nalyrox contribute to Blping reve theveieoes
Data Captore Designt
‘The fns-captre effi! is reel: We huve deployed too test anit ecistion sengurs shag an anal
esdly datrenprre pogram for tining #74 dexelopmeot nupoees: nd have deployed bea opeeionat
aera Song marian capure progam Eoeed 6a pump To elne cea time a near res)-tine nets
ray,
‘The fet of the two eperssondl easers Ariadne i deployed "247° w the Uriverity frewal, and is
Taleciog cats in prosiscuous mide ever too fiberoptic eles olf © skrpont TAP a the Urivorsiy
Gateway one ttf i-coming afc nd one fro gg cof, Fe sessr Gel a RedHat Lines bok
rem tondurye based SHLATION! progats oveloped a the NSWC in algren, wig caprred tata ~~
tions RaBuenabled ecZvve esenbly, degved for bowly 4ovmlond to the malt sation Pheseut
jocuted'in the CCS folk lab lor rast eabime intucer Suvelenee onalsis and siumclvec
sar aks wale, Teteur ie cveflgwed with ao “12 eat RAID-ele hatdasvezsaerbly, and
Gaiam toby sle to dowmbnad and sere sucainaly 8 cele vcr of TCRAE tei header dt
“The sed operational yetor Dcedalu is dul bonaabe nt ether Widows XP ov Redfiat Lins, 1 ai
whic tie WonlywaXD cavrsneneiokaDy. Aue 4a avwilbitty of gryic Ubnaries. The design ito
nuttseoeely cece and malsee sremaing dt ih pouscaour wove in realaime over a single LAN
annesian over the CCS Gatasy tode ven an it tandem conga ef he MS Windews datusletion
bly windan anda C4 tage moteype program doveled in-hovse,
‘The to fet sal eveiaton sensors ire Busdohe when upeating in Radat Linas environment sing be
[elicvely High-rad vars fiendly dave sina joogzam elon and Dagiiaaer sing Saat 07 wun it
ka Windows 2000 Procavionmern, Both Pave and Deymatte are ecocsel with Those in dhe CCS
[Releceot ub, Doymosier alao hues the Teed Viewer {gait af the Knesies Project daabare tverane
fchovue), hich fe used 10 pepare skor-Sention da “esta” for doce
ped Costad Viton al Sanessoaalvauelization prugr,
nplietien sete ovo
Sanpete ec Fe
the ula: est and esslniun sens is Perthows 9 duabbactble lop'op sith both WindowsXP ronning
vada eed Snort ant RedHat Lint nin tcpdanp and ether. Pertous i wed to merit afie
‘Wades 2099 Poo serve het planned co beng pat of a prcllyperaining tnget amy
Kaowledge Development Desir
perstioel anlis of eaptored TCAD Meader calc was done in te modes: near realtine alysis of
nivenity-side TEDIP Neader wale: um the tialysisrsion Pout located ia the COS Hofnderk lab,
‘ehre sone 2! gigaiyton of data is 1 bs dowloaded huuny Fore lade atthe University fra; and
deabsnne isfyss tm Beséalug of sreucing PCAP beat tate on the CCS Galeny ned. Ta gain more
Jneiah irlo sltica) pererna gd ater "outer Selvin” the near seabtime abalyses perkeined oo
Theseus wil: supploncsed by war siedeph divealeecdy comprzetive analyses. The sucaming dia
Tieng seatz20 en Deda sing varius wr ne visualisation preg st under development should
{rovideanuher pospoetive cr Sesgn inet detection ecoigues and ceboduloges,
‘Ovel time, tis amtzsed thet roe iu tLe forms of ulesets sa he Tike witl emerge that allow
oes tod ancy of the aioe eve ince active, twigs Bath sensor Ser eaipulton a De
Shier wad sleéopment and puy'est He dee compression, anayicel qrocedures, and pesenaion
ehuncms a 2 aly sion,
Te cunusiry, i aticipatod that hull he more deliberate, sysccide taétéonal upp wsitg Univesity.
‘vide diay and the mere nerpaniaote, adapt rataie spszch sing 2. dats teaming methodology
Cnvched ith evblcnary gree techniques wl povidea degree of syncigim Gat citer wlone might
sp vcady seven
KQUIPMENT PURCTIASES,
‘August, 2002 — Purdove uf TGs dn!-CPU od 16B RAM cod DD. ROM “sniff” with halter
FRAND sezage with vo NIC for simincusaatonerscus Tslening a ighespocd data ans capably.
Deployed cnsice the University gecnay firewall wil Eber-opie connestiviy for vankimie stnsemous
“TCE header tei coleten op the University's in and out gotey vs
‘Apust, 2002 — Punches of J.67GHx dvsi-C) w! 269 RAM and DVD ROM “analysis stotio” wb 1.
{eats RAID slrege with 4.6708 DYD bare and 30 recordable CDs.
Doployei in he OFS main Inberstny CHotedsck") with TAN eonnectsty for peti downloading of the
‘TePAD header solic vallscted ky the “srifey” doplosed ot Ge University firewall, nd eontneons
autcnomone snzeming JCP herder tole elton the CCS Gaby Node,
AL'YLACATIONS DEVELOPMENT
August, 2002 — Praject Knoesos Database Taterface Appication
MS Acwse apnliation designed and wainsined by D. Fever, estahng and linking the equipment
prefieaions, nonwork topolgies, conerpral [ayeut, technica sues, pruect ebvonnlogy, report and
Sbafivence'wodkshap Secretion, ad = propa data detionwisavd nevrencata.
‘his uppliatin has ovctve nto ain datubaee Bunton, 2 pesswordprcieied backeut datlane for key
bakle an aellary minfdsioane of the local wpslogy (ato pucevord protected), ard a dat-viewsr
pplication that preperet st date for diet ingot Into the Center's iimensinnal dat visusizaion
sofrease, Coalition
‘he Min Database PruntRad (Figure Lo) pss an interface te Projet Konssasintadctory muti
fauipnert eperfewins, lucal fepology_ 4nd cansopial laseog project chrorcngy, bey inves and
Shefcracen, and catsel mmeties vod sovicwl incu fer TCPAD dela caprte ropestories, For
refecave peeses a ypesdings puede w the Tce Stow» Cencer for ap-to-se sans of rusian and
Telsed civ and tends, ung. with an inkisetive eum glewoty of (DS temminelogy, The Pejest
Tecunertsian Wistow {Figur Jb) vorides ey acces to eral Ted aticles as indeed btw i he
satan compet ent escnvation gion seston below, a8 ol a5 9 related WebCams proved nat by
fhe SANS Iosiine. The Key tables eed inthe Min Dutebuse Kec! Tu ase maintained an a passe
jueeced bch el eaebace Zor wae ot up-dee ad Koy dats neg
The ke opolegy ie pnuvide in a seprnte salary pooswcrdpscccted Jeatase de to the semitity of
eve specific snformelion whish it cuntns (Figure 2), Pot refrsnee pumpuses, a hotlink to mma for
Menhpt the una-besed ali osc to caicelanmp halle en x nensenk ie uko evi a wad t the
Gnuyor This allows ane o nly identify cal deve nls? and device Scenes. nant captned
Goa
[And ints the Tethee Viewer (Tguce 3) iso sgplcaion ft proves captureé TPA eal ce data
{ent cf eet short ro the slave deta vnuliztion softwere pram, Cesta, which
loys inerstive pale cuosfinarereater Hol visualization we:hnisues with greod touring eapsbity to
Sees se ecled se seta Uinem cembinatione of vaabes) head “nrosting saétical sere” To
feist he mist, ec fom sfews of to orginal dstacct apd ube prapatedlacned Utiee (ypieny a sobect
Sribe ov gital Ge) ace provided far sic by-4de comparison slong eta a convenience to modify lhe
Cited wd to ake te pod-Tied deve, nelary nce such a8 ussigoed pur ahd vendor oumbers,
unm ses, ae te ike on alan ready ecb.
weer? pois
ree wollt@imry| B88!
Se
Figure 18, Mato Database Frant-nd
a
Pare cauiae
es teston
pee"
eaters (TRE vant mts
agai a tea
igore 2.1oeal Topology Window:
Testbed Viewer
Frsininary Data Neier oe
pret
ue
Alternative Organization of
"Pack" Data
Figure 3 Teabed Vioner With “Packs” Supple Test Data Window,
‘March 7003 (on gafug)= Exolutiunaey Graphics CH Applica
is apstcaton is a C+ developneita envionment sppicetion design end womuined by the CCS
ANoguetineng group comprising E. Ween, D. Faron, 2D. King. Thi piston is desi wo aaly7e
id claplay smcning TERA bende ati acing 4 suite of ederored cvatodonscy graphic displays. The
Socom fat ist Be pipe-in ina data cape bi such 38 windup i @ WindowsXP covioameat or
Spdimp in a Reetiat Linon environment, iaplay design gectceion. level of resultnn, and degree of
ols acafoypleton taneparablity wl depend prsualy oo avai of eraphie Hires.
Sone examples of deta sisaliciion on Reeder dnt ce piven below, N ese diggs, we use ara. on 0476
Done and IP adder (ast too eetes) and detiration per set TP adresses (Bat ne owt). We are
Interested in cnnnely detection sttocsted wth posite intrusion atts
DIP.
Figure 4s Waterti Biogiam of Pstnarton IP vrai Tine. Bates epoch nrar bettors, newest epoch
‘noir Une inp, Pesistent verte sping iniates significant vontnning contse, Low ur der 1P
abaresses (to de le ype long De large Inlernet service providers
wey
SPort
‘Waterfall ingyen of Source Port versus Fime, The Tat 024 address are reserved for
jet such a8 rp snd fp services, cally telact and te Ske, Typically operating testes
increment the sovree port with each Hem paves, ence the sngonal Ines. Slope ofthe diagonals are
mice ave m digstie or periting stems. Ln tis
‘characteris of sflevent operating sys,
‘Sigraim mee port re trancated at 10,000 (here are senie 63,000 source ports).
