DTIC ADA293563: Sensor Fusion for Autonomous Outdoor Navigation using Neural Networks

27 Pages·2.3 MB·English

by  Defense Technical Information Center

.'. -,., I -.ýW- , .,.,.. , -, .ýý, . I.., , I -- .......:. . .I1 . ....1- ....,, ý, - -.! j .4ý -.. ý:. .. .........' ý . W%-. .,. t ., 1-.; ;
Iý,,;,%'ýý-..,4-I,- -'ý,-,i...I.;ý-ý..1ý.... ..'ý,ý4.ý.,.,',.I,-.ýW.ý1 .-?ý ý.,- ý. ýI ý ... ýý,I, ,.,.M.1,q., , v;..1. .,..; ".'1jr1 ;ý4,.ý." ..'-.ý : ý.-"zý" _;1, ý,,,,ý .ý,..ýj. ý .. ýýZ? . :;ý ,ý;,.,,-i;,:.s".L. ..Y -. ý,. i . ,",.I.ý .-.,..I . .,,-ý",. :,; .J'zý,. .;-. 'N"L-,._.:.,- I,ý . .; ý ;,ýW.Z-"" %', o.'. ;,- .ý,7. . -.ý-...M.; ýý ..,"t.ý'. ,,.ý .'..;ý" ...% 1... .".ý,.,,ý; .ý,. ,Y'ý.ý; ...;, ,.' -. . .% .."" ý -. .,;,'..-...1.,...' . .5ý, " 4 :,I z...ý,. ;_,. "'.ý. .,--;... ..ý ! . . .,ý.. :.;.: . .. ;.'"......-:.:.:... :.. ý .--ýiý ' ,-. ....ý. :ý,Iý-. ý ..-.,-'...iý..,-' o: ,I--,., Iý -w,.:ý ..ý, r.ý .,L.- I,.. ý,Iý, ýý..:.:I, , ".ý.., . , 1 ...K, ý .. .-.,:,. .; .: : .'u,;ýo...,1:..,,: .. :. .. ..-. !ý,'. - ; .:ý !,.-.:..ý.ý., ,. , ...rý.:...I'... ' .1ý-::. "Ij'..ý,-'J...--1 ..ý : ;,". ..ý1,"..J.,, :I ,1',... .r.: rý. ,ý ý ". ..,"- '.. ,. 1 ,X ,z,0-"ýý;..IL'I .,- _. !ý--.Ai" -.i.4.' ý!,. ;.'.ý'ý .5ýi .jA. : a ý.,- .ý-,I. -ý,ý_. :ý..,ý 'ý ý, . . ý. ; wz.,ýý, ýý .: ,-.JI,.ý. ,Y,; .. ý ,.ý -eM'' t, ý. z - _,.ý.--..5ý::,., ,'.;.,,-,. " ý ýL! '.-j. : ý,- 4R.%_'1.,. ' I "-..-,.-.ý.-I. . k . '.".:-ý ýM.7, ý ,,_ . fr;eý . ý7M.!,,t ý -. . .1., .... , Pý -ým1 Ar Pý;, M71 .-1M, ..q'-ý I ý ;: ,ýýV 'lv,: ýL! ý.".- n .W-.I. -..IV - .k',-... _ Ii.4,;W....ý..",.,; . _pý,. ..-Z41:.;,1-M-,:".,I,1,".,,E.' ..-1-ý.t.3-,Qý ..-,-.ý1.-M .., ,_,1 ý . ý1. ý,,.. ý6ý-,ý ý .ýO ,-'1 "1_,,A . I" i,. .g,ý,.'..'.,ýý R..,".P .ý'.,.- -"'.ý .%ý. ',` . i ;ý- . ýI,']I.%k ý'ýý. ., .. ;.'i -- I1-ý- : .X!.iý.4,,ý.ýZ .'I;r .. :..k:A ._,-ýý -ýý.I.,.ýc l ý-.I.-- '.ý 4'ý. .ý ý,. !ý.1l- . .ý,ý.ý.-.ý , .,- ...,..1'., ý.. .,,ý;;.ý-I, .. ý.ý,,k ý.-...%;,.- . ,1_.:.-A, 4.- A.ý-." ,-:.' _n,,.... : . ; .., ý., .!.,.,. ;1I.,;,-ý, .7f"o--ýý.;,I ýý , I: ,. -..I? ý _ý!.I 3 .".-1",ýýý,- . ; i ,.ý, ý-'.I,I', 1-V-,L..1ý. -" ; ý,ý: `;ý: '... .o.;,,.%I-ý_.ý_'..---Zý',ý1, ý, I1., ý A 1ý .ý,I, . , .. zý., ý... --ý.: , r.;ý.0.,k. . .";'..:. - ý;'ý ý': 1,,!. O;F,,. ,.-..;ý.;..-.N;i :; a;,ý, :I .. -..,ýý,,:O ' .-, ,ýý.,ýý1.ý... ,E, ýW- -ý: ý,fýý ..I, :L.ý I 51; ; ,k-, ý.,',-..-j''Iýý.,., , .. , '.,.1."ý:.A:. ._, .. .,.t " ýýý ..ýýý. ;1,ý', .._ý ...I!ý..L'.;:ý.. ! ..ý ,.!..;-ý.-ý Iý-.ý . .ý . . M_- -.,..:.I,ý, ...4 ., . ! ....."ý.i:, ,!;ý :ý , :ý :,-:: .e ..:I. . .;,-V ._ ý% . .' ý_.. ,%I,.r-ý.ý 1 ,: . e..VM ',-m.ý. .', , . 'ý. ý,.., %ý, - . ,.:ý_!, ,.i:. -- ý...,,, , .;.... .,..:"..,.ý e-1-. .,,-.'-_W.ýý. -.;._.. ., m. -,. ýý . ...'.ýý- ' ,, , ..:, ...r.-..Z-..- 1.:ýý.....' ..-L. .e.,ý ý.ýýý'.-. , ;,4,r ý .l4!_ , ý..... -,ý..ý., .4 % ý...,.. :.k,ý ,ý :, .&' - _. L ,_.....,.--ý ýN .'. ,.ý .- ý. -. ,.J'1",ý4I "' .......L..-,,ýý.,:.r' !%.4.) ý-..1:.. .'.,, L5 7 ,j. ,. ,-L ýýý.. .,ýý : :.;,: .t-...,j 'z- 1,ý":. ý...;'; ý;Z ; ,6:_ . : I.c.ý';i-. .-ý ,-,...._... . .. _Ix.V,ýZI,...ý.2 i;i;4V, . ..-ý,-- .'% ;..a ' .%. .ý.',ýýý..,._ .', , ".;?..-.,", e ,. . Z; ..,f.ý.,ý1..- .n.,...-.LI . Ma,:,4,, .,,,,..... I. _1,_- ,; .'ýF.,,,,;..-, '': " :.ý I .'I,ý." .. . ,.i ., ý L;W":Z.&jý.,.? : ý.;"ý .ý-" ý ý " ..,ý ,-..J..ý.L ý ,',''I,.'.1 -' ý -.ý !z.. .:., .ýý vý ..z. .,,.. .-ýrIý.., ,,!.. . .. ?,,--,,4 -' ý :,v . N,.'%.. ý . 4ý;ý.ý; '-..'.ý....ýý-.-. ..-.ý-,< ,,ý1ý. ý _ý,..,2,..' ý- ;,7,.; ý--ý-, -:,. -'' .ý 1 ý". -4,p..ýý:I; , ýý,. ..;." ;ý.- ý .i'''."N.-,.'I;.'.,,ý-1 -, ., .,,-I ... .. ,.. 1, 0 " _., .J r,I .. - ,, Y ý.ý,;.:I- ý . ý. g -ýU, . j.-ý..-i:NAý. .-..,"1....-.; .. I.. ,:.ý..:!1, .. l,;,., . ;: : ..- ,.ý1.--. ý..,--...-;A . ., --a ,,-;ý -U .,.-ý.." . .:: -ý6 .,I : 4 ,.tI, _..b,, ý0 1 .; ..U".. 4 ,ý 1i.-b1,-. ..I ." . ý Jw , i..,.% o_.ý,I...ýL..l, .0_ ,w..4.. I,:I0 4' .n....,Iý1ý-ý. ým . ," '1.ýiý:, ý.., ýk.I. ý ,".ý1 ..; ý. .:.: ..-.. I.-, e" . -4 ::.I;-,, .ýýý.ý-U. ,Pý . ý ._r i.-"!ý.ý".ýý ._.ý .'..-: , , ...4,...J ,:-..,L ..,....'.` I.,.._ý, . I ýI, .s..I ,oM' ..,.- ,-,..ý. A.ý..I , ,,iI.;.ýý.A! ,""ý;_rý'.. ný-l,.e _ý z-.%:ý."I.., ýlk ,'.-.,,- -X. ýi .:.,i,;,M .g rAý,."ý;1;,-.. ',--ý..ý, .._. -,Lý.ýý ?:ýý.._Oý: ý-,::Iý . :-.-.'-.., .I%iý .rý ..L0 , .fý-".-;1,ý :ý_ .,-,. 'Svý,-.ý.!.. - " .; -:Iz,,_' .," -%- .u.-! :; Y. .,Iýý'N. IýR..ý_e%. , ,,,N.. .-, '. ,ýM :-,.ý ý,.--ýý . v. ..ý,.. , ,.ýý.1,-!%ý ý:. . ý- fI .i1 L t O.I-;-- ,' ,,: ,_ ., . ý kX'. : o-.eM"ýI1ý..' -_.1".! ýý ý , ý.ý-'4, _, ', _u.ý ýi.n.ý,_,,.- . I, ý... ýY_. r-o",.ýýýM.1,.,, a.,1.,.:. ,;. O ..:.ý!:.".: .'-1lý -.."ý.' 'M1aý 1 ,ýi. : ..;L :IN :,, 'Io-ý.-'.-', .r, feý 1ý z. .., ,,-, ýI_W Iý; . .. Ru0 ...1.,. ý.,; ..1ý.-, .e W, ,4 - 1'. .r_,i: FL.. ., Ms..,, 1ý ,. .,;s,--,. .ýýIý. ý"1 ,. ...,iý:.t -I ....ý.%;.. . . ,,. ,%.ý,.-,-.ý.I ý.. " ý. -.,,1ý..I. iýýý ,iR7.,I".v ý ;.. i.ý ,A .--.-.- ., 7ý1 ý..-.. .tý w ..-, ý.-F.,- . ..-I I.-" ý 7, .. .....-... :...-_I_. .I . ,rý, ",ý ý. ,.ý,1,I !ý :;.ým .iý- ..':..O - ;ý";, . :.ý.-.-.... ý . ,-.6". .., .-FF.ý ý,..:.":I,"ý:..'.z%-...,& ". .;.1',:1T-, -.i. , ., -';".ý..I_.::ý%ýI..F 1Iý ; ,.ý!: ... I-j.':.: I. 1ý .. ."---.'I.. . "..ý.4 . , , ,i ...,.'!a . Lr.E,' ,;, ý.,".t Ij . .,ý-o..ý1 1 ....ý ..':A. ..-.!. .R :, . .,,,.- . 'Qý,.".4 ,.7,1.,ý i 1e ýý",r;, -:,ý -.ý'-. ý'." -ý.I.-,u9. ý. ._e1 ;"; .ý,,_.ý' -ý'! 7 .. ,L.ý. ..-,,_-7...V t, ýý .:.,i.ý.. ,-.,ý:- ý , ,1r.. . .ýý,ý,ý,1. _ý ýIý 7. .J..,. .;ý . r-ý.',1,.,- ý ýv1_ 'V.;,. . z .7,;-ý '....- -ý.,,t .,..ý .-ý..lý,i,l, 1.k,;..ýýk..ý.,.'-, ;ýý! ý_,,.-ý-ý , ý, 0 ; - ":.1.I'_ ;.ý ,. 1. ý-%Iýý..,tý.. " s ,1 .d', ."-, ý .ý.t t .Lý",ýI ..L ..,y.- . ., - ý . , -1_ .,ý. ' I .,,O_ ,ý .%''., ,:.... ,l.ný;ý . -- fE7ý,.ý ",ý' ý,ý.%.ý ý.,-;s'.e, . ý P. .ý...f;;'ý- . .Y .', IZ,,_:. ..' ,..ý"..z, -. o., z.ý.,1' ý.ýZ: .;'ý ýý,,' ''L.., Z -ý.. ý -..,.A, ...,:?., .Yý ,,I4ý` , :ý,,"..4V:. -::. ;.. . ý,ý .._.1,-1,ý?ý:!.I",.ý ,-ýý . . , ý.'-,. ,..:..!.;'ýýI1,..ý ,;4.... ýýZ ý'.." ý; Iý :I-ý ,ý ý ,ý .T"ý. Ji.,ý ,:::Z .:m'. :,.ý,ý, ,,--.: ,.ýý_ ý. ,..:.ý;_.! ý,ý...ý.-.;.ýi'.'.l I: ý . " ,." _. 1ý..-": .: ...: ,. W'i:1:.ý. .-L 'ý .;ý ,,r.-.,,I : : ý :,. ..,. _ý., - , O . .,-ý.ý '".... -"I7 ý - '.';'M. ý, .,.;,_.ý. _ý!. ýý ý'. . .eý...- .-. . .%1 ,.,-ý ,,ý:.'"... '-.: ,, ý'7........c,.... , ',1.' ..ý- . .f; '.. .......;v.z ý., ..-".ý,..e. ::. , .;e',.. ,,,. - ý.ý-,.. ýý . ..1 ,-K..,ý': , ;'. ". .-...:.. . .... ,I, . ,; ..,ý. -.-..2 -,..ý.ý. ,,- ':... j ý-..-..1-..., ;,.,..-MýTý .7, .:..!,'... A .: .., I._.ý1.. ._ i... . , ... - .L.I ._% ... .,...ý .. ,',. . ..I',. .,-.;-7.. .. ....- ..ý ý1- ......-.`. ..,,ý.,.I,.ýý..... .,. .ý1:. .I -- -:T..I,.,I - " 4, 1..4,..%q . Iý"PIMýVi1CYIý.ýýM.:..,. . .."...k...z;-.i..1;-. "..4 ,ý,... ... ý!aU0ý;.,N4 ..ý .ý .''.Al .0,e .I,,,"J1ý r_;,1 Z ., ý.Zi. ý.ý'4 -I"Aý ;ý-"ý54 . 4.-.ýI-.,___t ' . ý.., ,.I .,4Z-.._- ,.c , N5 , ý.: ý1.A'....ý %.a ... -;, .ki,.ý, ý,ý'.ý._.,-1. .r- ..., ,,ý,: c.',ý..-,,ý :C''ý:..' ..;-,,.ýý- . ý,?N..r. J.. ý: .-:ý"ý C,I,-.-;,.."' ,,%ý'"ýý : 'I ..ýý? ,5ýýý; ý i_i _ eý,"ý P 0ýý., ý ,ý-z,., ý . .ý-ýLý _z.,ýý'' zýý. ,,7,; , ..ý,i,..,ý. ! o.IL..ý. ý,J,.:f1 ,ýý.,ý ..-ýý. ..,.,.ý, ;ý ,,, .:1o.ý 'ý. .,,. z,-X...":..--.. ;; ý,- , ý ..ý8 ,"-,1 ý .ý-,AI.,-.,,.. ,ý;L K,!,,ýý _..,.. .%.,. ,ý".-nýIý,,ý;. _'.-,ýý -.., "%..ý. -: ýý ,:. . ý , ;:'..ý,.I,.ýz,"ý.dýf..ýý.., ..1. :. .ý ý..,1.',. ;- ...; v ý "-ý .'j_.;_ýýý7_ .n I.: ;,'ý%... -,;.ý,...-..ý...e1;.."ý, ..,". ,,.."-..,.."-"_..,ý,.M.,%ý .. .. %... .,,ý .'.ý.. ...ý ,,. . ý ,. "..- 'r.. ,..,;.. . ý ,'. :.ý...... I, I" .-. . . . ý,-;!; ..,.ý'-.I1 .ý '': % ...ý'..ý ....-.....-...;-L.:_ ...; . ,. ."......,..;.-.ý%"'z' ."i.7,7 ýt..,-.. ,ý ....Lý,ý'- ...; .._. ",ýK.,W ._:..-.,.;....ý.: .. .,, ... ý "-.. ., .v......I._ . ._" ... .., .; ý,:.ý 0. . iýC:ý. ..ý..-L,.,. ý .... .;,-....z.ýc..., i-4. . ..,I ,.z.ýý-ý.i. J.. .,-:. .. 1ý...,.. . - . ,ý .Z-'...z ,,;,- ý -' ý.. .:.m.: ., , .-..ýOýý. , ý..:.I. ,.. j .V.-.,.-' ýý.,. ,.... ...i. . 1ý0 ;.ý. . . ,.,.;I.......:...ý.- .. ;,..ý'm . . .-I..ý..... .,.-.,-. , ;. ýj .-..:........ -;.'.,"Z_.."',.ý L,ýý,. ;..,.ý.I.ý . ... .. . ......., ,..,ý,. .: ..' ; .ý .,...ý;. ,. .ý:,. ,';;ý.,' . ý... k... '.. ,.I.. -ý._ý.ý. .. . ý ;-.;;--..1... -.. ,_ý..ý . ý:ý .ý eý...I-.....'.. . . ý ,. ý,.ý.....ý..ý,. 1...ý' ; ý:.. ..,'...I-.1 , . .... .....:ý1.. ý.1;. .;t . ,Z. ,:.,.. .e-..-. !:.,ý1.- '.. , .Iý . 1.4.,,. .,' % .- .,,.ý'. ...iI: . . ý. .:.!,..IJ-ý;,,ýL..5_, ,. ý.ý', j_.,.ý,ý2ýý'!Iý-- 1' , ..;ý ..ý...... .,,! ý-; ';! :ýý. ý .1 ...!: ýýý,... .i:j-.., .ýýý1,,;'ý ,' ý 1. i. .. .;. , ,4ý'- F- ,ý .,.L.!.... ý,:,''., ,i ý,ý..L.!, % .-.. ý ,.ý 4 ;:::,_..5qo. i.",.4ý, 1.' ..;-.- !., I..!'!. ,.4ýý;l-,ý. ý ;ý. 1 . ..." , 4.._ ýr-.,' mL,ýL ýý ý_ % ...._ ,.., .,ý...ý:.%ýwý ,,. '. ý'..Z.1 - --ý"i-ý ,.ý.... ._...'.ý--.: ý..ý . 1 , .''.-,: .;ý.,...,,ý ý1ý;.-ý`..,.....,.-,..,,-.. ;.. ý! %.-......, -.. - .,1' .ý,' .ý ,. .,-- :. ,ý.. ., -ý"A. ,:.-ý,.!...ý.ý.-. .'.c-,,;. .,:. "..:...,, ,-,ýý . , .%-.. ... .:ý.1... ý;!,,ýý.;...e-.ýIF.;.c., . . . ,.-!,,.-I:.... .. ;.ý .. ,..;'.... ,%.'-'.!Iýý,4. .,'.";ý.., ;.i-.-. .1,ý.,ý,.1 -k , ý. ...,, ..ý,L4 .1 . .. , '... ý,L'1 I 7 , 4-.-ý.e;9.ý,,ý.ý', Iý,,. ý'., ;ý ý.L ý, .. ,ý? ý-, , : ý .;;.- ..? I . ;,..-ý., . ij.Iý-,.ýý*1 :: ,.- ...p.x- ,,.. -;) ., A.!.I.ý-.. ';.-,..ý.. ',Il,...- '.i....m ;ý_.,.r:.:ý.. ;', jý,i. ,ý....,1- ;ýiý..I .ý-Wi_-a ;' ý.'.1_-,W.I..,_.Cý i ..-. .. :.., .ý .7 .,.-I,.:.ij .-ý:.,ý.jJ,......ý. .ý.i ý.1. !: ,.? .: : ..ý,:; tý,ýI4.j.1.ýI...;.I14.,.. ..4-t1..ý.- 1.P-..,..,:...:;ý,. ý-ýýZ...ýý!L;;.-'':.ý,ý.----,,..6;-ýý.;.,,',ý.;',,,ýý:ýL.,ýý '.i.r--',ý-.,ýý,l:,.;ý;.,..
iIýirIII; . I ýi-`:-..--"..:...,.ýW,-1,.,II,.%,.Iý- '-,-I,,;,ý. -ý.Z... ._-..." .ý-4. ý ..-1,..ý.i .ý,. ", ,,.3.^`ý-..v_'-.:"."d . ý " '-ý.... eI.._ ' .'- -,4ý1i,ý., Iýý..,i_;ý' ý;,:ýe.ýT.-ý_-,I.;, , .I-- "5/.;ýý. .e.);ýý ,..; 1-. 1..ý--: ý ,.-"- . V,.N;l..-.ý_.....ý1 l ..,,..ýI.'"ýa . ýJ, . ,',k, ". . .- ý.",. ".ý_, ,"ý:z;NýI_ý.--. ,- r.., ,I.'ý ý--I ..ý;A',--ýý:ý 1::,,.:.ýt' .ý...;.,.,ý, ,"ý ,.' ,"...ýý:.'ý.i - ý ;:Iý,ý ý .jI1;ýý,,.. ...ý_... ý'l-..:.I,"1",-.-: 1 . .ý ý.;i '- .. I ....,Ic-'..-ý%-ý_ . :. ý.i-,ý,ý....I . V, .....n:: .;.,c.. . i _...ýý ...'..IZ..,Pý.. .... :; . ý.,...,.,. . :,..- ,'.; ;I .Z .ý."..,I-ý,ý:..., . ,ý...I..,ý.::. . ,ý1 .. ,_,.. ..,..- ý .Io-:'%. .,..., ,' .. ý,....,;..ý.;,%,ý.`._. :-.I.,.; : ..".ý.-. ýý',, ,-..ý.- . .o'--.::.".-' '-. % . ý - - ýý . ....-...:-. , :'- :.: .-..-ýL-ýI...: .ý-.... ...-'I :..,.z. .ý. .; , ._..;.,.:...',. ' ..:. . :. ..1--,T,ý..-,? :._,;. . ý . ...I....i ý. .ý' ý._..; ,...,.:..q,.- ...ý. . .- ý ,..ýý" "1..",. ..oý.. ` ....-ý.5 :.ý.ýýý, ,.. .", ,_d.:,ý.;. ,.,ý.. .6.,..r. . : , . ..;.1.1ýI.,:..,'.i . ý.,-- ...J_,..ýýI.,,, -Z.ý .iý".: . -...1.ý._. .,ý..". .. ý ,, .ý..,.. .,ý .!W .,. ::7,ý ..ý4.ý.,!ý"'..-..- .', . ý..'. .,-.--.,ý-.:;..:- ý-,.ý'z..IA--1ý.- , ;r: . ý.. ;x..: , .,:,IL,, ý-. -1.j. ,ý ý,--. '"- %-, ý.,,4..ý ,,.Aný; .-_.I; .,ýf' ._ ý..ý,. . ý"WIL-.ý,.'1. ý . .:. .ý.%,ý ý." ,Ný,;..'ýq-A.:.., , . ý....,.ý.,-ý _ý -,-.N: -ý .v': .ý,Z"._, ,-,-:.R1 ';, . .'I.. . ,..ý ' .; ir ,.ý,' . ..ý,ý :,. 'ý I._% 1... .ý .ýý- ,.I.ý,17A,L.,' 1. f E._ . ,,i.,ý.. -? ,ý . ,ýV.,Iý -1 ;.,3ý. Ný Ný rýr.'.. .._ !ý".,: . .I.Fý.%'. ,1I : '. ý ý. ,.' R"., ý,. .: -,,,I. ',4ý-.ý ý , . ,Px ý,-'Ký;ý ,9 ý.. >,,W-- "ý%-:.. , -ý-. ._1_iý ,%,., W., ;ý ,,ý. ,--,ý "pý !":.., ,- :ý. ý....! Ar- ý,,,,% f,k'ýj." .%- , '"-- :.,1ý'I,., -i1-0,-, "Zl- .l_.:r n . -ý '.- -:,ý ' ý!"""M-",: ..G, ,R.,jý. ,. ; . 1.ý ; ,ý :.1ý, . ..ý,.I4 j.Q. ._- '1.':" ."II, ýgý m-.,:..I j_ !rr I ý" .ý% ..,,,;: ...,,.f-I-,.-- e, ".ýJ,:Z.'.i. :N,1 ýýP.I*% W .,ý" :ýý.IýI-.,"-;.,-,4'.;,§e ý.0 ý '- ,.; ,, , ,-. ý.,ý.I...A, ..ý,ý iýI 1;.- 4,ý .I - -.,ý ,L1 .'.iIý1IIýý,Iý ,,,-:, , ý-.,- ýý f .,5-. ýII,.' *;1I.':._,, .,..i,. ;, ý ,-. .; ; ., ..I- ý " ..:. ...--I:- -",,.;-" . --' .. . %',ý:.- i. , ..,.ý:._.--ý,"ý -r..,.,.. _"ý...ý..-ý..i. ,,e. .. , .' K,.,'- L,.. ,. 7;:'ý-.ý" .",.., , ýý._ _ 4 ;.,: ý;._-..- ;- .; ý,I.:ý...,I...;,ý .;ý._.". . . jý.,k I.' P,ý ..1,ý. iz .ý- -... -,ý., ,. ýý"-.,- ý L;1.3.,,a,.:.ý...,t,;.: :-" .'-.-. -I[.'-.. , , 3 . .:5ý,,." ý ,ý -L --I.4-.. :- :ý.:.-rýI,..". .....,-.j ..:: % ..I.ý., ý. -_.ý: ý..k r,ý;: ;..'_ýj".!"ýý.j:!; ...-'.'.ý-.. Ii. . ý,ýý4E--,,..ý.; ý.ýý.:.lý.- ý,ýme; '., 1 I r... ..:ý q . ; .%,.,_j..'.I.; ..-.,,i ...Sv...ý.,4 .:,, ..ý .ý. ,4.i r-.. :,. ý. .z". .ý ý.,.. .. ,-l. ..ý,z-ý. -. ..-,.:. . ,-. ...;.-.,.";..'ý-. .., :. .,zý. .- '-ý...1,,. ,,, P ýý'":.m :..,.ýý .". ,:,-. ,..,ý_ ý., ,Z-, ,- ? ý,i. ý,ý' -:,.,.,I'-%.'. -: '. : J ;,ýý,--,-..z .:. _-; i._,i -.I , ý.ý.A .,ý.ý. .L '= , " ..ýý- %,- ,ý.d,"1i- ..ý.,;,4 ":,,'..'ý_ ...-,, ý !ý1ý : ' .ý..ý.ý-;ý.,,ýIýC;y ý.11. j, -m .:. ,. Mj.-: 1ý,ýfý ;.ý4 . 1-.1 ,-ý:,, ,,, IZýý r'.1- ,<ý.K'.. ..,.,, .. ;-Aý: % .- '-ý...,ý-,.: .--.-.I,,",ý-,"ý,i.ý1 "1ý' -, .-...".,ýz4Mk, ,, .ýJ ý' -...! ,.ý ý ' iý' ý,ý.mý :.ý,'k I: % ý.-"..-, . .rr;.1.ý.ý ,L-.;,. -. ! ,:ý.A;ý,' ,-,r.-,r ýý ýV,ý- - ,w. r,,,_ %.;ý ý,i: . -..,..ý I"..,iýz. ,i. -ý.,,. I ....ý- : ;z.:..: "._:-...!: !1".i". - . ., .- ..ý1 .-,.. .',ý-,L%jý, r-. ...ýý -L-..",jý.,.1.1 . i. -.ý.ý1 , ...ý , .'.1; :.,." . ýT: ..i ý,"ý-.. - :- . 0iý:j_7.ý..1,ý..;ý. ý.;ý..ýý'-ý.. ýI-.I..,ý z z, . . ,: ..:..ý jý" ',1.,ý..1,..: ,'' ýý.-. ,-. ,?';.1ý ý ..__?..,ýr,, ;ý ; .-; . _E ,-.-1 .ý.ýZ ;..-ý .,1.:,ý ý1,_..ý' ý--.ý;!..ýý.ýýý' ,. :_--'y-..-..-,i,.-; ýý.,Iý41ýý. ,",.i ý1 ,ý-,týý _!,ý. .-.ý: ., ._. - , ýý- , .,: 7:IL Ný -:,-L..."-'.,- .'.' -,.ý.t ....ýI..ýý . -.ý,ý-'.7-,: ,t ..ý , .7,.ý-..Wý,'ýý-c,.:.',,ý.!;ý,:C" .-,. "ý,ý, ý,," %.-.. - ," ý; .i..,% ..4ý...ý. :,_.,!c-.' ý .ýI ': L6ý. -`ýi qý":-.ý,.ý... ., .,.. f.- Mý ýý:, -"I..ý.-i-_-..ý-Y.,t- .ý,-, I ,I..-,'11 '....-."'-z1, ý,.ý .,;_ýý.Iýý.1.I .'L',-iý-a1 iI .-ý:ý.ý._... ;-.." .,.i, ...ý..ý. . .. :, ý,.. k.U,. ' ý,ý.,.- ,n '4..-.,4.,..,.. .-ý:; ,,.,ý ."ý 4I:ý ; .,, .:C.ý...- .;4ý: ý .-" .. .,i z.., , .."ý..,...ý.'-,K:... ",.ý. . - "-. L-ý .._. ýLý........!,.;, , ...,ýR 1,, .''..z. v..: ý ,ý--.--!..ý..ý :..._LL - ,ý. .,.".ý,ý,ý. .;'ý- --,-C'ýý,,a,..,'.. , ý-ý; 4. r, .ý 1 :ý" ,I '.. ;r:._iý. 4ý..r ::. -=;,:,:,.,,,.,,ý-.?ý,. :.:v. -n,! . ,ý9j- ,..7..,..I4 , ý.';_.,ý. ; r: ._; ." .,.,,,,.,ý ý'ýL',,. .,:,;;;ý T: ý !O.,". .ý e1. ,,..%- _.I:. ý,,..-.-ýj - ;",.ýIk,..c '."7,. ý. . ,,,tý' ..,.i.I.' '. ý..; ý.'.. ý.... ,,I.Rýz " .-- Z ý.t,.,.;4.,.,D- T.-.i,ýy ý ý.. ý.,- ._... -; ; j.-._.,-..S;-... h - j'-:.ý.. ..,:-.-.....1:. '2 ..-: _.ýý .ý., ; .-ýý. W-ý .::.a 'ý .9.';. . .ý.._7 ý. ; - F._;,.L . .I.r.: .- . .,- ;.v_.-, ..ý.-ý,;. ...,5 .-.ý ,"- .,:...-ýA,;.,...K_..,;ý. L '% .ý- lLý. .. .i. .i .'...- I :..,,k -z, .-,;., s,-r. ..ý,rý. ., .0ý '., -ý ,.. ,,oý-k.., ..: q,.ý.L ..., ',-1. ..I 'j4i-'5ý ý.,"1-?1:. ý -ý .I.j :..'I ý W; .."s,ý..'..ýý. ! h;1__ ,.,,ý ý, ý-.ý: ::.:..' ..ý- ýIýi .! ý4?ý - .'',..,jý-ý ý,.ý;,Z,,ý..zýý.,',4! ý,. -.. : -.;z ý..,ýý : j "-z",-'-. ý,:.. .,"ýL,.,LQ,,< 1Rf ,E._,I,, ,ý' .;,z;., ,. ;- l.,"';:ýL :,::: ,ý.., .%:ý.t ,i_ I ,4;. 6-ýý;,.,k , ýý-," 7",,.-ý,ýSN.,ý.1-ý" ,ý ,,_.ý ' ,. _ 1ý -,,ý., .1 I ,,..Rý ý -,, :A ý , R,. -ý,ý' :,..., . . , . .4 ,.i.. :ý, ' _ .:ý-ý,=.,,0,ý.: ,..r! ýI.k .1 ý.I:. L. .':.%.-.r.-;.. : _ . !..' ..%!..!..I.. . I -1ý1 I .." .IIII..%ý-ý-?i,T,'.,;.;-,_;ý,*§ .__4m ,Iý .,!ý,M!. ý'Iý.i,:;ý,.'.-........,ý, ý.-..-.,.x.,"_ _;.j' 1ý..-. ý, --..Aý ' ,, ,.I-.-.(..-..,-"' S.;',--L_'; .ý.. 14.-..,.,;i.. ,-I".. " m,._ "., ...: '..-i. ,.-1'.-..1,:...Iý.'.ý-.1.- . . . ,r__."; ý.. .ý-. . ,..:"...;..-..;, 1.j ... .-.4.ý --.;.",..:.I... :..I.".'ý.;..C ...; .-,', .,.,1:..ý ..'.- ....- . .,.ý..ý....- ......-,j'.-.! ....,.i..ý..:.-_-.. ....ý-,. ;-. ,_,-,1' ýý.. ,,... ..%-.1ý....: . -y . . ý. ;.ý.-.,Z ýý.. ,:, : .,ý'ý-.x-.iýý. _.,. - ,.... ..:,.ý,. ý. 1,.. --.,.-.':'.;,.ý _ .,,..Eý''.,.ýj ý1ý .,.ý..,,.,:... ".....- ,.IL. L.-, ý.....V,,Lý. ' ;- j; ý ..I.I..'.1, . ý,ý''.'.ý._;.,.-.,;.i ý.'ýý- ...ýý,...,-. V'.ý_-xý ,.ý .. M -I.-. ..ý;.,.ý . .Ký.ý...ýý.:.ým.?ý! , ,. ý...,i.-,,ý.-ý" ' .'ý- I . ,. .4.... "ý. ..,44, .: c.....,.,:,,_:-.4.._ý , 4:I,",,I"..ý-l4, .:Vý ý .. ,,_r 6 ,. ý - .ý..1.... -,.,v...ýý. -- .. .'..-ý. ý.ýý 4ý.- ý .,,ý,...,ý-I-1....-ý1...'.. ..A. ._.r"...:ý ..,.;3..-jL ..mý:.._.".ý.a'- 'ýý, l _ : .Aý-'.ý.,-,..,,ýv..--,.l';;'- . S ..:7 '-'ý-""_._.,. .,14r. ,;t.. .;.ýiZ4.""' .,,,..x..Q,0 .ý ',..ýýNL-.,ý ....,..' .I-1,I.: ..' ". ý ,,,;,:'.;-ý-.,..ý-,., .' .,. :;ý.., .L1'... __ ;:% ,.I.-.ýý, ,- x.. ..k,ý m.:,.. ,; ,.,--2ý-ýg ý, :- p.," KP ,7- l.--', - .-1.ý,, .1.,!..I,.I,, .i1ýýý. ýýI---'.."ii"", "N. 1.n..ý"ý. ýI,",:'r-I1ý'.ý':;. iigp.1,,zo.. -,.,1 .,.: ,.-" . .1. ,,ý,ý ", 'ýý1ý" ."9 -..':,I.,.,.!'4...Z4 1.;, ,. .,.;',z.ý - _.,..lL_.,,.41ý , : ..r,,,I. ..ý", " 1,,-..1,.xC.vg-ýý- ýk;y,. A4:, .- ,"ýAý.. ý-".Eý' pýýýN: .. ,ý,ý..,., ..ý1-....;;.- .ý.'n.,ýL _ ,. , ,'k. , w.ý."-.. - .., ..,!;e.!ý-..-. . 1 ,.-X, "_..LL.Z,aIrs '. .-1,.ý ý. ý. .,;ý.ý .l '...ý-",.;; ;..,ý,Lm,.",-ýZ_;;.m.. ." :."._.ý ... k:l;,.lý. . ý.- .., 0."ý;i-' ..q:.-,Iýý.. ý. .:..-..ý _ .'-,ýý-*.':ý,-.-.. .I.":% 4-z"ý,..- ,. ý-.'' .e_--.. .K,1 4,.. ,ýý,ýý,'.,,.tA-,ý ..;ý-., A ý.i .-.1, ., ,ýýý ý.ý ý -v...,...,-N!11...m,-- o4'ý;:. . .ý-1.;,..;.,,.,, ,.ý.:;ý- .Iý ,._,I'*S Fu;L.jýýa rý ý: 0..."p,L ý-,;..-1ý1... )',,. ..aS; q ._. ,lij '..,., ,. ia..---..W..; .. .J.,'. .1 i,; .-,:ý:i1.,-'ýh"" . ý,- )I ;!!ý-..-.' ,..'l1.. .ýL.. eý.';I.17-..Mý' -..;"- .m.Qýý :I-.."-,.il.ý .ý ..ý--.I1ý..ý.,. -J; . " Iýý. ..._.'..: -.-,,.,.,. ý - -!M......'..'L ' p!-I v"-.I%-ýj , 1,;," .,-..- W1A !1.1.-.._.ý.. Pz. , & 2.,',.,l...-.-týý ! ;, .ý.l1 a,.ý....,ýý4I ..uýý7L:I.Iý,.,., '..1-R__.---.,I"l..:,ý..".''.,-- ý!i-,... ý..ý.... ,., 4-...7t.-.,..__4..,. ;..-...1R .4.;:k .- PI-....-i.I? .1:;,.,I ; ; : ,.....1. 1;,,.... ...ý ;,.; ..i-j ,....ý ,. . 1 .,. ý. .t.._.. ..IiI..,.ýý".I1:II..II.1,..i..IýI-ý .L:.ýI:,..:I.1 ......,..,.,..ý,1.%-. . .",ý.f.:",.ýý
II1' I ;ý.,.N:%ý.,. .ý ý ,. :,.-,:.,.,.- . ...,_..,....._.;', ,,'. . ý.. ! , _.54., 1 ' 'ý,,ýi,"-..;".." ýý.... I I . tI ý -,_ ýJ.:,.;,ýý ,. ,. ,ý T: -:,,ýP' ý."ý -,-ý!- ..ý.ý.".._..-..'I...,-.L ýr. .- A.ýF-f...ý.,,. -ý ;,-, ý -,8, w ý,`. -..'-,,., .ý.,", , ý. ,,.,,Lz ý,ýw.. I"l. . lL" .,-, .. ý,I.,,r .,.-. ýLa..ý,ý-, -ý'., ., ,,ý '7 ..% ,.t,.4,.Iý.'--L..!,,I . .. .. I.,.. . %ý-% ,ý:...., ..-:. l" r.iý"1I, ý!.,. .O." 1.1ý-;._ .I.ý V -..., .:. , :-.,-, : .-... - .1ý: . ,%... ;1. .%, e. m, .ý... i... 'l .,-.Lý ý. , ý1ýi-..' .. ..1...1- .1.ýý.12..-.. , ý 2.. . ' ý iýII II
Iý1 I: ý,,.rU'.ýI! lF..tt.ý.F. Iý.. ý.ýý.,ý,,".r,I.-"..rýý,,.Pýj4,. N,. ..-l-i,;7;r.ý,."1,.-:,AI;.ý- ý.',g ý ý1 .,, .' 1 4 j-" 4 -j'-,ýý, ýý k-_ý ýi-I,,:ýoýýý.',ý.l.!v:.ý`" , .ý.-.-,.o ý .ý!,..?'ýý.. --,- . --Iý4jr, ý15. VtP. ,-.: 6 "-.:Iý!...`ýý..,t.'O,..I"ý, :-_t:,! ; . .'ýý.ýýý, n..!ý ,ý..,. i,' "z',1,-1-,.,-.. P. . .. ,4I .;' -. !-.. ý,,ýý: ý- l.ý, ,,1ý.ý...-,Y--ýI 1. .r,.. 'I , A;'R'as ,P ,-j!rýý,! ,l:,.e ý,-,f-:...,ý.. .ý ýýf 'ýý l.ý.:ýý'.,,ýý, .:ý,%i,.'...,%,..: ý . :I'ý,. ýýo 5-.-.-...ý.: ..,......4,-.ý. . .iý. ! . ....;r; ý!,ý..,,: . ;. ;1;! ý?-..L;:.,5?,'"17;;:s....'ý..' ,j. . .1c:i.. ..:.._ 1-,, ,-M...I.,.,,. 1..-. ýýý. .---. . ,:,. . 1.7. 4V.. 4., ... 1,.M . .. .. , M i,'ý..,.LK..eý0.4 ..,ýA'-.ý --:..-.,.- ý-t,_. ?.,i*ýj.,,,,`ý.ý .i. . I .ýý 'e . ý : I.,ý -., .:.,Iý'. .,ýý. ,5,-. , ',-'ý%,.0-ijI . . ,,ý ;. . ý.ý ., -.;,,F.. .'ý1ý ý'1A.14.k,ý. :,L' ,I1" 1ý. -:!, .ý ,. F.if. -. .'1i; ý--ý _1 M_;p..iI. ,q ,. . - -.Mý:..ý1,ta0 .eLt _ 4..,,-; ý 7 II. ý,'',."ý.....;t. -Kk., .,-ýý -:ý.,,-I,ý "'ý-_.; -,. ',..:%4."ýý ,- ! .'ý!, I,"- ý_ %, -.f ý'ý.'ý., ILb... .!...." I ., V .c. ,h,ý' ,.. . i' ý -., -d-, ., .nW..,:. ..k ,:- : I.U1.;.., t 1L,I.:.I.-. .n,.I 's Iý n;. : I I n " Isi.ý,I,- i ,it :: -, ýl .t ki,1- , j,.-,,- l -' :ý.V- ý..'. " s,L .ý. 1m!ý Ij.-,",. ý- , -.,,-I .v--, . 1,,ý,'ýO ,:'-, ,' , . ..; -;,c." "ý1 -: z,. .;,"."n-.,. -I1kI. 0.. 0 ,' ' ý."-.I. z,," .-'v. - -, t ,-.ý,.ý.ý ., -.,.-I,; : -. i,- , q, .._ýý -.ý.,;I_-:-. ..'_.,ý, - ý. ýý'...g . 1"',W 1 '._-..,i.,, ,1rýýý,ý..1 I,-, , .% rýW- ; a. ;,.w.-,ý l 4,?,M-' ý-' __. ,ý,ý '.ýý ., .,-Iý'_-, _ý,. 2! ý .L ,. .:.-." I.,:ýý1:I-, ";:t , . _" ..1' 1-,.-.', .. ý-'. .1", .. .... .ý%-1,I.:ý: .1ý::-:4,;..;? 11 '..;.,';: .".-- ...';4'1ý ,, .ý . .,I-1. ..ý':- ....'%.. ý...-.,..LI i.1.,'. ..._,"% .-;-',...O.. _ ..':1.. .1ý4ý.1. ,.;M.^,,ý.. ,,;'..,1.,," 1.4,'.,ýý -. ý .1,1c.." .ý ;V:, ;g.ý:-l..:;. ý .:. 1 ý,-,M--lýr, -;M::ý Wý-. .ýý1.-c1_% ý,ý-,;M-.ýý,.1 ;-. .;m..;,ý . , ,.-., ,,i,' ",-.-',.,.ý.ý .ý,Eý '...,. ý,;_- :,.ý,- ,-.-..%.,ý.,_"'ý,ý.Iý ý:- _,!..a.,a, z ...m , ,.. ,' .-L .ý- z,ý..,ý .,.; .,,,1 ýV .ý"-I1ý-ý.ý_ . .:.. .,.-`ý- .a,; ,,%.-P.ý;". .j-.,,-.... .;_].-.,ý .4 ý.:,ý .._. ;.ý,. ,?.!, ..-'.e-.'ý: ý ,;. -,,I.,.,.',...' .,ý. ,' . ý:- ý ...:ý-'-,...._..; .t- . , .. ..., ".L.t,%.',7..._. :.jý'.. ,_.,_,'ý,...,'.;4_ ýý np :_ %. ..,L,ý,_"..ý.'. .. ,nl_..- p.. ..,.1.,,:i. ý. '. I. 6 ý...iýýýj.:.,,, .- .z.... . p--.-.;ý.ý--.. I., ý,:o,ý; -._.OL"ýr-...ý- ,.;jýý ý ._-ý..,-,;ý._1ý".- ýýa ,.. ý,'ýý,t. .ý 'ý ý,, .1--. -:-I::L: . ý.ý% -. ..-..: -- ,m.:;i.-.,?..,.ý ..;ý..i -,ý. .,,,,.,. .I,Z" y_.ýTF. . ýý" .y-'a...-..,. ,, .ý.-P.ýg,.-., '-j.._LN% ,,j.;"ý"r . 1-,_L.,ýIi.M,o_,1 WT...,-.ý- .'.-,. .I 1,,.,. ?.e-4,i.:,r.-,ý L,.a .:ý" ý ,r:,.W ,mý:6. .ý ..,;ýIý-ý.ýý,s..W.:....'. " ,;a:: ,.:-..-.i-ý 1 .,'.."--ý._ 7 ... ý;ý-- ...,..- .ý. 4.P1. 1 -.11 ,.ýN.:ý.. -4,... ,. 1.%1.ý.. ..1_.ý1 I,%%i..,1 ".: ;_- ."...ý,A;.,-1:.. .w. .. ..:...,"k:-ý.,... "1-_N '_.: -. --.,.ý l..-1'1.ýIý . .. ý: -_- .o ý-.1.:- ,!ý-Iý.,. 1 ý Y....:4I,ý......,1,! : ..... . !.... e..j :_I1;ýI:I .ý .1 m '.ý,ý1;,"i ,I:I"1ý II,ý-1;: ., ........,.§ __.,.:ý. ý.'...ýý:-,.,.,-.ý-.""":..,.ý.,,,.."
Sensor Fusion For Autonomous Outdoor
Navigation Using Neural Networks
Ian Lane Davis
CMU-RI-TR-95-05
The Robotics Institute
, Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA 15213
Accesion For
January, 1995 NTIS CRA&l
DTIC TAB
Unannounced 11
Justification .................
By
© 1995 Carnegie Mellon University Distributionf
Availability Codes
Avail andIor
Dist Special
This work has been supported in part by a National Science Foundation Graduate Research Fellowship. The
equipment used is provided in part by DACA76-89-CO014, Topographic Engineering Center, Perception for
Outdoor Navigation and DAAE07-90-C-R059, TACOM, CMU Autonomous Ground Vehicle Extension.
The views and conclusions expressed in this document are those of the author and should not be interpreted
as representing the official policies, either expressed or implied, of the U.S. government.
'D-I- - rUT-ON S ATEM
Approved for public release;
Distribution Unlimited
Sensor Fusion For Autonomous Outdoor
Navigation Using Neural Networks
Ian Lane Davis
CMU-RI-TR-95-05
Abstract
For many navigation tasks, a single sensing modality is sufficiently rich to accomplish the desired motion control
goals; for practicala utonomous outdoor navigation, a single sensing modality is a crippling limitation on what tasks
can be undertaken. In the research detailed in this paper, we open the door for a whole new suite of real-time autono-
mous navigation tasks previously unattainable. Using neural networks, including a neural network paradigm particu-
larly well suited to sensor fusion, and Carnegie Mellon University's HMMWV (High Mobility Multi-Wheeled
Vehicle) off-road military ambulance, we have successfully performed simulated and real-world navigation tasks that
required the use of multiple sensing modalities.
Table of Contents
1.0 Introduction .............................................................................................................. 1
1.1 Limitations of the state of the art ................................................................................. 1
1.2 Fully self-contained autonomous navigation ................................................................ 1
1.3 Prior work & our approach: the MAMMOTH system ................................................. 2
2.0 The task at hand .................................................................................................. 5
2 .1 Th e robot ............................................................................................................................. 5
3.0 System issues .................................................................................................. 7
3.1 Image resolution ............................................................................................................. 8
3.2 Network architecture ...................................................................................................... 8
3.2.1 Monolithic architectures used .......................................................................... 9
3.2.2 The specific MAMMOTH architecture ........................................................ 10
4.0 Experiments and results ...................................................................................... 11
4.1 Two interesting simulated experiments ........................................................................ 12
4.1.1 Road with road-colored obstacles ................................................................... 12
4.1.2 Road with brightly colored obstacles ........................................................... 15
4.2 Two equally interesting real-world experiments ......................................................... 16
4.2.1 The additive obstacle test ................................................................................ 16
4.2.2 The conflicting obstacle test ........................................................................... 19
5.0 Conclusions ............................................................................................................ 20
5 .1 M ilestones ......................................................................................................................... 20
5.2 Future Directions ........................................................................................................ 21
6.0 Acknowledgments ............................................................................................. 22
7.0 References .............................................................................................................. 22
CMU RI TR 05 05
1.0 Introduction
The autonomous land vehicle group at CMU has for years been studying autonomous navigation for wheeled vehi-
cles using the Navlab I and the HMMWV (Navlab II). Some of the most successful work to come out of the group
has included road following using CCD (charge coupled device) cameras [Pomerleau92, Jochem93] and cross coun-
try navigation using a scanning laser range finger [Langer93, Kelly93]]. The long term goal of each of these projects
has been to engineer a vehicle capable of performing fully autonomous tasks in a given environment (e.g. on a high-
way or in the wilderness).
1.1 Limitations of the state of the art
Earlier systems have had some obvious limitations. Primarily, a single CCD camera is sufficient for road following,
but insufficient for extended road driving tasks, and a laser range finder is sufficient for obstacle avoidance, but insuf-
ficient for cross country (XC) navigation in "interesting" terrains. Road following capability alone will not allow a
robot to drive on a roadway on which there are obstacles, and simple range-based obstacle avoidance can be both
inefficient and ineffective in the presence of vegetation, water, or other "unusual" obstacles.
One direction of research to explore is the augmentation of current capabilities with "top-down" information. For
road following this might mean using limited access roads and having all vehicles on the road signal each other about
their whereabouts. The obvious difficulty is dealing with unexpected obstacles, such as fallen branches, animals, and
even stalled or damaged cars. For cross country navigation, top-down augmentation might involve the use of maps in
coordination with some sort of global positioning/intertial navigation system. Such a system is limited in previously
unexplored regions. Top-down augmentation in either case is subject to failures due to communications breakdowns
or incorrect information which cannot be directly confirmed via available local sensing.
1.2 Fully self-contained autonomous navigation
In both the road following and the cross country navigation tasks, as well as in many other tasks for which the use of
robots is needed, such as space exploration, fully self-contained navigation is either very desirable or entirely neces-
sary. The techniques which have worked so well to this date for the limited scenarios (in ALVINN and Ranger) have
been "bottom-up" techniques, which we can define as techniques which take the rawest of locally available data and
CMURITR_05 05 1
turn this into the necessary motion control commands for the task. Inherent in this concept is rich sensing, and for the
tasks at hand the broadest way to expand the sense-space of the robot is via additional local sensing modalities.
Thus, our goal is to integrate multiple sensors to achieve capabilities not previously attainable in autonomous naviga-
tion systems. These capabilities are achievable with "bottom-up" techniques such as those described later. For the
reasons mentioned above, using such an approach is desirable and often necessary.
1.3 Prior work & our approach: the MAMMOTH system
The approach taken in our research is informed by much of the work in cross country naviation [Daily88, Brumitt92,
Langer93, and Kelly93], work in neural networks for navigation [Pomerleau92, Meeden93, Marra88, and Wright89],
and work in modular neural networks [Jacobs9l, Smieja92, Hampshire89, Fahlman9l, Gluck93, and Jochem93].
Our current experiments with sensor fusion use backpropagation neural networks [Rumelhart86]. The basic elements
of a neural network are nodes and connections. Nodes have activation levels, and connections have weights. In a typ-
ical network, there are a number of input nodes, each of which has some level of activation (these can represent the
pixels in an image with their activation level being their intensity in the image). Most often, there is a next level of
nodes, called hidden units. Each hidden unit is connected to each input unit (unit and node are used interchangeably)
by a connection with a given weight. The activation level of a hidden node is the sum of the activation of each node
it's connected to multiplied by the weight of the connection. The activation is limited to being between -1.0 and 1.0
(or zero and one) by a sigmoidal function. Hidden units come in layers usually and often there is only a single layer
which connects to a group of nodes known as the output nodes. The network learns by having some pattern put into
the input nodes and adjusting the weights of all of the connections until the output nodes show the desired output pat-
tern.
Why do we choose to use neural networks? For an excellent discussion of the suitability of neural networks for the
general navigation task see [Pomerleau92]. The primary issue is that we do not necessarily know all of the features in
sensor space which affect our navigation decisions. The fallout is that if we use a model-based system of navigation
we would need to explicitly take all of the features into account and experimentally (or arbitrarily) set the relative
importance of each feature. We cannot expect performance in the presence of a feature rich environment such as the
CMURITR_05 05 2
real outdoor world, and we certainly can expect to go back to the drawing boards in any new environment. Neural
networks when properly trained can automatically select and weight the most important features of an environment,
with the added bonus of being able to tackle a radically different environment with just a change of training data.
The architectures of the networks we are examining in this work are of two flavors: monolithic networks and modular
networks. In the typical monolithic neural network approach, we give the network all of the sensor data as inputs and
allow the network to develop internal representations which allow it to perform adequately within the context of the
experiences we expose it to. We might even train the network in different situations to allow us a broader range of
"safe" navigation terrains. A monolithic network may be seen in Figure 1, on page 3.
I
InteArnal --
Representations
I
Inputs
FIGURE 1. A typical monolithic neural network in which the inputs feed forward into internal
representations (developed during training) which in turn feed forward into the outputs. For driving,
inputs might be the pixels of a video image and the outputs might be some representation of a steering
direction.
One potential shortcoming of this approach is that, while in a model-based system we can't guarantee performance
over all possible inputs, here, we cannot guarantee adequate performance in any situation, although we might expect
adequate performance in most cases. This is because we do not always know how to interpret the internal representa-
tions that the neural network generates in complicated scenarios.
What if we know that some feature in the sensor space is useful for navigation? For example, we know that the edges
of the road are tremendously good clues to use in on-road navigation. A typical neural paradigm might result in a net-
work learning to find the edges of the road. Or it might not. But even if it does, we might have directed the learning
towards less obvious but also important features if we first shared our a priori knowledge of the task and sensor
CMURITR_05 05 3
domain. Exactly how much of our resources (time, memory, computing power, patience) are we devoting to learning
what we already know?
This work uses monolithic neural networks for navigation, but the above concern, as well as the concern that mono-
lithic neural networks might not scale easily as we increase the demands on our system in terms of input space and
output (actuation) space, has led us to examine modular neural networks as an alternative. The modular architecture
we use attempts to address the issue of integrating a priori knowledge and hints that it may prove to be more flexible
as the complexity of the tasks increase. Our modular system is the Modular Architecture Multi-Modal Theory net-
work (MAMMOTH).
A MAMMOTH system consists of two segments, the feature level and the task level. The feature level is a set of Fea-
ture Neural Networks (FeNN) trained to recognize hand-picked features in whatever sensor modality serves as their
input source. Thus, there might be one or more networks in the feature level devoted to the color video modality and
each of these would be trained to recognize features in color video space, and their hidden units would be encoding
these features in color video space. Other networks in this level might address features in laser range finder sensor
space or in the input space of any modality we care to employ.
On top of this feature level is the task dependent network which unites the feature networks, the Task Dependent
Neural Network (Task Net). The task level uses the encoded information from the feature level networks as inputs to
a larger network trained to perform the navigation task. These encoded feature inputs can be supplemented with task-
related inputs. For a sketch of a MAMMOTH network see Figure 2, on page 5.
CMURITR_05 05 4
Task Dependent
Neural Network
/
Additional
Hand-Picked Feature Networks
FIGURE 2. General MAMMOTH architecture
This paper represents the first use of both the monolithic neural network and the MAMMOTh network on the outdoor
navigation tasks described later. For a more detailed description of the MAMMOTH paradigm and of its advantages,
see [Davis93b].
2.0 The task at hand
2.1 The robot
Our specific task is to achieve autonomous navigation with the Navlab II, or HMMWV, four-wheel-drive military
ambulance (see Figure 4, on page 7). In our early experiments we are focussing on a "wander" behavior. We want the
vehicle to wander around the terrain and not get into danger. This involves avoiding obstacles, but not heading to par-
ticular goals (this will be added later). We want our system to choose an appropriate steering direction. At this point
in time, the vehicle's throttle is not actuated, so the safety driver (a person in the vehicle who can take control at any
moment) controls the gas pedal and brakes.
The -MMAVW (pronounced "humvee") has all of its computing on-board, a gas generator for power, a battery back-
up for the computing power, an air-conditioned computing cage for custom cards (often for sensing) as well as the
workstations, and a whole suite of sensors, including a color video camera, a scanning laser range sensor, and a multi-
CMU RI TR 05 05 5
baseline stereo rig. The vehicle weighs 6 tons and can be driven at up to 70 miles per hour. For work on the vehicle,
three SPARCTM 10 computers, named Moe, Larry, & Curly, handle all of the processing.
FIGURE 3. The HMMWV
Most of our work is tested in one of two environments: at 2 kilometer by 2 kilometer outdoor testing site on top of an
old slag heap or on a square kilometer of a "virtual" 2.5D world.
The slag heap in the real world has sections that range from very sparsely vegetation with dry grass to intraversible
due to trees and thick bushes. There are natural and man-made ridges of earth, man-made mounds of earth, cliffs,
large semi-permanent puddles, and remnants of the foundations of a few old buildings. A dirt access road runs
through a large section of the slag heap (looping around to almost 4k total length). A rough estimate would be that
less than half of the total surface area of the slag heap is traversible with the -MMvWV (i.e. the vehicle itself is capa-
ble of safely moving over it). Most current systems are very limited in which sections they can maneuver in because
they do not use multiple sensors (and correspondingly cannot deal well with vegetation).
The simulator, called Alsim [Kelly93], used by the cross country navigation group at CMU consists of a map with
elevation and color (RGB) in each cell. The initial elevations are generated with a triangle based fractal, and colora-
tion is based on a similar fractal. This creates a world with varying elevation and patches of "vegetation" (a.k.a. green
stuff). We can add onto this world a variety of geometric shapes, such as gaussians, cylinders, boxes, pyramids,
CMURITR_05 05 6