Moving object detection apparatus and moving object detection method

72 Pages·2013·5.32 MB·English

US008599252B2
(12) United States Patent (10) Patent N0.: US 8,599,252 B2
Komoto et a1. (45) Date of Patent: Dec. 3, 2013
(54) MOVING OBJECT DETECTION APPARATUS FOREIGN PATENT DOCUMENTS
AND MOVING OBJECT DETECTION
METHOD JP 8-214289 8/1996
JP 2006-31114 2/2006
(75) Inventors: Ayako Komoto, Osaka (JP); Kunio JP 2008-146185 6/2008
Nobori, Osaka (JP); Masahiro IWasaki, WO 2010/050110 5/2010
Kyoto (JP) WO 2010/079556 7/2010
(73) Assignee: Panasonic Corporation, Osaka (JP)
OTHER PUBLICATIONS
(*) Notice: Subject to any disclaimer, the term of this
patent is extended or adjusted under 35 International Search Report issued Nov. 1, 2011 in corresponding
U.S.C. 154(b) by 86 days. International Application No. PCT/JP201 1/004173.
P. Anandan, “A Computational Framework and an Algorithm for the
(21) Appl. No.: 13/454,577 Measurement of Visual Motion”, International Journal of Computer
Vision, vol. 2, Jan. 1989, pp. 283-310.
(22) Filed: Apr. 24, 2012
(Continued)
(65) Prior Publication Data
US 2012/0206597 A1 Aug. 16,2012 Primary Examiner * Dave CZekaj
Assistant Examiner * Tsion B Owens
Related US. Application Data
(74) Attorney, Agent, or Firm * Wenderoth, Lind & Ponack,
(63) Continuation of application No. PCT/J P201 1/ 004173, L.L.P.
?led on Jul. 25, 2011.
(30) Foreign Application Priority Data (57) ABSTRACT
Jul. 27, 2010 (JP) ............................... .. 2010-168704 A moving object detection apparatus includes: a stationary
measure calculation unit calculating, for each of trajectories,
(51) Int. Cl. a stationary measure representing likelihood that the traj ec
H04N 7/18 (2006.01) tory belongs to a stationary obj ect; a distance calculation unit
G06K 9/00 (2006.01) calculating a distance representing similarity between traj ec
G06K 9/34 (2006.01) tories; and a region detection unit (i) performing a transfor
(52) US. Cl. mation based on the stationary measures and the distances
USPC .......................... .. 348/135; 382/103; 382/173 between the trajectories, so that a ratio of a distance between
(58) Field of Classi?cation Search a trajectory on stationary object and a trajectory on moving
USPC ................................. .. 348/135; 382/103, 173 object, to a distance between trajectories both belonging to
See application ?le for complete search history. stationary object becomes greater than a ratio obtained before
the transformation and (ii) detecting the moving object region
(56) References Cited by separating the trajectory on the moving object from the
trajectory on the stationary object, based on a geodesic dis
U.S. PATENT DOCUMENTS
tance between the trajectories.
2011/0002509 A1 1/2011 Nobori et al.
2011/0013840 A1* 1/2011 Iwasaki et al. .............. .. 382/173
2011/0228987 A1 9/2011 Iwasaki et al. 32 Claims, 32 Drawing Sheets
Camera I 110
.................. "4.1.991"...
iMoving object detection
:apparatus
101
; Image receiving unit
l 102
' I Trajectory calculation unit
j 103 104 \L 105
Stationary measure Distance Subclass
calculation unit calculation unit classi?cation unit
J 106
i I Weighted distance calculation unit I
107
Segmentation unit
Display I 120
US 8,599,252 B2
Page 2
(56) References Cited Edsger W. Dijkstra, “A Note on Two Problems in Connexion with
Graphs”, Numerische Mathematik, vol. 1, Dec. 1959, pp. 269-271.
Pedro F. FelZensZwalb et al., “Ef?cient Graph-Based Image Segmen
OTHER PUBLICATIONS tation”, International Journal of Computer Vision, vol. 59, No. 2,
2004, pp. 1-26.
Vladimir Kolmogorov et al., “Computing Visual Correspondence Carlo Tomasi et al. “Detection and Tracking of Point Features”, KLT:
with Occlusions via Graph Cuts”, International Conference on Com An Implementation of the Kanade-Lucas-Tomasi Feature Tracker,
puter Vision, Mar. 7, 2001, pp. 1-37. http://www.ces.clemson.edu/"stb/klt/, version 1.3.2, Mar. 28, 2006,
pp. 1-20.
Jianbo Shi et al., “Good Features to Track”, IEEE Conference on
Kenichi Kanatani, GaZo rikaii3 jigen ninshiki no suuri4(Image
Computer Vision and Pattern Recognition, Jun. 1994, pp. 1-8.
UnderstandingiMathematical Principle of Three-dimensional Rec
Richard Hartley et al., “Multiple View Geometry in Computer
ognition), Morikita Publishing Co., Ltd., May 24, 1990, pp. 78-99
Vision”, Second Edition, Cambridge University Press, 2003, pp. (with partial English translation).
87-131, 239-261, 279-309 and 365-407. Richard Hartley et al., “Multiple View Geometry in Computer
Chang Yuan et al., “Detecting Motion Regions in the Presence of a Vision”, Second Edition, Cambridge University Press, 2003, pp.
Strong Parallax from a Moving Camera by Multiview Geometric 87-131, 239-261, 279-309, 365-389 and 391-407.
Constraints”, IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. 29, No. 9, Sep. 2007, pp. 1627-1641. * cited by examiner
US. Patent Dec. 3, 2013 Sheet 1 0f 32 US 8,599,252 B2
FIG. 1
110
Camera /
/10()
EMaopvpianrgat uosbj ect detection \ /10]_ ;
1 Image receiving unit I
5 y /102 5
i Trajectory calculation unit i
\ 103 v 104 v 105 I
;I cSatlactuiloantairoyn muneiats ure cDailsctualnactei on unit Scluabscsliafiscsa tion unit :
\ \ \ r/1O6 E
Ii Weighted distanc\e calculatio/n1 u0ni7t E
i Segmenta\ tion unit
/12o
Display
FIG. 2
200
I. _____________________________ ..... -.|
I Computer:
/110 /201 202
Camera > I/F >< CPU r4
: 203 i
; 206 < > ROM ’‘ :
/120 : Vd/ :
D- | / : l e0 , 204 ;
‘Sp ay ; carcl ’ \ RAM K
a A I
l I \ V/ZOS I
HDD '
e _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ . . _ _ - _ - _ - - _ - - - _ _ _ _ -_
US. Patent Dec. 3, 2013 Sheet 2 0f 32 US 8,599,252 B2
FIG. 3
Receive image data on a plurality (T number) of consecutive images
i, 5302
Calculate motion information on pixels using temporally different
images to generate trajectories over the images
V 5303 v 5304 \ /S3O5
Classify the
Calculate a stationary Cakulate a
measure indicating distance trajectories
llkEIlhODCl of belonglng representing Into subclasses
which
to a stationary object, Simi'arity between
for.each .Of the the trajectories are subsets of
trajectones the trajectories
\y ,/S306
Calculate a geodesic distance based on the stationary measures of
the trajectories, the distance between the trajectories, and the
subclasses
\/ S307
Output a result of segmentation obtained by separating or
integrating the subclasses based on the geodesic distance between
the subclasses
US. Patent Dec. 3, 2013 Sheet 3 0f 32 US 8,599,252 B2
FIG. 5A
L)
UH
FIG. 5B
O
UU
FIG. 5C
MI\
/Wl
US. Patent Dec. 3, 2013 Sheet 4 0f 32 US 8,599,252 B2
FIG. 6A
Pixel i 603a
Input picture 601
Pixel k 603i) Motion vector information 602
t t+n
FIG. 65
T number of pictures 604 included in input video sequence
Trajectory
Pixel i 605a
xi 606a
Pixel k 505i) Trajectory xk 606D
1 2 T ' 1 T (Frame)
FIG. 7
Stationary measure / 103
calculation unit
Geometric constraint /_’7O1
estimation unit
\
Error calculation unit
US. Patent Dec. 3, 2013 Sheet 5 0f 32 US 8,599,252 B2
FIG. 8
Stereo Trinocular
Epipolar Triiinear
constraint constraint
Homography Structure
constraint consistency
constraint
FIG. 9A
Stationary point X
Second frame
First frame
Point {)1 in image ‘:> Line l2 in image of
of first frame second frame: epipoiar line
FIG. 9B
US. Patent
iDec.3,2013 Sheet60f32 US 8,599,252 B2
FIG. 10
®TT
\
1101
1102
zgEk
‘///T
Ei
FIG. 115
T T T
E Smal|er<-———> Greater
US. Patent Dec. 3, 2013 Sheet 7 0f 32 US 8,599,252 B2
FIG. 12
Distance calculation unit ’’ 104
Euclidean distance / 1201
calculation unit
V
Geodesic distance ’ 1202
calculation unit
FIG. 13A
I
Trajectory i
Trajectory s
Trajectory j
FIG. 135
Data pointJ' Data point s
I
US. Patent Dec. 3, 2013 Sheet 8 0f 32 US 8,599,252 B2
FIG. 13C
Data point 5
Data pointj j,,iégééééigéiééé?égi
:90, J') 1302 '
FIG. 14
Euclidean distance 1401 Geodesic distance 1402
Trajectory xi /
I i
Trajectory xS
(a) Euclidean distance between (D) Geodesic distance between
trajectories trajectories