视觉里程计原理(二)特征匹配与追踪(LK光流法)

rich feature descriptors
brute-force Comparing each feature in the first set to each feature in the second set
FLANN (Fast Library for Approximate Nearest Neighbors) K-means tree K-dimension tree
Machine Perception and Interaction Group (MPIG)
www.mpig.com.cn
Pyramid Implementation
Initial guess
Machine Perception and Interaction Group (MPIG)
www.mpig.com.cn
Machine Perception and Interaction Group (MPIG)
www.mpig.com.cn
Assumptions
3rd assumption
Machine Perception and Interaction Group (MPIG)
www.mpig.com.cn
TermCriteria termcrit=TermCriteria(TermCriteria::COUNT+TermCriteria::EPS, 30, 0.01);//指定光 流法搜索算法收敛迭代的类型 calcOpticalFlowPyrLK(img1, img2, points1, points2, status, err, winSize, 3, termcrit, 0, 0.001);
www.mpig.com.cn
optical flow
Lucas-Kanade (LK or KLT) [1]
three assumptions
• Brightness constancy • Temporal persistence or small movHale Waihona Puke Baiduments
• Spatial coherence
MPIG Seminar 0046
Feature matching
陈伟杰
Machine Perception and Interaction Group (MPIG) www.mpig.com.cn
cwj@mpig.com.cn
The main steps of Visual Odometry
images parameters
Machine Perception and Interaction Group (MPIG)
www.mpig.com.cn
rich feature descriptors
example
code FlannBasedMatcher matcher; std::vector< DMatch > matches; matcher.match(descriptors1,descriptors2,matches); Mat img_matches; drawMatches( img1, keypoints1, img2, keypoints2, matches, img_matches ); //-- Draw matches imshow("Matches", img_matches ); //-- Show detected matches
Machine Perception and Interaction Group (MPIG) www.mpig.com.cn
example
Machine Perception and Interaction Group (MPIG)
www.mpig.com.cn
thanks
Machine Perception and Interaction Group (MPIG)
Point matching using
rich feature descriptors
brute-force
FLANN
optical flow
Horn-Schunck
Lucas-Kanade
Machine Perception and Interaction Group (MPIG)
www.mpig.com.cn
We write as follow for convenience
Machine Perception and Interaction Group (MPIG)
www.mpig.com.cn
Standard KLT algorithm
Machine Perception and Interaction Group (MPIG)
www.mpig.com.cn
Code of LK based openCV
Address: github.com/sunzuolei/vo_basis
calcOpticalFlowPyrLK ()
vector<float> err; vector<uchar> status; Size winSize=Size(21,21); //设定金字塔层搜索窗口尺寸
KLT algorithm
Machine Perception and Interaction Group (MPIG)
www.mpig.com.cn
Pyramid Implementation
Standard KLT algorithm deal with small pixel displacement Solution for this is a pyramidal implementation
Assumptions
Brightness Constancy Assumption translational model:
Machine Perception and Interaction Group (MPIG)
www.mpig.com.cn
Assumptions
Taylor
2nd assumption
Pyramid Implementation
matching error function:
Machine Perception and Interaction Group (MPIG)
www.mpig.com.cn
Standard KLT algorithm
Define new images A and B
KLT algorithm
u v
gray value u is I(x,y)
The goal is to find v on J, where I(u) and J(v) are similar The way is to compute d
Machine Perception and Interaction Group (MPIG) www.mpig.com.cn
Eliminate errors
example
Machine Perception and Interaction Group (MPIG)
www.mpig.com.cn
optical flow
What’s optical flow?
Machine Perception and Interaction Group (MPIG)
Feature Extraction
Feature matching
Compute E or F for [R|t]
Drawing path
Machine Perception and Interaction Group (MPIG)
www.mpig.com.cn
second
Feature matching
www.mpig.com.cn
Eliminate errors
code double max_dist = 0; double min_dist = 100; for( int i=0; i<descriptors1.rows; i++ ) { double dist = matches[i].distance; if( dist < min_dist ) min_dist = dist; if( dist > max_dist ) max_dist = dist; } //-- Draw only "good" matches (i.e. whose distance is less than 2*min_dist ) std::vector< DMatch > good_matches; for( int i = 0; i < descriptors1.rows; i++ ) { if( matches[i].distance < 2*min_dist ) { good_matches.push_back( matches[i]); } }
Machine Perception and Interaction Group (MPIG)
www.mpig.com.cn
Eliminate errors
Getting rid of points for which the KLT tracking failed or those who have gone outside the frame int indexCorrection = 0;//初始化参数 for( int i=0; i<status.size(); i++) { Point2f pt = points2.at(i- indexCorrection); if ((status.at(i) == 0)||(pt.x<0)||(pt.y<0)) { if((pt.x<0)||(pt.y<0)) { status.at(i) = 0;//将对应的这组光流置零,等同于未发现该光流 } points1.erase (points1.begin() + (i - indexCorrection));//删除该点 points2.erase (points2.begin() + (i - indexCorrection)); indexCorrection++; } }
FLANN
Machine Perception and Interaction Group (MPIG)
www.mpig.com.cn
rich feature descriptors
example
FLANN
Machine Perception and Interaction Group (MPIG)
[1] B. D. Lucas, T. Kanade, An Iterative Image Registration Technique with an Application to Stereo Vision .
Machine Perception and Interaction Group (MPIG) www.mpig.com.cn
www.mpig.com.cn
Standard KLT algorithm
Machine Perception and Interaction Group (MPIG)
www.mpig.com.cn
Standard KLT algorithm
Machine Perception and Interaction Group (MPIG)
Machine Perception and Interaction Group (MPIG) www.mpig.com.cn
//-- Draw only "good" matches Mat img_matches; drawMatches( img1, keypoints1, img2, keypoints2, good_matches, img_matches, Scalar::all(- 1), Scalar::all(-1),vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS ); //-- Show detected matches imshow( "Good Matches", img_matches );
www.mpig.com.cn