非线性分析期末大作业(硕士博士非线性分析)

《非线性分析》期末作业A monocular vision 3D measurement methodbased on rotating lens姓名：xxx学号：xxxx学院：xxxx专业：xxxxx主讲教师：xxxxxA monocular vision 3D measurement method based on rotating lensCheng Xi KangAbstract——In this paper, a monocular vision measurement method based on rotating lens is proposed.A special optical lens is placed between the target and the camera, which causes light refraction during the measurement process. By analyzing images taken after different light refraction, the 3D coordinates of the target can be obtained. Using this method, 3D measurement of the full field can be completed with no laser assisted. Besides the 3D information of a target can be measured without requirements for a priori knowledge of feature points on the target or the location information of the lens. Finally experiments are conducted in the laboratory, and results show that excellent accuracy can be obtained with proposed method.Keywords—refraction; machine vision; measurementI.I NTRODUCTIONWith its advantages of non-contact, high precision and strong adaptability, visual measurement has been widely applied to the geometrical measurement areas such as space, navigation and so on. Using visual measurement methods, the 3D information of a target can be measured by analyzing its 2D images. Depth acquisition has a long history in computer vision. Based on the number of viewpoints required, existing methods can be broadly classified into multi-view and multiple exposure approaches. Multi-view methods exploit stereo information to recover the depth of a scene [1]. The location of a 3D point can be estimated by finding and triangulating correspondences across images.The work we’ve done is more closely related to [2, 3, 4, 5, 6, 7]. In [2], Lee and Kweon used a transparent biprism to obtain the geometry of an object. In [3] and [4], a transparent planar plate is used by the authors to estimate the depth of a scene, with whose 2 opposite faces being parallel to the image plane. In [5], Gao and Ahuja made a further step by setting the faces of the transparent medium non-parallel to the image plane and capturing images while rotating the medium about the camera’s principal axis. Before that, an extra step is essential that the pose and the refractive index of the medium must be estimated using a calibration pattern. In [6, 7], Shimizu and Okutomi proposed a new method to estimate depth by triangulation. They use either reflective or refractive medium for the implementation to record the scene from a fixed viewpoint without or with the reflection light paths which create individual images. Besides, a complex calibration setup required by their method is described in [8]. Similarly, our method uses a transparent plane-parallel medium as well, but with no requirement such as the medium plane being parallel to the image plane, a careful calibration of the medium’s pose or refractive index which are mentioned in the afore methods. [9,10] indicate that many researchers pay attention to the shape recovery of transparent objects which is also referred to as refractive objects recently. In [11], Zhihu Chen proposed a depth acquisition method based on refraction of light. In conclusion, using the existing binocular measurement methods, the relative relationship between two cameras has to be obtained, and images must be taken from two different perspectives. Moreover, most of the monocular systems require for a prior knowledge of the feature points on the target.The existing coordinate measurement methods based on light refraction require that the position and attitude of the transparent medium have to be known in advance.To solve above problems, we proposed a monocular vision measurement method based on rotating lens in this paper. This paper is organized as follows: Section 2 introduces the measurement method for 3D information of a target. Section 3 introduces the camera imaging model and solving internal and external parameters. Section 4 introduces the imaging model based on light refraction principle. Section 5 introduces the accuracy verification experiments in the laboratory. Section 7 introduces the conclusion we have achieved in this paper.II. MEASUREMENT METHODIn this paper, a special rotatable optical lens is placed between the target and the camera, whose thickness and refractive index are known before. Its shape is shown in Fig. 1. Deviation is produced on the images taken at different locations as light refract when the lens is rotated. And then we can calculate the 3D information of the target using deviations on different images.Figure 1. Measurement methodIII. CAMERA IMAGING MODELThe spatial points are projected onto a 2D image plane through the imaging lens of the camera. This projection can be described by the image transformation known as the camera model. The position of space points in the image plane can be approximated by the pinhole model, as shown in Fig. 1. The world point P is projected through the projection center of the lens to the point p in the image plane; p lies on a straight line from P through the projection center OC, as indicated by the dotted line. Generally, this kind of cameramodel is called the linear camera model as well.CameraLens Region A Region BRotating lensFigure 2. The principle of binocular stereo imagingGiven that lens may have serious distortion due to its manufacturing process, we introduce distortion coefficients into the binocular vision system model, and the equation can be written as:2212222121ˆˆˆˆˆ0ˆˆˆˆˆ0()00111c c u k u k u u v Z v Z k v k v u v M P ⎡⎤+⎡⎤⎡⎤⎢⎥⎢⎥⎢⎥++=⎢⎥⎢⎥⎢⎥⎢⎥⎢⎥⎢⎥⎣⎦⎣⎦⎣⎦(1) w P RP T =+ (2)where ()ˆˆuv represents the real point caused by image distortion; 1k , 2k stand for the first and second order distortion coefficient, which are known already; P represents the 3D coordinates of one point in camera coordinate; c Z is the scale factor; 1M represents the intrinsic parameter of the camera; rotationmatrix R and translation matrix T are the extrinsic parameters. w P is the corresponding coordinate of P in the world coordinate system.In this paper, a high precision machining calibration target is utilized to calibrate the cameras which are motionless during the calibration process. The calibration of binocular vision measurement system is implemented according to the Zhang’s 2D method [12] to achieve the high calibration precision, the calibration targets are shown in Fig. 3.Fig. 3. Calibration targets[]P X Y Z =IV. R EFRACTION OF L IGHTA. Light refract occurs on Region ARefraction of light refers to the change in the direction of a light ray due to a change in its speed. This is commonly observed when a light ray passes from one medium to another (e.g., from air to water). The refractive index of a medium is defined as the ratio of the velocity of light in vacuum to the velocity of light in the said medium. Therefore, the light path changes after the glass is added, light refraction will occur on the glass surface, Assuming that the lens is located randomly and its normal vector is n, the light path with refraction is shown in figure 3.Fig. 4. Imaging model of Region A With the theory of refractive geometry, the 3D coordinate of a particular point P is given by001122012T T T d d d P v v v v n v n v n =--- (3)where 2v denotes the direction vector of the light-path in the segment closest to the target, 1v denotes the direction vector of the light-path in the lens, 0v denotes the direction vector of the light-path in the segment closest to the camera, 0d ,1d ,2d denote the thickness of each segment, respectively, n is the normal vector of the lens.111i i i i a b +++=+v v n (4)11/i i i a u u ++= (5)11i i b ++= (6)where 1u , 3u is the air refractive index, we assume 311u u == without loss of generality. 2u is therefractive index of the lens.OB. Light refract occurs on Region BFig. 5. Imaging model of Region BWith the theory of refractive geometry, when light refracts on Region B as a result of lens rotated, the 3D coordinate of a particular point P is given by001122012T T T d d d P v v v v n v n v n ''''''=---''' (3) where 2v ' denotes the direction vector of the light-path in the segment closest to the target, 1v ' denotes the direction vector of the light-path in the lens, 0v ' denotes the direction vector of the light-path in thesegment closest to the camera, 0d ',1d ',2d 'denote the thickness of each segment, respectively, n is the normal vector of the lens.111i i i i a b +++''''=+v v n (4)11/i i i a u u ++'= (5)11i i b ++'= (6)where 1u 、3u is the air refractive index, we assume 311u u == without loss of generality. 2u is therefractive index of the lens.C. 3D reconstructionIn this paper, for the static objects, information of the same feature point in images after light refraction on Region A and B is extracted. Then equations after light refraction on Region A and B are set up respectively to obtain the 3D coordinates of targets. First, image features in the image taken after light refraction on Region A is extracted. Since the lens is flat, 0v equals to 2v , and we have (3)012P d d d =++. With eq.(1), we can write eq.(5), establishing therelationship between the image features and the 3D coordinates in the camera coordinate:O P(3)011001()T T P d P -=--v v v v n v n (10)Then image features in the image taken after light refraction on Region A is extracted. As mentionedabove, 0'v equals to 2'v , and we have (3)(3)012Pd d d P ''''=++=. With eq.(2), we can write eq.(6): (3)011001()T T P d P ''''-=--''v v v v nv n (11) With eq.(10) and (11), the 3D information of P in the camera coordinate system can be solved, and it can be further transformed to the world coordinate system through eq.(12). w P is the required 3D coordinate offeature points in the world coordinate system.w P RP T =+ (12)V. EXPERIMENTSIn this paper, corresponding experiments are conducted in the laboratory with the proposed method. Experiments are conducted in the laboratory to verify the feasibility of the proposed method and the measurement precision of the system. Results show that the system has good measurement accuracy.A. Experimental systemThe monocular measuring system proposed here is simple in structure, which only requires a single camera with rotating lens. With the above system, the 3D measurement of target can be completed.A single camera (phtron UX50), a lens (Nikon 18-24 mm), a rotating lens with thickness of 10 mm and 40mm, and a computer serves as graphic workstation are used to finish the experiment.B. Measurement experiment in the laboratoryThe 3D coordinates of feature points on a calibration target are measured in the laboratory using the equipment shown in figure 5. Firstly, Zhang's method is used to calibrate the internal and external parameters the camera, and results are shown in table 1.TABLE I. C ALIBRATION RESULTS OF THE GLOBAL OPTIMIZATIONThen, the 3D coordinates of feature points on the calibration target whose size and accuracy are 300 mm*400 mm and 0.02 mm, respectively, are measured. Images are taken before and after light refraction, as shown in figure 6.Then, the 3D coordinates feature points on the calibration target are restored, of which the results are shown in figure 7. Finally, the 3D measurement based on the refraction principle is achieved with relatively stable results. According to the calibration target, the average error of the results is 0.1 mm.Fig. 7. Measurement targetC.Accuracy verification experimentIn this paper, verification experiments for the accuracy of measurement system are conducted in the laboratory. Using a calibrated camera, repeated tests are conducted to measure the length of a high precision 1D target whose length is 350.0177 mm. Results show that the accuracy can of the system can reach 1 um.Fig. 8. Measuring the 1D target 20 tests are conducted to measure length of a known 1D target with the proposed method, of which results are shown in figure 8. As in shown in the figure, the average error of the measurement is 0.24 mm.D i s p l a c e m e n t D e v i a t i o n (m m )TestFig. 9. The precision of the measurement systemThe measuring accuracy of the proposed methods will only be effected by three elements: the calibration precision of internal and external parameters of the camera, the thickness of the lens, and the measuring accuracy of the normal vector of the lens. Error of the proposed system mainly comes from distortion of the optical lens and the inaccurate extraction of feature points.VI. C ONCLUSIONA monocular vision 3D measurement method and measuring system based on a special rotatable lens have been proposed. With the proposed method and system, the measuring process of monocular vision measurement was simplified, and cost for the measurement was reduced.∙Automatic measurement of 3D information can be achieved by rotating special lens, which improves the efficiency.∙Verification experiments were conducted in the laboratory, and results show that the system has high measurement precision. The measurement accuracy can reach 0.95 mm, which shows the possibility for real application.A CKNOWLEDGMENTThis study was co-supported by the National Natural Science Foundation of China (Nos. 51375075, 51227004), the National Basic Research Program of China 973 Project (No.2014CB046504).R EFERENCES[1]S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski. A comparison and evaluation ofmulti-view stereo reconstruction algorithms. In Proc. Conf. Computer Vision and Pattern Recognition, volume 1, pages 519–528, 2006.[2]D. Lee and I. Kweon. A novel stereo camera system by a biprism. IEEE Transactions on Roboticsand Automation, 16:528–541, 2000.[3]H.-G. Maas. New developments in multimedia photogrammetry. Optical 3-D MeasurementTechniques III, 1995.[4]C. Gao and N. Ahuja. Single camera stereo using planar parallel plate. In Proc. Int. Conf. on PatternRecognition, volume 4, pages 108–111, 2004.[5]C. Gao and N. Ahuja. A refractive camera for acquiring stereo and super-resolution images. In Proc.Conf. Computer Vision and Pattern Recognition, volume 2, pages 2316–2323, 2006.[6]M. Shimizu and M. Okutomi. Reflection stereo – novel monocular stereo using a transparent plate.In Proc. On Canadian Conference on Computer and Robot Vision, pages CD–ROM, 2006.[7]M. Shimizu and M. Okutomi. Monocular range estimation through a double-sided half-mirror plate.In Proc. on Canadian Conference on Computer and Robot Vision, pages 347–354, 2007.[8]M. Shimizu and M. Okutomi. Calibration and rectification for reflection stereo. In Proc. Conf.Computer Vision and Pattern Recognition, pages 1–8, 2008.[9]N. Morris and K. Kutulakos. Dynamic refraction stereo. In Proc. Int. Conf. on Computer Vision,volume 2, pages 1573–1580, 2005.[10]N. Morris and K. Kutulakos. Dynamic refraction stereo. In Proc. Int. Conf. on ComputerVision, volume 2, pages 1573–1580, 2005.[11]Zhihu Chen, Kwan-Yee K. Wong, Yasuyuki Matsushita, Xiaolong Zhu, and Miaomiao Liu,Self-Calibrating Depth from Refraction. In Proc. Int. Conf. on Computer Vision, pages 635 - 642, 2011.[12]Z.Y. Camera calibration with one-dimensional objects. IEEE Trans. Pattern Anal. Mach.Intell. volume 26, pages 892–899, 2004.。