2外文原文1

Seam Tracking of Articulated Robot for Laser Welding Based on Visual Feedback ControlWenzeng Zhang, Qiang Chen, Guoxian Zhang, Zhenguo Sun, and Dong DuKey Laboratory for Advanced Materials Processing Technology, Ministry of Education, Department of Mechanical Engineering, Tsinghua University, Beijing 100084, P.R. China Abstract. Laser welding has a quite high demand on trajectory accuracy of articulated robot. To improve dynamic trajectory accuracy of articulated robots for laser welding, a novel 3D seam tracking method is proposed based on stereo visual feedback control. The method con-structs a visual feedback control system with two convergent cameras mounted at the end of in-dustrial robot. A tool coordinate system is constructed to transfer the end-link displacement of robot to that of the tool. A proposed GPI transform method, binocular vision technologies and a modified algorithm of line-line matching are utilized to calculate the positions of laser focus and weld seam, which makes the dynamic trajectory error between laser focus and weld seam can be calculated. The robot is finally commanded to move and decrease the trajectory error as soon as possible based on robot kinematics. Experimental results show that the method can ef-fectively improve the trajectory accuracy of industrial robot for laser welding.Keywords: industrial robot, visual feedback, seam tracking, trajectory accuracy.1 IntroductionNowadays, articulated robots for sale only provide customers their positioning accu-racy, never offering or guaranteeing their trajectory accuracy. However, along with the development of manufacturing and processing, some high-speed and high-accuracy jobs, such as laser welding and cutting, have a quite high demand on trajec-tory accuracy. Moreover, current industrial robots could only move under scheduled instructions under a strictly structured environment, which limits their applications. Lots of research schemes are proposed to improve the ability of industrial robots in environmental understanding. As an important sensing method, vision plays an im-portant role in improving the industrial robots’ ability in environmental understanding and variance adaptation.In reference [1], a visual servoing system of industrial robots based on position was constructed, and algorithm to estimate the object position and pose with a camera is proposed when the distances of several characteristic points were known beforehand. In reference [2], an eye-on-object method was realized in planar moving object track-ing based on an eye-in-hand visual servoing structure. In reference [3], the problem that the actual position of end link of robot is far from the anticipant one was solved with a method of spatial path planning and image-based control. In reference [4], an industrial flame tracking system was developed to cut planar drawings based on vision. T.-J. Tarn et al. (Eds.): Robot. Weld., Intellige. & Automation, LNCIS 362, pp. 281–287, 2007. © Springer-Verlag Berlin Heidelberg 2007282 W. Zhang et al.In this paper, in order to improve the moving trajectory accuracy of industrial ro-bots for laser welding, a GPI transform method for seam recognition [5] in trajectory tracking was utilized, and an improved algorithm was proposed to reconstruct the tra-jectory line and laser focus more precisely. Meanwhile a tool coordinate system was constructed skillfully to make the tool displacement calculation easily and quickly. The modified displacement of the robot is calculated and the robot is finally com-manded to move and decrease the trajectory error as soon as possible based on robot kinematics.2 3D Reconstruction of Laser Focus and Trajectory LinesThe hardware of the established robot visual servoing system is shown in Fig.1. A la-ser processing tool and two cameras are mounted on the end of articulated robot. The laser focus and trajectory line are imaged in both cameras.Fig. 1. System hardware componentsFig. 2. Coordinate framesThe control structure of system is shown in Fig. 2. The established frames of sys-tem is shown in Fig. 3.Where, {}R{}E--the base and end-effecter frames of robot ,CR--the frames of left and right camera,{}CL{}T--the frames of tool,{}Seam Tracking of Articulated Robot for Laser Welding 283M , N --the laser focus and the desired point to be welded,A B T --the transformation matrix between frame A and B.The inner and outer parameters of cameras [1], stereo vision coordinates, transform matrix of coordinates between the robot hand and eye [6], and relation matrix be-tween the robot hand and tool are all calibrated beforehand.Fig. 3. Coordinate framesThe laser focus and trajectory line in two images are detected by a threshold seg-mentation method and GPI transform [5]. Since images in left camera and right camera are matched, the position of laser focus in the coordinates of left camera can be recon-structed based on the principle of stereo vision reconstruction.In reference [7], a line-line matching method was proposed to reconstruct the 3D line from the images in left and right cameras. Experimental results show that this method usually causes errors, which may reduce when the camera goes away from the object observed. The reason is that the model of pinhole imaging is always used. The model of pinhole imaging is righter when the camera is farer from the object, which formula is:CL CL Z f Z −≈.However, to improve the image resolution, it is not suitable to put the camera far away from the object. Then an improved line-line matching method is proposed which is established based on:CL CL Z f Z −≠. The improved projection imaging formula is:00()()CL i i CL i x CL i i CL i y f X u u f Z S f Y v v f Z S ­=+°−°®°=+°−¯(1) where, (,)i i u v --a point coordinates in image, pixel,284 W. Zhang et al.00(,)u v --the center point coordinates of image, pixel,,x y S S --the width and height of a pixel, mm,f --the focus length of lens, mm,(,,)CL CL CL i i i X Y Z --the coordinates of a point on the line in {}CL , mm. The equation of spacial line in {CL} can be gained as follows: 000CL CL CL CL CL CL X X Y Y Z Z l m n −−−== (2)where, 000(,,)CL CL CL X Y Z --coordinates of a point on the line in {}CL , mm, (,,)CL CL CL l m n --the direction vector of line detected in {}CL , mm.The purpose of the control system is to keep the laser focus on the trajectory line by moving the industrial robot. In order to control the robot movement, the pedal point of laser focus on trajectory line should be calculated beforehand.Because the curvature of curve weld seam is generally small, the curve weld seam can be considered approximately as multiple segments of straight lines. The pedal-finding problem is changed to calculating the pedal N of the laser focus M in a straight line. Construct a plane that covers point M and is perpendicular to the trajec-tory line. Therefore, the cross point of this plane and trajectory line is the pedal.000()()()0CL CL CL CL CL CL M M M CL CL CL CL CL CL l X X m Y Y n Z Z X X Y Y Z Z l m n ­−+−+−=°®−−−==°¯(3) where, (,,)CL CL CL M M M X Y Z --coordinates of the laser focus in {}CL , mm.With equation (3) the pedal position (,,)CL CL CL N N N X Y Z in {}CL is gained.3 Displacements CalculationChange the coordinates of calculated pedal and laser focus from {}CL to {}T , then calculate the displacements in joint space of robot with Jacobian matrix in {}T . Con-struct the tool coordinates {}T (shown in Fig. 2) as follows:(1) Choose the laser focus O T as the original point,(2) Choose the laser beam as Z T , pointing from the fiber torch to object welded,(3) Axis X T is set by equation (4):/E E E E E T T CL T CL =××x z x z x (4)Seam Tracking of Articulated Robot for Laser Welding 285(4) Axis Y T is set by equation (5):EE E T T T =×y z x (5) where, ECL x --the description ofCL x in {E}, ,,E E E T T T x y z --the description of ,,T T T x y z in {E}. The displacement T MN p in {}T can be calculated by (6):()T T E CL CL MN E CL T T N M =−p(6) The calculation of Jacobian matrix in {}T is similar with that in {}E , the only difference is that using 11n n E n n T B A T −−= to replace the 1n n A −. With this method,the displacement calculation of tool end is changed to the displacement calculation of robot end, which would be easier and quicker. (1n n A −is the transformation matrixfrom the n-1 coordinates to the last coordinates.)Keep the orientation of robot end-link unchangeable when tracking. Calculate the displacement in joint space with Jacobian matrix in {}T :[]()1116610n T T T MN i n n q q q J ×−×תº⋅⋅⋅⋅⋅⋅=«»¬¼p (7) where, ,(1,2,,)i q in =⋅⋅⋅--displacement of the robot joint i, n --number of the robot joint, T J --Jacobian matrix in {}T .4 Experiments and AnalysisThe experimental results show that the trajectory detecting and control system of welding robot based on stereo vision can be well used in trajectory recognizing, de-tecting and automatic tracking. The error of tracking is less than 0.5 mm in real-time. For example, two steel sheets are butted closely without groove. The tracking speed of robot end is 20 mm/s. The position and orientation of weld seam (trajectory line) is unknown beforehand. Fig.3a is the image captured before tracking. In this im-age two short parallel white lines is the edges of sub-window searched; the long white line is the trajectory line detected by GPI transform method, and the white circle point is laser focus; the black square point is the pedal of laser focus and welding line cal-culated in 3D space with stereo vision algorithm. Fig.3b shows the image of tracking result, where the laser focus has already moved to the pedal and goes along with the welding line.286W. Zhang et al.(a) image before tracking (b) image after trackingFig. 4. Image before and after tracking in left camera\m mX ∆\m m s \m mY ∆\m ms(a) error in X(b) error in Y \m mZ ∆\m ms \m md ∆\m ms(c) error in Z (d) spacial distance error Fig. 5. Tracking errorThe errors in X, Y, Z and the spacial distance error of tracking are shown in Fig. 4. Experimental results show that the errors in X, Y, Z are less than 0.3 mm, the spacial distance error is less than 0.4 mm. The system could not only recognize the welding seam automatically, but also meet the welding requirement in real-time trajectory tracking and control.5 ConclusionsIn this paper, a 3D seam tracking method of articulated robot for laser welding based on stereo visual feedback control is proposed. The method can be used to improve dynamic trajectory accuracy of articulated robots in precise materials processing such as laser welding and cutting. With the proposed algorithm the trajectory line and laserSeam Tracking of Articulated Robot for Laser Welding 287focus can be reconstructed accurately by 3D vision. The needed tool displacement is calculated easily and quickly by a suitable tool coordinate frame. The robot is finally commanded to move and decrease the trajectory error as soon as possible based on robot kinematics. Experimental results show that the method can effectively improve the trajectory accuracy of industrial robot for laser welding, and the system used the method could meet the demand of high accuracy trajectory tracking.AcknowledgementsThis research was supported by Chinese National Natural Science Foundation (No. 50275083), and The Research Fund for the Doctoral Program of Higher Education (No. 20020003053).References1.Mana Saedan, Marcelo H, 3D Vision-based Control On an Industrial Robot[A]. Proc ofIASTED Intel Conf on Robotics and Applications [C]. Florida, USA. Nov, 2001: 152-157 2.Wang Jianhua, Zhao Xiaoguang, Tan Ming. Research on Tracking Control of a 5-DOF Vis-ual Servo Robot [J]. Robot. 2005, 27(3): 215-219 (in Chinese)3.Mezouar Y, Chaumette F. Path Planning for Robust Image-based Control [J]. IEEE Transon Robotics and Automation, 2002, 18(4): 534549.4.Xu Kanlie, Fei Renqing, Wang Xiaogang etc. Research on 2D Line Segment Tracking Sys-tem [J]. Journal of Shanghai University. 1999, 5(4): 348-352 (in Chinese)5.Zhang Wenzeng, Chen Qiang, Du Dong etc. Gray Projecting Integral Method for Line De-tection [J]. Journal of Tsinghua University(Science and Technology). 2005, 45(11): 1446-1449 (in Chinese)6.Shiu Y C, Ahmad S. Calibration of Wrist-mounted Robotic Sensors by Solving Homogene-ous Transform Equations of the Form AX=XB [J]. IEEE Trans of Robotics and Automa-tion, 1989, 5 (1): 16-277.Ma Songde, Zhang Zhengyou. Computer Vision: Computation Theory and Algorithm Basic[M]. Beijing: Science publishing company. 1998 (in Chinese)。