Mixed Control of Multi-Fingered Haptic Interface Robot with operator’s uneasy feeling in m

Mixed Control of Multi-Fingered Haptic InterfaceRobot with operator’s uneasy feeling in mindHaruhisa Kawasaki and Tetsuya Mouri(Gifu University, 1-1 Yanagido, Gifu, 501-1193 Japan, {h_kawasa, mouri}@gifu-u.ac.jp) Abstract—This paper presents a mixed control of multi-fingered haptic interface robot consisting of a 6 DOF arm and a 15 DOF hand. The interface is placed opposite the human hand and presents force and tactile feeling to the fingertips. In order to reduce the uneasy feeling in the operator, a mixed control consisting of a finger force control and an arm position control intended to maximize the control performance index, which consists of the hand manipulability measure and the norm of the arm-joint angle vector, is proposed. Experimental results demonstrate the effectiveness of the mixed control.Keywords—Haptic interface, Force control, Manipulability, Design, Virtual reality1.INTRODUCTIONHaptic interfaces that present force and tactile feelings have been utilized in the areas of telemanipulation [1], interaction with micro/nano scale phenomena [2], medical training [3], and so on. A multi-fingered haptic interface has a greater potential for these applications than does a single point haptic interface. A haptic interface consisting of an arm and fingers [4]-[9] can be used in a wide space. Most of them consist of a hand and arm exoskeleton system. With this system, it is hard to represent the weight of virtual objects through the fingertips because the hand mechanism is mounted on the back of human hand. Fixing the haptic interface to the hand binds the hand and creates an oppressive sensation in the operator. Moreover, the operator will have a strong feeling of unsafety when the system shows an abnormal performance.A haptic interface must be safe, function in a wide operation space, and represent not only force at the contact points but also the weight of virtual objects. In addition, it should not cause an oppressive feeling when it is attached to humans and should not represent its own weight. In response to these issues, we developed a five-fingered haptic interface named HIRO II [10], which consists of a 6 DOF arm and a 15 DOF hand. The interface is placed opposite the human hand, which ensures safety and freedom of movement. The controller of the haptic interface opposite the human hand is required to take redundancy of the mechanism into consideration in general. At the same time, to enhance the operator’s sense of identification with the hand’s movement, the excessive arm motion had to be reduced.This paper presents a mixed control consisting of finger force control and arm force control for multi-fingered haptic interface robot, which is an upgraded version reference [13]. In the arm control, a desired hand pose is determined that will maximize the control performance indexconsisting of the hand manipulability measure and the norm of the arm joint angle vector, which is introduced to reduce the operator’s uneasy feeling. Several studies have addressed the manipulability measure for a robot arm [14], for cooperative arms [15], and for dexterous hands[16]. Most of them have utilized the manipulability measure for their mechanism design and off-line path planning, but they have not used it for real-time control of the hand and arm robot systems. In the proposed control method, the manipulability measure is used in real-time control. A great potential of the multi-fingered haptic interface robot utilizing the mixed control method is demonstrated experimentally.2. M UTI -FINGERED HAPTIC INTERFACE ROBOTMulti-fingered haptic interfaces joined to the arm can provide a wide operation space. However, most of them are mounted on the back of a human hand in the manner of the CyberForce [9]. Fixing the haptic interface to the hand binds the hand and creates an oppressive sensation in the operator. In order to reduce this feeling and increase safety, we have developed a Haptic Interface RObot named HIRO II [10]. Figure 1 shows the developed five-fingered haptic interface where it is connected to the five fingers of an operator‘s hand. The haptic interface consists of an interface arm, a haptic hand with five haptic fingers, and a controller. When the operator moves his/her hand, the haptic interface follows the motion of operator’s fingertips and presents the sensation of force. The operator feels only a small constriction because the coupled part between the human hand and the haptic interface is centered only on the fingertips of the operator.The haptic hand consists of 5 fingers, which are designed to be similar to human fingers in geometry and range of motion. The developed haptic finger is shown in Figure 2. It is designed based on an anthropomorphic robot hand, named Gifu Hand III [17]. Each finger has 3 joints allowing 3 DOF. The first joint, relative to the hand base, allows abduction/adduction (rotation about the y axis shown in Fig. 2). The second joint and the third joint allow flexion/extension (rotation about the z axis shown in Fig. 2). All joints are driven by DC servomotors with geartransmissions and rotary encoders. Maximum output joint torques from the 1st to the 3rd finger joints are 0.96, 0.52, and 0.20 Nm, which provide force on a fingertip of 7.8, 4.2, and 3.9 N, Fig. 1. Five-fingered haptic interface robot: HIRO II2nd link1st link 1st motor xForce sensor spherical jointFinger holder z Fig. 2. Haptic fingerrespectively. In order to read the finger loading forces, the force sensor (NANO sensor made by BL AUTOTEC, LTD.) in the second link of each finger is installed. The resolution of the force sensors F x , F y , and F z , are 32, 32, and 98 mN, respectively.To manipulate the haptic interface, an operator has to wear a finger holder on his/her fingertips as shown in Figure 1. The finger holder contains an iron sphere, which is attached to the permanent magnet at the force sensor tip to form a passive spherical joint. This passive spherical joint plays two roles. One, is to adjust to differences between the human and haptic fingers orientations. Each human finger attached to the arm has 6 DOF spatially, while each haptic finger has 3 DOF. Hence, additional passive 3 DOF are needed. The second role is to ensure that the operator can remove his fingers from the haptic interface if it malfunctions. The suction force created by the permanent magnet is 5 N.The interface arm has 6 joints allowing 6 DOF. The lengths of the upper arm and the forearm are 0.3 and 0.31 m, respectively. The arm joints are actuated by AC servomotors equipped with rotary encoders and gear transmissions. The maximum output force is 45 N and the maximum velocity at the haptic hand base is 0.8 m/s. The interface arm has a size similar to the human arm, and a workspace of the wrist compatible with it.3. M IXED C ONTROLThe control of HIRO II adopts a mixed finger force control and arm position control to follow the operator‘s fingertip motion and to present the sensation of force while allowing the operator to identify with the device’s motion.3.1 Finger Force ControlLet us denote a kinematic Jacobian, joint torque, joint angle of the i -th finger, contact force, and desired contact force between the human and the haptic i -th fingers as 33×∈R Fi J , 3R Fi ∈τ, 3R Fi ∈q , 3R i ∈F , and 3R di ∈F , respectively. Then, the haptic finger is controlled by joint torque based force feedback control as follows:())(321F F F F d e F e F T F F dt q g qK F F K F K J τ+−++=∫&, (1) whereT F T T F ),,,(521ττττL =, T T F T F T F F ),,,(521q q q q L =, ),,(51F F F diag Block J J J L =, T T T T ),,,(521F F F F L =, T T d T d T d d ),,,(521F F F F L =, i d e F F F −=, K F1 >0 is the force feedback gain matrix, K F2 >0 is the force integral feedback gain matrix, K F3 is the velocity feedback gain matrix, and 15)(R F F ∈q g is the gravitational compensator term. Each finger joints is controlled to follow the desired finger force independently. In this control, a finger that reaches the limit of the movable range is switched to a position control to keep the joint angle in the movable range, while the remaining fingers are controlled by the force control. After reaching the limit of the movable range, the finger that switched to position control is switched back again to force control when the direction of the joint torque input is in the same direction of that of the joint angle apart from the limit of the movable angle.3.2 Arm Position ControlIn order to avoid the large arm motion at the small motion of the operator’s fingers, we considerthe following control performance index added an arm joint angle norm to the hand manipulability measure:()∑=++=51i A i i i i Q P W CPI βα, (2) where i α and i β are weighting coefficients and W i is a manipulability measure of the i -th finger, P i is a penalty function of the i -th finger to keep the joint angle within the range of movement, and A Q is a penalty function to avoid large arm motion. The manipulability measure of the i -th finger is given by()21)det(Fi T Fi i W J J = (){}322213cos cos sin i i i i i i q q l q l q ++= , (3)where l ij is the j-th link length of i -th finger and q ij is the j-th joint angle of the i -th finger. Equation(4) shows that the manipulability measure of finger does not depend on the first joint angle q i1. Figure 3 shows values of W i by using the finger length of HIRO II. However, this maximum point is near the limit of the movement range of q i 2. Moreover, there is a line of singular points at q i3 = 0 rad, which leads to a local extreme of W i . In order to keep the finger joint angles within the movement range, the following penalty function is added:(){}(){}[]∑=−+−−−=31exp exp j ij ij ij ij j i b q a q P μμγ,(4) where j γ is the weighting coefficient of the j -th joint, μ is the parameter to adjust an exponential function, ij a and ij b are the lower limit and the upper limit of the j -th joint angle of the i -th finger,respectively. Figure 4 shows the values of i i i i P W βα+under the condition of 0.1=i α, 5104.3×=i β , 2.11=γ, 0.12=γ, 1.13=γand 0.10=μ. The value of i i i i P W βα+ has one localextreme for q i3 > 0. The hand manipulability measure given by the first term of the left hand side of equation (2) leads to the optimum position and orientation of the hand base; however, this sometimes creates a θ3(rad)M a n i p u l a b i l i t y m e a s u r e (m m 3)Maximum pointθ2(rad)Fig. 3. Manipulability measure of finger W iθ3(rad)M a n i p u l a b i l i t y m e a s u r e (m m 3)Maximum pointθ2(rad)Fig. 4. Modified manipulability measure of finger i i i i P W βα+large arm movement even if the operator moves his/ her finger only slightly. This phenomenon make the operator unsafe because operator usually thinks that the haptic arm does not move widely when his/her finger motion is small. To prevent a large change of arm joint angle, the following penalty function is added:()()A Ad T A Ad A Q q q Γq q −−−=21, (5) where 6R A ∈q is the current arm joint angle vector, 6R Ad ∈q is an desired arm joint angle vector, and 0>Γis a weighting matrix. The desired arm joint angle is defined to maximize equation (2) under a constraint condition in which the five haptic fingertip positions F q are fixed to the operator’s fingertip positions. Equation (2) can be rewritten as a function of F q and A q . Hence, the optimum arm joint angle Ad q is computed by the conjugate gradient method on line with the convergence criterion CPI[k ]-CPI[k -1] < δ , where k is the iteration number and δ is the scalar constant. At the same time, a maximum iteration number is limited to 30 because of the limitation of the control’s sampling time. Figure 5 shows the relation between the value of the control performance index and the iteration number at 2100.1×=δ. In most cases, the optimum arm joint angle Ad q is obtained at some iteration number, because the current arm joint angle A q is treated as an initial point near the extreme of the control performance index.The haptic arm is controlled by the following PD control with gravitational and external force compensators:Ae T A A A A Ad A A Ad A A F J q g q qK q q K τ++−+−=)()()(21&&, (6) where 6R A ∈τ is the haptic arm joint torque, K A1 is the position feedback gain matrix, K A2 is the velocity feedback gain matrix, )(A A q g is the gravitational compensator term, 66×∈R A J is the kinematic Jacobian of the haptic arm, and Ae F is a external force compensator term given by⎟⎟⎟⎟⎠⎞⎜⎜⎜⎜⎝⎛×=∑∑==5151i i Fi i i Ae F p F F , (7) where Fi p is the position vector from the origin of hand base coordinate to the i -th haptic fingertip.This term is the force and moment equivalent to the total external forces at the haptic fingertips. 4.Finger responses induced by the force control when pushing a virtual wall are shown in Figure 6. In this case, the operator’s index finger is connected to the haptic interface. Initially the finger approaches the wall, expressing a response in the free space. The desired force at the finger is set to zero. The average force error is 0.08 N. The finger then pushes the wall, expressing a response in the constraint space. The desired force at the finger is set to penetration depth of the finger into the wall, K (= 2.0 N/mm) is the stiffness of the wall, and d (=5.5x10-5 Ns/mm) is the damping coefficient of the wall. The average force error is almost the same as that in free space. We constructed a virtual piano system using HIRO II as shown in Figure 7 to evaluate the 051.52x106C o n t r o l p e r f o r m a n c e i n d e x Iteration number 1.50x1061.48x106 Fig. 7. Virtual piano systemmotion performance of the device. An operator can play piano with his/her right hand. Sounds are generated when the operator’s fingertips touch the surface of the virtual keyboard. Figure 8 shows position and force responses of the thumb and the little finger on the condition of the most rapid tap timing, in which the arm motion as well as the thumb and the little finger motions are generated. This shows that an acceptable tempo is about 5 times per second. Most of the applications will be used at less than this motion rate. Moreover, the proposed mixed control reduce the feeling of insecurity in the operator as compared with a arm control based on hand manipulability measure only .5.C ONCLUSIONA mixed control of a multi-fingered haptic interface in consideation of the operator’s uneasy feeling has been presented. A mixed control method involving finger force control and arm position control has been evaluated experimentally. In the arm control, a control performance index to reduce the feeling of insecurity in the operator has been presented. This interface can be utilized in a wide range of applications as a master or a slave in real and virtual environments. Future research will be directed toward the implementation of the interface in various applications such as the medical training and the teleoperation of the anthropomorphic robot hand.A CKNOWLEDGMENTThe authors’ thanks are addressed to the members of the haptic interface research project of Gifu University.R EFERENCES[1]I. Ivanisevic and V. J. Lumelsky, “Configuration space as a means for arugmenting humanperformance in teleoperation tasks,” IEEE Trans. on SMC, Part B, vol. 30, no. 3, pp. 471-484, 2000.[2]M. Guthold, et. al. ,“Controlled manipulation of molecular samles with the nanomanipulator,”IEEE/ASME Trans. on Mechatronics, vol. 5, no. 2, pp. 189-198, 2000.[3]C. Basdogan, C. H. Ho, and M. A. Srinivasan, “Virtual environments for medical training:graphical and haptic simulation of laparoscopic common bile duct exploration,” IEEE/ASME Trans. on Mechatronics, vol. 6, no. 3, pp. 269-286, Sept. 2001.[4]M.S. Shimamoto, ``TeleOperator/telePresence System (TOPS) Concept Verification Model(CVM) Development,'' N.K. Saxena, ed., Recent Advances in Marine Science and Technology, '92, Pacon International, pp. 97-104, 1992[5]M. Bergamasco, B. Allotta, L. Bosio, L. Ferretti, G. Parrini, G.M. Prisco, F. Salsedo, G. Sartini,An arm exoskeleton system for teleoperation and virtual environments applications, Proc. of 1994 IEEE International Conference on Robotics and Automation, pp 1449 – 1454, 1994 [6]H. Maekawa and J. H. Hollerbach, Haptic display for Object Grasping and manipulating inVirtual Environment, Proc. of 1998 IEEE Int. Conf. On Robotics and automation, pp.2566-2573, 1998[7]A. Nakai, Y. Kunii, and H. Hhashimoto, Development of 7 DOF Exoskeleton Type HapticInterface, Journal of RSJ, vol. 17, no.8, pp.1126-1133, 1999 (in Japanese)[8]Y. Adachi, et. al., “Development of a haptic device for multi fingers by Macro-MicroStructure,” Journal of the Robotics Society of Japan, vol. 20, no. 7, pp. 725-733, 2002 (in Japanese)[9]Immersion Corporation, “CyberForce,” url:/3d/products/cyber_force.php[10]H. Kawasaki, T. Mouri, M. O. Alhalabi, Y. Sugihashi, Y. Ohtuka, S. Ikenohata, K. Kigaku, V.Daniulaitis, K. Hamada and T. Suzuki, “Development of five-fingered haptic intereface: HIRO II,” Proc. of ICAT 2005, Christchurch, New Zealand, pp. 209-214, 2005[11]J.E. Colgate, P.E. Grafing, M.C. Stanley, G. Schenkel, Implementation of stiff virtual walls inforce-reflecting interfaces, Proc. of IEEE Virtual Reality Annual International Symposium, pp.202-208, 1993[12]R.J. Adams and B. Hannaford, Control Law Design for Haptic Interfaces to Virtual Reality,IEEE Trans. on Control System Technology, vol. 10, no.1, pp.3-13, 2002[13]T etsuya Mouri, Haruhisa Kawasaki, Kazugige Kigaku, and Yoshio Ohtsuka, Novel ControlMethods for Multi-fingered Haptic Interface Robot, Proc. of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2006), Beijing, pp. 1576-1581, 2006 [14]T. Yoshikawa, “Measure of manipulability for robot manipulator,” Trans. of JRM, Vol. 2,No.1, pp. 63-67, 1983 (in Japanese)[15]S. Lee, “Dual redundant arm configuration Optimization with task-oriented dual armmanipulability,” IEEE Trans. on Robotics and automation, Vol. 5, No.1, pp. 78-97,1989 [16]A. Bicchi, C. Melchiorri, and D. Balluchi, “On the mobility and manipulability of generalmultiple lim Robots,” IEEE Trans. on Robotics and automation, Vol. 11, No.2, pp. 215-228, 1995[17]T. Mouri, H. Kawasaki, K. Yoshikawa, J. Takai, and S. Ito, “Anthropomorphic robot hand:Gifu Hand III, Proc. of Int. Conf. ICCAS2002,” pp.1288-1293, 2002。