虚拟拔河力觉模拟应用系统设计 - 北京航空航天大学

Modeling of Intelligent Behaviour in Haptic Enabled Tug-of-War Simulation Dangxiao Wang Yuru Zhang Yonggang CaoRobotics Institute, Beihang University, Beijing, P. R. ChinaE-mail: hapticwang@, yuru@AbstractVirtual environment (VE) is a prime component in haptic interaction system. Traditional passive VE is not suitable to model human-machine interaction intelligence in haptic system such as a virtual tug-of-war system. Novel taxonomy of virtual environment is proposed according to different interaction tasks in physical world. Based on the difference between passive and active VE, intelligent VE is defined as a special type of nonlinear and active VE. The components of intelligent VE are identified and challenges within modeling and rendering method are discussed. Based on a one-dimensional haptic device, a virtual tug-of-war prototype is developed and different rendering algorithms are compared. Experiment results illustrate that the intelligent VE enhance the fidelity of the tug-of-war process and the rendering algorithm could ensure stability of the haptic system. The developed prototype can be utilized in entertainment areas.1. IntroductionHaptic-enabled human-computer interaction is a new area originated from virtual reality and robotics field, which opens new potential for surgical simulation, entertainment, rehabilitation etc [1]. Virtual environment is the fundamental element of haptic interaction system. According to the diversity of simulated tasks, there are various virtual environments in the haptic systems. Classification and modeling of various virtual environments are an important issue for stability analysis and fidelity evaluation of haptic systems [2,3].Kirkpatrick et al. compared different operating tasks and proposes the concept of operating mode [4]. Bloomfield et al. classified virtual environments via the haptic action in disassembly task [5]. Carignan et al. classified the haptic system according to the characteristic of the control system [6]. In these systems, VE are usually assumed to be passive, and the modeling method mainly focused on physically based rendering. However, under active or intelligent VE, for example, when it is needed to simulate real time tug-of-war between human and a virtual rival within computer, physically based rendering method is not suitable. It is necessary to reconsider the classification and modeling method of virtual environment.Haptic systems based on intelligent VE can be found in many application areas. Entertainment industry is one promising area. Immersion Inc. and Microsoft Inc. have introduced force feedback effects in their computer games [7,8]. In entertainment systems, the intelligent interaction between human and machine need to be simulated. Because the human’s intelligence participates in the interaction process, the VE should response to human’s input in real time. The VE is no longer passive and static. It becomes “intelligent”. The potential of intelligent VE on research and application is a relatively new topic [9].In this paper, new classification of VE is proposed and modeling method of intelligent VE is explored. The reminder of the paper is organized as follows. Section 2 proposed the concept of intelligent VE based on classification of various VE. Section 3 introduces development of a virtual tug-of-war system. Section 4 gives experiment results of the virtual tug-of-war system. Section 5 gives conclusion and future work.2. Concept of intelligent VE2.1 Preliminary taxonomy about VEThe function of VE is to transform the information from human operator into the dynamical response of virtual objects, and to produce the virtual force rendering information to control the haptic device.Because virtual environment is coupled with haptic device, the component and information-mapping model of virtual environment is different according to different haptic device and different interaction task.There are two existing classification method for VE. The first method classified the VE into two types: impedance and admittance [10]. The second method divides VE into linear and nonlinear, time delay and no time delay, passive and non-passive [7]. It can be seen the above classification method cannot reflect the relationship between VE’s parameters and human’s input.In above classification method, the evolvement of the VE is not been considered. For example, in virtual tug-of-war system, the rival of the operator will possess itsown strategies. It will adjust its strategies real time according to the change of the human operator’s strategies and the property of the virtual object is evolving according to the operator’s movement. With consideration of such intelligence, the haptic interaction system will be more interactive and more interesting.According to the different type of the VE’s dynamic model and according to the different reason that lead to the change of the parameter, classification of virtual environments can be established. New taxonomy or classification of VE is proposed as figure 1 shown. In time invariant environment, the dynamic model and its parameters are constant; in time variant environment, the parameters are evolving according to the operator’smovement or force changes.Fig. 1 New classification of VEFrom figure 1 it can be seen that intelligent VE represents a large branch in VE. Therefore it will own great potential in haptic simulation.2.2 Model of intelligent VECompared with typical VE, mathematical model of intelligent VE can be derived. In typical impedance displayed type VE, the transform function is()()()s I s O s G = (1)where is Laplace transform of output signal, ()s O ()s I is Laplace transform of input signal.Considering the typical virtual wall, the system dynamical model is()⎩⎨⎧<=≥−=0000X X F X X X X K F h e h h e e (2) Therefore, the transformation function is()()()e he K s X s F s G =='(3) where .0'X X X h h −=It can be seen the virtual wall’s property is decided by the virtual stiffness e and the wall boundary 0, which are both constant and independent upon the input form human operator. The transform function is single input/output, linear and time invariant. When the freedom of I/O signal increases, or simulating task become complex, the influencing parameters of the VE will increaseaccordingly. As equation (4) gives, the transform function and the influencing parameter of the impedance displayed type VE can be expressed in a matrix formK X ()()()()[]()nj m i s X s X F P P g s X s G s F h h h pj g i h e ,,1;,,1,,,,66166616L L v v v vv ====×××× (4)where is geometric parameters of the VE, is physical parameters of the VE, is the number of geometric and physical parameters respectively. gi P pi P n m ,()⋅g is the dynamical model reflecting the dynamical behavior of the VE. h h X F vv , means the force and movement of human operator.Characteristics of intelligent VE can be concluded: z Parameters of signal-mapping model are evolvingwith the change of the operator’s input parameters (movement/ force or both).z The virtual environment measure human’s movementand force tendency, and produce real time response according to human’s operation.z VE’s response cannot be predicted according todefinite physics equation, however it can be produced randomly within a specified group. z The intelligent VE may not be passive.Therefore, in intelligent VE, the dynamical model and the geometrical and physical parameter will become nonlinear, time variant, and will evolve and depend on the change of the operator’s input. According to the different reason causing model/parameter change, the intelligent VE can be classified into following three types and their combination. Equation (5-7) means geometric parameters evolving type, geometric parameters evolving type and dynamical behavior evolving type respectively. In thefollowing equations, ()⋅frepresent the relationship function between the parameters and human operator’s input. ()t X F f P h h gi ,,1v v = (5) ()t X F f P h h pi ,,2v v = (6) ()()t X F f g h h ,,3v v =⋅ (7)Furthermore, addition to transparency and stability, interactivity is proposed as new performance evaluation-index to describe the characteristics of intelligent VE. It enhances the unknown property of VE, and the human operation dependent features.2.3 Problems in intelligent VE simulationThe problems of intelligent VE exist in following aspects: modeling, rendering, computation and evaluation of fidelity, stability and intelligent implementation of the system.Modeling method of the VE can be classified into polygon model, volume model and Nurbs etc [11]. The common of these methods is to describe the geometrical and physical properties of the virtual objects, to enable real time collision detection and force computation according tothe status of the haptic device. In intelligent VE, the geometrical and physical properties of virtual objects may be evolving, which proposes requirement for new modeling method.Rendering method can be classified into three kinds: constraint-based, penalty-based and pulse-based [11,12]. For intelligent virtual environment, the dynamic information of the human operator needs to be considered in the rendering algorithm. In this paper, the traditional constraint-based method is modified to achieve interdependence between virtual force and human operator’s input.The challenge for fidelity evaluation means that no consistent model is suitable for describing the fidelity. It is variable according to input of different human operator. Therefore, it should be evaluated according to experiment of specified operator.Stability problem occurs because system passivity is usually destroyed. It is a challenge to maintain stability under time evolving and active VE.Interactivity is utilized as new evaluation index. Some intelligent learning method, such as artificial neuro-network, may be used to mimic learning process of human operator. Intelligent hierarchy should be established for modeling the dynamic behavior in VE.3. Virtual tug-of-war system3.1 Concept of tug-of-warBased on component of intelligent VE, the virtual tug-of-war system is designed.The goal of the tug-of-war system is to simulate the tug process between real operator and virtual operator, and the system can be extended to tug process between two real human operators through network. Relationship among fidelity, stability and simulation algorithm in intelligent VE can be discussed on the designed platform of the tug-of-war system.As shown in figure 2, components of the tug-of-war system include: human operator, haptic device and computer. Critical task of the system can be decomposed into modeling of virtual objects, task planning of the tug-of-war process, force rendering, graphical rendering. Real time computation, including force computation, force rendering and graphical rendering etc, are all running onthe computer.Fig. 2 Prototype of a tug-of-war system3.2 Hardware platform designBecause the motion of the virtual rope and the output force of human operator are all one-dimensional variables, one degree of freedom haptic device is developed to simulate the tug-of-war system. Physical prototype is shown in figure 3.Power from DC torque motor is transmitted through conical gear and synchronous belt, then to drive the slider. Slider’s position is measured by optical encoder collocated with the motor. Human operator imposes force on the ring of the slider and force sensor inside the ring record the resistance force between human finger and the ring.Client-server architecture is utilized in the haptic system. The component of the control system is illustrated in figure 4. Because the PMAC cannot accept analogy signal, the signal of the force sensor is collected via an AD card [13]. Current amplifier is utilized to achieve high-resolution force control of the motor. The client computer runs on Windows2000 OS, and the softwaresystem is developed using Visual C++6.0 and OpenGL.Fig. 3 Physical prototype of the haptic deviceFig. 4 Component of the control system3.3 Software architectureIn order to enhance the extensibility of the system, the software is constructed using four-level hierarchical architecture illustrated in figure 5. Correspondingly, object-oriented class family combined with Microsoft Foundation Class (MFC) architecture is defined as figure 6 shows.Fig. 5 Four-level architecture of software system The task-planning module gives the motion status series of the tug process, and supplies the criteria for deciding which side is winning. The module also gives the method for handling exceptional cases. The force properties in the process and the performance index for evaluating the fidelity of the tug process are also be analyzed in this module.The force-rendering module is responsible for mapping the motion from device’s coordinate to the position of the rope and the virtual person in the scene and computing the virtual force. Furthermore, the rendering module ensures the consistency between the capability of the haptic device and the virtual human. Stability control method needs to be utilized and evaluation of the fidelity needs to be carried out.Graphical design is responsible for displaying realistic virtual human, including its pose’s change during the tug process, real time motion of the rope, and giving alarmwhen one side is going to lose the game.Fig. 6 Class definition of the software system3.4 Intelligent control strategyTwo rendering strategies are used. One is impedancedisplay, and the other is admittance display. Within each method, the level of intelligence is varied to compare the effect of intelligent VE on fidelity of the tug process. That is to say, parameters of the VE are varying according to the input force and movement of human operator.In impedance display, the position of the human/slider ismeasured and the force model is()()()()()()()()()⎪⎩⎪⎨⎧≤−=>−−⋅=δδk X k X k F k X k X k X k X K k F h e h h e e 0000(8)where()()()()()()()()()()()⎪⎪⎩⎪⎪⎨⎧≤−−−=>−−−−+−=ηη111110000000k X k X k X k X k X k X K k X k X k X k X h h m h(9) In above two equations, means the virtual force signal at current sampling period. ()k F e ()k X h means the slider’s position signal at current sampling period. ()()1,00−k X k X mean the virtual rope’s neutral position signal at current sampling or previous samplingperiod. δis the threshold to sense human’s input movement. η is the intelligent threshold to change the virtual rope’s neutral position. e is the virtual stiffness to map human’s motion to virtual force. m is the intelligent coefficient to compute the displacement of the virtual rope’s neutral position.K K In admittance display, the output force of the human on the slider is measured and the virtual force model isF K F F d h e Δ⋅+= (10)Where is the measured force signal from human’s hand. d is the game winner choice selection coefficient. h F K F Δ describe the force difference between the human operator and its virtual rival.. The level of intelligence can be varied through selecting different model of and d K F Δ. One possible model is⎪⎩⎪⎨⎧≤−>=0101hhd X X K && (11)[][]⎩⎨⎧−∉−∈=Δb b h b b h X X X X X X F ,10,2(12)where his the velocity of the slider in the haptic device. The positive direction means the slider is moving toward the human operator’s side instead of the virtual rival’s side. b is the displacement boundary to define the game victory criteria. When the slider moves through the boundary, the game will be stopped and one side will win.X &X Other choice of d and will cause different feeling of tug process. For example, K F Δ1=d K means the virtual rival is always stronger than human operator, and on the contrary 1−=d K means the virtual rival is always weaker than human operator.4. Experiment4.1 Goal and methodThere are two goals in experiments. The first goal is to evaluate the fidelity of the tug-of-war process. The second is to validate the stability under active VE.To evaluate the fidelity of the tug-of-war process, different human operators is required to feel the system and give scores on their feeling. To find the stable control method for simulating the tug-of-war process, three kinds of control strategy is utilized to compare the stability and fidelity difference.4.2 Experiment resultThe force data under different control modes and different intelligent level are given. Figure 7 and 8 is force signal under impedance control using equation (8) with the parameters in table 1. Through the force curve, it can be found that the parameters in the rendering model should be selected carefully in order to ensure stability of the intelligent tug process simulation. Only using the first parameter group can the stability be ensured. Figure 9 is force signal under admittance control using equation (10) when . Figure 10 and 11 is graphical scene of starting the game and winning the game.2,1=Δ=F K dTable 1 Parameters in impedance controlParameters 1 2 34 (mm N K e /) -10-10 -10 -100(mm N K m /)20 20 10 200 ()mm δ 0 3 0 3 ()mm η3 10 3 10Fig. 7 Parameter group 1 in table 2Fig. 8 Parameter group 3 in table 2Concerning about the fidelity of the tug process, 5 operators are required to feel the process without known the utilized strategy. It is found that most operators think admittance control strategy is more realistic than impedance control. In order to reflect the gaming characteristics between computer and human, intelligent admittance control strategy increases interactivity of the system. However, the admittance control strategy requires fairly good performance of force sensor, which will reduce reliability and increase cost of the system.Fig. 9 Force under simplified admittance controlFig. 10 Graphical scene of starting the gameFig. 11 Graphical scene of winning the game5. ConclusionIn this paper, intelligent VE is defined to reflect intelligent behavior during human-machine interaction. Challenges in modeling of intelligent VE compared with passive VE is identified. Modeling and rendering method in intelligent VE are discussed.Based on characteristic of intelligent VE, virtual tug-of-war system is developed to explore the design problems of intelligent VE. The fidelity requirements of the simulation are analyzed and subtasks are decomposed to ensure the system requirement.Based on a novel one-dimensional haptic device, prototype of virtual tug-of-war system is developed. Experiment results proved that intelligent VE is effective for modeling real-time tug-of-war between human and virtual rival within computer. The stability of the system can be ensured under arbitrary human movement. Different operators give positive evaluation on fidelity of the tug-of-war process simulation.In future work, more intelligent strategies will be induced into the virtual rival to increase the interest of the system. New application system will be discussed utilizing the concept of intelligent virtual environment.AcknowledgementThis research received support from the Science Foundation of China Post-doctoral Research and the National Natural Science Foundation of China under the grant No. 50575011. Their support is greatly appreciated. References[1]Mandayam A. Srinivasan and Cagatay Basdogan. Haptics in virtualenvironments: taxonomy, research status, and challenges. Comput. &Graphics, Vol.21, No. 4, pp. 393-404, 1997[2]K. Salisbury, F. Conti, and F. Barbagli, "Haptic rendering:Introductory concepts," in IEEE Computer Graphics and Applications, vol. 24, 2004, pp. 24--32.[3]Brian E. Miller, J. Edward Colgate, Randy A. Freeman, On the Roleof Dissipation in Haptic Systems, IEEE Transactions on Robotics,VOL. 20, NO. 4, August 2004, pp. 768-771[4]Arthur E. Kirkpatrick, Sarah A. Douglas, Application-basedEvaluation of Haptic Interfaces, Proceedings of the 10th Symp. OnHaptic Interfaces For Virtual Envir. & Teleoperator Systs.(HAPTICS02), 24-25 March 2002, pp: 32 – 39[5]Bloomfield A., Yu Deng, Wampler J., Rondot P., Harth D.,McManus M., Badler N., A taxonomy and comparison of hapticactions for disassembly tasks, Proceedings of IEEE Virtual Reality, 2003, 22-26 March 2003, pp: 225 – 231[6]Craig R. Carignan，Kevin R. Cleary，Closed-loop Force Controlfor Haptic Simulation of Virtual Environments, Haptics-e, Vol. 1, No. 2[7] B. Miller, J. Colgate, R. Freeman, Guaranteed Stability of HapticSystems With Nonlinear VE. IEEE Transaction on Robotics and Automation, Dec. 2000. pp: 712-719[8][9][10]R. J. Adams, B. Hannaford, “Stable Haptic Interaction with VirtualEnvironments”, IEEE Transcations on Robotics and Automation, VOL. 15, NO. 3, 1999[11] C. B. Zilles and J. K. Salisbury, “A constraint-based God-objectmethod for haptic display,” in Proc. IEEE/RSJ Int. Conf. Intell.Robots Syst., Pittsburgh, PA, 1995, pp. 146–151.[12]Brian Vincent Mirtich, Impulse-based Dynamic Simulation of RigidBody Systems, Dissertation for Doctor of Philosophy in Computer Science, University of California at Berkeley, 1997[13]User manual, PMAC – Programmable Multi-Axis Controller,Delta Tau Data systems Inc. 2003. 。