Design of an Immersive Teleconferencing Application

Design of an Immersive Teleconferencing Application
G. Y. Georgiannakis, P. E. Trahanias, C. N. Nikolaou and S. C. Orphanoudakis
Institute of Computer Science, FORTH and Department of Computer Science, University of Crete
1 Introduction
In the last few years, an outburst has taken place in research concerning multimedia systems. Multimedia systems o er the possibility to combine text, computer graphics, video and audio, as well as other forms of information in computer displays. This research has been extented to teleconferencing applications to support remote conferencing with systems that feature the above capabilities. Such systems o er substantial money and time savings, being an interesting substitute to actual conferences between remotely located parties. Teleconferencing is a broad term applicable to a wide spectrum of communication sessions, where two or more parties at di erent locations may collaborate

Abstract. This paper concerns the design of a system supporting tele-
conferencing applications. This system aims at creating an immersive environment by compositing a synthetic background with the reconstructed images of remote participants. It also presents natural scenes by employing viewer tracking and rendering images from the vantage point of the viewer. Processing requirements are met by exploiting existing networks of workstations by means of a quality of service based resource management scheme. This approach o ers advantages with respect to the application itself (teleconferencing) and resource requirements. By restricting the transmitted images to the participants only, the required bandwidth becomes much smaller and video quality can be achieved. The employed resource management scheme limits computational resources to general purpose workstations. On the other hand, a synthetic background that matches the real meeting room can be easily designed, giving the impression of a virtual meeting room as an extension of the real one. In this way, remote conference sessions that resemble real life meetings can be held.
2 Teleconference Applications
Recently there has been an increasing interest in media spaces as a means to support synchronous collaboration. These systems allow simultaneous, two-way transmission of auditory and visual information and, thus, simulate the every day media that people use to collaborate. The term media space actually indicates the analogy between these arti cial environments to the actual environments, where collaborating people communicate every day. There are two di erent approaches to create such systems. The rst approach is a computer centered approach, where collaboration is realized by means of shared window systems, and applications where images of collaborating partners are presented in a small window on the computer displays. The second approach creates shared workspaces based on audio and video communication. Such systems handle information that is external to the computers, such as voices and images of people. They aim at providing a sense of
by exchanging and/or manipulating information. A particular term, namely media space 4], aims at supporting the inherently social and technical collaborative work by existing technology means. In such a setting, groups of people may work together, although they may be separated by long distances, by exploiting visual and acoustic environments that resemble actual meetings. Unfortunately, the audio and video media that are used in teleconferencing convey only a limited subset of visual and auditory information when compared to actual face-to-face meetings. Therefore, media spaces can be currently regarded as an inferior substitute for actual meetings. In fact, in media space environments the perceived information is partial when compared to the rich information grasped in real life meetings. In this paper we are primarily concerned with a teleconferencing application, where audio and visual communication between persons at di erent locations is immersive. In other words, the whole setting creates a virtual environment where people involved may meet and discuss as if they were face-to-face. Such an environment exploits vision and stereo techniques, so that participating partners have the illusion of being next to the other without the need of special virtual reality equipment. This application exploits available computational resources within an organization, so that image processing techniques can isolate participating persons while rendering and compositing may incorporate participants in arti cially created backgrounds. Such an environment is particularly useful in meetings where body language, gestures and location of partners are essential. The rest of this text is structured as follows. In the next section we review teleconference applications that aim at creating a virtual representation of the actual meeting place and participants. Then we present the major issues that should be addressed by teleconferencing applications. The remainder of the paper is dedicated to the Immersive Teleconferencing Application (ITA). Initially, the major components of ITA and their interconnection are presented. Then, resource management issues are discussed and concluding remarks are made.