Audiovisual 3d rendering as a tool for multimodal interfaces - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2008

Audiovisual 3d rendering as a tool for multimodal interfaces

Résumé

In this talk, we will start with a short overview of 3D audiovisual rendering and its applicability to multimodal interfaces. In recent years, we have seen the generalization of 3D applications, ranging from computer games, which involve a high level of realism, to applications such as SecondLife, in which the visual and auditory quality of the 3D environment leaves much to be desired. In our introduction will attempt to examine the relationship between the audiovisual rendering of the environment and the interface. We will then review some of the audio-visual rendering algorithms we have developed in the last few years. We will discuss four main challenges we have addressed. The first is the development of realistic illumination and shadow algorithms which contribute greatly to the realism of 3D scenes, but could also be important for interfaces. The second involves the application of these illumination algorithms to augmented reality settings. The third concerns the development of perceptually-based techniques, and in particular using audio-visual cross-modal perception. The fourth challenge has been the development of approximate but "plausible", interactive solutions to more advanced rendering effects, both for graphics and audio. On the audio side, our review will include the introduction of clustering, masking and perceptual rendering for 3D spatialized audio and our recently developed solution for the treatment of contact sounds. On the graphics side, our discussion will include a quick overview of our illumination and shadow work, its application to augmented reality, our work on interactive rendering approximations and perceptually driven algorithms. For all these techniques we will discuss their relevance to multimodal interfaces, including our experience in a urban design case-study and attempt to relate them to recent interface research. We will close with a broad reflection on the potential for closer collaboration between 3D audiovisual rendering and multimodal interfaces.
Fichier non déposé

Dates et versions

inria-00606818 , version 1 (07-07-2011)

Identifiants

Citer

George Drettakis. Audiovisual 3d rendering as a tool for multimodal interfaces. 10th international conference on Multimodal interfaces (ICMI'08), Jan 2008, Chania, Greece. ⟨10.1145/1452392.1452394⟩. ⟨inria-00606818⟩

Collections

INRIA INRIA2
47 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More