Home

Design of a User Interface for Multimodal Navigation in Virtual 3D Environments

 

Rant to the author:

    

Check for a response

In winter '01/'02 I finished my diploma thesis on navigation in 3D environments. The greatest support I received came from Frank Althoff and all the coworkers at former blaxxun interactive where I could integrate some of my concepts in the VRML browser blaxxun Contact (now developed as BS Contact VRML by Bitmanagement Software GmbH).

The thesis finds a way of increasing the efficiency of human computer interaction in the context of VR environments. In the first part it provides two extensions to the VRML standard for reading user input and for implementing custom navigation modes. In the second part it develops a frame work for VR applications that allow a user to select among various haptic and semantically higher level input devices according to their personal preferences and to the application needs.

The thesis is written in German language, but the paper I've written together with Frank Althoff is in English and it covers parts of it in a very dense way.

After four introductory chapters that motivate the work (chapter 1 and 2), narrow the topic to 3D applications (chapter 3) and analyze the VRML technology (chapter 4) my diploma thesis covers four main chapters:

  • Chapter 5

  • develops some mathical formalisms that allow to describe and talk about navigation.

  • Chapter 6

  • discusses an extension to the VRML standard that allows to represent arbitrary input devices in a VR application.

  • Chapter 7

  • develops an extension to the VRML standard that allows a VR application to precisely control the navigation of the user in 3D space.

  • Chapter 8

  • develops a frame work and a communication formalism that allow a VR application to provide the user multiple semantic and haptic input devices for performing navigation tasks according to their personal preferences and the application needs.

The closing chapter 9 describes some further work that would complete the frame work.

Both extensions to the VRML standard have been implemented in the VRML player BS Contact VRML (www.bitmanagement.de).

 

 

Read the thesis (4.2 MB, PDF, German):

PDF

Design einer Bedienschnittstelle für die multimodale Navigation in virtuellen 3D Welten (4.08 M Byte)

Download as ZIP (3.80 M Byte) (should the above link not work)

 

 

The table of content, translated to English:


1 Introduction


2 User Interfaces

2.1 Available Modalities

2.2 Developement of User Interfaces

2.3 Requirements on 3D User Interfaces


3 VR Applications

3.1 Examples

3.2 Applikation Structure

3.3 Interaktion Paradigms
3.3.1 Navigation
3.3.2 Manipulation
3.3.3 Communikation

3.4 Cathegorizing Input Devices

3.5 Multimodal Interface for Virtual Scenarios, MIVIS


4 The VRML Technology

4.1 Mindset for the Language

4.2 Execution Model

4.3 Concept of User Interaction
4.3.1 Manipulation
4.3.2 Navigation


5 Customizble Navigation

5.1 Underlying Formalisms
5.1.1 Types of Movement
5.1.2 Coordinate Systems
5.1.3 Direction Systems

5.2 Movements
5.2.1 Representing
5.2.2 Filtering

5.3 Possibilities of Existing VRML
5.3.1 A Typical Browser
5.3.2 Customizable Navigation

5.4 Resolving the Problem
5.4.1 Objectives
5.4.2 Conzept


6 Representing Input Devices

6.1 Requirements

6.2 Overview

6.3 Node Definition

6.4 Explanations
6.4.1 Aktivating the Device
6.4.2 Standardisation of Devices

6.5 Discussion
6.5.1 Input Focus Multitasking Systems
6.5.2 Flexibility through Usage of Proto Mechanism
6.5.3 Methods for Accessing the Device
6.5.4 DeviceSensor as bindable Node

6.6 Typical Devices and their Implementation
6.6.1 General Implementation Details
6.6.2 Space Mouse
6.6.3 Joy Stick
6.6.4 Mouse and Keyboard
6.6.5 TCP Connections


7 Controlling Navigation

7.1 Requirements on Nodes for Navigation
7.1.1 Support for Velocity based Devices
7.1.2 Support for Position based Devices
7.1.3 Support for Referring Navigation Styles
7.1.4 Support for Diskrete Navigation Styles
7.1.5 Controlling Basic Navigation Parameters

7.2 Node Definition
7.2.1 The NavigationInfo2 Node
7.2.2 The NavigationSensor Node
7.2.3 The Navigator Node
7.2.4 Cancelation of Viewpoint Animations

7.3 Combining Movements
7.3.1 Represented as an Information Flow Diagram
7.3.2 Represented as Pseudo Code


8 Multimodal Interaction

8.1 Existing Software
8.1.1 A Formalism for Navigation Commands
8.1.2 The Structure of the Original MIVIS System

8.2 Design Decisions
8.2.1 A Communication Channel for Time Continuous Signals
8.2.2 Navigation Modes
8.2.3 The Haptic Interpreter
8.2.4 The Continuous Integrator
8.2.5 Feed Back to the User

8.3 System Architektur
8.3.1 System Overview
8.3.2 Input Modules
8.3.3 Kommunikation Channels
8.3.4 Diskrete Integrator
8.3.5 Navigator
8.3.6 Kontinuous Integrator
8.3.7 Feed Back Modul

8.4 Extended Functionality
8.4.1 Quasi Continuous Navigation
8.4.2 Referenzing Navigation
8.4.3 Control Comands
8.4.4 Formalism for Status Indication

8.5 Implementation
8.5.1 Usage of VRML/VrmlScript
8.5.2 Communikation Channels
8.5.3 Diskrete Integrator
8.5.4 Continuous Integrator
8.5.5 Navigator
8.5.6 Haptic Interpreters


9 Continuative Work

9.1 Examples of Use

9.2 Extending the System Architecture
9.2.1 Undoing Movements from Haptic Devices
9.2.2 Continuous Pointing Gestures

9.3 Extending the Functionality
9.3.1 Application Access
9.3.2 Referencing Moving Objects
9.3.3 Multimodal and Threedimensional Realisation of the Context Menu

9.4 Extending to the Manipulation Paradigm
9.4.1 System Architecture
9.4.2 Funktionality
9.4.3 Connecting the Application


10 Summary


Indexes
Figures
References
Internet Pages


Appendix A Conventions

Appendix B Example Scenarios for Customizable Navigation

Appendix C Extended Communikation Formalism

Appendix D DeviceSensor SDK for blaxxun Contact

 

__.-.__
end of document