Technological advances cause real worlds and simulated worlds to merge into what is called augmented reality. Augmented reality ranges from real museums augmented with a virtual guide to largely virtual worlds in which only your own hand is real, and to fully virtual gaming environments. In these augmented worlds natural navigation and interaction is a real challenge due to limitations in multimodal control and feedback technology and due to limitations in the understanding and true-to-life modeling of physical, biological and psychological processes. 
     Traditional interfaces for navigation and interaction consist of keyboards, mouse and joysticks, steering wheels for control, and desktop displays and dashboards for feedback. These interfaces are generally not natural (even clumsy) and demand substantial cognitive resources, in particular for inexperienced users. In order to enable people to navigate and manipulate in a natural manner in augmented worlds and thereby free their cognitive resources (memory, attention) for higher-level tasks we have to move away from dashboards and joysticks. Interaction concepts are needed that comply to the user’s perceptual and psychomotor systems, for instance using body mounted interfaces, and gesture recognition that are easy to control and interpret, fast and intuitive. With egocentric body mounted interfaces the simulated and real worlds can be matched intuitively and control and interpretation become natural. Further, augmented worlds become more fun!
     The GATE theme “Interacting with the World” has been divided into three work packages:

---

WP 3.1 Animating Navigation and Manipulation

One of the most practical and important properties of a virtual environment is the possibility to interact with it by navigating through it or by picking up or pushing objects ... more

---

WP 3.2 Detecting, interpreting and affecting user behavior

In many applications, computer vision based analysis of motion, gestures, and facial expressions of groups of people is desired. Current techniques are still limited to a single person, and to fixed position in front of a camera. ... more

---

WP 3.3 Brain Connection Devices

The full title of this work package is: A Brain Connection Device for Education, Feedback, Gaming, Handsfree Interaction, Joy, Know-how, Learning and More… ... more