Knowledge Transfer Project
Sound Design in serious games

Audio is essential in the experience of entertainment games. Sound effects and music influence emotion and behaviour, and immerse users in the environment and events. Sound design for serious games and virtual training applications faces a dilemma: can non-realistic sounds be used to enhance the experience?

In the development of serious games, the design of soundtracks receives little attention: most effort is directed towards creating realistic and flexible scenarios and high quality graphics. Sound design requires specialist knowledge, with which developers of serious games are not familiar. The goal of this project is to develop an accessible toolbox of methods and techniques for design, development and evaluation of soundtracks in serious games. The tools are focused on important user related aspects of the training: engagement of the trainee, realism and learning goals of the training, and the training context.

Focus on user experience and training goals
In the first phase of this project, two case studies are performed with training applications developed by VSTEP for safety personnel in hospitals and detention centers. Different classes of sounds in training environments are distinguished, such as events (for instance explosions), feedback on actions, ambient sounds that create awareness of the location, communication, and sounds to increase stress and arousal. Some sounds are important in every training level, but complex soundtracks may interfere with the learning process in introductory training. They may also confuse the communication with the instructor, which is an important element in this type of training. For assessing the trainees' knowledge in unexpected and new circumstances, sounds are used to create confusing and complex incidents. For training under stressful circumstances, screams of victims can be added, or non realistic sounds, such as a heart beat, breathing sounds, played at high volume. The training must however still be experienced as convincing. The effect of these additions on the emotional response and on learning will be assessed, and methods for design and testing are evaluated.

Tools and techniques for sound design
Known methods and techniques for requirements analysis, design, user testing and evaluation will be adjusted and extended. They will include specific characteristics of this modality, user response relevant for serious gaming, and characteristics of the context of the training. The project will add to the understanding of the effects of sounds in serious gaming and training, especially on the emotional response of the trainee. It will provide companies with a methodology to develop and evaluate soundtracks, which will improve the quality of the soundtracks and the training environment, and will improve the manageability of the development process.

Partners
Utrecht University
VSTEP BV

Contact
Joske Houtkamp, Utrecht University
J.M.Houtkamp(at)uu.nl