An Audio Architecture Integrating Sound and Live Voice for Virtual

197 Pages·2002·2.91 MB·English

NAVAL POSTGRADUATE SCHOOL
Monterey, California
THESIS

AN AUDIO ARCHITECTURE INTEGRATING SOUND
AND LIVE VOICE FOR VIRTUAL ENVIRONMENTS

by

Eric M. Krebs

September 2002

Thesis Advisor: Russell D. Shilling
Co-Advisor: Rudolph P. Darken
This thesis done in cooperation with the MOVES Institute

Approved for public release; distribution is unlimited
THIS PAGE INTENTIONALLY LEFT BLANK
REPORT DOCUMENTATION PAGE
Form Approved OMB No. 0704-0188
Public reporting burden for this collection of information is estimated to average 1 hour per response, including
the time for reviewing instruction, searching existing data sources, gathering and maintaining the data needed, and
completing and reviewing the collection of information. Send comments regarding this burden estimate or any
other aspect of this collection of information, including suggestions for reducing this burden, to Washington
headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite
1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project
(0704-0188) Washington DC 20503.
1. AGENCY USE ONLY (Leave blank) 2. REPORT DATE 3. REPORT TYPE AND DATES COVERED
September 2002 Master’s Thesis
4. TITLE AND SUBTITLE: An Audio Architecture Integrating Sound and Live 5. FUNDING NUMBERS
Voice for Virtual Environments
6. AUTHOR(S) Eric M. Krebs
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING
Naval Postgraduate School ORGANIZATION REPORT
Monterey, CA 93943-5000 NUMBER
9. SPONSORING /MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSORING/MONITORING
N/A AGENCY REPORT NUMBER
11. SUPPLEMENTARY NOTES The views expressed in this thesis are those of the author and do not reflect the official
policy or position of the Department of Defense or the U.S. Government.
12a. DISTRIBUTION / AVAILABILITY STATEMENT 12b. DISTRIBUTION CODE
Approved for public release; distribution is unlimited
13. ABSTRACT (maximum 200 words)
The purpose behind this thesis was to design and implement audio system architecture, both in hardware and in
software, for use in virtual environments. The hardware and software design requirements were to provide the ability to add
sounds, environmental effects such as reverberation and occlusion, and live streaming voice to any virtual environment
employing this architecture.
Several free or open-source sound APIs were evaluated, and DirectSound3D was selected as the core component
of the audio architecture. Creative Labs Environmental Audio Extensions (EAX) was integrated into the architecture to
provide environmental effects such as reverberation, occlusion, obstruction, and exclusion.
Voice over IP (VoIP) technology was evaluated to provide live, streaming voice to any virtual environment.
DirectVoice was selected as the voice component of the architecture due to its integration with DirectSound3D. However,
extremely high latency considerations with DirectVoice, and any other VoIP application or software, required further research
into alternative live voice architectures for inclusion in virtual environments. Ausim3D’s GoldServe Audio Localizing Audio
Server System was evaluated and integrated into the hardware component of the audio architecture to provide an extremely
low-latency, live, streaming voice capability.

14. SUBJECT TERMS 15. NUMBER OF
Virtual Environments, Audio, Voice Over IP (VoIP), Sound, Spatialized Sound PAGES
197
16. PRICE CODE
17. SECURITY 18. SECURITY 19. SECURITY 20. LIMITATION
CLASSIFICATION OF CLASSIFICATION OF THIS CLASSIFICATION OF OF ABSTRACT
REPORT PAGE ABSTRACT
Unclassified Unclassified Unclassified UL
NSN 7540-01-280-5500 Standard Form 298 (Rev. 2-89)
Prescribed by ANSI Std. 239-18
i
THIS PAGE INTENTIONALLY LEFT BLANK
ii
This thesis done in cooperation with the MOVES Institute

Approved for public release; distribution is unlimited

AN AUDIO ARCHITECTURE INTEGRATING SOUND AND LIVE VOICE FOR
VIRTUAL ENVIRONMENTS

Eric M. Krebs
Commander, United States Naval Reserve
B.S., United States Naval Academy, 1985

Submitted in partial fulfillment of the
requirements for the degree of

MASTER OF SCIENCE IN MODELING, VIRTUAL ENVIRONMENTS AND
SIMULATION

from the

NAVAL POSTGRADUATE SCHOOL
September 2002

Author: Eric M. Krebs

Approved by: Russell D. Shilling
Thesis Advisor

Rudolph P. Darken
Co-Advisor

Rudolph P. Darken
Chair, MOVES Academic Committee
iii
THIS PAGE INTENTIONALLY LEFT BLANK
iv
ABSTRACT

The purpose behind this thesis was to design and implement audio system
architecture, both in hardware and in software, for use in virtual environments. The
hardware and software design requirements were aimed at implementing acoustical
models, such as reverberation and occlusion, and live audio streaming to any simulation
employing this architecture.
Several free or open-source sound APIs were evaluated, and DirectSound3D
was selected as the core component of the audio architecture. Creative Technology Ltd.
Environmental Audio Extensions (EAX 3.0) were integrated into the architecture to
provide environmental effects such as reverberation, occlusion, obstruction, and
exclusion.
Voice over IP (VoIP) technology was evaluated to provide live, streaming voice
to any virtual environment. DirectVoice was selected as the voice component of the
VoIP architecture due to its integration with DirectSound3D. However, extremely high
latency considerations with DirectVoice, and any other VoIP application or software,
required further research into alternative live voice architectures for inclusion in virtual
environments. Ausim3D’s GoldServe Audio System was evaluated and integrated into
the hardware component of the audio architecture to provide an extremely low-latency,
live, streaming voice capability.

v
THIS PAGE INTENTIONALLY LEFT BLANK

vi
TABLE OF CONTENTS

I. INTRODUCTION........................................................................................................1
A. SOUND IN VIRTUAL ENVIRONMENTS...................................................1
B. RESEARCH OBJECTIVE .............................................................................2
C. THESIS ORGANIZATION............................................................................4
II. BACKGROUND ..........................................................................................................7
A. SPATIAL HEARING AND SOUND .............................................................7
B. SOUND AND EMOTION .............................................................................10
1. Linking Performance with Optimum Stress or Arousal ................10
2. Linking Arousal with Audio .............................................................11
C. SOUND AND TRAINING ............................................................................12
1. Linking Audio with Performance.....................................................12
2. Linking Performance with Memory, Expertise and Training.......13
3. Selected Task Analyses ......................................................................15
4. Summary.............................................................................................17
D. VOICE OVER IP (VOIP) TECHNOLOGY ...............................................17
E. LIVE VOICE IN VIRTUAL ENVIRONMENTS.......................................22
F. ARCHITECTURAL ACOUSTICS..............................................................27
III. CURRENT ARCHITECTURE DESIGNS..............................................................29
A. INTRODUCTION..........................................................................................29
B. OPEN AUDIO LIBRARY (OPENAL) ........................................................29
C. DIRECTSOUND3D....................................................................................31
D. EAX 3.0........................................................................................................33
E. SOFTWARE API SUMMARY ....................................................................35
F. AUDIO RESOURCE MANAGEMENT......................................................35
G. AUSIM3D GOLDSERVER ..........................................................................37
H. OVERALL SYSTEM ARCHITECTURE...................................................39
IV. VOICE LATENCY ANALYSIS...............................................................................45
A. INTRODUCTION..........................................................................................45
B. EXPERIMENTAL DESIGN.........................................................................46
1. Apparatus ...........................................................................................46
2. Procedures ..........................................................................................46
C. RESULTS AND ANALYSIS ........................................................................47
D. SUMMARY ....................................................................................................49
V. SOFTWARE IMPLEMENTATION .......................................................................51
A. INTRODUCTION..........................................................................................51
B. GFAUDIO.......................................................................................................52
1. gfAudioGlobal ....................................................................................52
2. gfListener ............................................................................................53
3. gfSoundObject....................................................................................56
vii
4. gfAudioEnvironment .........................................................................61
5. gfAudioEnvironmentTransition .......................................................62
6. gfAudioEnvironmentManager .........................................................64
7. gfNetVoice...........................................................................................67
C. AUSERVERLIB.............................................................................................70
1. auSystem .............................................................................................70
2. auBase .................................................................................................70
3. auSource..............................................................................................71
4. auListener ...........................................................................................72
5. auSound ..............................................................................................75
6. auChannel...........................................................................................76
7. auNotify...............................................................................................77
8. auTools ................................................................................................78
9. Summary.............................................................................................78
VI. CONCLUSIONS AND RECOMMENDATIONS...................................................79
A. SUMMARY ....................................................................................................79
B. RECOMMENDATIONS...............................................................................79
C. FUTURE WORK...........................................................................................81
APPENDIX A. GFAUDIO DOCUMENTATION ..............................................................83
A. GFAUDIOENVIRONMENT CLASS REFERENCE ................................83
1. Public Types .......................................................................................83
2. Public Methods...................................................................................83
3. Public Attributes ................................................................................84
4. Detailed Description...........................................................................84
5. Member Enumeration Documentation............................................84
6. Constructor and Destructor Documentation...................................85
7. Member Function Documentation ...................................................85
8. Member Data Documentation ..........................................................86
B. GFAUDIOENVIRONMENTMANAGER CLASS REFERENCE ...........86
1. Public Methods...................................................................................86
2. Public Attributes ................................................................................87
3. Detailed Description...........................................................................87
4. Constructor and Destructor Documentation...................................87
5. Member Function Documentation ...................................................87
6. Member Data Documentation ..........................................................87
C. GFAUDIOENVIRONMENTTRANSITION CLASS REFERENCE.......88
1. Public Methods...................................................................................88
2. Public Attributes ................................................................................88
3. Detailed Description...........................................................................88
4. Constructor and Destructor Documentation...................................89
5. Member Function Documentation ...................................................89
6. Member Data Documentation ..........................................................90
D. GFAUDIONET CLASS REFERENCE.......................................................90
1. Public Methods...................................................................................90
2. Public Attributes ................................................................................90
viii