Robert Ciesla - Sound and Music for Games: The Basics of Digital Audio for Video Games

Sound and Musicfor GamesThe Basics of Digital Audiofor Video GamesRobert CieslaSound and Music for Games: The Basics of Digital Audio for Video Games Robert Ciesla HELSINKI, Finland ISBN-13 (pbk): 978-1-4842-8660-9 ISBN-13 (electronic): 978-1-4842-8661-6 https://doi.org/10.1007/978-1-4842-8661-6 Copyright 2022 by Robert Ciesla This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made.

The publisher makes no warranty, express or implied, with respect to the material contained herein. Managing Director, Apress Media LLC: Welmoed Spahr Acquisitions Editor: Spandana Chatterjee Development Editor: Spandana Chatterjee Coordinating Editor: Mark Powers Cover designed by eStudioCalamar Cover image designed by Freepik (www.freepik.com) Distributed to the book trade worldwide by Springer Science+Business Media New York, 1 New York Plaza, Suite 4600, New York, NY 10004-1562, USA. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@springer-sbm.com, or visit www.springeronline.com. Apress Media, LLC is a California LLC and the sole member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation. For information on translations, please e-mail booktranslations@springernature.com; for reprint, paperback, or audio rights, please e-mail bookpermissions@springernature.com.

Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook versions and licenses are also available for most titles. For more information, reference our Print and eBook Bulk Sales web page at http://www.apress.com/bulk-sales. Any source code or other supplementary material referenced by the author in this book is available to readers on GitHub (https://github.com/Apress). For more detailed information, please visit http://www.apress.com/source- code. Printed on acid-free paper Table of Contents About the Author xiii About the Technical Reviewer xv Acknowledgments xvii Introduction xix Chapter 1: Bits, Sample Rates, and Other Fundamentals of Digital Audio 1 On Audio Bit Depth 2 Pulse-Code Modulation (PCM)4 More on Quantization 4 On the Human Hearing Range 5 Fundamentals, Harmonics, and Overtones 5 The Audio Spectrum 6 Decibels and Normalization 7 Loudness Standards 8 Max Headroom 10 The Nyquist Theorem and Frequency 11 The Perils of Aliasing 11 Going Beyond 441 kHz 11 On-the-fly Oversampling 12 Delight in Dithering 12 A Few Words on Word Clocks 15 iii Table of ConTenTs Common Audio File Formats 15 What Exactly Is a Codec? 15 Lossy vs Lossless 16 WAV and AIFF, the Undefeated Duo of Mixing 17 The Mighty MP3 18 Xiphorg Presents: Ogg Vorbis, Ogg Opus, and FLAC 18 Advanced Audio Coding (AAC) 19 Apples ALAC 19 Constant vs Variable Bitrate 20 Beyond PCM: Direct Stream Digital (DSD) 21 The Pest Called Phase Cancellation 22 The Power of Transients 23 In Closing 24 Chapter 2: Current Generation Audio Hardware 25 Balanced vs Unbalanced Cables 25 XLR and 1/4" Cables 26 Dynamic and Condenser Microphones 27 The Fundamentals of Vocal Microphones 28 Microphone Polar Patterns 28 The All-in-One Podcast Microphones 30 Pops and Plosives 31 More Common Audio Cables 31 Digital Interfacing 33 A Few Thoughts on S/PDIF and AES3 34 Monitoring Hardware: Headphones 35 Closed vs Open Headphones 37 Active Noise Cancellation37 iv Table of ConTenTs Studio Monitor Speakers 37 Tweeters and Woofers 38 Nearfield vs Farfield Speakers 38 Speakers and Watts 40 Connectivity Protocols: The Sweet Flow of Data 40 USB 3 Specification Confusion 41 Firewire: Going, But Not Gone 42 Audio Interfaces and You 43 Operating System Support 44 Headphone Amplifiers 45 Computer Matters 46 RAM Affairs 47 CPUs and Cores 48 Core Multiplication 48 Those Important CPU Extensions 49 Storage Hilarity 50 Computer Shopping 52 Intel vs Apple M1 54 In Closing 54 Chapter 3: Manipulating the Spectrum57 Equalization (EQ) 57 Mastering 59 The Fast Fourier Transform (FFT) 59 EQ Filter Varieties 61 Implementing EQ 62 EQ and Color 63 More Filters: Butterworth and Chebyshev 64 v Table of ConTenTs EQ Limitations 65 The Three Universal Rules of Equalization 66 Dynamics: Compressors and Limiters 67 Attack and Release 68 Make-Up Gain 69 The Five Types of Compressors69 Serial and Parallel Compression 71 The Wizardry of Magnetic Tape 72 Tube Temptations 74 Power Classes 76 Software Saturation 77 Sound Effects Best Practices 79 In Closing 81 Chapter 4: More Effects and Digital Audio Workstations 83 More on Plugin Formats 83 Plugin Management 84 Reverb 101 85 Components in Reverb 87 Reverb Parameters 87 Send vs Insert 90 Best Practices: Reverb in Sound Effects 90 Best Practices: Reverb in Soundtracks 91 Modulation Effects: Chorus, Flanger, Phaser 93 Retro Sound FX Creation 94 A Quickie Reminder on MIDI 95 Taking Latency Seriously 96 vi Table of ConTenTs Audio Workstation Characteristics 98 Garageband by Apple 99 Recording Quality 100 Garageband Multitracking and Effects 100 Exporting Audio 101 Virtual Instruments and Samples 101 Virtual Amplification 102 Garageband: A Summary 103 Ardour by Paul Davis and Team104 Scripting in Ardour 104 Recording Quality 106 JACK: Virtual Cabling 107 Ardour Multitracking and Effects 107 Virtual Instruments and Samples 108 Exporting Audio 108 Ardour: A Summary 108 Logic Pro by Apple 110 Recording Quality 110 Logic Pro Multitracking and Effects 111 Virtual Instruments and Samples 112 Exporting Audio 112 Logic Pro: A Summary 113 FL Studio by Image Line Software 114 FL Studio Multitracking and Effects 115 Virtual Instruments and Samples 116 Exporting Audio 116 vii Table of ConTenTs FL Studio: A Summary 117 LMMS 117 LMMS Multitracking and Effects 119 Virtual Instruments and Samples 119 Exporting Audio 120 LMMS: A Summary 121 Waveform FREE by Tracktion 121 Virtual Instruments and Samples 122 Recording Quality 123 Waveform-Free Multitracking and Effects 123 Exporting Audio 123 Waveform Free: A Summary 124 In Closing 126 Chapter 5: Voice Acting: Hardware and Techniques 127 Elements of Voice Recording 127 Setting Up Your Microphone 128 Microphone Impedance 129 Preamps 130 Some Hardware Preamps 133 Dynamics Processors 133 Portable Phantom Power 135 Soundproofing 135 Voice Material Production Essentials 137 The Typical De-essing Procedure 138 Software for Voice Artists 140 Audacity 141 viii Table of ConTenTs Ocenaudio 142 WavePad 144 In Closing 146 Chapter 6: MIDI and Composing in the Digital Age 147 Introduction: Composing for Video Games 147 A Smidgeon of Music Theory 149 Bars and Time Signatures 151 On Notes and Rests 152 The Early Days of MIDI 153 Notes in MIDI 154 MIDI, Dynamics, and Tempo 155 Elaborating on MIDI Messages 156 MIDI Channels and Modes 159 Cello with Kazoo: Going Multitimbral 160 MIDI Specifications 160 General MIDI (GM) 161 Roland MT-32 162 Roland GS163 General MIDI 2 164 Yamaha XG 164 MIDI Connectivity 165 The ADSR Envelope 166 Meet the Waveforms 167 Additive and Subtractive Synths 169 Synths vs Controllers 169 Automation 171 ix Table of ConTenTs Quantization 173 The Power of Virtual Consoles 174 Getting Started with Symphonic Soundtracks 179 A Few Words on Samplers 180 Merrymaking with Articulations 181 Symphonic Soundtrack Best Practices 183 In Closing 185 Chapter 7: Introducing MIDI 20 187 MIDI 20 at a Glance 187 MIDI 20 in Detail 188 MIDI Capability Inquiry (MIDI-CI) 189 Universal MIDI Packet (UMP) 190 Jitter Reduction Timestamps (JRT) 191 MIDI Polyphonic Expression (MPE) 192 MIDI 20 and You 194 In Closing 196 Chapter 8: Chiptunes 197 Chiptunes: The Beginning 197 Traversing Through the Eras 199 The 8-Bit Era 200 The SID Sound 203 PAL vs NTSC 205 Frequency Modulation (FM) Synthesis 205 The 16-Bit Era 206 Chipmusic Today 210 On Music Trackers 210 Tracker File Formats 212 x Table of ConTenTs Module Components 213 Effect Enumeration 213 Milkytracker 214 ModPlug Trackers: OpenMPT and Schism Tracker 216 Sound Chip Emulation Trackers 218 FamiStudio (NES) 218 DefleMask (Multiple Systems) 221 GoatTracker (C64/SID) 222 SID Instrument Building Blocks 224 SID Instrument Examples 226 Back to GoatTracker 229 In Closing 230 Chapter 9: Audio Solutions for Video Games 231 Video Game Audio Approaches 231 Sound Occlusion and Obstruction 233 HRTF: 3D in Your Headphones 233 Virtual vs True Surround 234 Ambisonic Audio 234 Dynamic Music: Horizontal vs Vertical 236 On Real-Time Mixing 238 Immersive Audio Elements239 The Doppler Effect 240 Audio Raycasting and Hardware Acceleration 242 Application Programming Interfaces (APIs) 243 Middleware 244 Some Modern Audio Solutions 244 Miles Sound System by Robert Miles244 xi Table of ConTenTs FMOD by Firelight Technologies 245 Wwise by Audiokinetic 248 OpenAL/OpenAL Soft 250 History Lesson: EAX and OpenAL 251 irrKlang by Ambiera 252 In Closing 254 Chapter 10: Selling and Licensing 255 Copyright 255 Publisher 256 Music Supervisor 256 Licensor vs Licensee 256 Synchronization License 256 Master Use License 257 Public Domain Music 257 Royalty-Free Music 257 Performance Rights Organizations 258 International Standard Recording Code (ISRC) 258 Royalties 259 Diegetic vs Non-diegetic Music 260 Preparing Your Music260 Music Licensing Businesses 262 Exclusive vs Non-exclusive Agreements 263 In Closing 263 Repeats and Endings 264 Index 265 xii