I’ll never forget the sheer wonder I felt when experiencing Bethesda’s latest entry in The Elder Scrolls series in 2011. I could wake up on a Saturday morning, inhale a bowl of cereal and boot up my PlayStation 3. I explored this new world of magic, dragons and endless opportunity, birthing in me an addiction to the fantasy genre in all its forms.

Time truly was an abstract concept when I held my controller. Hours passed in moments as I cleared dungeons and saved the citizens of every lucky town I stumbled upon.

Through it all, the classical, Nordic-inspired soundtrack plucked on the strings of my primitive monkey brain, enhancing tense moments and immersing me in the wild beauty of Skyrim’s landscape.

Even now my heart rate spikes when I hear the first note of the battle score. This is usually accompanied by a dragon’s ferocious roar or a woefully arrogant bandit telling me that I “never should’ve come here.”

I got a bit carried away there but Skyrim is my favourite game. Moreover, the point I was trying to make is this:

Audio is intrinsic to our gaming experience.

But, if audio is so important, why did I have to look up the name of the composer that wrote my favourite game’s soundtrack? On the one hand, I’ve not paid attention to the sound design before. On the other hand, I’m sure plenty other gamers are in the same boat.

In a year in which the games industry surpassed the size of both the film and music industries, I think that it’s only fair that we should start paying a bit more attention to how our beloved games are developed and appreciate just how much work it takes to make a good game.  

Brian Schmidt has been working with audio in games since arcade games and pinball machines. He has seen the growth of the gaming industry and witnessed the births and deaths of multiple generations of consoles. It almost goes without saying that Brian is as passionate as he is knowledgeable about sound design.

His long list of achievements includes designing of the original Xbox start-up theme and founding of GameSoundCon in 2009, the first convention exclusively dedicated to the gaming audio industry.

“Sound plays a lot of different roles in gaming.”

Brian said: “similar to films, there’s sound that replicates an environment and creates emotion. Then there’s the way sound rewards players, like playing a tune when completing objectives. And then there’s the level of detail that sometimes people don’t appreciate that is purely to create immersion.”

In addition, Brian has seen games become increasingly captivating and expansive as technology developed in leaps and bounds.

“I think of immersion as not being broken out of the spell,” he said.

“And the banes of immersion are things like loading screens and cinematics used to cover up load times. As the hardware has progressed, those loading screens have shortened and in some cases completely disappeared.”

I can most definitely vouch for that. The load times and frame rate of the remastered edition of Skyrim on PS4 are remarkably better than the original PS3 version. (Before you ask, yes, I bought Skyrim twice, I promise I really do have a social life.)

Critics compare most games through the scale of graphics. Games studios are often lauded for having ‘realistic’ visuals as if being realistic directly correlates to being an enjoyable game. Some games in fact make their wacky graphical style a staple of their franchise, such as Borderlands’ unique semi-cartoonish style.

Brian explained to me the reasoning behind the trend.

“There was a visual arms race where every generation of graphics processor was more and more realistic.”

In particular, he explained that as hardware progressed, graphics saw massive improvements that were easily identifiable to the average gamer. For the most part, you could tell which games were more polished and better-funded immediately by their graphics.

However, the capabilities of game audio are not as easily measurable. Thus sound was often overlooked.

“The one thing that always impresses me is the amount of detail that companies like Riot put into their sound design.”

“They’re very deliberate when creating sounds for their character’s spells. All their ice-based characters all have a series of sounds that are deliberately grown so that you can hear that the base energy for these attacks is the same, it all sonically fits.”

“Another great example is Mortal Kombat, where there’s an effort put into the sound design that blind gamers can compete at tournament level against sighted players just by tracking the audio.”

Accessibility

For blind players, good sound design is essential for a game to be playable and enjoyable.

TJ Vining, known as tjtheblindgamer, is the world’s first blind Master Prestige player on Call of Duty. Master Prestige is the maximum level possible in one of the world’s most recognisable gaming franchises.

“I was two years old when I started gaming, and I didn’t let going blind stop me at all.”

TJ has almost five thousand subscribers on his YouTube and mostly plays Call of Duty, Mortal Kombat and Diablo 3.

“It sounds weird in premise, right? A completely blind person playing video games is unexpected for most people, but because of accessibility, good audio design and dedication we can have fun and compete.”

TJ has recently focused on Call of Duty’s infamous Zombies game mode. Treyarch has particularly improved its’ sound design this year.

“I’ve played Zombies so much that I can guide sighted people around the maps, and my friend calls me Blind Deadpool because I count all my bullets.”

TJ has been working hard to make gaming more accessible for those with disadvantages for some time. In addition, he appealed for additional features from Treyarch in Call of Duty Blackout after a fellow player with one hand was struggling to use the controller properly.

After TJ lobbied the game developers, Treyarch added a patch to create an alternative button layout to help all players.

“It’s a sign of good accessibility when you make a feature for a disabled person but it benefits the majority of players,” he said.

The Evolution Championship Series, called EVO, is the world cup of fighting games. In 2019, over 200,000 people watched the tournament, with 9,000 competitors battling for a prize pool of $200,000.

TJ said OBS Rattlehead, a player who shocked the world in the Mortal Kombat EVO series in 2013, inspired him.

The blind gamer competed at an international level and consistently beat sighted players. Rattlehead didn’t reach the finals, but just standing with the best in the world was a massive achievement.

“In the comments of Rattlehead’s matches, people were saying that his opponents should just let him win and that really pissed me off,” TJ said.

“We don’t play competitive games just to be handed a win because we’re blind.

“We play to either get our asses kicked or kick someone else’s ass, that’s the whole point of gaming.”

TJ expressed to me how much gaming meant to him. It got him through some tough times dealing with a disability.

“At the end of the day, there’s still a missing link. I can’t sit here with a straight face and say that I can experience an entire game, because I miss out on all the cool visuals, but thanks to good audio design and accessibility I can still enjoy gaming the way I used to.”

The process of making game audio

The accessibility for the blind gaming community is all determined by a developer’s attention to detail in the audio design of their games. Michael Spicer is a lecturer at the University of Las Vegas and has been working in videogame audio for 2 years.

He has composed two albums of gaming soundtracks (available on Spotify), and also works in audio design.

Michael uses what are known as ‘Foley’ techniques to create sound effects in games. It’s a highly creative process and involves the use of various props to create the perfect audio for specific actions. Footsteps, rustling leaves and even attack animations.

“Foley is used to create low-key background noises that don’t necessarily stand-out, but they give scenes depth and realism. Hearing a bird chirping far away or chairs scraping on a tavern floor subconsciously improves immersion in games.”

“It’s all this stuff that you’d never pay attention to but if you don’t hear those sounds it becomes unnatural, like there’s a void,” Michael said.

Foley techniques are extensively used by NetherRealm Studios, the developers of Mortal Kombat. They create unique and detailed audio for their character’s attacks. Cracking walnuts to imitate breaking bones and pulping fruit to simulate internal organ damage.

Playing Mortal Kombat is a visceral experience for your ears, and combined with their infamously gory graphics, I wouldn’t recommend the fighter game for squeamish players.

Furthermore, The life of an audio designer can involve using bizarre props to create that ‘perfect’ sound and Michael is no stranger to experimenting.

One of his clients needed audio for their ‘slime’ character. If the term is new to you, think Slimer from Ghostbusters. These vague amorphous blobs of goop are an enemy-type popularised by Dragon Quest, Dungeons and Dragons and more recently, Minecraft.

“I took a metronome and a massive jug of vegetable oil into my walk-in closet. I sat there shuffling this oil back and forth trying to make a sound that was vaguely gloopy and gelatinous.”

“My wife and cats often give me strange looks as I carry around weird objects and repeatedly grunt into my microphone.”

Working on game audio presents lots of unique challenges. The highly imaginative nature of gaming means that audio designers have to create sounds for things that don’t have any basis in reality. Magic systems and fictional animals are a great example of this.

“With things like a frost spell you’re actually hearing broken glass being stepped on slowly with water mixed in the background, so it’s like this crunchy but also flowing type of sound,” Michael said.

“It’s almost like chemistry, you have a lot of experimentation that goes in to get the final product.”

“That’s why I like video games, they’re the cross-section of all art forms because they combine visuals, audio and interactivity,” he said.

The multitude of choices available to sound designers can shape a user’s experience, similarly to the use of audio in films and television.

Michael said: “if you add pizzicato strings to a horror movie it becomes a comedy with the same visuals. Damien becomes a cute little toddler and not the spawn of Satan.”

 “Audio controls people’s brains a little bit, you can really influence emotion with music.”

The impact of audio in gaming is difficult to measure emotionally, but a few studies have attempted such a feat.

In 2012, a study by Abertay University found that the inclusion of sound dramatically increased the range of gamers’ heart rates compared with those who played games without audio. Subsequently, this evidence of a physical reaction to sound in games reinforces the idea that audio enhances immersion and improves a gaming experience.

Despite the significance of audio, some believe that it is destined to remain in the shadow of graphics in gaming.

In this, I bow down to the experience of Brian Schmidt:

“When you’re doing sound for a game, the idea is to create a complete experience. For the most part, sound is in support of that experience, and if you notice the sound too much you could be taking away from the overall game.”

Brian said: “the right combination of music and visuals can be breathtaking, and that’s what we try and provide as sound designers.”

With the past year’s boom in the gaming industry, there will be more opportunities than ever for game developers to focus on their sound design, and hopefully more recognition of their work by the gaming community.