Changing Pitch of Sound Effects to Match Background Music

By Tracey 2 min read

Sky: Children of the Light

In the game Sky: Children of the Light, a lot of sound effects in the game will change their pitch to match the background music. This includes the game's musical instruments, and the "chirping" of the player — instrument noises that the player can emit as a form of communication.

This was done as a deliberate choice to reduce any unintended dissonance that the player might hear, as the audio designer elaborated on in an interview.

Which technical detail are you most proud of?

Ritsu: The tone of many sound effects are pitch shifted in real-time according to the music key. This avoided subconscious player discomfort caused by discord between music and sound effect.

[Source: FMOD interview on Sky: Children of the Light]

This can be automated.

This is an example of the instrument notes, for C major in Sky

And this was the range of instrument notes you can get in the game, from a diagram I made (not up-to-date but still accurate)

However, in Sky, I found that it was a little too automated — at times the key for sound effects changes very often and it kept changing the key of musical instruments, making for a frustrating playing experience.

Here is a video of mine illustrating how quickly it can change at times. It’s an older video but it still remains the same in the current game.

Most of the time it is relative ok and doesn’t change too quickly. But other players and I found that this renders it a bit useless to play any songs during those moments. And also it’s not very useful for choosing the key of your instruments, unless you happen to be able to use perfect pitch / rapidly compare notes against a reference. (I can name notes and even I found it a bit too fast, had to pause many times before making that video)

Journey

In the game Journey, sound effects also change as the key of the music changes, most distinctly also the "chirping" of the player — this time, it’s cello and flute noises that the player can emit as a form of communication. I think the chirps would pick from the 1st note, 2nd note, 5th note, 7th note of the key.

In this case, the key of the sound effect remained pretty consistent for a sub area, never rapidly switching between different keys. This made it feel a bit less jarring for me.

Here is an example of chirping in two different areas.

It stays consistent in A minor during at least most of this desert area:

And it stays consistent in D minor for this tower:

Summary

Probably it is most comfortable for the player if the key of sound effects stays the consistent for whole parts of a track, rather than changing to match a few chords that have notes outside the key.