WAV vs MP3 Seek Performance

[kunle-e]

Is there some sort of overhead when seeking sounds that are MP3-backed vs WAV-backed because of MP3 decompression or something similar?

I have a feature where these events happen in an instant:

• while the engine is running
• the engine_start time is set to a startAtFrame, which means that any sounds with a start time equal to or after that frame will play.
• sounds get their seek position changed relative to their start position and the startAtFrame
• This is basically like a 'rewind' to a specific point (or fast forward) so that all sounds are heard where they should be as if we started from the beginning and came to that point (`startAtFrame'). The engine is never stopped when this happens.

What I noticed:

• I notice that the timing between a backing sound(MP3) and other sounds (WAV) is sometimes off when the 'rewind' is pressed. The backing sound is not seeking to exactly where it should be seeking, it's slightly off. This happens more on iOS/Core Audio.
• As things are playing along, this problems happens more consistently when hitting 'rewind' towards the end of the backing track when there is more seeking to do (assuming that seeking starts at the beginning).
• if starting from the beginning where there is no seeking involved (engine time is frame 0, backing sound start time is frame 0) there are no issues with timing between the backing sound and upcoming sounds.
• I swapped out the MP3 for a WAV file and the problem does not happen.

Does any of this make sense or perhaps I could be doing something wrong?
I remember back in the day of using PC DAWs, we always worked with uncompressed audio, so I'm wondering if this is just how it's going to be when trying to seek MP3s on the fly? I guess I could convert the MP3 to WAV on the mobile and use the WAV within the app operations?

[mackron]

The seeking situation with MP3 is an annoying one. There is no accelerated seeking with MP3 if you're not loading sounds with the DECODE flag. For MP3, whenever you seek it will be done in terms of a brute force seek-and-discard which is just always going to be slow. That also means that the seek will indeed take longer when you seek further towards the end of the file. If you were to use the DECODE flag in ma_sound_init*() it should become instant.

There is some support for configuring a seek table to accelerate this, but unfortunately it's not directly exposed by the ma_sound API. If you are willing to bypass the engine's built-in resource management, you could use a ma_decoder directly and configure the seekPointCount config option in ma_decoder_config. Once you have the ma_decoder object, you can then plug it into ma_sound_init_from_data_source(). Then when you seek, the seek points will be taken into account and will enable some accelerated seeking.

[kunle-e]

Thanks, this is good to know because it was driving me crazy!
I do use MA_SOUND_FLAG_DECODE everywhere I use ma_sound_init_from_file(), specifically these flags:

ma_sound_init_from_file(&g_engine, outputFile, MA_SOUND_FLAG_DECODE | MA_SOUND_FLAG_STREAM | MA_SOUND_FLAG_NO_SPATIALIZATION, &soundGroup, NULL, &g_sound_1);

I might want to look at that second option, but afraid that I can't do it now because of other pieces of the app that needs to be built. I think for now , I will convert the mp3s to wav and document this as something to revisit. It's just good to know that it is a thing.

[mackron]

The DECODE flag will be ignored when using with STREAM, so unfortunately in this case the DECODE option is not viable. I also forgot to mention that DECODE (without STREAM) will also use up a lot more memory.

I'll add a TODO to my list to look at exposing the seek point thing to ma_sound_config.