Repository navigation
Make audio processors MediaItem aware #418
Description
My app support gapless and everything works fine.
But my app also support Replay Gain that it applies from extracted metadata. The gain is applied from onTracksChanged and this works nicely for the first song (And also when applying for albums since the replay gain is constant).
The problem arise when users want to play random songs with random replay gain values. In that case due to pre buffering for gapless, onTracksChanged is called too late and the replay gain is applied late (For medium quality mp3 up to 2 seconds). This obviously is not nice.
Since they play random tracks there's no need for gapless, and they prefer a small delay for the next song than a wrong volume for a few seconds.
The question is how can I achieve that (disable gapless, so that I can apply the replaygain at the proper moment) in normal usage of exoplayer and media sources and everything.
I'd like to avoid rewrite a complete different path to not use ExoPlayer playlists and handle everything manually.
Activity
tonihei commentedon May 25, 2023
Could you point out how exactly you apply the replay gain once you have the metadata? Is this a simple call to
setVolume()or something more advanced than that? And also, where exactly do you get the metadata from - is itFormat.metadata?The reason for these questions is that waiting for
onTracksChangedmay work in practise, but it is strictly speaking also too late because you likely miss a few audio samples at the beginning of the stream. I wonder if whatever needs to be done should be implemented as anAudioProcessoror as an adaptation/wrapper of an existing audio processor. They unfortunately don't see the fullFormatobject, but just a simplified view of it, but we may be able to make some adaptations to set the fullFormat(including the needed metadata). The advantage is not only that the first start should be better, but also all transitions (gapless/prebuffered or not) can apply the right effect very accurately to each audio sample.Tolriq commentedon May 25, 2023
Yes I get the RG values from the Format.metadata. (But I could also get them before if necessary).
And I use most of the time an AudioProcessor for that but there's no calls to configure or anything on media change if they are the same codec and everything. (And it's also an issue for example to enable / disable audio processor on demand, currently we can only bypass by doing a simple copy). The other way I use sometimes is to apply an Android DynamicsProcessing at higher level when I also apply some EQ stuff) but this is even less precise.
If there was a way with AudioProcessor then I would always use it but this is not the case currently.
For the first song or when pressing next (so a new prepare) this works perfectly the metadata is extracted before the codec is initialized and applied correctly before first audio sample or at least the flush works soon enough to ensure no wrong sample is played.
But with gapless we are not talking about 1 audio sample but could be up to 2 seconds that is already passed through the processor and the flush is wayy too late to recover. With flac or high res media this is a lot less but with mid quality MP3 this is 2 seconds.
Anyway if there's no way to disable the current gapless behavior, an audio processor solution would be nice.
For this need and some others (like enabling / disabling processor) having a way at the processor level to know when there's a media change would help a lot. (Including a way to disable the processor based on the media for a small performance gain and avoid an useless buffer copy).
Tolriq commentedon Aug 2, 2023
Bump ? Currently applying the RG via ffmpeg as a secondary solution but this is not efficient at all. A future native solution would be better.
Samrobbo commentedon Aug 3, 2023
AudioProcessoris the ideal place to handle this replay gain.It sounds to me like the problem is around the fact that
AudioProcessorinstances do not get theconfigurecall when the media item changes. This is an intentional design because avoiding configuring/flushing theDefaultAudioSinkmeans there are no audio discontinuities.@tonihei - You're more familiar with ExoPlayer components. Is there functionality in audio renderers to force the sink to be configured when the media item changes? I can't determine what the trigger is for MediaCodecAudioRenderer/DecoderAudioRenderer to configure the underlying sink is - It all seems based on codec reuse.
Once/if @tonihei confirms this, my recommendation would be for @Tolriq to pass in a custom
ForwardingAudioSink(ofDefaultAudioSink), and intercept theAudioSink#configurecall. Use theFormatpassed in to check for replay values in themetadata, and pass that information to a customAudioProcessor.DefaultAudioSink#configurewill then reconfigure and flush allAudioProcessorinstances, which will make active the new replay value.31 remaining items
Tolriq commentedon Apr 4, 2025
Is this somehow still in the radar and planned in a medium timeline ?
If not would be nice to know so I hack something for my use case.
Tolriq commentedon Jun 13, 2025
Small bump again, I got frequent requests to support server based replay gain values, and this is currently not possible.
Tolriq commentedon Feb 12, 2026
Any hope to have something in 1.10 ?
Forward Timeline+period uid to AudioSink and StreamMetadata
Set correct position offset in StreamMetadata in DefaultAudioSink
tonihei commentedon Mar 13, 2026
The two commits above should address this feature request. The
AudioSinkgets theTimelineandMediaPeriodIdfor full context on which item is playing, and it already had the stream timestamp offset for time-dependent processing. Both of this is now also forwarded to theStreamMetadataofAudioProcessor.flush.Tolriq commentedon Mar 13, 2026
Thanks a lot seems to work nicely.
Can you just clarify in what cases the MediaPeriodId will be null ?
tonihei commentedon Mar 16, 2026
If it's unknown. The issue that the values rely on forwarding of information through the
Rendererclasses, so there could be custom implementations not forwarding the value correctly, orDefaultAudioSinkcould be used in other context where this information is not available. If you are using the default renderers in a regular ExoPlayer playback, these values should always be available and never null.Tolriq commentedon Mar 16, 2026
Thanks, that sink and related audioprocessor are only used in the regular exoplayer case.