FFmpeg arnndn filter has bugs?

[VoodooFX]

Test files: https://we.tl/t-TrGUOgj5IN

Command:

ffmpeg.exe -i out-full.wav -filter_complex [0:a]arnndn=m=sh.rnnn[s0] -map [s0] out-full.sh.wav

Original:


1 case) Spectrograms of result look exactly same on i5-2500M and i7-13700K:


2 case) But for i7-8700K and Ryzen 5 5500 results look like this [both have similar look, but not same]:


Anyone knows what is going on? Or how to debug it?

I tried:

ffmpeg.exe -i out-full.wav -filter_complex [0:a]arnndn=m=sh.rnnn[s0] -map [s0] o_0.wav -cpuflags 0

For "1" case it didn't had influence, for "2" case it produced ~slightly different spectrograms.

Tried FFmpeg v5.1.2 vs v6.1.1, no influence on "1" case, for "2" case spectrograms ~slightly different.

[richardpl]

That second image looks wrong, -cpuflags allows to select some CPU SIMD instructions instead of using all available on CPU.
Perhaps some of SIMD code is buggy in ffmpeg on specific SIMD set for certain CPUs.
Probably AVX2 and higher (AVX512, etc) related. Dunno if float vector dsp is buggy or just FFT TX code.
If on linux you could run perf (on ffmpeg cli with debug symbols not stripped) or similar to inspect which functions get executed on which CPUs.

[VoodooFX]

Shouldn't "-cpuflags 0" disable all CPU instructions?

"i5-2500M" - doesn't have avx2 [it's oldest cpu from all tested], but i7-13700K has it and it has same output.

Originally Posted by richardpl

That second image looks wrong

I actually like case "1" behavior, it works well on normal audios (out-full.wav is muffled poor quality, probably vhs recording), when in case "2" it performs weak denoise and is very inconsistant.

Originally Posted by richardpl

If on linux you could run perf (on ffmpeg cli with debug symbols not stripped) or similar to inspect which functions get executed on which CPUs.

Could you post an exact command and what cli to run?

[VoodooFX]

Btw I've tested:

ffmpeg.exe -i out-full.wav -filter_complex [0:a]arnndn=m=sh.rnnn[s0] -map [s0] o_1.wav -cpuflags +mmx+sse+sse2

Similar results as with "-cpuflags 0".

[richardpl]

With that specific .wav file I get much better results with this one:

afftdn=nr=22:tn=1:tr=0:om=o

instead of

arnndn=m=sh.rnnn

If you have linux like OS: run: perf record <rest of ffmpeg command>

To get performance results of above run: perf report

The ffmpeg app and dynamic linked ffmpeg libs (libavfilter, libavutil and others) need to have debug symbols to have results meaningful.

[VoodooFX]

Not interested in afftdn. [I didn't noticed positive effects for Whisper]

Originally Posted by richardpl

The ffmpeg app and dynamic linked ffmpeg libs (libavfilter, libavutil and others) need to have debug symbols to have results meaningful.

I've no idea how to do it, I guess that's where my participation on this ends.

EDIT:
Later I'll check where ffmpeg bugs should be reported.

[VoodooFX]

Found the culprit of the bug -> Stereo.
With mono input on all CPUs spectrogram looks exactly as the "1" case.

Those RNN models are trained on 48000 mono wavs, first I assumed that audio needs to be pre-converted to that, but ffmpeg didn't complained on input with different Hz/channels and tests on i5-2500M showed no difference on results.

Btw, I don't see input Hz influence on the results with all CPUs.
Still a mystery left why stereo works on i5-2500M and i7-13700K, but doesn't work on i7-8700K and Ryzen 5 5500.

[richardpl]

Try with arnndn=m=sh.rnnn:threads=1

[VoodooFX]

Originally Posted by richardpl

Try with arnndn=m=sh.rnnn:threads=1

That fixes it too, on stereo input the results are same as with mono input.

[richardpl]

That is really silly bug.

[VoodooFX]

There is ticket -> https://trac.ffmpeg.org/ticket/10863#ticket