Is there any way to "trick" aomenc, or preferably aom-av1-psy, into utilizing grain synthesis by analyzing a grainy, untouched source but actually encoding video that was de-grained or de-noised
externally?
I use
--enable-dnl-denoising=0 with aom-av1-psy, per BlueSwordM and many others' suggestions, because the encoder's own denoising is a bit much, and having only 1 parameter to tune for it isn't enough. It leaves a lot to be desired.
Something like SMDegrain with a KNLMeansCL pre-filter pass, and appropriate parameters for both, can output a de-grained/DNR video that is vastly superior to what sometimes seems like either completely automated or excessive "professionally done" smeary and waxy DNR present in finished Blu-Rays. Adding some dither noise back in after working in 16-bit color and optionally de-banding if needed can help prevent new banding, with any number of decent AVS/VapourSynth debanding filters.
If I take video processed in
this manner and pass it on through to aomenc with
--enable-dnl-denoising=0, the grain synthesis function seems like it only acts on that dither noise, if it even acts at all. It doesn't know about the grain that I removed, because...well, how could it? It never saw the grain. If I try to crank
--denoise-noise-level up, it starts to look like x265 does at slower presets with
--psy-rdoq turned up too high: ugly artificial noise.
Does the encoder make the grain synthesis "decisions" - or at least the pattern and strength, exclusively during the 1st pass? If so, would it be feasible to conduct the first pass pointed at an AVISynth/VapourSynth script that has all the de-band/de-grain processing commented out, and then pull the old switcheroo and actually encode the 2nd pass pointing at a fully pre-processed source?
If that could work, would actually using
--enable-dnl-denoising=1 during this jape of a first pass help it make better rate control decisions?
Can --enable-dnl-denoising even be set differently on pass 1 and pass 2? I wouldn't want to actually encode the 2nd pass with it set to anything but
=0, especially if I'm doing my own external filtering. If it can't be switched, forcing SMDegrain or KNLMeansCL to run with lower spatial filtering might leave a "static" grain "texture", albeit likely pretty blotchy, in the source that it could use?
I hope that makes sense. I also thought of an alternative,
ridiculous strategy, but it involves:
- Fully encoding the video twice, once with no external filtering and --enable-dnl-denoising=1, once with the external filtering and --enable-dnl-denoising=0, both with the same --denoise-noise-level,
- Using a likely non-existent tool to check the AV1 bitstream of both,
- Hack the frame level grain synthesis data from the former encode into the latter.
So uh, unlikely, if not impossible. And also excessively time consuming.
I tried searching for an answer, and all I could find was excitement about using photon grain synthesis. That doesn't seem to help with the idea of encoding externally filtered video but trying to use grain synthesis to mimic the original grain pattern and strength, unless I just can't make the connection there.