f3kdb dithering up to 16bit and DGDecodeNV

[killazys]

Not sure if this is the proper forum, but in my encoding testing I've been doing lately I'm seeing strange errors when encoding from x264-encoded 8bit sources to lower resolution x264 10bit files that are dithered up to 16bit by flash3kyuu, using DGDecodeNV.

This does not happen with raw BD sources.

Any ideas as to why? I'd like to take advantage of the speed increase that Nvidia's CUVID can offer me.

[Guest]

Open your script directly in VirtualDub. Do you see errors?

[killazys]

Opening the avs for the BD raw using both DGSource and AVCSource output correct video in VirtualDub.

Switching to .mkv container for 1080p x264-encoded 8bit source; I am now using FFVideoSource vs. DGSource, using stacked 16bit output with f3kdb.

Here's example DGSource output: http://i.imgur.com/73xK7.jpg

Here's example FFVideoSource output: http://i.imgur.com/cE5UT.jpg

When I encode to x264 and specify input depth as "16", FFVideoSource will output properly but DGSource leaves me with a video that appears exactly as the frame in the picture.

[Keiyakusha]

Can you explain more what is your goal? It is not clear whats wrong on your screenshots. Maybe post the _same_ frame before encoding, avisynth output and after encoding?
You feed stacked output to x264 and getting stacked result - this is what should be expected.
But encoding stacked output as 10bit is pointless... Also stacked output is always 8bit-double-height, it can't be 16bit. So if you have some visual corruption - maybe it is because you fooled x264 by setting input depth to 16.
Also make sure you use custom x264 build (not the one from x264nl) or something like avs4x264 in between of x264 and avisynth, otherwise it won't understand what you feed to him, cause both stacked and interleaved ways to output 16bit are non standard ugly hacks.

[killazys]

Hm. Here's the goal: better output quality.

I feed video to x264_64 10bit builds by JEEB via something similar to this:

avs2yuv -raw "in.avs" -o - | x264_64.exe --level 5.1 --input-depth 16 --input-res 1920x1080 --demuxer raw -o "out.mkv" -

And Keiyakusha, I'm saying that the result AFTER x264 has encoded, be it using the stacked or interleaved format, is the same stacked or interleaved format that you see in the screenshots for video that is output by DGSource from x264 8bit video. The resulting video from x264 is correct for BD raw sources using the same avisynth commands.

EDIT: http://i.imgur.com/0HslA.jpg vs. http://i.imgur.com/TIeuT.jpg

The first one obviously has a lot of problems, not limited to the horrible blocking, washing out of the colors, and strange "shadows" in the bottom frame. This is exactly how x264 output appears using the command line in the above code. The second one appears as a normal 1080p video, with no bottom frame.

[golagoda]

I hardly understand what you're trying to do but just remember Nvidia GPU's can't decode Hi10P h.264, if you're trying that at all with dgdecnv or anything that uses the GPU it won't work properly.

[killazys]

Er, I'm dithering up to 16bit first, feeding that result to x264 to encode to a 10bit output in order to achieve slightly better picture quality. And DGDecodeNV isn't decoding 10bit (theoretically) because I feed 8bit frames before f3kdb command occurs. It's passed to x264 via avs2yuv anyway.

Right now I'm doing some encodes, but I'll update soon.

[sneaker_ger]

It's not really clear what your problem is. In the DGSource screenshots you see these artifacts because the Nvidia decoder cannot decode Hi10p, there is nothing you can do about that except from switching to a decoder that actually does support Hi10p (like ffmpegsource). Are you 100% sure that your "BD raws" are not in fact 10 bit encodes?
If you think that DGSource does something wrong, upload a sample of your source.

[killazys]

Video
ID : 4113 (0x1011)
Menu ID : 1 (0x1)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Codec ID : 27
Duration : 51mn 34s
Bit rate mode : Variable
Bit rate : 38.2 Mbps
Maximum bit rate : 40.0 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.769
Stream size : 13.8 GiB (86%)


It's pretty offensive of you to assume that I'm not working with a real BD raw. I would rather not upload a 153MiB video for you to download, but I will say that the source looks exactly like the FFVideoSource image that I link to.

And that encodes using DGDecodeNV with f3kdb work with the BD.

EDIT: Figured out my problem. Issue wasn't with f3kdb at all.

[sneaker_ger]

It's pretty offensive of you to assume that I'm not working with a real BD raw. I would rather not upload a 153MiB video for you to download, but I will say that the source looks exactly like the FFVideoSource image that I link to.

I don't really care what your source is, just trying to help you. I see artifacts that point to Hi10p being decoded with an 8 bit decoder, so that would have to be my logical conclusion.

Figured out my problem. Issue wasn't with f3kdb at all.

It is custom to share your findings in case someone else stumbles upon the same problem.

[the_weirdo]

Hi killazys, what do you use to decode and take screenshots from outputs?

[killazys]

Apologies, especially to sneaker_ger. I misread your post. Indeed, the issue was that I was decoding a 10bit source x264 source that I had blindly assumed to be 8bit.

Of course, the BDs were 8bit anyway and I never thought to compare.

Luckily, DGDecodeNV will still frameserve 8bit video and allow f3kdb to dither up to 16bit later in the Avisynth command line.

the_weirdo, I load an avs file into VirtualDub and "copy source frame to clipboard."

[the_weirdo]

Actually, I wanted to know which decoder(s) you're using to decode encoded videos (which are 10-bit H.264) for those screenshots. So according to your latest post, you're using DGDecodeNV to decode an encoded video (the one using DGDecodeNV as source filter), right? Glad you sorted that out.

[sneaker_ger]

Apologies, especially to sneaker_ger. I misread your post. Indeed, the issue was that I was decoding a 10bit source x264 source that I had blindly assumed to be 8bit.

Of course, the BDs were 8bit anyway and I never thought to compare.

No problem, I think I misread your first post, too, because you clearly stated that you were using both BD raws and x264 encodes as sources, not just BD raws. I kinda overlooked that because I was generally confused about the essence of your question. But all that matters is that you were able to figure it out.

In general you should not assume that f3kdb (or any other filter for that matter) works better or worse depending on the source filter, because at the end, all it gets is uncompressed video from AviSynth, it does not have to know or care about the preceding (source) filters.

[killazys]

Right. And Avisynth goes line by line, yes?

[sneaker_ger]

Basically yes. For the most parts all those filters don't interact with each other, they are just fed uncompressed video and audio by AviSynth and in return output uncompressed video and audio back to AviSynth, which in turn passes the data on to the next filter. They don't care what happens before or after them.

[killazys]

Thanks for the clarification :)
SetMTMode has also been confusing to me; if the output looks fine, can I assume that whatever filter I'm using is compatible with the mode?

[sneaker_ger]

Yes, if the output looks fine it's OK. The bigger problem with these MT modes is that they are often unstable, so encodes start crashing at random times, something which you don't notice in short tests.

[killazys]

Hm.. my encodes usually run for around four hours each and I haven't experienced any crashing thus far. Lucky me, I guess ^^