fmtconv: resize, bitdepth and colorspace conversions

[asarian]

Originally Posted by poisondeathray

Did you get an error message ?

No error messages; just the x264 process that reports a broken pipe on the first frame it processes:

encoded 1 frames, 12.82 fps, 238.61 kb/s
Error: fwrite() call failed when writing frame: 0, plane: 0, errno: 32
Output 9 frames in 2.06 seconds (4.38 fps)
Core freed but 24883200 bytes still allocated in framebuffers

I got an error messge complaining that didn't have nnedi3_resample.py , so I needed to download it and put it into site-packages , then it worked

I got nnedi3_resample.py in site-packages (and nnedi3_weights.bin in the plugins64 directory), as per usual.

use_fmtc=False by default if you look at muvsfunc

I saw that, yeah; but it was not caused the breakage.

[poisondeathray]

Originally Posted by asarian

No error messages; just the x264 process that reports a broken pipe on the first frame it processes:

encoded 1 frames, 12.82 fps, 238.61 kb/s
Error: fwrite() call failed when writing frame: 0, plane: 0, errno: 32
Output 9 frames in 2.06 seconds (4.38 fps)
Core freed but 24883200 bytes still allocated in framebuffers

1) Did you convert it to the final pixel format in the script? eg. YUV420P8 if encoding 8bit 4:2:0 etc... YUV420P10 if 10bit 4:2:0 . Or whatever you're doing

2) Are you using correct matching vspipe with your vapoursynth version ?

Post commandline if neither of those is the cause

[asarian]

Originally Posted by poisondeathray

1) Did you convert it to the final pixel format in the script? eg. YUV420P8 if encoding 8bit 4:2:0 etc... YUV420P10 if 10bit 4:2:0 . Or whatever you're doing

I'm just trying to downsize it to a regular Blu-ray format, 8-bit, compressed with x264.

I use dgdecodenv, which, if I understood it correctly, will output the UHD input as 8-bit YV12 by default.

2) Are you using correct matching vspipe with your vapoursynth version ?

Yes, matching vspipe is being used. The whole new setup works flawlessly (already done 2 discs with it), but it's just this SSIM_downsample that won't budge for me.

[poisondeathray]

You have to convert it back to the final pixel format. That filter converts up to float

[asarian]

Originally Posted by poisondeathray

You have to convert it back to the final pixel format. That filter converts up to float

Ah, that would explain it. Thx!

[asarian]

Originally Posted by poisondeathray

1) Did you convert it to the final pixel format in the script? eg. YUV420P8 if encoding 8bit 4:2:0 etc... YUV420P10 if 10bit 4:2:0 . Or whatever you're doing

Bloody h*ell! Looks like I got the colorspace messed up, after all: flaccid colors

The bottom image is the result of extracting the HEVC stream with TsMuxer (as eac3to can't extract to .mkv for HEVC yet), so prior to me re-encoding it. The colors are all flaccid, and not vibrant at all.

Is there any way I could change the header of the encoded file somehow, and set a different colorspace? Or am I too late already?

And perhaps more importantly, as TsMuxer apparently screws up the stream, and eac3to can't do it, what is the best way to extract the HEVC stream properly? (to feed to the VS input filters).

[asarian]

^^ Ok, I did some comparison:

stream comparison

The pane on the right is from the resultant tsMuxer stream, the one of the right of the original Blu-ray m2ts. Far as I can see, they're identical! No bit-depth is lost, no colorspace conversion took place, etc. Yet playing/encoding the stream muxed by tsMuxer yields all these flaccid colors.

[asarian]

Hmm, after the initial shock, I think the UHD 'Oblivion' disc is simply encoded with flatter colors. (Which is possible, of course) I can't see another reason. I got 2 other UHD discs so far (I intend to make Pan @ Scan HD versions, from UHD, for some of my fave movies), and they look about the same, color-wise (demuxed) as their original. So, no weirdness was going on, after all. Maybe I should get my money back on the Oblivion UHD. (Or make the colors a bit more vibrant myself).

[poisondeathray]

Originally Posted by asarian

Hmm, after the initial shock, I think the UHD 'Oblivion' disc is simply encoded with flatter colors. (Which is possible, of course) I can't see another reason. I got 2 other UHD discs so far (I intend to make Pan @ Scan HD versions, from UHD, for some of my fave movies), and they look about the same, color-wise (demuxed) as their original. So, no weirdness was going on, after all. Maybe I should get my money back on the Oblivion UHD. (Or make the colors a bit more vibrant myself).

You probably have to tonemap it . It's HDR10 release, correct?

https://www.blu-ray.com/movies/Oblivion-4K-Blu-ray/156371/


	
	

			
				HDR: HDR10
Universal's 2160p, HDR-enabled presentation...
			
		

One option is DGHDRtoSDR , another is vapoursynth tonemap

[videoh]

Originally Posted by poisondeathray

One option is DGHDRtoSDR

You may like this one as it is GPU accelerated. Apply it after resizing on the GPU (if needed) for maximum performance.

[MonoS]

I do tonemap that way

den = core.resize.Bicubic(clip=den, format=vs.RGBS, filter_param_a=0.0, filter_param_b=0.75, range_in_s="limited", matrix_in_s="2020ncl", primaries_in_s="2020", primaries_s="2020", transfer_in_s="st2084", transfer_s="linear",dither_type="none", nominal_luminance=mcll)
den = core.tonemap.Mobius(clip=den, exposure=exposure, transition=0.3, peak=mcll)
den = core.fmtc.resample(den, resw, resh, kernel="bilinear", invks=True, invkstaps=3, taps=4)
den = core.resize.Bicubic(clip=den, format=vs.YUV444PS, filter_param_a=0.0, filter_param_b=0.75, matrix_s="709", primaries_in_s="2020", primaries_s="709", transfer_in_s="linear", transfer_s="709")
res = core.fmtc.resample(den, resw, resh, invks=True, kernel="bicubic", invkstaps=3, css="420").fmtc.bitdepth(bits=8, dmode=3, flt=False)

the parameters you need to pass are resw and resh for the final resolution, mcll for the mcll parameter of your stream, you can get it with mediainfo, and exposure you need to set as you like it, i suggest you to compare with the SDR bluray, as the tonemap filter i use this one https://github.com/ifb/vapoursynth-tonemap

[asarian]

Originally Posted by videoh

You may like this one as it is GPU accelerated. Apply it after resizing on the GPU (if needed) for maximum performance.

This looks like just what the doctor ordered! Unfortunately, the pertinent thread points to an invalid rar: https://forum.doom9.org/showthread.php?t=175624


EDIT: Nevermind: found a newer version in the directory itself.

[videoh]

Latest version is always linked on my web page:

http://rationalqm.us/mine.html

[ChaosKing]

@videoh could you make a vapoursynth section on your webpage too? Or mark filters with native vapoursynth support somehow?

[videoh]

OK, will do. Thanks for the suggestion.

[asarian]

Originally Posted by videoh

OK, will do. Thanks for the suggestion.

And, while we're at it, could you please also make DGDenoise native to vapoursynth? (so that DGDecodeNV.dll doesn't need to be loaded as legacy avs filter any more for it, with LoadPlugin).

[videoh]

Originally Posted by asarian

And, while we're at it, could you please also make DGDenoise native to vapoursynth? (so that DGDecodeNV.dll doesn't need to be loaded as legacy avs filter any more for it, with LoadPlugin).

The problem for me is that implementing both native Avisynth and native Vapoursynth versions creates a lot of code duplication and extra maintenance. I have not found a solution to this. Myrsloik has stated, and testing bears out, that there is no performance penalty for using the avscompat mechanism. So unless a compelling argument is advanced, I'll prefer to just make one version. If Avisynth could import Vapoursynth filters then I would reverse things.

I had already implemented native Vapoursynth for DGSource() before I settled on this policy.

[asarian]

Originally Posted by videoh

The problem for me is that implementing both native Avisynth and native Vapoursynth versions creates a lot of code duplication and extra maintenance. I have not found a solution to this. Myrsloik has stated, and testing bears out, that there is no performance penalty for using the avscompat mechanism. So unless a compelling argument is advanced, I'll prefer to just make one version. If Avisynth could import Vapoursynth filters then I would reverse things.

I had already implemented native Vapoursynth for DGSource() before I settled on this policy.

Okay, thx. That sounds entirely reasonable. I tried it out last night on a re-encode, and was worried that maybe the avs route would be less stable somehow, but turned out to be rocksolid after all.

[asarian]

^^ P.S. I'm continually amazed at how fast the GPU stuff is. Naturally, my GTX 1080 Ti is a beast, but especially the GPU load (measured by GPU-Z) is extremely low: along with OpenCL support for x264, the gpu-assisted resize, and DGDenoise, the entire GPU load only dips into a few percent, every now then. nVidea, ftw!

[videoh]

Originally Posted by asarian

^^ P.S. I'm continually amazed at how fast the GPU stuff is.

Yes, it clearly outperforms CPU implementations and if you can eliminate unnecessary CPU<->GPU transfers (think CUDASynth) it blows CPU away into the next galaxy.

Naturally, my GTX 1080 Ti is a beast, but especially the GPU load (measured by GPU-Z) is extremely low: along with OpenCL support for x264, the gpu-assisted resize, and DGDenoise, the entire GPU load only dips into a few percent, every now then. nVidea, ftw!

Who can argue with that? I like to remind people to look at the CPU utilization too, because savings can be applied to CPU things, especially encoding.