fmtconv: resize, bitdepth and colorspace conversions

[adsun701]

Hi there,
When I was testing if fmtconv could make transforms from 709 content to ciexyz content, I may have encountered a possible bug.
I either get a segmentation fault or completely black output when converting 709 primaries to ciexyz primaries.

import vapoursynth as vs
core = vs.get_core()
core.std.LoadPlugin(path=r'/usr/local/lib/vapoursynth/libimwri.so')
clip = core.ffms2.Source(r'/home/andy/Sample.mkv')
clip = core.std.Trim(clip, 2300, 2300)
clip = core.fmtc.bitdepth(clip, bits=32, dmode=6)
clip = core.fmtc.matrix(clip, mats='709', matd='RGB', col_fam=vs.RGB, bits=32)
clip = core.fmtc.transfer(clip, transs='709', transd='linear', bits=32)
clip = core.fmtc.primaries(clip, prims='709', primd='ciexyz')
clip = core.fmtc.transfer(clip, transs='linear', transd='2084', cont=1000/10000, bits=32)
clip = core.imwrif.Write(clip, 'EXR', 'image%04d.exr', firstnum=0)
clip.set_output()

Is this a bug? If so, I would appreciate that it be fixed. If not, then what am I doing wrong?

[m3sh]

Any plans to add HLG transfer function support in an upcoming release? By my reading of the documentation, I can't see anything referencing ARIB-STD-B67 (Hybrid Log Gamma, HLG) EOTF.

Would love to have it, Technicolor guys gave me an asset in HLG and I'm hoping to use fmtconv to do the conversion to HDR10 to feed into my currently limited STB client - the broadcom chipset doesn't support HLG afaik.

[Cary Knoop]

Originally Posted by m3sh

Any plans to add HLG transfer function support in an upcoming release? By my reading of the documentation, I can't see anything referencing ARIB-STD-B67 (Hybrid Log Gamma, HLG) EOTF.

Would love to have it, Technicolor guys gave me an asset in HLG and I'm hoping to use fmtconv to do the conversion to HDR10 to feed into my currently limited STB client - the broadcom chipset doesn't support HLG afaik.

I wonder if this, as a workaround for now, can be achieved by using Lut2 function mode.

[Myrsloik]

Originally Posted by m3sh

Any plans to add HLG transfer function support in an upcoming release? By my reading of the documentation, I can't see anything referencing ARIB-STD-B67 (Hybrid Log Gamma, HLG) EOTF.

Would love to have it, Technicolor guys gave me an asset in HLG and I'm hoping to use fmtconv to do the conversion to HDR10 to feed into my currently limited STB client - the broadcom chipset doesn't support HLG afaik.

See the documentation for the internal VS resizer. It can actually do a lot of things...

[m3sh]

Originally Posted by Myrsloik

See the documentation for the internal VS resizer. It can actually do a lot of things...

Ahh yes... took a look at this though and while I'm still using R32 (and probably need to upgrade to VS R36) just want to be sure that upgrading will address the fact that R32 doesn't seem to like the "transfer_s" parameter...? Right now I can't get it working due to specifying the transfer output function as a string, seeing as I don't know the enumerated value for st2084.

[Myrsloik]

It wasn't implemented back then

[m3sh]

Of course... I am a sad (and lazy) panda. Thanks, I'll get 36 installed and try again.

[Cary Knoop]

Originally Posted by adsun701

Hi there,
When I was testing if fmtconv could make transforms from 709 content to ciexyz content, I may have encountered a possible bug.
I either get a segmentation fault or completely black output when converting 709 primaries to ciexyz primaries.

import vapoursynth as vs
core = vs.get_core()
core.std.LoadPlugin(path=r'/usr/local/lib/vapoursynth/libimwri.so')
clip = core.ffms2.Source(r'/home/andy/Sample.mkv')
clip = core.std.Trim(clip, 2300, 2300)
clip = core.fmtc.bitdepth(clip, bits=32, dmode=6)
clip = core.fmtc.matrix(clip, mats='709', matd='RGB', col_fam=vs.RGB, bits=32)
clip = core.fmtc.transfer(clip, transs='709', transd='linear', bits=32)
clip = core.fmtc.primaries(clip, prims='709', primd='ciexyz')
clip = core.fmtc.transfer(clip, transs='linear', transd='2084', cont=1000/10000, bits=32)
clip = core.imwrif.Write(clip, 'EXR', 'image%04d.exr', firstnum=0)
clip.set_output()

Is this a bug? If so, I would appreciate that it be fixed. If not, then what am I doing wrong?

It is my understanding you need to convert the clip to 4:4:4 first, I assume the mkv file is 4:2:0 but perhaps that assumption is wrong.

Also when I use the bits parameter in transfer() from '709' to 'linear' I get: matrix: output bitdepth not supported.

[adsun701]

Thanks for your help!

I actually discovered a bug in the primaries conversion code. When converting to ciexyz primaries, there is a divide by zero error due to the coordinates of the primaries. Hopefully it gets fixed!

[adsun701]

Also, could SMPTE ST 2085 matrix conversion and ICtCp matrix conversion be implemented? They are supported in the latest version of HEVC, and used for HDR content. SMPTE ST 2085 is used with CIEXYZ primaries, while ICtCp can be used with any primaries.

Thanks!

[TheFluff]

The VS builtin resizer (zimg) has support for ICtCp being worked on as of right now. You should probably consider porting existing code to zimg as well; it's pretty much just flat out better than fmtconv, and it's in active development (unlike fmtconv which hasn't had any activity at all in the last year). It also has more contributors and users than fmtconv since it's a general-purpose C/C++ library and not tied to a filtering framework like VS or Avisynth. It's available in ffmpeg via libavfilter's vf_zscale, for example.

I don't think there are any features fmtconv has that the internal resizer (that is zimg) lacks, anymore. If there are, submit a feature request. http://www.vapoursynth.com/doc/functions/resize.html

[Selur]

I don't think there are any features fmtconv has that the internal resizer (that is zimg) lacks, anymore. If there are, submit a feature request. http://www.vapoursynth.com/doc/functions/resize.html

May be I'm just overlooked it, but I didn't find anything in the documentation about:

How to handle interlaced content? (In fmtconv there is interlaced&interlacedd. Do the resizers automatically handle and keep the interlacing state the source is in?)
Whether there are more resizers available than Bilinear/Bicubic/Point/Lanczos/Spline16/Spline36 through zimg? (fmtconv has point/box/bilinear/cubic/bicubic/lanczos/blackman/blackmanminlobe/spline16/spline36/spline64/spline/gauss/sinc )

[TheFluff]

Originally Posted by Selur

How to handle interlaced content? (In fmtconv there is interlaced&interlacedd. Do the resizers automatically handle and keep the interlacing state the source is in?)

It's handled automatically, yes, assuming that the _Field or _FieldBased frame props are set correctly. Unlike Avisynth, VS (and zimg) understands that an interlaced clip that you call separatefields() on does not magically become progressive, so if you feed a field-separated clip to the internal resizer, resize it horizontally and then weave it again it'll still be handled correctly (there won't be any jumping up and down).

Originally Posted by Selur

Whether there are more resizers available than Bilinear/Bicubic/Point/Lanczos/Spline16/Spline36 through zimg? (fmtconv has point/box/bilinear/cubic/bicubic/lanczos/blackman/blackmanminlobe/spline16/spline36/spline64/spline/gauss/sinc )

No. None of the extra ones are all that useful anyway.

[WolframRhodium]

Originally Posted by TheFluff

It's handled automatically, yes, assuming that the _Field or _FieldBased frame props are set correctly. Unlike Avisynth, VS (and zimg) understands that an interlaced clip that you call separatefields() on does not magically become progressive, so if you feed a field-separated clip to the internal resizer, resize it horizontally and then weave it again it'll still be handled correctly (there won't be any jumping up and down).


No. None of the extra ones are all that useful anyway.

However, as an example, some people has been widely using gauss for years, and this phenomenon will continue. How can you come down to your conclusion?

[TheFluff]

Originally Posted by WolframRhodium

However, as an example, some people has been widely using gauss for years, and this phenomenon will continue. How can you come down to your conclusion?

The Gauss resizer isn't even a resampler at all, mathematically speaking. As far as I can recall I don't think people really use it all that much for actually resizing things; they use it to blur things. Use something else for that.

Blackman is a bad windowed sinc; use lanczos instead.

[feisty2]

Originally Posted by TheFluff

It's handled automatically, yes, assuming that the _Field or _FieldBased frame props are set correctly. Unlike Avisynth, VS (and zimg) understands that an interlaced clip that you call separatefields() on does not magically become progressive, so if you feed a field-separated clip to the internal resizer, resize it horizontally and then weave it again it'll still be handled correctly (there won't be any jumping up and down).


No. None of the extra ones are all that useful anyway.

gaussresize is more useful than you think, it could be used as a non-ringing convolution filter, and it's far more powerful than std.convolution, cuz:
a. it works with any radius (using the "taps" parameter)
b. it does not require you to manually calculate the coefficient matrix, you could simply control the filtering strength using the "p" parameter.

[WolframRhodium]

Originally Posted by TheFluff

The Gauss resizer isn't even a resampler at all, mathematically speaking. As far as I can recall I don't think people really use it all that much for actually resizing things; they use it to blur things. Use something else for that.

Blackman is a bad windowed sinc; use lanczos instead.

The combination of lanczos and gauss in upsampling is popular. But zimg doesn't have gauss, that's a problem.

Another example is feisty2's Vine/Plum.

[TheFluff]

Originally Posted by feisty2

gaussresize is more useful than you think, it could be used as a non-ringing convolution filter, and it's far more powerful than std.convolution, cuz:
a. it works with any radius (using the "taps" parameter)
b. it does not require you to manually calculate the coefficient matrix, you could simply control the filtering strength using the "p" parameter.

Isn't that exactly what I said? Write an actual Gaussian convolution filter if that's what you want, instead of shoehorning it into a resampler-that-isn't.

[feisty2]

Originally Posted by TheFluff

Isn't that exactly what I said? Write an actual Gaussian convolution filter if that's what you want, instead of shoehorning it into a resampler-that-isn't.

gaussresize is already highly optimized as a part of the resample function in fmtc, I could write a Gaussian filter any day of the week but I'm NOT a professional programmer, so that's gonna be a lot worse (performance-wise) than the existing gaussresize approach anyways, and I'm definitely not wasting my time just to mess with stupid things like that.

[WolframRhodium]

Originally Posted by TheFluff

Isn't that exactly what I said? Write an actual Gaussian convolution filter if that's what you want, instead of shoehorning it into a resampler-that-isn't.

What's more, fmtconv contains various dithering mode and custom resample kernel, but what about zimg?