fmtconv: resize, bitdepth and colorspace conversions

[Myrsloik]

Try other filters and you'll see it's probably a general speed degradation. It's because I've had to temporarily stop using tcmalloc. It will be back some day soon probably. Once it has the weird VS2015 issues fixed.

[an3k]

Originally Posted by cretindesalpes

Here is another plug-in for Vapoursynth.

>>> fmtconv-r17.zip <<<

Thank You

I just noticed the autogen.sh in the zip-file doesn't has chmod u+x and is additionally in DOS format, thus Linux cannot execute it, only after manual chmod and conversion into "Linux" format. The file in the git repo is fine. However both libraries are installed into $PREFIX/lib/ instead of $PREFIX/lib/vapoursynth/

[jackoneill]

Originally Posted by an3k

Thank You

I just noticed the autogen.sh in the zip-file doesn't has chmod u+x and is additionally in DOS format, thus Linux cannot execute it, only after manual chmod and conversion into "Linux" format. The file in the git repo is fine. However both libraries are installed into $PREFIX/lib/ instead of $PREFIX/lib/vapoursynth/

Use the --libdir parameter.

[mawen1250]

I encountered a really weird bug that might relate to fmtc.bitdepth.
In fact it was someone reporting bug about mvf.BM3D then did I go to figure out what's going wrong. After testing for nearly a whole day, finally I found a minimum script to reproduce this issue.

video example and the test script

environment:
CPU is Ivy Bridge (no AVX2)
Windows 7
VS R30 64bit
fmtconv-r17
BM3D-r4
L-SMASH-Works-r859-20151231

import vapoursynth as vs

core = vs.get_core(threads=0)
core.max_cache_size = 8000

src_path = r'Madoka_01.mkv'
src = core.lsmas.LWLibavSource(src_path, threads=1)
src = core.fmtc.bitdepth(src, bits=32) # Comment this line will not reproduce the issue
src = core.fmtc.resample(src, kernel='bicubic', a1=0, a2=0.5, css='444') # Catmull-Rom can reproduce the issue
#src = core.fmtc.resample(src, kernel='spline', css='444') # Spline can reproduce the issue
#src = core.fmtc.resample(src, kernel='spline36', css='444') # and some other kernels will not
src = core.fmtc.matrix(src, mat='709', col_fam=vs.RGB)

radius = 1
param = {'sigma':[3,3,3], 'radius':radius, 'profile':'fast', 'matrix':100}
sample = vs.FLOAT # vs.INTEGER will not reproduce the issue
src = core.bm3d.RGB2OPP(src, sample)
flt = core.bm3d.VBasic(src, **param).bm3d.VAggregate(radius, sample)
flt = core.bm3d.VFinal(src, flt, **param).bm3d.VAggregate(radius, vs.FLOAT) # vs.INTEGER will not reproduce the issue
flt = core.bm3d.OPP2RGB(flt, vs.FLOAT) # vs.INTEGER will not reproduce the issue

final = flt
#final = core.fmtc.matrix(final, mat='709', col_fam=vs.YUV) # optionally, uncomment this line will also reproduce the issue
final = core.fmtc.bitdepth(final, bits=8, dmode=3) # with float input, integer output and dmode=3~7, the output is corrupt

out = final # src and flt are OK
out[111:].set_output()

From what I've seen the corrupt result is produced by the last fmtc.bitdepth, and only with float input+integer output+dmode=3~7.

The weird thing is that only under specific circumstance will this issue be reproduced. With a tiny change in the script (e.g. different resample kernel, vs.FLOAT->vs.INTEGER, remove the BM3D part, change dmode to 0~2) the result will be correct. Different environment may also affect it but I'm not sure.
More specific information is written in the comments.
I've tried converting source to lossless format and the result is the same, thus it should not be a problem relates to source filter.

[mawen1250]

After more testing, I found out that it's related to the final estimate of BM3D, which can produce very large or even NaN/INF results, and break the error diffusion algorithm of fmtc.bitdepth.

Adding either one like these before fmtc.bitdepth I'll get different black areas:
final = core.std.Expr(final, 'x -4294967296 max')
final = core.std.Expr(final, 'x -16777216 max')
final = core.std.Expr(final, 'x -65536 max')

Anyway, I've fixed this issue in BM3D-r5, but I'm not sure if it's a problem for fmtc.bitdepth.

[speedyrazor]

Is there a way to reverse the filed order is VapourSynth?

[MonoS]

Originally Posted by speedyrazor

Is there a way to reverse the filed order is VapourSynth?

If i've undestood what you're asking, it should be

reversed = core.std.SelectEvery(InterleavedField, 2, [1,0])

[speedyrazor]

Originally Posted by MonoS

If i've undestood what you're asking, it should be

reversed = core.std.SelectEvery(InterleavedField, 2, [1,0])

Hi, thanks for the suggestion, but this is not what I was after.
Lets say I have some footage which is top field first and I want to convert it to bottom field first. How would I do that in VapoutSynth please?

[jackoneill]

Originally Posted by speedyrazor

Hi, thanks for the suggestion, but this is not what I was after.
Lets say I have some footage which is top field first and I want to convert it to bottom field first. How would I do that in VapoutSynth please?

Probably by deinterlacing it and then interlacing it again. If you just take the fields as they are and display them backwards, you'll get jerky motion. Or if you put the top field at the bottom, and the bottom field at the top, it's going to look wrong. Therefore, slow processing (QTGMC) is the way to go.

[speedyrazor]

Originally Posted by jackoneill

Probably by deinterlacing it and then interlacing it again. If you just take the fields as they are and display them backwards, you'll get jerky motion. Or if you put the top field at the bottom, and the bottom field at the top, it's going to look wrong. Therefore, slow processing (QTGMC) is the way to go.

What I am looking for is the equivalent to ffmpeg's filter:

-vf phase=b

Just tested and this works correctly, any equivalent in VapourSynth?

[Myrsloik]

Simply delete the first field, same as in avisynth as usual...

core.std.DoubleWeave(clip.std.SeparateFields(tff=whatevertheinputis)[1:])[::2]

Untested but it's the general idea at least.

[speedyrazor]

Originally Posted by Myrsloik

Simply delete the first field, same as in avisynth as usual...

core.std.DoubleWeave(clip.std.SeparateFields(tff=whatevertheinputis)[1:])[::2]

Untested but it's the general idea at least.

I am using it like this:

ret = core.std.DoubleWeave(ret.std.SeparateFields(tff=True)[1:])[::2]

But am getting the error:

vapoursynth.Error: DoubleWeave: argument tff is required

[stax76]

can fmtconv convert to the same format used in the code below?

clip.resize.Bicubic(format=vs.COMPATBGR32)

[cretindesalpes]

fmtconv doesn’t output to COMPAT* formats. However you can convert to RGB24 with fmtconv and output to a COMPAT format with the builtin VS functions.

[cretindesalpes]

fmtconv r18:

• Added the primaries function to convert between gamuts.
• The “full” range is now closer to what is specified in the standards.
• A recent Vapoursynth is now required because API headers were updated to version 3.1.4.
• transfer: added the Adobe RGB and ProPhoto / ROMM curves.

[benmanw]

just forward J1Man's request from GitHub.

https://github.com/EleonoreMizo/fmtconv/issues/7

Feature Request, High Dynamic Range Video Conversion #7


J1Man commented 2 days ago

Dear Laurent,

I would like to request a feature. Can you add the capability to do UHD compliant Standard Dynamic Range (SDR) to High Dynamic Range (HDR) conversion (and also the opposite HDR to SDR conversion) to Fmtconv plugin?

I also sent you a long email with all the technical research that I could find. In summary, the required new features are listed below.

1) A custom HDR-1000 system compatible SMPTE ST 2084 transfer function that maps to "0 cd/m2 to 1000 cd/m2" luminance range. (Note: current 2084 function included in Fmtconv maps to 0 to 10,000 cd/m2)

2) Add DCI-P3 color primaries as a preset to primaries function. This is not very urgent since "gd=[0.265, 0.69], bd=[0.15, 0.06], rd=[0.68, 0.32], wd=[0.3127, 0.329]" command can be used to manually define DCI-P3 primaries in fmtconv r18.

Once these two new features are implemented, the workflow in Fmtconv will be the following for SDR to HDR conversion.
-Convert source video to "Linear light RGB 4:4:4"
-If necessary, change primaries to one of the HDR compatible primaries (BT709, BT2020, or DCI-P3)
-Convert transfer function from Linear to the custom SMPTE ST 2084 that maps to "0 to 1000 cd/m2" luminance range
-Resample to YUV 4:2:0 10 bit using BT2020 or BT709 matrix (which ever matrix is appropriate)
-Send YUV to x265 encoder. Specify the 2084 transfer fuction, appropriate matrix and appropriate primaries in the x265 command line.

The opposite steps can be used to convert HDR to SDR.

Please let me know what you think.

Best Regards

[benmanw]

One more message from J1Man. He won't be able to post on Doom9 until he has been registered for 5 days (until 3/19/2016).

I attached the source code patch zip file for fmtconv r18 that modifies SMPTE-ST-2084 function to "0 cd/m2 to 1000 cd/m2" luminance range and adds the "dci-p3" preset to primaries function. I also attached a zip file that contains the complete fmtconv r18 source code that is already patched. I would appreciate if somebody with Visual Studio can compile the 64bit fmtconv.dll for Windows 7. No matter how hard I tried in the last 2 days, I could not compile fmtconv.dll on Windows 7 by using MSYS2 and mingw64.

I saw that fmtconv uses the "Dolby Labs recommended version of SMPTE-ST-2084 function" specified on page 17/58 of the following PDF document.

http://www.mediaandbroadcast.bt.com/...HDTV-final.pdf

I think that was good choice because Dolby Labs version is designed for easy scalability between luminance ranges. Upper bound of the range can be changed by dividing or multiplying the right hand side of the function shown on page 17/58 of the PDF document. That's how I modified lines 78 and 83 of the TransOp2084.cpp file.

I added "dci-p3" primaries preset after line 356 of Primaries.cpp function.

fmtconv-r18-HDR1000-patch-only.zip
fmtconv-r18-patched-with-HDR1000features.zip

[cretindesalpes]

J1Man:

fmtconv does not need any modification to support 1000 cd/m² as HDR peak white. If your 100% light value represents 1000 cd/m², you can divide the components by 10 so they fit in a scale where 100% means 10000 cd/m². For this, reduce the contrast directly in the transfer command by setting cont=0.1.

clip = core.fmtc.transfer (clip, transs="linear", transd="2084", cont=1000.0/10000.0)

BTW in your email you showed me a Vapoursynth code snippet where the transfer functions had fulls=False and fulld=False. This is not right because the R’G’B’ input/output for matrix is full by default.

I added the DCI-P3 colorspace to fmtconv and will release an update in the next few days.

[cretindesalpes]

fmtconv r19:

• primaries: refined the values for the Adobe Wide gamut and BT.2020 primaries.
• primaries: added DCI-P3, ACES AP0/AP1, S-Gamut, S-Gamut3.Cine, ALEXA and V-Gamut presets.
• transfer: added ACEScc, ERIMM, S-Log2, S-Log3 and V-Log curves.

[J1Man]

Cretindesalpes:

Thanks for your reply and thanks for adding DCI-P3 to the primaries. I will use cont option, as you suggested, to change the luminance range. I will test the videos on my UHD tv and let you know if there are any HDR related issues.


Benmanw:

Thanks for forwarding my messages to the doom forum.