fmtconv: resize, bitdepth and colorspace conversions

[cretindesalpes]

Here is another plug-in for Vapoursynth.

>>> fmtconv-r31.zip <<< (https://ldesoras.fr/src/vs/fmtconv-r31.zip)

Fmtconv is a format-conversion plug-in for the Vapoursynth and Avisynth+ video processing engines. It does:

Resizing.
Bitdepth conversion with dithering.
Colorspace conversion (matrix, transfer characteristics and chromatic adaptation).

Supports:

8-–12-, 14- and 16-bit integer, 32-bit float
Colorspaces: RGB, Y, YUV, YCoCg, YDzDx and ICtCp in 4:4:4, 4:2:2, 4:2:0 and 4:1:1 chroma subsampling factors.
Progressive and interlaced content.

Fmtconv is focussed primarily on quality and exactness rather than execution speed. This does not mean it is slow or unoptimized, but fmtconv is clearly not on par with the fastest equivalent 8-bit filters.

The full documentation is included in the zip file.
New requirement from r29: Vapoursynth r55 or above (support for API v4 only).

If you’re curious, you’ll see undocumented functions in this plug-in. However they are temporary and will be removed later or moved to another plug-in, so please don’t use them.

Source code is also available as Git repository (https://gitlab.com/EleonoreMizo/fmtconv).

[sneaker_ger]

Thank you, but do you really think it is a good idea to offer stacked? Maybe we should be getting rid of that hack once and for all.

[cretindesalpes]

The stacked stuff is only meant to offer interoperability with Avisynth plug-ins, before they all get ported.

[kolak]

Thanks a lot- great work :)

:thanks:

[Keiyakusha]

Here it comes first question from someone who have no clue about this plugin.

With it can I convert lets say 320x2 (or 2x320) YV12 clip into 320x2 YV24 and without rising bitdepth? For example 8bit in - 8 bit calculation - 8 bit out, not 8-16-8

Edit: i'm not an expert on YCgCo, does current implementation offers lossless conversion to and from RGB?

[kolak]

You probably can't as internal processing is done at high precision, but this is only a good thing :)

[cretindesalpes]

Keiyakusha: no it's not possible. All the resizing calculations are done in float anyway so in the end a bitdepth conversion has to be done (if you don't want a float clip).

I haven't tested if the YCgCo can restore lossless RGB. It would need at least 9 bits in theory.

[Keiyakusha]

You probably can't as internal processing is done at high precision, but this is only a good thing :)

Unfortunately I have different opinion on this. Outside of professional 3D and 2.5D editing software, where high bitdepth often used for better control in manipulating with color or may represent something like strength of light that goes through less than 100% opaque obstacles (and also maybe x264's 10bit streams too), I don't believe in high bitdeph. In other words using more than 8bits for things like resize, denoise or whatever in my opinion gives so little improvements, that it not worth even one extra cpu cycle spent on it.

This rises another question. Is it possible to estimate how big is speed difference between similar colorspace (or some other) conversions (8bit in - 8bit out) using this plugin compared to avisynth? (assuming everything is done in 1 thread)

[cretindesalpes]

Therefore this plug-in is most likely not for you. It’s much slower than the Avisynth resizers. You can check the speed difference by setting only one thread in vapoursynth (use core = vs.Core (threads=1) ) and do a benchmark with a simple blankclip as input. Or you can do everything in Avisynth, using Dither_resize16 which uses the same algorithms and should give a comparable speed, making sure that avstp is disabled (call avstp_set_threads(1) at the end of the script).

[Myrsloik]

Unfortunately I have different opinion on this. Outside of professional 3D and 2.5D editing software, where high bitdepth often used for better control in manipulating with color or may represent something like strength of light that goes through less than 100% opaque obstacles (and also maybe x264's 10bit streams too), I don't believe in high bitdeph. In other words using more than 8bits for things like resize, denoise or whatever in my opinion gives so little improvements, that it not worth even one extra cpu cycle spent on it.

This rises another question. Is it possible to estimate how big is speed difference between similar colorspace (or some other) conversions (8bit in - 8bit out) using this plugin compared to avisynth? (assuming everything is done in 1 thread)

If course it is, just set vs to use one thread and use ffms2 as the video source in both and it should be very easy to compare 1:1.

[Keiyakusha]

Thanks for your answers. I kind of expected this answer but had to make sure. I mainly asked because right now using vapoursynth's internal resizer it seem to be impossible to do 320x2 YV12 -> 320x2 YV24 (and back) at all. (or Yv12->rgb, or whatever). And color conversion seems to be not included in separate plugin with internal avisynth functions. Thus right now I have to use ffms2 in avisynth, do conversions there and only then pass this to vapoursynth... (using awesome plugin by Chikuzen). Unless I missed something.

[kolak]

Unfortunately I have different opinion on this. Outside of professional 3D and 2.5D editing software, where high bitdepth often used for better control in manipulating with color or may represent something like strength of light that goes through less than 100% opaque obstacles (and also maybe x264's 10bit streams too), I don't believe in high bitdeph. In other words using more than 8bits for things like resize, denoise or whatever in my opinion gives so little improvements, that it not worth even one extra cpu cycle spent on it.

This rises another question. Is it possible to estimate how big is speed difference between similar colorspace (or some other) conversions (8bit in - 8bit out) using this plugin compared to avisynth? (assuming everything is done in 1 thread)

I deal with different type of data and done many test, which actually surprised me.

a= take 10bit HD file and scale at 8bit to SD.
b= take 10bit HD file, dither it and scale to SD (at 8bit).
c= take same file scale at 10bit to SD and than dither.

There is actually quite visible difference in all of them, but I was surprised that even b and c show visible difference. I though that there will be no real difference, but there is.

Just a not- we're talking about proper 10bit source- eg shot on RED, Alexa etc cmera.

[Keiyakusha]

I deal with different type of data and done many test, which actually surprised me.

a= take 10bit HD file and scale at 8bit to SD.
b= take 10bit HD file, dither it and scale to SD (at 8bit).
c= take same file scale at 10bit to SD and than dither.

There is actually quite visible difference in all of them, but I was surprised that even b and c show visible difference. I though that there will be no real difference, but there is.

Just a not- we're talking about proper 10bit source- eg shot on RED, Alexa etc cmera.
Yes there can be quite big visual difference when going from higher to lower bitdepth one way or another. Also depends on the actual content though. I'll do "c" if it is possible. But in this particular case I was talking about 8bit source to begin with. Sorry for not making it clear, this was kind of continuation of my initial post. Also what you say applies more to genuine 10bit sources than to upconverted.

[sneaker_ger]

A few questions:
1.) Does resample output 16 bit int or 32 bit float by default?
2.) "Bitdepth conversion with optional dithering."
How do I know if dithering is on or off?
3.) Is "MPEG2" the default chroma placement?

[cretindesalpes]

A few questions:
1.) Does resample output 16 bit int or 32 bit float by default?
The default is 16 bits for integer input, and float for floating point input. The 16-bit integer output is a straight conversion from the intermediate float results without dithering.

2.) "Bitdepth conversion with optional dithering."
How do I know if dithering is on or off?
Dithering is used when:
- Reducing the bitdepth of integer data, or converting from float to integer
- Doing a full-range ↔ TV-range conversion between integer formats, because the resulting values haven't an exact representation.

3.) Is "MPEG2" the default chroma placement?
Yes.

[sneaker_ger]

The default is 16 bits for integer input, and float for floating point input. The 16-bit integer output is a straight conversion from the intermediate float results without dithering.

Does it make any sense to choose 32 float output for 16 bit source? (Assuming I have more of your filters after that?)

Where can we find info on the dithering algos? Is there any good comparison that keeps video encoding in mind? Or is "Filter Lite" generally a good idea?

[ajp_anton]

Isn't YCgCo just YUV with different coefficients than 601 and 709? Because you're mentioning it as if it's an alternative to YUV.
And how do you use it? =)

Thread title is wrong BTW, "ftmconv".

[cretindesalpes]

sneaker_ger:

I’d say that keeping everything in 16 bit is sufficient, there is no need to use float. Float could be useful when working with linear light, because dark areas need more precision. Or when working with non-perceptually uniform colorspaces like CIE XYZ. Keeping a pipeline in float also avoids the conversion overhead, but the gain has to be balanced with the memory bandwidth doubling and related cache issues.

For the dithering algorithms, you’ll find relevant information here:
http://caca.zoy.org/wiki/libcaca/study/2
http://caca.zoy.org/wiki/libcaca/study/3
http://caca.zoy.org/wiki/libcaca/study/4
Filter Lite (Sierra 2-4A) is OK as a general purpose dither algorithm and is very similar to Floyd-Steinberg. Anyway if you want to dither to 8 bits before encoding to avoid colorbanding, you’ll probably prefer using ordered dithering.

ajp_anton:

Technically, YCgCo just is another linear combination of the RGB values into signals approximating perceptual luminance and chrominance. But the result is different enough to create another category. I think it’s suited more for encoding than for processing, as it is reported to compress a bit better than YUV. It is specified by MPEG-4 part 10 (H.264) but I think players and encoders supporting it are still rare.

Thank you for the typo report, I’ll fix it.

[mandarinka]

Nice addition!

So I take it that if one is paranoid, it'S best to pass around the float format and dither to 8/16 bit integer with bitdepth() as the last step (when the followup filters/encoder require it).

[sneaker_ger]

@cretindesalpes

Thanks for the detailed answers and your work on the plugin. I tested it out today and it worked really great.

[cretindesalpes]

fmtconv r2 (http://forum.doom9.org/showthread.php?t=166504):

resample: optimized paths involving float input or output
resample: fixed white/magenta screen with 8-bit input and float output
bitdepth: implemented fast dither mode (but not in SSE2 yet)
bitdepth: optimized float-to-integer path
bitdepth: faster dithering when ampo = 1 and ampn = 0
matrix: enabled the SSE path for float operations


Note: I'll rename the "bitdepth" argument in the bitdepth() function in "bits" in the next release.

[Mug Funky]

this is beautiful. thankyou very much.

any speed losses in this plugin are made up for by the parallelism gained from using VS, and it can only get faster. i'm breaking realtime on BD transcodes for the first time ever, and in higher quality than i was ever able to achieve.

[Keiyakusha]

any speed losses in this plugin are made up for by the parallelism gained from using VS, and it can only get faster.
I don't understand how this is possible.
If that is true, this means x264 or whatever software you use fails to fill all available CPU resources.
If we assume that A filter twice as fast compared to B filter (both in one thread), then under 100% cpu load A still will be twice faster, regardless of how much threads you put in filter B. Even if B will output result faster, it will eat cycles that otherwise will go to x264, so overall there shouldn't be speed gain or unchanged speed, there should be speed loss.

[Mug Funky]

i'm comparing to encoding the same blu-ray through avisynth, which is single threaded. often it's the bottleneck when encoding, especially if using intermediate formats (because sometimes x264 isn't the final destination :))

[Keiyakusha]

i'm comparing to encoding the same blu-ray through avisynth, which is single threaded. often it's the bottleneck when encoding, especially if using intermediate formats (because sometimes x264 isn't the final destination :))
If you were some new user, I could understand this response, but you're not, so it's kind of trolling. Anyhow, I got your point.

[kolak]

cretindesalpes: thank you for your work. Quick test are promising- all seams to be working well and speed is very good. Are you going to implement adding noise? I found that whatever dithering method you use it's always good put put a bit of noise anyway.

Great job :thanks:

[kolak]

Have crash on these lines:

ret = core.ffms2.Source(source=r'S:\test.mov') -10bit DNxHD source
b= core.fmtc.resample(clip=ret, w=720, h=576)
c= core.fmtc.matrix (clip=b, mats="709", matd="601", col_fam=vs.YUV)
a= core.fmtc.bitdepth (clip=c, bitdepth=8, dmode=3)

It's matrix line, as without it all was working fine.
It also looks like dithering mode=5 does not work at all- video looks like it's without any dithering.

Any idea why (vs R16)?

[cretindesalpes]

I couldn’t reproduce your problems here (with both VS r15 and r16). Your script works and dmode=5 is effective. Are your sure your input is 4:4:4? Matrix needs 4:4:4 to work, but it should issue an error message, not a crash.

[kolak]

I couldn’t reproduce your problems here (with both VS r15 and r16). Your script works and dmode=5 is effective. Are your sure your input is 4:4:4? Matrix needs 4:4:4 to work, but it should issue an error message, not a crash.

No- source is 4:2:2 10bit (most likely YUV422P10 color space).
So I need to add line with conversion to 4:4:4 before matrix?

It crashes Vdub badly- straight away, no log etc.

Another question - how lossless is/can be YUV->RGB->YUV with your tool?
If source is YUV will it stay in YUV for all conversions?

[Keiyakusha]

kolak
Maybe I miss something but you not supposed to open in VDub anything that is not V210 or 8bit formats. Sure it crashes,closes,whatever, it doesn't supports whats coming into it.
Edit: and yes in theory it should give an error, but on practice for me just after some message pops-up, the process terminates.

[Myrsloik]

kolak
Maybe I miss something but you not supposed to open in VDub anything that is not V210 or 8bit formats. Sure it crashes,closes,whatever, it doesn't supports whats coming into it.

Stop trolling around. Vdub doesn't crash on unsupported formats. It simply says it can't decode that fourcc.

[Keiyakusha]

Stop trolling around. Vdub doesn't crash on unsupported formats. It simply says it can't decode that fourcc.
It does. Not sure if it's a crash or whatever but it stops working

[kolak]

I convert to v210 or YUY2 if I want to see it in Vdub- I know Vdub very well. Read above.

[cretindesalpes]

No- source is 4:2:2 10bit (most likely YUV422P10 color space).
So I need to add line with conversion to 4:4:4 before matrix?
Yes, just add css="444" in your resample line.

Another question - how lossless is/can be YUV->RGB->YUV with your tool?
It's not lossless. For the matrix operation, it will be bound by numerical error noise. So the higher the bitdepth, the better. Anyway a 16-bit chain should be enough to process a 10-bit input.

And if you add inverse and direct chroma subsampling, it will add more errors because of the aliasing and the limited kernel bandwidth. Even using a very large kernel, the losslessness will not be guaranteed.

If you want more chance to recover the original pixels, dither with dmode=1 at the very last step. But if you inserted some operations in your processing like color or level correction, use a real dithering and forget about losslessness.

If source is YUV will it stay in YUV for all conversions?
I'm not sure if I correctly understand your question. We now have real high-bitdepth and planar RGB colorspaces in Vapoursynth, so there is no need to use the same tricks as in the Dither tools for Avisynth. When you specify a RGB colorspace in matrix, the clip is really converted to this colorspace.

[kolak]

If import filter reads file as eg. YUV422P10 and I want to scale it to SD staying in YUV mode is this possible (or will it always go to RGB during internal conversions)?
Like in the case above- DNxHD scaled to PAL with 709->601 conversion- will this stay in YUV ?

I don't want to go to RGB when source file is read as YUV.

Thanks for your clear answers :)

[cretindesalpes]

Yes of course, resample and matrix will keep the same colorspace family by default. For matrix, you can specify another family as target colorspace.

If your input is YUV422P10 it must be converted to 4:4:4 before applying the matrix, and converted back to 4:2:2 after. This is how it should be done. This will yield a more reliable result than for example ColorMatrix, which processes the chroma without taking the luma into account (well, this is not important for the specific 601<->709 case), and processes the luma by point-resizing the chroma on the fly. This approximation is fast but not very accurate.

If resampling the chroma twice is not acceptable for you, there is another possibility, a bit more complex to set up: use the basic 4:4:4 conversion to generate the luma plane, resample the luma to match exactly the initial chroma (half-sized 4:4:4) to generate the chroma planes, and merge the resulting planes.

[kolak]

Well- if it stays in YUV or RGB depending on incoming format by default, than this is good for me :)

I did try YUV->RGB->YUV but it end up quite far from being lossless.

Source is YUY2, than:

ret= core.fmtc.resample(clip=ret, css="444")
ret= core.fmtc.matrix (clip=ret, mat="601", col_fam=vs.RGB)
ret= core.fmtc.matrix (clip=ret, mat="601", col_fam=vs.YUV)
ret= core.fmtc.resample (clip=ret, css="422")
ret = core.fmtc.bitdepth (clip=ret, bitdepth=8, dmode=1)

dmode=-1 keeps crashing.

[cretindesalpes]

Hmm right it's dmode=1 now, it was -1 in the Dither tools for avisynth so I got confused. I fixed my previous message. Also, for this kind of stuff, you can resample with a better kernel than the default (a 8-tap Blackman for example).

[kolak]

Are all kernels as in dither, should I look in dither docs? I used spline36 I think (it was just a guess :)).

Sorry- I used spline36 when I tested resizing.

[Reel.Deel]

Hi cretindesalpes, like always thank you very much for another awesome tool. :)
For testing purposes only, I used one of your resizing examples in the Dither documentation. I don't know if I'm doing something wrong but I'm getting some strange results.

Example in Dither doc:
Sharpening the luma using the convolver of the resizer:

Dither_convert_8_to_16 ()
Dither_resize16 (Width (), Height () / 2, kernel="impulse -1 6 -1",
\ fh=-1, fv=-1, cnorm=true, center=false, y=3, u=2, v=2)
DitherPost ()
My Script:
import vapoursynth as vs
import sys
core = vs.Core()

# Load Plugins
core.std.LoadPlugin(path=r'C:\Vapoursynth\FMTConv\fmtconv.dll')
core.avs.LoadPlugin(path=r'C:\AviSynth 2.5\plugins\DGDecodeNV.dll')

# Blu-ray source cropped to 1920x1080 by DGSource
src = core.avs.DGSource(dgi=r'X:\Test.dgi')

# Processing
src = core.fmtc.resample(clip=src, w=960, h=540, impulse="-1 6 -1", fh=-1, fv=-1, cnorm=1, css="420", planes="3,2,2", center=2)
src = core.fmtc.bitdepth(clip=src, bitdepth=8)

# VDub Output
last = src
The problems I'm encountering:
Sometimes it opens without resizing at all.
Sometimes it resizes to 960x1080 or to 1920x540.
Sometimes (rare) it resizes correctly.
But most of the time VDub gives me the following error:
Avisynth open failure:
python exception: 'resample: argument clip is required'
I tried removing some parameters to try to find the problem. The following still produces the problems mentioned above.
.....
src = core.fmtc.resample(clip=src, w=960, h=540, impulse="-1 6 -1")
src = core.fmtc.bitdepth(clip=src, bitdepth=8)
.....
I'm using VS r16 on 32-bit Windows XP SP3.

*edit*
Using the following, the correct results are more consistent but the other problems still remain.
src = core.fmtc.resample(clip=src, w=1920, h=540, impulse="-1 6 -1")

[sneaker_ger]

If your input is YUV422P10 it must be converted to 4:4:4 before applying the matrix, and converted back to 4:2:2 after. This is how it should be done. This will yield a more reliable result than for example ColorMatrix, which processes the chroma without taking the luma into account (well, this is not important for the specific 601<->709 case), and processes the luma by point-resizing the chroma on the fly. This approximation is fast but not very accurate.


So matrix on something other than 4:4:4 is not just not yet implemented, but is not implemented on purpose?

[cretindesalpes]

Reel.Deel:

Try with the syntax: impulse=[-1, 6, -1]. This is no longer a string but a wonderful datastructure called array.

However I don't understand what's happening with these error messages that don't match the actual error. I need to investigate this.

sneaker_ger:

Yes, more or less. As mentioned previously, I could implement specific cases differently but at the moment it works as designed. It needs to be optimized, though.

[ajp_anton]

Would it be possible to add support for the Jinc resizer (in madVR)?

[kolak]

cretindesalpes:

Source is YUY2:

b= core.fmtc.resample(clip=ret, w=720, h=576, kernel="spline36", css="444")
c= core.fmtc.matrix (clip=b, mats="709", matd="601", col_fam=vs.YUV)
a= core.fmtc.bitdepth (clip=c, bitdepth=8, dmode=1)

I seams to have some issues with dither tools- resizing seams to mess with chroma placement, specially with css=444, which is required for matrix conversions. There are issues without css=444 also, but less visible.
Test on HD bars going to SD shows chroma shifts compared to Vdub and internal Edius resizing (they are both about the same). There is something wrong going on with dither tools. Another issue are dithering (higher modes, eg 3) artefacts, which seams to break solid nature of bars (specially green one).
This was also visible a lot when I tested YUV->RGB->YUV conversion- avisynth was way more lossless, which is not the way how it should be taking into account dither precision.
I will try to put some grabs later.


update: it's all related to last line in my script which is: last=core.resize.Spline(clip=h,format=vs.COMPATYUY2) - it's not dither tools, but this line messing with good output of dither tools. Is this part of swscale?

[cretindesalpes]

ajp_anton:

Not at the moment. Jinc is quite different of the other kernels (the transform is not separable) so it would need a significant rewrite to add it. Currently I focus on making everything work correctly.

kolak:

Yes, core.resize.Spline uses swscale. Do I understand that finally you haven't any more problem with fmtconv?

— - —

Time for a new release:

fmtconv r3 (http://forum.doom9.org/showthread.php?t=166504) bitdepth: changed the "bitdepth" parameter to "bits"
bitdepth: added SSE2 optimizations for upconversions.
resample: added interlaced resizing ("interlaced" parameter)
resample: now sets "_ColorRange" and "_ChromaLocation" properties when known
resample: fixed the "planes" parameter previously interpreted as 0 (black or green screen).

[kolak]

Yes- fmtconv is absolutely fine (at least bits which I tested)- all problems were related to swscale!

Question- how interlaced resizing is handled- resize per field?

[cretindesalpes]

kolak:

For interlaced resizing, you have to separate the fields first. Then the relative field position is automatically taken into account when resizing.


fmtconv r4 (http://forum.doom9.org/showthread.php?t=166504):

Added a documentation.
Filters now write some frame properties when known.
Fixed the code so it can be compiled on Linux. Thanks to Jackoneill/Nodame for testing.
bitdepth: no need to specify any bitdepth or colorspace (for simple range conversions).
matrix: Added SSE2 implementation for integer processing.
matrix: Allows the destination bitdepth to be higher than the input (added the bits parameter).
matrix: col_fam completes csp instead of replacing it.
resample: Added interlacedd to specify if output is interlaced (allows simple bobbing).
resample: Added tff and tffd to specify field parity.
resample: Added scale, scaleh and scalev for easier magnification.
resample: Added a two-digit mode to css.
resample: Fixed a typo preventing to select 4:1:1 chroma subsampling.
Added nativetostack16.

[Revgen]

Documentation needs a fix. The resample() parameters scalev and scaleh are defined as scalex and scaley.

I like the new bobbing method. Way easier to implement.

[cretindesalpes]

Oh right! I fixed the doc without changing the version number, just download again the zip file.

[Robert Martens]

Just noticed a small typo in one of your error messages, it looks like fmtc/Resample.cpp, on line 251, says 'w' where it should say 'h' to indicate that the user's provided an invalid height for the output colorspace.

[cretindesalpes]

Thanks for reporting it. Copy/paste is the most evil invention ever.

[sl1pkn07]

howto build in linux? (to linux)

greettings

[jackoneill]

howto build in linux? (to linux)

greettings


g++ -O3 -shared -msse2 -fPIC -o libfmtconv.so AvstpWrapper.cpp main.cpp fmtc/*.cpp fstb/*.cpp vsutl/*.cpp -I.
Add -march=your_cpu_here if you get "undefined symbol __sync_val_compare_and_swap_16" or something like that.

clang from svn should work as well.

[sl1pkn07]

thanks man.

[kolak]

kolak:

For interlaced resizing, you have to separate the fields first. Then the relative field position is automatically taken into account when resizing.


fmtconv r4 (http://forum.doom9.org/showthread.php?t=166504):

Added a documentation.
Filters now write some frame properties when known.
Fixed the code so it can be compiled on Linux. Thanks to Jackoneill/Nodame for testing.
bitdepth: no need to specify any bitdepth or colorspace (for simple range conversions).
matrix: Added SSE2 implementation for integer processing.
matrix: Allows the destination bitdepth to be higher than the input (added the bits parameter).
matrix: col_fam completes csp instead of replacing it.
resample: Added interlacedd to specify if output is interlaced (allows simple bobbing).
resample: Added tff and tffd to specify field parity.
resample: Added scale, scaleh and scalev for easier magnification.
resample: Added a two-digit mode to css.
resample: Fixed a typo preventing to select 4:1:1 chroma subsampling.
Added nativetostack16.

Is random noise coming? Small amount after dithering always helps :)

[kolak]

Another thing:
just had a case where I had to convert interlaced 10bit 422 to 8bit 420 for QTGMC processing- it needs 5 additional lines to achieve. Even if it's not very complicated than it's way "to long"- it should be done with one (or 2) line. Can't we have most of the processing included in one command?

[cretindesalpes]

fmtconv r5 (http://forum.doom9.org/showthread.php?t=166504):
Added 12-bit support for all the functions.
Added matrix2020cl to convert between linear RGB and Y’Cb’Cr’ colorspaces using the BT.2020 constant luminance matrix.
matrix: Added the BT.2020 matrix, non constant luminance mode only.
matrix: Added single-plane output with the singleout parameter.
resample: allows x:y:z form for specifying the colorspace subsampling css.
resample: added SSE2 integer calculations for slight speed improvement. Activated by default, use flt=1 to compute everything in float (previous operating mode).


Kolak:

I’ll add simple wrapping functions when the library is stabilized.

[Reel.Deel]

cretindesalpes, the binary is still the same one as r4.

Edit:
Thanks for the update! :)

[cretindesalpes]

Doh! Sorry. I replaced the .dll file. Just download again ftmconv-r5.zip.

[cretindesalpes]

My bad. fmtconv r6 (http://forum.doom9.org/showthread.php?t=166504) should fix it.

[cretindesalpes]

Got it. The sum of the integer coefficients was sometimes off by 1 or 2 units. The bug is now fixed and the change will be available in the next release.

[Mystery Keeper]

Wouldn't build until I included <algorithm> in all sources with min/max.
Wouldn't build with MinGW at all.

[Are_]

It builds fine with clang/gcc http://forum.doom9.org/showpost.php?p=1606550&postcount=53

[cretindesalpes]

fmtconv r7 (http://ldesoras.free.fr/src/vs/fmtconv-r7.zip): Warning—I left an important bug in the resizer, you’d better wait for the next version.

64-bit version for Windows
resample: A few optimizations for special cases.
resample: fixed the coefficients used in integer resizing, whose sum was sometimes off by a few units.
Should compile with MinGW

[Mystery Keeper]

This script
from __future__ import print_function

vapoursyth_plugins_path = 'E:\\vapoursynth-plugins\\x64\\'
import vapoursynth as vs
import sys
sys.path.append(vapoursyth_plugins_path + 'E:\\vapoursynth-plugins\\py\\')
core = vs.get_core(threads=8)

core.set_max_cache_size(16000)

import os
for filename in os.listdir(vapoursyth_plugins_path):
if filename[-4:] != '.dll':
continue
try:
core.std.LoadPlugin(vapoursyth_plugins_path + filename)
except Exception as e:
print("Error: ", e, end='\n', file=sys.stderr)

core.std.LoadPlugin(path = 'D:\\Programming\\TempLinearApproximate-VapourSynth\\build\\release-x64\\templinearapproximate.dll')

#fun = core.list_functions()
#print(fun, end='\n', file=sys.stderr)

d2vfile = 'F:\\Video to Process\\Riaru Onigokko 2\\VTS_01_1.d2v'

ret = core.d2v.Source(input=d2vfile)

ret = core.vivtc.VFM(ret, order=1, mode=5)
ret = core.vivtc.VDecimate(ret)

def tvToPC(input):
c = core.fmtc.resample (clip=input, css="444")
c = core.fmtc.matrix (clip=c, mats="601", matd="709")
c = core.fmtc.resample (clip=c, css="420")
c = core.fmtc.bitdepth (clip=c, bits=8)
return c

z = ret
#z = tvToPC(z)

ret = tvToPC(ret)

diff = core.std.MakeDiff(z, ret)
line1 = core.std.StackHorizontal([z, ret])
line2 = core.std.StackHorizontal([diff, ret])
compare = core.std.StackVertical([line1, line2])
compare.set_output()
gives this output

http://s019.radikal.ru/i620/1311/1a/c9c8845119af.png

I've also managed to crash VirtualDub with this crash report (http://paste.org.ru/?jgrt8q).

[cretindesalpes]

ftmconv r8 (http://forum.doom9.org/showthread.php?t=166504):
resample: fixed bugs introduced in r7.

[Mystery Keeper]

'Tis fixed. Thank you!

[lansing]

I wanted to convert color space from YV12 to RGB to load some Virtualdub filters, and I'm getting the error "VFW module doesn't support RGB24 output". How should I do this?


ret = core.fmtc.resample (ret, css='444')
ret = core.fmtc.matrix (ret, mats="709", col_fam=vs.RGB)
ret = core.fmtc.bitdepth (ret, bits=8)

[lansing]

I guess it only accepts RGBs in interleaved format

which one are you referring to? fmtconv or virtualdub?

[jackoneill]

Is the internal threading of any use when VapourSynth runs filters in parallel? Or is it simply a leftover from Dither?

[cretindesalpes]

Yes it is more or less a leftover from Dither, because I want to keep the same base code to backport it from fmtconv to Dither later. It should be harmless anyway.

[alexxdls]

import vapoursynth as vs
core = vs.get_core()
import os
ext = '.png'
dir = r'PNG/'
srcs = [dir + src for src in os.listdir(dir) if src.endswith(ext)]
v = core.imgr.Read(srcs,24,1)
v = core.fmtc.matrix(v, mat="709", col_fam=vs.YUV, bits=16, fulls=1, fulld=1)
v = core.fmtc.bitdepth(v, bits=10, dmode=1)
enable_v210=True
v.set_output()Failed to evaluate the script:
Python exception: matrix: only constant pixel formats are supported.
Traceback (most recent call last):
File "vapoursynth.pyx", line 1148, in vapoursynth.vpy_evaluateScript (src\cython\vapoursynth.c:18441)
File "D:/TRAILERS/DCP/MOCKINGJAY-PT1_TLR-1_S_RU-XX_RU_51_2K_LION_20140729_DLA_IOP_OV/myVapourScript.vpy", line 8, in <module>
v = core.fmtc.matrix(v, mat="709", col_fam=vs.YUV, bits=16, fulls=1, fulld=1)
File "vapoursynth.pyx", line 1071, in vapoursynth.Function.__call__ (src\cython\vapoursynth.c:17443)
vapoursynth.Error: matrix: only constant pixel formats are supported.

[MonoS]

In the documentation of dither for avs there is a very handy quick explanation of what a dithering algo is good for.
I used to use the stucki error diffusion because seemed the best of the list.
In the vs version you added two new algos: Sierra-2-4A error diffusion and Ostromoukhov error diffusion.

Can you please give me a quick explanation like the ones on the avs doc??

[cretindesalpes]

Sierra-2-4A is comparable to Floyd-Steinberg, but slightly faster.

Ostromoukhov helps fighting the artefacts seen on Floyd-Steinberg, giving more homogenous flat areas. See also: A Simple and Efficient Error-Diffusion Algorithm (http://www.iro.umontreal.ca/~ostrom/publications/publications_abstracts.html#SIGGRAPH01_VarcoeffED)

[MonoS]

Ok thanks, i read the article [even i didn't understood it very well], i even tried to make a 2bit image to compare stucki and ostromoukhov, but vs doesn't allow me to output 2bit image afaik.

Then i tried to see the difference between the different ED methods and i was unable to see any difference in the LSB, but very little in the MSB [on a 16bit image decomposed using nativetostack16, with 8bit image i didn't saw any difference at all].

Probably i made my tests in an erroneous way, so i'd like to know your opinion if the difference are really so unnoticeable or very little differences can make an image looks better

I used this script for my tests and the first two exr frames from BBB
src = core.imwri.Read("C:/Users/MonoS/Desktop/%04d.exr", firstnum=1).fmtc.bitdepth(flt=1)

yuv = core.fmtc.matrix(src, mats="RGB", matd="709", bits=32, col_fam=vs.YUV)

core.std.MakeDiff(core.fmtc.bitdepth(yuv, flt=0, bits=16, dmode=7, fulls=False, fulld=False).fmtc.nativetostack16(),\
core.fmtc.bitdepth(yuv, flt=0, bits=16, dmode=4, fulls=False, fulld=False).fmtc.nativetostack16()).set_output()

[cretindesalpes]

You are trying to dither floating point data to 16 bits, the difference between dithering methods will be insignificant. Actually I don’t even bother dithering when converting down to 16 bits. If you want to see the difference, dither to 8 bits. And even at 8 bits, it will be quite hard to spot any quality difference. Only Bayer dithering can really stand out on low-res pictures because of its specific pattern.

[jackoneill]

Try something like this:

import vapoursynth as vs

c = vs.get_core()

c.std.LoadPlugin(".libs/vszimg.so")

shades = []

fmt = c.register_format(color_family=vs.RGB, sample_type=vs.INTEGER, bits_per_sample=14, subsampling_w=0, subsampling_h=0)

for i in range(1200):
shades.append(c.std.BlankClip(width=1, height=120, format=fmt.id, color=[i] * 3, length=1))

src = c.std.StackHorizontal(shades)

none = c.z.Depth(src, depth=8, dither="none").text.Text("none")
ordered = c.z.Depth(src, depth=8, dither="ordered").text.Text("ordered")
random = c.z.Depth(src, depth=8, dither="random").text.Text("random")
ed = c.z.Depth(src, depth=8, dither="error_diffusion").text.Text("error diffusion")

src = c.std.StackVertical([none, ordered, random, ed])

src.set_output()

With some minor changes, you can replace z.Depth with fmtc.bitdepth. The differences between the dithering methods will be visible.

[MonoS]

You are trying to dither floating point data to 16 bits, the difference between dithering methods will be insignificant. Actually I don’t even bother dithering when converting down to 16 bits. If you want to see the difference, dither to 8 bits. And even at 8 bits, it will be quite hard to spot any quality difference. Only Bayer dithering can really stand out on low-res pictures because of its specific pattern.

At least with BBB i saw no difference at all using 8bit, so i preferred comparing the 16bit version [redoing it using hist.Luma i can see some differences].

So in the end, if i understood correctly, it's better to use a compression resilient algo if i really need to dither down and only for 8bit output [but i usually pipe the 16bit video directly to x264/5], at 16bit differences are almost unnoticeable and i can use the one i like most.

If so, why bother implementing 7 different algos??

[MonoS]

Thanks jackoneill, i was able to see some difference between the different algos, my favorite remain the stucki ED, it seems pretty similar to Ostromoukhov and Sierra 2-4A, but stucki seemed to be a bit more random in adding dithering and maybe is a plus, the other ones seemed to generate some artifact [like the worm like "artifact" of Atkinson].

Are my observation right in some sense??

[metyo]

Hi everybody,
I made a fresh install to try the vapoursynth plus HAvsFunc.
the compiling from git was success.
When i try to use the fmtc functions it gives Assertion failed:

vspipe --y4m zod.vpy - | ffplay -i pipe:
-----
vspipe: ./conc/Interlocked.hpp:181: static int64_t conc::Interlocked::cas(volatile int64_t&, int64_t, int64_t): Assertion `is_ptr_aligned_nz (&dest)' failed
----

archlinux 32bit, ubuntu 32bit as well

my script is simple :
-----------------------------------
import vapoursynth as vs
core = vs.get_core()
c = core.std.BlankClip(width=100, height=200, format=vs.YUV420P8, color=[64, 64, 64])
c = core.fmtc.resample(c, w=100, h=100)
c.set_output()
------------------------------

when I leave the original size :
c = core.fmtc.resample(c, w=100, h=200)
I see the green box


what can I miss ?

thanx -- metyo

/* EDIT : AchLinux x86_64 , packages from AUR everything works ok ! thank sl1pkn07 for maintaning */

[cretindesalpes]

fmtconv r9 (http://forum.doom9.org/showthread.php?t=166504):
Added the transfer function.
resample: Most kernel-related parameters are now arrays, allowing to specify different values for each plane.
resample: Allows horizontal and vertical values for taps and invkstap.
resample: AVX2 optimizations.
matrix: Tries to deduce the target colorspace from simple matrix settings
matrix: Sets the new _Matrix frame property
matrix: AVX2 optimizations.
bitdepth: SSE2 optimizations for the “fast” algorithm.

[sl1pkn07]

fail build on linux (gcc 4.9.x)

http://sl1pkn07.wtf/paste/view/bcfcec5a

also tested with -std=c++11: same resoult

greetings

[RTW47]

getting "vapoursynth.Error: Core only supports API R3.1 but the loaded plugin requires API R3.2;" message, then trying to load fmtconv-r9 [log (http://pastebin.com/raw.php?i=jJ8SGqZD)]

[Myrsloik]

getting "vapoursynth.Error: Core only supports API R3.1 but the loaded plugin requires API R3.2;" message, then trying to load fmtconv-r9 [log (http://pastebin.com/raw.php?i=jJ8SGqZD)]

I'll post a new VapourSynth build soon that it works with.

[cretindesalpes]

Uh I didn’t realized that I used an unreleased VS version, sorry for that…

sl1pkn07:
Try to compile the *_avx.cpp and *_avx2.cpp files with the -mavx2 option.
I think you should #include <cstddef> in ToolsAvx2.hpp too.
However for the “error: inlining failed in call to always_inline (…) target specific option mismatch”, I really don’t know what to do. I have the same kind of errors with G++ on Cygwin64, but on different intrinsics (_mm_min_epi32 and _mm_max_epi32).

[sl1pkn07]

and if my CPU not support avx2 instrucctions? (Westmere-EP)

greetings

[cretindesalpes]

This shouldn’t be a problem. Instruction sets are checked at runtime before mapping the functions.

EDIT to reply to your edit: Add
#include <cstddef>
to fstb/ToolsAvx2.hpp and it should compile better.

EDIT2: beware, the -mavx2 should not be added to all files, only the avx ones.

[sl1pkn07]

doh. sorry my edit :( (the 'sed' is applied bad)


EDIT2: beware, the -mavx2 should not be added to all files, only the avx ones.


how get this?, my ""makefile"" is:


all:
g++ -shared ${CXXFLAGS} -msse2 -mavx2 -fPIC -std=c++11 -o lib${_plug}.so AvstpWrapper.cpp main.cpp fmtcl/*.cpp fmtc/*.cpp fstb/*.cpp vsutl/*.cpp -I. $(pkg-config --cflags vapoursynth)


greetings

EDIT: like this? http://sl1pkn07.wtf/paste/view/33b6cfc7

[cretindesalpes]

fmtconv r10 (http://forum.doom9.org/showthread.php?t=166504) should fix these compilation problems and allow using previous Vapoursynth versions.

sl1pkn07: You have to compile each file (or set of files) separately, then link everything. I’m not good at writing makefiles because of the dependencies to maintain, or you need makefile generators. Anyway when I want to test the G++ compilation on Cygwin, I use a script like this (http://sl1pkn07.wtf/paste/view/9b437617).

[sl1pkn07]

oh. thanks! (sorry for all, but im not a ceder, I only a user)

now build ok


greetings

EDIT: fail load


import vapoursynth as vs
core = vs.get_core()
video = core.lsmas.LWLibavSource("00000.MTS")
video = core.fmtc.resample(video, 1280, 720)
video.set_output()


└───╼ vspipe --info 00000.vpy -
vspipe: symbol lookup error: vspipe: symbol lookup error: /usr/lib/vapoursynth/libfmtconv.so: undefined symbol: _ZN4fstb18conv_to_lower_caseERSs


EDIT2: edit output with 8ca84c (https://github.com/vapoursynth/vapoursynth/commit/8ca84c1bd25aa98ad183ef47deae56171685fd4b) VS

[cretindesalpes]

sl1pkn07: Some files located in different directories have the same name, hence generating the same object files. The compiler will overwrite them if you put all the object files in the same directory with their default name. You should put them in different directories or rename the object files.

Also, I fixed an SSE2 bug in fmtconv r11 (http://forum.doom9.org/showthread.php?t=166504).

EDIT: I previously tried to include a build tool but had to give up due to complexity and lack of motivation (I tried gyp and autotools). If someone has an idea how to do it without generating a mess in the source tree and the VC++ projects, easy to update and to test on Windows, feel free to help.

[feisty2]

how about adding RGBToXYZ and XYZToRGB to do a gamut (primaries) convert?

[cretindesalpes]

feisty2: Gamut conversion is planned.

[feisty2]

feisty2: Gamut conversion is planned.

music to my ears :)

[sl1pkn07]

sl1pkn07: Some files located in different directories have the same name, hence generating the same object files. The compiler will overwrite them if you put all the object files in the same directory with their default name. You should put them in different directories or rename the object files.


oh. :S thanks again for the info

now build and fix my issue

greetings

[foxyshadis]

EDIT: I previously tried to include a build tool but had to give up due to complexity and lack of motivation (I tried gyp and autotools). If someone has an idea how to do it without generating a mess in the source tree and the VC++ projects, easy to update and to test on Windows, feel free to help.

x265 uses CMake, and it's turned out to be the easiest damn project/makefile generator I've ever used. As complex and as many build options as x265 has, the CMake files are still small and readable; for you it should be as simple as keeping an up-to-date list of new source files.

[cretindesalpes]

HolyWu: thanks for the report. fmtconv r12 (http://forum.doom9.org/showthread.php?t=166504) fixes the crash.

foxyshadis: thanks for the suggestion. I’ll check CMake too.

[djcj]

I can't compile the following files on Linux: main.cpp AvstpFinder.cpp AvstpWrapper.cpp fmtc/Convert.cpp fmtc/Resample.cpp fmtc/Transfer.cpp fmtcl/FilterResize.cpp fmtcl/Scaler.cpp
Was anyone able to build this plugin on Linux yet?

[Are_]

https://github.com/dubhater/fmtconv with autotools or you can use this cheap script compile.sh (https://paste.kde.org/pzaox1x6n)

Notice that repo is not up to date, so you would prefer to just grab the build directory from there.

[sl1pkn07]

all:

g++ -c -fPIC -std=c++11 -mavx2 -o fmtc/Matrix_avx.o fmtc/Matrix_avx.cpp
g++ -c -fPIC -std=c++11 -mavx2 -o fmtc/Transfer_avx2.o fmtc/Transfer_avx2.cpp
g++ -c -fPIC -std=c++11 -mavx2 -o fmtcl/BitBltConv_avx2.o fmtcl/BitBltConv_avx2.cpp
g++ -c -fPIC -std=c++11 -mavx2 -o fmtcl/Scaler_avx2.o fmtcl/Scaler_avx2.cpp
g++ -c -fPIC -std=c++11 -mavx2 -o fstb/ToolsAvx2.o fstb/ToolsAvx2.cpp

g++ -c -fPIC -std=c++11 -msse2 -o fmtc/Bitdepth.o fmtc/Bitdepth.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtc/ConvStep.o fmtc/ConvStep.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtc/Convert.o fmtc/Convert.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtc/Matrix.o fmtc/Matrix.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtc/Matrix2020CL.o fmtc/Matrix2020CL.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtc/NativeToStack16.o fmtc/NativeToStack16.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtc/Resample.o fmtc/Resample.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtc/Stack16ToNative.o fmtc/Stack16ToNative.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtc/Transfer.o fmtc/Transfer.cpp

g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/BitBltConv.o fmtcl/BitBltConv.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ChromaPlacement.o fmtcl/ChromaPlacement.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/CoefArrInt.o fmtcl/CoefArrInt.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ContFirBlackman.o fmtcl/ContFirBlackman.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ContFirBlackmanMinLobe.o fmtcl/ContFirBlackmanMinLobe.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ContFirCubic.o fmtcl/ContFirCubic.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ContFirFromDiscrete.o fmtcl/ContFirFromDiscrete.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ContFirGauss.o fmtcl/ContFirGauss.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ContFirInterface.o fmtcl/ContFirInterface.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ContFirLanczos.o fmtcl/ContFirLanczos.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ContFirLinear.o fmtcl/ContFirLinear.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ContFirRect.o fmtcl/ContFirRect.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ContFirSinc.o fmtcl/ContFirSinc.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ContFirSnh.o fmtcl/ContFirSnh.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ContFirSpline.o fmtcl/ContFirSpline.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ContFirSpline16.o fmtcl/ContFirSpline16.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ContFirSpline36.o fmtcl/ContFirSpline36.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ContFirSpline64.o fmtcl/ContFirSpline64.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/DiscreteFirCustom.o fmtcl/DiscreteFirCustom.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/DiscreteFirInterface.o fmtcl/DiscreteFirInterface.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ErrDifBuf.o fmtcl/ErrDifBuf.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ErrDifBufFactory.o fmtcl/ErrDifBufFactory.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/FilterResize.o fmtcl/FilterResize.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/KernelData.o fmtcl/KernelData.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ResampleSpecPlane.o fmtcl/ResampleSpecPlane.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ResizeData.o fmtcl/ResizeData.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/ResizeDataFactory.o fmtcl/ResizeDataFactory.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fmtcl/Scaler.o fmtcl/Scaler.cpp

g++ -c -fPIC -std=c++11 -msse2 -o fstb/CpuId.o fstb/CpuId.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fstb/ToolsSse2.o fstb/ToolsSse2.cpp
g++ -c -fPIC -std=c++11 -msse2 -o fstb/fnc.o fstb/fnc.cpp

g++ -c -fPIC -std=c++11 -msse2 -o vsutl/FilterBase.o vsutl/FilterBase.cpp
g++ -c -fPIC -std=c++11 -msse2 -o vsutl/PlaneProcCbInterface.o vsutl/PlaneProcCbInterface.cpp
g++ -c -fPIC -std=c++11 -msse2 -o vsutl/PlaneProcessor.o vsutl/PlaneProcessor.cpp
g++ -c -fPIC -std=c++11 -msse2 -o vsutl/fnc.o vsutl/fnc.cpp

g++ -c -fPIC -std=c++11 -msse2 -o AvstpWrapper.o AvstpWrapper.cpp
g++ -c -fPIC -std=c++11 -msse2 -o main.o main.cpp

g++ -shared -fPIC -std=c++11 -o libfmtconv.so fmtc/*.o fmtcl/*.o fstb/*.o vsutl/*.o *.o "


save this with name 'Makefile' in the 'src' dir and type 'make' in console

waiting to merge dubhater branch to mirsloik branch (or release with "stable" tarball)

[cretindesalpes]

djcj: AvstpFinder should be build only on Windows (please refer to fmtconv/doc/fmtconv.html#compiling). For the other files, what are the errors?

[djcj]

I got a lot of errors like these: error: ‘__m256’ does not name a type

But compiling the files that refused to build with -mavx2 instead of -msse2 solved it for me.

Here's my modified Makefile.am:
warningflags = -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers
includeflags = -I$(srcdir)/src
commonflags = -O3 $(MFLAGS) $(warningflags) $(includeflags)
AM_CXXFLAGS = -std=c++11 $(commonflags)

lib_LTLIBRARIES = libfmtconv.la

libfmtconv_la_SOURCES = \
src/fmtc/Bitdepth.cpp \
src/fmtc/ConvStep.cpp \
src/fmtcl/BitBltConv.cpp \
src/fmtcl/ChromaPlacement.cpp \
src/fmtcl/CoefArrInt.cpp \
src/fmtcl/ContFirBlackman.cpp \
src/fmtcl/ContFirBlackmanMinLobe.cpp \
src/fmtcl/ContFirCubic.cpp \
src/fmtcl/ContFirFromDiscrete.cpp \
src/fmtcl/ContFirGauss.cpp \
src/fmtcl/ContFirInterface.cpp \
src/fmtcl/ContFirLanczos.cpp \
src/fmtcl/ContFirLinear.cpp \
src/fmtcl/ContFirRect.cpp \
src/fmtcl/ContFirSinc.cpp \
src/fmtcl/ContFirSnh.cpp \
src/fmtcl/ContFirSpline16.cpp \
src/fmtcl/ContFirSpline36.cpp \
src/fmtcl/ContFirSpline64.cpp \
src/fmtcl/ContFirSpline.cpp \
src/fmtcl/DiscreteFirCustom.cpp \
src/fmtcl/DiscreteFirInterface.cpp \
src/fmtcl/ErrDifBuf.cpp \
src/fmtcl/ErrDifBufFactory.cpp \
src/fmtcl/KernelData.cpp \
src/fmtcl/ResampleSpecPlane.cpp \
src/fmtcl/ResizeData.cpp \
src/fmtcl/ResizeDataFactory.cpp \
src/fmtc/Matrix2020CL.cpp \
src/fmtc/Matrix.cpp \
src/fmtc/NativeToStack16.cpp \
src/fmtc/Stack16ToNative.cpp \
src/fstb/CpuId.cpp \
src/fstb/fnc.cpp \
src/fstb/ToolsSse2.cpp \
src/vsutl/FilterBase.cpp \
src/vsutl/fnc.cpp \
src/vsutl/PlaneProcCbInterface.cpp \
src/vsutl/PlaneProcessor.cpp

#if WINDOWS
#libfmtconv_la_SOURCES += src/AvstpFinder.cpp src/AvstpWrapper.cpp
#endif

libfmtconv_la_LDFLAGS = -no-undefined -avoid-version $(PLUGINLDFLAGS)


noinst_LTLIBRARIES = libavx2.la

libavx2_la_SOURCES = \
src/fmtc/Convert.cpp \
src/fmtc/Matrix_avx.cpp \
src/fmtc/Resample.cpp \
src/fmtc/Transfer.cpp \
src/fmtc/Transfer_avx2.cpp \
src/fmtcl/BitBltConv_avx2.cpp \
src/fmtcl/FilterResize.cpp \
src/fmtcl/Scaler.cpp \
src/fmtcl/Scaler_avx2.cpp \
src/fstb/ToolsAvx2.cpp \
src/main.cpp

libavx2_la_CXXFLAGS = $(AM_CXXFLAGS) -mavx2


libfmtconv_la_LIBADD = libavx2.la


edit:

Since the AvstpFinder stuff is only meant for Windows, a quick patch that avoids undefined references on Unix systems:
diff --git a/fmtconv/src/fmtcl/FilterResize.cpp b/fmtconv/src/fmtcl/FilterResize.cpp
index e5b66a7..fc07af4 100644
--- a/fmtconv/src/fmtcl/FilterResize.cpp
+++ b/fmtconv/src/fmtcl/FilterResize.cpp
@@ -61,8 +61,12 @@ namespace fmtcl


FilterResize::FilterResize (const ResampleSpecPlane &spec, ContFirInterface &kernel_fnc_h, ContFirInterface &kernel_fnc_v, bool norm_flag, double norm_val_h, double norm_val_v, double gain, SplFmt src_type, int src_res, SplFmt dst_type, int dst_res, bool int_flag, bool sse2_flag, bool avx2_flag)
+#if defined(__USE_AVSTP)
: _avstp (AvstpWrapper::use_instance ())
, _task_rsz_pool ()
+#else
+: _task_rsz_pool ()
+#endif
/*, _src_size ()
, _dst_size ()
, _win_pos ()
@@ -517,7 +521,9 @@ void FilterResize::process_plane_normal (uint8_t *dst_msb_ptr, uint8_t *dst_lsb_
assert (stride_dst > 0);
assert (stride_src > 0);

+#if defined(__USE_AVSTP)
avstp_TaskDispatcher * task_dispatcher_ptr = _avstp.create_dispatcher ();
+#endif

TaskRszGlobal trg;
trg._this_ptr = this;
@@ -612,19 +618,25 @@ void FilterResize::process_plane_normal (uint8_t *dst_msb_ptr, uint8_t *dst_lsb_
tr._work_dst [d] = work_dst [d];
}

+#if defined(__USE_AVSTP)
_avstp.enqueue_task (
task_dispatcher_ptr,
&redirect_task_resize,
tr_cell_ptr
);
+#endif
} // for Dir_H
} // for Dir_V

+#if defined(__USE_AVSTP)
_avstp.wait_completion (task_dispatcher_ptr);
+#endif

// Done
+#if defined(__USE_AVSTP)
_avstp.destroy_dispatcher (task_dispatcher_ptr);
task_dispatcher_ptr = 0;
+#endif
}


diff --git a/fmtconv/src/fmtcl/FilterResize.h b/fmtconv/src/fmtcl/FilterResize.h
index f49e280..c9a0e2f 100644
--- a/fmtconv/src/fmtcl/FilterResize.h
+++ b/fmtconv/src/fmtcl/FilterResize.h
@@ -42,6 +42,10 @@ http://sam.zoy.org/wtfpl/COPYING for more details.

#include <cstdint>

+#if defined(__CYGWIN32__) || defined(__CYGWIN__) || defined(__MINGW32__) || defined(__MINGW64__) || defined(_MSC_VER) || \
+defined(__WIN32) || defined(__WIN32__) || defined(_WIN32) || defined(__WIN64) || defined(__WIN64__) || defined(_WIN64)
+# define __USE_AVSTP
+#endif


namespace fmtcl
@@ -158,7 +162,9 @@ private:

static void redirect_task_resize (avstp_TaskDispatcher *dispatcher_ptr, void *data_ptr);

+#if defined(__USE_AVSTP)
AvstpWrapper & _avstp;
+#endif
conc::CellPool <TaskRsz>
_task_rsz_pool;

[jeremy33]

Hello,

Jackoneill and me found a bug : http://forum.doom9.org/showthread.php?p=1722246#post1722246

We found that if you use something like sx=-0.5, sy=-0.5 (eg. clip = c.fmtc.resample(clip, kernel="spline36", sx=-0.5, sy=-0.5)) the image is really bad :
http://img11.hostingpics.net/thumbs/mini_464396screenshot.jpg (http://www.hostingpics.net/viewer.php?id=464396screenshot.jpg)

[jackoneill]

Hello,

Jackoneill and me found a bug : http://forum.doom9.org/showthread.php?p=1722246#post1722246

We found that if you use something like sx=-0.5, sy=-0.5 (eg. clip = c.fmtc.resample(clip, kernel="spline36", sx=-0.5, sy=-0.5)) the image is really bad :
http://img11.hostingpics.net/thumbs/mini_464396screenshot.jpg (http://www.hostingpics.net/viewer.php?id=464396screenshot.jpg)

It does not happen with r12 (or r8) on a Core 2 Duo.

[jeremy33]

It does not happen with r12 (or r8) on a Core 2 Duo.
Thanks for the info, so I build it myself and it works!
False alarm ;)

[jackoneill]

Not sure if related to any crashes or garbled output, but I found these out-of-bounds reads:

==13632== Thread 4:
==13632== Invalid read of size 8
==13632== at 0xD64BBC1: _mm_set_epi64x (emmintrin.h:583)
==13632== by 0xD64BBC1: _mm_set_epi64 (emmintrin.h:589)
==13632== by 0xD64BBC1: _mm_loadl_epi64 (emmintrin.h:700)
==13632== by 0xD64BBC1: load_8_16l (ToolsSse2.hpp:93)
==13632== by 0xD64BBC1: read_i16 (ProxyRwSse.hpp:70)
==13632== by 0xD64BBC1: void fmtcl::BitBltConv::bitblt_ixx_to_x16_sse<fmtcl::ProxyRwSse<(fmtcl::SplFmt)1>, fmtcl::ProxyRwSse<(fmtcl::SplFmt)3>, 16, 8>(fmtcl::ProxyRwSse<(fmtcl::SplFmt)1>::Ptr::Type, int, fmtcl::ProxyRwSse<(fmtcl::SplFmt)3>::PtrConst::Type, int, int, int) (BitBltConv.cpp:808)
==13632== by 0xD6541C2: void fmtcl::FilterResize::process_tile_transpose<unsigned short, (fmtcl::SplFmt)1>(fmtcl::FilterResize::TaskRsz const&, fmtcl::FilterResize::TaskRszGlobal const&, fmtcl::ResizeData&, int*, int, fmtcl::FilterResize::Dir&, int&, int*) (FilterResize.cpp:1002)
==13632== by 0xD652C37: fmtcl::FilterResize::process_tile(conc::LockFreeCell<fmtcl::FilterResize::TaskRsz>&) (FilterResize.cpp:691)
==13632== by 0xD5DD75F: AvstpWrapper::fallback_enqueue_task_ptr(avstp::TaskDispatcher*, void (*)(avstp::TaskDispatcher*, void*), void*) (AvstpWrapper.cpp:272)
==13632== by 0xD653186: fmtcl::FilterResize::process_plane_normal(unsigned char*, unsigned char*, unsigned char const*, unsigned char const*, int, int) (FilterResize.cpp:619)
==13632== by 0xD688035: fmtc::Resample::process_plane_proc(VSFrameRef&, int, int, void*, VSFrameContext&, VSCore&, vsutl::ObjRefSPtr<VSNodeRef, &VSAPI::cloneNodeRef, &VSAPI::freeNode> const&) (Resample.cpp:842)
==13632== by 0xD6881DA: fmtc::Resample::do_process_plane(VSFrameRef&, int, int, void*, VSFrameContext&, VSCore&, vsutl::ObjRefSPtr<VSNodeRef, &VSAPI::cloneNodeRef, &VSAPI::freeNode> const&, vsutl::ObjRefSPtr<VSNodeRef, &VSAPI::cloneNodeRef, &VSAPI::freeNode> const&, vsutl::ObjRefSPtr<VSNodeRef, &VSAPI::cloneNodeRef, &VSAPI::freeNode> const&) (Resample.cpp:622)
==13632== by 0xD698A93: vsutl::PlaneProcessor::process_frame(VSFrameRef&, int, void*, VSFrameContext&, VSCore&, vsutl::ObjRefSPtr<VSNodeRef, &VSAPI::cloneNodeRef, &VSAPI::freeNode>, vsutl::ObjRefSPtr<VSNodeRef, &VSAPI::cloneNodeRef, &VSAPI::freeNode>, vsutl::ObjRefSPtr<VSNodeRef, &VSAPI::cloneNodeRef, &VSAPI::freeNode>) (PlaneProcessor.cpp:283)
==13632== by 0xD687633: fmtc::Resample::get_frame(int, int, void*&, VSFrameContext&, VSCore&) (Resample.cpp:490)
==13632== by 0x6ED8827: VSNode::getFrameInternal(int, int, VSFrameContext&) (in /usr/lib/libvapoursynth.so)
==13632== by 0x6EE6E08: VSThreadPool::runTasks(VSThreadPool*, std::atomic<bool>&) (in /usr/lib/libvapoursynth.so)
==13632== by 0x50F4DEF: execute_native_thread_routine (in /usr/lib/libstdc++.so.6.0.20)
==13632== Address 0x199c6f7d is 307,197 bytes inside a block of size 307,200 alloc'd
==13632== at 0x4C2C526: memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==13632== by 0x4C2C641: posix_memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==13632== by 0x6ED7BBD: VSPlaneData::VSPlaneData(unsigned long, MemoryUse&) (in /usr/lib/libvapoursynth.so)
==13632== by 0x6ED9955: VSFrame::VSFrame(VSFormat const*, int, int, VSFrame const*, VSCore*) (in /usr/lib/libvapoursynth.so)
==13632== by 0x6ED9C56: VSCore::newVideoFrame(VSFormat const*, int, int, VSFrame const*) (in /usr/lib/libvapoursynth.so)
==13632== by 0x6ED05EC: newVideoFrame(VSFormat const*, int, int, VSFrameRef const*, VSCore*) (in /usr/lib/libvapoursynth.so)
==13632== by 0xD93B664: ??? (in /usr/lib/libffms2.so.3.0.0)
==13632== by 0x6ED8827: VSNode::getFrameInternal(int, int, VSFrameContext&) (in /usr/lib/libvapoursynth.so)
==13632== by 0x6EE6E08: VSThreadPool::runTasks(VSThreadPool*, std::atomic<bool>&) (in /usr/lib/libvapoursynth.so)
==13632== by 0x50F4DEF: execute_native_thread_routine (in /usr/lib/libstdc++.so.6.0.20)
==13632== by 0x60A2373: start_thread (in /usr/lib/libpthread-2.21.so)
==13632== by 0x564727C: clone (in /usr/lib/libc-2.21.so)
==13632==
==13632== Invalid read of size 8
==13632== at 0xD65FE82: _mm_set_epi64x (emmintrin.h:583)
==13632== by 0xD65FE82: _mm_set_epi64 (emmintrin.h:589)
==13632== by 0xD65FE82: _mm_loadl_epi64 (emmintrin.h:700)
==13632== by 0xD65FE82: load_8_16l (ToolsSse2.hpp:93)
==13632== by 0xD65FE82: read (ProxyRwSse.hpp:118)
==13632== by 0xD65FE82: process_vect_int_sse2<fmtcl::ProxyRwSse<(fmtcl::SplFmt)1>, 16, fmtcl::ProxyRwSse<(fmtcl::SplFmt)3>, 8> (Scaler.cpp:756)
==13632== by 0xD65FE82: void fmtcl::Scaler::process_plane_int_sse2<fmtcl::ProxyRwSse<(fmtcl::SplFmt)1>, 16, fmtcl::ProxyRwSse<(fmtcl::SplFmt)3>, 8>(fmtcl::ProxyRwSse<(fmtcl::SplFmt)1>::Ptr::Type, fmtcl::ProxyRwSse<(fmtcl::SplFmt)3>::PtrConst::Type, int, int, int, int, int) const (Scaler.cpp:688)
==13632== by 0xD64F836: fmtcl::FilterResize::process_tile_resize(fmtcl::FilterResize::TaskRsz const&, fmtcl::FilterResize::TaskRszGlobal const&, fmtcl::ResizeData&, int*, int, fmtcl::FilterResize::Dir&, int&, int*) (FilterResize.cpp:901)
==13632== by 0xD652CC1: fmtcl::FilterResize::process_tile(conc::LockFreeCell<fmtcl::FilterResize::TaskRsz>&) (FilterResize.cpp:683)
==13632== by 0xD5DD75F: AvstpWrapper::fallback_enqueue_task_ptr(avstp::TaskDispatcher*, void (*)(avstp::TaskDispatcher*, void*), void*) (AvstpWrapper.cpp:272)
==13632== by 0xD653186: fmtcl::FilterResize::process_plane_normal(unsigned char*, unsigned char*, unsigned char const*, unsigned char const*, int, int) (FilterResize.cpp:619)
==13632== by 0xD688035: fmtc::Resample::process_plane_proc(VSFrameRef&, int, int, void*, VSFrameContext&, VSCore&, vsutl::ObjRefSPtr<VSNodeRef, &VSAPI::cloneNodeRef, &VSAPI::freeNode> const&) (Resample.cpp:842)
==13632== by 0xD6881DA: fmtc::Resample::do_process_plane(VSFrameRef&, int, int, void*, VSFrameContext&, VSCore&, vsutl::ObjRefSPtr<VSNodeRef, &VSAPI::cloneNodeRef, &VSAPI::freeNode> const&, vsutl::ObjRefSPtr<VSNodeRef, &VSAPI::cloneNodeRef, &VSAPI::freeNode> const&, vsutl::ObjRefSPtr<VSNodeRef, &VSAPI::cloneNodeRef, &VSAPI::freeNode> const&) (Resample.cpp:622)
==13632== by 0xD698A93: vsutl::PlaneProcessor::process_frame(VSFrameRef&, int, void*, VSFrameContext&, VSCore&, vsutl::ObjRefSPtr<VSNodeRef, &VSAPI::cloneNodeRef, &VSAPI::freeNode>, vsutl::ObjRefSPtr<VSNodeRef, &VSAPI::cloneNodeRef, &VSAPI::freeNode>, vsutl::ObjRefSPtr<VSNodeRef, &VSAPI::cloneNodeRef, &VSAPI::freeNode>) (PlaneProcessor.cpp:283)
==13632== by 0xD687633: fmtc::Resample::get_frame(int, int, void*&, VSFrameContext&, VSCore&) (Resample.cpp:490)
==13632== by 0x6ED8827: VSNode::getFrameInternal(int, int, VSFrameContext&) (in /usr/lib/libvapoursynth.so)
==13632== by 0x6EE6E08: VSThreadPool::runTasks(VSThreadPool*, std::atomic<bool>&) (in /usr/lib/libvapoursynth.so)
==13632== by 0x50F4DEF: execute_native_thread_routine (in /usr/lib/libstdc++.so.6.0.20)
==13632== Address 0x1b1204bc is 691,196 bytes inside a block of size 691,200 alloc'd
==13632== at 0x4C2C526: memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==13632== by 0x4C2C641: posix_memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==13632== by 0x6ED7BBD: VSPlaneData::VSPlaneData(unsigned long, MemoryUse&) (in /usr/lib/libvapoursynth.so)
==13632== by 0x6ED8DDB: VSFrame::VSFrame(VSFormat const*, int, int, VSFrame const* const*, int const*, VSFrame const*, VSCore*) (in /usr/lib/libvapoursynth.so)
==13632== by 0x6ED8FFD: VSCore::newVideoFrame(VSFormat const*, int, int, VSFrame const* const*, int const*, VSFrame const*) (in /usr/lib/libvapoursynth.so)
==13632== by 0x6ED0372: newVideoFrame2(VSFormat const*, int, int, VSFrameRef const**, int const*, VSFrameRef const*, VSCore*) (in /usr/lib/libvapoursynth.so)
==13632== by 0x6EB6FBC: makeDiffGetFrame (in /usr/lib/libvapoursynth.so)
==13632== by 0x6ED8827: VSNode::getFrameInternal(int, int, VSFrameContext&) (in /usr/lib/libvapoursynth.so)
==13632== by 0x6EE6E08: VSThreadPool::runTasks(VSThreadPool*, std::atomic<bool>&) (in /usr/lib/libvapoursynth.so)
==13632== by 0x50F4DEF: execute_native_thread_routine (in /usr/lib/libstdc++.so.6.0.20)
==13632== by 0x60A2373: start_thread (in /usr/lib/libpthread-2.21.so)
==13632== by 0x564727C: clone (in /usr/lib/libc-2.21.so)


This is fmtconv r12 running in 64 bit Arch Linux.

[cretindesalpes]

Thanks for the bug report. Do you know what is the input video format and the resize parameters?

[jackoneill]

Thanks for the bug report. Do you know what is the input video format and the resize parameters?

For the first one:

clip = VideoNode
Format: YUV420P8
Width: 640
Height: 480
Num Frames: 100
FPS Num: 30003
FPS Den: 1001
Flags: None
w = 960
h = 720
sx = 0.0
sy = 0.0
sw = 0.0
sh = 0.0
kernel = spline36
taps = 4
a1 = None
a2 = None
css = None
planes = [3, 3, 3]
cplace = mpeg2
cplaces = None
cplaced = None
interlaced = 2
interlacedd = 2
flt = False

For the second one:

clip = VideoNode
Format: Gray8
Width: 960
Height: 720
Num Frames: 100
FPS Num: 30003
FPS Den: 1001
Flags: None
w = 640
h = 480
sx = 0.0
sy = 0.0
sw = 0.0
sh = 0.0
kernel = spline36
taps = 4
a1 = None
a2 = None
css = None
planes = [3, 3, 3]
cplace = mpeg2
cplaces = None
cplaced = None
interlaced = 2
interlacedd = 2
flt = False

[cretindesalpes]

Hmmm… It seems that the allocated space corresponds exactly to the frame size. I’m just checking the VS source code and I see this VS_FRAME_GUARD which is not defined by default. Actually I always thought there was a few bytes left after the frame data to make SIMD code simpler (same as in Avisynth). But it seems that VS_FRAME_GUARD exists only for debugging purpose and strict frame bounds should be enforced. Sigh… so consider all the fmtconv releases broken at the moment.

[djcj]

Just want to report you that a build with debug level 2 or higher fails for me. Building with -g1 is no problem.
g++ -c -O2 -fPIC -Wall -Wextra -Wno-unused-parameter -Wno-unused-result -Isrc -g -Wno-reorder -std=c++11 -mavx2 -msse2 -march=native -pipe -I/usr/local/include/vapoursynth -I/usr/include/vapoursynth -o src/fmtc/Transfer.o src/fmtc/Transfer.cpp
src/fmtc/Transfer.cpp: In static member function ‘static void fmtc::Transfer::MapperLog::find_index(const fmtc::Transfer::FloatIntMix*, __m128i&, __m128&)’:
src/fmtc/Transfer.cpp:791:24: warning: unused variable ‘val_max’ [-Wunused-variable]
static const float val_max = float (int64_t (1) << LOGLUT_MAX_L2);
^
src/fmtc/Transfer.cpp: At global scope:
src/fmtc/Transfer.cpp:1547:1: error: ‘static void fmtc::Transfer::MapperLog::find_index(const fmtc::Transfer::FloatIntMix*, __m256i&, __m256&)’ conflicts with a previous declaration
} // namespace fmtc
^
src/fmtc/Transfer.cpp:780:6: note: previous declaration ‘static void fmtc::Transfer::MapperLog::find_index(const fmtc::Transfer::FloatIntMix*, __m128i&, __m128&)’
void Transfer::MapperLog::find_index (const FloatIntMix val_arr [4], __m128i &index, __m128 &frac)
^
In file included from src/fmtc/Transfer.cpp:31:0:
src/fmtc/Transfer.h:321:18: note: -fabi-version=6 (or =0) avoids this error with a change in mangling
find_index (const FloatIntMix val_arr [8], __m256i &index, __m256 &frac);
^
src/fmtc/Transfer.cpp:1547:1: error: ‘static void fmtc::Transfer::MapperLin::find_index(const fmtc::Transfer::FloatIntMix*, __m256i&, __m256&)’ conflicts with a previous declaration
} // namespace fmtc
^
src/fmtc/Transfer.cpp:689:6: note: previous declaration ‘static void fmtc::Transfer::MapperLin::find_index(const fmtc::Transfer::FloatIntMix*, __m128i&, __m128&)’
void Transfer::MapperLin::find_index (const FloatIntMix val_arr [4], __m128i &index, __m128 &frac)
^
In file included from src/fmtc/Transfer.cpp:31:0:
src/fmtc/Transfer.h:301:18: note: -fabi-version=6 (or =0) avoids this error with a change in mangling
find_index (const FloatIntMix val_arr [8], __m256i &index, __m256 &frac);
^
I'm using g++ 4.8.2 64 bit on Ubuntu 14.04.

[jackoneill]

-fabi-version=6 does take care of those errors.

[cretindesalpes]

fmtconv r13 (http://forum.doom9.org/showthread.php?t=166504):
matrix: optimized the SSE2 and AVX2 paths for integer data.
Added cpuopt to some functions, to manually limit the instruction set optimizations.
Added build files for the unix-like systems, thanks to jackoneill.
Fixed a buffer overflow bug in the SSE2 and AVX2 code of bitdepth and resample, thanks to jackoneill for reporting it.
Removed the int16tofloat and floattoint16 temporary functions.

[sl1pkn07]

Added build files for the unix-like systems, thanks to jackoneill.


missing

[cretindesalpes]

Oh right, fixed.

[sl1pkn07]

tnx bro!

[metyo]

funny

this module went wrong

vspipe: symbol lookup error: /usr/lib/vapoursynth/libfmtconv.so: undefined symbol: __sync_val_compare_and_swap_16

ArchLinux 64bit

metyo

[sl1pkn07]

mmm

try to commenting the line 16 in the pkgbuild

greetings

[jackoneill]

funny

this module went wrong

vspipe: symbol lookup error: /usr/lib/vapoursynth/libfmtconv.so: undefined symbol: __sync_val_compare_and_swap_16

ArchLinux 64bit

metyo

"-march=native" may fix that. Other values must work as well, but I don't know which. (Obviously you shouldn't use "native" if you're making a binary for other people.)

[metyo]

hi
you mean to disable the

rm -fr src/VapourSynth.h --- line ?

i used the vapoursynth-plugin-fmtconv r13-1 cos it was the more fresh..

info - i reinstalled the 20131130.ce9e577-1-x86_64.pkg.tar.xz, and it works as it did

[sl1pkn07]

please paste the script

edit: try to use the non-git package

[metyo]

the fmtconv is a mission for me always

the srcipt is


import vapoursynth as vs
core = vs.get_core()

c = core.std.BlankClip(width=100, height=200, format=vs.YUV420P16, color=[64, 64, 64])
c = core.fmtc.resample(c, w=200, h=200)
c.set_output()


so i think i don't miss too mutch:)

i use havsfunc - QTMGC , so the fmtc is a dependency.
But don't you spend too much free time of yours, the previous versions was working good...

edit

sorry the same
Csomagok (means packages hungarian..:):) ) (1) vapoursynth-plugin-fmtconv-r13-1

[sl1pkn07]

works for me with r11.6.f796ebd-1 and with non-git r13 with VS-git r27.7.ge00d0de-1

[cretindesalpes]

"-march=native" may fix that. Other values must work as well, but I don't know which. (Obviously you shouldn't use "native" if you're making a binary for other people.)
Actually, on x64 architectures, fmtconv needs the CMPXCHG16B instruction, which is lacking on early AMD CPUs (http://en.wikipedia.org/wiki/X86-64#Older_implementations). So you need to specify a more recent arch. For example -march=nocona should do the job. Maybe there are other ways to specify this constraint to GCC?

EDIT: just found the -mcx16 (https://gcc.gnu.org/onlinedocs/gcc-4.9.2/gcc/i386-and-x86-64-Options.html#i386-and-x86-64-Options) option which should be more appropriate.

[jackoneill]

EDIT: just found the -mcx16 (https://gcc.gnu.org/onlinedocs/gcc-4.9.2/gcc/i386-and-x86-64-Options.html#i386-and-x86-64-Options) option which should be more appropriate.

That works!

[lo1t3yu]

Compiled fmtconv doesn't work :)
Used fmtconv: R13 from git: https://github.com/EleonoreMizo/fmtconv
Due to this makefile and -mavx2 instead of -msse2 fmtconv successfully linked and compiled. Thanks to sl1pkn07! But if use fmtconv...
Vspipe breaks error:
vspipe: symbol lookup error: /usr/lib/vapoursynth/libfmtconv.so: undefined symbol: _ZN5vsutl6CpuOptC1ERNS_10FilterBaseERK5VSMapRS3_PKc

[cretindesalpes]

Unfortunately I cannot help with Linux builds, as I haven’t access to a Linux system at the moment. Anyway you should use -mavx2 only on *_avx2 files, as specified in the doc, unless you have an AVX2-capable CPU and don’t plan to share the library.

[lo1t3yu]

Exist different branches and forks of this plugin. What's the difference between them?

https://github.com/EleonoreMizo/fmtconv
https://github.com/vapoursynth/fmtconv
https://github.com/dubhater/fmtconv

[Myrsloik]

https://github.com/EleonoreMizo/fmtconv is the official one now.

[lo1t3yu]

Thanks to all, it works.
Full Makefile.am: https://github.com/dubhater/fmtconv/blob/master/build/unix/Makefile.am
Also need to use -march=native flag.

[cretindesalpes]

fmtconv r14 (http://forum.doom9.org/showthread.php?t=166504):
matrix: fixed a bug introducing wrong offsets in custom matrix coefficients, thanks to mawen1250 for the report.

[metyo]

It works for me too.

Compiled from non git version Archlinux 64 bit

[feisty2]

http://i.imgur.com/R9g6SVM.png
the following function gets me a weird line stuck in the middle (or lower middle? whatever) of the image, boxed out with a red rectangle

def cblur16 (src):
core = vs.get_core ()
w = src.width
h = src.height
blur = core.fmtc.resample (src, w*4, h*4, kernel="cubic", a1=1, a2=0, fulls=True, fulld=True)
sharp = core.fmtc.resample (src, w*4, h*4, kernel="cubic", a1=-1, a2=0, fulls=True, fulld=True)
dif = core.std.MakeDiff (blur, sharp)
dif = core.fmtc.resample (dif, w, h, kernel="gauss", a1=100, fulls=True, fulld=True)
clip = core.std.MergeDiff (src, dif)
return clip

[cretindesalpes]

feisty2: indeed. I think it’s fixed now.

fmtconv r15 (http://forum.doom9.org/showthread.php?t=166504):
resample and bitdepth: fixed a bug creating dark lines or weird patterns. Was introduced in r13 while trying to fix the buffer overflow problem. Thanks to feisty2 for spotting it.
resample: fixed the non-SIMD code path, causing crashes.

[mawen1250]

With core.fmtc.matrix(src, mat="709", col_fam=vs.RGB), I've got slightly brighter output if input is YUV444P16 (compared with YUV444PS input for fmtc.matrix and z.Colorspace).

EDIT:
More tests reveal that it's a problem related to vsedit when previewing floating point frames. When I manually convert output depth to 8bit, all the results are visually the same.

[jeremy33]

Hello,

I had a problem with fmtconv from the djcj PPA (https://launchpad.net/~djcj/+archive/ubuntu/vapoursynth) as you can see here:
http://forum.doom9.org/showthread.php?t=165771&page=81

We found that it was compiled incorrectly. So djcj now use the original Makefiles from "https://github.com/EleonoreMizo/fmtconv/tree/master/build/unix" to build fmtconv on the PPA but there is a problem.

The compilation works for Ubuntu 15.04 and the problem we had with fmtconv is gone but the compilation doesn't work with Ubuntu 14.04. Djcj opened an issue on github :

https://github.com/EleonoreMizo/fmtconv/issues/3

I hope that can be fixed.

Thank you

[cretindesalpes]

jeremy33: It looks like the errors come from the declarations using AVX types in compilation units limited to SSE2. Because they are only declarations, they have no consequence on the generated code and GCC 4.9+ accept them, but I agree it’s not very clean. I’ll see if I can refactor the whole AVX business.

[jeremy33]

Great, thank you very much !

[cretindesalpes]

I did the modification, pushed in the git repository. I am unable to run the unix build, so if someone could test it, any help is appreciated.

[Are_]

Tested, it builds without "march" with everything.

[sl1pkn07]

build for me with gcc5. (non AVX machine. -march=native)

[cretindesalpes]

Thanks for your report.

[jeremy33]

I did the modification, pushed in the git repository. I am unable to run the unix build, so if someone could test it, any help is appreciated.
Now it builds with g++-4.8, g++-4.9, clang++-3.5 and clang++-3.6 without problem.

Thank you very much !

[mawen1250]

I've got a problem with fmtc.resample:
When an array is specified for sx/sy, only the first one in the array is actually used.

[cretindesalpes]

Right. It should be fixed now. Thanks for the report.

fmtconv r16 (http://forum.doom9.org/showthread.php?t=166504):
bitdepth: added support for 11-bit and 14-bit integer input.
bitdepth: Fixed a slight plane inconsistency when dithering grey multi-plane pictures using an error diffusion algorithm.
matrix2020cl: added SSE2 optimisations for the floating point path.
resample: sx, sy, sw and sh parameters passed as arrays are now correctly taken into account.
transfer: added the blacklvl parameter.

[mawen1250]

Thanks for the update! It works correctly now.

[feisty2]

source
http://i.imgur.com/hxMRuOt.png

clp = xxx
clp = core.fmtc.bitdepth(clp, fulls=False, fulld=True, bits=32, flt=True)
clp = core.fmtc.transfer(clp, transs="709", transd="linear", fulls=True, fulld=True, cont=3)
clp = core.fmtc.transfer(clp, transs="linear", transd="709", fulls=True, fulld=True, cont=1/3)
clp.set_output ()

http://i.imgur.com/8w07oNl.png
looks like nothing ever happened, good


clp = xxx
clp = core.fmtc.bitdepth(clp, fulls=False, fulld=True, bits=32, flt=True)
clp = core.fmtc.transfer(clp, transs="709", transd="linear", fulls=True, fulld=True, cont=3)
clp = core.fmtc.bitdepth(clp, fulls=True, fulld=True, bits=16)
clp = core.fmtc.bitdepth(clp, fulls=True, fulld=True, bits=32)
clp = core.fmtc.transfer(clp, transs="linear", transd="709", fulls=True, fulld=True, cont=1/3)
clp.set_output ()

http://i.imgur.com/Znwhy9r.png
what the *beep*? looks totally wrong and miles away kind of different from the source clip, is it like, a bug, or more of, a failure of 16bits?
guess it's the latter one, 16bits failing, imho, told ya, 16bits sucks, 32bits rules!

EDIT: I'll keep doing "Placebo" (scripting sort of thing) if it's just a bug and wait it to be fixed
or, well, I'll try to find a way to stuff float support to nnedi3 and mvtools if it's 16bits sucking ass.

[cretindesalpes]

feisty2: After the x3 contrast operation, the luminance channel contains values > 1. They just get clipped when converting to integer data and cannot be restored to their original value when applying the 1/3 contrast. Nothing wrong here; keep working with floating point data when building such a processing graph.

[cretindesalpes]

fmtconv r17 (http://forum.doom9.org/showthread.php?t=166504):
bitdepth: added “Void and cluster” dithering method and its patsize parameter.
bitdepth: added floating point implementation for the Ostromoukhov dithering
bitdepth: added SSE2 optimizations for halftone modes (0, 1 and 8).
bitdepth: fixed incorrect conversion from float to 8-bit integer using the “fast” modes with SSE2 instruction set.

[jose1711]

i am still getting the following error:
$ LANG=C LC_ALL=C vspipe test.vpy stream.y4m --y4m & mpv stream.y4m
[1] 9942
Playing: stream.y4m
vspipe: ./../../src/conc/Interlocked.hpp:183: static int64_t conc::Interlocked::cas(volatile int64_t&, int64_t, int64_t): Assertion `is_ptr_al
igned_nz (&dest)' failed.
Failed to recognize file format.

the problem goes away as soon as i remove this line from test.vpy:

ret = haf.QTGMC( ret, Preset='Slow', TFF=True)


does it mean that this feature is limited to 64bit only processor/OS? i am on arch linux i686. thank you, jose

[cretindesalpes]

jose1711: I fixed something in the git repository, but at this point I’m not sure it fixes everything. Please check it and tell me if it works better, or differently.

[jose1711]

jose1711: I fixed something in the git repository, but at this point I’m not sure it fixes everything. Please check it and tell me if it works better, or differently.

thank you, i assume i was supposed to recompile vapoursynth-plugin-fmtconv-git. unfortunately, there was no change or improvement. this is the backtrace:

Program received signal SIGABRT, Aborted.
0xb7fdbbc8 in __kernel_vsyscall ()
(gdb) bt
#0 0xb7fdbbc8 in __kernel_vsyscall ()
#1 0xb7c67d66 in raise () from /usr/lib/libc.so.6
#2 0xb7c69397 in abort () from /usr/lib/libc.so.6
#3 0xb7c60e07 in __assert_fail_base () from /usr/lib/libc.so.6
#4 0xb7c60e8b in __assert_fail () from /usr/lib/libc.so.6
#5 0xb720b8f9 in conc::LockFreeStack<fmtcl::ErrDifBuf*>::push(conc::LockFreeCell<fmtcl::ErrDifBuf*>&) ()
from /usr/lib/vapoursynth/libfmtconv.so
#6 0xb720bade in conc::CellPool<fmtcl::ErrDifBuf*>::allocate_zone(int, unsigned int, conc::AtomicPtr<conc::LockFreeCell<fmtcl::ErrDifBuf*> >&) () from /usr/lib/vapoursynth/libfmtconv.so
#7 0xb720bc4a in conc::CellPool<fmtcl::ErrDifBuf*>::expand_to(unsigned int) () from /usr/lib/vapoursynth/libfmtconv.so
#8 0xb715aa4f in fmtc::Bitdepth::Bitdepth(VSMap const&, VSMap&, void*, VSCore&, VSAPI const&) () from /usr/lib/vapoursynth/libfmtconv.so
#9 0xb714bcf4 in vsutl::Redirect<fmtc::Bitdepth>::create(VSMap const*, VSMap*, void*, VSCore*, VSAPI const*) ()
from /usr/lib/vapoursynth/libfmtconv.so
#10 0xb754734a in VSPlugin::invoke(std::string const&, VSMap const&) () from /usr/lib/libvapoursynth.so
#11 0xb7534bb2 in invoke(VSPlugin*, char const*, VSMap const*) () from /usr/lib/libvapoursynth.so
#12 0xb7605ebe in __pyx_pw_11vapoursynth_8Function_3__call__ () from /usr/lib/python3.4/site-packages/vapoursynth.so
#13 0xb79dc973 in PyObject_Call (func=0xb2aa8874, arg=0xb76ed02c, kw=0xb7645cec) at Objects/abstract.c:2040
#14 0xb7a95905 in do_call (nk=<optimized out>, na=<optimized out>, pp_stack=<optimized out>, func=<optimized out>) at Python/ceval.c:4466
#15 call_function (oparg=<optimized out>, pp_stack=<optimized out>) at Python/ceval.c:4264
#16 PyEval_EvalFrameEx (f=0xb75981cc, throwflag=0) at Python/ceval.c:2838
#17 0xb7a98f9b in PyEval_EvalCodeEx (_co=0xb2a95070, globals=0xb73a9aac, locals=0x0, args=0x82b3ab4, argcount=4, kws=0x82b3ac4, kwcount=0,
defs=0xb2aa8790, defcount=3, kwdefs=0x0, closure=0x0) at Python/ceval.c:3588
#18 0xb7a95f8e in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=<optimized out>, func=<optimized out>)
at Python/ceval.c:4344
#19 call_function (oparg=<optimized out>, pp_stack=<optimized out>) at Python/ceval.c:4262
#20 PyEval_EvalFrameEx (f=0x82b365c, throwflag=0) at Python/ceval.c:2838
#21 0xb7a98f9b in PyEval_EvalCodeEx (_co=0xb2a98160, globals=0xb73a9aac, locals=0x0, args=0xb7675d6c, argcount=1, kws=0xb7675d70, kwcount=2,
defs=0xb73b0188, defcount=78, kwdefs=0x0, closure=0x0) at Python/ceval.c:3588
#22 0xb7a95f8e in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=<optimized out>, func=<optimized out>)
at Python/ceval.c:4344
#23 call_function (oparg=<optimized out>, pp_stack=<optimized out>) at Python/ceval.c:4262
#24 PyEval_EvalFrameEx (f=0xb7675c2c, throwflag=0) at Python/ceval.c:2838
#25 0xb7a98f9b in PyEval_EvalCodeEx (_co=0xb76596b0, globals=0xb7687ecc, locals=0xb7687ecc, args=0x0, argcount=0, kws=0x0, kwcount=0,
defs=0x0, defcount=0, kwdefs=0x0, closure=0x0) at Python/ceval.c:3588
#26 0xb7a99070 in PyEval_EvalCode (co=0xb76596b0, globals=0xb7687ecc, locals=0xb7687ecc) at Python/ceval.c:775
#27 0xb75ee9a4 in vpy_evaluateScript () from /usr/lib/python3.4/site-packages/vapoursynth.so
#28 0xb75f0be3 in vpy_evaluateFile () from /usr/lib/python3.4/site-packages/vapoursynth.so
#29 0xb7f8659c in vsscript_evaluateFile () from /usr/lib/libvapoursynth-script.so.0
#30 0x08049fe2 in main ()

[cretindesalpes]

jose1711: Thanks, the stack trace was really enlightening. I fixed a few other things. Please give it another try now.
Edit: the “official” git I update is here (https://github.com/EleonoreMizo/fmtconv). I don’t know which copy you were checking.

[jose1711]

jose1711: Thanks, the stack trace was really enlightening. I fixed a few other things. Please give it another try now.
Edit: the “official” git I update is here (https://github.com/EleonoreMizo/fmtconv). I don’t know which copy you were checking.

thank you, i am using the correct git source then. anyway, still the same error but the stacktrace changed a little:

Program received signal SIGABRT, Aborted.
0xb7fdbbc8 in __kernel_vsyscall ()
(gdb) bt
#0 0xb7fdbbc8 in __kernel_vsyscall ()
#1 0xb7c67d66 in raise () from /usr/lib/libc.so.6
#2 0xb7c69397 in abort () from /usr/lib/libc.so.6
#3 0xb7c60e07 in __assert_fail_base () from /usr/lib/libc.so.6
#4 0xb7c60e8b in __assert_fail () from /usr/lib/libc.so.6
#5 0xb71493b7 in conc::LockFreeStack<fmtcl::ErrDifBuf*>::push(conc::LockFreeCell<fmtcl::ErrDifBuf*>&) ()
from /usr/lib/vapoursynth/libfmtconv.so
#6 0xb7209c54 in conc::CellPool<fmtcl::ErrDifBuf*>::allocate_zone(int, unsigned int, conc::AtomicPtr<conc::LockFreeCell<fmtcl::ErrDifBuf*> >&) () from /usr/lib/vapoursynth/libfmtconv.so
#7 0xb7209db2 in conc::CellPool<fmtcl::ErrDifBuf*>::expand_to(unsigned int) () from /usr/lib/vapoursynth/libfmtconv.so
#8 0xb7159799 in fmtc::Bitdepth::Bitdepth(VSMap const&, VSMap&, void*, VSCore&, VSAPI const&) () from /usr/lib/vapoursynth/libfmtconv.so
#9 0xb7149ea4 in vsutl::Redirect<fmtc::Bitdepth>::create(VSMap const*, VSMap*, void*, VSCore*, VSAPI const*) ()
from /usr/lib/vapoursynth/libfmtconv.so
#10 0xb754734a in VSPlugin::invoke(std::string const&, VSMap const&) () from /usr/lib/libvapoursynth.so
#11 0xb7534bb2 in invoke(VSPlugin*, char const*, VSMap const*) () from /usr/lib/libvapoursynth.so
#12 0xb7605ebe in __pyx_pw_11vapoursynth_8Function_3__call__ () from /usr/lib/python3.4/site-packages/vapoursynth.so
#13 0xb79dc973 in PyObject_Call (func=0xb2aa3874, arg=0xb76ed02c, kw=0xb7645cec) at Objects/abstract.c:2040
#14 0xb7a95905 in do_call (nk=<optimized out>, na=<optimized out>, pp_stack=<optimized out>, func=<optimized out>) at Python/ceval.c:4466
#15 call_function (oparg=<optimized out>, pp_stack=<optimized out>) at Python/ceval.c:4264
#16 PyEval_EvalFrameEx (f=0xb75981cc, throwflag=0) at Python/ceval.c:2838
#17 0xb7a98f9b in PyEval_EvalCodeEx (_co=0xb2a90070, globals=0xb73a9aac, locals=0x0, args=0x82b4374, argcount=4, kws=0x82b4384, kwcount=0,
defs=0xb2aa3790, defcount=3, kwdefs=0x0, closure=0x0) at Python/ceval.c:3588
#18 0xb7a95f8e in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=<optimized out>, func=<optimized out>)
at Python/ceval.c:4344
#19 call_function (oparg=<optimized out>, pp_stack=<optimized out>) at Python/ceval.c:4262
#20 PyEval_EvalFrameEx (f=0x82b3f1c, throwflag=0) at Python/ceval.c:2838
#21 0xb7a98f9b in PyEval_EvalCodeEx (_co=0xb2a93160, globals=0xb73a9aac, locals=0x0, args=0xb7675d6c, argcount=1, kws=0xb7675d70, kwcount=2,
defs=0xb73b1188, defcount=78, kwdefs=0x0, closure=0x0) at Python/ceval.c:3588
#22 0xb7a95f8e in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=<optimized out>, func=<optimized out>)
at Python/ceval.c:4344
#23 call_function (oparg=<optimized out>, pp_stack=<optimized out>) at Python/ceval.c:4262
#24 PyEval_EvalFrameEx (f=0xb7675c2c, throwflag=0) at Python/ceval.c:2838
#25 0xb7a98f9b in PyEval_EvalCodeEx (_co=0xb76596b0, globals=0xb7687ecc, locals=0xb7687ecc, args=0x0, argcount=0, kws=0x0, kwcount=0,
defs=0x0, defcount=0, kwdefs=0x0, closure=0x0) at Python/ceval.c:3588
#26 0xb7a99070 in PyEval_EvalCode (co=0xb76596b0, globals=0xb7687ecc, locals=0xb7687ecc) at Python/ceval.c:775
#27 0xb75ee9a4 in vpy_evaluateScript () from /usr/lib/python3.4/site-packages/vapoursynth.so
#28 0xb75f0be3 in vpy_evaluateFile () from /usr/lib/python3.4/site-packages/vapoursynth.so
#29 0xb7f8659c in vsscript_evaluateFile () from /usr/lib/libvapoursynth-script.so.0
#30 0x08049fe2 in main ()

[cretindesalpes]

jose1711: I’m not sure what’s wrong now. I did another modification. If it still doesn’t work, could you do the following?
– Add the -fdump-class-hierarchy option for the compilation. Modify the AM_CXXFLAGS line at the beginning of makefile.am for example.
– Compile fmtconv
– Locate the file beginnig with Bitdepth.cpp and ending with the .class extension. It should be in the same directory as the object files.
– Send me the content (it’s a text file) via a pastebin or something, or compressed on any file host if it’s too big.

[jose1711]

jose1711: I’m not sure what’s wrong now. I did another modification. If it still doesn’t work, could you do the following?
– Add the -fdump-class-hierarchy option for the compilation. Modify the AM_CXXFLAGS line at the beginning of makefile.am for example.
– Compile fmtconv
– Locate the file beginnig with Bitdepth.cpp and ending with the .class extension. It should be in the same directory as the object files.
– Send me the content (it’s a text file) via a pastebin or something, or compressed on any file host if it’s too big.

different error now:

Python exception: No attribute with the name scd exists. Did you mistype a plugin namespace?
Traceback (most recent call last):
File "src/cython/vapoursynth.pyx", line 1469, in vapoursynth.vpy_evaluateScript (src/cython/vapoursynth.c:24755)
File "test.vpy", line 22, in <module>
ret = haf.QTGMC( ret, Preset='Slow', TFF=True)
File "/usr/lib/python3.4/site-packages/havsfunc.py", line 878, in QTGMC
if TR0 > 0: ts1 = TemporalSoften(bobbed, 1, 255<<shift, CMts<<shift, 28<<shift, 2) # 0.00 0.33 0.33 0.33 0.00
File "/usr/lib/python3.4/site-packages/havsfunc.py", line 3688, in TemporalSoften
clip = set_scenechange(clip, scenechange)
File "/usr/lib/python3.4/site-packages/havsfunc.py", line 3710, in set_scenechange
sc = core.scd.Detect(sc, thresh)
File "src/cython/vapoursynth.pyx", line 1090, in vapoursynth.Core.__getattr__ (src/cython/vapoursynth.c:18950)
AttributeError: No attribute with the name scd exists. Did you mistype a plugin namespace?

Failed to recognize file format.

[cretindesalpes]

Ah. It looks like it’s not an fmtconv error anymore. Vapoursynth is looking for scd.Detect. Check that you have the scene change detection plug-in (http://forum.doom9.org/showthread.php?t=166769) installed.

[jose1711]

Ah. It looks like it’s not an fmtconv error anymore. Vapoursynth is looking for scd.Detect. Check that you have the scene change detection plug-in (http://forum.doom9.org/showthread.php?t=166769) installed.

wow, thank you so much. after compiling scene change detection plugin everything just works!

[jose1711]

and i just came across another problem. while most of the videos work i just stumbled upon a clip that throws a segmentation fault. gdb:

Using host libthread_db library "/usr/lib/libthread_db.so.1".
[New Thread 0xb1001b40 (LWP 11289)]
[New Thread 0xb1802b40 (LWP 11288)]
[New Thread 0xb29b3b40 (LWP 11287)]
[New Thread 0xb2003b40 (LWP 11286)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb1001b40 (LWP 11289)]
0xb7d65740 in __memcpy_ssse3 () from /usr/lib/libc.so.6
(gdb) bt
#0 0xb7d65740 in __memcpy_ssse3 () from /usr/lib/libc.so.6
#1 0xb70deeb9 in vs_bitblt(void*, int, void const*, int, int, int) () from /usr/lib/libffms2.so.4.0.0
#2 0xb70e08e6 in VSVideoSource::OutputFrame(FFMS_Frame const*, VSFrameRef*, VSAPI const*) ()
from /usr/lib/libffms2.so.4.0.0
#3 0xb70dfa14 in VSVideoSource::GetFrame(int, int, void**, void**, VSFrameContext*, VSCore*, VSAPI const*) ()
from /usr/lib/libffms2.so.4.0.0
#4 0xb7540d78 in VSNode::getFrameInternal(int, int, VSFrameContext&) () from /usr/lib/libvapoursynth.so
#5 0xb754f72a in VSThreadPool::runTasks(VSThreadPool*, std::atomic<bool>&) () from /usr/lib/libvapoursynth.so
#6 0xb75506d0 in std::thread::_Impl<std::_Bind_simple<void (*(VSThreadPool*, std::reference_wrapper<std::atomic<bool> >))(VSThreadPool*, std::atomic<bool>&)> >::_M_run() () from /usr/lib/libvapoursynth.so
#7 0xb7eadaee in std::(anonymous namespace)::execute_native_thread_routine (__p=0xb2d00544)
at /build/gcc/src/gcc-5.2.0/libstdc++-v3/src/c++11/thread.cc:84
#8 0xb793f1c3 in start_thread () from /usr/lib/libpthread.so.0
#9 0xb7d24e8e in clone () from /usr/lib/libc.so.6


any idea? much obliged, jose

[cretindesalpes]

I only see Vapoursynth code here. You can post this in the main Vapoursynth thread.

[cretindesalpes]

No idea why. What were the changes between test1 and test2?

[Myrsloik]

Try other filters and you'll see it's probably a general speed degradation. It's because I've had to temporarily stop using tcmalloc. It will be back some day soon probably. Once it has the weird VS2015 issues fixed.

[an3k]

Here is another plug-in for Vapoursynth.

>>> fmtconv-r17.zip <<< (http://ldesoras.free.fr/src/vs/fmtconv-r17.zip)

Thank You :)

I just noticed the autogen.sh in the zip-file doesn't has chmod u+x and is additionally in DOS format, thus Linux cannot execute it, only after manual chmod and conversion into "Linux" format. The file in the git repo is fine. However both libraries are installed into $PREFIX/lib/ instead of $PREFIX/lib/vapoursynth/

[jackoneill]

Thank You :)

I just noticed the autogen.sh in the zip-file doesn't has chmod u+x and is additionally in DOS format, thus Linux cannot execute it, only after manual chmod and conversion into "Linux" format. The file in the git repo is fine. However both libraries are installed into $PREFIX/lib/ instead of $PREFIX/lib/vapoursynth/

Use the --libdir parameter.

[mawen1250]

http://i683.photobucket.com/albums/vv197/mawen1250/Madoka_01_test.vpy%20-%200_zps6opssy2k.jpg
I encountered a really weird bug that might relate to fmtc.bitdepth.
In fact it was someone reporting bug about mvf.BM3D then did I go to figure out what's going wrong. After testing for nearly a whole day, finally I found a minimum script to reproduce this issue.

video example and the test script (https://mega.nz/#!xA5BVIrL!zZEjn5KZNoN5WPWIM2WPiON5WtAQesCjH3SOv7XJk4o)

environment:
CPU is Ivy Bridge (no AVX2)
Windows 7
VS R30 64bit
fmtconv-r17
BM3D-r4
L-SMASH-Works-r859-20151231

import vapoursynth as vs

core = vs.get_core(threads=0)
core.max_cache_size = 8000

src_path = r'Madoka_01.mkv'
src = core.lsmas.LWLibavSource(src_path, threads=1)
src = core.fmtc.bitdepth(src, bits=32) # Comment this line will not reproduce the issue
src = core.fmtc.resample(src, kernel='bicubic', a1=0, a2=0.5, css='444') # Catmull-Rom can reproduce the issue
#src = core.fmtc.resample(src, kernel='spline', css='444') # Spline can reproduce the issue
#src = core.fmtc.resample(src, kernel='spline36', css='444') # and some other kernels will not
src = core.fmtc.matrix(src, mat='709', col_fam=vs.RGB)

radius = 1
param = {'sigma':[3,3,3], 'radius':radius, 'profile':'fast', 'matrix':100}
sample = vs.FLOAT # vs.INTEGER will not reproduce the issue
src = core.bm3d.RGB2OPP(src, sample)
flt = core.bm3d.VBasic(src, **param).bm3d.VAggregate(radius, sample)
flt = core.bm3d.VFinal(src, flt, **param).bm3d.VAggregate(radius, vs.FLOAT) # vs.INTEGER will not reproduce the issue
flt = core.bm3d.OPP2RGB(flt, vs.FLOAT) # vs.INTEGER will not reproduce the issue

final = flt
#final = core.fmtc.matrix(final, mat='709', col_fam=vs.YUV) # optionally, uncomment this line will also reproduce the issue
final = core.fmtc.bitdepth(final, bits=8, dmode=3) # with float input, integer output and dmode=3~7, the output is corrupt

out = final # src and flt are OK
out[111:].set_output()
From what I've seen the corrupt result is produced by the last fmtc.bitdepth, and only with float input+integer output+dmode=3~7.

The weird thing is that only under specific circumstance will this issue be reproduced. With a tiny change in the script (e.g. different resample kernel, vs.FLOAT->vs.INTEGER, remove the BM3D part, change dmode to 0~2) the result will be correct. Different environment may also affect it but I'm not sure.
More specific information is written in the comments.
I've tried converting source to lossless format and the result is the same, thus it should not be a problem relates to source filter.

[mawen1250]

After more testing, I found out that it's related to the final estimate of BM3D, which can produce very large or even NaN/INF results, and break the error diffusion algorithm of fmtc.bitdepth.

Adding either one like these before fmtc.bitdepth I'll get different black areas:
final = core.std.Expr(final, 'x -4294967296 max')
final = core.std.Expr(final, 'x -16777216 max')
final = core.std.Expr(final, 'x -65536 max')

Anyway, I've fixed this issue in BM3D-r5, but I'm not sure if it's a problem for fmtc.bitdepth.

[speedyrazor]

Is there a way to reverse the filed order is VapourSynth?

[MonoS]

Is there a way to reverse the filed order is VapourSynth?

If i've undestood what you're asking, it should be

reversed = core.std.SelectEvery(InterleavedField, 2, [1,0])

[speedyrazor]

If i've undestood what you're asking, it should be

reversed = core.std.SelectEvery(InterleavedField, 2, [1,0])


Hi, thanks for the suggestion, but this is not what I was after.
Lets say I have some footage which is top field first and I want to convert it to bottom field first. How would I do that in VapoutSynth please?

[jackoneill]

Hi, thanks for the suggestion, but this is not what I was after.
Lets say I have some footage which is top field first and I want to convert it to bottom field first. How would I do that in VapoutSynth please?

Probably by deinterlacing it and then interlacing it again. If you just take the fields as they are and display them backwards, you'll get jerky motion. Or if you put the top field at the bottom, and the bottom field at the top, it's going to look wrong. Therefore, slow processing (QTGMC) is the way to go.

[speedyrazor]

Probably by deinterlacing it and then interlacing it again. If you just take the fields as they are and display them backwards, you'll get jerky motion. Or if you put the top field at the bottom, and the bottom field at the top, it's going to look wrong. Therefore, slow processing (QTGMC) is the way to go.

What I am looking for is the equivalent to ffmpeg's filter:

-vf phase=b

Just tested and this works correctly, any equivalent in VapourSynth?

[Myrsloik]

Simply delete the first field, same as in avisynth as usual...


core.std.DoubleWeave(clip.std.SeparateFields(tff=whatevertheinputis)[1:])[::2]


Untested but it's the general idea at least.

[speedyrazor]

Simply delete the first field, same as in avisynth as usual...


core.std.DoubleWeave(clip.std.SeparateFields(tff=whatevertheinputis)[1:])[::2]


Untested but it's the general idea at least.
I am using it like this:

ret = core.std.DoubleWeave(ret.std.SeparateFields(tff=True)[1:])[::2]

But am getting the error:

vapoursynth.Error: DoubleWeave: argument tff is required

[stax76]

can fmtconv convert to the same format used in the code below?

clip.resize.Bicubic(format=vs.COMPATBGR32)

[cretindesalpes]

fmtconv doesn’t output to COMPAT* formats. However you can convert to RGB24 with fmtconv and output to a COMPAT format with the builtin VS functions.

[cretindesalpes]

fmtconv r18 (http://forum.doom9.org/showthread.php?t=166504):
Added the primaries function to convert between gamuts.
The “full” range is now closer to what is specified in the standards.
A recent Vapoursynth is now required because API headers were updated to version 3.1.4.
transfer: added the Adobe RGB and ProPhoto / ROMM curves.

[benmanw]

just forward J1Man's request from GitHub.

https://github.com/EleonoreMizo/fmtconv/issues/7


Feature Request, High Dynamic Range Video Conversion #7


J1Man commented 2 days ago

Dear Laurent,

I would like to request a feature. Can you add the capability to do UHD compliant Standard Dynamic Range (SDR) to High Dynamic Range (HDR) conversion (and also the opposite HDR to SDR conversion) to Fmtconv plugin?

I also sent you a long email with all the technical research that I could find. In summary, the required new features are listed below.

1) A custom HDR-1000 system compatible SMPTE ST 2084 transfer function that maps to "0 cd/m2 to 1000 cd/m2" luminance range. (Note: current 2084 function included in Fmtconv maps to 0 to 10,000 cd/m2)

2) Add DCI-P3 color primaries as a preset to primaries function. This is not very urgent since "gd=[0.265, 0.69], bd=[0.15, 0.06], rd=[0.68, 0.32], wd=[0.3127, 0.329]" command can be used to manually define DCI-P3 primaries in fmtconv r18.

Once these two new features are implemented, the workflow in Fmtconv will be the following for SDR to HDR conversion.
-Convert source video to "Linear light RGB 4:4:4"
-If necessary, change primaries to one of the HDR compatible primaries (BT709, BT2020, or DCI-P3)
-Convert transfer function from Linear to the custom SMPTE ST 2084 that maps to "0 to 1000 cd/m2" luminance range
-Resample to YUV 4:2:0 10 bit using BT2020 or BT709 matrix (which ever matrix is appropriate)
-Send YUV to x265 encoder. Specify the 2084 transfer fuction, appropriate matrix and appropriate primaries in the x265 command line.

The opposite steps can be used to convert HDR to SDR.

Please let me know what you think.

Best Regards

[benmanw]

One more message from J1Man. He won't be able to post on Doom9 until he has been registered for 5 days (until 3/19/2016).

I attached the source code patch zip file for fmtconv r18 that modifies SMPTE-ST-2084 function to "0 cd/m2 to 1000 cd/m2" luminance range and adds the "dci-p3" preset to primaries function. I also attached a zip file that contains the complete fmtconv r18 source code that is already patched. I would appreciate if somebody with Visual Studio can compile the 64bit fmtconv.dll for Windows 7. No matter how hard I tried in the last 2 days, I could not compile fmtconv.dll on Windows 7 by using MSYS2 and mingw64.

I saw that fmtconv uses the "Dolby Labs recommended version of SMPTE-ST-2084 function" specified on page 17/58 of the following PDF document.

http://www.mediaandbroadcast.bt.com/wp-content/uploads/D2936-UHDTV-final.pdf

I think that was good choice because Dolby Labs version is designed for easy scalability between luminance ranges. Upper bound of the range can be changed by dividing or multiplying the right hand side of the function shown on page 17/58 of the PDF document. That's how I modified lines 78 and 83 of the TransOp2084.cpp file.

I added "dci-p3" primaries preset after line 356 of Primaries.cpp function.

fmtconv-r18-HDR1000-patch-only.zip (https://github.com/EleonoreMizo/fmtconv/files/175990/fmtconv-r18-HDR1000-patch-only.zip)
fmtconv-r18-patched-with-HDR1000features.zip (https://github.com/EleonoreMizo/fmtconv/files/175991/fmtconv-r18-patched-with-HDR1000features.zip)

[cretindesalpes]

J1Man:

fmtconv does not need any modification to support 1000 cd/m² as HDR peak white. If your 100% light value represents 1000 cd/m², you can divide the components by 10 so they fit in a scale where 100% means 10000 cd/m². For this, reduce the contrast directly in the transfer command by setting cont=0.1.
clip = core.fmtc.transfer (clip, transs="linear", transd="2084", cont=1000.0/10000.0)
BTW in your email you showed me a Vapoursynth code snippet where the transfer functions had fulls=False and fulld=False. This is not right because the R’G’B’ input/output for matrix is full by default.

I added the DCI-P3 colorspace to fmtconv and will release an update in the next few days.

[cretindesalpes]

fmtconv r19 (http://forum.doom9.org/showthread.php?t=166504):
primaries: refined the values for the Adobe Wide gamut and BT.2020 primaries.
primaries: added DCI-P3, ACES AP0/AP1, S-Gamut, S-Gamut3.Cine, ALEXA and V-Gamut presets.
transfer: added ACEScc, ERIMM, S-Log2, S-Log3 and V-Log curves.

[J1Man]

Cretindesalpes:

Thanks for your reply and thanks for adding DCI-P3 to the primaries. I will use cont option, as you suggested, to change the luminance range. I will test the videos on my UHD tv and let you know if there are any HDR related issues.


Benmanw:

Thanks for forwarding my messages to the doom forum.

[J1Man]

Cretindesalpes:

You stated in your message that the RGB input/output for matrix is "full range" by default. I ran some tests with Big Buck Bunny clip (link below).
http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_60fps_normal.mp4

When I use the transfer function (even with only the input clip option), I can not get accurate colors from the output clip unless I manually define clip range conversion from "limited to full" or "limited to limited" either in transfer or matrix functions. I wrote the test scripts below along with their results. Let me know if this is an indication of a bug in fmtconv.

Script 1 (Input and Output look different. Output colors are slightly dull.)
clip = core.fmtc.resample (clip, css="444")
clip = core.fmtc.matrix (clip, mat="709")
clip = core.fmtc.transfer (clip)
clip = core.fmtc.matrix (clip, mat="709")
clip = core.fmtc.resample (clip, css="420")
clip = core.fmtc.bitdepth (clip, bits=10)


Script 2 (Input and Output look different. Output colors are slightly dull.)
clip = core.fmtc.resample (clip, css="444")
clip = core.fmtc.matrix (clip, mat="709")
clip = core.fmtc.transfer (clip, fulls=True, fulld=True)
clip = core.fmtc.matrix (clip, mat="709")
clip = core.fmtc.resample (clip, css="420")
clip = core.fmtc.bitdepth (clip, bits=10)



Script 3 (Input and Output look identical.)
clip = core.fmtc.resample (clip, css="444")
clip = core.fmtc.matrix (clip, mat="709")
clip = core.fmtc.transfer (clip, fulls=False, fulld=False)
clip = core.fmtc.matrix (clip, mat="709")
clip = core.fmtc.resample (clip, css="420")
clip = core.fmtc.bitdepth (clip, bits=10)


Script 4 (Input and Output look identical.)
clip = core.fmtc.resample (clip, css="444")
clip = core.fmtc.matrix (clip, mat="709")
clip = core.fmtc.transfer (clip, fulls=False, fulld=True)
clip = core.fmtc.matrix (clip, mat="709")
clip = core.fmtc.resample (clip, css="420")
clip = core.fmtc.bitdepth (clip, bits=10)



Script 5 (Input and Output look identical.)
clip = core.fmtc.resample (clip, css="444")
clip = core.fmtc.matrix (clip, mat="709",fulls=False, fulld=True)
clip = core.fmtc.transfer (clip)
clip = core.fmtc.matrix (clip, mat="709", fulls=True, fulld=False)
clip = core.fmtc.resample (clip, css="420")
clip = core.fmtc.bitdepth (clip, bits=10)

[cretindesalpes]

Only Script 4 gave me a different output with a regular TV-range video sample, which was expected. However the Big Buck Bunny clip you linked seems to be BT.601 and not BT.709 (the stream is not even tagged and some red colors are clipped when using a 709 matrix), which would explain why converting back and forth to R’G’B’ generates some differences.

[J1Man]

Thanks for getting back to me.

How did you understand that the original clip uses BT601 matrix? Does it use BT601-525 primaries? I got BT709 matrix and primaries from MadVR's OSD but it seems like MadVR automatically assumes BT709 matrix/primaries for all HD size clips. Does fmtconv display the original matrix/primaries/transfer_function info for the clip? Or did you use another program?

In order to prevent color clipping, is it safe to use BT2020 matrix for back and forth conversion between RGB and YUV (without primaries gamut conversion), when I am not sure about the source matrix?


I am still getting some unexpected behavior in the output even with BT601 matrix. I generated screenshots for the unprocessed original clip, output for scripts 1-5 with BT601 matrix and output for scripts 1-5 with BT709 matrix. All screenshots are from the same frame that is located approximately at time code 06:22/10:35 of the clip. I used 64bit fmtconv r19, 64bit ffms2 ver 2.20 and 64bit vapoursynth r31 via 64bit Staxrip 1.3.2 to generate these screenshots. The link for the screenshots zip file is shown below:

https://github.com/EleonoreMizo/fmtconv/files/182947/BigBuckBunny_FmtConv_RangeTestScripts_Screenshots.zip

I noticed that Script 4 gives different output in my tests as well. Color of the Bunny is incorrect in the screenshots for Script 4 both with BT601 and BT709 matrices. There is a very small difference in output of Script 4 that I did not notice before I posted my previous message.

When you look at my screenshots, you will see that Scripts 1, 2 and 4 with BT601 matrix give different outputs that are not identical to input. Only Scripts 3 and 5 with BT601 matrix give outputs that are identical to input. (Note: In this particular test frame, BT601 and BT709 did not make a noticeable difference in the screenshots for any of the scripts. That's why it is very difficult to visually notice the incorrect usage of BT709 matrix by looking at the screenshots of this test frame.)

Were you able to replicate my results by using Scripts 1 and 2 with BT601 matrix on the Big Buck Bunny clip near the same frame/scene that I used in my screenshots? What might be the reason for input/output differences caused by Scripts 1 and 2? The results might not be noticeable at some other scenes of the same clip.



Is this Big Buck Bunny clip limited range based on your analysis? Is there a way to tell if a source clip is full range or limited range? MediaInfo does not always show this information. I probably should have asked this question before doing all the work for generating screenshots.

[kolak]

Cretindesalpes:

You stated in your message that the RGB input/output for matrix is "full range" by default. I ran some tests with Big Buck Bunny clip (link below).
http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_60fps_normal.mp4

When I use the transfer function (even with only the input clip option), I can not get accurate colors from the output clip unless I manually define clip range conversion from "limited to full" or "limited to limited" either in transfer or matrix functions. I wrote the test scripts below along with their results. Let me know if this is an indication of a bug in fmtconv.

Script 1 (Input and Output look different. Output colors are slightly dull.)
clip = core.fmtc.resample (clip, css="444")
clip = core.fmtc.matrix (clip, mat="709")
clip = core.fmtc.transfer (clip)
clip = core.fmtc.matrix (clip, mat="709")
clip = core.fmtc.resample (clip, css="420")
clip = core.fmtc.bitdepth (clip, bits=10)


Script 2 (Input and Output look different. Output colors are slightly dull.)
clip = core.fmtc.resample (clip, css="444")
clip = core.fmtc.matrix (clip, mat="709")
clip = core.fmtc.transfer (clip, fulls=True, fulld=True)
clip = core.fmtc.matrix (clip, mat="709")
clip = core.fmtc.resample (clip, css="420")
clip = core.fmtc.bitdepth (clip, bits=10)



Script 3 (Input and Output look identical.)
clip = core.fmtc.resample (clip, css="444")
clip = core.fmtc.matrix (clip, mat="709")
clip = core.fmtc.transfer (clip, fulls=False, fulld=False)
clip = core.fmtc.matrix (clip, mat="709")
clip = core.fmtc.resample (clip, css="420")
clip = core.fmtc.bitdepth (clip, bits=10)


Script 4 (Input and Output look identical.)
clip = core.fmtc.resample (clip, css="444")
clip = core.fmtc.matrix (clip, mat="709")
clip = core.fmtc.transfer (clip, fulls=False, fulld=True)
clip = core.fmtc.matrix (clip, mat="709")
clip = core.fmtc.resample (clip, css="420")
clip = core.fmtc.bitdepth (clip, bits=10)



Script 5 (Input and Output look identical.)
clip = core.fmtc.resample (clip, css="444")
clip = core.fmtc.matrix (clip, mat="709",fulls=False, fulld=True)
clip = core.fmtc.transfer (clip)
clip = core.fmtc.matrix (clip, mat="709", fulls=True, fulld=False)
clip = core.fmtc.resample (clip, css="420")
clip = core.fmtc.bitdepth (clip, bits=10)

Your source file has no full range flag, so by the default nothing will expect full range.

These files are full range and have proper flagging, so you can try:
http://www.dpreview.com/reviews/canoneos5dmarkii/19

[cretindesalpes]

How did you understand that the original clip uses BT601 matrix?
By trying to display it with a BT.601 matrix and suddenly seeing flat colors turning into gradients. Look at the fruit hanging on the tree at around 1:45.

Does it use BT601-525 primaries?
No idea. As I said, the stream isn’t tagged at all with color specifications. To be sure, you should ask the people that encoded this movie.

In order to prevent color clipping, is it safe to use BT2020 matrix for back and forth conversion between RGB and YUV (without primaries gamut conversion), when I am not sure about the source matrix?
There is no safe way. If you assume a certain colorspace and if your stream has Y’Cb’Cr’ component out of its legal range, you’ll have a loss when going back to Y’Cb’Cr’. In any case the R’G’B’ representations will be wrong so I can’t see why you would do this.

When you look at my screenshots, you will see that Scripts 1, 2 and 4 with BT601 matrix give different outputs that are not identical to input.
You’re assuming the source is full range whereas it is TV-range.

Is this Big Buck Bunny clip limited range based on your analysis? Is there a way to tell if a source clip is full range or limited range?
Unless specified explicitly in the stream, you can assume TV-range for most Y’Cb’Cr’ signals. To make sure (some streams are wrongly tagged), you can use an histogram. However this is not trivial, a lot of cameras generate TV-range streams but uses levels over 235 to store HDR data. But the black level should be located around 16 anyway.

[J1Man]

Kolak:

Thanks for the full range test cases. I ran the tests using the following video from the page that you sent me:
http://download.dpreview.com/canon_eos5dmkii/5D2_Portrait.MOV

This video has Full Range color, BT.601 matrix coefficients, BT.709 primaries and BT.709 transfer function tagged in the stream. No guess work this time. Everything is tagged.

Since this video is full range, I added script 6 shown below.

Script 6 (Input and Output look identical for a full range source video.)
clip = core.fmtc.resample (clip, css="444")
clip = core.fmtc.matrix (clip, mat="601",fulls=True, fulld=True)
clip = core.fmtc.transfer (clip)
clip = core.fmtc.matrix (clip, mat="601", fulls=True, fulld=True)
clip = core.fmtc.resample (clip, css="420")
clip = core.fmtc.bitdepth (clip, bits=10)

I only ran test scripts 1,2 and 6 all with BT.601 matrix on this full range video. It does not make sense to run scripts 3,4 and 5 on this full range video (but in case anybody is wondering, outputs in test scripts 3, 4 and 5 do not look like the input, as expected). Screenshots can be downloaded from the link below. All screenshots are taken from the first frame of the video.

https://github.com/EleonoreMizo/fmtconv/files/183269/Portrait_FmtConv_RangeTestScripts_Screenshots.zip

Only script 6 gives an output that is identical to the input. Scripts 1 and 2 have different outputs. I would have expected Scripts 1 and 2 to have identical inputs & outputs for this full range video, but they don't.


The reason why I am running these tests and bringing this issue to Cretindesalp's attention is because I am trying to figure out if there is a bug in fmtconv or not. Based on Cretindesalp's messages, test scripts 1 and 2 should have identical inputs and outputs. But my screenshots (both for limited range Big Buck Bunny and full range Portrait video) show that this is not the case.


Cretindesalps:

Based on your message, I understand that the tests that I did with Big Buck Bunny clip were not very useful because we do not know for sure what the original specifications of the clip are.

This Portrait clip generated by Canon EOS 5D Mark II camera is a better test case because all information is tagged in the stream. What is your opinion on these results? Shouldn't scripts 1 and 2 give outputs that are identical to the input for this full range clip?

The most important question is, is there any unexpected behavior in these test results that indicate the presence of a bug in fmtconv? If you think that there is no problem, I am good too. I can just use script 5 for limited range clips and script 6 for full range source clips.

If you are in doubt, I can re-run the same test with other clips that you specify. Just send me the links.

[cretindesalpes]

If you use a full-range video, you have to indicate to fmtc.matrix that you are using full-range Y’Cb’Cr’. Default is TV-range. That’s why script 1 and 2 don’t give the right result, whereas script 6 works as expected.

[J1Man]

Cretindesalps:

I am also having trouble with primaries conversion. I ran the test using the same video:
http://download.dpreview.com/canon_eos5dmkii/5D2_Portrait.MOV

This video has Full Range color, BT.601 matrix coefficients, BT.709 primaries and BT.709 transfer function tagged in the stream.

I used the following script to convert it to a 10bit full range clip with BT2020 primaries, BT2020 matrix and BT2020-10 transfer function.

Script 7 (Input and Output look different for a full range source video.)
clip = core.fmtc.resample (clip, css="444")
clip = core.fmtc.matrix (clip, mat="601",fulls=True, fulld=True)
clip = core.fmtc.transfer (clip, transs="709", transd="linear")
clip = core.fmtc.primaries (clip, prims="709", primd="2020")
clip = core.fmtc.transfer (clip, transs="linear", transd="2020_10")
clip = core.fmtc.matrix (clip, mat="2020", fulls=True, fulld=True)
clip = core.fmtc.resample (clip, css="420")
clip = core.fmtc.bitdepth (clip, bits=10)


I encoded the video by 10bit build of x265 and tagged the stream properly with BT2020 info. Screenshots can be downloaded from the link below. I took the screenshots at the first frame using MadVR. The girl's face looks very red. I am getting similar results when I play the clip on my UHD TV.

https://github.com/EleonoreMizo/fmtconv/files/188854/5D2_Portrait_BT2020_conversionTest.zip


Am I doing something wrong? Is there a mistake in my script?

[cretindesalpes]

Indeed there is a problem in the primaries code. I’m going to check what’s wrong.

[cretindesalpes]

fmtconv r20 (http://forum.doom9.org/showthread.php?t=166504):
primaries: fixed a bug preventing to set all primaries individually without specifying any preset.
primaries: fixed a bug in the color conversion, thanks to J1Man for having spotted it.

[J1Man]

Cretindesalpes:

Thanks for quickly releasing the new R20 version of fmtconv. I did more testing (using the same portrait clip) and it seems like the issue is not completely resolved. It seems like a big part of the problem is due to BT2020 matrix conversion. I ran the tests listed below using fmtconv r20. The screenshots can be downloaded from the link shown below.

https://github.com/EleonoreMizo/fmtconv/files/190266/5D2_Portrait_BT2020_StepByStepConversionTest2.zip

I first ran Script 6 to test for regressions. Script 6 converts the clip to RGB and back to 10bit YUV without modifying the matrix, primaries and transfer function. I took screenshots with MadVR and confirmed that input and output look 100% identical. This also proves that there is no distortion introduced by x265 or MadVR.

I also ran a modified version of Script 6 that only changes the transfer function from BT709 to BT2020-10 (keeping Matrix and primaries the same). I confirmed that screenshots taken by MadVR were identical. This rules out transfer function related mistakes.

I ran Script 7, to convert everything to BT2020 colorspace, took screenshots with MadVR and saw that input and output were different. I also ran a modified version of Script 7 to convert everything to BT709. Again there was a noticeable difference. The surprising thing is BT709 and BT2020 conversion screenshots look very very similar. As a result, I decided to run other tests.

I created Script 8 (shown below) that only changes the matrix (keeping primaries and transfer function identical). The input and output were different. There is a very big and noticeable difference.

Script 8 (just changes the matrix, Input and Output look different)
clip = core.fmtc.resample (clip, css="444")
clip = core.fmtc.matrix (clip, mat="601",fulls=True, fulld=True)
clip = core.fmtc.matrix (clip, mat="2020", fulls=True, fulld=True)
clip = core.fmtc.resample (clip, css="420")
clip = core.fmtc.bitdepth (clip, bits=10)


There might be a problem with the matrix conversion function used by fmtconv. Since we are going from the smaller color space of BT601 to much larger color space BT2020, there should not be any clipping. There is a BT2020 matrix defined in the source code of x265. I wrote the links below in case they can be useful to you.

https://bitbucket.org/multicoreware/x265/commits/2860b4d8bd682e8d0fe2a08b1e16198feb696230
https://bitbucket.org/multicoreware/x265/src/2860b4d8bd682e8d0fe2a08b1e16198feb696230/source/common/constants.cpp?at=default&fileviewer=file-view-default




I created Script 9 (shown below) that only changes the primaries (keeping matrix and transfer function identical). The input and output were different. There is a small (but still noticeable) difference visible on girl's face, lips and the ceiling.

script 9 (just change primaries, Input and Output look different)
clip = core.fmtc.resample (clip, css="444")
clip = core.fmtc.matrix (clip, mat="601",fulls=True, fulld=True)
clip = core.fmtc.transfer (clip, transs="709", transd="linear")
clip = core.fmtc.primaries (clip, prims="709", primd="2020")
clip = core.fmtc.transfer (clip, transs="linear", transd="709")
clip = core.fmtc.matrix (clip, mat="601", fulls=True, fulld=True)
clip = core.fmtc.resample (clip, css="420")
clip = core.fmtc.bitdepth (clip, bits=10)


I also ran script 9 using the previous primaries (from fmtconv r18) for BT2020 using the command (gd=[0.170, 0.797], bd=[0.131, 0.046], rd=[0.708, 0.292], wd=[0.3127, 0.3290]) and the results did not change.


Please let me know what you think. I ran all the tests that I could think of. I can run more tests if you have any other test ideas.

[J1Man]

Cretindesalpes:

I stumbled on a more serious bug. When "1886a" (alternative approximation of 1886) transfer function is used before and after primaries conversion, it creates heavy pink artifacts on bright areas of the video. I attached screenshots from Tears of Steel movie. You can replicate the bug on any movie with a bright outdoor scene. Artifacts appear mostly on bright parts of the image.

https://github.com/EleonoreMizo/fmtconv/files/195533/tears_of_steel_BugReport_1886a_transferfunction.zip


I used script 10 shown below to generate the screenshots. Script 11 that changes transfer function from 1886a to pure 2.3 gamma curve (without primaries conversion) also creates artifacts that almost posterize the output.

Script 10 (Primaries conversion with 1886a transfer function creates pink artifacts)
clip = core.fmtc.resample (clip, css="444")
clip = core.fmtc.matrix (clip, mat="709",fulls=False, fulld=True)
clip = core.fmtc.transfer (clip, transs="1886a", transd="linear")
clip = core.fmtc.primaries (clip, prims="709", primd="2020")
clip = core.fmtc.transfer (clip, transs="linear", transd="1886a")
clip = core.fmtc.matrix (clip, mat="2020", fulls=True, fulld=False)
clip = core.fmtc.resample (clip, css="420")
clip = core.fmtc.bitdepth (clip, bits=10)

Script 11 (changing transfer function from 1886a to pure 2.3 gamma curve creates artifacts)
clip = core.fmtc.resample (clip, css="444")
clip = core.fmtc.matrix (clip, mat="709",fulls=False, fulld=True)
clip = core.fmtc.transfer (clip, transs="1886a", transd="linear")
clip = core.fmtc.transfer (clip, transs="linear", transd="linear", gcor=1/2.3)
clip = core.fmtc.matrix (clip, mat="709", fulls=True, fulld=False)
clip = core.fmtc.resample (clip, css="420")
clip = core.fmtc.bitdepth (clip, bits=10)



Based on these two scripts, I think the bug is in 1886a transfer function code only.



Most probably you already have it, but I found the latest 1886 standard (with formulas) on the following page:
http://www.itu.int/rec/R-REC-BT.1886-0-201103-I


In theory, Should an accurate implementation of BT.1886 transfer function standard (1886a in fmtconv) provide more accurate linearization before primaries conversion for the latest movies?

[Selur]

'Small' feature request:
Would be nice/helpful if fmtconv would contain a 'helperfunction' which would convert the current color format&matrix to the target color format and matrix.
something like core.fmtc.convert(clip, srcmat, tarfor, tarmat)

so that when having to do a RGB to YUV420P10 conversion instead of having to call:
c = core.fmtc.matrix (clip=c, mat="601", col_fam=vs.YUV, bits=16)
c = core.fmtc.resample (clip=c, css="420")
c = core.fmtc.bitdepth (clip=c, bits=10)
one could simply call:
c = core.fmtc.bitdepth (clip=c, srcmat="601", tarfor="YUV420P10")

or when doing YUV420P10 601 to YUV420P8 709 instead of having to call:

c = core.fmtc.resample (clip=c, css="444")
c = core.fmtc.matrix (clip=c, mats="601", matd="709")
c = core.fmtc.resample (clip=c, css="420")
c = core.fmtc.bitdepth (clip=c, bits=8)

one could simply call:
c = core.fmtc.bitdepth (clip=c, srcmat="601", tarfor="YUV420P8", tarmar="709")
I understand that this would be some lengthy wrapper function to handle all the different possible conversions, but I think it would help the general pain of writing Vapoursynth scripts a bit and not everybody would have to write always those 3-X format conversion lines.

Cu Selur

[jackoneill]

'Small' feature request:
Would be nice/helpful if fmtconv would contain a 'helperfunction' which would convert the current color format&matrix to the target color format and matrix.
something like core.fmtc.convert(clip, srcmat, tarfor, tarmat)

so that when having to do a RGB to YUV420P10 conversion instead of having to call:
c = core.fmtc.matrix (clip=c, mat="601", col_fam=vs.YUV, bits=16)
c = core.fmtc.resample (clip=c, css="420")
c = core.fmtc.bitdepth (clip=c, bits=10)
one could simply call:
c = core.fmtc.bitdepth (clip=c, srcmat="601", tarfor="YUV420P10")

or when doing YUV420P10 601 to YUV420P8 709 instead of having to call:

c = core.fmtc.resample (clip=c, css="444")
c = core.fmtc.matrix (clip=c, mats="601", matd="709")
c = core.fmtc.resample (clip=c, css="420")
c = core.fmtc.bitdepth (clip=c, bits=8)

one could simply call:
c = core.fmtc.bitdepth (clip=c, srcmat="601", tarfor="YUV420P8", tarmar="709")
I understand that this would be some lengthy wrapper function to handle all the different possible conversions, but I think it would help the general pain of writing Vapoursynth scripts a bit and not everybody would have to write always those 3-X format conversion lines.

Cu Selur

The built-in resizers work the way you describe.
http://www.vapoursynth.com/doc/functions/resize.html
Swscale is not involved anymore, so you can use them without worries.

[Selur]

Nice! That helps! Thanks!

[groucho86]

Hi there,
I'm trying to (ultimately) create an MP4 with HDR metadata. My source is generated in Resolve in ACES with an ODT of P3D60 PQ (1000 nits) so it's HDR-Ready. I've outputted as both an OpenEXR image sequence and as a ProRes422HQ.

Having read threads on both doom9 and Blackmagic forums, I think x265 is expecting a YUV 420 10bit y4m stream.

I was doing the following in FFMPEG but was not getting correct results:
ffmpeg -nostats -start_number 00086400 -framerate 24000/1001 -i "/Volumes/TEST/p3d60/p3d60-%08d.exr" -strict -1 -vf \\
scale=out_color_matrix=bt2020:out_h_chr_pos=0:out_v_chr_pos=0,format=yuv420p10 -f yuv4mpegpipe - | x265 - --y4m --crf 13 \\
--tune grain --colorprim bt2020 --transfer smpte-st-2084 --colormatrix bt2020nc --master-display \\
"G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)" --max-cll "1000,400" --output "/Volumes/TEST/hevc/test.hevc"

I'm now trying ffmtconv but I'm unable to create the YUV 420 10bit stream after creating an smpte-2084 version. This is what I have:
c = core.lsmas.LibavSMASHSource(source="/Volumes/Test/ test01.mov")

c = core.fmtc.resample(clip=c, css="444")
c = core.fmtc.bitdepth(clip=c, bits=10)
c = core.fmtc.matrix2020cl(clip=c, full=True )
c = core.fmtc.transfer(c, transs="2020", transd="2084", cont=1000.0/10000.0)

c.set_output()

I then try something like
c = core.fmtc.resample(clip=c, css='420')

But get the following error:
File "src/cython/vapoursynth.pyx", line 1383, in vapoursynth.Function.__call__ (build/temp.macosx-10.9-x86_64-3.5/pyrex/vapoursynth.c:25204)
vapoursynth.Error: : couldn't get a pixel format identifier for the output clip.

Thank you in advance for your help.

[dipje]

Another thing I noticed in r20. Am I doing something wrong, making wrong assumptions or is there something wrong in fmtconv somewhere?

I read a simple movie file (YUV420P8).

I resample to YUV444P16, use matrix to go RGB48 (from BT709), I transfer to linear, I use primaries to go from 709 to 2020, I transfer from linear to 2048 with a contrast value of 1000.0 / 10000.0 (0.1), then matrix to YUV with BT2020 (non-constant) matrix.

As a test, I do the reverse directly in the script. Matrix to RGB from BT2020nc, transfer to linear from 2084 with a contrast value of 10000.0 / 1000.0 (10.0), I use primaries to go from 2020 to 709, I transfer from linear to regular 709, and I use matrix to go to YUV with BT709 matrix.

The result should be pretty much exact the same as the input (with some rounding errors here and there or something), but what I get is a clear 'levels' difference. It seems as if somewhere in the process there is a 'limited range / full range' mix up. The colors seem right, it's just a little bit of levels / contrast that seems different.

I ended up added 'fulls' and 'fulld' everywhere to make sure it does what I expect it to but I still get the same.

Am I doing wrong? Fmtconv wrong? Or is this just the result from going from 709 to 2084 and back since it's not a pure 1-to-1 mapping or something?

My code copy & pasted to be clear:

orig = c
c = core.fmtc.resample(c, w = 1920, h = 1080, css = '444', kernel = 'spline36')
c = core.fmtc.matrix(c, fulls = False, fulld = True, mats = '709', col_fam = vs.RGB)
c = core.fmtc.transfer(c, fulls = True, fulld = True, transs = '61966-2-1', transd = 'linear')
c = core.fmtc.primaries(c, prims = '709', primd = '2020')
c = core.fmtc.transfer(c, fulls = True, fulld = True, transs = 'linear', transd = '2084', cont = 1000.0 / 10000.0)
c = core.fmtc.matrix(c, fulls = True, fulld = False, matd = '2020', col_fam = vs.YUV)
c = core.fmtc.resample(c, css = '420')
c = core.fmtc.bitdepth(c, fulls = False, fulld = False, bits = 10, dmode = 5)

z = c
z = core.fmtc.resample(z, css = '444', kernel = 'spline36')
z = core.fmtc.matrix(z, fulls = False, fulld = True, mats = '2020', col_fam = vs.RGB)
z = core.fmtc.transfer(z, fulls = True, fulld = True, transs = '2084', transd = 'linear', cont = 10)
z = core.fmtc.primaries(z, prims = '2020', primd = '709')
z = core.fmtc.transfer(z, fulls = True, fulld = True, transs = 'linear', transd = '709')
z = core.fmtc.matrix(z, fulls = True, fulld = False, matd = '709', col_fam = vs.YUV)
z = core.fmtc.resample(z, css = '420', kernel = 'spline36')
z = core.fmtc.bitdepth(z, fulls = False, fulld = False, bits = 8, dmode = 5, ampo = 1, ampn = 1)

# At this moment 'orig' and 'z' look different although they should look pretty much the same I guess

[age]

you have used different transfer values,probably you have to go in linear space using 709 transfer
or come back from linear space using "61966-2-1"

orig = c
c = core.fmtc.resample(c, w = 1920, h = 1080, css = '444', kernel = 'spline36')
c = core.fmtc.matrix(c, fulls = False, fulld = True, mats = '709', col_fam = vs.RGB)
c = core.fmtc.transfer(c, fulls = True, fulld = True, transs = '709', transd = 'linear')
#c = core.fmtc.transfer(c, fulls = True, fulld = True, transs = '61966-2-1', transd = 'linear')
c = core.fmtc.primaries(c, prims = '709', primd = '2020')
c = core.fmtc.transfer(c, fulls = True, fulld = True, transs = 'linear', transd = '2084', cont = 1000.0 / 10000.0)
c = core.fmtc.matrix(c, fulls = True, fulld = False, matd = '2020', col_fam = vs.YUV)
c = core.fmtc.resample(c, css = '420')
c = core.fmtc.bitdepth(c, fulls = False, fulld = False, bits = 10, dmode = 5)

z = c
z = core.fmtc.resample(z, css = '444', kernel = 'spline36')
z = core.fmtc.matrix(z, fulls = False, fulld = True, mats = '2020', col_fam = vs.RGB)
z = core.fmtc.transfer(z, fulls = True, fulld = True, transs = '2084', transd = 'linear', cont = 10)
z = core.fmtc.primaries(z, prims = '2020', primd = '709')
z = core.fmtc.transfer(z, fulls = True, fulld = True, transs = 'linear', transd = '709')
#z = core.fmtc.transfer(z, fulls = True, fulld = True, transs = 'linear', transd = '61966-2-1')
z = core.fmtc.matrix(z, fulls = True, fulld = False, matd = '709', col_fam = vs.YUV)
z = core.fmtc.resample(z, css = '420', kernel = 'spline36')
z = core.fmtc.bitdepth(z, fulls = False, fulld = False, bits = 8, dmode = 5, ampo = 1, ampn = 1)

[dipje]

D'oh.. stupid mistake on my part it seems. Will try it out, but srgb and 709 isn't always the same of course :S.

[age]

Is it possible to add/port some new resizer like ewa_lanczossharp (mpv player) ?

[adsun701]

Hi there,
When I was testing if fmtconv could make transforms from 709 content to ciexyz content, I may have encountered a possible bug.
I either get a segmentation fault or completely black output when converting 709 primaries to ciexyz primaries.

import vapoursynth as vs
core = vs.get_core()
core.std.LoadPlugin(path=r'/usr/local/lib/vapoursynth/libimwri.so')
clip = core.ffms2.Source(r'/home/andy/Sample.mkv')
clip = core.std.Trim(clip, 2300, 2300)
clip = core.fmtc.bitdepth(clip, bits=32, dmode=6)
clip = core.fmtc.matrix(clip, mats='709', matd='RGB', col_fam=vs.RGB, bits=32)
clip = core.fmtc.transfer(clip, transs='709', transd='linear', bits=32)
clip = core.fmtc.primaries(clip, prims='709', primd='ciexyz')
clip = core.fmtc.transfer(clip, transs='linear', transd='2084', cont=1000/10000, bits=32)
clip = core.imwrif.Write(clip, 'EXR', 'image%04d.exr', firstnum=0)
clip.set_output()

Is this a bug? If so, I would appreciate that it be fixed. If not, then what am I doing wrong?

[m3sh]

Any plans to add HLG transfer function support in an upcoming release? By my reading of the documentation, I can't see anything referencing ARIB-STD-B67 (Hybrid Log Gamma, HLG) EOTF.

Would love to have it, Technicolor guys gave me an asset in HLG and I'm hoping to use fmtconv to do the conversion to HDR10 to feed into my currently limited STB client - the broadcom chipset doesn't support HLG afaik.

[Cary Knoop]

Any plans to add HLG transfer function support in an upcoming release? By my reading of the documentation, I can't see anything referencing ARIB-STD-B67 (Hybrid Log Gamma, HLG) EOTF.

Would love to have it, Technicolor guys gave me an asset in HLG and I'm hoping to use fmtconv to do the conversion to HDR10 to feed into my currently limited STB client - the broadcom chipset doesn't support HLG afaik.
I wonder if this, as a workaround for now, can be achieved by using Lut2 function mode.

[Myrsloik]

Any plans to add HLG transfer function support in an upcoming release? By my reading of the documentation, I can't see anything referencing ARIB-STD-B67 (Hybrid Log Gamma, HLG) EOTF.

Would love to have it, Technicolor guys gave me an asset in HLG and I'm hoping to use fmtconv to do the conversion to HDR10 to feed into my currently limited STB client - the broadcom chipset doesn't support HLG afaik.

See the documentation for the internal VS resizer. It can actually do a lot of things...

[m3sh]

See the documentation for the internal VS resizer. It can actually do a lot of things...

Ahh yes... took a look at this though and while I'm still using R32 (and probably need to upgrade to VS R36) just want to be sure that upgrading will address the fact that R32 doesn't seem to like the "transfer_s" parameter...? Right now I can't get it working due to specifying the transfer output function as a string, seeing as I don't know the enumerated value for st2084.

[Myrsloik]

It wasn't implemented back then

[m3sh]

Of course... I am a sad (and lazy) panda. Thanks, I'll get 36 installed and try again.

[Cary Knoop]

Hi there,
When I was testing if fmtconv could make transforms from 709 content to ciexyz content, I may have encountered a possible bug.
I either get a segmentation fault or completely black output when converting 709 primaries to ciexyz primaries.

import vapoursynth as vs
core = vs.get_core()
core.std.LoadPlugin(path=r'/usr/local/lib/vapoursynth/libimwri.so')
clip = core.ffms2.Source(r'/home/andy/Sample.mkv')
clip = core.std.Trim(clip, 2300, 2300)
clip = core.fmtc.bitdepth(clip, bits=32, dmode=6)
clip = core.fmtc.matrix(clip, mats='709', matd='RGB', col_fam=vs.RGB, bits=32)
clip = core.fmtc.transfer(clip, transs='709', transd='linear', bits=32)
clip = core.fmtc.primaries(clip, prims='709', primd='ciexyz')
clip = core.fmtc.transfer(clip, transs='linear', transd='2084', cont=1000/10000, bits=32)
clip = core.imwrif.Write(clip, 'EXR', 'image%04d.exr', firstnum=0)
clip.set_output()

Is this a bug? If so, I would appreciate that it be fixed. If not, then what am I doing wrong?
It is my understanding you need to convert the clip to 4:4:4 first, I assume the mkv file is 4:2:0 but perhaps that assumption is wrong.

Also when I use the bits parameter in transfer() from '709' to 'linear' I get: matrix: output bitdepth not supported.

[adsun701]

Thanks for your help!

I actually discovered a bug in the primaries conversion code. When converting to ciexyz primaries, there is a divide by zero error due to the coordinates of the primaries. Hopefully it gets fixed!

[adsun701]

Also, could SMPTE ST 2085 matrix conversion and ICtCp matrix conversion be implemented? They are supported in the latest version of HEVC, and used for HDR content. SMPTE ST 2085 is used with CIEXYZ primaries, while ICtCp can be used with any primaries.

Thanks!

[TheFluff]

The VS builtin resizer (zimg (https://github.com/sekrit-twc/zimg)) has support for ICtCp being worked on (https://github.com/sekrit-twc/zimg/issues/69) as of right now. You should probably consider porting existing code to zimg as well; it's pretty much just flat out better than fmtconv, and it's in active development (unlike fmtconv which hasn't had any activity at all in the last year). It also has more contributors and users than fmtconv since it's a general-purpose C/C++ library and not tied to a filtering framework like VS or Avisynth. It's available in ffmpeg via libavfilter's vf_zscale, for example.

I don't think there are any features fmtconv has that the internal resizer (that is zimg) lacks, anymore. If there are, submit a feature request. http://www.vapoursynth.com/doc/functions/resize.html

[Selur]

I don't think there are any features fmtconv has that the internal resizer (that is zimg) lacks, anymore. If there are, submit a feature request. http://www.vapoursynth.com/doc/functions/resize.html
May be I'm just overlooked it, but I didn't find anything in the documentation about:

How to handle interlaced content? (In fmtconv there is interlaced&interlacedd. Do the resizers automatically handle and keep the interlacing state the source is in?)
Whether there are more resizers available than Bilinear/Bicubic/Point/Lanczos/Spline16/Spline36 through zimg? (fmtconv has point/box/bilinear/cubic/bicubic/lanczos/blackman/blackmanminlobe/spline16/spline36/spline64/spline/gauss/sinc )

[TheFluff]

How to handle interlaced content? (In fmtconv there is interlaced&interlacedd. Do the resizers automatically handle and keep the interlacing state the source is in?)

It's handled automatically, yes, assuming that the _Field or _FieldBased frame props are set correctly. Unlike Avisynth, VS (and zimg) understands that an interlaced clip that you call separatefields() on does not magically become progressive, so if you feed a field-separated clip to the internal resizer, resize it horizontally and then weave it again it'll still be handled correctly (there won't be any jumping up and down).

Whether there are more resizers available than Bilinear/Bicubic/Point/Lanczos/Spline16/Spline36 through zimg? (fmtconv has point/box/bilinear/cubic/bicubic/lanczos/blackman/blackmanminlobe/spline16/spline36/spline64/spline/gauss/sinc )
No. None of the extra ones are all that useful anyway.

[WolframRhodium]

It's handled automatically, yes, assuming that the _Field or _FieldBased frame props are set correctly. Unlike Avisynth, VS (and zimg) understands that an interlaced clip that you call separatefields() on does not magically become progressive, so if you feed a field-separated clip to the internal resizer, resize it horizontally and then weave it again it'll still be handled correctly (there won't be any jumping up and down).


No. None of the extra ones are all that useful anyway.

However, as an example, some people has been widely using gauss for years, and this phenomenon will continue. How can you come down to your conclusion?

[TheFluff]

However, as an example, some people has been widely using gauss for years, and this phenomenon will continue. How can you come down to your conclusion?

The Gauss resizer isn't even a resampler at all, mathematically speaking. As far as I can recall I don't think people really use it all that much for actually resizing things; they use it to blur things. Use something else for that.

Blackman is a bad windowed sinc; use lanczos instead.

[feisty2]

It's handled automatically, yes, assuming that the _Field or _FieldBased frame props are set correctly. Unlike Avisynth, VS (and zimg) understands that an interlaced clip that you call separatefields() on does not magically become progressive, so if you feed a field-separated clip to the internal resizer, resize it horizontally and then weave it again it'll still be handled correctly (there won't be any jumping up and down).


No. None of the extra ones are all that useful anyway.

gaussresize is more useful than you think, it could be used as a non-ringing convolution filter, and it's far more powerful than std.convolution, cuz:
a. it works with any radius (using the "taps" parameter)
b. it does not require you to manually calculate the coefficient matrix, you could simply control the filtering strength using the "p" parameter.

[WolframRhodium]

The Gauss resizer isn't even a resampler at all, mathematically speaking. As far as I can recall I don't think people really use it all that much for actually resizing things; they use it to blur things. Use something else for that.

Blackman is a bad windowed sinc; use lanczos instead.

The combination of lanczos and gauss in upsampling is popular. But zimg doesn't have gauss, that's a problem.

Another example is feisty2's Vine/Plum.

[TheFluff]

gaussresize is more useful than you think, it could be used as a non-ringing convolution filter, and it's far more powerful than std.convolution, cuz:
a. it works with any radius (using the "taps" parameter)
b. it does not require you to manually calculate the coefficient matrix, you could simply control the filtering strength using the "p" parameter.

Isn't that exactly what I said? Write an actual Gaussian convolution filter if that's what you want, instead of shoehorning it into a resampler-that-isn't.

[feisty2]

Isn't that exactly what I said? Write an actual Gaussian convolution filter if that's what you want, instead of shoehorning it into a resampler-that-isn't.

gaussresize is already highly optimized as a part of the resample function in fmtc, I could write a Gaussian filter any day of the week but I'm NOT a professional programmer, so that's gonna be a lot worse (performance-wise) than the existing gaussresize approach anyways, and I'm definitely not wasting my time just to mess with stupid things like that.

[WolframRhodium]

Isn't that exactly what I said? Write an actual Gaussian convolution filter if that's what you want, instead of shoehorning it into a resampler-that-isn't.

What's more, fmtconv contains various dithering mode and custom resample kernel, but what about zimg?

[TheFluff]

What's more, fmtconv contains various dithering mode and custom resample kernel, but what about zimg?

Dithering: Bayer pattern, random noise (magnitude 0.5) and Floyd-Steinberg are supported. Do you have any particularly convincing arguments as to why anything else is vastly superior? Having more options just for the sake of having more options is Avisynth mentality.

Custom convolution kernels: what is the actual practical use case for this? Is there any reason you can't just use a separate convolution filter?

I mean, yeah you can do a ton of interesting shit with a highly customizeable general-purpose FIR filter, but customization and flexibility is kinda hard to combine with ease of use, optimization and maintainability. If you want a filter that can do anything, it probably won't be very well suited to being a fast and reliable image resizer for real world inputs. The reason I'm trying to convince people to switch to zimg is that for 95% of the most common use cases it does what fmtconv does but is faster and easier to use. I know I told you to submit feature requests, but please keep them within the scope of what is reasonable to have in standard image resizer. For exotic stuff, write exotic filters.

[feisty2]

Having more (not significantly worse, like x86 is worse than x64, xp is wore than win10, so I never compiled x86 binaries or obsolete xp binaries for my plugins) options is always better, simply because people could ignore opinions they don't need, and anyone gets to choose what's useful to him/her (which is also why I love c++ and dislike java :))

[WolframRhodium]

Dithering: Bayer pattern, random noise (magnitude 0.5) and Floyd-Steinberg are supported. Do you have any particularly convincing arguments as to why anything else is vastly superior? Having more options just for the sake of having more options is Avisynth mentality.

Custom convolution kernels: what is the actual practical use case for this? Is there any reason you can't just use a separate convolution filter?

dithering: “Filter Lite”. See madvr's thread for why someone love it

kernel: https://forum.doom9.org/showthread.php?t=166080

[TheFluff]

Having more (not significantly worse, like x86 is worse than x64, xp is wore than win10, so I never compiled x86 binaries or obsolete xp binaries for my plugins) options is always better, simply because people could ignore opinions they don't need, and anyone gets to choose what's useful to him/her (which is also why I love c++ and dislike java :))

You're wrong, and very predictably so. You will probably understand this when you have more experience, and I'm probably telling you this completely in vain because I think it's hard to actually internalize this before you've actually experienced it. I was wrong in exactly the same way when I was younger and I don't think anyone could have convinced me to understand that, back then.

More code (and hence more combinations of possible code paths) isn't free, it's bought with the most expensive resource of all - developer attention. More code paths means more tests and more maintenance. More complex code with more paths through it becomes harder to reason about, understand and predict. With more code, it becomes harder to change your mind, refactor and redesign.

You and I - and all other programmers - are much dumber than we'd like to think we are. Things need to be kept simple in order for us to be able to reasonably keep working with them.

[TheFluff]

dithering: “Filter Lite”. See madvr's thread for why someone love it

kernel: https://forum.doom9.org/showthread.php?t=166080

I'm sure a lot of madvr users love a lot of things but I care exceedingly little about them. Adding more dithering options is a pain in the butt and has basically no visible effect on output bitdepths >= 8. I myself sure as heck can't see any meaningful difference between madvr's different error diffusion modes with 8-bit output, and I definitely can't say which one looks better. If you want dithering because you want things to look nice on your shitty 6-bit monitor, then perhaps you should in fact dither on playback instead of trying to encode it into your (probably lossily compressed) video? Especially when dither pattern preferences are so subjective. Filter lite is also supposed to be faster but zimg's Floyd-Steinberg implementation is already really optimized so who cares.

Custom kernels is in a similar spot. Yes you can get marginally improved results in some ways on certain (mostly small) images, but almost nobody uses it and there are tradeoffs with every kernel. If you care about this kind of custom tweaking a standard resizer probably isn't right for you.

If you want fmtconv for its customizeability, you can keep using it! It's fine! It's just that for the vast majority of common use cases (like most of the ones posted in this thread) zimg does the same thing faster and with a bit less effort. That's all I'm saying.

[junh1024]

I'll just leave this here:

Functionality is an ASSET.
Code is a LIABILITY.

(tldr so having the LEAST code that does the MOST stuff is desireable)

[juhok]

Now on FreeBSD Ports! :) https://www.freshports.org/graphics/vapoursynth-fmtconv/

[MonoS]

What is the correct way to downscale in linear light (gamma corrected) an HDR stream?
I tried to come up with a script, but i'm not so confident it is correct as i already screwed up multiple times in this regards :D

[fadedmaple]

Hi,I find a problem.When i use fmtc like this ,it will tell me "transfer: unsupported color family."

src16 = core.fmtc.bitdepth(src,bits=16)
UV = core.fmtc.resample(src16,1920, 1080, kernel="Spline36")
gray = core.std.ShufflePlanes(src16, 0, colorfamily=vs.YUV)
gray = core.fmtc.transfer(gray, transs="2020", transd="linear")
gray = core.fmtc.resample(gray, 1920, 1080, kernel="Spline36")
gray = core.fmtc.transfer(gray, transs="linear",transd="2020")
src = core.std.ShufflePlanes([gray,UV], [0,1,2],vs.YUV)


It works only when the colorfamily specified it with GARY 。
I am a noob in this , can explain why?THX

gray = core.std.ShufflePlanes(src16, 0, colorfamily=vs.GRAY)

[sneaker_ger]

See fmtconv doc.
As input, the function accepts only RGB and grayscale colorspaces.

[fadedmaple]

See fmtconv doc.

Thanks for you repley.
Maybe my expression is wrong,What is the difference between Y plane and GRAY?

[asarian]

Speaking of resizing, if I want to downsize an UHD Blu-ray, would this be the best way to go about it?

core.std.LoadPlugin("DGDecodeNV.dll")
video = core.dgdecodenv.DGSource("i:/jobs/test.dgi", fulldepth=False)
vid = core.fmtc.resample (clip=vid, w=1920, h=1080)

Or does core.fmtc.resample need a special parameter, optimized for reduction?

[Wolfberry]

The default kernel of fmtconv is bicubic with b=c=1/3 a.k.a. "Mitchell-Netravali" which should be enough for normal use.

If you are using the dll from the official release, I will recommend to use the one from ChaosKing's FATPACK or the one in my signature as it is compiled with a newer compiler and might save you from random crashes.

Try muvsfunc's SSIM_downsample (https://github.com/WolframRhodium/muvsfunc/blob/master/muvsfunc.py#L3507) if you are not satisfied with the results of plain bicubic.

[asarian]

The default kernel of fmtconv is bicubic with b=c=1/3 a.k.a. "Mitchell-Netravali" which should be enough for normal use.

If you are using the dll from the official release, I will recommend to use the one from ChaosKing's FATPACK or the one in my signature as it is compiled with a newer compiler and might save you from random crashes.

Try muvsfunc's SSIM_downsample (https://github.com/WolframRhodium/muvsfunc/blob/master/muvsfunc.py#L3507) if you are not satisfied with the results of plain bicubic.


Cool. Thx. :)

[asarian]

The default kernel of fmtconv is bicubic with b=c=1/3 a.k.a. "Mitchell-Netravali" which should be enough for normal use.

If you are using the dll from the official release, I will recommend to use the one from ChaosKing's FATPACK or the one in my signature as it is compiled with a newer compiler and might save you from random crashes.



Seems my fmtconv.dll is dated March 2016. So, thx for the new binary. :) N.B. Is it safe to replace all my old binaries (.dll's) with yours? (For VS R45).

[asarian]

Hmm, someone please explain this to me. I'm using:

import muvsfunc as muf
...
vid = muf.SSIM_downsample (clip=vid, w=1920, h=1080, use_fmtc=False)

It throws the following error, though:


encoded 1 frames, 12.82 fps, 238.61 kb/s
Error: fwrite() call failed when writing frame: 0, plane: 0, errno: 32
Output 9 frames in 2.06 seconds (4.38 fps)
Core freed but 24883200 bytes still allocated in framebuffers

[Wolfberry]

It should be safe if you want and can use my dlls (may require AVX2 and additional dependencies), if not, there are always official ones.
For you second question, errno 32 means broken pipe. Try using only source filters and see if the error persists. (Post your full script might help as well)

[asarian]

It should be safe if you want and can use my dlls (may require AVX2 and additional dependencies), if not, there are always official ones.
For you second question, errno 32 means broken pipe. Try using only source filters and see if the error persists. (Post your full script might help as well)


Pretty much happens with just this single input line:

vid = muf.SSIM_downsample (clip=vid, w=1920, h=1080, use_fmtc=False)


Couldn't really tell whether it's "clip=vid" or just "vid", or that w and h are to be set with an appended '=x' or that they are just the values. But I tried all combinations, to no avail.

Also, I didn't see any dependency missing. Putting 'import muvsfunc as muf' should suffice, right?

[asarian]

Ok, can anyone show me a script snippet of a working muf.SSIM_downsample then, please? Really love to try out this new and smooth reduction method. Thx.

[poisondeathray]

Ok, can anyone show me a script snippet of a working muf.SSIM_downsample then, please? Really love to try out this new and smooth reduction method. Thx.

Did you get an error message ?

I got an error messge complaining that didn't have nnedi3_resample.py , so I needed to download it and put it into site-packages , then it worked


import muvsfunc as muvs
clip = whateversource
clip = muvs.SSIM_downsample(clip, w=1920, h=1080)
clip.set_output()


use_fmtc=False by default if you look at muvsfunc

[asarian]

Did you get an error message ?

No error messages; just the x264 process that reports a broken pipe on the first frame it processes:

encoded 1 frames, 12.82 fps, 238.61 kb/s
Error: fwrite() call failed when writing frame: 0, plane: 0, errno: 32
Output 9 frames in 2.06 seconds (4.38 fps)
Core freed but 24883200 bytes still allocated in framebuffers


I got an error messge complaining that didn't have nnedi3_resample.py , so I needed to download it and put it into site-packages , then it worked

I got nnedi3_resample.py in site-packages (and nnedi3_weights.bin in the plugins64 directory), as per usual.

use_fmtc=False by default if you look at muvsfunc


I saw that, yeah; but it was not caused the breakage. :)

[poisondeathray]

No error messages; just the x264 process that reports a broken pipe on the first frame it processes:

encoded 1 frames, 12.82 fps, 238.61 kb/s
Error: fwrite() call failed when writing frame: 0, plane: 0, errno: 32
Output 9 frames in 2.06 seconds (4.38 fps)
Core freed but 24883200 bytes still allocated in framebuffers





1) Did you convert it to the final pixel format in the script? eg. YUV420P8 if encoding 8bit 4:2:0 etc... YUV420P10 if 10bit 4:2:0 . Or whatever you're doing

2) Are you using correct matching vspipe with your vapoursynth version ?

Post commandline if neither of those is the cause

[asarian]

1) Did you convert it to the final pixel format in the script? eg. YUV420P8 if encoding 8bit 4:2:0 etc... YUV420P10 if 10bit 4:2:0 . Or whatever you're doing

I'm just trying to downsize it to a regular Blu-ray format, 8-bit, compressed with x264.

I use dgdecodenv, which, if I understood it correctly, will output the UHD input as 8-bit YV12 by default.

2) Are you using correct matching vspipe with your vapoursynth version ?

Yes, matching vspipe is being used. The whole new setup works flawlessly (already done 2 discs with it), but it's just this SSIM_downsample that won't budge for me.

[poisondeathray]

You have to convert it back to the final pixel format. That filter converts up to float

[asarian]

You have to convert it back to the final pixel format. That filter converts up to float

Ah, that would explain it. :) Thx! :goodpost:

[asarian]

1) Did you convert it to the final pixel format in the script? eg. YUV420P8 if encoding 8bit 4:2:0 etc... YUV420P10 if 10bit 4:2:0 . Or whatever you're doing


Bloody h*ell! Looks like I got the colorspace messed up, after all: flaccid colors (https://1drv.ms/u/s!AhSxhQ9g_mrMlBx7PAVJJhq8lNRd)

The bottom image is the result of extracting the HEVC stream with TsMuxer (as eac3to can't extract to .mkv for HEVC yet), so prior to me re-encoding it. The colors are all flaccid, and not vibrant at all. :(

Is there any way I could change the header of the encoded file somehow, and set a different colorspace? Or am I too late already?

And perhaps more importantly, as TsMuxer apparently screws up the stream, and eac3to can't do it, what is the best way to extract the HEVC stream properly? (to feed to the VS input filters).

[asarian]

^^ Ok, I did some comparison:

stream comparison (https://1drv.ms/u/s!AhSxhQ9g_mrMlB5wxbZgx_9ewbV3)

The pane on the right is from the resultant tsMuxer stream, the one of the right of the original Blu-ray m2ts. Far as I can see, they're identical! No bit-depth is lost, no colorspace conversion took place, etc. Yet playing/encoding the stream muxed by tsMuxer yields all these flaccid colors. :(

[asarian]

Hmm, after the initial shock, I think the UHD 'Oblivion' disc is simply encoded with flatter colors. :) (Which is possible, of course) I can't see another reason. I got 2 other UHD discs so far (I intend to make Pan @ Scan HD versions, from UHD, for some of my fave movies), and they look about the same, color-wise (demuxed) as their original. So, no weirdness was going on, after all. Maybe I should get my money back on the Oblivion UHD. :) (Or make the colors a bit more vibrant myself).

[poisondeathray]

Hmm, after the initial shock, I think the UHD 'Oblivion' disc is simply encoded with flatter colors. :) (Which is possible, of course) I can't see another reason. I got 2 other UHD discs so far (I intend to make Pan @ Scan HD versions, from UHD, for some of my fave movies), and they look about the same, color-wise (demuxed) as their original. So, no weirdness was going on, after all. Maybe I should get my money back on the Oblivion UHD. :) (Or make the colors a bit more vibrant myself).

You probably have to tonemap it . It's HDR10 release, correct?


https://www.blu-ray.com/movies/Oblivion-4K-Blu-ray/156371/


HDR: HDR10
Universal's 2160p, HDR-enabled presentation...


One option is DGHDRtoSDR , another is vapoursynth tonemap

[videoh]

One option is DGHDRtoSDR You may like this one as it is GPU accelerated. Apply it after resizing on the GPU (if needed) for maximum performance.

[MonoS]

I do tonemap that way
den = core.resize.Bicubic(clip=den, format=vs.RGBS, filter_param_a=0.0, filter_param_b=0.75, range_in_s="limited", matrix_in_s="2020ncl", primaries_in_s="2020", primaries_s="2020", transfer_in_s="st2084", transfer_s="linear",dither_type="none", nominal_luminance=mcll)
den = core.tonemap.Mobius(clip=den, exposure=exposure, transition=0.3, peak=mcll)
den = core.fmtc.resample(den, resw, resh, kernel="bilinear", invks=True, invkstaps=3, taps=4)
den = core.resize.Bicubic(clip=den, format=vs.YUV444PS, filter_param_a=0.0, filter_param_b=0.75, matrix_s="709", primaries_in_s="2020", primaries_s="709", transfer_in_s="linear", transfer_s="709")
res = core.fmtc.resample(den, resw, resh, invks=True, kernel="bicubic", invkstaps=3, css="420").fmtc.bitdepth(bits=8, dmode=3, flt=False)

the parameters you need to pass are resw and resh for the final resolution, mcll for the mcll parameter of your stream, you can get it with mediainfo, and exposure you need to set as you like it, i suggest you to compare with the SDR bluray, as the tonemap filter i use this one https://github.com/ifb/vapoursynth-tonemap

[asarian]

You may like this one as it is GPU accelerated. Apply it after resizing on the GPU (if needed) for maximum performance.

This looks like just what the doctor ordered! :) Unfortunately, the pertinent thread points to an invalid rar: https://forum.doom9.org/showthread.php?t=175624


EDIT: Nevermind: found a newer version in the directory itself. :)

[videoh]

Latest version is always linked on my web page:

http://rationalqm.us/mine.html

[ChaosKing]

@videoh could you make a vapoursynth section on your webpage too? Or mark filters with native vapoursynth support somehow?

[videoh]

OK, will do. Thanks for the suggestion.

[asarian]

OK, will do. Thanks for the suggestion.

And, while we're at it, could you please also make DGDenoise native to vapoursynth? (so that DGDecodeNV.dll doesn't need to be loaded as legacy avs filter any more for it, with LoadPlugin).

[videoh]

And, while we're at it, could you please also make DGDenoise native to vapoursynth? (so that DGDecodeNV.dll doesn't need to be loaded as legacy avs filter any more for it, with LoadPlugin). The problem for me is that implementing both native Avisynth and native Vapoursynth versions creates a lot of code duplication and extra maintenance. I have not found a solution to this. Myrsloik has stated, and testing bears out, that there is no performance penalty for using the avscompat mechanism. So unless a compelling argument is advanced, I'll prefer to just make one version. If Avisynth could import Vapoursynth filters then I would reverse things. :)

I had already implemented native Vapoursynth for DGSource() before I settled on this policy.

[asarian]

The problem for me is that implementing both native Avisynth and native Vapoursynth versions creates a lot of code duplication and extra maintenance. I have not found a solution to this. Myrsloik has stated, and testing bears out, that there is no performance penalty for using the avscompat mechanism. So unless a compelling argument is advanced, I'll prefer to just make one version. If Avisynth could import Vapoursynth filters then I would reverse things. :)

I had already implemented native Vapoursynth for DGSource() before I settled on this policy.

Okay, thx. That sounds entirely reasonable. :) I tried it out last night on a re-encode, and was worried that maybe the avs route would be less stable somehow, but turned out to be rocksolid after all.

[asarian]

^^ P.S. I'm continually amazed at how fast the GPU stuff is. Naturally, my GTX 1080 Ti is a beast, but especially the GPU load (measured by GPU-Z) is extremely low: along with OpenCL support for x264, the gpu-assisted resize, and DGDenoise, the entire GPU load only dips into a few percent, every now then. nVidea, ftw! :)

[videoh]

^^ P.S. I'm continually amazed at how fast the GPU stuff is. Yes, it clearly outperforms CPU implementations and if you can eliminate unnecessary CPU<->GPU transfers (think CUDASynth) it blows CPU away into the next galaxy.

Naturally, my GTX 1080 Ti is a beast, but especially the GPU load (measured by GPU-Z) is extremely low: along with OpenCL support for x264, the gpu-assisted resize, and DGDenoise, the entire GPU load only dips into a few percent, every now then. nVidea, ftw! :) Who can argue with that? I like to remind people to look at the CPU utilization too, because savings can be applied to CPU things, especially encoding.

[asarian]

I like to remind people to look at the CPU utilization too, because savings can be applied to CPU things, especially encoding.

Consider my following script:

import vapoursynth as vs

core = vs.get_core ()
core.max_cache_size = 8192

core.avs.LoadPlugin ("C:/Program Files (x86)/dgdecnv/x64 Binaries/DGDecodeNV.dll")

vid = core.avs.DGSource (r'i:\jobs\inter.dgi', resize_w=1920, resize_h=1080, fulldepth=True)
vid = core.dghdrtosdr.DGHDRtoSDR (clip=vid, fulldepth=False)
vid = core.avs.DGDenoise (vid, strength=0.12, searchw=9)

vid.set_output ()



The typical CPU-utilization for a job like this is 99% CPU (for the x264 process), and about 0.3% for vspipe (YAY GPU!)

On a non-GPU accelerated jon, CPU saturation for the vspipe process is around 50% (with haf.QTGMC). So, yeah, whatever can be done in GPU, I'll take it! :)

[videoh]

So, yeah, whatever can be done in GPU, I'll take it! :) Have a look at DGBob() and PVBob() instead of QTGMC for deinterlacing if you are after blazing speed. Decomb is also implemented for GPU (DGTelecide and DGDecimate).

Maybe getting OT for this thread.

[asarian]

You may like this one as it is GPU accelerated. Apply it after resizing on the GPU (if needed) for maximum performance.

I'm loving it! :)

I noticed, however, that sometimes DGHDRtoSDR still leaves things too washed out, as it were. See: https://1drv.ms/u/s!AhSxhQ9g_mrMlEJQ6vC4Z8ExAKlg

Those are stills from the movie Divergent. The overlapped image (on the right, with the black bars) is from the original blu-ray; the underlying, big image is what DGHDRtoSDR produced (with default values).

In Photoshop, I needed +48 contrast to have it look kinda like the blu-ray. Sadly, I found no contrast parameter for DGHDRtoSDR (brightness didn't seem to affect much of anything). Is there a way to achieve this?

Thanks.

EDIT: 'what DGHDRtoSDR produced' from the UHD source, of course. :)

[videoh]

Sadly, I found no contrast parameter for DGHDRtoSDR (brightness didn't seem to affect much of anything). Is there a way to achieve this? Use Tweak() in your Avisynth script.

[asarian]

Use Tweak() in your Avisynth script.

Thanks. I see it's part of havsfunc itself. :)

[RainyDog]

Is it possible to use fmtconv to resize and crop together?

For example, in avisynth you can the below script when the exact crop is a row of 21 black pixels at both the top and bottom to avoid either overcropping or undercropping by 2 pixels.

Spline36Resize(1920, 1038, 0, 21, -0, -21)

Thanks.

[sneaker_ger]

See documentation about "resample" "sx", "sy", "sw", "sh" parameters.

sx, sy

Coordinate of the top-left corner of the picture sub-area used as source for the resizing. They can be fractional. If negative, the picture is extended by replicating the left pixel column.

These parameters are arrays, so it’s possible to specify a different value for each plane. The last value is used for the unspecified planes. The coordinates are always related to the pixel dimensions, you don’t need to scale them with the chroma subsampling.

sw, sh

Size in pixels of the sub-area to resize. They can be fractional. If 0, the area has the same size as the source clip. If negative, they define coordinates relative to the bottom-right corner, in a Crop-like manner. These parameters are arrays like sx and sy.

[zorr]

If you are using the dll from the official release, I will recommend to use the one from ChaosKing's FATPACK or the one in my signature as it is compiled with a newer compiler and might save you from random crashes.

I'm using the FATPACK version but fmtconv.dll is dated March 2016 and has the exact same date as the official version. Is the more recent (and hopefully less buggy (https://forum.doom9.org/showthread.php?p=1866388#post1866388)) version still available somewhere?

[asarian]

I'm using the FATPACK version but fmtconv.dll is dated March 2016 and has the exact same date as the official version. Is the more recent (and hopefully less buggy (https://forum.doom9.org/showthread.php?p=1866388#post1866388)) version still available somewhere?

I'm using the one Wolfberry linked to (his own version), but the link doesn't seem valid any more. Sad, because that one is very stable.

[zorr]

I'm using the one Wolfberry linked to (his own version), but the link doesn't seem valid any more. Sad, because that one is very stable.

Could you share the file? It could be included in the next FATPACK.

[asarian]

Could you share the file? It could be included in the next FATPACK.

Here it is: fmtconv.dll (https://1drv.ms/u/s!AhSxhQ9g_mrMlgeOR7S66IbtuWOw)

It's for 64-bit, btw.

[poisondeathray]

wolfberry has 3 public shared fmtconv versions, which one are you linking to? or is it a different one ?
https://drive.google.com/drive/folders/1xZQABtoaSFgGu11YstmHKYLzO3elemlC

[asarian]

wolfberry has 3 public shared fmtconv versions, which one are you linking to? or is it a different one ?
https://drive.google.com/drive/folders/1xZQABtoaSFgGu11YstmHKYLzO3elemlC

If I recall correctly, he had a separate vapoursynth directory there (which I couldn't find any more earlier either). The fmtconv version I took was dated 8 Dec 2018 (which I don't see any more either). Said directory had things like a new libdfttest too.

[wonkey_monkey]

A warning, as people still seem to be posting directly to threads and may not be aware:

It appears that the forum may have been hacked. There is a suspicious "test" announcement, apparently from tebasuna51 (but probably not), parts of the forum are not working, and there appear to be some malicious javascript files.

I'm not speaking in any official capacity here, but I would recommend, at the very least, NOT entering your password anywhere on Doom9 for the time being.

Please refer to this post: https://forum.doom9.org/showthread.php?goto=newpost&t=176128

[asarian]

A warning, as people still seem to be posting directly to threads and may not be aware:

It appears that the forum may have been hacked. There is a suspicious "test" announcement, apparently from tebasuna51 (but probably not), parts of the forum are not working, and there appear to be some malicious javascript files.

I'm not speaking in any official capacity here, but I would recommend, at the very least, NOT entering your password anywhere on Doom9 for the time being.

Please refer to this post: https://forum.doom9.org/showthread.php?goto=newpost&t=176128


The guy that did speak in an official capacity ('Swede'), is he legit?! He has locked his announcement, so I can't react, but suggests ppl change their password... which would be an ideal hack (as you need to enter your old password on the form there too) if it's illegit.

[videoh]

Swede is legit.

[asarian]

Swede is legit.

Puts my mind at ease. :) Thanks.

[groucho86]

Hi everyone,
Just opened up an issue on fmtconv's github (https://github.com/EleonoreMizo/fmtconv/issues/12), wondering if anyone had any insight here.

I have a SGamut3.Cine Linear EXR. It seems like the Linear to Slog3 conversion is clipping in the highlights.

import vapoursynth as vs
import muvsfunc
core = vs.get_core()

input_path = 'source_sgamut

c = core.imwri.Read(input_path,float_output=True)
c = core.fmtc.transfer (clip=c, transs='linear', transd='slog3')
c.set_output()

Test material is here:
https://github.com/EleonoreMizo/fmtconv/files/3691493/linear_to_slog3_tb.zip

Includes:
source_sgamut3cine_linear.exr <-- source file
resolve_delinearized.tif <-- expected result
vs_delinearized.png <-- fmtconv result

[age]

Yes it looks so

import vapoursynth as vs
import muvsfunc
core = vs.get_core()

input_path = 'source_sgamut

c = core.imwri.Read(input_path,float_output=True)
#c = core.std.Expr(clips=[c], expr=" x 0.9 *")
c = core.std.Expr(clips=[c], expr=" x 0.01125000 >= x 0.01 + 0.19 / log 10 log / 261.5 * 420 + 1023 / 171.2102946929 95 - x * 0.01125000 95 + / 1023 / ?")
c.set_output()


Not sure if the resolve result is right either, maybe there is need to scaling from linear to reflection multiply by 0.9

https://github.com/videovillage/Lattice-Issues/issues/17

https://pro.sony/s3/cms-static-content/uploadfile/06/1237494271406.pdf

[groucho86]

Thanks age! Your expr matches well to the DCTL I am using in Resolve. The 0.9 does not seem relevant in this case.

Any idea why fmtconv would be clipping? I looked at TransOpSLog3.cpp and nothing's jumping out at me. Maybe I'm not looking in the right place.

[Cary Knoop]

Hi everyone,
Just opened up an issue on fmtconv's github (https://github.com/EleonoreMizo/fmtconv/issues/12), wondering if anyone had any insight here.

I have a SGamut3.Cine Linear EXR. It seems like the Linear to Slog3 conversion is clipping in the highlights.

What is the bitdepth of the EXR and it is a float representation?

[groucho86]

Hey Cary,
EXR is 16bit half float. Sample file is downloadable here (https://github.com/EleonoreMizo/fmtconv/files/3691493/linear_to_slog3_tb.zip).

exrheader inspection:
$ exrheader source_sgamut3cine_linear.exr

file source_sgamut3cine_linear.exr:

file format version: 2, flags 0x0
channels (type chlist):
B, 16-bit floating-point, sampling 1 1
G, 16-bit floating-point, sampling 1 1
R, 16-bit floating-point, sampling 1 1
compression (type compression): none
dataWindow (type box2i): (0 0) - (319 239)
displayWindow (type box2i): (0 0) - (319 239)
framesPerSecond (type rational): 24/1 (24)
lineOrder (type lineOrder): increasing y
pixelAspectRatio (type float): 1
screenWindowCenter (type v2f): (0 0)
screenWindowWidth (type float): 1
timeCode (type timecode):
time 00:00:00:00
drop frame 0, color frame 0, field/phase 0
bgf0 0, bgf1 0, bgf2 0
user data 0x0
type (type string): "scanlineimage"

[redbtn]

Is there any differences between fmtc Spline36 and internal Vapoursynth Spline36? If yes, what is better for HDR downscale? Thanks!

[Jukus]

better for HDR downscale
There is information on these pages:
http://www.vapoursynth.com/doc/functions/resize.html
http://avisynth.nl/index.php/Resize
But some say it is out of date.
I don’t know, I use bicubic with b=0,c=0.5.

[cretindesalpes]

Any idea why fmtconv would be clipping? I looked at TransOpSLog3.cpp and nothing's jumping out at me. Maybe I'm not looking in the right place.

I just fixed this issue. The clipping was caused by an inappropriate lookup table selection. There is a new release, check fmtconv r21 (http://forum.doom9.org/showthread.php?t=166504).
Sorry for the trouble! It’s been years since I worked on fmtconv so it was a bit difficult to dive into the code again.

[redbtn]

I just fixed this issue. The clipping was caused by an inappropriate lookup table selection. There is a new release, check fmtconv r21 (http://forum.doom9.org/showthread.php?t=166504).

Sorry for the trouble! It’s been years since I worked on fmtconv so it was a bit difficult to dive into the code again.Thank you for new release!
Glad to see you!

[ChaosKing]

Could you also make a release on github? :thanks:

[Pat357]

If you are using the dll from the official release, I will recommend to use the one from ChaosKing's FATPACK or the one in my signature as it is compiled with a newer compiler and might save you from random crashes.

I checked the files in your signature but couldn't find the newly compiled fmtconv.dll.
Can you tell me the name of the file ?
Thank you !

[ChaosKing]

Just use the new r21 version posted by cretindesalpes.

[cretindesalpes]

I actually discovered a bug in the primaries conversion code. When converting to ciexyz primaries, there is a divide by zero error due to the coordinates of the primaries. Hopefully it gets fixed!
Fixed. Thanks for your report!

[cretindesalpes]

When "1886a" (alternative approximation of 1886) transfer function is used before and after primaries conversion, it creates heavy pink artifacts on bright areas of the video.
Fixed now.

[Pat357]

Fixed now.

Any link to the "fixed" version ? (Windows binaries)

The link to r21 still has 8/12/2019 date-stamp :o

[cretindesalpes]

There is no new release yet, but there will be soon a r22 with all the fixes. In the meantime, it is possible to compile the source code from the git repository.

[cretindesalpes]

Here it is. fmtconv r22 (http://forum.doom9.org/showthread.php?t=166504):
bitdepth: upconversions for full range data now scale to the maximum value instead of shifting bits. Thanks to Z4ST1N for the report.
matrix: added support for the YDzDx, ICtCp-PQ and ICtCp-HLG colorspaces.
matrix: fixed a case issue for matrix identifiers.
primaries: added support for P3-DCI, P3-D65 and P3-D60. The old DCI-P3 was actually P3-D65. The code "dcip3" remains for backward compatibility but is deprecated.
primaries: added support for the EBU 3213-E colorspace.
primaries: added a "2100" alias for the BT.2020/BT.2100 colorspace.
primaries: fixed a divide-by-0 crash when using CIEXYZ as destination colorspace
primaries: fixed a bug preventing to use the "uhdtv" string for BT.2020 primaries
transfer: added support for hybrid log-gamma (HLG) curves.
transfer: for the SMPTE SM 2084 curve, the specific value of 0 now gives a true 0 in both directions (minor change).
transfer: BT.2020 curve for 12-bit data now uses exact values giving a continuous derivative (minor change).
transfer: now uses more accurate constants for the sRGB curve, giving a better derivative continuity (minor change).
transfer: fixed a bug with the alternate formula for BT.1886 curve.
Should now build correctly on FreeBSD (patch by jbeich)
Fixed compilation for Linux on ARM or aarch64. Binaries not tested yet.

[redbtn]

Here it is. fmtconv r22 (http://forum.doom9.org/showthread.php?t=166504)
Thank you!
Can I ask why spline64 and spline taps=4 doesn't produce the same result?

[cretindesalpes]

Can I ask why spline64 and spline taps=4 doesn't produce the same result?
This is because the splineNN algorithms are hardcoded plain cubic splines (see here (https://forum.doom9.org/showthread.php?t=147117) for details), while the spline + taps are natural cubic splines, as implemented by Wilbert in SplineResize 0.2 (http://wilbertdijkhof.com/).

[Pat357]

I have some problems building fmtconv for x64 using Mingw64.

Created an issue on your git : https://github.com/EleonoreMizo/fmtconv/issues/14

After applying the patch I attached at the issue https://github.com/EleonoreMizo/fmtconv/files/3964030/fmtc.patch.txt , the building works ok, but I'm not sure my patch doesn't mess up anything.....

[cretindesalpes]

I have some problems building fmtconv for x64 using Mingw64.
Thank you, the patch was correct. I did several other modifications. Please check the git repository.

[dev-null]

One of the primary issues I run into, when building myself a VapourSynth setup for my outdated OS X system (10.10), was a broken/ non working build of "fmtconv".

I fetched the r22 release and build it with clang ("clang-700.1.81", as included in XCode 7.2, last available for Yosemite). It compiled w/o a hitch, just had the known linking problem with a missing libatomic (the GCC thingy). Removing the respective -latomic flag fixed that. But calling any "fmtc" stuff from a VapourSynth script made vspipe never return. It was busy looping a single thread @ ~100% CPU usage. A build of the older r21 release works fine. Also a build of r22 with gcc 9.2 works fine too (no -latomic linking problem this way either.) So there's some sort of change in r22 that makes older clang mess up the build.

Interestingly I can't build the current master version from git with old clang nor gcc 9.2. The compilation fails rather similarly on stuff in src/conc/Interlocked.hpp. Here's the gcc9.2 output: https://pastebin.com/3NWWBJKW . Sadly I can't wrap my brain around this C++ stuff, so I have no idea what it wants to do there and how I could fix it. :D Also interesting is that this Interlocked stuff seems to be the stuff where the broken r22 clang build has trouble with. That's according to a process sample taken with Activity Monitor: it seems to hang around some of those "OSAtomicCompareAndSwap" functions in the stack trace.

Just reporting. Maybe someone has run into the same issue and knows details why it happens exactly: compiler bug, fmtconv's code broken, etc.? Anyway… to make it short: if you got busy-looping issues with fmtc[onv] stuff on an outdated Mac OS X system, try a build of the lib with gcc-9.2! :)

[Cary Knoop]

Any idea why this fails on a ProRes4444 format? I presume it is the alpha channel.


c = core.ffms2.Source("prores4444.mov")
c = core.fmtc.bitdepth(c, bits=16)


Error:
c = core.fmtc.bitdepth(c, bits=16)
File "src\cython\vapoursynth.pyx", line 1862, in vapoursynth.Function.__call__
vapoursynth.Error: bitdepth: argument clip is not of array type but more than one value was supplied

Edited to add:

Found out the solution myself, if an alpha channel is present two clips will be returned.
This works fine:

c = core.ffms2.Source("prores4444.mov")
c = core.fmtc.bitdepth(c[0], bits=16)

[feisty2]

Found out the solution myself, if an alpha channel is present two clips will be returned.


fun fact: python has language level support for functions with multiple return values, so this works even better (for the sake of readability)


c, alpha = core.ffms2.Source("prores4444.mov")

[Cary Knoop]

fun fact: python has language level support for functions with multiple return values, so this works even better (for the sake of readability)


c, alpha = core.ffms2.Source("prores4444.mov")

Cooool!

[Blue_MiSfit]

THANK YOU for making this tool and the impeccable documentation. I've been remastering some old SD content recently into HD and this has been extremely helpful for doing the transfer and primaries conversion!

[Selur]

A Hybrid user reported this issue to me and I can reproduce it on my system.
using https://github.com/EleonoreMizo/fmtconv/releases/tag/r22
with
# Imports
import vapoursynth as vs
core = vs.get_core()
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# source: 'F:\TestClips&Co\files\YUV\yuv422_16bitBitEndianStereo.mov'
# current color space: YUV422P8, bit depth: 0, resolution: 848x352, fps: 23.976, color matrix: 709, yuv luminance scale: limited, scanorder: progressive
# Loading F:\TestClips&Co\files\YUV\yuv422_16bitBitEndianStereo.mov using LWLibavSource
clip = core.lsmas.LWLibavSource(source="F:/TestClips&Co/files/YUV/yuv422_16bitBitEndianStereo.mov", format="YUV422P8", cache=0, prefer_hw=0)
# making sure input color matrix is set as 709
clip = core.resize.Point(clip, matrix_in_s="709",range_s="limited")
# making sure frame rate is set to 23.976
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
original = clip
# cropping the video to 842x348
clip = core.std.CropRel(clip=clip, left=0, right=6, top=2, bottom=2)
# cropping the video to 842x348
original = core.std.CropRel(clip=original, left=0, right=6, top=2, bottom=2)
clip = core.fmtc.resample(clip=clip, kernel="spline16", w=1280, h=530, interlaced=False, interlacedd=False)
original = core.fmtc.resample(clip=original, kernel="spline16", w=1280, h=530, interlaced=False, interlacedd=False)
# adjusting output color from: YUV422P16 to YUV420P8 for x264Model (i420@8-bit)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, range_s="limited")
# adjusting for FilterView
#if original.format.id != clip.format.id:
# if (original.format.color_family == vs.RGB and clip.format.color_family != vs.RGB):
#original = core.resize.Bicubic(original, format=clip.format.id, matrix_s="470bg", range_s="limited")
#elif (original.format.color_family == clip.format.color_family):
# original = core.resize.Bicubic(original, format=clip.format.id, range_s="limited")
# else:
# original = core.resize.Bicubic(original, format=clip.format.id, matrix_in_s="470bg", range_s="limited")
stacked = core.std.StackHorizontal([original,clip])
# set output frame rate to 23.976fps
stacked = core.std.AssumeFPS(clip=stacked, fpsnum=24000, fpsden=1001)
# Output
stacked.set_output()
and
I:\Hybrid\64bit\Vapoursynth>VSPipe.exe c:\Users\Selur\Desktop\test.vpy e:\Test.avi


I:\Hybrid\64bit\Vapoursynth>VSPipe.exe c:\Users\Selur\Desktop\test.vpy e:\Test.raw


I:\Hybrid\64bit\Vapoursynth>VSPipe.exe c:\Users\Selur\Desktop\test.vpy e:\Test.raw

Output 80 frames in 0.29 seconds (274.31 fps)

I:\Hybrid\64bit\Vapoursynth>VSPipe.exe c:\Users\Selur\Desktop\test.vpy e:\Test.raw

Output 80 frames in 0.29 seconds (272.45 fps)

I:\Hybrid\64bit\Vapoursynth>VSPipe.exe c:\Users\Selur\Desktop\test.vpy e:\Test.raw


I:\Hybrid\64bit\Vapoursynth>VSPipe.exe c:\Users\Selur\Desktop\test.vpy e:\Test.raw

Output 80 frames in 0.29 seconds (272.80 fps)

I:\Hybrid\64bit\Vapoursynth>VSPipe.exe c:\Users\Selur\Desktop\test.vpy e:\Test.raw


I:\Hybrid\64bit\Vapoursynth>
it randomly crashes,... every few times,..
Changing the source filter from LWLibavSource to FFMS2

core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")
clip = core.ffms2.Source(source="F:/TestClips&Co/files/YUV/yuv422_16bitBitEndianStereo.mov",cachefile="E:/Temp/mov_f1203bb1621d87009c15418dab3b00fb_853323747.ffindex",format=vs.YUV422P8,alpha=False)

doesn't help, it even seems that it crashed more often with ffms2.

removing:
clip = core.fmtc.resample(clip=clip, kernel="spline16", w=1280, h=530, interlaced=False, interlacedd=False)
original = core.fmtc.resample(clip=original, kernel="spline16", w=1280, h=530, interlaced=False, interlacedd=False)
and everything works.
Changing "spline16" to "spline64" same thing happens, but it seems to happen less frequently.
(same for spline36)
When using 'cubic' is seems to always crash.

Cu Selur

Ps.: Uploaded the clip I used to my GoogleDrive (https://drive.google.com/file/d/1X8gLxWSXqbsWe4QJPqwCUBcy93zjhfiN/view?usp=sharing) (~46MB)

[groucho86]

Yes it looks so

import vapoursynth as vs
import muvsfunc
core = vs.get_core()

input_path = 'source_sgamut

c = core.imwri.Read(input_path,float_output=True)
#c = core.std.Expr(clips=[c], expr=" x 0.9 *")
c = core.std.Expr(clips=[c], expr=" x 0.01125000 >= x 0.01 + 0.19 / log 10 log / 261.5 * 420 + 1023 / 171.2102946929 95 - x * 0.01125000 95 + / 1023 / ?")
c.set_output()


Not sure if the resolve result is right either, maybe there is need to scaling from linear to reflection multiply by 0.9

https://github.com/videovillage/Lattice-Issues/issues/17

https://pro.sony/s3/cms-static-content/uploadfile/06/1237494271406.pdf

age, for posterity's sake, I just realized that the second half of the lin to slog3 expression (the one dealing with values below 0.01125000) was not working as expected.

The proper RPN expr for linear to slog3 is:

x 0.01125000 >= x 0.01 + 0.19 / log 10 log / 261.5 * 420.0 + 1023.0 / x 171.2102946929 95.0 - * 0.01125000 / 95.0 + 1023.0 / ?

[MysteryX]

Running this code in VsEdit gives this error


video = core.resize.Point(video, format=vs.GRAY16)
video = core.fmtc.transfer(video, transs="709", transd="linear", fulls=0, fulld=1)
video = core.resize.Point(video, format=vs.GRAY8)



Error on frame 0 request:
Resize error -1: invalid graph state L910: !m_state.has_chroma() || m_state.plan

[Selur]

@MysteryX: I can reproduce the issue unless I set the _ColorRange property beforehand:

# Setting color range to TV (limited) range.
video = core.std.SetFrameProp(clip=video , prop="_ColorRange", intval=1)


# Setting color range to PC (full) range.
video = core.std.SetFrameProp(clip=video , prop="_ColorRange", intval=0)


Cu Selur

[cretindesalpes]

Running this code in VsEdit gives this error

video = core.resize.Point(video, format=vs.GRAY16)
video = core.fmtc.transfer(video, transs="709", transd="linear", fulls=0, fulld=1)
video = core.resize.Point(video, format=vs.GRAY8)


Can be fixed by removing the _ColorRange property before sending it to output:
video = core.std.SetFrameProp (video, prop="_ColorRange", delete=True)
video.set_output()
The problem exists even without calling fmtconv, just by setting the property manually to 0 (full range). There’s nothing in the VS doc that forbids to attach a _ColorRange property set to 0 to a GRAY8 or GRAY16 clip, so I think it’s probably a bug related to VS Editor.

[cretindesalpes]

fmtconv r23 (http://forum.doom9.org/showthread.php?t=166504):

transfer: added an Exposure Index (EI) parameter for the Arri Log C Alexa 2.x and 3.x curves.
bitdepth: properly sets the _ColorRange attribute.
Doesn’t output a debug message when AVSTP is not found.
Fixed a concurrency issue by using a more recent toolkit when compiling with MSVC.
Windows XP is not supported any more.

[kedautinh12]

Please can you port to avisynth??

[feisty2]

dither is the equivalent toolkit for avisynth.

[kedautinh12]

dither is the equivalent toolkit for avisynth.

I don't think so
https://forum.doom9.org/showpost.php?p=1943559&postcount=1131

[cretindesalpes]

I already ported bitdepth, matrix and matrix2020cl to Avisynth+, so we’re halfway now. I’ll start a thread in the Avisynth forum when it is ready.

[cretindesalpes]

fmtconv r24 (https://forum.doom9.org/showthread.php?t=166504):
bitdepth: added dithering mode 9: quasirandom sequences.
bitdepth: added a triangular probability distribution function (TPDF) for the dithering patterns and noises, along with the associated parameters tpdfo and tpdfn.
bitdepth: added corplane parameter to prevent colored noise in RGB processing.
bitdepth: fixed crash when trying to change the range (full or TV) without reducing the bitdepth.
matrix: deletes the _Matrix and _ColorSpace properties if a non-standard matrix is used.
matrix2020cl: sets the _Matrix and _Transfer frame properties.
matrix2020cl: fixed hideous colors when converting slightly out-of-colorspace values from integer Y’Cb’Cr’ to integer linear RGB.
resample: totalh and totalv are now correctly taken into account, added total parameter too.
resample: added a planes parameter.
transfer: sYCC curve (similar to sRGB) now supports signed values.
transfer: added a planes parameter.
Added support for Avisynth+
Added information on functions reading and writing frame properties.
Added more support for 14-bit data.
Vapoursynth: still uses API v3.6 but can use Python constants from future API v4.0.

I did a lot of internal changes in this release because of the Avisynth+ port so I hope I haven’t broke too many things. The next thing I’ll do is probably adding reference OOTFs (direct and inverse) to transfer to fully support the BT.2100 HDR pipeline and scene- or display-referred linear RGB.

[kedautinh12]

Thanks

[cretindesalpes]

I’m very likely to remove the planes parameter from fmtc_transfer introduced in r24. I added it because it was almost free, but with the upcomping addition of OOTFs, planes are no longer independant, and the parameter is making the implementation too complicated for my taste. Any objection?

[poisondeathray]

Some issue with either transfer or primaries function in r24, that worked ok with r22 in vapoursynth version x64. I didn't test x86 or avs version

When converting ProPhoto (romm) or AdobeRGB to sRGB, it works ok in r22 . But in r24, there is some clipping or some errors in some channels

Best way is to see example:

Demo package ,

https://www.mediafire.com/file/k5rfblwvpi7tkwp/OxbowBoots_AdobeRGB,ProPhoto,sRGB.zip/file

(src photos from https://furbo.org/color/WideGamut/)


srgb = core.imwri.Read(r'Oxbow_Boots_sRGB.jpg')
prophoto = core.imwri.Read(r'Oxbow_Boots_ProPhoto.jpg')
adobe = core.imwri.Read(r'Oxbow_Boots_AdobeRGB.jpg')

atos = core.fmtc.bitdepth (adobe, bits=32)
atos = core.fmtc.transfer (atos, transs="adobergb", transd="linear")
atos = core.fmtc.primaries (atos, prims="adobe98", primd="srgb")
atos = core.fmtc.transfer (atos, transs="linear", transd="srgb")
atos = core.resize.Point(atos, format=vs.RGB24)

ptos = core.fmtc.bitdepth (prophoto, bits=32)
ptos = core.fmtc.transfer (ptos, transs="romm", transd="linear")
ptos = core.fmtc.primaries (ptos, prims="romm", primd="srgb")
ptos = core.fmtc.transfer (ptos, transs="linear", transd="srgb")
ptos = core.resize.Point(ptos, format=vs.RGB24)

i = core.std.Interleave(clips=[srgb, atos, ptos])

i.set_output()

[cretindesalpes]

Thank you, I can reproduce the issue here (dark tones with transfer). I’m going to check what is wrong.

EDIT: it was the linear part of the sRGB curve that was wrong from r24. Fixed in commit f7cd019, will be available in the next full release or hot fix update.

[cretindesalpes]

fmtconv r25 (https://forum.doom9.org/showthread.php?t=166504) is mostly a bug-fix release:
bitdepth/Avisynth+: fixed I420 input that couldn’t be converted to higher bitdepth. Thanks to StvG for the report.
matrix: fixed the output colorspace autodetection which failed with specific matrix combinations.
resample/Avisynth+: fixed fulls and fulld parameters which were wrongly defined as int instead of bool.
resample/Avisynth+: fixed cplaced which was wrongly copied from cplaces. Thanks to TbtBI for the report.
transfer: automatically adapts the output format (16-bit integer) if the input is a low-bitdepth clip and nothing is specified in bits nor flt, instead of emitting an error.
transfer: fixed the BT.2100 HLG curve that was inverted.
transfer: fixed the sRGB curve linear part and extended its positive range. Thanks to poisondeathray for the report.
Avisynth+: fixed a crash occuring when using data bitdepths between 9 and 14. Thanks to tormento for the report.

[Etroxamin]

hey, i have a (hopefully) small problem:

i've converted YUV 8bit 420 to RGB 32bit 444, looks like this:

src = core.fmtc.bitdepth(src, bits=32)
src = core.fmtc.resample (src, css="444")
src = core.fmtc.matrix(src, mat="709", col_fam=vs.RGB, bits=32)

that works without problems.

now change it to YUV 10bit 420:

src = core.fmtc.bitdepth (src, bits=10)
src = core.fmtc.matrix(src, mat="709", col_fam=vs.YUV, bits=10)
src = core.fmtc.resample (src, css="420")

that works, but gives me a brighter picture, perhaps because of full range instead of tv.

i just started with VS scripts and could need some help here. How to do that?
Thanks in advance.

[poisondeathray]

hey, i have a (hopefully) small problem:

i've converted YUV 8bit 420 to RGB 32bit 444, looks like this:

src = core.fmtc.bitdepth(src, bits=32)
src = core.fmtc.resample (src, css="444")
src = core.fmtc.matrix(src, mat="709", col_fam=vs.RGB, bits=32)

that works without problems.

now change it to YUV 10bit 420:

src = core.fmtc.bitdepth (src, bits=10)
src = core.fmtc.matrix(src, mat="709", col_fam=vs.YUV, bits=10)
src = core.fmtc.resample (src, css="420")

that works, but gives me a brighter picture, perhaps because of full range instead of tv.

i just started with VS scripts and could need some help here. How to do that?
Thanks in advance.



You can specify fulls and fulld (s for source, d for destination) in the fmtc.matrix lines

[Etroxamin]

You can specify fulls and fulld (s for source, d for destination) in the fmtc.matrix lines

hey, thanks for your help, but i just started with vs scripts and this doesnt't really help me to know what to do, could you eventually provide me with a full example?

thanks in advance.

[poisondeathray]

hey, thanks for your help, but i just started with vs scripts and this doesnt't really help me to know what to do, could you eventually provide me with a full example?

thanks in advance.


Why are you doing this in that manner ? Why not 8bit420 to 10bit420 directly ?

fmtc.Resample always outputs 16bit int or 32bit float. So it does not make sense to use fmtc.bitdepth to downconvert to 10bit then up, then down again

From the documentation:

Output is always 16-bit integer (default for integer input) or 32-bit float. Use fmtc.bitdepth to convert the result to a lower bitdepth.



eg. if your input is limited range, fulls=False (limited range YUV) to full range RGB. Then the reverse when you go from RGB (full) to YUV (limited)


src = core.fmtc.bitdepth(src, bits=32)
src = core.fmtc.resample (src, css="444")
src = core.fmtc.matrix(src, mats="709", fulls=False, fulld=True, col_fam=vs.RGB)

src = core.fmtc.matrix(src, mat="709", fulls=True, fulld=False, col_fam=vs.YUV)
src = core.fmtc.resample (src, css="420")
src = core.fmtc.bitdepth (src, bits=10) #set your dither mode, default is 3 Sierra-2-4A error diffusion

[Etroxamin]

Why are you doing this in that manner ? Why not 8bit420 to 10bit420 directly ?

Well, i just try out a few things and see how they work.

The source is 8bit yuv 420, anime with a lot of ugly artefacting, as it has dynamic grain added. I convert it to 16 bit, with hope that certain filters will work better in 16 bit, than convert it to 32bit rgb 444, cause i try with Waifu up/down sampling to remove some grain, but on the other side also to give more sharpness at the same time, I tried to mimic this sharpening, but it looks better with waifu up/down, or else i would need to use dehaloing, but that may will destroy fine white lines that are no halos. But i still need to test if waifu suits for videos ...

btw, someone recommended this:

src = core.resize.Bicubic(src, format=vs.RGBS, matrix_in_s='709')

src = core.resize.Bicubic(src, format=vs.YUV420P10, matrix_s='709', dither_type='none')

when i use this the higher brightness doesn't appear, but i will try your example as well, thanks a lot for that btw.

And ... there seem to be a few more ways to do these conversions, can you recommand a "way" eventually? As said, i am quite new to VS scripts, so any recommendations would be helpful.

[poisondeathray]

And ... there seem to be a few more ways to do these conversions, can you recommand a "way" eventually? As said, i am quite new to VS scripts, so any recommendations would be helpful.

Yes, more than one way to do the same thing

I prefer internal core.resize , for general use because fewer lines to write.

But there are some functions that fmtc has that resize does not (eg. some transfer functions, some kernels, some different dithering patterns, a few other things)

[AMED]

Sorry a bit of a stupid question but is there any reason to use core.fmtc.resample over core.resize.Spline36 for resizing the blurays from 1080p to 720p?

Possibly just overkill?

[Yomiko]

In fmtc you can specify cropping arguments for each plane, so if the source has chroma shift to fix, you will see the convenience.

[MysteryX]

When using bitdepth, do I need to set fulls and fulld on every call? Or does fulld take fulls by default? Or does it read frame properties?

[cretindesalpes]

Like other fmtconv functions, bitdepth assumes that the range is full for RGB and YCgCo, and narrow for the others (namely Y with or without UV). So there is no need to set the full* parameters if you know the default values are valid for your use. At the moment, the frame properties are not used to detect the range, I don’t know if there is a need for this.

[MysteryX]

Like other fmtconv functions, bitdepth assumes that the range is full for RGB and YCgCo, and narrow for the others (namely Y with or without UV). So there is no need to set the full* parameters if you know the default values are valid for your use. At the moment, the frame properties are not used to detect the range, I don’t know if there is a need for this.
When converting bitdepth, it doesn't change the format and shouldn't change the range. After reading the frame property, for YUV conversion, do I need to set BOTH fulls and fulld on every call?

So far it has been converting YUV full-range clips with default fulls=false and fulld=false. Output seems "fine", but what's the impact of that?

[poisondeathray]

When converting bitdepth, it doesn't change the format and shouldn't change the range. After reading the frame property, for YUV conversion, do I need to set BOTH fulls and fulld on every call?

So far it has been converting YUV full-range clips with default fulls=false and fulld=false. Output seems "fine", but what's the impact of that?

In terms of code values, no problems

But FrameProps for _ColorRange will be changed to "1" (limited range) . So if earlier source was flagged full range "0" for FrameProps, it will now say limited, which might be problematic. You can SetFrameProp to "0" to mark it full, or use fulls=True, fulld=True

[MysteryX]

Under what authority is it setting frame props if it isn't reading them?

If it's just that, fulld sets it; what is fulls for?

[cretindesalpes]

You can do a range conversion in the same step as bitdepth change. Or you can even change the range without changing the bitdepth.

Using a TV-range conversion with a full range signal works but the results will be slightly off: code for 100 % white is 255 in 8 bits and 1023 in 10 bits. With a TV-range conversion, 255 becomes 1020. It is generally not a big deal when converting between large bitdepth because the difference is really tiny.

[MysteryX]

That could explain the very slight luma shift I was seeing; as I'm performing several conversions.

IMO by default it should read and preserve frame properties, and setting fulls should also set fulld's default. Applying TV-PC range conversion should only be done when explicitly said.

[cretindesalpes]

fmtconv r26 (https://forum.doom9.org/showthread.php?t=166504):
matrix: fixed the output colorspace when using singleout.
matrix: fixed the output colorspace when using a custom matrix. Thanks to vxzms for the report.
matrix/Vapoursynth: _Matrix and _ColorSpace frame properties are actually deleted when a custom matrix is used or the final colorspace is unknown. Thanks to mysteryx93 for the report.
resample: fixed empty custom impulse that could cause a crash.
resample: added top-left chroma location.
resample/Avisynth+: interlacing detection now uses the global stream information when the frame properties are not available.
resample/Avisynth+: fixed the planes parameter.
transfer: Linear light can now be display- or scene-referred. Added sceneref, lw, lws, lwd, lb, ambiant and deprecated blacklvl.
transfer: By default, automatically matches the reference white levels for source and destination transfer curves. This may cause some backward incompatibilities. match parameter added.
transfer: Removed the planes parameter introduced in r24 because planes are no longer independent of each other.
transfer: debug provides information about the transfer operation and levels as a frame property.
transfer: BT.470M characteristic is now a pure power curve, instead of a copy of the sRGB curve.
transfer/Avisynth+: fixed flt + bits combination that was ignored. Thanks to DTL for the report.
transfer/Avisynth+: fixed case sensitivity to transs and transd.
transfer/Vapoursynth: fixed logceis and logceid that were missing from the function registration since their introduction in r23… Thanks to groucho86 for the report.
Fixed compatibility with Avisynth+ 3.7.1.
Avisynth+: fixed "RGBPxx" colorspace values that weren’t correctly interpreted as planar.

[cretindesalpes]

fmtconv r27 (https://forum.doom9.org/showthread.php?t=166504):
matrix: fixed coefficient errors when using singleout.
primaries: added Sony P22 primaries.
transfer: better selection between linear and logarithmic input for LUTs, LUT size reduced.
transfer: fixed crash with grey colorspace. Thanks to vxzms for the report.

[lansing]

I got an error trying to convert a GRAY8 to YV12,
"Error: Filter resample didn't set videoinfo", what does it mean?


clip = core.std.ShufflePlanes(clip, 0, colorfamily=vs.GRAY)
clip = core.fmtc.resample(clip, css="420")

[cretindesalpes]

resample cannot change the format like this. You can change the chroma subsampling using css only for a format which already has chroma channels. I’m going to make the error message more explicit.

[MysteryX]

For specifying matrix(mat=...), how can I convert INT value from frame property into a string value it will understand?

btw I did a performance test on 5K clip with built-in Bicubic vs FMTC

List of heaviest filters with vspipe --filter-time

Built-in Bicubic:

Script evaluation done in 7.41 seconds
Output 30 frames in 73.84 seconds (0.41 fps)
Filtername Filter mode Time (%) Time (s)
Bicubic parallel 73.07 53.96
Bicubic parallel 57.85 42.72
BM3D parallel 56.85 41.98
KNLMeansCL parreq 53.74 39.68


FMTC

Script evaluation done in 6.83 seconds
Output 30 frames in 63.83 seconds (0.47 fps)
Filtername Filter mode Time (%) Time (s)
resample parallel 64.69 41.29
KNLMeansCL parreq 56.66 36.17
matrix parallel 55.09 35.17
BM3D parallel 55.02 35.12


Considerably better! But still heavy. For 720p videos, resize is way down in the list and not an issue. Any idea what could be causing this performance bottleneck? I posted about the issue here. (https://github.com/vapoursynth/vapoursynth/issues/824)

The only thing that comes to my mind is CPU cache saturation as explained here (https://forum.doom9.org/showthread.php?t=173772).

[cretindesalpes]

fmtconv r28 (https://forum.doom9.org/showthread.php?t=166504):
bitdepth: Faster void and cluster pattern generation. Larger patterns are allowed.
matrix: Added an alias for "fcc"
primaries: New presets: Free Scale-gamut, DaVinci Wide Gamut, DRAGONcolor, DRAGONcolor2, REDcolor, REDcolor2, REDcolor3, REDcolor4 and REDWideGamutRGB.
primaries: Fixed Sony P22 primaries.
primaries: Removed the deprecated "dcip3" alias for "p3d65".
transfer: Added DaVinci Intermediate, RED Log3G10, REDlog, Cineon and Panalog.
Vapoursynth and Avisynth+ plug-ins are now contained in the same binary file.

[asarian]

I was wondering whether there's way to convert regular Bluray HD to UHD HDR10, using fmtconv? I saw several attempts here, but in all those examples something went wrong. A clean example would be nice. :)

[Blue_MiSfit]

I'm struggling a bit with some HDR conversions:

Source:
16 bit full range RGB
PQ, BT. 2020

Target:
12 bit limited range YCbCr 4:4:4
PQ, BT. 2020

(for ProRes XQ encoding)

With this script:


import vapoursynth as vs
core = vs.core

source = core.ffms2.Source("flattened_video.mov")

c = source

c = core.fmtc.matrix(source, mat="2100", bits=16)
c = core.fmtc.bitdepth(c, bits=12)

c.set_output()


I get something that's mostly right, but doesn't fully match the source in Resolve. Suggestions?

[cretindesalpes]

What kind of difference do you have? How could I reproduce your issue in Resolve?

[cretindesalpes]

fmtconv r29 (https://forum.doom9.org/showthread.php?t=166504):
Vapoursynth: switched to API v4. Fmtconv now requires Vapoursynth r55 or later, API v3 is not supported anymore.
resample: Fixed degnerated cases with invks causing a crash. Thanks to Dogway for the report.
resample/Avisynth+: fixed frame property writing when only cplaced is defined, thanks to Dogway for the report.
transfer: Added sigmoid curve as a transfer function.
transfer: Fixed the float to integer path.

[Blue_MiSfit]

Ok, here's a sample frame:

https://drive.google.com/file/d/1vJgzmUQmoAS02TikcwOaar80M-cVSLv3/view?usp=sharing

Again, this is 16 bit full range RGB in HDR with BT. 2020 primaries and ST. 2084 / PQ transfer. Things have evolved a bit since my last post, and now I want to produce two different types of output:

1) Limited range 4:2:0 10 bit (for HDR10 encoding with HEVC)

2) Limited range 4:4:4 12 bit (for ProRes XQ encoding)

Both seem totally fine if I keep full resolution. However, if I downscale (e.g. to 1080p) I'd like to do the scaling in linear light since this is best practice. When I do, a lot of the higher code values are clipped.

Examples: https://imgur.com/a/1K8iKG3

Good (... I think):

import vapoursynth as vs
core = vs.core

a = core.ffms2.Source("HDR_Ramp.tif")

#a = a.fmtc.transfer(transs="pq", transd="linear", fulls=True, fulld=True)
a = a.fmtc.resample(w=1920, h=1080)
#a = a.fmtc.transfer(transs="linear", transd="pq", fulls=True, fulld=True)
a = a.fmtc.matrix(mat="2020", fulls=True, fulld=False)
a = a.fmtc.resample(css="420")
a = a.fmtc.bitdepth(bits=10)

a.set_output()


Bad (clipped):

import vapoursynth as vs
core = vs.core

a = core.ffms2.Source("HDR_Ramp.tif")

a = a.fmtc.transfer(transs="pq", transd="linear", fulls=True, fulld=True)
a = a.fmtc.resample(w=1920, h=1080)
a = a.fmtc.transfer(transs="linear", transd="pq", fulls=True, fulld=True)
a = a.fmtc.matrix(mat="2020", fulls=True, fulld=False)
a = a.fmtc.resample(css="420")
a = a.fmtc.bitdepth(bits=10)

a.set_output()


Am I doing something wrong? The same pattern holds true if I omit the final resample (keeping 4:4:4) and change bitdepth to 12 to make 4:4:4 12 bit for ProRes XQ.

[poisondeathray]

Bad (clipped):

import vapoursynth as vs
core = vs.core

a = core.ffms2.Source("HDR_Ramp.tif")

a = a.fmtc.transfer(transs="pq", transd="linear", fulls=True, fulld=True)
a = a.fmtc.resample(w=1920, h=1080)
a = a.fmtc.transfer(transs="linear", transd="pq", fulls=True, fulld=True)
a = a.fmtc.matrix(mat="2020", fulls=True, fulld=False)
a = a.fmtc.resample(css="420")
a = a.fmtc.bitdepth(bits=10)

a.set_output()


Am I doing something wrong? The same pattern holds true if I omit the final resample (keeping 4:4:4) and change bitdepth to 12 to make 4:4:4 12 bit for ProRes XQ.

Do calcs in 32bit float
a = core.ffms2.Source...
a = a.fmtc.bitdepth(bits=32)
.
.
.

[Blue_MiSfit]

Wow, yeah that totally fixed it.

Why is this necessary? I understand how 32 bit float is certainly more precise than 16 bit integer, but I'd have imagined the differences being a whole lot more subtle than this!

On the off-chance, is it possible to reproduce this filter chain in ffmpeg?

[poisondeathray]

Wow, yeah that totally fixed it.

Why is this necessary? I understand how 32 bit float is certainly more precise than 16 bit integer, but I'd have imagined the differences being a whole lot more subtle than this!




Yes 16bit has higher precision, but 16bit int will still clip to [black,white] or [min,max] , or 0 to 65535 for 16bit code values. "Higher precision" because there are more "steps" described between black to white, but "higher precision" still clips

But the ST2084 PQ to linear transfer function will have values >1 in float ( > 65635 in 16bit) , or "whiter than white"



On the off-chance, is it possible to reproduce this filter chain in ffmpeg?


Yes, using zscale and format=gbrpf32le for float
https://ffmpeg.org/ffmpeg-filters.html#zscale-1


-vf zscale=w=1920:h=1080:transferin=smpte2084:transfer=linear,format=gbrpf32le,zscale=matrix=2020_ncl:transferin=linear:transfer=smpte2084,format=yuv420p10le

[Blue_MiSfit]

Thanks very much for the ffmpeg filter chain! I'll do some testing with that.

Thanks also for the explanation. After I thought about it a bit more this did indeed occur to me :)

[Blue_MiSfit]

Hmm ok so I had to add a couple of parameters to

a) Identify the input primaries as bt2020

and

b) Specify limited range output


This worked great for a TIF input. The output matches the source in Resolve perfectly:


ffmpeg -i HDR_Ramp.tif -vf zscale=w=1920:h=1080:pin=bt2020:p=bt2020:tin=smpte2084:t=linear,format=gbrpf32le,zscale=m=2020_ncl:tin=linear:t=smpte2084:r=limited,format=yuv420p10le -c:v libx265 -crf 10 reference2.mp4


However, as soon as I swapped over to using the IMF MXF J2K wrapped version of this (same image, just a lossless compressed version of the full range 16 bit RGB TIF), I get the same clipped whites:


ffmpeg -f imf -i .\CPL_FOX-TPS-1MIN-HDR10-ML7_20181105_OV.xml -map 0:v -vf zscale=w=1920:h=1080:pin=bt2020:p=bt2020:tin=smpte2084:t=linear,format=gbrpf32le,zscale=m=2020_ncl:tin=linear:t=smpte2084:r=limited,format=yuv420p10le -c:v libx265 -crf 10 -preset superfast -x265-params keyint=24 -t 1 scale_linear_08.mp4


That clipped white is visible both in MPC-HC and in Resolve.

Both formats seem to insert the same auto scaler from rgb48le to gbrp16le:


[auto_scale_0 @ 00000157ac51b040] picking gbrp16le out of 48 ref:rgb48le alpha:0
...
[auto_scale_0 @ 00000157ac51b040] w:3840 h:2160 fmt:rgb48le sar:0/1 -> w:3840 h:2160 fmt:gbrp16le sar:0/1 flags:0x0


I tried disabling auto conversion and explicitly adding format=gbrp16le to the head of the filter graph and that errors out:


[format @ 000001f0f4c5ea00] Setting 'pix_fmts' to value 'yuv420p|yuvj420p|yuv422p|yuvj422p|yuv444p|yuvj444p|gbrp|yuv420p10le|yuv422p10le|yuv444p10le|gbrp10le|yuv420p12le|yuv422p12le|yuv444p12le|gbrp12le|gray|gray10le|gray12le'
The filters 'graph 0 input from stream 0:0' and 'Parsed_format_0' do not have a common format and automatic conversion is disabled.
Error reinitializing filters!

[poisondeathray]

Hmm ok so I had to add a couple of parameters to

a) Identify the input primaries as bt2020

and

b) Specify limited range output


This worked great for a TIF input. The output matches the source in Resolve perfectly:


ffmpeg -i HDR_Ramp.tif -vf zscale=w=1920:h=1080:pin=bt2020:p=bt2020:tin=smpte2084:t=linear,format=gbrpf32le,zscale=m=2020_ncl:tin=linear:t=smpte2084:r=limited,format=yuv420p10le -c:v libx265 -crf 10 reference2.mp4



Interesting, I would have thought that in tif without any flags or metadata, pin=bt2020:p=bt2020 should be a no-op and do nothing, and that RGB to YUV limited range would be default setting... But you're right, they are required or the proper flags need to be set

But for current ffmpeg behaviour - it appears is the metadata for primaries in the output file isn't written , unless you set it in the zscale arguments, or the encoder settings. The actual video data is unaffected and still a no-op
Color primaries : BT.2020

(I guess it's better to be explicit and check assumptions)



However, as soon as I swapped over to using the IMF MXF J2K wrapped version of this (same image), I get the same clipped whites:


ffmpeg -f imf -i .\CPL_FOX-TPS-1MIN-HDR10-ML7_20181105_OV.xml -map 0:v -vf zscale=w=1920:h=1080:pin=bt2020:p=bt2020:tin=smpte2084:t=linear,format=gbrpf32le,zscale=m=2020_ncl:tin=linear:t=smpte2084:r=limited,format=yuv420p10le -c:v libx265 -crf 10 -preset superfast -x265-params keyint=24 -t 1 scale_linear_08.mp4


That clipped white is visible both in MPC-HC and in Resolve.



Not sure, there might be other metadata or flags in the imf that overrides some settings or assumptions. what pixel format for the imf ?

You can add -report to the command line and look ath the ffmpeg log for clues

[Blue_MiSfit]

Thanks. It's all still 16 bit full range RGB BT. 2020 / PQ in the IMF.

I'll start a new thread elsewhere about the ffmpeg stuff. Continuing on the VapourSynth track here I realized I'd missed some really nasty halos in the high contrast areas. Forcing kernel=bicubic helped this quite a lot.

Resting the eyes now, hopefully I'll get that sorted tomorrow :)

[mastrboy]

I have some unexpected behavior when doing 709->601 conversion having to run fmtc_matrix twice to get the expected result when compared to "old" dither based conversion.
I'm updating some of my ancient script and started to move to fmtc from Dither.

On the other hand 601->709 works as expected, the following two snippets create visually similear color output:
# old method
Dither_convert_8_to_16()
Dither_convert_yuv_to_rgb(matrix="601", output="rgb48y", lsb_in=true)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convert_rgb_to_yuv (r, g, b, matrix="709", lsb=false, mode=6)

# new method:
fmtc_resample (css="444")
fmtc_matrix (mats="601", matd="709")
fmtc_resample (css="420")
fmtc_bitdepth (bits=8)

But the following 709->601 snippets does not produce the similar results:
# old method
Dither_convert_8_to_16()
Dither_convert_yuv_to_rgb(matrix="709", output="rgb48y", lsb_in=true)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convert_rgb_to_yuv (r, g, b, matrix="601", lsb=false, mode=6)

# new method:
fmtc_resample (css="444")
fmtc_matrix (mats="709", matd="601")
fmtc_resample (css="420")
fmtc_bitdepth (bits=8)

But, if I call fmtc_matrix twice, like "fmtc_matrix (mats="709", matd="601").fmtc_matrix (mats="709", matd="601")" I get similar results to Dither.

What am I doing wrong here with fmtc? Or is it Dither that is wrong?

Edit: Seems the issue is with AVSPmod and not fmtc, after downgrading AVSPmod v2.6.2.8 I'm not able to reproduce this issue anymore. (Seems to be a bug introduced in AVSPmod somewhere between v2.6.2.9 - v2.7.1.3 where it looks like it fetches a cached frame somewhere in the filter pipeline...)

[cretindesalpes]

fmtconv r30 (https://forum.doom9.org/showthread.php?t=166504):
matrix: The _ColorRange frame property is now set when a matrix preset is used.
transfer: Added ACEScct transfer function.
primaries: Added DCI P3+ and Cinema Gamut presets.
primaries: Added wconv parameter for full conversion.
Changed the configure options to compile with Clang.
Updated datatypes in the examples.

[cretindesalpes]

fmtconv r31 (https://forum.doom9.org/showthread.php?t=166504):
bitdepth/Vapoursynth: Can work on clips of variable frame size. The function was already designed to handle this feature, but the parameter check was more strict than necessary. Thanks to Vardë for the report.
matrix: Added the SMPTE ST 2128 IPT-PQ-C2 transform to and from L’M’S’.
resample: Fixed 14 to 16 bit AVX2 conversion path, thanks to NSQY for the report.
resample: Fixed chroma placement: U and V vertical positions for interlaced PAL-DV were swapped and vertical subsampling > 2 was not handled at all. Thanks to Chortos-2 for the report.
transfer: Added L* transfer function.
transfer: Added Arri LogC4 transfer function.
transfer: Rescaled the S-Log3 curve to be used with limited-range code values, like the other S-Log curves.
primaries: Added Arri Wide Gamut 4 colorspace.
primaries: Fixed white XYZ coordinates for the NTSC-J colorspace (exact D93 location), thanks to Chortos-2 for the report.
Program path without x86 SIMD: fixed wrong conversions affecting a lot of functions (noticed on ARM/Apple). Thanks to SaltyChiang for the fix.
On Windows/x64, the Vapoursynth plug-in can be installed with the command line pip install vapoursynth-fmtconv. Requires Vapoursynth r74 or above.

[Adub]

Are there any plans to support Linux installs via PyPi ("pip install") and/or sdist (source distribution) packaging so that at the very least users can compile things on their own machine if you don't want to bother with making Linux binaries yourself?

Many plugin authors have been using Meson as their build system since it simplifies much of this work, in case you are looking for a tool.

[Z2697]

Inline assembly code use 32bit register to store memory address and causes crash on 64bit Windows.
Possible fix: https://gist.github.com/Mr-Z-2697/a761228cd0189b50e8022088d780e4c5
(Or, _InterlockedCompareExchange64 is available in MinGW, just use that instead? By appending || defined (__MINGW32__) to #if defined (_MSC_VER))
(Inline assembly seems to be slightly faster...)

src/conc/Interlocked.hpp
int64_t Interlocked::cas (int64_t volatile &dest, int64_t excg, int64_t comp) noexcept
{
...
#if defined (_MSC_VER)
...
#elif defined (_WIN32) || defined (WIN32) || defined (__WIN32__) || defined (__CYGWIN__) || defined (__CYGWIN32__)
...
asm volatile (
" lea %[comp], %%esi \n"
" mov (%%esi), %%eax \n"
...


Story:
I wanted to build fmtc with MSYS2 for a long time, because MSVC is so, err, massive. (but I still have MSVC anyway...)
But the plugin just keeps crashing. So I had to use MSVC.
Today I finally decided to sit down and find out what's wrong.

[ChaosKing]

Are there any plans to support Linux installs via PyPi ("pip install") and/or sdist (source distribution) packaging so that at the very least users can compile things on their own machine if you don't want to bother with making Linux binaries yourself?

Many plugin authors have been using Meson as their build system since it simplifies much of this work, in case you are looking for a tool.

I hope we get official linux/mac releases. For now there is https://github.com/Jaded-Encoding-Thaumaturgy/vs-wheels#fmtconv--fmtc

[Selur]

I agree a pip release would be nice, but https://github.com/Jaded-Encoding-Thaumaturgy/vs-wheels#fmtconv--fmtc seems to be working atm.