How to do 3D Lut color correction (.3DL .Cube) with avisynth(+) ?

[K.i.N.G]

I wasn't too happy with the current HDR -> SDR filters available so I decided to create some luts but I can't find anything online on how to use them in avisynth?

Surely this must be possible? Since luts are such commonly used files for this kind of thing.

[videoh]

I wasn't too happy with the current HDR -> SDR filters available Which ones did you try and why were you unhappy with them?

[TheFluff]

There is a VapourSynth filter (https://github.com/sekrit-twc/timecube/) that can apply 3D LUT's in .cube format (some Adobe thing I believe), FWIW.

[K.i.N.G]

Which ones did you try and why were you unhappy with them?

DGHable and DGReinhard ... i get the impression some tones are problematic.
With my current test im having problems with orange reds (getting too orange yellow). Making skin tones not as good as i'd like.
Maybe its just me? I've only tested 2 discs for now.

Also, I'm not a big fan of exaggerated tinting of a whole movie (there are exceptions) or remasters ruined by bad grading.
Good examples are "the good, the bad and the ugly remastered" or the lord of the rings trilogy extended edition... where the grading looks much worse than the original.

So also in those cases it would be sweet if I could just use/load a lut with corrections I did (to my personal taste).

[K.i.N.G]

There is a VapourSynth filter (https://github.com/sekrit-twc/timecube/) that can apply 3D LUT's in .cube format (some Adobe thing I believe), FWIW.

Nice, I'll have to try that out later...
Im using DGhable & DGreinhard in avisynth (through StaxRip) though so I'd be more usefull for me if there was a solution using avisynth

[FranceBB]

I came across this topic looking for a LUT plugin in Avisynth/Avisynth+.
I'm actually using ffmpeg to apply my LUTs, but it would be really useful to have the VapourSynth filter ported to Avisynth.

[Yanak]

I once asked here if somehow possible to port this into avisynth http://forum.doom9.org/showthread.php?p=1837229#post1837229

Seems like it's something that will be difficult... it's really something i miss to make avisynth complete for my use, like you i use ffmpeg to do this task for now when i need it.

[TheFluff]

I doubt it would be hard to port, the plugin interface is separated from the actual processing.

[videoh]

I've started working on it. :)

[Yanak]

You cannot imagine how sweet those words sound to my hears :)

If you manage to get this working on avisynth it will be a game changer for me and i imagine many others.

Thank you very much.

[videoh]

What formats do you need, in order of priority?

[Yanak]

For me I use mostly .cube files, not sure what others use the most, .3dl it's very rare to use those for me, i use more haldclut files than .3dl to say :p

Thank you.

[videoh]

I was referring to the input and output pixel formats of the filter itself. Can you specify those please?

Assuming the filter exists, what would your script look like?

[FranceBB]

Definitely .cube as cube LUT format.
As to the color space and bit-depth, I would say 4:2:0 8bit / 4:2:0 10bit input and output.
LUTs are generally used to apply a specific type of color correction in post-production, but in a 2018-Avisynth scenario, I would say that they would be mainly use them for HDR.
In other words, I think people would use them to tone-map HDR to SDR, HLG to HDR10 and so on.
So I would say that Rec.709 and Rec.2020 should be supported as well.

Is it possible?

If you wanna test out two .cubes, you can find mine here:



BT709_to_PQ_400: https://mega.nz/#!3N0TSSLT!KkhslQPudhXBZMW0yKBGzhwm6lS5AYxUcIV2BZzxuw0
BT709_to_PQ_800: https://mega.nz/#!DdMhUCgC!djQpewXxZr8YXQoeBhz-KnBRAhnZ-T7FbHnVoYKV2l0

Output limited to 1000nit.


https://i.imgur.com/YFjToim.png

[Yanak]

^ What he said 420p8 mostly as input, I guess some use it on 422 too, and mostly 420p8/10 as output i think.

Also used on raw footage ( slog from drones for example) to correct colors, or to apply some effects like with CineStyle_S_Curve lut, or Kodachrome (https://frankglencairn.wordpress.com/2014/01/15/everything-looks-better-on-kodachrome-k-tone-lut/) effect, can be used to turn stuff recorded in summer with loads or greens into autumn colors (https://www.smallhd.com/community/free-lut-arapaho) etc etc

[videoh]

OK, guys, thank you.

[WorBry]

i use more haldclut files than .3dl to say :p


Easy to convert HaldCLUT to CUBE with IWLTBAP_LUT_Generator. Did that with Pat David's G'MIC Film Emulations:

https://gmic.eu/film_emulation/negative_new_sample_1.shtml#browse

Would be nice to use those in AVISynth(+).

[Yanak]

Thanks for the link, i'll give it a try soon, G'mic plugin for Gimp is what i often use to create and generate custom haldcluts from static images i modify, then later apply them on videos via FFmpeg, this will certainly help a lot if videoh manage to get this working on avs+ it will be very nice indeed.

Thank you :)

[WorBry]

You're welcome.

...videoh manage to get this working on avs+ it will be very nice indeed

Yes, it would open up many possibilities.

[StainlessS]

I have no idea if these couple of files are of any interest to anyone (had them for years, never really looked at them much).

Adobe Cube Lut Specs, v1.0(~228KB):- http://www.mediafire.com/file/ptd2lt3xludabbo/cube-lut-specification-1.0.pdf.7z/file

yCMS_x86, excerpt from manual, (maybe there is an x64 version somewhere)
yCMS - Color Management System
v1.12 2012/11/07

________________________________________________________________________________

INTRODUCTION
________________________________________________________________________________


yCMS is a Color Management System (CMS) that uses 3-dimensional lookup
tables ("3D LUTs") to perform the conversion between the input and output
color representations. You can read a little more about CMS here:
http://en.wikipedia.org/wiki/Color_management

The idea of using 3D LUTs was due to the intensive computing that needs to
be performed when converting video data between different color spaces. When
using 3D LUTs all computing is performed offline during its creation. The
computed 3D LUTs (three, one for each component) are then written to a file
which can later be used, when playing videos or viewing photos, simply by
mapping the input data through the 3D LUTs. This way, the conversion can be
performed at runtime using lower computational power and leaving more CPU
available for other tasks. Since all the computing is performed offline its
precision can be increased by using 64bit floating point (per component)
through out all the conversion chain.

(~239KB):- http://www.mediafire.com/file/8ppce1j87qj6f2c/yCMS_x86.rar/file
EDIT: Above yCMS_x86 (home site is now dead) and thread here on D9 not active since 2013.
https://forum.doom9.org/showthread.php?t=154719&highlight=ycms


EDIT:
Apply two 3x3 matrices to each pixel? :- https://forum.doom9.org/showthread.php?p=1706626

RGB/XYZ Matrices:- http://www.brucelindbloom.com/index.html?Eqn_RGB_XYZ_Matrix.html

[videoh]

I have vscube ported to Avisynth+ and have successfully applied a 3D LUT using an Avisynth+ script. I have to properly implement the parameters and test thoroughly before I can give it to y'all. Meanwhile, I have a few questions:

1. In Vapoursynth, this gives me a red clip:

clip = core.std.BlankClip(width=640,height=480,format=vs.RGB24, length=1000, keep=True, color=[255, 0, 0])

but this gives me a black clip:

clip = core.std.BlankClip(width=640,height=480,format=vs.RGB48, length=1000, keep=True, color=[255, 0, 0])

So is it a bug or am I missing something?

2. Can anyone give me a HDR10 PQ->SDR 3D LUT that I can test with?

Thank you.

[poisondeathray]

1. In Vapoursynth, this gives me a red clip:

clip = core.std.BlankClip(width=640,height=480,format=vs.RGB24, length=1000, keep=True, color=[255, 0, 0])

but this gives me a black clip:

clip = core.std.BlankClip(width=640,height=480,format=vs.RGB48, length=1000, keep=True, color=[255, 0, 0])

So is it a bug or am I missing something?


In 16bit, 100% red would be 65535

[videoh]

Duh, thank you, poisondeathray!

[StainlessS]

In 16bit, 100% red would be 65535

Yep, that is a little bit of a nasty, its pretty horrible that one has to cater to differing ranges dependant upon
bit depth, cant suggest anything better, but is still horrible.

[huhn]

2. Can anyone give me a HDR10 PQ->SDR 3D LUT that I can test with?

https://displaycal.net

you can create 3D lut from what ever source profile with what ever target nit you want with this tool.

[WorBry]

DaVinci Resolve 15 (including free version) has a series of HDR ST 2084 (300 - 4000 nits) <-> Gamma (2.4, 2.6, 2.8) conversion LUT's (cube). I've not tested any of them outside of Resolve though.

With the program installed, the folder location is C:\ProgramData\Blackmagic Design\DaVinci Resolve\Support\LUT\HDR ST 2084.

[wonkey_monkey]

Yep, that is a little bit of a nasty, its pretty horrible that one has to cater to differing ranges dependant upon
bit depth, cant suggest anything better, but is still horrible.

One solution is to allow a different way of specifying colour like fcolor={1.0, 0.0, 0.0}

[videoh]

DaVinci Resolve 15 (including free version) has a series of HDR ST 2084 (300 - 4000 nits) <-> Gamma (2.4, 2.6, 2.8) conversion LUT's (cube). I've not tested any of them outside of Resolve though.

With the program installed, the folder location is C:\ProgramData\Blackmagic Design\DaVinci Resolve\Support\LUT\HDR ST 2084. That's great, WorBry, thank you!

[videoh]

Here is a test version of 3D LUT support for Avisynth+. I will provide source code when it is tested OK. Please refer to the user document for usage details. Your feedback will be appreciated.

http://rationalqm.us/misc/avscube_1.0.rar

[WorBry]

Done a few quick tests - applied a few color grade 3D cube LUT's (created in Resolve) with avscube to some (linear) Rec709 HD test footage and greyscale ramps - and it looks to be working as expected. Excellent.

[FranceBB]

It works as expected.
By the way, you wrote to use avsresize to convert to RGB 16bit, but you can convert without using it like so:

video3=DGDecode_MPEG2Source("I:\Production\RAW\424-430\427.d2v")
audio3=FFAudioSource("I:\Production\RAW\424-430\427 T81 2_0ch 224Kbps DELAY -134ms.ac3")
AudioDub(video3, audio3)

tfm(mode=1,pp=5,slow=2,micmatching=2,clip2=tdeint(mode=2,type=3))
tdecimate()
ConvertFPS(23.976)

trim(0, 500)

Crop(4, 0, -4, -0)

Spline64Resize(848, 480)

ConvertBits(16)

ConvertToPlanarRGB()

Cube("BT601_to_BT709.cube")

Converttoyuv420()

I tested it with a simple BT601 to BT709.
I don't really do BT601 to BT709 conversion using a LUT 'cause I generally use ColorMatrix for 8bit and Dither_Tools for 16bit, but in order to test your new plugin, I did and it was fine.

Thank you very much indeed.

[WorBry]

Quick question for ye high bit-depth AVS processing experts (me not being one).

I have some 10-bit 422 DNxHD.mov clips with 'full range' (0-1023 in 10-bit) luma. I want to test a 3D LUT.cube (created with DaVinci Resolve) that applies a 'soft clip' i.e. clips 'Full' to 'Limited' range (64 - 940) but with a soft roll-off - offers an alternative to 'hard-clipping' (clamping) or compression when constraining full range material to 'broadcast safe' range.

Came up with this basic script, but I'm not sure what 'colorspace_op' string expressions to apply in z_ConvertFormat so that 'Full' (PC.709) range coefficients are applied in the conversion to RGBP16 and then 'Limited' (Rec.709) range coefficients are applied in the output conversion to YV16:

LSMASHVideoSource("{Path}:\Test_DNxHD_10bit_422_Full_Range.mov", format="YUV422P10")
ConvertFromDoubleWidth(bits=10)
z_ConvertFormat(pixel_type="RGBP16",colorspace_op ="?????")
Cube("{Path}:\Soft_Clip_64_940_20.cube")
z_ConvertFormat(pixel_type="YV16",colorspace_op ="?????")

Could anyone point me to a reference that gives a full description of the 'colorspace_op' parameter terms and definitions.

Thanks.

[videoh]

https://forum.doom9.org/showthread.php?p=1784247#post1784247

One thing I am not clear on is whether the 3D LUT expects linear RGB. If so, then maybe something like:

z_ConvertFormat(pixel_type="RGBP16",colorspace_op="709:709:709:l=>rgb:linear:709:f")
Cube("{Path}:\Soft_Clip_64_940_20.cube")
z_ConvertFormat(pixel_type="YV16",colorspace_op="rgb:linear:709:f=>709:709:709:l")

But the cube name suggests it is for limited range, so wouldn't you want to keep it limited and set fullrange=false for Cube()? Just a thought, I'm probably more clueless than you about this stuff. ;)

Thanks, guys, for your testing and feedback.

[WorBry]

Thanks for reference link. It's clear now. Works as expected. Example:

LSMASHVideoSource("{Path}:\GSRamp_DNxHD_10bit_422_Full_Range.mov", format="YUV422P10")
ConvertFromDoubleWidth(bits=10)
z_ConvertFormat(pixel_type="RGBP16",colorspace_op ="709:709:709:f=>rgb:709:709:f")
Cube("{Path}:\Soft_Clip_64_940_20.cube")
z_ConvertFormat(pixel_type="YV16",colorspace_op ="rgb:709:709:l=>709:709:709:l")
#
LanczosResize(720,720)
TurnRight()
Histogram(mode="classic")

http://i.imgur.com/K9nDlqPm.png (https://imgur.com/K9nDlqP)

Resolve actually has a dedicated tool for creating custom Soft-Clip LUT's but they are 1D LUT's with a proprietary file extension (.olut) for in-program use only - as far as I know they cannot be converted externally. Only workaround is to apply the Soft-Clip LUT to a clip (full range greyscale ramp) on the timeline and generate an exportable 3D Cube LUT from that. But it works. Cool.

[videoh]

Thanks for the feedback, WorBry. And FranceBB. Great to hear it is working as expected.

@all

I added the source code to the linked file. It's just one new file and the Avs+ headers added to the vscube directory. Maybe sekrit-twc can absorb it. Otherwise I will link it at my website.

[Revan654]

Do any of the 3D Lut Filters Support iluts, cube_shaperlut or olut? These are mainly generated through Blackmagic.

[WorBry]

As mentioned earlier:


Resolve actually has a dedicated tool for creating custom Soft-Clip LUT's but they are 1D LUT's with a proprietary file extension (.olut) for in-program use only - as far as I know they cannot be converted externally. Only workaround is to apply the Soft-Clip LUT to a clip (full range greyscale ramp) on the timeline and generate an exportable 3D Cube LUT from that. But it works. Cool.

That's the only workaround I know. I use the Grey Scale generator (Edit Page>Effects Library> Generators) to create the Grey Scale clip on the timeline (convert to New Compound Clip). On the Color Page, I then apply the 1D LUT to the clip (Node Graph) and right click the clip thumbnail (Timeline) to select Generate 3DLUT (cube).

[WorBry]

..cube_shaperlut..

I think a .cube_shaperlut is just some internal system intermediary. If you look in:

C:\ProgramData\Blackmagic Design\DaVinci Resolve\Support

....you'll see that there are two folders, LUT and .LUT

The LUT folder is where the actual 3D.cube and ID (.ilut/.olut) files reside and where externally created 3D cube LUT's should be placed for import.

The .LUT folder is a private system folder. In it you'll see there is a corresponding .cube_shaperlut extension file and a .cube_shaperlut.png for each 3D.cube LUT found in the LUT folder. For each ID (.ilut/.olut) LUT in the LUT folder however there is a like-named file (but with no extension) and .png file.

Looks like these files are generated automatically when the LUT Listing is updated.

Edit: There was some suggestion that other programs (Nuke) are able to read these .cube_shaperlut files, when the extension is changed to .cube.

https://www.liftgammagain.com/forum/index.php?threads/lut-export-in-rcm-linear-timeline-colorspace.5644/

Out of curiosity, I put that to the test. Copied the .cube_shaperlut file corresponding to one of the 3D.cube LUT's that I had tested earlier, changed the extension to .cube and tried it with AVSCube. No chance - 'Unhandled C ++ Exception'. The file size was also smaller (421kb) than the bone-fide .cube LUT (971 kb).

Also tried placing the .cube_shaperlut file (with extension changed to .cube) in the Support> LUT folder. After updating the LUT listing, the file did appear in the 3D LUT drop down menu and LUT gallery, but applying it to a clip had absolutely no effect.

I don't think these .cube_shaperlut files hold any value outside of Resolve.

Incidentally, the .png file that is created in the .LUT folder is the stock thumbnail image that is presented in the LUT Gallery with the LUT effect applied. Maybe these .cube_shaperlut files are operators for that.

[TomArrow]

This plugin is amazing, works very fine. Thank you.

Any way you could implement an alternate function that, instead of a path to a .cube file, accepts a HALD CLUT?

For reference, it's this: http://www.quelsolaar.com/technology/clut.html

Now, I know I can just convert a CLUT to a CUBE file, but what I would really wanna do is be able to load a non-static CLUT.

Imagine this: Your friend is working on the same project as you and has done a color correction on an entire movie. You wanna continue his work, but dont want to send terabytes of data around. So your friend simply renders his corrections into a Lagarith RGB HALD CLUT AVI and you simply load it over the source material and BAM, you have all his corrections, no matter what software he used.

Could also play around with that a lot and create funny effects I'm sure. Shouldn't be too difficult to implement, right? It would just have to accept a 16bit RGB 4096x4096 clip. That could come from an Image (ImageSource) or from a video, it wouldn't matter.

[Stereodude]

Thanks for the plugin videoh I got a black screen when I set fullrange=false. It works okay with fullrange=true.

LoadPlugin("C:\HDTV Tools\DGDecNV\DGDecodeNV.dll")
LoadPlugin("C:\HDTV Tools\avscube\x64\vscube.dll")

DGSource("00005.dgi").crop(0,114,-0,-114).ConvertBits(16)

ConvertToPlanarRGB(matrix="PC.709").crop(0,1,0,-1)

Cube("C:\HDTV Tools\avscube\U-CANON-C-LOG-V1-LTR.cube", fullrange=true)

ConvertToYUV420(matrix="PC.709").ConvertBits(8, dither = 0)

[videoh]

I'll look into it, Stereodude. Thanks for pointing it out. Can you please give me your cube file? Thank you.

[Stereodude]

I'll look into it, Stereodude. Thanks for pointing it out. Can you please give me your cube file? Thank you.
It's in this zip file.

http://stereodude.net/LUT.zip

[videoh]

Got it, thank you.

Can you give me a mediainfo dump on your source file, please? Is it HDR, etc?

[videoh]

Seems that this was fixed in timecube at some point:

https://forum.doom9.org/showthread.php?p=1850130#post1850130

The code link there is broken so I'll try to hunt down the r2 changes.

[videoh]

Latest code is here:

https://github.com/sekrit-twc/timecube

I'll do a re-port to Avisynth+ starting with this latest code.

[Stereodude]

Got it, thank you.

Can you give me a mediainfo dump on your source file, please? Is it HDR, etc?
General
Complete name : E:\TempBD\TIM\05\00005.h264
Format : AVC
Format/Info : Advanced Video Codec
File size : 618 MiB
Overall bit rate mode : Variable

Video
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, RefFrames : 4 frames
Bit rate mode : Variable
Maximum bit rate : 38.0 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
It is FHD directly from a Blu-ray. It is not HDR. It looks very flat. I believe that they shot the footage in C-Log (Canon Log) and didn't bother to grade it. Hence my interest in applying a C-Log to Rec709 3D LUT to it. I'm not sure if it should be fullrange=false or true. I wanted to try both and look at the output visually when I noticed that fullrange=false gave black screen output.

[videoh]

OK, thanks. As I mentioned, I'll re-port the latest code.

[videoh]

Rather than upgrade to r2a I ported the fullrange fix only. Here is the revised version:

http://rationalqm.us/hdr/avscube_1.1.rar

[Stereodude]

Rather than upgrade to r2a I ported the fullrange fix only. Here is the revised version:

http://rationalqm.us/hdr/avscube_1.1.rar
I get an error on both the x86 and x64 versions.

x64:
'C:/HDTV Tools/avscube/x64/vscube.dll' cannot be used as a plugin for AviSynth.

x86:
'C:/HDTV Tools/avscube/x86/vscube.dll' cannot be used as a plugin for AviSynth.

[videoh]

Hmm, that's strange. I will look into it. This doesn't happen with my version 1.0 DLL? What version of Avisynth+ are you using?

[Stereodude]

Hmm, that's strange. I will look into it. This doesn't happen with my version 1.0 DLL? What version of Avisynth+ are you using?
No, the 1.0 version works fine in x64 & x86 (aside from the bug I reported earlier). I have AviSynth+ 0.1 (r2772, MT, x86_64) (0.1.0.0)

[videoh]

Somehow avscube.dll got excluded from the build after I updated VS 2017. I have fixed it. Please re-download to get the fixed version.

[FranceBB]

Somehow avscube.dll got excluded from the build after I updated VS 2017. I have fixed it. Please re-download to get the fixed version.

Tested on WinXP SSE4.1 and it works fine with fullrange=true now.
Thanks again. ;)

[Stereodude]

Somehow avscube.dll got excluded from the build after I updated VS 2017. I have fixed it. Please re-download to get the fixed version.
Seems to be working okay with the updated version. No more black screen and the fullrange switch works.

[videoh]

Great. Thanks for your testing and for reporting the issue.

[FranceBB]

Hi Donald,
I'm resurrecting this topic as it seems to be the only one which is AVSCube related and I'd like to ask you something about it.
I've been using it all the time at work, in production, on my Xeon 28c/56th machines, but I'd like to speed things up a bit as all my server run fairly recent NVIDIA Quadro cards.
You're a developer, you know CUDA, you made many hardware accelerated plugins, are you tempted even just a little bit to make this CUDA accelerated as well?

[FranceBB]

I know that we all took this as granted, but just to reassure everyone, I made a simple test for Cube() with the identity matrix.

Test1:

#Indexing
video=LWLibavVideoSource("Z:\YUV_pattern-2fr.v210-14-2020PQ.mov")
audio=BlankClip(length=1, channels=8, audio_rate=48000, fps=25)
AudioDub(video, audio)

#From 4:2:2 10bit to 4:2:2 16bit planar
ConvertBits(16)

#From 4:2:2 16bit planar Narrow Range to RGB Planar 16bit Narrow Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:2020:2020:limited=>rgb:2020:2020:limited", dither_type="none", resample_filter="Spline64")

Test2:

#Indexing
video=LWLibavVideoSource("Z:\YUV_pattern-2fr.v210-14-2020PQ.mov")
audio=BlankClip(length=1, channels=8, audio_rate=48000, fps=25)
AudioDub(video, audio)

#From 4:2:2 10bit to 4:2:2 16bit planar
ConvertBits(16)

#From 4:2:2 16bit planar Narrow Range to RGB Planar 16bit Narrow Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:2020:2020:limited=>rgb:2020:2020:limited", dither_type="none", resample_filter="Spline64")

#From PQ to HLG with 16bit precision
Cube("Z:\IDENTITY.cube", fullrange=true)


I called "Test1" as noLUT and "Test2" as IdentityMatrix.

Now, since anything times the identity matrix returns itself and therefore applying it or not shouldn't change anything (unless linear algebra and my whole life is a lie xD), I expected Test1 and Test2 to be the same, so I did:

noLUT=ImageSource("\\mibctvan000\Ingest\MEDIA\temp\Test4_PQ_Avisynth_no_LUT_no_dithering_RGBP16.tiff").ConverttoYUV444()
IdentityMatrix=ImageSource("\\mibctvan000\Ingest\MEDIA\temp\Test6_PQ_Avisynth_IdentityMatrix_no_dithering_RGBP16.tiff").ConverttoYUV444()

StackHorizontal(noLUT, IdentityMatrix)
TurnRight.Histogram.TurnLeft #luma

and indeed it returned:https://i.imgur.com/YFBVZTn.png

and since I couldn't see differences at first glance, I tried with Subtract:

noLUT=ImageSource("\\mibctvan000\Ingest\MEDIA\temp\Test4_PQ_Avisynth_no_LUT_no_dithering_RGBP16.tiff").ConverttoYUV444()
IdentityMatrix=ImageSource("\\mibctvan000\Ingest\MEDIA\temp\Test6_PQ_Avisynth_IdentityMatrix_no_dithering_RGBP16.tiff").ConverttoYUV444()

Subtract(noLUT, IdentityMatrix)

which shouldn't have been returning differences, in fact: https://i.imgur.com/3UTpdiB.png

So I thought: Uhm, maybe it's just my eyes but I can't see any difference, let's try with SSIM and...

noLUT=ImageSource("\\mibctvan000\Ingest\MEDIA\temp\Test4_PQ_Avisynth_no_LUT_no_dithering_RGBP16.tiff").ConverttoYUV420().ConvertBits(bits=8, dither=-1)
IdentityMatrix=ImageSource("\\mibctvan000\Ingest\MEDIA\temp\Test6_PQ_Avisynth_IdentityMatrix_no_dithering_RGBP16.tiff").ConverttoYUV420().ConvertBits(bits=8, dither=-1)

SSIM(noLUT, IdentityMatrix, "results.csv", "globalSSIM.txt", lumimask=1, scaled=1)

returned Y=1.0000, U=1.0000, V=1.0000
so a perfect score, which means that the two are exactly the same: https://i.imgur.com/uxznR68.png

so I can confirm that Cube() works as expected.

The reason why I've made this test is to be 100% sure as I've been having some stuff processed through an analyzer and although the result was... "fine", there were some slight variations from how it should have been, so I was trying to find one function to blame but apparently it's not Cube() eheheheheh.

Anyway, I think it was worth sharing to reassure everyone that the function is mathematically correct and does what it's supposed to do.

[StvG]

@videoh, can you add support for passing frame properties when they are available?

[kedautinh12]

Latest ver timecube i think avs ver don't port it
https://github.com/sekrit-twc/timecube

[videoh]

@videoh, can you add support for passing frame properties when they are available? Sure, I'll work on that today.

[videoh]

Latest ver timecube i think avs ver don't port it
https://github.com/sekrit-twc/timecube I'll see if there is anything we need in r3.

[videoh]

Here is a port of r3 to Avisynth. Frame properties passthrough has also been added. Please advise if this is working properly.

http://rationalqm.us/misc/vscube_13.zip

[Reel.Deel]

Here is a port of r3 to Avisynth. Frame properties passthrough has also been added. Please advise if this is working properly.

http://rationalqm.us/misc/vscube_13.rar

Not sure if the file is corrupted or what but I can't open the file, the size is only 34k.

[videoh]

That's the correct size. Works fine for me. What happens when you try to open it?

Anyone else having problems?

[Reel.Deel]

That's the correct size. Works fine for me. What happens when you try to open it?

Anyone else having problems?

I can open other rar files just fine, I tried downloading it in a different browser and same problem :confused:.


7-Zip
---------------------------
Can not open file 'C:\Users\Downloads\vscube_13.rar' as archive
---------------------------

Edit: not sure was going on but https://extract.me/ opens the file just fine. The rar file has one binary named "vscube.dll".

[videoh]

Edit: not sure was going on but https://extract.me/ opens the file just fine. The rar file has one binary named "vscube.dll". That's correct. Is the DLL working?

I deleted the rar and put a zip instead. Is it better?

[Reel.Deel]

That's correct. Is the DLL working?

I deleted the rar and put a zip instead. Is it better?

Zip works just fine, thanks. I will test it and get back to you if I find any problems.

:thanks:

[videoh]

Thank you. Could be a RAR versus RAR4/5 thing.

Are you using WinRAR or 7zip?

[Reel.Deel]

Thank you. Could be a RAR versus RAR4/5 thing.

Are you using WinRAR or 7zip?

I'm using 7zip. I tested the DLL and used a simple LUT and it's working like it is supposed to. Have not checked frame properties.

[videoh]

OK, thank you.

[StvG]

Here is a port of r3 to Avisynth. Frame properties passthrough has also been added. Please advise if this is working properly.

http://rationalqm.us/misc/vscube_13.zip

Thanks. It's working properly here.

[kedautinh12]

That only x86 or x64 ver? Can you support both? I remember old ver support both. Thanks

[StainlessS]

RD,
I just noticed that 7zip posted update Dec 2021,
v19.00 was latest [EDIT: stable] for a long time, then updated in Nov 21 and again in Dec 21,
perhaps part reason was some RAR thingy.
7zip:- https://www.7-zip.org/

EDIT: Non beta, ie stable, v21.07 [26 Dec 2021]

[Reel.Deel]

RD,
I just noticed that 7zip posted update Dec 2021,
v19.00 was latest [EDIT: stable] for a long time, then updated in Nov 21 and again in Dec 21,
perhaps part reason was some RAR thingy.
7zip:- https://www.7-zip.org/

EDIT: Non beta, ie stable, v21.07 [26 Dec 2021]

You're probably right. My 7-zip was super old. I went ahead and updated. Thanks for pointing that out.

Here is a port of r3 to Avisynth. Frame properties passthrough has also been added. Please advise if this is working properly.

http://rationalqm.us/misc/vscube_13.zip

Can you provide the source code please?

[joearmstrong]

The example from Avisynth wiki seems to be wrong:

DGSource("THE GREAT WALL.dgi", fulldepth=true)
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020ncl:st2084:2020:l=>rgb:linear:2020:f", dither_type="none")
Cube("...\test.cube")
z_ConvertFormat(pixel_type="YUV420P16", colorspace_op="rgb:linear:2020:f=>2020ncl:st2084:2020:l", dither_type="none")

Converting to linear produces errors. Maybe a bug? Without linear the reults are fine for me:

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020ncl:st2084:2020:l=>rgb:st2084:2020:f", dither_type="none")
Cube("...\test.cube")
z_ConvertFormat(pixel_type="YV12", colorspace_op="rgb:st2084:2020:f=>709:709:709:l", dither_type="ordered")

[Reel.Deel]

The example from Avisynth wiki seems to be wrong:

I copied the example that was included in avscube v1.1. I did not test it nor do I have a lot of experience with hdr. If it's indeed wrong I will update the wiki accordingly.

[videoh]

That only x86 or x64 ver? Can you support both? I remember old ver support both. Thanks Yes, of course. When I release it there will be both versions.

[videoh]

Can you provide the source code please? Yes, of course, when I release it.

[videoh]

The example from Avisynth wiki seems to be wrong:

DGSource("THE GREAT WALL.dgi", fulldepth=true)
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020ncl:st2084:2020:l=>rgb:linear:2020:f", dither_type="none")
Cube("...\test.cube")
z_ConvertFormat(pixel_type="YUV420P16", colorspace_op="rgb:linear:2020:f=>2020ncl:st2084:2020:l", dither_type="none")

Converting to linear produces errors. Maybe a bug? Without linear the reults are fine for me:

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020ncl:st2084:2020:l=>rgb:st2084:2020:f", dither_type="none")
Cube("...\test.cube")
z_ConvertFormat(pixel_type="YV12", colorspace_op="rgb:st2084:2020:f=>709:709:709:l", dither_type="ordered") Thank you, I will update the example. Note also that DGSource() no longer takes a fulldepth parameter.

[videoh]

Released as version 1.3 (to track timecube version):

http://rationalqm.us/board/viewtopic.php?f=5&t=463&p=15239#p15239

[FranceBB]

The example from Avisynth wiki seems to be wrong:

Converting to linear produces errors. Maybe a bug? Without linear the reults are fine for me:

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020ncl:st2084:2020:l=>rgb:st2084:2020:f", dither_type="none")
Cube("...\test.cube")
z_ConvertFormat(pixel_type="YV12", colorspace_op="rgb:st2084:2020:f=>709:709:709:l", dither_type="ordered")

It depends on what the LUT is doing.
If the LUT is converting to BT709 SDR, then you don't need to put st2084 to 709 in z_ConvertFormat, otherwise you would be converting it twice!

This is what I use at Sky for PQ to HLG:

#BBC HDR PQ to HDR HLG LUT

#From 4:2:2 16bit planar Narrow Range to RGB Planar 16bit Full Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:st2084:2020:limited=>rgb:st2084:2020:full", resample_filter_uv="spline64", dither_type="error_diffusion")

#From PQ to HLG with 16bit precision
Cube("\\mibcisilonsc\avisynth\Server\encoder\encoder\Processors\avs_plugins\LUTs\1a_PQ1000_HLG_mode.cube", fullrange=true)

#From RGB 16bit planar Full Range to YUV422 10bit planar Narrow Range with dithering
z_ConvertFormat(pixel_type="YUV422P10", colorspace_op="rgb:std-b67:2020:full=>2020:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")

and it's what I believe should be a good example.

[joearmstrong]

You're right, FranceBB.
I think this should be correct:

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020ncl:st2084:2020:l=>rgb:st2084:2020:f", dither_type="none")
Cube("C:\Downloads\HDR to SDR\LUT von der rabe\Lut Eigen_HDR to SDR 02.cube", fullrange=true)
z_ConvertFormat(pixel_type="YV12", colorspace_op="rgb:709:709:f=>709:709:709:l", dither_type="ordered")

[videoh]

National squirrel appreciation day today :tip: gonca. Cube is the ham and the convert stuff is the bread. The ham tastes great.

[FranceBB]

Yeah the last one you posted is correct. :)

National squirrel appreciation day today.

I thought you were just fooling us all, Donald, but I googled it and it's actually really the Squirrel Appreciation Day O_O

[Dogway]

My monitor profile doesn't work anymore in latest version v1.3. Here's a link (https://www.sendspace.com/file/hptmlg). It was working at least up to v1.1. Do I need to conform the .cube file to another format?

[videoh]

Doesn't work means what? And post your script.

[Dogway]

Means white output. This is what I have been using the past years:
ConvertBits(16)
ConvertToPlanarRGB()
Cube("MonitorLUT.cube", 2, true)
ConvertBits(8,dither=1)

[videoh]

Thank you. I'll look into it.

[poisondeathray]

Means white output. This is what I have been using the past years:
ConvertBits(16)
ConvertToPlanarRGB()
Cube("MonitorLUT.cube", 2, true)
ConvertBits(8,dither=1)

Vapoursynth vscube_r3 produces "white" output too , unless the input is RGBS . r2a worked ok with RGB48

[videoh]

What is RGBS and how would one arrange the input to be RGBS?

[Reel.Deel]

What is RGBS and how would one arrange the input to be RGBS?

32-bit float and ConvertBits(32).

[videoh]

Thank you. Investigating...

[StvG]

I can reproduce the issue with the white output with only Cube("....", 2). cpu 0 and 1 are fine. Using just Cube("....") is fine and it seems AVX2 is used.

[poisondeathray]

I can reproduce the issue with the white output with only Cube("....", 2). cpu 0 and 1 are fine. Using just Cube("....") is fine and it seems AVX2 is used.

for avscube 1.3 and vscube 1.3 , cpu 0 and 1 are ok for me too. But it picks 2 by default for me, and white screen (unless RGBS in put for VPY)

[videoh]

Looks like a bug in lut_avx2.cpp, as when I replace that with the version from r2a, it works. Intrinsics and assembly are Greek to me, so I posted an issue at sekrit's site:

https://github.com/sekrit-twc/timecube/issues/7

Meanwhile, don't use cpu=2. You can try cpu=3 for AVX512, which seems to work for me.

[StvG]

Meanwhile, don't use cpu=2. You can try cpu=3 for AVX512, which seems to work for me.

That's why for me Cube("...") (w/o specifying cpu) is working (AVX512). But cpu=3 isn't mentioned in the readme/avs wiki.

[videoh]

Yeah, I need to update the help file.

[videoh]

sekrit committed a fix for this. I'll update the Avisynth build.

[videoh]

I updated the 1.3 release with the fix for cpu=2:

http:\\rationalqm.us\hdr\avscube_1.3.zip

[StvG]

Thanks.

[ErazorTT]

I'm having a lut which when using ffmpeg produces the expected behaviour, but when using the cube function returns unexpected things.
The following image shows the result, on the right part there are these waves which are unexpected, and are also not there using ffmpeg.
https://i.postimg.cc/ykYZnHbn/v101-33-size.png (https://postimg.cc/ykYZnHbn)

However when using ffmpeg I'm getting this:
https://i.postimg.cc/tYSVMc0b/Grey-Ramp16bits-converted.png (https://postimg.cc/tYSVMc0b)

Here are all files I am using:
hlg_test.7z (https://cloud.rascanu.de/index.php/s/32eFwN5r3sDyBTy)

The command for ffmpeg is:
ffmpeg -i GreyRamp16bits.png -vf lut3d=lut_4000_3.0.cube -pix_fmt rgb48be GreyRamp16bits_converted.png

What is going on here, why the different results?

[Reel.Deel]

@ErazorTT

Does DGCube have the same problem? https://www.rationalqm.us/board/viewtopic.php?p=16092#p16092

[poisondeathray]

For reference, avs version , vpy version that it's based on , and Adobe get similar results

The difference is the default interpolation type for ffmpeg (tetrahedral) . Trilinear , pyramid, prism all produce the pattern to an extent

eg.
ffmpeg -i GreyRamp16bits.png -vf lut3d=lut_4000_3.0.cube:interp=trilinear -pix_fmt rgb48be GreyRamp16bits_converted_trilinear.png

[tormento]

Does DGCube have the same problem?
Is it working for you? What card? Drivers?

[Reel.Deel]

Is it working for you? What card? Drivers?

No, I am also getting access violation. I have not updated my drivers in a while so I was going to do that and see if it fixes the issue.

[ErazorTT]

For reference, avs version , vpy version that it's based on , and Adobe get similar results

The difference is the default interpolation type for ffmpeg (tetrahedral) . Trilinear , pyramid, prism all produce the pattern to an extent

eg.
ffmpeg -i GreyRamp16bits.png -vf lut3d=lut_4000_3.0.cube:interp=trilinear -pix_fmt rgb48be GreyRamp16bits_converted_trilinear.png

Oh that is a nice finding! So its the interpolation. Could this better interpolation be implemented into vscube (or DGCube for that matter)?

[ErazorTT]

@ErazorTT

Does DGCube have the same problem? https://www.rationalqm.us/board/viewtopic.php?p=16092#p16092

I just found that DGCube needs 65x65x65 sized LUTs, so I tried again with one of that size. And yes, DGCube behaves that same as vscube.

[poisondeathray]

Could this better interpolation be implemented into vscube (or DGCube for that matter)?

It's debatable if a certain interpolation is going to be "better" for all situations;

But having more options is always welcome

You can try submitting a feature request
https://github.com/sekrit-twc/timecube/issues

[tormento]

No, I am also getting access violation.
Generic size version has just been released.

[tormento]

Here is a new version supporting tetrahedral interpolation. It addresses ErazorTT's issue, as the artifacts do not occur with tetrahedral. Please read the new DGCube.txt file for details and be aware that the filter is now invoked as DGCube(). I'll add this to the timecube-derived Cube() filter as well. Also need to add Vapoursynth support to DGCube().

https://rationalqm.us/misc/DGCube.zip

[tormento]

Notice that in the latest version, no more avsresize is required and it got scorching fast.

[ErazorTT]

@tormento: Yup, this does now produce the intended results with these luts. Thanks!

[tormento]

Thanks!
Much better to thank the author. :)

Any bug report and testing is the most welcome.

[kedautinh12]

https://rationalqm.us/misc/DGCube.zip

I wonder why don't DGCube have 2 ver x32 & x64

[tormento]

I wonder why don't DGCube have 2 ver x32 & x64
'cause x86 is getting more and more useless, especially with 4k.

[FranceBB]

Probably cause old x86 OS like my beloved XP don't really have drivers for modern NVIDIA GPUs so the DG plugins would have nowhere to run except for the old not so powerful NVIDIA GPUs... :(
I wish NVIDIA kept releasing drivers for XP x86 but it didn't happen, so nowadays I do most of my stuff via RDP from my XP to the remote Windows Server 2019 instance, especially after Tormento made me upgrade my stable Windows 10 Enterprise workstation to the unstable crap called Windows 11 Enterprise which one day upgraded itself only to end up blue screening in loop, thus making me lose every program installed.

[tormento]

especially after Tormento made me upgrade my stable Windows 10 Enterprise workstation to the unstable crap called Windows 11 Enterprise which one day upgraded itself only to end up blue screening in loop, thus making me lose every program installed.
Oh, dear.

You use crappy hardware and you blame me. :D

Windows 11 Enterprise x64 here since day 1 (on Insider Channel too) and no major hiccups.

[FranceBB]

Oh, dear.

You use crappy hardware

-> I'm broke, as simple as that xD

[FranceBB]

Hi there Donald,
currently it isn't possible to pick which kind of interpolation is used to produce the result in Cube() (I'm talking about the CPU version).
Would it be possible to introduce an option in Cube() like String Interpolation where it can have one of the following values Interpolation="Tricubic", Interpolation="Trilinear", Interpolation="Tetrahedral", where the default is Tetrahedral?

So I imagine the usage to be like:

Cube("myLUT.cube", fullrange=true, Interpolation="Tetrahedral")


The reason is that I'm currently dealing with a brand new BBC LUT for HLG to BT709 SDR.
I used to use the 8a, which performs clipping, but some majors want me to use the new 9c which works in Studio RGB (Limited TV Range) and preserves overshootings (so the BT709 will have out of range whites and blacks and it won't clip them).

The issue I have with it is that the interpolation Cube() is doing is different from the interpolation the major is using and they will therefore reject the result.


AVS Script:

LWLibavVideoSource("D:\4K_Ama\05_DI SERVIZIO - EBU UHD HDR COLOR BARS\EBU_HDR_COLOUR_BARS_2160p_v210.mov")

propClearAll()

#From 4:2:2 16bit planar Narrow Range to RGB Planar 16bit Narrow Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:std-b67:2020:limited=>rgb:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion", use_props=0)

#From HLG to BT709 with 16bit precision
Cube("A:\avdb\Server\encoder\encoder\Processors\avs_plugins\LUTs\9c_HLG_BT709_Type3_Display_DownMapping_SuperWhite_nocomp-v1_5.cube", fullrange=true)

#From RGB 16bit planar Narrow Range to YUV420 8bit planar Narrow Range with dithering
z_ConvertFormat(pixel_type="YUV420", colorspace_op="rgb:709:709:limited=>709:709:709:limited", resample_filter_uv="spline64", dither_type="error_diffusion", use_props=0)


Here's the result (Left interpolation done by Cube() - Right interpolation done by the major who's selling us the content):

https://i.imgur.com/hyky5sZ.png



Before you say "CUDA!" please note that our workflows are based on 100+ scalable VMs in the cloud that are shut down and only woken up when they need to perform a job and then they're shut down and they're CPU only, so we have no way to use CUDA other than on the legacy MAM on prem which will be discontinued soon-ish.


Thank you in advance,
Frank

[kedautinh12]

DGCube 22/09/30
https://rationalqm.us/misc/DGCube.zip

[FranceBB]

Hi Donald,
I've been trying DGCube() with a dual GPU configuration at work, however it only really seems to be using 1 GPU only.
Is there a way to make DGCube() use both GPUs?

Here's the script:

video=LWLibavVideoSource("A:\MEDIA\temp\Test.mxf")
FL_ORI=LWLibavAudioSource("A:\MEDIA\temp\Test.mxf", stream_index=1)
FR_ORI=LWLibavAudioSource("A:\MEDIA\temp\Test.mxf", stream_index=2)
CC_ORI=LWLibavAudioSource("A:\MEDIA\temp\Test.mxf", stream_index=3)
LFE_ORI=LWLibavAudioSource("A:\MEDIA\temp\Test.mxf", stream_index=4)
SL_ORI=LWLibavAudioSource("A:\MEDIA\temp\Test.mxf", stream_index=5)
SR_ORI=LWLibavAudioSource("A:\MEDIA\temp\Test.mxf", stream_index=6)
Left_ORI=LWLibavAudioSource("A:\MEDIA\temp\Test.mxf", stream_index=7)
Right_ORI=LWLibavAudioSource("A:\MEDIA\temp\Test.mxf", stream_index=8)
FL_ITA=LWLibavAudioSource("A:\MEDIA\temp\Test.mxf", stream_index=9)
FR_ITA=LWLibavAudioSource("A:\MEDIA\temp\Test.mxf", stream_index=10)
CC_ITA=LWLibavAudioSource("A:\MEDIA\temp\Testmxf", stream_index=11)
LFE_ITA=LWLibavAudioSource("A:\MEDIA\temp\Test.mxf", stream_index=12)
SL_ITA=LWLibavAudioSource("A:\MEDIA\temp\Test.mxf", stream_index=13)
SR_ITA=LWLibavAudioSource("A:\MEDIA\temp\Test.mxf", stream_index=14)
Left_ITA=LWLibavAudioSource("A:\MEDIA\temp\Test.mxf", stream_index=15)
Right_ITA=LWLibavAudioSource("A:\MEDIA\temp\Test.mxf", stream_index=16)
audio=MergeChannels(FL_ITA, FR_ITA, CC_ITA, LFE_ITA, SL_ITA, SR_ITA, Left_ITA, Right_ITA, FL_ORI, FR_ORI, CC_ORI, LFE_ORI, SL_ORI, SR_ORI, Left_ORI, Right_ORI)
AudioDub(video, audio)

propclearall()

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:std-b67:2020:limited=>rgb:std-b67:2020:full", resample_filter_uv="Spline64", dither_type="error_diffusion", use_props=0)

DGCube("HLG_to_BT709.cube", in="full", out="full", lut="full", interp="tetrahedral")

z_ConvertFormat(pixel_type="YUV422P10", colorspace_op="rgb:709:709:full=>709:709:709:limited", resample_filter_uv="Spline64", dither_type="error_diffusion", use_props=0)


however this is the result:

https://i.imgur.com/i2TWlcW.png


I know that I can get rid of z_ConvertFormat() on top and bottom and allow DGCube() to perform the YUV -> RGB -> YUV conversion to speed things up, however before I do that I wanna make sure that it's possible to use both GPUs at the same time.

[poisondeathray]

Is there a way to make DGCube() use both GPUs?


You can specify device # with DGCube


device: GPU device number to use. If device is 255 (default), then the GPU device will be automatically selected.


Untested, but maybe mp_pipeline to process 2 parallel processes (selecteven, selectodd), each with a different device #, then interleave(even,odd) . Or maybe you can do it without mp_pipeline, it might thread slower (or it might be faster)

[poisondeathray]

BTW teh donald does not post here anymore . You can post and ask in his forum too

[tormento]

Added support to tethrahedral in VSCube.

[kedautinh12]

DGCube 22/09/30
https://rationalqm.us/misc/DGCube.zip

But i still download old ver from here

[FranceBB]

But i still download old ver from here

Not DGCube, VSCube.
As you can see from those commits here: https://github.com/sekrit-twc/timecube/commit/af682b6f7a5ad302bd59ab7e7a29312334be63ac the new tethrahedral interpolation version has been added, although it's not yet "official" 'cause it needs further testing.
Sekrit added it only as plain C code for debug reasons so that I could test it on Monday on my broadcast equipment.
If it works, he'll go on with writing the assembly / intrinsics to make use of AVX, AVX2, AVX512. After that, it will be time to update the old port with the new version, so AVSCube (i.e Cube() ).

If everything goes well, we'll have tetrahedral interpolation in Cube() in Avisynth along with VapourSynth.


Anyway the current plan for the week is:

- Test the current C only VapourSynth only implementation by Sekrit
- Report back if the results are par with what I expected on a Sony reference monitor + Tektronix Hardware Waveform Monitor
- Allow Sekrit to work on the SSE2, AVX, AVX2, AVX512 intrinsics / manually written assembly for VapourSynth
- Ask sekrit to merge the two projects (VSCube and AVSCube) so that they can coexist in the same repository so that next time they'll be updated at the same time


I'll keep you guys posted.


Bye,
Frank

[FranceBB]

In the meantime, in case people want to test, I've made a build with the latest changes from Sekrit (VapourSynth only, for now): vscube.zip (https://github.com/sekrit-twc/timecube/files/11946440/vscube.zip)

Inside the package you'll find the normal vscube_x86.dll and vscube_x64.dll compiled with v143 and of course the "special" v141_xp /Zc:threadSafeInit msvc compiled version called "_XP":

- vscube_x86.dll
- vscube_x64.dll
- vscube_XP_x86.dll
- vscube_XP_x64.dll

The idea is to test the new interpolation method with:

timecube.Cube(cube="Test.cube", fullrange=true, interp=1, cpu=0)

which is, hopefully, what I'm gonna do today.

[tormento]

I was chatting with FranceBB a few minutes ago and we noticed that NVEnc can apply 3D lut in tethraedral with hw acceleration.

I wrote a wake up call thread in development section, as an AVS plugin based on NVEnc would be awesome.

[poisondeathray]

In the meantime, in case people wants to test, I've made a build with the latest changes from Sekrit (VapourSynth only, for now): vscube.zip (https://github.com/sekrit-twc/timecube/files/11946440/vscube.zip)


Thanks.

I tried on ErazorTT's test in post 102.

The syntax is interp=<int>

0 Trilinear
1 Tetrahedral

Looks ok so far, Tetrahedral matches ffmpeg's output

[tormento]

Well. I did some tests applying SDR to HLG (1080p video) and PQ to HLG LUTs (2160p video). I tested with lossy compression as at the moment I don't have enough space to go HEVC lossless.

NVEncC (x64) 7.30 (r2572) by rigaya, Jun 24 2023 02:43:35 (VC 1929/Win)
OS Version Windows 11 x64 (25393) [UTF-8]
CPU Intel Core i7-2600K @ 3.40GHz [TB: 4.51GHz] (4C/8T)
GPU #0: NVIDIA GeForce GTX 1660 SUPER (1408 cores, 1830 MHz)[PCIe3x16][545.31]
NVENC / CUDA NVENC API 12.1, CUDA 12.2, schedule mode: auto
Input Buffers CUDA, 16 frames
Input Info AviSynth+ 3.7.3 r3996(yv12(16bit))->p010 [SSE2], 3840x2064, 24000/1001 fps

Width : 1920
Height : 1032
Frames : 5544
Time : 03:51.231
Framerate : 23.976023 (24000/1001)
Format : YUV420P8

NVEncC64.exe --cqp 18 --codec h265 --output-depth 10 --aq --aq-temporal --vpp-colorspace lut3d=D:\Programmi\Media\AviSynth+\cube\5a_BT709_HLG_UPCONVERT_DISPLAY_mode_nar_in_nar_out_nar_nocomp.cube,lut3d_interp=tetrahedral -i "N:\In\3_12 Avatar 2\avatar sdr to hlg_temp\avatar sdr to hlg.avs" -o "N:\In\3_12 Avatar 2\avatar sdr to hlg_temp\avatar sdr to hlg_out.h265"

encoded 5544 frames, 160.65 fps, 18601.23 kbps, 512.74 MB

Width : 3840
Height : 2064
Frames : 5544
Time : 03:51.231
Framerate : 23.976023 (24000/1001)
Format : YUV420P16

NVEncC64.exe --cqp 18 --codec h265 --output-depth 10 --aq --aq-temporal --vpp-colorspace lut3d=D:\Programmi\Media\AviSynth+\cube\1a_PQ1000_HLG_mode-nar_in-nar_out-nar_nocomp.cube,lut3d_interp=tetrahedral -i "N:\In\3_12 Avatar 2H\HDR10 to HLG_temp\HDR10 to HLG.avs" -o "N:\In\3_12 Avatar 2H\HDR10 to HLG_temp\HDR10 to HLG_out.h265"

encoded 5544 frames, 42.90 fps, 57277.81 kbps, 1578.86 MB

I haven't yet set the correct color spaces and ranges but it seems to work fine. I have yet to try to feed video directly to NVEnc, bypassing any frameserver.

So curious to see the numbers from modern cards.

[FranceBB]

it seems to work fine

Indeed, in your HLG sample converted from BT709 SDR with the NVEnc lut3d filter using tetrahedral interpolation and the 5a LUT the black was sitting at 0.0V and the white was correctly peaking at 0.52V as per HLG standard of 75% reference white.

https://i.imgur.com/42Gdvnc.png


It was peaking lower when you converted from PQ with the 1a LUT, but that's only because such a LUT assumes a source at 1000 nits, however your PQ version of Avatar 2 was made straight out of the DCI P3 XYZ MJPEG2000 4:4:4 12bit cinema master whose reference white is at 48 (so the actual PQ MaxCLL is actually much lower than what was specified in the metadata). This won't result in a wrong output, though, however it will be a slightly worse experience for those watching the BT2020 HLG version on a BT2020 SDR 100 nits display.


https://i.imgur.com/310Dzva.png

[tormento]

This won't result in a wrong output, though, however it will be a slightly worse experience for those watching the BT2020 HLG version on a BT2020 SDR 100 nits display.
Any way to "expand" somehow that lower nit value?

[DTL]

If there is no need any headroom for superwhites - simple ColorYUV() can be added as some gain to Y to reach the required white level at SDR.

[tormento]

If there is no need any headroom for superwhites - simple ColorYUV() can be added as some gain to Y to reach the required white level at SDR.
Please elaborate with some examples. How and how much should I apply?

[DTL]

For example using Videotek tool look at the white peaks level - add Y gain (ColorYUV gain_y >1) so white peaks reach 0.7V (nominal white SDR). Select scene with most powerful white peaks.

Or simply use master control monitor and adjust Y gain so that no whites reach clipping.

[FranceBB]

Also please keep in mind that such a thing is fine for SDR. If you're on HLG, you shouldn't exceed 0.52V for the white while you expand.
Compressing and expanding is the besting thing ever, though, so you could use HLG Tools (https://github.com/wswartzendruber/hlg-tools) to generate a specific LUT for that very MaxCLL + reference white. https://forum.doom9.org/showthread.php?t=182499

For Avatar 2, something like:

pq2hlg --max-cll 280 --ref-white 48 --size 64 avatar2.cube

should achieve a pretty decent result, but you can ask William and I'm sure he'll help you. ;)

[DTL]

May be already exist some PC software simulators of AUTO KNEE function of cameras ? So using HDR source user can get best SDR possible with good enough compressed superwhites as it possible with real hardware good quality SDR camera. Though it is not linear transform as LUT but in best case adaptive per-scene highlight compression engine.

So again 4K/(4K+) HDR footage may be treated as RAW source to master HD SDR footage using offline software tools. Realtime AUTO KNEE can not analyse full future scene and offline software can. So offline AUTO KNEE expected to be better it its robotic activity. And users will not be limited with static HDR to SDR LUT-based or static math-based transforms.

[FranceBB]

Though it is not linear transform as LUT but in best case adaptive per-scene highlight compression engine.

yep, we talked about this in 2021 and it is possible, but only with libplacebo I'm afraid as there's nothing else that provides dynamic scene-based tonemapping according to a tweakable threshold.
What I had in mind in 2021 was tonemapping on a scene by scene basis by using the HDR Compute Peak, the HDR Peak Decay Rate and the HDR Scene Threshold parameters.
Those three parameters aren't coming out of the blue, but rather they're what MPV is using already (and has been for years):

--hdr-compute-peak=<auto|yes|no>
Compute the HDR peak and frame average brightness per-frame instead of relying on tagged metadata. These values are averaged over local regions as well as over several frames to prevent the value from jittering around too much. This option basically gives you dynamic, per-scene tone mapping.


--hdr-peak-decay-rate=<1.0..1000.0>
The decay rate used for the HDR peak detection algorithm (default: 100.0). This is only relevant when --hdr-compute-peak is enabled. Higher values make the peak decay more slowly, leading to more stable values at the cost of more "eye adaptation"-like effects (although this is mitigated somewhat by --hdr-scene-threshold). A value of 1.0 (the lowest possible) disables all averaging, meaning each frame's value is used directly as measured, but doing this is not recommended for "noisy" sources since it may lead to excessive flicker. (In signal theory terms, this controls the time constant "tau" of an IIR low pass filter)


--hdr-scene-threshold-low=<0.0..100.0>, --hdr-scene-threshold-high=<0.0..100.0>
The lower and upper thresholds (in dB) for a brightness difference to be considered a scene change (default: 5.5 low, 10.0 high). This is only relevant when --hdr-compute-peak is enabled. Normally, small fluctuations in the frame brightness are compensated for by the peak averaging mechanism, but for large jumps in the brightness this can result in the frame remaining too bright or too dark for up to several seconds, depending on the value of --hdr-peak-decay-rate.

[DTL]

" These values are averaged over local regions as well as over several frames to prevent the value from jittering around too much. "

The words 'several frames' looks too scary. The cutscene duration may take > 10 seconds. It looks it is not full offline scene analysis but only some decode-ahead realtime playback engine. If decoding too much frames to make real full cutscene analysis takes too much RAM (in 4K and larger) - it can be simple 2-pass engine with collecting all peaks data from footage at 1st pass and making per-scene adjustment at second pass.

[FranceBB]

The words 'several frames' looks too scary. The cutscene duration may take > 10 seconds. It looks it is not full offline scene analysis but only some decode-ahead realtime playback engine.

Yeah... they have some kind of limited lookahead but they certainly don't go over the whole file at once.



If decoding too much frames to make real full cutscene analysis takes too much RAM (in 4K and larger) - it can be simple 2-pass engine with collecting all peaks data from footage at 1st pass and making per-scene adjustment at second pass.

That's a very good idea and it's close to what I'm already doing.
Even though I'm just applying a static LUT, when a provider doesn't send the MaxCLL value for a PQ content, I use a two step approach in which Avisynth goes through the whole file saving the info about the MaxCLL of each frame 'till the end, then my logic gets rid of outliers (and potential compression overshooting errors) and then gets the output to populate the MaxCLL value and apply the right LUT. For instance, if a content is 4322 nits, it will use the PQ_to_BT709_4322.cube so that 4322 nits will become 100.

Of course this is all just with a static LUT, but given that I'm already "wasting" time with a first pass + second pass, I wouldn't really mind if this led to a glorious scene-aware tonemapping.
I tried to talk about this to Jean Philippe back in the days, but he was pretty busy back then and he's still pretty busy right now, just like everyone else (me included, unfortunately :( ), so he didn't really have time to look at this, but ideally this is something that should be implemented in HDR Tools.

Anyway, even with the current static methods we have today, we're not getting terrible results, so we somehow managed to live 'till 2023 without dynamic scene-aware tonemapping and I can keep living without it, but yeah, it would be nice to have, one day, in the future. :)

[FranceBB]

Hi everyone,
as a little update, sekrit made a new commit here: https://github.com/sekrit-twc/timecube/commit/5f34d7a061b8898707594a5ef84dfd3e75216741
in which he committed the manually written intrinsics in assembly for SSE4.1, AVX2 and AVX512, so that the new tetrahedral interpolation method is no longer limited to the slow C++ code only.
I tested it on my Intel Xeon Gold 6238R and it seems to be working ok, but please test, 'cause the more feedback we can give him, the better. :)

Here's the new build I made with sekrit's latest changes: vscube_07072023.zip (https://github.com/sekrit-twc/timecube/files/11977404/vscube_07072023.zip)

just like before, inside the package you'll find the normal vscube_x86.dll and vscube_x64.dll compiled with v143 and of course the "special" v141_xp /Zc:threadSafeInit msvc compiled version called "_XP":

- vscube_x86.dll
- vscube_x64.dll
- vscube_XP_x86.dll
- vscube_XP_x64.dll


It's still VapourSynth only, for now, but you can see the development of the thread here (https://github.com/sekrit-twc/timecube/issues/8) and that sekrit is willing to incorporate the Avisynth version into the main repository as soon as someone (perhaps Donald, who also commented on the thread) commits the latest changes required. Anyway, looks like we're finally getting somewhere. :)

Please test and report back if you find any issues.

[kedautinh12]

Very pleasure if you port it to avs+ :D

[FranceBB]

Very pleasure if you port it to avs+ :D

If you're talking to Donald, I'm not sure he regularly checks here, so it might take a while before he sees your post; I was under the impression that he was willing to do it just like the other versions, though, but I'm not him, so I don't know what his intentions are.
If you're talking to me, instead, well, if and only if no one is willing to do it, then I'll try to do it myself ('cause I don't wanna have two people doing the same thing twice xD).
We'll see, but it would be better if everyone was here on Doom9 regularly, so that we didn't have to speculate :(

[gonca]

I believe he has a test version ready

[FranceBB]

I believe he has a test version ready

You're correct. Donald just released an Avisynth test version: Link (https://rationalqm.us/misc/AVSCube_test.rar)

I'll test it on Monday morning as soon as I get back to work and see if the results are consistent with the VapourSynth version.


The new syntax accepts:

interp: int, 0 = linear, 1 = tetrahedral

like in the VapourSynth version.


By the way, this is applied to a real life football content to go from UHD XAVC Intra Class 300 50p BT2020 HLG 1000 nits to an FULL HD XDCAM-50 25i BT709 SDR 100 nits:


#Indexing UHD 50p BT2020 HLG 1000 nits live feed
LWLibavVideoSource("Test_HLG.mxf")

#Screw frame properties
propclearall()

#From 4:2:2 10bit planar Narrow Range to RGB Planar 16bit Narrow Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:std-b67:2020:limited=>rgb:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion",
use_props=0)

#From HLG to BT709 with 16bit precision
Cube("A:\LUTs\9c_HLG_BT709_Type3_Display_DownMapping_SuperWhite_nocomp-v1_5.cube", interp=1, fullrange=1)

#From RGB 16bit planar Narrow Range to YUV422 8bit planar Narrow Range with dithering
z_ConvertFormat(pixel_type="YUV422", colorspace_op="rgb:709:709:limited=>709:709:709:limited", resample_filter_uv="spline64", dither_type="error_diffusion", use_props=0)

#Downscaling to FULL HD
SinPowResizeMT(1920, 1080)

#Interlace to 25i
assumeTFF()
separatefields()
selectevery(4,0,3)
weave()



if tests go fine, I'll report back.
Speaking of which, I hope he'll release the source code given that I'll compile the x86 version as well to include it in FFAStrans (https://forum.doom9.org/showthread.php?t=176655) as I have to support both 64bit and 32bit users as always ;)

[kedautinh12]

Latest DGCube
https://www.rationalqm.us/misc/?C=M;O=D

[FranceBB]

Something doesn't feel right.


LWLibavVideoSource("K:\4K_Ama\05_DI SERVIZIO - EBU UHD HDR COLOR BARS\EBU_HDR_COLOUR_BARS_2160p_v210.mov")

propClearAll()

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:std-b67:2020:limited=>rgb:std-b67:2020:limited", resample_filter_uv="Spline64", dither_type="error_diffusion", use_props=0)

tetrahedral=Cube("B:\avdb\Server\encoder\encoder\Processors\avs_plugins\LUTs\9c_HLG_BT709_Type3_Display_DownMapping_SuperWhite_nocomp-v1_5.cube", interp=1, fullrange=1).VideoTek()
trilinear=Cube("B:\avdb\Server\encoder\encoder\Processors\avs_plugins\LUTs\9c_HLG_BT709_Type3_Display_DownMapping_SuperWhite_nocomp-v1_5.cube", interp=0, fullrange=1).VideoTek()
donald=DGCube("B:\avdb\Server\encoder\encoder\Processors\avs_plugins\LUTs\9c_HLG_BT709_Type3_Display_DownMapping_SuperWhite_nocomp-v1_5.cube", in="full", out="full", lut="full", interp="tetrahedral").VideoTek()

StackVertical(trilinear, tetrahedral, donald)


They produce three VERY different results.

https://user-images.githubusercontent.com/18946343/252718564-6cc33ab9-bffc-4dd7-8a57-a1636488c8b5.png

What I don't understand is why Cube() tetrahedral and DGCube() tetrahedral are so much different.

Souce: https://we.tl/t-xqG6XosULK
Cube_Tetrahedral: https://we.tl/t-GLjsVvJ6Ad
DGCube_Tetrahedral: https://we.tl/t-5L8KCVRfmg
Cube_Trilinear: https://we.tl/t-iBhR03kbFP

[FranceBB]

Also, DGCube with Tetrahedral is actually the one that gets it right.
In the Cube version, however, it looks like a bit of both versions is applied. You can see the ripples of the trilinear and also the smoothness of the tetrahedral. Why?

Cube Trilinear (rightly showing ripples):
https://user-images.githubusercontent.com/18946343/252724560-114173bd-d2a6-4c1f-9fe4-e7fe04cdc2af.png

DGCube Tetrahedral (no ripples, actually correct):
https://user-images.githubusercontent.com/18946343/252724797-2b7fda60-c6a9-44de-9875-063610838967.png

Cube Tetrahedral (something in-between, NOT correct):
https://user-images.githubusercontent.com/18946343/252724954-6e1a6f5e-d075-4089-94fa-19452abee4e9.png

Cube Tetrahedral is something in between...? O_O

[poisondeathray]

Do you get the same results with the vpy vscube version ?

[FranceBB]

@poisondeathray I found the culprit. It's the manually written assembly code for AVX2. I reported it back to both Donald and Sekrit.

The normal C++ code is actually correct.
https://user-images.githubusercontent.com/18946343/252745440-f84911cc-7ea4-43ff-be50-f807ac813d83.png

The SSE4.1 assembly code is also correct.
https://user-images.githubusercontent.com/18946343/252745542-dedb44eb-04da-42b2-9416-c5970c31690d.png

The AVX2 assembly code is wrong.
https://user-images.githubusercontent.com/18946343/252745656-13b452db-b2bf-48dc-afb7-36304615360a.png

The AVX512 assembly code is correct.
https://user-images.githubusercontent.com/18946343/252745744-c4833162-f814-4a6f-84ac-1640bc4ea382.png

So it looks like there's something wrong only in the AVX2 code.

[FranceBB]

new version by Donald and Sekrit for Avisynth with the AVX2 code fixed: Link (https://rationalqm.us/cube/AVSCube_test.rar)


Changelog

v1.4
2023/07/12
Added tetrahedral interpolation.

v1.3
2022/01/21
Updated to timecube version 1.3 base code.
Added frame properties passthrough.
Updated the help document.

v1.1
2019/01/25
Rather than upgrade to r2a I ported the fullrange fix only.

v1.0
2018/08/16
Initial release


Usage:

Cube (clip, string "cube", int "cpu", int "interp", int "fullrange")

clip =
Input clip; the filter receives and delivers the RGBP16 color space.

string cube = ""
Full path of the 3D LUT file. This must be an Adobe *.cube file.

int cpu = INT_MAX
cpu: default INT_MAX, which selects best SIMD.
0 : SIMD_NONE
1 : SIMD_SSE41
2 : SIMD_AVX2
3 : SIMD_AVX512

int interp = 0
interp: default 0 selects the worse interpolation method for backwards compatibility reasons.
0 : trilinear interpolation
1 : tetrahedral interpolation


int fullrange = 1
0 : limited range
1 : full range


Trilinear vs Tetrahedral:

Trilinear interpolation is used by default only for historical reasons.
When Cube() was first developed it only had one interpolation method, namely trilinear.
Although this was far from ideal as it can produce ripples, plenty of people used it (as they were using this function), therefore when tetrahedral was introduced it was agreed to leave trilinear as default to allow people to still obtain the same results when upgrading the version.
However, in practice, tetrahedral should always be used and it's the suggested method.

Source (left) - Trilinear (center) - Tetrahedral (right):

https://i.imgur.com/uzoYkFm.png


As you can see, in the right hand side waveform (tetrahedral) there are no ripples, unlike the ones you can see in the trilinear interpolation at the center.



Usage example:

Example 1:


#Indexing
LWLibavVideoSource("Z:\00_INGEST_MAM\ENCODER\BACKUP LIVIO\SAMPLE-NEW\Sky UK HDR HLG Test Pattern\T3_LUT_LP_UHD_BT2020_50_v210.mov")

propClearAll()

#From Limited Range 10bit YUV422 to 16bit Limited Range Planar RGB
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:std-b67:2020:limited=>rgb:std-b67:2020:limited", resample_filter_uv="Spline64", dither_type="error_diffusion", use_props=0)

#From BT2020 HLG to BT709 SDR
Cube("\\avdb\Server\encoder\encoder\Processors\avs_plugins\LUTs\9c_HLG_BT709_Type3_Display_DownMapping_SuperWhite_nocomp-v1_5.cube", interp=1, fullrange=1)

#From 16bit Limited Range Planar RGB to Limited Range 10bit YUV422 with dithering
z_ConvertFormat(pixel_type="YUV422P10", colorspace_op="rgb:709:709:limited=>709:709:709:limited", resample_filter_uv="Spline64", dither_type="error_diffusion", use_props=0)



Example 2:


# Source
DGSource("THE GREAT WALL.dgi")

# From Limited Range 10bit YUV420 to 16bit Full Range Planar RGB
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:st2084:2020:limited=>rgb:st2084:2020:full", dither_type="error_diffusion")

# From PQ to HLG
Cube("PQ_to_HLG.cube", fullrange=1, interp=1)

# From 16bit Full Range Planar RGB to Limited Range 10bit YUV420 with dithering
z_ConvertFormat(pixel_type="YUV420P10", colorspace_op="rgb:std-b67:2020:full=>2020:std-b67:2020:limited", dither_type="error_diffusion")

[rgr]

I'm not very knowledgeable, so I need help understanding something :)

I created a LUT that, when applied to a clip, changes the Y range from 16-254 to 16-235 while keeping the UV within 16-235. Target: videos from cameras with Y values ​​in the 16-255 range.

In AviSynth, it works great with 'Cube("lutCreatorJs-64.cube",interp=1,fullrange=1)'. A camera file loaded as full-range and treated with this command looks identical to a limited-range file created with Y changed from 16-255 to 16-235 (AviSynth and ApplyGradationCurves(lumapoints="0,0,16,16,254,235")).

But it doesn't work in Vegas, Resolve, or Premiere. In Vegas the file looks too bright on the graph (exceeds the limited range), while playing in Resolve and Premiere the brightness ends around 90%.
Does anyone know why?

[rgr]

Cannot upload LUT file here:
https://files.fm/u/aa3vkrgcv8