Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 2nd August 2022, 17:03   #221  |  Link
TomArrow
Registered User
 
Join Date: Dec 2017
Posts: 82
I've a quick question about z_ConvertFormat. Does it write any usually not visible metadata to the clip that a software like ffmpeg can use? A friend reported that in conjunction with ffmpeg, using this filter (just doing a straight conversion to the same color space) can result in changes to the output file in terms of metadata, transfer characteristic specifically.

Of course this doesn't have to be a bad thing per se, but it would be good to know.
TomArrow is offline   Reply With Quote
Old 2nd August 2022, 18:56   #222  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Metropolitan City of Milan, Italy
Posts: 2,273
I think it does make use of frame properties, so if you're using a recent version of Avisynth and a frame properties enabled build of ffmpeg, it will read them.

This is just an example:

Code:
#Indexing ProRes HQ 4:2:2 23,976p 10bit planar HDR PQ source
video=LWLibavVideoSource("KFH00862.mov")
ch1=LWLibavAudioSource("KFH00862.mov", stream_index=1)
ch2=LWLibavAudioSource("KFH00862.mov", stream_index=2)
ch3=LWLibavAudioSource("KFH00862.mov", stream_index=3)
ch4=LWLibavAudioSource("KFH00862.mov", stream_index=4)
ch5=LWLibavAudioSource("KFH00862.mov", stream_index=5)
ch6=LWLibavAudioSource("KFH00862.mov", stream_index=6)
ch7=LWLibavAudioSource("KFH00862.mov", stream_index=7)
ch8=LWLibavAudioSource("KFH00862.mov", stream_index=8)
ch9=LWLibavAudioSource("KFH00862.mov", stream_index=9)
ch10=LWLibavAudioSource("KFH00862.mov", stream_index=10)
ch11=LWLibavAudioSource("KFH00862.mov", stream_index=11)
ch12=LWLibavAudioSource("KFH00862.mov", stream_index=12)
ch13=LWLibavAudioSource("KFH00862.mov", stream_index=13)
ch14=LWLibavAudioSource("KFH00862.mov", stream_index=14)
ch15=LWLibavAudioSource("KFH00862.mov", stream_index=15)
ch16=LWLibavAudioSource("KFH00862.mov", stream_index=16)
audio=MergeChannels(ch1, ch2, ch3, ch4, ch5, ch6, ch7, ch8, ch9, ch10, ch11, ch12, ch13, ch14, ch15, ch16)
AudioDub(video, audio)

#From 4:2:2 10bit to 4:2:2 16bit planar
ConvertBits(16)

#From 4:2:2 16bit planar Narrow Range to RGB Planar 16bit Narrow Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:2020:2020:limited=>rgb:2020:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")

#From PQ to HLG with 16bit precision
Cube("WarnerBros_PQToHLG_MaxCLL_2752.cube", fullrange=true)

#From RGB 16bit planar Narrow Range to YUV422 16bit planar Narrow Range
z_ConvertFormat(pixel_type="YUV422P16", colorspace_op="rgb:2020:2020:limited=>2020:2020:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")

#Filling the right frame properties with HLG
propset("_Transfer", 18)

#Speed-up + pitch adjustment
ResampleAudio(48000)
AssumeFPS(25, 1, false)
TimeStretch(tempo=100.0*25.0/(24000.0/1001.0))
ResampleAudio(48000)

#Clipping 
Limiter(min_luma=4096, max_luma=60160, min_chroma=4096, max_chroma=60160)
as you can see here I'm populating the transfer characteristics value with the right one, in fact this is the important bit:

Code:
#Filling the right frame properties with HLG
propset("_Transfer", 18)
which will set the arib-std-b67 transfer, so HLG, so that when FFMpeg is gonna read the script with -i "AVS Script.avs" it will read them correctly.
If you wanna be absolutely sure that frame properties are set correctly, you can check them with AVSPmod and eventually populate them accordingly yourself, otherwise you can always use propClearAll() right after indexing and everything will be business as usual (I generally prefer the latter).

Last edited by FranceBB; 4th August 2022 at 07:52.
FranceBB is offline   Reply With Quote
Old 3rd August 2022, 09:14   #223  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 301
It's probably better that a moderator to move posts from #109 to #213 (including) in a separate thread to avoid confusions between the two builds.

At the moment the main diff between the builds(r2 and r14):
Code:
- how alpha plane is treated: r2 - ZIMG_ALPHA_STRAIGHT; r14 - ZIMG_ALPHA_PREMULTIPLIED

- r14 has additional pixel types aliases - RGBP8, RGBAP8, YUV444, YUV444P8, YUVA444P8, YUV422, YUV422P8, YUVA422P8, YUV420, YUV420P8 , YUVA420P8, YUV411, YUV411P8

- r14 has additional chroma locations - TOP_LEFT, TOP, BOTTOM_LEFT, BOTTOM
- r2 has additional chroma location alias - dv (LEFT)

- r2 has additional resample aliases - gauss (BICUBIC), lanczos4 (LANCZOS), blackman (LANCZOS), sinc (LANCZOS), z_BlackmanResize (LANCZOS), z_GaussResize (BICUBIC), z_SincResize (LANCZOS)

- r14 has additional args: cpu_type, nominal_luminance, approximate_gamma, use_props

- r14 has support for frame properties

- r14 has additional args for the compatibility wrappers (z_xxResize) - chromaloc_op, dither
Btw r2 still has the parsing bug when colospace_op values with dash are used - for example std-b67/st431-2...

Last edited by StvG; 3rd August 2022 at 09:19.
StvG is offline   Reply With Quote
Old 3rd August 2022, 09:56   #224  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 551
As the jpsdr post at https://forum.doom9.org/showthread.p...87#post1970987 -

I don't know what the others plugins output, neither what they expect in input. "linear", but wich one ? scene linear light and scaled by camera exposure or displayed linear ?

Can the z.lib switch between different 'linear' conversions from HDR formats ?

Following tests at https://forum.doom9.org/showthread.p...95#post1970995 shows that the OOTF on/off at the process of conversion system transfer to 'linear' looks like make the main difference. The difference is not great and may be too few persons now make processing in linear to detect this difference at real footage.
DTL is offline   Reply With Quote
Old 4th August 2022, 13:32   #225  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 301
Quote:
Originally Posted by DTL View Post
As the jpsdr post at https://forum.doom9.org/showthread.p...87#post1970987 -

I don't know what the others plugins output, neither what they expect in input. "linear", but wich one ? scene linear light and scaled by camera exposure or displayed linear ?

Can the z.lib switch between different 'linear' conversions from HDR formats ?

Following tests at https://forum.doom9.org/showthread.p...95#post1970995 shows that the OOTF on/off at the process of conversion system transfer to 'linear' looks like make the main difference. The difference is not great and may be too few persons now make processing in linear to detect this difference at real footage.
zimg itself can switch between scene-referred and display-referred case (here) but this option is not api exposed so the avs/vs plugin is using only the display-referred one.
StvG is offline   Reply With Quote
Old 4th August 2022, 14:32   #226  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 551
Can the OOTF stage be simulated as some AVS scripting after 'first stage conversion to linear' with zimg-AVS plugin ? To get 'scene-referred' too if required ?

"but this option is not api exposed"

May be it can be feature-request and API supplement and finally go into next plugin version ?

So for sources 'conditioned' in 'scene-linear light' it can be used as a one-step conversion from HDR transfer encoded form into linear for scaling or other processing.

Last edited by DTL; 4th August 2022 at 14:37.
DTL is offline   Reply With Quote
Old 4th August 2022, 23:23   #227  |  Link
TomArrow
Registered User
 
Join Date: Dec 2017
Posts: 82
Quote:
Originally Posted by FranceBB View Post
I think it does make use of frame properties, so if you're using a recent version of Avisynth and a frame properties enabled build of ffmpeg, it will read them.

This is just an example:

Code:
#Indexing ProRes HQ 4:2:2 23,976p 10bit planar HDR PQ source
video=LWLibavVideoSource("KFH00862.mov")
ch1=LWLibavAudioSource("KFH00862.mov", stream_index=1)
ch2=LWLibavAudioSource("KFH00862.mov", stream_index=2)
ch3=LWLibavAudioSource("KFH00862.mov", stream_index=3)
ch4=LWLibavAudioSource("KFH00862.mov", stream_index=4)
ch5=LWLibavAudioSource("KFH00862.mov", stream_index=5)
ch6=LWLibavAudioSource("KFH00862.mov", stream_index=6)
ch7=LWLibavAudioSource("KFH00862.mov", stream_index=7)
ch8=LWLibavAudioSource("KFH00862.mov", stream_index=8)
ch9=LWLibavAudioSource("KFH00862.mov", stream_index=9)
ch10=LWLibavAudioSource("KFH00862.mov", stream_index=10)
ch11=LWLibavAudioSource("KFH00862.mov", stream_index=11)
ch12=LWLibavAudioSource("KFH00862.mov", stream_index=12)
ch13=LWLibavAudioSource("KFH00862.mov", stream_index=13)
ch14=LWLibavAudioSource("KFH00862.mov", stream_index=14)
ch15=LWLibavAudioSource("KFH00862.mov", stream_index=15)
ch16=LWLibavAudioSource("KFH00862.mov", stream_index=16)
audio=MergeChannels(ch1, ch2, ch3, ch4, ch5, ch6, ch7, ch8, ch9, ch10, ch11, ch12, ch13, ch14, ch15, ch16)
AudioDub(video, audio)

#From 4:2:2 10bit to 4:2:2 16bit planar
ConvertBits(16)

#From 4:2:2 16bit planar Narrow Range to RGB Planar 16bit Narrow Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:2020:2020:limited=>rgb:2020:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")

#From PQ to HLG with 16bit precision
Cube("WarnerBros_PQToHLG_MaxCLL_2752.cube", fullrange=true)

#From RGB 16bit planar Narrow Range to YUV422 16bit planar Narrow Range
z_ConvertFormat(pixel_type="YUV422P16", colorspace_op="rgb:2020:2020:limited=>2020:2020:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")

#Filling the right frame properties with HLG
propset("_Transfer", 18)

#Speed-up + pitch adjustment
ResampleAudio(48000)
AssumeFPS(25, 1, false)
TimeStretch(tempo=100.0*25.0/(24000.0/1001.0))
ResampleAudio(48000)

#Clipping 
Limiter(min_luma=4096, max_luma=60160, min_chroma=4096, max_chroma=60160)
as you can see here I'm populating the transfer characteristics value with the right one, in fact this is the important bit:

Code:
#Filling the right frame properties with HLG
propset("_Transfer", 18)
which will set the arib-std-b67 transfer, so HLG, so that when FFMpeg is gonna read the script with -i "AVS Script.avs" it will read them correctly.
If you wanna be absolutely sure that frame properties are set correctly, you can check them with AVSPmod and eventually populate them accordingly yourself, otherwise you can always use propClearAll() right after indexing and everything will be business as usual (I generally prefer the latter).
Thanks! That could explain it.
TomArrow is offline   Reply With Quote
Old 5th August 2022, 19:39   #228  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 301
avsresize_r15 (pass: gjFnhBCshVL3):
- relaxed restrictions for use_props=0;
- added scene_referred parameter (default false).

Quote:
Originally Posted by FranceBB View Post
... otherwise you can always use propClearAll() right after indexing and everything will be business as usual (I generally prefer the latter).
If you don't want the source frame properties to be read use "use_props=0". The performance would be better than propClearAll() and checking for source frame properties.

Quote:
Originally Posted by DTL View Post
Can the OOTF stage be simulated as some AVS scripting after 'first stage conversion to linear' with zimg-AVS plugin ? To get 'scene-referred' too if required ?

"but this option is not api exposed"

May be it can be feature-request and API supplement and finally go into next plugin version ?

So for sources 'conditioned' in 'scene-linear light' it can be used as a one-step conversion from HDR transfer encoded form into linear for scaling or other processing.
Try the new scene_referred parameter.
StvG is offline   Reply With Quote
Old 5th August 2022, 21:57   #229  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 551
Quote:
Originally Posted by StvG View Post
Try the new scene_referred parameter.
It finally work about as expected https://forum.doom9.org/showthread.p...23#post1972723 but looks the float RGB values are very different for each plugin and require deep correction with 'magic numbers' like about 3.0 gain for one plugin and 0.157 for avsresize. But finally it can convert 10bit HLG YUV codevalues into rec709 via RGB-linear-float path with 'exact' precision in 8bit R'G'B' output. And the intermediate values forming transient looks about good too.

Last edited by DTL; 5th August 2022 at 22:01.
DTL is offline   Reply With Quote
Old 6th August 2022, 16:22   #230  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Metropolitan City of Milan, Italy
Posts: 2,273
Quote:
Originally Posted by StvG View Post
If you don't want the source frame properties to be read use "use_props=0". The performance would be better than propClearAll() and checking for source frame properties.
Gotcha, will do!


Code:
#From 4:2:2 16bit planar Narrow Range to RGB Planar 16bit Narrow Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:2020:2020:limited=>rgb:2020:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion", use_props=0)

#From PQ to HLG with 16bit precision
Cube("\\myserver.local\avdb\Server\encoder\encoder\Processors\avs_plugins\LUTs\WarnerBros_PQToHLG_MaxCLL_2752.cube", fullrange=true)

#From RGB 16bit planar Narrow Range to YUV422 16bit planar Narrow Range
z_ConvertFormat(pixel_type="YUV422P16", colorspace_op="rgb:2020:2020:limited=>2020:2020:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion", use_props=0)

#Filling the right frame properties with HLG
propset("_Transfer", 18)
FranceBB is offline   Reply With Quote
Old 6th August 2022, 18:58   #231  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 301
Quote:
Originally Posted by FranceBB View Post
Gotcha, will do!


Code:
#From 4:2:2 16bit planar Narrow Range to RGB Planar 16bit Narrow Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:2020:2020:limited=>rgb:2020:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion", use_props=0)

#From PQ to HLG with 16bit precision
Cube("\\myserver.local\avdb\Server\encoder\encoder\Processors\avs_plugins\LUTs\WarnerBros_PQToHLG_MaxCLL_2752.cube", fullrange=true)

#From RGB 16bit planar Narrow Range to YUV422 16bit planar Narrow Range
z_ConvertFormat(pixel_type="YUV422P16", colorspace_op="rgb:2020:2020:limited=>2020:2020:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion", use_props=0)

#Filling the right frame properties with HLG
propset("_Transfer", 18)
The last line propset("_Transfer", 18) can be omitted by using z_ConvertFormat(pixel_type="YUV422P16", colorspace_op="rgb:std-b67:2020:limited=>2020:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion", use_props=0).
StvG is offline   Reply With Quote
Old 8th August 2022, 12:45   #232  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Metropolitan City of Milan, Italy
Posts: 2,273
Thanks, I have updated my scripts!

I have another question, though.
I have a BT601 SD 25i TFF source that I want to convert to BT709 but I want to leave interlacing alone.
Of course I could just bob, use z_ConvertFormat() and then separatefields() etc, but I'd like to avoid that.
I noticed that in z_ConvertFormat() there's "interlaced" as a parameter that can be either true or false, however if I set it to "true", it says "Interlaced operation not supported":

Quote:
video=FFVideoSource("G:\txmaster\sd\PRM7_0247234_SD_Remuxed_v2.mxf")
ch1=FFAudioSource("G:\txmaster\sd\PRM7_0247234_SD_Remuxed_v2.mxf", track=1)
ch2=FFAudioSource("G:\txmaster\sd\PRM7_0247234_SD_Remuxed_v2.mxf", track=2)
audio=MergeChannels(ch1, ch2)
AudioDub(video, audio)

propclearall()

z_ConvertFormat(colorspace_op="470bg:470bg:470bg:limited=>709:709:709:limited", pixel_type="YUV422P10", resample_filter="lanczos", dither_type="error_diffusion", interlaced=true, use_props=0)


why?

this works but I'd like to avoid it:

Quote:
video=FFVideoSource("G:\txmaster\sd\PRM7_0247234_SD_Remuxed_v2.mxf")
ch1=FFAudioSource("G:\txmaster\sd\PRM7_0247234_SD_Remuxed_v2.mxf", track=1)
ch2=FFAudioSource("G:\txmaster\sd\PRM7_0247234_SD_Remuxed_v2.mxf", track=2)
audio=MergeChannels(ch1, ch2)
AudioDub(video, audio)

propclearall()

Bob()
z_ConvertFormat(colorspace_op="470bg:470bg:470bg:limited=>709:709:709:limited", pixel_type="YUV422P10", resample_filter="lanczos", dither_type="error_diffusion", interlaced=false, use_props=0)
assumeTFF()
separatefields()
selectevery(4,0,3)
weave()

Last edited by FranceBB; 8th August 2022 at 12:54.
FranceBB is offline   Reply With Quote
Old 9th August 2022, 00:34   #233  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 301
The support for interlaced video isn't added. Btw why propclearall() when you have use_props=0?
StvG is offline   Reply With Quote
Old 9th August 2022, 05:48   #234  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Metropolitan City of Milan, Italy
Posts: 2,273
I see...
Will it ever be added?
As to propclearall (), if I don't use it, even if avsresize won't use them with use_props=0, it won't nuke them, so they would still be there which is even worse 'cause at that point they would just be wrong and not updated.

EDIT: Actually, you're right, use_props=0 make it not read them but still set them, so they're gonna be correct, my bad.

Last edited by FranceBB; 9th August 2022 at 09:11.
FranceBB is offline   Reply With Quote
Old 10th August 2022, 01:52   #235  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 301
avsresize_r16 (pass: ZrDjU0BAGRha):
- improved use_props=-1;
- throw error when use_props=0 and _FieldBased/_Field > 0;
- added support for interlaced clips: when interlaced=true, always use_props=0.
StvG is offline   Reply With Quote
Old 10th August 2022, 08:19   #236  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Metropolitan City of Milan, Italy
Posts: 2,273
What a lovely morning present.
Yesterday Donald gifted the community with a CUDA porting of Cube(), today you introduced interlaced = true support in avsresize.
I wonder what's gonna be next...
FranceBB is offline   Reply With Quote
Old 10th August 2022, 13:47   #237  |  Link
kedautinh12
Registered User
 
Join Date: Jan 2018
Posts: 1,270
Quote:
Originally Posted by FranceBB View Post
What a lovely morning present.
Yesterday Donald gifted the community with a CUDA porting of Cube(), today you introduced interlaced = true support in avsresize.
I wonder what's gonna be next...
Thanks for info of CUDA ver of cube but where's the link?

Last edited by kedautinh12; 10th August 2022 at 13:53.
kedautinh12 is offline   Reply With Quote
Old 10th August 2022, 15:04   #238  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,243
Quote:
Originally Posted by kedautinh12 View Post
Thanks for info of CUDA ver of cube but where's the link?
Here
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 10th August 2022, 15:35   #239  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Metropolitan City of Milan, Italy
Posts: 2,273
ehehehe, you know, Tormento is my drug-deal--- ehm I mean DLL dealer too! Especially for GPU-related stuff eheheheh
FranceBB is offline   Reply With Quote
Old 13th August 2022, 17:12   #240  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 301
avsresize_r17 (pass: OPtHxyD16GmI): use_props=-1 improved a bit more.
StvG is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 23:35.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.