Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 30th January 2020, 23:00   #41  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Metropolitan City of Milan, Italy
Posts: 1,935
Quote:
Originally Posted by videoh View Post
It's certainly something I could do. May I ask what source filter you using to deliver 4:2:2 and 4:4:4? A sample script would be helpful.
My indexer is simply FFMpegSource2 and my source files are generally either AppleProRes masterfiles or DNxHR masterfiles delivered directly by the production company that made the movie.
I can also provide a very short sample (few seconds) if you want but it has to be on a secured FTP provided that you will use it for test only and that you will never ever share it with anyone else. (Copyright is not a joke).

Last edited by FranceBB; 31st January 2020 at 00:26.
FranceBB is online now   Reply With Quote
Old 31st January 2020, 01:04   #42  |  Link
Richard1485
Guest
 
Posts: n/a
Quote:
Originally Posted by manolito View Post
FFmpeg (Atak version)
Code:
zscale=s=704x396,zscale=tin=smpte2084:min=bt2020nc:pin=bt2020:rin=tv:t=smpte2084:m=bt2020nc:p=bt2020:r=tv,zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p,colormatrix=bt709:bt601
FFmpeg (Tebasuna version):
Code:
zscale=s=704x396,zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p,colormatrix=bt709:bt601
Is there a difference between Atak and Tebasuna's solutions? The outputs looked pretty similar to me, and I guessed that the difference between the two was that the settings that Atak spells out are (or have become) default settings that are implied in Tebasuna's example.
  Reply With Quote
Old 31st January 2020, 04:15   #43  |  Link
videoh
Useful n00b
 
Join Date: Jul 2014
Posts: 1,614
Quote:
Originally Posted by FranceBB View Post
I can also provide a very short sample (few seconds) if you want but it has to be on a secured FTP provided that you will use it for test only and that you will never ever share it with anyone else. (Copyright is not a joke).
PM sent. Thank you.
videoh is offline   Reply With Quote
Old 31st January 2020, 11:36   #44  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,592
What method/program would anyone recommend if you want to convert HDR range to SDR and not reduce the color gamut to rec709? It looks like HDRTools can do it.

FranceBB,
I was looking at your LUT collection and while I have no idea how to use them yet (I've been meaning to work that out for a while) is there never a reason to convert HDR to SDR without changing the colorimetry?

Last edited by hello_hello; 31st January 2020 at 11:48.
hello_hello is offline   Reply With Quote
Old 31st January 2020, 11:51   #45  |  Link
richardpl
Registered User
 
Join Date: Jan 2012
Posts: 184
Quote:
Originally Posted by Richard1485 View Post
Is there a difference between Atak and Tebasuna's solutions? The outputs looked pretty similar to me, and I guessed that the difference between the two was that the settings that Atak spells out are (or have become) default settings that are implied in Tebasuna's example.
Stop using colormatrix filter - it is bad and evil. Instead use zscale options directly.
richardpl is offline   Reply With Quote
Old 31st January 2020, 12:20   #46  |  Link
Richard1485
Guest
 
Posts: n/a
Quote:
Originally Posted by richardpl View Post
Stop using colormatrix filter - it is bad and evil. Instead use zscale options directly.
Thanks for your input, but I don't use colormatrix. That must be something that was tacked on to those examples, which I was only quoting. My question related to the difference in syntax.
  Reply With Quote
Old 31st January 2020, 15:54   #47  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 3,013
Quote:
Originally Posted by richardpl View Post
Stop using colormatrix filter - it is bad and evil. Instead use zscale options directly.
The FFmpeg command lines I posted here:
https://forum.doom9.org/showthread.p...52#post1897752

use ColorMatrix for the final change from 709 to 601 for a very simple reason: Replacing 709 by 601 in the zscale parameters does not work, dozens of error messages. So you would have to tell me how to modify these parameters to get a bt601 output. BTW I would not be willing to pay money for this advice...
manolito is offline   Reply With Quote
Old 31st January 2020, 17:00   #48  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Metropolitan City of Milan, Italy
Posts: 1,935
Quote:
Originally Posted by hello_hello View Post
What method/program would anyone recommend if you want to convert HDR range to SDR and not reduce the color gamut to rec709? It looks like HDRTools can do it.

FranceBB,
I was looking at your LUT collection and while I have no idea how to use them yet (I've been meaning to work that out for a while) is there never a reason to convert HDR to SDR without changing the colorimetry?
Well, so basically you wanna go from BT2100 HDR PQ to BT2020nc SDR, is that right?
But... what for? I mean, if it's for TV compatibility, then it would make much more sense to go from PQ to HLG so that both SDR TVs and HDR ones will be able to play it without compromising the original HDR too much. If it's for FULL HD SDR TV compatibility, then they won't be able to interpret the new matrix anyway so it wouldn't make sense.
Anyway, I can definitely make a LUT from BT2100 PQ to BT2020 SDR and from BT2020nc HLG to BT2020 SDR.
As to how to use my matrices of linear transformation, it's pretty easy:

Code:
FFVideoSource("example.mxf")

ConvertBits(16)

ConvertToPlanarRGB()

Cube("C:\Programmi\AviSynth+\LUTs\example.cube", cpu=1, fullrange=false)
where "example.cube" is the matrix of linear transformation. Essentially a matrix of linear transformation is something that maps a point to another. I don't know whether you're familiar with linear algebra or not (I personally love it). If you are not familiar and you wanna have a visual intuition about what it, I strongly suggest you this series of videos made by Grant Sanderson: Link and check out "Linear Transformation". It's gonna be so much fun as it's extremely well animated. If you get very passionate about it there are real lectures made by Doctor Gilbert Strang (MIT).


EDIT: I made such a matrix: Link

Last edited by FranceBB; 31st January 2020 at 17:52.
FranceBB is online now   Reply With Quote
Old 31st January 2020, 17:03   #49  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,094
Quote:
Originally Posted by manolito View Post
The FFmpeg command lines I posted here:
https://forum.doom9.org/showthread.p...52#post1897752

use ColorMatrix for the final change from 709 to 601 for a very simple reason: Replacing 709 by 601 in the zscale parameters does not work, dozens of error messages. So you would have to tell me how to modify these parameters to get a bt601 output. BTW I would not be willing to pay money for this advice...
That's because it's not called bt601, it's called bt470bg (PAL) or bt470m (NTSC).

See this thread for some of the subtleties of analog SD TV (non-)standardization.

e: note that for color matrix there is no bt470m though.

e2: actually let me make a little table or something of the potentially relevant values.

For color primaries, there is
- bt709
- bt470m (old NTSC, I mean like ancient)
- bt470bg (equivalent to Rec.601 for PAL and SECAM)
- smpte170m and smpte240m (different metadata values but identical functionally; also equivalent to Rec.601 for NTSC)

For transfer characteristics ("gamma"), there is
- 601 and bt709 (different metadata values but functionally identical, note no "bt" before 601; you can also use smpte170m as an alias for 601)
- bt470bg (old PAL, assumed display gamma 2.8)
- bt470m (old NTSC, assumed display gamma 2.2)

For color matrix, there is only
- bt709
- bt470bg (equivalent to Rec.601 for both PAL, NTSC and SECAM)

So assuming you're targeting a PAL TV that is at least newer than fall of the Berlin Wall, you probably want primaries=bt470bg, transfer=601 and matrix=bt470bg.

The fact that colormatrix only touches the matrix, and PAL and NTSC happen to use identical values for the matrix in Rec601, is probably what has led to the somewhat confusing use of "601" as a generic name for all SDTV colorimetry even though it's ambiguous what it means for transfer and primaries.

Last edited by TheFluff; 31st January 2020 at 17:41.
TheFluff is offline   Reply With Quote
Old 1st February 2020, 01:00   #50  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 3,013
Did some more tests with the FFmpeg command lines for HDR to SDR conversions. Still a little bit confused...

First of all I do not want to get into another dispute about using ColorMatrix. I did try using zscale instead of colormatrix, it did work in the end, but the result was indistinguishable from the colormatrix result.

What really bothers me is that the FFmpeg command lines I posted do work sometimes with good results, but for other source files I get results like this:
https://forum.doom9.org/showthread.p...73#post1875073

It does work nicely for the LG New York clip, but for the Sony Camping in Nature clip it did not work. To get a watchable result I had to change the tonemap algo from hable to clip or mobius. No way to make it work for hable or reinhard.

And both the clip and mobius methods result in very ugly colors. Quite unusable...

I first suspected that it had something to do with the FFmpeg versions I used, but after many tests with all kinds of different older versions I know that this is not true.

The clip properties of these two clips are very similar. Here is what MediaInfo has to say:
Quote:
LG New York
=============


Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format_Commercial_IfAny : HDR10
Format-Profil : Main 10@L5@High
Codec-ID : V_MPEGH/ISO/HEVC
Dauer : 1 min 12s
Bitrate : 48,1 Mb/s
Breite : 3 840 Pixel
Höhe : 2 160 Pixel
Bildseitenverhältnis : 16:9
Modus der Bildwiederholungsrate : konstant
Bildwiederholungsrate : 25,000 FPS
ColorSpace : YUV
ChromaSubsampling/String : 4:2:0
BitDepth/String : 10 bits
Bits/(Pixel*Frame) : 0.232
Stream-Größe : 414 MiB (100%)
verwendete Encoder-Bibliothek : ATEME Titan KFE 3.7.3 (4.7.3.1003)
Default : Ja
Forced : Nein
colour_range : Limited
colour_primaries : BT.2020
transfer_characteristics : PQ
matrix_coefficients : BT.2020 non-constant
MasteringDisplay_ColorPrimaries : Display P3
MasteringDisplay_Luminance : min: 0.0500 cd/m2, max: 1200 cd/m2




Sony Camping in Nature
========================

Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format_Commercial_IfAny : HDR10
Format-Profil : Main 10@L5.1@High
Codec-ID : hvc1
Codec-ID/Info : High Efficiency Video Coding
Dauer : 2 min 7s
Bitrate : 75,6 Mb/s
maximale Bitrate : 123 Mb/s
Breite : 3 840 Pixel
Höhe : 2 160 Pixel
Bildseitenverhältnis : 16:9
Modus der Bildwiederholungsrate : konstant
Bildwiederholungsrate : 59,940 (60000/1001) FPS
ColorSpace : YUV
ChromaSubsampling/String : 4:2:0 (Type 2)
BitDepth/String : 10 bits
Bits/(Pixel*Frame) : 0.152
Stream-Größe : 1,12 GiB (100%)
Kodierungs-Datum : UTC 2016-02-03 07:59:49
Tagging-Datum : UTC 2016-02-03 08:01:32
colour_range : Limited
colour_primaries : BT.2020
transfer_characteristics : PQ
matrix_coefficients : BT.2020 non-constant
MasteringDisplay_ColorPrimaries : R: x=1.000000 y=1.000000, G: x=1.000000 y=1.000000, B: x=1.000000 y=1.000000, White point: x=1.000000 y=1.000000
MasteringDisplay_Luminance : min: 0.1000 cd/m2, max: 1 cd/m2
The only way I found to convert the Sony clip with FFmpeg with very good quality was to preencode it first to the final resolution using X264 lossless. The resulting file could be converted with hable tonemapping without any issues.

So I am still clueless why the original Sony clip could not be converted with hable tonemapping. There was no error message during the conversion, it was just that the result was borked.

Any ideas?
manolito is offline   Reply With Quote
Old 1st February 2020, 01:37   #51  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,094
Try replacing npl=100 with npl=1200. No idea if I'm on the right track or not, haven't dealt with HDR much, but I have a hunch.
TheFluff is offline   Reply With Quote
Old 1st February 2020, 02:50   #52  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 3,013
I did that, the colors are different from the original npl=100, but still not usable.

Meanwhile I tested different lossless preprocessing options, and it turned out that it does not matter if the lossless file was made with a different frame size or frame rate. Even when the lossless X264 intermediate file kept all the source properties, it did convert flawlessly to SDR using the hable tonemapping.

Could it be that current FFmpeg versions have trouble decoding some HEVC source files?


//EDIT//
Almost forgot to mention that I am using 32-bit FFmpeg exclusively. Could this have anything to do with it?

Last edited by manolito; 1st February 2020 at 02:58.
manolito is offline   Reply With Quote
Old 1st February 2020, 03:17   #53  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,592
Quote:
Originally Posted by FranceBB View Post
Well, so basically you wanna go from BT2100 HDR PQ to BT2020nc SDR, is that right?
But... what for? I mean, if it's for TV compatibility, then it would make much more sense to go from PQ to HLG so that both SDR TVs and HDR ones will be able to play it without compromising the original HDR too much. If it's for FULL HD SDR TV compatibility, then they won't be able to interpret the new matrix anyway so it wouldn't make sense.
Yes, now I see it that way it makes sense.

Quote:
Originally Posted by FranceBB View Post
Anyway, I can definitely make a LUT from BT2100 PQ to BT2020 SDR and from BT2020nc HLG to BT2020 SDR.
As to how to use my matrices of linear transformation, it's pretty easy:
Sorry, I didn't mean to put you to any trouble, the question was rather theoretical, but thanks for the info. I'll check out the video and make time to have a play later.

Quote:
Originally Posted by FranceBB View Post
EDIT: I made such a matrix: Link
Did you do all that because I asked a silly question??

I'll have to come back to it, but I'll have a read of that post too.

Thanks a lot!
hello_hello is offline   Reply With Quote
Old 4th February 2020, 19:37   #54  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 3,013
Followup to this previous post:
https://forum.doom9.org/showthread.p...69#post1897869

I think I figured out why some HDR sources work and others don't when trying to apply a HDR to SDR conversion with FFmpeg. The difference is the chroma subsampling of the source. Sources where MediaInfo reports:
ChromaSubsampling/String : 4:2:0
work correctly. Other sources where:
ChromaSubsampling/String : 4:2:0 (Type 2)
is reported do not work when using the source file directly as the input for FFmpeg. This applies ony for "hable" and "reinhard" as the tonemap algorithm.

I have no idea what "Type 2" means for chroma subsampling, and I have not found any quick way to get rid of it in a HDR file. But there are 2 workarounds:

If AviSynth is installed, you can use an AVS script with just the source filter as the FFmpeg input. The filter line has to be the complex one posted by Atak (developed by dipje). The simpler line posted by Tebasuna will crash FFmpeg for an AVS input.

The second workaround requires to preconvert the source to a lossless intermediate file and use this intermediate file as the input for FFmpeg.
manolito is offline   Reply With Quote
Old 4th February 2020, 20:20   #55  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 9,883
Mani, havin' a helluva prob loading D9 webpage[or any web page] in less than 2 mins.

But see here (can also have 4:2:2 type 2).
https://forum.doom9.org/showthread.p...65890&page=216

EDIT:
And "ChromaSubsampling_Position": "Type 2":- https://github.com/mdhiggins/sickbea...tor/issues/968
Quote:
"ColorSpace": "YUV",
"ChromaSubsampling": "4:2:0",
"ChromaSubsampling_Position": "Type 2",
"BitDepth": "10",
EDIT: Type 2 ChromaSubsampling, U,V swapped over ???
ChromaSubsampling_Position, MPEG1, MPEG2 ???
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 4th February 2020 at 21:45.
StainlessS is offline   Reply With Quote
Old 4th February 2020, 23:41   #56  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Metropolitan City of Milan, Italy
Posts: 1,935
@manolito... FFmpeg can convert from classic 4:2:0 to 4:2:0 type 2 'cause it's what I use to encode with x265 after I fed ffmpeg with Avisynth.
Avisynth -> ffmpeg 4:2:0 to 4:2:0 type2 -> x265.
In other words, if FFmpeg can do it, I'm sure it can do the opposite as well.
Unfortunately I'm gonna be stuck with my mobile only and no access to my computer whatsoever otherwise I would have pasted what I use to convert from 4:2:0 to 4:2:0 type2 in FFmpeg so that you could invert it and do the opposite.


P.s if no one else replies (I doubt it) remind me to paste my bat here after February the 8th when I'll be sitting at my desk again (in front of my computer).
FranceBB is online now   Reply With Quote
Old 4th February 2020, 23:51   #57  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 9,883
Could 4:2:0 Type 2, ChromaSubsampling_Position be Progressive chroma position (matched to vertically adjacent pairs of luma scanlines, not interlaced alternate luma scanlines).
or maybe that is the ChromaSubsampling type 2 whotsit.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 5th February 2020 at 00:03.
StainlessS is offline   Reply With Quote
Old 8th February 2020, 22:53   #58  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Metropolitan City of Milan, Italy
Posts: 1,935
Quote:
Originally Posted by hello_hello View Post
Did you do all that because I asked a silly question??
Well, mostly because I realized that there are some scenarios in which it could actually be useful.

Quote:
Originally Posted by manolito View Post
MediaInfo reports:
ChromaSubsampling/String : 4:2:0
work correctly. Other sources where:
ChromaSubsampling/String : 4:2:0 (Type 2)
is reported do not work when using the source file directly as the input for FFmpeg. This applies ony for "hable" and "reinhard" as the tonemap algorithm.

I have no idea what "Type 2" means for chroma subsampling, and I have not found any quick way to get rid of it in a HDR file.

This is how I go from normal 4:2:0 to 4:2:0 type 2 and encode with x265:

Code:
ffmpeg.exe -i "avs_script.avs" -vf scale=out_color_matrix=bt2020nc:out_h_chr_pos=0:out_v_chr_pos=0 -pix_fmt yuv420p16le -strict -1 -an -f yuv4mpegpipe - | x265.exe --y4m -
You can use -vf scale in ffmpeg to do the opposite and go from type2 to normal 4:2:0 (MPEG2 position); try this for BT709 with MPEG2 standard 4:2:0 chroma position:

Code:
-vf 'scale=out_color_matrix=bt709:out_h_chr_pos=0:out_v_chr_pos=128'
FranceBB is online now   Reply With Quote
Old 9th February 2020, 02:04   #59  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 3,013
Thanks FranceBB for your advice, but I could not get it to work...

I added your scale parameters at the front of my usual FFmpeg call:
Code:
zscale=s=704x396,zscale=tin=smpte2084:min=bt2020nc:pin=bt2020:rin=tv:t=smpte2084:m=bt2020nc:p=bt2020:r=tv,zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p,colormatrix=bt709:bt601
but it did not make any difference.

Last edited by manolito; 9th February 2020 at 03:41.
manolito is offline   Reply With Quote
Old 18th February 2020, 03:37   #60  |  Link
JoyBell
Registered User
 
Join Date: Jun 2017
Posts: 18
I have been trying to work with DGHDRtoSDR my self as our family TV does not do HDR at all and MadVR does not look correct either. We have an HDR 4K Laser Projector as well, but for (swear words) can not get it to do HDR in Windows. Just for now want my content 4K and SDR please.


Code:
z_ConvertFormat(pixel_type="YUV420P16",colorspace_op="2020ncl:2020:2020:l=>2020ncl:2020:2020:l", dither_type="None")
DGHDRtoSDR(hue=8.0, tm=0.9, r=1.00, g=1.00, b=1.15, roll=0.7, gamma=0.42, fulldepth=true)
This does not look correct. I loaded up the HDR sample which for some reason looks much much better than the shows and movies after conversion.

This is a screen grab from the Windows HDR player (noticed the laptop set the panel brightness to insane when this ran).


This is what the above code gave. Much duller.


It does tell that the DGHDRtoSDR needs an adjustment, but after months, have had no luck using Tweak or anything else in Avisynth. The HDR->SDR output always looks much darker and duller than the SDR native content.

Also adding any of the listed parameters to DGHDRtoSDR() causes an script error.

DGHDRtoSDR(mode="pq",white=1800)
gives
Script Error: DGHDRtoSDR does not have a named argument "mode"


I have no idea how to get LUTs or ColorLike or other things working.

Sorry, about the image, how to make them show reasonable size syntax would be welcome.

Last edited by JoyBell; 18th February 2020 at 07:07.
JoyBell is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:13.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.