Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
3rd June 2024, 23:33 | #61 | Link |
Registered User
Join Date: Apr 2018
Posts: 51
|
Anyone knows something about OETF and EOTF functions?
The interesting thing here for me is: From their description in docs I'v found that OETF is used to encode original light and EOTF is reverse. So first I thought that if I use EOTF on data I will get "original lightness of the scene". But EOFT function makes everything MUCH darker. And OETF makes brighter. It can't be that original scene lightness was so dark. Or it is just a misunderstanding because of "light" differs from "electric signal" - so this functions are not designed to be used such way (maybe they are designed to be used only inside camera and inside display hardware). But then it is strange that OETF function gives great results on some content. For example Divergent 2 and 3 blurays processed by it (Rec.709 limited -> full range -> OETF) results it bright picture but with good contrast. While I'v tested on other BDs and result are just bright picture with lost contrast (its possible to get better contrast if limited -> full conversion is used twice before OETF, but this burns a lot of darks). So it looks like for some content there should be another OETF function. But same OETF is specified for Rec 601 and Rec 709. |
4th June 2024, 08:23 | #62 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,207
|
"But EOFT function makes everything MUCH darker. And OETF makes brighter. It can't be that original scene lightness was so dark."
Your display device doing EOTF for you (and it is at current PCs typically rec.709/sRGB EOTF). So if you apply EOTF to data in the distribution domain and feed to standard display - you got EOTF applied twice and darker displaying. To check linear light values you need to understand how digital values maps to physical light. And inspect digital values directly (or switch your E->O display device to linear EOTF if possible). " looks like for some content there should be another OETF function." Most of cine-titles (not direct live broadcasts) are mastered not with real scene light but after complex colour-grading process for artistic reasons and by choice of product director. So usase of standard EOTF and standard display only guarantees you will see same optical displaying of the content as was expected at production. For live broadcasts depending on camera settings you can get real OETF only tracking and can restore linear light using EOTF transform. But real OETF tracking for physical production camera rarely usable (only in some 100% controlled studio lighting) so for real ENG and other open air shooting cases some (unknown) non-linearity added like KNEE (and AUTO KNEE) control to soft-compress highlights and this data not sent as metadata with encoded content so you can not restore full linear light with EOTF only. Only for special closed digital imaging like medical highly linear path may be used from camera to display so doctor can see as much real image as possible. Most of other digital imaging are somehow distorted at production to have 'more nice look'. Same and much more changes applied to colours view. You can take some digital camera with RAW output and look how awful is it looks in both colour and tone in most of real life scenes. Last edited by DTL; 4th June 2024 at 08:39. |
16th June 2024, 00:31 | #63 | Link |
Registered User
Join Date: Apr 2018
Posts: 51
|
I'v tested my beelink android TV box and MPC player again. This time with my own specially created files.
Results: TV box does not care about HEVC flag - full or limited color space. If video is in full space but flagged limited - it will show it full anyway. But looks like it cares for range itself. So I'v created image with half RGB(16,16,16) and half RGB(235,235,235). And then it shows it as black and white (really white not some gray nearly white). So looks like TV box can fail in limited color space determinations if first frames will contain values < 16. MPC player: * MPC Video Renderer cares about full or limited color space flag and displays correctly. * MadVR is same. (But with decoding AV1 its faster and does not freeze). So I better use it. * EVR Renderer does not care about it. It always rerange YUV from limited to full. If video is already full - it will rerange it again. Same in Staxrip (avisynth) - if source is YUV - preview will always be reranged to full. Last edited by Argaricolm; 16th June 2024 at 00:47. |
16th June 2024, 01:14 | #64 | Link |
Registered User
Join Date: Apr 2018
Posts: 51
|
What bothers me now is that why twice reranged video to full looks more like it should be so:
Limited -> Full Full -> Twice Full |
16th June 2024, 10:44 | #65 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,207
|
Content mastered at master control monitor. See EBU Tech 3320 and 3325 for requirements and testing as example.
https://tech.ebu.ch/docs/tech/tech3320.pdf https://tech.ebu.ch/docs/tech/tech3325.pdf These documents also list EOTF and test methods and requirements for different tiers of quality. To see how it was mastered you need to simulate master control monitor and viewing environment (different for SDR and HDR too). Also you can tweak as you want with local display controls - it will be your local version of image. |
17th June 2024, 02:46 | #66 | Link |
Registered User
Join Date: Apr 2018
Posts: 51
|
New release v1.16.
Now you can easily get limited range back like so: softlight(8) = full range softlight(3) = average softlight(9) = back to limited You can change default range parameters for limited range like so: Softlight(8, rangemin = 16, rangemax = 235) Its needed on some sources to play with OETF function (it needs black to be absolute - or contrast will be lost). Also added softlight(7) mode. Some video sources with limited range contain values < 16 and > 235. This mode will change them to 16 and 235. This will ensure correct range identification by your decoding hardware (seems it looks at first frame for values outside limited range to identify video as full range). Plus it will allow to compress your video better. |
18th June 2024, 17:47 | #68 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,207
|
It looks like you do not understand why users need some levels and colour correction plugins. The studio mastered titles are always perfectly mastered for colour and tone so do not need any corrections. You can read also the article from Ken Rockwell how studio shootings always get perfect colour and tone - https://www.kenrockwell.com/tech/highlight-shadow.htm
But the job of studio colour grading costs a lot. On some days (like 19xx..201x) there were home end users video unexperienced shooting with bad natural lighting on cheap hardware and bad auto-exposure and bad auto-colour etc. This result in bad colour and tone in most cases. So users typically want to make old home and amateur recordings better using some free tools. Thus to show how the tool can help end users with badly recorded home video it is better to use the real samples - not perfectly mastered studio titles. Some real content may be downloaded from forum posts at videohelp forum where users show some bad footages for repair. Last edited by DTL; 18th June 2024 at 17:52. |
18th June 2024, 18:21 | #69 | Link | |
Registered User
Join Date: Oct 2001
Location: Germany
Posts: 7,468
|
I'm a bit confused what mode 8 (tv2pc) does.
Quote:
Code:
clip = core.std.Levels(clip=clip, min_in=16, max_in=235, min_out=0, max_out=255) but: Code:
clip = core.std.Levels(clip=clip, min_in=16, max_in=235, min_out=0, max_out=255) clip = core.Argaricolm.Softlight(clip, mode=11, yuvin=601, yuvout=601) while: Code:
clip = core.Argaricolm.Softlight(clip, mode=8, yuvin=601, yuvout=601) clip = core.Argaricolm.Softlight(clip, mode=11, yuvin=601, yuvout=601) I also looked at: Code:
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full") clip = core.Argaricolm.Softlight(clip, mode=11, yuvin=601, yuvout=601) => any inside on this? What does tv2pc do? Cu Selur Last edited by Selur; 18th June 2024 at 18:37. |
|
21st June 2024, 04:43 | #70 | Link |
Registered User
Join Date: Apr 2018
Posts: 51
|
TV2PC changes range this way for 8 bit (in RGB for each pixel):
(R/G/B - 16) / 219 * 255 if input is yuv - it is converted to rgb for processing (without changing range) As for core.std.Levels maybe you need to rerange channels in yuv differently (from vp docs): clip = std.Levels(clip, min_in=16, max_in=235, min_out=0, max_out=255, planes=0) clip = std.Levels(clip, min_in=16, max_in=240, min_out=0, max_out=255, planes=[1,2]) As for examples. I see difference here too. AviSynth and VapourSynth give different results. Btw you should rerange your result back to limited range using softlight(9) Otherwise you will have full range in yuv. If you preview it in staxrip (and maybe other software) you will see it twice reranged to full (for me preview is always reranged from limited to full - if I already have full - in preview I see it even more dark). To not have such issue you should try to work in RGB space. As for different result in VapourSynth looks like a bug in VapourSynth (thou its strange). Here what I'v found. Using this script: import vapoursynth as vs from vapoursynth import core core.std.LoadPlugin(path='c:\\..\\softlight.dll') clip = core.std.BlankClip(width=640,height=480, format=vs.YUV420P8, length=500, fpsnum=2997, fpsden=125, color=[16, 128, 128]) clip = core.Argaricolm.Softlight(clip, mode=8) clip = core.Argaricolm.Softlight(clip, mode=11) clip.set_output() When I open in VirtualDub - mode 8 is never executed. Y = 16 is passed to mode 11 resulting in 55 Then 55 is passed again to mode 11 resulting 115 So it does mode 11 twice without doing mode 8. I don't see how it can be from my code. If only mode 8 is in script - it is executed correctly resulting 0 in Y. If I do mode 8 then 11 then 9 - only mode 9 is executed 3 times. I'll try to update VapourSynth (have R65). Last edited by Argaricolm; 21st June 2024 at 06:40. |
21st June 2024, 14:13 | #71 | Link | ||
Registered User
Join Date: Oct 2001
Location: Germany
Posts: 7,468
|
Quote:
Quote:
|
||
21st June 2024, 20:37 | #72 | Link |
Registered User
Join Date: Apr 2018
Posts: 51
|
Fixed it.
v1.17 The problem was in different parameters passing in VapourSynth than in AviSynth. AviSynth uses different objects for each call - so there I use object variables to store parameters. But VapourSynth uses same object for different calls. It first calls filterCreate for each call (in same object). So last call replaced previous parameters. And after that it starts to call filterGetFrame. So I made it to pass parameters for each vsapi->createVideoFilter call. Looks like it is not a bug. Strange, that filter skeleton does not have parameters passing example. |
21st June 2024, 20:53 | #73 | Link | |
Registered User
Join Date: Apr 2018
Posts: 51
|
Quote:
What I don't understand is that why nearly all latest titles are so dark. Same goes for old titles too. I don't think the problem was with the lack of light. |
|
21st June 2024, 21:24 | #74 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,207
|
To display dark scenes - the lower code values of standard 8bit encoding used. The sRGB and rec.709 dynamic range with 8bit encoding is about 1000:1 (as ratio from first non-zero code value of 17 to nominal white of code value 235)so for daylight scenes higher part used and for night scenes - lower part. The total range is fixed in relative lighting (not as possilble with Dolby HDR with dymanic metadata ?) so it is easy enough to show dark scenes as dark and bright as bright. But this only work best in required viewing conditions - see viewing conditions for HDTV ITU-R BT.2022 https://www.itu.int/dms_pubrec/itu-r...8-W!!PDF-E.pdf and ITU-R BT.2035 https://www.itu.int/dms_pubrec/itu-r...7-I!!PDF-E.pdf and also check your display gamma-tracking for rec.709 EOTF first and also display nominal white level (or peak luminance) if you do not see dark scenes correctly.
Typically for end-users HDTV and rec.709 - (BT.2022) 1.1 General viewing conditions for subjective assessments in a laboratory environment The assessors’ viewing conditions should be arranged as follows: a) Room illumination: low b) Chromaticity of background: D65 c) Peak luminance1: 70-250 cd/m2 (See § 1.7.2) d) Monitor contrast ratio: < 0.02 (See § 1.7.1) e) Ratio of luminance of background behind picture monitor to peak luminance of picture: ~ 0.15 1.2 General viewing conditions for subjective assessments in a home environment a) Environmental illuminance on the screen (incident light from the environment falling on the screen, should be measured perpendicularly to the screen): 200 lux b) Peak luminance1 : 70-500 cd/m2 (See § 1.7.2) c) Ratio of luminance of inactive screen to peak luminance monitor contrast ratio: < 0.02 (See § 1.7.1) (BT.2035) 1.1 Viewing environment for subjective assessment a) Room illumination: 10 Lux b) Chromaticity of background: D65 (optionally D93 in some regions) c) Ratio of luminance of background behind picture monitor to peak luminance of picture: ≈ Between 10% ±2% of reference white value 10 Lux of room illumination is really very dim lighting (indoor and after sunset typically in room with windows). "I don't think the problem was with the lack of light." It may be with too many light in your viewing environment and too low display contrast (or too low display peak brightness too). Too much external lighting shifts viewer's brightness of adaptation to high levels and viewer lost ability to see low brightness levels (if even display tracks EOTF completely perfect). Last edited by DTL; 21st June 2024 at 21:31. |
9th July 2024, 16:12 | #76 | Link |
Registered User
Join Date: Apr 2018
Posts: 51
|
Another question.
Does anyone know why some sources have incorrect range? For example this: Solo A Star Wars Story. This is a frame from bluray. As you can see on the frame with a lot of blacks the lowerst value is 22. But should be 16 (at least some pixels should be absolute black). For comparison same frame reranged: softlight(8,rangemin=22,rangemax=235) softlight(9) Screenshots are from YUV (so you see both of them as they will appear on TV). And this is a "new" movie. So this is not a result of something wrong. By unknown reason this range was intended. I'v already seen some of such sources where lowerst range limit is ~22 instead of 16. Example By the way. If you want normal blacks after softlight(1 or 3 or 11) you should rerange such sources this way. |
9th July 2024, 18:50 | #77 | Link | |
Big Bit Savings Now !
Join Date: Feb 2007
Location: close to the wall
Posts: 1,705
|
Quote:
Watched on a OLED it makes sense, watched on a TFT you may want to lower that black because TFT blacks are poor to begin with. But tying that to 16 for all audiences means loss of shadow detail, and crushed blacks on any slighty misadjusted monitor. Just had such a source where this was attempted, the real blacks were sitting around 8. Was hard to reverse without banding.
__________________
"To bypass shortcuts and find suffering...is called QUALity" (Die toten Augen von Friedrichshain) "Data reduction ? Yep, Sir. We're that issue working on. Synce invntoin uf lingöage..." |
|
2nd September 2024, 19:44 | #78 | Link |
Registered User
Join Date: Apr 2018
Posts: 51
|
I'v found that some sources are twice converted to limited range (full -> limited -> limited).
To determine such issue you should check first black frames of video for levels (can do so using ShowChannels plugin). Normally first black frame should be (Y = 16, U = 128, V = 128). It may differ a little due to compression. And normally each video should contain such totally black frame or frames at the begining. This signals hardware that video is in limited range. When your hardware decodes video and displays - it always converts limited range to full before displaying. But in case of twise converted range - hardware converts twise limited range to limited range and displays limited range to you. For example is the first season of Rick & Morty that has Y = 30 in first key frames. I have not checked all seasons, but 6 & 7 do not have such issue. Last edited by Argaricolm; 4th September 2024 at 16:42. |
28th October 2024, 21:08 | #79 | Link |
Registered User
Join Date: Apr 2018
Posts: 51
|
New version 1.19.
Critical fix in TV2PC 10 bit color range conversion function. It was working wrong. Added fullrange option. By default functions will treat source as limited color range. It will be converted inside. Only OETF & EOTF functions are same. So now you can do just: softlight(3) instead of: softlight(8) softlight(3) softlight(9) If you want to do same as before: softlight(8) softlight(3,fullrange=1) softlight(9) This is so only for YUV color space, because it is treated as limited color range by default. I'm thinking to make same changes for RGB color space (add ability to change color space inside call). But by default it is treated as full range. Maybe I should do it reverse way (do not change it for RGB by default, but change it if fullrange = 1). Last edited by Argaricolm; 29th October 2024 at 17:19. |
1st November 2024, 17:03 | #80 | Link |
Registered User
Join Date: Oct 2001
Location: Germany
Posts: 7,468
|
My 2 cents about this:
when:
|
Thread Tools | Search this Thread |
Display Modes | |
|
|