Using smoothlevels on NTSCJ Japanese DVD

[Logan9778]

Hi guys.

I'm trying to experiment with levels, but I don't know if I'm doing it right. I'm trying to get a faded looking NTSCJ Japanese encode to show up right on my lcd monitor. I think it's outputing 16-235 luma, so I'm using smoothlevels to change it to 0-255. Looking at the built in histogram in AvsPmod, dark blacks now reach the 16 limit line, where before they were showing around max 25 to 30. But the problem is, they now cross over it at points, looking like they hit around 12 over even 10. I know Ghitulescu said that PAL and NTSCJ can have darker blacks than the 16 limit, so I guess my question is, how should I deal with the blacks that go beyond 16?

In PAL it is possible to have a blacker black as the pedestal is not required (so for NTSC in Japan).


Do PAL and NTSCJ use some other luma scale?

This is the smoothlevels code I'm using.

smoothlevels(Output_low=0, Output_high=255)

Thanks!

Here are pics of my AvsPmod with and without smoothlevels

No smoothlevels

https://picload.org/image/doodcddr/reglevels.png

Smoothlevels

https://picload.org/image/doodcddl/smooth.png

Blacks going over the 16 limit line

https://picload.org/image/doodcdlr/over.png

[Asmodian]

It is fine to have black go below 16 after your range adjustment. You might also do a bit less of a stretch, maybe outputlow=4 or something. The error is most likely not a true levels issue because it is far away from normal limited range. It seems like it is simply low dynamic range with a high black level; use as much stretch to get it corrected as you can without going over very often.

[Emulgator]

In the end you got to know what your playback chain is gonna do with Y<16.
Make a DVD with flickering test bars and watch the result.
For most of DVD players you want to touch Y=16 only for pitch black, this shall yield R=G=B=0.
Y of 12 or 10 would yield the same black, so you would have effectively crushed your shadows.
If such movie exhibits different blacks per scene, and it doesn't look like artistic intention then I would consider that a grading fault.
I suggest to go through by scene and consequently adjust shadow values by scene.
To see what the called function is actually doing it can be recommended to keep debug view open.
SmoothCurve is often my friend here.

[Logan9778]

Thanks guys!

In the end, I used

SmoothLevels(input_low=24, input_high=229, output_low=16, output_high=235)

I realized I should still output 16-235 for YUV, and that the luma had been over-compressed ( I was getting 24 for solid black in ColorYUV). At least I think this is what is happening. I wonder if the 229 should have been 227, since pitch black was off by 8. 235-227=8. But it doesn't seem to make any huge difference.

I had been using AvsPmod's %YUV luma showing in the bar, and for some reason it showed blacks too high, and whites too low. I went with ColorYUV() and used the loose min/max values and it seems to be correct.

After using the Smoothlevels() code,
Avisynth histogram showed pitch black as 16 ( the first black frame of the video) while the brightest scenes hit 255 max in the histogram. So, I guess it's good now?

Should I do anything with the colors since I stretched the luma?

[Asmodian]

Great solution, much better than my suggestion. Best to use 16-235 for the output.

You could check the loose min/max values for the chroma channels as well, remember to use 16-240 for them. :)

Edit: This probably won't work at all, the video is very unlikely to have 100% values for min/max U or V. I am not sure what should be done.. thinking in U and V is hard. Maybe do the range fix in RGB? :p

[Logan9778]

Thanks Asmodian!

Yeah, I might try converting it all to RGB later and experiment there. A LOT more learning to go!

EDIT: Well, in the end, I found out that there was no mistaken double compression of the levels when they compressed the levels to YUV.

But it was interesting learning about levels, and personally, I think they are a little compressed ( probably for an artistic reason to fade them a little) and that it is possible to stretch or change them to what you think they should be.

[foxyshadis]

Probably less for an artistic reason than that someone who didn't know what they were doing did another 0-255->16-235 compression after it had already been done. Very common in low-budget stuff.

Digital NTSCJ can indeed be 0-235, as an incorrect interpretation of the NTSCJ 0-714mV in Japan, generally by capture cards only developed for the American market. The correct way to handle it, and any other level weirdness, would be to scale it to 16-235 (or better yet, 64-960), where you use your best judgment to decide where the zero-level lies. BTB isn't uncommon due to analog and DCT noise, but black is still black; just scale and threshold it wherever it looks like it should be.

[Logan9778]

Thanks Foxy.

64-960?

It doesn't seem to be double compression, but I think someone did screw up along the line somewhere. Looks like the first frame is just a black frame, yet it shows to be 23 in the 16-235 scale.

[foxyshadis]

Thanks Foxy.

64-960?

It doesn't seem to be double compression, but I think someone did screw up along the line somewhere. Looks like the first frame is just a black frame, yet it shows to be 23 in the 16-235 scale.

64-960 is the same scale in 10-bit. Makes a lot of sense to process and encode anime especially in high bit depth, to prevent the broad banding you often get in flat areas.

[Logan9778]

Oh Thanks! Something I will remember! I was wondering why I always see old anime in 10-bit.