UpscaleChecker

[VoodooFX]

Upscale checking, prompted by Fake 4:4:4 (https://forum.doom9.org/showthread.php?t=184237).

Quick alpha draft for tests so far.
Ideas how to get better magic number [aka 'VFXNo'] are welcome. :)

Download: https://github.com/Purfview/UpscaleChecker

[FranceBB]

Looks like PointResize is throwing curve balls to this algorithm... (colorbars also "trick" the algorithm).

ColorBars(848, 480, pixel_type="YV24")

UpscaleChecker()

Upscale: No

ColorBars(848, 480, pixel_type="YV12")

Converttoyv24()

UpscaleChecker()

Upscale: No

ColorBars(848, 480, pixel_type="YV12")

Converttoyv24()

PointResize(1280, 720)

UpscaleChecker()

Upscale: No


LWLibavVideoSource("I:\Kyou kara Ore wa\Kyou kara Ore wa!! EP01 1080i.mxf")

UpscaleChecker()

Upscale: No

LWLibavVideoSource("I:\Kyou kara Ore wa\Kyou kara Ore wa!! EP01 1080i.mxf")

PointResize(3840, 2160)

UpscaleChecker()

Upscale: No

LWLibavVideoSource("I:\Kyou kara Ore wa\Kyou kara Ore wa!! EP01 1080i.mxf")

LanczosResize(3840, 2160)

UpscaleChecker()

Upscale: Maybe

LWLibavVideoSource("I:\Kyou kara Ore wa\Kyou kara Ore wa!! EP01 1080i.mxf")

Spline64Resize(3840, 2160)

UpscaleChecker()

Upscale: Maybe

[kolak]

444 content can be found here:

http://download.opencontent.netflix.com.s3.amazonaws.com/SolLevante/hdr10/SolLevante_HDR10_r2020_ST2084_UHD_24fps_1000nit.mov, but this is very obvious 444.

Any Netflix IMF master from ( just video MXF is enough)
http://download.opencontent.netflix.com.s3.amazonaws.com/index.html?prefix=
Eg. Sparks is not so obvious.
Not sure if avs can read IMF MXF files?
Big files.

https://www.arri.com/en/learn-help/learn-help-camera-system/camera-sample-footage

Most ProRes files will be 444 straight from camera, so good test files.

https://sonycine.com/testfootage/

Bottom of this page you have ProRes422 and XAVC-I ( those are most likely 422 as well).

You can also use RAW files from Sony website and from here
https://www.red.com/sample-r3d-files
But then you need to load them to Resolve and export to 444 or 422 codec.

[VoodooFX]

Looks like PointResize is throwing curve balls to this algorithm... (colorbars also "trick" the algorithm).
Yep, PointResize damages image too much. :D
Colorspace tests do nothing there as there is no chroma checking yet, and it checks progressive UHD <-> HD <-> SD, no middle stuff like 720p, but can be added such I think.

This doesn't fail:

ColorBars(848, 480, pixel_type="YV12")
BicubicResize(1920, 1080) # and BilinearResize(1920, 1080)
UpscaleChecker()




444 content can be found here:
Thanks, I'll check them out.

[kolak]

Getting:
"ScriptClip does not have a named argument args"

[StainlessS]

You need Grunt for Scriptclip(Args) arg.

[kolak]

Thx, worked.

[kolak]

Upscale checking, prompted by Fake 4:4:4 (https://forum.doom9.org/showthread.php?t=184237).

Quick alpha draft for tests so far.
Ideas how to get better magic number [aka 'VFXNo'] are welcome. :)

Download: https://github.com/Purfview/UpscaleChecker

First tests show that getting those thresholds may be very difficult.

[VoodooFX]

First tests show that getting those thresholds may be very difficult.

Did you meant suitability thresholds? Post image at https://imgbox.com, it doesn't degrade quality, or just upload to Wetransfer.

[kolak]

Yes, even CGI title which should be best case scenario shows very different values for different frames.
Issue may be fact it's HDR?

Analyse Sol Levante from Netflix (ProResXQ 444).

For real content I'm getting range from eg 0.3-1000+ and for fake most frame says Unsuitable (if good then it does say upscaled).

[VoodooFX]

I dunno about HDR much, and that sample is 35GB, I can download only much smaller samples.

[kolak]

50Mbit h265 422 version of it (Rec.709):
https://we.tl/t-Wd24cTAREw

[VoodooFX]

For real content I'm getting range from eg 0.3-1000+ and for fake most frame says Unsuitable (if good then it does say upscaled).

Looks like it works like intended, I think, not sure what you meant by "0.3", problem is that somewhere "Yes" <-> "No" things should overlap, it's not possible to pin point that for example below 100 it's upscale and above 100 it's not. So grey area needs to exist, probably it can be reduced by some "normalization". It's rough draft, now focused mostly on "sure things", that middle area needs to be investigated first by collecting stats from zillion frames.

I think if you see "No" on some frames then it's not upscale, if you can't find "No" then it's upscale.


Anyway, I updated it to v0.2 with chroma testing, debug and laxed some restrictions. Things tested only on 10s clip, no suitability restrictions on chroma.
UpscaleChecker( ChromaTest=0, Debug=0 )

50Mbit h265 422 version of it (Rec.709):
https://we.tl/t-Wd24cTAREw

I can download less than 1GB in one file, can you split it with rar?

[kolak]

0.3 as low value of VFXNo. It can go lower as well.
Grey area seems to be very wide :)
I think better approach would be to look at spectrum of the difference. 444 is "sharp" and more gradual. 422 is coarser, specially horizontally, with makes sense. Measuring how much of the difference is there is not the best approach (not sure of this is what your script does). I'm just not smart enough to help more :)

https://we.tl/t-4eK6al21uL

[VoodooFX]

0.3 as low value of VFXNo. It can go lower as well.
And that "0.3" frame you selected as good "detailed" frame to test?
Can you tell that that frame is not upscale by visual inspection?

EDIT:
Think this way: Not-upscaled video can have "blurry" frames that looks like upscale, upscaled video can't have "detailed" frames that looks like real res.
Broad grey area is there so people wouldn't jump to the wrong conclusions by looking at some crappy frames to test, I think, if there is better formula I'm all ears.
And I didn't tested much of this in practice, so I don't want to state Yes/No when actually I dunno where those thresholds are, if someone knows I'm happy to hear about them. :)

EDIT2:
Btw, both chroma channels are tested separately, and channel with highest 'VFXNo' is selected for stats.



4:2:2 test:
v=LWLibAvVideoSource("D:\A003C025_150830_R0D0.mov").ConvertTo8bit
u = v.ConvertToYV12().ConvertToYV16()
o = v.UpscaleChecker(ChromaTest=1, Debug=1).crop(0,0,420,100)
u = u.UpscaleChecker(ChromaTest=1, Debug=1).crop(0,0,420,100)
stackvertical(o,u.AddBorders(0,2,0,0,$F0F080))

https://i.imgur.com/4Cu0lOi.jpeg

ColorBars:
v=ColorBars(pixel_type="YV16")
u = v.ConvertToYV12().ConvertToYV16()
https://i.imgur.com/d3pP0aW.jpeg

v=ColorBarsHD(pixel_type="YV24")
u = v.ConvertToYV16().ConvertToYV24()
https://i.imgur.com/v2wUbRn.jpeg

v=ColorBars(pixel_type="YV24")
https://i.imgur.com/aCsyPFd.jpeg

[kolak]

Hmmm- overall getting many unsuitable frames. Hard to find those which do work.
Netflix CGI title is mix of Yes/No.
Title which I suspected to be 422 is actually all Yes (at least those frames which are suitable).

[VoodooFX]

Hmmm- overall getting many unsuitable frames. Hard to find those which do work.
Netflix CGI title is mix of Yes/No.
Expected, as there is no suitability checks for chroma, and whole chroma thing is semi-randomly set by looking at "A003C025_150830_R0D0.mov" clip [which practically is no better than one frame...].

I can add a second debug option that would parse whole clip and would write stats to txt log, then some thresholds could be derived from there [and log can be easily shared too].


Title which I suspected to be 422 is actually all Yes (at least those frames which are suitable).
At least this one works very well. :)

[kolak]

Yes, that suspected file showing only Yes is a good sign.
In the same time it's quicker to actually look and judge by eye :)
Script is terribly slow.

[FranceBB]

Ok, so, I grabbed version 0.2 and I started testing assets that the good old algia (Livio, now retired) collected for testing over the years.

Test Case 1:
Arri 4K LogC HDR 4:4:4 25p 12bit (Apple ProRes XQ 2.1 Gbit/s)

https://i.imgur.com/osM3kb4.png

It says "maybe", however this is definitely not an upscale.

Test Case 2:
Canon C300 FULL HD CLog3 HDR 4:2:2 25p 10bit Full PC Range (H.264 Intra Class 150 Mbit/s)

https://i.imgur.com/YhdWZRe.png

Any frame results in "Unsuitable frame" across the whole take. I tried with another take, but it keeps saying "Unsuitable frame". Perhaps it doesn't like Clog3 as transfer? Dunno.

https://i.imgur.com/1ASPyGH.png - https://i.imgur.com/XIVGHGW.png

Test Case 3:
Sony A7 SIII FULL HD Slog3 HDR 4:2:2 25p 10bit Full PC Range (H.264 Intra Class 100 Mbit/s)

https://i.imgur.com/D510Ls3.png

Yay, finally I've got a "No"! :D


And just when I had hope, test case 4 blew my results... :(

Test Case 4:
Z Cam FULL HD ZLog2 HDR 4:2:2 25p 10bit (Apple ProRes 422 116 Mbit/s)

https://i.imgur.com/QafcyrW.png
https://i.imgur.com/Jp5Kqvw.png

Upscale: Yes... followed by some "Unsuitable frame" :(

https://i.imgur.com/HmpCuZM.png

At that point I got suspicious and I thought: "Uhmm, maybe it just doesn't like HDR and totally logarithmic curves, so no PQ, no Slog3, no Clog3, no ZLog and ZLog2, no LogC etc etc etc. What I've done is to apply a LUT to get to BT709 SDR 100 nits et voilà, UpscaleChecker() suddenly worked like it should:

https://i.imgur.com/rv0WjEP.png
https://i.imgur.com/tljB7MH.png

So I re-did all the other tests and just like magic...

https://i.imgur.com/Ofq9URS.png
https://i.imgur.com/a1Tf9hB.png



https://i.imgur.com/hJIenFM.png - https://i.imgur.com/i7aKUUP.png


So... does it work? Yes.
Is it limited to SDR? Yes.

So the results are very promising as long as you stick to SDR sources.

[tormento]

As reference, you can use getnative (https://github.com/Infiziert90/getnative).

[kolak]

I've also noticed it doesn't like LOG or PQ/HLG curves.
I had idea to actually boost saturation a lot so it's easier for script to work, but once you overdo it then everything is no upscale :)

Problem is as VoodooFX said- you cant just have Yes/No.
Real 444 can show Yes, but 422 should never show No.
Problem is that you have to test many frames (or all), so it's time consuming.

[VoodooFX]

Yep, no HDR testing, probably Anime is not good too.

Problem is that you have to test many frames
Check v0.03.

[kolak]

Will check.

[mastrboy]

Yep, no HDR testing, probably Anime is not good too.


For anime, just use Getnative: https://github.com/Infiziert90/getnative (Though it uses Vapoursynth rather then Avisynth.)