Dogway's Filters Packs

[Dogway]

I don't know what scalers SmartTVs use but I thought they were some kind of edge directed upscalers which would be similar to nnedi3. A good upscaler is one that antialiases so this is not a problem of downscaling but upscaling.

A reference "cheap" upscaler is typically lanczos4 not spline36 (although I do like bicubic "precise" also). lanczos antialiases a bit better than spline36 and 4 taps adds more sharpness at the cost of almost no additional halo because the 4th lobe is negative which in gamma encoded images is almost invisible.

I mean I didn't invent DPID, you can read the paper. All "recognized" downscalers would be pretty aliased in your terms; DPID, SSIM_downsample, "Didée", "Zopti"...

CResize discards too much information by blurring, it can be nearly replicated with RobiSoft:

bicubicresize(720,480,0.679623,0.160189)
spline36resize(1920,1080)

[Ceppo]

Thanks for your answer.

I did try your suggestion, but all of them seem to output too much aliasing on the character's edges. Then I just did CResize(720,480,aastr=1) and got a close result with the details and less aliasing on the edges.

Do you have some idea to better detect "visible" aliasing?

What I do now, is to just use a sharp resize and a soft one then I make the difference. I apply antialiasing to the difference then I add the difference to the soft one to recover details. Sangnom2 and nnedi3 seem too strong for the job, so I had to make CAntialiasing for it, but it still doesn't cut it, as you said.

[Dogway]

To detect visible aliasing use your preferred antialias.

AA = SantiagMod(strh=1,strv=1) # My favorite (nnedi3 at its core)
ex_makediff(AA) # diff centered at range_half

Your current method is what is commonly used to remove ringing, the difference of Hermite vs your kernel.

[Ceppo]

Thanks

[anton_foy]

No rush just eagered for the Zopti ootimization of SMDegrain, when is it going to be released approximately? And is this optimization going to affect modes such as temporalsoften aswell?

[Dogway]

Originally Posted by anton_foy

No rush just eagered for the Zopti ootimization of SMDegrain, when is it going to be released approximately? And is this optimization going to affect modes such as temporalsoften aswell?

I said I would start end of this week, I noticed small issues in SceneStats so I will work on it tomorrow and probably start with Zopti correlations on Monday.

[anton_foy]

Originally Posted by Dogway

I said I would start end of this week, I noticed small issues in SceneStats so I will work on it tomorrow and probably start with Zopti correlations on Monday.

Sorry I missed that okay I will await with great anticipation

[DTL]

"what scalers SmartTVs"

For cheap end-users displays it mostly probably something close to bicubic (with low kernel size). Because it is most universal and cheap and not show too much ringing at badly conditioned content. For more expensive end-user displays it may be also some non-linear and if user need best result - the downscaler need to be tuned for given display's scaler.

[kedautinh12]

I think they just use fastest and don't show too much ringing scalers in SmartTVs

[Dogway]

I think for most SmartTVs and for some years now (since 4K implementation) they include a dedicated chip for upscaling even for mid tier models (Quantum Processer 4K for Samsung, LG Tru Engine 4K for LG, etc).
LG and Samsung

[Dogway]

I finished ScenesPack to what I consider a stable version so please give it a try. I know there are some false positives/negatives scene changes, but overall it should give more matches than SCSelect, or other SC filters. There are ways to improve the SC algo but I'm done here for now (will only update as soon as pinterf adds 'average' stat to PlaneMinMaxStats() )
I will start today with the Zopti correlations by porting CoS to Python and doing the correlations there. It will take me some time though. Then I will try to draw some non-linear functions that correlate different parameters for MVTools.

[tormento]

The remastered Godfather box arrived me since a couple of days and even if they recovered the tint, that was yellowish in the last set I had, they did a terrible, horrible work with compression and perhaps some denoising.

Just look at the clusters of noise in these two images:





@Dogway: here you can find some clips of the movies.

Godfather_desk
Godfather_doorface
Godfather_greenwall
Godfather_greysuit
Godfather_hug
Godfather_phone
Godfather_picture
Godfather_shades
Godfather_table

I have tried to play a little with SMDegrain switches but I can't find consistent settings that can be applied to a whole movie recode. The worse parts are in flat areas of colors and in dark scenes.

To get rid of the noise on the desk of Godfather_desk, I would lose tiny details in the lighter scenes. I was thinking: as darker scenes suffers from higher film grain noise (perhaps higher ISO film?), wouldn't be possibile to adapt denoising strenght to the general darkness of a scene?

In the mean while, what kind of script could I use on such a difficult movie?

[Dogway]

Woow, what happened to the doorface scene! looks like they copy-pasted the face from another source without postpro of any kind.

In these old films grain typically shows when a high ISO stock is used, for example for dark scenes, so you can run per-scene stats to detect them and perform more aggressive denoising like explained in a previous post.

My recommendation for performance is to do a stats prepass with:

SceneStats("Range+Stats+StdErr",lookahead=90,path="D:\mystats.log")

Then load the stats with:

ReadStats(path="D:\mystats.log")

In any case I found SceneStats() has issues with long scenes (over 20 seconds) as it's very slow.
I was waiting for pinterf to update PlaneMinMaxStats(), but I will upload what I have now and try to check feedback to see if I can speed things up.

You can detect dark scenes with

ScriptClip(function [] () {

sts = propGetAsArray("_SceneStats")

avg = sts[4]
med = sts[5]
skw = sts[6]

dark = avg < 80 && med < 60 || skw > 0.8
} )

[Julek]

You might be interested in this:
https://github.com/dnjulek/jvsfunc/b...xpr.py#L10-L55

I did ccd using only Expr, I saw that AVS does not yet have a native plugin and I think this might help.

To help with vs-expr: https://github.com/AkarinVS/vapoursy...pr-expressions

[Dogway]

Thanks for the note. I might need to read a bit more about CCD denoise, is it some kind of debayer denoise?
Looks like it can be implemented, as for the rest we lack 'sort' operator in Expr. I'm curious to know the performance gap compared to my sorting network approach.

Can you compare:

ExtractY()
ex_median("median5")
Prefetch(6) # for 4C/8T or tune according to CPU

to

y_clip = core.std.ShufflePlanes(planes=0, colorfamily=vs.YUV)
medianblur(y_clip,2)

And also with "median7" and n=3


---------------


As for SceneStats() code wise it is correct now, but it suffers a lot with long scenes/shots (>20s). This is mainly because due to lack of a loopback, for each frame we do a lookahead and a lookback to set start and end scene frames. Ideally we should only do this once at start of scene and then copy _SceneRange props from previous frame (here's what we need a loopback/feedback for) if current frame is not a SC.

[Julek]

Originally Posted by Dogway

Thanks for the note. I might need to read a bit more about CCD denoise, is it some kind of debayer denoise?

You can find a more complete explanation here: https://github.com/DomBito/VapourSyn...main/README.md


Tests:

BlankClip(length=5000, width=1920, height=1080, pixel_type="Y16")

ex_median("median5")
Prefetch(12)

Result: https://i.imgur.com/VcISpEt.png

from jvsfunc import medianblur
import vapoursynth as vs
core = vs.core

src = core.std.BlankClip(None, 1920, 1080, vs.GRAY16, 5000)
blur = medianblur(src, 2)

blur.set_output()

Result: https://i.imgur.com/IUCU4cZ.png

BlankClip(length=5000, width=1920, height=1080, pixel_type="Y16")

ex_median("median7")
Prefetch(12)

Result: https://i.imgur.com/zab2gdQ.png

from jvsfunc import medianblur
import vapoursynth as vs
core = vs.core

src = core.std.BlankClip(None, 1920, 1080, vs.GRAY16, 5000)
blur = medianblur(src, 3)

blur.set_output()

Result: https://i.imgur.com/Tc8hpLI.png

CPU: AMD 3700X

#EDIT: now that I noticed the note about Prefetch, I will redo the tests, as I don't use AVS I'm not familiar with the tool
#EDIT2: updated tests

[Dogway]

Wow, about 2.5x the performance, could be very worth it, not sure if it makes use of internal optimizations with LLVM or something though.

[Dogway]

I updated ScenesPack to deal with issues on long scenes, now it only performs lookahead on scene start frames (not every frame as previous versions). I also optimized other code blocks and generally performs about 30% faster. There are still some issues with StdErr mode, so avoid that mode or 'ALL'.

Later I realized that the culprit for long scenes was a low memory assignment, so for good measure I recommend going no less than setmemorymax(2048*3).

I might ask you now what would you consider a "dark" frame, ideally 'dark' as synonymous of high ISO film stock. We might agree that a low scene's average and median would depict a dark frame. But "dark scenes" could also show a bright spotlight. I didn't find any paper talking about ISO or dark scene detection but would you consider a frame/scene with 90% below 8-bit 80 avg/median value and 10% <200 bright a dark frame/scene?

In other words:

dark = avg < 80 && med < 60
or
dark = avg < 80 && med < 60 && mx < 150

Now I also feel confident to finish the SAD analysis tool, it's not an unsupervised tool though nor a fast one, but can help on defining scene's grain density.


I also implemented reading '_SceneStats' on several filters like ex_retinex, softlimiter, greyworld, and made ex_autolevels. Will be uploading soon.

[anton_foy]

Originally Posted by Dogway

I updated ScenesPack to deal with issues on long scenes, now it only performs lookahead on scene start frames (not every frame as previous versions). I also optimized other code blocks and generally performs about 30% faster. There are still some issues with StdErr mode, so avoid that mode or 'ALL'.

Later I realized that the culprit for long scenes was a low memory assignment, so for good measure I recommend going no less than setmemorymax(2048*3).

I might ask you now what would you consider a "dark" frame, ideally 'dark' as synonymous of high ISO film stock. We might agree that a low scene's average and median would depict a dark frame. But "dark scenes" could also show a bright spotlight. I didn't find any paper talking about ISO or dark scene detection but would you consider a frame/scene with 90% below 8-bit 80 avg/median value and 10% <200 bright a dark frame/scene?

In other words:

dark = avg < 80 && med < 60
or
dark = avg < 80 && med < 60 && mx < 150

Now I also feel confident to finish the SAD analysis tool, it's not an unsupervised tool though nor a fast one, but can help on defining scene's grain density.


I also implemented reading '_SceneStats' on several filters like ex_retinex, softlimiter, greyworld, and made ex_autolevels. Will be uploading soon.

Now this is interesting! But did you take into consideration the red channel? With cmos red tint combined with low exposures makes for noisy footage. I mean high ISO usually gets more noise when light is on the red side. Blue light (daylight balanced LED lights for example) is less noisy. EDIT: Sorry I read wrong (tired) you said film stock and I talked about high ISO video.

[Dogway]

Thanks, yes that is an idea, probably for grain detection.
I was thinking a bit and researching, and made up a 2D matrix for evaluating scene exposure.
Raw exposure takes the scene's median and classifies it within the 5 blocks/zones (ymax/5)
Contrast does the same but takes the range (th_max - th_min) and classifies it in 5 zones as well. It acts as a bias for Raw exposure.

|  \    |        |        |        |        |        |
|   \Med| V.Dark | Dark   | Mid    | Bright |V.Bright|
|Rng \  |        |        |        |        |        |
|-------|--------|--------|--------|--------|--------|
|       |        |        |        |        |        |
| Flat  |   0    |   1    |   2    |   3    |   4    |
|       |        |        |        |        |        |
|-------|--------|--------|--------|--------|--------|
|       |        |        |        |        |        |
| Mid   |   0    |   1    |   2    |   3    |   4    |
|       |        |        |        |        |        |
|-------|--------|--------|--------|--------|--------|
|       |        |        |        |        |        |
| Cont  |   1    |   2    |   3    |   4    |   4    |
|       |        |        |        |        |        |
|-------|--------|--------|--------|--------|--------|
|       |        |        |        |        |        |
| V.Cont|   2    |   3    |   4    |   4    |   4    |
|       |        |        |        |        |        |
|-------|--------|--------|--------|--------|--------|
|       |        |        |        |        |        |
| Bimod |   3    |   4    |   4    |   4    |   4    |
|       |        |        |        |        |        |
|-------|--------|--------|--------|--------|--------|

I think I should decouple 'th' from the calculation of th_min and max and lock it at 1%. I also built a grid to show what 'th' represents in pixels in 'show=true' mode. Uploading in a moment.


EDIT: Updated to v3.2 to fix Exposure expression. Also converted divisions to multiplication for performance.
Here's a snapshot of the 'show' panel where it shows all the possible stats.
Each square of the grid represents the percentage of 'th' so you can figure out the size of discarded pixels for 'th_min' and 'th_max' (in this case 1%).
Pearson's skewness wasn't accurate for Exposure detection, as you can have negative skewness (meaning bright) in the lower portion of the histogram, and it was sensitive to outliers.