Need help on a fancy avisynth script

[Dark Shikari]

Right now, this is a proof of concept--ideally if this works properly I could make it into a real filter. Here's what needs to be done.
//note this could work for any size block--I'm actually thinking about 8x8 blocks at the moment.
for macroblock in frame
{
B = macroblock
A = previousFrame.macroblock(macroblock.x,macroblock.y)
if( (A-B)^4 < threshold)
{
B = A + (average(B) - average(A))
}
}

So far I have this atrociously ugly script:
w = clip.width
h = clip.height
prevclip = clip.Trim(0,-1).DuplicateFrame(0,0,0,0,0,0,0,0,0)
result = mt_lutxy(clip, prevclip, "x y - x y - * x y - * x y - * 256 /")
threshold_mask = mt_luts(result, result, mode = "avg", pixels = mt_square(8), expr = "y").pointresize(w/8,h/8).pointresize(w,h).Levels(0, 15.0, 255, 0, 255).Tweak(cont=2.0).Greyscale()
mt_merge(prevclip, clip, threshold_mask)
This has a few problems:

1. It doesn't implement the shifting-by-difference-of-averages which is very important to avoid visual blocking.

2. Most importantly, it doesn't actually work; the "prevclip", which is supposed to be a reference to the previous frame, is currently just a bunch of duplicates of the first frame, for testing. The thing is, let's say I run the function on frame 6 and it modifies frame 6. Then, when it runs on frame 7, I need it to use the modified frame 6, not the original, in its calculations. How do I do this?

Advice from the experts welcomed :cool:

[Didée]

Maybe not 100% what you want, but it's a start:

clip = last
th = 128
THR = string(th)

global w = clip.width()
global h = clip.height()
global prevclip = clip.selectevery(1,-1)
global clip_avg = clip.dctfilterD(14).pointresize(w/8,h/8).pointresize(w,h)

ScriptClip("""
diff = mt_lutxy(clip, prevclip, "x y - x y - * x y - * x y - * "+THR+" > 0 255 ?")
prevclip_avg = prevclip.dctfilterD(14).pointresize(w/8,h/8).pointresize(w,h)
global prevclip = mt_merge(clip, prevclip.mt_adddiff(mt_makediff(clip_avg,prevclip_avg)), diff, U=2,V=2)
prevclip
""")


Notes:

DCTFilterD(14) is a very fast way to average 8x8 blocks. Alas DCTFilter has a small bug, that's why the additional 2xPointResize is there.

With having mt_lutxy working inside the conditional environment, things become unnecessary slow: a fresh 256*256 LUT is created for every single frame.

Better to write a plugin with optimized math. :)

[Dark Shikari]

Your version goes very fast and doesn't seem to work at all... if I return just the diff it shows an almost completely white mask with a few dots of black early on. Note that I need the mask to be dealing with blocks, not individual pixels.

Also, what does THR do?

[Didée]

THR is, like you suggested, the threshold in
if( (A-B)^4 < threshold).

Could you be a bit more verbose on what you want to achieve at all? Your initial description is not perfectly clear ... to me, at least.

For one, (A-B)^4 < threshold) is a block difference. This can be defined in several ways, there is not one fixed one. You know from x264's motion search. ;)

Then, B = A + (average(B) - average(A)) will completely discard any high-frequency information of block B, if there should be any.


Are you after stability in dark/uniform areas?

[Dark Shikari]

What I'm doing is trying to retain grain spatially, but not temporally. That is, I want the grain to be constant over time in areas that aren't moving. This would allow the grain to be kept far more easily by the encoder.

I intentionally want to ignore all data in block B with the exception of the average color.

[Didée]

Hmmh, it's not my day today, or something ... now I tried it like this:

th = 0
THR = string(th)

global clip = last
global w = clip.width()
global h = clip.height()
global prevclip = clip.selectevery(1,-1)
global clip_avg = clip.dctfilterD(14).pointresize(w/8,h/8).pointresize(w,h)

ScriptClip("""
diff = mt_lutxy(clip, prevclip, "x y - x y - * x y - * x y - * 1 /")
\ .dctfilterD(14).pointresize(w/8,h/8).mt_lut("x "+THR+" > 0 255 ?").pointresize(w,h)
prevclip_avg = prevclip.dctfilterD(14).pointresize(w/8,h/8).pointresize(w,h)
global prevclip = mt_merge(clip, prevclip.mt_adddiff(mt_makediff(clip_avg,prevclip_avg)), diff, U=2,V=2)
prevclip
""")


Basically that seems correct ... only problem is that it doesn't work correct, either.:rolleyes: The diff mask is reasonable on the very first frame, but cumulates to 255 after the 2nd frame in progress.
Perhaps someone is seeing where it goes wrong ? I don't.

However it might be, I have some doubts about your idea. Grain usually is temporally random, how do you expect to turn it into some pure-spatial / temporally static? (I had this as a side-effect in some heavy denoising scripts, and didn't like the effect.)
Also, some noticeable switching-effect is to expect, when in a low motion scene you're below the threshold most of the time, but every now&then the threshold is exceeded...

[Dark Shikari]

Also, some noticeable switching-effect is to expect, when in a low motion scene you're below the threshold most of the time, but every now&then the threshold is exceeded...The point is that its a sort of psy optimization. I got it working with my old script on a couple frames and it seemed good. Here's why I think it'll work.

1. If something moves, you won't notice the change in grain pattern on it anyways.

2. You need extremely high bitrates to accurately keep framewide grain at a high quality. Since such bitrates are usually not used, one is either forced to settle for removing the grain, or converting it to ugly DCT noise. IMO, static grain is better than both--it avoids the banding caused by removing the grain, and also focuses the eye on the actual scene rather than the background. A really grainy encode looks quite ugly IMO and basically forces the encoder to cover the frame with useless intra blocks.

The switching effect may cause an issue, but it should be possible to solve IMO.

[Didée]

Solved the error in my script. The following works as it should. However, there seems to be a memory problem. Probably a mem leak, perhaps mt_lutxy, I can't tell for now.

global clip = last
global w = clip.width()
global h = clip.height()
global modclip = clip
global clip_avg = clip.dctfilterD(14).pointresize(w/8,h/8).pointresize(w,h)

ScriptClip("""
prevclip = modclip.selectevery(1,-1)
diff = mt_lutxy(clip, prevclip, "x y - x y - * x y - * x y - * 16 /")
\ .dctfilterD(14).pointresize(w/8,h/8).mt_lut("x "+THR+" > 0 255 ?").pointresize(w,h)
prevclip_avg = prevclip.dctfilterD(14).pointresize(w/8,h/8).pointresize(w,h)
global modclip = mt_merge(clip, prevclip.mt_adddiff(mt_makediff(clip_avg,prevclip_avg)), diff, U=2,V=2)
modclip
""")

[Dark Shikari]

I managed to implement my script in x264 as a preprocessing filter. Through a whole ton of modification and tweaking, I got it to be effective--it lowers bitrate by roughly 15-25% at the same QP for highly grainy sources with minimal visual lossiness.

Now using x264 as a filter framework is certainly one of the most terrible things I've ever done. :p However, I'm somewhat daunted by the Avisynth sample filter, since I don't program in C++ at all.... advice?

[Leak]

advice?
Learn C++

[Didée]

Well, I can't judge how similar or different your implementation is to my script ... but assuming that the script basically does what it should (it seems so to me), I can't quite agree with your enthusiasm.

I used the script on this very grainy DVB-S capture: sample#1 (http://home.arcor.de/dhanselmann/_samples/Alien2_scene1_source.m2v), sample#2 (http://home.arcor.de/dhanselmann/_samples/Alien2_scene2_source.m2v), and came to the expected conclusion:
As soon as the threshold gets high enough to provide the wished "freezing" effect to the background grain, the overall perceptual quality dropped by a lot.
Also, it looks a bit strange when some single-frame defects (e.g. small dirt speckles) get freezed ... they better should not, obviously.

The main problem is the same as mentioned several times in the past: one can't reliably measure "noise" with such simple methods. Here: the single fact that the actual block difference is below an arbitrary threshold won't guarantee that there is only noise/grain in that block. It still might true detail.

[Dark Shikari]

Well, I can't judge how similar or different your implementation is to my script ... but assuming that the script basically does what it should (it seems so to me), I can't quite agree with your enthusiasm.

I used the script on this very grainy DVB-S capture: sample#1 (http://home.arcor.de/dhanselmann/_samples/Alien2_scene1_source.m2v), sample#2 (http://home.arcor.de/dhanselmann/_samples/Alien2_scene2_source.m2v), and came to the expected conclusion:
As soon as the threshold gets high enough to provide the wished "freezing" effect to the background grain, the overall perceptual quality dropped by a lot.
Also, it looks a bit strange when some single-frame defects (e.g. small dirt speckles) get freezed ... they better should not, obviously.

The main problem is the same as mentioned several times in the past: one can't reliably measure "noise" with such simple methods. Here: the single fact that the actual block difference is below an arbitrary threshold won't guarantee that there is only noise/grain in that block. It still might true detail.I have changed the script a lot. Mostly tuning, the basic concept is similar, but I did a whole lot of weighting, heuristic crap, etc. I also changed the block size to 4x4.

My main trick is that for each individual block of the frame, I set a maximum "fluctuation length"--after X number of frames of being consecutively run on that block, the algorithm is not run on that block for one frame, to reset the block. What this does is cause blocks to get out of sync with each other, making the video still appear grainy while having far less actual grain.

The one problem with my current version of the algorithm is that while it works well for grainy sources, if the source is not sufficiently grainy, it has a negative effect.

I agree that the original script was pretty crap--but the new version is better :)

[Didée]

Well, okay. It starts getting too hypothetical (on my side) to argue any further. We'll see.

[Dark Shikari]

Well, okay. It starts getting too hypothetical (on my side) to argue any further. We'll see.
Sample results: linkage (http://www.mediafire.com/?bdyaymnyu23). They're at a visually lossless CRF (10) so the only real noticable difference is one has the grain optimization and one doesn't. Notice the differing filesize.

Source is Cruncher's high-grain HD source material.

[Didée]

Oh my dear. Mediafire is running strongly at ~8 kB/s for me ... that'll take a while, ETA 1h40m ...

In the meantime, if you're interested, I've put together a few short samples (http://maxupload.com/C9FDBFFD) of one of my 'historical' testing objects. No shiny HD, just good old PAL SD.:p
Still, a reasonable mixture of scenes where grain sucks up major bitrate (1 & 3 inparticular), while in some scenes "avoiding" grain endangers unwanted detail loss.
Some scenes with floating fog would be interesting as well, but I can't find any right now.

[Dark Shikari]

Oh my dear. Mediafire is running strongly at ~8 kB/s for me ... that'll take a while, ETA 1h40m ...

In the meantime, if you're interested, I've put together a few short samples (http://maxupload.com/C9FDBFFD) of one of my 'historical' testing objects. No shiny HD, just good old PAL SD.:p
Still, a reasonable mixture of scenes where grain sucks up major bitrate (1 & 3 inparticular), while in some scenes "avoiding" grain endangers unwanted detail loss.
Some scenes with floating fog would be interesting as well, but I can't find any right now.It works well on some of them and not as well on others--for example, 1 is a perfect candidate, while 5 (the fog one) results in a bit of blocking.

The main issue is I need to create a metric to decide whether or not to use the algorithm.

One idea would be to couple it with motion prediction (in the style of MVTools' Truemotion); if a block moves, don't act on it, for example. Speaking of which, are there any good algorithms for measuring if something has moved or not (not how much or where it moved, but just whether it moved) that are not affected strongly by grain? I mean, I coula always bruteforce it: fft3dftiler the clip and use that as a secondary clip for motion checking.

[Dark Shikari]

Oh my dear. Mediafire is running strongly at ~8 kB/s for me ... that'll take a while, ETA 1h40m ...

In the meantime, if you're interested, I've put together a few short samples (http://maxupload.com/C9FDBFFD) of one of my 'historical' testing objects. No shiny HD, just good old PAL SD.:p
Still, a reasonable mixture of scenes where grain sucks up major bitrate (1 & 3 inparticular), while in some scenes "avoiding" grain endangers unwanted detail loss.
Some scenes with floating fog would be interesting as well, but I can't find any right now.
ZShare mirror (http://www.zshare.net/download/40147356429bf5/).

[Didée]

OT - Funnily, ZShare also gives me ~8kB/s. Are there any still problems with the connection over the atlantic (somewhen in the past there were), are those hosters located in america? It's not my setup or connection - bandwidth down is above 1 MB/s, and I just downloaded my own sample above from MaxUpload at ~120 kB/s. Not great, but it's okay.

Sorry for the OT, but sometimes I'm really amazed by filehosters being very slow for unknown reasons.

ETA at MediaFire: 5 minutes. Light at the end of the tunnel! :D

[Dark Shikari]

OT - Funnily, ZShare also gives me ~8kB/s. Are there any still problems with the connection over the atlantic (somewhen in the past there were), are those hosters located in america? It's not my setup or connection - bandwidth down is above 1 MB/s, and I just downloaded my own sample above from MaxUpload at ~120 kB/s. Not great, but it's okay.

Sorry for the OT, but sometimes I'm really amazed by filehosters being very slow for unknown reasons.

ETA at MediaFire: 5 minutes. Light at the end of the tunnel! :D

MaxUpload went 15KBps-20KBps for me... :p . My connection varies wildly, I can literally max out my hard disk write speed on some websites (those with a direct Internet2 link, such as the server hosting the Elephant's Dream lossless files) and on some international ones I get as slow as 1 megabit or less.

[Dark Shikari]

Thanks a lot for those clips--I came up with a new heuristic that can adaptively decide whether a 16x16 block should be acted on or not. The result is less of an effect on compression on "difficult" sources like your fog scene, but without any of the blocking noticeable with my previous algorithm. In other words, it works on those now :) . Sources like Cruncher's that didn't have the problem don't seem to be affected much by the change.

Though I haven't done that much analysis yet, all sources I've tried with the new heuristic appear to have no noticeable artifacts from my algorithm at all.

[Dark Shikari]

With a major bugfix my filter now no longer craps out on non-grainy encodes at all. It appears to work in basically all cases now.

It has one minor difficulty left that I can find: it actually has two effects on the bitrate.

1. It lowers the bitrate if it is effective on that video (i.e. if there's enough unmoving areas with grain).

2. It gives an overall rise in bitrate of 3-10% or so.

If 1) isn't strong enough, 2) takes over, and the bitrate actually rises slightly... this may be an algorithm issue but if I can make a heuristic to avoid it, so much the better. I'm not very confident I can fix it though--it seems to be inherent in the algorithm.

[foxyshadis]

Does it raise or lower the average quantizers in a bitrate-based encode, though?

[Dark Shikari]

Does it raise or lower the average quantizers in a bitrate-based encode, though?Without a doubt lower, as it means fewer bits at the same quantizer.

[Dark Shikari]

http://i24.tinypic.com/2nvzeoh.png

:cool:

No real visual quality difference.

(CRF10 to allow for visual quality comparison of the filter, independent of the encoding. Bitrate results are similar at higher CRFs though).

(Test footage: "teil.vob")

A 200 frame sample of Cruncher's source: filtered (http://www.mediafire.com/?7gydo3zedyn), unfiltered (http://www.mediafire.com/?5mmb5wy2dur). The difference is huge--many frames go from 90% intra blocks to 10% intra blocks with the filter, and bit distribution is vastly improved.