Filter for eliminating DCT blocks

[SansGrip]

After some testing with my noise-producing filters (see
this thread (http://forum.doom9.org/showthread.php?s=&threadid=37003)), in particular using GaussianNoise in combination with Layer to add noise to a clip, I noticed a couple of problems. First the overall luminosity of the clip was increased (obviously as a side-effect of using Layer's "add" mode), and second it appeared to cause a very slight shift towards green. I'm not sure what might be responsible for this.

It occurred to me that in this specific application (adding noise to a clip), and given that Gaussian noise with mean=0 is both negative and positive, it makes more sense, would be more efficient, and would produce better results if one applied those values directly to a clip rather than layering them together later. So I wrote another filter called, funnily enough, AddNoise.

The reason one would want to add noise to a clip, as suggested in this thread's title, is that it can completely remove once and for all those awful DCT blocks (also known as dark blocks) that MPEG encoders enjoy liberally scattering through our otherwise-flawless encodes.

For a much more detailed rationale and usage instructions see the included documentation.

(Note to moderators: I made this a separate thread because it's a separate filter from my Noise_Generators package, and conceptually is more focused on artifact removal than general noise generation. Feel free to merge the two if you think it more appropriate.)

Edit: Removed links to old version.

[OUTPinged_]

mmm


Did a minor check and still cant get how you are getting better result with noise added.


Maybe you can post here a couple of pictures where the improvement is clearly seen?

[SansGrip]

Maybe you can post here a couple of pictures where the improvement is clearly seen?

Here's an example (http://www.jungleweb.net/~sansgrip/AddNoise.png) from American Pie (I happen to have the vobs sitting on my hard drive right now). It's a montage of four grabs of the same P-frame.

Script:

AudioDub(Mpeg2Source("ap.d2v"), WavSource("ap.wav"))
Crop(11, 8, 700, 462)
LanczosResize(352, 174)

opening = Trim(0, 533)
opening = opening.TemporalSoften(3, 8, 30)

movie = Trim(534, 131644)
movie = movie.AddNoise(yv=?, loop=4, mode="round")

credits = Trim(131645, 0)
credits = credits.Greyscale().TemporalSoften(3, 8, 30)

last = opening + movie + credits

AddBorders(0, 33, 0, 33)

LegalClip()
ConvertToRGB()

Settings:

TMPGEnc MPEG-1 352x240 500-2564kbps CQ-100

Details:

You'll want to zoom in about 300%, particularly on the expanse of wall to the right of the guy. You might want to nudge your monitor's brightness/contrast up a little if the DCT blocks aren't clearly visible.

The top-left grab had no noise applied. As you can see, the whole wall is composed entirely of DCT blocks. They're fairly subtle in this frame, but still very noticible especially on a TV.

The top-right grab had yv=1 noise applied. You should be able to see that the DCT blocks are almost completely eliminated by the added noise, with negligible effect to the rest of the frame. Personally, I think the detail elsewhere is actually enhanced, though that's a subjective thing (as poptones said here (http://arstechnica.infopop.net/OpenTopic/page?a=tpc&s=50009562&f=67909965&m=3890938134&r=5470927074), I find the undithered grab to be somewhat "cartoonish" in comparison. Incidentally, he posted some grabs in that thread that illustrate the benefit of dithering a lot more effectively than my example. I'll continue looking for a better one).

The bottom frames had yv=1.5 and yv=3 noise applied. yv=3 is definitely too strong with this source, and yv=1.5 is borderline at best.

Bear in mind that this encode is intended to be viewed on a TV. If I were encoding for monitor viewing with XviD I would likely need less strong settings. I may not need to add noise at all, though those DCT blocks do tend to crop up in dark areas.

[soulfx]

I'm going to check this out. Hehe, through all the trouble I have to remove noise from my captures now I'm going to see about adding noise.

I do know what is meant by "cartoon" looking and DCT blocks as when I get done applying the noise filters I have running on my captures I get a little bit of each. Right now all the filters are working good at keeping the noise down and detail up, but something just doesn't look right about the faces. They are too clean, almost like everyone is wearing streatched masks.

Yeah! I guess I'm the first one to give this filter a download. I'll run some test and try it out.

[SansGrip]

I'm going to check this out. Hehe, through all the trouble I have to remove noise from my captures now I'm going to see about adding noise.

I didn't believe it either until I saw the results. The good news is that adding noise is much quicker than removing it ;) :D.

Right now all the filters are working good at keeping the noise down and detail up, but something just doesn't look right about the faces. They are too clean, almost like everyone is wearing streatched masks.

Exactly. I think those "invisible" details that get smoothed away to aid compressibility aren't so unimportant after all, especially in faces and fine textures. For example, look without zooming at the cushions on the couch and the detail around the edge of the chair in the grabs I provided. In the grab with noise 1.0 I think they are sharper and (for want of a better phrase) more "three-dimensional".

It seems to me that MPEG isn't designed for very smooth sources; this is precisely the reason that cartoons can, at least in my experience, look so awful when run through the encoder. I've not tried it yet, but I have a hunch that adding some noise to a cartoon might improve that significantly.

I'll run some test and try it out.

I'll be very interested to hear your opinion :)

[Blight]

I don't see the point of adding noise prior to encoding as that just raises the bitrate.

If you want to add noise to compensate for frailties in the video-encoding, do it on the decode (i.e. see: ffdshow).

[SansGrip]

I don't see the point of adding noise prior to encoding as that just raises the bitrate.

Of course. That's what it's meant to do :).

If you want to add noise to compensate for frailties in the video-encoding, do it on the decode (i.e. see: ffdshow).

ffdshow is great, and if Panasonic would include it in my DVD player's firmware I'd stop adding noise during pre-processing ;).

[stickboy]

Originally posted by SansGrip
ffdshow is great, and if Panasonic would include it in my DVD player's firmware I'd stop adding noise during pre-processing ;).What if you use crappy cables? :D

[Xenoproctologist]

Hmm...the next step here would be to create an inverted unsharp mask and use it to weight the noise so it's only added to areas with flat shading.

[SansGrip]

What if you use crappy cables? :D

That's a good idea, but I think constantly switching back and forth depending on whether I'm watching a DVD or VCD would be bad for the connectors :D ;).

[SansGrip]

Hmm...the next step here would be to create an inverted unsharp mask and use it to weight the noise so it's only added to areas with flat shading.

Funnily enough I was just reading up on spatial frequency, because it occurred to me that the problem is worst in areas of low spatial frequency (i.e. flat areas with little detail).

I've not come across any comprehensible (to me anyway) explanation of it, so I was just about to ask in General Discussion :).

[Xenoproctologist]

Hmm...I just had a thought...

What if the noise were only added to the pixels on the edge of the macroblocks? Would this be enough for mpeg4 codecs to latch on to, while having the added benefits of (1) consuming less bandwidth than full-frame noise and (2) having the noise be smoothed away during deblocking?

[SansGrip]

What if the noise were only added to the pixels on the edge of the macroblocks?

If someone (maybe me -- I'll think about it ;)) could come up with an effective algorithm for detecting just DCT blocks, then this would probably help with re-encoding MPEG sources.

However bear in mind that almost all the DCT blocks you see in the grabs I posted above were generated by the second encode (that is, from MPEG-2 to MPEG-1). Since those DCT blocks don't exist yet it would be hard to blend them :).

[SansGrip]

I was thinking about the spatial frequency thing and I remembered an experimental filter a wrote quite some time ago to do "detail detection" on a block by block basis. It wasn't very successful but I decided to take another look at the code and see if I could resurrect it.

After some playing I spotted what was wrong with it and, to my surprise, it actually works really quite well. Here (http://www.jungleweb.net/~sansgrip/AddNoiseShow.png) is a shot of it in "show" mode displaying where the new version will add noise to the same frame I used to illustrate the problem in my post above.

You can see that it thinks there's quite a lot of areas where there's insufficient detail (this is of course configurable; I used detail_threshold=12), particularly that wall which we know gets totally splattered with DCT blocks when there's no noise added.

I've done some frame-by-frame comparisons and it seems where the filter detects insufficient noise is almost always exactly where one gets DCT blocks occurring in the final encode, including dark areas (lair of the evil dark blocks).

Anyway, play around with it (and look at the source if you like). I consider this to be rather a good candidate for a compromise between the yes-add-noise and what-are-you-nuts camps ;).

Edit: Removes links to old version.

[MaTTeR]

SansGrip,

Looking at your screenshot, it seems like the filter is definitely displaying the areas where noise would need to be added based on my experience. So to me this looks very promising and I'm anxious to do some testing in about 12hrs or so. I've been fighting the DCT blocks on stable backgrounds for sometime now and I'm glad to see your effort in the resolution. Thx!

[Xenoproctologist]

Originally posted by SansGrip
What if the noise were only added to the pixels on the edge of the macroblocks?

If someone (maybe me -- I'll think about it ;)) could come up with an effective algorithm for detecting just DCT blocks, then this would probably help with re-encoding MPEG sources.
Pre-existing macroblocks are irrelevant, as what you're trying to do here is prevent macroblocking in the current encode. Macroblock size and offset would have to be passed to the filter as parameters.

Also, the current spatial detail detection could result in some thresholding artifacts in very clean sources. You might want to consider fading the edges of the noise out.

[Didée]

Originally posted by Xenoproctologist
You might want to consider fading the edges of the noise out.
This is true for almost every filter that processes only a part of the frame!

Anyway, what about this:
IMHO, the blocking in flat areas is caused by the high quant coefficients the codec decides to use *because* of the little detail in the respective area.
So, adding noise seems to be mainly needed when encoding to mpeg-1 - when encoding to XviD, the usage of a custom quantizer matrix with less agressive coefficients for the low frequencies should fight this blocking problem as well.

?

[SansGrip]

it seems like the filter is definitely displaying the areas where noise would need to be added based on my experience

It does seem that way, which is surprising since the algorithm is so simple. I wasn't expecting it to work so well. But it needs testing on a variety of sources before I'll get too excited ;).

I'm anxious to do some testing in about 12hrs or so.

I'll be very interested to hear your results.

I've been fighting the DCT blocks on stable backgrounds for sometime now

What version of MPEG do you encode to?

[SansGrip]

Macroblock size and offset would have to be passed to the filter as parameters.

Well AFAIK the offset should always be 0,0 and the DCT blocks always 8,8 for MPEG. But someone might like to correct me on this. I think it's still an interesting idea that I'm going to play with some.

Also, the current spatial detail detection could result in some thresholding artifacts in very clean sources. You might want to consider fading the edges of the noise out.

I was thinking the same thing after writing the "show" code and before I'd finished the actual noising part, but with reasonable means and variances (i.e. those most people would need to use) it's very difficult to spot even when you look real close. I noticed it a couple of times over the course of an hour's testing, but it was only where I already knew (from "show") that a block of noise would appear in an area.

Of course if you jack up the variance to something unrealistic (say, >30) then it starts getting noticible, but I would say that's enough of a noise difference that feathering wouldn't really help.

[SansGrip]

IMHO, the blocking in flat areas is caused by the high quant coefficients the codec decides to use *because* of the little detail in the respective area.

I've not done any testing, but my gut says this is exactly the cause.

So, adding noise seems to be mainly needed when encoding to mpeg-1

And that's what I encode to almost exclusively, hence the filter ;).

when encoding to XviD, the usage of a custom quantizer matrix with less agressive coefficients for the low frequencies should fight this blocking problem as well.

There's a definite chance that would work too. While the docs I've read on MPEG encoding say that it compresses more in high frequency areas, I'd say based on my experience (certainly with MPEG-1 at all bitrates and with MPEG-2 at lower bitrates) the default matrix is too aggressive on very low frequency areas.

I've only done a couple of XviD encodes (generally when I've captured something to test my capture card) and got weird results so I've not really played with it much, but this now has me wondering about the Quantize Matrix section of the TMPGEnc settings... :)

Edit: It seems TMPGEnc's default matrix is far too aggressive with low frequency areas. The "MPEG standard" matrix is significantly better at reducing DCT blocks (except in I-frames, which seems to be unfixable), and there are other non-standard matrices that perform even better. It still can use a little dithering, but nowhere near as much as with the default matrix.

[MaTTeR]

Originally posted by SansGrip
at version of MPEG do you encode to?

90% of the time I'm using MPEG-4/XviD and the other 10% I'm using TMPEG/MPEG-1. Watching the MPEG-1 encodes on my standalone player, well I never see any DCT blocks at all, not even on the stable backgrounds such as walls. However, for MPEG-1 I'm also using the excellent KVCD/XVCD templates which IMHO give excellent results but that's another topic.

[SansGrip]

Watching the MPEG-1 encodes on my standalone player, well I never see any DCT blocks at all, not even on the stable backgrounds such as walls.

This could also be due to contrast/brightness/sharpness settings on your TV as well as the ambient light levels when you watch it. Is your TV calibrated?

However, for MPEG-1 I'm also using the excellent KVCD/XVCD templates which IMHO give excellent results but that's another topic.

Hmmm, I've not heard of those. Where did you find them? Edit: Never mind. www.kvcd.net ;).

[MaTTeR]

Originally posted by SansGrip
Is your TV calibrated?

Yes, I calibrated it about a month ago when it was first bought. In terms of brightness/contrast, it's somewhat close to my monitor. The only difference I can think of is that my standalone is using component outputs which hides the blocks somewhat. I can take the same video, play it from HTPC via S-Video/GeForce2 and the blocks can be seen easily.

I'm going to start a two pass rip on a pretty clean DVD source using AddNoise. This source is pretty sharp and cleaner than most DVD's but does have some smoke/fog scenes which loves to look blocky after encoding. I'd like to use the LegalClip filter along with AddNoise2 with a conservative variable. Would this script be a nice starting point?

LoadPlugin("C:\Program Files\AviSynth2\plugins\MPEG2DEC3.dll")
mpeg2source("D:\DIvX RIPs\Romeo is Bleeding\romeo.d2v",cpu=1,moderate_h=45,moderate_v=65,lumoff=-2,iDCT=2).crop(8,4,704,464).GrapeSmoother(10).UnFilter(3,3).AddNoise(yv=1,show=false,fast=false,block_size=16,loop=4).BicubicResize(624,336,0.333,0.333).LegalClip()

Hmm...maybe a bit too much filtering.

[SansGrip]

In terms of brightness/contrast, it's somewhat close to my monitor.

Then I would suggest your monitor is too bright or your TV is too dark ;). They shouldn't really be the same. The monitor should be darker.

The only difference I can think of is that my standalone is using component outputs which hides the blocks somewhat.

Wow, wish I had component in/out on my stuff :).

...your script...

It looks ok, except I would say that you might not need lumoff=-2 and UnFilter if you use AddNoise. I've always thought the lumoff/UnFilter combination to be something of a hack. It's a clever hack that seems to work, but why? No-one's really pinned that down yet. Also it'll change your levels slightly.

As for AddNoise, I'd say try a short clip with various settings before deciding what you want to use. For example with the TMPGEnc default matrix I would use 0.75 with American Pie, but with the new matrix I just made (and also with the MPEG standard one) I'd use 0.3 instead. (You don't need yv=1 because 1 is the default variance for all components.)

Also try with mode="round" and mode="floor". The former will add slightly more noise because it rounds to the nearest integer instead of towards zero.

I always use loop=5, since it seems to work just as well as generating new noise for each frame, at least for this purpose.

Finally, fiddle with the detail_threshold with show=true to make sure it's in line with your thinking on what already has enough detail. 12 seems a reasonable default but I only tested it with a couple of sources.

Apart from that, there's really nothing to tweak ;) :D.

Oh, and WRT LegalClip, you can include it if you like. It all depends on what XviD does with out-of-range values.

[MaTTeR]

Thanks for the input but right now it seems to be a bit of a struggle to find good values. Guess it's going to take awhile for me to get some conclusive results.

I haven't figured out what's causing it but my speed has dropped from 62FPS using YV12 and no filtering to a whopping 10FPS with AddNoise(yv=0.5,fast=true),GrapeSmoother(8) & LegalClip() in YUY2 mode of course. I'm thinking LegalClip couldn't cause that much of a speed penalty or am I wrong? It must be AddNoise which it made it drop so much.

So far I played around with detail_threshold and it's resulted in the file being way oversized. Lots of variables to play with here:D

Edit- Yep, my monitor is darker but only slightly. Since I just recently switched over to component cables, I can honestly say they are well worth the $80 I paid for them. The colors are so clean and the details are superb when viewing a clean source such as a SuperBit DVD.

Edit2- I've taken Unfilter and lumoff=-2 out of the script for now and doing no preprocessing with MarcFD's filter. In regards to LegalClip, I have no idea how XviD handles this but I'm using it now to see the difference if any.

[SansGrip]

I haven't figured out what's causing it but my speed has dropped from 62FPS using YV12 and no filtering to a whopping 10FPS with AddNoise(yv=0.5,fast=true),GrapeSmoother(8) & LegalClip() in YUY2 mode of course.

The culprit (or one of them) is AddNoise. You'll definitely want to use the loop parameter to pre-generate and loop over frames of noise instead of generating new noise for each frame. Making Gaussian noise is very time-consuming and the code is unoptimized.

Benchmark the script without AddNoise and then with AddNoise(..., loop=x, ...). You shouldn't see a big difference.

I'll be interested to hear the result.

So far I played around with detail_threshold and it's resulted in the file being way oversized. Lots of variables to play with here:D

Yep, and there might be more in the future ;). I was thinking of adding weighting to the noise based on the detail level, and the option of only noising every nth frame. I'm also going to investigate Xenoproctologist's suggestion of noising only at DCT block boundaries.

As far as finding good values for a particular clip, I would suggest first turning on show mode and setting detail_threshold to something that looks right to you (in other words make sure it's highlighting only areas you think need more noise). Then turn off show mode and set the variances to, say, 1.0. This might be too much, so decrease it by say 0.2 each time until you start seeing unacceptable DCT blocks in low frequency areas. This should give you an idea of the minimum amount of noise necessary for a particular clip.

Edit: I forgot to mention there's plenty of other things to try out, such as setting it to noise the luma very gently and the chroma stronger (i.e. set yv to something like 0.25 and uv and vv to something in the range of 0.75-3). This doesn't seem to work too well with MPEG-1, but MPEG-4 is a different beast.

Another thing you can try is smoothing before or after adding noise. I'm guessing the latter would tend to negate the benefit of dithering in the first place, but it's something I've not got around to testing yet. Yet another possibility is noising the whole frame (detail_threshold=0) and then smoothing. I don't know what the result will be, but you never know :).

You can also set loop=1 which will produce a single frame of noise that is added to every frame. This is in effect spatial noise as opposed to regular spatiotemporal noise. At high settings it looks rather like you're watching it through a very speckled window, but at low settings it might reduce the DCT blocks without being so noticible to the eye. I don't know, I've not tried it yet. Let me know ;).

[MaTTeR]

Originally posted by SansGrip
The culprit (or one of them) is AddNoise. You'll definitely want to use the [i]loop parameter to pre-generate and loop over frames of noise instead of generating new noise for each frame.

Well the loop parameter definitely fixed the speed issue. I lost about 4hrs of downtime last night due to avisynth acting strangely but finally got it sorted out this morning with the help of some coffee:)

I've tested alot of different settings on this clean source DVD and I'm still not satisifed with the results that the UnFilter(x,x) and Lumoff=-x givves me. So far I've had to find a middle of the road setting that doesnt cause the codec to eat excess bitrate, in doing so the DCT blocks aren't completely disappearing but I did notice an improvement. Here's what I'm currently using and will continue to tweak-
LoadPlugin("C:\Program Files\AviSynth2\plugins\MPEG2DEC3.dll")
mpeg2source("D:\DIvX RIPs\Romeo is Bleeding\test\romeo.d2v",iDCT=2)
crop(8,4,704,464)
GrapeSmoother(8)
AddNoise(yv=0.25,vv=0.65,fast=true,loop=5,detail_threshold=4)
BicubicResize(624,336,0.333,0.333)
LegalClip()

I like where I have the detail_threshold set at but I'm thinking I need to possibly play with the yv,vv and block_size parameters. Have you seen how much varying the block_size affects the encode? I'm thinking of playing around with it and trying the floor/round modes as well.

[SansGrip]

So far I've had to find a middle of the road setting that doesnt cause the codec to eat excess bitrate, in doing so the DCT blocks aren't completely disappearing but I did notice an improvement.

Of course there's always going to be a path to be struck between bitrate and DCT blocks, especially using this filter. I'm beginning to think that AddNoise achieves much better results with MPEG-1 than with MPEG-4.

The question now is, what exactly is the lumoff/unfilter combo doing to the clip? I think I shall make it my mission today to find out ;).

I like where I have the detail_threshold set at but I'm thinking I need to possibly play with the yv,vv and block_size parameters.

I think the key to it is finding the right variances for the clip. 0.25 in floor mode isn't very much, so I'm not surprised you're still seeing some blocks. Look at it this way:

With a variance of 0.25, 68% of the values generated will be within -0.25 and 0.25. 95% will be within -0.5 and 0.5, and 99% will be within -0.75 and 0.75. The closest-to-zero number that produces a change in the pixel is, obviously, -1.0 or 1.0, which is four standard deviations away from zero when variance=0.25. The chance of getting a -1.0 or 1.0 is very small, and something like 99.9999% of the numbers you're generating will result in a zero, or no change to the pixel.

If you were to use mode=round, though, the closest-to-zero number that affects the pixel is 0.5. Since 0.5 is only 2 standard deviations away from the mean of zero, it's much more likely to occur. Indeed, about 5% of the values you generate should be <= -0.5 or >= 0.5.

As you can see, when using small variances the rounding mode can make a big difference :).

You could also try a higher luma variance with loop=1 and see what you think of the effect. Or try just greyscale noise, with yv=whatever, uv=0, vv=0.

Have you seen how much varying the block_size affects the encode?

To be honest I've never tried. I'd be interested to hear your conclusions.

[iago]

The question now is, what exactly is the lumoff/unfilter combo doing to the clip? I think I shall make it my mission today to find out ;).

:D LOL!

Good luck, pal! I'm sure it "does" something "good", though I -like everyone else- don't know what the real magic is! ;)

Sorry for shifting off-topic, but I just couldn't resist! ;)

kind regards,
iago

[MaTTeR]

The only reasoning I've come up with for the nice results of the Unfilter/Lumoff combo is that UnFilter is doing the majority of the work. With a setting of UnFilter(4,4) for example, maybe it slightly sharpens the existing noise forcing the encoder to add a few more bits to those areas which essentially is what were also trying to do with AddNoise(xx). The only problem with my theory is that I think UnFilter is more designed for sharpening object edges instead of so called mosquito noise:)

Whatever the reasoning, for MPEG-4 it's the best solution I've used up until this point but more testing continues.

[iago]

@MaTTeR

When I first encountered the magic UnFilter does together with lumoff=-2 in one of my test encodes, I also thought just like you. That is, I concluded that the key point is sharpening with UnFilter(+,+). But then, when I discovered that UnFilter with (-,-) parameters is also as effective as (+,+) parameters against black-blocking when used with lumoff=-2, this reasoning, which sounded quite reasonable first, simply collapsed! ;)

best regards,
iago

[SansGrip]

Whatever the reasoning, for MPEG-4 it's the best solution I've used up until this point but more testing continues.

Does it get rid of all DCT blocks, or just those in dark areas? I tried UnFilter(5, 5) with an MPEG-1 clip and it did work somewhat against DCT blocks in "non-dark" areas, but not as effectively as AddNoise (for me anyway).

It also produced, as it's supposed to, a general sharpening that seemed to encourage mosquito noise. Perhaps the ideal solution might be to implement lumoff=-a and an unsharp mask only in the areas lacking detail...?

[SansGrip]

But then, when I discovered that UnFilter with (-,-) parameters is also as effective as (+,+) parameters against black-blocking when used with lumoff=-2, this reasoning, which sounded quite reasonable first, simply collapsed! ;)

That is weird. I can see lumoff=-2 helping in that, perhaps, superblack compression artifacts might be shifted out of the range where they get noticed by the encoder, but I have no idea why smoothing would help if the problem really is a lack of detail.

I'm still pretty confused by this whole thing. By its nature MPEG should compress low-frequency areas much less than high-frequency ones, so why it's only low-frequency areas that exhibit DCT blocks is anyone's guess.

According to this, smoothing the low-frequency areas (thus reducing the frequency even more) should improve the blocking since the encoder should compress those areas even less. But test results do not bear this out. If anything, smoothing low-frequency areas makes the problem worse, while adding noise (increasing the frequency) makes them go away.

One possibility is that higher frequency areas get allocated significantly more bits and thus, even with the higher levels of compression, do not suffer from this problem.

Any MPEG experts out there?? ;)

[MaTTeR]

Originally posted by SansGrip
[i]Perhaps the ideal solution might be to implement lumoff=-a and an unsharp mask only in the areas lacking detail...? You read my mind man*evil grin*:D

The detection algo your using in AddNoise seems to pick out the problematic parts such as walls, doors and in general....stable backgrounds. Those areas are where we usually see the blocks floating around that distract us from what the main object of the frame is suppose to be, say for instance a persons face. Combining your detection scheme with an UnSharp Masking filter might be somewhat of a silver bullet, at least it certainly sounds good when I think about it;) How difficult would such a beast be to code? I'm not a coder so I'm just curious.

Edit- iago is right though, even using (-x,-x) with UnFilter still reduces the blocking problem:confused:

[SansGrip]

The detection algo your using in AddNoise seems to pick out the problematic parts such as walls, doors and in general....stable backgrounds.

Pretty cool considering all it does is count unique luma values ;).

Combining your detection scheme with an UnSharp Masking filter might be somewhat of a silver bullet, at least it certainly sounds good when I think about it;)

I was thinking the same thing. There shouldn't be any of the problems normally associated with sharpening such as mosquito noise and ringing, since those are high-frequency issues.

How difficult would such a beast be to code? I'm not a coder so I'm just curious.

I have a vague idea how unsharp mask works (first you produce a blurred version of the image then use that to make the sharpened version) but I've not looked into it in any depth. It would be slower than AddNoise with loop=x though.

I will ask Dr Google right now and try to get something implemented tonight. I think I'll start by adding it as an alternative "mode" (though I've already used that parameter name, dammit ;)) for AddNoise. If it does turn out to be the "silver bullet" then maybe I'll make it a separate filter.

Edit: After looking at UnFilter more closely it seems it's not an unsharp mask after all, but (as the name suggests, now I think about it) a way of compensating for common filtering techniques that might have been applied to the source.

Unfortunately I don't believe I can use Tom's code directly in my filter because his is released under the GPL and mine is not (I might be wrong here though). For the moment I'll work on adding an unsharp mask for y'all to play with :).

iago is right though, even using (-x,-x) with UnFilter still reduces the blocking problem:confused:

Over the whole frame, or just in dark areas?

[MaTTeR]

Originally posted by SansGrip
It would be slower than AddNoise with loop=x though. Well if that's the price we have to pay then so be it :-)

I will ask Dr Google right now and try to get something implemented tonight.That would be fantastic, I'll be sure to monitor the thread. In regards to the name, well you could always name it "Blockbuster". lol

Over the whole frame, or just in dark areas? Sorry I missed the first time you asked this. Well since I mainly see the blocks only in dark areas anyways then that's where I see them disappear after the UnFilter combo. Maybe I only see them in darker to mid-tone areas because I'm encoding at higher bitrates than others do, not sure:rolleyes:

I'm not so sure that UnSharp masking would need to be used in this filter your writing though. I'm thinking that even a "generic" type of sharpening might even work. I mean after all, the parts that were mainly interested in sharpening have very little detail which is why the encoder somewhat ignores them. It seems a more general sharpening might be faster than UnSharp but I'm only speaking from my old Photoshop(2D) experience. What are your thoughts on this?

Edit-
Yep that's what I was thinking about UnFilter, I think "msharpen" might be the UnSharp masking filter. Tom's filter seems to focus more on the edge detection of frames itself. I think it's specifically to get rid of the nasty "EE" effects from poorly filtered sources such as the crappy "Born on the Fourth of July" DVD. Great movie ruined by EE:devil:

[SansGrip]

In regards to the name, well you could always name it "Blockbuster". lol

hehehe I like it ;).

Well since I mainly see the blocks only in dark areas anyways then that's where I see them disappear after the UnFilter combo.

I would say that MPEG-4 has pretty much solved the DCT block issue except in darker areas. That's why I'm getting better results with AddNoise than you guys, since I'm using MPEG-1 with puts them all over the place (as does MPEG-2 to a certain extent).

It seems a more general sharpening might be faster than UnSharp but I'm only speaking from my old Photoshop(2D) experience. What are your thoughts on this?

I think you're referring to something like the built-in Sharpen filter, which IIRC uses a simple 3x3 matrix. I guess the speed difference would depend on how the blurring was implemented with an unsharp mask. If a 3x3 blur was used the speed difference wouldn't be that much, but if implemented as a Gaussian blur it would be quite a lot slower. Either that or I'm talking nonsense ;). I'll get back to you when I've done some actual research...

Edit: Stop editing in response to the edit I wrote while you were writing, while I'm writing! :D

I don't know if MSharpen uses an unsharp mask. It definitely is edge-constrained though.

[MaTTeR]

Hmm...I'm having second thoughts about the initial idea:D

Personally I'd much rather blur the areas that the detection algo has selected rather than sharpen them for obvious bitrate savings. This is where UnFilter has come in to use I guess because we can either sharpen or blur depending on the source material.

In regards to Gaussian blurring with UM, would we really need that process? I guess my thinking is that even a 3x3 average when blurring or sharpening might suffice for what were trying to accomplish. In my thinking, the most difficult part is successful and that's the detection algo that you implemented.

BTW- Do you have Blockbuster chain reantal stores in Ontario? They are on every street here in the states. That name would be real fitting for this type filter.

Edit- (yet again):D
I mispoke earlier, Donald's "msharpen" filter is not using UM. Have a look here (http://forum.doom9.org/showthread.php?s=&threadid=32757&highlight=unsharp) for more info.

[trbarry]

Tom's filter seems to focus more on the edge detection of frames itself. I think it's specifically to get rid of the nasty "EE" effects from poorly filtered sources such as the crappy "Born on the Fourth of July" DVD. Great movie ruined by EE

UnFilter just approximates an inverse of a simple 3 tap sharpen or soften filter. It doesn't look for edges or try to guess what convolution was previously used. It's just optimistic, figuring the original encoders were lazy and the user is smart enough to set the strength. That is, if it causes artifacts then back it off a bit. ;)

And I still have no idea why it helps the blocking problem. It's probably just a serendipitous bug that cancels one in Xvid. But it makes me afraid to ever change it. It will be interesting to see if the YV12 version still helps. :devil:

- Tom

[Kaiousama]

Thanks Great filter Sansgrip ! :D
Two considerations:
1) it's slow (but this is a secondary problem ;)
2) I'm testing it on anime (btw argento soma :P) and if i've well understood the filter apply the noise to the green areas leaving unchanged the non green areas according to the detail_threshold bcause in that area (the non green ones) the detail is higher than that value.... looking at the attached picture you'll see that the non-noised (non green) areas in this way are the main edges' ones, and irony of the fate are the main parts that in anime are affected by dct artifacts (because the main edges in the scene are like a wavefront when the motion starts), request: can you insert an option in the filter to exactly invert the assegnation of green and non-green (that will be noised & non-noised) areas, so in that way the noise will be added only to the edges&near-edges area (giving a better DCTproblems reduction & better quality in the parts that are not affected by DCTartifacts)? Many thanks! Bye

[edit] i've tryied to attach an image but probably it needs a bit to be authenticated ^_^

[SansGrip]

1) it's slow (but this is a secondary problem ;)

Well, it's very slow when generating new noise for each frame. But with the loop option it shouldn't be too bad. It's all still unoptimized of course.

if i've well understood the filter apply the noise to the green areas leaving unchanged the non green areas according to the detail_threshold

That's right. In show mode areas with insufficient detail (that will have "detail" added) are highlighted in green.

the main parts that in anime are affected by dct artifacts (because the main edges in the scene are like a wavefront when the motion starts)

That's interesting, I didn't know this. To be honest I'm surprised that you get DCT blocks in high-detail (high-frequency) areas, but while I've never tried it myself I know anime encoding is a different beast entirely from live action :).

can you insert an option in the filter to exactly invert the assegnation of green and non-green (that will be noised & non-noised) areas

It will be in the next release, which will also include 3x3 sharpening and blurring. I've written the code, I just need to tidy it up :).

[Kaiousama]

That's right. In show mode areas with insufficient detail (that will have "detail" added) are highlighted in green.
is it possible to make the noised areas( now higlighted in green) highlighted in an alpha blended colour (the same green for instance); so we can see what's under the green to better decide if that part has to be noised or if it's better to reduce the noise threseold?

That's interesting, I didn't know this. To be honest I'm surprised that you get DCT blocks in high-detail (high-frequency) areas, but while I've never tried it myself I know anime encoding is a different beast entirely from live action :).
It's difficult to say with words (mainly for a -bad english speaking- italian ^_^) but the issues in anime are mainly two (in my opinion): the artifacts (blocks) created in an overall fast scene movement (in the sliding or zooming ones) and the artifact created near the edges in movement (slow or fast, it doesn't matter); this second one is the most eyes' sensed problem, i think it's more important in anime than movies because anime has definite and massive edges for each thing, while in reality movies the edges are less focused by eyes (because in proportion to the rest of the picture are only a little and trascurable part)

It will be in the next release, which will also include 3x3 sharpening and blurring. I've written the code, I just need to tidy it up :).
faboulous, amazing, wow!!
i'm thinking: can you code a parameter to decide what kind of noise strenght and blurring effect add separately to the -over the noise threseold- parts and other values to the -under the noise threseold- parts ?

You are doing a great work... Thanks:D

p.s. a little curiosity: why only 3x3 blur?isn't there a way to implement variable diameter blur (it would be very useful to have a high diameter blur in the colour uniform areas and a little one in the detailed parts)?

[SansGrip]

is it possible to make the noised areas( now higlighted in green) highlighted in an alpha blended colour

I already fixed that yesterday. Affected blocks are now outlined in grey and the contents are unaltered.

It's difficult to say with words (mainly for a -bad english speaking- italian ^_^)

Don't worry, your English is molto bene -- definitely much better than my Italian which I "studied" for three weeks ten years ago, both for the pure love of learning and because the two best-looking girls in the school were also in the class :D.

can you code a parameter to decide what kind of noise strenght and blurring effect add separately to the -over the noise threseold- parts and other values to the -under the noise threseold- parts ?

That would create an awful lot of parameters, at least for method="noise", and I'm not sure what benefit there would be in adding noise to areas already sufficiently detailed. What did you have in mind?

why only 3x3 blur?isn't there a way to implement variable diameter blur (it would be very useful to have a high diameter blur in the colour uniform areas and a little one in the detailed parts)?

Because it's fast and easy ;). I agree, though, that while a 3x3 sharpen is more than strong enough, a 3x3 blur isn't very effective. I'm still doubtful, though, as to the positive impact of reducing the frequency in already low-frequency areas.

I just had a thought that is sort-of connected to this reply but would work better as a separate reply for people who haven't read this far ;). I'll write that in a moment.

P.S. As to why the new release isn't out yet, I took some time to (algorithmically) optimize the sharpen and blur methods and gained several fps in the process. I also did a pretty major structural overhaul to make the code more modular and easier to maintain, and managed to (entirely accidentally) gain another few fps. I still have to add the noise generation code back into the restructured version, but that won't take long. Then I have to update the docs. Then I'll release it :).

[SansGrip]

Before I forget it, here's what popped into my head while I was writing the last reply...

As I said I'm not totally sure of the benefits of blurring low-frequency areas (except for extra compressibility), but I can see the benefit of being able to apply separately different filters to low- and high-frequency parts of the frame.

One thing I don't want AddNoise/Blockbuster to become is a filter that tries to do everything and please everyone. I conceived it as a way to eliminate DCT blocks and want it to stay focussed on that job.

For this reason, might it not be a good idea to write something along the lines of:

DetailMixer(threshold=x, low_filter=UnFilter(...), high_filter=SmoothHiQ(...))

where the output of low_filter is used for low-frequency areas and the output of high_filter is used for high-frequency areas and then blended together?

Just a thought.

[Kaiousama]

can you code a parameter to decide what kind of noise strenght and blurring effect add separately to the -over the noise threseold- parts and other values to the -under the noise threseold- parts ?
That would create an awful lot of parameters, at least for method="noise", and I'm not sure what benefit there would be in adding noise to areas already sufficiently detailed.
i think 2 more options are sufficient : a highFreq_strenght and a lowFreq_strenght, and the entropy doesn't become like in my bedroom :D

What did you have in mind?
Simple... maybe your filter can reduce even the mosquito effect,
think: the mosquito effect borns (and it's more visible) in the detailed part of the movie that are moving, think of it as an expansion of the block concept but inverted.
I mean: in high frequency areas, a movement create a zone of temporaneal low quality, the codec has to analyze by macroblocks and zzap! in this macroblock the innocent watcher see a mosquito that surrounds the movement of the edge.
Now, what if in this high_freq moving zones there is some noise that make them more like a uniform moving zone instead of a -surrounded by high frequency edge- moving zone? (yes, the detail goes a little down but remember, this zone is in movement, so a little noise is hard to see with eyes).
Maybe even this bad effect can be attenuated in one shoot ;) (what do you think? i think the best way is to make some compression tests on difficult movie and anime parts)

P.S. As to why the new release isn't out yet, I took some time to (algorithmically) optimize the sharpen and blur methods and gained several fps in the process. I also did a pretty major structural overhaul to make the code more modular and easier to maintain, and managed to (entirely accidentally) gain another few fps. I still have to add the noise generation code back into the restructured version, but that won't take long. Then I have to update the docs. Then I'll release it .
Don't worry for the release... i think is better to have a big changed release (with documentation) instead of 3-4 little releases in a day made in hurry ;)

I already fixed that yesterday. Affected blocks are now outlined in grey and the contents are unaltered.
your solution is even better than the alphablended one, good idea

Don't worry, your English is molto bene -- definitely much better than my Italian which I "studied" for three weeks ten years ago, both for the pure love of learning and because the two best-looking girls in the school were also in the class
Knowledge & women .... the two real people's drugs :D :D (glad to party the new upcoming release with a good glass of chianti :sly: )

Goodnight

[SansGrip]

i think 2 more options are sufficient : a highFreq_strenght and a lowFreq_strenght, and the entropy doesn't become like in my bedroom :D

That's a possibility. But I'm still not convinced about increasing the frequency of high-frequency areas in an attempt to reduce artifacts :).

maybe your filter can reduce even the mosquito effect,
think

I've not tried it, but I doubt it very much. Mosquito noise is a high-frequency artifact, which is why you only see it around edges etc.. Increasing the frequency of those areas will almost certainly make the problem worse, not better. Smoothing (i.e. reducing the frequency) in those areas might work, though.

Knowledge & women .... the two real people's drugs :D :D

hehehe yes, and now I know a handful of Italian words that I wouldn't otherwise. Didn't get anywhere with the girls, though ;).

[High Speed Dubb]

Sansgrip,
I take it you&rsquo;ve figured out what a variance is. ;)

Here&rsquo;s a speed trick you could use. A normal distribution can be closely approximated as a binomial. That can let you replace a slow normally distributed random number generator with a few draws from a uniform distribution.

In other words, get n uniform[0,1] random numbers (i.e., the usual random function), add them together, subtract (n/2), and multiply by a number chosen to get you the desired variance. The bigger then n, the more closely this will resemble a normal distribution. 4 is probably a reasonable n to use.

[SansGrip]

I take it you&rsquo;ve figured out what a variance is. ;)

heheh yes, it took me a while but I got there. I still have a bunch of image processing-related PDF's lying around that I wish I could understand though ;).

In other words, get n uniform[0,1] random numbers (i.e., the usual random function), add them together, subtract (n/2), and multiply by a number chosen to get you the desired variance. The bigger then n, the more closely this will resemble a normal distribution. 4 is probably a reasonable n to use.

How would one normalize this to zero mean unit variance?

I read about this technique (with a recommended n of 12) on Usenet but whoever was writing said it's a fairly poor way of getting a normal distribution. Probably for this filter it would be more than adequate, though.

The speed of the Gaussian PRNG isn't really an issue any more, since I now pre-generate and loop over the cached noise. I also have a trick for the new version which dramatically reduces memory requirements (I make one-quarter-sized frames of noise, and cycle through the cache for each quarter for each component).

[High Speed Dubb]

Subtracting (n/2) gets you a 0 mean. If you need to adjust to variance = 1.0, try multiplying the output by sqrt(12/n). (Maybe that&rsquo;s where the poster came up with an n of 12?) n=4 was my rating of &ldquo;looks pretty much like a normal,&rdquo; since that&rsquo;s the goal, here.

[Kaiousama]

For this reason, might it not be a good idea to write something along the lines of:
DetailMixer(threshold=x, low_filter=UnFilter(...), high_filter=SmoothHiQ(...))
where the output of low_filter is used for low-frequency areas and the output of high_filter is used for high-frequency areas and then blended together?

mhhh.... this is a great idea, because let everyone apply the better filter combination for each case :eek: (i'm imploring to implement this option, it would be too much useful :p :D)

[Wilbert]

For generation of samples of a normal variable, the math folks use the Box-Muller transform:

If X1, X2 are uniformly (independent) distributed on the interval (0,1) then Z1,Z2 are normal distributed (mean zero and variance one):

Z1 = sqrt( -2*ln(X1) ) * cos(2*pi*X2)
Z2 = sqrt( -2*ln(X1) ) * sin(2*pi*X2)

see also in mathworld (http://mathworld.wolfram.com/Box-MullerTransformation.html).

So you generate samples for X1 and X2 and compute the samples for Z1, and scale it if necessary.

[yaz]

hi all !

i enjoy this thread very much. it blurbs my mind all the time (never mind :-)

q1: would it be possible to tune the extent of noise (intended to add) somehow to the average luma round the pixel a/o to the luma noise of the block/frame ?
- sometime i'd like more noise in a certain part of the picture or i'd add more to the darker ones or so.

q2: would it be possible to draw the noise somehow from the picture?
- imho, noise is a part of the picture holding relevant information. if this part were reinvented (to a certain extent), that (part of) information would be regained. or am i completely wrong ? of course, this noise wont have any 'presettable' characterisctics.
- for such purpose, i usually do a strong smoothing & then i merge back a part of the luma from the orig frame. it's very helpful, especially with tv capts & vhs sources. i was just musing about something like that.

y

ps don't stop your activity just because of my stupidity :-)

[SansGrip]

Well I finally got the documentation in order this morning and have uploaded Blockbuster v0.3a to my web site along with its source code.

Many changes in this release, including substantial downsizing of parameters and significant speed boosts. In particular note that the way detail_threshold works has been changed. Check out the docs (http://www.jungleweb.net/~sansgrip/avisynth/Blockbuster-readme.html) for more information.

Enjoy :).

Edit: Changed link to point to 0.3a.

Edit 2: Remove links to old version.

[SansGrip]

(i'm imploring to implement this option, it would be too much useful :p :D)

It wouldn't be an option, it would be a separate filter. Once I come up with a good -- and fast -- way of blending parts of two clips together seamlessly I'll write it.

Any suggestions?

[SansGrip]

For generation of samples of a normal variable, the math folks use the Box-Muller transform

This is what I'm using right now, at least the fast version of it (I think it's called the Polar method).

Thanks for showing me mathworld -- I think it'll come in very handy :).

[SansGrip]

q1: would it be possible to tune the extent of noise (intended to add) somehow to the average luma round the pixel a/o to the luma noise of the block/frame ?

Sure. It could be weighted based on luma quite easily, though there would of course be a performance hit.

q2: would it be possible to draw the noise somehow from the picture?

Check out v0.3. It's new sharpen method will do exactly that: amplify the already existing details instead of simply making new noise.

imho, noise is a part of the picture holding relevant information.

Well, technically it's not. Signal holds the information we want, and noise is the information we don't want ;). That said, I know what you mean. Noise, as far as the encoder is concerned, adds detail, and that seems to make them behave better under certain circumstances.

[Xenoproctologist]

How about checking so that pixel values don't roll over/under when (pixel+noise) < 0 || (pixel+noise) > 255, m'kay?

[SansGrip]

How about checking so that pixel values don't roll over/under when (pixel+noise) < 0 || (pixel+noise) > 255, m'kay?

Oops ;).

Edit: Removed links to old version.

[Xenoproctologist]

Originally posted by SansGrip
Oops ;).

Heh. Groovy. Now I can see if this helps get rid of the DCT Blocks From Hell in this anime encode I'm doing.

Well...after I see what removing the dot-crawl does, anyway. (Guava! Wai!)

[Kaiousama]

Hi, thanks for the release, i've had the time to do only a few tests, and for now i've had:
- The filter is painfully speedy (i've tried even to call it more times into the same script and the speed reduction is very acceptable, yatta!)
- The Blur mode is very very useful for anime: instead of a constant blur in the image it'll be possible to have a constand or scaled blur gradient simply creating a combo of blockbuster with a decreased invert threseold :) (the same concept is obviously applicable on sharpening)
- The noise adding was the weak point in my tests (*ouch*) because it hasn't removed at all the DCT blocks with standard options, i have to make more tests on it to be sure... can you explain a little more in detail how the noise is added in every block?

Bye

[SansGrip]

The filter is painfully speedy

Glad to hear the optimizations worked ;).

can you explain a little more in detail how the noise is added in every block?

Basically the random number generator produces a value based on the mean and variance you give it. The mean will generally be zero otherwise you'll introduce an overall change in brightness, but the variance depends on how much noise you want to add. When variance=1 68% of the values will be between -1 and 1, 95% between -2 and 2, and 99% between -3 and 3. Values outside this range will be generated, but very rarely. The values are simply added to the current value for luma or chroma.

So to vary the "strength" of the noise, leave the mean at zero and change the variance.

[iago]

@SansGrip and MaTTeR

Well, imho, unlike the lumoff/UnFilter combo, BlockBuster is a great noising filter to eliminate DCT blocks throughout the "whole encode", regardless of the luma range, with its "block size", "detail threshold" and "noise strength" parameters, which all can be configured to suit the needs and taste of the user.

As far as I can tell, the lumoff=-2/UnFilter combo that I've been using for a long while in my encodes is mostly effective against blocks in "dark areas", not in the other parts of the encode.

Therefore, I guess, rather than considering BlockBuster as an alternative to the lumoff/UnFilter combo, it might be much better to regard/use it as a complement to it (or vica versa of course ;)), since in my limited BlockBuster tests with XviD (using b-frames) it seems that BlockBuster is not as effective against DCT blocks in dark areas as the lumoff/UnFilter combo. However, as I said above, with its all-configurable great parameters, it might do miracles in other parts of the encode where the lumoff/UnFilter combo fails.

So why not give them together a try? I absolutely will! ;)

Btw, I agree with Matt that 4 seems to be a good value for detail threshold. And now I'm trying "block_size=4" together with "lm=0, lv=2.5, cm=0, cv=0.5". However I have no idea how it will effect compressibility, which I don't want to sacrifice completely indeed. Let's try and see! ;)

Anyway, I want to thank SansGrip one more time for his great effort and the color (noise :D) he has introduced to the encoding scene! ;)

best regards,
iago

[SansGrip]

@iago:

I've been waiting impatiently for feedback from you, the "dark block master" ;).

since in my limited BlockBuster tests with XviD (using b-frames) it seems that BlockBuster is not as effective against DCT blocks in dark areas as the lumoff/UnFilter combo.

Did you try lumoff and Blockbuster together, without UnFilter? If that helps I'll add a luma offset parameter, perhaps configurable so it only offsets luma within a certain range (so you can apply it only to areas already dark).

By the way, have you tried the other methods (sharpen and blur) yet? I'm not sure how useful they are.

However I have no idea how it will effect compressibility, which I don't want to sacrifice completely indeed. Let's try and see! ;)

Yes, let me know -- I'd be very interested to hear.

Anyway, I want to thank SansGrip one more time for his great effort and the color (noise :D) he has introduced to the encoding scene! ;)

heheh thanks. It feels strange working on one filter to add noise and one filter to take it away simultaneously though ;).

[iago]

@SansGrip

Due to many reasons, one of them being XviD and AviSynth 2.5 development going lightning fast recently ;), I haven't been able test BlockBuster extensively yet.

I have just started doing some compressibility tests and playing with the common and method-specific parameters to achieve some sort of compromise between effectiveness and compressibility. I guess, soon I'll start more specific tests as well. For the time being, I can only say that I'm very pleased to see BlockBuster work with MPEG-4 successfully and do its job against DCT blocks throughout the "whole" encode.

Also, having a (configurable) luma offset parameter with the filter can really be interesting.

Btw, there is an important thing I wonder: Can we expect an AviSynth2.5 / YV12 implementation of BlockBuster, which would really be great imho? ;)

best regards,
iago

[MaTTeR]

Originally posted by iago
Btw, there is an important thing I wonder: Can we expect an AviSynth2.5 / YV12 implementation of BlockBuster, which would really be great imho? ;)This is actually why my Blockbuster testing hasn't taken place yet. Had too many YV12 rips queued up:)

Glad to hear Iago had some success with BlockBuster. Hmmm, the luma offset affecting the "detailed_threshold" areas would be very cool indeed.

[SansGrip]

Due to many reasons, one of them being XviD and AviSynth 2.5 development going lightning fast recently ;), I haven't been able test BlockBuster extensively yet.

hehehe that's fine, I was just kidding ;).

For the time being, I can only say that I'm very pleased to see BlockBuster work with MPEG-4 successfully and do its job against DCT blocks throughout the "whole" encode.

I'm very glad to hear that it has an affect on MPEG-4 as well -- I was beginning to think it was only useful against MPEG-1...

Also, having a (configurable) luma offset parameter with the filter can really be interesting.

MaTTeR's suggestion (luma offset affecting only areas within detail_threshold) is an interesting one, though my initial thought is that the offset would need to be small in order not to be noticible (I guess -2 is small enough).

Btw, there is an important thing I wonder: Can we expect an AviSynth2.5 / YV12 implementation of BlockBuster, which would really be great imho? ;)

If Blockbuster is definitely working for you then I'll certainly implement a YV12 version. I'll need to figure out how to make 2.5 co-exist with 2.0 first though :).

[SansGrip]

By the way, there's an interesting thread (http://www.kvcd.net/forum/viewtopic.php?t=1604) on the KVCD forum with reports of people getting good results.

</shameless plug> ;)

[MaTTeR]

Wow, Blockbuster's speed is obviously much better than AddNoise now. Using BB, I hardly lost 1FPS during encodes:D

I've been running a few Quant=2 tests with Koepi's latest dev build(XviD) from today. So far I have to agree with Iago, overall the DCT blocks are being reduced. My results were a bit unexpected using Iago's initial parameters in regards to compressibility. For a 6min DVD clip at Quant=2, I'm showing the file size to increase about 11MB. Now, I might have expected this if using method="sharpen" but my results show almost the same file size if blurring or sharpening.
LoadPlugin("C:\Program Files\AviSynth2\plugins\MPEG2DEC3.dll")
mpeg2source("D:\night.d2v",lumoff=-2,iDCT=2,cpu=1,moderate_h=35,moderate_v=55)
crop(4,4,712,472)
TemporalSoften(1,5,0)
Blockbuster(method="sharpen",detail_threshold=6,lm=0,lv=2.5,cm=0,cv=0.5,cache=384)
LanczosResize(624,336)
File size= 63,498K

I then changed the method to blur and the file size came out as 63,520K. As you can see not much difference in the file size between the two. For reference, I encoded the clip without Blockbuster and the size is 56,726K.

I then viewed all the clips using TV-Out to see the visual difference. I have to admit that the clip with method="sharpen" looked best overall. Using "blur" seems to have helped a bit but as a side effect the blocks seemed to jump around a bit more so again visually I chose the "sharpen" clip. My source isn't the best for this test so I've got 3 others here that I'll try.

BTW- I didn't even try the tests without lumoff=-2, I was just assuming it could only help the situation. It would be interesting to see how the clip looked if the affected detail_threshold areas were the only part of the frame with luminance lowered though. Anyway, I'll post more comments later as my tests complete. Great work SansGrip, looks as though MPEG-1 certainly benfits from BlockBuster after reading user comments over at Kwag's.

Edit- I understand why my file size increased now. I obviously was adding noise while blurring/sharpening:D Don't mind me, I'll continue my aimless ramblings later after more thorough testing.

[MaTTeR]

Just finished up with more XviD Quant 2 tests on different DVD sources. The tests confirm my initial visual thoughts, method="sharpen" indeed gives a more overall eye pleasing encode compared to "blur" IMO.

The most impressive thing I've noticed is the file size doesn't increase on some movies but instead decreases for some reason:confused: This is a very good thing but I'm not sure why it happens. Some of my clips did increase when using "sharpen" but only very slightly, maybe less than 1%. Again, my results are for MPEG-4 only; in regards to MPEG-1/2(prefer 1) I've never really had a problem with DCT blocks. Maybe that's because the KVCD templates work so well:p

One last thing, using lumoff=-1 or even 2 definitely seems to help further reduce DCT blocks even more depending on the source material. I'm not sure how this affects the overall compressibility if any, I suspect it should help some in that regard. I have not tested adding noise with BlockBuster, perhaps Iago had some luck with it. Right now I'm happy with what "sharpen" is producing, good stuff indeed.

[SansGrip]

Blockbuster(method="sharpen",detail_threshold=6,lm=0,lv=2.5,cm=0,cv=0.5,cache=384)


When using method="sharpen" only the common parameters (detail_threshold, block_size, invert) as well as the sharpen-specific parameter (strength) are used. If you use sharpen or blur, no noise is added so you don't need the mean, variance, and cache settings.

[SansGrip]

method="sharpen" indeed gives a more overall eye pleasing encode compared to "blur" IMO.

That's what I would predict. I'm surprised that blur even works at all for DCT block removal. It might work on mosquito noise if combined with invert though :).

I have not tested adding noise with BlockBuster, perhaps Iago had some luck with it.

I think you'll find even better results from adding noise, depending on the source. There's only so much you can do with sharpen before things start to look artificial.

Glad to hear you're getting some pleasing results. I'm going to implement lumoff tonight and hopefully will have something ready by tomorrow :).

[MaTTeR]

Originally posted by SansGrip
[i]I think you'll find even better results from adding noise, depending on the source. There's only so much you can do with sharpen before things start to look artificial.

Think I'll do some some tests with doing both sharpen and adding noise. I need to call the filter twice for this right? I tried doing it on one line but no luck. heheh

I've yet to do any full 2-pass encodes yet but I'll set some up before going to sleep. I'm anxious to see an entire movie with Blockbuster. Thx for looking into the luma feature.

[SansGrip]

MaTTeR: Think I'll do some some tests with doing both sharpen and adding noise. I need to call the filter twice for this right?

Yep.

I tried doing it on one line but no luck. heheh

I was thinking of adding this functionality but couldn't come up with a way of doing it without confusion. I've been talking with the guys on the KVCD forum and a couple of them were confused by the docs with regards to which parameters go with which method, so I think making the calling syntax even more complicated might be a bad idea ;).

Thx for looking into the luma feature.

I roughed out in my head how it'll work last night while I was watching TV, so I'm going to code that now, along with better parameter-checking, while I wait for the cable guy (free installation and $1 for two months of basic cable... You'd have to be a moron to say no to that :D).

[SansGrip]

I just released 0.4 (http://www.jungleweb.net/~sansgrip/avisynth/Blockbuster-0.4.zip) (and source (http://www.jungleweb.net/~sansgrip/avisynth/Blockbuster-0.4_src.zip)) with the following changes:

* Added luma_offset and luma_threshold parameters
* Split detail_threshold into detail_min and detail_max
* Removed redundant invert mode

See the documentation (http://www.jungleweb.net/~sansgrip/avisynth/Blockbuster-readme.html) for more info.

Have fun :).

[iago]

@SansGrip

You may be sure that BlockBuster works and does a pretty good job with MPEG-4 too ;). So, parallel to the great AviSynth 2.5 development, it will really be nice to have a YV12 version of the filter. Thanks a lot for working on it!

Btw, to see its effect on blockiness (which is absolutely more visible with SBC encodes) and maybe pushed by some feeling of nostalgia as well ;), I also did a "Nandub SBC" encode using BlockBuster with some experimental/heavier settings such as (method="noise",block_size=4,detail_threshold=4(in some parts 2),lm=0,lv=3,cm=0,cv=0), which gave me a very good encode as a result, though I'm sure it decreased compressibility considerably ;).

Well, in short, I'm looking forward to the YV12 version of BlockBuster actually!

best regards,
iago

edit: wow! a new version has already arrived while I was typing! ;)

[SansGrip]

You may be sure that BlockBuster works and does a pretty good job with MPEG-4 too ;).

That's a relief :).

gave me a very good encode as a result, though I'm sure it decreased compressibility considerably ;).

heheh well like I say in the docs, sometimes it's worth it ;).

Well, in short, I'm looking forward to the YV12 version of BlockBuster actually!

I'm going to check out the YV12 threads to see if it's possible to have 2.0 and 2.5 co-existing on the same Windows install. Unfortunately my current hard drive situation means I can't do another install just for YV12 development :(.

[MaTTeR]

Originally posted by SansGrip
I'm going to check out the YV12 threads to see if it's possible to have 2.0 and 2.5 co-existing on the same Windows install. Both versions co-exist fine here for the past few weeks. I simply have VdubMod 1.4.11 with it's own AVS 2.5 dll in the same folder. Otherwise the rest of the system is still using AVS 2.06. You should have no problems at all.

[SansGrip]

Both versions co-exist fine here for the past few weeks.

Excellent news, and a clever solution.

YV12-enabling all my filters for 2.5 is now high on my TODO list, just below "Find out how the 2.5 API differs from 2.0" and "Learn what YV12 is" ;).

[iago]

@SansGrip

All I can say, based on my ongoing tests, is that this filter is a fantastic tool, with its highly configurable parameters, which can all be tweaked according to the nature of source, to get decent encodes with the least possible DCT blocks.

"luma_offset" and "luma_threshold" parameters are just great to keep the original levels of the picture untouched, with a capability of affecting only the dark parts of the encode as desired (where the default 25 or something like 20 would be reasonable values imho).

"detail_min" and "detail_max" are also amazing parameters, which can be set as to suit the needs of the user, and also more comprehensible than the previous inverty ;) ones.

And finally, you can determine the strength of the noise to be applied to luma and/or chroma planes (while I prefer to use only luma noise and set cm and cv to 0).

Absolutely great! Keep up your great work, man! I really appreciate it ;).

(And please do not delay much releasing the YV12 version of this magic tool, which might even make me change my long-time favourite combo ;))

regards,
iago


edit: soon, I'll report some test results as well.

[High Speed Dubb]

What amount (variance) of noise addition are people finding to be effective at breaking up blocks for various kinds of MPEGs?

I&rsquo;m asking for my own purposes, since this will help me come up with defaults and docs for noise filtering. But it&rsquo;ll probably be useful for Blockbuster, too.

[SansGrip]

@iago:

Wow! Thanks for the high praise -- I'm very glad this finally evolved into something useful :).

(And please do not delay much releasing the YV12 version of this magic tool, which might even make me change my long-time favourite combo ;))

Well, I just ported NoMoSmooth (see other thread) to YV12, and am waiting for feedback from 2.5 testers on whether it even works properly.

Once I know I understand how it works I'll make YV12 versions of all my filters, including (first, of course :)) Blockbuster.

[SansGrip]

What amount (variance) of noise addition are people finding to be effective at breaking up blocks for various kinds of MPEGs?

I find it varies significantly depending on the source. For MPEG-1 I've received good reports with anything from 1 to 3.

[kwag]

Originally posted by High Speed Dubb
What amount (variance) of noise addition are people finding to be effective at breaking up blocks for various kinds of MPEGs?

I&rsquo;m asking for my own purposes, since this will help me come up with defaults and docs for noise filtering. But it&rsquo;ll probably be useful for Blockbuster, too.

Hi High Speed Dubb,

Try this combination in your script:

Blockbuster(method="sharpen", detail_min=60, detail_max=99, strength=20) # Only sharpen material above 60%.
Blockbuster(method="noise", detail_min=1, detail_max=20, lv=1.5) # Only apply noise below 20% complexity.
# Mid tones (frequencies) from ~21% to ~59% will not be processed.

Be sure to use Blockbuster 0.4 for this.

-kwag

[High Speed Dubb]

Cool &mdash; Thanks for the estimates. This means I should start recommending lower Stability values for Peach Smoother, at least if the output is going to be used for MPEG-1 encoding.

At its default stability value it tends to drop the variance by about a factor of 13. With broadcast material that gets you pretty near to variance = 1.0, which is already on the low side. Cleaner sources would end up with too little noise.

You could use BlockBuster to add noise back in afterward. That would let you increase it especially in low contrast areas. On the other hand, it does sound slightly strange to do that if the problem could have been avoided just by using less aggressive smoothing.

[SansGrip]

You could use BlockBuster to add noise back in afterward. That would let you increase it especially in low contrast areas. On the other hand, it does sound slightly strange to do that if the problem could have been avoided just by using less aggressive smoothing.

I've been playing with this in-between YV12 investigations and spending a few minutes per day with my family ;).

I've found that using NoMoSmooth (which IMHO with the correct settings leaves just about as noiseless-yet-detailed a source as I've seen) followed by Blockbuster to put noise back in to insufficiently detailed areas and tame those DCT blocks produces extremely good results.

Unfortunately I've not had the chance to really give this a thorough testing. I've slotted it in fairly high on my to-do list though.

[drebel]

SansGrip,

Sorry for the delay, but i had to take an aspirin after rereading the whole thread...
A few comments for Blockbuster now:

1.It achieves its goal to reduce dct blocks.To be exact ,it almost eliminates them.It can't be considered as a complement to unfilter/lumoff combo(i've been using this method for quite sometime now and there is a BIG difference in the final result.I can confirm the sucess of the combo when the prob is not that obvious.)

2.The only method that gave good results so far was "sharpen".("blur" makes no difference and "noise" is second better in order)

3.Block_size really depends on the problem;a small size selection had better results in my encodings(especially in dark areas with high contrast

4.About "detail" level anything else but processing the WHOLE frame(detail_min=1,detail_max=100) gave artifacts:i remember a roof that was flashing(small ~white~ blocks allover the roof in a very low motion frame>There were also some white blocks around vertical-most of the time-edges.I dunno but it might have to do with the detail detection params(i tried everything in between) because the problem was solved by processing the whole frame

5.After iago's (and the rest of the community) suggestion i applied luma_offset=-2 .Great result!...except for the threshold.Everything less than 100 made SOME blocks look darker->more distinguishable to their neighbours(like tiny dots)

6.Strength is another thing worthy of mentioning.The default or less has no probs and produces clear frames.Anything above 30 gives "grid" effect and "scaling" artifacts on every edge(hope i make some sense..)

7.About compressibility i can't say much.I had to encode those movies at constant quant 2 with crystal clear final result but no predictability at all...

8.Given the final results and the minor speed penalty,i feel the need to congratulate you for all the effort.

best regards,
george

[drebel]

Ps:I still cant get acess to your site.Think i retry later...

[SansGrip]

Ps:I still cant get acess to your site.Think i retry later...

Should be fixed now. Sorry about the downtime.

(I'm in the middle of porting Blockbuster to YV12 right now, but I'll respond to your previous post later today :).)

[drebel]

Fix confirmed.Thx

[SansGrip]

Does anyone actually use chroma noise, or thinks it's at all useful? My tests show the benefits of the noise method comes from the luma noise, with chroma noise not seeming to make any difference.

It would be much simpler to port to YV12 if I got rid of chroma noise and just had all the methods working in luma. If I don't hear any calls for it to be retained in the next couple of days I'll take it out :).

[iago]

@SansGrip,

An immediate vote from me: throw it out and port to YV12 asap! ;) I'm looking forward to it!

best regards,
iago

[MaTTeR]

I agree with iago, forget about it and let's see how the YV12 port does:)

Edit- drebel has some valid points above and seems to match my observations as well though I never tried processing the entire frame. Can't wait to test the new build when it comes out though.

[SansGrip]

Ok, I think two votes is enough. I'm as impatient as anyone to get the YV12 port out ;). Working on it for 0.5, then for 0.6 we'll hopefully thrash out some ideas to address the issues raised above.

[SansGrip]

Here's 0.5 with minor changes for the Avisynth 2.0/YUY2 version but now includes an Avisynth 2.5/YUY2/YV12 version.

Changes:

* Removed chroma noising (and cm and cv parameters)
* Renamed lm and lv to mean and variance respectively
* Some small optimizations here and there

Even though not much has changed in the Avisynth 2.0/YUY2 version, I'd appreciate if everyone could upgrade in order to test this one works okay. You'll probably find it a bit faster too :).

Have fun.

Edit: Removed links to old version.

[MaTTeR]

Wow, that was really fast. Well of course I just loaded an encode in the queue and anxious to see what happens.

luma_offset & luma_threshold could be a God send if they work like I'm thinking they do. I'd much rather have the threshold areas darkened than the entire frame, this should be cool. Thx again!

[iago]

@SansGrip

Thanks a lot! I'll start playing with it immediately! ;)

@MaTTeR

What settings are you considering to try? I wanna compare it with mine! ;)

regards,
iago

[SansGrip]

2.The only method that gave good results so far was "sharpen".("blur" makes no difference and "noise" is second better in order)

I assume you're encoding with XViD. From what reports I've had I'm beginning to think that sharpen works better for MPEG-4 than noise, probably because MPEG-4 is so much better wrt DCT blocks than previous verions.

On the other hand, reports from others and my own experiences indicate that adding noise is the preferable solution for MPEG-1.

3.Block_size really depends on the problem;a small size selection had better results in my encodings(especially in dark areas with high contrast

Interesting. I've still not tried changing the block size, and I guess I really should ;). Can you be more specific about what sizes seem to work best with what source material?

4.About "detail" level anything else but processing the WHOLE frame(detail_min=1,detail_max=100) gave artifacts

This is quite possibly related to the sharpen method. The algorithm used is unsophisticated and fairly harsh, and better results would probably be achieved with an unsharp mask method. As far as light noise goes (variance=1-2) I definitely don't notice any artifacts in the encode (other than that the DCT blocks are gone ;)).

:i remember a roof that was flashing(small ~white~ blocks allover the roof in a very low motion frame>There were also some white blocks around vertical-most of the time-edges.

Sounds like the sharpening algorithm.

5.After iago's (and the rest of the community) suggestion i applied luma_offset=-2 .Great result!...except for the threshold.Everything less than 100 made SOME blocks look darker->more distinguishable to their neighbours(like tiny dots)

I wondered if that would be the case. Do you notice the same difference (and benefit) when using luma_offset=-1? As for the threshold, do you mean when you set it to 100 there are no noticible brightness changes?

6.Strength is another thing worthy of mentioning.The default or less has no probs and produces clear frames.Anything above 30 gives "grid" effect and "scaling" artifacts on every edge(hope i make some sense..)

Again, the algorithm is less than ideal. The sharpen and blur methods were really just a quick experiment, but if people are actually using them I'll take a look at improving them (maybe selecting a different matrix for sharpen) and also implementing an unsharp mask method.

8.Given the final results and the minor speed penalty,i feel the need to congratulate you for all the effort.

Thanks :). It's been a fairly long and frustrating process, but it's nice to see people actually using it for its intended purpose ;).

[SansGrip]

Wow, that was really fast.

I did say without chroma processing it would be simple :D.

Well of course I just loaded an encode in the queue and anxious to see what happens.

I'm anxious too! hehe

luma_offset & luma_threshold could be a God send if they work like I'm thinking they do.

Yet another feature I've not tried. I'm hoping that -1 doesn't result in noticible darkened blocks like drebel reported with -2. If it does I'm going to have to rack my brains to come up with a way of reducing the effect.

[SansGrip]

I'll start playing with it immediately! ;)

heheh excellent. Let me know what you think.

[iago]

hehe ;)

And know what! Just before BlockBuster I have FluxSmooth in my test scripts too, with some more aggressive settings than the defauls! ;)

iago

[MaTTeR]

Originally posted by iago
@MaTTeR

What settings are you considering to try? I wanna compare it with mine! ;)

Well the 4:3 source I'm testing has quite a few noticeable blocks with lots of scenes containing walls:( Overall this movie is soft and not very sharp so I thought I'd use UnFilter for faces and main objects. Walls are becoming a dirty word for me these days. lol So I started my encode with the following-
mpeg2source("D:\DIvX RIPs\Apostate\apost.d2v",iDCT=2)
crop(4,4,712,472)
Convolution3d(preset="movieHQ")
Blockbuster(method="sharpen",block_size=4,luma_offset=-1,luma_threshold=25,detail_min=1, detail_max=16)
Unfilter(3,3)
BicubicResize(528,392,0,0.4)
undot()
Limiter()

[cult]

@MaTTeR
what is Limiter() ??

[SansGrip]

And know what! Just before BlockBuster I have FluxSmooth in my test scripts too, with some more aggressive settings than the defauls! ;)

Awesome! What settings? How do you like it?

[MaTTeR]

cult,

From sh0dan
If anyone is testing the alpha, you could also try the new version, with the new "limiter" filter. It works like SansGrips Legalclip, except you can specify parameters, and it's much faster (ISSE optimized).

Usage:

Limiter(minimum luma, maximum luma, minimum chroma, maximum chroma)

Default parameters are the same as LegalClip (default YUV range). Funny thing is that it can also be used for color correction.

In short it's "clamping" the color scale for TV output(16-235/240) respectively. I haven't tested it that much yet, sort of waiting on the subtract filter to see the difference.

[drebel]

SansGrip,

million thx for the fast reflexes...

Ok.Back to business!(eventhough the aspirin didnt help much..)

For "torture" movie i selected the Spanish production "Todo Es Mentira" with Penelope Cruz.Garbage quality, dark,blocky and slightly blurry (but no other "noise-like" artifacts.

I've tried many times to encode those vobs with xvid,usind Convolution3d and many many combination of filters.The result was not good enouph : one has the feeling that he 's waching the movie through a thick glass(hyperfiltered to avoid those blocks,but with no luck).PP didnt help either and the lack of details was obvious.

It gives me great pleasure to test it with Blockbuster in YV12.I'll try to focus on "noise" method this time.About luma_offset=-1,the effect i was talking you about was softer but,from my point of view, it's better to pinpoint a problem than to cover it if we're looking for solutions(sorry,i'm not a coder... )

For now ,a first asharp tryout is in the "oven" waiting to face the jury.So the only thing i can do for now is preparing the .avs and some partial-testing.:eek:


best regards to everyone prelonging our insomnia,
george

[MaTTeR]

Well my full 2-pass encode finsihed awhile ago and watched some of the scenes that had been giving me problems. All I can say right now is that I'm very encouraged by what I seen:)

Overall the encode was nice and clean with very few left over blocks moving around on walls. I noticed a nice side effect as well, BlockBuster seems to help get rid of the XviD B-Frames noise smearing problem. I've encoded this movie 8 times in 3 days and this new BlockBuster is by far the most pleasing to my eyes. I think with some more agressive settings I might be able to knock this movie out after all. I just wanted to drop this quick note before I went to bed. I'm very anxious to test more settings and hear what others were testing with! Great work SansGrip!

Setting this up in the queue now-
crop(4,4,712,472)
TemporalCleaner(2,2)
Blockbuster(method="sharpen",block_size=3,luma_offset=-1,luma_threshold=28,detail_min=1, detail_max=25)
Unfilter(3,3)
BicubicResize(528,392,0,0.5)
undot()
Limiter()

Edit- Oh and I'm also very curious to see where we can get optimal quality by placing BlockBuster in the chain. I'm starting to think maybe it might work better before a smoother type filter now, I'll try it tomorrow.

[kilg0r3]

i just had a look on the american pie screen shots with the different noise settings. as far as i can say, it seems to me that the blocks are still there although they will be less visible with noise when watching the movie. Still, i don't see the advantage to add noise during the encode sacrifcing bitrate and consequentially causing higher quantizers to be used when it is possible to achieve the same effect by using ffdshow's noise feature, which admittedly needs some improvement.

the only case that might justify using a noise generator is the black blocking issue. yet, for this, it would be very nice to have a filter that adds the noise only in areas with a low luma value; 20 and less.

[MaTTeR]

Why add more noise? Well for me I absolutely refuse to use Post Processing to cover artifacts up for a crap rip. I wan't the best possible encode from the start even if it does require me to use more space. Why not have the best if you can afford it?

@all
I really don't see any problems using the luma offset parameter. In fact, when darkening it's very difficult to see any change at all with -1 or -2. The only way I was able to notice any difference in the Vdub preview window was to use a crazy setting such as -40. So it seems safe to use but I'm not sure how much it's going to help us.

My 2nd encode finishe dup over night and I'm still fighting the same problem for now...ringing artifacts. This seems to be a side effect of using YV12 but it's also amplified when using BlockBuster IMO. Has anyone else seen this yet?

My final thought is that the second script I posted above is working much better on this movie. More agressive settings were needed but that might be depending on the rest of our filter chains. In theory, UnDot or UnFilter might just be UnDoing our BlockBuster settings:D

[SansGrip]

My 2nd encode finishe dup over night and I'm still fighting the same problem for now...ringing artifacts. This seems to be a side effect of using YV12 but it's also amplified when using BlockBuster IMO.

When you say ringing artifacts, what exactly do you mean? Can you post a framegrab?

My final thought is that the second script I posted above is working much better on this movie.

I have my doubts about using smoothers etc. after Blockbuster, because sharpening then smoothing is never going to be as good as simply sharpening less in the first place. And wrt to adding noise then smoothing, the second step is never going to make the noise more random, always less. Less random means perhaps noticible patterns appearing in what used to be noise. The brain is very good at ignoring noise and very good at spotting patterns :).

[SansGrip]

i just had a look on the american pie screen shots with the different noise settings. as far as i can say, it seems to me that the blocks are still there

DCT blocks will always "be there", because that's what an MPEG frame is made up of. However, the blocks in the grabs with noise added contain more details, are internally graduated consistently with the entire wall, and have much less noticible boundaries between each other.

Still, i don't see the advantage to add noise during the encode sacrifcing bitrate and consequentially causing higher quantizers to be used

Generally sacrifices are made for a reason, and this is one of those YMMV things. If you think encodes look fine with noticible DCT blocks all over smooth surfaces then that's great. I personally don't think they look fine, I think they look awful, and they jump around and flash on and off in what I find to be a most distracting manner. Thus I am willing to raise the bitrate slightly to improve quality by dithering.

Yes, more bits will be used but since the problem is caused by over-compression in the first place, that's to be expected.

when it is possible to achieve the same effect by using ffdshow's noise feature, which admittedly needs some improvement.

First, ffdshow doesn't work on my standalone DVD player. Second, not everyone has it. Third, are you really arguing that destroying details then faking them later is better than keeping the existing details in the first place? That doesn't make sense to me.

This argument could easily be extended to resolution. Why use all those bits to encode at 640x480 when we could encode at 320x240 and just use our player's double-size feature? Of course, because it doesn't look as good. As always, the trade-off is quality for bits.

Dithering is used in many places to improve quality. Almost all music is recorded, edited and processed with at least 20 bits these days, but CDs are only 16-bit. When this 20-bit+ music is transferred to CD it is dithered during conversion. This retains more details in the material --noticible ones -- than simply truncating to 16 bits, and is standard industry practice. Admittedly it's a much more targeted application of noise (at least in the more sophisticated/expensive systems), but then a lot more research has been done into audio dithering than video dithering -- at least, research I have access to :).

[MaTTeR]

Originally posted by SansGrip
[i]When you say ringing artifacts, what exactly do you mean? Can you post a framegrab?

I have my doubts about using smoothers etc. after Blockbuster Yeh, I'll find a good example of ringing and post it tomorrow after I get back from work. Ringing artifacts are basically what appears to be large multi-colored rings most commonly seen on darker stable backgrounds. It's hard for me to describe but a picture is worth a thousand words:)

Your right, I've ran multiple tests tonight and indeed I get better visual results when using smooothers before BlockBuster. Also, I'm convinced drebel was correct in using the smallest block size. In fact if I could go lower than 3 I would. For whatever reason larger block size are showing diminished returns. Well here's my 6th revision of my script for this 4:3 flick I've been encoding-
mpeg2source("D:\DIvX RIPs\Apostate\apost.d2v",iDCT=2,cpu2="oxoxoo",moderate_h=50,moderate_v=70)
crop(4,4,712,472).undot().Blockbuster(method="sharpen",block_size=3,luma_offset=-2,luma_threshold=30,detail_min=1,detail_max=35).BicubicResize(512,380,0,0.4).Limiter()

Edit- Grhh...when will Mozilla ever fix these friggin line breaks?!?! Doom9 will get a kick out of this if he spots it:D

[drebel]

Hope i waited long enough for all other tests to be completed before posting some new hints....
About blockbuster in YV12 now:

- Algo errors seem to be fixed!No more flashing pixels,regardless of the params selection(thx SansGrip).Looks like you didnt just port the filter to YV12...:)

- Method "sharpen" is still giving the best result and ,of course ,sharper image(no nedd for extra sharpenning most of the times)."Noise" is reducing artifacts but also gives a "motion" on walls etc.This effect could be used to give a retro feeling to certain B&W old movies while keeping soome of the noise(director's opinion)

- Block size should,i think, be kept as low as possible when encoding movies.This could prevent several "mistakes" at the selection of the perfect parameters to be shown clearly at the final result(the effect of tiny dots and the chesstable-like grid are far less obvious).This doesnt neccessary mean that bigger blocksize is unuseful.Certain anime films(not tested)could benefit from size 6 or 8(reminds me of another old thread about "solid coloured areas"...

- As detail now is(?) without errors,a detail_min=1,detail_max=30 is enough.When human skin contains blocks ,detail_max could be raised to 40 or even 50

- Strength more than 40 is aggressive and not beautiful (for my eyes)


Edit- Oh and I'm also very curious to see where we can get optimal quality by placing BlockBuster in the chain. I'm starting to think maybe it might work better before a smoother type filter now, I'll try it tomorrow.

- I totally agree with MaTTeR on this one.I just wanted a second opinion and your test just gave me what i needed.Thanks :)


-Your opinion is always a great help and a step forwards.The filter is now mature enough for everyone to use it.


best regards to all,
george

[SansGrip]

- Algo errors seem to be fixed!No more flashing pixels,regardless of the params selection(thx SansGrip).Looks like you didnt just port the filter to YV12...:)

I think that's all I did ;). Would you be able to do my a favour and try a ConvertToYUY2() before Blockbuster in your script? If the flashing pixels are still gone then I guess I did fix a bug too...

"Noise" is reducing artifacts but also gives a "motion" on walls etc.

I find this happens when the variance isn't set correctly, either too low or too high.

Block size should,i think, be kept as low as possible when encoding movies.

Interesting. I'll have to remember to include this in the docs.

Strength more than 40 is aggressive and not beautiful (for my eyes)

Yes, it's a pretty powerful sharpening matrix.

The filter is now mature enough for everyone to use it.

Well, normally I would take this kind of remark to mean next release I should go ahead and bump the version to 1.0, but I just got a book on assembly language so I don't want to come out of the beta stage just yet...... :D

[SansGrip]

In fact if I could go lower than 3 I would.

The only reason I set that as a minimum is because the sharpen/blur matrices are 3x3. Since you're using a block size of 3, only the very centre pixel is being affected. I'm surprised it's working like that, tbh ;).

Hmmm, this gives me an idea for an algorithm that would process every pixel and not require the image to be split up into blocks at all...

I'll have to think about that one :).

[iago]

@SansGrip and all

I must admit that I have also got the best results with the filter using the "sharpen" method so far. I agree that a strength value of more than 40 (imho default 25 is also fine) would really hurt compressibility a lot and lead to unpleasant results.

As for the luma_offset and luma_threshold parameters, I haven't experienced any problems with "luma_offset=-2", but if you set the luma_threshold too high (>25-30) it may lead to some artifacts at shady parts, some sort of very visible annoying border lines between the luma_offset=-2 parts and the neighbouring parts of the picture to these areas. Personally I prefer to use a luma_threshold of 20 to avoid this side-effect.

Btw, I'm still sceptical about using BlockBuster "before" smoothers, since this "may" totally undo what BlockBuster will achieve (?).

Regards and many thanks again for this powerful filter! Imho, with its current situation and its great flexibility it's more than ready for real encodes! ;)

iago

[drebel]

Would you be able to do my a favour and try a ConvertToYUY2() before Blockbuster in your script? If the flashing pixels are still gone then I guess I did fix a bug too...

Done!No flashing pixels anywhere......
Good luck with assembly ;)
If your wish is to incorporate any new ideas to the already great filter,no complaints from here :D

regards,
george

[SansGrip]

Done!No flashing pixels anywhere......

Excellent. I must admit I've never had any reports of flashing pixels from anyone else -- perhaps it was a problem with the small block sizes.

Good luck with assembly ;)

It's coming along. I'm starting with LegalClip, my simplest filter, and I think I've come up with something significantly more efficient than the compiler generated from the C++ code (but no MMX/SSE yet). Blockbuster shouldn't be too difficult to rewrite in assembler -- or at least should be easier than the smoothers.

If your wish is to incorporate any new ideas to the already great filter,no complaints from here :D

I have an idea involving a moving window instead of discrete blocks. I need to let it percolate through my brain for a few days before I look into implementing it though :).

[SansGrip]

Here's a new release which fixes a bug and adds a parameter. It'll probably be the last 0.x version -- I think I'll bump it up to 1.0 for the next release since lots of people are using it and there are very few reports of any problems.

Changes:


0.6 - Added seed parameter. Fixed a bug that might have been responsible for the access violations when using with high resolutions and the default cache size.
The new seed parameter allows one to specify the seed value for the pseudo-random number generator in the noise method. Previous versions always produced a slightly different file size over multiple encodes using the same settings (obviously because of its random nature). By specifying a seed value one can force Blockbuster to produce the same "random" noise each time it is run, which helps in certain applications that require a more predictable outcome.

As usual the source and docs are on my site.

Please let me know if you have any problems.

Edit: Removed links to old version.

[onesoul]

Originally posted by MaTTeR
mpeg2source("D:\DIvX RIPs\Apostate\apost.d2v",iDCT=2,cpu2="oxoxoo",moderate_h=50,moderate_v=70)

I shouldn't ask here but I am too curious, what does those parameters mean? btw what does limiter() do?

[SansGrip]

Originally posted by onesoul
btw what does limiter() do? Limiter is the Avisynth 2.5 built-in version of LegalClip -- it clamps the pixel value ranges to CCIR-601. If you do a search for "CCIR" you should find more information than you probably want to read ;).

[MaTTeR]

Originally posted by onesoul
I shouldn't ask here but I am too curious, what does those parameters mean? btw what does limiter() do? SansGrip answered about Limiter() and you can find all the other parameters documented on MarcFD's MPEG2Dec3 documents.

@SansGrip
I'm pulling my hair out here trying to figure out what's causing me access violations with the YUY2 (v0.6a) filter. My script looks like- mpeg2source("D:\Seals\seal.d2v",idct=2,lumoff=-1)
Telecide(guide=1)
Decimate(mode=1)
crop(4,4,712,472)
Blockbuster(method="noise",detail_min=1,detail_max=35,cache=512,block_size=3,luma_offset=-2,luma_threshold=25)
BicubicResize(544,404,0,0.5)
LegalClip()
The violations dont happen immediately when I open my script, only when I use Vdub to start seeking through the video or start an actual encode (Happens about 6th or 7th frame). Also, if I move the "BB" line to after resizing then all works very well. Sounds similar to the problem I was having in this thread. (http://forum.doom9.org/showthread.php?s=&threadid=40056&pagenumber=3)

I'm using XviD and the latest AVS 2.07 11-25-2002 build. Any ideas? Let me know if I can provide any other information. I might be slow to respond since I'm leaving the city for the holidays.

Happy Holidays Everyone!

[SansGrip]

Originally posted by MaTTeR
The violations dont happen immediately when I open my script, only when I use Vdub to start seeking through the video or start an actual encode (Happens about 6th or 7th frame). Also, if I move the "BB" line to after resizing then all works very well. Sounds similar to the problem I was having in this thread. (http://forum.doom9.org/showthread.php?s=&threadid=40056&pagenumber=3) If it is the same bug then the fact that this always seems to be fixed by running the filter after resizing suggests that it's the resize filter trying to request out-of-range frames.

While I'm still not convinced that pretending it never happened is the right way to "fix" the problem, I can add a range check and see if it helps.

I've never tried running Blockbuster before the resize myself, but shouldn't it really go after anyway? I would think that by having it before you'll at least be reducing its effectiveness and so will need a higher variance to compensate...

[MaTTeR]

Originally posted by SansGrip
I've never tried running Blockbuster before the resize myself, but shouldn't it really go after anyway? Well I'm not exactly sure to be honest. I've been testing it extensively on this crap DVD source for the 48hrs and I can't make my mind up if I like it before or after. FWIW, using BB's "add noise" feature along with XviD's Qpel option seems to help quite a bit.

Maybe it's not worth looking into if BB performs better after resize? Guess I'll know shortly since my encode just just finished, need to view it on TV now.

BB is adding random color noise correct? I was wondering how difficult it would be to add random neutral colored noise(ie. grey or black). The only reason I mention this is because the noise is very noticeable when processing full frames (detail_max=100%). I don't know, was just thinking aloud if neutral colored noise would be less noticeable to the human eye.

[SansGrip]

Originally posted by MaTTeR
Maybe it's not worth looking into if BB performs better after resize? Oh, it's definitely worth it. I'd be very interested to hear your conclusion since I've only ever used it after the resize :).

BB is adding random color noise correct? Nope... As of 0.4 (I think) it only adds noise to luma.

The only reason I mention this is because the noise is very noticeable when processing full frames (detail_max=100%). Can't you just lower the variance until it reaches a satisfactory level?

[iago]

Maybe it's not the place, nor much relevant to the topic, but DivX5 decoder's Film Effect at level 3 (out of 4) is imho the most pleasing-to-the-eye noise pattern to enhance the look of an encode and cover the encoding artifacts/blocks almost perfectly. I had to admit that somewhere, and it just happened to be here ;).

Btw, I use DivX5 only to decode Nandub SBC encodes, and never for encoding purposes! ;)

regards,
iago

[bilu]

Should I use lumoff=-2/Unfilter(5,5) with BlockBuster, or as a replacement? I read the thread but haven't seen a conclusion.
Guess I'll test it tonight :)

Please look at this thread also http://forum.doom9.org/showthread.php?s=&postid=230458#post230458 , I'm curious if the MPEG quantizing could afect the luma/chroma range for TV-output

[bilu]

@iago

After this reply (http://forum.doom9.org/showthread.php?s=&postid=230512#post230512) I really got confused :o

[Didée]

@ bilu

Once upon a time, there was a thread ... :D

... about "black-blocking" on TV-sets, and what to do about.

In this thread, as well as in some others, it was mentioned that Tom's "unfilter" adds a small amount of noise to the source. This is the main reason why it helps against the "black-blocking" problem.
And that's also why it helps with negative values as well as with positive: the noise gets added in both cases.

[iago]

Once upon a time, there was a thread ... :D (Didée)

Yeah, and what a thread it was! :D

Btw, I'm not sure if UnFilter does its nasty job by adding or keeping noise (with "+,+" values), or by darkening or something (with "-,-" values), or by some other mysterious ways which might be even much more beyond our understanding (Tom, any new theories about this? :D), but it certainly does something to help defeat the black-blocking issue! ;)

iago

[SansGrip]

I just uploaded to my site (see sig) version 0.7, which adds a new method: dithering. Basically this is very similar to the noise method, except it will add the same noise to each frame, creating a kind of "unchanging noise" effect.

I added this method because I wanted to see if such dithering would result in less artificial "movement" in otherwise static areas than with the noise method. It should be considered experimental.

Let me know :).

[iago]

@SansGrip

Welcome back, man! And thanks for the new version, gonna try asap! ;)

regards,
iago

[iago]

@SansGrip,

I have played a bit with the new "dither" method and compared the results to the old "noise" method. Yes, the results of "dither" are really very similar to the "noise" method, but that's only when you do a frame-by-frame comparison in VirtualDub. But during playback, the results of the "noise" method is definitely superior to and provides a much more natural/pleasing viewing experience than the "dither" method imho ;).

best regards,
and thanks again for this great filter,

iago

[Kramerica]

When used with xvid, dither produces a result which looks very similar to a divx 3.11 rip. That is to say, dark areas keep their pattern, even when the area housing this pattern is moving. I don't really think this is necessarily a bad and can look better than either sharpen or noise methods in some instances.

[kwag]

Originally posted by iago
@SansGrip,

But during playback, the results of the "noise" method is definitely superior to and provides a much more natural/pleasing viewing experience than the "dither" method imho ;).


Unless you're looking at a still background, where you'll notice that if you use "Noise", there will be some movement. Like on walls, etc. But if you use "dither", the backgrounds will look still, and that is a big difference. :cool:

-kwag

[MaTTeR]

Almost 2 weeks later and I'm still trying to get a good encode of US Navy Seals, this flick is driving me nuts.

"Dither" doesn't seem to help under water scenes at all but the "noise" method does to some degree. I can see what kwag means about stable backgrounds using "dither" though, definitely less movement.

[SansGrip]

Sounds like it's a good job I marked it "experimental" :D. When I coded it I really had no idea how it would look or work, I just figured it might fix the "artificial movement" problem inherent with noise.

Now I think about it, it makes sense that noise looks more natural, since anyone with cable is already "tuned" to ignore noise anyway :).

[iago]

Btw, "dither" seems to be much more compressibility-friendly compared to "noise" ! ;)

[SansGrip]

Originally posted by iago
Btw, "dither" seems to be much more compressibility-friendly compared to "noise" ! ;) Yep, I noticed that too :).

[MaTTeR]

Anyone have any comments/advice on block_size? I'm typically using a value around 3-5 and 6-8 on darker movies like the evil one I mention above.

@SansGrip,
Indeed BlockBuster works more effeciently when put at the end of the script as you suggested. You stated that as of v0.4 that your only adding noise to the luma plane. Does some type of hidden parameter exist to still add noise to the chroma plane? I suppose this might kill compressibility in general...Did you find the effect to not be helpful? Thx again for your continued effort.

Edit- fixed vBB typo.

[SansGrip]

Originally posted by MaTTeR
Anyone have any comments/advice on block_size? I'm typically using a value around 3-5 and 6-8 on darker movies like the evil one I mention above. I've only ever used the default (except when checking for bugs of course ;)), but would be interested to hear others' input on this.

Does some type of hidden parameter exist to still add noise to the chroma plane? Nope. I did quite thorough testing on this before dropping chroma noise, but it seemed to be almost invisible to the encoder. That said, it wouldn't be difficult to add it back in if there's enough demand :).

BTW, are you the same MaTTeR mentioned in the HeadAC3he docs as a beta tester?

[MaTTeR]

Originally posted by SansGrip
That said, it wouldn't be difficult to add it back in if there's enough demand :).Well that's what I figured, so it's prolly not worth the effort. I was just curious more than anything else :-)

BTW, are you the same MaTTeR mentioned in the HeadAC3he docs as a beta tester? Yep, that would be me. Actually I'm not sure if I ever read those docs myself. LOL

[Didée]

To be honest, I did not play very much with block_size. But since we're fighting mpeg blocks, I feel the need to process only 8-pixel-blocks, or maybe 16.
With other values, I'd fear to alter the information within a block not equally, endangering some visual degration of the block's content.

I'm more curious, what strenghts are you actually using?

-----

For quite some time, I am fiddling on a completely different approach.
Unfortunately, I don't find enough time to test this thoroughly.

Basic concept:
(similar as the built-in, but completely different approach)

Apply noise to areas with little information, leave areas with plenty information alone, and a smooth transition between these two.


1. Find the edges in a frame

I do this by computing the difference between the (slightly blurred) original frame and a stronger blur of it. This might be called "edge detection by unsharp masking). Most other methods of other filters deliver "sharp" edges. Here, we need a B/W image of the edges that correspond to the "strength" of the edges in the original frame.
Alas, since current filters don't really suite (and I'm not a coder), I have to fiddle a lot to make a proper mask here.
(Waiting for MarcFD to implement "subtract" for YV12. He made a little promise of a new parameter for that ... ;) )

2. Perform "blockbuster" with "noise" and "max_detail=100"

3. Layer the fully blockbustered frame onto the areas with no edges, by using the result of step 1 as a mask.


This way, I hope to avoid that the added noise disturbs the image.
My biggest "problem" with blockbuster and its approach is, that the blocks that are processed are flickering around: try "show" mode, and let it play. You see what I mean. Also, this popping-around of the processed blocks perhaps could disturb the codec's motion estimation. (?)
And, in case one is using mpeg-quants with XviD, on I- and P-frames that use quant 2, the noise may be preserved by some amount. Then, the block-wise adding of noise might get visible.

Doing the above with current AviSynth weapons is slow. Perhaps if one of our awesome filter developpers is bored and doesn't know what else to do ...


Comments to this would be very welcome.


Didée

[MaTTeR]

Originally posted by Did�e
My biggest "problem" with blockbuster and its approach is, that the blocks that are processed are flickering around: try "show" mode, and let it play. You see what I mean. Also, this popping-around of the processed blocks perhaps could disturb the codec's motion estimation. (?) Well I have to say I've this problem lately too. Using show I can see that the mask will change from one extreme to another between 2 frames that barely differ(very low motion). I was thinking the same thing about it affecting the codec's motion estimation also but from what I can see in terms of compressibility it doesn't seem to harm much. I have to admit I'm using Xvid's Qpel function though, that should help:)

Doesn't strength just affect the sharpen parameter or did I read the docs wrong?

[Didée]

No, you have read right. Stupid me, of course I meant "variance".
Excuse me, 36 hours without sleep ...

[bilu]

Could the use of a smoother such as fluxsmooth after blockbuster help?

[SansGrip]

Originally posted by MaTTeR
Yep, that would be me. Well, in that case, perhaps you know more details about the Surround 2 downmix mode other than, as the docs say, it uses "a matrix I found"? ;)

[SansGrip]

Originally posted by Didée
But since we're fighting mpeg blocks, I feel the need to process only 8-pixel-blocks, or maybe 16. I feel the same, though I tend towards the former. DCT blocks are always 8 pixels. Macroblocks are 16, and are the symptom of a different problem.

I'm more curious, what strenghts are you actually using? I usually use between 0.5 and 1.0, though I sometimes go higher if the source is very blocky. Interestingly, I just watched the three-disc AC3-Guru version of Minority Report and it looks like it had Blockbuster(method="noise", detail_min=1, detail_max=100, variance=3) applied to it. Weird effect, and not surprising it had to go on three discs ;).

Apply noise to areas with little information, leave areas with plenty information alone, and a smooth transition between these two. I like this idea. Sort of like a combination of MSmooth's edge constraints and Blockbuster's noise. It would require reworking the code, though, since it would no longer be based on blocks.

Perhaps it could be activated via "block_size=0", though this would be fairly opaque. Alternatively one could use, say, mode="edges" for this approach or mode="blocks" for the old way...

I'd appreciate some feedback on the best way to implement this wrt parameters. Oh, and whether it's even a good idea ;).

My biggest "problem" with blockbuster and its approach is, that the blocks that are processed are flickering around This is indeed a problem, and is one of the reasons I added the "dither" method. Dither helps somewhat but doesn't address your concern specifically (i.e. that the blocks considered "not detailed enough" are constantly changing).

Doing the above with current AviSynth weapons is slow. Perhaps if one of our awesome filter developpers is bored and doesn't know what else to do ... Well I'm not bored and certainly have lots of things to do, but I'll give this some thought and see what comes out the other side ;).

Comments to this would be very welcome. I second that.

[esby]

Someone (maybe Sangrip, if he want / can) can compile again the last version in order to use it with avs 2.51... ? ^^
(Blockbuster-2.5.dll saying that it is not a filter for avs 2.5 etc.)

Thanks

esby

[Boulder]

Someone already recompiled it a while ago, search the forums and you'll find it attached somewhere. You might also find a link at www.avisynth.org .

[esby]

mmm
I'll search the forums...

For information... the link on www.avisynth.org
redirects here and on sangrip page, both with a non compliant version to the last build as far i tested.

esby

[Garfield]

Originally posted by esby
mmm
I'll search the forums...

For information... the link on www.avisynth.org
redirects here and on sangrip page, both with a non compliant version to the last build as far i tested.

esby

Maybe you should try the avistynth 2.5 faq right in this section ;)

[esby]

Well one click being better than a lot of clicks...

http://forum.doom9.org/attachment.php?postid=255587

for those who'll arrive here in the future...

esby

[Garfield]

Tiens, un francais, j'avais pas fait gaffe ;)