Need help with some kind of gridding effect(?)

[Abyssal]

Anyways, I'm new to the forums (as you probably can see on my profile) but I'm also new to AviSynth.

So therefor I've decided to make a thread hoping to get some help with a few questions I have related to AviSynth usage.

I've been encoding a lot of anime lately and I've started to dig myself into the wondrous world of AviSynth and there are some terms I need help with.

This first one is something that generally shows up on darker areas of the encode, no - not banding but actually flat dark colors. Like a completely blue/black wall with no particularly varying color schemes.

It is some kind of gridding effect that can sometimes get a bit blocky as well.

I've cut out the area of the effect from screenshots;

Here is the raw BDMV (Sorry if the cutouts are slightly off by a few pixels but I think you'll get the idea);
http://tpspic.me/i/ThpRW.png - Yes, I know the shirt is not present and this is a different frame but it shows the wall (in reality a locker but whatever :P)

Here it is, encoded but unfiltered:
http://tpspic.me/i/K0yp.png

Here it is with some debanding and sharpening:
http://tpspic.me/i/a9Xs.png


As you can see, it's not present in the source so it's only in the encode but as you can see, debanding makes it slightly more visible and introduces some blocking to it - I did some sharpening just to make it easier to see.

So my first question - Is there any term for this so I know what to call it in the future(?) and secondly, is there any way to repair this with any particular filter?

[Guest]

Your screenshots are very unclear. I diffed the first two PNGs and I see what looks like shot noise in one, but nothing I would characterize as "gridding". Can you show a clearer example? Also, please use the same frame for before and after comparisons!

[Abyssal]

I know it's not gridding but I'm not sure what it is.

Here are new screenshots:

RAW:
http://tpspic.me/i/Vn8.png

With some debanding/sharpening:
http://tpspic.me/i/4he5q.png

Do note that this is also visible unfiltered as well but it's more visible with some debanding and sharpening.

Keep in mind that I am not talking about the banding in the middle-ish area but at the very top around the shadows where you can see the heavy noise in the shadows. At certain areas it gets a bit blocky and around that you can see some some gridded dots. Also note that this is not only visible when removing noise, it's generally in dark areas - also in the shadows where there is no noise at all. I just took these shots cause I thought it was more visible in these. :P

[Guest]

Here is the diff between your two PNGs:

http://neuron2.net/misc/diff.png

Right click and download it to your PC to avoid permission issues.

I don't see anything like what you are describing.

[Abyssal]

Hmm... strange... because to me it's pretty visible.

Let me ask you, what do you see?

[Guest]

Just looking at your PNGs with my eyes, I see nothing!

Diffing with Beyond Compare produces the result I just posted.

[Abyssal]

Just looking at your PNGs with my eyes, I see nothing!

Diffing with Beyond Compare produces the result I just posted.

Can't seem to be able to open that one up after saving. Both when saving it directly and when adding the .png extension manually when saving.


Btw, let's see if someone else finds the threads and maybe they see it. :P

[Guest]

Maybe a JPG will work for you.

http://neuron2.net/misc/diff.jpg

[Abyssal]

Nope, still can't open it...


Anyway, Here I have cut out the area and zoomed it in a bit:

RAW:
http://tpspic.me/i/TwDBU.png
Encoded:
http://tpspic.me/i/rxHZo.png

Even more zoomed in
RAW:
http://tpspic.me/i/euiqw.png
Encoded:
http://tpspic.me/i/qPEG.png

If you still can't see, try moving closer to your monitor.

[poisondeathray]

If you're seeing a fine grid pattern, that's from the dithering filter you used. It's easier to see if you raise the gamma, or use a levels filter

(And you shouldn't be seeing it, this means your setup isn't calibrated properly)

If it gets blocky, it's because you're not using enough bitrate. Although some of the screenshots were not of the same frame, there is a big difference in quality between your encodes and the original, that can only be attributed to low bitrate

[Bloax]

The "encoded but not filtered" one looks like it has some bayer dithering going on for some reason.
That is, stuff like the alpha on this guy:
http://i.imgur.com/rmxOs.png

[Abyssal]

The "encoded but not filtered" one looks like it has some bayer dithering going on for some reason.
That is, stuff like the alpha on this guy:
http://i.imgur.com/rmxOs.png


Exactly like that. And no, I'm confident I did not to any dithering to it.

but so it's called dithering? since I kinda wanted help with the term as well. :P

[Guest]

@Abyssal

How did you encode it?

[Didée]

And no, I'm confident I did not to any dithering to it.
Post#1 says that you used debanding. Debanding uses dithering.

[Abyssal]

@Abyssal

How did you encode it?

x264 + AviSynth I guess?

This was quite some time ago so I only have the encoded files left and the BDMVs left. No scripts or anything nor do I know if I batch encoded or used a GUI.


Post#1 says that you used debanding. Debanding uses dithering.

Oh, yeah, true. But I posted 3 different examples. :)

[mp3dom]

Maybe you've used a GUI with a 'deband' check ticked (or an AVS with dithering enabled?)... Those shots highly resemble the result of an ordered dithering algorithm.
Also, never seen that kind of problem on an x264 encode (starting from a non-dithered source), even with too low bitrates.

[Abyssal]

How do I know if I used an AVS with dithering enabled?

[Guest]

He means in your script. So what was the script you used?

You are treading on rule 6 territory here. Generally, we don't accept the "I forgot how I encoded it" line. You can avoid problems with the rules by starting again from your original disk.

[Abyssal]

He means in your script. So what was the script you used?

You are treading on rule 6 territory here. Generally, we don't accept the "I forgot how I encoded it" line. You can avoid problems with the rules by starting again from your original disk.

I see, then lock the thread since I do no have the script anymore.

But either way, I did not use any dithering then because this can sometimes be visible without any filtering.

Anyway, since he said debanding introduces dithering - is it just to try out any dithering filter and see what works out? Or is it just one of the downsides of debanding?

[Guest]

I see, then lock the thread since I do no have the script anymore. If you have the original disk, you can re-encode it and see if the artifacts recur. If so, we will be able to tell you what step in your process caused them.

If you have a source stream that already shows the artifacts, please post a sample of it.