Macroblocking in anime...

[SPiKA]

I just can't get rid of the macroblocking no matter what I try. Already tried many different x264 settings, and lots of different denoisers (MVDegrain, TemporalDegrain, MC_Spuds, etc...) with no luck. Bitrate is about 1400kbps.

Here's a sample of the source:
http://www.mediafire.com/?mfjdf17j9bz

[SPiKA]

Here's a sample of the encode:
http://www.mediafire.com/?92n00umdmim

It's strange... for some reason, the encode plays perfectly fine without any kind of blocking on my secondary PC. :S
Maybe a decoder problem?

[45tripp]

probably i don't see blocking in either.

there is awful juddering though.

search deshaker.
also seem to remember dark shikari had a clip of bebop with shaking,
search that thread, maybe get a script idea

[Adub]

If you get blocking on one computer, and not another, it is a decoder issue. Make sure that you enable deblocking in the all of the decoder's that you use.

[[P]ako]

Sometimes CRT monitors mask what you can see in some LCDs.

[Comatose]

I also don't see any blocking, and I'm using a LCD.

By the way, which anime is that? o.o

[cestfait]

decoder is important, but you could use better motion estimation or lower your deadzones. also, I would use some aq on that. that would be the most direct route to getting rid of your blocking.

your filtering looks interesting. did you use mvdegrain after all? would you mind posting your script?

[Mug Funky]

@ [P]ako:

it's the other way round. LCDs tend to introduce rounding errors and crap like that, where a CRT will not mess with it's input so much. this is especially true when going from YUV to RGB in 8 bits. if you're lucky enough to plug a decent broadcast monitor into your output you'll see the banding is not there because the YUV is going straight into the monitor's analog electronics before any rounding error can bugger things up.

[[P]ako]

@ [P]ako:

it's the other way round. LCDs tend to introduce rounding errors and crap like that, where a CRT will not mess with it's input so much. this is especially true when going from YUV to RGB in 8 bits. if you're lucky enough to plug a decent broadcast monitor into your output you'll see the banding is not there because the YUV is going straight into the monitor's analog electronics before any rounding error can bugger things up.

Gotcha. I always thought it was the opposite.

[SPiKA]

By the way, which anime is that? o.o

Cowboy Bebop.

decoder is important, but you could use better motion estimation or lower your deadzones. also, I would use some aq on that. that would be the most direct route to getting rid of your blocking.

your filtering looks interesting. did you use mvdegrain after all? would you mind posting your script?

Lower the deadzones to... 9-9 or something like that?
AQ? what is that? xD

DGDecode_mpeg2source("D:\Encodes\cb1\1.d2v",info=3)
ColorMatrix(hints=true,interlaced=true)
deint = TDeint(mode=2, mtnmode=3, blim=100)
TFM(cthresh=4, slow=2, clip2=deint)
TDecimate(mode=1)

crop( 8, 6, -8, 0)
Spline36Resize(704,480)

source = last
backward_vec2 = source.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec1 = source.MVAnalyse(isb = true, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec1 = source.MVAnalyse(isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec2 = source.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
source.MVDegrain2(backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400,idx=1)

fft3dfilter(bt=-1,sharpen=0.7)

dull = last
sharp = dull.LimitedSharpenFaster( ss_x=1.25, ss_y=1.25, strength=60, overshoot=1 )
Soothe( sharp, dull, 40 )

[SPiKA]

Ah, and x264 settings are:

--pass 2 --bitrate 1500 --stats ".stats" --ref 16 --mixed-refs --no-fast-pskip --bframes 16 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 7 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --threads auto --thread-input --cqmfile "C:\Program Files\megui\extra\eqm_avc_hr.cfg" --progress --no-psnr --no-ssim --output "output" "input"

[Dark Shikari]

I'd try ditching the CQM, personally.

[cestfait]

you might try the prestige cqm, alternatively. aq means "adaptive quantization." It was originally created to fight blocking in large open areas by allocating bits to "flat" areas. The poster above me has since invented a new method (or methods, I guess) to do the same thing more efficiently. Some people think that the older aq does better on anime. Many feel that aq shouldn't be used at all on anime (although from what I've seen of that style of encoding, I wouldn't take that view too seriously unless you plan to smooth the living hell out of your picture). I think that the new aq works better than ever on anime, you just have to treat it a bit more like video and less like anime-- don't try to get rid of ALL of the noise. With a little bit of useless noise to munch on, the newer x264 builds can produce some really nice anime compression.

the bottom line-- update your x264.exe!

p.s. you might want to go a bit easier on the sharpening. there was a lot more visible haloing in the finished encode than in the original dvd.

Cheers!

[SPiKA]

Is there a way to use AQ with MeGUI?

[Dark Shikari]

Is there a way to use AQ with MeGUI?Its on by default.

[cestfait]

if it is one of the older builds with the original aq though, you can put your "--aq-strength" and "--aq-sensitivity" commands in the custom command line options box under "zones."

Good luck!


p.s. I just realized that I was pretty much ignoring the topic of this thread to begin with. at least on the computers I have had, most of the effect you are talking about has to do with how your screen is calibrated. on a mac, there are a number of ready made profiles that change how your screen "interprets" color values. Their "Color LCD" preset, for example, is much more discerning when it comes to dark-area-blocking, for example, whereas "Adobe RGB" is not.

All the same, it's better to avoid the macroblocking to start with, right?

p.p.s. as for deadzone settings, it's usually better to keep the recommended ratio (unless you like useless noise). that means 9-5 would be a better choice than 9-9 (although it would cost more bits, so maybe a more equivalent setting would be something like 11-6). just play until you think you're retaining the right amount of detail. still, not much beats lower quantizers, so in your case I would just boost motion estimation, leave deadzone at 21-11 and lower my crf from 18.0 to 15.0 or something like that.

[SPiKA]

if it is one of the older builds with the original aq though, you can put your "--aq-strength" and "--aq-sensitivity" commands in the custom command line options box under "zones."

I'm using the latest build (Jarod's 819). Can you recommend me values for aq-strength and aq-sensitivity?

so in your case I would just boost motion estimation

Using MVDegrain3 instead of MVDegrain2?

leave deadzone at 21-11 and lower my crf from 18.0 to 15.0 or something like that.

crf? I can't find that option...

PD: thanks to all of you for your help... as you've already noticed, I'm pretty n00b. xD

[Blue_MiSfit]

CRF is "constant quality". It's kind of like CQ mode in Xvid, but it allows for some quantizer variation.

CRF ranging from 18 to 22 is usually transparent - and most folks who use CRF do so in this range.

~MiSfit

[cestfait]

instead of "Automated 2-pass" use the constant quality mentioned above.

yeah, mvdegrain3 would help, but I meant using exhaustive or SATD exhaustive search instead of uneven multi-hexagon. that has visual effects on anime that most people don't notice, but can really help against macro-blocking (because the bits go into "more correct" places).

not to mention that compression with a better search method is much higher, so you can lower crf values even more and get a better quality picture without needing a huge bitrate.

by the way, crf is one-pass, so if you needed justification for the slower motion-search and really slow filters like mvdegrain3, imagine encoding without 2-passes, but with the same quality.


p.s. SPiKA, the funny thing is that I started encoding a several months after you did. I remember KB. ;)

[foxyshadis]

The speed decrease that switching from mvdegrain2 to 3 would cause would be better put towards switching to mc_spuds, temporaldegrain, or possibly one of the 'fast' temporaldegrains.

[cestfait]

I wouldn't say I totally agree there. maybe temporal degrain, but mcspuds does a lot of things to the video that I don't approve of. not to mention that in my experience a clean vector clip doesn't really help in anime sources post 1995. I also think that if you're going to be using temporal averaging along the lines of temporalsoften, ttempsmooth, or even mvdenoise, you need at least four radius frames or really high thresholds, and the thresholds on temporal degrain have caused me problems before.

besides, a couple of blocks that slip through an mvdegrain are usually made up for by the motion searching in the final encode, at least in his case, using x264.

I'm a bit biased, but I hold the personal opinion that you are sinking fast the more filters you use on a single source. it's annoying, first of all, not to know exactly what's going on. secondly, I find that many filters conflict with each other in unexpected, subtle, and sometimes inefficient ways.

I wouldn't go beyond adding a vector clip to some motion compensation in terms of complex denoising, and usually not even that. I'm not a fan of using more than one denoiser on any source, except if one is removegrain(1) [undot].

[SPiKA]

I've sent the encode to some friends... some say that it plays fine, but others say that they can see some nasty blocking.

Can't figure out why this is happening... :S

[Dark Shikari]

I've sent the encode to some friends... some say that it plays fine, but others say that they can see some nasty blocking.

Can't figure out why this is happening... :SDo some have LCDs and others have CRTs? :p

[SPiKA]

Do some have LCDs and others have CRTs? :p

It has nothing to do with that...
I have CRT's in my 2 PC's, but I can see macroblocking only in my main rig.

[sixide]

I played it with deblocking disabled and AutoLevels on, and I couldn't find any blocking of note. I don't think this is an encoding problem.

[Spuds]

but mcspuds does a lot of things to the video that I don't approve of. not to mention that in my experience a clean vector clip doesn't really help in anime sources post 1995.

Not that I'm looking for your approval :) but what does it do that you object to that you can't easily turn on / off or change a value to? The script is highly configurable so its easy to experiment with options and I've always been open to constructive criticism and adding tunning options.

Don't like a clean vector clip? preprocess=false don't like derining edges? edgeclean=false, don't like sharpening sharpp=0, don't like extra denoise in high motion areas? ml=255 you like temporal processing? temporal=true like your own thsad values thsad=???, don't like truemotion truemotion=false, etc ...

[cestfait]

absolutely right. I'm sorry for bashing your script; it really is impressive coding. all I meant was that because I prefer to use only a few of the functions (although I push that method: the more you mess with the video, the worse it looks) your script is too large and clunky for my personal use... efficiently. yes, I can disable whatever I want, but I might forget to disable something, or end up disabling so much that I would be better off just scripting:

function mvaverage(clip c)
{
vbw = mvanalyse(c, isb = true, delta = 1, blksize = 16, overlap = 0)
vfw = mvanalyse(c, isb = false, delta = 1, blksize = 16, overlap = 0)
vbw2 = mvanalyse(c, isb = true, delta = 2, blksize = 16, overlap = 0)
vfw2 = mvanalyse(c, isb = false, delta = 2, blksize = 16, overlap = 0)
vbw3 = mvanalyse(c, isb = true, delta = 3, blksize = 16, overlap = 0)
vfw3 = mvanalyse(c, isb = false, delta = 3, blksize = 16, overlap = 0)
vbw4 = mvanalyse(c, isb = true, delta = 4, blksize = 16, overlap = 0)
vfw4 = mvanalyse(c, isb = false, delta = 4, blksize = 16, overlap = 0)
vbw5 = mvanalyse(c, isb = true, delta = 5, blksize = 16, overlap = 0)
vfw5 = mvanalyse(c, isb = false, delta = 5, blksize = 16, overlap = 0)

bw = mvcompensate(c, vbw)
fw = mvcompensate(c, vfw)
bw2 = mvcompensate(c, vbw2)
fw2 = mvcompensate(c, vfw2)
bw3 = mvcompensate(c, vbw3)
fw3 = mvcompensate(c, vfw3)
bw4 = mvcompensate(c, vbw4)
fw4 = mvcompensate(c, vfw4)
bw5 = mvcompensate(c, vbw5)
fw5 = mvcompensate(c, vfw5)

interleave(bw5, bw4, bw3, bw2, bw, c, fw, fw2, fw3, fw4, fw5)
temporalsoften(5, 4, 4, 32, 2)
selectevery(11, 5)
}

...or rewriting part of mcspuds, but the above is simpler. It's just personal preference, although I may have been pushing my own opinion a bit too forcefully in my comments. ;)


@ SPiKA: perfectionist that I am, I wouldn't be satisfied until NOBODY could see the blocks. besides, working to get rid of them will make you a better encoder in the end. or, at least, it will allow you to come to terms with where you are willing to let things go. it's pretty hard to compress something a lot without corrupting the video a bit, if you're trying to fix EVERYTHING. I let some of the less annoying problems past, but strive for perfection when it comes to ivtc and blocking, for example.

[SPiKA]

The thing is, I've tried every suggestion you've made but I can't get rid of them... :p

[Spuds]

Thanks for the explanation, the original wording made me think there was a incorrect or improper processing step vs a personal preference on how to approach a video issue which I understand.

I see why one would prefer the 'hand' code as its more easy to see the processing steps although more work to change things when you want to quickly explore options.

I must say that long compensate and temporalsoften could be an artifact generator at scene changes as mvflow like mvcompensate can 'drag' info from frames that are no longer relevant, plus it can mutate gradients, that said I use it as its very powerful :)
mc_spuds(frames=4,flow=true,blocksize=16,overlap=0,strength=1,sharpp=0,ml=255) is 'similar' in high level 'concept' to what you posted but you would have to be familiar with all those settings :p

[cestfait]

SPiKA: if that's not getting rid of your blocks, you have 2 options. Screw around with more filters or add more bitrate. I find that I never use more than an average of 1500-1600kbps, but without "me=tesa" I would need more. you can also boost your m.e. range, but I wouldn't think you'd need that if your denoising is motion compensated.

IVTC or deinterlacing, the above averaging script, some sharpening, and cropping are really the only filtering I do on sources with no special artefacts.

[SPiKA]

SPiKA: if that's not getting rid of your blocks, you have 2 options. Screw around with more filters or add more bitrate. I find that I never use more than an average of 1500-1600kbps, but without "me=tesa" I would need more. you can also boost your m.e. range, but I wouldn't think you'd need that if your denoising is motion compensated.

I'm getting the blocks even without any filtering at all, and moar bitrate doesn't seem to help either.
Guess it's a problem with the decoder.

Maybe I should give up... xD

[cestfait]

have you really tried aq? and don't expect an unfiltered picture to act BETTER after encoding!

I doubt it's a decoder problem at this point, or even monitor. you didn't have so many blocks to begin with, and the ones you had could have been addressed with some aq and a bitrate under 1600.