Major Blocking Issue...

[Caspar]

Ok, this is not my first encode using XviD, but it is the first time I encounter this problem to such an extent. I am using the latest XviD build from Koepi and as per usual, using avisynth as the frame server. There are lots of frames in the resulting encoding showing the blocking problem, and I have tried different settings (using aggressive High bitrate degradation and Low bitrate improvement values) and it doesn’t seem to help. The only time the I don’t see the blocks is when I set the bitrate to 3,000+ but that’s not going to work for me since I am looking at a target size of 175MB after muxing the audio (25min video). I would assume that the bitrate I am using is too low...

Source Frame:
http://img.photobucket.com/albums/v415/LightFusion/Original_XviD1.png

Resulting Frame (2pass encode):
http://img.photobucket.com/albums/v415/LightFusion/After_XviD1.png


XviD Settings:
http://img.photobucket.com/albums/v415/LightFusion/XviD_Profile_Config.jpg

http://img.photobucket.com/albums/v415/LightFusion/XviD_Advanced_Config.jpg

http://img.photobucket.com/albums/v415/LightFusion/XviD_Encoding_Config.jpg

http://img.photobucket.com/albums/v415/LightFusion/XviD_Size_Config.jpg

http://img.photobucket.com/albums/v415/LightFusion/XviD_Zone_Config.jpg


Here's the AVS script used:

#load plugins
loadplugin("d:\encode\filters\mpeg2dec3dg.dll")
loadplugin("d:\encode\filters\kerneldeint151.dll")
loadplugin("d:\encode\filters\BlendBob.dll")
loadplugin("d:\encode\filters\Decomb521.dll")
loadplugin("d:\encode\filters\deen.dll")
loadplugin("d:\encode\filters\temporalcleaner.dll")
loadplugin("d:\encode\filters\Convolution3DYV12.dll")
loadplugin("d:\encode\filters\VSFilter.dll")
loadplugin("d:\encode\filters\LoadPluginEx.dll")
loadplugin("d:\encode\filters\DustV5.dll")

mpeg2source("d:\encode\my_encode.d2v")

KernelBob(1,8,false)
BlendBob(show=false,badFrames=0)
decimate(cycle=5,quality=3)

temporalcleaner(ythresh=2, cthresh=3)
convolution3d(preset="animehq")
deen("a2d",2,8,6)
converttoyuy2()
pixiedust(3)
converttoyv12()
deen("a2d",2,6,1)
temporalcleaner(ythresh=1, cthresh=1)

converttorgb32()
crop(8,0,-10,-5)
LanczosResize(640,480)
converttoyv12()

textsub("d:\encode\my_encode.ssa")


Please let me know if you require anymore information. Appreciate any response.

[Sharktooth]

converttorgb32()
crop(8,0,-10,-5)
LanczosResize(640,480)
converttoyv12()


A little offtopic but... why you convert to RGB for cropping and resizing?
Remove those converttorbg32 and converttoyv12 ...

[Manao]

Sharktooth : Because he wanted to crop to non mod4 values ( which is usless imho )

Caspar : what was the quantizer of the picture you choosed to show ? Was it a b-frame ? You can get this information with ffdshow and its OSD.

[Sharktooth]

Originally posted by Manao
Sharktooth : Because he wanted to crop to non mod4 values ( which is usless imho )
didnt saw it... i assumed he was cropping in "the right way"...

[Teegedeck]

2x temporalcleaner, 1x convolution3d, 2x deen and 1x pixiedust - isn't that a bit much denoising and possibly one potential cause of the problem?

[olavitur]

Try smaller resolution without those denoisers, etc.
What was your clip's first pass size?

By the way, why didn't you use Cartoon-mode?

[Sirber]

Maybe MPEG quants would help...

[Sharktooth]

it's 2x deen or 2x temporalcleaner producing artifacts on moving objects...
i would also use lanzcoz4resize and MOD4 crop without colorspace conversions.
No pixiedust() but DeGrainMedian() and no filtering before resize...

[Caspar]

Sorry for the late reply, I thought I should try some of the suggestions first before I make a reply. So here’s my results from the suggestions given thus far...

@Manao
As you instructed, I checked the quantizer for the frame using ffdshow and the frame is a quant 5 b-frame.

@olavitur
The projected file size using the script in my first post is 238MB so I am looking at a ~67% compression. I have already tried shooting for a bigger final size of 233MB but it still produced blocking on the same frames.

Here’s the new script used after looking at all the suggestions. I didn’t use lanzcoz4resize because it will produce a file that’s way too big for my purpose.

mpeg2source("d:\encode\my_encode.d2v")

KernelBob(1,8,false)
BlendBob(show=false,badFrames=0)
decimate(cycle=5,quality=3)

crop(8,0,-8,-6)

convolution3d(preset="animelq")
deen("a2d",2,8,6)

BilinearResize(640,480)

textsub("d:\encode\my_encode.ssa")


I have enabled Cartoon mode this time. The reason why I didn’t enable this option is because a lot of people claims anime != cartoon, so I am just going with the flow not wanting to test it out myself.

With this script, the first pass yield a projected file size of 243MB.

After the 2nd pass, nothing has changed, still blocking on the same frames, and some that didn't show blocking now have slight blocking.
http://img.photobucket.com/albums/v415/LightFusion/After2_XviD1.png

None of the settings where changed except for Cartoon mode. Even with the new script the frame is still encoded as a quant 5 b-frame.

Edit: typos

[Teegedeck]

I'd be curious what happens if you throw out all denoisers (deen, conv3d). You can do just a constant quantizer=3 encode on that clip and see whether it helps at all before you do the whole 2-pass all over again.

Another idea would be to see if using quarterpixel-accuracy gets you ahead.

[Caspar]

@Teegedeck
I did a 1pass CBR (default settings) with quant 3 just as your suggested (note that the p-frames are quant 3 but b-frames are quant 5, b-frame quant ratio and offset default). The fisrt time with all the denoisers removed and the frame turned out to be fine without blocking... :confused: I checked what the quant for the frame is and it turns out to be a quant 3 p-frame. Ok this make sense, so I did another 1pass CBR with quant using the 2 denoisers. The result was a frame with blocking... (quant 5 b-frame) Just to make sure I didn't do something stupid, I did another 1pass without the denoisers and the frame shows blocking again... (quant 5 b-frame).

I will do a 2 pass with quarter pixel to see if it will help.

[ChronoCross]

Actually I'm almost thinking it might be a deinterlacing issue. I've seen problems like that whenever I use a wrong deinterlacing setting. it's not a deen issue, nore is it a convolution issue. A source like Tenchi Universe has near the same problem witht he blocky lines. if you deinterlace it and try to return it to Film speed then it blocks the lines. but if you use just use Telecide(order=1,guide=0) it comes out looking fine. that is my suggestion. a script like this purely for testing.

MPEG2Source("source.d2v")
Telecide(order1,guide=0).Decimate(mode=2, cycle=5) //possibly hybrid, will work on normal sources as well.
crop(8,0,-8,-6) ///maybe this ais the problem since your using an older avisynth with DUST? try the standard (8,0,x_res-crop,y_res-crop) format
Deen("a2d",2,4)//spatial Cleaning low thresholds
BilinearResize(640,480)
textsub("subtitle.ssa")

that shouldn't have any adverse affects. ooh I had another thought..you changed the curve compression handling

In the 2nd pass Xvid config Screen yu should set your settings
Overflow Control Strength %: 0
Max overflow improvement %: 5
Max Overflow Degregation %: 5

Setting them anything over than this might be a bad idea for anime sources. IMHO letting xvid decide all of those factors( by setting the first one to 0) is the best way to go.

Regards,
ChronoCross

EDIT: Don't use 1 pass encoding on anime sources. from my experience with anime it NEVER looks good. 2 pass is the way to go.

[Caspar]

@Teegedeck
More results from using quarter pixel. The settings pretty much remained the same as in my first post with the exception of enabling quarter pixel. Cartoon mode is disabled because it didn't make a difference in the above tests.

Script:

mpeg2source("d:\encode\my_encode.d2v")

KernelBob(1,8,false)
BlendBob(show=false,badFrames=0)
decimate(cycle=5,quality=3)

crop(8,0,-8,-6)

#denoisers

BilinearResize(640,480)

textsub("d:\encode\my_encode.ssa")


Denoisers:

convolution3d(preset="animelq")
deen("a2d",2,8,6)


quarter pixel with denoisers
Overflow Control Strength %: 10
Max overflow improvement %: 10
Max Overflow Degregation %: 10
result: blocking, same as above

quarter pixel with denoisers
Overflow Control Strength %: 0
Max overflow improvement %: 5
Max Overflow Degregation %: 5
result: blocking, same as above

quarter pixel without denoisers
Overflow Control Strength %: 10
Max overflow improvement %: 10
Max Overflow Degregation %: 10
results: blocking but not as bad (first pass projected size 261MB ~60% compression)
http://img.photobucket.com/albums/v415/LightFusion/After3_XviD1.png

quarter pixel without denoisers
Overflow Control Strength %: 0
Max overflow improvement %: 5
Max Overflow Degregation %: 5
results: blocking but not as bad

Encoding without the denoisers did help on the blocking problem some what, but it caused ringing in frames that were ok with the denoisers.

Edit: typos

[ChronoCross]

try using something other than

KernelBob(1,8,false)
BlendBob(show=false,badFrames=0)
decimate(cycle=5,quality=3)

try Telecide and Decimate. or Tomsmocomp and Decimate, or sagnom. or something. Try hitting "defaults" in xvid.

there is absolutely no reason for those blocks to appear with the settings your using for denoisers.

It has to be the deinterlacer. you could also try just using the denoisers and skip deinterlacing all together. see if the blocks still appear.

Quarter pixel is simple motion estimation. and in that scene has absolutely no "real" effect(on that particular scene as it's not even a real motion scene). if you really want to find the problem do the following

Start with a simple script
MPEG2Source("source.d2v")

and simply test out each filter one at a time. add one. encode...remove encode again and compare the results.

[Manao]

ChronoCross : it is not the deinterlacer. It's simply that q5 bframe is too much for that particular frame. Perhaps motion isn't well catched ( dark blue, dark brown --> very hard to catch motion )

Caspar : try to change b-frame settings ( lower the ratio, or the offset )

[ChronoCross]

I actually have an additional question...why do you have your iframe interval set to 240? isn't that too low? IDK might just be my imagination or perhaps I read it somewhere.

[dragongodz]

you could also try msmooth plugin instead of the other denoisers. i have found it does pretty good on anime. i usually use lower than its default values though so be warned you may want to play with its settings before you find what you are happy with. testing it on defaults though should give you an idea though.

you could also try reducing B frames to 1 and see what happens(i always use 1 myself :) ).

why do you have your iframe interval set to 240? isn't that too low? IDK might just be my imagination or perhaps I read it somewhere.
i read before that its best to leave it large(240 isnt small) since xvid will decide when to insert I frames anyway. so if you have a static scene for a long period of time you wont have lots of I frames used when they are not needed. the difference between 240 and the default 300 will make bugger all difference really though. infact its also been said before that a good idea is fps x 10seconds so if the film is 24fps then 240 is a reasonable number. :)

Quarter pixel is simple motion estimation. and in that scene has absolutely no "real" effect(on that particular scene as it's not even a real motion scene).
for anime suposed flat colour areas can have shifting hues though. these can sometimes be picked up as motion. so trying it without Qpel is again a reasonable thing to do.

[Teegedeck]

@Caspar: It is not quite so easy to catch the same frames in VDub; don't simply load the avi, but make an avs like

directshowsource:("C:\your.avi")

Activate ffdshow's OSD, so it will print frametype and framenumber - load that script into VDub for your screenshots.

[lamer_de]

If it's only that particular scene, why not use a zone for it, when you discovered it looks ok @ quant 3? Setting the b-vop sensitivity to negative values and using a low fixed quant should do the trick.

If it's through the episodes,it seems that your denoisers are too strong and xvid overcompresses specific dark frames. Use less denoising (or other denoising methods) then.

CU,
lamer_de

[Caspar]

@lamer_de
The problem is throughout the whole clip, it's not limited to that one scene. Yes, I have tried using less denoisers and even none at all, but it doesn't help. :(

Ok, after more testing, the following solutions worked for me:
1. Lowering the B-frame quant ratio or offset
2. Using 1 B-frame instead of 2

I used the latter and the results are pretty good! I figured that b-frame are supposed to be highly compressed frames, but what's the point in using them if I had to use the same quant as my p-frames on them (please correct me if I am wrong)? :confused:

Thanks for all the input people, really appreciate your help. :thanks:

[Aktan]

For anime encodes, I use the follow b-frame settings:

max 1 bvop
quant ratio 1
quant offset 0
bvop sensitivity -19.

Since u tried pretty much everything that i just said except the bvop sensitivity, maybe just try plain that and see if that helps.


With this setting ~44% of the encode is still b-frames, and the avg b-frame size is ALWAYS lower than the average p-frame size.

[ChronoCross]

what anime are you encoding? I'm considering buying it just so I can work on it hands on.

[Teegedeck]

Originally posted by Aktan
For anime encodes, I use the follow b-frame settings:

max 1 bvop
quant ratio 1
quant offset 0

This isn't recommendable. It simply makes both compressibility and picture quality worse. Please do a search to learn the reasons.

[Aktan]

Originally posted by Teegedeck
This isn't recommendable. It simply makes both compressibility and picture quality worse. Please do a search to learn the reasons.

It is well known that encoding "anime" is different than encoding "live" material. I have not search the forum, (I'm lazy :D ) but I think the conclusions were made for only "live" material. You said the quality is worse and yet Caspar has "fixed" his problem due to the (almost) same settings as I use. If indeed the problem is "fixed", the quality is acually better! Though your probably still right about the compressibility.

[Manao]

Caspar lowered the b-frame offset and quant ratio, he didn't say it lowered them down to 0 and 1 ( which makes b-frames have the same quantizer as p-frames, which is half the time worst than having only p-frame ).

[Aktan]

Originally posted by Manao
Caspar lowered the b-frame offset and quant ratio, he didn't say it lowered them down to 0 and 1 ( which makes b-frames have the same quantizer as p-frames, which is half the time worst than having only p-frame ).

But Casper did say:

Originally posted by Caspar
I figured that b-frame are supposed to be highly compressed frames, but what's the point in using them if I had to use the same quant as my p-frames on them (please correct me if I am wrong)?

And from what I have seen, the avg b-frame size is always smaller than the avg p-frame size. Unless the xvid stat window is off. (Sorry for resaying what I said above)

[Manao]

My bad, I overlooked that part of his post.

Indeed, even at same quant, b-frames are smaller than p-frames, because they are predicted better. However, they increased the distance between two p-frames, which means p-frames will be predicted worse than without b-frames ( hence higher filesize than p-frames without b-frames, and than b-frames in the same clip )

The two effects tends to nullify each other when quants are the same ( but sometimes quality is improved ), however, with a lower quant for b-frames, overall quality at fixed filesize improves almost always.

Perhaps Caspar's clip is one exception to the rule.

[Aktan]

As you know, anime has a lot of repeat frames. So if a b-frame is a repeated frame, the prediction acually has not changed much. (This is a reason why N-VOPs should be here). On the parts where there is a change from frame by frame, I have (I think) set the b-frame sensitivity low enough so that the prediction are not huge two frames forward and still save with b-frames without the bad frames Casper has gotten.

I think I have a clip where it seems worse with B-Frames at lower quants too, so it may acually happen more in "anime" vs "live" video.

[dragongodz]

actually Casper said this
Ok, after more testing, the following solutions worked for me:
1. Lowering the B-frame quant ratio or offset
2. Using 1 B-frame instead of 2

I used the latter and the results are pretty good!
that is lowering the ratio and offset when using 2 B frames worked OR using just 1 B frame with default ratio and offset worked. 2 solutions, you do not need to do both.
i would reccomend just using the 1 B frame instead of screwing around with the ratio and offset. infact i already did earlier in this very thread. :D

[stephanV]

Originally posted by Aktan
As you know, anime has a lot of repeat frames. So if a b-frame is a repeated frame, the prediction acually has not changed much.
On still images b-frames are actually less efficient than p-frames. There is nothing you can gain from using an extra (similar) image to predict a still image. Certainly not when you use the same quantizer.

Using the same quantizer for p and b frames usually lets the p-frame sizes explode.

While I'm not doubting Caspar's results, i'm wondering if using no b-frames at all might also be a workable solution. In general such b-frame settings are not recommended. Using only one b-frame is still ok too though. :)

[Aktan]

Originally posted by stephanV
On still images b-frames are actually less efficient than p-frames. There is nothing you can gain from using an extra (similar) image to predict a still image. Certainly not when you use the same quantizer.

I'm not too sure your right about that. During a black screen area, I usually see a quant 2 p-frame around 169 in size while a quant 2 b-frame around 9 in size. This leads me to believe b-frames still are smaller.

Originally posted by stephanV
Using the same quantizer for p and b frames usually lets the p-frame sizes explode.

I understand that it may explode due to a bigger prediction, but as i said above, a repeated frame does not make a bigger prediction.

Originally posted by stephanV
While I'm not doubting Caspar's results, i'm wondering if using no b-frames at all might also be a workable solution. In general such b-frame settings are not recommended. Using only one b-frame is still ok too though. :)

I have forgotten to test plain no b-frames. You might be right, but i think you save a little bit still with b-frames enabled, even if the p-frames are more bloated for the scenes with out repeated frames.

@dragongodz

yes, I saw he said OR :sly: and was hoping noone else noticed it. But I have seen b-frames create the bad looking frames still with only 1 b-frame, which was why i recommended both.

[stephanV]

Originally posted by Aktan
I'm not too sure your right about that. During a black screen area, I usually see a quant 2 p-frame around 169 in size while a quant 2 b-frame around 9 in size. This leads me to believe b-frames still are smaller.

Yes b-frames are smaller than p-frames... but the p-frames are much larger than they should be.

Just did a small test

avisynth script

blankclip #produces black screen for 10 seconds at 24 fps, 640x480


I did a quant 4 encoding of this file in VirtualDub 1.5.10 with XviD (1.0.2 of 5 september by koepi), all setting were "out of the box" except the b frame settings, results:

no b-frames ---> 4 kbps
2/1.5/1 ---> 13 kbps
2/1/0 ---> 13 kbps

i will try on a real image too if you want :)

as a side note, if XviD b-frames are adaptive... this is also a bit weird behaviour...

[Aktan]

Your right. On a pure black no change area, pure p-frames are smaller. (I tested it too just now). But I also tested a pan with my settings vs pure p-frames, and the results were different.

So please try my settings on an anime source (i never said these work for live material :) ) and compare. This includes the change in b-frame sensitivity to -19 and max consect b-frame to 1. (I think now that it doesn't work well unless all the settings are changed, though I have not tested it yet.)

By the way, I think when VHQ is implemented for b-frames, then b-frames can be quantizied higher, but until then.... :D

[Teegedeck]

I've been using 1.1 with VHQ for B-frames since July now. Wouldn't want to go back.

[stephanV]

Originally posted by Aktan
Your right. On a pure black no change area, pure p-frames are smaller. (I tested it too just now). But I also tested a pan with my settings vs pure p-frames, and the results were different.
But pans are not still images, far from :p

anyway, ive never done any anime, so i cant tell, but... stills are stills i think. it's nice to have discussion though. :)

BTW - VHQ for b-frames doesnt change this i think. as the loss lies in the p-frames, not in the b-frames.

[Aktan]

@Teegedeck

I guess I should try a newer build, though 1.0.2 is considered more stable (I think), which is why I was waiting for 1.1 to be more stable first.

@stephanV

Sure, pans are not stills, but hardly anything is completely still. Even the anime "repeat" frames have some tiny movement.

I guess I'll try these settings on a still anime scene and report back :) .

[Aktan]

Well, I have tested on a very small clip of 3 seconds of "still" anime, and found the size of the encode with the b-frame settings I usually use to me smaller than pure p-frames.

Done with 1-pass with "discard 1-pass" disabled (quant 2), GMC and qpel enable, though there were no s-frames, VHQ 4, chroma motion enabled

With 1/1/0 B-frame (-19 b-frame sen): 286 kb
Only P-frame: 312 kb

I know this is inconclusive and I will probably do a full encode, but if the trend is the same, then i can see some savings still.

Sorry, I don't have a place to post the clips.

[dragongodz]

yes, I saw he said OR and was hoping noone else noticed it. But I have seen b-frames create the bad looking frames still with only 1 b-frame, which was why i recommended both.
i would still reccomend trying with just 1 B frame BEFORE reccomending screwing with settings that some people(especially newbies) may not understand. :D

Even the anime "repeat" frames have some tiny movement.
yes shifting hues on what is meant to be essentially a large flat area can be picked up as motion. its best to reduce that sort of thing when possible. also my personal experience says that Qpel is not always anime freindly so i never use it.

[Caspar]

Originally posted by Teegedeck
I've been using 1.1 with VHQ for B-frames since July now. Wouldn't want to go back.

Are you saying I have been using VHQ 4 for no reason up till now? :o

[Manao]

In 1.0 branch, VHQ only applies to p-frames. B-frames can benefit from VHQ, but indirectly, because p-frames' quality is slightly higher, which makes that b-frames are slightly better compensated. But it's only a side effect, whereas in 1.1 branch, RD optimizations ( aka VHQ ) are made while compressing b-frames.

[Aktan]

Originally posted by dragongodz
also my personal experience says that Qpel is not always anime freindly so i never use it.

What does Qpel sometimes do that makes you say that?

[Peter Cheat]

Because Qpel tends to make the resulting image more noisy. This is good for 'real' sequences (makes it appear sharper?), but for anime it doesn't look good (smooth is preferable). Qpel will almost always make first pass larger than without for anime/cartoon, requiring increased quantisers for second pass that will reduce quality a bit.

[Aktan]

Originally posted by Peter Cheat
Because Qpel tends to make the resulting image more noisy. This is good for 'real' sequences (makes it appear sharper?), but for anime it doesn't look good (smooth is preferable). Qpel will almost always make first pass larger than without for anime/cartoon, requiring increased quantisers for second pass that will reduce quality a bit.

I see. Now for a low resolution, would qpel then be more helpful due to more possible savings, even if it is at the cost of being more noisy?

Results from a 21 min encode:

p-frame only: 297957 KB
with b-frame 1/1/0: 285574 KB

I guess the savings isn't much even though it is there.

[Manao]

Aktan : filesize alone doesn't matter. You should check visually if you see a difference between the two clip.

For the qpel, it's the more versatile option of XviD, imho. It often helps, but they are cases when it makes things worse. In doubt, try it by yourself.

[Aktan]

Originally posted by Manao
Aktan : filesize alone doesn't matter. You should check visually if you see a difference between the two clip.

Visually, they both look the same, though it is expected since it is constant quant 2. Plus a b-frame with the same quant as a p-frame, I thought, is suppose to be a higher quality than a p-frame.

Obviously this is just one test. I don't know if the b-frame settings 1/1/0 on anime is always smaller than pure p-frames. But from all the small tests i have done, it seems the only time pure p-frames is more compressible is when there is absolutly no motion, like the black screen.


It seems that I have taken over the thread :p . My bad. Casper, is everything all fixed now?

[Manao]

Plus a b-frame with the same quant as a p-frame, I thought, is suppose to be a higher quality than a p-frame.Indeed, but then the surrounding p-frames will have a lower quality, because they are not consecutive temporally. So it's in fact very hard to judge. The same test could be done at quant 4 or 5, in order to perceive a visual difference. Or psnr could be used, to judge them objectively.

[COREiP]

Blockbuster filter can prevent blocking http://www.indeus.com/sansgrip/avisynth/
If you use it right with other filters such as deen and msmooth u can achive great results.