BLOCKY ARTIFACTS in cartoons... Need quick & dirty solution!!

[_Vengeance_]

I need a quick-and-dirty solution to make these artifacts go away! :scared:

EXAMPLE CLIP:
http://www.acm.vt.edu/~clint/download/dexter-test-1200kbps-resize-512x384.avi

It is only 2 megs. Shouldn't take long to download. :)


My problem is these blocks. They occur on almost any black/white animation frame. Notorious shows are The Brak Show, Dexter's Lab, Family Guy, even PowerPuff Girls.

I think I must be missing something obvious.

This doesn't occur if I use Xvid CBR.
But I want to continue doing things the way I've been doing them.
I've encoded some 250G of video and want to be consistent.

[_Vengeance_]

Another sample. (Ear)

[_Vengeance_]

Last sample.

[_Vengeance_]

Okay, I don't see my samples, so here are some URLs where they can be grabbed at:

Bad Ear:
http://www.acm.vt.edu/~clint/download/dexbad0.bmp

Bad Thumb & Fingers:
http://www.acm.vt.edu/~clint/download/dexbad1.bmp

Bad Head!:
http://www.acm.vt.edu/~clint/download/dexbad2.bmp

Notice how these are typically in dark/bright border areas.

I encode 1200kbps CBR. But this has happened at all bitrates and resolutions I've ever used. This doesn't happen with Xvid.

But I want to continue using divx3/4. I don't want to change. I have a 6000 show VHS collection and I've already encoded hundreds of shows, over 250 gigs of video. I want it to be consistent.

This has happened at all resolutions. My old, 352x240, and my new, 320x384-resized-to-512x384 (because I drop frames if I cap directly to 512x384 and these look AWESOME.)

I'm not intersted in the slighest in saving space; that's why I only do 1 pass encoding. I have 24000 times the storage space now that I had in 1988 (480G vs 20M). In 2016 I expect to have 24000 times the sapce I have now. :devil: I think concentrating on making files small is a waste of time.

But I'd really like the files to LOOK GOOD. These blocks are killing me. Everytime I watch a show, one of them comes on the screen, and I find my eyes drawn to that part, and I stare only at the small block and ignore the rest of the (nicely encoded) screen. IT'S REALLY KILLING ME........... :stupid:

Please, someone, anyone, help me fix this.

Don't suggest for me to change programs. I only use virtualdub.

I'm just wondering if there is some setting I could change that I am missing. I use quality 76. Anything else I should say?

[Acaila]

In case you didn't know, attachments have to be approved before they show up...

As for your pics, all areas in a frame that have a high color saturation (as red areas usually have) are notoriously hard to compress without artifacts.

Since you're encoding cartoons, why don't you use some smoothing? Cartoons/Anime always benefit greatly from that, because it makes the large even-colored planes look much better.
I suggest some TemporalSmoother and/or SmartSmootherHiQ to clean everything up. No need to use a high setting, that would blur too much, but even low settings can do wonders for cartoons.

Also, if you're not concerned about filesize why are you then only using 76% quality? That would result in an average quantizer of what? 4? 4.5? That's actually quite bad. If space is no concern go for at least 90%, and the smoothers I mentioned above should help to reduce the size even more.

Capture in HuffYUV (you can probably capture at a higher resolution now) and later compress and resize into what you want.

[_Vengeance_]

Those examples WERE with a SmartSmoother AND a Temporal Smoother, and WERE captured using HuffYUV codec. :P

I haven't tried SmartSmootherIQ yet though.

So... Next suggestion anyone? :confused:


As for quality...I used to use 100% quality; Way too blocky. Looks much worse. That's why I now use about 76%. Are you saying raising it would make the artifacts go away? Because that's all I'm really interested in at the moment. I've encoded about 250G of shows and am happy with the quality.

[Teegedeck]

A thought: You already stopped 'being consistent' when you moved from DivX3 to DivX4 as DivX4 is (nearly) MPEG4-compliant while DivX3 isn't at all. This is a big step. DivX4 and XviD on the other hand are much closer to each other than DivX 3 & 4; they are compatible MPEG4-codecs. And as you've twice said that the problem doesn't occur with XviD - so I don't think you really have a problem where you see one.

[_Vengeance_]

Thanks for all the sugestions.

That is a good point about the codecs, I didn't know that. I generally fear change. Still... I'd like to be able to find a solution using my existing system rather than revising my system. I REALLY like my existing divx codec, except for the artifacts.

Noneteless, I am researching revising my system anyway.
(ie recompressing my problem-raw-huffyuv-100M clip with different parameters to see which ones will make the artifact go away)

[[[Since I am *constantly* compressing stuff, I don't like to stop long enough to revise my system. I'm more concerned with getting it all done ... (in 1.5 yrs of near-constant compression I've done about 10% of what I want to do.)]]]

:devil: Quality 90 (vs 72, I incorrectly said 76 above) doesn't help.

:devil: SmartSmootherIQ (vs SmartSmoother 1.1) doesn't help.

Xvid helps tho. I'm wondering what is different.

(Yes yes I know Xvid > Divx but I'm stubborn and don't want to use it nonetheless).

[As an aside, do people have to install an Xvid-codec.exe to be able to view xvid-encoded stuff?]

[Teegedeck]

Originally posted by _Vengeance_

[As an aside, do people have to install an Xvid-codec.exe to be able to view xvid-encoded stuff?]

Not really, no. You only have to change the fourcc-code of an XviD-.avi (with a program like 'fourcc-changer') to make the DivX-directshowfilter play back the XviD file - and vice versa. There's also an option in XviD's config-tab that lets you decide to produce XviD-.avis that have a DivX5 or 4-fourcc from the start. I'm not sure though if DivX5 finally is able to decode MPEG-quantization correctly. If not, you have to use H.263-quantizer, always. But H.263 is preferable for Animation, anyway.

Though I would always use XviD's own DSF as the quality is better.

[EDIT:] And of course ffdshow plays both Xvid and DivX. It could be interesting to find out whether your files exhibit the same artifacts if played back by the XviD-DSF or ffdshow.

[Acaila]

The only thing left that I could find was:
I'm not interested in the slighest in saving space; that's why I only do 1 pass encoding.
Using 2-pass mode isn't about saving space, it's about making efficient use of the space you've got. Right now with your 1-pass mode the codec can look ahead as far you've set your Rate Control settings. When you use 2-pass mode the codec will be able to look ahead and behind to the entire video.
This way it can handle bit allocation much more intelligently, and the overal quality will increase. The frames you've shown here are very hard to compress and thus will need a lot of bits to look good. Using 1200 CBR on this frame obviously isn't enough, but with 2-pass it can strip easier frames to give the hard frames better quality.

Since you've saved your captures in HuffYUV anyway, it's not much extra effort to run 2 passes (except for the time of course) to increase the quality.

Another thing, DivX3 sucks with 1-pass CBR mode. That's a plain fact. The only way DivX3 can compete with XviD/DivX5 is by also using a 2-pass mode, SBC. In every codec the 2-pass mode is superior in quality over the 1-pass modes.

[_Vengeance_]

Put it this way: If I had to make 2 passes, it would take me twice as long to encode my stuff, right? I'm already being held back by the amount of time it takes; it is going to take me at least 5 years to do all my VHS tapes, if not 10. Many times I have trouble encoding one day's captures by the next day -- and I need the space back to get the next day's captures. I'm missing stuff daily. Taking twice as long to encode would make me miss even more stuff.

I'm not interested in making it take longer just for better quality-per-byte. I'm simply interested in making the artifacts go away, and mass-encoding as much stuff as possible. I've run some statistics on my encodes and have made 700M of AVIs every day (sustained average) for almost a year....

ANYWAY... I'm not here to discuss 1-pass vs 2-pass, that is for another forum. I want to make the artifacts go away. I have a hard time believing it's impossible to do in divx 1-pass mode, since Xvid make them go away in 1-pass mode.

Also, about the bitrate comments: increasing the bitrate doesn't make the artifacts go away, I've done several experiments encoding that exact same clip at many different bitrates. Long before I ever came here for help. (BTW, using 2-pass divx 5, it simply blows goats. And I'm sure it's because I haven't messed with the 14,000 settings, but I'm simply not intersted in messing with that much stuff just to encode.)

Just for reference, I make a template VCF file that i apply to ALL my encodes (Well, 4 actaully, vidcap-toon, vidcap-live, vhs-toon, vhs-live.)

<tangent>
And before anyone says it, I'll acknowledge myself that I'm stubborn. Then again, I attribute my stubbornness as one of the main reasons I have 6000 shows on VHS, 1200 cds of shows offline, and 250G of my own encodes ... so I think it's a good trait. :devil:
</tangent>

[Teegedeck]

Originally posted by _Vengeance_

I want to make the artifacts go away. I have a hard time believing it's impossible to do in divx 1-pass mode, since Xvid make them go away in 1-pass mode.

Heh. Why? Codecs do have bugs... Huffyuv it was, you said? That kind of surprised me. DivX4, on the other hand, is reknown for its bad motion-estimation




And before anyone says it, I'll acknowledge myself that I'm stubborn. Then again, I attribute my stubbornness as one of the main reasons I have 6000 shows on VHS, 1200 cds of shows offline, and 250G of my own encodes ... so I think it's a good trait. :devil:


Heh (2). I hope you don't find yourself saying some day 'WAHHHH! I could have done all this in much better quality - I have to do it AGAIN!!' ;) Just joking. But I really think Acaila is right warning you of CBR; stay with quality-based encoding.

[_Vengeance_]

Is there a way I can force a divx3 and not 4 encode with Virtualdub?


Anyway, to me, CBR and quality-based do not necessarily mean different things. If I use a high enough bitrate, it is indistinguishable from the raw HuffYUV capture... I'm more concerned about speed.


And yes... some of my earlier stuff I already find myself redoing... But I figured something like this would happen, so I made it a point not to focus on the stuff I wanted to encode the most (as much as I could have) and spent a lot of time encoding stuff that I didn't really WANT to encode as much, just to get practice. And I can always re-watch the VHS tapes and re-encode them if I want. But VHS is already crappified anyway. :)

My new vidcap captures look good enough for me... I watch them on a 36' TV and they are better looking than a VHS tape. I'm happy.

You all have valid points..It's just those damn blocks that are killing me, hehe. If it weren't for them, I would be 100% happy and not change the way I'm doing it at all, EVER.

[_Vengeance_]

P.S. The artifacts do not show up in the HuffYUV capture, just the divx recompress. And not xvid either.

It seems to me that using xvid but forcing a FourCC of divx would be bad. It just seems like cheating...

[Teegedeck]

[laughter] I like your way of thinking. Anyway, OFF YOU GO THEN, download XviD and forget about DivX3/4, you have no choice! :D The only thing that matters is, you can turn both your DivX4 and XviD captures into MPEG4-streams that you can watch on a hardware-player, someday. Then you won't have to worry about fourccs, anymore.

And CBR IS much worse then quality-based encoding: Say you have a bitrate of 1600kbps; the encoder then will watch out not to use more than that over a given timespan that has to be quite short because otherwise it takes the risk of producing files that are seriously bigger than expected. So logically it has to be quite carefully about how much bits it can assign to a big frame (many bits equal high quality equal low quantizer). Whereas both quality-based encoding and fixed-quant-encoding don't give a toss about bitrate but about quantizers only - small frames receive few bits, big frames receive many bits. Two-pass basically does the same thing but also hits a desired filesize.

And yes, you can enforce constant quantizers on DivX3. Use Nandub, go to the DRF-table and set both min. and max. DRF to your desired quantizer. Or so I think. It seems a terribly long time ago since I last used Nandub for anything but muxing or cutting. I have no idea what happens if you use it like that without a .stats-file from a first pass. You could of cause get a constant DRF=2-encoding, eaasily by keeping the 1st pass' avi.

[_Vengeance_]

Well I definitely get your point... I mean after all, I use VBR encoding when ripping cds. :D

I just don't WANT to... I'm a stubborn mofo...As Homer Simpson said "I'm happy with things the way they are" (Except when there is an ugly block on homer's face!) c'mon man... Give me a break :) :) :) :)

<tangent>
One thing: I don't believe in hardware based players. :devil: I will never buy another cd player again after Best Buy did not honor their warrantee and fix my $400 100-cd changer (they insist it's not broken when it is -- after taking it in 6 times over 2 years until the 4-year warrantee expired -- they never even TRIED to fix it). I also will never buy another vcr again, and have not and never will buy a standalone dvd player (maybe a dvd-rom someday).
</tangent>

Die blocks, die! Maybe I'll have to use Xvid.. NOooooooooooooooooooooooooooooooo

[_Vengeance_]

Oh also, maybe a definition of "Quantizer" would help me out. I've seen a lot of talk but nothing in terms *I* have as of yet comprehended. :)

[Teegedeck]

Either that or draw your own cartoons... ^_°
Duh...!

[_Vengeance_]

<tangent>
That would be my dream job, except I can't draw. :)
I could be a writer, but there's no glory in that. Look at what happened to the Futurama writers. :(
</tangent>

[Teegedeck]

Oh, quantizers are kinda the 'filters' by which this frequency-stuff that video consists of get simplified after it underwent 'digitization'. The higher the quantizer ('DRF' in Nandub), the more 'simplification' gets acted upon the frequencies, especially the high frequencies that stand for small details. Hope I got all that right. In XviD's config-tab, you can actually have a look on what a so-called 'quantizer-matrix' looks like and you can alter it too (of course :) - that's just so typical of XviD where you can alter just about anything so that new users get really, really scared...).

[Acaila]

Ok...last suggestion.

Increase the Rate Control Averaging Period (for DivX4 anyway). This will give the codec more frames to base the average bitrate on, and 'might' increase those unicolored areas by taking away bits from other frames.

I suggest setting it to half the number of frames the entire video has, increase beyond that if you don't get the desired result. However this could totally mess up your average bitrate and filesize, but you can't have everything :).

XviD can do this correctly, because it has much, much better rate control and motion estimation build in.

EDIT: Wow! You two are discussing things faster than I can think and write up an answer :D

Oh, quantizers are kinda the 'filters' by which this frequency-stuff that video consists of get simplified after it underwent 'digitization'. The higher the quantizer ('DRF' in Nandub), the more 'simplification' gets acted upon the frequencies, especially the high frequencies that stand for small details. Hope I got all that right.
lol!

That's the short, short version :D

[_Vengeance_]

I must be encoding in divx3 because I don't even have that option... All I have is bitrate & quality.

[_Vengeance_]

I must be encoding in divx3 because I don't even have that option... All I have is bitrate & quality.

One thing that has made me avoid the more sophisticated encoders is I really don't want to get into the specific-settings-for-each-encode business.... I just want a template that I can shove everything into, forever.

[Teegedeck]

simple and forever...

Yeees, that's why I carve my letters into stone instead of using a text-processor! :)

Ahh, today was such a dead day on the forum, it has really been fun to talk to you; and thanks for your patience, Acaila!

[manono]

Hi-

Interesting thread. I must have skipped over something, but has no one suggested using anti-shit in the DivX 3.11 encodes? Try 16, and if that doesn'r get rid of most of those inverted blocks, then try 20 or so and you should be OK. And maybe set Min. Qual. to 1. Can't you do that in the one pass encodings he's doing?

But that's one reason I switched to Xvid-don't have to worry about the damn things anymore.

[PeterTheMaster]

yes, this is a case for anti-shit, but doesnt this make the encodes take longer? or is this only for dvds since one cant fast recompress?

what are you doing with all these shows? do you ever have time to watch them again? do you record all tv stations all day long?