CQMs and high motion animated footage: a good mix?

[Chainmax]

I about to start encoding my X-Men DVDs (the few one that have been released of the 1992 series) and will do them at 640x480 with an average bitrate of about 1440kbps. Given that this is a mid/high bitrate but the cartoon has a fair amount of high motion, do you think it would be beneficial to test CQMs on it? If so, which ones would you recommend me to try?

[*.mp4 guy]

You should only consider using a cqm if there is no ringing at the bitrate your using, or if you (unlike most people) don't mind ringing as much as banding/blocking. I very rarely encode animated material, so I don't have any suggestions as far as matrices are concerned.

I tried making a matrix for cartoons, but I couldn't get the ringing under control, and the gain in sharpness was of negligable usefullness on all of the material I tested with. Unless you are shooting for transparency and need that last little bit of extra quality in gradients and other subtle areas, I really don't think using a matrix on animated content is a good idea; but then again, people use aq on low bitrate anime, which I would also reguard as a bad idea, (aq increases ringing, due to harsher quantization of things that aren't flat, or close to flat) and they aparently like the results.

[Dark Shikari]

In anime, the main indicator of quality is the sharpness and cleanness of edges.

Anything that increases ringing, therefore, is probably not a very good idea.

[Chainmax]

But ringing shouldn't occur if one is using a high enough bitrate, right? I am planning to use VAQ though, and I'd rather use that than a CQM if I need to choose between both.

By the way, has MeGUI not implemented VAQ settings dialogue boxes or have do I need more powerful glasses?

[LoRd_MuldeR]

By the way, where in MeGUI can I see/set the VAQ parameters? I looked through the options and haven't found it.

You have to use "Custom Commandline Options" for the time being, as there are no AQ-specific GUI controls yet.
It's hidden on the "Zones" Tab inside the video configuration dialog...

[Chainmax]

What CLI command (if any) should I include in order to use the default VAQ parameters?

[LoRd_MuldeR]

What CLI command (if any) should I include in order to use the default VAQ parameters?
You should only modify "--aq-strength" and only if necessary. Default is "0.5" and more than "1.0" may cause trouble.
There also is "--aq-sensitivity" with default of "13", but it should not be touched (will be removed when VAQ hits SVN).

[Chainmax]

I think you misunderstood me. I intend to use default values, does that mean I don't have to enter any custom parameters or do I still have to add --aq-strength 0.5?

[Adub]

If you don't specify values, then the defaults are used, meaning that AQ is on. At least, that is what DS said once.

[LoRd_MuldeR]

I think you misunderstood me. I intend to use default values, does that mean I don't have to enter any custom parameters or do I still have to add --aq-strength 0.5?

Since v0.48, AQ is on by default. Specifying "--aq-strength 0.5" is useless, as that's just the default value.

0.48: AQ strength 0.5, sensitivity 13 made the defaults. Updated to r736. Qcomp is now scaled based on AQ strength automatically.

[Chainmax]

Got it. Thanks for all the replies :).

[Adub]

However, AQ is usually not recommended for anime. At least, not DS's AQ, that is coming from the man himself. He infact recommend Haali's old AQ for anime as it is extremely strong on areas with large amounts of color. However, since this is a high motion anime, it is hard to be sure. Best to make your own tests and compare.

[Dark Shikari]

However, AQ is usually not recommended for anime. At least, not DS's AQ, that is coming from the man himself. He infact recommend Haali's old AQ for anime as it is extremely strong on areas with large amounts of color. However, since this is a high motion anime, it is hard to be sure. Best to make your own tests and compare.I've heard very conflicting statements on my AQ with anime; some people say its amazing, other people say its terrible, so it likely depends heavily on the source.

[*.mp4 guy]

I've heard very conflicting statements on my AQ with anime; some people say its amazing, other people say its terrible, so it likely depends heavily on the source.

Or on the bitrate. Phsychovisualy some things are a good trade off at high bitrates, but not low bitrates, or vice versa (deadzones vs trellis 2 comes to mind).

[Yoshiyuki Blade]

I've heard very conflicting statements on my AQ with anime; some people say its amazing, other people say its terrible, so it likely depends heavily on the source.

Yeah sources that aren't completely digitized with super clean-ness (as most animes are today) definitely see an improvement with AQ. I really haven't come across a source that doesnt benefit from VAQ in some sort of way.

I've also found a benefit from using Sharktooth's CQM, at least at lower bitrates.

[Chainmax]

I'm going to encode the intro with both VAQ defaults and disabled VAQ (I assume that's done by using --aq-strength 0.0?) and see how they compare. If you guys want, I can post both encodes so you can also have a look.

I am going to encode with the AE-MaxQuality profile but with Trellis 1, MSP 6 and No dct decimation, by the way.

[Terranigma]

Sure, i'd be interested in checking them.
Could you do 2 more encodes: 1 with the Prestige CQM, and the other one with Sharktooth's eqm_avc_hr CQM (using the same settings, same bitrate)?
Sharktooth has some more CQM's, but he didn't release them. Hopefully this'll be an incentive for him to release them to the public. =P

[Chainmax]

Sure, no problem. I assume Sharktooth's eqm_avc_hr can be downloaded from his signature, but where can I download the Prestige one?

[Terranigma]

Prestige can be found here (http://forum.doom9.org/showthread.php?p=989184#post989184). :)

[DeathTheSheep]

If worst comes to worst, you could always try AQ 0.46 (use strength 1.0) and compare it against results of using 0.48 and/or the Prestige matrix. AQ 0.46 + higher qcomp actually works rather well on animated material.

[Terranigma]

Actually, from my testing, 0.48 performs better than 0.46, but then again, I don't do 1 pass encodings; I use the the aq's like how they were meant to be used. :)

[hkazemi]

If worst comes to worst, you could always try AQ 0.46 (use strength 1.0) and compare it against results of using 0.48 and/or the Prestige matrix. AQ 0.46 + higher qcomp actually works rather well on animated material.

Where is AQ .46 available for download? Is it buried on your website somewhere (as you mentioned you intended to take over that version/branch)?

[DeathTheSheep]

Actually, from my testing, 0.48 performs better than 0.46, but then again, I don't do 1 pass encodings; I use the the aq's like how they were meant to be used. :)

Can I ask you what settings/source you used? Also, 1-pass doesn't necessarily represent a way AQ isn't "supposed" to be used (and neither is adapting your threshold for best visual and metric results, I might add).

Where is AQ .46 available for download? Is it buried on your website somewhere (as you mentioned you intended to take over that version/branch)?
Patch here (http://files.x264.nl/force.php?file=./AQ/AQ_0.46.diff), binary here (http://www.fileducky.com/rIhWCgXL/), includes my updated me-prepass and some other nice things, courtesy of wizboy11.

[Chainmax]

Ok, here are the clips. As stated, they were encoded with an up-to-date MeGUI using the AE-MaxQuality profile with an average 1440kbps video bitrate, Trellis 1, MSP 6 and No dct decimation. Since no one told me otherwise, VAQ was disabled by setting --aq-strength 0.0 as a custom CLI command.


http://rapidshare.com/files/92436294/VAQ_NO_CQM.mp4.html
http://rapidshare.com/files/92447318/VAQ_EQM.mp4.html
http://rapidshare.com/files/92452885/VAQ_Prestige.mp4.html
http://rapidshare.com/files/92503157/NO_VAQ_NO_CQM.mp4.html
http://rapidshare.com/files/92506542/NO_VAQ_EQM.mp4.html
http://rapidshare.com/files/92510199/NO_VAQ_Prestige.mp4.html

I apologize for using RapidShare if it's inconvenient to some, but I was using BestSharing and they seem to no longer be active. If anyone can recommend a better file sharing website, I'd appreciate it.


[edit]There are some spots that look quite awful, but that is (probably) due to inconvenience in the IVTC process I'm looking to solve.

[Yoshiyuki Blade]

Mediafire seems to be the most convenient choice at the moment imo. Anyhow, downloading the samples now...

[Chainmax]

So, has anyone been able to download the samples?

[Sharktooth]

i cant get the files from rapidshare...
im behind a NAT and my IP is not public, since i share it with other FastWeb customers rapidshare says im already downloading some other files when in fact im not...
... a small price to pay for a very damn fast DSL connection...

[Chainmax]

I'll upload them somewhere else then. Is MediaFire (like Yoshiyuki Blade suggested) ok? Also, could you suggest me some extra options?

[*.mp4 guy]

You might aswell put them all together, they aren't big enough to warrant separating them.

Anyway, I'll rate the ones I looked at.

Aq: a little rough around the edges sometimes, not usually to the extent of being annoying.

no aq: edges are a little better then the version with aq, but still not perfect, annoying banding in only two scenes (both look more or less perfect in the aq clip), the most noticible banding is on the blue background of the last scene.

aq+eqm: edges are very bad in some places, no noticible improvement in gradiants over aq+no cqm (there is hardly any room for improvement).

The source is so completely devoid of detail you could probably get away with much higher inloop settings without loosing anything. I would strengthen alpha deblocking until the aq and no-aq encodes exhibit the same level of ringing and leave it at that.

[Chainmax]

*.mp4 guy: thanks for the feedback, I'll post a version with only AQ and stronger inloop soon.


Everyone: I uploaded the samples to mediafire as a single package. Here's the download link: http://www.mediafire.com/?9typ9mniiyy

[Sharktooth]

mediafire is ok, you can try also http://wikifortio.com/
however using VAQ and my matrix is not a good idea though... since both do about the same thing (but in different way).
prestige should be used ONLY on grainy sources when everything else fails.

[Chainmax]

Were you able to download the files then? If so, what do you think of each result?

[Chainmax]

Anyone else?

[Sharktooth]

ooops... i missed your post... ok. ill download them tomorrow and report back.

[Sharktooth]

there are garbled frames...
however i prefer (first look) the no-VAQ no-CQM encode.

[Chainmax]

Yeah, those garbled frames are due to what I presume are errors in the IVTC I'm still trying to correct. So, you do not see any banding in the no-VAQ no-CQM encode that is corrected in the VAQ no CQM one?

[Sharktooth]

well, the overall quality is low for all the clips, however the VAQ encode shows blurry edges...

[*.mp4 guy]

well, the overall quality is low for all the clips, however the VAQ encode shows blurry edges...

It really depends upon how many scenes there are that will have trouble with banding, If there are a lot, then vaq is better (imo, obviously), if there are only those few in the begining, then, no vaq is better.

The overall low quality of the clips is due to the souce, which was obviously not very high quality; overall X264 was not the most significant source of quality issues.

[Chainmax]

Ok then, once I get the IVTCing issues ironed out I'll post two new VAQ_no_CQM encodes: one with a higher bitrate and one at current bitrate but higher deblocking settings.

[foxyshadis]

You could always make an encode with vaq off, find gops that have severe banding/blocking, and re-encode them with vaq on. vaq isn't a zone option so you can't use it like that, for now, but you can manually splice the new gops in.

[Chainmax]

That would be far too much work considering it would have to be done on 77 episodes.