x264 matrices advice

[egrimisu]

Hi i encoded an anime and my settings where :

DGDecode_mpeg2source("D:\work conan\01\Mirai_Shounen_Conan_01.DVD(MPEG2.AC3)[Misu]_Track1.d2v")
setmtmode(2)
source = last
backward_vec1 = source.MVAnalyse(isb = true, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec2 = source.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec1 = source.MVAnalyse(isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec2 = source.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
source.MVDegrain2(backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400,idx=1)
fft3dfilter(sigma=0.4, bt=1, bw=32, bh=32, ow=16, oh=16, plane=4, dehalo=0, ncpu=2)
gradfun2db(thr=1.2)
crop (4,0,-4,0)
spline36resize(720,480)


and my encode settigns where:

program --pass 2 --bitrate 2000 --stats ".stats" --ref 16 --mixed-refs --no-fast-pskip --bframes 4 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 7 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me tesa --threads 2 --thread-input --progress --no-psnr --no-ssim --output "output" "input"


What matrices to use for gaining video quality for anime at 1500-2200 bitrate( i have anime that i compress to 1550, other 1700, other to 200, and othes to 2200).

if i set a the bitrate so the final file to be 400 per example i get 320, WHY ??? how shall i tweak quantitizers to get 400mb file. i post a screen so you could se that those 80mb could be used.i get blocks in dark areeas

http://img147.imageshack.us/my.php?image=snapshot20080731234501bm4.png

[Ranguvar]

1.) This should be in the MPEG-4 AVC forum.
2.) Anime has almost always done best with the flat quantization matrix.
3.) Custom quant matrices are being phased out anyways; improvements like VAQ, Psy RDO, and FGO have made CQMs obsolete pretty much.
4.) I recommend using --threads auto, it will use more threads than your CPU number and usually improve CPU usage. Also, --thread-input usually slows down multithreaded scripts like yours in my experience.
5.) --weightb isn't recommended for anime.
6.) What revision of x264 are you using? What patches? I recommend the build bob0r recently posted in the custom patches thread, of r928 with Psy RDO 0.5.
7.) You might want to use BicubicResize instead of Spline36; a softer resize will help out the encoder a lot, as it doesn't need to encode so many sharp edges.
8.) I would boost --bframes up to 16; it lets the encoder make smarter decisions about how many bframes to use in a row, since you obviously aren't targeting standalone compatibility with 16 refs.
9.) --me tesa is complete and utter overkill. I highly doubt you'll be able to tell a difference between that and --me umh or perhaps --me umh --merange 32 without looking at frame-by-frame comparisons.
10.) Again, since you aren't targeting standalone compatibilty, try adding p4x4 - it may help very slightly, since you seem to be wanting the best possible quality.

[egrimisu]

Thanks, i'l try in the weekend these and i will come back with a feedback

[Sharktooth]

custom matrices are no longer necessary since AQ (VAQ) is now on by default.
also custom matrices dont make miracles. you cant GAIN quality. bits are subtracted from some areas to be added to other areas of the picture, so some qulity is subtracted somewhere to be added elsewhere. that's what x264 AQ (VAQ) makest too and the reason why CQMs and VAQ may conflict. VAQ is simply better since an algorithm decides where to syubtract and where to add how much bits, even across multiple frames. While CQMs work on frequencies in a single frame with fixed coefficients.
the same is true for xvid builds with VAQ.
also default quntization is usually better than any custom matrix for anime content...

[egrimisu]

got a good tip for setting qunt for anime? i cand see that the defaul is 10/51/4

[Ranguvar]

That should be good in 99% of all cases with any source.

[gav1577]

@ Ranguvar so its best for quality to remove --weightb from both passes of a 2 pass encode if your source is anime ? :)

[Ranguvar]

Yes for now, but with the new b-frame decision patch, if the anime has lots of fades, then use --weightb on both passes :) weightb can hurt the choosing of optimal references for anime/cartoons, as I understand it, but if the source has lots of fades, and you're using the new b-frame decision, weightb can help.

[gav1577]

Yes for now, but with the new b-frame decision patch, if the anime has lots of fades, then use --weightb on both passes :) weightb can hurt the choosing of optimal references for anime/cartoons, as I understand it, but if the source has lots of fades, and you're using the new b-frame decision, weightb can help.

Thanks for the advice much appreciated :D

[Ranguvar]

Course :D Just don't sue me if I'm wrong :p

[*.mp4 guy]

Quantization matrices are really only able to trade off between artifacts, so they work best when you have lots of "artifact Y" that you would rather replace with, for instance, equal amounts of several artifacts.