x264: extreme quality on anime! [Archive]

Sirber

19th July 2005, 02:16

Hi

Just made my first real encoding on naruto episode 143 using x264 with realanime's defaults (all the fancy stuff, 3bf, inloop +3), 3-pass ABR. I used 536kbps with a 64kbps HE-AAC soundtrack. Watching the source (170MB) and the result (100MB), I couldn't say which one is the source.

Many thanks to x264 devs!!! :D :D :D
Keept the excellent work!!!

http://www.detritus.qc.ca/files/x264/snapshot20050718211820.jpg
http://www.detritus.qc.ca/files/x264/snapshot20050718211711.jpg
http://www.detritus.qc.ca/files/x264/snapshot20050718211949.jpg

leowai

19th July 2005, 06:42

Nice work! I almost can't find any blocky picture here.

By the way, is these encoded with default settings in x264 with exception of inloop increased to 3?

Japhsoncross

19th July 2005, 10:12

i also did a lot of tests these days, and x264 is amazing in anime materials.

but i also found that x264 blurs the details in natural materials......i don't know the reason.
my settings are:
x264 --keyint 240 --min-keyint 24 --bframes 2 --b-pyramid --ref 16 --filter -3:-3 --ipratio 1.1 --pbratio 1.1 --pass 1 --stats "xx.log" --analyse all --weightb --me umh --merange 32 --subme 6 --8x8dct
x264 --keyint 240 --min-keyint 24 --bframes 2 --b-pyramid --ref 16 --filter -3:-3 --ipratio 1.1 --pbratio 1.1 --pass 2 --stats "xx.log" --analyse all --weightb --me umh --merange 32 --subme 6 --8x8dct

extreme slow settings, but the details of encoded frames don't satisfy me even if the bitrate goes higher(1.2M).

Sirber, do you have any tests on natural materials?

dinolib

19th July 2005, 10:22

If you use inloop -3 (instead of +3) you blur details! <= EDIT:FALSE!! It's the opposite

cheers

Japhsoncross

19th July 2005, 11:52

THX, i'll have a try.

----------------------
i replaced +3 with -3, it blurs more...

Sirber

19th July 2005, 12:19

Nice work! I almost can't find any blocky picture here.

By the way, is these encoded with default settings in x264 with exception of inloop increased to 3?+3 is default :)
If you use inloop -3 (instead of +3) you blur details!

cheersCan you post screenshots?

Japhsoncross

19th July 2005, 13:15

+3 is default :)
Can you post screenshots?

i compressed the three files(source, loopfilter-3, and loopfilter+3) in jpeg format at max quality.

Sirber

19th July 2005, 13:19

Images in pending approval. Which value do you recommend for RealAnime?

Caroliano

19th July 2005, 14:30

I'm having impressive results with x264 in animes too. I aim for quality and found that x264 can have a similar quality with almost two times lower bitrates. 500 ~ 600 is now the bitrate for high quality animes instead of 800 ~ 900 of Xvid.

I use the defaut adaptative in-loop filter (in others words filter=0) for VFW sharktooth's build. It leave all the details I want. I tried to play a bit with it but the defaut seems good for HQ anime.

And about the quantizers? I think that q24 is very good quality and q22 is almost perfect. I cap my quantizers and don't let the encoder down to less than q22. It would be a wast of bits for me. What you guys think?

Sirber

19th July 2005, 15:49

I use default quants. x264 is pretty adaptive. I get similar results with no bframes!!! :D

(from my first 400kbps comparison, which bframes were disabled by error)

Caroliano

19th July 2005, 16:33

But you see any quality improvement in anime content beyond q22?

And this no-bframe thing is witch this naruto episode at this bitrate? Or was in the one piece clip?

Sirber

19th July 2005, 16:43

But you see any quality improvement in anime content beyond q22?
I don't know. I'll try ASAP.
And this no-bframe thing is witch this naruto episode at this bitrate? Or was in the one piece clip?
One piece. Did not try on a full encode yet. I got crash (exitCode -1 at 100% encode) on my other naruto #143 test encode at 400kbps.

Isochroma

19th July 2005, 22:45

I don't use anything greater than q20, and I can see quality improvements at even lower quantizers. I am working with very sharp, clean hi-res DVD originals, and I have a 21" Flat Trinitron monitor. So, I can see all the details that everyone else is missing.

Caroliano

19th July 2005, 23:29

Hum. For me in lower quants will only represent better the noise of the source because the in-loop filter is lower...Of course, I don't have a 21" Flat Trinitron monitor and a perfect source, only a good one. And I'm not aim to SHQ encodes, only HQ...

IgorC

20th July 2005, 00:30

x264: extreme quality on anime!

The same I can say for Nero H.264.
These two codecs are best right now.

Japhsoncross

20th July 2005, 02:04

Images in pending approval. Which value do you recommend for RealAnime?

i remember a small test you'd posted here: http://forum.doom9.org/showthread.php?t=97075 , i used your source and tested myself, gave also 400kbps, the first time i forgot to use loopfilter, and the result was quite blocky, then i tested the loopfilter@-3, and the result could be quite acceptable. IMO, at exteme low bitrates the value of loopfilter can be higher, while the bitrate goes higher, the value can be lower for more details.

Sirber

20th July 2005, 02:19

On my tests, +3 was pretty effective (400+ kbps).

http://www.detritus.qc.ca/files/x264/snapshot20050719211830.jpg
http://www.detritus.qc.ca/files/x264/snapshot20050719214207.jpg
Millenium Actress, 720x400, x264, 930kbps

ChronoCross

20th July 2005, 03:52

well one of the better things about anime is there isn't much noticeable detail loss even when you try to get rid of details cause the details aren't as overtly obvious as real life action. especially with skin gradients. Anime is smooth rather than rough and it is very hard for a codec to whipe out enough of the details to be noticeable.

I don't see a point to encoding at 400kbps in AVC as of yet. it may become smaller but it won't be as good as a high bitrate encode. and even at 400kbps it doesn;t become streamable as AVC isn't at that stage. It's good for testing. Have you done any high bitrate tests and it's effect on noise?

Japhsoncross

20th July 2005, 04:31

well i generally test at the bitrates 400kbps, 750kbps, and 1200kbps, and the codec x264 and ND MP. til now, i find x264 is much better on anime sources at low bitrate, for higher bitrate and real life sources, ND MP gives better result on details. i wonder if it's my mistake, my command line has been posted at #3.

Chainmax

20th July 2005, 07:04

Sirber: OMFG, these images look amazing, and they correspond to a 0.075 bpp :eek:

Much like the gnomes in the world of Krynn, I have a Life Quest(tm). My life quest is obtaining an excellent rip out of my mercilessly crappy R4 NTSC Simpsons 1st Season DVD set. I tried numerous combinations of filters and while I solved a good many problems, moving blocks still haunt me. I moved from latest Koepi Xvid to latest Nero Recode, and my latest script:

MPEG2Source("X:\wherever\SimpTest.d2v",cpu2="ooooxx")
DeDot()
TFM(d2v="X:\wherever\SimpTest.d2v",mode=3,PP=7,mChroma=true,chroma=true,mi=50)
TDecimate(mode=1)
FixChromaBleeding()
LumaYV12(lumoff=-2,lumgain=1.0)
ColorYUV(levels="pc->tv")
Deblock()
Deen()
HQDering()
Crop(12,0,700,476,align=true)
aWarpSharp(depth=16,cm=1)
BicubicResize(640,480,0,0.5)
BlindPP(quant=0,cpu=0,cpu2="ooooxx")
LimitedSharpen()

while producing somewhat acceptable results (not sure anymore though, I'm fed up looking at the same clip for the umpteenth time), still has noticeable moving blocks in many scenes. I haven't really searched for most used settings, so I used what I could recall people said some time ago: 5 Reference frames, Max GOP size = 10xFPS, 1 B-Frame, vector range -256 to 255.75, Chroma Optimization, Psy Enhancements High. Since the source has some really horrible blocking, I used either Adaptive Deblocking @ Automatic Smooth or plain Deblocking @ strength 6. Average bitrate is ~1620 kbps and resolution is 640x480.

Now, onto the questions:

1) What is the difference between -x deblocking and +x deblocking?

2) It seemed to me that inloop (deblocking, right?) wasn't as effective at block removal than just using a deblocker on the script. I know that the definition is "prevents blocking at the encoder stage" but don't quite get it, does it mean that it just avoids blocking due to using a too low bitrate, meaning that in my case it's useless?

3) Could you guys please post your preferred settings for x264? I'd like to give it a whirl too and would like to know some good starting values.

Japhsoncross

20th July 2005, 07:28

1) it means the strength of the deblocking filter.
2) the filter will improving the apearance of the decoded frames. and also in the encoding stage, the filtered image is used for motion-compensated prediction, which can improve compression performance.
3) i'm still searching for a better setting for x264, which not only does well on anime sources, but on natural sources.

Sirber

20th July 2005, 12:35

http://www.detritus.qc.ca/files/x264/setting.jpg
I have to switch "weaker" and "stronger", since -3 is stronger than +3.

Sirber

20th July 2005, 12:58

I tryed 436kbps. Shapes are a little less sharp, but I save 20MB :)

yaz

20th July 2005, 14:23

... I have to switch "weaker" and "stronger", since -3 is stronger than +3.dunno what u mean by 'stronger' and weaker' here but ... from mencoder readme:
... parameter of deblocking filter (default: 0) ...
A positive value reduces blocking artifacts more, but will also smear details ...
For encodes that are intended to be reasonably high quality, you might want to turn it down a little bit. However, if your source material already has some blocking or noise which you would like to remove, or if it is animation, it may be a good idea to turn it up a little bit. imho, it still stands.

the bests
y

Sirber

20th July 2005, 14:32

From the screenshots source vs -3 vs +3, -3 seems blurier.

akupenguin

20th July 2005, 16:48

When comparing deblocking settings, pick a source that has detail to begin with, and compress it enough to see the difference.
http://img302.imageshack.us/img302/5704/loopsrcyr7.pnghttp://img175.imageshack.us/img175/2940/loop6aw3.png
http://img49.imageshack.us/img49/6116/loop0by4.pnghttp://img105.imageshack.us/img105/9900/loop6ua9.png

Chainmax

20th July 2005, 17:30

Wow, deblock -6 looks awful. Deblock +6 blurs a lot, but that's probably what I need in my case. Too late for change though, I'm already making an encode with the following settings:

x264 --pass 1 --bitrate 1620 --keyint 240 --min-keyint 24 --bframes 3 --b-pyramid --ref 8 --filter -6:-6 --analyse all --weightb --me umh --merange 16 --subme 5 --8x8dct -o NUL x264Try.avs
x264 --pass 2 --bitrate 1620 --keyint 240 --min-keyint 24 --bframes 3 --b-pyramid --ref 8 --filter -6:-6 --analyse all --weightb --me umh --merange 16 --subme 5 --8x8dct -o Pruebax264.mp4 x264Try.avs

I will try with --filter 6:6 later. One thing: bond posted a complete list of x264 switches a while ago, and in that list the max value for subme was said to be 5. Japhsoncross, however, used 6. Which is the actual max value?

Isochroma

20th July 2005, 17:34

Athlon64 for H264!

Yes, anime is very beautiful compressed with this codec! It has been a miracle for me, x264 in particular.

ChronoReverse

20th July 2005, 17:55

Heh, yeah, I'm finding anime sources to be nicely compressed with H.264

It's rather unfortunate that the big H.264 release of the weekend (FMP:TSR) was using a poor raw and had artifacts.

It was rather reminiscent of when XviD was starting to be used how people were going "What codec do I have to use?"

I can't believe how many people are still downloading the June 2002 FFDShow from the Sourceforge site and then thinking that it sucks because it doesn't work.

If we move on to AVC+HE-AAC+MP4 (using MP4 softsubs) it'll be great :P

Chainmax

20th July 2005, 18:18

Oy, the settings I'm using resulted in a ~0.5fps second pass :scared:.
I tried the script I posted without Deen or Deblock and the results were about on par with Nero. I'll try including Deblock() and --filter 3:3.
In any case, what should I change in the settings I used for a speed boost with the lowest impact on quality?

Caroliano

20th July 2005, 18:20

lordreign

20th July 2005, 18:25

Hey sirber I'm just curious, this is abit off topic, but that naruto is it your own recorded source or is it a raw or something? (dattabayo release is only 111meg not the 170 your source is) Because I'd love to reencode naruto to x264 but I can only get either dattabayo's releases or the raw (which isnt much use cause me no understand japanese :confused:) so I was wondering how u got the subs on yours if its not a subbed release (external sub file perhaps)?

Cheers.

Ishan

20th July 2005, 18:32

I usualy set deblocking to 0 because for me it always hurt quality (I'm only doing encode @ full DVD res @ 2mbps) If someone would be kind enough to explain how it works it'll be easier to setup. thx :)

ChronoReverse

20th July 2005, 18:38

@ChronoReverse: But the decode speed is bad. This is for me the worst thing about x264...

Once I can decode a Naruto opening in realtime and full resolution with my Celeron 1.7MHz I will aproval a move to H.264...

The same opening in Xvid with Qpel and GMC @3000kbps takes 50% ~ 60% in my processor.

PS: I use the latest FFDShow to decode both.

Yeah, I do understand about that.

Afterall, a 1.7GHz Celeron is like worse than a 800Mhz Pentium 3 (judging by the fact that a similarly clocked Pentium 4 is worse than a Pentium 3 1.0GHz Tualatin).

Still, I've heard reports of people playing back the FMP:TSR encode with 866MHz cpus so it's not too bad (640x352).

Doom9

20th July 2005, 18:39

--ref 8 --filterYou're going overboard here...
--merange 16 --subme 5Those are default values and thus not needed. MeGUI would give you a commandline without those ;)
And a fast first pass speeds up things considerably without noticable quality loss.

Sirber

20th July 2005, 18:43

Chainmax

20th July 2005, 20:56

Doom9: yeah, I didn't notice the defaults for merange and subme (is 5 the maximum for subme or is it 6?). I lowered bframes to 1 and ref to 5 but I'll keep filter 3:3 because the source is really awful. I'll report what speed gains (if any) I get. Also, what switch enables fast first pass?

unmei

20th July 2005, 21:01

I usualy set deblocking to 0 because for me it always hurt quality
Partially agreed. First i think to diable the inloop filter you need to actually uncheck it. The "0" here only means "use default" - and the positive/negative numers are "use more/less than default". Well correct me if necessary, i can't even look at the config dialog right now..
Now, i too think stronger deblocking is not partially nice to look at, i never increased it so far. BUT i have some DVDs that have pretty poor quality (the usual filed blending stuff a bit strong and other dirt) - there x264 fails to excel. In a few cases i was unable to filter it in a way to make x264 behave "normal", so i went for a XviD encode for a start. While i totally agree that x264 usually rocks for anime, it has some real problems with lots of stuff that should not be there - the noise/grain like in cowboy bebop is in my experience less of a problem, but field blending and ringing/macroblocks from the mpeg-2 can generate ugly persistent blocks.

Doom9

20th July 2005, 21:12

(is 5 the maximum for subme or is it 6?).it's 6, 6 enables RDO. You'd knew that if you were using MeGUI (why aren't you btw.. it really simplifies your life and the two, three issues that are still open are not a showstopper for anyone)

Also, what switch enables fast first pass?It's called Turbo in MeGUI ;) If you enable commandline preview, you'll see what it does (it does a bunch of things, lowering subme, limiting the number of references, limits macroblock options, motion estimation and I think that's it. It's definitely easier to have one checkbox to do that all for you instead of having to remember it.

Chainmax

20th July 2005, 21:58

Ok, ok, I'll use MeGUI, just don't hurt me :p.

Ishan

20th July 2005, 23:05

what's the default deblocking x264 use? because I find those perfect for me, I just don't set deblocking :)

Revgen

20th July 2005, 23:12

@ChronoReverse: But the decode speed is bad. This is for me the worst thing about x264...

Once I can decode a Naruto opening in realtime and full resolution with my Celeron 1.7MHz I will aproval a move to H.264...

How expensive are computers in Brazil?

There are a lot better processors out there that should be fairly inexpensive to upgrade, unless the prices are different over there.

Caroliano

20th July 2005, 23:18

what's the default deblocking x264 use? because I find those perfect for me, I just don't set deblocking :)
Hum... how you expect I answer it...

The defaut is 0 (the range is -6 to +6). The deblocking strength is adaptative. For high quantizers, high deblock strength. For less than q15, no debloking is made. I agree with you, the defaut is perfect to me :)

-> A question that poped in my mind now. If I set the debloking strenght to +6 it will deblock in q14 for example?

Caroliano

20th July 2005, 23:40

How expensive are computers in Brazil?

There are a lot better processors out there that should be fairly inexpensive to upgrade, unless the prices are different over there.
Well, to make a good upgrade of processors you almost aways need to upgrade the motherboard too. But that's not the point...

The computers here isnt much more expansive here than USA computers in absolute values. The problem is to get the money to paid a upgrade every year. My family earns less than 1.200 dolars per month, and is in high middle class for brazilians standarts. We can buy a month food for less than 70 dolars but a basic computer (sempron 2200) is more than 500 dolars. It is too much for us, but probabily not for you. I don't know if you understand my point. We have less money here than you there.

PS: You posted while I was writing the last mensage... and I write too slow in english...

ChronoReverse

20th July 2005, 23:45

No I understand just fine. After all, I only recently moved to a AthlonXP 2500+ (Sempron 2500+ basically) from a 1.0GHz Celeron (a Tualatin one so ironically it'd be faster than your 1.7GHz Celeron). I would love to have a sparkling Athlon64 system, but I'll make do with what I can get.

I'm not trying accuse you of anything, but rather just pointing out how a Celeron is a poor choice (unfortunately).

You would've ended up with faster system at around the same (or lower) price going the Duron (Applebred) 1.6GHz route =/

In any case, enough of this diversion from the topic.

I have to figure out why this DVD encode I did ended up blue (using megui and mencoder to encode AVC).

Sirber

20th July 2005, 23:51

x264 encoding with a 1800+ (at work) at 720x400, IT + C3D, gives a big 2 FPS :( Looking for DualCore...

Caroliano

21st July 2005, 00:54

@ChronoReverse: I was not talking to you... i was answering to Revgen ... but all right. This celeron 1.7 is not a so bad choice. It is faster than your old Celeron 1.0 MHz, the diference of clock rendiment is not that big. Intel never launched a P4 slower than a existent P3. They jump from PIII 1GHz to P4 1.4GH instead.
It is a fairly good processor of intel. It is cheaper and don't heat too much, but I agree, an AMD would be better.

A bit more in topic: I don't have problems with 640x352 clips. Only Last Exile was a bit tricky but wachable... the problem is 640x480.

@Siber: Well come to my world :p