The quest for true constant quality with x264

[Kuukunen]

There has been lots of threads about these things, and I think I read through the most relevant ones. They still didn't give me the answers I wanted.

However, I feel this topic is important, since I think constant perceived quality is what people usually really want, since there really is no reason to go to higher bitrates if you can achieve acceptable quality on a lower bitrate. (Unless you want to fit exactly one file per CD etc.)

First I will list some bits of information I have gathered and on which I am basing my assumptions and questions. I may have lots of misinformation and wrond ideas, so if I'm wrong about them, I hope someone could fix my flawed facts.

For a long time I was wondering if 2pass is really better than "Constant Quality" of the same bitrate. I mean logically it shouldn't be: it tries to achieve constant perceived quality, but 2pass does exactly the same. Only difference is that 2pass calculates the level of quality that gives the wanted filesize. (Approximately.)

So if 2pass really is better than CFR, it must mean 2pass's definition of "constant quality" is different from CFR's definition.

I really can't understand why 2pass encoding is inherently so much better than one pass. The only reason I've gotten for it is because in 2pass the 1st pass makes a stats file for later use in 2nd pass. On the other hand one pass constant quality can't predict what are the properties of the frames ahead of the frame currently worked on. This gives 2pass only one advantage: If there is a frame ahead that needs more quantizer and that frame is using the current frame as a reference, the current one should be given more too. I don't want to use the expression "more complex frame" since I'm still not quite convinced if complex frames need more quantizier or not.

If there is some other reason, PLEASE tell me. All I've seen is stuff like "2-pass mode is superior given an equal file size..". A few times even something like "I tried both, and 2pass was slightly better." Never the reason. (Other than the one I already mentioned.) I'd REALLY like to know what is this magical quality that makes 2pass encoding "superior".

I'm under impression when the bitrate goes up, CFR comes very close to 2pass. And I guess it wouldn't be too much of a quality loss if I just did CFR, but being the perfectionist I am, I would want to achieve the best possible quality while still maintaining the philosophy of having constant quality for all encodes. (Of one series for example.) The way I see it there are four ways to solve this:

1) 1st pass CRF with stats file. Then somehow try to figure out a good bitrate from that, then do 2nd pass with that bitrate. I still have pretty much no idea how to get a good bitrate estimate from it, so if someone could please tell me, it would be nice, as this seems to be currently the most feasible way.

2) A normal one pass CRF outputting a stats file, then pick the final bitrate of the resulting video file, and do a normal second pass with that bitrate. Now this is the only one I really know how to do with reasonable accuracy, but it seems like a lot of wasted CPU time.

Now we move into the hypothetical methods.

3) 1st pass constant quality mode with stats file. Then 2nd pass constant quality mode. I read that "crf inherently doesn't work in 2pass", but is there a reason for that? When a normal 2nd pass is given a stats file and a bitrate, it somehow magically decides on a quality everything is going to be encoded in. Is it really impossible to give that magical quality factor as an argument to x264?

4) This suggestion makes even more assumptions, mainly assuming I'm right about the fact that makes 2pass encoding better than one pass. Or to be specific I mean there is really no reason to know stuff about the frames that aren't near the one currently under progress. (If there is, please tell me why.) Therefore, would it be completely impossible to calculate stats of the few frames ahead? There might be frames using frames as a reference that are using frames as a reference etc, but I'm not even sure how far this is possible, and even if it is possible to carry on from the first frame to the last frame, it should be taken care of by a reasonable buffer. Now, this isn't too different from the #3, so why go through the difficulties? Well, there are situations where rendering of the video is really expensive, like having crazy filters in Avisynth or maybe some other frameserver. Or you might be encoding real time. So why would it be impossible to render some frames in advance, store them in memory, then calculate stats for those, then encode the current frame (which is many frames before the one rendered last) using the given CMF? (Constant Magic Factor)

[akupenguin]

If there is some other reason, PLEASE tell me. All I've seen is stuff like "2-pass mode is superior given an equal file size..". A few times even something like "I tried both, and 2pass was slightly better." Never the reason. (Other than the one I already mentioned.) I'd REALLY like to know what is this magical quality that makes 2pass encoding "superior".
* Better QP choices if we know the content of future frames. Especially when choosing how much to boost the QP of I-frames. Yes, this could be solved by 1pass lookahead ratecontrol.
* --direct=auto doesn't work very well in 1pass. In particular, 2pass direct=auto can choose the mode of each B-frame based on that frame's own stats, while 1pass direct=auto chooses the mode of each B-frame based on the previous frames' stats. This could be solved by re-encoding the B-frame if its optimal mode differs from the guess. This would be somewhat faster than 2pass, though still many frames would be encoded twice. This also has some issues when combined with pyramid + threads.
* The 2pass "complexity" metric is bits, while the 1pass "complexity" metric is based on SATD. This may or may not be an improvement at all.

The combined effect of the above reasons is somewhere between 0 and 0.1 dB PSNR. So you might be better off just spending the extra cpu-time on slightly slower 1pass settings.

3) 1st pass constant quality mode with stats file. Then 2nd pass constant quality mode. I read that "crf inherently doesn't work in 2pass", but is there a reason for that? When a normal 2nd pass is given a stats file and a bitrate, it somehow magically decides on a quality everything is going to be encoded in. Is it really impossible to give that magical quality factor as an argument to x264?
http://forum.doom9.org/showthread.php?t=103348
Though that doesn't mean it can't be done. After all, CRF doesn't inherently have anything to do with QP either, the values of --crf are only similar to the QP they produce because I tuned a formula to translate the value from --crf to the internal rate_factor. So I suppose I could write a similar translation for the 2nd pass, and a given --crf in 1pass would usually be similar bitrate to the same value of --crf in 2pass.
Or I could just store the SATD values used in 1pass CRF in the 2pass file, and use them. Then 2pass CRF would be identical to 1pass CRF except for the points specified above.

CMF? (Constant Magic Factor)
It's called "rate factor", hence the name "constant rate factor".
Actually, "called" is a bit strong. It's stored in a variable named "rate_factor", hence the name "constant rate factor".

[Kuukunen]

First, thanks for answering, it really cleared some suspicions and gave me hope. Also, congrats on the 1337th post. ;P
* Better QP choices if we know the content of future frames. Especially when choosing how much to boost the QP of I-frames. Yes, this could be solved by 1pass lookahead ratecontrol.
Sounds reasonable. But how far does 2pass encoding look anyways? As in how many frames would it have to cache?
* --direct=auto doesn't work very well in 1pass. In particular, 2pass direct=auto can choose the mode of each B-frame based on that frame's own stats, while 1pass direct=auto chooses the mode of each B-frame based on the previous frames' stats. This could be solved by re-encoding the B-frame if its optimal mode differs from the guess. This would be somewhat faster than 2pass, though still many frames would be encoded twice. This also has some issues when combined with pyramid + threads.
Even if some of them were encoded twice, I would guess it's possible to predict which direct mode would be more probable using the results from the previous decisions. Besides, even in the absolute worst case scenario when the thing is completely random, only half of the frames would be guessed wrong. (Unless I misunderstood the whole thing completely.)
* The 2pass "complexity" metric is bits, while the 1pass "complexity" metric is based on SATD. This may or may not be an improvement at all.
So let me get this straight... Simply by implementing a lookahead buffer that stores stats and frames into memory, then implement guessing heuristics for the direct=auto, you get one pass encoding that is in no way worse quality than two pass encoding of the same file size?

However, even though it's the same quality, it's a lot faster... Let's not forget expensive rendering either, I have an Avisynth script I've used that renders at the rate of 3fps. And not only that... It would also help a lot in trying to figure out the best encoding settings...
http://forum.doom9.org/showthread.php?t=103348
Though that doesn't mean it can't be done. After all, CRF doesn't inherently have anything to do with QP either, the values of --crf are only similar to the QP they produce because I tuned a formula to translate the value from --crf to the internal rate_factor. So I suppose I could write a similar translation for the 2nd pass, and a given --crf in 1pass would usually be similar bitrate to the same value of --crf in 2pass.
Or I could just store the SATD values used in 1pass CRF in the 2pass file, and use them. Then 2pass CRF would be identical to 1pass CRF except for the points specified above.
The reason I called rate_factor in 2pass encoding "constant magic factor" was because it's apparently different from the CRF rate_factor. Right? The thing I'm really seeking here is how to define the "CMF". I hope what you meant in that quote was something like that... As in it would be a REAL 2nd pass. Meaning it would have all the QP lookahead and already figured out decisions for direct=auto?

As I said... What I was after in this thread was a way to create "true" predefined constant quality. The suggestions/possibilites you gave look very promising indeed, and I guess all I could hope for was that my meditations on this could be proven to be possible/correct. Still... If the suggestion #4 or at least #3 would at any point become part of the actual implementation, it would be the best thing since DCT. I feel that it wouldn't be too hard to implement, since you already have the 2nd pass code ready with all bells and whistles for those things and only things needed are the two things I mentioned.

[akupenguin]

But how far does 2pass encoding look anyways? As in how many frames would it have to cache?
It's exponentially weighted with a time constant of cplxblur. So a 20 frame lookahead is good enough.

So let me get this straight... Simply by implementing a lookahead buffer that stores stats and frames into memory, then implement guessing heuristics for the direct=auto, you get one pass encoding that is in no way worse quality than two pass encoding of the same file size?
Yes. The "2pass is always better" idea stems from previous codecs (xvid, divx, lavc) where it was definitely true. But it's not inherent in all video codecs, and x264 comes pretty close to fixing 1pass.

Let's not forget expensive rendering either, I have an Avisynth script I've used that renders at the rate of 3fps.I'm assuming you don't have enough HDD space for huffyuv?

The reason I called rate_factor in 2pass encoding "constant magic factor" was because it's apparently different from the CRF rate_factor. Right?
1pass's rate_factor and 2pass's rate_factor represent exactly the same quantity, they're stored in the same variable, and used in the same code. They just differ in their units. The code that uses rate_factor is insensitive to the units if you enable a target bitrate (ABR or 2pass), but the units do matter for CRF.

[Kuukunen]

Yes. The "2pass is always better" idea stems from previous codecs (xvid, divx, lavc) where is was definitely true. But it's not inherent in all video codecs, and x264 comes pretty close to fixing 1pass.
Yes, it's very close, but I guess it's still not quite there. Of course the idea isn't new, I ran into this once:
http://www.undercut.org/Nandub_OnePass/
No idea how well it works or anything though, didn't look into it too much.
I'm assuming you don't have enough HDD space for huffyuv?
170Mbps for my poor lil laptop... You assume correctly. :P
That was never the point though. Point wasn't really how to fix MY next encode. Point was just that I wondered about how it could possibly be fixed altogether for everyone, as with just the 20 frame buffer thing and one pass encoding, I don't really see a reason for decoding to huffyuv anymore.
1pass's rate_factor and 2pass's rate_factor represent exactly the same quantity, they're stored in the same variable, and used in the same code. They just differ in their units. The code that uses rate_factor is insensitive to the units if you enable a target bitrate (ABR or 2pass), but the units do matter for CRF.
Ok. I haven't looked into the code so I don't know how it works internally. The reason I talked about them differently was because they acted differently. (Apparently the only real practical difference was the QP lookahead thingy, though.)

Thank you for answering to all these questions. I really wish I you got at least some ideas based on my ramblings.

[PuzZLeR]

Fascinating topic if I may jump in.

This debate can spurn on forever because it really doesn’t come down to quantitative metrics such as PSNR to conclude a fair comparison between the two, especially when this measurement is almost negligible anyway.

Other than the fact that 1-pass’s advantage is less CPU time and a rather accurate quality forecaster and 2-pass’s advantage will be very predictable file size and (maybe) optimal quality per file size there may not be a definite quality advantage to either, especially with x264.

A believer of 2-pass can conclude that the bits are being allocated to areas of the clip most needed. However, assuming this logic, this gives more margin for 1-pass to delegate more bits to other areas maybe deprived in 2-pass (assuming the same amount of bitrate).

So what would be elements in the decision criteria between these schemes in data distribution?

Akupenguin mentioned the older codecs, DivX, Xvid, etc, having a definite advantage with a second pass. Very true. However, using this methodology as a model, wouldn’t some of the theory apply as well to x264?

Please let me explain.

With these older codecs, what was apparently noticeable was in scenes of higher motion.

One pass was obviously short-sighted, and would allocate bits on a frame-by-frame basis. Every frame, without knowledge of the next, would be given its “due”.

However, with the advantage of foresight, a 2-pass scheme would better allocate data to the higher motion scenes in the video, something a frame-by-frame encoding scheme would not be able to recognize. This would especially make sense when such activity can last for hundreds of frames at a time and 1-pass cannot predict this in advance. Even a 20-frame head start can be a significant advantage for 2-pass in motion scenes.

Qualitatively speaking, given this hypothesis, the result was video that had a higher spread of visual quality among the more stationary frames in 1-pass, maybe even the majority of the frames in total, while 2-pass would have far less blur in the scenes with higher movement.

I do understand that x264 was better designed, and the algorithms implemented ease this variance, however wouldn’t this play a role as well? Forgive me if I’m wrong, I can’t help but think that maybe a 2-pass scheme, even with x264, may have an edge, albeit small, in higher motion scenes.

Thanks for reading.

[akupenguin]

Quantitative metrics such as PSNR are sufficient for comparing 1pass to 2pass, because they have exactly the same target bit distribution and differ only in how close they get to that target. PSNR can't compare macroscopically different bit distributions, but those aren't the issue here. I'm not saying x264's 2pass is the optimal distribution, only that you can get the same distribution in 1pass.

The reason xvid, divx, etc have a definite advantage with 2pass is because they have no unrestricted 1pass VBR (aside from CQP, which is suboptimal for other reasons). The difference between 1pass ABR and 2pass is most visible in high motion because ABR doesn't give each frame its due, it limits the bitrate.

1pass doesn't have to be short-sighted. A small lookahead buffer is sufficient to get all the benefit of 2pass bitrate distribution, because the inter-prediction dependencies between frames (and psy motion masking, if your ratecontrol uses that) are limited to a reasonably small radius of effect. Even if you don't believe my estimate of 20 frames for x264, inter prediction is definitely limited to 1 GOP. And psy effects can't possibly last more than a few seconds.

If even a 20-frames head start helps, then that's evidence that you don't need 2pass, because you can get a 20 frame head start in 1pass just be spending a little extra RAM.

[Kuukunen]

This debate can spurn on forever because it really doesn’t come down to quantitative metrics such as PSNR to conclude a fair comparison between the two, especially when this measurement is almost negligible anyway.
That doesn't really matter. Like I've been trying to say here, it's possible to use pretty much the exact same methods of 2pass in 1pass, giving exactly the same quality. (But of course, you can't predict the file size.)

A believer of 2-pass can conclude that the bits are being allocated to areas of the clip most needed. However, assuming this logic, this gives more margin for 1-pass to delegate more bits to other areas maybe deprived in 2-pass (assuming the same amount of bitrate).
A believer? Remember, blind faith is baaad. I think it really doesn't go like that. By principle, one pass CRF and a normal 2pass both try to achieve a contant quality. The difference is that in 2pass, the amount of that constant quality is calculated beforehand to the value that would give the wanted filesize, in CRF it's given.

So what would be elements in the decision criteria between these schemes in data distribution?
Speaking on a theoretical level (it's not in any implementations yet, although x264 is pretty close) it doesn't matter. You design the best decision criteria you can think of and use it in both 2pass and one pass.

Akupenguin mentioned the older codecs, DivX, Xvid, etc, having a definite advantage with a second pass. Very true. However, using this methodology as a model, wouldn’t some of the theory apply as well to x264?
The final bitstream doesn't care at all if it had been encoded in one pass or two pass, or maybe 1.5 pass where the hard parts are re-encoded, anything really. There really is no real reason why the fact older codecs didn't implement proper one pass encoding should apply to newer ones too. Also, did you read the link I gave about Nandub OnePass?

With these older codecs, what was apparently noticeable was in scenes of higher motion.

One pass was obviously short-sighted, and would allocate bits on a frame-by-frame basis. Every frame, without knowledge of the next, will be given its “due”.

However, with the advantage of foresight, a 2-pass scheme would better allocate data to the higher motion scenes in the video, something a frame-by-frame encoding scheme would not be able to recognize. This would especially make sense when such activity can last for hundreds of frames at a time and 1-pass cannot predict this in advance. Even a 20-frame head start can be a significant advantage for 2-pass in motion scenes.
akupenguin said: "It's exponentially weighted with a time constant of cplxblur. So a 20 frame lookahead is good enough."
2pass encoding doesn't really look farther than 20 frames. Why should it? How on earth would a frame 100 frames away affect the current one? And even if it does, how does it matter in 1pass vs 2pass if 2pass doesn't care either? It doesn't.

I do understand that x264 was better designed, and the algorithms implemented ease this variance, however wouldn’t this play a role as well? Forgive me if I’m wrong, I can’t help but think that maybe a 2-pass scheme, even with x264, may have an edge, albeit small, in higher motion scenes.
Once again... Why should it? It is perfectly possible to make 1pass use the exact same methods 2pass uses.

[CruNcher]

Yep indeed the concept of 2pass is outdated i do most of my encodes in ABR (especialy high bitrate ones HDTV,since H.264) and the difference between a X264 ABR with more tools compared to a less tools but 2 pass encode is almost non existant and both need the same time, also i never managed it yet to overflow with H.264 (if you call +-5 kb overflow/underflow ;) (wich happens from source to source with XviD and others more often,but then the overflow is most of the time in the Mb area).
Lookahead would be perfect so finaly 2pass could retire once and for all :P

[akupenguin]

x264's ABR does still have some disadvantages. Even if the extra tools you can enable with the time saved make up for the worse bit distribution, a compressibility test followed by CRF will be better yet. i.e. the optimal method when you do have a filesize constraint is 2pass but with a sparsely sampled 1st pass. I would integrate that into x264 if it weren't so dependent on avisynth.

[PuzZLeR]

Old ASP habits do die hard don’t they?

In my experiments with x264, I admit that I have failed to notice any (meaningful) quality advantage to either 1-pass or 2-pass (with same file size). I would only look for it because of what my experience with DivX was where it was obvious. It appears that the 2-pass mindset is a thing of the past, especially regarding the abilities of the H.264 standard.

In fact, even with x264, I would first encode in 1-pass and then feed that bitrate into 2-pass. Sometimes to alleviate some of the long H.264 encode times, I would even use the DivX quantizers to encode, and the resulting bitrate would serve as an index and to a proportionally equal bitrate for 2-pass x264 encoding.

I am grateful to hear from this thread that this is more a waste of time than productive. Even if I did get any added quality with 2-pass, it would be so minute and not worthy of the extra couple of steps.

2-pass is king when needing a certain amount of video time, or clips, for a given amount of storage unit and I will continue to use it as such.

However, after reading this thread, when a certain “fit” is not important, and bitrate calculators are not necessary, I’m going 1-pass all the way. Thank you to all.

[audyovydeo]

I am grateful to hear from this thread that this is more a waste of time than productive. Even if I did get any added quality with 2-pass, it would be so minute and not worthy of the extra couple of steps.

However, after reading this thread, when a certain “fit” is not important, and bitrate calculators are not necessary, I’m going 1-pass all the way. Thank you to all.

As a recent addition to the x264 user base I'm dead in the middle of intensive testing to determine the "best" settings.
I'd essentially tried only 2-pass following the general wisdom that "it's the best", but all my testing is pointing to the fact that I'd get 200% the encode time for less than 1% quality increase (SSIM) and visually identical content.

... I would first encode in 1-pass and then feed that bitrate into 2-pass

I'd like to know if/how you managed to automate that in Windows : I dont think x264 sets env variables with the output parms, at least in win32. That said, 99% of my encodes I don't even specify a bitrate, I let x264 go with the "quality" settings II give it.

[akupenguin]

It appears that the 2-pass mindset is a thing of the past, especially regarding the abilities of the H.264 standard.
x264's ratecontrol has nothing to do with H.264 (except insofar as it has to translate the internal units to/from QP for the final output). If I were writing a MPEG-1 codec, 2pass would be just as obsolete.

I'd get 200% the encode time
There's your first mistake. 2pass is not twice as slow as 1pass.

[audyovydeo]

There's your first mistake. 2pass is not twice as slow as 1pass.

My respects, akupenguin : it was a question of brevity. Real-life testing on my PC gives about 183% on average.

[PuzZLeR]

I'd like to know if/how you managed to automate that in Windows : I dont think x264 sets env variables with the output parms, at least in win32. That said, 99% of my encodes I don't even specify a bitrate, I let x264 go with the "quality" settings II give it.

I didn't actually. I would just simply run a batch job overnight using 1-pass. Then, after jotting down the resulting average bitrates per clip, I would run a batch with the same source the next night, this time using 2-pass, with the 1-pass average bitrates from the night before as respective bitrate parameters in 2-pass.

Actually, regarding HandBrake, a GUI for x264, you may find this thread interesting in their forums. They are attempting to accomplish just that, an automated process that implements the "benefits" of 1-pass combined with 2-pass:

http://handbrake.m0k.org/forum/viewtopic.php?t=287


I'd essentially tried only 2-pass following the general wisdom that "it's the best", but all my testing is pointing to the fact that I'd get 200% the encode time for less than 1% quality increase (SSIM) and visually identical content.


It does make "philosophical" sense to believe going over content a second time gives better results as I still somewhat do. However, hypothetically speaking, even if a 1% improvement is the result, in terms of logistics, and as an analogy, I personally now realize that it would be foolish to devote 300%, 283%, 200% or even 183% more effort into my job for a 1% pay increase now that I think of it, even if I'm doing it while asleep.

[legoman666]

Why not do a 2 pass encode with x264? I do a lot of 1080p/1080i to 720p conversions. For a 43 minute episode of Firefly, using megui's HD-Slowest profile, the first pass takes an hour. The second pass takes 12. In my case, I might as well do it since the extra time added is negligable and the quality is better.

[Kuukunen]

Quantitative metrics such as PSNR are sufficient for comparing 1pass to 2pass
What about the psy stuff, like assigning less quality in fast motion where you can't notice it? Now I'm not sure if that's done in 2pass in a predicted manner, (As in, knowing that the next frame is high motion affects the current one.) or if information about that is saved in the stats files at all. Actually, if it isn't, and predicting stuff like that would have a positive effect on quality, wouldn't lookahead buffer help in this too?
It appears that the 2-pass mindset is a thing of the past, especially regarding the abilities of the H.264 standard.
Well, if you want the absolute best quality 2pass is still the way to go. (See lower.) And as I tried to say earlier, the bitstream format (or ASP or AVC) doesn't have anything to do with 2pass or 1pass, it just says "do this and this and this to decode this frame". It's all in the encoder. Of course it's possible to make an encoder that considers every frame when encoding the first one, which would be basically impossible for traditional one pass encoding, but it would be just silly and useless.
It does make "philosophical" sense to believe going over content a second time gives better results as I still somewhat do. However, hypothetically speaking, even if a 1% improvement is the result, in terms of logistics, and as an analogy, I personally now realize that it would be foolish to devote 300%, 283%, 200% or even 183% more effort into my job for a 1% pay increase now that I think of it, even if I'm doing it while asleep.
In a way, it does make sense. But more in the way of: "more time spent means more quality". Of course I've debated this whole 2pass vs. 1pass thing with people who have argumented with things like: "you do realize it's impossible to make variable bitrate in one pass, right?"

On the 1% quality increase... Usually analogies suck. So does this. :p When you work, you get money, but with money the thing is, you can only spend it once. When you encode, the final product might be used many times. And my last 2pass encode didn't take 183% of 1pass. The first pass was 3.2FPS and the second was 0.71FPS. (I'm not sure, but I would think the second pass is pretty close to one pass.) Also, if I could 1% more money for work I do while I sleep, of course I'd get it. And finally, usually it's not about the amount of quality increase, but a matter of the sizes of the files that look the same. I don't know how much that difference is with current x264's CRF vs. 2pass. (But I've understood it has a lot to do with bitrate and/or the level of quality.)
Why not do a 2 pass encode with x264? I do a lot of 1080p/1080i to 720p conversions. For a 43 minute episode of Firefly, using megui's HD-Slowest profile, the first. pass takes an hour. The second pass takes 12. In my case, I might as well do it since the extra time added is negligable and the quality is better.
This was what this thread was all about. 1) Because there is really no need to do two passes if one pass can do the same. 2) With current implementations, 2pass encoding takes bitrate as an argument. I don't know beforehand how much bitrate a video needs to look acceptable. Of course, the quality setting needed might vary too, and might be dependant on other encoder settings and filters, but it will surely give a better estimate. Of course I could run some kind of compressibility test, that gives approximate bitrate needed for certain quality, but that's not a full solution.

[delacroixp]

I generally use 18-CQ-CRF but I have also experimented with Insane-2-pass... but the overall quallity is still some way from the original... possibly only 60%...
I took 'Band of Brothers' and tried Q18-CQ-CRF on DAResolution (1024x576) comparing the results to an Q18-CQ-CRF Anamorphic (720x576) encode...


Anamorphic Text http://souls-online.net/delacroixp/AutoMKV/BoB/BoB_Text_Anamorphic.png ... DAR http://souls-online.net/delacroixp/AutoMKV/BoB/BoB_Text_DAR.png ... VOB http://souls-online.net/delacroixp/AutoMKV/BoB/BoB_Text_VOB.png

Nobody is interested in great looking text... but it is a relative indicator of the quality of the rest of the encode... look especially at the bottom of the angled leg of the capital R...


Anamorphic http://souls-online.net/delacroixp/AutoMKV/BoB/BoB_ Anamorphic_Thumb.jpg (http://souls-online.net/delacroixp/AutoMKV/BoB/BoB_Anamorphic.png) ... DAR http://souls-online.net/delacroixp/AutoMKV/BoB/BoB_DAR_Thumb.jpg (http://souls-online.net/delacroixp/AutoMKV/BoB/BoB_DAR.png) ... VOB http://souls-online.net/delacroixp/AutoMKV/BoB/BoB_VOB_Thumb.jpg (http://souls-online.net/delacroixp/AutoMKV/BoB/BoB_VOB.png)

If you download the VLC snapshots and watch the jeep, particularly between the Anamorphic and DAR encode... the clarity and detail will just jump right out at you...


This DAR quality, however, does not come cheep... the Anamorphic encode is 45% less-than the VOB while the DAR encode is only 24% less-than the VOB original... and encodes times are also 25% slower.
A further consideration is that your movie has been correctly resized during encode rather than, in realtime on plaback... but larger files and higher res can also impact negatively on playback unless you have a dual-core system with RAID... especially if 'seeking' is involved.
I used the Lanczos4 resize filter though there may well be a filter better suited since the movie hasn't been resized, as much as, re-mapped to it's originally intended resolution.

I get the impression that Q18 reaches a quality-threshhold per-pixel... and that the only way to achieve further gains is by 'spreading the love' and letting more pixels carry the load...
I've just finished 'Lord of the Rings 1' at 1152x480 (2.4:1) originally (720x576 PAL / 16:9, black-space top and bottom)... The MKV is 5 GB (with direct copy of audio and subs) compared to 6.3 GB of the Mpeg-2 orginal...

Hardly a great saving, but an interesting experiment in the quest for Absolute Quality ...


:):D:eek:
Pascal


BTW
I took this a step further and over-sampled a BoB clip to 1280x720 and there was still a significant increase in quality... though it's not quite there with the original...
Original Post (http://forum.doom9.org/showpost.php?p=989749&postcount=1888)

[akupenguin]

This DAR quality, however, does not come cheep... the Anamorphic encode is 45% less-than the VOB while the DAR encode is only 24% less-than the VOB original
Let me get this straight... you made two encodes at different bitrates, and the one with the higher bitrate looked better. WTF else did you expect to find?

A further consideration is that your movie has been correctly resized during encode rather than, in realtime on plaback...
Unless your encoded resolution exactly matches your monitor resolution, it will be scaled in realtime on playback anyway. So the only question is: scale it both before encoding and during playback, or scale it only on playback. All else being equal, the latter is obviously better.
Pre-scaling could help if and only if all of the following are satisfied: You're encoding only for your own monitor so you know in advance the playback resolution. You use a non-realtime scaling algorithm that's even slower than decoding high resolution h264. And you use high enough bitrate that the extra sharpness introduced by the scaler isn't lost in the encoding artifacts.

I get the impression that Q18 reaches a quality-threshhold per-pixel... and that the only way to achieve further gains is by 'spreading the love' and letting more pixels carry the load...
More leftover preconceptions from mpeg4asp. There was something special about mpeg4 Q2. There is nothing special about h264 Q18. Maybe your source doesn't actually contain any more quality than Q18 can represent, but in that case upscaling won't help either.

[delacroixp]

This DAR quality, however, does not come cheep... the Anamorphic encode is 45% less-than the VOB while the DAR encode is only 24% less-than the VOB original... and encodes times are also 25% slower.
Let me get this straight... you made two encodes at different bitrates, and the one with the higher bitrate looked better. WTF else did you expect to find?

Almost, but not quite... Well, actually, I did an Q17-CQ-CRF and Q16 anamorphic encode with no significant increase in quality from Q18. In other words, H264 used a much higher average-bitrate with no discernable difference...
However, for the same filesize, by increasing the resolution while maintaining Q18, there was, and is, a clear increase in quality...

Sure, it's not 10 movie encodes on a single 4.5 GB DVD with beautifull efficiency... but a DAResolution encode does bridge the, not-inconsiderable, quality-gap between an anamorphic encode and the orginal VOB movie material... all else being equal...



A further consideration is that your movie has been correctly resized during encode rather than, in realtime on plaback...
Unless your encoded resolution exactly matches your monitor resolution, it will be scaled in realtime on playback anyway. So the only question is: scale it both before encoding and during playback, or scale it only on playback. All else being equal, the latter is obviously better.

Point taken...



I get the impression that Q18 reaches a quality-threshhold per-pixel... and that the only way to achieve further gains is by 'spreading the love' and letting more pixels carry the load...
More leftover preconceptions from mpeg4asp. There was something special about mpeg4 Q2. There is nothing special about h264 Q18. Maybe your source doesn't actually contain any more quality than Q18 can represent, but in that case upscaling won't help either.
Well Band of Brothers (http://en.wikipedia.org/wiki/Band_of_Brothers) was never a regular full-feature movie release... but it was the most expensive TV-series to date, at $250 million. My DVD looks pretty regular to me though I have no idea what process, other than being fully digital, it took to get from raw, un-compressed data... to the final few GB's in DVD format...
I can only comment on what I've seen and perceived given the short time I've used H264 or even given my limited encoding experience...

It certainly seams to me that you have a better chance of increasing quality beyond Q18-CQ-CRF by increasing resolution after this point... even beyond Anamorphic or DAResolution if you so choose... which is a farcry from DVDShrink if all you need is a 10% or 20% reduction in filesize...


You're the Guru... you explain the anomaly...



:):D:eek:
Pascal


BTW
I had another look at the PNG's and the difference in quality is almost palpable... Even if over-sampling is a total crap-shoot... Q18-Anamorphic and even Q18 encodes in general, have a long way to go before they match original quality.

[foxyshadis]

btw, I revisited one of my older recommendations, for very high quantizers use two-pass instead of crf. The two main things 2pass handles much better are slow fades and cuts to low-motion high-detail scenes. Most of the time they're pretty equivalent with small tradeoffs in smearing and artifacting. So even if you're really pushing the compression level (for streaming or whatever), crf can handle it, although crf causes somewhat greater degradation on frames that would violate vbv. 2pass seems to be better at spreading the love to compensate, probably the longer lookahead window, but not by a large amount.

[akupenguin]

2pass doesn't use any lookahead for vbv either. 2pass vbv was an afterthough, which just happens to sorta work because the same vbv code from 1pass is still enabled in 2pass. But there have also been reports that it doesn't work, in which case those frames probably look better because they do violate the vbv.

[audyovydeo]

@delacroixp
I for one would love to know whatyour SSIM and PSNR readings are for these three encodes.

@everyone, when you encode, do you aim at a particular SSIM range ? I've read the thread on metrics so I do compare only different encodes of the same source.

With my "fast" and "best" command lines, I always seem to land between 0.92 and 0.95, with wildly differing bitrates depending on the video source of course.

Do other people consistently have 0.98+ readings, or what ?


audyovydeo

[akupenguin]

SSIM is no better than PSNR in that sense - the absolute value of each is more an (inverse) measure of how much detail was in the source, rather than how much is left after encoding.
SSIM is good at comparing two encodes of the same content, and is less accurate but still useful for comparing different pieces of the same movie. It is useless for comparing two movies from different sources.

[audyovydeo]

@all

Being in a testing mood, I want to run 3 different 1st passes, a "crf", a "qp" and a "straight", creating different stats files. I'll then feed each stat file to the 2nd pass and record the results. We're bound to see how the stats are used by the 2nd pass.
Let's go :


testA pass1
x264 --crf 25 --pass 1 --progress --stats stats_crf.txt -o dummy.mp4 basic.avs

SSIM Mean Y:0.8805476
1252.67 kb/s


testA pass1 control
x264 --pass 1 --crf 25 --progress --stats stats_crf2.txt -o dummy.mp4 basic.avs

SSIM Mean Y:0.8805476
1252.67 kb/s


testB pass1
x264 --pass 1 --qp 30 --progress --stats stats_qp.txt -o dummy.mp4 basic.avs

SSIM Mean Y:0.8773727
1108.11 kb/s


testC pass1 (use all the defaults)
x264 --pass 1 --progress --stats stats_pass1.txt -o dummy.mp4 basic.avs

SSIM Mean Y:0.9112998
3105.93 kb/s


the stat files *are* different :
24,954 stats_crf.txt
24,841 stats_pass1.txt
24,767 stats_qp.txt



testA pass2
x264 --pass 2 --progress --stats stats_crf.txt -o dummy.mp4 basic.avs

SSIM Mean Y:0.9110670
3090.43 kb/s



testB pass2
x264 --pass 2 --progress --stats stats_qp.txt -o dummy.mp4 basic.avs

SSIM Mean Y:0.9111075
3090.45 kb/s


testC pass2
x264 --pass 2 --progress --stats stats_pass1.txt -o dummy.mp4 basic.avs

x264 [info]: SSIM Mean Y:0.9112998
3105.93 kb/s



Mmmmh, not sure that pass2 is using the important info from pass1 to best advantage ...
But I haven't pushed x264 to the limit. Let's throw in a last run, with a custom-made cmdline.
I'll exploit nearly all weapons in x264's arsenal :



testD
x264 --pass 1 --ref 3 --mixed-refs --bframes 3 --b-pyramid --bime --weightb --trellis 2 --no-dct-decimate --no-fast-pskip --subme 6 --b-rdo --partitions all --8x8dct --direct auto --me umh --no-psnr --progress --stats stats_custom.txt -o dummy.mp4 basic.avs

pass1 :
SSIM Mean Y:0.9134420
2856.89 kb/s

pass2 :
SSIM Mean Y:0.9135212
2858.52 kb/s



Mind-blowing ?
And I spared you - because you all know them by heart - the differences in encoding speeds.


My conclusions :

1. lots of options to play around with (I love cmd-lines)
2. playing with options I can obtain a 1% quality benefit for a 500% encoding time price
3. lots of highly-qualified highly-fragmented information
available online
4. missing formal reference doc (especially felt on mutually exclusive options)
5. Despite BluRay HD-DVD et al, a fundamental problem exists while H264.mp4 cannot play back on DVD-players the way DivX.avi
does. Reminds me of the time when I was using OS/2. I stuck with OS/2 almost a decade. I'll stick to x264 for the next decade
with the certainty, like my doom9 peers, that I'm applying the state-of-the-art in compression technology to my videos.
I just need to ask friends to buy dual cores to see my holiday videos.
6. I may have missed something out uttely and completely : I beg to be set straight.

happy encoding !
audyovydeo

[akupenguin]

testA pass2
You didn't specify a bitrate on the 2nd pass. 2pass CQP is identical to 1pass CQP. Yes, that's a feature.

playing with options I can obtain a 1% quality benefit for a 500% encoding time price
how did you calculate that?

[audyovydeo]

You didn't specify a bitrate on the 2nd pass. 2pass CQP is identical to 1pass CQP. Yes, that's a feature.


how did you calculate that?

Akupenguin,

I'm sure it is a feature, but how is a lost soul like me to know ?
When I started playing with x264 I wanted my encodes to be sort-of-compliant to something. so i looked up the parameters in sharktooth's megui High profile, derived a cmd-line from that, and started playing around. I'd started out by specifying a bitrate out of thin air. My sources are DV/PAL. I'd give it either 2000 or 3000 kb/s, but kept wondering what was the "reasonable" rate (knowing I dont encode for streaming, which is a different subject altogether of course). One day I think "let's not give --bitrate" and bingo : x264 applies the parameters and spits out what the bitrate should be for that source, with those parameters. For me that's fine, and who/what checks if it's right or wrong ?
Heck, I even cat * | grep -E "warning" all the source files to find what the warning messages were looking for a clue on the options.

re: 500%, I have boiled things down to a "fast" and a "best" cmd line. "fast" clocks between 30 and 35 fps, depending on the source, "best" comes in at 3 to 7 fps. 2nd pass figures for both, of course.


kind regards
audyovydeo

[akupenguin]

re: 500%, I have boiled things down to a "fast" and a "best" cmd line. "fast" clocks between 30 and 35 fps, depending on the source, "best" comes in at 3 to 7 fps. 2nd pass figures for both, of course.
No, I mean how did you calculate the 1%. If it's your testC vs testD, that saved 8% bitrate and the SSIM improvement is equivalent to another 3%.

[audyovydeo]

No, I mean how did you calculate the 1%. If it's your testC vs testD, that saved 8% bitrate and the SSIM improvement is equivalent to another 3%.

hello
no it's SSIM only.
Here is an example for a low motion encode I did last week :


[fast]

pass 1
24.04 fps, 1709.48 kb/s
SSIM Mean Y:0.9275593

pass 2
28.37 fps, 1709.48 kb/s
SSIM Mean =


[best]

pass 1
5.68 fps, 1806.07 kb/s
SSIM Mean Y:0.9324594


pass 2
5.87 fps, 1806.07 kb/s
SSIM Mean =


comparing :
speed 517% slower
SSIM 0.5% larger
bitrate 5.6% higher


same, adding --bitrate 1806 to each cmd :

24.08 fps, 1827.58 kb/s
SSIM Mean Y:0.9288506

28.17 fps, 1830.42 kb/s
SSIM Mean Y:0.9291035



5.66 fps, 1803.20 kb/s
SSIM Mean Y:0.9325334

5.86 fps, 1828.27 kb/s
SSIM Mean Y:0.9328857


"Best" is essentially the cmd-line in my previous post. "Fast" is same less bframes, trellis, --subme 1 and --ref 1, --partitions none.
I'm not here to whine or pester you. I'm just surprised that all the bells and whistles do not allow for a greater spread of SSIM values. I'd expected to when I read YUVSoft's very good & exhaustive report.

cheers
audyovydeo

[akupenguin]

Yuv=hpiv_720x480.yuv
Fast="-m1 -r1 -A=none"
Slow="-m6 -r3 -A=all -8 -t2 -w --me=umh -b3 --b-rdo --b-pyramid --bime --direct=auto --mixed-refs --no-dct-decimate --no-fast-pskip"
x264 $Yuv -o xf1.264 --stats xf.2pass $Fast -B500 -p1
x264 $Yuv -o xf2.264 --stats xf.2pass $Fast -B500 -p2
x264 $Yuv -o xs1.264 --stats xs.2pass $Fast -B500 -p1 -b3 --direct=auto
x264 $Yuv -o xs2.264 --stats xs.2pass $Slow -B500 -p2

xf1: bitrate=485.64 psnr=41.263 ssim=0.9535066 fps=56.45
xf2: bitrate=500.41 psnr=41.605 ssim=0.9561926 fps=66.41
xs1: bitrate=473.52 psnr=41.405 ssim=0.9541903 fps=45.50
xs2: bitrate=499.87 psnr=43.343 ssim=0.9671792 fps=9.28

now, how much extra bitrate do we have to give to the fast encode in order to match the quality of the slow encode?
xf3: bitrate=788.00 psnr=43.303 ssim=0.9671580 fps=61.63
xf4: bitrate=797.10 psnr=43.341 ssim=0.9673887 fps=61.50

That's 58% extra bitrate by ssim, or 59% by psnr. Not 1%.
Maybe you're just underestimating the magnitude of a "small" change in ssim?

[audyovydeo]

Maybe you're just underestimating the magnitude of a "small" change in ssim?

Maybe. But when I compare xf2 to xs2 :


xf1: bitrate=485.64 psnr=41.263 ssim=0.9535066 fps=56.45
xf2: bitrate=500.41 psnr=41.605 ssim=0.9561926 fps=66.41
xs1: bitrate=473.52 psnr=41.405 ssim=0.9541903 fps=45.50
xs2: bitrate=499.87 psnr=43.343 ssim=0.9671792 fps=9.28


I get a :
0.9671792 / 0.9561926 x 100 = 101.148% increase in SSIM
for a
66.41 / 9.28 x 100 = 715% increase in encoding speed

for practically the same bitrate (x264 achieves target bitrates marvellously well).

seen and dloaded v655, thanks.
audyovydeo

[nm]

I get a :
0.9671792 / 0.9561926 x 100 = 101.148% increase in SSIM
for a
66.41 / 9.28 x 100 = 715% increase in encoding speed
You mean 1.148% and 615% increase. However, as akupenguin proved very clearly, the quality difference is quite large for the small SSIM value difference. If you wan't to compare percentages in any meaningful way, you're best off targeting the same SSIM value for the two encodes and then compare the bitrates or file sizes.

[audyovydeo]

You mean 1.148% and 615% increase.

I mean a 1.148% increase and a 7 (dot 15)-fold increase. 9.28 times 6.15 doesn't give you 66.41.

the quality difference is quite large for the small SSIM value difference.

Huh ? Since the "SSIM value" is a "measure of quality", your sentence reads :
"the quality difference is quite large for the small quality difference"

which is senseless.


If you wan't to compare percentages in any meaningful way.

What makes you say so ?


you're best off targeting the same SSIM value for the two encodes and then compare the bitrates or file sizes..

Which is exactly what I've been doing, my way. By *not* giving a --bitrate parm to x264, I obtain its "best effort" bitrate. All I've been saying is that I'm amazed at the small differences in SSIM readings vs the large differences *expected by me*.
Now that akupenguin's explained away how a small chunk of SSIM represents a largish chunk of data rate, I am satisfied, and we can all go back to sleep.

Thanks for the time & trouble, akupenguin.
a/v

[nm]

Huh ? Since the "SSIM value" is a "measure of quality", your sentence reads :
"the quality difference is quite large for the small quality difference"

which is senseless.It doesn't read that way. I was referring to the small absolute difference in SSIM values which you were expressing as a percentage. That small change can still be visually very significant: a large quality difference.

Which is exactly what I've been doing, my way. By *not* giving a --bitrate parm to x264, I obtain its "best effort" bitrate.In that case you are using QP 26, which means that with two different sets of x264 parameters, both the bitrate and the PSNR/SSIM values of the encodes are different and therefore it is difficult to make valid quality comparisons. It is better to standardize on something (like the bitrate or the SSIM value) when testing different parameters.

Off topic:
I mean a 1.148% increase and a 7 (dot 15)-fold increase. 9.28 times 6.15 doesn't give you 66.41.
English is not my native language so I could be wrong here, but 7.15 * x is not the same as increasing x by 715% but by 615%.

[delacroixp]

Maybe your source doesn't actually contain any more quality than Q18 can represent, but in that case upscaling won't help either.
I still don't know what that means... since the DAResolution encode is significantly superior to the Q18-Anamorphic...
It certainly seams anachronistic, though having said that, I have discovered no other means to 'significantly' increase 'absolute quality' and bridge the gulf between Original and Transcode quality...
If I make a photocopy... the quality is almost identical... given equal-filesize, H264 should be identical to Mpeg2... but since H264 is more efficient, a 90% or possibly even 80% filesize remake of the original should equate to 'equal quality'.


@delacroixp
I for one would love to know whatyour SSIM and PSNR readings are for these three encodes.

I have downloaded DmitriyV2's MSU Video Quality Measurement Tool! (http://forum.doom9.org/showthread.php?t=95573) but it doesn't seam to allow for, comparison to the original VOB file...
I'm still trying an HD encode of the same episode but I've never worked with PSNR, Delta, VQM, SSIM, et al, though I've become a big fan of consistency, be it constant-quality as defined by DivX or constant-quality as defined by CQ-CRF and perceived by most of the human species...

I'm sure there's some dude out there... Superman, Spiderman, X-man, whatever... who has superior senses, specifically, audio-visual, perhaps nurtured by increased appreciation... who will recognize the defects in the movies that most of us fail to see...

I guess they get to see IMAX movies in 3D... where it's always better on the big screen...


:):D:eek:
Pascal

[akupenguin]

given equal-filesize, H264 should be identical to Mpeg2... but since H264 is more efficient, a 90% or possibly even 80% filesize remake of the original should equate to 'equal quality'.
Try 50% or even less. But that's only if you encode both the MPEG-2 and H.264 from the same lossless source. If you encode H.264 from MPEG-2, it will always be lower quality, no matter how high the bitrate. Because that's how lossy codecs work.

[Manao]

audyovydeo : here's why your reasoning is wrong. Let's say the SSIM computation had an additionnal step, and that SSIM2 = 1 / (1 - SSIM). that would give, for xf2, SSIM2 = 22.83, and for xs2, SSIM2 = 30.47. Suddenly, it's not a 1% increase, it's a 30% increase.

That means you can't do what you want with metrics. Especially, SSIM - nor PSNR - are made to be used relatively.

The proper way to use a metrics, without knowing beforehand how it behaves - which is your case with SSIM - is to try to change the bitrate till both encoding have the same metrics. That way, you can see the bitrate gain at the same quality. And, that way, you can say it's 30% bitrate saving ( which is your case, actually ).

The second way to use metrics needs you to know how the metrics behave. For example, PSNR can be compared by substracting one PSNR score to another. It's usually agreed that a 0.05 dB difference amounts to 1% bitrate ( that formula only works for small differences - less than 2dB in any case, since the PSNR / bitrate relationship is inherently logarithmic ).

In your case, as I said, the so called 1% quality increase is actually a 1.7 dB increase which amounts to 30% bitrate savings. Lots of people here will say that 30% bitrate are worth a 7 folds speed decrease.

[audyovydeo]

@Manao

voila' a thorough explanation ! Coupled with akupenguin's, you prove at least one of my original points was right :


6. I may have missed something out uttely and completely : I beg to be set straight.

many thanks
audyovydeo

[delacroixp]

given equal-filesize, H264 should be identical to Mpeg2... but since H264 is more efficient, a 90% or possibly even 80% filesize remake of the original should equate to 'equal quality'.
Try 50% or even less. But that's only if you encode both the MPEG-2 and H.264 from the same lossless source. If you encode H.264 from MPEG-2, it will always be lower quality, no matter how high the bitrate. Because that's how lossy codecs work.
I'm sure that you are right... it certainly makes sense to me...
So, in other words...
IF your DVD has only 60% of the original (source material) quality... THEN you can only expect to achieve 60% quality in a Transcode (at best)...
ELSE copy directly...


:):D:eek:
Pascal


btw
The belief in the Flat Earth Society (http://en.wikipedia.org/wiki/Flat_Earth_Society) also makes sense to me...

[delacroixp]

@Manao
6. I may have missed something out uttely and completely : I beg to be set straight.


voila' a thorough explanation ! Coupled with akupenguin's, you prove at least one of my original points was right :

I doubt that you we're wrong... true quality is rare and consistency even moreso...
But what's happenned to Kuukunen... it's like being guests to party... and the owner has left town...


:):D:eek:
Pascal

[PuzZLeR]

But what's happenned to Kuukunen... it's like being guests to party... and the owner has left town...


After entertaining some guests, Kuukunen found spiritual salvation from their insight, jumped on the 1-pass train to bliss, and has never been heard from again...

[delacroixp]

But what's happenned to Kuukunen... it's like being guests to party... and the owner has left town...

After entertaining some guests, Kuukunen found spiritual salvation from their insight, jumped on the 1-pass train to bliss, and has never been heard from again...
LoL...
I guess the 1-pass Orient (http://en.wikipedia.org/wiki/Orient_Express) Express (http://www.orient-express.com/web/vsoe/vsoe_a1a_splash.jsp) can be pretty intoxicating...
I might have to rename this thread 'The quest for true absolute quality with H264' during Kuukunen's absence and travels 'Around the world (http://en.wikipedia.org/wiki/Around_the_World_in_Eighty_Days) in 80 Days (http://www.palinstravels.co.uk/static-7)'.

Hopefully, when he gets back, he'll see the light and realise that there's always a next step on the 'Stairway (http://en.wikipedia.org/wiki/Stairway_to_Heaven) to (http://www.brave.com/bo/lyrics/stairhea.htm) Heaven (http://www.youtube.com/watch?v=_mNjd-hnxbs)' ...


:):D:eek:
Pascal


btw
It's my birthday today... I'm a little dissapointed that he's not here to share the joy... and I make for a very lousy host...

[PuzZLeR]

From one fine young old-timer to another:

HAPPY BIRTHDAY my dear friend.

Geordie.

[amdxp]

Anyone who know which program is best to encoding to x264 for the best quality and encoding time? I've used MeGUI with a profile "HQ-slowest" and it take 22 hours to encode the one movie. :confused:

My PC: athlon 64 3500+

[Manao]

It's that one (http://forum.doom9.org/forum-rules.htm), especially #12. You really should look it up.

[amdxp]

I know that. But it's impossible that all programs is the same. I need the one to encode the movie to best quality ( ok, one of the best).

And the rules (http://forum.doom9.org/forum-rules.htm) not helping me to do that.

[akupenguin]

No, really. x264 is x264, no matter what frontend you use. The only difference is the user interface.

[amdxp]

No really. x264 is x264, no matter what frontend you use. The only difference is the user interface.

Thanks for the answer. Now I have another problem. I have the DVD movie with resolutions 720x576 16:9, so what I must set to in MeGUI for good quality? 720x576? Or 640x480? Is there a big difference between 720x576 and 640x480?

[Dr.Khron]

What you are talking about has nothing to do with the x264 encoder. All resizing/cropping is done with Avisynth before the encoder touches the video, so you should be posting this question in the Avisynth forum.
I know this stuff is confusing at first, but you need to make an effort to understand the different steps involved before you ask for help.

As for your question...
Keep in mind that the DVD resolutions have little to do with the "best" size to encode at... what matters is the aspect ratio of the movie once you've completely removed the black bars. If you are using MeGUI to build your Anisynth script, it has options for cropping and previewing before you start fooling with the resize. A good choice is to do your cropping first, and then invoke the "smart anamorphic" mode, which will then calculate your resizing for you.

Here is some good info on Avisynth:
http://avisynth.org/mediawiki/Main_Page

Here is an excellent page that describes how to crop and resize virtually any kind of DVD content:
http://www.animemusicvideos.org/guides/avtech/aspectratios.html

[amdxp]

Ok, thank you.

[PuzZLeR]

x264 is x264 and any GUI that has better output than another is only due to its settings.

However, having said that, if you want to try a nice Windows GUI for x264 you can try HandBrake.

It's easy and has good settings that give excellent speed and great quality for both 1-pass and 2-passes. Both handled very well.

And it even handles anamorphic resize with nice results.

If you like to have control over every little detail then go with MeGUI and AutoMKV, however if you want simple, HandBrake does alot of your thinking for you.

[delacroixp]

If you like to have control over every little detail then go with MeGUI and AutoMKV, however if you want simple, HandBrake does alot of the thinking for you.
Simple is good... That's definitely made for me...


:):D:eek:
Pascal

[delacroixp]

From one fine young old-timer to another:

HAPPY BIRTHDAY my dear friend.

Geordie.
Thanks much... really appreciated...
I suppose that you did notice that the DivX stock price (http://seekingalpha.com/by/symbol/divx) is at an alltime low...


:):D:eek:
Pascal


btw
Sorry about the 2nd post... neuron2 (http://forum.doom9.org/showthread.php?t=125441) and I are competing for the $million 6 posts per day challenge.

[delacroixp]

No, really. x264 is x264, no matter what frontend you use. The only difference is the user interface.
Perhaps, it's not so much the actual video encoding time... but the GUI layout which affects how fast you can get the whole thing going... not to mention multiple jobs if you have 20 or 30... and possibly even the home-grown filter defaults, de-interlacing procedures and suchlike...


:):D:eek:
Pascal

[Kuukunen]

Oh, I'm still here. I was reading your silly/boring arguments about how to calculate percentages correctly and didn't want to interfere with that. :P Then I was just a bit busy with stuff and also wanted to run some tests first, but now I'm back with another megapost, sorry, but don't you dare to tl;dr me, I spent lots of time on this..

Of course tests like these have been done a lot, but more is even better, so I thought I'd give it a try too. This time with cartoon sources. (So I don't know how well the results apply to normal movies etc.)

I did tests on a 20 min clip, but I was using heavy Avisynth filters and x264 settings. (it wasn't just a test) It took me 10 hours each so I did just two. One CRF and after that, 2pass with the same bitrate.

Command lines were:
--crf 22 --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter 2,2 --subme 7 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads 2 --thread-input --progress

--pass 2 --bitrate 839 --stats ".stats" --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter 1,1 --subme 7 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads 2 --thread-input --progress

Actually, I just noticed I used different in loop deblocking for some reason, but does that even matter to the actual encoding too much or just decoding? Or... I might be way off here. Well of course it matters to the metrics if they're calculated from the deblocked image. Quick test gave only a 0.00008 increase to SSIM when comparing 2:2 to 1:1, so it shouldn't matter too much.

I managed to crash my machine right after the first pass in the 2pass test, so I don't have a log from the first pass, but I was just using the MeGui's automated 2pass... Anyways... results...

CRF:
SSIM Mean Y:0.9908159 PSNR Mean Y:47.639 U:50.942 V:51.045 Avg:48.428 Global:47.839 kb/s:836.65

2pass:
SSIM Mean Y:0.9906545 PSNR Mean Y:47.749 U:50.816 V:50.916 Avg:48.496 Global:47.709 kb/s:839.76

So the CRF's SSIM was slightly higher! This is weird... But first some more tests. I thought of the same thing that Manao already said: the best way to use metrics is to try to somehow get the same metrics for two encodes and then compare the bitrates. I used a 40 second clip that had parts from the same source as the first test. Other than bitrate etc, same settings as before. First I did CRF 23.5, then bunch of 2passes with slightly different bitrates.

CRF:
SSIM Mean Y:0.9875892 PSNR Mean Y:44.741 U:47.747 V:47.698 Avg:45.464 Global:44.339 kb/s:1862.46

2passes:
SSIM Mean Y:0.9871943 PSNR Mean Y:44.998 U:47.797 V:47.701 Avg:45.696 Global:44.102 kb/s:1856.15
SSIM Mean Y:0.9870436 PSNR Mean Y:44.936 U:47.763 V:47.641 Avg:45.638 Global:44.019 kb/s:1826.60
SSIM Mean Y:0.9868893 PSNR Mean Y:44.852 U:47.689 V:47.595 Avg:45.560 Global:43.929 kb/s:1795.30
SSIM Mean Y:0.9867403 PSNR Mean Y:44.778 U:47.657 V:47.534 Avg:45.489 Global:43.843 kb/s:1766.14
SSIM Mean Y:0.9865961 PSNR Mean Y:44.710 U:47.593 V:47.478 Avg:45.424 Global:43.761 kb/s:1736.42
SSIM Mean Y:0.9863962 PSNR Mean Y:44.628 U:47.547 V:47.419 Avg:45.347 Global:43.667 kb/s:1705.71
SSIM Mean Y:0.9862169 PSNR Mean Y:44.554 U:47.483 V:47.381 Avg:45.280 Global:43.570 kb/s:1674.80

I thought I'd test with lower bitrates so I'd hit the CRF's metrics with one of the 2pass settings, but surprisingly CRF had AGAIN higher SSIM! Ok, the bitrate was slightly higher too, and maybe not so surprisingly, as settings and source were the same, although a cut version with a lot more motion was used, as can be seen from the bitrates.

So why does CRF have higher SSIM? I can only guess, but I think it might be related to the question I asked earlier, but didn't get an answer for. I mean... SSIM and PSNR were created for still images, so intuitively all sorts of psy effects such as using lower bitrate for high motion scenes are ignored when doing comparison using only metrics. Comparison with just metrics is sort of a bad idea anyways. This might mean 2pass looks better because it uses future frames to predict the best settings for the current frame so the low SSIM zones are in the high motion scenes where they go more easily unnoticed.

And quite right, I checked it out visually and it seemed CRF had a bit more detail in high motion scenes and 2pass a bit more in low motion. Well I only checked three frames, but I also used MSU VQM and it gave the same results: SSIM for CRF was higher in high motion and for 2pass in low motion. Or at least in some parts, mostly they were practically the same.

So from these tests I can draw two conclusions: 1) CRF doesn't lose to 2pass at least too much, at least in the higher range of metrics and at least for cartoons. 2) Don't rely just on the metrics. :P

As an added bonus test, I was wondering how well x264 re-encodes. I mean in theory, with the exact same settings, it COULD be possible to get the exact same video, but that's not too realistic. So anyways, I re-encoded using CRF 23.5 settings and using the CRF 23.5 encode as a source. :P

SSIM Mean Y:0.9946396 PSNR Mean Y:48.241 U:52.142 V:52.320 Avg:49.085 Global:47.578 kb/s:1769.13

Guess it's pretty acceptable, at least when comparing to the original SSIM.

SSIM is no better than PSNR in that sense - the absolute value of each is more an (inverse) measure of how much detail was in the source, rather than how much is left after encoding.
I don't know what you're after with that, but wasn't SSIM designed just for that? For measuring how much of the source is left in the compressed image?
http://en.wikipedia.org/wiki/SSIM
I guess it's easier to compress an image with less noise, if that's what you were after, but in this case I think noise should be considered detail and the more noise is lost, the worse the compression is.

Then onto some miscellaneous things...

I didn't get the total encoding times from the first tests. Mainly because I was doing other stuff on the computer too, but I'd guess it was something like 2pass was 130-150% of CRF's time. This reminds me... is it somehow possible to get x264 report the actual CPU time it used? This would be a lot more accurate than just checking the start and end times.

Another minor x264 related thing... Why is PSNR and SSIM calculation on by default? I mean... if someone checks the PSNR and SSIM usually he knows very well even before encoding he's gonna do it, but on the other hand, an average user who just wants to get video packed as quickly as possible usually doesn't care about them. By the way in case anyone's interested, I also did a quick test with lighter settings and the same 40 sec clip. I got 6.57 FPS without metrics and 6.51 with them.

And yet again... (Ok, this one's not really x264 related.) I haven't paid much attention to it before, but in the MeGui log, I can find three different bitrates:
x264 [info]: PSNR Mean Y:48.241 U:52.142 V:52.320 Avg:49.085 Global:47.578 kb/s:1769.13

encoded 1208 frames, 0.90 fps, 1769.40 kb/s

desired video bitrate of this job: 23 kbit/s - obtained video bitrate (approximate): 1771 kbit/s

Avinaptic gives 1769.29kb/s. Why is this? Which one is the right one? They're all so close it doesn't really matter, though, and this whole questions is pretty boring... I guess the last one in MeGui could be explained because I was using mp4 for container and it counted the container too. And the others are just slightly different ways of calculating it?

Now, yet again rambling about hypothetical encoder settings... Would it be sensible or even possible to have encoder setting for hitting a specified SSIM? I'm talking about average/mean SSIM, so the psy things and such would still be allowed. (Which would mean that this time it definitely needs two passes to be accurate.) Of course, the whole idea is again to try to define the quality of the compressed video beforehand, so if CRF does better job at it than target SSIM, it would be useless. Actually, as indicated by those tests I did, the rate factor should be at least sometimes more accurate. Especially in a video with mostly high motion vs. video with mostly low motion. In high motion video CRF should use less bitrate because it's not noticed anyways and the SSIM thing would just waste it. (CRF doesn't do it as well as 2pass, though, at least not yet.)

Lastly, since this thread turned into talk about metrics, here's a thought... I'm not an expert in metrics and haven't looked into different ones, so sorry if this has already been talked about. I noticed that all the metrics I saw were for still images, not for video. This gave me an idea: motion compensated SSIM. Yea I know what motion compensation is, but I couldn't think of a better term. I mean SSIM that lets high motion scenes to be lower quality and still result in higher numbers. Of course this can be hard to judge, but since the encoders are already doing it when they do the psy effects, I guess someone should have some kind of idea how the motion should be weighed. (Is there any research behind this or has it just been estimated with lots of testing?) Naturally it would still be only a rough estimation, but I guess it should be pretty easy to make it better than the normal SSIM.

[Sagittaire]

Well you simply can't use metric like that: PSNR is able to say that encoding A is better encoding B with high probability if delta is 1.5 dB and not with 0.15 dB in your case. Same thing with SSIM. You must use metric with a confidence threshold and +/- 0.15 dB for PSNR or +/- 0.0001 for SSIM are not good confidence threshold.

[Kuukunen]

Well you simply can't use metric like that: PSNR is able to say that encoding A is better encoding B with high probability if delta is 1.5 dB and not with 0.15 dB in your case. Same thing with SSIM. You must use metric with a confidence threshold and +/- 0.15 dB for PSNR or +/- 0.0001 for SSIM are not good confidence threshold.
Of course... The whole point wasn't that "CRF is better than 2pass", it was "CRF ISN'T worse than 2pass". (with those settings and source.)

But still, on your point... When you compare two images that are compressed with the same codec, I think requiring 1.5dB difference before saying anything about the quality difference is a bit too much. Also notice that the second test had only 1.2dB difference between encodes and had 200kb/s difference in bitrate. (And again, remember, same codec, very similar settings.) Is that meaningless too?

The other main point was also already in my post: don't trust the metrics. Did you read it at all? :(

[Manao]

As you pointed out, PSNR and SSIM are meant to measure the distorsion of a picture. So they have to be slightly modified to measure the distorsion of a video.

The problem is, how to modify it. For example, for PSNR, you could say PSNR(video) = average(PSNR(frames)). But, since PSNR(frame)=log(frame square error), you could also say PSNR(video) = log(video quare error).

Of course, both formulas give widely different results, and both are reported by x264 : Avg:45.464 Global:44.339 kb/s:1862.46
Avg:45.696 Global:44.102 kb/s:1856.15And - what a surprise - average and global PSNR move in opposite ways - in that case.

So, the process "metrics frame" -> "metrics video" isn't a straightforward process. For SSIM, the same applies, especially since I don't think anybody reflected on the way to apply SSIM for a video. I don't think a plain average - as it is done currently - works as it should.

Back to the average/global PSNR. The difference in behavior put the emphasis on what can be done and can't be done when using metrics with a codec. Metrics are fine as soon as they are used properly, and judging a rate control by metrics isn't necessarily the best of uses.

Average PSNR, for example, will prefer a rate control that increase the quality(PSNR) of a lots of frames, even if it means decreasing the quality(PSNR) of some. It is likely, for example, to be higher for CBR than for CQ.

Global PSNR, on the other hand, is mostly influenced by the lowest quality(PSNR) frames. So global PSNR prefers rate control that keeps the PSNR relatively constant.

That matches what you observed.

As for average SSIM, I don't know which rate control it would like most, but from the figures you gave, it seems it prefers a more constant quality.

[PuzZLeR]

This is exactly what I meant earlier when I felt that quantitative testing PROBABLY may not be sufficient to determine a winner between 1-pass and 2-pass. These readings are like a balance sheet, which measures a point in time, not an income statement which shows a length of unit time.

Do these readings quantify a “blurriness coefficient” in higher motion scenes? If so, then please show me how and I will admit I’m wrong.

Remember, we are comparing 1 pass vs 2-pass with the same bitrate, so each scheme is limited to a certain fixed amount of data to work with. Qualitatively what was happening with ASP codecs is that more frames ended up looking nicer in 1-pass because 2-pass was adding more of the bitrate to motion.

How can this be properly measured/quantified when only using pictorial info?

Then again, with x264, I don’t notice any difference in the two with my eyes like I certainly did with ASP codecs. If there is an edge, I’ll give it to 2-pass only slightly, but that’s because I’m probably biased I admit from previous habits. Akupenguin has implemented features in the x264 codec to quite likely deem 2-pass an option only if a “fit” is needed.

Even though I’m still a 2-pass kind of guy, and if maybe it still gives (slightly) better quality, I have a high amount of video yet to encode, so I’m now going 1-pass for the ease of workload.

Depending on people’s estimates, 2-pass is about 175% the work of 1-pass. But in my case, that would be 275%, because I would still need to run 1-pass to determine the quality bitrate needed so I can then feed that into 2-pass for the final clip.

IMO, to get a 1% quality boost (IF you do) is not worth it for almost 3 times the work, even though I have a late-model D-core CPU. The rate of quality increase is much slower than the rate of workload increase.

Kuukunen, you raised an interesting point where a useful feature could be a parameter of PSNR/SSIM reading, but I have another one.

Is there a feature in the x264 codec (or is it a GUI thing?) that can automate the process from 1-pass to 2-pass? What I’m talking about is that the first pass would be quality-based and output a workfile such as a stats/log file. Then the second pass would be ABR, however it would use the average bitrate determined from the workfile and finish the job.

This would personally reduce my workload down to less then 200% instead of 275% (+the extra work setting up the 2-pass). We also get the best of both worlds and it may be the optimal solution and the clear winning scheme between the two.

Akupenguin, is this possible? Or is this a job that is GUI/profile-based for someone like buzzqw, berrinam or Sharktooth?

[Manao]

Do these readings quantify a “blurriness coefficient” in higher motion scenes? If so, then please show me how and I will admit I’m wrong.There's no such things as "blurriness coefficient" for average/global PSNR nor for average SSIM.

The closest to what I guess you want - a metrics that cares more about low motion frames than high motion ones - is the average PSNR.

However, that doesn't mean you should use that metrics.

[PuzZLeR]

"Bluriness coefficient" is just a hypothetical name. If I knew how to quantify this I would pick a better name than this one.

I don't think an average of PSNR would be ideal either to measure this.

The true measure would probably be one of higher mathematics, such as calculus. We would need to measure the rate of change of a continuous/real curve in some way, not an average for a discrete set of points. We are observing motion not points on a graph.

Then again, it's been awhile since I've been academic so I may be rusty here.

[akupenguin]

Is there a feature in the x264 codec (or is it a GUI thing?) that can automate the process from 1-pass to 2-pass? What I’m talking about is that the first pass would be quality-based and output a workfile such as a stats/log file.

Just specify --crf instead of --bitrate in the first pass.
And 175% is still too much. My 2pass encodes take 120% of the time compared to a 1pass encode with the same settings as the 2nd pass.

[jbrjake]

You can automate the process somewhat, at least in Mac OS X and other *nix environments. You have to save the standard error output from the first pass to a file. This is a bit of a pain if you still want to see it in the terminal -- you have to juggle input and output around to use tee by adding this to the end of the command line:
3>&1 1>&2 2>&3 | tee /tmp/crf-pass.log 3>&1 1>&2 2>&3
You can use some cli tools like cat and grep and awk and sed to find the line where x264 gives the video bitrate at the end:
cat /tmp/crf-pass.log | grep 'kb/s' | awk '{print $NF}' | awk -F\: '{print $2}' | sed -e 's/\...//'
Plug it in for the bitrate on the second pass:
--bitrate $(cat /tmp/crf-pass.log | grep 'kb/s' | awk '{print $NF}' | awk -F\: '{print $2}' | sed -e 's/\...//')
Then you can use a semicolon to string the two commands together and run both passes one right after the other without further user intervention. Here's an example using mencoder (which always seems to store the first pass's frame stats in divx2pass.log in the working directory). The only thing to fill in is the input file path:
mencoder INPUT -vf format=i420 -nosound -ovc x264 -of rawvideo -o crf-pass.h264 -x264encopts stats=divx2pass.log:pass=1:crf=15 3>&1 1>&2 2>&3 | tee /tmp/crf-pass.log 3>&1 1>&2 2>&3 ; \
mencoder INPUT -vf format=i420 -nosound -ovc x264 -of rawvideo -o abr-pass.h264 -x264encopts stats=divx2pass.log:pass=2:bitrate=$(cat /tmp/crf-pass.log | grep 'kb/s' | awk '{print $NF}' | awk -F\: '{print $2}' | sed -e 's/\...//')

[Kuukunen]

So, the process "metrics frame" -> "metrics video" isn't a straightforward process. For SSIM, the same applies, especially since I don't think anybody reflected on the way to apply SSIM for a video. I don't think a plain average - as it is done currently - works as it should.
Actually... I was thinking about it, and I'd guess average SSIM isn't too bad. From Wikipedia: "0.95 SSIM, for example, would imply half as much variation from the original image as 0.90 SSIM." So if we have two frames, the other 0.90, and the other 0.95, average SSIM would be 0.925. But wouldn't it be logical to think that's how much the whole video has left of the original?

I wanted to know more about it, so I read the paper for SSIM. (http://www.cns.nyu.edu/pub/eero/wang03-reprint.pdf) Ok I didn't read it all, it's long, boring and involves lots of math. (Kinda like my posts, sans real math.) The main thing I wanted to check was how the index is calculated for the whole image from the small parts. It seems they DO use simple mean. This would mean that if you put those 0.90 and 0.95 frames side by side in one frame you'd get 0.925. I'm not quite sure what would give better results than plain average if what we're after is an index for Structural SIMilarity over the whole video.

Reading the paper I made some other observations too: SSIM was designed for JPEG and JPEG2000. Or at least that's what they use as examples. What I'm trying to get to here is it should work pretty well for other DCT based stuff too... like H.264. (Yea, JPEG2000 is wavelet-based.) Other observation was that SSIM seems to be a lot better than PSNR. Check the stuff at page 11 and 12. I already trusted SSIM a lot more, but this makes me even more confident.

Back to the average/global PSNR. The difference in behavior put the emphasis on what can be done and can't be done when using metrics with a codec. Metrics are fine as soon as they are used properly, and judging a rate control by metrics isn't necessarily the best of uses.

...

As for average SSIM, I don't know which rate control it would like most, but from the figures you gave, it seems it prefers a more constant quality.
But constant quality was exactly what we're after here! When I was writing that post, I kind of thought of the SSIM based rate control in middle of writing it, so I didn't get to think it through properly. After a bit of thinking, I'm getting more and more confident it could be a good idea.

Think about it like this... All of the encoding schemes (ABR, CRF, 2pass) are about constant quality. Problem is, how to define this quality. Currently it's defined by some method in the depths of x264. Now I don't mean to disrespect that method, but I would guess SSIM gives better estimate about quality, at least for still images. Of course we all agree that SSIM is NOT the absolute and definite value for perceived quality, but I'm thinking currently might be the best one we've got. Meaning if we have constant SSIM, it would better match the "constant quality" scheme I'm striving for.

That was all without taking the psy effects into consideration. As was pointed out, the current SSIM was designed for still images. But, as opposed to PSNR, it was very much designed for human visual system. So basically, it's already considering the psy effects inside one frame. The video version (VSSIM?) should consider the psy effects in a group of pictures. The only one I can think of here, is the one I've talked about a lot: bitrate saving in high motion scenes. I'm not sure what would be the best way to do this, but here's one proposition:
SSIM = VSSIM * ( 1 - MB * MI )
SSIM: the acceptable SSIM of the frame
VSSIM: the number we want to keep constant for all frames
MI: motion index, on scale from 0 to 1 how much motion is in the frame
MB: motion boost, how much of the SSIM is affected by the motion index on a scale from 0 to 1 (yes, the name sucks)

So for example for desired VSSIM of 0.95 in a scene with motion index of 0.5 and and motion boost of 0.5, a SSIM of 0.7125 would be acceptable. MI of 0.4 and MB of 0.2 would give 0.874 for acceptable SSIM. And the whole idea is that for motion index of 0, and any MB, the SSIM would be the same as VSSIM. This would of course need exhaustive tests, especially for finding good values for MB.

In the end this boils down to two things. First, is SSIM really a lot better than the rate factor currently used? If there isn't any real difference, there's no need to go through all the trouble. Second, how feasible is it to try to get to a target SSIM? It might be pretty hard and in the worst case scenario one frame would have to be encoded multiple times. However, from the tests I did for the last post, I noticed that SSIM doesn't change too much from frame to frame. And it should correlate pretty well to the rate factor, so it should be somewhat easy to make some estimations.

This all is yet again just hypothetical and I AM pretty sure the rate control does a good job. I'm just throwing ideas around of how it might be possible to improve it. (Even if just a bit.) I'm tempted to play around with the code to try to make some sort of proof of concept hack for the ideas I've presented in this thread.
This is exactly what I meant earlier when I felt that quantitative testing PROBABLY may not be sufficient to determine a winner between 1-pass and 2-pass. These readings are like a balance sheet, which measures a point in time, not an income statement which shows a length of unit time.
Of course it's sufficient. We just need good enough metrics. :P
Do these readings quantify a “blurriness coefficient” in higher motion scenes? If so, then please show me how and I will admit I’m wrong.
Do you mean something like: http://www.compression.ru/video/quality_measure/info_en.html#ybluringmeasure
Remember, we are comparing 1 pass vs 2-pass with the same bitrate, so each scheme is limited to a certain fixed amount of data to work with. Qualitatively what was happening with ASP codecs is that more frames ended up looking nicer in 1-pass because 2-pass was adding more of the bitrate to motion.
Wha? Are you sure? I would've guessed exactly the other way around, like it should be... Too lazy to tests now.
How can this be properly measured/quantified when only using pictorial info?
It can't, hence the whole VSSIM thing.
Kuukunen, you raised an interesting point where a useful feature could be a parameter of PSNR/SSIM reading, but I have another one.

Is there a feature in the x264 codec (or is it a GUI thing?) that can automate the process from 1-pass to 2-pass? What I’m talking about is that the first pass would be quality-based and output a workfile such as a stats/log file. Then the second pass would be ABR, however it would use the average bitrate determined from the workfile and finish the job.
That's very much GUI (or UI) thing. I mean, as far as I know, x264 doesn't give any automation for any 2pass. But it's already very possible to do. I even mentioned this in my first post. (the method #2)
This would personally reduce my workload down to less then 200% instead of 275% (+the extra work setting up the 2-pass). We also get the best of both worlds and it may be the optimal solution and the clear winning scheme between the two.
On the other hand with lookahead buffer thing, it would still be reduced by half.

I was going to propose something like jbrjake already did. I use Windows, but I have Cygwin for stuff like shell scripts. So jbrjake's method can be done on Windows too with Cygwin. I guess it might be possible to do it with the standard batch scripts, but I don't know how sensible it would be.

[akupenguin]

From Wikipedia: "0.95 SSIM, for example, would imply half as much variation from the original image as 0.90 SSIM." So if we have two frames, the other 0.90, and the other 0.95, average SSIM would be 0.925. But wouldn't it be logical to think that's how much the whole video has left of the original?
Circular reasoning. Insofar as SSIM correlates with perceived quality, .925 is by definition halfway between .90 and .95. But that doesn't mean SSIM is scaled well; the same could be said of PSNR.

There are three levels on which a distortion metric can be calibrated:
1) Monotonic. This means that D(encode1) > D(encode2) implies encode2 looks better than encode1.
2) Quantitatively correct for a single subject. This means that D(encode1)=2*D(encode2) implies encode2 looks twice as good as encode1.
3) Quantitatively correct for multiple subjects. This means that D(encode1 of scene1)=2*D(encode2 of scene2) implies encode2 looks twice as good as encode1.

Note: in quantitative calibration I'm including any meta-psychovisual effects, so I can assume that after calibration the total quality of a movie is equal to a simple temporal average of instantaneous quality. If the most distorted scene in a movie has a greater impact on the total perceived quality, then the metric should increase the spacing between highly distorted quality levels, so that an average works.

AFAIK, SSIM is only calibrated on part (1).
SSIM is theoretically correct on part (2), but only by the same theory that says PSNR is correct, and we've already determined that's inaccurate.
I haven't heard of even an attempt to calibrate any metric on part (3). And (3) matters if you want to use a distortion metric for ratecontrol.

Furthermore, even assuming a perfectly calibrated metric, you don't want to keep quality constant throughout the movie. Instead, you want to maximize total quality / minimize total distortion in a given bitrate. This is equivalent to minimizing total distortion+bitrate*lambda for some constant lambda ("RDO"). Which is in turn equivalent to holding d(distortion)/d(bitrate)=-lambda. The result of optimizing for that is that quality varies over time, modulated by how many bits it would take to improve any given scene, and the optimal distribution depends on the compression format. And such optimization can only be practically used for tuning a heuristic ratecontrol algorithm; it's ridiculously slow to optimize for directly.

[PuzZLeR]

You can automate the process somewhat, at least in Mac OS X and other *nix environments.

What about Windows?

BTW - Jbrjake, welcome to Doom9 with your first post. It's great having one of the devs for HandBrake, an awesome GUI IMHO, give us some insight.

[foxyshadis]

My plan was to weight frames based on their amount of texture, since in most cases more texture=more motion (or more noise, which is also unwanted as far as a quality measure is concerned). The problem is I can't decide on a good way to weight itex vs ptex. Alternately, I could base it on the ratio of i/p/skip blocks, but I haven't tested that yet. Heavy quantization would probably throw this measurement off, but then again it also seems to reduce the amount of visual action in fast scenes.

A more direct method would be measuring distortion before and after applying texture, but modifying lavc or x264 for a vague hypothesis is a lot more work than slapping together scripts and spreadsheets.

[delacroixp]

IMO, to get a 1% quality boost (IF you do) is not worth it for almost 3 times the work, even though I have a late-model D-core CPU. The rate of quality increase is much slower than the rate of workload increase.

Unless you have a 'reson d'etre' to transcode, it all becomes pretty academic... since some quality will be lost in the works...
However, there may be one movie (your alltime favourite) that merits extra attention... in which case you might apply 3-pass or Q16-CQ-CRF...


:):D:eek:
Pascal

[Kuukunen]

Circular reasoning. Insofar as SSIM correlates with perceived quality, .925 is by definition halfway between .90 and .95. But that doesn't mean SSIM is scaled well; the same could be said of PSNR.

Ok, I had one problem in that post. At times I was talking about pure similarity of two videos and at times I was talking about how to make SSIM match the impression left to viewers. Sorry about that.

I mean, when using SSIM for a picture it's assumed that the picture is of the same quality all over. We can't assume that for video. (As in, same quality frames all over the video.) But still I wanted to make a point that the "structural similarity" would indeed be 0.925.

On the other hand, if you think about a video in which half of the frames are perfect quality and half are horrible mess, then compare that to something with the same average SSIM that's constant quality all over the video. Which one would be better? I went with intuition and chose the latter. So with these assumptions I thought the constant SSIM could be pretty good idea.
There are three levels on which a distortion metric can be calibrated:
1) Monotonic. This means that D(encode1) > D(encode2) implies encode2 looks better than encode1.
2) Quantitatively correct for a single subject. This means that D(encode1)=2*D(encode2) implies encode2 looks twice as good as encode1.
3) Quantitatively correct for multiple subjects. This means that D(encode1 of scene1)=2*D(encode2 of scene2) implies encode2 looks twice as good as encode1.

*snip*
AFAIK, SSIM is only calibrated on part (1).
SSIM is theoretically correct on part (2), but only by the same theory that says PSNR is correct, and we've already determined that's inaccurate.
I haven't heard of even an attempt to calibrate any metric on part (3). And (3) matters if you want to use a distortion metric for ratecontrol.

Actually with constant SSIM, the one I was after would be "D(encode of scene1)=D(encode of scene2) implies they seem to be same quality".

But even still, here's a fun diagram from the SSIM paper, from the pages I mentioned in the last post:
http://www.niksula.cs.hut.fi/~ajantti/ssim.png
In this MSSIM is Mean SSIM, meaning it's a mean over the SSIM map of the whole image. MOS is Mean Opinion Score. 175 JPEG images and 169 JPEG2000 images made from 29 source images were rated by ~20 students. (Read the paper for more details.) To me it seems SSIM does pretty good job at values higher than 0.85. And actually for low quality images PSNR is better. Of course it's pretty small survey, but at least it's something. The point isn't about how good SSIM is, the question I'm interested in is: "is it better than the method currently used?"

Note: in quantitative calibration I'm including any meta-psychovisual effects, so I can assume that after calibration the total quality of a movie is equal to a simple temporal average of instantaneous quality. If the most distorted scene in a movie has a greater impact on the total perceived quality, then the metric should increase the spacing between highly distorted quality levels, so that an average works.
You must mean you don't include meta-psychovisual effects? Maybe you should. This is related to the example I made earlier about the two different videos with same average SSIM. Or maybe an even better example: transparent encoding vs. perfect encoding with 5% of frames horrible mess. (Or something like that.) Yes, I was thinking about just weighing the worse scenes too, but if you force constant motion weighed SSIM over the whole video, there are no most distorted scenes. Again, following the "weakest link" mindset.
Furthermore, even assuming a perfectly calibrated metric, you don't want to keep quality constant throughout the movie. Instead, you want to maximize total quality / minimize total distortion in a given bitrate.
As I said, I don't care about setting bitrate, I care about setting the level of total quality to an acceptable level. As for maximizing quality, I'd really like to keep it constant. When you have an almost transparent encoding, you don't notice the errors everywhere. Usually there are certain hard to encode scenes that give it away. In my opinion, the rate control would be perfect when by setting the quality level, the noticable errors would be either everywhere, or nowhere to be found. Of course I'm not saying there should be artifacts everywhere, I'm saying that the optimal level of quality would be just a tad higher than that quality. Why? Because I'd guess it's pretty much the lowest bitrate you can do for a transparent encoding. This is mostly just theoretical, but should be strived for. I'm not saying using SSIM for rate control would do this, once again, all I'm saying it might bring that a bit closer.

Hmm... Come to think of it, if you're going for a transparent encoding MSSIM itself might not be enough. The noticable errors aren't all over the worst frame after all... So you should also take into consideration the parts of the frame with the lowest SSIM values, indicating some very hard to encode spot. I don't know if it's even possible to boost a bitrate of a certain spot with H.264, so I guess it could just mean the whole frame would have to be encoded in higher quality.
This is equivalent to minimizing total distortion+bitrate*lambda for some constant lambda ("RDO"). Which is in turn equivalent to holding d(distortion)/d(bitrate)=-lambda. The result of optimizing for that is that quality varies over time, modulated by how many bits it would take to improve any given scene, and the optimal distribution depends on the compression format. And such optimization can only be practically used for tuning a heuristic ratecontrol algorithm; it's ridiculously slow to optimize for directly.
I can see what you're after, but are you sure that's always the best way? Yes, it might be counterintuitive to spend loads of bitrate on a small set of the frames just to get them to high enough quality, but if that's the only possible way to get transparent encoding, I say it should be done. (lambda=0)
What about Windows?

Read last part from my previous post.

[akupenguin]

(not responding to the first part because I can't really articulate my objections)
I don't know if it's even possible to boost a bitrate of a certain spot with H.264, so I guess it could just mean the whole frame would have to be encoded in higher quality.
It's possible. If you really want to, you could enforce that every single macroblock has the same SSIM.

I can see what you're after, but are you sure that's always the best way? Yes, it might be counterintuitive to spend loads of bitrate on a small set of the frames just to get them to high enough quality, but if that's the only possible way to get transparent encoding, I say it should be done. (lambda=0)
Sure, a perfect quality metric would essentially max out at transparent, so both RDO (in the degenerate case of lambda~=0) and constant metric would produce the same results there. They differ in non-transparent encodes. And if you don't have enough bitrate to get transparency everywhere, then yes I do say it's counterintuitive to want artifacts everywhere instead of artifacts in just a few of the most complex frames. I have seen movies which spent half their total bits on one scene if encoded in CQP, and that scene still had lower SSIM than the rest.

[Kuukunen]

My plan was to weight frames based on their amount of texture, since in most cases more texture=more motion (or more noise, which is also unwanted as far as a quality measure is concerned).
I mentioned this already earlier, but it's not codec's job to do denoising. And if noise is lost, it's a bad thing. (As it might be actual detail.) I don't know if that's what you meant by it, or if you meant it's just a good side effect, but you're saying noisy frame should get less bitrate? Anyways, I haven't thought about the practical way of weighing the frames too much so I dunno.
(not responding to the first part because I can't really articulate my objections)

It's possible. If you really want to, you could enforce that every single macroblock has the same SSIM.
Hmmm..... That would be very interseting. At least as a test. I'd guess it could cause lots of overhead though?
Sure, a perfect quality metric would essentially max out at transparent,
Or actually, I'd like to have a metric that would create that constant borderlines of "tiny errors everywhere or no errors at all" for different viewers and different equipment. Oh and linear correlation to perceived quality regardless of image or video, please. :P
so both RDO (in the degenerate case of lambda~=0) and constant metric would produce the same results there. They differ in non-transparent encodes. And if you don't have enough bitrate to get transparency everywhere, then yes I do say it's counterintuitive to want artifacts everywhere instead of artifacts in just a few of the most complex frames.
But the whole point was to try to find the forementioned borderline. And if every frame is just over that line, there's loads of bitrate saved in the easy frames while still attaining transparency.
I have seen movies which spent half their total bits on one scene if encoded in CQP, and that scene still had lower SSIM than the rest.
Of course there are difficult cases like that. But if it was high motion (it probably was.) lower SSIM is acceptable without noticing too many errors.

Anyways, I'm not saying that non-zero lambda is useless, I'm just saying if you want the same quality everywhere, you might want to set the lamba to 0.

This whole thing is pretty much hypothetical anyways, and I'd guess there is very little hope to see an actual implementation. I've been exaggerating for the sake of argument, so I was expecting strong objections.

[Manao]

I'm just saying if you want the same quality everywhere, you might want to set the lamba to 0Would you agree that, if you want the same quality everywhere, then the only thing that matters is that quality, and then that you don't really care about filesize ?

If you were caring about bitrate, what would you prefer : a movie with a constant quality, or a movie with 90% of the scenes with a higher quality, and 10% with a lower quality ?

[Kuukunen]

Would you agree that, if you want the same quality everywhere, then the only thing that matters is that quality, and then that you don't really care about filesize ?
I've purposely avoided the sentence "I don't care about filesize". Of course I care about filesize... I want the minimum filesize for certain level of constant quality. Of course in pathological cases it might be justified to relax the constant quality requirement a bit.
If you were caring about bitrate, what would you prefer : a movie with a constant quality, or a movie with 90% of the scenes with a higher quality, and 10% with a lower quality ?
Constant quality, ...? There are times when you need certain bitrate. There are even times you need CBR. But this thread has been about constant quality from the beginning. Well, as I said, (and aku too) it might not be feasible always, in theory sometimes the minor quality increase could suck horrible amounts of bitrate, but those are high motion scenes and don't need that much weight anyways. (Still constant quality, if the metrics used compensates for motion.) I haven't done any tests on this though, I don't even have any pathological source. Or one with very high-motion opening with rest in low-motion. Maybe it's just me, but if there's a scene with noticeably worse quality, I get annoyed.

[akupenguin]

Then I have misunderstood your goals from the beginning. You started by talking about CRF and 2pass, both of which try for the RDO I described, not constant quality.

[Manao]

I didn't make myself understood.

Do you agree that :
- There's the minimum watchable quality, under which you wouldn't want to watch the video. There's the transparency quality, over which you can't see any improvements. And there are qualities in between.
- Those qualities form a continuum, and two nearby qualities are very hard to distinguish from one another.
- If your constraint is a filesize and you want a constant quality, you can't choose that quality, it is chosen by the bitrate.
- Given that you don't choose the constant quality, and that the obtained quality is in between the minimum watchable quality, would you prefer that encoding, or an encoding with the same size, but with a somewhat more variable quality, which makes the overall quality noticeably higher, but, on some scenes, a bit lower than the constant quality which - I repeat - you didn't choose ?

[Kuukunen]

Then I have misunderstood your goals from the beginning. You started by talking about CRF and 2pass, both of which try for the RDO I described, not constant quality.
Yes, some of the stuff I just thought of while writing a post, so it might not all make sense together without explaining... Some of it is a bit of a mess. And I think I might've had some initial misconceptions, sorry about that... But that still doesn't make me feel I don't have a point. :P
I didn't make myself understood.

Do you agree that :
- There's the minimum watchable quality, under which you wouldn't want to watch the video. There's the transparency quality, over which you can't see any improvements. And there are qualities in between.
Yes, but video quality really isn't quantitative in one dimension. Or... of course you can ask 100 people to rate the video and get one average number, if that's what you meant, but more on this lower. (I'll use the term MOS, or Mean Opinion Score from the SSIM paper.)



- Those qualities form a continuum, and two nearby qualities are very hard to distinguish from one another.
If by quality you meant that MOS-quality, no. Or, depending on what you mean by "distinguish", but if you mean "the ratings are very close", then your sentence doesn't say anything, so I assume you're talking about distinguishing a video from another with the same MOS. Of course it's usually true if you have same encoder and almost the same settings, but one of the main points I've had here is that you can distribute the quality all over the video in different ways, so two videos with the same MOS can be very distinguishable. Also, there might be huge a deviation in the score, while another video with same mean has much smaller deviation. Usually you'd choose a certain MOS and then minimize the bitrate, but counterintuitively sometimes video with a lower MOS is better. It might be better to get a smaller deviation with the cost of a slightly lower mean so more viewers would be over the minimum watchable quality line. It was just my intuition that constant quality would give smaller deviation.
- If your constraint is a filesize and you want a constant quality, you can't choose that quality, it is chosen by the bitrate.
Yes. Or if I choose that constant quality, I can't choose the bitrate. You can affect the bitrate with settings, but I assume you're talking about the best possible settings. (And most likely slowest.)
- Given that you don't choose the constant quality, and that the obtained quality is in between the minimum watchable quality, would you prefer that encoding, or an encoding with the same size, but with a somewhat more variable quality, which makes the overall quality noticeably higher, but, on some scenes, a bit lower than the constant quality which - I repeat - you didn't choose ?

This is the main question... Of course I would be crazy to say that I ALWAYS want constant quality, even though it means 90% of bitrate is in ten seconds of the video which causes everything else to look horrible. Usually I DO want to choose the quality, and I haven't at any point really talked about choosing the filesize, but the same principles should work there too.

Let's talk about transparency a bit. As you can see from my post count, I haven't been on these forums for too long, so I have to admit I don't know exactly what people mean by "transparency". As far as I'm concerned, there are two kinds of transparency, one where you can't distinguish the video when you watch it under normal viewing conditions, and one where you take a screenshot of a very high-motion scene then zoom it and compare it to the original side to side.

For BOTH of these the constant quality method is the optimal one. Or lambda=0 in RDO. I don't think you can disagree with me on this... Do you?

Of course in those cases the "constant quality" is defined with differnet parameters. As I've been saying all the time, for the first kind of transparency you should also take into consideration the psy effects and steal some bitrate from high-motion scenes. For the second kind of transparency, you can't do this and you have to make every single frame near perfect. Sorry for making up my own abbrevations as I go, but on the VSSIM thing I presented, this would mean MB=0. I can think of only very few situations where this is actually desirable, so when I talk about transparency I mean the first kind... If this a huge misunderstandment and misuse of standard terms, please correct me.

Also, I realize SSIM most likely won't tell this transparency-level, but once again, all I'm saying, it might be better estimate.

So sometimes (very often?) one might want strict constant quality. But... sometimes there are scenes which would hog loads of bitrate for small quality increase. If you're not going for transparency, this might be a big draw back. However, it's very possible to combine the best from both worlds.

Here, I drew a fun picture to illustrate my point:
http://www.niksula.cs.hut.fi/~ajantti/capvssim.png
I think you guessed where I'm getting at, but I'll explain. If there are scenes that are VERY hard to encode, you could still use constant quality, but with slight modifications. Or you might view it as RDO with slight modifications. I mean the distortion+bitrate*lambda -formula is used, but only for scenes where the bitrate requirement would be just too big. I think it would be silly to go over a certain specified level of quality, so if you could use RDO and cap the distortion to that quality, you could save loads of bitrate. I assume the distortion here is the negative/opposite number of VSSIM/whatever. Yes, I understand this bitrate is saved mainly from the easy to encode scenes which don't take too much bitrate anyways, but there are usually lots of those.

[delacroixp]

I'm just saying if you want the same quality everywhere, you might want to set the lamba to 0.
Would you agree that, if you want the same quality everywhere, then the only thing that matters is the quality, and then that you don't really care about filesize ?

If you were caring about bitrate, what would you prefer : a movie with a constant quality, or a movie with 90% of the scenes with a higher quality, and 10% with a lower quality ?
Everyone cares about filesize to some degree... as to constant quality I think that we rely on a 'homogenous human group', biologically and mentally... to define constant quality... nobody's interested in quality that they can't perceive.
Most people dislike smelly French cheese but over time they could learn to enjoy and appreciate... equally, people can become very quickly accustomed to crackly WW II radio and actually enjoy the experience...

Personal history, experiences, tastes and preferences differ...

Maybe it's just me, but if there's a scene with noticeably worse quality, I get annoyed.
I like to give the example of a military parade... you only notice something wrong when 1 idiot marches out of step... likewise if everyone is marching badly, you probably notice the 1 guy who is actually in tune with the music...
I prefer perceived constant quality... but inasmuch as you're not dealing with a completely homogenous group, biologically and mentally... your best hope is probably for some average that will only be accurate much of the time...

No man is an island... but with H264 encoding, you can customise the settings to suit your personal requirements best...


:):D:eek:
Pascal

[delacroixp]

@delacroixp
I for one would love to know what your SSIM and PSNR readings are for these three encodes.

Which tool do you use to measure SSIM and PSNR... does it also include support for VOB Mpeg2 material... ???


:):D:eek:
Pascal

[audyovydeo]

@delacroix :

I merely take the values that x264 dumps at the end of the encodes.
I did download MSU's tool, but haven't yet gotten around to use it. Time's the problem, as always. (it's why we're all spending small fortunes on ever-faster CPUs ???? ;-)

Incidentally, do you or anyone know of a way of redirecting x264's output to a file under windows ? In unix, as was discussed in another thread, one can tee the output then play around with it. I tried the same with cygwin, but tee did not work as expected.
It would make stats collection (and a couple of other things) that much easier...

audyovydeo

[Kuukunen]

Which tool do you use to measure SSIM and PSNR... does it also include support for VOB Mpeg2 material... ???You can use MSU VQMT to measure anything you can load in Avisynth. It also gives per-frame readings. That's what I used... And yea, you can also enable PSNR and SSIM readings from the commandline/MeGui/whatever options. Or more like "undisable", they're on by default.Incidentally, do you or anyone know of a way of redirecting x264's output to a file under windows ? In unix, as was discussed in another thread, one can tee the output then play around with it. I tried the same with cygwin, but tee did not work as expected.
It would make stats collection (and a couple of other things) that much easier...Seems x264 uses stderr for displaying information. So the standard way of "x264 --blah blah 2> file" should work both in Windows batch file and Cygwin shell scripts. If you really want to tee it instead of directing the output, in Cygwin you'll have to do "x264 --blah blah |& tee file". I don't know if there's an equivalent in windows, but unless you want to get the progress in real time, you could first redirect stderr to a file like I described, and then do "type file" in the batch file.

[0gg]

and for MeGUI

=> is it possible to have a .avs script ?
=> is it possible to have jobs ?

... in order to have the best encoding quality !!

thx

[audyovydeo]

"x264 --blah blah 2> file"

indeed it was stderr, not stdout, thanks !
I know it's been said before, but "DOS batch sux".
What I'm trying to do is capture the SSIM and bitrate values of pass1 to variables, do some operations and give it to pass2.
Am breaking my head against brick wall : DOS batch has klunky handling of env variables.
If anyone has managed to [set /p bitrate = cat log.txt | grep blah blah], I'm listening ...

audyovydeo

[delacroixp]

I encoded Band of Brothers (http://en.wikipedia.org/wiki/Band_of_Brothers), chapter 1 in 4 ways... and compared the results... (2857mb = 100%)
I did all the encodes using the AutoMKV GUI, without any filters and resized with Lanczos (downwards) and with LanczosPlus (upwards)...
I also used Photoshop to upscale the VLC png snapshots where necessary... to make an equal size comparison... I did incremental and one-off resizing with different filters to get the best results possible...
HD encodes are at 1280x720 (Original movie is PAL 720x576 \ 16:9)


Q18 Anamorphic (1726mb 60%)... http://souls-online.net/delacroixp/AutoMKV/BoBTest2/1_Q18_Anamorphic_Q23_1280_HD/BoB1_Q18_Anamorphic_1726mb.png (http://souls-online.net/delacroixp/AutoMKV/BoBTest2/1_Q18_Anamorphic_Q23_1280_HD/BoB1_Q18_Anamorphic_1726mb_2.png) Q23 HD (1443mb 50%)... http://souls-online.net/delacroixp/AutoMKV/BoBTest2/1_Q18_Anamorphic_Q23_1280_HD/BoB1_Q23_1280_1443mb.png (http://souls-online.net/delacroixp/AutoMKV/BoBTest2/1_Q18_Anamorphic_Q23_1280_HD/BoB1_Q23_1280_1443mb_2.png)

Q23 HD looks better than Q18 Anamorphic eventhough it's almost 300mb and 10% smaller...



Q22 LD (640x360) (481mb 17%) ... http://souls-online.net/delacroixp/AutoMKV/BoBTest2/2_Q22_640__Q30_1280_HD/BoB1_Q22_640_481mb.png (http://souls-online.net/delacroixp/AutoMKV/BoBTest2/2_Q22_640__Q30_1280_HD/BoB1_Q22_640_481mb_2.png) Q30 HD . (465mb 16%)... http://souls-online.net/delacroixp/AutoMKV/BoBTest2/2_Q22_640__Q30_1280_HD/BoB1_Q30_1280hd_465mb.png (http://souls-online.net/delacroixp/AutoMKV/BoBTest2/2_Q22_640__Q30_1280_HD/BoB1_Q30_1280hd_465mb_2.png)

Q30 HD looks better (just) than Q22 LD eventhough it's almost 20mb and 1% smaller...



It would seam that H264 reaches an 'absolute quality' peak at Q18 but reaches an 'efficiency peak' very much earlier... a bit like a car reaches an absolute top-speed at, let's say, 180 kmh (112 mph) but reaches a fuel-efficiency peak at 90 kmh (56 mph).

The mid point is where Quality(anamorphic encode) = Quality(DAR encode).
After this point, H264 is running fat but efficiency can be restored by increasing DAResolution.
Before this point, H264 is being choked/strangled and efficiency can only be restored by reducing resolution below DAR and even Total-anamorphic-resolution (415 kpixels in the case of Mpeg2 PAL).

At high bitrates... it would seam that... you are better off upscalling anamorphic movies... while maintaining some kind of constant Bitrate-to-Resolution = β(DAR) (β > 1)(β=RealNo)... in other words, more bitrate, more resolution...
You're encodes will undoubtedly take longer... but while the encoder and your computer work harder (and longer) it probably achieves greater levels of efficiency with more, rather than fewer, pixels...

At Low bitrates (β < 1) and a fraction of original resolution.
This is probably also true for 1440x1080 Anamorphic material...


:):D:eek:
Pascal


btw
The layout looks best on a 1280x1024 PC screen resolution or greater...

[akupenguin]

Surely in order to get comparable results you want to use the same scaling filter before and after encoding? All I see is ringing in the HD versions and no ringing in the anamorphic one.

[delacroixp]

Surely in order to get comparable results you want to use the same scaling filter before and after encoding? All I see is ringing in the HD versions and no ringing in the anamorphic one.
I should imagine that to get truly comparable results you should also include figures for PSNR and SSIM.
However, my skills-level is a bit limited though I did post (http://forum.doom9.org/showthread.php?p=1008690#post1008690) a request for suitable software...

I din't use any resizer (scaling filter) for the anamorphic encode since it was encoded at original resolution... VLC player did the upscaling in realtime... as for the 2 HD versions, LanczosPlus was used...


:):D:eek:
Pascal

[akupenguin]

Right. So all you've determined is that you like LanczosPlus better than whatever VLC uses by default, which is probably whatever scaler is built into your video card.

[ToS_Maverick]

my experience with the 24 season 1-4 (Pal) i coded was:

the real resolution of the series was quite low, so you could resize to 640x352 without problems. i was at the same point as you are now... code anamorphic or resize. tests showed me, that the 640x352 with quants ~18 looked more detailed and harmonic than the anamorphic version at ~24.

[delacroixp]

Right. So all you're determined is that you like LanczosPlus better than whatever VLC uses by default, which is probably whatever scaler is built into your video card.
Well.. you seam to be much more determined than I, way better equipped to know whether you are right or wrong and your logic is flawless... so I'll just stick to the simple stuff... (they say that statistics can prove anything)...
I have a pretty crappy graphics card here anyway... but I can only compare Anamorphic and DAR (or HD) by upscalling and then somehow comparing the results... perhaps, taking an anamorphic snapshot (720x576) would exclude GPU intervention to some extent...

I think I should I try to find where Quality(Anamorphic)=Quality(DAR)... 1024x576 in this case... it negates the complexities of upscaling with Photoshop...
I assume you believe that an Anamorphic encode will always maintain more quality than a DAR encode of equal filesize... regardless of bitrate... ie, Q10, Q25 or Q35...

My point doesn't seam to be about the efficiency of player upscale(ability) but whether H264 is encoding less efficiently after some point... perhaps Q25 is an efficiency peak...
Few encoders doubt a quality peak at Q18 but what about an efficiency peak... it may be a case of diminishing marginal returns...


:):D:eek:
Pascal

[delacroixp]

my experience with the 24 season 1-4 (Pal) i coded was:

the real resolution of the series was quite low, so you could resize to 640x352 without problems. i was at the same point as you are now... code anamorphic or resize. tests showed me, that the 640x352 with quants ~18 looked more detailed and harmonic than the anamorphic version at ~24.
Q24 is probably pretty close to some H264 efficiency peak anyway... maybe Q23 in your case...
But if you compared Q22-Anamorphic to Q24-DAR... you might find that the DAR encode comes out the winner in terms of filesize, quality and Q18-640...


:):D:eek:
Pascal


/************************ NOTE ***************************
/* Is the Q18-640 encode, in fact, smaller than the Q24-Anamorphic... ???
/* Try Q25-DAR...
/********************************************************

[kikker]

Given that you will inevitable be scaling either the DAR or the anamorphic encodes upon playback assuming you do not have a monitor that natively displays those resolutions, wouldn't it be more appropriate to compare the 2 encodes with the different quants at the same bitrate/file size?

In other words, you shold be able to generate an xy plot of quant vs resolution where bitrate is identical. Then, genererate an additional 2 curves at other bitrates. Finally, compare the encodes at various frames resizing the anamorphic frame with the same resizing filter as was used in the HD encode.

The graph would look like something as follows where the thick black line (and this wouldn't necessarily be a straight line nor would the constant bitrate curves, but you get the idea) would represent a constant perceived quality.

http://img211.imageshack.us/img211/9350/quantresjy8.png

[Dayvon]

Just throwing this out there...

But I have a DVX100B that and I could record some RAW sample clips if they would be helpful. The reason I mention it is because you currently you are going from a MPEG2 VOB to x264, but I could provide either 24fps@480p clips or 30fps@480p clips if you'd like? Could help with the accuracy on any studies/tests.

[delacroixp]

Given that you will inevitable be scaling either the DAR or the anamorphic encodes upon playback assuming you do not have a monitor that natively displays those resolutions, wouldn't it be more appropriate to compare the 2 encodes with the different quants at the same bitrate/file size?

In other words, you should be able to generate an xy plot of quant vs resolution where bitrate is identical. Then, genererate an additional 2 curves at other bitrates. Finally, compare the encodes at various frames resizing the anamorphic frame with the same resizing filter as was used in the HD encode.

The graph would look like something as follows where the thick black line (and this wouldn't necessarily be a straight line nor would the constant bitrate curves, but you get the idea) would represent a constant perceived quality.

http://img211.imageshack.us/img211/9350/quantresjy8.png
Looks good... I've avoided 'identical bitrates' since 2-pass takes soo long and it was sufficient to show that...
movie1 is better than movie2 IF Quality(movie1) > Quality(movie2) as long as Bitrate(movie1) <= Bitrate(movie2)

Perhaps it's possible to make 10 anamorphic encodes @ 720x576 from Q18 -> Q28... and then 10 HD and 10 640x360 encodes based on the same filesize...
The y-axis would represent quality while the x-axis would show 10 discreet points representing (Q18-Q28 ... also, bitrate1, bitrate2, ... bitrate10)
The anamorphic encodes would be the default (horizontal line) standard quality while the other encodes would be arranged above or below depending on their perceived quality...

Theoretically, the HD and 640 encodes would intersect the anamorphic line at some point...


:):D:eek:
Pascal

[delacroixp]

Just throwing this out there...

But I have a DVX100B that and I could record some RAW sample clips if they would be helpful. The reason I mention it is because you currently you are going from a MPEG2 VOB to x264, but I could provide either 24fps@480p clips or 30fps@480p clips if you'd like? Could help with the accuracy on any studies/tests.
Nice to see MeGUI getting involved... nice icon... I wish AutoMKV could also have a kewl face on the world...


:):D:eek:
Pascal


btw
Seams like Doom9 has a pretty hotrod PC... but where's the RAID stuff !!!