Elephants Dream SD (360p) challenge

[Sharktooth]

true, but VC-1 seems to have an edge at high bitrates since the devs paid particular attention in preserving fine details.

[Inventive Software]

Which is still quite important. ;) This test at 1 Mbit, both VC-1 and H.264 look much the same, only real difference is speed, with WMV9 AP being slower than x264.

And Sharky: I'd have a look at that VC-1 encode again. Seems to be the best VC-1 encode I've done to date on this source at 500 Kbits, but at this bitrate, H.264 wins out, so it seems. I'm waiting on analysis from Ben Waggoner and Zambelli for more suggestions, if they can eek out better quality from WMV9 AP.

[Sharktooth]

note: the "edge" i was referring to is meant about the bitrate/quality scaling...

[bobololo]

Since I made the suggestions in my original post, I've done a number of patches to x264, which I'll try when encoding for this thread once all the patches are compiled and ready:

Great, just keep us posted as soon as you can show your results. It'll be a good thing to check for real all your improvements and evaluate how much they visually enhance.

[Dark Shikari]

Great, just keep us posted as soon as you can show your results. It'll be a good thing to check for real all your improvements and evaluate how much they visually enhance.I'm guessing overall they won't be too noticable; a few % here and a few % there isn't overall that obvious except perhaps at low bitrates.

What would be very noticable would be some sort of new psychovisual options.

[benwaggoner]

New VC-1 encode is on it's way. Powertoy settings were as follows:
Note that you:

Don't need Lookahead on for 2-pass encodes
Don't need to specify Macroblock Mode Cost (it's set correctly via Complexity)
Don't need to set Motion Vector Cost, since that's set by default.

Also, for this clip, I haven't always seen DQuant and Perceptual Option pay off that well at low bitrates. You might try one or the other.

[Inventive Software]

OK, time to try the other way round then. ;)

[Dark Shikari]

x264 encode (http://www.mediafire.com/?fj2chyednxo), 500.51kbps.

commandline:

x264 --bframe 16 --b-pyramid --ref 8 --fpel-cmp satd --me imh --me-prepass --mixed-refs --merange 16 --subme 7 --b-rdo --threads 2 --trellis 2 --bime --weightb --partitions all --direct auto --progress --8x8dct --no-fast-pskip --bitrate 500 --pass 3 -o ED360_IMH.mkv ED360.avs

The comparison of this to VC-1 is night and day.

[bobololo]

x264 encode (http://www.mediafire.com/?fj2chyednxo), 500.51kbps.

The comparison of this to VC-1 is night and day.

I compared your result to "Inventive Software". By watching very closely (i.e. almost frame by frame) I can observe a little bit more details in yours. Hence good point, your patches actually brought some improvements !

However those improvements are too elusive (at least for my eyes) to provide a feeling of a better details preservation or sharpness when watching the sequence live.

Do you have an idea of how slow is your encode compared to Inventive Software's one (your settings appear insane ;)).

[Dark Shikari]

I compared your result to "Inventive Software". By watching very closely (i.e. almost frame by frame) I can observe a little bit more details in yours. Hence good point, your patches actually brought some improvements !

However those improvements are too elusive (at least for my eyes) to provide a feeling of a better details preservation or sharpness when watching the sequence live.

Do you have an idea of how slow is your encode compared to Inventive Software's one (your settings appear insane ;)).
Those settings went at about 2.5 FPS on a Core 2 Duo 2Ghz :p

I wasn't trying for speed; if you want a non-insane encode speed, use one of the following (top is slowest, bottom is fastest):

(8-10% better)
--ref 4 --bframes 16 --b-pyramid --weightb --bime --direct auto --analyse p8x8,b8x8,i8x8,i4x4 --8x8dct --subme 7 --no-fast-pskip --mixed-refs --me imh --me-prepass --trellis 2

(6-8% better)
--ref 4 --bframes 16 --b-pyramid --weightb --bime --direct auto --analyse p8x8,b8x8,i8x8,i4x4 --8x8dct --subme 7 --no-fast-pskip --mixed-refs --me umh --trellis 2

(My favorite for balance of speed/quality)
--ref 3 --bframes 16 --b-pyramid --weightb --bime --direct auto --analyse p8x8,b8x8,i8x8,i4x4 --8x8dct --subme 6 --mixed-refs --me hex --trellis 1

(4-5% worse)
--ref 2 --bframes 16 --b-pyramid --weightb --bime --direct auto --analyse p8x8,b8x8,i8x8,i4x4 --8x8dct --subme 5 --mixed-refs --me hex --trellis 1

(7-9% worse)
--bframes 16 --weightb --direct auto --analyse p8x8,b8x8,i8x8,i4x4 --8x8dct --subme 4 --me hex --trellis 1

[bobololo]

As Mainconcept announced their brand new reference encoder. I gave a try and here is the result (ftp://mood.ateme.com/algo/ed360p-mc-vbr-500k.mp4).

To me it's quite hard to make the difference compared to your result.

I used some similar settings (2pass vbr), I selected all "complex" parameters when available and disabled all "fast" options.

[Sharktooth]

I see big differences between the x264 and MC encodings.
First, MC is more blocky on the backgrounds but in the same scene it has more foreground details. x264 shows banding in place of blocks.
also the initial fade is handled better by the MC encoder.

[zambelli]

And Sharky: I'd have a look at that VC-1 encode again. Seems to be the best VC-1 encode I've done to date on this source at 500 Kbits, but at this bitrate, H.264 wins out, so it seems. I'm waiting on analysis from Ben Waggoner and Zambelli for more suggestions, if they can eek out better quality from WMV9 AP.
The Rapidshare page was showing that the download limit had been exceeded so I couldn't download your VC-1 sample, but I did encode a few of my own. Using our new VC-1 Encoder SDK I was able to fix those pesky fades and improve upon many of the gradients and dark areas.

Here's my VC-1 encode: http://www.citizeninsomniac.com/video/ED_360p_500k_eSDK_DQ.wmv

Dquant is definitely necessary for this encode with VC-1 because of all the smooth and dark areas, but striking a reasonable balance between sending all the bits to those areas versus the textured areas is very challenging. Your encode is likely a better compromise (I haven't seen it, but I can guess that based on your encoding settings and my own results), whereas my linked VC-1 encode has a greater bias for smooth areas and gradients. Neither encodes are perfect - I really don't like the amount of blocking and loss of detail that's happening in the textures, regardless of how detailed the smooth areas might look.

x264 encode, 500.51kbps.
I checked out your x264 encode - and it is excellent. Very impressive. Great sharpness and good decisions about where to lose details and where to preserve them in the context of 500k. x264's deblocking really pays off here, too, because it's hard to actually find blocking artifacts in your encode - whereas I wish I could say the same for the VC-1 encode.

There are a few places where VC-1 eSDK does better, such as in the light gradients between #6300 and #7000, but that's not surprising given the previously mentioned VC-1 dquant bias for smooth areas.

BTW, I've noticed a weird artifact that you might want to investigate: check out frames around #3087 and notice there's some strange chroma ghosting to the right of the character wearing the green coat. There are hints of green all over those wires. Not sure where that's coming from.

BTW, I think you might've used the wrong color matrix for the source conversion - your encode looks greener than my source which I converted using Rec601 since this is an SD source. Did you use Rec709?

[Dark Shikari]

BTW, I've noticed a weird artifact that you might want to investigate: check out frames around #3087 and notice there's some strange chroma ghosting to the right of the character wearing the green coat. There are hints of green all over those wires. Not sure where that's coming from.

BTW, I think you might've used the wrong color matrix for the source conversion - your encode looks greener than my source which I converted using Rec601 since this is an SD source. Did you use Rec709?Yeah, I think I used the wrong color matrix.

The chroma ghosting is very strange, also.

One thing I may try is re-encoding this with my new AQ algorithm--I'm not sure if it'll help at this low a bitrate, but it could decrease blocking even more.

[Sergey A. Sablin]

As Mainconcept announced their brand new reference encoder. I gave a try and here is the result (ftp://mood.ateme.com/algo/ed360p-mc-vbr-500k.mp4).

To me it's quite hard to make the difference compared to your result.

I used some similar settings (2pass vbr), I selected all "complex" parameters when available and disabled all "fast" options.

did you use any AQ? (I didn't check the encode, otherwise I'd know this)
btw disabling "fasts" mostly has influence on speed, which slows significantly, while quality increase isn't that noticeable. Well you probably know...

btw2 - do you mind to have a try to yours encoder?

[Sergey A. Sablin]

did you use any AQ? (I didn't check the encode, otherwise I'd know this)

ok, had a look - no AQ, but looks like it is just 2 plain B-frames instead of 3 hierarchical B-frames comparing to x264.

[Dark Shikari]

ok, had a look - no AQ, but looks like it is just 2 plain B-frames instead of 3 hierarchical B-frames comparing to x264.x264 actually has 16 bframes enabled (my encode) so it can use as much as it wants, basically.

[IgorC]

MC seemed to me more sharp but also a bit blocky than x264.
I did fast comparing x264 and MC to old Ateme High Profile encoder from 2005 with full settings and psy 2. My visual preference is still to Ateme however it has a lower OPSNR and SSIM2 0.24a with lumimask = true.

Bobololo It would be interesting to see your encoder

Can anybody do ASP encode with Xvid, Divx or Libav to compare it to VC-1? Its pretty seems like VC-1 is still on par with ASP and no chance to be compared with H.264

zambelli Your encode is 510 kbit/s. The target bitrate was 500 kbit/s. 2% isn't much but can make difference.

[Sergey A. Sablin]

x264 actually has 16 bframes enabled (my encode) so it can use as much as it wants, basically.

I don't think more that 3 b-frames anyhow useful for real life encoding (probably besides anime), that's why I've limit encoder to 3 b-frames at max.

[Dark Shikari]

I don't think more that 3 b-frames anyhow useful for real life encoding (probably besides anime), that's why I've limit encoder to 3 b-frames at max.In fades more b-frames are useful and so forth, though x264 doesn't have proper fade frame decision yet. Its bad habit to not use 16 bframes IMO in general encoding, because it doesn't slow things down at all to give the encoder the option, and there are cases when it does benefit.

[Sergey A. Sablin]

In fades more b-frames are useful
correct, but using some b-frames and explicit weighted prediction for at least P-frames even more useful.

Its bad habit to not use 16 bframes IMO in general encoding, because it doesn't slow things down at all to give the encoder the option, and there are cases when it does benefit.
how many B-frames in a row were used in this particular stream in x264 encoding at max?
it won't make any difference in speed using 1 or 100 b-frames as settings are rather insane and as far as I know x264 lowres uses fast decisions to construct pb pattern. using more faster settings one should see a small difference in performance.

[shon3i]

IMHO Ateme is now golden mean, it have very sharp picture and less blocking than mainconcept and x264. Psy on macroblock in ateme is very good option which make encoding realy free blocking. Mainconcept/Elecard's encoders realy make more sharper and more blocky encode, but that can be soloved with Adaptive Q and negative values for strenght.

[CruNcher]

yep i can absolutely agree with shon3i my extreme low bitrate HD test shows absolutely stunning visual results, you can really see that tweaking for streaming scenarios (broadcast) really helps overall, and im almost sure that we would also have seen this in the 4th anual MSU test if they would have decided to participate it's really sad.

[bobololo]

did you use any AQ? (I didn't check the encode, otherwise I'd know this)
btw disabling "fasts" mostly has influence on speed, which slows significantly, while quality increase isn't that noticeable. Well you probably know...

Well I'm absolutely not familiar with MC encoder and all I tried to do was to find what could be the highest quality config without any compromise about the speed in order to match with Dark Shikari encode. Actually I'd strongly suggest you to encode it on you own using the best fitted settings rather than letting me fumbling ;)

btw2 - do you mind to have a try to yours encoder?

I wouldn't be honest if i say no ;) As you can imagine my first objective was to evaluate different H.264 implementations compared to ours. And if I haven't made any comments, it was on purpose. Actually I don't plan to post any streams from us (for many good reasons you can believe me). Hence it would be totally meaningless for me to claim anything without any real material to back my statements.

yep i can absolutely agree with shon3i my extreme low bitrate HD test shows absolutely stunning visual results, you can really see that tweaking for streaming scenarios (broadcast) really helps overall, and im almost sure that we would also have seen this in the 4th anual MSU test if they would have decided to participate it's really sad.

The reason why we haven't taken part to MSU's new round is that their comparison conditions are totally out of scope with regards to the research we led those past years. It wouldn't surprise me if our current version even have lower psnr/ssim than the latest beta while MSU evaluation is based on it. Beside, you all know that we're more and more involved in broadcast encoding. Our encoder consequently has been largely tuned for that usage and unfortunately there were not a single test that could reflect the encoding for the TV: interlaced, CBR, short gop, constrained CPB, strict HRD compliance, etc. And moreover, even I've considerable respect for MSU's initiative and work, I have to admit that I've been a little bit disappointed when after explaining them several times that our encoder would not be well suited for their tests (for the reasons described just before). When I also told them that we'd be very happy to take part to their comparison if it would offer some test cases that match the applications our encoder is designed for. They got back to me saying that "You can take part to the comparison privately, you need to pay xxxx euros and your scores will be removed from the public results" !

[DeathTheSheep]

you need to pay xxxx euros and your scores will be removed from the public results
Wow.

Well that puts you in a somewhat awkward situation indeed. :eek:

I was actually wondering if there was a trial/demo/test version of that Mainconcept Reference software somewhere, because I'm a bit surprised by what I saw in the test clip.

[Sergey A. Sablin]

Wow.

Well that puts you in a somewhat awkward situation indeed. :eek:

just to make things clearer - what has been offered by Dmitry (and I believe what bobololo said) is that if someone wants to participate, but don't want to publish the results (for any reason), that this is possible with some payment.

I was actually wondering if there was a trial/demo/test version of that Mainconcept Reference software somewhere, because I'm a bit surprised by what I saw in the test clip.

yes, it is - check out the web site.

[DeathTheSheep]

You are correct, but it's hard to test a codec's full potential (especially concerning minute quality differences) when there's a huge flickering watermark added to every frame. :rolleyes:

[Sergey A. Sablin]

You are correct, but it's hard to test a codec's full potential (especially concerning minute quality differences) when there's a huge flickering watermark added to every frame. :rolleyes:

sure, but anyway better than nothing ;)

[DeathTheSheep]

Maybe. ;)

[IgorC]

you can try Elecard H.264. It has the same MC encoder but instead of watermark there is 15000 frames limitation.

[DeathTheSheep]

On the download page, is it the "Elecard Converter Studio" or "Elecard Mobile Converter"?

[Sergey A. Sablin]

On the download page, is it the "Elecard Converter Studio" or "Elecard Mobile Converter"?

the former one.

[DeathTheSheep]

from elecard website (http://www.elecard.com/products/products-pc/consumer/converter-studio/)
The demo versions have limitations of 15000 frames for conversion, demo period duration of 30 days and logo on the files being encoded.
:rolleyes:

[IgorC]

Elecard just add the watermark this month. hehehe :p not lucky

[DeathTheSheep]

The Sheep of DEATH is never lucky!

[IgorC]

Yeah, I think I used the wrong color matrix.

The chroma ghosting is very strange, also.

One thing I may try is re-encoding this with my new AQ algorithm--I'm not sure if it'll help at this low a bitrate, but it could decrease blocking even more.
What matrix did you use? It is not bad for this source.
I'm testin here with different color matrices. rec709 is too
red and hard to encode due to artifacts on this color

[DeathTheSheep]

Aha! Finally testing out the ol' AQ on low bitrates! Now I'm excited...

[zambelli]

What matrix did you use? It is not bad for this source.
I'm testin here with different color matrices. rec709 is too
red and hard to encode due to artifacts on this color
Rec709 is probably not the suitable color matrix for SD 360p. The default Rec.601 matrix should be used for converting the RGB PNG source to YV12 after resizing, IMO.

[check]

Rec709 is probably not the suitable color matrix for SD 360p. The default Rec.601 matrix should be used for converting the RGB PNG source to YV12 after resizing, IMO.
When you say less suitable, do you mean it actually impacts compressibility, or simply it's the converting to the wrong colours?

[Inventive Software]

@zambelli: Now I'm interested, since mostly it's been a lil off-topic. Currently, the (mostly crude) way of converting the RGB source to YV12 is "ConvertToYV12()" in AviSynth. That's it. With just "ConvertToYV12()", is that using Rec.601 or Rec.709?

[IgorC]

It says that default is Rec601 in avisynth doc.

[zambelli]

When you say less suitable, do you mean it actually impacts compressibility, or simply it's the converting to the wrong colours?
It's converting to the wrong colors because at sub-SD resolutions most drivers (which end up doing the final YUV-->RGB conversion in playback) will assume BT.601. So if you convert source video to YV12 using BT.709, but the video driver assumes BT.601 in playback - you end up with different colors than you started with.

[IgorC]

Libav ASP 500.4 kbits (http://www.mediafire.com/?bbugyyxywrh)

mencoder settings 3passes:
mencoder.exe eld_rec601.avs -o pass3_large.m4v -ovc lavc -lavcopts vstrict=1:vpass=3:vbitrate=500:vb_qfactor=1.5:vb_qoffset=0.0:vqcomp=0.75:vmax_b_frames=2:bidir_refine=4:preme=2:dia=-6:predia=-6:mbd=2:precmp=6:cmp=6:subcmp=6:v4mv:mv0:mv0_threshold=0:trell:cbp:qprd:last_pred=3:psnr:vrc_strategy=0 -of rawvideo -ffourcc MP4V



For watching I used ffdshow decoder with high quality settings for postprocessing:
http://img70.imageshack.us/img70/8562/ffdshowtw6.png

I've evaluated zambelli's new SDK VC-1 video ( http://www.citizeninsomniac.com/video/ED_360p_500k_eSDK_DQ.wmv ).

Visually I prefer ASP

[zambelli]

I've evaluated zambelli's new SDK VC-1 video ( http://www.citizeninsomniac.com/video/ED_360p_500k_eSDK_DQ.wmv ).

Visually I prefer ASP
Any particular scenes or features look better in ASP? I'm not a big fan of this low-bitrate VC-1 quality and I'd love to take such feedback back to the dev team. VC-1 should ideally look better than ASP, if not AVC.

[Sharktooth]

VC-1 HAS TO look better than ASP and expecially at low bitrates. It's more advanced and there's no reason why it shouldnt. However, the current implementation is not so shining comparing it to current ASP encoders.

[IgorC]

Any particular scenes or features look better in ASP? I'm not a big fan of this low-bitrate VC-1 quality and I'd love to take such feedback back to the dev team. VC-1 should ideally look better than ASP, if not AVC.

Sorry, zambelli I provided an ASP video as answer to your vc-1 sample encoded by extra new sdk. And ASP looks really better than current vc-1.
Yeah, we've already heard tale about how VC-1 is inferior at low bitrates and how vc-1 shines at high bitrate?
Sorry what is compression all about? What point to use VC-1 at high bitrates when there is MPEG-2?

There isn't only x264.
Don't subestimate H.264 codecs at high bitrates saying how good vc-1 preserve film grain. There is still some advanced commercial H.264 encoders that use psy models to handle filmgrain and misc. details at middle/high bitrates.
VC-1 is perfectly inferior to full featured AVC at all bitrates.

[zambelli]

Sorry, zambelli I provided an ASP video as answer to your vc-1 sample encoded by extra new sdk. And ASP looks really better than current vc-1.
Yeah, we've already heard tale about how VC-1 is inferior at low bitrates and how vc-1 shines at high bitrate?
Sorry what is compression all about? What point to use VC-1 at high bitrates when there is MPEG-2?
And didn't I just say that Microsoft encoder's low-bitrate quality could use some improvements? I wasn't making any excuses - I was asking for concise feedback. :confused:
Well, thanks for the ASP encoding.

[CruNcher]

The problem is really to find the right balance (especialy if you go into the HVS direction (psycho)visual optimization without takeing metrics into account) between all the factors that play a role, im doing this since day 1 that im here and it is a very time consumeing process each generation, the're even ways to lower the complexity of the streams (for the decoder) so they are almost on paar with VC-1 complexity and that @ lower bitrate keeping still better quality/speed :) imho Microsoft balanced to much for HD (High Bitrate) and totaly forgot about Low Bitrate (this is really bad for VOD and Mobile usage). Apple Engineers do really great in that direction :) even if people won't belive it here but Quicktime is very well balanced.

[Sharktooth]

CrapTime has other problems... like slowness (playback and encoding), huge memory footprint, a sh!tload of bugs, standards incompatibilities (partial implementation), etc.

[Dark Shikari]

Quicktime is very well balanced.Yes, it balances bugginess with a terrible interface, lack of support for standards, and memory bloat.