Ateme H.264 Beta - Quality Feedback

[bobololo]

This new thread is dedicated to the quality feedback only.

Please report here everything related to quality comments about the tools, optimal settings, prefered configurations, etc. Anything that could help us to improve the encoder quality wise should come here :)

-- bobololo.

[superdump]

With the release of Beta 3, the Ateme AVC encoder is maintaining a fairly stable state and now quality tuning can really begin. As such, bobololo requested that I produce an impartial review of the preceding quality-related discussion.

Please note that in this summary I have attempted to convey the tester’s view as accurately as I could. If I have made any misinterpretations or otherwise, don’t hesitate to contact me.

Initial findings were that not only did this codec have huge potential but it has a huge advantage over all other h.264 codecs to date in that it is both fast AND maintains high quality. One user notes that Ateme AVC is more watchable than x264. Motion more fluid and natural, blocking artefacts less pronounced when in-loop deblocking not used. Link (http://forum.doom9.org/showthread.php?s=&postid=539447#post539447)

The codec appears to be very scalable coping with a full range of bit rates and resolutions in most combinations. There were many findings of high quality at HD resolutions with surprisingly low bit rates or very good quality with high bit rates.

Some people noted that an output similar to XviD with constant Q2 with something like half the file size. Link (http://forum.doom9.org/showthread.php?s=&postid=539442#post539442) Another user noted similar results with a HD source. Link (http://forum.doom9.org/showthread.php?s=&postid=540704#post540704) Notice that even when really stressing the codec at 1000kbps for this HD clip, Ateme AVC still holds its own and maintains top-rank quality. Only WMV9 can keep up at this bit rate and XviD has fallen behind.

With beta 2 a comparable looking clip to ~Q3 XviD was made with Q27 in Ateme's AVC producing a bit rate saving of ~40%. Link (http://forum.doom9.org/showthread.php?s=&postid=541323#post541323) Again with beta 2 we have some impressive quality at very high bit rates. Link (http://forum.doom9.org/showthread.php?s=&postid=541065#post541065) And yet more comparisons made, this time between XviD Q4 with the 6of9 HVS matrix and Ateme AVC revealed that Q20 was similarly transparent with a 15-30% bit rate reduction.

Constant quant isn't really recommended however as it doesn't deliver the best quality at a certain file size which partially invalidates some further tests by Teegedeck of XviD with 6of9 HVS at ~Q4-Q5 versus "-qp 23 -adaptdeblock -deblock 3 -qual extra" show similar sized files with generally similar quality aside from a few artefacts in the AVC clip. Link (http://forum.doom9.org/showthread.php?s=&postid=540808#post540808)

Moving on from this the lower resolution, low bit rate findings were also very spectacular. Two 300kbps encodes of the same clip at 640x256 with 48kbps HE AAC audio can still look good. The former clip uses -deblock adapt, the latter -cartoon -deblock -2. The update was made as the latter setting gave a more agreeable output to JohnV on his TFT. Link (http://www.hydrogenaudio.org/stuff/OutOfTime_Ateme-h264_300kbps.mp4) Link (http://www.hydrogenaudio.org/stuff/OutOfTime_Ateme-h264_300kbps_inloop2.mp4)

Across four clips RBF found that Ateme AVC gave a more detailed output with less artefacts than VP6.2/VSS h.264/XviD at 400kbps with a resolution of 720x352. RBF had only used -qual normal for these clips, so it should be noted that -qual extra would make them look even better. :) Link (http://forum.doom9.org/showthread.php?s=&postid=540704#post540704)

One user even thought that their AVC encode looked better than the original! Their source was quite blocky on some mist and as such the deblocking filter dealt with this to the users liking. In beta two we had an incredible clip at 400kbps. Link (http://nero.ateme.com/beta_encodes/cathedral-beta2-400extra-crop-avc.mp4)

There was a lot of feedback about various settings and options and a number of bugs and suggestions for improvement. Remembering that this IS a beta codec we all expected this. One of the main issues with beta 1 and 2 has been the b-frame skipping decision being too aggressive and producing an output with jumpy blocks and jerky motion. This has been fixed for beta 3 and will be very welcome among the testers. The problem only occurred at low bit rates and seemed to be compensated when using high bit rates.

There has been a large amount of discussion regarding the level of deblocking that looks good with certain resolutions/quantisers/bit rates and this discussion continues. Beta 1 had an adaptive mode but the strength of it was completely out of the user's hands. Beta 2 introduced a graded adaptive deblocking method to allow tuning to suit the individual. This gave a good improvement but some suspect that there is something wrong with the implementation and that it sometimes acts too aggressively when not needed and sometimes too weakly when needed. Hopefully this will be looked at soon.

Deblocking is certainly a very powerful feature and, when tweaked, will be a valuable asset to this codec and other h.264 codecs. Teegedeck has an adaptive deblocking behaviour suggestion, as different quants require different levels of deblocking to maintain good quality. Quoted values were - qp 19: none, qp 20: -3, qp 21: -2, qp 22: 0, qp 23: 3. Link (http://forum.doom9.org/showthread.php?s=&postid=541431#post541431)

Not surprisingly, at low bit rates, increasing quality level DOES increase quality level drastically. Sometimes the best/extra quality improvements were debatable but most often extra was clearly better. At high bit rates (704x288 @ 1200kbps) one user found that -qual normal gave a better quality suggesting that it was sharper than good/best/extra.

Cartoon mode seems to work better all round, not just in cartoons, as it takes chroma information into account for various decisions. It may well be recommended to always use it if speed is not the primary consideration.

There were various comments about the rate control, both good and bad and sometimes a mix of the two. :) Bad quality in the last 10 seconds of a clip was noticed and it was reported by babayaga that this was to prevent an oversized output. This would not normally be noticeable in a full film situation as the end is very dark with credits but this method will be removed (due to 'unpopular demand' I suppose).

I personally noticed in my clip that low motion quality was much better than high motion quality and as such suggested an adjustment to the bias. The two-pass rate control was considered to not give constant quality according to some PSNR test comparisons made against the main rival codecs. There was a surprisingly bad quality output produced from a very high motion anime clip that was handled much better by other codecs. I suspect this is due to the aforementioned b-frame skipping but would be dealt by good rate control if the entire anime episode were to be encoded rather than just the start credits. The rate control in beta 3 has been improved so I look forward to testing this.

Hardly anyone could spot any difference between using one reference or multiple references; as such it seems to be a waste of time. Comments suggest that psycho visual level 1 helps with fades and faces and psycho visual level 2 looks about the same as psy1 but possibly slightly less quality due to removed detail. Poor quality was produced from a sharp/grainy/very detailed slow motion source with unnatural contrasts. Link (http://forum.doom9.org/showthread.php?s=&postid=541189#post541189)

Soulhunter noticed some strange artefacts on an 8000kbps 'Shuttle' encode but these have been fixed for beta 3.

Sagittaire tested some settings with SSIM and gave his opinions on which looked best to him. Link (http://forum.doom9.org/showthread.php?s=&postid=541561#post541561)

Finally, some blocks noticed on still areas and backgrounds but there is reduced ringing compared to XviD and good detail preservation noted. Link (http://forum.doom9.org/showthread.php?s=&postid=541348#post541348)

All in all it's been a very busy week or so at Ateme. They seem responsive to our feedback and very helpful with any queries, also the speed of development is shocking. Most of all I would like to thank them for producing a high quality next-generation codec.

[Sagittaire]

@ superdump
waaaaouu ... good job

@ Andrey

Originally posted by Andrey
On low bitrates - definitely. :)
Inloop filtering produce really unexpected (in good meaning) results.

Still can not do good medium-to-high bitrate (~1Mbit) encode - too blurry. Seems that deblocking must be disabled for such a bitrates...
Will check it...

SSIM for deblock adapt (beta 2)

source HPII 640*272
~1700 Kbps with XviD q2 default setting


bitrate 600 800 1000
deblock off 75.26 79.59 83.12
adapt -6 75.46 79.68 83.11
adapt -4 75.46 79.68 83.11
adapt -2 75.47* 79.68 83.11
adapt 0 75.84 79.76* 83.11
adapt 2 76.45 80.21 83.24*
adapt 4 76.96 80.75 83.57
adapt 6 76.71 80.88 83.92

* Deblock activation treshold


Deblock for H264 is "adaptative". For low bitrate (high quant) treshold activation is low and for high bitrate (low quant) treshold activation is high. I think that for very high bitrate deblock is in practice off ...

[Sirber]

On my test clip, it's still heavily squary but it's less blicking. I can say it improved :)

encavc.exe -i test.avs -o test.mp4 -qual extra -rcmode 2pass -br 600000 -psy 2 -maxb 3 -cartoon -ref 3 -adaptdeblock

[RadicalEd]

Happy to report that the B-frame adjustments in beta-3 did the trick; they now work flawlessley with animation. Quality is excellent, surpassing XviD and, in a closer race, WMV9. Real10 is still king of the hill in my initial tests, but I'll get back to that later when the -deblock 6 encode is finished.

Current settings are: defaults + -qual extra -psy 2 -deblock 4 -adaptdeblock -ref 10 -cartoon -rcmode 2pass

[superdump]

I just tested to see what the RC was like now. It's much better.

http://www.swains.plus.com/superdump/256fix.png

Of course, the red line is beta 3.

[Sirber]

It seems to use less in the begining and lots more at the end.

[everwicked]

Originally posted by superdump
I just tested to see what the RC was like now. It's much better.

http://www.swains.plus.com/superdump/256fix.png

Of course, the red line is beta 3.

Is the filesize close to identical?

The lines are almost identical so that could mean 2 things:
- the encoder used more bits
- the coding effiency increased since beta2

[superdump]

Originally posted by everwicked
Is the filesize close to identical?

The lines are almost identical so that could mean 2 things:
- the encoder used more bits
- the coding effiency increased since beta2 Filesizes are almost identical.

1.0.1.17: 21.5 MB (22,647,187 bytes)
1.0.1.19: 21.6 MB (22,653,868 bytes)

I used identical settings for both encodes, and 1.0.1.17 had the b-frame fix.

[Sagittaire]

@ Superdump and everwicked

http://www.swains.plus.com/superdump/256fix.png

if you observe in detail H264 beta 3 is just a little below for 75% of the frames then passes largely above for the last 25%: it's simply a very better RC repartition. I think Average and Overall PSNR will be very better for beta 3 in this case. Visually the end will be also very better quality ... ;-)

[superdump]

Originally posted by Sagittaire
@ Superdump and everwicked

http://www.swains.plus.com/superdump/256fix.png

if you observe in detail H264 beta 3 is just a little below for 75% of the frames then passes largely above for the last 25%: it's simply a very better RC repartition. I think Average and Overall PSNR will be very better for beta 3 in this case. Visually the end will be also very better quality ... ;-) Yes, yes, I can see that. I was just posting it to point out that the changed RC was working properly.

[SeeMoreDigital]

It's all getting rather exciting now.

I wish I could take part more in these tests but my main "encoding" PC is not behaving it's self properly and I have not had the time to format and start again!

Still, I wonder if bobololo could confirm whether the new NVE release will include a seeking fix for H.264/AVC in NeVideo.ax filter. And if NeAudio.ax filter will be accessible via all software players.

Plus... can we see some downloadable sample encodes in this thread please?


Cheers everyone

[bobololo]

Originally posted by SeeMoreDigital
Still, I wonder if bobololo could confirm whether the new NVE release will include a seeking fix for H.264/AVC in NeVideo.ax filter. And if NeAudio.ax filter will be accessible via all software players.

Plus... can we see some downloadable sample encodes in this thread please?

This is not a official confirmation, but I don't see any reason why the seek issue with ShowTime filters wouldn't be fixed by Ahead. I don't know when but hopefuly it should be done in the next update.

Regarding the samples, well a few clips were already posted. Did you catch them ?

[SeeMoreDigital]

Originally posted by bobololo
Regarding the samples, well a few clips were already posted. Did you catch them ? I got just two.

One by your good self and another by plonk420


Cheers

[babayaga]

Originally posted by superdump
Yes, yes, I can see that. I was just posting it to point out that the changed RC was working properly.

There was a significant change in the RC that might help "unexpected clips" to be much better, especially at the end.

At the same time, the strengh of the servo loops has been reduced since some of you prefer to have a more constant quality at the expense of a slightly higher size error.
We expect that the size error wil be kept bellow something like 150kB but there was not enough time to test extensively.

On some "pathological clips" (much too low target bitrate) like the one provided by Sirber, the quantiser climbs up to 51 and the RC is very confused. In this case there is a large oversize/undersize.
Handling such situation is on the way :-)

[Bulletproof]

Originally posted by RadicalEd
Happy to report that the B-frame adjustments in beta-3 did the trick; they now work flawlessley with animation. Quality is excellent, surpassing XviD and, in a closer race, WMV9. Real10 is still king of the hill in my initial tests, but I'll get back to that later when the -deblock 6 encode is finished.

Current settings are: defaults + -qual extra -psy 2 -deblock 4 -adaptdeblock -ref 10 -cartoon -rcmode 2pass

I believe the current cap on -ref is 5, I think someone from nero/ateme said that in the main thread. That should probably be written in the next beta's encavc.txt

[Sagittaire]

At the same time, the strengh of the servo loops has been reduced since some of you prefer to have a more constant quality at the expense of a slightly higher size error.

Acceptable error for me: 0.5% for maxi error target size


Bitrate 500 Kbits/s 1000 Kbits/s 1500 Kbits/s 2000 Kbits/s
size 88 092 Ko 176 185 Ko 264 277 Ko 352 370 Ko
+/- 440 Ko 880 Ko 1 320 Ko 1 760 Ko

XviD 87 920 Ko 175 928 Ko 264 232 Ko 352 360 Ko ... :)
RV10 88 362 Ko 176 397 Ko 264 780 Ko 353 382 Ko ... :)
VP6 89 542 Ko 178 532 Ko 266 106 Ko 354 322 Ko ... :o
WMV9 88 362 Ko 176 482 Ko 264 550 Ko 352 730 Ko ... :)
DivX5 88 614 Ko 176 972 Ko 265 094 Ko 353 100 Ko ... :)
DivX3 88 518 Ko 176 516 Ko 264 516 Ko 352 520 Ko ... :)


PS: test in progress for H264 ... :devil:

[RadicalEd]

Originally posted by Bulletproof
I believe the current cap on -ref is 5, I think someone from nero/ateme said that in the main thread. That should probably be written in the next beta's encavc.txt

I recall it being said that 5 is the practical maximum and gains above that were insignificant, but as far as I know there's no actual limit.

[bobololo]

Originally posted by SeeMoreDigital
I got just two.
One by your good self and another by plonk420

JohnV also posted some good stuff, check at the first page of the initial beta feedback thread.

[bobololo]

Originally posted by RadicalEd
I recall it being said that 5 is the practical maximum and gains above that were insignificant, but as far as I know there's no actual limit.

The max value supported by the spec is 16. But actually values higher than 5 don't help very much.

[Soulhunter]

Originally posted by SeeMoreDigital
I got just two.

One by your good self and another by plonk420


Cheers

Could mail ya a encode of the Matrix lobby chapter -> " if its allowed " !!!


Bye

[Teegedeck]

Originally posted by bobololo
The max value supported by the spec is 16. But actually values higher than 5 don't help very much.
Just to support that: As I still don't have time for testing perceived quality, I've made a simple filesize comparison (again the Matrix Revolutions HD trailer). Here are the bitrate-savings compared to the encode that only used 1 reference frame:

4 reference frames: 0.12% savings
8 reference frames: 0.14% savings
16 reference frames: 0.15% savings

Maybe the overall a bit disappointing results are due to the material. Has anybody else made comparisons on the effect of the number of reference frames?

I hope to have time in order to test whether reference frames perhaps influence quality in a positive way, soon.

And as I'm at it: Has anybody encoded clips with and without the -psy switch and compared the results? I could only see that -psy blurred the picture a bit but unfortunately it didn't seem to do anything else but that.

@bobololo & babayaga: Can you tell me how -adaptdeblock calculates? I'd like to know what deblocking-strength the -adaptdeblock settings I've used actually triggered.

[708145]

Originally posted by Teegedeck
Maybe the overall a bit disappointing results are due to the material. Has anybody else made comparisons on the effect of the number of reference frames?


Maybe someone has a high quality source of a music video? Should be easy to make use of several ref. frames with fast cuts.

bis besser,
Tobias

[Tommy Carrot]

I can confirm that the b-frames are working flawlessly, the pans are as smooth as they should. :)

Btw, i wondered which method does this codec use for b-frames: overquantizing, like in the mpeg1-2-4 codecs, or using the same quantizers as in the p-frames? Seems to me the latter, because i cannot find any quality-level difference between the continous frames, which would be sign of overquantized b-frames. In this case this codec can get compression gain with b-frames without any quality loss. :eek:

Bobololo, if this is not a secret, could you clarify this?

[Teegedeck]

XviD 1.1 doesn't have a quality loss with b-frames at b-frame-quant = (p-frame-quant x 1.5) + 1 either... B-frames shouldn't necessarily trigger a quality-loss when compressed stronger than p-frames; they're supposed to be compressed stronger.

[Tommy Carrot]

Originally posted by Teegedeck
XviD XviD 1.1 doesn't have a quality loss with b-frames at b-frame-quant = (p-frame-quant x 1.5) + 1 either... B-frames shouldn't necessarily trigger a quality-loss when compressed stronger than p-frames; they're supposed to be compressed stronger.

Perhaps not visible loss, but the b-frames still have higher quantizers. With the reference h.264 encoder, i could get significantly smaller filesizes with equivalent b-frame quantizers (xvid cannot do this), and i'm curious if this is the case here too.

[Lobuz]

Maybe without postprocessing in XviD it's not so noticable but with sharp elements with postprocessing it's still visibly chenging from sharp P frame to blured Bframe etc. Or maybe I will have to make a better test with that XviD 1.1

Regards
Lobuz

[Teegedeck]

I think this would move the thread off-topic. Please come over to the XviD forum if you want to discuss this thoroughly.

[bobololo]

Originally posted by Teegedeck Maybe the overall a bit disappointing results are due to the material. Has anybody else made comparisons on the effect of the number of reference frames?

I hope to have time in order to test whether reference frames perhaps influence quality in a positive way, soon.

Your observations match with our early findings that showed that only very special clips benefit from multi-reference. For instance the foreman_cif sequence gets an huge coding efficiency improvement when we have several references. In most of the cases, it helps very slightly and requires much more cpu power. That's why in our default configuration, we recommand only 1 reference frame.

Originally posted by Teegedeck
And as I'm at it: Has anybody encoded clips with and without the -psy switch and compared the results? I could only see that -psy blurred the picture a bit but unfortunately it didn't seem to do anything else but that.

You're right the psychovisual effects are relatively weak in our current implementation and we still need to work a lot in this field. This could explain why there is no much difference with or wihtout. Basically, with -psy 1 fade transition should be better encoded than without. Btw note that when evaluating psy enhancements (which are subjective), it's very important to look at the clip normally and not doing frame by frame comparison.

Originally posted by Teegedeck
@bobololo & babayaga: Can you tell me how -adaptdeblock calculates? I'd like to know what deblocking-strength the -adaptdeblock settings I've used actually triggered.

The final strength offset used for the deblocking is computed as follow : strength = base + delta[Qp]. Where <base> is the value passed to the -deblock argument, <Qp> the quantizer and delta[] an array defined as following :


-6, -6, -6, -6, -6, -6, -6, -6, -6, -6,
-6, -6, -6, -6, -6, -6, -6, -6, -6, -6,
-6, -6, -6, -6, -6, -5, -5, -4, -4, -3,
-3, -3, -2, -2, -2, -1, -1, -1, -1, -1,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
1, 1


Those values were choosen without any tuning, and we wanted to adjust them depending on your feedback :)

Regarding the deblocking, please check at my next post.

[bobololo]

Originally posted by Tommy Carrot
I can confirm that the b-frames are working flawlessly, the pans are as smooth as they should. :)

Btw, i wondered which method does this codec use for b-frames: overquantizing, like in the mpeg1-2-4 codecs, or using the same quantizers as in the p-frames? Seems to me the latter, because i cannot find any quality-level difference between the continous frames, which would be sign of overquantized b-frames. In this case this codec can get compression gain with b-frames without any quality loss. :eek:

Bobololo, if this is not a secret, could you clarify this?

Ok without going in-depth, we quantized bframes more than other frames :)

[bobololo]

Now that the encoder has achieved a fairly stable state, we can at last discuss about quality tuning :) And I'd like to first have your feedback about the famous inloop deblocking filter and especially when the adaptive deblocking is enabled.

As it was already said, the adaptive deblocking is based on the quantizer used to encode a frame. The very simple idea behind this is to say that when the frame is highly compressed, it has more chance to show blocking, so I need to increase the delobking strength to hide the artefacts. The same applies to the opposite case.

Considering this, bframes have a higher deblocking strength. Do you feel that when watching at the encode ? Do you have the impression that there is a smoothness variation between different frame type or is it invisible for you ? The reason for this query is to know if we need to have separate strength for different frame type.

Secondly I'd like your opinion about this adapative scheme. Have you ever meet cases where the picture is higly quantized but you prefer that only a light deblocking to be applied to it ? In such case we should take into account other parameters than quantizer to compute the deblocking strength.

Last but not least, I was thinking about providing the users with the abilities to set "custom deblocking strength matrix" ;). This means that the users can choose themselves the strength to apply function of the picture quantizer. Do you think this can be useful ?

[Bulletproof]

It might be a good idea for the adaptive deblocker to test to see how dark or how bright a frame is, very dark scenes tend to handle deblocking badly and bleed/blur away into the dark background. Or maybe an option to specify when the deblocker kicks in, like an initial QP setting to set, for example I want to encode something and only want the deblocker to work if a quantizer of 32 or more is being used, and if above 32 the adaptive engine will determine how much to deblock and what deblocking cap i specified. Different deblocking settings for I,P,B frames, like -2 for I frames, -1 for P frames, 1 for B frames, this may make quality unstable however, I'm just bouncing thoughts which you may be able to improve on.

[plonk420]

Originally posted by 708145
Maybe someone has a high quality source of a music video? Should be easy to make use of several ref. frames with fast cuts.

bis besser,
Tobias

i've been working on this very thing... and it's murdering h264 ;) .. it's pretty unique for a musicvid, tho.

bobololo, i'm uploading it to the ftp now or pretty soon. be careful posting this clip; it appears to be released on an RIAA-controlled label, so posting the entire clip might not be the greatest idea (i could care less if you post the whole thing or not .. it's a freakin' sweet musicvideo and has some codec-wrecking sections). i think HydrogenAudio has found 30 seconds to be safe for any music. you could probably get away with as many 30 second clips as you want to, but i'd suggest 0:53 to 1:24 and 2:55 to 3:25

http://www.magnetbox.com/riaa/search.asp?searchtype=AsinSearch&keyword=B0000641QR

[yaz]

vow vow vow ... what's on ? just gone for some holiday and coming back for this ... how could i miss it ?
having read all threads of 'ateme' i found it quite exiting. is it possible to join in to this testing somehow ? i know i know, deadline's over but maybe ... (pls pls:-)
thx
y

[Sagittaire]

@ African Brothers ... ;-)

I have always problem with block flicking in low motion ... sniff
-qual extra -rcmode 2pass -br 1024000 -psy 2 -deblock 4 -adaptdeblock -ref 5 -priority idle

Perhabs deblock transition bframe-pframe problem ... ???

For this particular problem a capture is useless... but I can post samples if you want ... ???

[bobololo]

Originally posted by Sagittaire
@ African Brothers ... ;-)

I have always problem with block flicking in low motion ... sniff
-qual extra -rcmode 2pass -br 1024000 -psy 2 -deblock 4 -adaptdeblock -ref 5 -priority idle

Perhabs deblock transition bframe-pframe problem ... ???

For this particular problem a capture is useless... but I can post samples if you want ... ???

We've worked on the block flicking aspect and we've obtained fairly good improvement in this field. The encodes are more stable now. This will be included in the next beta !

[Teegedeck]

Originally posted by bobololo
Last but not least, I was thinking about providing the users with the abilities to set "custom deblocking strength matrix" ;). This means that the users can choose themselves the strength to apply function of the picture quantizer. Do you think this can be useful ?
Very useful indeed! That way I would feel way more comfortable, knowing that I can assign just as much deblocking as necessary/justifiable to a quantizer. Also, I'm lousy at maths and couldn't derive anything from the formula you've posted. :o

[Selur]

did some extrem clip watching the last hours and for psychvisuals I can say:

I prefer psy 1&2 over psy 0 and I can't see a difference between psy1 and psy2 unless I do a fram-to-frame comparison. So for me psy 1 is the way to go. ;)

I also prefer maxb 1 oder maxb 2&3, since they seem sometimes to be too smooth. :)

Cu Selur

[superdump]

Originally posted by bobololo
The final strength offset used for the deblocking is computed as follow : strength = base + delta[Qp]. Where <base> is the value passed to the -deblock argument, <Qp> the quantizer and delta[] an array defined as following :


-6, -6, -6, -6, -6, -6, -6, -6, -6, -6,
-6, -6, -6, -6, -6, -6, -6, -6, -6, -6,
-6, -6, -6, -6, -6, -5, -5, -4, -4, -3,
-3, -3, -2, -2, -2, -1, -1, -1, -1, -1,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
1, 1


Those values were choosen without any tuning, and we wanted to adjust them depending on your feedback :)

Originally posted by Teegedeck
Very useful indeed! That way I would feel way more comfortable, knowing that I can assign just as much deblocking as necessary/justifiable to a quantizer. Also, I'm lousy at maths and couldn't derive anything from the formula you've posted. :o

Right, say you use -deblock -4 -adaptdeblock, -4 is the base. delta[Qp] is a delta value taken from that list bobololo posted which is related to the quantiser used for the frame. The list starts from Qp 0 and should go to Qp 51. So to decide the frame deblocking strength the delta[Qp] is referenced from that quantiser-delta list and then individual frame deblocking strength = base + delta[Qp].

Originally posted by Teegedeck
Maybe it would be good to change its adaptive behaviour a bit in the long run. For me, the same adaptive deblocking strength that was too strong for quant 20 was too weak for quant 23. I'll hopefully have the time today to do one or two more hours of testing.

ATM strengths like these seemed to preserve a maximum of detail while providing enough smoothing:

qp 19: none
qp 20: -3
qp 21: -2
qp 22: 0
qp 23: 3

Your deblocking as such is not adaptive as all frame in your sequences have the same quantiser. The delta[Qp]s for your suggested quants are ALL -6. As such your suggestions for the level of deblocking imply strengths of:

Qp 19 : none (I imagine you mean -clref deblocking was used)
Qp 20 : strength = -3 + (-6) = -9 = off? (or does it just clip to -6?)
Qp 21 : strength = -2 + (-6) = -8 = off?
Qp 22 : strength = 0 + (-6) = -6
Qp 23 : strength = 3 + (-6) = -3

So, if when the strength is below the defined lower bound of -6 the deblocking is disabled, deblocking only kicked in at Qp 22. However I obviously haven't considered b-frames and their overquantisation. As such it would probably kick in earlier and this would be the effect that you'd notice. I hope that helped. :) Sorry if I was at all patronising, I just intended to be thorough.

EDIT: I have been informed by bobololo that the strengths of -9 and -8 would be clipped to -6.

[Sagittaire]

@ Superdump

re-good job ... :D

@ Bobololo

with adaptdeblock
strength = base + delta[Qp]

Without adaptdeblock
strength = base ? ... if correct it's possible to make our personal matrix ... ;-)

You want that we done our personal preference for the matrix adaptdeblocking?


Considering this, bframes have a higher deblocking strength. Do you feel that when watching at the encode ? Do you have the impression that there is a smoothness variation between different frame type or is it invisible for you ? The reason for this query is to know if we need to have separate strength for different frame type.


perhaps that two custom different matrix for the I,P frame and the bframes could be a good idea

strength = base + delta-p[Qp]
strength = base + delta-b[Qb]

[CruNcher]

0-24 = -6
25-26 = -5
27-28 = -4
29-31 = -3
32-34 = -2
35-39 = -1
40-51 = 1

for all who can't read it it's simple as that :)

[plonk420]

i've been playing with
-best
-extra
-extra -psy1
-extra -psy2
i also start at a moderate bitrate and increase/decrease by 200kbps (or start at a low bitrate and increase it by either 80 or 200kbps) for each trial but i'm getting a bit bored with those...

what combination of settings are you guys playing around with and comparing? i'd like to set up a batch and let it chug away as i'd rather play with complete clips (2pass vbr because the encoder can distribute the bitrate better). i don't really understand the deblocking modes or quantizer (haven't played with them); could someone give me a set of switches that might be good with whatever bitrate would go well to test them with (high, med, low bitrates considering the clip)?

[bobololo]

A quick post to provide some fresh info :)

- We've squeezed out a little bit more coding efficiency by tuning encoder internals. As a result our latest encodes showed even more sharpness than before.

- We'll be at the IBC Show at Amsterdam for the week end. So if you have the chance to be there, don't hesitate to visit us. We'll be present on several booths including Ahead, AVC Alliance, Texas Instrument, etc...
We should demonstrate cool stuff there ;)

- The next beta is expected next week with all our latest improvements.

-- bobololo.

[Sagittaire]

Last but not least, I was thinking about providing the users with the abilities to set "custom deblocking strength matrix" . This means that the users can choose themselves the strength to apply function of the picture quantizer. Do you think this can be useful ?


Very good idea ... ;-)

and if possible integer PSNR metric command with name in output ... for exemple -PSNR "H264-500"

[RBF]

I made comparison on very difficult-coding material.
Bitrate - 700kbit/s,
At XviD almost all frames are compressed with quantizer 31
At VP6 many frames are compressed with quantizer above 31
At Ateme the majority of the frames are compressed with quantizers from 30 up to 40

Core encoder version 1.0.1.19

Input file : night.avs
Output file : night700.mp4
Resolution : 688x544 @ 25.00 fps
Length : 3002 Frames
Process Priority : Normal
Rate Control : 2pass
Target Bit Rate : 700 kb/s
Quality : Extra
Init Quantiser : 24 [0 - 51]
Max Consecutive BFrames : 3
Deblocking Strength : -2
Num Reference : 1
Psychovisual : 0
Cartoon mode : Off
Features : ipred ppred bpred cabac deblock hpel qpel part

-- Start processing pass 1 / 2
* 100.00% completed
* 3002 frames processed @ 5.08 fps
-- Start processing pass 2 / 2
* 03001: encoding @ 5.81 fps - bitrate 917.03 kb/s - 100.00% completed
* 3002 frames encoded @ 5.19 fps - average bitrate 700.98 kb/s

Encoding complete

Xvid with the switched - off postprocessing - http://rbf.nm.ru/xvid_700.jpg
Xvid with maximum postprocessing - http://rbf.nm.ru/xvid_700_max.jpg
VP6 with the switched - off postprocessing - http://rbf.nm.ru/VP6_700.jpg
VP6 with automatic postprocessing - http://rbf.nm.ru/VP6_700_auto.jpg
Ateme - http://rbf.nm.ru/nero_700.jpg>
The original - http://rbf.nm.ru/original_700.jpg

[JohnV]

This may be a bit offtopic for this thread, but the other thread is very long and this is a quality picture anyway. :D

Here's a part of the Nero Digital team on friday 10th 2004 at IBC conference in Amsterdam:
http://www.hydrogenaudio.org/stuff/NeroDigitalTeam2.jpg

From left to right:
Tchi Southivong (bobololo) : Ateme
Xavier Castellan : Ateme
Pierre Larbier (babayaga) : Ateme
Ivan Dimkovic : Ahead
Cyril Fombonne :Ateme
Menno Bakker : Ahead
Jim Corbett : Ahead
Patrick Peeters : Ahead

[SeeMoreDigital]

Nice one JohnV :D

Hey, could you amend your link so their "ugly mugs" are visible 24/7?

It's nice to link names with faces...

Does anybody have any photos like this of other forum members? Maybe we should start a new thread?

I wonder if we can get a photo of Doom9, from "over the rainbow"?


Cheers

[aketon]

Originally posted by RBF
I made comparison on very difficult-coding material.
Bitrate - 700kbit/s,
At XviD almost all frames are compressed with quantizer 31
At VP6 many frames are compressed with quantizer above 31
At Ateme the majority of the frames are compressed with quantizers from 30 up to 40

Core encoder version 1.0.1.19

Input file : night.avs
Output file : night700.mp4
Resolution : 688x544 @ 25.00 fps
Length : 3002 Frames
Process Priority : Normal
Rate Control : 2pass
Target Bit Rate : 700 kb/s
Quality : Extra
Init Quantiser : 24 [0 - 51]
Max Consecutive BFrames : 3
Deblocking Strength : -2
Num Reference : 1
Psychovisual : 0
Cartoon mode : Off
Features : ipred ppred bpred cabac deblock hpel qpel part

-- Start processing pass 1 / 2
* 100.00% completed
* 3002 frames processed @ 5.08 fps
-- Start processing pass 2 / 2
* 03001: encoding @ 5.81 fps - bitrate 917.03 kb/s - 100.00% completed
* 3002 frames encoded @ 5.19 fps - average bitrate 700.98 kb/s

Encoding complete

Xvid with the switched - off postprocessing - http://rbf.nm.ru/xvid_700.jpg
Xvid with maximum postprocessing - http://rbf.nm.ru/xvid_700_max.jpg
VP6 with the switched - off postprocessing - http://rbf.nm.ru/VP6_700.jpg
VP6 with automatic postprocessing - http://rbf.nm.ru/VP6_700_auto.jpg
Ateme - http://rbf.nm.ru/nero_700.jpg>
The original - http://rbf.nm.ru/original_700.jpg


VP6 is much better than the others!!! It keep more details!! I believe that ateme has a very good codec too!

[JohnV]

He has cartoon mode off which would take chroma values into account in decision making, doesn't use adaptive deblocking and maybe a bit higher offset for deblocking could be used. I'd try using these things and psy 1 for better quality.

Accoding to bobololo Beta4 will have again higher efficiency so it will be interesting. :)

[RBF]

Originally posted by aketon
VP6 is much better than the others!!!

I disagree! You have not mixed a picture? Ateme keep more details than VP6.