DivX Pro “Kauehi” version [Archive]

Gej

2nd July 2003, 18:16

http://labs.divx.com/archives/000010.html

What’s New:
- Decoder work:
* Huge Speed improvements
. Decoding is from 10% to 30% faster on all platforms range
* Automatic Post Processing
. Post Processing adapt itself to the highest possible level attainable in current
decoding conditions, without slowdown or dropping frames.
* Experimental Post Processing Decisions
. This is an experimental feature that allow user to manually change the Post
Processing thresholds and strength.
- Encoder:
* Various Tahanea bug fixes
. Random skipped blocks in Slow/Very Slow mode
. Lower Q=31 frequency in 2nd pass
. Various cosmetic fixes
* Quality improvements with B frames
. Better B frames Motion Estimation
. Change the B frame quantizer modulation
* New Psychovisual Mode
. Based on Human Visual System
. Cortex Texture Masking

A little issue has been found with the installer it doesn’t set the Decoder Deblocking default threshold values

Default: Gen=1 Hor=20 Ver=40
LighterPP: Gen=0 Hor=30 Ver=50

kenobi51

4th July 2003, 04:32

Well, i'm loading it for test.
In fact, i want to ask you one thing :
why format of the logfiles created during first pass
has changed with 5.0.3 version of the codec ?
It's very annoying because i wrote a tool (look at http://membres.lycos.fr/kenobi51/) to ehance the keyframe detection, it's based on IA algorythms and bring adaptative BFrame support (trough a similar algo)
i didn't understand the new format of the log so i didn't successed adapting my soft.
All the more so as in the informations given by the old format, there is the size obtained during first pass which is important to adjust quality during second pass.

Is there any mean to retrieve the old format log (perhaps an option) ??

Excuse me for my bad english but i'm french so.

Morbo

6th July 2003, 04:16

Any Implementation of "full search",is a "Huge Hit",with an XP CPU.

Very promising results so far though....lowbit rate improved greatly at higher res.

cheers

Insolit

6th July 2003, 07:13

With this version I got much worse decoding quality than with the 5.0.5
I used the default settings changed by the registry key that could be downloaded from the divx labs.
I'vent tried encoding with this codec though.

Morbo

6th July 2003, 17:59

The speed selection now is the same as the VHQ mode in XVID..NOrmal is best all around,but very slow is recommended for archives.

Im also not fond of defualt decoder settings(why is de-ring and block on by default?!).

Radeon support is looking better(gotta have a 9700 to enjoy it on crappy encodes(not recomended fro better ones)).

Ive still got shimmering and wacky problems with Qpel at high bit rates.But overall a quality improvement in the encoding section.Considering the decoder issues a nit-pick,but one other nastyness Id thought Id mention.

Using the fastest selection on the performance /Quality produces flashy artifacts at lower bit rates.....

You can get away with faster settings for higher rates,but lower rates do produce some weird effects.

Cheers......

Worthy release gej! even if it is a beta:D

SeeMoreDigital

7th July 2003, 10:37

I am generating some low bitrate tests now!

However it's vital that the 'Decoder Default Setting Reg File' is installed.
If you don't you will not get the best out of this version of the DivX codec. The reg file is available here: -

http://labs.divx.com/archives/000011.html

There's also some useful info listed on this page too.

SeeMoreDigital

8th July 2003, 13:53

I've just completed my first series of tests and I have to say that this version of the DivX Pro codec can offer better quality looking encodes.

I very much like the fact that there is greater scope for adjustment under the 'performance/quality' settings tab.

Using this version of the DivX Pro codec, I have found that on my PC, the 'Fastest' setting compares to 'Slow' with the original DivX Pro codec. And the 'Standard' setting compares to 'Slowest' with the original DivX Pro codec.

Some of you, who are familiar with my posts will no that I am quite fond with creating 'low bitrate' encodes. And this is where 'Kauehi' excells.

It quicly became apparent that when encoding using the 'standard' setting both foreground and more importantly, background images are more detailed.

The 'Slow' setting improved detail further. And helped stabilize the 'vertical background shakes' that can often be seen with low bitrates DivX encodes!

I have not fully tried out the 'Very Slow' setting yet. But if the detail improves on this scale it's fair to assume that 'Kauehi' may even surpass WMV9 VCM encodes at the same low bitrate.

On the downside is that 'Very Slow' encoding is vveerryy ssllooww. Slower even that WMV9, at full bar. And that's slow!

temporance

8th July 2003, 14:58

Have you tried the new psychovisual mode yet? I can't see any difference (maybe that's a good thing!)

Morbo

8th July 2003, 15:21

I avoid those settings....

I will second the low-rate encode quality....much better!

With very slow enabled,Im amazed at how far down I can go @ DVD resolutions now....

About 750k and 640x480 Animes looks awsome detail wise now.

Koepi and Umanimac now have direct competition:D

Cheers!

SeeMoreDigital

8th July 2003, 17:48

temporance,

Yes, I generated, from the same source, a couple of 5min 720x576 'video only' encodes at 640kbps. One using the default (light) psychovisual mode. And the other using full strength.

There is a difference but I'll be damned if I can tell you it's for the better!

Maybe encoding at even lower bitrates would reveal some clear differences. I'll have to give it a go!

What is a bit of surprise though, is that when you use the 'very slow' setting, even the first pass is - very slow. Much, much slower than the first pass of RealMedia Video9 and WinMedia Video9!

I would hope that this will benifit the second pass greatly.

My plan is to encode my 'test movie' StarWars2 (PAL) 137mins using this 'very slow' mode. But at nearly 20hours for the first pass and god knows how long for the second pass. It begs the question... will anybody ever use this setting for full movie encodes?

Cheers

Gej

8th July 2003, 20:18

Hi,

I just add a post relating to the new psychovisual mode at DivX Labs: http://labs.divx.com/archives/000012.html

A good tweak: In order to use all the features giving quality boost and still achieve acceptable encoding time, here is what I do:
Using multi pass mode
1st pass, B frames only, no psy, Standard
2nd pass, B frames only, new psy, Standard, updating the log files during the 2nd pass
3rd pass, B frames only, new psy, Very Slow

The encoded file at the end of the 3rd pass will benefit form the rate distortion algorithm as well as from the New Psy and the optimized B frames. It’ll have been trough the Rate Control algorithm two times allowing taking care of all potential sudden quality drops.

Oh, and forget the "Full Search" this is only for reference purpose, the encoding time is ridiculiously slow and the gain is debatable.

LordIntruder

8th July 2003, 22:58

Gej: thanks for the small article that enlightens us on this new psy option and what results we can get.

However your suggestion on how to increase speed encodings lets me fuzzy. I thought we should never change the parameters on next Nth passes to not get weirds results as a lower quality or targeted file size oversized/undersized.

Is this solution gets the same result as if all passes would have been made like the 3rd one (B frames only, new psy, Very Slow) or are we suppose to see a noticeable difference in quality?

Thanks for info. :)

SeeMoreDigital

9th July 2003, 00:25

Wow Gej, thanks for your post.

However, I feel a bit confused too, as I too (like LordIntruder) understood that you should not fiddle with the encoders settings between passes!

I am happy you mentioned something about the 'Full Search' setting. As I was going to contact DivX Labs to find out what this was supposed to be used for!

I will certainly give your setting a try, using my 6min test file.

Many thanks, Cheers.

Owen

9th July 2003, 03:43

Has anyone used this new biuld to do real time capture.
I get a "divide by zero" error when I try to capture.
It works for transcoding in Vdub.

Regards,

Owen

dTb

9th July 2003, 08:29

Originally posted by SeeMoreDigital
Using this version of the DivX Pro codec, I have found that on my PC, the 'Fastest' setting compares to 'Slow' with the original DivX Pro codec. And the 'Standard' setting compares to 'Slowest' with the original DivX Pro codec.

I'm not trying to be a smart arse but it basically says as much on the divx labs page. Read the stuff on Tahanea too as afaik it's largely the same as Kauehi, at least when it comes to most of the settings.

If only I had some more time for testing this baby out:(

SeeMoreDigital

9th July 2003, 14:07

Well thanks for your input dTb!

One of the purposes of having a forum such as this, is to share your findings with others.

Although in a perfect world it would be nice to believe everything that was published, via its creator. I don't - Well not all the time anyway!

Anyway, I have now encoded a short 'video only' 6min 17sec (9431Frames) Chapter from a PAL DVD using the 'very slow' setting and with everything else at default.

The bitrate selected was 627kbps. The output of the 'frame size' is the same as the input at 720w x 576h pixels.

Both the first pass and the second pass took virtually the same time to do. At around 58mins!

Now here's the thing. The encoded image is, without doubt, better looking than previous versions of DivX. However, in my opinion it still does not match the quality of WMV9 (VCM).

To be fair I did not use any of the psychovisual modes. So with this in mind I will now move onto encoding the same file again but this time following Gej's Psy recommendations.

I will post my finding in a few hours.

SeeMoreDigital

9th July 2003, 20:27

Well it's done!

I followed Gej's instuctions to the letter and I am happy to report that the quality of the output image is better than using my previous settings.

Using my 'video only' 6min 17sec test file I can confirm the following: -

1st pass: 00hrs 06min 11sec
2nd pass: 00hrs 20min 05sec
3rd pass: 01hrs 08min 20sec

However, unfortunately the encode still does not look as good as WMV9 (VCM).

dTb

10th July 2003, 02:55

I hear what your saying SeeMoreDigital but I think Gej is probably the best source of info were gonna get. ;)

Anyway it's good to hear some of your results, hopefully I can do some testing later today but I might not be able to report much, I'm off on holidays in a few days, will be 25 on the 19th.

Btw, you may want to post in the divx labs forum if you want more feedback from the DARC guys.

SeeMoreDigital

10th July 2003, 08:44

Thanks dTb,

Maybe I should do as you say and post my findings to the divx labs forum.

However, it's been my experience that many people don't take encoding at low bitrates too seriously. Which I think is a shame.

All the emphasis seems to be with regard to creating a 'perfect copy of a source'. Which is ofcource useful and interesting but in my opinion not as challenging as getting the best out of low bit.

I also recognize that many people intend to keep their encoded creations for all time. Which is fine. But if they were able to take a look at some old (2 years or more) DivX encodes they would quickly see how far and how quickly the technology moves on and maybe would not get so attached to their creations.

Many users also seem to be fond of cropping and resizing (reducing the pixel count of) their encodes. Well, I on the other hand have been doing the opposite. I've been very surprised at just how good a low bitrate 1024w x 576h (PAL DVD only I'm afraid) encode can look!

So I guess I am very much in the minority!

Darrius "Junto" Thompson

10th July 2003, 16:45

Originally posted by SeeMoreDigital
Well it's done!

I followed Gej's instuctions to the letter and I am happy to report that the quality of the output image is better than using my previous settings.

Using my 'video only' 6min 17sec test file I can confirm the following: -

1st pass: 00hrs 06min 11sec
2nd pass: 00hrs 20min 05sec
3rd pass: 01hrs 08min 20sec

However, unfortunately the encode still does not look as good as WMV9 (VCM).

SeeMoreDigital,

I just encoded quite a few clips using WMV9 and the DivX Kaeuhi build and used a PNSR tool (psnr4avi) to measure the results. It's quite promising given we are implementing improvements that will keep decoding complexity low so hardware can support our changes well. Here are my results. We'll test some more and get them posted at the labs. Oh and I did test at a lower bitrate as I generally try to keep a very high level goal in my mind of what we might want to achieve. I really like and dislike using PNSR. It is quite good when you want to compare a large clip and can't look at every single frame as it can point you to problematic scenes. The problem is sometimes with PSNR, PSNR will say the quality is low in one scene when in fact subjectively it looks rather good and has allowed us to add data to more problematic areas. If your not familiar with PSNR the general rule is if you see a 0.5db improvement it is worth considering adding the feature that caused this improvement into a standards based codec. Also you can create an empty log file in c: called framelevelcontrol.txt and it will tell you motion complexity and PSNR, however I don't use this PSNR since I tested against WMV9 and needed to use an identical tool for testing purposes. However, I use it to plot motion complexity compared to PSNR for comparing all the clips as I easily see where we might be better or worse. With Kaeuhi, Hi-Motion scenes are really improved.

Clip: Inspector Gadget, 720x480, 500Kbps. ( I consider that low at that resolution)

Here is my summary:

DivX Kaeuhi vs DivX 5.0.5 Using 2 Pass Mode
--------------------------------------------

Kaeuhi Simple Profile over DivX 5.0.5: 1.32db Improvement with Peaks at 4.5db in high motion areas.

Kaeuhi Simple Profile Slow Mode vs DivX 5.0.5: 1.50db with similar peaks.

Kaeuhi Simple Profile Slow Mode + B-Frames:1.55db Improvement.

Kaeuhi Simple Profile Slow Mode + B-Frames + New Psychovisual: 1.57db. (remember psychovisual generally improves subjective quality but lowers PSNR, yet the new Psychovisual Improves both)

DivX Kaeuhi vs WMV9 (Both using 2 Pass Mode)
-----------------------------------------------
So far I've only tested the Highest Quality Mode (HQ-1 what I call it in my testing so I can remember)and the middle mode (HQ-2). The reason is I'm trying to compare quality at a similar performance. HQ-1 is slow and full HQ is really really slow.

WMV9 HQ-2 PSNR:41.95
DivX Kaeuhi Slow PSNR:41.94 (That is close!)
DivX Kaeuhi Slow + B frames:41.99 (Now we're ahead)
DivX Kaeuhi Slow + B + New Psychovisual Mode: 42.01

The new slow mode has yet to be optimized and is about 3fps slower but by release I'm assuming it will be the same if not faster. The results are really promising and can be validated using any PSNR tool today.

WMV9 HQ-1 PSNR:42.12 (about 3fps slower)

What I find most exciting is that the DARC team has created a codec based on MP4 that can beat WMV9 while we had many people saying WMV9 cannot be beat unless you stray from MP4 compliance. Many also said this type of improvement could not be made unless we were willing to allow decode time to increase substantially. Well with these first results I'm quite excited to say we've achieved both and still within MP4 while not adding CPU overhead to decode time. I encourage anyone to validate these results and test more clips as I'll be doing for the rest of the week. I have a ton of graphs we can post shortly.

Darrius "Junto" Thompson
Director R&D DivX

DevilsChild

10th July 2003, 16:59

I find the "slow" and "very slow" performance/quality settings to be practically unusable. Took me 13 hours to do the first pass of an 85 min. movie ("Sasquatch") near 1000kBit/s. The "standard" setting works very well, but I still don't see much change in speed (forgot to save the log file :scared: ).

The quality on the other hand is awesome. Most definitely an improvement.

Darrius "Junto" Thompson

10th July 2003, 17:32

Originally posted by DevilsChild
I find the "slow" and "very slow" performance/quality settings to be practically unusable. Took me 13 hours to do the first pass of an 85 min. movie ("Sasquatch") near 1000kBit/s. The "standard" setting works very well, but I still don't see much change in speed (forgot to save the log file :scared: ).

The quality on the other hand is awesome. Most definitely an improvement.

Happy you agree the quality has improved. Our approach has always been, implement, test\verify, and then optimize the hell out of it. Don't worry, optimization will be next on our roadmap :)

Darrius

SeeMoreDigital

10th July 2003, 19:17

Darrius,

Let me begin by saying I'm so very pleased that both your good self and Gej are so approachable via the forum.

To provide you with a little more information, let me confirm that all my 'short tests' have been made using the PAL version of StarWars 2 Chapter 41 as a source. Which, in my opinion is a particularly difficult image to encode.

As I mentioned in my previous posts for the purposes of my tests I only encode the video stream of the DVD. This makes it much easier for me to compare the finished file size of say, your codecs encode against the finished file sizes of other codecs encodes. Both the DivX and WMV9 VCM encodes are viewed on a PC via the DivX Player 2.1.

I'm sure you will agree that although it should a simple matter of setting the same bitrate for all codecs. When you do do this, you don't necessarily obtain the same output file size!

Fore instance, if I encode with your codec at say, 627kbps, I would have to set WMV9 VCM codec to just 615kbps to obtain the same file size!

As I hope you will already be aware, I have a great fondness for DivX. So nobody was more upset and surprised than me to find that my WMV9 tests revealed a superior looking image at low bitrates. So nobody would be happier than me if the 'Kauehi' version of the DivX codec will prove to surpass WMV9.

My only fear is that in order to obtain comparable quality with WMV9 the source stream will have to spend an agonizing amount of time in the DivX encoder.

If you wish I would be very happy to mail you some of my test encodes on CD-R(s). Just send me a PM.

murattttt

11th July 2003, 01:24

It is good to see the coders of the divx codec here and havachance to toss ideas. You guys are so admiringly humble and great.

I made some long testing of Kauehi and found the results very promising.
My avs file looks like this:

LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec3.dll")
mpeg2source("M:\THE_BODYGUARD_16X9_FF_PAL_D5\BodyGuard.d2v")
crop(2,0,716,576)
LanczosResize(576,320)

The source is very old, noisy and highly compressed in terms of mpeg2(4 GBs with ac3 5.1 - it is the original though)
I encoded it at 800 kbits, new psy,b frames, 2 multipasses at slow setting. The encoding time is really slow at my machine. I get around 3 f/ps im my PC. I can not think at encoding very slow setting, it doesn't look like my PC can finish it in 3 days. Nevertheless I would be willing to try very slow if someone said it had wonders in allover look of the video.
I compared it to divx 505 and Xvid (Koepi's latest). Xvid seems to be a more aggressive rival but Kauehi surpasses Xvid in terms of esp. high motion scenes and the general contrast of the video (Maybe because of the new decoding prmtrs).
However I lack the tools to objectively compare the codec in terms of encoding quality (i.e. numbers of macroblocks in the produced video).
I just can not think of not resizing the video because a full framed video will not play smooth in my or my friends' machines and find it useless unless I have a standalone divx player.
When it comes to WMV9 VCM codec, I think it is for streaming purposes only, much blurry and unconfigurable for my taste(and closed - I can not compare M$'s sound codecs to ogg let alone AAC). On the other hand I can not deny that when it comes too that low bitrates (below 600) it has no serious rivals (that includes real video).

The quality seems to become better and better in Divx and I begin to wonder what are the limits of MPEG4. By the way I do not see much of a future in MPEG4 because what puts the standards in digital video nowadays is Divx networks itself (Any standalone MPEG4 container player around?). Maybe divx will also branch out from the standards of MPEG4. These are the out of curiosity questions if you may forgive me.

dTb

11th July 2003, 06:49

I've done one test so far and I think the results are very pleasing.

AVS as follows

LoadPlugin("i:\PROGRA~2\GORDIA~1\mpeg2dec3.dll")
LoadPlugin("i:\PROGRA~2\GORDIA~1\undot.dll")
mpeg2source("G:\DVD Movies\Undercover Brother\Undercover Brother.d2v")
trim(0,15000)
crop(10,12,702,552)
undot()
LanczosResize(640,472)
undot()
Temporalsoften(2,3,3,mode=2,scenechange=6)

10mins of the start of Undercover Brother PAL Region 4
Used a bitrate of 1000kbps, Comp test of 38.42% in enc1.02 which I consider fairly low. For backups I aim for 60% or above.

Settings

DivX 5.0.5
b-frames, normal pve, sct 47%, Max Keyframe 250, -0.02 modulation
3 passes with same setting for each.

Kauehi
Method outlined by Gej in this thread for 3 passes.
1st- b-frames, no pve, standard search
2nd- b-frames, new pve, standard search
3rd- b-frames, new pve, very slow search
sct 47%, Max Keyframe 250 & -0.02 modulation throughout.

When comparing I use ffdshow with no post-processing

At first comparing these two files by watching them they seemed similar although with 5.0.5 there seemed to be more artifacting on edges with motion. In normal viewing conditions I would say the first 5 mins would generally be regarded as pretty much the same quality with Kauehi being just slightly better.
A bit after 5 mins in we have a fight scene with a fair bit of motion, this is where Kauehi shines. The 5.0.5 version suffers from a lot of macroblocking whereas in the Kauehi version this is greatly reduced, macroblocks are almost non existent being slightly evident in only a few of the very fast scene changes.

So it seems to me that in this test at least Kauehi has maintained a similar level of quality as 5.0.5 during low motion scenes but has improved a lot in the high motion scenes and also somewhat in frames with small areas of high motion.

I might try and host some frame comparisons so you can get an idea of what I mean.

A couple of things I've noticed that some might find interesting. The new psy mode results in quantisers that are whole numbers (eg. 4 as opposed to 4.25 you might see in the old version). I've also noticed bframe quantisers are closer to the pframe quantiser, so for a typical pbpbpb sequence it might be say 4,6,4,6 instead of 4,8,4,8 typically seen in 5.0.5.

I decided to compare the psnr of the two files but it's the first time I've used psnr4avi and I don't understand it's output. I'll give you the info and maybe someone can tell me what it means.

Command Line entry
psnr4avi.exe 505test02.avi kauehitest.avi

Result
Average PSNR (Y U V): 1.#J, 1.#J, 1.#J
Total Average PSNR: 1.#J

:confused: :confused:

Hopefully what I've done here will be of some use to someone:)

dTb

11th July 2003, 08:07

I decided I might as well host some screenshots

Comparison 1
505comparison01 (http://www.ozemail.com.au/~duaneb/505comparison01.jpg)
Kauehicomparison01 (http://www.ozemail.com.au/~duaneb/Kauehicomparison01.jpg)

Frame # 3404
5.0.5 - b-frame, q 8
Kauehi - b-frame, q 7

Look at the left arm.

Comparison 2
505 Comparison02 (http://www.ozemail.com.au/~duaneb/505comparison02.jpg)
Kauehi Comparison02 (http://www.ozemail.com.au/~duaneb/Kauehicomparison02.jpg)

Frame # 7840
5.0.5 - b-frame, q 8
Kauehi - p-frame, q 5

Comparison 3
505 Comparison03 (http://www.ozemail.com.au/~duaneb/505comparison03.jpg)
Kauehi Comparison03 (http://www.ozemail.com.au/~duaneb/Kauehicomparison03.jpg)

Frame # 7842
5.0.5 - b-frame, q 9
Kauehi - p-frame, q 5

You should be able to notice a lot less artifacting and macroblocks in the Kauehi versions. You might say it's not fair to compare frames when frame types are different and quants are also different but they are just to illustrate what I commented on above and if Kauehi has made a better decision on what frame type and quant to use I think that's valid. Also consider that while using lower quants in these high motion frames it's maintaining the quality of the low motion scenes.

I'm open to criticism just be gentle;)

SeeMoreDigital

11th July 2003, 10:01

Thanks dTb.

Your findings seem similar to mine

At first comparing these two files by watching them they seemed similar although with 5.0.5 there seemed to be more artifacting on edges with motion. In normal viewing conditions I would say the first 5 mins would generally be regarded as pretty much the same quality with Kauehi being just slightly better.

This is interesting. Maybe I should go for it this weekend and encode my entire test movie using Gej's settings!

Anyway, I thought you were off on holiday. Don't even think about packing your laptop. Just sit back, sip a few cold ones and enjoy yourself. Oh and happy 25th for next week!

Eh gods I remember my 25th. At one point some friends of mine got a bit out of control with some cans of whipped cream. It went all over the lounge ceiling, not to mention the floor and walls. But the next thing I remember seeing was one of them holding my vacuum cleaner 'upside down' trying to suck the stuff off the ceiling!

Needless to say he spent much of the next day painting over the black wheel marks!

Happy days!

Darrius "Junto" Thompson

11th July 2003, 16:26

Originally posted by murattttt
The quality seems to become better and better in Divx and I begin to wonder what are the limits of MPEG4. By the way I do not see much of a future in MPEG4 because what puts the standards in digital video nowadays is Divx networks itself (Any standalone MPEG4 container player around?). Maybe divx will also branch out from the standards of MPEG4. These are the out of curiosity questions if you may forgive me.

Yes, it seems everytime we think we've reached the limits of improving video quality within the boundaries of MP4 someone on the team comes up with a great idea that has the potential for making improvements that are worth our efforts. At the same time we know there are limits and we constantly think we are close and then someone on the team proves this thought wrong :) However, we know that other algorithms can be developed that would enable us to take an even larger leap than "might" be available within MP4 whether the idea\algorithm is proprietary or based off of newer standards and we are working on these everyday, so at no time are we confining ourselves just to MP4, we have to see what will give us the best jump in quality along with performance and keep in mind what the average configuration of a video PC user is and then also correlate that with hardware. To make a long story short, you can expect that we are looking at many new ideas not within MP4 as our goal is not to be MP4 but to be DivX...high quality\useability across software and hardware. Also I'm not sure I've ever mentioned this publically but our goal was never to be MP4 our goal was always quality\performance\scalability. We then looked in a toolbox to see what we thought were the best video tools to build on top of and it happened that the tools we believed would give us the best video quality also happened to be part of MP4 and this is how the MP4 route was chosen by DivX. It was not....let's be MP4. It was, "How do we create quality".

Hope that satisfies your curiosity :) Good things to come in the future and as we're doing now you'll see these ideas early in the lifecycle at divxlabs and of course we'll get feedback from everyone here as well, and now that we've done this once we've learned quite a bit from our mistakes which will hopefully make the experience and resulting technology much better this 2nd time around :)

Darrius "Junto" Thompson
Director R&D DivX

gigah72

11th July 2003, 16:43

is the videostream output of future versions of divx compatible with
actual players like the elta 8882 and compareable devices ?

what do i need to make a ultra hq encode of a dvd, to put 2 or 3 on one
dvd-rw ?

thanx, earthings and usually i read all faq's and dont't bore
you all,

gigah72

Darrius "Junto" Thompson

11th July 2003, 16:45

Originally posted by dTb

So it seems to me that in this test at least Kauehi has maintained a similar level of quality as 5.0.5 during low motion scenes but has improved a lot in the high motion scenes and also somewhat in frames with small areas of high motion.

I decided to compare the psnr of the two files but it's the first time I've used psnr4avi and I don't understand it's output. I'll give you the info and maybe someone can tell me what it means.

Command Line entry
psnr4avi.exe 505test02.avi kauehitest.avi

Result
Average PSNR (Y U V): 1.#J, 1.#J, 1.#J
Total Average PSNR: 1.#J

:confused: :confused:

Hopefully what I've done here will be of some use to someone:)

dTb,

This is definitely helpful and confirms the results we've been seeing. Thanks! High Motion scenes are definitely much better with all the clips I've tested as well. Given this you may want to move the bitrate modulation so it is a little stronger in low motion which may give the overall video an even better improvement in quality. I"m trying different thresholds today to see if there might be a better balance here.

>Command Line entry
>psnr4avi.exe 505test02.avi kauehitest.avi

That sshold work fine. The first file should be your original
source file and the 2nd file should be your encoded file. To test 505 vs Kaeuhi you would need to first run the PSNR of 505 vs. Source and then get the PSNR of Kaeuhi vs. Source and then look at the difference.

I would suggest piping out the results to a text pad since DOS will only keep up to 50 lines and you will need much more than this and if you want to graph the results in a tool like excel you can just copy paste or import of you open the data from a txt file. So I dump the data to a text file just using the '>' command and nameing a text file. I copied my last test below so you can see what I mean.

H:\>"C:\psnr4avi.exe" "C:\inspectorgadget_huffy.avi" c:\mytests\MS\Ins-1000-WM9-HQ-2.avi > c:\WM9-1000-2pass-HQ-2.txt

Also if you create an empty text file in c: called Framelevelcontrol.txt Kaeuhi will dump data there that tells you about motion complexity and PSNR. Don't use the PSNR in this file if you are comparing other files from older builds or comparing different codecs since you want to use the same PSNR tool whenever possible so there are little chances of error. However, I take the motion complexity from this log file as I can easily plot the high or low motion areas and see how PSNR fluctuates when compared to motion complexity which verifies high-motion is better.

Hopefully that helps.

I just tried 5.0.5 2Pass @1000kbps vs. Kaeuhi at same settings and the improvement is even much better now at 1000kbps vs my 500kbps test. It's really amazing.

@1000kbps my psnr increase with Kaeuhi is 3db better when taken as an average over the whole file, which is huge and my eyes can definitely confirm the PSNR result. And I didn't use any new tools or advanced tools. Just Simple Profile.. the Rate Control and probably a few other minor thigns are just that much better!

Also at 1000kbps the Kaeuhi Build PSNR using the standard mode and no advanced features beats WMV9-HQ-2(middle setting).

Kaeuhi PSNR:43.86
WMV9 PSNR:43.75

I'll try the WMV9 highest mode next. It also confirms your findings as the high motion areas are where Kaeuhi really stands out when I compare the 2 visually.

I'll post more here and at divxlabs.com when ready.

Darrius "Junto" Thompson
Director R&D DivX

Darrius "Junto" Thompson

11th July 2003, 16:51

Originally posted by Darrius "Junto" Thompson

>Command Line entry
>psnr4avi.exe 505test02.avi kauehitest.avi

That should work fine. The first file should be your original
source file and the 2nd file should be your encoded file. To test 505 vs Kaeuhi you would need to first run the PSNR of 505 vs. Source and then get the PSNR of Kaeuhi vs. Source and then look at the difference.

I would suggest piping out the results to a text pad since DOS will only keep up to 50 lines and you will need much more than this and if you want to graph the results in a tool like excel you can just copy paste or import of you open the data from a txt file. So I dump the data to a text file just using the '>' command and nameing a text file. I copied my last test below so you can see what I mean.

H:\>"C:\psnr4avi.exe" "C:\inspectorgadget_huffy.avi" c:\mytests\MS\Ins-1000-WM9-HQ-2.avi > c:\WM9-1000-2pass-HQ-2.txt

I forgot to mention that when comparing PSNR I then average the PSNR for Y U and V for every frame. The DivX Codec Framelevelcontrol.txt output only gives you 'Y' (Luma)

Darrius

Darrius "Junto" Thompson

11th July 2003, 18:51

Ok so now I just tested WMV9 at the fullest high quality mode
at 1000kbps. It seems DivX at the higher bitrates starts
to get better than WMV9. We're thinking that at lower bitrates
WMV9 uses some sort of inloop deblocking which likely helps
with PSNR and washing out the blocking but at higher bitrates
this is less useful and so the DivX PSNR really starts to get
even better when compared... interesting. I didn't even use
b-frames or the new psy mode (which by the way some more work
internally may have made this even better... coming soon.

Kauehi 2 Pass 1000kbps Slow Mode PSNR:44.03
WMV9 2 Pass 1000kbps Highest Quality Mode:43.82

I'd be interested in seeing if anyone else has tried the full mode on WMV9 and compared it to Kaeuhi Slow Mode and even Kaeuhi with Bframes and the new PSY Mode.

Darrius

LordIntruder

11th July 2003, 20:23

Each message got new crusty details. ;)

It's goog to see what can be done with small video tests but I wanted to see what I could get from a whole movie encode. So I've encoded Terminator 2 Director's Cut (02h35min long) in both 5.05 and Kauehi.

#########
LoadPlugin("mpeg2dec3.dll")
LoadPlugin("Convolution3dyv12.dll")
mpeg2source("C:\Terminator2.d2v")

trim(0,213611)
crop(2,70,714,436)
BicubicResize(672,288,0,0.5)
Convolution3d(preset="moviehq")
#########

I use the latest optimized tools available: YV12, Fast Recompress, AviSynth 2.52, etc... to get the fatest encoding times.

Resolution was 672 x 288 as indicated in the script above. Gknot gave me a "60%" at the compressibility test (4% used at Q=2) for that resolution and 2CD (700Mb).

Bitrate used was 1168 for the video. I encoded credits separately at 300K but I will only speak and give figures about the main movie, not credits.

-- 5.05:

Bi-Directional & GMC: on (Quarter Pixel: off)
Psy: normal
Slowest
Bitrate Modulation: 0.1

Each pass was made with those settings, around 3.5 hours/pass so 3 passes = 10,5 hours of encoding time.

-- Kauehi:

Bitrate Modulation: 0.1

I used Gej's tweak like this:

- 1st Pass: Bi-Directional & GMC: on, no psy, Standard
- 2nd pass, Bi-Directional & GMC: on, new psy, Standard
- 3rd pass, Bi-Directional & GMC: on, new psy, Slow

I didn't want to try "Slowest" with Kauehi because it's really and definitively too slow. I insist I first made my test with 5.05 then I installed Kauehi and make the test again.

1st pass took exactly 4 hours and 14 min. ~ 14fps
2nd pass took exactly 7 hours and 29 min. ~ 7,9fps
3rd pass took exactly 24 hours and 29 min ~ 2,4fps
Total encoding time = 36 hours and 12 min

Made on an Athlon XP1800+ with 1Gb Ram. The least we can say is that you have to be patient. ;)

So if I compare the encoding time, it has increased from 5.05 to Kauehi if we look at the first pass where we have similar settings and even psy off with Kauehi doesn't help to reduce it. Maybe I should make a third test to see if Kauehi Standard is a bit better than 5.05 slowest? I'll try if I get time.

If I compare both movies, yes I prefer Kauehi version but it depends where we look at. I should say that both are very closed and you have to take your magnifying glass to see differences but on some others areas with 5.05 I got scenes macroblocked (mainly high-motion) and now they are gone with Kauehi, incredible!!!! Of course if Kauehi would have not been available I would have use EKG and reallocate bits on those macroblocked scenes but now the codec Kauehi do its job better along the whole movie.

However even with a 2CD and a 1168 bitrate I find some scenes not as good as I would like they are with Kauehi. So EKG is the answer? Examine scenes, increase bit takes a long time and redo a full 24h pass start to becomes very exhausting. Maybe with Slowest mode in Kauehi I could have get a better result but it's impossible to use it right now for full movies encode.

I would say that people that are in a hurry could stay in 5.05 and use EKG and those who are ready to waiiiit could use Kauehi as the result is very nice but not perfect. Of course using EKG with Kauehi will bring a very very good result but better own a top-of-the-art 3.2Ghz. ;)

Of course Divx team will work to optimize the encoding time so that we could use EKG to correct imperfections in a near future without having passes that last 24h.

Cheers to all Divx team, Kauehi is a very good codec. :)

Darrius "Junto" Thompson

11th July 2003, 20:40

Originally posted by LordIntruder

So if I compare the encoding time, it has increased from 5.05 to Kauehi if we look at the first pass where we have similar settings and even psy off with Kauehi doesn't help to reduce it. Maybe I should make a third test to see if Kauehi Standard is a bit better than 5.05 slowest? I'll try if I get time.

Cheers to all Divx team, Kauehi is a very good codec. :)

LordIntruder, thank you very much for your feedback. Please do try just using Kauehi Standard vs 5.0.5 slowest, I think you'll be pleasantly surprised to see that the quality difference is also very noticeable here, especially in action scenes, while still keeping the encoding speeds fast. Maybe try a 2000-3000 frame clip with some action and you'll see what I mean. Even I have a hard time believing the nice jump in improvement when using Kauehi in Standard mode vs. 5.0.5 :)

LordIntruder

11th July 2003, 23:44

Oh thanks a lot for the hint Darrius. You confirme Kauehi Standard is a bit better than 5.05 slowest. That's a good thing to hear. So that the increased time is not wasted and the codec use it to enhance quality.

That sounds really good. :) It worths the really small extra time.

By the way I've just encoded the menu of the DVD and compared it with my old one in 5.05 done some days ago.

Bitrate was 1000K same settings that 5.05 but this time as it is a very small encode (less than 1000 frames) I used Kauehi at 'slowest' & new psy for all the 3 passes. The result is amazing!!! I had some macroblocks with 5.05 and now all of them are almost gone. With 5.05 you were able to notice them without difficulty, now you have to watch very carefully to notice some.

Also in the video there is a lot of red colour and with that thing codecs are pushed to the limits as we see pixelisation and macroblocks in those red areas. The same scene with Kauehi is great and only in a specific area I have been able to view some macroblock but smoothed and almost unoticeable. Kauehi handle red color like a charm.

The bad thing is that at slowest mode with Kauehi i'm around 1fps. So a pass would take around 48 hours at this speed to encode the 213611 frames of T2. You better not make a mistake and miss your targeted size!!! ;)

I'm really impressed!! Cheers again! :)

dTb

12th July 2003, 03:39

Originally posted by SeeMoreDigital
Anyway, I thought you were off on holiday. Don't even think about packing your laptop. Just sit back, sip a few cold ones and enjoy yourself. Oh and happy 25th for next week!

Eh gods I remember my 25th. At one point some friends of mine got a bit out of control with some cans of whipped cream. It went all over the lounge ceiling, not to mention the floor and walls. But the next thing I remember seeing was one of them holding my vacuum cleaner 'upside down' trying to suck the stuff off the ceiling!

Needless to say he spent much of the next day painting over the black wheel marks!

Happy days!

lol thanks, sounds a bit wild, hopefully I have just as good a time, I'll keep a close eye on any whipped cream cans ;)

I managed to find some time for a quick test but I'm flying out in a few hours, back to where I grew up, Alice Springs, in the centre of Australia about 500kms from Ayers Rock/Uluru.

Thanks for the kind words and good luck with any further testing.

dTb

12th July 2003, 03:46

Originally posted by Darrius "Junto" Thompson
>Command Line entry
>psnr4avi.exe 505test02.avi kauehitest.avi

That sshold work fine. The first file should be your original
source file and the 2nd file should be your encoded file. To test 505 vs Kaeuhi you would need to first run the PSNR of 505 vs. Source and then get the PSNR of Kaeuhi vs. Source and then look at the difference.

I would suggest piping out the results to a text pad since DOS will only keep up to 50 lines and you will need much more than this and if you want to graph the results in a tool like excel you can just copy paste or import of you open the data from a txt file. So I dump the data to a text file just using the '>' command and nameing a text file. I copied my last test below so you can see what I mean.

H:\>"C:\psnr4avi.exe" "C:\inspectorgadget_huffy.avi" c:\mytests\MS\Ins-1000-WM9-HQ-2.avi > c:\WM9-1000-2pass-HQ-2.txt

Doh, yeah I should have known, compare with the source not between the two test files. I wasn't sure how to compare an avs but I see that outputting to huffy is the go.
Thanks for the help and I guess if I'm lucky in a couple weeks time when I get back from holidays I might have a nice new version of divx to play with :)

Keep up the good work DARC

Owen

14th July 2003, 03:03

I have tried to use Kauehi in FlyDS for real time capture as I normally do with 5.05 and I get a "divide by zero" error.
What gives?
Is real time capture broken?
Vdub encoding works ok, but I don't do re encoding. Only 1 pass realtime.

Regards,

Owen

bond

14th July 2003, 12:27

Originally posted by dTb
I wasn't sure how to compare an avs but I see that outputting to huffy is the go.try it that way:
file="movie.avi"

clip1=AviSource(file).converttoyuy2().trim(1,0)
clip2=Import("C:\movie.avs").converttoyuy2()
compare(clip1,clip2,"YUV","psnr_"+file+".txt")

SeeMoreDigital

15th July 2003, 10:37

This is all really great info.

I too have done some more encoding tests using higher bitrates because I remembered reading somewhere on the forum, that at around 900-1000kbps the std DivX 5.0.5 codec begins to surpass WMV9 (VCM).

The DVD I decided to use was the PAL version of Toy Story 2 which just happens to be the only true 1.77:1 DVD we (the family) happen to have. And at 89mins would'nt take too long to encode.

Also another advantage of using a true 1.77:1 source is that the image occupies all 720 x 576 (414,720) pixels, so it seemed a good idea to encode an image that occupies every pixel.

Anyway the DivX '5.0.5' and DivX 'Kauehi' encodes where generated using a bitrate of 963kbps. The WMV9 VCM encode was generated using a bitrate of 948kbps. And I used an Mp3 audio bitrate of 128kbps for all three encodes.

I also decided to use, what I believe would be each codecs most common settings. Which are as follows: -

DivX '5.0.5': -
2pass Bitrate - 963kbps
Psy Enhancements - None
Pre Processing Source - None
Performance/Quality - Slowest
1st Pass - 02hrs 15min
2nd Pass - 02hrs 10min
Encoded File Size - 711,475KB

DivX 'Kauehi': -
2pass Bitrate - 963kbps
Psy Enhancements - None
Pre Processing Source - None
Performance/Quality - Standard
1st Pass - 01hrs 55min
2nd Pass - 01hrs 53min
Encoded File Size - 711,379KB

WMV9 VCM: -
2pass Bitrate - 945kbps
Decoder Complexity - Main
Pre Processing Source - N/A
Performance - 4 (out of 5) 'Default'
1st Pass - 02hrs 10min
2nd Pass - 05hrs 50min
Encoded File Size - 711,576KB

I am still happy to confirm that DivX 'Kauehi' generates a more detailed 'overall' looking image than DivX '5.0.5'. There are less noticable artifacts, the foreground images are sharper, the background images are less shaky and the colours more stable. So on this basis alone I would recommend that DivX 'Kauehi' be used.

I am also happy to confirm, that the rumours are true! DivX does begin to out perform WMV9 at the above bitrates. Now the eagle eyed of you would say. Sure, ofcource DivX out performed WMV9, you did'nt encode at the same bitrate for both - which is true. However, the important aspect here is to use the finished file size as the comparison and not the bitrate setting.

As different codecs generate different file sizes at the same bitrate!