DivX Kaukura Codec Beta

[Gej]

This is the last stop before reaching our final destination. The island we are going to explore today is Kaukura.

This beta version offer stability improvements, cosmetic fixes and speed optimization over Manihi. Give it a test run and provide us with your feedback.

http://www.divx.com/divx/windows/beta/



What's New:

* Psychovisual mode:
- Speed Optimization
- Psychovisual mode is now capable of operating on Chroma. This option is not enabled by default for performance reasons. In order to enable it you must create an empty file in C:\ called "DivxPvShapingChroma.txt" for "fast" mode and "DivxPvSimpleChroma.txt" for slow mode. Results show a very little gain (around 1-2%) in JND tests.

What's Fixed:

- A bug with different number of frames between passes with VirtualDub
- GUI improvements
- A bug with MV (reuse) and psychovisual mode used together
- Installer issues
- Crashes involving setting crop/resize values

[dTb]

Cool, I'll get stuck into some testing monday.

One question, will the old psychovisual mode be available in the final release? I primarily used it when encoding credits where I thought it helped a lot in improving the overall quality at the low bitrates I was using.
The way quantisers were used in the old mode seemed to work well with credits, I'll do some testing with the new pve to see how it goes.

[phrentec]

Hello Gej. Before reaching your final destination is there any chance you can let DivX operate in a color depth of 8-bit( or 256 colors)? It seems that DivX (and all other compression codecs I can think of) can't work in just 8bit of color and instead works in 24 and 32 bit. In fact I think DivX doesn't even work in 16bit from what I'm seeing here. Well in many cases video looks better dithered(ie 256 colors) instead of blocky when at lower bitrates.
Thanks.
Regards.

[Gej]

No way DivX or other codec works in 8bit (they actually not even work in RGB 16bit or 24bit, but in 12bit YUV)

Sorry, but if you want genuine 8bit suport you have to use cinepak !

[phrentec]

thanks for the clarification.

[nuked]

original 2 pass still produces files that are less than 1/3 of correct size.

[nuked]

For the record I'm getting about 4.5 frames per second on slow with normal preprocessing, bframes, no gmc, no qpel, no physc at 720x 304 resolution on a 2.0 ghz p4. Definitely faster than manihi but still too slow for me. I wont use slow mode unless it or my hardware gets to about twice that.

[SeeMoreDigital]

Blimey!

I've just carried out a short encoding speed test.

Source
Harry Potter 2 trailer @ 02m08s

01 test @ 570kbps
Settings Used - None
1st pass 'standard' - 02m10s
2nd pass 'slowest' - 00m56s

02 test @ 570kbps
Settings used - BiDi only
1st pass 'standard' - 03m07s
2nd pass 'slowest' - 12m03s

03 test @ 570kbps
Settings used - GMC only
1st pass 'standard' - 03m52s
2nd pass 'slowest' - 00m58s

So on this basis. If you don't use any settings at all. It will encode faster than a 'Borg cube at maximum warp'!

Images are not too bad either..... More detailed tests to follow!

EDIT...
Still not sure if the post-processing works properly.

[nuked]

Wow, the first test is indeed impressive. Bidi really slows it down by like a factor of 12!? Even if it does though... is it really worth ditching Bidi to get the improvements of slow mode?

[SeeMoreDigital]

Well, this is indeed strange.

I've tried twice now to generate an DivX Kaukura encode of StarWars2 with video at 647kbps and Mp3 audio at 64kbps (24,000Hz).

The 1st pass using 'standard' setting took 02h 38m 10s.
But the 2nd pass kept failing. Generating an 'Mpeg audio sync error' after 173,489KB had been encoded.

I tried using both the 'standard' and 'slowest' settings for the 2nd pass - but no joy as yet!

I will try again. However this time I will set the audio to 96kbps(48,000Hz).

It will be interesting to see what happens now!

[silver_cpu]

Most likely, the resampling will fix your problem. At 96kbps, the mp3 encoder resamples down to 32khz. I think the speed improvements are phenomenal, they really do make a big difference. If we could see similar improvements in b-frame compression, DivX would enter a whole new world of professional encoding. Thanks, DivX team! Keep up the good work :)

Edit: BTW, is it possible to make the first two passes using no GMC, no qPel, and no Bframes, and then include them in the last pass? Or do they need to be in each pass to work correctly?

[SeeMoreDigital]

Originally posted by silver_cpu
Most likely, the resampling will fix your problem. At 96kbps, the mp3 encoder resamples down to 32khz. I think the speed improvements are phenomenal, they really do make a big difference.
Yep that was my thought. Strange that Kauihi did'nt suffer from this problem. I don't know about Manihi, as I never tried it at 64kbps!

Originally posted by silver_cpu
Is it possible to make the first two passes using no GMC, no qPel, and no Bframes, and then include them in the last pass? Or do they need to be in each pass to work correctly?
Now I'm glad you've asked this because I was thinking the same. I was also thinking that if the Gej's (and DigitAl56K's) info is correct regarding the first pass settings. ie don't bother using 'slow' and 'slowest' modes. Why bother having them at all!

[Gej]

Now I'm glad you've asked this because I was thinking the same. I was also thinking that if the Gej's (and DigitAl56K's) info is correct regarding the first pass settings. ie don't bother using 'slow' and 'slowest' modes. Why bother having them at all!

The 1st pass generate a log file that record several data about the video to be encoded, 1st pass doesn't actually generate any watchable file. So using "standard" during the 1st will generate a log file and gather the required data (though theses will not be as accurate than if generated using "slow" or "slowest" but the difference is really really low thus can be neglected)
Then using this log file during the 2nd pass to actually encode using "slow" or "slowest" mode can be a huge time saver, and still using all the advantages of the advanced modes in term of visual quality.
WARNING: YOU CAN'T DO THAT IF YOU USE A MV FILE !!
If you plan to use a MV file during the Nth pass, you HAVE to use the same performance settings.

original 2 pass still produces files that are less than 1/3 of correct size.

Yes, actually something is broken on the original RC, It'll not be fixed as it'll be removed in the final release.


My favorite’s settings so far:

Common setting:

Home Theater profile: on
B frames: on
Psy: fast (no chroma hack)
NO MV file

1st pass
performance: Standard

2nd pass
performance: Slowest

(okay the 2nd pass is insanely long, but the quality is really great, the amount of blockyness is REALLY reduced with "slowest" and Psy "fast", and when playing on a 1st generation DivX enabled DVD player (they do not have any deblocking) even a 2 hour movie on 1cd do not look blocky at all during the higher action scenes)

[nuked]

"Yes, actually something is broken on the original RC, It'll not be fixed as it'll be removed in the final release."


:( I guess I'll smile again anyway though if these new speeds become available with b-frames, but I'm still stuck with 2 pass for now and original was better.

edit: I meant stuck with standard 2 pass.

[Gej]

I'm not sure original RC was better, honestly, the original has the fatal disadvantage not to take care of the VBV, that make it rather dangerous when it come to hardware compatibility.

If only difference between original and new, beside the VBV enforcement is the fact that original 2pass mode was slightly biased toward low motion (it allocate proportionally more bit to low motion part than Nth pass RC does when motion modulation is set on 0)
If you want to achieve the same effect you should use the bitrate modulation to value around 0.10-0.15.

[silver_cpu]

So if I wanted to use an MV file, then I'd need to make sure that all passes, including the last, were encoded at the same speed (slowest, or standard, or whathaveyou), and that qpel, gmc, and bframe settings would have to be the same for each pass? Or could I enable those after the log file was created?

[Gej]

No, no, no

Qpel, GMC and B frames must be consistent from passes to passes, MV reuse or not.

[silver_cpu]

ok, thanks for the info :)

Edit: But let's say I didn't want to wait for 3 "slowest" passes. Is "standard" really quite decent? Or do you get a substantially noticeable difference when you encode in "slowest" mode? I've got about 800kbps in my video stream.

[Gej]

I rather just do 2 passes and use "Slow" than 3 passes and use "standard"
check here a PSNR graph of the differences:

http://labs.divx.com/archives/000007.html

Slow and slower really does make a difference (as well as PSY "fast")

[SeeMoreDigital]

My StarWars 2 test encode is now finished. I did'nt use any Qpel, GMC and BiDi etc. Just used the read/write log and MV file.

Source
StarWars 2 02h 16m 39s (8199sec) PAL

Video @ 750kbps
Audio @ 96kbps
Settings Used - None
1st pass 'standard' - 02h 38m 01s
2nd pass 'slowest' - 01h 31m 12s
3rd pass 'slowest' - 01h 32m 06s

I have to say that I am very impressed with the results. The encoded image is much cleaner than DivXPro5.0.5. And as you can see I managed to do 3 passes in around the same time as I would have expected to do 2!

I appreciate that most users prefer to use BiDi (that's B-frames for you newbies). So not to be put off, I'm going to generate the encode again with Gej's favourite setting ie Home Theater profile: on / B frames: on / Psy: fast (no chroma hack)/ NO MV file / 1st pass: Standard / 2nd pass: Slowest.

Cheers

[Sumster]

Gej...

Would you suggest doing the following. Running in "STANDARD" mode for 4 passes USING the MV file, and then do the final pass in SLOWEST without the MV file?? That way, the final pass is not dependent on the MV from the previous 'standard' passes, but the standard passes will all be quick (and allow more info through the rate algorithm). Of course, ALL passes would have to be the same in regard to B-frames, Qpel, etc.

Thanks,

Sumster

[SeeMoreDigital]

OK boys and girls.

I've just carried out two more tests with Kaukura. Again without using any Qpel, GMC and BiDi etc. Just creating an read/write log and MV file.

This time I encoded Planet Of The Apes.

For my first test I decided to encode the 1st pass using 'standard' and the 2nd and 3rd passes in 'standard' also.

For my second test I decided to encode the 1st pass using 'standard' and the 2nd and 3rd passes in 'slowest' mode.

I was expecting the second test to take longer to do than the first test but I was very surprised when they took the same time to encode. The other strange thing is that they looked exactly the same. Which inturn also looked exactly the same as my 2pass Manihi encode - It was even the same file size!

Gej, somebody, anybody, please explain?

[Gej]

MV file stand for Motion Vector file, it's a file that contain all reference to all motion vectors for all blocks in all frames. If you generate this file during the 1st pass with performance “standard” when you enable MV reuse during the Nth passes, even if you select performance “slowest” it will use the data from the file, so “slowest” will be useless.

[SeeMoreDigital]

Originally posted by Gej
MV file stand for Motion Vector file, it's a file that contain all reference to all motion vectors for all blocks in all frames. If you generate this file during the 1st pass with performance “standard” when you enable MV reuse during the Nth passes, even if you select performance “slowest” it will use the data from the file, so “slowest” will be useless.
So, are you also saying that when you encode using 'MV' that it's pointless to use more than 2 passes?

And don't you guy's ever stop working?

[sapient]

From what gej is saying, I figure that the speed tests that digital56k did some days ago and got us so excited are actually invalid. If I remember corectly he did 4 passes in standard and a final one in slowest using the mv file. If I understood gej correctly, that would actually result in a fifth standard pass. My own tests show that kaukura is still very slow, with or without b-frames (second pass @ slowest w/ psy fast, w/o b-frames = 3xtime of 1st pass standard)

sapient

[Flame10]

Here is a strange issue.

Whenever I try to do a batch directory processing or running a batch .vcf file processing in Virtualdub, it crashes in divx.dll, now this crash also happens if a single file is opened and just after loading a Configuration file the encoding is started, however just by choosing the Audio codec manually, the encoding works w/o any crash. I'm using Mp3 codec and the setting is surely stored in the .VCF config file, as it remains autoselected when I try to manually configure the audio codec again. I was thinking that this may be an issue with the radium codec I'm using but this crash never happens with Divx 5.05 version or prior, so it seems that some sort of bug must have cropped in.

Gej/any other developer can you please check this out.

[Sumster]

Gej...

can you use MV re-use for the first 4 passes in standard, and then in the 5th "slowest" pass simply NOT use MV?? Then, at least, you are using MV to speed up the first four passes.

[SeeMoreDigital]

Originally posted by sapient
...My own tests show that kaukura is still very slow, with or without b-frames (second pass @ slowest w/ psy fast, w/o b-frames = 3xtime of 1st pass standard)
This is a little odd sapient. When I did some speed tests, without Bi-Di, GMC, Qpel etc I found Kaukura to be the fastest DivX codec ever. Are you sure you've deselected everything including the Profile, Psychovisual enhancements and disabled the feedback window?

If you missed them, my tests were as follows: -

Source
Harry Potter 2 trailer @ 02m08s

01 test @ 570kbps
Settings Used - None (except Write/Read Log/MV file)
1st pass 'standard' - 02m10s
2nd pass 'slowest' - 00m56s

02 test @ 570kbps
Settings used - BiDi & Write/Read Log/MV file
1st pass 'standard' - 03m07s
2nd pass 'slowest' - 12m03s

03 test @ 570kbps
Settings used - GMC & Write/Read Log/MV file
1st pass 'standard' - 03m52s
2nd pass 'slowest' - 00m58s

As you can see the second pass and subsequent passes are really quick!

[SeeMoreDigital]

Originally posted by Flame10
Here is a strange issue.

Whenever I try to do a batch directory processing or running a batch .vcf file processing in Virtualdub, it crashes in divx.dll, now this crash also happens if a single file is opened and just after loading a Configuration file the encoding is started, however just by choosing the Audio codec manually, the encoding works w/o any crash. I'm using Mp3 codec and the setting is surely stored in the .VCF config file, as it remains autoselected when I try to manually configure the audio codec again. I was thinking that this may be an issue with the radium codec I'm using but this crash never happens with Divx 5.05 version or prior, so it seems that some sort of bug must have cropped in.

Gej/any other developer can you please check this out.

There is something different about how both Manihi and Kaukura encode audio. I too have the Radium codec installed and experienced a problem. But mine was encoding audio at just 64kbps - which I have to admit not alot of people here would do!

[sapient]

@seemoredigital

You are using the MV file! This means, according to gej, that your second pass is actually done at standard, not slowest. And as I said I also used psy fast, but nothing else

[BRMSlash]

Well, it took me a few attempts to actually get this to install. As soon as the installer tried to access the registry, the mouse pointer sat there flashing between normal and the hour glass and progressed no further. I then tried installing it on my old computer, which installed just fine. So I returned back to my new computer and ran Ad-aware and it found Gator from my previous installation of divx5.05pro, which is fine. I then deleted all gator files that Ad-aware found. DivX Kaukura then installed fine. I also tried with Manahi and I had the same problem until I deleted Gator(the Gator folder was the critical thing I found), any of the old official releases had no problem installing with Gator though. Weird?

I can handle that slight annoyance, but now it is impossible for me to finish a 1-st multipass with Kaukura. My computer just locks up and I have to hard reboot. I am using GordianKnot 0.28.5.2 to create the avs. I then add ConvertToRGB32() to the last line (this reduces red blockyness greatly). I then queue up two passes using Gej's favourite settings in either Vdub or Vdubmod. It then locks my computer at a random spot in the first pass. I revert to Manahi and then all is fine, no more lock-ups. This is very annoying, especially when this is close to the "final destination". Maybe it's just my machine, but it's only a few weeks old (AthlonXP3000+)? Any ideas anyone? Any assistance would be greatly appreciated.

[SeeMoreDigital]

sapient,

Yes, I did use the MV file setting to generate my 2 an 3 pass test file encodes.

As I said before. Just have a go at generating some short test encodes without using any settings.

Then introduce various settings, one at a time. To see how it affects the speed of your encodes.

Cheers

[BRMSlash]

I take back what I said about Manahi working. My comp just froze around 85%. I think it must be the new bios I updated to. Damn you EPOX. I will investigate further, I had Manahi working last week.

[sapient]

@SeeMoreDigital

I will try what you suggest as soon as I get home.

Your results indicate that standard encoding with hinted mv results in a great speed increase. But slowest mode is still very slow. You can't speed it up using mv, or you 'll end up doing a standard pass instead. You would have to do the 1st pass @ slowest wich would not help things. To sum up, if I understood gej correctly, the options for slowest right now are the following 2:
-1st pass standard, second pass slowest, no mv
-1st pass slowest, second pass slowest with mv

(@Gej - Am I correct?)

Both options require at least one full length, unaccelerated slowest pass that is still pretty darn slow. Adding b-frames to the mix seems to make things even slower. I will test this more thoroughly later today and report back.

sapient

[Fr4nz]

Yeah bidirectional encoding is slow like hell now!!! :(

[nuked]

Hey Gej.. see the link below for something I think would be a really neat feature to have in a final divx. Wouldn't it be wonderful if we could all quit worrying about screwed up hybrid telecine jobs? I hope this doesn't bend any cross posting rules too badly, but I'm not sure where to plant this idea to see if there is any interest. Maybe I'm just naive and crazy though. I think this would be a feature that people just don't realize how much they'd like cause they don't have it yet. Of course some support would be needed in dvd2avi and avisynth or such but it has to start with codecs I think.

http://forum.doom9.org/showthread.php?s=&threadid=59294

by the way.. I've done minimal testing of the interlace support in kaukura (I assume it's the same as before) and I don't see it doing anything useful, in fact it increases file size of quality based encodes of both interlaced and telecined 30 fps content just a tiny bit over the progressive option. I still need to test on lower bit rates though to see if it improves visual quality. Some filters in the playback codec for swap, rate-doubling bob, and blend would be very nice too.

[SeeMoreDigital]

I found time today to carry out some more low-bitrate encode tests. However this time I decided not to create/use the MV log setting. The results are as follows: -

Source
Harry Potter 2 trailer @ 02m08s

04 test @ 570kbps
Settings Used - None, except Write/Read Log
1st pass 'standard' - 02m06s
2nd pass 'slowest' - 19m00s

05 test @ 570kbps
Settings used - BiDi only + Write/Read Log
1st pass 'standard' - 03m18s
2nd pass 'slowest' - 19m50s

06 test @ 570kbps
Settings used - GMC only + Write/Read Log
1st pass 'standard' - 02m50s
2nd pass 'slowest' - 20m21s

07 Gej test @ 570kbps
Settings used - Home Theater profile, Psy: fast + Write/Read Log, No MV file
1st pass 'standard' - 02m46s
2nd pass 'slowest' - 19m36s

This really does confirm what Fr4nz, sapient, etc and Gej have said about encoding with BiDi etc..... It's hellishly slow.

Even on my PC, a 2nd pass using the 'slowest' setting would take 10hours for ever 1hour of A/V. What a shame!

However, as I've said before, using the MV setting really does speed things up. And produces a very good looking image for viewing on a TV. When you begin to encode above 950kbps using MV alone, it's a totally different story...... As you obtain great looking encodes at fast encoding speeds!

I really like the idea of incorporating MV (motion vector) support. As this process makes far more sense to me than having to use a multitude of other settings!

But that's just my opinion!

[Gej]

When MV reuse is enabled, in the Nth pass it does not matter what performance is selected, In practice, MV reuse is disabled for B-frames, so the performance selected will be used to perform Motion Estimation for the B-frames.

[SeeMoreDigital]

Hi Gej

May I ask again please. Given that encoding with MV is very quick indeed, when no other settings are activated.

Is it pointless to use more than 2 passes?
Would using 'Psy: Fast' help?

Cheers

[silver_cpu]

Gej: are you saying that when we enable Bframes, the nth passes disregard the MV file? If so, one would wonder why, and if the MV file effectively replaces bframes, since it wouldn't need to estimate backwards...

Also, I'm very interested in what people are saying about the MV file.

When MV reuse is enabled, in the Nth pass it does not matter what performance is selected

So are you saying that no matter what performance setting you select on the second pass, the MV file dictates the quality setting for the encode (and therefore the quality setting is dependant on the MV file's data)? Or are you stating that the quality setting is independant of the MV file, and that it simply uses the data for its own processing purposes? If this were true, then all passes would need to be in the desired quality, i.e. "slowest."

And does it make any difference to encode for more than two passes, and what can the codec do in subsequent passes that it can't (or doesn't yet) do in the second?

[dTb]

Hmm, my understanding from what Gej has said is that the use of the mv file makes the performance setting irrelevant in an nth pass and that a b-frame will not use any mv info from the mv file and will therefore make use of whichever performance setting was selected to obtain this information.
Hope this clears things up for you silver_cpu, I'm fairly sure I've understood what Gej means but I guess he can clarify further if needed.

The Performance setting part makes sense to me, the motion estimation done determines the motion vector information stored in the mvinfo.bin file from my understanding. The purpose of this file is so that the process of determining this info is not repeated when it's not necessary. When the mvinfo.bin file is enabled the motion estimation may simply be disabled altogether during an nth pass, the rest of the work done by the codec probably remains the same regardless of the performance setting. I hope I'm making sense here.

From the b-frame part it seems that there's a possibility you can gain a benefit while using b-frames by setting the performance slider higher on an nth pass. I guess some testing would need to be done to decide if there is indeed any quality benefit and also if there is any speed benefit.

[silver_cpu]

So I guess my next test will be to guage quality with and without bframes at a certain bitrate. Anyone have a good idea on how to go about this in a good way?

[SeeMoreDigital]

silver_cpu
So I guess my next test will be to guage quality with and without bframes at a certain bitrate. Anyone have a good idea on how to go about this in a good way?
Well, as I've said previously, I've already carried out such tests (and more besides the ones posted). Which is why I'm a little confused about how to setup Kaukura's. Or should I really say I'm confused about which settings are useless/don't function when encoding with the MV log file.

Last night I posted a reply to one of Sagittaire's threads (in the 'new A/V formats section) which I think is relevant here:-
With regard to DivX Kaukuru I think it is DivX's finest codec so far.
However, this version has even more user settings than previous versions, which in my opinion will confuse many users. And as such cause much argument regarding 'what setting is the best to use etc'!

I've generated many low bitrate tests using all manor of Kaukuru's settings. But at the end of the day I would dump the lot and just set up the codec to write/read MV files - when you do this the encoder 'goes into warp drive'!

And there you have it..... I know it's only my opinion, but I really do think that most users will soon get bored/fed up with using the 'slowest' settings and only use the fastest method - ie write/read MV log files.

So I'm sure I'm not alone in wanting to know. When you use MV, what other settings are you able to use?

Come on Gej. Put me out of my misery!

[Fr4nz]

Gej optimizations for the new b-frames quantizations are really needed since bidi is too slow now.

[Acaila]

When you use MV, what other settings are you able to use?
1- When you use enable MV files during 2nd pass the codec will just copy those motion vectors, i.e. the codec will use whatever performance setting you've used during the 1st pass.

2- B-frames do not use the info from the MV file, so the B-frames will perform a new motion estimation/compensation at the performance setting you're using during the 2nd pass.

e.g.
1st pass: performance standard, disable B-frames, enable MV
2nd pass: performance slow, disable B-frames, enable MV (all frames will use same performance as 1st pass = standard)

1st pass: performance standard, enable B-frames, enable MV
2nd pass: performance slow, enable B-frames, enable MV (P-frames will use standard, B-frames will use slow performance setting)

And I disagree about the new versions having more settings, all it has is more performance options, it really doesn't take very long to understand what they do. But then I also think more control is usually better. For the newbies it's just a matter of not messing with the settings and using the defaults until you know what you're doing.

[SeeMoreDigital]

OK, I think I'm getting it.

Gej originally wrote...
"No, no, no. Qpel, GMC and B frames must be consistent from passes to passes, MV reuse or not."

And then he wrote...

"When MV reuse is enabled, in the Nth pass it does not matter what performance (setting) is selected. In practice, MV reuse is disabled for B-frames (BiDi), so the performance selected will be used to perform Motion Estimation for the B-frames".

Maybe the wording was a little confusing?


You mention the following setting: -
1st pass: performance standard, enable B-frames, enable MV
2nd pass: performance slow, enable B-frames, enable MV (P-frames will use standard, B-frames will use slow performance setting)

All of which I used during my second test. Using these settings was indeed faster than using no MV file.

Gej also wrote: -
"...I rather just do 2 passes and use "Slow" than 3 passes and use "standard"... Slow and slower really does make a difference (as well as PSY "fast")..."

I wonder if better encodes would be generated by using the following settings or would the benefits be too small: -
1st pass: performance slow, enable B-frames, enable MV
2nd pass: performance slow, enable B-frames, enable MV

I will do a speed test for this setting but it's obvious it will be long.

You also wrote: -
"And I disagree about the new versions having more settings, all it has is more performance options, it really doesn't take very long to understand what they do. But then I also think more control is usually better. For the newbies it's just a matter of not messing with the settings and using the defaults until you know what you're doing."

Granted it's good to have options but DivX has so many of them now, it's hard to tell 'visually' which one makes any significant difference or which option cancels out another option! Maybe there should be little 'pop up warnings' saying something like 'Be advised - selecting this option makes no significant difference to your encode' or 'Be advised - selecting this option is not possible'!

I notice RMV9 and WMV9 don't give you as may chances to 'get it wrong'! So it must be just an Mpeg4 thing. Which is probably why XviD is full of people discussing the same 'image quality versus settings issues' that we are doing here

One other thing I have noticed is with regard to post-processing (PP) when viewing your encodes via the DivX player 2.1. And that is, if you watch an encode with PP and then immediately select another encode (without closing down the player) the PP for the second encode is lost!

Still it's always fun to learn from each other - Keep coding!

[sapient]

The problem with kaukura is not that there are too many options, it's that half of the options do not work or in some cases do not work in the "obvious" way. If these issues are not resolved for the final, there should be a thorough redesign of the interface to make clear which options work and which don't. For example, when mv reuse is selected, the performance slider should be disabled. And the b-frame options should probabley be moved. Another idea is to have separate preformance sliders for p-frames and b-frames, since different performance levels are possible. In any case you have to make the options more straight-forward. One must be able to understand what's going on without having to search the discussion forums!

sapient

[Acaila]

I wonder if better encodes would be generated by using the following settings or would the benefits be too small: -
1st pass: performance slow, enable B-frames, enable MV
2nd pass: performance slow, enable B-frames, enable MVI believe that should give a nice improvement since now all frames will get to use the 'slow' setting. But you're right, it might take a while to complete :), though MV will still speed things up a bit here.
I notice RMV9 and WMV9 don't give you as may chances to 'get it wrong'! So it must be just an Mpeg4 thing. Which is probably why XviD is full of people discussing the same 'image quality versus settings issues' that we are doing here
IMO having a lot of settings is the result of the evolution of a codec. The more features get added, the more settings you need to control those, because not everything can and should be handled internally. I think that once RV9 and WMV9 get to the same level of 'maturity/age' of DivX and XviD, they will also have a lot more settings than they do now.
One other thing I have noticed is with regard to post-processing (PP) when viewing your encodes via the DivX player 2.1. And that is, if you watch an encode with PP and then immediately select another encode (without closing down the player) the PP for the second encode is lost!That's because the PP is controlled by the DirectShow overlay, and there can only be one instance of this at any given time. So if you open a new video while another one is already occupying the overlay, the second video won't have access to PP. It's a windows thing, nothing the codec can do about it.

[SeeMoreDigital]

Acaila,

Many thanks for the info.

I'm sure your words here are very useful for a lot of people - not just a thicko like me!

Going back to the settings issue. I can't imagine RV9 and WMV9 ever going down this route.
RV9 has recently introduced EHQ, which has vastly improved what is already a very capable codec, (at just about every bitrate and pixel frame size) without resorting to 'sliders' and 'check boxes'!
And as for WMV9. Well what can you say here... Would they really want to be bombarded with loads of 'what does this do?' or 'this does'nt seem to do anything' emails.

Maybe it is wishful thinking on my behalf to think that DivX should keep things simple. I would just hate the idea of potential new DivX users moving onto a different product because they find the quantity of setting too daunting!

I also think sapient makes a good point. Maybe the DivX interface should be totally redesigned. Maybe it should be split up into 'Novice User' and 'Expert User'.
But I do, really strongly believe that pop up 'help explaination panels' wouldn't go a miss!

We can only hope that DivX are looking and listening to both sides of the user coin.

Cheers and thanks again Acaila.

[bond]

gej
would be great if there can be an option in the codec interface which makes the feedback window not load automatically!

btw. i cant disable the feedback window by clicking on the right up corner "x" in windows

[Fr4nz]

You have to go in the additional settings window, you'll find the funtion which disable it.

[bond]

thanks :)

[SeeMoreDigital]

Originally posted by bond
gej
would be great if there can be an option in the codec interface which makes the feedback window not load automatically!

btw. i cant disable the feedback window by clicking on the right up corner "x" in windows
You might be lucky now, as your encodes should be quicker to do!

[dTb]

I've just completed testing a couple of methods, coincedently the two your wondering about SeeMoreDigital.

Rules of Attraction, 10mins starting around the end of the opening credits. PAL region 4

LoadPlugin("i:\PROGRA~2\GORDIA~1\mpeg2dec3.dll")
mpeg2source("G:\DVD Movies\Rules of Attraction\ROA.d2v")
trim(22500,37499)
crop(4,12,712,552)
LanczosResize(644,468)

Compress test of 47.16%

Converted to a Huffyuv file.

Standard settings: 1150kbps, bframes, sct 47%, mkt 250, fast pve

1st file
1st pass, standard, no mv: 19mins
2nd pass, slowest, no mv: 1hr57mins
Total: 2hrs16mins

2nd file
1st pass, slowest, write mv: 1hr54mins
2nd pass, slowest, use mv: 1hr14mins
Total: 3hrs8mins

Times taken from vdubmod job control, I was browsing during most of the encoding time so the times will be a little inaccurate but still pretty close.

So even using bframes there's a substantial time reduction with mv reuse although the first method is a lot quicker.
The quality of both files is good but I haven't had a chance to properly evaluate them yet which I'll do later today with psnr values included, just thought I'd post the times for those curious.

Edit: Doh, forgot to mention I used fast pve for both files.

[silver_cpu]

So what's the purpose of more than two passes, if I may ask? Does it simply increase accuracy/quality by a small measure? Or are there actual processes that are only enabled (or gain significant benefits) with more than 2 passes?

And btw, thanks to all of you for helping with the questions about which features do what and when :)

And while I'm at it, what's a good PSNR to aim for, in an encoding? Or is it something of an arbitrary number? Do the psy enhancements change this?

[LordIntruder]

So what's the purpose of more than two passes?

With another pass you can have a small increase in quality but the main target is to use the EKG program. With this tool you can reallocate bits wherever you want. Often the codec doesn't encode a scene correctly and You could do 10 passes in slowest mode it would be the same. So you do 2 or 3 passes, you load your video in EKG, you watch and you increase bits in bad encoded areas and can let EKG decrease bits by itself OR (what I do) decrease bits by yourself in good encoded scenes. It takes a very long time of work if you control your whole movie but quality rarely means quick task. ;)

I always do 3 passes but if I get macroblocks, etc... I load my movie into EKG, play with bits, do another pass and i'm done with a beautiful rip. :)

However even without EKG Divx used with good settings brings you a very good quality, but if you want to get the most you know what to do now. :)

[nuked]

and the best part is by the time you're done with all that, you never want to see the movie again... so you can throw the whole thing away and save 100% file space! :) Don't mind, me... I just kid. Each to his own.

[silver_cpu]

nuked: LOL! You don't know how many times this has happened to me. I don't have one of the fancy machines most of the enthusiasts here have, so by the time I finish messing up *another* encode, I can get kinda frustrated. I've figured out pretty much everything now, though, and mostly it's just the continually morphing DivX codec which gives me any headaches (and not many at that). Basically, I just wanted to know what these features were intended for, so I could get a better idea what would work with it. Maybe I'll try the EKG utility, but I have to ask: is Dr. DivX better than EKG, besides the addition of some features? Also, is EKG difficult to work with? Thanks, all!

[LordIntruder]

nuked: ROTFL. A good one! :D

EKG has nothing to do with Dr Divx. Dr Divx is to let people that have no knowledge in encodings to make their own with some clicks. EKG is to let people change the way bits are distributed by the codec. Two programs for two differents things. However I don't know if EKG can be used with Dr Divx.

EKG is very easy to use and there is a guide on divx.com. In less than 10min you'll be able to use it. However keep in mind that EKG doesn't do magic things. If you increase bits in some areas to get a better quality, it means equals bits will be remove somewhere else. It may be clear for you but I already see people increasing bits along all their movies believing it would work. hehe. ;)

To use it the first time I strongly suggest to encode a 1 min clip and to increase bits on the first 30 sec and then make another pass and observe how your last 30sec clip change. The most difficult part is to determine how much percents you have to increase the bad encoded scenes to get a good one because this part is yours and you have no help except your eyes to make a decision and feel, from the current quality, how much you should add in % to reach a good visual quality.

However it's not very hard. I often play between 10% to 25% and most of the time it's sufficient. Nevertheless I no longer use it (EKG) till I play with the beta divx codec in "slow" mode. Redo another 17h pass to get rid of 4 macroblocks in the upper-left corner in two scenes makes me nervous. ;)

[dTb]

In regards to my test, after visually comparing the two files I find it hard to find much of a difference between the two, the psnr results are also very similar. File 1 total average psnr using psnr4avi.exe = 42.63, File 2 = 42.55.
Also comparing the quant distribution with DRFAnalyser the distribution is very similar, file one with an avg. quant of 4.06, file 2 4.13. The percentage of the quants used was also very similar.

For future use of this codec I think I'll use a method similar to file 1, maybe two passes standard and one slowest, I'm unsure about pve but it didn't introduce anything noticeably bad.

[LordIntruder]

By the way, speaking of EKG and I've just noticed that since I installed Kaukura, this tool is gone. :( It was still there with Manihi. I don't care because I don't play with it right now due to the encoding times of "slow" passes but others people could need it.

Am I the only one that experience/notice this? Is it intended or a bug?

Also, did anyone tried the chroma option included in the Kaukura codec? I wonder if we can gain a noticeable visual quality. I'm afraid we have to deal with longer encoding times, so.

I quote divx site:

Psychovisual mode is now capable of operating on Chroma. This option is not enabled by default for performance reasons. In order to enable it you must create an empty file in C:\ called "DivxPvShapingChroma.txt" for "fast" mode and "DivxPvSimpleChroma.txt" for slow mode. Results show a very little gain (around 1-2%) in JND tests.

[Owen]

I may be the only person in on the planet doing real time Divx encoding, :D as no one ever seems to talk about it.
My findings with the Manihi and Kaukura builds compared to 5.05 is that quality of the new builds is great but I cant get low enough bit rates.
Where 5.05 gives me about 2.3gig per hour with good quality (except when it looses the plot and gets stuck on low quants for a second or two.), the new builds give about 6gig per hour when set to 1000kbs on Home theater profile.
Quality is very good, as it should be for that bitrate, but I want 2gig per hour.
What is happening with the new beta’s preventing lower bitrates.
I am encoding 720x576 PAL interlaced with Bidirectional encoding. No QPEL or GCM.
At leased interlaced encoding seems to work now. 5.05 was brocken.

Regards,

Owen

[silver_cpu]

Just wanted to state that I had not, in fact, tried DrDivx yet when I made that post...I stand corrected, if I'd tried it out already I wouldn't have made the post ;)

And to owen: if you would please tell me how in the world you're getting 25fps on your machine while encoding, I'd love you forever. The closest I've ever come was about 15-18fps. What kind of machine do you have? And what software do you use to encode your movies, and at what resolution?

[temporance]

Originally posted by Owen
Where 5.05 gives me about 2.3gig per hour with good quality (except when it looses the plot and gets stuck on low quants for a second or two.), the new builds give about 6gig per hour when set to 1000kbs on Home theater profile.
1000kbit/s = 125000 bytes / sec = 450,000,000 bytes / hour = 429MB/hour.

Something strange is happening if this is not what you're getting at 1000kbit/s. Which RC mode are you using?

[nuked]

Does it hurt the rc or anything to change rate between first and second pass. I managed to make an avs with gknot and some modifications where it was 24fps but gknot was asuming 30 in it's first pass it corrected the bit rate in between passes. The last time this happened I got a file size that was 20% off as well, although I think this has hapened before getting the correct file size, the only difeernce I know was having fast psyc on. I'm not gonna report a file size problem without more research though, but I just want to know.. if I go do another pass and change the bit rate, what effect will that have? I would kinda guess it should make it effectively about as good as a second pass even if it's a third or fouth pass, but I'm guessing.

[BoNz1]

@ nuked, if the frame rate changes between passes I can see how that would seriously mess up an encode. Just think about it. Say if at frame 1000-1200 you need a lot of bitrate and at 1200-1400 you don't need much. Now, you changed the frame rate from 30 to 24 in the second pass. Now we have a problem. In the second pass the RC will allocate bitrate from the first pass file for those high bitrate scenes in frames 1000-1200 only problem is that those frames don't require that much bitrate any more since they are now mostly the frames 1200-1400 in the first pass. Maybe if you did another pass it would help, but I would just start all over again.

[Owen]

silver_cpu,
I am using FlyDS (WDM capture app) to capture PAL 720x576 resolution from a MSI TV@nywhere card (WDM drivers) and 196kbs MP3 audio (Radium codec) on a P4 2.8Gig overclocked to 3.3Gig on 932Mhz front side bus with 1Gig Corsair duel channel DDR on ASUS P4P800 M/B.
My old system with P4 3.06Gig overclocked to 3.3Gig with single channel DDR and ASUS P4PE had no problem capturing this as well.

Temperance,
I have tried using Speed/Quality set to Standard, Slow and Slowest (if that is what you mean by RC), but nothing seems to get the bit rate down. I also tried capture at 4000kbs but that resulted in even higher bit rates. (at least that is consistent)
I want to get about 2 hours of video on a DVDR. That would need approx 4000kbs average.
Something is definitely not right with rate control for 1pass real time encoding.
As I said in my last post. 5.05 did give me close to 2Gig per hour. But a rate control bug would often ruin my captures with occasional bursts of macrobocks (low quants).
I have just built a new system with a clean install of WinXP SP1 and Kaukura and it has not helped. So it is not a system related problem.
If I could have the quality I was getting with 5.05 at 4000kbs without the rate control bugs and with working interlaced encoding I would be happy.
I can’t compare quality of new builds with 5.05 because the bit rates are so different.

Regards,

Owen

[nuked]

only problem is that those frames don't require that much bitrate any more since they are now mostly the frames 1200-1400 i

no, sorry, I was not clear. I did not change the frame rate between passes. I only changed the bitrate. The avs file remained the same. Actually what happened was gknot had a 30fps source of x length in time. Said your-file-size/x=bitrate. But I put telecining in teh avs by hadn cause I wanted some extra options. This should be ok because it doesn't change time or file-size so bitrate should stay the same. However after gknot saw the first pass results, it saw fewer frames than it expected. It assumed this meant the movie time was shorter (of course it was not) and incorectly adjusted the bitrate to compensate in teh second pass.
So actually it turns out the first pass was the correct bit rate and that's why my second pass file size came out too big.

I still have the same question but for a diferent reason. Can I do a thrid pass now at the right bitrate, will it be a) as good a normal thrid pass, b) as good as a normal second pass, or c) worse than all of the above and I should just start over.
Thanks.

[LordIntruder]

I already made a test of a short clip once to see what would happen if I change the bitrate between passes. Quality was still very good, no artefacts. No difference between the original (which was already very good). However I think (didn't verify it but it's simple to do) if you increase bitrate in the next pass you are going to get an oversized avi and if you decrease it an undersized one from your targeted size.

Gej showed us than we could make a first pass in standard, a second pass in standard and psy and a third pass in slow and psy so that we save time and even if modes are different, each pass gets gain from the precedent one. So I assume it's same here but the size will be different. But I repeat I never verify this point so I may be wrong.

Try and tell us what happens. If you don't want to lose your current work, copy both your current AVI and LOG files in a safe directory. From thoses files you can make as much tests as you want (assuming you know how to do with virtualdub).

[BoNz1]

nuked, you can change bitrate between passes without any problems. Just make sure of course that the change isn't really drastic like say in the first pass your bitrate was like 1500kbps and now you are changing it to 200kbps. Anyhow this doesn't sound like it is the case so IMO it would have to be a) as good a normal third pass.

[U977]

Agreed with Sapient:

I already fllow the XviD forum to know about all the advices and bugs.
Yet, I'm very interested in looking at hose new DivX builds.
But I definitively don't have enough time to follow any topic around (avisynth, GKnot, XviD, DivX...).

Even a web page that would summarize the status of teh coded would be fine. It's quite hard to have to read every forum to find out how healthy the codec is.

Anyway, Kaukura is still beta... We are warned that some things may not work. But good advices are still welcome, I guess :-)

[nuked]

Bonz1, ok.. that's about what I thought. I figured basically the passes are just iterative determinations of quality vs. compression which should be pretty much linear within a reasonable range of bitrate anyway.. More passes just add non-linear corrections when the linear predictions don't quite work out as predicted. That's why I was thinking it may be at least 2 pass quality, as the linear correction should be right already. I don't know if this logic is anywhere close to right, but seemed good to me. I haven't had a chance to try anything yet... actually I'm not sure if I'll be able to recover my settings this time anyway. I'll see, but good to know for reference at the very least that it should work. Thanks.

[silver_cpu]

U977:

Even a web page that would summarize the status of teh coded would be fine. It's quite hard to have to read every forum to find out how healthy the codec is

Well, if I understand what you're wanting, it's basically a more in-depth version of this site. Doom9 is great for supplying important, pertinent info to the digital video community, but unfortunately the staff is limited, and therefore can't bring you the kind of bleeding-edge news that you're looking for. To do so would require time to visit all of the boards you don't have time for yourself, and to coalate this information into a web update. Either way you go, it's hours' worth of work, and frankly few people have enough spare time for that. Of course, I'm not affiliated with the staff of Doom9 in any way, so they may have their own reasons for not providing the constant coverage that you're asking for.

Owen: that's a mean system, friend. Afraid I'm running an Athlon-tbird 850 OC'd to 950, with 1G of 133Mhz SDRAM. Phew :P I suppose if I were processing 3 times the info I am now, I'd be able to encode real-time, too! (time to upgrade, it's next on my priority list, just after paying the IRS and feeding my face!)

[nuked]

U977 isn't making anyone do anything. Was just a suggestion, and he didn't say D9 should do it either... maybe DivX or santa-clause could do it.

Lets just say, "not a bad idea IF anybody has the time and feels like doing it" and leave it at that. Cause it's not inherently a bad idea is it? Are we not allowed to present ideas unless we're willing to execute them ourselves?

[nuked]

if you click on the mv file selection button.. you get a file browser with the file filter set to *.bin. Seems alot like like having it set to *.mv would make more sense, no? The only options are *.bin and *.* with *.bin being the default. Of course *.* will work.

[LordIntruder]

It is not a bug, "****.bin" has always been here since the dawn of version 5.

[nuked]

why? all my MV files are labeled as blah.mv. Maybe that's just something gordian knot caused. are they supposed to be labeled blah.bin?

well I did thaat rencode in the right bit rate. Not happy with the results, but there could be a couple of reasons. This encode was already inteded to see if I could get improvement in a particularly tough flick by using fast pve which I had never used before. So I have one done without the bitrate switch with 2 passes at normal, no pve, no MV. Now I have this sorta 3 pass version but with the bitrate reduced in the last pass, with normal and fast pve, read/write MV. The bitrate of the final pass was the same for each and both had b-frames. I really expected this 3 pass attempt to come out better, but there are many dusty and hazy scenes where it looks significantly worse (no blocks, just less detail) and I can't find any place where it looks noticeably better. Oh well. I'd need much more testing to see exactly what's making it worse.

[SeeMoreDigital]

Well, talking of multipass.

I decided to see how an encoded image would change from pass to pass. The DVD I chose to encode was the PAL version of Star Trek Nemesis (112min).

The settings used were as follows: -
Codec Bitrate - 775kbps
Settings Used - MV/log file only + No other settings
1st pass 'standard'
Nth passes 'standard'

I generated 6passes in all. And saved each pass (5 in total).

Using this method I have concluded that 3 passes is OK but anything after that begins to blur the encode. And when you get up to the 6th pass, quite a bit of detail is lost.

I also have to say that there's not much quality improvement between the 2nd and 3rd passes either (using this bitrate speed anyway). So it's questionable whether the extra pass is worth the effort. Even though it's quite quick to generate!

Cheers

[ThePanther]

Guys,

I want to encode about 10 DVDs, should I use 5.05 or wait for the final release of "Kaukura". When is there likely to be a final release of "Kaukura"

Thanks A Million

[temporance]

@SeeMoreDigital,
I've never found that more passes blurs the encode. Are you sure that you never used the output of pass n as input for pass n+1 ?!?!

[SeeMoreDigital]

Originally posted by temporance
@SeeMoreDigital,
I've never found that more passes blurs the encode. Are you sure that you never used the output of pass n as input for pass n+1 ?!?!
Now you come to mention it. I could have quite easily made that error. I'll have to check. Thanks for the reminder.

Cheers

[silver_cpu]

Panther: this is on the beta download site:

The Quality Assurance (QA) period will run for about three weeks starting 2003-07-25. All the results must be e-mailed back to us by 2003-08-25 at the latest.

Therefore, it's safe to assume that the final release will be after they've been able to incorporate the various and sundry bug reports into their work, and clean up/modify the code as they desire. So maybe another month? Maybe a little longer, I'm not sure how their release schedule (if they have one) goes.

Also, does anyone know the name of the linux MP4 video codec? It's supposed to be really good, and I've been wanting to try it out, but I can't seem to find it on the web.

[zyrill]

i have to report a bug in kaukura. i encoded a movie using vdubmod 1.5.4.1 and the file has a black block in the lower right corner. it seldomly keeps changing the size and in the beginning it wasn't there at all... http://mitglied.lycos.de/zyrill/Die%20Hard_24847.JPG

[nuked]

Hey that's amusing. I don't think any of the rest of have seen this though. Are you so sure it's the codec? Can you look at the input to the codec directly... like if you're using a .avs and virtualdub, do you see this block when viewing the avs in virtualdub? This looks a little like the left-side effects I'm used to seeing with probelms arising from the video size not being a multiple of 16. It's a little diferent though. I doubt there was anything important in that corner anyway.

[SeeMoreDigital]

Agreed.

It looks more like a VirtualDubMod (VDM) problem rather than an Kaukura problem!

Have you at any time experimented making your own logo's etc with VDM. Maybe something has wondered into one of your 'plugins' folders by mistake - or wherever they have to go!

Cheers

[nuked]

This may not be peculiar to kaukura, just I noticed it now. I have a few scenes where the whole picture is jittering. In one case at least it was a problem with the transfer because the image border is shifted about 20 pixels over and I can see the edge of the picture moving with the whole picture. Anyway.. that's not my problem. My problem is this: This jittering scene is encoded with your basic IPBPBPBPBPB type pattern. There are hard edges in the image. In the B-frames the edges get blockwise jagged and start to dance. I seem to have plenty of bits for these scenes as mostly they look beautiful. My guess is some blocks on the edge are taken as P's and some as B's and somehow errors in the diferent motion vectors for the two is causing them to line up slightly diferently. Not sure there is anything much that can really be done about this though. I suppose I could try GMC or qpel. Not sure if I really have a question... just rambling observations.

[nuked]

update: gmc and qpel don't help. This is a pretty high contrast edge.. at least in color, maybe not in brightness. I'd expect to motion estimation to lock on pretty hard and acurate to this contrast. There is some noise.. but I'd think it would be litteraly in the noise compared to edge.

[nuked]

hehe.. that's kinda subtle.. bright sunlight, pale suface... sky and surface are the same brightness. I just ran "greyscal()" on it. Amazingly enough.. the edge is very low contrast when viewed this way. Kinda makes one think about color based motion vectors( I asume they currently only use limina), but then I'd probably need a faster computer.

[SeeMoreDigital]

Hi Nuked

Can you confirm your source's 'image pixel frame size' and your encodes 'image pixel frame size'. And the encoding application you use?

I've noticed that some applications are able to encode the pixels between the matte and the image, better than others!

Cheers

[nuked]

crop(4,6,710,268)
BicubicResize(632,268,0,0.5)

I already cropped once before this to keep noise in the black from affecting previous filters. This crop all in all ends up a multiple of four but not on a boundary of 4. I find it safest to do crops like that at the end to avoid some color glitches involving field based filters(I ahve a few pattern glitches to worry about and a cople that must be deinterlaced, but not in this scene. I have a recent post in avisynth about this crop issue.. I think it's on their todo list to fix)... hence the 2 crops. Anyway.. for size all you need is the last 2 numbers.. this is the only resizing done and this crop is exactly to the image....except in teh one scen where the mmiage shifts.. but I see this type fo affect in ohter scenes too. I don't follow why you ask really though. Are you saying the stuff between the matt has it's own fixed motion property? I think I may ahve noticed this type of effect... noise, that isn't quite noise... kinda sits there statically in front of the moving image. I can see how this could screw up motion estimation.

As for software... well now you see I'm using a home brewed avisynth script.. and then just encoding with vdub. I'm not sure how you mean about some software doing a better job of this than others. Do you just mean like using diferent noise filters to filter out this stuff? I've made a 10 second clip with the problem so I can try some things quickly.

ps.. this crop is exactly cropped to the image +/- one pixel maybe.
edit: actually I cropped an extra 2 pixels off one side cause the original had color issues in the last 2 pixels.

[SeeMoreDigital]

Hi nuked

Thanks for the detailed reply.

The reasons why I asked the questions is because I'm interested in the methods and the applications people use to create their encodes.

Personally I don't use scripts very often but I do like to use VirtualDubMod and more recently MPEGMediator.

However what I have noticed is, that even when you use the same settings in each application the images they create look slightly different. And as such generate different file sizes. VirtualDubMod, forinstance, seems to do a much better job at encoding the first few horizontal lines of pixels between the image and the matte that MPEGMediator. Especially if you encode at full frame, which is what I tend to do.

[nuked]

c-more,

ahh.. I see, I wasn't quite sure what you meant by matte... I was taking your thoughts in completely the wrong direction. You mean the pixels on the edge of the image joining the the black borders right? I have not used anything other than vdub or vdubmod(actualy using vdubmod here.. not vdub as I said above.) If it changes things I always work in yv12 or ocasionaly carefully thought out conversions into yuy2 and back for some filters... but then I use fast-recompress in vdubmod. I have also found these film edges to be handled pretty well and have never quite understood why some people make such a big deal out of problems with them.. I can see some blending with the matte but it's pretty minor.

The dancing edges I'm talking about are of course not atually on the matte edge, but on an edge in the actual image... I suppose the effect is a similar motion estimation problem though. I would think the diference would be the matte edge has higher contrast. I have suggested on the xvid.org board the possibility of allow the user to define an image box and restrict the sign of motion vetors for all blocks along this box so you only mix in stuff from the interior and never from the exterior, and use no motion vector at all for any block entirely outside the box(to prevent the oposite). I got no response. Probably not something your average joe user wants to mess with and honestly not something I care much about either.

Entersting though that the app would make such a diference. I assume if you don't use avisynth that you use filters in your app, like vdub filters. Cropping and resising in combination with yv12 can have issues for instance. I imagine that even with the same "settings" the actual filters used may be a little diferent. Just thoughts though.. As I mentioned, I've mostly stuck to one general method. I'm having a hard enough time learning this one.

edit: of course if your matte edge isn't at a 16 boundary there will always be some mixing even with no motion vectors, no?.. cause quantized dct's just don't handle hard edges perfectly I think..

[nuked]

oh and dont' thank me for detailed replies... my long replies are mostly just cause I'm full of..... Don't take anything I say seriously.. I'm ususualy just making stuff up that sounds good :) Every once and awhile I even get soething right.

[SeeMoreDigital]

The dancing edges I'm talking about are of course not atually on the matte edge, but on an edge in the actual image... This is why I asked the question about the source. As from time to time some movies are poorly transfered to DVD. You may have already noticed on your PC, that some 1.85:1 DVD's have vertical black borders as well as small horizontal mattes.

Now because these vertical black borders are not as crisp (hard edged) as the horizontal ones, the encoding application finds them more difficult to render.

Well that's the theory anyway.

Personally I can't understand why 1.85:1 DVD's with vertical black borders are released. As they really should be 1.77:1. But that's the film industry for you!

Probably does'nt solve your problem but it might be useful for you to know!

Cheers

[nuked]

hehe.. experience says this topic of dvd cropping and sizing leads to long useless heated arguments very quickly with poorly written standards and little hard evidence or support for anything... So I'm gonna say let's not go further with that... but I hear ya.

I had the oposite impression of sharp edges being encodeable though. I actually have read threads about adding egde smoothing borders to make them encode better. This makes sense to me since a square wave in coordinate space takes an infinite number of terms in frequency space to define exactly(edit: okay actually most any arbitray shape will, but particularly making an edge pretty sharp is I thought kinda dificult in frequency space). That's why audiophiles love the square wave test.. it tests the whole frequency range of their equipment in one go. I'm more familiar with continous transforms than discrete one's so maybe it's not quite infinite in this case. I guess this is all going pretty off topic now.. oops.

[SeeMoreDigital]

I had the oposite impression of sharp edges being encodeable though. I actually have read threads about adding egde smoothing borders to make them encode better I have not used such tools but it makes sense that they would work. As anything that can create a nice straight hard edge exactly along a row of pixels would be useful.

Wow, please don't go too mad with the technical speak as it gives me a headache. Especially at the weekend. I hear far too much of it at work. Most of it coming out of my mouth!

My real interest is mostly hardware, encoding for me is a sideline activity we do at work. Although converting customers home film and VHS movies to CD-R or DVD R is on the increase at the moment.

I prefer to keep the techno speak simple so the newbies can understand

Cheers

[nuked]

oh sorry..., math is half what my work is about.. or it's supposed to be anyway, I forget someitmes that frequency transforms are not considered as the basics in all lines of work :). Not trying to sound arrogant.. it's not hard stuff if I can understand it, but you're right it's stuff not everyone is familiar with. To put it simply I actually was under the impression that slow edges, like fades are much easier to encode thatn a quick sharp edge.. that's what I meant by smooth, not smooth as in unjagged, smooth as in unsharp, slow transition...

I think what these edge filters do is make the transition gradual, ramp up from black to the image color; and it makes a border that's easy on the eye and since it's a slow transition.. easier to encode and that's what I've heard, and that makes complete sense from how I understand the math, but I don't have first hand experience or evidence.

To make up for confusing any newbies, if you want, here's the long, hopefully easier to understand explanation. If you're more in the mood to have a beer than learn something (quite understandable on a saturday) then I've already said my point and you can quit reading here.

Mpeg 4 encoders use soemthing called DCT(discrete cosine transform). This is basically a way to describe a picture as a sum of waves starting with low frequeny waves and going up to high frequency waves instead of describing it as a bunch of individual pixels. Every wave has a value everywhere in the "block"(little chunk of the picture) and by adding waves of diferent frequencies which interfere diferently you can make every pixel be different. In theory a picture can be represented this way with no loss of information. Low frequency waves correspond to gradual changes. Many solid color areas like sky or whatnot can easily be described by only the lowest waves because they change slowly, so much information can be cut by essentially not including the higher frequencies, hence compression!(edit, actually the compression mechanism a bit more complicated than this) But a sharp edge is a sudden change.. it's not a wave at all actually, but it's sudden, low frequencies are slow and don't do sudden. You need high frequencies. How you can describe an edge at all with waves is complicated and reauires summing alot of high frequencies as well as some low ones, but it's easy enough for a computer.. and isn't important.. but the point is a sharp edge requires many more pieces to describe it and thus takes more data to encode. Now, I'm familiar with this type of math from diferent contexts entirely and there are many technical fields that use these principles, but I don't see why it would be diferent here and I have read a little about how the mpeg4 encoders actually work, but if I'm off base anywhere here someone pipe in.... Or, if as happens once and while, I'm more or less right.. wouldn't mind getting a "yup" either.

It's not just the math of the encoding transfomation, but might even make motion estimation more effective... like if something is coming slowly from off screen then you can encode it as a small diference singal of a block from the preious frame, where you chose to use a block that was hanging just a little off the screen before to compare to. All the parts that were showing then will match quite well and the parts that were not may not match so terribly badly either cause at least the area out there was basically the right shade and color. All you need to encode your picture then is a small difference signal and a simple 2 number vector that points to the place in the previous frame where the image came from. How much does this help in practice?.. beats me.

[dTb]

Originally posted by zyrill
i have to report a bug in kaukura. i encoded a movie using vdubmod 1.5.4.1 and the file has a black block in the lower right corner. it seldomly keeps changing the size and in the beginning it wasn't there at all... http://mitglied.lycos.de/zyrill/Die%20Hard_24847.JPG

You've reminded me, I experienced something very similar only for me it was actually square rather than rectangular. The res is 556x400 so not multiples of 16 but I think it's actually a decoder error as it's only visible when kaukura is doing the decoding. Decoding with ffdshow and the problem goes away.

A problem has been discovered with Kaukura, it seems if you use slowest p/q, slow pve and the mvinfo.bin file you might encounter problems, more info here
http://forums.divx.com/viewtopic.php?topic=52998&forum=23

[zyrill]

dTb - you are absolutely right. it's a decoder bug - even if i watch it in VDubMod it isn't visible anymore. nevertheless it's very annoying... let's hope gej reads through here and fixes it - my account on divx.com is borked, i'm waiting for my confirmation email since days.

[SeeMoreDigital]

Hi nuked,

Well what can I say. You've certainly made the science of Mpeg4 an interesting read!

Although I'm know maths wiz I'm quite proud of the fact I can still do some long division without the use of a calculator! He He.

Well it would seem from the last couple of posts that there is a bug in Kaukura. As I have'nt seen this bug myself, it makes me wonder if this only happens during cropping and resizing when using MV!

Cheers

[zyrill]

all cropping and resizing was done before encoding with avisynth furthermore i did not reuse the mv-file...

[nuked]

I still think it may be a 16 bug. I just went back and looked at a couple of old avi's where I had 16 glitches. I have often found that not using multiples of 16 causes a small black box or stripe in the top left and causes teh whole colum uder it to be shifted and wierd. I remember concluding that I could get by with breaking the 16 rule sometimes if I turned fast recompress on.. strage, but that's what I remember(been awhile, don't hold me to it). Since then I ahve always used fast recompress and I think I've probably made some illegal crops, but I haven't seen the problem. Anyway, I went back and found an old encode where I had that problem and played it with kaukura. Sure enough, you guys aren't seeing things... there's the block in the lower right along with the left side effects... only my lower right block is cooler than yours... it's green. I just played a BUNCH of other avi's that didn't have this left side problem and none of them had the block in the lower right. Ok.. so your's didn't have the left side problem... but still maybe it's 16 related... obviously though whatever it is, it could be fixed in the decoder.

So i figured, hey.. if ffdshow fixes the lower right block, maybe it fixes the left side problem... nope. tried libavcodec and xvid codec with every idct... left side problem still there. but the right side block is indeed gone.

[nuked]

ps.. my problem encode did not use an mv file either and oh yeah.. it was made using 5.02.

[SeeMoreDigital]

This problem is getting stranger and stranger.

I wonder if users can confirm if this problem occured after they had cropped and/or resized their encodes!

[nuked]

well zyrill's image is 712x304 so I was assuming he cropped it... and it's a multiple of 8 but not 16.. I wonder if zyrill wants to try to encode a few minutes of it with it recropped to 720... on the other hand.. this probably shouldn't happen anyway and he already found a cure. As far I care there's nothing wrong at all with using ffdshow... kinda has alot of neat features actually.

[SeeMoreDigital]

Hi nuked,

This is a wierd problem. I generated some cropped and resized encodes using MPEGMediator and did not experience any problems at all!

I'll do some more tests tomorrorow.

Cheers my friend!

[nuked]

it is strange. I just looked again.. the one clip, I can find a problem with is 652x272. 652 is not even divisible by 8... I kinda thought I might have remembered there actually being an issue with 8 even more than 16... but then zyrill's clip is divisible by 8. Oh yeah.. of course being an avisynth guy.. I always had my clips cropped and resized before sending them into vdub... not using the vdub filters. Who knows what diference this makes. Good luck with the tests.

edit: and back when I made that one.. I probably really wsa using vdub, not vdubmod

[dTb]

I just discovered with my clip that enabling 'YUV Extended Mode' fixes the problem so give it a try on your files guys and see if you get the same result.

I have other clips that are not multiples of 16 or 8 which don't exhibit the problem so I'm unsure if that really is the cause.

[nuked]

no dice... YUV doesn't fix it for me.. still a green block. I don't know that it's size related... but I do know that my left side probelm was size related. I could very reproducibly crop or resize to a mulitple of 16 and it would go away without changing anything else. All I know for sure about this green block was that out of about 15 divx's that I tried.. the only one with the green block happened to be the same one with a left-side problem... coincidences do happen, but more often they don't happen.

[zyrill]

what the? now my block is green as well! alright something is really really weird! it used to be black... i don't understand this... maybe because i installed the matroska filter again? well unfortunately i don't have the source files anymore because at first when i checked the file in vdubmod there were no visible errors so i deleted the vobs :( i might be able to rip it again but it'll take time. anyway: i also used avisynth 2.5 (nuked seemed to believe i didn't) for cropping and resizing.

[fccHandler]

Just a comment: I used to have a little flickering green line (maybe 8 pixels wide) in the upper left corner of all my DVD2AVI to Mpeg2Dec3 to VirtualDub encodes, and I wasted a lot of time trying to track it down. It vanished when I installed the latest version of Avisynth, and I haven't seen it since.

I don't know if this is related to your problem, but it demonstrates how difficult it can be to place the blame when so many components are involved...

[zyrill]

well when i watch the plain avisynth-created file, there are no errors and even if i play the encoded file with vdubmod there are no errors - only kaukura decoder creates those nasty blocks. did i mention the blocks are not there when watching the movie with ffdshow?

[nuked]

zyrill... I was referencing SeeMoreDigital about the avisynth cuts.. not you, not that it matters. fccHanlder.. that thing you're talking about in the upper left is exactly what I've been talking about. I should find a screen shot of it. That is an equally elusive little bugger.. I think they are all combinations of issues.. as are most bugs. I mean.. you probably never saw that little bar in vdub before you encoded.. so you can't say it's entirely an avisynth issue if vdub could view the avisynth's just fine. I think it's a subtle combination of a at least a couple of things. Of course the new avisynth works in yv12 if by new you mean the 2.5 versions... and I found that using fast recompress helped the issue and fast recompress is also a color space issue. Might be related... who knows. Maybe Gej has ben unresponsive recently casue he's working on getting all these little issues hammered out for the final release.

[valipod]

This is really strange:

With slowest performance, Q-Pel makes encoding more than twice faster than without Q-Pel, while with standard performance, things are the other way around, it takes a little longer to encode with Q-Pel. I have tried this with several movies (BiDi and GMC always used, no MV).

Wether we like Q-Pel or not, this another story. I never use it, and this is why this thing with speed kind of annoys me...

Anyone have an idea, it feels like a bug...

[mikeson]

@valipod: Qpel is deactivated in slow and slowest modes ATM.

[calinb]

Originally posted by Gej
My favorite’s settings so far:

Common setting:

Home Theater profile: on
B frames: on
Psy: fast (no chroma hack)
NO MV file

1st pass
performance: Standard

2nd pass
performance: Slowest
Gej, Just curious--would these be your favorite settings, if you didn't care about simple profile-only hardware playback? Seems like there isn't much of a downside (other than hardware playback) by using GMC. As you said in your 5.02 guide:Originally posted by Gej in his DivX 5.02 guide on www.divx.com

Beside possible incompatibilities with some simple profile only hardware device, this option has the green flag for everyday use.

[nuked]

GMC is "compatible" with my laptop... but it runs the battery down like twice as fast last I checked(which wasn't recently).

[valipod]

Originally posted by mikeson
@valipod: Qpel is deactivated in slow and slowest modes ATM.

Well... I don't know. Since I have made several encodes with slowest performance, in pairs, with and without QPel, and first: there is a big difference in speed, QPel being faster, and second: there is a clear difference in quality (the "having a life of their own" artifacts apear when using QPel). So I am quite sure, QPel is not deactivated when using slowest performance.

[SeeMoreDigital]

Originally posted by calinb
Gej, Just curious--would these be your favorite settings, if you didn't care about simple profile-only hardware playback? Seems like there isn't much of a downside (other than hardware playback) by using GMC. As you said in your 5.02 guide: I am happy to announce that hardware players, such as Kiss range and the SigmaDesigns Xcard will now spin DivX files encoded with GMC. However, Qpel has not been implemented as yet!

Cheers