XVID and VBV

[gatormac]

I have been using XVID 1.0 RCx to backup movies succesfully for some time now, but have come across this problem:

I just bought a Philips DVP642 standalone which is Divx certified and plays all other MPEG4 ASP spec compliant streams as well. My XVID encodes look beautiful and for the most part play back without a hitch...its hard to tell it apart from the actual DVD source. However, during bright high motion scenes there is a brief hitch or delay that is very annoying. I encode with fairly default settings except for 1 b-frame, unpacked, and a 1 pass at a Q of 2. I tried adjusting encoding settings, but came to the conclusion that it was a frame size/bitrate limitation issue since it was only happening in bright motion scenes (dark motion sequences were fine). I came on here and did a search and found this in another thread:

Do you mean the VBV requirement of ASP? Xvid does not yet respect VBV - this could cause problems in high-action segments of a movie encoded at high bitrate. An xvid encode could cause a "MPEG-4 ASP@L5" compliant decoder to stall (jerky/slow playback) while it recovers from a buffer underflow. I believe DivX is the only MPEG-4 encoder that respects VBV, although the xvid people are working on a VBV rate control for sometime after the 1.0 release. Until this is done, xvid can't really claim to comply to the MPEG-4 profiles (like SP or ASP).

When can we expect for the XVID team to implement this? I'm very excited about actually having a standalone that can play my XVID encodes so well and hate the thought of going back to the evil empire Divx.

[sysKin]

Originally posted by gatormac
When can we expect for the XVID team to implement this? I'm very excited about actually having a standalone that can play my XVID encodes so well and hate the thought of going back to the evil empire Divx. Well 1.0 is not out yet. After that, I really can't tell you much because I will not be the guy who implements it (I know absolutely nothing about rate control).

Let's hope it will come soon because it is really important.

In the meantime you might try using two-pass encoding with some "high-motion reduction", it would chop off the highest bitrate scenes. You might also increase b-frame threshold, because most of the bitrate does not come from high motion but rather because b-frames get disabled there.

If not, you might just decrease quality in general - I doubt you will see a difference between quant 2 and 3 on a TV. But what I said before is better.

Good luck,
Radek

[gatormac]

Thanks for the response Radek. I did a few test runs on the section of video that caused the problem by lowering the quant to 3, 4, and 5. The Q3 vid still had the problem, but not nearly as frequent. The Q4 vid did even better, but still had a spot every now and then that would hiccup. Q5 was improved even more, but still had 2 spots that delayed. Q3 looked good enough, but I could detect some loss in quality with Q4 and Q5, but they as you said would probably be good enough on a TV. However, I don't really care to go through every video scene by scene to determine whether or not I need to change the quant by setting a zone, nor do I want to lower the quality of my entire encode by lowering the quant for the entire film.

Your suggestion for doing a 2-pass encode with high motion reduction sounds like a possibility, but I have no idea how to do high motion reduction (so I will do some searching). I have no idea what setting the b-frame threshold will do either, nor what numbers to try so I will search for that as well.

This issue makes me wonder if XVID is superior to Divx only in that it has no limitations for bitrate per frame. I encoded Forrest Gump with Divx back before I switched to XVID and noticed that some pixelation would occur at some high motion scenes, yet when I tried with XVID it didn't have that problem. I can only assume that this is due to the issue we are discussing here. If that is the case, if and when XVID becomes spec compliant will it have the same quality problems that Divx does by being limited to a bitrate ceiling? I hope not.

Before I make this next statement I would like to thank the devs of XVID for all the hard work they do and the great product they have created. This is not a complaint, but a thought. I was just wondering why they would wait for the 1.0 final release before tackling the issue of VBV. It doesn't make much sense to me to release a product that doesn't even comply with the standard that has been set. I understand that the release is way too close to go messing with things now, but it is odd to me that it wasn't dealt with earlier.

[bond]

Originally posted by sysKin
Let's hope it will come soon because it is really important.yep, especially for standalones

would love to see if xvid could encode in divx5 profiles :D

[gatormac]

I tried increasing B-VOP sensitivity to 10, 20, and 30....the video still had the bitrate spikes. I tried reducing high motion/bitrate by 10, 20, 30 and 70%....the video still had bitrate spikes. Are these options even working?

[gatormac]

I'm not sure why this isn't being discussed more....it is IMO the biggest problem facing XVID, much larger an issue than the typical issues such as whether or not somebody can figure out how to get their A/V in sync or which is the best quantization matrix to use.

Also, I don't think the high bitrate scene degredation works at all the way it is supposed to. The movie I have been having the bitrate spiking issues with I first encoded like this:

AS@L5
H.263
AQ,IE,Qpel,GMC--ALL OFF
B-VOPs 1,1,1 unpacked, closed GOV
16:9 NTSC PAR
1 pass @ Q2
MSP 6
VHQ 1
Chroma motion on
Frame drop ratio 0
Max I-frame interval 240
Quants 1-31
Trellis

The resulting frame size was 688X464
The min/avg/max/total key frame sizes were 3481/39504/157157/55168k
The " " " delta frame sizes were 9/19458/182881/1309237k
average bitrate was 3810kbps

I needed to limit the higher range of frame size to keep the video from exhibiting the delay that I detailed earlier, so I did everything the same except that I did a 2 pass encode, I did a 1:1 AR instead of anamorphic, I-frame boost at 0, and high bitrate scene degredation set at 30%.

The resulting frame size was 704X384.
min/avg/max/total keys were 2956/36290/186415/46675k
" " " deltas were 9/20425/149232/1376608k
average bitrate was 3975kbps

The video was worse than the first attempt.

If the max frame size is ~1620mbs, then why would it be so hard to cut off the bitrate distribution at that max per frame and distribute the excess to other frames that don't require as much?

[RadicalEd]

The reason it hasn't been on the forefront is because XviD is predominantly a computer-based codec at the moment, and few (if any) PC decoders require the rate adjustments as defined in the levels. I'm sure even a quick max bitrate hack that simply raises quantizer on a frame until it fits wouldn't be too difficult to implement, but that sort of thing hasn't been an immediate concern. I haven't the slightest idea what it would take to implement all of the controls, as I don't possess the standards papers nor have enough secondary research behind me to even be sure of what they all are, but I'm sure the task will be tackled in full when possible. Sorry, if you don't have the coding abilities to contribute, you just have to sit tight :| that's GNU for you.

Don't get me wrong, I'd like to see levels working too. But 1.0 is "Just A Number"™ and there's plenty of time for advancement.

[sysKin]

No, the reason it hasn't been on the forefront is because we can't do it. There are people who can, but they didn't do it. Simple as that.

So, if you think it's important, code it. If you think it should have been coded before, as yourself why you didn't.

Ah, the simplicity of opensource :D :D :cool:

I was kinda kidding, but not too much.
Radek

[gatormac]

I'm not trying to sound full of complaints...believe me, I love this proggy and think the world of the developers. If I had any programming ability at all I would be all over this though. I feel by the time I taught myself the basics of programming and caught up to speed in the video coding world I would be releasing my spec compliant MPEG4 XVID codec about the time everyone else was releasing their MPEG-873642 holographic neurosensory codec with true tactile immersion, somewhere around the year 2055.

I have great worrys about this as I encoded the clip in spec compliant Divx....I had forgotten how much detail is lost using Divx. It looked awful to me. Maybe this is why the devs aren't tackling it...not that they can't, but that they know the quality will take too much of a hit.

[Teegedeck]

Excuse me if I butt in, maybe something's gotten past me - but how do you intend to play a video with an average bitrate of 3975kbps? That's about the upper limit the latest Mediatek MPEG-4 chip will decode and it is your average, not your max. Encoding at quant=2 doesn't seem like a good idea.

Also you write about your 1-pass results and wonder why changing the curve-compression settings doesn't have any effect. Curve compression only works in two-pass.

As I've said, maybe I've overlooked something in your posts.

[gatormac]

Originally posted by Teegedeck
Excuse me if I butt in, maybe something's gotten past me - but how do you intend to play a video with an average bitrate of 3975kbps? That's about the upper limit the latest Mediatek MPEG-4 chip will decode and it is your average, not your max. Encoding at quant=2 doesn't seem like a good idea.

Also you write about your 1-pass results and wonder why changing the curve-compression settings doesn't have any effect. Curve compression only works in two-pass.

As I've said, maybe I've overlooked something in your posts.

I have no idea about upper levels of any chip and I have no idea what chip this player uses (but I realize the latest Mediatek is considered the best ATM). The entire movie encoded at Q2 plays beautifully....except for the bright high motion scenes. I would like to keep as much quality as possible while omitting the erroneous playback. The 2-pass encode was aimed at a filesize that was 80% of the compression check in Gknot and using the curve compression. I know that changing the curve compression settings only work in 2-pass (and in fact are only available in 2-pass). However, my first encode at a constant Q of 2 would use more or less a default curve compression, therefore I should be able to compare the 1-pass with the 2-pass with curve compression (shouldn't I?). In any event, these problems were occuring even at a constant Q of 5, although much less frequently. I guess I will try a Q of 6....but you are really starting to lose alot of quality by that point.

Using Divx and Xvid at a target bitrate of 4000kbps yielded ugly results. Very washed out with lack of detail. That is why I have been encoding at a Q of 2 on 1-pass and at 80% of Gknot compression on 2-pass. It gives good results.

[Teegedeck]

What kind of source are you encoding? HDTV? ATM I'm pretty lost as to how anything can look washed out at a two-pass target-bitrate of 4000 kbps. This could fill a DVD-R for a movie. Well, almost... 3.5 GB for an average movie, I guess.

[gatormac]

Originally posted by Teegedeck
What kind of source are you encoding? HDTV? ATM I'm pretty lost as to how anything can look washed out at a two-pass target-bitrate of 4000 kbps. This could fill a DVD-R for a movie. Well, almost... 3.5 GB for an average movie, I guess.

You're point is right and it is also wrong. The entire movie encoded ~3900kbps looks sweet....its just the bright action sequence that needs more for the frame size. I have been encoding the action sequence in question to test out different settings on the player and the scene was averaging 8000-10000kbps whereas the rest of the movie was probably in the 1500-2500kbps range. So I have been trying to lower the bitrate of the action sequence without losing too much detail....that is why 4000kbps looked washed out....it was a scene that initially wanted 10000kbps. And the fact that I was using crappy Divx.

My last attempt I used XVID 2-pass method set at a target bitrate of 4000kbps, 0 I-frame boost, overflow control strength 0 (letting XVID decide), and high bitrate scene degredation at 20%, B-VOP sensitivity at 50. The result was an average bitrate of 3998kbps, quants ranging from 2 to 8 with an average of 3.96, the video looked much better than the sorry Divx attempt at the same bitrate....not perfect, but definately acceptable. Some detail was lost, but not enough to worry about and the delay problem was gone. I am still going to try to refine this method, first by seeing if it plays OK without the degredation for high bitrate scenes, and also by upping the bitrate a bit at a time until the problem recurs so that I will know a ballpark of the players limitations. Any suggestions regarding settings are greatly appreciated.

The video in question is Band of Brothers Episode 2, region 1, NTSC. I have been testing frames 44000 to 49000, the scene where they attack the German guns in a fixed position. Very bright and fast.

[gatormac]

UPDATE

The 2-pass encode that worked yesterday was only when I encoded the high bitrate scene alone. Once I used those settings on the entire movie the same problems happened again. The largest frame size when I encoded the scene only at 4000kbps was 90602. Using the same settings for the entire movie (except actually lowering the target bitrate to 3300kbps) yielded a largest frame size of 166221. Heck, I encoded Gladiator at 1650kbps and it had large frames in action sequences too...126297 to be exact.

Basically, encoding a 2-pass @ 3300kbps (or 1650kbps or whatever) does fine if you encode scene by scene and then append the segments together, but if you encode the entire movie @ 3300kbps, the high bitrate scenes will try to use X amount over the average 3300kbps that the low bitrate scenes didn't use....if you remember the rest of the movie was probably encoding in the 1500kbps range. Encoding 90% of the movie at the low range will give the high bitrate scenes ALOT of extra bits to play with to reach that average of 3300kbps, thus far overshooting the 3300kbps average for that scene.

I would think that the current XVID bit allocation method would be fine, but to give users the option of staying compliant, just put a cap on the frame size regardless of what method you are encoding. It could be a setting called "Standalone compliance" and work exactly the same way, giving users all the options that XVID currently has, but when noticing that XVID wants to allocate X amount of bits to a frame or sequence of frames it will cut off the top and put the extra bits in storage for use in lower bitrate scenes. My lack of understanding of how it all works probably makes that suggestion totally useless, but it would be nice. I think probably 90-95% of all frames encoded even at a Q of 2 fall within the specs, but its that 5% that can cause problems.

[gatormac]

I was just thinking as I headed out the door to work, what is the bit limitation per frame @ and average of 4000kbps? Well, correct my math if I'm wrong, but 4000kbps would be 4Mbps = 4,000,000bps. For a 24fps source that would come out to 4,000,000bps/24fps = 166666 bits per frame. What am I doing wrong here as this should be plenty for almost any source?

[gatormac]

I encoded Band of Brothers Episode 2 last night using AutoGK setting the file size at 1/5 DVD (896MB) and using 2-channel AC3. Here is the log file:

[4/16/2004 12:03:22 AM] Input dir: D:\DVD\Rips\BOB2_VTS_01_PGC2
[4/16/2004 12:03:22 AM] Output file: D:\DVD\Rips\BOB2_VTS_01_PGC2\autobob.avi
[4/16/2004 12:03:22 AM] Audio: English
[4/16/2004 12:03:22 AM] Subtitles: none
[4/16/2004 12:03:22 AM] Codec: XviD
[4/16/2004 12:03:22 AM] Target size: 896Mb
[4/16/2004 12:03:22 AM] Custom resolution settings: fixed width of 704 pixels
[4/16/2004 12:03:22 AM] Started encoding.
[4/16/2004 12:03:22 AM] Demuxing and indexing.
[4/16/2004 12:04:48 AM] Analyzing source.
[4/16/2004 12:04:48 AM] Writing the following script to D:\DVD\Rips\BOB2_VTS_01_PGC2\gk_tmp\lastjob.vcf
===========================================================
VirtualDub.Open("D:\\DVD\\Rips\\BOB2_VTS_01_PGC2\\gk_tmp\\_.avs",0,0);
VirtualDub.RemoveInputStreams();
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(0);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetCompression();
VirtualDub.video.filters.Clear();
VirtualDub.subset.Delete();
VirtualDub.SaveAVI("D:\\DVD\\Rips\\BOB2_VTS_01_PGC2\\gk_tmp\\_.avi");
===========================================================
[4/16/2004 12:09:04 AM] Source has percentage of interlacing in motion areas: 36.49
[4/16/2004 12:09:04 AM] Source has percentage of telecined patterns: 91.33
[4/16/2004 12:09:04 AM] Source has percentage of progressive patterns: 8.67
[4/16/2004 12:09:04 AM] Source has percentage of interlaced patterns: 0.00
[4/16/2004 12:09:04 AM] Source is considered to be hybrid (mostly FILM).
[4/16/2004 12:09:04 AM] Writing the following script to D:\DVD\Rips\BOB2_VTS_01_PGC2\gk_tmp\lastjob.vcf
===========================================================
VirtualDub.Open("D:\\DVD\\Rips\\BOB2_VTS_01_PGC2\\gk_tmp\\_test.avs",0,0);
VirtualDub.RemoveInputStreams();
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(0);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetCompression();
VirtualDub.video.filters.Clear();
VirtualDub.subset.Delete();
VirtualDub.SaveAVI("D:\\DVD\\Rips\\BOB2_VTS_01_PGC2\\gk_tmp\\_test.avi");
===========================================================
[4/16/2004 12:09:04 AM] Looking for optimal hybrid thresholds.
[4/16/2004 12:13:10 AM] Found threshold of: 0.24
[4/16/2004 12:13:10 AM] Found 74377 frames
[4/16/2004 12:13:10 AM] Audio size: 74,883,072 bytes (71.41 Mb)
[4/16/2004 12:13:10 AM] Overhead: 516,992 bytes (0.49 Mb)
[4/16/2004 12:13:10 AM] Video size: 864,124,032 bytes (824.09 Mb)
[4/16/2004 12:13:10 AM] Running compressibility test.
[4/16/2004 12:13:10 AM] Writing the following script to D:\DVD\Rips\BOB2_VTS_01_PGC2\gk_tmp\autobob_comptest.avs
===========================================================
LoadPlugin("C:\PROGRA~1\AutoGK\filters\mpeg2dec3dg.dll")
LoadPlugin("C:\PROGRA~1\AutoGK\filters\autocrop.dll")
LoadPlugin("C:\PROGRA~1\AutoGK\filters\decomb.dll")
LoadPlugin("C:\PROGRA~1\AutoGK\filters\undot.dll")

function getOrder(clip c) {
order = GetParity(c) ? 1 : 0
return order }

movie = mpeg2source("D:\DVD\Rips\BOB2_VTS_01_PGC2\gk_tmp\autobob.d2v")
cropclip = autocrop(movie,mode=0,wmultof=4,hmultof=4,samples=10,aspect=0,threshold=35)
c_width = width(cropclip)
c_height = round(height(cropclip) / 1.21518987341772)
input_par = float(c_width)/float(c_height)
fixed_aspect = input_par > 1.4 ? 1.21518987341772 : 1.18518518518519
c_height = round(height(cropclip) / fixed_aspect)
input_par = float(c_width)/float(c_height)
out_width = 704
out_height = round(float(out_width) / input_par)
hmod = out_height - (floor(out_height / 16 ) * 16)
out_height = (hmod > 4) ? (out_height + (16 - hmod)) : (out_height - hmod)
new_aspect = (float(out_width) / float(out_height)) / fixed_aspect
Telecide(movie,order=getOrder(movie),guide=1,post=2).Decimate(mode=3,threshold=0.24)
autocrop(mode=0,wmultof=4,hmultof=4,samples=10,aspect=new_aspect,threshold=35)
Undot()
LanczosResize(out_width,out_height)
SelectRangeEvery(300,15)
===========================================================
[4/16/2004 12:13:10 AM] Writing the following script to D:\DVD\Rips\BOB2_VTS_01_PGC2\gk_tmp\lastjob.vcf
===========================================================
VirtualDub.Open("D:\\DVD\\Rips\\BOB2_VTS_01_PGC2\\gk_tmp\\autobob_comptest.avs",0,0);
VirtualDub.video.SetMode(1);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x64697678,0,10000,0);
VirtualDub.video.filters.Clear();
VirtualDub.subset.Delete();
VirtualDub.SaveAVI("D:\\DVD\\Rips\\BOB2_VTS_01_PGC2\\gk_tmp\\autobob_comptest.avi");
===========================================================
[4/16/2004 12:20:55 AM] Duration was: 7 minutes 44 seconds
[4/16/2004 12:20:55 AM] Speed was: 8.00 fps.
[4/16/2004 12:20:55 AM] Compressibility percentage is: 54.83
[4/16/2004 12:20:55 AM] Using softer matrix.
[4/16/2004 12:20:55 AM] Using softer resizer.
[4/16/2004 12:20:55 AM] Chosen resolution is: 704x400 ( AR: 1.76 )
[4/16/2004 12:20:55 AM] Predicted comptest value is: 64.38
[4/16/2004 12:20:55 AM] Running first pass.
[4/16/2004 12:20:55 AM] Writing the following script to D:\DVD\Rips\BOB2_VTS_01_PGC2\gk_tmp\autobob_movie.avs
===========================================================
LoadPlugin("C:\PROGRA~1\AutoGK\filters\mpeg2dec3dg.dll")
LoadPlugin("C:\PROGRA~1\AutoGK\filters\autocrop.dll")
LoadPlugin("C:\PROGRA~1\AutoGK\filters\decomb.dll")
LoadPlugin("C:\PROGRA~1\AutoGK\filters\undot.dll")

function getOrder(clip c) {
order = GetParity(c) ? 1 : 0
return order }

movie = mpeg2source("D:\DVD\Rips\BOB2_VTS_01_PGC2\gk_tmp\autobob.d2v")
cropclip = autocrop(movie,mode=0,wmultof=4,hmultof=4,samples=10,aspect=0,threshold=35)
c_width = width(cropclip)
c_height = round(height(cropclip) / 1.21518987341772)
input_par = float(c_width)/float(c_height)
fixed_aspect = input_par > 1.4 ? 1.21518987341772 : 1.18518518518519
c_height = round(height(cropclip) / fixed_aspect)
input_par = float(c_width)/float(c_height)
out_width = 704
out_height = round(float(out_width) / input_par)
hmod = out_height - (floor(out_height / 16 ) * 16)
out_height = (hmod > 4) ? (out_height + (16 - hmod)) : (out_height - hmod)
new_aspect = (float(out_width) / float(out_height)) / fixed_aspect
Telecide(movie,order=getOrder(movie),guide=1,post=2).Decimate(mode=3,threshold=0.24)
autocrop(mode=0,wmultof=4,hmultof=4,samples=10,aspect=new_aspect,threshold=35)
Undot()
BicubicResize(out_width,out_height,0,0.5)
===========================================================
[4/16/2004 12:20:55 AM] Writing the following script to D:\DVD\Rips\BOB2_VTS_01_PGC2\gk_tmp\lastjob.vcf
===========================================================
VirtualDub.Open("D:\\DVD\\Rips\\BOB2_VTS_01_PGC2\\gk_tmp\\autobob_movie.avs",0,0);
VirtualDub.video.SetMode(1);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x64697678,0,10000,0);
VirtualDub.video.filters.Clear();
VirtualDub.subset.Delete();
VirtualDub.SaveAVI("D:\\DVD\\Rips\\BOB2_VTS_01_PGC2\\autobob.avi");
===========================================================
[4/16/2004 1:25:08 AM] Duration was: 1 hour, 4 minutes 12 seconds
[4/16/2004 1:25:08 AM] Speed was: 19.31 fps.
[4/16/2004 1:25:08 AM] Expected quality of first pass size: 58.83
[4/16/2004 1:25:08 AM] Running second pass.
[4/16/2004 1:25:08 AM] Writing the following script to D:\DVD\Rips\BOB2_VTS_01_PGC2\gk_tmp\lastjob.vcf
===========================================================
VirtualDub.Open("D:\\DVD\\Rips\\BOB2_VTS_01_PGC2\\gk_tmp\\autobob_movie.avs",0,0);
VirtualDub.video.SetMode(1);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x64697678,0,10000,0);
VirtualDub.video.filters.Clear();
VirtualDub.subset.Delete();
VirtualDub.RemoveInputStreams();
VirtualDub.stream[0].SetSource("D:\\DVD\\Rips\\BOB2_VTS_01_PGC2\\gk_tmp\\AUTOBOB AC3 T03 2_0ch 192Kbps DELAY 0ms.ac3",0x00000203,0);
VirtualDub.stream[0].SetMode(0);
VirtualDub.stream[0].SetInterleave(1,96,96,1,0);
VirtualDub.stream[0].SetClipMode(1,1);
VirtualDub.stream[0].SetConversion(0,0,0,0,0);
VirtualDub.stream[0].SetVolume();
VirtualDub.stream[0].SetCompression();
VirtualDub.SaveAVI("D:\\DVD\\Rips\\BOB2_VTS_01_PGC2\\autobob.avi");

The compression test yielded a 54.83, so AutoGK changed the matrix to H.263 (ESS compatible) and to a softer resizer...it estimated the changes to put the compression at 64.38, but after the first run was actually 58.83....the resulting movie still had bitrate spikes that I described before. This is not at 4000kbps, or even 3300kbps, but at 2200kbps. So you see, it isn't my desire to have extraordinarily high quality....I don't think 58% is too much to ask for. Opening the file information in VDubMod I found that the largest frame size was 109000. I encoded Gladiator with AutoGK...the largest frame size was 119000, but there wasn't any bitrate spiking during playback like the BOB2 movie.

I don't normally use AutoGK, but I wanted to try it to see if the results would be any different and to possibly help me find a solution to this. Could it possibly be that Band of Brothers is a hybrid source? Reading the log file it states that there is 36.49% interlacing in motion areas. That is a bit misleading though as the credits are interlaced.

I know I seem to be talking to myself here, but any help would be greatly appreciated.

[gatormac]

I realize that I've been talking to myself for a couple of days, but unless thats against the rules I guess I will continue as I would like to resolve this issue.

I downloaded DRFanalyzer to see if that would give any more info on the problem. I ran it with the AutoGK file from above:

http://home.hiwaay.net/~dthiot/DRF2.jpg

http://home.hiwaay.net/~dthiot/DRF3.jpg

http://home.hiwaay.net/~dthiot/DRF1.jpg

I have searched and haven't found anything to help me understand what the graph is telling me or how I should take that info to solve my problem.

[gatormac]

I understand what the graph is telling me now, but I still don't have a clue as to what is actually causing the problem nor how to fix it. It isn't just my encoding methods as the same thing happened with relatively conservative settings with AutoGK. I have even tried various Q settings, target bitates, 1-pass, 2-pass, single B-VOPS, 2 B-VOPS, 8 B-VOPS...10,20,30,80,100,10000 B-VOP sensitivity, 0% I-frame boost...10,20,30,40,70% high bitrate scene degredation, default settings across the board....you name it, I've tried it. I've searched these boards back to the middle of 2002 for answers. And you know what, nothing comes close to working. XVID is broken and I don't think anybody knows how to fix it. Nor does anybody seem to care. Everyone is more interested in how cool Soulhunters HD resolution encodes look than whether or not XVID actually works. Even Manono (or Syskin, I can't remember which) has stated that he approaches encoding with the understanding that XVID's bitrate control is broken. I understand some people aren't concerned as XVID is a "computer-based codec", but I believe most people spend their time encoding to backup their DVDs....that they can play on their DVD player .

I am sorry for the jaded post....I have no ill will towards anybody, especially the devs whom I know do this on their own free time. (nor Soulhunter....I think your HD encodes are cool too! Try them on a standalone, however :D ) I am simply nearing the point of bashing my PC and standalone with a hammer, shaving my head and running off to join the Peace Corps.

@ keopi

Is it possible to make XVID compliant, or is the code too far gone to implement it before a total overhaul? Are you guys working on h.264/avc instead of wasting time on a codec thats you consider to be going the way of the Dodo? I only ask you because of your history of developing the XVID curve compression algorithm, which I think has something to do with what I am talking about.

[gatormac]

I'm going for the record for most consecutive posts in a thread :D

I have found something out during my testing...maybe you already know this and maybe its normal, but it seems to be the reason for the problem I am having. The clip that I have been talking about and testing over and over has a high number of bits for certain I-frames. These are whats causing the problems with playback on my standalone. I took Syskins advice and tried to chop off the highest bitrate scenes using high bitrate scene degredation, but it hasn't worked because it only affects P and B frames. The I-frames remain unaltered. Checking the graph in DRFanalyzer shows a definite overall lowering of the overall peaks, but the I-frames remain unchanged leaving the problem intact.

Is there anyway to limit or chopoff the I-frames without having to limit I-frame quants to a low of 6 for an entire movie?

[sysKin]

Originally posted by gatormac
Is there anyway to limit or chopoff the I-frames without having to limit I-frame quants to a low of 6 for an entire movie? Come to think of it, negative i-frame boost would do the job..... but is not possible. Must be a gui problem. Will fix it.

Setting i-frame boost to zero would be the first step - but no, I have no idea what else you might do. Limiting quants (maybe not all the way down to 6, but for example to 4 or 3) would fix that, possibly even without visible quality degradation...

[Zhnujm]

@gatormac:
Do you play these files from DVD or from CD ? With most (all) players you can use higher bitrates if you play it from DVD.

Also, even if it may not be very helpful, if you do a lot of these high bitrate encodings you should think about buying another player.
The Philips uses the ESS chipset wich is by far the worst when it comes to high bitrates.

[gatormac]

@ Syskin

Yeah, I tried putting a negative value in for I-frame boost, but it wouldn't have it.

@ Zhnujm

I play them from DVD and believe me, when there is a player available I will be all over it. I bought this one because it was readily available and cheap. I don't know where you live, but in the USA it is hard to find a standalone MPEG4 player.

I haven't been able to find the chipset or its limitations for this player.

Since this problem seems to require 2-passes to resolve, yet I want to maintain quality throughout I think I might try encoding in 1-pass mode twice. First I will encode in 1-pass at a Q of 2, then open the file up in an XVID analyzer like DRFanalyzer (although one that shows the bits used per frame would be nice...anybody know of one?) to look for trouble spots such as large frames. Then I could take that info, start a second 1-pass encode at a Q of 2, but setting zones to deal with the trouble spots. Its a bit more tedious, but should work and might be the only solution ATM.

Thanks for responding!

[Jackzeripper]

Hello :)

I’ve had exactly the same problem than Gatormac. It would be great for MPEG4 standalone to apply VBV and bitrate limiter (like in poor divx) in XviD1.0 final !

I’ve got « jerks » and « freezes » on my KiSS DP-1000 whith the encoding of « Matrix Reloaded » (on every very complex green scenes, « point of view of the matrix », like the first 2 minutes).

I encode with RC4 (usualy in AS@L5 with H263 or « hvs best pic. » and quant2) with a very simple AVS script at 656*304 @ 1200k (bit/pixel = 0.24). but the very speedy scenes are largely over 5000/6000 kbps for sometimes 5 or 10 seconds (KiSS only support less than 4000k or a bit over but just for 1 or 2 seconds).

I try with AS@L4, I try with quant3, I try with a bitrate modulation « high BR scenes degradation » at 40%. I even try with a zone options encoding @ 0.50 for speed scenes (it’s a mad job !) But the bitrate is still over 5000k with the green scenes and the quality of the remaining scenes is quite bad… And this green scenes are full of P-frames that go from 20k to more than 100k/frame without I- or B-frame (at the difference of DivX encoding using bitrate limiter @ 3800k)

Another time, I’ve got 13.000 kbps on the 2 minutes of a start credit scene very speed and contrasted (but it was with MPEG matrix at quant1)

Otherwise, no problem of oversize or overbitrate with RC3 or 4 even in quant1 with my KiSS standalone ! XviD allowed bitrates higher than DivX on KiSS.
XviD 1.0 is my favorite MPEG4 codec for its sharpness and smoothness of background (no blocks at all !). For its compressibility.

Sorry for my poor english :( and long live to XviD :)

[gatormac]

I have seen in many other threads the same issue.....as MPEG4 standalones become more and more available, more people are having problems with bitrate spikes. Hopefully this will be resolved soon as it is the only issue I have with this awesome codec.

[Jackzeripper]

it's so depressing to use divx and its limiter for these (very few) cases of bitrate spikes...:(

[CruNcher]

@gatormac and others
i agree with Zhnujm try to get a MediaTek based Player they can Play most everything (MPEG-4) and are not bound really to VBV problems i heared the first MediaTek players arived in the US allready :)

Sigma and ESS Chips of the 1st Generation are not really the way to go Chips of the 2nd Generation like the MediaTek and soon the new Zoran ones are, but surely there will be another 3rd Generation following soon.

[Jackzeripper]

a friend had exactly the same problem than me with RELOADED on a Mediatek 1389DE chipset (H&B DX3220), that's seems to be the best standalone...

[Zhnujm]

Did your friend play it from cd or dvd ?
Most Mediatek players have a very poor performance if the files are read from cd.

[Jackzeripper]

on DVD (he burns 3 XviD on each DVD-R). He has got BR spikes over 7000k on the begining of the movie (the green numbers) for 1400k of targer BR.

[gatormac]

I think I'll wait for the 4th generation player as I've heard problems with all standalones, not just ESS or Sigma based sets. If XVID had the VBV my ESS player would be awesome (I still think it is).

[Zhnujm]

Originally posted by Jackzeripper
on DVD (he burns 3 XviD on each DVD-R).

Me too :)
Good luck i dont have problems so far with my elta 8883.
I also have a lot 1 DVD encodes and some 9MBit test files that work perfect but maybe thats because they have a relativ constant high bitrate and not some high spikes.

If you could make a part available for download i could try it.

[Jackzeripper]

sorry i delete the file (and keep DivX with max BR limiter @ 3800 kbps). And it would have been to heavy for my little french connection...:(

[BoNz1]

http://www.xvid.org/modules.php?op=modload&name=phpBB2&file=viewtopic&t=2016 for those who haven't seen this looks like gruel may add VBV support soon but not to 1.0 as there is a feature freeze. Hopefully, he will fix the rest of the rate controller too since it works alright on high bitrates but does quite poorly on lower bitrates.

[gatormac]

Awesome. Thats the news I wanted to hear. Now they just need to release 1.0 final so we can get on with things.

Its been awhile since RC4 has been released....maybe the final is just around the corner (please!).