QuEnc 0.60 Released!

[Guest]

Two points to add to Hank's correct statements:

1. A coded GOP order of IBBPBBP... can still be closed if the leading B frames are coded with macroblocks not using forward prediction.

2. The coded order IBBPBBP... is normally preferred for GOPs because it has a higher compression ratio than IPBBPBB...

[DarkFoon]

@Nic
you asked, and I have delivered: A test showing how different encoders treat the test clip I created yesterday.
They all used the same parameters (as far as I could control):
gop-18
b-frames=2
closed-gop-ON
bitrate-7000
max bitrate-8000
CQM-one I whipped up last night

So here goes(ignore all of the filename except the last part, that's the encoder):

Source
http://rapidshare.de/files/5046118/Random_Clip.zip.html

Quenc
http://rapidshare.de/files/5071994/Robot_Chicken_0.8_0.5_Quenc.m2v.html
size: 7,403kb

HC
http://rapidshare.de/files/5071829/Robot_Chicken_0.8_0.5_HCenc.m2v.html
size: 10,791kb

TMPG
http://rapidshare.de/files/5071900/Robot_Chicken_0.8_0.5_TMPG.m2v.html
size: 8,388kb

(sorry, no CCE. I don't have it)
All filesizes reported by window's "details" view

Attached are the log files from HC and TMPG, if you are interested.

OT:
In light of hank315's last post, I realize that I know nothing :eek:

[video]

DarkFoon no :) an open GOP allways starts with a B-VOP. Only colsed GOPs can be started with I-VOPs

[communist]

Ok I'm a little bit confused as to where the error lies but I *think* its in QuEnc.
I recently tried to encode a DV avi file using a simple script as followed:

AviSource("d:\street.avi")
ConvertToYV12(interlaced=true)

This opens up fine in VirtualDub (1.6.10) and I can even add filters to the script and it will encode the whole file. I'm using the PanasonicDV codec, all other DV avi files work but this one wont encode.

However the same script will crash QuEnc (closes itself) as soon as I hit Encode. When I select separate passes for Audio/Video QuEnc will encode the Audio but not the video. So the Audio seems to be not the problem. When I select No Audio in muxing profile it will only create a 0 byte mpg file but not encode anything. When I select No Video it will create the mpeg file with audio only.
All other options like MPEG1/2, VBR, HQ, Extreme, DC precision, Interlaced etc. dont seem to have any impact at all on the problem.

Using DirectShowSource("d:\street.avi",25) instead of AviSource fixes this problem (QuEnc encodes the file) but all other DV avi files work fine with AviSource.

Here is a 10frame long sniplet from the file that exihibts the same problem.
crash.avi (http://www.stud.uni-goettingen.de/~s304280/D9/crash.avi)

The PanasonicDV codec can be found here (http://www.stud.uni-goettingen.de/~s304280/D9/pdvcodec.zip).

[Nic]

@communist: How very odd! Thanks for the good bug report and test files. It crashes here for me too (I'm at work so can't find out where and why it crashes yet). I'll try and fix it for the next release. Thanks again :)

[communist]

Nice, I was already thinking that the file was cursed by the usual windows works for all except file x. I had already reverted to a older windows installtion from an image file just to find out that it was indeed the file. Good to know its not my setup :)

[dragongodz]

However the same script will crash QuEnc (closes itself) as soon as I hit Encode.
using the MS DV codec it just doesnt encode, 0 size file, rather than crash. atleast for me.
here is another example that works with QuEnc 0.60RC2 fine, just as the 24fps crash clip did, but doesnt with 0.60 and 0.61 release versions. so the change/bug for both these has happened between 0.60RC2 and 0.60 release.

[Nic]

http://nic.dnsalias.com/QuEnc_PrettyExperimental.zip

Just an update for people that are testing these builds as to where I am going with the encoder for the next update.

a rough changelog (since the VeryExperimental build):

1) Drop down codec list (Includes SNOW and MPEG-4)
2) Can output an MP4 (with AAC or MPEG-4 or both)
3) Can output an AVI (with SNOW)
4) Can encode to MP3 and M4A
5) Tweaked rate control for 2 pass
6) Tries to give better options depending on the content of the source (so detects if has video/audio/both and only displays relevant options)
7) Now works on 24 frames per second video (plus others perhaps broken last build)
8) Pulldown should now be working for MPEG-2
9) Should now work on communists problem clip (hopefully! please test)
10) The MP4 that is created is PSP profile. Works on a PSP/K750/W800* as well as it does in something like QuickTime.

-Nic

*As long as you keep within the encoding parameters for those devices

[communist]

Great! My problematic clip no longer crashes QuEnc :)
MP4-Output? That looks pretty cool and already works to some degree as expected.

Also I dont know if you already know it or whether I'm doing something wrong:
When MPEG4 as codec is selected:
QuEnc will crash at encoding start when ("MPEG2 ;)" ) Mux-Profile is set to MP4
QuEnc wont crash when No Audio or No Video is selected, output files are ok.
I tried with several clips but this script will do the same:

ColorBars(720,576)
ChangeFPS(25)
Trim(0,250)
ConvertToYV12()

[DarkFoon]

Great release!
no more 24fps bug.
but I have noticed some interesting things.
It accepts the RGB24 test clip AVI as is (without any conversion to YV12.)
And the output size of the test clip is larger than the previous release.
(8,122 vs. 7,403)
Will the next version fix the problem with compressing the fade to black too much?

[Nic]

@communist: Well spotted. Depends on the resolution of the clip. (it's to stop the user making incompatible PSP MP4 files inside movenc.c). It does not apply here, so i'll remove that limitation and reupload :)
Thanks again!

EDIT:
Ok, reuploaded. Download http://nic.dnsalias.com/QuEnc_PrettyExperimental.zip again and make sure it's the new version by right clicking on the exe, goto properties->version and check it's version 0.6.1.4. That should fix the MP4 bug you were having (although there are plenty more i'm sure ;) )

@Darkfoon: Yup 24 fps fixed. It uses more bitrate on your clip now like it should have done. The black = blocks problem is a tricky one (not only on fades). I'll be trying to fix it soon, but don't really know where to start on that one.
When fed an AVI QuEnc creates a temporary AVS to load the AVI and use YV12 colorspace :)

-Nic

[DarkFoon]

A quick note for anybody who has been using the test clip I provided for testing:
frames 73-76 seem to show the most compression.
In my experimenting with CQM's, I have found that certain ones reduce the ammount of artifacting in those frames.

Hmmm... upon further investigation, it appears to me that the artifacting I'm noticing might be related to the B-frame compensation.
Would any body care to look at this and verify?

@Nic
It seems to me that Quenc removes 1 frame total from my clip. I have 304 frames in the original, and the encoded version has 303. I seem to remember a post about this in one of the earlier versions, but I thought it had been fixed?

[JoeBG]

@ Nic

Great idea to add mp4 output to QuEnc. I always was a QuEnc fan for my mpeg2 Encodes but changed last year to mpeg4.

Testet Version 0.6.1.4. Very Blocky results with 700 kbps. Much better with 1800 kbps (without b-frames) but not in action scenes. Here also very blocky when looking from frame to frame.

The first pass ist as fast as Xvid is on my system in the second pass and the second pass of QuEnc is as fast as the first pass in Xvid => Why is the second QuEnc pass so fast?



In the moment QuEnc produces mpeg4 ASP. Do you plan to add the possibilities for mpeg4 settings?

[dragongodz]

for those interested in testing if having a value for headers makes any difference to the rate control there is a new 0.61 test version at
http://www.pcpages.com/dragongodz2/
you should only compare it to 0.61 release version and NOT the experimental versions. also dont expect much difference as its a just a small hardcoded value. the point is to see if there is any up or down(size/bitrate wise) variance because of it. testing with long clips, expecially ones that undersize with 0.61 release, would be highly desirable.

[Mug Funky]

hmm. CBR mode seems to be b0rk, no sure since what version (i'm using the latest "pretty alpha")

i'm doing 2-pass CBR at 7000, and average quant is about 16, peaking at 22 or so (if you trust bitrateview).

contrast that with a CQ 7 encode that comes out average 3236 kbps with a peak of about 6500.

QLB matrix used on both.

i can provide a test sample in huffy if need be. i'm testing this on a real-world problem sample, because i have a few short captures of the source lying around, and i just figured out how to get them into avs without any colourspace conversion...

btw, 1-pass CBR doesn't seem to suffer this, but appears to hit b-frames a little too hard. (GOP structure is non-negotiable, unfortunately) looks the best so far though, not counting the CQ one.

[Nic]

@MugFunky:
In theory, there is no such thing as 2 pass CBR. But i've heard it mentioned a lot. Could people explain it to me? CBR means constant bitrate. 2 pass is only used to determine how to distribute bitrate, but CBR restricts any sort of distribution.

I can imagine a kind of 2 pass ABR where the range of bitrate is tightly limited. But 2 pass CBR makes no sense to me.

I think i'll turn off 2 pass mode when CBR is selected for the next version.

-Nic

[Mug Funky]

yeah, i thought that too... it could in theory improve matters, but only in a very limited set of cases (and it'd probably suffice to do it per-GOP then).

though i wonder what is happening there - there's a marked difference in how the 2 modes behave.

btw, for CBR encoding, would it make sense to allow b-frames to have the same quants as p-frames? 2-pass mode doesn't limit b-frame quants, which is why i was using it. looks like it backfired :)

nice work with adding mpeg-4 and aac encoding for PSP :)

[dragongodz]

In theory, there is no such thing as 2 pass CBR. But i've heard it mentioned a lot. Could people explain it to me?
if you insist. ;)
you are correct in saying that 2 pass CBR should never really be needed(i have said it myself plenty of times aswell) however the way some encoders work is they they get as close to CBR as possible and pad frames to make the CBR. sometimes however some do not pad correctly, example over padding or not raising a quant so the bitrate actually rises above the target CBR. its usually very small and really of no matter. also if an encoder doesnt pad properly the bitrate can of course also drop below the CBR. now a second pass can come through and correct any such minor mistakes.thus is born 2 pass CBR. :D
also some do not pad at all but just use the closest it can get so you see up and down frame sizes which is really more a constrained/tight ABR. i imagine an encoder could choose to do padding on such streams in a second pass or change the ABR if it thinks its not the most effective.
as i said though it is pretty pointless IMHO as the encoder should be checking the frames before they are written and making any corrections on the spot for CBR.
i can name 1 comercial program by the way that supports it, CCE 2.70. BASIC version supports 2 pass CBR and SP support multi-pass CBR.
http://www.cinemacraft.com/eng/basic.html

now as it relates to QuEnc, actually libavcodecs RC is of course far from great. infact CBR shows this just as good as anything. with one of the previous versions someone was trying to make a vcd or svcd or something and using CBR but got spikes. i told him to do it 2 pass and sure enough it reduced the spiking and gave him a usable file.

btw, for CBR encoding, would it make sense to allow b-frames to have the same quants as p-frames?
thats a RC decision. that is you have to consider that B frames could have a lot more motion vectors and be larger than the P frame at the same quant ,so to get it back down to CBR you would up the quant. :)
now with 1 pass we have set some settings to try and reduce spiking and this does hit the B frames but it is better than the alternative. maybe later we can add mpeg2enc's(modded TM5) 1 pass CBR and VBR RC which should prove much better. or even turn 1 pass VBR in to more of an ABR mode maybe.

[Mug Funky]

ah, okay. but the MVs wouldn't add that much would they?

if you saw this sample all would make sense (you got anywhere big i could upload it to?). if you're familiar with the r4 studio ghibli DVDs, you'll know they're multiangled with a storyboard, and so their max BR is limited to 7000, and their GOP structure is immovable (segment re-encodes on multiangle features is nightmarish... even when all the numbers work out, spruce finds something to complain about).

well, after the encode of Pom Poko got QC'd, there were a couple of bits that just looked awful. every encoder has its limits i guess... so i decided to grab the bad bits uncompressed and use them as "problem samples". i plan to collect a few more of these and use them as generic encoder test clips, rather like the problem samples we all know from audio encoding - applause, castanets, fatboy slim, etc. so far all i have is 2 anime clips of beaver things running through thick scrub. one normal, one fading into a still scene. typically anything with sharp edges, high motion (with no motion blur) and lots of green is going to completely mess up an encoder...

well, anyway, when i tested quenc on these samples, it was the b-frames that were the big problem. everything else was fine, and changing the matrix didn't help much. encoding at constant quant, though dramatically lowering the bitrate, seemed to prevent the bad scenes going really bad. so it would seem that CBR mode could certainly spare a lower quant on the b-frames in these cases, considering that at such a high quant, it must have been padding these frames anyway (CQ 7 looked better and peaked at 6500 after all).

perhaps it's more a b-frame quant ratio thing? when the p-frame quants get a little high, the b-frames turn into a blockfest.

btw, QuEnc is great on everything else, so not to worry :) i can send this sample if need be (it's 65 megs in huffy, considerably bigger in that stupid "blackmagic" uncompressed 8 bit uyvy format though...).

[edit] btw, 1-pass VBR is actually really good... i wouldn't be replacing that unless there was something better to replace it with. hasn't been any spiking since the 0.6 release. also, isn't mpeg2enc's RC macroblock based?

[DarkFoon]

@dgz
I have seen 2-pass CBR in one other place: Windows Media Encoder 9
but that's for WMV.
[edit] I like 2-pass CBR, for the record (meaning that I use it occasionaly)

@Nic
I don't think I have mentioned this, but Quenc is the fastest encoder on my system. :)
It used to peak out at ~11fps with DivX on 352x240(30 fps)
Quenc working on 720x480(24fps) goes up to 15fps!!!!
Keep up the good work!
Will there ever be a constant Q mode that is constrained by max bitrate?

[manolito]

btw, 1-pass VBR is actually really good... i wouldn't be replacing that unless there was something better to replace it with. hasn't been any spiking since the 0.6 release. also, isn't mpeg2enc's RC macroblock based?
Lately I do a lot of AVI2DVD conversions using QuEnc or FreeEnc 0.31 in 1-pass VBR mode, and I am quite happy with the results. (For low bitrates the quality is noticeably better than with CCE 2.67 in 1-pass VBR-RoBa mode). I do have a question though about which formula for rate control equation would be most suitable in 1-pass VBR mode.

In FreeEnc I have experimented with "rc_eq=tex^qComp" and "rc_eq=tex". With "rc_eq=tex^qComp" I had hoped that the amplitude of the bitrate spikes and dips would be somewhat lower, but BitrateViewer tells me that the peaks are about the same. Using "rc_eq=tex" I can tell by looking at FreeEnc's graph that the bitrate does fluctuate more freely, and encoding speed is a little higher. Visually I was never able to put my finger on any differences in quality between the two methods.

Which method does QuEnc use in 1-pass VBR mode? And which of the two methods should theoretically be able to deliver a better rate control pattern in 1-pass VBR mode? And does the "^" really mean "XOR"? (You see I am totally ignorant about FFMPEG / MENCODER).

Cheers
manolito

[GoDuke]

Nic & Co. --

As requested, I've run a couple of tests with both the current Experimental Version (.6.1.4) and the current release version (.61).

Settings:

Avg Bitrate: 7000
VBR
2-Pass
High Quality
Interlaced
Max Bitrate 9500

Input: Simple Script, VHS capture (352x480) 00:01:42 long

RESULTS (Bitrate Viewer):

Version .6.1.4
Avg Bitrate 4414
Peak Bitrate 6714
Avg Q 2.06
Peak Q 5.74

Version .61
Avg Bitrate 6045
Peak Bitrate 8619
Avg Q 1.07
Peak Q 5.64

Comments
I'm not sure why the new version does so much more poorly. Perhaps it has to do with the poor quality input. In either case, the target average of 7000 is missed pretty severely.

Most interesting to me is that both versions severely cut the bitrate in the last 2 seconds or so. This isn't a function of the source. The clip actually cuts off in the middle of a fairly high motion sequence. There is suddenly a lot of blocking in the background with no apparent cause. If it would be helpful, I can post / send the frames in question for comparison.

I plan to run an encode of an entire 20 min segment tonight (using the Experimental Verion). I'll post those results in the morning.

Let me know if any additional information would be of use.

[freelock7]

-Manolito
rc_eq=tex^qComp doesn't change really something in FreeEnc.
Many undeflows reported by ifoedit with freeEnc031 led me to tweak some values in the incredible template to reduce underquantization.
Lmin = 1 and spatial_cplx_masking=0.1 change the bitrate distribution with a better quantization.
But the 031 version uses the QuEnc051 libavcodec known for its bitrate spikes. So, do not set the high bitrate above 7000!
For now, QuEnc062 experimental has a very good VBR1Pass and seems to correct undersize (tested with LADDER_49).

[manolito]

Thanks freelock7,
I already read your post in VMesquita's FreeEnc forum and did some test encodes with the values you proposed for Lmin, Lmax and spatial_cplx_masking. The concept of "lagrange multiplier" is way beyond my math capabilities, but 1-pass VBR tests with your tweaked values came out pretty good. Will do some more testing....

Cheers
manolito

[dragongodz]

if you're familiar with the r4 studio ghibli DVDs, you'll know they're multiangled with a storyboard
did you forget who you are talking to ? ;) of course i know the studio ghibli dvds, i even own some(not all), so yes i know what they are like. :)

you got anywhere big i could upload it to?
how big ?

btw, 1-pass VBR is actually really good... i wouldn't be replacing that unless there was something better to replace it with.
thats cool. there was plenty of problems in the past but maybe its ok now. havent tested it recently. if everyone is happy with it then there is indeed no need to change it.

I have seen 2-pass CBR in one other place: Windows Media Encoder 9
i suspect since that uses techniques from mpeg4 it uses the same type of definition for CBR(more like ABR) which is not exactly the same as mpeg1/2 CBR. as such 2 passes may indeed help it.

GoDuke - it would be nice to see what the test version i mentioned also produced in that test you did aswell. especially compared to QuEnc 0.61 release version.

[DarkFoon]

Encoder test using the same parameters.
Sample 1 (http://rapidshare.de/files/5237333/TMPG-SVCD-QLB-2pass-VBR_2000avg-2400max_.m2v.html)

Sample 2 (http://rapidshare.de/files/5236991/Quenc-SVCD-QLB-2pass-VBR_2000avg-2400max_.m2v.html)

Both have their pro's and con's:
Sample 1 doesn't compress the fade as bad, but has worse overall quality (blocking in almost every motion frame).
Sample 2 has better overall quality (less blocking/the blocking is concentrated in a few frames) but certain luminosities are hit really hard, and have lots of blocking. (last time I chcked, Nic was working on that)

Both samples are MPEG2, closed GOP, QLB matrix, anime, and SVCD (Yes, I know its customary to IVTC anime, but I really don't want to go through the trouble of hand tuning it to look perfect, and its going to end up on TV anyway, so it won't matter.)
I wonder why the Quenc sample has the kind of blocking that Blockbuster() was created to reduce (DCT blocking?) on the color areas, but not the other sample.
I can't wait for 0.7! :D

[GoDuke]

dragongodz:

Your wish is my command . . . :)

Using your experimental version with the same script and parameters as before resulted in much improved rate control:

Avg Bitrate: 6597
Peak Bitrate: 9533
Avg Q: 1.75
Peak Q: 8.07

Interestingly, however, the problem with the last few second of the clip seems even more pronounced. In a low motion, fairly dark scene, the bitrate plummets from 6895 to 696.

EDIT: Apparently, I forgot to post the earlier version on this full 23:34 clip with version .6.1.4. All settings were the same as the shorter clips above:
Avg Bitrate: 4728
Peak Bitrate: 8676
Avg Q: 2.01
Peak Q: 8.07
This full clip ends with a fade out, but the blocking problem is huge here on the last 10 frames or so. At full speed, it's not very noticable, but if you watch it slow, it's really bad.

Hope this helps . . .

[dragongodz]

GoDuke - thank you very much for testing it. hmm bigger difference than i expected.

anyone else intereseted in testing ? not only for reproducable undersizes but also ones where 0.61 release version sized fine, that is does my test version then oversize or is it still close etc.

see i really need multiple tests on different sources etc to know exactly if its more generally good or bad and that only comes from more tests.

[Koepi]

Requested bitrate: 5188 kbps (2pass).

0.61 (Nic): 4100 kbps
0.61+Headers-dgz: 4100 kbps
0.61-without headers-dgz: 3000 kbps

Cheers
Koepi

[dragongodz]

thanks Koepi. how long(time) was the source ?

[Koepi]

It was round about 1 h 40 mins, a full movie encode.

I didn't look into your changes, but if you add a "theoretical" overhead, make it i.e. 1/8 of real frame size for b-frames and maybe 1/4 for p-frames. Just for testing let's assume i-frames aren't scalable, thus entering full frame size as "static" could be ok.

Those values are an average in mpeg4 asp. Maybe they work well for mpeg2, too.
(I just mention this because you wrote you added static overhead to the stats).

Cheers
Koepi

[dragongodz]

well the ideal would be to get the header stats from libavcodec. will have to look back in to passing that through when i get time. ;)

as such though this test version is meant to see if a header value increased the bitrate on undersizing and/or oversized correct sizing encodes. since it isnt a big change i dont expect large variation in general but more of a general direction adding the header takes things. so if a few more people want to test it would be nice. if you are interested in the bits i have found for this so you can see what i am talking about Koepi then PM me. :)

[Mug Funky]

one thing on the undersizing in 2-pass mode: i've noticed that open/closed GOP makes a large difference. the open-GOP 2-pass file (all other settings identical to the closed one) was about half the size of the closed one. strange.

there's no marked difference between open and closed in 1-pass mode (bitrate behaviour is about the same, just a little different).

i'm not sure how reproducible this is on other clips (need more testing), but it could be a place to look :)

[dragongodz]

Nic - please read this thread if you havent already.
http://forum.doom9.org/showthread.php?p=715272

i'm not sure how reproducible this is on other clips (need more testing), but it could be a place to look
i will run some tests(hopefully tomorrow) and see if i can reproduce it aswell.

[dragongodz]

Mug Funky - ok ran a couple of tests with both QuEnc 0.61 release and experimental versions on a 10 minute clip with open and closed GOP. result was closed GOP files a few hundred K bigger. considering the final files sizes were nearly 200MB its not really much, certainly not half the size as you experienced. so i dont know what happened with your clip to have such a major difference. how long was it ?

[Mug Funky]

how long was it?

quite short actually :) about 200 frames. i guess it's not surprising that it'd behave unpredictably on such a short sample, but it's also strange that closed-GOP made such a difference, considering there was no scenechange detection. there would be a slightly higher number of i-frames, but could that explain all the difference?

if there's no difference over long clips i wont worry. IMHO a more pressing concern is getting the last second of reatecontrol to behave nicer.

[DarkFoon]

IMHO a more pressing concern is getting the last second of reatecontrol to behave nicer.
Agreed! :)

[Nic]

Been in NYC (working sadly), so haven't looked at QuEnc. Should get time this week. I'll fix that last second problem (shouldn't be too tricky, I hope ;) )

Thanks for all the feedback :)

-Nic

[freelock7]

-DGZ
Tested your <QuEnc061-testonly> with a 100min movie lenght(2Pass):
set average=5000
Encoding Average =4882
Set Max Bitrate=8200
Bitrate spike=9010
Muxing with IfoEdit= No buffer- under- runs detected

[boombastic]

if i use the /? switch for quenc to have the list of the possible command to use i can't see the way to encode the audio to 48000 Hz sampling rate instead of the default 44.1 KHz,is there a way?

[dragongodz]

freelock7 - thanks for the test. can you also do a comparison against 0.61 release version so i can see the actual difference ?

i can't see the way to encode the audio to 48000 Hz sampling rate instead of the default 44.1 KHz,is there a way?
QuEnc encodes at the rate it is given, feed it 44khz it encodes to 44khz. to change the sample rate with Avisynth read the Avisynth docs that are installed with it.

[freelock7]

Tested another movie
QuEnc-DGZ
Wanted avg=4500
Encoded avg=4010
Global QLevel=1.40

QuEnc061(original)=exactly the same!

Well, strange QLevel with many flat lines at quant=1 (underquantization).
Fine quality picture (sure, for this average!)
Both of them work the same. Did you change something?

[dragongodz]

Both of them work the same. Did you change something?
yes but it is only a small change to try and see if it has a posative or negative impact. on some it seems posative while others negative and others no difference. so i think there are more fundamental changes needing to be made for any major improvement. i will be trying to look at some other changes this week as time permits. thanks again.

oh and for that second test, if you have a lot of Q1 and na average of 1.40 then you are practically saturating the compression. so it would struggle to get up to that bitrate unless it used padding etc which of course doesnt add any quality just size. :)

[DarkFoon]

unless it used padding etc which of course doesnt add any quality just size. :)

That's something that's been puzzling me. A few weeks ago (probably a month or two), it sounded like you were just adding some kind of padding to increase the bitrate, but what is the point of that? Isn't smallest size at the highest quality (possible) the purpose of compression?
No offense, but I can't think of any situations where undersizing would be undesirable.

[dragongodz]

it sounded like you were just adding some kind of padding to increase the bitrate, but what is the point of that?
no, not adding padding. the things i have been changing are to do with bit distribution and rate control calculations.

No offense, but I can't think of any situations where undersizing would be undesirable.
when the rate control has failed to distribute as much bitrate as possible. for the above example where the average quant is around 1 then you are practically at saturation point and wont get much more in either size or quality. however say the average quant had been more like 4 or 5 and you were still under your target bitrate. that suggests that there should be areas where a lower quant me be able to be used, thus raising bitrate towards target and ,atleast in theory, increasing quality. of course the distribution still has to adhere to any restrictions such as max bitrate and VBV etc.

[DarkFoon]

I stand corrected.

By the way, how is max bitrate determined by quenc/libavc? I remember come confusion on the subject a while back.
I always thought it was the number of bits used to encode 30 frames (NTSC) or 25 frames (PAL). And when a new frame was encoded, the oldest frame in the comparison group (the 30 or 25 frames used to determine max bitrate) was dumped, and the newest frame could have at most as many bits that would make the group total the max bitrate.

hmmm, but that wouldn't work too well... if there were 29 frames of 0 bits(unlikely), the 30th frame could by 8.5 megabits in size (assuming max bitrate=8500) which would totally mess up set top playback devices.

[Mug Funky]

of course, if you're getting an undersized file at average quant 1, then you can squeeze a few more bits with a higher bitrate matrix - the numbers in the quant matrix behave the same as the global quantizer - lower numbers = more bits = more detail.

of course, some stuff undersizes no matter what... i encoded "son of godzilla" with a flat 8 matrix (all 8's for intra and non intra) and it still came out at almost constant quant 1. pre-compression filtering can often be too efficient (maybe these movies were mastered for divx rather than DVD?)

[DarkFoon]

@Mug Funky
That's an interesting idea. My MPEG-2 capture card uses different quant matrices. I'm not sure what causes it to change, but when I run it through DGDecode with the "show quant matrices" option, I get a large file with about 10 to 20 matrices.

I'd be kinda neat if a user could specify a directory with a bunch of matrices, and quenc could switch to another matrix if it maxes out on quant 1 with the first one the user specifies. Or the user could specify 4 or 5 matrices for quenc to choose among for highest quality/smallest size, depending on which the user prioritizes.

[Mug Funky]

i'd love it for CBR stuff. it'd be easier to implement i think - if quant=1 and bitrate < target, then multiply matrix by some calculated number (based on compressibility stuff i have no idea about, and also a ratio of current bitrate and target bitrate), then round to integers. this wont affect the "shape" of the matrix much (rounding excluded), but will allow quant 1 to keep more detail.

[dragongodz]

I always thought it was the number of bits used to encode 30 frames (NTSC) or 25 frames (PAL). And when a new frame was encoded, the oldest frame in the comparison group (the 30 or 25 frames used to determine max bitrate) was dumped, and the newest frame could have at most as many bits that would make the group total the max bitrate.
correct. though not all rate controls will go through all that. some will just go by GOP and target that to the relevant percentage per frames per max bitrate etc. Nics latest experimental versions is infact trying to do it the way you describe. ;)

hmmm, but that wouldn't work too well... if there were 29 frames of 0 bits(unlikely), the 30th frame could by 8.5 megabits in size (assuming max bitrate=8500) which would totally mess up set top playback devices.
actually no it shouldnt. that is if the dvd player has that little dvd symbol on it then it is meant to be able to handle such a case as that seeing as it is technically legal. ok reality may show that not all players would handle it great but reality should also show that it would have to be an extreme case, and i mean really extreme and rare, for an encoder to ever produce such an encode. so its not actually something you should need to worry about.

I'd be kinda neat if a user could specify a directory with a bunch of matrices, and quenc could switch to another matrix if it maxes out on quant 1 with the first one the user specifies. Or the user could specify 4 or 5 matrices for quenc to choose among for highest quality/smallest size, depending on which the user prioritizes.
well yes that is possible but it would be much simpler to simply adapt the chosen matrix if needed. that is if after 1st pass at Q2 the size/bitrate estimated is smaller than half the target size/bitrate then reduce/halve the values in the matrix(limiting to 8 and 16 values for intra and non-intra). then adapt the GOP/frame sizes from the first pass to a rough(you would need another real first pass to be actually accurate) new value from what the new matrix should produce. then encode second pass based on that. want to guess how much fiddling around that would be ? ;)

i'd love it for CBR stuff. it'd be easier to implement i think
i agree, for CBR it should be easier to implant aswell. still fiddley though.