QuEnc 0.54 released

[Nic]

? The difference in encoding time should be non existant with different dc precision...basically, higher dc precision will take up slightly more bits but can make for a higher quality picture. (This is my understanding, i may be talking cr*p ;) )

-Nic

[Kika]

The DC-Precision affects the way, the DCT/Quantisation works.
With 10 or 9 Bit you can expect a better quality, yes, but you will also need slighty more Bitrate.
9 Bit DC is always a good choice for Movies.
For my self, i only use 10 Bit for my private Videos (from Hi8), if i plan to put it in highest possible Quality onto DVD (at Max. Bitrate).

[Mug Funky]

doesn't DC precision only affect the very first coefficient (like the "base colour" for the block)? if that's the case, then it would make almost negligible difference to quality and filesize.

or have i got it wrong? damn mysteries.

[dragongodz]

"intra_dc_precision: the MPEG-1 DC value is mandatory quantized to a
precision of 8 bits. MPEG-2 introduced 9, 10, and 11 bit precision set
on a picture basis to increase the accuracy of the DC component, which
by very nature, has the most significant contribution towards picture
quality. Particularly useful at high bit rates to reduce
posterization. Main and Simple Profiles are limited to 8, 9, or 10 bits
of precision. The 4:2:2 High Profile, which is geared towards higher
bitrate applications (up to 50 Mbits/sec), permits all values (up to 11
bits)."

from

http://www.faqs.org/faqs/mpeg-faq/part3/

:D

[Kika]

doesn't DC precision only affect the very first coefficient

No, it affects all elements of the Matrix.
While DCT, all elements are being roundet, at higher precision, you will get lot's more of different results. That lowers the risc of getting Rainbow-Effects but needs more Bitrate.

That all is becoming very complicated, if you want to use a special Matrix for quantisation. And if the bitrate isn't high enough, you may get LOWER quality at 10 Bit...

[Mug Funky]

hmmm.... come to think of it there's nowhere to set the field-order in the GUI. i'm assuming it's TFF by default (as most mpeg-2 is).

[dragongodz]

well i just had a quick look at ffmpeg.c etc. ok theres a couple of flags that can be set, CODEC_FLAG_INTERLACED_DCT and CODEC_FLAG_INTERLACED_ME. there does appear to be a top field first option aswell where you can set top, bottom or automatic.

just some more stuff for Nic to look at when he gets time. :)

[freelock7]

Strange MPEG2 results after encoding a movie at 4000 VBR:
On my PC, the movie works fine and on my DVD player it shows a lot of ugly pixels.
What does it means?

[dragongodz]

is it bad the whole way or just in parts ?
how did you author the dvd ?
are the pixels there when played by your pc from the burnt dvd ?

could be anything from a bad encode to bad authoring to a bad dvdr.

[freelock7]

the hole part.
I'm using DGindex for the .d2v file and the avs works well with CCE on my DVD player.
I'll test 0.51 to compare.
DC precision is fixed at 11.
Maybe the wrong choice?

[dragongodz]

DC precision is fixed at 11
GAK! read what i said above about what DC precision can be used for what. 11 is for HighProfile@ only. DO NOT use it for dvd etc. the max for dvd is 10.

[freelock7]

sorry!
It's fine.
The picture is very accurate without blocs on the edges-with asharp(2,5).
Now, it's hard to see a difference between 0.54 and CCE!

Settings:
-VBR>3500(*)
-High quality
-2Pass
-treillis quant
-interlaced
-detect scene change(**)
-limit high bitrate:7500(***)
--------------
(*)Advice: Not under for best performance
(**)seems to be necessary for better results in high speed motion picture.
(***)More precise & stable than 0.51.

[Amnon82]

@dragongodz: I added support for QuEnc 0.54 to ESD 0.6. The only Problem is this:

the options: -cmatrix and -nocmatrix doesn't work. Can You tell me the right strings?

[RobertR]

Originally posted by Amnon82
@dragongodz: I added support for QuEnc 0.54 to ESD 0.6. The only Problem is this:

the options: -cmatrix and -nocmatrix doesn't work. Can You tell me the right strings?
It's hidden somewhere in this thread :)
Try -qlb and -noqlb

[Amnon82]

I try this. Thx.

Also I found out that -scene and -noscene won't work!

K -qlb and -noqlb works. But if I check -qlb it shows QuEnc quick and disappear again. Nothing happend.

This happens also when I used the avcodec.dll compiled by Peter Cheat.

[Edit] With the avcodec-mem.dll it works. There is still the -scene/-noscene matter.

@nic and dragongodz: When do you guys add a custom matrix support. I for example want to test my own matrix with quenc for example. In FreeEnc it works realy good. I tested in the newest ESD build the FreeEnc with my CYDVD 2.0EX Settings against the Settings of 0.54 using the Custom Matrix. In QuEnc I got a higher avg.Bitrate during prediction but not so a good Quality with FreeEnc using a lower avg.Bitrate to get the same size.

[dragongodz]

yes mentioned when i released 0.54 i missed a couple of changes to for the cmtrix commands to work. so the qlb commands were still active. not a big issue since jdobbs had just released a new version of DVD-RB using the qlb commands. :)

-scene and -noscene won't work!
hmm thats strange. i will test that out later and see whats up.

When do you guys add a custom matrix support
i personally do not like to say when things will and wont happen. otherwise people get expectations and they start to ask "when?" as you just have. so it will happen when it happens. :D

a lower avg.Bitrate to get the same size
you lost me here. if you have 2 encoded files that are the same size they cant have a different average bitrate.

[Amnon82]

a lower avg.Bitrate to get the same size
you lost me here. if you have 2 encoded files that are the same size they cant have a different average bitrate.

My program ESD is a Prediction Routine for QuEnc and FreeEnc.
I tested the same Movie with both Encoder.

First I encoded the Movie in FreeEnc and got a file with the size of 798,56 MB and a Sample of 9,86 MB. The Quality with a avg.Bitrate of 751 was good.

Then I encoded the same Movie with QuEnc with a avg.Bitrate of 981 and got also a Sample of 9,86 MB and a 797,56 MB big file with less quality as the one encoded with FreeEnc.

The avg.Bitrates are calculated by ESD. In the QuEnc-Mode it takes always higher Bitrates than in FreeEnc. I don't know why. The Result is always the same. This is realy strange.

Only thing I did was using my settings in FreeEnc (CyDVD 2.0 EX).
In QuEnc I only can use the KVCD Matrix or the buildin one.

If I can change the Matrix, maybe it will get the same Quality.
I used for both encodes the avcodec.dll You called avcodec-mem.

[dragongodz]

ah you mean avg. bitrate you enter. i meant the real average bitrate of the produced mpegs. the same size means they should be the same bitrate(or close since the example you gave they are not exactly the same).

QuEnc uses the standard matrix or custom(QLB, Quenc Lower Bitrate)) matrix. kvcd matrix was replaced as of QuEnc 0.53.

simply replacing the matrix will not change things to make them the same. you would also have to use other settings that QuEnc uses to reduce bitrate spiking. this is no doubt where the quality loss comes from aswell.

i did a quick test to use as an example. i did a short clip(1500frames), 720x576 ,25fps, avg bitrate 3000, max bitrate 4000.
freeenc using default matrix(which is notch i believe) 10.2mb, pulsing and bad to be honest.
freeenc using incredible matrix(had to delete default matrix file for this to actually work), 20.5mb, quality ok.
QuEnc no custom matrix, 20.5mb, quality ok.

now when viewed with bitrate viewer it shows another important detail. freeenc(incredible) spiked to 4786 bitrate while QuEnc max remained just below 4000. so during those spikes freeenc will look better since its using more bitrate for those frames. QuEnc however is trying to stay below the max bitrate(no it wont 100% of the time but it does try :) ).

now when you are making a video for vcd/svcd/dvd that max bitrate is rather important. there are set standards for them and if not adhered to then some hardware players can have problems playing them. thats why QuEnc's settings are the way they are.

so how does this relate to the avg. bitrate you enter. hmm well its possible that the spiking is actually taking the bitrate higher than what you enter so you have to enter a lower value to begin with.

it will be nice when Peter finishes the new rate control and settings dont have to be messed with to try and force avcodec to do what it is meant to. :D

[dragongodz]

ok just tested the -scene and -noscene commandline options. both worked fine. i checked the advanced options and it was changed both times correctly. i then checked the produced files and again they were correct.

[Amnon82]

I retested it and got this:

http://www.brckomania.net/CYBERNETSOFTWARE/images/quesd.png

And here the created File by ESD 0.61:

@ECHO off
ECHO Batchfile created with EasySVCD/DVD 0.61 - visit: www.cydvd.tk
ECHO Start encoding Movie Fullencode Pass
C:\Programme\ESD\Encoder\QuEnc.exe -i "D:\esd\test.avs" -o "D:\esd\test.m2v" -b 3550 -maxbitrate 7550 -dc 10 -1 -mpeg2 -hq -vbr -scene -trell -cgop -nointerlaced -noextreme -gopsize 15 -maxbframes 2 -qlb -aspectratio 16:9 -auto -close
exit

So what is wrong?

[dragongodz]

dont know. i tested from the command line.

i have downloaded ESD and will try to test it sometime tommorow. will let you know if i can repeat it once i have.

[lamster]

Originally posted by dragongodz (to "DVD Rebuilder > QuEnc encoding completed in 0 minutes!")
well its basically the sse2 parts of avcodec that are the problem. you either have to disable sse2 or use the memalign hack to make it work properly with a P4. you would think one of the FFMpeg guys would have tracked down the problem. apparently though its not meant to be so bad under linux so they havent.

I understand that the problem is that an exception is occurring in avcodec.dll, because the structure isn't properly aligned, and the movdqa instruction generates a protection exception if the memory address isn't 16-byte aligned. (I had reported my experience at http://forum.doom9.org/showthread.php?s=&postid=539053#post539053 )

The fix is to either ensure that the address is properly aligned, or use movdqu instead of movdqa.

My point was that the symptom reported in the other thread was that QuEnc was only doing one pass, and nobody (but you) had a clue as to what was going on. If you were to wrap the calls to avcodec.dll (possibly just the call to avcodec_encode_video) in a try / catch(...) block, you could catch the exception and display a message saying that avcodec wasn't behaving nicely. (If you wanted to get fancy, you could add a structured exception handler and see exactly what the exception was, to give a better error message - "Dude - you're running on a P4 and getting a Priviliged instruction exception - RTFM !!!", and log some diagnostic information, but that's not really necessary.)

[dragongodz]

Amnon82 - ok did some quick testing. the problem is closed gop and scene detection, they do not work together, try selecting both in the QuEnc gui. so its either 1 or the other. try turning off closed gop in ESD and you will see scene change works fine from command line.
i also noticed you dont put quotes "" around the path to the ESD exe. if installed in a drawer with a space in the name the bat wont work.

lamster - yes i understand but since all versions should use the memaligned compile from now on is there a real need ? the only problems i have seen reported have been fixed with that.

[lamster]

If QuEnc will be shipping with the memaligned avcodec.dll, then I agree that it's less likely that people will run into it.

[Amnon82]

THX for the replay dragongodz.

[Peter Cheat]

@dragongodz
There is a mistake in the code for setting aspect ratio. In file "AVSEnc.cpp" the line:

c->sample_aspect_ratio = av_d2q(frame_aspect_ratio,255);

should actually be:

c->sample_aspect_ratio = av_d2q(frame_aspect_ratio*c->height/c->width,255);

This fixes the aspect ratio problems. (Look in FFMPEG source, this is how it is done in there).

[dragongodz]

This fixes the aspect ratio problems.
that was already fixed on 0.54. what version of the source code are you looking at ? in 0.54 it has

c->sample_aspect_ratio = av_d2q(frame_aspect_ratio*sdh/sdw,255);

its interesting though that 0.51 didnt need the height and width. it however used a much older checkout of FFMpeg.

[Trahald]

How hard would it be to add support for Chapter setting (ie forced I-frame insertions)? (I was whipping up something in c++ to use quenc with big3)

[dragongodz]

i remember there was some asking about I frame insertion on the FFMpeg mailing list. went nowhere. so dont expect to see it soon.

[darkluna]

How difficult would it be to save the 1st pass info file to a specific directory so that you could run (or re-run) the 2nd-pass separately at a later time? This would be a very handy feature.

Thanks for all the work

[zilog jones]

Someone probably noticed before how QuEnc encodes TFF interlaced video as BFF, but is there any way of fixing this? Is there just some byte in the file header that needs be changed or what? I'm fimiliar with hex editors if it is something like that.

[Mug Funky]

a little program called "ReStream" may be what you're looking for.

i think it's just a stream flag thing, but it may not be, so don't hit me if it doesn't help you (i do a lot of testing, but not many actual encodes, and as such haven't tried interlaced QuEnc on my teev yet).

[Peter Cheat]

Originally posted by dragongodz
that was already fixed on 0.54. what version of the source code are you looking at ? in 0.54 it has


I thought I was using 0.54 source. Now I am not sure.

@Mug Funky
If it is only a flag that has to be changed/set, then it can be fixed in QuEnc easily.

[dragongodz]

I thought I was using 0.54 source. Now I am not sure.
it also says QLB matrix in the gui etc so i am guessing its actually 0.53 you may be using. :)

If it is only a flag that has to be changed/set, then it can be fixed in QuEnc easily.
yes a page or so back its already mentioned that there is an option in FFMpeg for TFF and BFF. so yes thats another thing on the list for the next version.

How difficult would it be to save the 1st pass info file to a specific directory so that you could run (or re-run) the 2nd-pass separately at a later time? This would be a very handy feature.
yes quite possible to do but its questionable of how much use it would be. ok you can encode passes at different times but on the same hand if you change anything between passes(such as in the avs) it makes the first pass pointless. i dont think its a priority to me.

[Mug Funky]

@ dragongodz: would it be beneficial to save the first pass, and on the 2nd and 3rd passes do something tantamount to a "rejig", that is re-encode using the motion vectors from the 1st pass? this would make "fast 2nd pass" REALLY fast.

or do the vectors change subtly in 2nd and 3rd passes?

[dragongodz]

hmm well it would need proper testing but seeing as the first pass is effectivly a constant quant encode i dont know how reliable they would be for using for the second pass, the motion vectors that is. however from second pass to third ? that may be worth investigating how much MVs change or not.

just as a note about re-using MVs. i notice that you can write an MV file with divx's first pass and read it for nth passes BUT thats only for standard mode. for slow and slowest that function is disabled and not ticked. kind of suggests it doesnt give the best results/quality.

so unless people are interested in a sub-optimal/lower quality quick encode its probably best not to re-use MVs from the first pass.

[Mug Funky]

mm. okay.

does this mean MVs are calculated after quantization? i thought they were calculated using the source picture (at least, the CCE manual says this, but i suppose libavcodec is entitled to do it differently).

because if MVs are calculated off the source, then re-use seems logical, but of course, it's quite possible the best rate/quality is achieved by calculating them after quantization.

i don't know enough about MPEG-2 :scared:

[dragongodz]

motion vector decisions are based on pre-encoded frames compared to the current frame. that is how much like is the previous I or P frame in the GOP to the frame we want to encode(actually this is all done by macroblock but i am just trying to simplify it). now with first pass encoding everything at say quant 2 you are going to get pretty good block matching. however when you do the second pass you are doing VBR and the quants will change as things like the rate control require. so a motion vector based on a P frame that was quant 2 first pass may not be ideal if the same P frame is encoded at quant 10 second pass. a frame referencing it may no longer be a very good match in other words.
hope i explained that well enough(i doubt it but that should give you some idea). :)

oh and i just installed divx 5.2 today(to check a problem a freind was having) and i notice it no longer has the reuse MVs. 5.1 did but i guess they decided it wasnt worth keeping. i have the feeling the same is true for QuEnc aswell.

[darkluna]

I was looking for the option to do the 2nd-pass separately, as a way of interrupting the process. My encodes take a long time to complete, and there have been times when I have wanted to stop after pass 1, do something else, and then pick up pass 2 at a later time

[Peter Cheat]

Going back a few posts...

Originally posted by Trahald
How hard would it be to add support for Chapter setting (ie forced I-frame insertions)? (I was whipping up something in c++ to use quenc with big3)

Actually, this isn't hard to do at all. It is easy to force FFMPEG to use any frame type, for any frame you want. If you explain exactly how you want to forced I frame insertion to work, I can check it out. (ie do you just want to be able to specify the frame number of a frame you want to turn into an I frame?)

The code to force an I Frame would just look like:

...(some condition)...
s->pict_type = I_TYPE;
...


Too easy. It's probably possible to force it externally with QuEnc.

About MVs, dragongodz is right. The MV decision will most likely change for second pass as P references I and B references P. If the quality (quantiser) of an I frame changes, MVs change for P, and pass on to B. You could do it, but it would most likely come out far below optimal.

[RobertR]

Originally posted by Peter Cheat
Actually, this isn't hard to do at all. It is easy to force FFMPEG to use any frame type, for any frame you want. If you explain exactly how you want to forced I frame insertion to work, I can check it out. (ie do you just want to be able to specify the frame number of a frame you want to turn into an I frame?)

The code to force an I Frame would just look like:

...(some condition)...
s->pict_type = I_TYPE;
...


I'm assuming that it would be good to follow CCE behaviour on that part too. With CCE it's just matter of supplying list of frame numbers. (this would also help very much in my simple private project :D )
I've done some more testing of your changes to libavcodec/ffmpeg (please remember that i don't use windows). Unfortunately i was too stupid and let ffmpeg overwrite files from passes 1 to 3 (so i have no evidence of that spikes i wrote ealier). I did 7 passess of same VOB ID.
4th and 5th pass are almost equal in size (difference is about 500KB) and have avg bitrate just below targeted 3500kbit/s.
Strangely at 6th and 7th pass both bitrate and size went up (for 6th pass diff in size is around 1MB, for 7th pass - 30MB - which is 10% of file).
In all above cases q level in BitrateViewer is almost flat (fluctuances (?) are very small). In NuEnc at some pass it almost followed bitrate line (4th pass i think). I can;t reproduce this here. I'll post the pictures from bitrateviewer later and will edit this post to add url. Today and tomorow i'll try to do same tests with patched ffmpeg 0.49.pre1 (the same version that your patches are based).

EDIT:
Ouch! Big big sorry! I just spotted that i wrote about patched libavcodec in wrong thread :scared:
If i can be forgive just this time i'll leave my comments here. If not please let me know and i'll remove them and repost in proper thread.

[Guest]

Hi,
Any plans of adding sound to your great tool? avs -> m2v+m2a?
That would be super cool :)
Tin2tin

[Mug Funky]

dragongodz and peter cheat: thanks for the info. believe it or not, i understood it perfectly well. i suppose if it were that easy, everyone would be doing it :)

i eagerly await the next version :)

[DMagic1]

Can someone tell me how to solve this "AVS File is not outputting
(Use ConvertToYV12() at end of script" error with RB+QuEnc.

I saw someone else taking about the same problem but it did really get answered. ConvertToYV12() is in my script.

[Peter Cheat]

Originally posted by Guest
Hi,
Any plans of adding sound to your great tool? avs -> m2v+m2a?
That would be super cool :)
Tin2tin

I could write audio encoding and muxing into the code. If I have the time, I might do that and pass the sources onto dragongodz, if he is interested. Libavcodec can encode mp2 audio already, and it can mux, however, not to a system stream but a program stream. I don't imagine it would be hard to change though.

[dragongodz]

Nic decided not to use the mp2 encoding from avcodec because it is very basic and not the best quality. using TooLame would provide better.

[Trahald]

Originally posted by Peter Cheat
Going back a few posts...
Actually, this isn't hard to do at all. It is easy to force FFMPEG to use any frame type, for any frame you want. If you explain exactly how you want to forced I frame insertion to work, I can check it out. (ie do you just want to be able to specify the frame number of a frame you want to turn into an I frame?)

I would be greatly appreciative if possible.. As robert said, reading from a list file of the frame #s would work. oh.. and btw.. it would also have to be a new GOP, not just an i-frame ;)

[DMagic1]

Originally posted by DMagic1
Can someone tell me how to solve this "AVS File is not outputting
(Use ConvertToYV12() at end of script" error with RB+QuEnc.

I saw someone else taking about the same problem but it did really get answered. ConvertToYV12() is in my script.

Anybody?....

[dragongodz]

did you try playing the avs to see if it reports an error ?

also how many frames is it ? QuEnc wont handle smaller than 3 frames for 2 pass for example.

[DMagic1]

Its not small and it plays fine. CCE handels the script fine also.