FreeEnc: New High Quality MPEG2 Encoder

[vmesquita]

Hi Everyone,

I just created a new libavcodec based encoder. This encoder takes AVS input and outputs MPEG2 like QuEnc, but the main difference is that every parameter is tweakable, plus it comes with the optimized parameters of MencodeMe, which are known to produce great quality. Download here:

http://www.downloadcounter.com/cgi-bin/download.pl?username=vmesquita&account=5

This encoder is also compatible with QuEnc command-line options, meaning it can replace QuEnc in any app that suppots it. Just rename the executable to QuEnc.exe and overwrite.

Enjoy! :D

EDIT:
0.2 is out
changes:
- Fixed the P4 crashes by disabling SSE2 code in avcodec. Finally! Thanks Mean from doom9 for pointing the problem.
- It's now possible to use different .ini templates
- new command-line option "-template". Usage: -template FreeEnc (do not put the .ini in the end, it's added automatically)
- New Status bar that shows if it's in first or second pass (to avoid confusion)
- Added Incredible KDVD alpha settings

[Paced]

Great to see yet another new tool, thanks. But, after a few tests, I have found that it doesn't respect the average bitrate (I'm not sure why) - I originally set it at 2113, but the output (according to VirtualDub / BitrateView) was reported to be approximately 1300.

[vmesquita]

Hi Paced,

This is knows as the "can't quantisize more" libavcodec feature. The encoder simply is already using the lowest quatisizer possible in every area of the video.
You may want to tweak the parameters scplx_mask and lmin to get higher bitrates, since the encoder comes tweaked to lower bitrates (you can find it in freeenc.ini)
In the old mencoder forum, you'll find tons of info and tests about libavcodec:
http://kvcd.net/forum/viewforum.php?f=81

[JDay]

Well, I tried an avs from DVD-RB that gave me the error "could not locate libavcodec mpeg2" (or something like that), and "could not open avs" when I click go again. Here's the avs in question:
#------------------
# AVS File Created by DVD Rebuilder
# VOBID:01, CELLID:01
#------------------
LoadPlugin("C:\DVD Backup\DVD-RB\MPEG2Dec3dg.dll")
mpeg2source("C:\DVDS\KATAKURI\AUDIO_TS\D2VAVS\V05.D2V",idct=7)
trim(0,6443)
converttoyv12()

This was actualy output for cce, I just added the "converttoyv12()" (works fine in QuEnc). Also, its 16:9, interlaced, 23.976fps.

[vmesquita]

Very wierd, this error should not happen. But I don't understand this:

"Also, its 16:9, interlaced, 23.976fps."

Interlaced 23.976 fps? Maybe this is creating some trouble. :confused:

[JDay]

It's a hybrid source. A lot of it is progressive, but RB encodes the whole thing as interlaced, and sets the flags on rebuild. I still seem to get the error when I try to encode as progressive, however. I tried it again, with some different settings (1 pass, progressive) and got the error "could not read from avs line 6". Line 6 is the "converttoyv12()" line (my avs changed a little since my last post, switched to dgdecode- still works with QuEnc, though). After that, I get the could not open avs error, and the program crashes when I cose it.

[JDay]

I also did some tests, just changing the size and fps of blankclip(). Often, I would get errors, and it would not encode or it would get through the first pass then give an error at the beginning of the second. Also, some clips that I did encode had missing frames (300 frames->297 frames). This was a problem in QuEnc, that Nic was able to solve. You might want to check into what the issue was there.

Edit: I had the most trouble with 720x480 23.976fps content. I was able to encode 29.97fps ok (for blankclip(), anyway).

[vmesquita]

Well, looks like the initial error is related to this 23.976 interlaced source. I'll test later with this.

But there's something really wierd here: messages regarding specifically the avisynth script like this "could not read from avs line 6" are generated by avisynth itself, FreeEnc just detects that Windows failed to open the script video source and gets the error message from AVISynth.

The crash must be because I forgot to free the pointer in case of error... My mistake, I'll fix later.

EDIT: Indeed, 23.976 interlaced causes the error message, I just tested.
EDIT2: Even 29.97 interlaced crashes. I must have enabled a wrong flag.
EDIT3: Yes, I did enable the wrong flag, just fixed. Now let's see the missing frames problem.

[JDay]

Don't know why I got that avs error. Same script works in QuEnc, VirtualDub, MPC, etc (although I only got the error in FreeEnc once). Surely that's a problem on my end, rather than in your program.

[Amnon82]

vmesquita my friend. I'll try your encoder. Good work so far.:D

[vmesquita]

Ok, 0.12 is out fixing the interlaced problem. :)

[yaz]

@vmesquita
great stuff ! thx a lot ! i've just made some short tests but seems to work. quality is fairly acceptable. however, i've got some questions. afais, the 'default' setting is that optimized for kvcd implementations (MencodeMe, PackShot, ... ) what if i wanted to use quite different settings ? so ...
- is it necessary to set all parameters in the ini file or is there some embedded defaults ? say, is there a default matrix or should i always set it in the ini file ?
- are these the only available parameters or can i add more ? say, i do prefer trellis optimization, is there a way of using/testing it with FreeEnc (or is it disabled) ?
- if there's some more options, is there a list of the opts available ?
- FreeEnc does not seem to accept bitrate settings strictly, i always got some deviation. is it a well known problem of libavcodec. have u improved this part too or just take it as it is ?
- ... :-)))

thx again for your efforts
y

[Amnon82]

I had some troubles. It crashed always. Lets see if the new version solved the chrashing problems with PAL movies ... Good work! Go on

[vmesquita]

afais, the 'default' setting is that optimized for kvcd implementations (MencodeMe, PackShot, ... )

Actually they are optimized for low bitrate. :D

what if i wanted to use quite different settings ? so ...

Be my guest, that's the whole design difference from QuEnc.

- is it necessary to set all parameters in the ini file or is there some embedded defaults ? say, is there a default matrix or should i always set it in the ini file ?

Everything must be set in the ini. Do not remove any line or the parameter you removed will go to zero, causing crazy results.

- are these the only available parameters or can i add more ? say, i do prefer trellis optimization, is there a way of using/testing it with FreeEnc (or is it disabled) ?

Every libavcodec parameter is there! But the name of it may be a little different from mencoder implementation. Trellis for instance is trell. You will find it disabled (trell=0), enable if you want.

- if there's some more options, is there a list of the opts available ?

All libavcodec options are tweakable via INI, only the name may be slightly different from mencoder notation. Teh defaults are set to mencoder defaults, except for the parameters with were optimized.

- FreeEnc does not seem to accept bitrate settings strictly, i always got some deviation. is it a well known problem of libavcodec. have u improved this part too or just take it as it is ?

It's just as it is, but:
The default settings are optimized for low bitrate, so it tend to produce undersize when you ask for higher bitrate, like 3000, 4000, etc (depends on source compressibility, of course). If you prefer high bitrate, tweak lmin and scplx_mask. You can find some info on this on the now locked Mencoder KVCD forum.

[yaz]

@vmesquita
thx for the quick reply. weeks (months?) ago i'd made a very thorough test of mencoder. i'm about to brush that up w/FreeEnc. i'm curious what of those findings will work here & what not. hmmm ... i hope i still have the same sources.
thx again
y

[lithoc]

I've happen to download Quenc & FreeEnc.
I've notice that both have avcodec.dll
Is this is the same?
Can I have Quenc & FreeEnc running on the same avcodec.dll?

I've run some test and found out that by using Quenc dll file.
The video size is bigger & lower quality.
Can anyone confirm this?

[vmesquita]

I compiled my avcodec.dll using the latest CVS, linking only the needed .o (i.e., I didn't include h263 and other unneeded codecs). I don't know how Nic created his and what version he used.

[Paced]

Originally posted by vmesquita
Hi Paced,

This is knows as the "can't quantisize more" libavcodec feature. The encoder simply is already using the lowest quatisizer possible in every area of the video.
You may want to tweak the parameters scplx_mask and lmin to get higher bitrates, since the encoder comes tweaked to lower bitrates (you can find it in freeenc.ini)
In the old mencoder forum, you'll find tons of info and tests about libavcodec:
http://kvcd.net/forum/viewforum.php?f=81

Thanks for the reply, much appreciated.

On another note, I too seem to be getting constant crashes (after the first pass) with FreeEnc (even with the latest version). I've found it happens when the Trim() command is used; it happens with SelectRangeEvery(1200,12) too. Not sure if this will help or not, but I have not been able to encode one PAL clip with the program. I have however, been able to encode some NTSC clips.

[dragongodz]

my compile for QuEnc 0.53 is CVS from the 1st of August(my time). i didnt disable any codecs from normal compile. Nics compile from 0.51 is from before that was released(of course) and he said that he uses an old version of gcc(dont know exact version).

so basically mine and vmesquita should give similar results while Nics dll may not.

[lithoc]

I've problem with crashing too.

After it crash once, it keep on crashing whenever I run the program & encode again.
I cannot confirm this but the problem goes away after I delete the files in prefetch folders in windows xp.

[vmesquita]

@dragongodz
Thanks for the info. :)

@Paced
I did some tests here using trim and PAL sources and still, no crash...
:(

@lithoc
I don't know how the prefetch thing can be related.

@everyone with crashing problems
I was thinking that maybe there can be a DLL conflict, so maybe it would be worth temporarily removing all avcodec.dll copies (including the ones from QuEnc), reboot the system and try again.

I did a build of the DLL using the latest CVS, downloaded completelly (I have used CVS update previously), keeping all codecs and using no upx. Please do the solution above, and if still doesn't work, try this DLL:
<edit>removed old link</edit>

EDIT:
Looks like QuEnc 0.52 does not work in my system: it keeps increasing the ammount of used memory till there's no more memory left. QuEnc 0.51 still works fine. Maybe there's a DLL conflict, indeed.

[vmesquita]

Just to make sure the DLL conflicts are out of the picture, I did a static build (does not use the DLL). Get it here:
<edit>removed old link</removed>

I can provie freeenc.o in case anyone wants to link with a different version of the library, as the LGPL requires. Let me know if this fixes the problem.

[Paced]

Hey, it sucks to be reporting this, but the static version still 'crashes' on my system :(. Basically, it does the first pass with no problems, but before starting the second pass, the program just suddenly closes itself - a majority of the time there is no error prompt/pop-up, etc. (it just closes).

//Edit

Now that I think about it, the only time I've seen an error window/pop-up (when the program quits itself after the first pass), is whenever I use SelectEvery(xx) in the script.

[vmesquita]

@Paced
Could you post a sample script that crashes on the start of the second pass?

[Paced]

Originally posted by vmesquita
@Paced
Could you post a sample script that crashes on the start of the second pass?

Sure, I'll give you two:

#------------------
# AVS File Created by DVD Rebuilder
# VOBID:01, CELLID:01
#------------------
mpeg2source("C:\WUTEMP\TEST\D2VAVS\V01.D2V")
trim(0,1649)
ConvertToYV12()
AudioDub(BlankClip())


and..


AVISource("C:\AVIHere.avi")
LanczosResize(720, 448, 19, 0, 602, 352)
AddBorders(0, 64, 0, 64)
ConvertToYV12()

[vmesquita]

@Paced
Thanks, this script did not crash on me but was making the encoder exit after the second pass, and I fixed.

@all
0.13 is out! Changelog in the first message. I have a hope that I might have fixed the problem (there were two leaking problems in the code), but don't get too happy, since this never happened to me, I can't garantee anything.

[Encoder Master]

Very nice VMESQUITA.:D

[Paced]

Originally posted by vmesquita
@Paced
Thanks, this script did not crash on me but was making the encoder exit after the second pass, and I fixed.

@all
0.13 is out! Changelog in the first message. I have a hope that I might have fixed the problem (there were two leaking problems in the code), but don't get too happy, since this never happened to me, I can't garantee anything.

Yay for vmesquita! It works now :D

[vmesquita]

Originally posted by Paced
Yay for vmesquita! It works now :D
It's great to hear that! :D

[Paced]

Oh wait, I spoke too soon :( I didn't notice it at first, but it still doesn't start the second pass for me. Now, instead of quitting the program after the first pass, it just acts like it has finished encoding; but, the problem is, it has only done one pass. At least it's not quitting anymore :D Or, the only other reason I can come up with is: did you change the way the progress bar works? Because in your very first version, I remember that the progress bar went through twice in the entire process (one for first pass, and two for the second pass). Or does it only go through once, but half = one pass, and the second half = 2 pass ? I hope that made sense.

[vmesquita]

How do you know it has not started the second pass? You see, when you select two pass, the progress bars shows both the first and second pass, so when it reaches 50%, means that it has finished the first pass and it's starting the second. You can notice that until about 50%, the filesize doesn't grow (stays in 0kb).

EDIT: Looks like you got it before I finished writtng my reply. :D

[Paced]

Originally posted by vmesquita

EDIT: Looks like you got it before I finished writtng my reply. :D

Hehe, too fast ;) Thanks for the clarification, and the work you've put in.

[dragongodz]

Looks like QuEnc 0.52 does not work in my system: it keeps increasing the ammount of used memory till there's no more memory left.
do you mean QuEnc using the dll that comes with it ? if yes then what cpu and windows are you running ?

[Wilbert]

I just created a new libavcodec based encoder. This encoder takes AVS input and outputs MPEG2 like QuEnc, but the main difference is that every parameter is tweakable, plus it comes with the optimized parameters of MencodeMe, which are known to produce great quality.
I guess you forgot to mention the most important difference?

[vmesquita]

@dragongodz
I am running Windows XP SP1 on an AtlhonXP 2000+, 512 Mb. I'll do some additional tests to confirm it's still hapening, altrought on the day I reported, it was happening in every single try, using default avcodec.dll.

@Wilbert
I guess this part of my explanation you highlightned is the most important difference. There are others, like the plataform used (I used MinGW+WxWidgets, QuEnc was developed in Visual C++). So could please point what is "the most important difference" in your oppinion?

@all
After some test at KVCD.NET while this forum was down, we came to the conclusion that the crashes are happening with some P4 processors, right after FreeEnc calls avcodec routine that encodes a frame. What is interesting is that the crash doesn't happen in the first frame encoded, but generally in the third. Also it's interesting that the old avcodec.dll from QuEnc works with FreeEnc in this "crashing machines" but give bad quality. :confused:
My last idea was try to cross-compile from cygwin (like Nic did in the past) but I still could not make it work... :( In theory it should produce the same results of a straight compilation in MSYS, but how knows... :rolleyes: I'll keep trying.

[dragongodz]

vmesquita - hmm i am running XP on an athlon 2400+ with 256mb ram and dont get that problem. also i havent seen anyone else mentioning the ram being eaten on other forums either. so very strange.

the old avcodec.dll from QuEnc works with FreeEnc in this "crashing machines" but give bad quality
i asume you mean Nics dll. as i said that is an older compile so output may not be the same as more up to date cvs. it shouldnt output really bad quality though just slightly different. maybe you could ask some of those people that have tried to also test with my dll and see if that does the same. if it also gives bad quality then that will be really strange, since mine is much more up to date.

[mean]

There is still some alignment issue with sse2 code on win32
The following worked for me (also using mingw) :

look in MPV_common_init_mmx and comment out

if(mm_flags & MM_SSE2){
s->dct_quantize= dct_quantize_SSE2;
} else

[Wilbert]

@Wilbert
I guess this part of my explanation you highlightned is the most important difference. There are others, like the plataform used (I used MinGW+WxWidgets, QuEnc was developed in Visual C++). So could please point what is "the most important difference" in your oppinion?
Will the source be available one day?

[vmesquita]

Originally posted by dragongodz
vmesquita - hmm i am running XP on an athlon 2400+ with 256mb ram and dont get that problem. also i havent seen anyone else mentioning the ram being eaten on other forums either. so very strange.

I just tested with 0.53 and it keeps increasing the memory usage till there's no memory left yet. But a important data I forgot to mention: I am using a DivX source to test. Anyway I'll test today at work (where I have another AtlohXP :D ) and report back.

i asume you mean Nics dll. as i said that is an older compile so output may not be the same as more up to date cvs. it shouldnt output really bad quality though just slightly different. maybe you could ask some of those people that have tried to also test with my dll and see if that does the same. if it also gives bad quality then that will be really strange, since mine is much more up to date.

Yes, I was talking about Nic's DLL. I'll ask people to test with your compile. :)
Originally posted by mean
There is still some alignment issue with sse2 code on win32
The following worked for me (also using mingw) :

Thanks! You're right, this makes a lot of sense! Actually I think I'll go further and disable SSE2 from CPU detection (cputest.c) on, just rule out SSE2 problems of the picture.

Originally posted by Wilbert
Will the source be available one day?

Probably it won't. I don't think that's "the most important difference".
:rolleyes:

[vmesquita]

Hi everyone,

Following mean's advice, I compiled a special version of avcodec.dll with SSE2 completelly disabled. Replace the one that comes with FreeEnc and let me know if it fixes the crashes. :)

http://www.jltoca.uaivip.com.br/files/avcodec_no_sse2.zip

[dragongodz]

I just tested with 0.53 and it keeps increasing the memory usage till there's no memory left yet. But a important data I forgot to mention: I am using a DivX source to test
hmm well divx source shouldnt make any difference really. are you using avisource() or directshowsource() ?
this could be a problem aswell
http://forum.doom9.org/showthread.php?threadid=47893
but its strange how it doesnt on all peoples machines etc.

oh and you could also try eliminating any filters from your script just to make sure they do not have anything to do with it.

[vmesquita]

dragongodz,

I'll post about it in QuEnc 0.53 thread. ;)

[vmesquita]

Hi everyone

O.2 is out! Check the changelog and get it in the first post. :)

[Amnon82]

You solved the crashing-problem on my P4. Sorry I didn't told You that I also use a P4 ...

Can You explain me how the predictionmethod (prediction_method=0) works?

[vmesquita]

Hi Amnom,

I'm glad it's now working for you. I think this parameter doesn't do much with mpeg encoding. From ffmpeg code

/**
* prediction method (needed for huffyuv).
* - encoding: set by user
* - decoding: unused
*/
int prediction_method;
#define FF_PRED_LEFT 0
#define FF_PRED_PLANE 1
#define FF_PRED_MEDIAN 2

[Amnon82]

I coding a addon for Your encoder. Still have one Problem:

If I wrote this into the batchfile:

FreeEnc.exe /i D:\testfreeenc\matrix2_pred2.avs /o D:\testfreeenc\mat.mpv /1 /aspectratio 16:9 /nointerlaced /b 3550 /maxbitrate 7550 /auto /close

It works... but if I do this into the batch:

FreeEnc.exe /i D:\testfreeenc\matrix2_pred2.avs /o D:\testfreeenc\mat.mpv /1 /aspectratio 16:9 /nointerlaced /b 3550 /maxbitrate 7550 /template CYDVD1_1Alpha /auto /close

It won't ...

I readed also at KVCD.net about it ... I'm using the newest version of Your encoder (0.21)

Here the link to my app ESD 0.1 (http://www.brckomania.net/CYBERNETSOFTWARE/FILES/ESVCDDVD.exe)

[Inc]

@ Avalon

/template "CYDVD1_1Alpha" ... as I do remember

Before investing that much time in an add on, have a look at V 0.3

[Amnon82]

@Incredible

Nice graph ...
The problem with the Templates is still there!
Why stopping the addon? It will be like CQMatic for FreeEnc ...

@vmesquita

Here is the fullworking Batchfile:

@ECHO off
ECHO Batchfile created with EasySVCD/DVD - visit: www.cydvd.tk
ECHO Start encoding 25 percent of movie for prediction
E:\sources\ESVCDDVD\Encoder\FreeEnc.exe /i D:\testfreeenc\drunk1.avs /o D:\testfreeenc\drunk1.m2v /b 6811 /maxbitrate 7550 /1 /interlaced /aspectratio 4:3 /auto /close
exit

And this don't... :

@ECHO off
ECHO Batchfile created with EasySVCD/DVD - visit: www.cydvd.tk
ECHO Start encoding 25 percent of movie for prediction
E:\sources\ESVCDDVD\Encoder\FreeEnc.exe /i D:\testfreeenc\drunk1.avs /o D:\testfreeenc\drunk1.m2v /b 6811 /maxbitrate 7550 /1 /interlaced /aspectratio 4:3 /template CYDVD1_1Alpha /auto /close
exit

When I tip on the Combobox in your encoder and choose again my template it works. Can You post me a working Batchfile?

[Inc]

Why stopping the addon? It will be like CQMatic for FreeEnc

a) Nobody told to stop but to look first ...

b) In libavcodec its not possible to do a prediction like when using CQmatic. I know u used the slicer() function in DRPenc appl. to do a prediction based 1pass VBR but many and many tests on libavcodec based binaries did show that a "known" prediction is not possible. You cant set minrate and 2pass is obligatory if results should look good. The quantizer curve at 1pass behaves crazy and will only be set optimal finally at the 2nd pass.

[Amnon82]

I do now a test. I'll take the "chase-scene" of Matrix2.
Lets see if I get the size I want ... even in 1pass.

[freelock7]

@VMesquita
__________

Great job with this new libavcodec engine.
FreeEncoder 3.0 works perfectly with CYDVD matrix with a great quality and flexibility!
thanks!

[vmesquita]

Hi freelock7,

It's nice to know that! :D :)

[Amnon82]

Great to hear that You like my Matrix.

[freelock7]

Fantastic at low bitrate (VBR:2500).
Never seen this quality with CCE original settings.

[snek_one]

wow, i have installed DIKO to convert some avi files to dvd's for friends.. i quickly ran a test using a 4 minute clip from the avi's to see what setting would be best for me..

now, i was greatly surprised to find out that freeenc @ 2pass was FASTER than cce @ 2pass!!
it actually made a full minute difference..
cce always does an extra pass to determine the quality?
now maybe if i would do a full 3 hours worth of film cce might actually beat freeenc again, i guess..

next to that the freeenc file was smaller and actually looked better??

i don't know how this happened, but if this is due to your freeenc, i am greatly impressed!

many thanks for this great encoder!
you have helped the community more than i think you can imagine!

system:
p4 530 (3GHz socket 775 / 1MB Cache) / Asus P5GDC-V / 2x256MB DDR333 CAS2.5 (my old ram, will upgrade to 4x512MB ddr400 double twin set soon) / windows xp sp2 with .net 1.1 / 1x120GB UDMA133 8MB Cache / 1x80MB UDMA133 8MB Cache / 1x40GB UDMA100 2MB Cache / NEC 2510A Dual Layer burner

[Guest]

Hi,
I really like this enc - it's much more stable for me than quenc and nuenc.

But I'm also having a problem calling templates in the commandline - the template shows up nice in the gui but when pressing "Go!" it says:"could not load libavcodec MPEG 2".

When selecting the template manually there's no problem - just when calling templates in the commandline I get this error.

Tin2tin

[Peter Cheat]

It might help if you post your commandline. It might shed some light on the situation.


I really like this enc - it's much more stable for me than quenc

Since FreeEnc and QuEnc are based on the same library, I doubt there is a difference in stability. (Maybe its the older avcodec FreeEnc is using?)

[Guest]

It's the same commandline problem as Amnon82 described above.

I don't know why but nuenc and quenc keeps exting without rendering - without any logic in when and what I'm doing wrong. I a p4 user and tried with the mem dll - but it's the same. Freeenc doesn't shut down like that ever for me.

:confused:
Tin2tin

[Ebobtron]

This encoder works well for me. Both Free and Qu ran in my Win98se machine, but QuEnc is very odd in WinXP.

Is there a referece for the params in the templates, I cann't make out most of them and if I could I wouldn't understand most either.

Looked over the ffmpeg docs but found little I was able to cross between the two

Thanks,

PS Thanks for the great work....

[freelock7]

@VMesquita
----------
1)How the setting of linear (or non-linear) quantizer scale value can be modified to produce the best value?
2)Can I change the DCT prcision value from 8 to 10 in the .ini file?

[freelock7]

No response?
So, I recall my question :
The Qlevel with a mpeg2 file encoded by CCE is more accurate (>6)than a file encoded with the same bitrate with Freeenc (under 3).
My DVD player seems to read with difficulty (not everytime) a video encoded by FreeEnc.
So, is it possible to adjust the QLevel in FreEnc to have more accurate results?

[Peter1234]

freelock7,
How are you measuring Q? I think that when using BitRate Viewer a lower Q number is better. Q is generally improved by increasing the video data rate. In my tests, FreeEnc produces a better Q than QuEnc for equal data rates and same script. I assume QuEnc will improve in the future since it is still in the development stage.

[Peter Cheat]

I think the problem you are experiencing is due to underflow (causes jerky playback). If you are doing 2-pass, NuEnc almost completely solves this issue. Its not a problem with the matrix or the peak bitrate (its a problem with maximum bitrate (not peak) - this is not reported by any commercially available tools AFAIK).

[freelock7]

Yes, I know the problem with the "scene change" option in QuEnc -you have resolved in NuEnc-but I like the flexible . ini file of FreeEnc witch bring me a lot of parameters to change (ffmpeg script). I'm working on it to find more accurate results.

[Peter Cheat]

Its not a "scene change" problem. Its a VBV compliance problem. The symptoms you have outlined seem to correspond to this common problem which I've tried to fix.

You can try to reduce the problem in FreeEnc. The parameter to modify is rc_buffer_aggressivity. Try 0.5 or maybe even smaller. But this doesn't help very much :(

Oh, and you can't compare quantisers from FreeEnc and CCE directly. FreeEnc uses linear quantisation and CCE uses non-linear. Its like trying to compare a car with a top speed of 200mph and a car with a top speed of 320kph and saying the car with a top speed of 320kph is faster. You have to convert units first!

[freelock7]

Thank you Peter to help me to configurate FreeEnc. I appreciate.
About Quantizer scale, It is right that CCE2.67 use a non linear scale but CCE2.5 use the linear scale in option(used by NuEnc, QuEnc &FreeEnc). So, you can compare easily CCE and FreeEnc.
To change quantizer scale setting in FreeEnc, <qcompress>, <Qmin>, <Qmax>& <qscale q> can be adjusted.
In correlation, VBV works well with the Qscale:
"Rate control attempts to adjust bit allocation if there is significant difference between the target bits (anticipated bits) and actual coded bits for a block of data. If the virtual buffer begins to overflow, the macroblock quantization step size is increased, resulting in a smaller yield of coded bits in subsequent macroblocks. Likewise, if underflow begins, the step size is decreased. The Test Model approximates that the target picture has spatially uniform distribution of bits. This is a safe approximation since spatial activity and perceived quantization noise are almost inversely proportional.
From:http://www.mpeg2.de/doc/mpegfaq/mpeg2.htm
So, it is not stupid to think that the Quantizer scale(1) can influence the VBV.
_____________
(1)QScale:"this is the quantization scale value that has a trade-off between quality and compression. Using different Q-scale values has a significant affect on amount of compression you get. The Q-scale values can be set for I, P, and B-frames separately. The scale can be from 1 to 31. Larger numbers will result in better compression but at the expense of worse quality. The image.param has the Q-scales all set to 1, giving the best quality/worse compression. Through a series of tests, increasing the Q-scale affected the amount of compression the most."
(Mpeg optimizations by Eric Young)

[Peter1234]

freelock7,
If you have not already done so, you may want to look at the FreeEnc forum on the KVCD website.
http://kvcd.net/forum
It has more information about FreeEnc.

[freelock7]

Yes but you have to pay for it! I do not agree !

[Asmodeus]

Go to www.vmesquita.com , creator site of FreeENC. It is still in development.

[Peter1234]

freelock7,
you wrote: "Yes but you have to pay for it! I do not agree !"

No. You do not have to pay, KVCD forum is free, but you do have to register.

[dragongodz]

No. You do not have to pay, KVCD forum is free, but you do have to register.
no. you do have to pay first. to quote from the kvcd main pages news(16th October)

Forum access now requires a very small "One Time" fee.

[Peter1234]

dragongodz,
Thanks for pointing that out. I was not aware that it was no longer an open forum. My apologizes to freelock7.

[Peter Cheat]

@freelock7
Increasing the minimum quantiser will reduce the probability of underflow. But it will reduce the quality also. The concept is to only increase the quantiser when necessary (to prevent underflow)keeping quality to a maximum, and to achieve a target bitrate. avcodec only checks for underflow once, and only during initialisation (before encoding). My current version checks 30 times (I've reduced this for next release for faster encoding) trying to make sure it won't happen (it still does, but very rarely).

Beware, some of the options in FreeEnc do not even apply to MPEG 1 or 2. Some info about what the options do is here at ffmpeg.org (http://ffmpeg.sourceforge.net/ffmpeg-doc.html#SEC8).

I wouldn't use my modified DLL with FreeEnc, I've made so many changes that it won't be compatible (it might encode, but the result will be poor). My next version of NuEnc will allow the minimum bitrate to be set.

You can't really compare quantisers even if they are both linear by looking at an average. It is easy to make a very low quantiser average by using very low quantisers for easy to encode scenes, and then using high quantisers on hard to encode scenes (to reduce bitrate). The average quantiser will be lower, but the quality would be very poor (this is done in CBR encoding). The best way to compare is to look at how different segments are quantised and decide which encoder made the better decision (do this visually).

And it does suck that KVCD now requires a fee, but I guess it does cost time and money to keep a board like that going. My only complaint is that it's just not very active.

[fozzieb]

the forum for FreeEnc and diko are now on www.vmesquita.com

and it is free.

[freelock7]

Here a good tool for freeEnc improvement:
http://forum.doom9.org/showthread.php?s=&threadid=80931
Thanks Amnon82 (for the matrix inside)!

[freelock7]

To correct the libavcodec RC underflow problem in FreeEnc, I'm using a template with a 448kb VBV size.
It's experimental. So,it has to be used with caution knowing the DVD compliance is 224kb(1).
It could be an alternate solution at low bitrate only.
This template can be downloaded here:
http://reinkinator.home.t-link.de/MDVD.ini
Thanks to MoleVCD.de
_________________________________
(1)This value specifies the size of the decoders "Video Buffering Verifier(2)". It represents the amount of coded video data that can be buffered by the decoder. At constant bitrates the buffer allows best use of the MPEG compression techniques. It is filled at a constant rate (that of the video stream) and partially emptied when a frame is decoded. The buffer fills up during sequences that compress well and empties during sequences that do not compress well.

Usual values are 40 for producing an MPEG-1 VCD , 112 for an MPEG-2 SVCD and 224 for DVD. These are the minimum allowed by the specifications, but if you are encoding for a particular player it may have a larger VBV buffer, so these could be increased (3).
_________________________________________________
(2)MPEG concept defined in ISO/IEC 13818-2 (MPEG-2, Annex C)
which employs a fixed size buffer to handle the transition of
the channel bit rate to the rapidly flucuating coded bit rate of
individual MPEG pictures. The scope of the VBV is only within a sequence.
The VBV is built upon a framework of several axioms of decoder behavious
which are unfortunately not very well described in the spec.
These include:

display pictures have known durations, but the time to download
the corresponding coded bits into the VBV varies. This time is
dependent on the bitrate of the delivery and the coded size of
the picture being delivered to the VBV. For example, I frames
will take an average of 4 picture periods to download, whereas
B pictures will take half an average picture period.

all coded pictures are not equal in size. Ratios of 15:5:2
are common for I, P, and B pictures respectively.

a picture is decoded instantenously in the VBV model, yet a real-world
decoder will account for the time it takes to decode a picture by
increasing the size of the VBV buffer.

a B picture is displayed almost as soon as it is decoded.
An I or P picture is not displayed until the next I or P picture
is decoded. This is known as the reorder delay.

the first vbv_delay of a sequence is arbitrary, but is usually
kept large. From there on, subsequent vbv_delay values are
directly related to the coded size of a pictures and the interval of
the picture being displayed while a picture is being decoded.

_______________________________________________
(3)"Changing the VBV buffer size will not affect quality in any way. The amount of bitate used by your video is set in stone during the encoding process by your AVG, MIN, and MAX values (or just AVG for CBR). It can not be changed on the fly by some magical setting to make your video better. The VBV curcuit controls the encoded stream to the encoder, ensuring it always outputs a constant stream for decoding to the encoder. It does this by throttling back on the mpeg input, if the decoder falls behind (say during a complex scene), or when it gets ahead (think 'On Golden Pond'). I used the funnel visual, because it works the best. Think of the VBV circuit as the person pooring the water. It maintains a constant amount of 'water' in the upper part of the funnel. No matter how big you make the funnel, you still get the same water out. It's not purified in the process.

Increasing the VBV buffer size would in essense give you a bigger 'funnel'. The problem here is that a larger funnel will increase the amount of data your dvd chipset must decode. If it is expecting a maximum amount of data for the medium its working on. It can choke up. Pushing more data doesn't make it look better. It just makes the chipset work harder to decode the info it's given(...)."
DJRumpy, VideoHelp Forum, july 2003.
Detailed VBV explanation:
http://petercheat.host.sk/libav/maxbitrate-truth.html

[Peter1234]

freelock7,
I compared your MDVD.ini with Incredible_KDVD.ini and found that your ini gave slightly better Q values at low bit rates with everything else the same. Very nice. Thanks.

Incredible_KDVD.ini: average kbps = 873, average Q = 10.13
MDVD.ini: average kbps = 869, average Q = 9.85

[Inc]

The less Q in average results from the MVCD Matrix. Reinkinator is a member of the MVCD Forum where Roc and I did assemble and release the Templates for TmpgEnc incl. that Matrix which was done by ours. So the Matrix is ported to the ini File for Freenc.

Also a very good try is the Bach01 Matrix from Ddog

[freelock7]

Thanks Incredible !
I'll test DDog matrix too.

Caracteristics of the BACH1 matrix:
http://forum.doom9.org/showthread.php?s=&threadid=60276&highlight=kvcdnotch

Other matrix
http://forum.doom9.org/showthread.php?s=&threadid=67975&highlight=ddog+matrix
@Peter1234
Did you see some reading problem by your DVD player with the MDVD matrix?

[Peter1234]

@incredible,
Thanks for the suggestion. Bach01 Matrix did give better q value.

Incredible_KDVD.ini: average kbps = 873, average Q = 10.13
MDVD.ini: average kbps = 869, average Q = 9.85
Bach01.ini: average kbps = 868, average Q = 9.26

Bach01 matrix
intra_matrix=8,16,19,22,26,27,29,34,16,16,22,24,27,29,34,37,19,22,26,27,29,34,34,38,22,22,26,27,29,34,37,40,22,26,27,29,32,35,40,48,26,27,29,32,35,40,48,58,26,27,29,34,38,46,56,69,27,29,35,38,46,56,69,83
inter_matrix=16,18,20,22,24,26,28,30,18,20,22,24,26,28,30,32,20,22,24,26,28,30,32,34,22,24,26,30,32,32,34,36,24,26,28,32,34,34,36,38,26,28,30,32,34,36,38,40,28,30,32,34,36,38,42,42,30,32,34,36,38,40,42,44

EDIT: DDogg called this matrix Bach1 not Bach01. I got the name Bach01 from Incredible post. DDogg original post is at http://forum.doom9.org/showthread.php?s=&threadid=60276&highlight=kvcdnotch

@freelock7,
I encoded using your ini (vbv=224) and made SVCD using VCDEasy and it played on DVD player but the last few seconds of video did not play. I also generated DVD files using DVDauthorGUI to verify that it would work (showed Vbv buffer size : 458752 bytes ) and the resulting files played in WinDVD. My experience is that a lower vbv works better. I normally use vbv=80 for SVCD.

[Peter Cheat]

@freelock7
Using 448KB as a buffer size to reduce underflow problems is crazy. You need to reduce the buffer size to reduce underflows on standalones, not increase it. If a DVD player only has a 224KB buffer, and you encode a stream that requires a 448KB buffer, it will not play correctly in some parts.

[freelock7]

True. I realised that IfoEdit shows a lot of DTS errors (data arrive too late) during muxing process.
It was not really a good idea to avoid underflow.
Thanks for advice.
The problem comes from the quantisation level I try to fix.
Here the quantisation Mencoder command line:
1)<q_min> at 2 increase the minimum quantisation but it decrease the average prediction.
2)<q_blur> may be increased for a better result (0-1).
"1.0 average the quantizer over all previous frames
larger values will average the quantizer more over time so that
it will be changed slower."
(Based on mplayers DOCS/tech/libavc-option)
3)<mb_qmin> (minimum macroblock quantizer) at 2 can help.
_________________________________
One finds the truth while seeking

[Yuri]

Great work!!!
Unfortunatelly, my encoding became 3 frames shorter that the input file. Don't know why, but the 3-4-5 frames at the beginning were somehow omitted.

[freelock7]

Caused by old FreeEnc0.3 libavcodec.
You can download the new modified libavcodec here:

FreeEnc0.31 is out: http://www.vmesquita.com/en/download.htm

[Yuri]

I'm afraid I had been using this very version which omitted frames. BTW, could NuEnc's dll interfere somehow, though it's in a seperate folder?

Best regards

[freelock7]

FreeEnc encodes from avs script.
Compare with QuEnc. They works in the same way.
There is not interference between them if you use a separate folder.

[fluffysnurgle]

I have downloaded the new(?) FreeEnc 0.41 but I have a problem with it.

In the past using version 0.31, using the "default" matrix, the encoding would ignore bitrate settings and always encode at around 1000kbps. But the problem went away by using the Incredible_KDVD matrix.

Now, however, the "Incredible" matrix is the default, and the low bitrate problem occurs with it also. So the new version is not usable.

[freelock7]

Please, post your question in the new FreeEnc0.41 thread