FreeEnc: New High Quality MPEG2 Encoder

[freelock7]

@VMesquita
__________

Great job with this new libavcodec engine.
FreeEncoder 3.0 works perfectly with CYDVD matrix with a great quality and flexibility!
thanks!

[vmesquita]

Hi freelock7,

It's nice to know that! :D :)

[Amnon82]

Great to hear that You like my Matrix.

[freelock7]

Fantastic at low bitrate (VBR:2500).
Never seen this quality with CCE original settings.

[snek_one]

wow, i have installed DIKO to convert some avi files to dvd's for friends.. i quickly ran a test using a 4 minute clip from the avi's to see what setting would be best for me..

now, i was greatly surprised to find out that freeenc @ 2pass was FASTER than cce @ 2pass!!
it actually made a full minute difference..
cce always does an extra pass to determine the quality?
now maybe if i would do a full 3 hours worth of film cce might actually beat freeenc again, i guess..

next to that the freeenc file was smaller and actually looked better??

i don't know how this happened, but if this is due to your freeenc, i am greatly impressed!

many thanks for this great encoder!
you have helped the community more than i think you can imagine!

system:
p4 530 (3GHz socket 775 / 1MB Cache) / Asus P5GDC-V / 2x256MB DDR333 CAS2.5 (my old ram, will upgrade to 4x512MB ddr400 double twin set soon) / windows xp sp2 with .net 1.1 / 1x120GB UDMA133 8MB Cache / 1x80MB UDMA133 8MB Cache / 1x40GB UDMA100 2MB Cache / NEC 2510A Dual Layer burner

[Guest]

Hi,
I really like this enc - it's much more stable for me than quenc and nuenc.

But I'm also having a problem calling templates in the commandline - the template shows up nice in the gui but when pressing "Go!" it says:"could not load libavcodec MPEG 2".

When selecting the template manually there's no problem - just when calling templates in the commandline I get this error.

Tin2tin

[Peter Cheat]

It might help if you post your commandline. It might shed some light on the situation.


I really like this enc - it's much more stable for me than quenc

Since FreeEnc and QuEnc are based on the same library, I doubt there is a difference in stability. (Maybe its the older avcodec FreeEnc is using?)

[Guest]

It's the same commandline problem as Amnon82 described above.

I don't know why but nuenc and quenc keeps exting without rendering - without any logic in when and what I'm doing wrong. I a p4 user and tried with the mem dll - but it's the same. Freeenc doesn't shut down like that ever for me.

:confused:
Tin2tin

[Ebobtron]

This encoder works well for me. Both Free and Qu ran in my Win98se machine, but QuEnc is very odd in WinXP.

Is there a referece for the params in the templates, I cann't make out most of them and if I could I wouldn't understand most either.

Looked over the ffmpeg docs but found little I was able to cross between the two

Thanks,

PS Thanks for the great work....

[freelock7]

@VMesquita
----------
1)How the setting of linear (or non-linear) quantizer scale value can be modified to produce the best value?
2)Can I change the DCT prcision value from 8 to 10 in the .ini file?

[freelock7]

No response?
So, I recall my question :
The Qlevel with a mpeg2 file encoded by CCE is more accurate (>6)than a file encoded with the same bitrate with Freeenc (under 3).
My DVD player seems to read with difficulty (not everytime) a video encoded by FreeEnc.
So, is it possible to adjust the QLevel in FreEnc to have more accurate results?

[Peter1234]

freelock7,
How are you measuring Q? I think that when using BitRate Viewer a lower Q number is better. Q is generally improved by increasing the video data rate. In my tests, FreeEnc produces a better Q than QuEnc for equal data rates and same script. I assume QuEnc will improve in the future since it is still in the development stage.

[Peter Cheat]

I think the problem you are experiencing is due to underflow (causes jerky playback). If you are doing 2-pass, NuEnc almost completely solves this issue. Its not a problem with the matrix or the peak bitrate (its a problem with maximum bitrate (not peak) - this is not reported by any commercially available tools AFAIK).

[freelock7]

Yes, I know the problem with the "scene change" option in QuEnc -you have resolved in NuEnc-but I like the flexible . ini file of FreeEnc witch bring me a lot of parameters to change (ffmpeg script). I'm working on it to find more accurate results.

[Peter Cheat]

Its not a "scene change" problem. Its a VBV compliance problem. The symptoms you have outlined seem to correspond to this common problem which I've tried to fix.

You can try to reduce the problem in FreeEnc. The parameter to modify is rc_buffer_aggressivity. Try 0.5 or maybe even smaller. But this doesn't help very much :(

Oh, and you can't compare quantisers from FreeEnc and CCE directly. FreeEnc uses linear quantisation and CCE uses non-linear. Its like trying to compare a car with a top speed of 200mph and a car with a top speed of 320kph and saying the car with a top speed of 320kph is faster. You have to convert units first!

[freelock7]

Thank you Peter to help me to configurate FreeEnc. I appreciate.
About Quantizer scale, It is right that CCE2.67 use a non linear scale but CCE2.5 use the linear scale in option(used by NuEnc, QuEnc &FreeEnc). So, you can compare easily CCE and FreeEnc.
To change quantizer scale setting in FreeEnc, <qcompress>, <Qmin>, <Qmax>& <qscale q> can be adjusted.
In correlation, VBV works well with the Qscale:
"Rate control attempts to adjust bit allocation if there is significant difference between the target bits (anticipated bits) and actual coded bits for a block of data. If the virtual buffer begins to overflow, the macroblock quantization step size is increased, resulting in a smaller yield of coded bits in subsequent macroblocks. Likewise, if underflow begins, the step size is decreased. The Test Model approximates that the target picture has spatially uniform distribution of bits. This is a safe approximation since spatial activity and perceived quantization noise are almost inversely proportional.
From:http://www.mpeg2.de/doc/mpegfaq/mpeg2.htm
So, it is not stupid to think that the Quantizer scale(1) can influence the VBV.
_____________
(1)QScale:"this is the quantization scale value that has a trade-off between quality and compression. Using different Q-scale values has a significant affect on amount of compression you get. The Q-scale values can be set for I, P, and B-frames separately. The scale can be from 1 to 31. Larger numbers will result in better compression but at the expense of worse quality. The image.param has the Q-scales all set to 1, giving the best quality/worse compression. Through a series of tests, increasing the Q-scale affected the amount of compression the most."
(Mpeg optimizations by Eric Young)

[Peter1234]

freelock7,
If you have not already done so, you may want to look at the FreeEnc forum on the KVCD website.
http://kvcd.net/forum
It has more information about FreeEnc.

[freelock7]

Yes but you have to pay for it! I do not agree !

[Asmodeus]

Go to www.vmesquita.com , creator site of FreeENC. It is still in development.

[Peter1234]

freelock7,
you wrote: "Yes but you have to pay for it! I do not agree !"

No. You do not have to pay, KVCD forum is free, but you do have to register.

[dragongodz]

No. You do not have to pay, KVCD forum is free, but you do have to register.
no. you do have to pay first. to quote from the kvcd main pages news(16th October)

Forum access now requires a very small "One Time" fee.

[Peter1234]

dragongodz,
Thanks for pointing that out. I was not aware that it was no longer an open forum. My apologizes to freelock7.

[Peter Cheat]

@freelock7
Increasing the minimum quantiser will reduce the probability of underflow. But it will reduce the quality also. The concept is to only increase the quantiser when necessary (to prevent underflow)keeping quality to a maximum, and to achieve a target bitrate. avcodec only checks for underflow once, and only during initialisation (before encoding). My current version checks 30 times (I've reduced this for next release for faster encoding) trying to make sure it won't happen (it still does, but very rarely).

Beware, some of the options in FreeEnc do not even apply to MPEG 1 or 2. Some info about what the options do is here at ffmpeg.org (http://ffmpeg.sourceforge.net/ffmpeg-doc.html#SEC8).

I wouldn't use my modified DLL with FreeEnc, I've made so many changes that it won't be compatible (it might encode, but the result will be poor). My next version of NuEnc will allow the minimum bitrate to be set.

You can't really compare quantisers even if they are both linear by looking at an average. It is easy to make a very low quantiser average by using very low quantisers for easy to encode scenes, and then using high quantisers on hard to encode scenes (to reduce bitrate). The average quantiser will be lower, but the quality would be very poor (this is done in CBR encoding). The best way to compare is to look at how different segments are quantised and decide which encoder made the better decision (do this visually).

And it does suck that KVCD now requires a fee, but I guess it does cost time and money to keep a board like that going. My only complaint is that it's just not very active.

[fozzieb]

the forum for FreeEnc and diko are now on www.vmesquita.com

and it is free.

[freelock7]

Here a good tool for freeEnc improvement:
http://forum.doom9.org/showthread.php?s=&threadid=80931
Thanks Amnon82 (for the matrix inside)!

[freelock7]

To correct the libavcodec RC underflow problem in FreeEnc, I'm using a template with a 448kb VBV size.
It's experimental. So,it has to be used with caution knowing the DVD compliance is 224kb(1).
It could be an alternate solution at low bitrate only.
This template can be downloaded here:
http://reinkinator.home.t-link.de/MDVD.ini
Thanks to MoleVCD.de
_________________________________
(1)This value specifies the size of the decoders "Video Buffering Verifier(2)". It represents the amount of coded video data that can be buffered by the decoder. At constant bitrates the buffer allows best use of the MPEG compression techniques. It is filled at a constant rate (that of the video stream) and partially emptied when a frame is decoded. The buffer fills up during sequences that compress well and empties during sequences that do not compress well.

Usual values are 40 for producing an MPEG-1 VCD , 112 for an MPEG-2 SVCD and 224 for DVD. These are the minimum allowed by the specifications, but if you are encoding for a particular player it may have a larger VBV buffer, so these could be increased (3).
_________________________________________________
(2)MPEG concept defined in ISO/IEC 13818-2 (MPEG-2, Annex C)
which employs a fixed size buffer to handle the transition of
the channel bit rate to the rapidly flucuating coded bit rate of
individual MPEG pictures. The scope of the VBV is only within a sequence.
The VBV is built upon a framework of several axioms of decoder behavious
which are unfortunately not very well described in the spec.
These include:

display pictures have known durations, but the time to download
the corresponding coded bits into the VBV varies. This time is
dependent on the bitrate of the delivery and the coded size of
the picture being delivered to the VBV. For example, I frames
will take an average of 4 picture periods to download, whereas
B pictures will take half an average picture period.

all coded pictures are not equal in size. Ratios of 15:5:2
are common for I, P, and B pictures respectively.

a picture is decoded instantenously in the VBV model, yet a real-world
decoder will account for the time it takes to decode a picture by
increasing the size of the VBV buffer.

a B picture is displayed almost as soon as it is decoded.
An I or P picture is not displayed until the next I or P picture
is decoded. This is known as the reorder delay.

the first vbv_delay of a sequence is arbitrary, but is usually
kept large. From there on, subsequent vbv_delay values are
directly related to the coded size of a pictures and the interval of
the picture being displayed while a picture is being decoded.

_______________________________________________
(3)"Changing the VBV buffer size will not affect quality in any way. The amount of bitate used by your video is set in stone during the encoding process by your AVG, MIN, and MAX values (or just AVG for CBR). It can not be changed on the fly by some magical setting to make your video better. The VBV curcuit controls the encoded stream to the encoder, ensuring it always outputs a constant stream for decoding to the encoder. It does this by throttling back on the mpeg input, if the decoder falls behind (say during a complex scene), or when it gets ahead (think 'On Golden Pond'). I used the funnel visual, because it works the best. Think of the VBV circuit as the person pooring the water. It maintains a constant amount of 'water' in the upper part of the funnel. No matter how big you make the funnel, you still get the same water out. It's not purified in the process.

Increasing the VBV buffer size would in essense give you a bigger 'funnel'. The problem here is that a larger funnel will increase the amount of data your dvd chipset must decode. If it is expecting a maximum amount of data for the medium its working on. It can choke up. Pushing more data doesn't make it look better. It just makes the chipset work harder to decode the info it's given(...)."
DJRumpy, VideoHelp Forum, july 2003.
Detailed VBV explanation:
http://petercheat.host.sk/libav/maxbitrate-truth.html

[Peter1234]

freelock7,
I compared your MDVD.ini with Incredible_KDVD.ini and found that your ini gave slightly better Q values at low bit rates with everything else the same. Very nice. Thanks.

Incredible_KDVD.ini: average kbps = 873, average Q = 10.13
MDVD.ini: average kbps = 869, average Q = 9.85

[Inc]

The less Q in average results from the MVCD Matrix. Reinkinator is a member of the MVCD Forum where Roc and I did assemble and release the Templates for TmpgEnc incl. that Matrix which was done by ours. So the Matrix is ported to the ini File for Freenc.

Also a very good try is the Bach01 Matrix from Ddog

[freelock7]

Thanks Incredible !
I'll test DDog matrix too.

Caracteristics of the BACH1 matrix:
http://forum.doom9.org/showthread.php?s=&threadid=60276&highlight=kvcdnotch

Other matrix
http://forum.doom9.org/showthread.php?s=&threadid=67975&highlight=ddog+matrix
@Peter1234
Did you see some reading problem by your DVD player with the MDVD matrix?

[Peter1234]

@incredible,
Thanks for the suggestion. Bach01 Matrix did give better q value.

Incredible_KDVD.ini: average kbps = 873, average Q = 10.13
MDVD.ini: average kbps = 869, average Q = 9.85
Bach01.ini: average kbps = 868, average Q = 9.26

Bach01 matrix
intra_matrix=8,16,19,22,26,27,29,34,16,16,22,24,27,29,34,37,19,22,26,27,29,34,34,38,22,22,26,27,29,34,37,40,22,26,27,29,32,35,40,48,26,27,29,32,35,40,48,58,26,27,29,34,38,46,56,69,27,29,35,38,46,56,69,83
inter_matrix=16,18,20,22,24,26,28,30,18,20,22,24,26,28,30,32,20,22,24,26,28,30,32,34,22,24,26,30,32,32,34,36,24,26,28,32,34,34,36,38,26,28,30,32,34,36,38,40,28,30,32,34,36,38,42,42,30,32,34,36,38,40,42,44

EDIT: DDogg called this matrix Bach1 not Bach01. I got the name Bach01 from Incredible post. DDogg original post is at http://forum.doom9.org/showthread.php?s=&threadid=60276&highlight=kvcdnotch

@freelock7,
I encoded using your ini (vbv=224) and made SVCD using VCDEasy and it played on DVD player but the last few seconds of video did not play. I also generated DVD files using DVDauthorGUI to verify that it would work (showed Vbv buffer size : 458752 bytes ) and the resulting files played in WinDVD. My experience is that a lower vbv works better. I normally use vbv=80 for SVCD.

[Peter Cheat]

@freelock7
Using 448KB as a buffer size to reduce underflow problems is crazy. You need to reduce the buffer size to reduce underflows on standalones, not increase it. If a DVD player only has a 224KB buffer, and you encode a stream that requires a 448KB buffer, it will not play correctly in some parts.

[freelock7]

True. I realised that IfoEdit shows a lot of DTS errors (data arrive too late) during muxing process.
It was not really a good idea to avoid underflow.
Thanks for advice.
The problem comes from the quantisation level I try to fix.
Here the quantisation Mencoder command line:
1)<q_min> at 2 increase the minimum quantisation but it decrease the average prediction.
2)<q_blur> may be increased for a better result (0-1).
"1.0 average the quantizer over all previous frames
larger values will average the quantizer more over time so that
it will be changed slower."
(Based on mplayers DOCS/tech/libavc-option)
3)<mb_qmin> (minimum macroblock quantizer) at 2 can help.
_________________________________
One finds the truth while seeking

[Yuri]

Great work!!!
Unfortunatelly, my encoding became 3 frames shorter that the input file. Don't know why, but the 3-4-5 frames at the beginning were somehow omitted.

[freelock7]

Caused by old FreeEnc0.3 libavcodec.
You can download the new modified libavcodec here:

FreeEnc0.31 is out: http://www.vmesquita.com/en/download.htm

[Yuri]

I'm afraid I had been using this very version which omitted frames. BTW, could NuEnc's dll interfere somehow, though it's in a seperate folder?

Best regards

[freelock7]

FreeEnc encodes from avs script.
Compare with QuEnc. They works in the same way.
There is not interference between them if you use a separate folder.

[fluffysnurgle]

I have downloaded the new(?) FreeEnc 0.41 but I have a problem with it.

In the past using version 0.31, using the "default" matrix, the encoding would ignore bitrate settings and always encode at around 1000kbps. But the problem went away by using the Incredible_KDVD matrix.

Now, however, the "Incredible" matrix is the default, and the low bitrate problem occurs with it also. So the new version is not usable.

[freelock7]

Please, post your question in the new FreeEnc0.41 thread