New tool: Enc (encode, compressibility test, size prediction with almost every codec)

[jonny]

http://jonny.leffe.dnsalias.com/enc/

This tool is able to encode, make a size prediction at a given quantizer and make a compressibility test with (almost) every codec.
As input an AviSynth file is required (the file doesn't have to contain "return" or "end" in order to make compressibility test or size prediction).
It's structured in a way that make it work with every codec without the need to be modified.
You can also put multiple jobs in the joblist and save/restore both settings and joblist.
Needs VirtualDub for working (the latest VirtualDubMod version is strictly required if you are going to use size prediction or compressibility test).

First time you run this tool, be sure to open the help menu to see the nice people that helped me (i'll never be able to thanks all as i should... but i try ).


Some important notes:

Every time you update to a new codec version, you must trash all the settings/jobs saved for the previous version of the codec (codec data structures could be changed, so previous settings doesn't work any more).
Another good idea is to restore codec defaults settings (only the first time you open the codec settings interface).

If you are going to make compressibility test or size prediction with XviD using bframes, be sure to use a version after 07-03-2003.

There are still some XviD bframes configuration where the error in the size prediction is high (something like 7%-8%), i'm working on this problem night and day .

Using AviSynth 2.5 + MPEG2Dec3 v1.00 seems to leak a lot of memory if you run a lot of jobs or you get AviSynth info on the file.
A reference can be found here:
http://forum.doom9.org/showthread.php?s=&threadid=47970
Nic have solved this problem, be sure to check this thread:
http://forum.doom9.org/showthread.php?s=&threadid=53164

Opening 32 times the DivX codec control panel crash the app on Win98SE (this doesn't happen with XviD or with DivX+Win2000 ... DivX bug?)

It's all for now, i hope this tool will be useful

[kaitsuburi]

Awesome timing! I just posted at the GKnot Dev forum last night about problems loading B-frames XviD stats files in Gknot:

http://forum.doom9.org/showthread.php?s=&threadid=50682

I used DivX Enc for quite some time before switching to XviD. I am anxious to try Enc when I get home this evening.

Many thanks for your work!!

-kaitsuburi

[kxy]

Originally posted by jonny
http://jonny.leffe.dnsalias.com/enc/
This tool is able to encode, make a size prediction at a given quantizer and make a compressibility test with (almost) every codec.
As input an AviSynth file is required (the file doesn't have to contain "return" or "end" in order to make compressibility test or size prediction).

You mean the script CANNOT "return" or "end" function?

Just tested with xvid with b-frame, and it works, give me a day to do a full encode and we see how much it deviate.

feature request: a slider for output resolution, so if the compression is too high or too low we can change it accordingly.

[hakko504]

Originally posted by kxy
You mean the script CANNOT "return" or "end" function?

Just tested with xvid with b-frame, and it works, give me a day to do a full encode and we see how much it deviate.

feature request: a slider for output resolution, so if the compression is too high or too low we can change it accordingly.

Exactly: the correct line should read:

As input an AviSynth file is required (the file cannot contain "return" or "end" in order to make compressibility test or size prediction).

[jonny]

@kaitsuburi:
Thanks! (i've switched to XviD too )

@kxy:
Usually for testing i (helped a lot by betatesters) do a serie of 11 jobs, running size prediction with 1%-10% of the movie and the last at 100%.
You can find some results here:
http://jonny.leffe.dnsalias.com/enc/sp/xvid.htm (there are still some tests left and i'll go to update the page soon)
This version have an option keep/delete a special log file.
If you have the time to run 11 jobs and you are able to send me the joblist+11logfiles, this will be a really helpful for me.

[Sharro]

Originally posted by kxy
...feature request: a slider for output resolution, so if the compression is too high or too low we can change it accordingly.

That would be great!! If we could have also have:

-aspect ratio into consideration
-different resizing filters
-cropping

I would not need anything else and be a happier person :-))))

Great job for a small and simple program without any dependence on .net.

All the best,


Sharro

PS: Will run a few tests tonight as I have already a few full first pass lying on my hd.

[jonny]

Thanks Sharro.

Yep, doing an integrated AviSynth generator will be great, i know.
This could require some time, anyway i'm working on something similar now (for a different project)... so, why not?

[d.co]

I'm getting a strange (though I'm guessing somewhat simple to fix) error in which when I run a compressiblity test usting Xvid (b-frames on VHQ=1 Motion search=6 et al) I get an error at the end of it that says "Cannot open vdenc.log" instead of the compressiblity info.

I have Vdubmod.exe and Enc on my C: (in differenet places) and the video files (vob, d2v and avs) on my D: drive. I'm also using one of Koepi's newest unstable builds. So for the life of me I cannot figure out where this error is coming from.

Thanks for your time and consideration and thank you for this tool... I remember asking about Compressiblity tests for XVID a short while back. Nice to see it finally has it (and then some)

DeeCo

NOTE: Whoops sorry... I noticed I didn't have the latest VirtualDubMod version. I put that in place and everything is working now.

Out of curiosity what is it about the latest versions of VirtualDubMod that make it work with ENC that slightly older versions won't?

[kxy]

d.co,

You kind of answer that question yourself, it is vdenc.log

[ookzDVD]

@jonny,

Your enc is rock!
I love this program, thank you for your great work!

[Kamui-Dash]

nice little tool, how about adding FFVFW into the game

[jonny]

You kind of answer that question yourself, it is vdenc.log

yes, basically the latest build have an option that generate this log file (that contains informations about encoded frames size)
I use this log instead of the XviD or DivX log... so there is no need to update my tool if XviD or DivX log file format change (and the method can be applied with every codec)

@Kamui-Dash:
You can try to use FFVFW fcc in order to use it (read the docs about using custom fcc), i'm not able now to see if it works correctly (probably later in the evening i'll try)

@ookzDVD:
Thanks

[jonny]

@Kamui-Dash:
Yep it works, simply put fvfw in one of the custom fcc slots and you are ready to use FFVFW.
(tested with ffvfw-20030325.exe)

[Siku]

@jonny

I love this program! Thank you, keep up the good work!

Regards,
Siku

[((( atom )))]

johnny: my respect!

-very useable tool, thx - i'll use it from now on.

[kaitsuburi]

@jonny-

A big THANKS from me too. Enc has been working like a charm so far on about 4 XviD encodes. B-frames, vhq, custom matrices -- no problem, compressibility tests have given me almost perfectly accurate results.

-kaitsuburi

[jonny]

@Siku, ((( atom ))), kaitsuburi:
Many thanks for the kind words

There is a tiny update on my site:

1.01 (13th April 2003)
- Added ffvfw and Microsoft WMV 9 codecs
- Removed ICConfigure and ICGetState error checking (some codecs doesn't return a correct value, it's better to completely remove the check)

PS: atm size prediction accuracy is not guaranteed with ffvfw+bframes

[jonny]

Updated the notes with this really important thing:

Every time you update to a new codec version, you must trash all the settings/jobs saved for the previous version of the codec (codec data structures could be changed, so previous settings doesn't work any more).
Another good idea is to restore codec defaults settings (only the first time you open the codec settings interface).

[TNM]

@jonny

I have some questions

- Does Enc work "mostly" with avs2.5 and above for compressibility test? With an avs 2.08 script, unless I choose to use 100% of the movie, Enc will create a temp avs script with the function SelectRangeEvery(...) which doesn't seem to be recognized by avs2.08...

- Is it more accurate to use more % of the movie for compressibility test? Or is 1-10% enough? I did some compressibility tests with 5% movie used and the predited filesizes at quant 2 are always very near to the original filesize of the .m2v files...I wonder if it's accurate or not... if I follow the suggestion in the readme file to let the final encode filesize at 50-60%, my encode would be huge...

Really sorry if the questions are dumb...I'm quite new to all this Thanks a lot for creating such a useful tool anyway

PS: oops, forgot to mention that I did all tests with XviD latest Koepi unstable build with b-frame "extended method"

[hakko504]

Originally posted by TNM
@jonny

I have some questions

- Does Enc work "mostly" with avs2.5 and above for compressibility test? With an avs 2.08 script, unless I choose to use 100% of the movie, Enc will create a temp avs script with the function SelectRangeEvery(...) which doesn't seem to be recognized by avs2.08...

Check the box marked 'Use SelectEvery' and you should be OK. Strange though, I thought SelectRangeEvery should be included in v2.08.

- Is it more accurate to use more % of the movie for compressibility test? Or is 1-10% enough? I did some compressibility tests with 5% movie used and the predited filesizes at quant 2 are always very near to the original filesize of the .m2v files...I wonder if it's accurate or not... if I follow the suggestion in the readme file to let the final encode filesize at 50-60%, my encode would be huge...

Well, you will get slightly better prediction if you raise the percentage, but the time invested is usually not worth it. The tests are quite accurate even at 5% as shown here even though in jonny's test setup, 5% seems to produce the worst results. And if you think the filesize is too big if you use 50-60% in the compression test, then you could lower the resolution, add some filtering, check your resize method, check your deinterlacing method (if any) or maybe a combination of all.