New tool: Enc (encode, compressibility test, size prediction with almost every codec)

[jonny]

http://jonny.leffe.dnsalias.com/enc/

This tool is able to encode, make a size prediction at a given quantizer and make a compressibility test with (almost) every codec.
As input an AviSynth file is required (the file doesn't have to contain "return" or "end" in order to make compressibility test or size prediction).
It's structured in a way that make it work with every codec without the need to be modified.
You can also put multiple jobs in the joblist and save/restore both settings and joblist.
Needs VirtualDub for working (the latest VirtualDubMod version is strictly required if you are going to use size prediction or compressibility test).

First time you run this tool, be sure to open the help menu to see the nice people that helped me (i'll never be able to thanks all as i should... but i try :)).


Some important notes:

Every time you update to a new codec version, you must trash all the settings/jobs saved for the previous version of the codec (codec data structures could be changed, so previous settings doesn't work any more).
Another good idea is to restore codec defaults settings (only the first time you open the codec settings interface).

If you are going to make compressibility test or size prediction with XviD using bframes, be sure to use a version after 07-03-2003.

There are still some XviD bframes configuration where the error in the size prediction is high (something like 7%-8%), i'm working on this problem night and day :D.

Using AviSynth 2.5 + MPEG2Dec3 v1.00 seems to leak a lot of memory if you run a lot of jobs or you get AviSynth info on the file.
A reference can be found here:
http://forum.doom9.org/showthread.php?s=&threadid=47970
Nic have solved this problem, be sure to check this thread:
http://forum.doom9.org/showthread.php?s=&threadid=53164

Opening 32 times the DivX codec control panel crash the app on Win98SE (this doesn't happen with XviD or with DivX+Win2000 ... DivX bug?)

It's all for now, i hope this tool will be useful :)

[kaitsuburi]

Awesome timing! I just posted at the GKnot Dev forum last night about problems loading B-frames XviD stats files in Gknot:

http://forum.doom9.org/showthread.php?s=&threadid=50682

I used DivX Enc for quite some time before switching to XviD. I am anxious to try Enc when I get home this evening.

Many thanks for your work!!

-kaitsuburi

[kxy]

Originally posted by jonny
http://jonny.leffe.dnsalias.com/enc/
This tool is able to encode, make a size prediction at a given quantizer and make a compressibility test with (almost) every codec.
As input an AviSynth file is required (the file doesn't have to contain "return" or "end" in order to make compressibility test or size prediction).


You mean the script CANNOT "return" or "end" function?

Just tested with xvid with b-frame, and it works, give me a day to do a full encode and we see how much it deviate.

feature request: a slider for output resolution, so if the compression is too high or too low we can change it accordingly.

[hakko504]

Originally posted by kxy
You mean the script CANNOT "return" or "end" function?

Just tested with xvid with b-frame, and it works, give me a day to do a full encode and we see how much it deviate.

feature request: a slider for output resolution, so if the compression is too high or too low we can change it accordingly. Exactly: the correct line should read:As input an AviSynth file is required (the file cannot contain "return" or "end" in order to make compressibility test or size prediction).

[jonny]

@kaitsuburi:
Thanks! (i've switched to XviD too :))

@kxy:
Usually for testing i (helped a lot by betatesters) do a serie of 11 jobs, running size prediction with 1%-10% of the movie and the last at 100%.
You can find some results here:
http://jonny.leffe.dnsalias.com/enc/sp/xvid.htm (there are still some tests left and i'll go to update the page soon)
This version have an option keep/delete a special log file.
If you have the time to run 11 jobs and you are able to send me the joblist+11logfiles, this will be a really helpful for me.

[Sharro]

Originally posted by kxy
...feature request: a slider for output resolution, so if the compression is too high or too low we can change it accordingly.

That would be great!! If we could have also have:

-aspect ratio into consideration
-different resizing filters
-cropping

I would not need anything else and be a happier person :-))))

Great job for a small and simple program without any dependence on .net.

All the best,


Sharro

PS: Will run a few tests tonight as I have already a few full first pass lying on my hd.

[jonny]

Thanks Sharro.

Yep, doing an integrated AviSynth generator will be great, i know.
This could require some time, anyway i'm working on something similar now (for a different project)... so, why not? :)

[d.co]

I'm getting a strange (though I'm guessing somewhat simple to fix) error in which when I run a compressiblity test usting Xvid (b-frames on VHQ=1 Motion search=6 et al) I get an error at the end of it that says "Cannot open vdenc.log" instead of the compressiblity info.

I have Vdubmod.exe and Enc on my C: (in differenet places) and the video files (vob, d2v and avs) on my D: drive. I'm also using one of Koepi's newest unstable builds. So for the life of me I cannot figure out where this error is coming from.

Thanks for your time and consideration and thank you for this tool... I remember asking about Compressiblity tests for XVID a short while back. Nice to see it finally has it (and then some) :)

DeeCo

NOTE: Whoops sorry... I noticed I didn't have the latest VirtualDubMod version. I put that in place and everything is working now.

Out of curiosity what is it about the latest versions of VirtualDubMod that make it work with ENC that slightly older versions won't?

[kxy]

d.co,

You kind of answer that question yourself, it is vdenc.log

[ookzDVD]

@jonny,

Your enc is rock!
I love this program, thank you for your great work!

[Kamui-Dash]

nice little tool, how about adding FFVFW into the game :D

[jonny]

You kind of answer that question yourself, it is vdenc.log

:) yes, basically the latest build have an option that generate this log file (that contains informations about encoded frames size)
I use this log instead of the XviD or DivX log... so there is no need to update my tool if XviD or DivX log file format change (and the method can be applied with every codec)

@Kamui-Dash:
You can try to use FFVFW fcc in order to use it (read the docs about using custom fcc), i'm not able now to see if it works correctly (probably later in the evening i'll try)

@ookzDVD:
Thanks :)

[jonny]

@Kamui-Dash:
Yep it works, simply put fvfw in one of the custom fcc slots and you are ready to use FFVFW.
(tested with ffvfw-20030325.exe)

[Siku]

@jonny

I love this program! Thank you, keep up the good work!

Regards,
Siku

[((( atom )))]

johnny: my respect!

-very useable tool, thx - i'll use it from now on.

[kaitsuburi]

@jonny-

A big THANKS from me too. Enc has been working like a charm so far on about 4 XviD encodes. B-frames, vhq, custom matrices -- no problem, compressibility tests have given me almost perfectly accurate results.

-kaitsuburi

[jonny]

@Siku, ((( atom ))), kaitsuburi:
Many thanks for the kind words :)

There is a tiny update on my site:

1.01 (13th April 2003)
- Added ffvfw and Microsoft WMV 9 codecs
- Removed ICConfigure and ICGetState error checking (some codecs doesn't return a correct value, it's better to completely remove the check)

PS: atm size prediction accuracy is not guaranteed with ffvfw+bframes

[jonny]

Updated the notes with this really important thing:

Every time you update to a new codec version, you must trash all the settings/jobs saved for the previous version of the codec (codec data structures could be changed, so previous settings doesn't work any more).
Another good idea is to restore codec defaults settings (only the first time you open the codec settings interface).

[TNM]

@jonny

I have some questions :)

- Does Enc work "mostly" with avs2.5 and above for compressibility test? With an avs 2.08 script, unless I choose to use 100% of the movie, Enc will create a temp avs script with the function SelectRangeEvery(...) which doesn't seem to be recognized by avs2.08...

- Is it more accurate to use more % of the movie for compressibility test? Or is 1-10% enough? I did some compressibility tests with 5% movie used and the predited filesizes at quant 2 are always very near to the original filesize of the .m2v files...I wonder if it's accurate or not... if I follow the suggestion in the readme file to let the final encode filesize at 50-60%, my encode would be huge...

Really sorry if the questions are dumb...I'm quite new to all this :D Thanks a lot for creating such a useful tool anyway :)

PS: oops, forgot to mention that I did all tests with XviD latest Koepi unstable build with b-frame "extended method" :)

[hakko504]

Originally posted by TNM
@jonny

I have some questions :)

- Does Enc work "mostly" with avs2.5 and above for compressibility test? With an avs 2.08 script, unless I choose to use 100% of the movie, Enc will create a temp avs script with the function SelectRangeEvery(...) which doesn't seem to be recognized by avs2.08...

Check the box marked 'Use SelectEvery' and you should be OK. Strange though, I thought SelectRangeEvery should be included in v2.08.
- Is it more accurate to use more % of the movie for compressibility test? Or is 1-10% enough? I did some compressibility tests with 5% movie used and the predited filesizes at quant 2 are always very near to the original filesize of the .m2v files...I wonder if it's accurate or not... if I follow the suggestion in the readme file to let the final encode filesize at 50-60%, my encode would be huge...
Well, you will get slightly better prediction if you raise the percentage, but the time invested is usually not worth it. The tests are quite accurate even at 5% as shown here (http://jonny.leffe.dnsalias.com/whywork.htm) even though in jonny's test setup, 5% seems to produce the worst results. And if you think the filesize is too big if you use 50-60% in the compression test, then you could lower the resolution, add some filtering, check your resize method, check your deinterlacing method (if any) or maybe a combination of all.

[jonny]

Strange though, I thought SelectRangeEvery should be included in v2.08.

This is what i was thinking too, but seems not true :( ( i have not yet included SelectEvery option in Enc, so this option could be still useful ;) ).

So, @TNM:
To use AviSynth 2.08 you need to include mpeg2dec.dll plugin ( SelectRangeEvery is hidden here :) ).
AviSynth 2.5x include SelectRangeEvery directly in the core ( so you don't need to include a plugin in order to use this function ).


For compressibility test, i usually go for 4%-5% of the movie.

As you can see from the tests around on my site, 1% give the worst results, 10% give the better results.
Going up with the % of analisys gives usually better results, but this is true only in the general case (it's not guaranteed that 10% is always more accurate than 9% for example).
For the others doubts about accurancy, compressibility test with 100% of the movie is your best friend to verify the errors in size prediction (there are no statistics here, the result is the real video size after a full encode).


PS: as hakko says, don't be surprised when you obtain a huge predicted size, this often happen if your movie is hard to compress and you are using a big resolution, lowering resolution and using more aggressive filtering (eg. bilinear resize or some light tempsmoother) can help a lot.

[TNM]

@jonny, hakko504
Thanks a lot, I'll follow your suggestions :)

[kaitsuburi]

@jonny

I've been happily using Enc for a couple of days now and I have a very slight humble feature request for future versions.

Could you include an option to set the priority class at once for all jobs in the joblist. Vdubmod has that option but not Enc (or I haven't encountered it at least, maybe I'm blind :o ) . My W2k sucks up too much processing power on "Normal" priority and I am always forced to change it manually. I liked being able to set a default "Low" priority for the job list in Vdubmod.

Of course, this is not a big deal, just a small small request...

thanks,

-kaitsuburi

[jonny]

Actually i'm working on a standalone encoder and i could add this option here (i want to replace VDub with this encoder).
If this will take too much time, i can think to release a version in the middle that handle VDub priority.
Let's see what happen :)

[hakko504]

Originally posted by jonny
Actually i'm working on a standalone encoder and i could add this option here (i want to replace VDub with this encoder). AVS2AVI? (http://forum.doom9.org/showthread.php?s=&threadid=36768)

[jonny]

I'm taking great inspiration from avs2avi, sure :)

But i'm writing all from stratch (i'm making experiments to read frames directly via avisynth.dll, like in axenc000.zip-released by Toff in the same thread).

For the writing part i was thinking to write avi from stratch (like VDub does, so without using vfw).
But matroska will be soon released (so a better choice could be writing in this format instead of avi).

Apart from some speed gain, i think that in this way i'll be able to get a reduced error in size prediction (i'm able to fast encode each snip as a separate clip, this will delete oversized frames at the end of each snip and will help a lot in the XviD field).

If all goes well, be prepared for a new betatesting session soon :)

[hakko504]

I see. Sounds interesting. Send me a note when you want someone to beta test it for you ;)

[N_F]

Originally posted by kaitsuburi
@jonny

I've been happily using Enc for a couple of days now and I have a very slight humble feature request for future versions.

Could you include an option to set the priority class at once for all jobs in the joblist. Vdubmod has that option but not Enc (or I haven't encountered it at least, maybe I'm blind :o ) . My W2k sucks up too much processing power on "Normal" priority and I am always forced to change it manually. I liked being able to set a default "Low" priority for the job list in Vdubmod.

Of course, this is not a big deal, just a small small request...

thanks,

-kaitsuburi
No need for this. Just open VDM that Enc calls and look around in the options and you'll find a place to change default priority. I have it set at idle here as I like to be able to watch movies, play games or whatever while encoding.

@jonny - You know where to find betatesters if you need them :)

[jonny]

@jonny - You know where to find betatesters if you need them

sure! ;)

[Nazgul]

Hey Jonny, just as a note: In Divx 5.04 it seems that 100% quality has been set to Q=1, rather than the Q=2 it's been until now. I'm assuming that to run the Compressibility Tests in Enc we still use Q=2, even though it says that's 97% quality now?

[jonny]

Yep, i think quant=1 it's an overkill, so it's better to consider quant=2 as 100% quality and use it for the compressibility test (this will give you results that are "compatible" with the previous versions).

[wotef]

hi - using enc 1.01 and huffyuv ccesp patch, i get this error after trying to configure a normal encode using custom fcc "hfyu"

icgetstatesize <=0

[jonny]

It seems that huffyuv doesn't have a (correct?) get/set state implementation.
Never used huffyuv, i'll try this later in the evening

[N_F]

@jonny - Will we see an Enc version where the program window will be accessible while encoding in the near future?

[jonny]

I think in the next version... with another nice surprise (i hope in this weekend)

... nice new avatar, N_F :D

[N_F]

Originally posted by jonny
I think in the next version... with another nice surprise (i hope in this weekend)
Can't wait :)

Originally posted by jonny
... nice new avatar, N_F :D
You should see her in full size :)

[hakko504]

Originally posted by N_F
You should see her in full size :) Yes please! :)

Who is it? Your girlfriend? You? ;) :D

[N_F]

Originally posted by hakko504
Your girlfriend? In my dreams... yeah :)

Sorry to disappoint you, but it's actually not a real person. It's a painting by Jennifer Janesko, an artist who focuses on very beautiful and semi-nude women. I've uploaded the full picture at http://www.freewebs.com/franic/jennifer_janesko_018-1.jpg

I'd make all her paintings available, but I don't know what the copyright says.

[acanthis]

Jonny,

I've been using Enc 1.1 for a few days now, and I'm hooked! I was a great fan of DivX5Enc, so I missed the automatic two-pass job generation at first, but the new Enc is more flexible and it's saving me a lot of time now that I've started using the Copy To and From Job buttons.

Good work!

[jonny]

@acanthis:
Many thanks :)

@wotef:
Sorry for the delay, i've tryed huffyuv, but the codec doesn't have an ICGet/SetState implementation, so it's not possible to use it with my app. (the same thing happen to VDub, if you save the parameters in a vcf file, codec state it's not saved, so it's not possible to queue jobs with different huffyuv settings)

@N_F:
We should ask to Jennifer Janesko where have taken the inspiration :D

[nexus]

First: thanx for this great tool. But it doesn't work with latest VirtualDubMod v1.5.1.1a. At least Size prediction and Compressebility test. Would be nice if this can be fixed.

--
nexus

[soulfx]

@jonny, thanks for this awesome program. I had used DivX Enc back in my n00b days and since switchin' to pure XviD the only thing I missed was the ease of which DivX Enc could do compression tests for me. I don't think I could have gotten where I am today without your early DivX Enc work and I am happy to have Enc for my XviD encodes now.

@nexus, until jonny gets the code updated you can always stick in the old VDMod 1.4 in the same directory as 1.5 (just rename it to VDMod14 or whatever) and point to that in Enc.

Happy encoding everyone.

[nexus]

@soulfx: I already did this. Just wanted to mention this problem

--
nexus

[jonny]

@nexus & soulfx: thanks to both :)

There is a quick update on my site (Enc 1.02) that fix the problem with the latest VDubMod version (and of course works with previous VDubMod versions too)

[nexus]

Thanx :-)

--
nexus

[jonny]

Using AviSynth 2.5 + MPEG2Dec3 v1.00 seems to leak a lot of memory if you run a lot of jobs or you get AviSynth info on the file.
A reference can be found here:
http://forum.doom9.org/showthread.php?s=&threadid=47970

Nic have fixed this problem, there is a new mpeg2dec3 version ready to be tested ^^
http://forum.doom9.org/showthread.php?s=&threadid=53164

[((( atom )))]

@jonny

i am still using it :) it is reliable.

wouldn't it make sense, adding audio-dub-support to it, so one would get a all-ready second pass?

when i do vdubmod by hand, i alsways load my audio-tracks into vdubmod before doing the second pass. it saves time, klicks and space.

[jonny]

I want first to finish some things on the video encoding side (this may take a bit... the thing i'm working on it's a bit complicated and require a good amount of testing).
I have specs defined for the next 3-4 versions :).
After this i'll work on the audio side.

[((( atom )))]

cool, i'll wait..

[jonny]

Enc 1.03 is out

- Added support for AVS2AVI, at least v1.27 is required (work with normal encodes, size prediction and compressibility test, this feature is still in beta stage)
- Added an encoding window, it shows the joblist with the running job selected and the status of the terminated jobs, you can stop after the current job finish and you can change the value of the shutdown option
- Option to run only the new jobs (useful if you add jobs after encoding or if you stop encoding and you want to run the remaining jobs)
- Added some command line options (see the docs)

AVS2AVI support is still in beta stage.
At least v1.27 is required (the new versions will work too).
There are some things to take care about AVS2AVI, atm XviD output is broken (the avi header is bad for XviD encodes) and atm i'm not able to make error checking (if AVS2AVI report an error).
Compressibility test and size prediction seems to work well (on my little tests avs2avi produce the same frame sizes, compared to VDub).
Many thanks to DaveEL for adding log generation (comp.test and size prediction works with AVS2AVI thanks to this).
Be sure to check for AVS2AVI updates here:
http://daveel.leffe.dnsalias.com
http://forum.doom9.org/showthread.php?threadid=36768

Enjoy with this new version :)

[JagPanzer]

- Added an encoding window, it shows the joblist with the running job selected and the status of the terminated jobs, you can stop after the current job finish and you can change the value of the shutdown option

That's realy cool. And other new features also. Thanks jonny! :)

[jonny]

Yep, this was one of the main requested features (at the time of divx5enc :))
It's really useful if you put a large amount of comp.test jobs, you can now see the results job after job and decide to stop when one of the result it's ok for you.
(one little dirty trick: if you want to stop without waiting for the current job to finish, you can press "abort" on VDub, the result for the current job will be not correct, but doing this don't compromise the stability of the app... i do this many times when i test Enc :D)

[iago]

jonny,

Thanks for this great tool! I've just found the time to try Enc 1.03 for a couple of XviD compressibility tests and I really like it. Very useful! :)

Also, I'm really sorry I couldn't reply to your PM a while ago when you first wrote to me and I couldn't provide any help during the XviD-testing stage of Enc, but I was incredibly busy those days because of work.

ciao,
best regards,
iago

[((( atom )))]

@ jonny,

is there any way to keep the output avi of the size-prediction? i would like to be able to use it for a quick check on the quality of the selected compression-options. since it gous through the whole movie, it would be very convinient to make use of it..

[jonny]

@iago:
Many thanks :)
(no problem for the betatesting, i understand you perfectly)

@((( atom ))):
Yes, this could be useful, i'll add this option in one of the next versions

[((( atom )))]

jonny,

big thanx again! - where would we be without all you guys really programming that amazing software?!?

[jonny]

:)

[jonny]

Quick update (for those that haven't noticed this): AVS2AVI 1.28 is out! go to leach it ^^ (the problem with the XviD output is gone)

[Kintaro]

Hmmm, it seems that the time to ask for some help has finally come :(
I just cant get this to work and Im pretty clueless what else to try. Heres my little summary:

Im using Enc 1.03 and tried everything from Normal Encode to Compressibility Test, but when im hitting Start all Jobs the program seems to loop using 100%CPU without doing anthing. The Status window also never appeared :(

For the encoders: Ive tried VDubMod 1.5.1.1a and 1.4.13 as well as AVS2AVI 1.28, but none of them worked for me.

Source: The first source was a small 100MB Vob which is an Intro from Anime. I thougt that 10% of an 100sec clip may be not enough for compressibility check, so I took a 2h movie and let Enc run overnight for compressibility with this one (but this morning it was still using 100%CPU without finishing the 10% of a movie which takes ~6h for 1-Pass encoding normally).
Im using Avisynth 2.51 with MPEG2Dec3v104 with IVTC and Bilinear Resize to get the correct AR. No additional filters are used. Ive also tried MPEGDecoder.dll, but the only thing it did was to spare me from terminating enc.exe via Taskmanager because it crasht instantly.

Codecs: Normally Im using XviD with h.263, VHQ=4, Chroma motion and 2 B-Frames. Ive changed this settings several times and even took DivX3.11a and DivX5 (which I havent used for quite a year now :)), but the codec also dont seem to be the problem.

Im running WinXP so Ive also tried to run enc.exe in Win2k and 98 compatibility mode which seemed to work well for some other programs, but (guess what) it didnt change anything.
Additional I have to say that my CPU is P4 what seems to be pretty rare for the Users of this Forum, dunno if this could be the problem.

As far as I just dont get it, Im hoping one of you will see what Im to blind for and help me out.

THX for your time reading this

[jonny]

Strange problem!

Ok, let's start:

1-Opening the avisynth file via VDub or MediaPlayer don't give any problem, isn't it?
2-Encoding directly in VDub don't give any problem, isn't it?
3-Try to use Enc 1.00 to see if you obtain the same effect.
4-Try, in the menu, Tools>Get informations from avisynth file, to see if report correct informations about the avisynth file.
5-Be sure to respect this:
Every time you update to a new codec version, you must trash all the settings/jobs saved for the previous version of the codec (codec data structures could be changed, so previous settings doesn't work any more).
Another good idea is to restore codec defaults settings (only the first time you open the codec settings interface).
6-If i understand well... when you press Start, Enc disappear and nothing popup (no VDub window or AVS2AVI window or Enc window), isn't it?
7-Posting your AviSynth file could be useful.
8-You could also try to reinstall AviSynth, VDub and the XviD codec.

The P4 should be not a problem (for the Enc point of view).
MPEGDecoder can't be used for size prediction (there are problems with seeking on this plugin and the size prediction & compressibility test can't work correctly).

After this, if you have still problems, i can try to detect it sending you a modified version.

@All:
I'm testing Enc mainly on Win2000, can someone test it on XP or report if there are problems on XP (but i really don't think the problem is releated to the OS used)?

[N_F]

I've been running it on XP, though not the absolute newest version. Every version from the first to the 1.03 "beta" (don't know if you'd label it beta) you sent me ~1 week ago have worked fine.

[Kintaro]

THX for your quick reply :)

The .avs causes no problems when loading directly into VDubMod or WMP6.4 and has just been encodet normally to XviD with VDubMod 1.5.1.1a.
--------------
#
LoadPlugin("C:\Programme\Videobearbeitung\AviSynth 2.5\plugins\MPEG2Dec3.dll")
#
mpeg2source("D:\Videobearbeitung\Chobits\vts_02.d2v",cpu=4,iPP=true,idct=5)
#("D:\Videobearbeitung\Chobits\vts_02.d2v") <- tried this also
#
Telecide()
Decimate(cycle=5)
BilinearResize(720,416)
#BilinearResize(640,352)
#
---------------

The Codecs settings have been reset to default just after the installation.
I tried Get informations from avisynth file 15mins ago, the program is not responding since then.
For Point 6 youre understanding it totally right: Enc disappears, nothing Pops up and CPU usage for enc.exe goes up to 97% or whatever is available.
Im going to try Enc 1.00 -and if theres no other way reinstall the other programs- this evening. Im a little busy now with my work :rolleyes:

[acanthis]

Originally posted by jonny
@All:
I'm testing Enc mainly on Win2000, can someone test it on XP or report if there are problems on XP (but i really don't think the problem is releated to the OS used)?

I run Enc 1.03 on XP Pro, XP Home and Win2K. Works on all platforms the same.

[jonny]

@acanthis & N_F:
Thanks for the answer (@N_F: yep, no need to test the latest version, i have only added AVS2AVI support from the version you are using now)

@Kintaro:
Ok, at least now i know where it stops :) (i read avisynth file informations before starting the encodes, so Enc freeze here)
A quick thing you can try is to reinstall AviSynth.
Another thing you can try is to use a simple script containing only:

Version()

To see if the app freeze in this case too.
In the worst case, later in the evening i'll be able to send you a modified version (i hope you'll have some time to help me on tracking this problem, anyway the test i need require 5 minutes :)).

Thanks!
jonny

[Kintaro]

Just tried v1.oo and same thing there.
Even an .avs with only Version() causes freezing.
I think 5mins of work shouldnt be a problem for me since youre trying to help me :D

[corigan]

1.03 and all the previous versions have worked great for me jonny. Running them on XP Pro Sp1. Just wanted to say thanks for a great tool, I really only use it for compressability tests, but it works wonders for that aspect of encodes. Thanks again for your hard efforts.

Corigan

[Garfield]

Same for me, i dind't have any problem with all version on a win xp sp1 system.

I use it for everything it can do because it do it good, thanks a lot johnny for this nice tool ;)




I have some ideas that came to my mind during my experiences with enc, as it is a very powerful tool for testing, maybe it could be even more powerful. That's not requests, just ideas that i'd like to share.

It would be cool to have the opportunity to give a name to each job you add in the job list as a reminder of your settings, i think it will be very useful for comp tests i.e if you want to test differents xvid matrices, once all jobs are done, you can see in the job list all results with your differents settings.

Also could it be possible to have Enc writting/upating a txt file wich will save all the joblist results for various encodes/tests with the same avs file and telling you wich .enj in the \jobs\logs\ folder correspond ?

And can those .enj files be named like this : avsname_yyyy-mm-dd hh.mm.ss.enj because sometimes it's hard to find the one you're looking for ;)

I apologize for bad english and i hope those ideas don't sounds stupid :D

[jonny]

@Garfield & corigan:
Many thanks :)
The problem is for sure not related to XP, i'm working with Kintaro to understand what is wrong (i'll post some infos after this).
If someone other have the same problem, a detailed post is really appreciated :)


It would be cool to have the opportunity to give a name to each job you add in the job list as a reminder of your settings, i think it will be very useful for comp tests i.e if you want to test differents xvid matrices, once all jobs are done, you can see in the job list all results with your differents settings.

I agree, i was thinking to add an optional description field too ^^


Also could it be possible to have Enc writting/upating a txt file wich will save all the joblist results for various encodes/tests with the same avs file and telling you wich .enj in the \jobs\logs\ folder correspond ?

You'd like to have different log for each avs file or a global log file?


And can those .enj files be named like this : avsname_yyyy-mm-dd hh.mm.ss.enj because sometimes it's hard to find the one you're looking for

I like this idea (i like more something like yyyy-mm-dd hh.mm.ss_avsname.enj, so the logs are ordered for date... perhaps i could put an hidden option for changing the order :D ... and this will be done only if all the jobs uses the same avisynth file)

Thanks for the ideas (i promise nothing because there are already a lot of things i'm working on right now... but if you have some patience...)

PS: no need to apologize, i'm not able to 100% distinguish from bad or good english :D

[Garfield]

You'd like to have different log for each avs file or a global log file?


Well, i have no idea,I like more the idea of a log file containing all jobs done for the same avs file but a global log file would also be nice.


I like this idea (i like more something like yyyy-mm-dd hh.mm.ss_avsname.enj, so the logs are ordered for date... perhaps i could put an hidden option for changing the order :D ... and this will be done only if all the jobs uses the same avisynth file)


That's not a problem :)
Maybe adding a description to the whole job list wich will appear in the enj name can be a good trick too.


Thanks for the ideas (i promise nothing because there are already a lot of things i'm working on right now... but if you have some patience...)

That's also not a problem at all, feel free to code what you want to code when you want to code it ;)

I can't wait for the next update even if i have no idea what it will came out with. I really like this tool, you have all the controls on settings, you can configure a whole week of encoding, and you can store all the settings you use

Thanks a lot for wasting your time reading my post & keep up the good work :)

[hubereevez]

Just wanted to thank u Jonny, I trusted you tool since divx4 and it always work for me.......
This new tool is great like the others (but Imust make more snapshot grrrr ;) )

Just wondering how u use you own tool, more like a gordianknot complement, or do u try to add your features into gordianknot?

Thanks a lot again

hubhub

[jonny]

Thanks hubereevez :)

Just wondering how u use you own tool, more like a gordianknot complement, or do u try to add your features into gordianknot?

Actually i use GKnot for creating the base AviSynth script, after this i play manually with the script running differents size prediction jobs, and i finally encode with Enc :).
Sometime i make QB encodes (for movies that have great compressibilty) and i adjust the audio to fit the CD (usually 2 CD).
I have some little tools to speed up audio encoding setup & muxing (but aren't usable for other than me atm :D)

Adding features to GKnot... i'm not skilled in delphi programming, all i can offer atm are methods for making a good comp. test :) (actually GKnot & other tools use some options i've added in VDubMod for the comp.test tasks, this is my contribute :))

[kaitsuburi]

@all

I have been unable to access Jonny's site for a couple of hours. Does anyone know of other sites that host Enc?

Thanks,

-kaitsuburi

[Garfield]

Get it here (http://www.efrance.fr/garfield900/enc_1_03.rar) ;)

[kaitsuburi]

Thank you Garfield! Google searches were absolutely futile -- I must have spent over an hour :o

-kaitsuburi

ps. Perhaps maybe someday Enc should be included in doom9's d/l area? I'd say it's worthy :cool: just a suggestion...

[jonny]

Thanks for the mirror Garfield :)

Yep, the site is not accessible atm, i'll setup a little mirror for downloading asap (until the site go up again ;))

EDIT: the site is up now ^^

[DvdKhl]

Hi

I hope you didn't stoped the development on Enc because I really like to use it.

Now to my Problem Enc seems to not work with the newest AVS2AVI 1.35 release. I get an error in Enc as follows:
-----------------------
[Error] File "D:\Encoding\Vobs\1x02.avi" not found Enc XviD (xvid) D:\Encoding\Vobs\1x02.avs D:\Encoding\Vobs\1x02.avi C:\Programme\AV Tools\Enc\bin\avs2avi.exe
-----------------------

I don't know why it searches after an Avi file but the same settings work with VDub just fine.

[Livesms]

Strange problem!

Ok, let's start:

1-Opening the avisynth file via VDub or MediaPlayer don't give any problem, isn't it?
2-Encoding directly in VDub don't give any problem, isn't it?
3-Try to use Enc 1.00 to see if you obtain the same effect.
4-Try, in the menu, Tools>Get informations from avisynth file, to see if report correct informations about the avisynth file.
5-Be sure to respect this:
Every time you update to a new codec version, you must trash all the settings/jobs saved for the previous version of the codec (codec data structures could be changed, so previous settings doesn't work any more).
Another good idea is to restore codec defaults settings (only the first time you open the codec settings interface).
6-If i understand well... when you press Start, Enc disappear and nothing popup (no VDub window or AVS2AVI window or Enc window), isn't it?
7-Posting your AviSynth file could be useful.
8-You could also try to reinstall AviSynth, VDub and the XviD codec.

The P4 should be not a problem (for the Enc point of view).
MPEGDecoder can't be used for size prediction (there are problems with seeking on this plugin and the size prediction & compressibility test can't work correctly).

After this, if you have still problems, i can try to detect it sending you a modified version.

@All:
I'm testing Enc mainly on Win2000, can someone test it on XP or report if there are problems on XP (but i really don't think the problem is releated to the OS used)?

I've got the same problem.

1. Work's ok
2. Encoding work's ok too.
3. Enc 1.00 gives me the same result - Enc window disapp. and no VDub or AVS2Avi window app.
4. Tools->Get informations from avisynth file hang's up my Enc 1.03
AviSynth script is
AviSource("test.avi")
6. Yes - when I press Start Enc disappear and nothing changed

What shall I do and how can I correct this problem.

[Livesms]

I've changed AviScript from

AviSource("test.avi")

to

DirectShowSource("test.avi",fps=25,audio=false)

and Tools->Get informations from avisynth file start working.

But now I've got another error http://dump.ru/files/3/3481623924/Untitled-1.png

Can't open "vdenc.log". What is that? I can't find option to disable it :sly:

[shpitz]

you should use vdubmod.

[MacAddict]

Great looking tool here and I really like the idea of size prediction with XviD! Unfortunately the prediction is usually somewhere between 25-80MB too large compared to my real output. Has development stopped? Many thanks

[jonny]

you could try to increase the % of analysis to get a better prediction and yes, development is stopped long time ago (the tool still works and i personally don't need updates :) )

[henryho_hk]

jonny, can you tell me how to run Enc in full batch mode w/o any user intervention?

[jonny]

not possible
depending on what you want to do, maybe you could explore vdub scripting capabilities

[henryho_hk]

I see. May you us tell how Enc comes up with the compressibility figure? Thanks.

Edit: Trying to read the Perl script now. ^_^

[jonny]

this old th. can help you understand some details:

http://forum.doom9.org/showthread.php?t=44621

note: if you aren't a programmer it would be difficult to merge all the infos in something that work
note2: don't expect to easly set codec parameters, at least using vdub/xvid-vfw

[mod]

Hi all. It seems that I'm not able to make Enc work from command line.
Using:
enc.exe -vd="C:\avs2avi\avs2avi.exe" -if="c:\avs2avi_input.avs" -fr=off -of="c:\avs2avi_output.avi" -mo=ct -us=5 -me=ex -dl=off -tm=br -br=400
opens the main form and all settings are ready, but I have to manually set the codec options, add the process to the job queue and make it start.
I'd like to know if it's possible to make Enc start just from command line and get back the result, because I want to use it to perform a compressibility check as call from another application without any user action.
Thanks.

PS: If this isn't possible, does anyone if there's any other tool that can do it?

[foxyshadis]

Another option is to try running a compressibility check with encraw; there was some good information about how to make one in the mpeg2 forum some time back. It probably wouldn't correspond to the presets unless you copied enc's algorithm for calculating as well.

Or you could bundle one of those windows macro programs to click the buttons for you automatically. :p

[mod]

Thanks for reply :)
I use xvid_encraw for my encodings, do you think that performing a single pass with q = 2 and vhq 4 on a small portion of my source could work? What matrix would you use?
I want to make a "1 click" compressibility check, from which result my tool generates a list of presets basing on Teegedeck ones.
Any help appreciated :)

[jonny]

the best way is to use SelectRangeEvery
(for example SelectRangeEvery(1400, 14) for an 1% of the movie)
you should find a way to read outputted frame sizes, this is needed because some frames need to be discarded in the final calculations (you should take a look at the thread i've linked above)

[mod]

Thanks for the tip.
At the moment I can't figure out how to read the size of the frames (do you?), I'll work on it next days.
English is not my language, so I'm not sure I understood properly the post you linked to, I try to make a to-do list, please correct me if I'm wrong :)

1- single pass: xvid_encraw q = 2 vhq = 4 bframes on|off with SelectRangeEvery(nx,x) to get 1/n encoded frames (is 14 the correct value for statistical reasons?)

2- find a way to get the dimensions of the frames (maybe I can get them from the xvid .stats? dunno.. going to investigate)

3- remove the frames that appear more than once between.. what? I'm sorry but I didn't understand this point

4- calculate the avg frame size and from this the expected size of the whole movie

5- getting the %ratio doing the ratio between target size and expected with q = 2

Am I missing something? I ask you to explain exactly what to do in point 3..

Thanks again for the help :)

[jonny]

in my example 14 frames every 1400 are encoded.
since we jump to another portion of the movie every 14 frames, it is high probable that XviD detect a scene change and put a keyframe at every jump (even if XviD doesn't put a keyframe, the encoded frame will be probably oversized compared to a standard encode).
so you'll have:

IPPPPPPPPPPPPPIPPPPPPPPPPPPPIPPPPPPPPPPPPP

those keyframes will not be present in the real encode, so you must discard the first frame every 14 frames to have a good prediction.
bframes complicates stuffs a lot, with bframes enabled i discard the first 3 frames and the last 3 frames in each 14 frames snip (enc's extended mode).

14 have to do with the mpeg2 gop size (the method is optimized for mpeg2 source), it's the length that gives good results in the minor amount of time (long time ago the value used was 13, i've changed it at the time of divx5 due to bframes introduction).
about getting frame sizes with xvid_encraw, i don't have idea :) (it's probably not that hard to get the source and add frame sizes logging - i've done something similar with vdubmod for enc)

[squid_80]

At the moment I can't figure out how to read the size of the frames (do you?), I'll work on it next days.
The current version of xvid_encraw starts with output like this:
Trying to retrieve width and height from input header
xvid [info]: Avisynth detected
xvid [info]: Input colorspace is YV12
xvid [warn]: Single pass defaulting to quant 4.00
xvid [info]: Input is 504 x 572, 25.000fps (25/1), starting from frame 0
xvid [info]: Number of frames to encode: 10
xvid [info]: xvidcore build version: xvid-1.2.0-dev
xvid [info]: Bitstream version: 1.2.-127

There's enough information there to work out the dimensions and stream length. Then the statistics start:
0: key=2, time= 15, len= 44437 | type=I, quant= 4, len= 44437
3: key=0, time= 31, len= 1718 | type=B, quant= 7, len= 212
4: key=0, time= 15, len= 238 | type=B, quant= 7, len= 238
5: key=0, time= 0, len= 6 | type=P, quant= 4, len= 1512
6: key=0, time= 31, len= 1248 | type=B, quant= 7, len= 240
7: key=0, time= 0, len= 214 | type=B, quant= 7, len= 214
8: key=0, time= 0, len= 6 | type=P, quant= 4, len= 1014
9: key=0, time= 16, len= 1223 | type=B, quant= 7, len= 232
-1: key=0, time= 15, len= 175 | type=B, quant= 7, len= 175
-1: key=0, time= 0, len= 6 | type=P, quant= 4, len= 997

Tot: enctime(ms) = 123.00, length(bytes) = 49271
Avg: enctime(ms) = 10.25, fps = 97.56, length(bytes) = 4105
I frames: 1 frames, size = 44437/ 44437, quants = 4 / 4.00 / 4
P frames: 3 frames, size = 1174/ 3523, quants = 4 / 4.00 / 4
B frames: 6 frames, size = 218/ 1311, quants = 7 / 7.00 / 7
(^average/total^) (^min/ave/max^)
The numbers in bold are the output frame sizes.

[mod]

@jonny: thanks a lot perfect explanation :)

The numbers in bold are the output frame sizes.
Yes I had already noted :) Now I have to choose between saving all the output to a file or working on the source to get just the info I need. Imo the second would be better, maybe with a -getfsize "fsize.txt" as input on the command line. This afternoon I'll take a look at the source and see if it's an easy job.
Thanks both, any other possible suggestion is still appreciated :)

Edit: @jonny: would it be possible to extract the infos I need from the last 3 output lines of xvid_encraw, discarding I and P frames?
I mean, if I encode 1 frame every 100, then the 1st, the 100th, the 200th etc encoded will be I frames, so the last lines would be, given n total frames:

I frames: n/100, size = ............../...............
P frames: n - n/100, size = ............../............... [without b-frames]

I frames: n/100, size = ............../...............
P frames: n - n/100 -x, size = ............../............... [with b-frames]
B frames: x, size = ............../............... with n/100 < x < n

Feel free to shoot me if I'm still wrong ^^

[jonny]

would it be possible to extract the infos I need from the last 3 output lines of xvid_encraw, discarding I and P frames?

if there are no bframes, you could use the average of P frames as a base value (so discarding all the I frames), this should still be a good approximation


I mean, if I encode 1 frame every 100, then the 1st, the 100th, the 200th etc encoded will be I frames, so the last lines would be ...

if you do this, there will be only I frames in the final encode and no good values for statistics
you should take 14 as a base, so:
14 frames every 1400 for an 1% of the movie
14 frames every 700 for an 2% of the movie
and so on ...

the xvid_encraw encode will have this sequence of frames:

1st snip, frame 0 ***discard always***
1st snip, frame 1 **discard if bframes are enabled**
1st snip, frame 2 **discard if bframes are enabled**
1st snip, frame 3
1st snip, frame 4
1st snip, frame 5
1st snip, frame 6
1st snip, frame 7
1st snip, frame 8
1st snip, frame 9
1st snip, frame 10
1st snip, frame 11 **discard if bframes are enabled**
1st snip, frame 12 **discard if bframes are enabled**
1st snip, frame 13 **discard if bframes are enabled**

2nd snip, frame 0 ***discard always***
2nd snip, frame 1 **discard if bframes are enabled**
2nd snip, frame 2 **discard if bframes are enabled**
2nd snip, frame 3
2nd snip, frame 4
2nd snip, frame 5
2nd snip, frame 6
2nd snip, frame 7
2nd snip, frame 8
2nd snip, frame 9
2nd snip, frame 10
2nd snip, frame 11 **discard if bframes are enabled**
2nd snip, frame 12 **discard if bframes are enabled**
2nd snip, frame 13 **discard if bframes are enabled**

3th snip, frame 0 ***discard always***
3th snip, frame 1 **discard if bframes are enabled**
3th snip, frame 2 **discard if bframes are enabled**
3th snip, frame 3
3th snip, frame 4
3th snip, frame 5
...

you should sum all the non-discarded frame sizes and divide by the total number of non-discarded frames to get a good average value for the prediction

[mod]

Thanks again for your time! :)

Ok, so here's what I have:

1) No B-frames:

IPPPPPPPPPPPPP IPPPPPPPPPPPPP IPPPPPPPPPPPPP

I get an avg of P-frames from xvid_encraw

2) B-frames

IXXPPPPPPPPXXX IXXPPPPPPPPXXX IXXPPPPPPPPXXX

I have to discard the size of underlined frames.

And now the n+1 question.. maybe it's the last one ^^

case 2a: all the "X" frames are type B (I don't think so)

- N = number of I frames reported by xvid_encraw
- 5N = number of B frames to remove from stats: I can do 5N * B-frames.avg and get an idea of what to remove.

case 2b: not all the "X" frames are type B

damn..

So.. with B-frames we are in case 2b aren't we?

[jonny]

1) No B-frames:

IPPPPPPPPPPPPP IPPPPPPPPPPPPP IPPPPPPPPPPPPP

I get an avg of P-frames from xvid_encraw

personally i would discard always the first frame in each snip, consider that in a real situation you can have:

IPPPPPPPPPPPPP PPPPIPPPPPPPPP IPPPPPPPIPPPPP

anyway there should be not much difference between the 2 methods



I have to discard the size of underlined frames.

and the first frame of each snip too:

IXXPPPPPPPPXXX IXXPPPPPPPPXXX IXXPPPPPPPPXXX



case 2a/2b

you can't assume this cases
all you have to do is:


sum all the non-discarded frame sizes and divide by the total number of non-discarded frames to get a good average value for the prediction

nothing more, nothing less :)

[mod]

and the first frame of each snip too:

IXXPPPPPPPPXXX IXXPPPPPPPPXXX IXXPPPPPPPPXXX

Of course, I forgot to write it :)

It seems that now I have all I need to make a try, I'll post my results when I'll have all the work done.

Thanks for your precious help :)

PS: I'm also working on making a modified build that outputs directly the result in a txt file, I'll see if it works..

[henryho_hk]

jonny, should I discard the same frames when max-b-frame = 1? And is the handling the same for packed and non-packed bitstream?

[henryho_hk]

It probably wouldn't correspond to the presets unless you copied enc's algorithm for calculating as well.

I wanna copy Enc's algorithm too. The problem is... I dunno Perl....

[jonny]

probably with max-b-frame = 1 you could discard less frames, you should run some tests and evaluate the errors in this case.
the handling suppose non-packed bitstream

[henryho_hk]

Thanks, jonny. I have made my first try here:

http://forum.doom9.org/showthread.php?p=879025#post879025

I made a 1st pass using xvid_encraw and parsed the stats to collect the frame sizes. For 1 and 2 max-frame cases, I dscard (0,1,12,13) and (0,1,2,11,12,13) respectively.

[shadell]

Thanks for Enc, I have been using it to do compressibility checks.

I would be interested to know what exactly Enc does when you ask for such a test. As well as the fundamentals, I'd like to know whether the tested section is taken from the start of the file, or does it take (for example) a couple of different sections from different parts of the file, further in, to avoid initial credits etc?

[henryho_hk]

jonny, do we need to cater for the AVI overhead when we are doing the file size prediction?

[jonny]

@shadell:
explained many times, try to look at the prev page

@henryho_hk:
i don't take this into account, you could :)

[Blue_MiSfit]

Still a very useful tool!

[Dams]

@jonny: compressibility test is usefull if I keep many other compressibility test of similar type of movie, and if I compare to average I obtain ?

[jonny]

i dont understand the question :)

[buzzqw]

@jonny

i think that Dams mean to do several compression test with different range/region of the same movie, and then do an average of all

in italiano: fa piu' test e poi fa la media...

ciao

BHH

[jonny]

i think that Dams mean to do several compression test with different range/region of the same movie, and then do an average of all

i see, the app already work in this way :)

[Carpo]

i have search the forum for this but kept going round and round in circles, could someone tell me how to setup xvid 1.1.2 and 1.2 to work with this tool - as i have read the docs that come with it but how it tells you to setup xvid it does not correspond to the setup options in xvid 1.1.2 and 1.2 - by that i mean the pictures in the docs do not look anything like how xvid options are set out now, and im a bit of a noob when it comes to doing encodes without the help of autogk or GK :o and the few encodes i have tried with megui seem to come out wrong due to encoder saturation, becuase im selecting the wrong preset

last time i tried this tool it told me 30.3% so i used that im megui and it was well short of what it should have been :(

edit: after a bit of reading elsewhere i think i may have it sorted :)

[dvdfernandes]

Hi! I'm geting an error when using Enc. I was trying to male a comp check, but the following error returned:

Predicted Size=0 (0 frames)

NEver happened to me before...
Could someone help me?

Here's the script in case it helps

LoadPlugin("C:\PROGRA~1\GORDIA~1\DGMPGDec\DGDecode.dll")

mpeg2source("D:\Hotel Ruanda\Hotel RWanda.d2v" )
crop(2,74,716,428)

LanczosResize(720,304)

LoadPlugin("C:\Program Files\GordianKnot\AviSynthPlugins\RemoveGrain\RemoveGrain.dll")
LoadPlugin("C:\Program Files\GordianKnot\AviSynthPlugins\RemoveGrain\Repair.dll")
LoadPlugin("C:\Program Files\GordianKnot\AviSynthPlugins\masktools-v2.0a30\mt_masktools.dll")
LoadPlugin("C:\Program Files\GordianKnot\AviSynthPlugins\mvtools-v1.6.2\mvtools.dll")
LoadPlugin("C:\Program Files\GordianKnot\AviSynthPlugins\MaskTools-v1.5.8\MaskTools-v1.5.8\MaskTools.dll")
LoadPlugin("C:\Program Files\GordianKnot\AviSynthPlugins\degrainmedian\degrainmedian.dll")

Import("C:\Program Files\GordianKnot\AviSynthPlugins\SeeSaw\Seesaw.avs")

a= last
b=a.degrainmedian(mode=3)
SeeSaw(a,b, NRlimit=2, NRlimit2=4, Sstr=1.5, Slimit=5, Spower=5, Sdamplo=6, Szp=16)

[buzzqw]

open your script in virtualdub, do you spot any errors ?

BHH

[Sharro]

It's been a few months since my last encode.

I'm getting an error in VirtualDub or VirtualDubMod when I start my compression test with ENC (a very simple avs just cropped .d2v created with latest DgIndex) and I'm using latest avisynth 2.5.7 and XVID 1.2.127 (SMTP).

When I do a start job I get and error in Virtualdub: "unknown switch / log"

Can somebody help me ?

EDIT: SOLVED!!

All the best,

Sharro

[chandragupta]

Hi!
i'm getting a 403 error accessing jonny's site
"http://jonny.leffe.dnsalias.com/"

neither does this work
"http://www.efrance.fr/garfield900/enc_1_03.rar"

please..can someone provide me with the download link for ENC.
Thanks!

[buzzqw]

http://www.64k.it/andres/data/e/enc_1_03.zip

BHH

[chandragupta]

@buzzqw
spent an entire day searching for this yesterday.
Thanks:thanks:

[telmoMRC]

hei guy i use the ENC to make compressibility test and it give this error what can i do to fix that ??

http://z1.zod.fr/z/lolitas-rry.html

edit :is fixed