A possible method for use with do cce 4 u

[baker]

I posted the below method a long time ago on another website that dealt with the program redvd. I have just reliased that it would also be a good method for do cce 4 u as well.

See what you think.

[quote]Tyris,

Remember I said I had a way to speed the whole CCE process up? Well I have it all figured out so here we go.

Now shouts out to kwag etc,, whom made this method for use with tmpegenc but its very appliable to CCE.

Now this is how this works. Imagine if we encoded only 1% of the film. Now obvisouly when we multiplied the file size by 100 we would get the final file size of the film when fully encoded. But if the film was say 100mins and we only encoded 1 min, obivouly this wouldn't work. The reason it wouldn't work is because the first min of the film is probably very different then the rest of the film. So here comes the magic avisynth line!!!!

SelectRangeEvery(x,y)

where x is every so many frames and y is the ammount of frames to use.

Now heres an example test script from collateral damage:
LoadPlugin("C:\(S)VCD~1\PROGRA~1\dvd2svcd\DVD2SVCD\MPEG2Dec\mpeg2dec.dll")
mpeg2source("C:\(S)VCD~1\PROGRA~1\dvd2svcd\DVD2SVCD\Movie\DVD2AVI_Project_file.d2v")
SelectRangeEvery(1565,25)

See how this works???

To get x multiply the time the film lasts by 60 to get the ammount of seconds. Then multiply that by 25 (or whatever) to get the ammount of frames in the film. Divide the ammount of frames in the film by 100 and presto! you have x!

Now for y, y is the ammount of frames in a gop structure. Now for pal the max ammount of frames allowed in a gop structure is 25. Whatever the ntsc value is put it in here.

Now on to encoding!

Now first off, you have been using the value of 20 for the image quailty prio!! come on man!!! 20 is way too low. Use 30. Relistically 40 is best, but we dont usually have a bitrate high enough so plz, change it to 30 or even just 25.

Next off. we will be using the one pass vbr mode in CCE. The goal is to get a q value for the film we are encoding so that it will fit into the ammount of space we desire.

Now lets say we have a 100 min film we want to get onto 2gbs.

Now we have our line, which for a 100 min pal film would be:

SelectRangeEvery(1500,25)

Now we open up CCE, set it to the one pass vbr mode, turn off vaf creation set the min bitrate to 500 the max to 7000 (or whatever you prefer) ,set the q to 30 and ENCODE!!! That should finish in about 3 mins!!! Now look at the size of the outputed mpv file, multiply that by 100 and you get what would be the size of the full film if you had of encoded the whole thing. Was it too big or too small? Our targeted size was 2gbs so lets say it came out at 3gbs. If you up the q past 30 the quailty starts to rapidly deteriate. so what do we do? Add the line temporalsmoother (2,2) and then start encoding again.

Now if it was too small say 1500mbs what we do then is start lowering the q until we come to a suitable value. Start lowering it by 10, so next off you would try a q of 20, still too small? try a q of 10.. BINGO but wait its too big! When it turns out to be too big start incresing the q by 2 until you get a Q which is under 2 gigs. And there you have your Q

All you have to do now is open up an avs file with the selected video file (without the selectrange command of course!) and maybe add temporal smoother if you seen earlier that you needed it. Open up CCE in one pass mode, set it up as you did before, enter in your q value and when its finnished encoding you should get a file very close to the predicted file size!!!!

All the best,
Baker
[quote/]

Can't wait to hear your opinion on this one!!!

Baker

[ShaneZ]

This method has already been suggested by Bach, its not a very new method. The one problem with the 1% slice of the mpeg and trying to calculate the Q% by this is that when you are encoding a VBR file its gonna leave a slight margin for error. Its a nice method to use when you want to knock out a quality video quickly.

Most useres would prefer a multipass of the file though to get higher quality and closer filesizes to what you were targeting. This is only my opinion tho, others will vary I am sure.

[VILLA21]

@baker, the method u described above is already implemented to DoCCE4u, it"s on the templates-->Roba-Bach.

[baker]

Right a few points to Make here.

1.I actually came out with this method before bach.(however I am sure he would have understood how all thiss works)

2. the roba-bach templet is NOT this method at all.

3. As somebody has sudjested this method isnt as accurate as the x pass, however dvd2svcd has managed to get it down to little or no file size difference!!

4. besides since it only needs one pass its tons faster then x pass (2-3 times faster!! we talking 6 hrs to convert a movie compared with 2 hrs!) and even if you don't want to use it on your main movie, then you can use it on your extras as it would speed the process up loads.

So BBwoof, whats your opinion on this?? ;-)

Baker

[baker]

Right I am sorry if I have done something I shouldn't have and now bbwoof is ignoring me!! But could I please have a response!!!

Baker

[Doom9]

well.. in this forum we're actually not using multipass to reach a certain size, but to improve quality... You can't beat a 5 pass RoBa encoding with any 1 pass method.

[baker]

well looks like this ones for the bin then.

It's really strange though cause I am usually the one who fights for the quailty method! I have been more then once quoted to say

"if somebody comes up with a way to create a perfect 9->5 backup and it took 24hrs on my athlon, I'd still do it as I am a quailty freak"

Its a good quote you put in there about trying to get quailty, not size.

But personally from what I am hearing now from reading posts made by bach and others the quailty is actually better!! can somebody please explain why??? I could understand if it was the same quailty as the x pass method as this would make sense but surely not better!! Something to do with high motion and what the eye can see is my guess to all this!

I think I'll let this one die, you can lock the thread if you want.

P.S. I still watn to be able to select "interlaced content" though!! ;-))

Baker

[BBWoof]

Originally posted by baker
I think I'll let this one die, you can lock the thread if you want.

P.S. I still watn to be able to select "interlaced content" though!! ;-))

Actually I read this thread right before I had to leave town for a bit. It's a method I may add in the future, but for now I'm just trying to satisfy those pointing out problems that can be implemented a bit quicker.

I also thought about adding the ability to determine "Upper field first" setting in DoCCE4U, but I don't see it as practical. That's because I don't receive the m2v file directly, I receive the avs. In order to get back to the m2v file I'd have to open the avs as a text file, parse the d2v file, open it up as a text file and parse the m2v file, before I'd be able to do any analysis on it. Then I'd run into the problem of multiple d2v files in one avs and just complicate the issue.

Since DoItFast4U is working directly on the m2v files, it's a much better location for the determination of "Upper field first".

BBWoof

[baker]

I am not looking for the upper field thingy.. there is actually interlaced content on some dvds, usually series dvds. This is better off encoded in interlace and currently your program dosn't support this.

Go on.. add it ;-)

Baker

[BBWoof]

Originally posted by baker
I am not looking for the upper field thingy.. there is actually interlaced content on some dvds, usually series dvds. This is better off encoded in interlace and currently your program dosn't support this.

So what you're saying is that you want a checkbox for interlaced source? That should be able to be done.

Is that a PAL thing?

BBWoof

[baker]

exactly! And I have never heard if nstc interlaced dvds exist, but I don't think they do.

Thnaks.

Baker

[kwag]

Hi Baker and all,

The file prediction method has been recently enhanced by Tenra. It is now extremely accurate and fast, and the quality obtained with CQ mode(at least with TMPEG), surpases any x-pass method. It was also integrated into "ToK" ( http://www.kvcd.net/forum/viewforum.php?f=36 ) just earlier today.
The method is described here: http://www.kvcd.net/forum/viewtopic.php?t=3495

-kwag

[baker]

it took a while, but I had a funny feeling you'd show up here!!!

And i am sure bbwoof, will take a look at that link.

Baker

[kwag]

Originally posted by Bach
man... this k*** is hilarious...


quote:
"use a script like this:"
interval = round((FrameCount/24)/60) # Interval spacing ( Full Sample )
#interval = round((FrameCount/24)/60) / 10 # Interval spacing ( 10% of full sample )
nFrames = round(24) # Frames per sample
SelectRangeEvery( (round(framecount/interval)),nFrames)

why do someone should round an integer?:eek:

why not simply:

x=24
selectrangeevery(60*x,x)#full sample
#selectrangeevery(600*x,x)


anyway: don't use k*** method with CCE. It doesn´t works since Q.factor and CQ_whatever are not the same thing.

btw: there is a new tool called d2sroba which implements roba/bach methods in DVD2SVCD in a smart way. Do a search for it.

T
Because interval = round((FrameCount/24)/60) can give non integer results, and that is invalid for SelectRangeEvery function. The constant 24 in nFrames = round(24) is there as an example, as a variable is sometimes used in that position, making the round necessary.
And BTW, the formula works just as accurate with CQ or with any encoder. Be it CCE, in one pass, etc. Q. Factor or CQ whatever are not the same, but file size is file size in WHATEVER file system you use :D.
You just never stop amusing me with your comments bach :rolleyes:

-kwag

[kwag]

Originally posted by Bach
file size prediction is NOT the same thing, my dear d*mb *ss. You are using a linear method, which (luck you) is good for the "almost" linear behaviour of tmpeg. CCE's q.factor have an exponential behaviour, so that your method of prediction makes wrong assumptions "a priori", resulting in wrong values.
You obviously haven't even tried the file prediction we use! The way it works is as simple as encoding a sample with ANY encoder, until you get the file size/ratio you want. You do this by adjusting the CQ in TMPEG, or quality or whatever in CCE or any other encoder. Then you remove the sampler timing script lines, and do the full encode. By taking one sample for every minute of a movie, we end up with ~2% accuracy every time. The speed of prediction to get correct CQ(TMPEG)/Q:Factor(CCE), etc. is now averaging ~5 minutes on a P4. BTW, there are many people already using this prediction method with CCE, and It works every time. So your comments are nonesense! :p

-kwag

[kwag]

Sorry bach, but you don't get it, and you are plain wrong (again!). We predict by file size!
I tried a manual prediction earlier with our method and CCE. Wanted size was 11,944KB. Actual encoded sample was 12,016KB after finding correct Q: value in MPEG-2 (ES, One pass VBR ) . I would say that's pretty damn accurate, don't you think?
The current formula has been integrated in the new ToK 0.0.5.3 just released a couple of hours ago ;)
ToK works with TMPEG, but the formula works manually with CCE just as well.
http://www.kvcd.net/forum/viewtopic.php?t=3096

-kwag

[kwag]

Strike three bach!
You just can't accept that there are thousands of users using our method, and when used correctly, the target is ALWAYS ~2% accurate.
Read my lips: We get a file size within 2% every time. Maybe you should change your signature to read:
"Great spirits have always encountered violent opposition from EVERYONE's mind, specially when they try to swim against the tide, and impose theoretical bologna which proves impractical to the masses" :D
Sorry, but I don't understand most of your mathematical theories. Maybe they're beyond my understanding (or they're FUBAR), but then again, maybe they're beyond the average understanding of (most) people in the world!
Enjoy your (complex) prediction method, while we enjoy our (practical) accurate and faster methods :p

-kwag

[baker]

I must say bach tone it down a bit... dont fly off the handle.

Can I say I wont side with anybody here cause I am a self confessed idiot and like both you guys! :D

But isn't the idea of sampler.dll to combat the effect bach is talking of?? By selecting a different 1% of the movie each time???

Baker

[kwag]

Hi Baker,
Sampler.dll was designed to take snapsots from the input material. It takes a "Window" of a specified width (in frames), and takes a one second snapshot for every minute of the picture. We found out some time ago ( SansGrip and many many more people at KVCD.Net in a humongous thread: http://www.kvcd.net/forum/viewtopic.php?t=2073 ) that this resolution (granularity) was good enough for file size prediction. So the method applies to any encoder that supports some CQ VBR or similar encoding format. So far, the results have excelled our expectations, and the latest (100% large sample/10% small sample ratio) as suggested by Tenra in the link I provided earlier in this thread, enhanced this method even further by speeding up the process. It's a tried and tested method ;)

-kwag

[jorel]

for me the most important in this thread:

bach and Kwag are for me,two big friends!!

if you join your inteligences,
everybody here and in kvcd forum get gifts.

and the best will be your friendship!!!

:o
;)

!

[kwag]

Originally posted by Bach


the '/' operator will ever give integer results when the values used are integer. Since FrameCount is integer, 24 and 60 are also integer, then (Framecount/24)/60 IS integer. This is something well know by any "hello world" coder :p

Really!, stop drinking before you post:

Example:
Framecount = 171,000
24 = 24
60 = 60


Now (171000 / 24) / 60 = 118.75 "Hello world"!, Oh say can you see, the ... decimal dots ... are blinding me :D

Take a nap, then come back and do your math !

Sure, in programming, if I define a variable as INT, then of course the result will be INT. Only if the receiving (assigned) variable is declared as Integer. This is not the case for AviSynth script. Again: Been there, done that. You obviously need some background on AviSynth's internals variables :p

-kwag

[kwag]

Ok, fair enough, but you missed the point and I missed the original reason of why we used "round" in the formula. Not "round(24)", that was a stray line from the actual original "round(framerate)" which is what we used.

Your sample here:

v=Colorbars(720,480)
v=v+v
v.Trim(1,171000)
interval = (FrameCount/24)/60
#interval = ((FrameCount/24)/60)/10
nFrames = 24
SelectRangeEvery(framecount/interval,nFrames)

Works correctly. But back to my original point about your "round" comment, which is what started this, the following code doesn't work:

v=Colorbars(720,480)
v=v+v
v.Trim(1,171000)
interval = (FrameCount/Framerate)/60
nFrames = 24
SelectRangeEvery(framecount/interval,nFrames)

But this one works:

v=Colorbars(720,480)
v=v+v
v.Trim(1,171000)
interval = (FrameCount/round(Framerate))/60
nFrames = 24
SelectRangeEvery(framecount/interval,nFrames)

And that is the reason why "round" must be used in the variable "Framerate".

-kwag