ripping using 1pass-quality-based - new way of ripping

[A_Pleite]

Hi guys out there

I didn't know where to put this threat in so I just put it under "general".
I got an idea how I can do 1pass-rips using constant quality. In fact it´s just the mix of 2 previous ideas:
1st: doing a compressibility-test to get to know about the source you are going to encode
2nd: If people are voting (e.g. fot the president) they show the results on tv even if there isn´t counted the 4th of the votes and the results do not much change when counting all the votes -other example: 1000 people can show the tv-watching-behavior of the whole nation.

So I wrote a short avisynth-function to make a short compressibility-check. But it doesn´t scan through the first five minutes, but takes 25 short parts out of it.
So I can create a short video out of the movie with 2 percent of its length and compress it with divx5-constant-quality. If this video has 2 % of the size I want to get for the whole movie I just use this sttings and compress the movie with it.
I have done 3 rips.. ups BACKUPS with this method and the movie was "only" 30 MB oversized onetime and 30 MB undersized another time ;). So this method is NOT accurate.
Tips for using this method:

-always create audio after video
-if you movie is to big, just reencode the endtitles with a very low bitrate or quant18
-calculate your rsolution with gknot (0.2 bit(pixel*frame) for 1cd)
-quant4 is ok for this resolution (1cd)
-use denoising with the internal denoiser in divx5 (I suggest "normal")
-During the encoding of the check-video You can see whether its going to fit the size more or less (in VDub with "projected file size")
-if your check-video is absolutely to big, then try to give higher values to quant and/or lower the resolution, the other way round for check-videos which are to small


pros:
-much faster than 2pass
-much better quality than constant-bitrate

cons:
-not much accurate (you may and in a 680MB-noaudio-movie)

I attach the "chack.avs" with the finctions "check" and "checkpercent"
Final word: I´m sorry if anyone has anywhere posted this idea, I got this on my own ;)

What do you think?


A_Pleite

[LotionBoy]

why? you've already said it's not accurate. The time you spend trying to match the desired file size, would be much better spent doing two passes. There is no way your method is going to match the quality of 2-pass, And it is inaccurate. I'm not trying to shoot you down from experimenting, because that is great, but I fail to see the advantage of this method.

LotionBoy

[Dark-Cracker]

can u post the time encode for your backup :)

with 1pass-CBR / 2-pass (both) / and the time with your methode.

try to get less size error (30 Mo is a little big) and you methode will be interested.

[A_Pleite]

time you spend trying to match the desired file size, would be much better spent doing two passes

Thats not true. for the 1st pass my pc usually needs 2 and a half hour, For the tests I needed 15 minutes

There is no way your method is going to match the quality of 2-pass

Well, I wouldn´t be to sure about this, because the divx5-2pass-engine is not the best, while constantquality-divx5 gives very good quality. Of course the quality is worse, but not as much as you may expect. Maybe I will do the Mtrix and compare it to the doom9-tests ;) (I would have to get the matrix dvd first :D )

And it is inaccurate
Of course it is. But when you don´t want to get an 700MB video its still ok, divx4 used to be much inaccurate, too (while xvid is accurate). You can try to compensate the size error with different ausio-bitrates (but you have to do the video first ;)). btw, my 3 backups had the size of 656, 680 and 701 MB (I used vbr-vorbis for the 2 -less accuracy- and constant mp3 for the 3rd rip).


try to get less size error (30 Mo is a little big) and you methode will be interested.
The more porcent of the movie you take for the check-video, the more accuracy you will have.

can u post the time encode for your backup

I have an athlon1.4c, 256MB ddr
I used smartripper, besweet, dvd2avi(1.76) or mpeg2avi, for the 3rd rip I used razorlame(because its mono and besweet doesn´t seem tosupport it) and Nandub, too

one word about dvd2avi(1.76)/mpeg2avi:
since I used the divx5 internal noise-filter I thought I could do the cropping and resizing in the codec, too. And I did.


My times(I didn´t check the times so its a bit inaccurate):
ripping (smartripper): 45 min.
finding the settings : 15 min.
encoding: 2:30h (it was almost as fast as with dvd2avi1.82(cropping, resizing with it) although I applied the noisefilter;))
ogg-encoding: 30 min. (not sure about the exact time)
multiplexing: 5 min.

all: 4:05

all plus 2:30 minus 15 (2npass but no testing - normal way): 6:20

This way is much faster, but I still use the normal way for the movies I really like ;)

A_Pleite

[LotionBoy]

your time numbers only work if the margin of error is a couple MB. What if I'm aiming for 670MB, and it ends up being 700MB (just the video). At this point, I have to reencode the entire movie again at a different constant quality and try and hit the desired file size. On the subject of quality. Yeah Divx5 2pass ain't great. It's still better than constant quality, and the Xvid 2pass is superior to both of them. If you can get the margin of error down to a couple MB, without drastically increasing the time spent, then it will be a nice option for a quick-and-dirty compression where time, not quality is the issue.

LotionBoy

[Dark-Cracker]

when i ask you the time for the rip i have think u have test the different time between 1passCBR / 2pass (first+second) / and the time take your methode. :)
but it's not important i suppose for 1pass CBR u take 2H30 for 2pass both (with MV file) 2H30 + 2H =4H30 and your methode : compresstest (2% of the movie) = 30 min + 1 pass quality 2H30

i think it a good methode if you want do some fast rip.

i will try to test it.

could u try to explain me to use your script ?

[A_Pleite]

What if I'm aiming for 670MB, and it ends up being 700MB (just the video). At this point, I have to reencode the entire movie again at a different constant quality and try and hit the desired file size.

1st I would advise you to aim for 640 (or so) MB - its not to bad to have a video which is 50 MB to small ;) - You have more space for the audio-part
2nd The good things about our motion-pictures is that they have so called endtitles at the end, for some they are interesting, but the average movie-watcher isn´t so mch intersted in every name. So you can drastically reduce the finalsize with reencoding them;)

Divx5 2pass ain't great. It's still better than constant quality, and the Xvid 2pass is superior to both of them.

I somehow believe you haven´t tried the divx5-constant-quality in any way, yet ;)

quick-and-dirty compression
i think it a good methode if you want do some fast rip
That´s waht it is :D

A_Pleite

PS: I would have made a tool out of it (got some ideas for more accuracy) but I'm just a very low-skilled Delphi-user :( (user, not coder;))

[Dark-Cracker]

try to explain me (very) well how to use your script and i will perhaps try to add this fonction in my tool if this interest you.
it's Autodub and support Divx(Pro) 5.x codec

go to :
www.dark-angel.does.it

for more informations and post your reply here.

[A_Pleite]

Sounds very interesting, but I have to think of a way to include this way of ripping in your tool. ;) stay tuned
I'm happy to see that ONE is interested. :D

btw, your site seems to be down currently

[[Toff]]

This is perhaps a dumb question, but why don't you use the SelectRangeEvery function of AVS ?

[theReal]

[Toff], I think it's not a dumb question...

Take for example a PAL movie, 120 minutes. It has 180000 frames, from which you want to take 25 small parts that equal 5% of the whole movie (this is how far I understood the script by A_Pleite).

180000 / 25 = 7200

5% of 7200 = 360

=> SelectRangeEvery(7200,360)


This should suffice for taking 25 snips with a size of 360 frames each, alltogether 9000 frames, exactly 5% of the whole movie.
Maybe I forgot something crucial here? I dunno, just a thought.

[Dark-Cracker]

humm theReal your logo seems to be a "Hoffman's trip" are u a drug addict ? :) :) :)

Else can some one could explain well how to use .avs script ?

bye all.

[theReal]

Dark-Cracker, how do you know ;)

I use this picture only to remember the good old times (and because it's a funny picture of Dr. Hofmann cycling home from work that one day in '43, for all of you who don't know this story...)

Nowadays I'm relatively opposed to such stuff (I even quit smoking cigarettes in favor of more effective workout) and divx is my only addiction :D

[A_Pleite]

Originally posted by theReal
[Toff], I think it's not a dumb question...

Take for example a PAL movie, 120 minutes. It has 180000 frames, from which you want to take 25 small parts that equal 5% of the whole movie (this is how far I understood the script by A_Pleite).

180000 / 25 = 7200

5% of 7200 = 360

=> SelectRangeEvery(7200,360)


This should suffice for taking 25 snips with a size of 360 frames each, alltogether 9000 frames, exactly 5% of the whole movie.
Maybe I forgot something crucial here? I dunno, just a thought.

There is no "SelectRangeEvery" - command in the avisynth reference guide :confused:

A_Pleite

[theReal]

it's part of mpeg2dec.dll (description here: http://users.win.be/dividee/avisynth.html )

it's used by GordianKnot for the compress test like this:

# COMPRESSIBILITY CHECK
# !Snip Size has to be 13 for use in GKnot!
SelectRangeEvery(260,13)

[TheWEF]

steady and dividee did huge test series to find out the best snip-size for a comp check. they found out that lots of small samples return the most accurate result (in comparison to a full first pass). 13 frames turned out to be the best sample lenght.

there is one little problem though: at the beginning of every "frame snip" you introduce a new artificial scene change, that uses a lot of bits. the smaller the snip size, the more "scene changes", the higher the error.

so we used a little trick:
frame 1 of each sample is not taken into account, only frame 2 - 13 (12 out of 13).
so you need to know the size of every single frame (and filter out every 13th) to get an accurate result.
you could get this information from an avi directly, or reading the stats- or log-file (like gknot does).
13 is hardcoded in gknot - that's why you have to use snip-size 13 if you are using gknots "load" button.

without this trick a comp-check is pretty useless.

wef.


p.s.: don't forget that we invented the comp-check to test filter settings and combinations before doing a full 2-pass-encode, not to replace the first pass.
i would say that accepting lower quality just to save 1 hour is not in the spirit of this board ;)
IMHO with your method you will never be able to predict file-size within an acceptable range, so i would hate it anyway.

[theReal]

Ah, thanks TheWEF for shedding some light on the way GKnot interpretes the comp. test logfile - that's very interesting.

I already had in mind that probably quite some experience and testing must have gone into that size of 13...

Whatever, I also like the comp. test like it is and I won't switch to non filesize-accurate 1-pass encoding only to save some time and I, like most people here would accept a 200% higher encoding time for 5% better quality, I guess ;)

[A_Pleite]

@toff and the real:
You´re right, I added this function to my check avs, because this is much easier ;)


Whatever, I also like the comp. test like it is and I won't switch to non filesize-accurate 1-pass encoding only to save some time and I, like most people here would accept a 200% higher encoding time for 5% better quality, I guess

Most of the people would, but I have only one PC and some of the movies I backup aren´t important enough to waste 2h wth them, but important enough to backup. btw, I think the results are much better than cbr-results (and like most of the movies floating around the web), but I see, here is the wrong place to suggest a way like this.

@dark-cracker:
still interested? I would send How I think Icould do it via pm.

A_Pleite

[Dark-Cracker]

@A_pleite
i am still interested, i wait your private msg with all the informations needed for integrate your methode.

@TheReal
i know your logo because a fiend have the same picture in it room with the year 1943 :)

[diji1]

Sure, we all believe you ... ;). mwahahaha
i know your logo because ...

[Dark-Cracker]

@A_pleide

Have u stop your methode ?
else check your pm.

@diji1
no hard drug for me :)
only alcohool, cigaret and pc are my sweet drugs :)

[diji1]

@dark-cracker
yeah, me too ... well, mostly, but it aint the forum i guess to talk of such things :P. sorry for the ot nature of post, hehe ...