Xvid for a 2GB rip? [Archive]

Satlover

3rd December 2002, 12:23

Hi All,

Before I start, I just want to say that I do read the forum and I am not lazy to test my own settings.

I normally go for a 3CD rip or a target of 2GB (so I can put 2 on 1 DVD-R) I used to use nandub and DivX 3.11 to do this with excellent results. However I moved to Xvid because Dix3.11 had problems sometimes when something moved in a scene change, like you would see a trail behind a moving object. Xvid does not have this problem.

At the moment I am following Iago's XviD 2-pass walkthrough http://nic.dnsalias.com/XviD/xvid-two-pass.zip

Question,
Does a 2GB rip make any sense at all if it does, then has anyone got an settings for this type of rip because we are always seeing settings for users who try to squash a film onto an 80min CD-R ?

Maybe some one has a guide for a 3CD rip .

Thanks
Dave
:D

sam_b

3rd December 2002, 13:13

I would suggest (assuming you know avisynth plugins):

A 720x or 704x resolution with lanczos resizing. Only if your decoding machine can handle it, of course. There are not many reasons to go over that, especially with qpel.

MPEG quants. Ne real reason to use H263 that I see, especially if you're used to SBC. Try H263 though if you get blocks anyway. Don't use lumi-masking.

2pass linear scaling. You stand the risk of saturating the codec on some films anyway, at least in my experience, so I see little point in curve compression. Try koepi's statsreader thing if you want, lets you know first pass size easily, and scales well.

You might like to set max bitrate up a bit.

And if you're using koepi's 0212 build, i'd use qpel and cm if you don't mind the speed. You can experiment with B-frames, poss 2/100/100 or 2/150/100, but you'll need to test if they help you.

Just my thoughts. Good luck.

iago

3rd December 2002, 13:32

@Satlover

Just a bunch of humble opinions for a 3CD rip:

* Use the highest resolution possible (if your system is powerful enough to decode it! ;))
* Do NOT use any extra features such as b-frames, qpel, lumi masking, etc.
* Do NOT use any extra filtering.
* Use MPEG quantization type for both passes (if the second pass is still necessary at all for a 3CD rip ;))
* Use Linear Curve Scaling!
* Use Motion Search Precision 6 (maybe together with Chroma Motion for even better PSNR ;))
* Leave Minimum I-frame interval at its default -> 1
* Do not use any special credits treatment.
* Cap the quantizers as you like! :D (2-4/2-8, 2-6/2-12, 2-31/2-31, etc.)
* Uncheck "Discard first pass" box, so you can use the first pass avi file if it's within your target size for 3CDs.

You should undoubtedly get perfect results!

regards,
iago

serbersan

3rd December 2002, 16:55

Originally posted by iago
@Satlover

* Do NOT use any extra features such as b-frames, qpel, lumi masking, etc.
....
regards,
iago

Sorry, but Qpel??? I thought qpel couldn't hurt the image like b-frames and lumi masking could do.

It's because it current development status or it's forever???

Thanks

edit: In XviD has qpel the same limitations that in Divx (limitations that are refered in Divx official page I mean a minimum bitrate for a resolution etc...)???

sam_b

3rd December 2002, 16:58

@iago

Do you think that qpel harms the image at high resolutions and low quants? I havn't tested much at such high bitrates, but I have found for me it to helps pretty much universally. Are you worried about stream-compliance?

Would be interested to know what you think.

iago

3rd December 2002, 17:34

@sam_b and serbersan

Regarding the QPel issue:

The reason why "I" wouldn't use QPel for a 3CD encode "for the time being" (until it proves its maturity ;)) is just that I find it a bit risky especially in terms of decoding, that's all! ;)

Of course it's worth a try and compare the results with and without it, not only frame by frame, but especially by playback, and go for what you like the best.

regards,
iago

mikeson

3rd December 2002, 18:58

@iago:

* Do NOT use any extra filtering.

In my experiences most DVDs were not mastered very well. I can see noise (most visible in static scene on solid blocks - for example wall). So in my encodings I mostly use some filtering. I prefer Convolution3D with movieHQ settings (or some tweaking). It improves visual quality because codec doesn't need to spend valuable bytes for reproducing noise in final encoding. But I know - it is very subjective and matter of taste.

I know 3CDs are plenty of space for movies that has 1 and 1/2 hour but what about Lord of the Rings EE or Pearl Harbor? I think it is more difficult situation. But I'm sure you definitely know what I'm talking about. ;)

iago

3rd December 2002, 21:20

@mikeson

Well, when it comes to noise, my point of view is usually contrary to the general opinion ;); that is, I prefer to keep the source noise as much as possible, which I think is one of the most effective ways against macroblocks. However, as you also have pointed out, it's a matter of taste.

And unfortunately those solid blocks and flat undetailed surfaces that you mention in your post are the first areas where macroblocks will appear in an encode, and even more annoyingly as soon as you filter/denoise those parts.

That's why I would prefer "MPEG quantization" and "no filtering" for pretty high bitrate encodes such as a 3CD one (though I personally never aim for more than 2CDs).

regards,
iago

mikeson

3rd December 2002, 22:22

@iago:

I agree with you, noise is very effective against macroblocks. But let me controvert a little bit. Noise that I want to remove with C3D is "moving" noise (ie noise, that is different in one frame then it was in previous). But noise, that prevents macroblocks (and I would like to keep it or "create" it) is "static" noise (ie noise pixels in current frame are on the same position as they were in previous). I think that this will spare bitrate but keep details, or am I wrong? Maybe I am... ;)

What do you think?

sam_b

3rd December 2002, 22:36

I can see where you're coming from, but since almost most source noise is vaguely gaussian, I do not believe the two can be so easily separated. Also, AFAIK, MPEG P/B frames only care about temporal noise, as static noise (ie. a pattern) would require no alteration from the previous frame. There would be no difference in blocking if it was present in the I-frame. It is the temporal noise which is needed to ensure the codec 'sees' enough difference between individual frames for each Mblock to be allocated bits other than for DC (flat colour).

May be way off with this though.

Gazza

4th December 2002, 03:05

I've just completed a small test that might have some relevance here. In the past I have been committed to 2 cd encodes - more interested in quality rather than space. I just did two encodes, one was 2 cd and the last was 1 cd. I used my copy of Deep Impact to do the test. Comparing the results, there wasn't too much visible difference between them in terms of visible quality. The 1 cd encode maybe had a little more movement on large areas of flat or similar video (e.g. walls & sky, etc).

The point is, with 3 cd encodes, I would expect to see almost dvd quality anyway (unless it was a very very long video source). With 2 cd's the quality I think is pretty damn good. With 1 cd the quality has improved remarkably in the last few weeks (thanks to the developers) and could be on par with 2 cd encodes? My assumption is that quality increases going from 1 to 2 to 3 cd's become less and less as you increase the numbers of cd's. Therefore is there a tool that you could use that will let you know what would be the best numbers of cd's to go for? I know there is a few factors that impinge on the final file size. But it would be nice to know that I can go for a 1 or 2 cd encode without multiple encode sessions.

By the way I used qpel, bframes (3/150), chroma motion and linear compression in the tests and I must say it looked very nice in 1 or 2 cd's.

cjv

4th December 2002, 03:55

Originally posted by Gazza
Therefore is there a tool that you could use that will let you know what would be the best numbers of cd's to go for?
Hmm..maybe I'm not understanding you fully, but isn't that exactly what GKnot and the Comp. check is designed for? With XviD, you just load the stats file manually.

cjv

soujir0u

4th December 2002, 06:15

Originally posted by mikeson
@iago:

I agree with you, noise is very effective against macroblocks. But let me controvert a little bit. Noise that I want to remove with C3D is "moving" noise (ie noise, that is different in one frame then it was in previous). But noise, that prevents macroblocks (and I would like to keep it or "create" it) is "static" noise (ie noise pixels in current frame are on the same position as they were in previous). I think that this will spare bitrate but keep details, or am I wrong? Maybe I am... ;)

What do you think?

Use a temporal noise remover then, such as TemporalSoften, TemporalSmoother, etc. That should remove the "moving" noise...

Gazza

4th December 2002, 07:20

Originally posted by cjv
Hmm..maybe I'm not understanding you fully, but isn't that exactly what GKnot and the Comp. check is designed for? With XviD, you just load the stats file manually.

cjv

cjv, thanks for the hint, I know that Gknot has this capability but I've never used it before. Up to now I could not see the value as I always went for 2 cd's and they seem pretty good anyway. The decision to go to 1 single cd is recent and based on the excellant advance of xvid. Therefore I really only use Gknot to generate the avs file which I then manually update. I'm in no way an expert on it's use as I only use it at the most rudimentary level.

I understand that Gknot doesn't handle xvid (only allows for dix3 & divx4). Therefore not sure that Gknot will give me a true indication as to the quality against final file size for a xvid encode.

When I tried the comp test (as per your much valued suggestion), unfortunately and for some unknown reason, Gknot crashes. I must be doing something wrong. I have searched and found some guidelines on using the Comp test within the forum but it still crashes after I followed them- weird. For some reason it doesn't like the d2v file outputted by dvd2avi. I'm using Gknot 0.23 and dvd2avi 1.86.

[edit] further explanation of problem - after using smartripper and dvd2avi to get the *.d2v, Gknot comp test crashes when trying to run. Error message comes back with 'unknown error'.
[edit 2] I've just done it again and Gknot reports that the d2v file is the wrong format. However the d2v can be used quite happily when doing normal encoding, etc and not the comp test. I've run out of ideas...

My comment regarding a simple tool for deciding if a 1 or 2 cd encode suits a particular video source was that something like this could be added into Nic's video calculator for example? Or maybe some of the recent xvid analysis tools? I'm sure that Gknot does a very good job but I'm having problems with it at the moment.

Anyway, thanks for the suggestion.

PS any suggestions on what might be causing my problems with Gknot would be much appreciated.

Satlover

4th December 2002, 10:57

Thanks allot :D to everyone who took time to reply. I have still got allot to learn! I tried a few things last night, here is what happened.

- I created a VFAPI AVI for frame serving and loaded the AVI into VirtualDub 1.4.13 and Xvid 0.9.0
- I used the built in resize filter in Virtual dub and changed it to lanczos3. Kept the resolution to same as the source which is around 718x485.
- I followed iago’s and SAM_B information
- I was not sure about the Capping of the quantizers like (2-4/2-8, 2-6/2-12, 2-31/2-31, etc.) so I left these as default.
- I could not find the Linear Curve Scaling :-0 (need to look again this evening)
- I used the Motion Search Precision 6 together with Chroma Motion
- I left the Minimum I-frame interval at its default -> 1
- Use MPEG quantization type for both passes

Playback of the AVI
-------------------
Normally when you open a Xvid AVI without any sound track into zoom player you get 2 video screens, one is a direct show window (I think) and the other one is from Zoom player. When you mux the audio and open it again it plays back as normal you don’t get the extra screen. To my surprise I could see the 718x485 AVI in the direct show window but not in Zoom player window which had colored lines across it. When I muxed the AVI with the MP3 sound track I had the same problem with the colored lines. (I need to test this with the new beta build of zoomplayer 300b2.zip) but I think it is a general problem and would occur in Windows media player aswell

Possible solution to the problem:
-> I think I need to get away from VFAPI and start using Avisynth, I could create the AVS file with Gordon Knot, keep the same resolution as the DVD and just crop the borders? Then I can use the fast recompess in Virtual dub instead of the full processing mode

->Do the resizing with Avisynth with the lanczos3 filter ? SAM_B mentions that I can use the lanczos3 filter with Avisynth – not sure how to do that.

->Do everything in Virtual dub. But I need to use a resize filter in order to crop the borders. Maybe there is just a filter just for cropping because I don’t need to resize if I want to keep the
orginal size.

Quality
-------
It was hard to tell what the quality was like because I could only expand the direct show window which did not keep the aspect ratio, so the screen was stretched a bit. The quality was not very good but I can work on that as soon as I get the AVIsynth sorted out and the playback to work with these high resolutions.

Ran on AMD AthlonXP 1900 and Geforce4 TI200

Personally I think CDR is old and we need to take advantage of 2GB (filesize limit for DVD-r)

Satlover (Dave):D

sam_b

4th December 2002, 11:43

@satlover

Yes, you really want to use AVISynth. Trust me. And you don't need GKnot.

Here's a valid example:

LoadPlugin("e:\program files\avisynth2\plugins\mpeg2dec3.dll")

vid=mpeg2source("e:\myfile.d2v",CPU=0,iDCT=2)
vid=LumaFilter(vid) #(for iago :-) same as lumaoff=-2
vid=Crop(vid,28,74,-24,-76) #(keep even crop values, at least horizontally)
vid=lanczosResize(vid,752,400) #keep as multiples of 16
#vid=bicubicresize(vid,752,400,0,0.5) #you might like this
Return vid

You'll need to dig up a recent copy of mpeg2dec3 and read the docs but this should get you started.

As for the weird playback, you probably have some rogue DShow filter around. Poss an old DVobSub???? The fact that your resolution is not made of multiples of 16 may cause you problems too. Use MOD16.

Linear curve scaling means using Koepi's statsreader (search) or settting No AltCC, Curve compression to 0%,0% during the second pass.

Assault

4th December 2002, 12:12

@satlover

as for the playback, download ffdshow from http://sourceforge.net/project/showfiles.php?group_id=53761&release_id=119280 and check both xvid and use xvid in ffdshow. Then your video should be played well.

Satlover

4th December 2002, 13:23

Thanks allot for your replys, yes I thought the ffdshow might do the trick, I will check that later. I found the mpeg2dec3 yv12.zip http://forum.doom9.org/attachment.php?s=&postid=203353
I hope I can use this like mpeg2dec3.dll

mikeson

4th December 2002, 21:28

Use a temporal noise remover then, such as TemporalSoften, TemporalSmoother, etc. That should remove the "moving" noise...

Why not use C3D? It works fine for me...

Satlover

4th December 2002, 21:32

@mikeson

I looked for the C3D filter for AVIsyth. I could not seem to find it
need to look into the forum for some links. I could not find the mpeg2dec3.dll either

thanks

cjv

4th December 2002, 21:56

Originally posted by Gazza
I'm using Gknot 0.23 and dvd2avi 1.86.
This might be the problem. Try using Gknot 0.26 and dvd2avi 1.76 (they can be used without disrupting your current install) Also, make sure you have the ORIGINAL mpeg2dec.dll that came with Gknot 0.26 in the same directory as Gknot (when trying to load the .d2v file)

Basically, just add the line
SelectRangeEvery(280,14) to the end of you current avs script. Do the first pass in VDub and call the stats file "Quick_640x272.stats" or something similar. It will only take approx. 5 min. Then go to Gknot, and load that stats file. It will switch to Divx3 in the upper checkbox, but that's ok. Just play with the resolution slider, # of CDs, audio bitrate, etc. I find that with the % around 50%, it makes a great 1CD rip. Taking it up to 70% makes a superb 2CD rip.

One more thing...after you have completed your entire first pass, you can load the whole file, and find out EXACTLY how well your second pass will compress.

I have never had gknot crash on me, although I rarely use it any more.

cjv

Gazza

4th December 2002, 23:11

Originally posted by sam_b
@satlover

Yes, you really want to use AVISynth. Trust me. And you don't need GKnot.

Here's a valid example:

LoadPlugin("e:\program files\avisynth2\plugins\mpeg2dec3.dll")

vid=mpeg2source("e:\myfile.d2v",CPU=0,iDCT=2)
vid=LumaFilter(vid) #(for iago :-) same as lumaoff=-2
vid=Crop(vid,28,74,-24,-76) #(keep even crop values, at least horizontally)
vid=lanczosResize(vid,752,400) #keep as multiples of 16
#vid=bicubicresize(vid,752,400,0,0.5) #you might like this
Return vid

@ sam_b, if you don't use gknot how do you decide on valid values for resolution and cropping....? Gknot seems to provide various feedback indicators that should help users with correct decisions here?

Gazza

sam_b

5th December 2002, 03:07

I load the unprocessed clip into vdub, and use it's filters to crop, then I copy the numbers across into to avisynth. I use a calculator to calculate the resulting aspect ratio after cropping, compensating for non-square pixels. Then I test compressibility by using fixed quality mode on samples, and looking at the size/quality of the clips to determine resize method/resolution/matrix etc...

After picking the resolution, I adjust the horizontal crop slightly to get the right aspect ratio, and encode. Quality of a clip is not determined solely by the bits/pixel/frame ratio.

Hope this make sense.

Gazza

5th December 2002, 07:48

@ cjv,

I used your advise with success and got the following results;

736x304 60.8%
704x288 65.3%
672x288 66.9%
640x272 72.1%

However, does this mean that a lower resolution (~70%) fits better on a 2cd encode (your advise was that 70% was good for 2 cd's)? I would have thought it was the other way round. Go for as big a resolution that you could squeeze out considering you have twice the space available compared to a 1 cd encode?

Anyway, if you don't use Gknot anymore then could I ask what methodology you use now? It seems that Gknot has been overtaken in past months as the preferred tool of choice.

cjv

5th December 2002, 12:37

Originally posted by Gazza
736x304 60.8%
704x288 65.3%
672x288 66.9%
640x272 72.1%
However, does this mean that a lower resolution (~70%) fits better on a 2cd encode (your advise was that 70% was good for 2 cd's)?,
These numbers are just approximations...I wouldn't get too hung up on comp. checks.
If these numbers are for Deep Impact, you mentioned blocks on walls, this is most likely due to a small amount of noise messing with the codec. You must either add more noise..or eliminate it entirely! I choose to eliminate noise for 1CD. I would personally pre-filter with c3d or fluxsmooth, try a Bicubic(0.5), and a resolution of 576x.

For 2 CDs, assuming your audio is already accounted for, I would double check you crop values..I bet you used Gknot Autocrop/Smartcrop :) I would try using 704x304, no prefiltering, bicubic(0.6), MPEG, Chroma, 3-bframes/100/150-200, NO qpel.

My method?...Do it all manually.
DVD2AVI->write initial .avs by hand.
Load in VDubMod, zoom 200%, Ctrl-E, crop top/botton by eye hitting F5 to refresh, then sides...make sure all crop values mod 4.
Find a close-up of human faces. Try different resizers/pre-filters..see filtering effects...hitting F5 like crazy to refresh! Do the same with a noisy scene, and again with a scene with lots of movement. Vdubmod is the best thing in a long time. Once I'm happy, just encode to XviD via VDubMod. I find that for 2CD encodes I like 3 bframes/100/150...and for 1CD encodes 3 b-frames/100/200. I always keep b-frame ratio at 100%, and only use the offset. I prefer to do my own prefiltering (instead of using filtering resizers), and almost always use Lanczos or Bicubic(0.6/0.65)..even for 1CD encodes. No qpel yet either...waiting for it to mature.
Hope some of this helps,

cjv

Gazza

5th December 2002, 23:31

cjv

Your approach certainly seems to be a more direct approach (which I like). A few weeks ago I started thinking that there is too many tools and steps in the 'generic' methodology (nothing against Gknot which is still a fine tool). I know I could use the 'one step' approach (autodub, etc) but they seem to lag behind the latest tools, etc that are updated on an almost daily basis. I will give your approach a thorough go and see how I go and what suits me best.

Thank you very much for sharing your experiences with us all.

Gazza

6th December 2002, 02:53

cjv

I like your approach - gives me more control on the final output. I still struggle with deciding what the starting point for the resolution should be. The vts_04_INFO.txt indicates a display mode of 720x576 (obviously 16:9 with black borders top and bottom). Gknot (yes you are correct I have been using Autocrop/Smartcrop) indicates a encode res of 704 x 288 and you have suggested 704 x 304. How do I know to go to 704 x 304 without using Gknot?

Gazza

cjv

6th December 2002, 03:54

@Gazza:

There is nothing wrong with using GKnot..especially if you are having trouble cropping by eye (which may not be the best for you...you asked me how I do it though)

Although it's possible that your movie maintains correct AR with 704x288, it is highly unlikely. It's most likely the combination of Gknot Smartcrop/Autocrop and having ITU-R checked that is screwing your AR. (On a side note, 704x288 will be easier to compress, but I bet Gknot cropped a lot of your movie away!)

To get proper AR with GKnot:
1) uncheck ITU-R standard. We will crop by hand so it doesn't matter which means NO smartcrop/autocrop.
2) Load your d2v, then go to Resolution and select Crop->Pixel. Start by cropping all your black bars from the top/bottom..making sure the values are mod2. A lot of movies will have like a weird "lip", even after you have cropped visible black bars. Crop that off too..it creates smearing and eats bitrate due to unclean crop. Now you can see why I zoom 200% or even 400%...it makes it easier.
3) Crop any crap from the sides..some have black bars and "lips."..keep your crop values mod2 at least.
4) Most likely, the resolution slider in GKnot will now show 704x304. :) Glance at the AR, if its < 2.5% don't sweat it. If you're still concerned, crop evenly off sides/top/bottom to reduce it to whatever you desire. Trying to attain 0.0% is silly and unnecessary.
5) If it's still 704x288, and you've cropped to your liking..then just leave it because you have a rare movie that really is 704x288.

I do it backwards now. I know that I will use 704x304 or 640x272 beforehand. My .avs starts out like this: (note: I don't trust FILM from DVD2AVI so I set it to 29.976 and IVTC manually with decomb)
w=704
h=304
src=mpeg2source("movie.d2v").Telecide(guide=1).Decimate().LumaFilter()
##src=src.crop(x,x,x,x)
credits=src.trim(xxx,xxx).TemporalSoften(1,10,25).BilinearResize(w,h)
movie=src.trim(xxx,xxx).LanczosResize(w,h)
return movie + credits

I then go back and put crop values in manually, refreshing with F5. Once cropped to my satisfaction, if the people looked screwed up and all, then I will go back and change my resize resolution (ie: to 704x288 or something), but I've never found it necessary.
Anyways, this is just my own personal way. Try it if you like it..or not..I'm just giving you options!

cjv

Gazza

6th December 2002, 04:18

cjv,

Thank you once again for taking the time to answer my request for advice. As first read my understanding of what is going on is now clearer. :) However I need to sit down for a couple of hours and play around with what you've told me and get more familiar with these issues. 'Practise makes perfect' as they say.

Reading other threads on the subject of 'Resolution', I think that others also struggle sometimes with these concepts. Maybe it would be useful for newbies (and those wishing to refresh their knowledge) to put this into a faq or 'how to' document (similar to the guideline recently published for xvid encoding)?

Cheers

Gazza

Satlover

6th December 2002, 12:01

After some great help from other in the forum I got some great results for my frame server.

LoadPlugin("C:\DVD\AviSynth2\plugins\MPEG2Dec3.dll")
LoadPlugin("C:\DVD\AviSynth2\plugins\Convolution3DYV12.dll")
mpeg2source("F:\1.d2v")
Sharpen(0.5) # Or tbarry's filter
LumaFilter(-2, 1.02)
Crop(2,70,-2,-70)
Convolution3d (preset="movieHQ")
LanczosResize(720,432)

Going to compress it with Koepi's build XviD-03122002. But maybe I should use the stable buid (to switch off B-frames)becuase I want to follow iago's recomendations

Just a bunch of humble opinions for a 3CD rip:

* Use the highest resolution possible (if your system is powerful enough to decode it! )
* Do NOT use any extra features such as b-frames, qpel, lumi masking, etc.
* Do NOT use any extra filtering.
* Use MPEG quantization type for both passes (if the second pass is still necessary at all for a 3CD rip )
* Use Linear Curve Scaling!
* Use Motion Search Precision 6 (maybe together with Chroma Motion for even better PSNR )
* Leave Minimum I-frame interval at its default -> 1
* Do not use any special credits treatment.
* Cap the quantizers as you like! (2-4/2-8, 2-6/2-12, 2-31/2-31, etc.)
* Uncheck "Discard first pass" box, so you can use the first pass avi file if it's within your target size for 3CDs.

You should undoubtedly get perfect results!

iago

6th December 2002, 12:44

@Satlover,

So finally you got a working script. Nice! ;)
Why don't you try with the following modifications to your script:

----------------------------------------------------
LoadPlugin("C:\DVD\AviSynth2\plugins\MPEG2Dec3.dll")
mpeg2source("F:\1.d2v")
Crop(2,70,-2,-70)
LumaFilter(-2, 1.02)
LanczosResize(720,432)
----------------------------------------------------

Give it a go. If you don't like the results (which is almost impossible imho), try sharpening and/or using Convolution3D then! Also, why not, of course you can use Koepi's 03122002 build.

regards,
iago