is VBR always better can CBR?

[ou8thisSN]

I shoot home DV video that I convert to DVD. I use CCE to do this, and lately i've been experimenting with filters to clean up some of the CCD grain added in my tapes. At present it takes close to 36 hours to encode for every 1 hour of DV video. This is for 5 pass CCE VBR 9500 max, 8000 avg, and 3000 min.

I was just thinking to myself, I only put 1 hour at most, on a DVD and I never go over.. the sizes are always around 3.5-3.8gb on each disc, meaning there is room left over most all the time.

Is it always better to do a multipass VBR encode over a constant rate CBR? Do people use VBR only as a balance between great quality and fitting a certain length on a DVD?

I could probably encode the whole 1 hour at 9000mbps @ CBR, and be done a lot faster....

basically all i'm asking is that is there an inherent benifit doing a VBR over CBR, when size of the final video file isnt a factor?
obviously I would like the answer to relate to getting the best quality possible.

[Boulder]

With such high bitrates, you won't see a difference between CBR and VBR. However, I'd do a one-pass VBR (not OPV) encode as it'll probably make it easier for your standalone to cope with the high total bitrate.

OPV's good too but it's harder to tweak so that it doesn't choke the standalone at high avg bitrates (sometimes the max bitrate is not respected).

[Mug Funky]

what's your audio? those bitrates look a bit high - not leaving much headroom for underflows/bad drives/bad discs.

5 passes is just too much. there's no point getting a fast encoder like CCE to go so slow, especially at high bitrates like that.

try CBR 8500. this has headroom for audio and muxing, and is high enough that difficult scenes will still look fine.

also, if you're doing noise reduction and insist on so many encoder passes, try export to a lossless file first so you're not wasting CPU time by denoising 5 times as well. this method takes more hard disk, but means you denoise once and encode as many passes as you like. bit of a time saver.

[ou8thisSN]

I dont have much on the disc, only 1 hour of video per DVD5... I only have 1 dd2.0 audio track, and a simple menu. size has never been an issue, thats why i encode so high...

how do I do what you're suggesting about denoising once, then encode? I have no idea how to do that....

right now, i capture the dv avi files using WinDV, join in Virtualdub, then encode in CCE using my script below, which would take 36 hours (only experimented with a 5 min test clip so far)

if it helps here's my script:

#LoadPlugin("D:\Filter\DeGrainMedian.dll")
LoadPlugin("D:\Filter\RemoveGrainS.dll")
LoadPlugin("D:\Filter\RepairS.dll")
LoadPlugin("D:\Filter\SSEToolsS.dll")
Import("D:\Filter\DeHalo_alpha.avs")
Import("D:\Filter\LRemoveDust.avs")
aviSource("D:\Test Avi\test.avi")
ConvertToYV12(Interlaced=True)
SeparateFields()
#DeGrainMedian(limitY=2,limitUV=3,mode=1,interlaced=true)
DeHalo_Alpha()
LRemoveDust_YV12(10,1)
Weave()

[Boulder]

how do I do what you're suggesting about denoising once, then encode? I have no idea how to do that....

It means opening the script in VirtualDub and saving in lossless format, HuffYUV is the most common one. Download and install a recent ffdshow build which includes a YV12-capable version of HuffYUV.

right now, i capture the dv avi files using WinDV, join in Virtualdub, then encode in CCE using my script below, which would take 36 hours (only experimented with a 5 min test clip so far)

Your script needs some tweaking. First of all you should download and install the Cedocida DV codec and set its decoding for YV12 compatible with DV (sorry, can't explain any better as I'm not on my own computer at the moment). This way you'll get proper YV12 output from AVISource. You should also use a smart bobber before filtering and then reinterlace afterwards.

#LoadPlugin("D:\Filter\DeGrainMedian.dll")
LoadPlugin("D:\Filter\RemoveGrainS.dll")
LoadPlugin("D:\Filter\RepairS.dll")
LoadPlugin("D:\Filter\SSEToolsS.dll")
Import("D:\Filter\DeHalo_alpha.avs")
Import("D:\Filter\LRemoveDust.avs")
aviSource("D:\Test Avi\test.avi")
AssumeBFF()
TDeint(mode=1)
DeHalo_Alpha()
LRemoveDust_YV12(10,1)
ConverttoYUY2() # for CCE
AssumeBFF()
SeparateFields()
SelectEvery(4,1,2) # to get TFF output
Weave()

The script won't be fast so you definitely should use the lossless intermediate file way if you do multipass encoding. But as Mug said, you can safely use CBR with high bitrates like yours.

[ou8thisSN]

okay, before you typed that my script needs tweaking, another forum member suggested that since I'm using NTSC, i should add the Reinterpolate411 function to my script. as it stands this is what i'm using:

#LoadPlugin("D:\Filter\DeGrainMedian.dll")
LoadPlugin("D:\Filter\RemoveGrainS.dll")
LoadPlugin("D:\Filter\RepairS.dll")
LoadPlugin("D:\Filter\SSEToolsS.dll")
Import("D:\Filter\DeHalo_alpha.avs")
Import("D:\Filter\LRemoveDust.avs")
aviSource("D:\DV Transfer\I Tape 1 Vdub.avi")
ConvertToYUY2
ReInterpolate411()
ConvertToYV12
SeparateFields()
#DeGrainMedian(limitY=2,limitUV=3,mode=1,interlaced=true)
DeHalo_Alpha()
LRemoveDust_YV12(10,1)
Weave()


thats the addition in italics. anyway before you posted your script, I used this script to transcode in virtualdub and saved it as a direct stream. is that not lossless? i think it said something like YUY2. (not very technical, i'm new to this stuff). Why do i need to install HuffYUV? isnt a direct stream lossless?

Your script needs some tweaking. First of all you should download and install the Cedocida DV codec and set its decoding for YV12 compatible with DV (sorry, can't explain any better as I'm not on my own computer at the moment). This way you'll get proper YV12 output from AVISource. You should also use a smart bobber before filtering and then reinterlace afterwards.

what are all the additions in your script? What makes you think that i'm not getting proper YV12 now? does it make it just more efficient or does it do some essential process?


finally, what script should i use when i encode in cce?

avisource("D:\Test Avi\filtered vdub output.avi")

?

[Boulder]

okay, before you typed that my script needs tweaking, another forum member suggested that since I'm using NTSC, i should add the Reinterpolate411 function to my script. as it stands this is what i'm using:

A-ha, you're an NTSC user. Reinterpolate is then the way to go, you'll get YUY2 out of that. Cedocida supports NTSC as well which would probably make ConverttoYUY2 and Reinterpolate unnecessary. I'm not sure as I'm from a PAL country.

thats the addition in italics. anyway before you posted your script, I used this script to transcode in virtualdub and saved it as a direct stream. is that not lossless? i think it said something like YUY2. (not very technical, i'm new to this stuff). Why do i need to install HuffYUV? isnt a direct stream lossless?

If you don't use any compression codec, it is lossless - it'll just consume HD space much more than using HuffYUV. If you're happy with it, by all means go that way. However, your script gives you YV12 due to ConverttoYV12().

what are all the additions in your script?

Basically smart bobbing (TDeint) and reinterlacing (Separatefields etc.)

finally, what script should i use when i encode in cce?

You can feed the avi file directly in CCE.

[ou8thisSN]

isnt the whole point that using a compression codec inherently makes it lossless? I mean, i dont really care if it takes up another 13gb on my HD as an intermediate file. Also, wouldnt using a compression codec also take additional time, than to leave it uncompressed? if there's anything i'm trying to save, its time.

If you're happy with it, by all means go that way. However, your script gives you YV12 due to ConverttoYV12().

so, leaving it as YV12 isnt suitable for CCE? CCE "likes YUY2" ? I'll just take your word for it.

so keeping in mind that i'll be using NTSC, would my final script look like this?

#LoadPlugin("D:\Filter\DeGrainMedian.dll")
LoadPlugin("D:\Filter\RemoveGrainS.dll")
LoadPlugin("D:\Filter\RepairS.dll")
LoadPlugin("D:\Filter\SSEToolsS.dll")
Import("D:\Filter\DeHalo_alpha.avs")
Import("D:\Filter\LRemoveDust.avs")
aviSource("D:\Test Avi\test.avi")
ConvertToYUY2 #### to make it acceptable for ReInterpolate
ReInterpolate411()
AssumeBFF()
TDeint(mode=1)
ConvertToYV12 ####to make it acceptable for LRemoveDust
DeHalo_Alpha()
LRemoveDust_YV12(10,1)
ConverttoYUY2() # converion again back for CCE
AssumeBFF()
SeparateFields()
SelectEvery(4,1,2) # to get TFF output
Weave()

I thought CCE couldnt handle AVI files natively, is there any quality difference in loading the AVI file directly in CCE vs through Avisource (in Avisynth)?

also, why is TFF output important?

Basically, i'm using Fred Thompson's script. I respect his input a lot and I am way over my head in this stuff, so I'm just a little confused that you suggest I should be changing that script. I asked him also what he thinks. I would just like to make the process as painless as possible.

[Boulder]

Also, wouldnt using a compression codec also take additional time, than to leave it uncompressed? if there's anything i'm trying to save, its time.

Yep, though your HD will have to work quite a bit harder. On a newer HD, this shouldn't be a problem.

so, leaving it as YV12 isnt suitable for CCE? CCE "likes YUY2" ? I'll just take your word for it.

CCE deals in YUY2. You can feed YV12 in it but it'll be converted to YUY2 before encoding (provided that your system has a codec that can do the conversion), which will then convert it back to YV12.

so keeping in mind that i'll be using NTSC, would my final script look like this?

#LoadPlugin("D:\Filter\DeGrainMedian.dll")
LoadPlugin("D:\Filter\RemoveGrainS.dll")
LoadPlugin("D:\Filter\RepairS.dll")
LoadPlugin("D:\Filter\SSEToolsS.dll")
Import("D:\Filter\DeHalo_alpha.avs")
Import("D:\Filter\LRemoveDust.avs")
aviSource("D:\Test Avi\test.avi")
ConvertToYUY2 #### to make it acceptable for ReInterpolate
ReInterpolate411()
AssumeBFF()
TDeint(mode=1)
ConvertToYV12 ####to make it acceptable for LRemoveDust
DeHalo_Alpha()
LRemoveDust_YV12(10,1)
ConverttoYUY2() # converion again back for CCE
AssumeBFF()
SeparateFields()
SelectEvery(4,1,2) # to get TFF output
Weave()

Yep, looks OK to me.

I thought CCE couldnt handle AVI files natively, is there any quality difference in loading the AVI file directly in CCE vs through Avisource (in Avisynth)?

There shouldn't be a difference.

also, why is TFF output important?

It is thought to be better compliant with standalone players as interlaced commercial DVDs are TFF most of the time.

[ou8thisSN]

well thank you for explaining that to me. I really appreciate it. I'm really trying to understand why plug-ins are needed and what they do, rather than just blindly using a script someone posts.

I was also wondering, does adding those suggestions to this script add any additional noise or artifacting or processing time or just makes things more compatible... for cce, because i see you have called AssumeBFF() twice. i'm sure there's a reason, but wont calling additional functions take a lot more time for vdub to process?

finally, what exactly does TDeint(mode=1) do? smart bobbing... that means nothing to me, since i'm functionally illiterate in this subject. That wont add any noise or anything right?

Thats the main reason i'm using Fred's script, is to get rid of CCD noise.

Also, my brain is fried, i knew what TFF (Top Field First) meant. I just am so tired i forgot what it was. I think i'd prefer to leave it BFF, as there's an option in CCE to convert it to TFF. if i convert it here, then i'll have to remember to turn that off in CCE. but if i were to leave it in, is there any difference between SelectEvery(4,1,2) and DoubleWeave().SelectOdd()

[Boulder]

As I said, the script will be slow. If you want quality, you need time..

AssumeBFF before TDeint tells it the field order and the second AssumeBFF makes sure Avisynth knows the field order before reinterlacing. Calling them doesn't add to the encoding time, they just set a specific flag in the decoded stream.

Smart bobbing means that the video is bob-deinterlaced - that is - fields separated and resized to full height. A stupid bob treats all parts of the frame as interlaced whereas a smart bob will detect areas in motion and will treat them as interlaced and the static areas as progressive. This improves quality. If you want a faster smart bob, you can use LeakKernelBob(order=0,sharp=true,threshold=4) instead of TDeint but the quality will not be as high (still, you might not notice the difference).

If you used certain other denoiser than LRemoveDust (like TemporalSoften), you could get away without smart bobbing and could use the SeparateFields - Weave combo you originally tried. Then again, colorspace conversions will be of better quality when they are used in a smart bobbed stream.

[ou8thisSN]

well, i just went to my desktop to implement your changes, and found my computer suffered an error in VirtualDub.

The source file was 13gb, but the HDD had run out of space, it finished at 26gb.

I dont understand how filtering a file would cause it to be so much bigger than the original... shouldnt it be around the same size?

i set virtualdubmod to "direct stream copy"

[Boulder]

Doublecheck in VDubMod that the framerate of the script is the same as in the source.

EDIT: if the source is DV and the output uncompressed, there's the difference.

[ou8thisSN]

the DV avi file itself gives me a 13.1gb file for ever 1 hour of video... when i edit this file together in vdub, and do a direct stream copy, its gives me another 13.1gb file. are you saying that this will be FURTHER uncompressed to a file larger than 13gb? if so, then I must use your lossless codec to compress.

i dont understand what you mean by "check your framerate". I make no mention of frame rate in the script, should i have? its NTSC standard of 29.976 or something... but i havent checked. how do i do this?

edit: i just checked my test avi file. it was being filtered in vdubmod, in "direct stream copy" and at 20% it was already about double the original file size of 1gb. That means for a 13gb file it would take 65gb???? thats insanely huge... can that be correct or am i doing something wrong?

[Boulder]

Because when you simply cut (or join) the video streams in VDub and use direct stream copy, there is no need for recompression. Avisynth will always decode stream which means that you have to compress the stream again even if you use direct stream copy. Using that just means that there won't be a YUY2->RGB32->YUY2 conversion but the video stream stays in YUY2 (or whatever the script outputs).

Nevermind the framerate, the reason for the filesize is the difference between DV and uncompressed streams. Using HuffYUV will save you some diskspace, most of the time the efficiency is about 2.0:1.

[ou8thisSN]

okay here's an update. 1. i installed the HuffYUV codec to save the space as you suggest.

2. few more changes were made to the filter script

#LoadPlugin("D:\Filter\DeGrainMedian.dll")
LoadPlugin("D:\Filter\RemoveGrainS.dll")
LoadPlugin("D:\Filter\RepairS.dll")
LoadPlugin("D:\Filter\SSEToolsS.dll")
Import("D:\Filter\DeHalo_alpha.avs")
Import("D:\Filter\LRemoveDust.avs")
aviSource("D:\DV Transfer\Test Avi\test.avi")
ConvertToYUY2 #### to make it acceptable for ReInterpolate
Reinterpolate411()
ConvertToYV12(Interlaced=True)
SeparateFields()
DeHalo_Alpha()
#DeGrainMedian(limitY=2,limitUV=3,mode=1,interlaced=true)
LRemoveDust_YV12(10,1)
weave()
ConvertToYUY2 ###ReverseFieldDominance requires YUY2 or RGB
ReverseFieldDominance()


The smartbob was removed because my displays are natively progressive anyway, is there a point to do all these interlace/deinterlace steps if the display itself will always deinterlace any input signal? i dont really understand a lot of whats going on in the script, but Fred suggested that in certain instances, the smartbob in conjunction with this filter can produce undesirable effects. I dont really have time to watch the whole clip on my computer before burnign to DVD-R.

I was pretty happy with the output before. All i really wanted was to remove some of the CCD "grain" that my camcorder introduced into the avi stream. if this can be softened or eliminated, then thats all i want. is your opinion, that Smart-bob is ABSOLUTELY essential to this for the filtering to work at its highest? or do you also agree that there is a change that: yes it could work very very well or in some cases it can look really bad depending on how it interacts with the filter.

i know very little about any of this stuff and i'm getting conflicting opinions from you and fred. judging by both of your advices so far, you both seem to know a great deal on the subject.

also, you mentioned using a Cedocida DV codec. How is this better or better suited than the Panasonic DV codec I already use?

thanks for all your help and attention in this thread. I really apperciate it.


EDIT: also, what mode should I use when saving the new output in virtualdub using the huffYUV codec? Full processing, fast recompress or Normal Recompress?

[Boulder]

If you wish to use the SeparateFields()-Weave() combo with those filters, do it this way:


#LoadPlugin("D:\Filter\DeGrainMedian.dll")
LoadPlugin("D:\Filter\RemoveGrainS.dll")
LoadPlugin("D:\Filter\RepairS.dll")
LoadPlugin("D:\Filter\SSEToolsS.dll")
Import("D:\Filter\DeHalo_alpha.avs")
Import("D:\Filter\LRemoveDust.avs")
aviSource("D:\DV Transfer\Test Avi\test.avi")
ConvertToYUY2 #### to make it acceptable for ReInterpolate
Reinterpolate411()
ConvertToYV12(Interlaced=True)
SeparateFields()
evn=SelectEven().DeHalo_Alpha().LRemoveDust_YV12(10,1)
odd=SelectOdd().Dehalo_alpha().LRemoveDust_YV12(10,1)
Interleave(evn,odd)
weave()
ConvertToYUY2(interlaced=true) ###ReverseFieldDominance requires YUY2 or RGB
ReverseFieldDominance()

You shouldn't use LRemoveDust on simply field-separated stuff as it's a spatio-temporal filter. Use it either this way or with smart bobbing.

You can search the forum for information on how to treat interlaced material properly, and smart bob-filter-reinterlace is the way to do it.

You can read about Cedocida in the DV forum, there's a sticky thread that contains all the information. I would definitely try it out, it might save you from the YUY2->Reinterpolate411->YV12 thing. Doing that many conversions might begin to show up in the red colored areas for example.

Use the fast recompress mode. It's the only one that allows you to use a codec for compressing and won't do any extra colorspace conversions.