Improved CCE Encoding using Huffy, can someone show us how...

[Roveer]

I've read a number of posts that say that using the Huffy codec you can dramatically improve your CCE encoding speed. You also must have gobs of disk space.

I want to try this, but I don't seem to be able to find a post that adequatly describes the process. If you know how, please share so I can give it a try.

Please mentione programs needed and their versions including Huffy. Please mention any custom settings that are required. Please mention process to make this work. I've never worked with huffy before, just dvd2avi, vfapi & avisynth & cce. So if we are going to work with virtualdub, I'll need lots of specific steps.

Thanks a bunch.

Roveer

[auenf]

instead of using avisynth and cce, get dvd2avi to write an AVI using the HUFFYUV codec (which you will have to install beforehand).

then when the file is written, import it into CCE like you would an avs or any avi.

Enf...

[Roveer]

Got it. I'm trying it on a 1gb vob1 right now. From the looks of it, just doing the DVD2AVI Piece would negate any time savings that you would get from faster CCE encodes. Add to that the jigunda disk space requirements and I'm hard pressed to see how we can derive any benefit from this method. But, I'm always one to experiment, so onward we go.

Roveer

[LB]

There is a much better way that that. See, the problem is that if you are going to apply any filters, then the benefit of your inital huffy avi has gone to waste in a sense because you now have filters being used in CCE. SO, how do you correctly use a huffy to maximise your gains and thus minimizing your time during the encode. Read on...

1. Download the huffy codec. I posted one a bit ago in these forums with a link. It's hacked to give you an incredible fast speed over the other versions you find on the net. So get that, unzip it, right click on the .inf? file and select install.

2. Open up virtualdub. You will render the huffy avi using this. Open your media file. It can be a avisynth file, or just a vfapi'd d2v file. Whatever. Just open your media and get your filters in order and get set to encode. Once you are all setup, goto the video tab and select compression codecs. Select the Huffy Avi Codec CCE Optimized.

**** If this is the FIRST time you have used this codec then it must be configured. I will explain how to configure the version I POSTED in these forums to dl. Each release is different so if you don't d/l mine then you have to figure out how to configure it on your own. What happens if you configure it wrong? You will notice a 80% drop in speed. ;p

So to configure mine, press the configure tab after you select that codec. You will notice three boxes. Select the upper two and leave the lowermost box de-checked. That's all there is to it.

3. Time to encode the avi. BEWARE: You need to encode to a segmented AVI file. What's this and why? Our avi will be 5gb for every 20mins you have. So a 2hr movie =~30gb. I have found that even having NTFS installed, CCE is sometimes anal and will not open the 30gb file and gives me some retarded error. The solution I found was to save it as a segmented avi in 2gb segments. So, goto the file tab and select "save as segmented avi". Name your segments and enter 2000mb for each segment. Easy. Sit back and wait now.

4. Now your huffy avi is done. We are going to feed it to CCE now. Open CCE and add your FIRST segment. Double click on the segment and bring up the configuration tab. Inside there is a box that I BELIEVE is called settings. Click on it. It will bring up another box that allows you to add the other segments so that CCE will join them to a final .mpv file. Once you add them all in, just continue using cce as you normally do...

That's all there is to it. Don't lose the forest for the trees. Remember our goal.

1.APPLY FILTERS BEFORE HUFFY ENCODE

2.ENCODE TO HUFFY AVI

3.FEED HUFFY AVI INTO CCE

There are a lot of variables in the equation on how great a difference in speed you will notice from this. Basically, doing a full movie, with moderate+ filtering, and 3+ VBR passes, you will notice a LARGE speed difference.

Hope this helps,

LB

[Roveer]

Thanks for the instructions. Looks like the moderator removed your link to the 'special' huffy.dll Can you provide it again. Maybe just a link, not the actual codec. I really want to give this a try and see what type of speed differences I see. I'm going to build a Dual system and would consider using it since AVISynth isn't going to give me SMP capability.

Roveer.

[LB]

Na, if that's what he (the mod) says he has to do then I'll have to agree with his choice. I didn't get it with any source ;( which is really unfortunate. If I ever talk to the guy that gave me it again I'll ask him about it. But still, huffy is the route to go. At least give it a try, but again, you won't notice any differences unless you time yourself in a FULL movie. A 5min sample won't give you a good indication. So anyway, d/l this copy. The one on doom9 doesn't include the .ini file and it assumes that you already have the huffy codec installed, which means if you are a new-huffy user, that version does you no good.

http://math.berkeley.edu/~benrg/huffyuv-2.1.1.zip

Get that. Install by right clicking on the .inf? file. Then, d/l the doom9 version, rename the dll to huffyuv.dll and overwrite your currently installed version in your system and/or system32 dir. Good, all done. Now the setup is a bit different.

In the configuration screen in vdub for the huffy codec:

Step1.
YUY2 Compression Method = Predict Median (BEST)
* better compression, 5% drop in performance over predict left. If you got the space, then use predict left. Else, if you need to save your HD space, use the Median. Here are the ratios if you care to see:

Predict median 2.77:1
Predict gradient 2.53:1
Predict left 2.71:1

So you decide.

Step2.
RGB Compression Method = Convert to YUY2
* I don't want to explain it, so just choose it. ;p

Step3.
Field Threshhold = 480
*NTSC IVTC'd/progressive material = 480 so pick that, pal progressive is 576. The other two options, 240 & 280 are for the respective interlaced source. But, I don't know why anyone would want to encode interlaced material to begin with. So, choose either 480 or 576.

Step4.
Enable full size output buffer = checked
*Never read up on this so I dunno what it does. Haven't tried de-selecting it ;p so I'd just leave it enabled.

Step5.
Finally, Suggest RGB. I had some compatability issues before and I *MAY* have checked this before encoding. Like I said before, I haven't used this particular version of the codec before so leave it disabled and do a 1min sample. If that sample will open in CCE, then you are good to go. If not, then check this option.
* NOTE: As I said before though, I have run into trouble opening even a 1min sample in huffy format with cce if the sample was not saved as a segmented avi even though it is obviously under 2gb. So, keep that in mind if you run into any errors opening your material in CCE.

That should do it. I can't remember the exact speeds in this huffy version, but I DO know that it is faster than the direct approach of attacking CCE with everything ya got in your .avs file. ;p Give it a shot and post your improvements in say a 2hr 3+ pass VBR movie with a least some attempt of filtering or ivtcin. I bet a lot of people would be interested in seeing it too. ;)

LB

[LB]

Hey rover, I forgot to mention this. This is even more of a reason to build a dual system if you do a lot of encoding. Ya see, as I said in a previous post, Avisynth and Vdub are SMP AWARE if used together. This means that while vdub will not use both of your processors, it will auto-set itself (if you are using WinXP or 2k?) on one processor and avisynth will set itself on the other. So, all your filtering will be done on one while the lossless compression to huffy will take place on the other.

Before I learned about avisynth, I was as avid tmpg user. I ivtc'd my source in there, saved the tpr, vfapid, opend the vfapi 8k avi in vdub and did a no-filter conversion to my huffy avi. I would average about 10-15FPS and it would obviously only be using 1 cpu.

So, because I was doing 20min clips, I would average 15mins for the ivtc and 45 minutes for the encode to huffy.

NOW...

I tried out avisynth a few months ago and have seen the light, or so to speak hah. I use either IT or Decomb for my ivtc, open the avs file in vdub and convert to huffy. My average now is 32FPS and it takes about 14minutes TOTAL. So basically, just in the huffy conversion I've saved 45minutes with avisynth. So yea, dual cpus are very incredible. Even if I didn't encode video, I'd never go back to a single cpu again. But, sticking to the topic of video encoding only, you made the right choice.

LB


**OT: You know, I have a DVD-R so I've graduated from the Divx realm (please dont flame me ;p) and moved on to mpeg2. But, I would be interested to find out if the same thing would happen converting to divx. I don't see why it wouldn't. Applying a ton of filtering and having avisynth (<-- sure is one hella hard word to spell over and over again) render on one cpu while nandub converts to divx on the other. It would be a *GREAT* reason for a lot of the divx users to consider switching over to dual cpus since they can see the benefit too.

[primo]

LB, regarding your CCE method of using multiple segmented AVI's for the final MPV file... what about importing 'chapter frames' into CCE to explicitly specify I-frames where the chapters will eventually be. I usually open up the original IFO in ChapterExtractor and copy and paste the CCE chapterlist to the .ecl file for the project. Would your segmented AVI method pose a problem for me doing this (it would be a pain if I have to offset chapter frames). BTW this all makes me want to go duallie but must wait for budget to catch up hehe!

[Roveer]

Followed your instructions to the T. Unfortunatly, the CCE encodes that I've made are actually taking longer. Here's my information.

Working with a P4 1.6ghz machine with multiple SCSI HD's. Created a 2.0gb huffy avi file with Virtual DUB following LB's instructions.

Keeping in mind, that I'm encoding for DVD, NOT DIVX or SVCD. I don't use filters. I usually just smartrip, DVD2AVI (no filters), VFAPI or AVISYNTH then into CCE for 3 pass encode. I usually get 1.2RT on CCE.

Using the Huffy method I'm getting .9RT on CCE. So I think I've gone in the wrong direction.

If you think I'm doing something wrong, please correct me. Otherwise, I think I'll be staying with my AVISYNTH method. Also, the virtualdub process would take hours on a full length movie. How are we accounting for that? Doesn't matter if we gain 15% in CCE if we lose 45% with vdub.

Roveer

[primo]

LB, regarding your CCE method of using multiple segmented AVI's for the final MPV file... what about importing 'chapter frames' into CCE to explicitly specify I-frames where the chapters will eventually be. I usually open up the original IFO in ChapterExtractor and copy and paste the CCE chapterlist to the .ecl file for the project. Would your segmented AVI method pose a problem for me doing this (it would be a pain if I have to offset chapter frames). BTW this all makes me want to go duallie but must wait for budget to catch up hehe!

EDIT: errrrr... oops sorry for the double post it was a mistake.

[LB]

First to Primo: No, that shouldn't pose a problem. CCE supports file segments and the file segments have 100% perfect segment joining accuracy (if that's even a description for what I'm referring to). But, I don't think it will be a problem. Never tried it before since I don't usually deal with normal movies and chapters. But try it out.


Now to the hard one Roveer: That is interesting. Well, here is the edit since I just figured it out. It's not good news. ;( Here is what I found:

That version on doom will give you .9 On the link I gave you, if you access the main page and download 1.3.1.1, you can get 1.1 with the first two boxes checked. And the unfortunate news is that with the modified version I have you can get 2.1 ;p. That kidna makes me feel bad now. But anyway, now we have an answer to this puzzle. So what we can conclude is, unless you are using heavy filters, such as noise reduction + sharpen + ivtc, then the two available huffy versions will not provide you with noticable speed gains. But, if you have a lot of poor quality movies and are going to be filtering the hell outta them, then download 1.3.1.1 and give it a shot.

Oh well... If I ever get ahold of the source for the version I have I'll post it. But until then... ;(

LB

[Zainal]

Without mass amounts of filtering and the fact that you are encoding at D1 resolution on 1 cpu using Huffyuv isnt really gonna help you. I being playing around with Gladiator using huffyuv, )mainly cuz it was the first dvd i grabbed outta my collection).
Heres the scrpit i got running for it

loadplugin("C:\Documents and Settings\Administrator\Desktop\MPEG2Dec3 v0.9\MPEG2Dec3.dll")
mpeg2source("C:\GLADIATOR\VIDEO_TS\vts1\gladiator.d2v")
converttoyuy2()

Forced Film was set in dvd2avi, source is quite high quality so no need to filter it and the last line is because im using Avisynth 2.5. Anyways to the point cinverting this to Huffyuv took about and hour and 20 minutes, then running the avi thru CCE 2.5 resulted in a speed of about 1.3. Loading the same script directly in CCE results in speed of about 1.8. When resizing to SVCD resolution which is 480,480 the ammount of time required to make the huffyuv av1 obviously took less although hom much less i couldnt tell ya I wasnt paying that close attention. At this resolution running the avi and and script thru resulted in speeds of about 2.2 for both of em. Then finally at half D1 resolution, which is 352x480, took about 45 minutes to create huffyuv avi and the CCE speed was about 3.6 whereas the script the speed was about 2.5
So in conculsion, if your running a single cpu and encoding at 720x480 dont bother encoding to Huffyuv first, the truth is it will take much longer to finish, this is also true of svcd resolution. In order to see any gain on a single cpu system you need to encoding to half d1 resolution or be using an ungodly number of filters.

edit: apparantly i dont know how to spell

[LB]

Well you don't want to resize before you goto CCE and after the huffy. You want to resize before you encode to huffy. That is how you can get a good indication of the performance gain. But unfortunatly, like I stated in my previous post, it appears that only the modified huffy that is no longer available gives you the massive speed gains on the order of 100% over the existing available to download huffy versions. So now, the only benefit where these *downloadble* huffy versions will help you is if you are doing massive filtering. Win some, lose some.

LB

[Zainal]

As far as the resizing was done i used bilinear in avisynth and converted to Huffyuv, from there i ran it thru CCE. I can imagine with Dual cpus converting to Huffyuv first is a big time saver, but at this point all i can do is imagine.
And for any mod who happens to read this thread sorry for the double post earlier, I dont know how i managed that.

[auenf]

Originally posted by Zainal
And for any mod who happens to read this thread sorry for the double post earlier, I dont know how i managed that.

i saw that the other day, and forgot about it ;)

i think you can delete a single post yourself anyway.

Enf...

[arman68]

Maybe you can find the modified huffy codec by doing a search in Google for "Huffy YUV w/ CCESP patch"

[LB]

Nope. The modified huffy is v1.31 and a normal v1.31 already says "w/ccesp patch". So its basically impossible to tell a v1.31 from a modified 1.31 except when it comes to the speed. ;( And I don't think the modified version is available anywhere on the web since a friend of a friend made it and I was told not to give it out for a good year. ;p Year has passed, I tried to give it out, oh well... ;p At least I get good speeds hehe.

[aftertaste]

well why doesn't this friend of yours want us to have it. humm seems weird to me. Does he feel it is unstable or something?

[catback]

I was an avid user of the Huffyuv codec along with AVI_IO for video captures. That is, until I ran into a problem where any static or other disruption in the video signal, even if it was minor, caused the capture to crash and render the captured file useless.

Not sure if AVI_IO or Huff was the culprit but I never found a solution. I emailed the author of AVI_IO but pretty much got a "sorry, nothing I can do" reply.

I've since started using PicVideo's MJPEG codec at a quality of 19 or 20 (disk space is not a concern for me) and I've had no crashes at all even over major signal disruptions. I do a lot of video captures from home VHS tapes and you often run into these signal disruptions.

[The_Flash]

I'm a big fan of MJpeg myself. Space is a concern for me and even on 20 MJpeg is a lot smaller than Huffyuv.

[digitalman]

Sorry for opening an old topic. I am waiting for my new hard drive to arrive so I have not tried this yet.
If I open the VOB's with VirtualDubMod and then directly save the file as a HuffYUV AVI file, will I get noticeably faster results than using DVD2AVI with MPEG2DEC3.DLL in AviSynth?
2.4Ghz(533Mhz bus) with 512MB mem.

[DDogg]

No :)

[LB]

Is that even possible? Vobs in vdub? (sure would be nice to build dvd2avi into vdub).