Hacked Huffy.dll with speeds of 3.2 in CCE

[LB]

^.o

What's this you say? Well aren't we in for a lill treat. Got this from a good buddy about a year ago. Don't ask, I'm not telling ;p. It's an older version of the huffy codec but it's also been modified (hax0red). Oooo baby. And the result? Going from a huffy avi file to cce, you will most likely see a 60% increase in whatever current speed you are running at. Yes, 60% I ain't lying. I went from 2.0 to 3.226. I've hit 3.8 on a good day too.

IMPORTANT: To set it up, in virtualdub, under codec settings, for the huffy codec there are THREE BOXES to check. Just check the TOP TWO, and leave the third/bottom one un-checked. Comprenda?

Great, now go play with it and report back here your results. Consider it an early XMAS gift from me to you. I'll send you the bill later. ;p

LB

Oh yea, and poopedypoop, this is how I got those whack speeds ;p Hope you find yourself over the 3.x mark too. ;)

Edit:I'm really sorry but since the source is not available to this mod. of Huff that's originally released under GPL the attachment *have* to be deleted. I'm the one approving it in the first place and I'm really sorry to have to do this.
/Swede

[Matthew]

I believe attachments have to be validated before they appear, for obvious reasons.doom9 couldn't have people attaching 'fixes' to CCE, for example, could he.

[Arky]

This is all well and good, but it doesn't help anyone who wishes to transcode a DV file to MPEG2, or anyone who wants to re-encode a ripped MPEG2 VOB to smaller MPEG2 stream. ...Or does it? Perhaps I am missing something here? (and let's face it, that's pretty damned likely since I've been up all night!) :eek: :D

Impressive claimed speed you have there, though, even without the hacked huffy codec. Wish I could get that kind of speed on my rig :(


Arky ;o)

[LB]

What you talking about f00 ;p J/kin

Sure it does. I assume DV=divx. Whenever you process through CCE, whether it be going to a SVCD compatible stream or a DVD compatible stream, you should always use the VBR option in CCE. Even with CBR set at the max bitrate, VBR has been shown (try if for yourself too) to produce better results at even LOWER bitrates than CBR. Maybe some bug in CCE, I don't know. But that's besides the point.Why use huffy for processing through CCE? Answer: Because it's fast as hell. If on one pass I can get 3RT with huffy versus 1RT with a divx avi with a lot of filters and crud, then if I were to do a 4 pass CCE encode, the huffy source would finish in say 2hr while the divx source would finish in say 6hrs. Non paragraph form:

Huffy AVI -> CCE 4Pass VBR -> 2hr
Divx AVI -> CCE 4Pass VBR -> 6hr

So, if your goal is to output a MPEG2 stream usinv VBR in CCE, then keep reading, if you are either totally confused or have no intention of creating a mpeg2 stream, ya can stop now.

Continuing, one of the misconceptions that a lot of people have is CCE speed. Yes, it is based upon your processor speed, but its ALSO based upon what type of file you are feeding it. So if I were to tell you I get 3RT on dual 1800s and you told me you get 1RT on a p4 2.4ghz, this means NOTHING. It doesn't provide the OTHER key factor. Which is? It also matter (almost more than your processor speed) what type of input file you are using. If you feed a straight divx 3.11 into cce you will get a different speed than sending in a vob file, a different speed than a filtered aut file from aviutil, etc etc. This again can be reasoned from the above comparison.

So what's all this mean to you? Well the BEST stream to use in CCE is an Huffy AVI stream. Why? It will maximize CCE's speed on your computer. No other compression codec that you could send CCE would process faster. And seriously, with this codec you may see a 1000% increase in processing speed over using a different codec, and a 60% increase in speed from using the regular huffy codec. Ok, now that I've convinced you that huffy is the way to go, how to you convert your vob files and divx files to huffy avi's to be processed in cce? Simple. Install the codec. Open vdub. Choose huffy codec as your, well, codec. ;) Also, configure it. There are three boxes to check. Only check the top two, not the bottom one. Finally, do all your settings or whatever resizes/filters you have to, then goto FILE, SAVE AS SEGMENTED AVI, and save it. The segmented avi is the trick since CCE has trouble opening non-segmented huffy avis, don't ask me why. It will also help if you don't have NTFS installed so that you can bypass the 2gb limit. Finally, you add your first segment in and inside the CCE popupbox you can add the rest of the segments. Then, you're done with the huffy part. ;) Please note though, a 20min huffy segment will cost you about 5gb.

Welp, there you go. This explanation should hopefully convince that even if you decide not to use his hacked codec, if you are doing vbr in cce, which you always should anyway, huffy will maximize your speed to ANY mpeg2 output stream. Heck, you could even process a .rm stream to a huffy avi and feed it to cce. ^.^

AH! 1 more thing to those of you that are NEW to using huffy compression. Yes this is re-encoding it before you even encode it in CCE. Doesn't that produce quality loss? NO! Huffy is a LOSSLESS compression codec. If you were to encode a divx file into a huffy avi file there is NO quality loss. (Another reason for why huffy avi's are large). So when it comes time to feed the sucker into CCE, it's identical in quality to the divx file.

LB

[Arky]

Originally posted by LB
What you talking about f00 ;p J/kin

Sure it does. I assume DV=divx.


No, "DV" does not mean "Div-X", it means DV. This is what you capture over firewire from a DV-camcorder.


Originally posted by LB

Whenever you process through CCE, whether it be going to a SVCD compatible stream or a DVD compatible stream, you should always use the VBR option in CCE.


I always DO - what's your point? I never mentioned this at all...

Originally posted by LB

Why use huffy for processing through CCE? Answer: Because it's fast as hell. If on one pass I can get 3RT with huffy versus 1RT with a divx avi with a lot of filters and crud, then if I were to do a 4 pass CCE encode, the huffy source would finish in say 2hr while the divx source would finish in say 6hrs. Non paragraph form:

Huffy AVI -> CCE 4Pass VBR -> 2hr
Divx AVI -> CCE 4Pass VBR -> 6hr


Erm...my point was that in order to encode from a Huffy stream, you have to transcode *to* a huffy stream from whatever your original was (and this would appear, in your case at least, to be Div-X, though God knows why you'd even bother transcoding a Div-x to anything else, as they are FULL of artifacts to begin with). This pre-transcoding to the Huffy codec rather seems to undermine the point of trying to save time when feeding it to CCE. The only exception I can think of is for those few people who capture directly to huffy from their TV cards etc.

Originally posted by LB

So, if your goal is to output a MPEG2 stream usinv VBR in CCE, then keep reading, if you are either totally confused or have no intention of creating a mpeg2 stream, ya can stop now.

Continuing, one of the misconceptions that a lot of people have is CCE speed. Yes, it is based upon your processor speed, but its ALSO
based upon what type of file you are feeding it.


FAR from being a misconception, most Doom9 members are only TOO aware of the difference the decompression of source codec takes. This is one reason why many people shy away from the VFAPI codec, because it insists on firstly converting to RGB, which absorbs time and processing power.


Originally posted by LB

So if I were to tell you I get 3RT on dual 1800s and you told me you get 1RT on a p4 2.4ghz, this means NOTHING. It doesn't provide the OTHER key factor. Which is? It also matter (almost more than your processor speed) what type of input file you are using. If you feed a straight divx 3.11 into cce you will get a different speed than sending in a vob file, a different speed than a filtered aut file from aviutil, etc etc. This again can be reasoned from the above comparison.


Yes, I know - and by the way, you missed 2 other factors:

1) input frame size (will you be interpolating upwards (YEUCH!! :( )

2) will you be OUTputting a smaller size than D-1 (i.e. SVCD is smaller)


Originally posted by LB

So what's all this mean to you? Well the BEST stream to use in CCE is an Huffy AVI stream. Why? It will maximize CCE's speed on your computer. No other compression codec that you could send CCE would process faster. And seriously, with this codec you may see a 1000% increase in processing speed over using a different codec, and a 60% increase in speed from using the regular huffy codec. Ok, now that I've convinced you that huffy is the way to go, how to you convert your vob files and divx files to huffy avi's to be processed in cce? Simple. Install the codec. Open vdub. Choose huffy codec as your, well, codec. ;) Also, configure it. There are three boxes to check. Only check the top two, not the bottom one. Finally, do all your settings or whatever resizes/filters you have to, then goto FILE, SAVE AS SEGMENTED AVI, and save it.


PRECISELY what I was getting at - you're going through an entire operation and pretending it doesn't take any time! By the time you've faffed about using VirtualDub and Huffy, I suspect you might just as well forget the whole idea and miss out this step. That way, you won;t have to find gigs and gigs of extra space for the Huffy-encoded files either. I appreciate that the low compression rate of Huffy means that there is negligible quality loss, so I won't take issue with you on that point. I just feel that your original post was sensationalising something which is a little misleading (not completely misleading, don't get me wrong). You are quick to point out that you are encoding to SVCD now, but you made no mention of this in your first post.


Arky ;o)

[LB]

Well I don't know if you thought I was trying to create a heated argument, but I wasn't. I was just trying to be informative. I'll keep it that way too. Let me address a few of your comments.

1. Why I discussed VBR

- The reason was that in CCE, if using VBR (which you said most Doom9 users use), it is incredibly important (as we all know) to obtain the best speed possible. I am not about to elaborate again as to why Huffy improves upon VBR in a more noticable way than CBR, but the reason I initially started the discussion with it has a simple answer: Why would people that don't use Huffy at the moment want to if they don't understand its benefits? Also, without understanding this, implementing Huffy for a CBR encode is almost pointless.

2. My Example of Divx v. Huffy

-That's exactly what it was, an example. No where did I mention all-inclusive. Say you made a bunch of home movies in divx and now want to convert them to a mpeg2 stream. Well, here's your answer if you have quite a number of them and would like to finish in under 10 years. But, I could have just as well suggested converting from a Vob to a Huffy Avi and then proceeding to cce. And yes, a Huffy avi is much faster than feeding a filtered/resized Vob, or any other source for that matter, into cce. If you were to implement Huffy with a Vob, you would probably notice the same scaling effects.

VOB --> Resizing etc --> CCE --> 1RT
VOB --> Huffy Avi --> CCE --> 3.2RT

3. VFAPI

- I did not even discuss vfapi, so I don't know why you brought it up. A Huffy AVI is in Yu2 format (hope I got the exact spelling right), not RGB.

4. Missing 2 Factors

a. I concede here. That is an important factor that people should enclude when they provide CCE speeds. A 720x480 encode versus a 480x480 encode is mighty different speed wise. So I guess when people provide speeds we need:

CCE SPEED xRT
CPU(s)
MEMORY
BUS MHZ
INPUT FORMAT (HUFFY AVI, AVS + FILTERS USED, ETC)
INPUT RESOLUTION

(Did I miss anything?)


b. Smaller file. I do not see how this is a metionable factor. In all cases (sending a VOB or a Huffy AVI) the result will likely be smaller. There could be the case of not using a Huffy avi and frame serving from Vdub with a divx source as the input, but again if you understood my explanation, that would waste time. But still, the fact that the file is smaller or lager should not be mentioned. I fail to see how creating a large or small file has any effect on the speed equation provided your HD can write as fast as CCE can output.

5. Last Comment

-A few things to address here. With 120gb hard drives becoming the norm, the average person can easily spare 30gb for a two hour movie for working space. That is almost assumed a pre-requisite to video encoding in general. Furthermore, the typical sub 80gb-HD individual will probably not be doing much video rendering to begin with. I would hardly call that unreasonable today. Second, I left out the huffy conversion part because its negligable. With an avisynth file I can convert to a Huffy avi WHILE real time IVTCing (with IT) at 34fps. Yes, it is worth mentioning I suppose, but if you are still doing a 3+ pass on CCE, this will save you an immense amount of time.

3pass = 4 actual runs. 1 calculation & 3 passes

2hour movie on my PC (but obviously directly scalable to other PCs)

---------WITH HUFFY------------
Huffy Conversion = 1hr 10min
CCE Calculation = 40min
CCE Three Runs = 2hr
TOTAL = 3hr 50min

----------WITHOUT HUFFY-----------
No Huffy Conversion = 0hr
CCE Calculation = 2hr
CCE Three Runs = 6hr
TOTAL = 8hr

So you be the judge, but if I'm any good at math, my money is on huffy.

Again, this is just an informative post, not argumentative. Any questions, holler back at me.

Edit edit edit, I have to stop and walk away, no more edits for today. Hah. Well I thought I should say one more thing to those that glance at this and say "pweh, I'll only save ~4hr". This is only applying an IVTC to your source. A LOT of doom9 users out there also enjoy filtering the hell out of their source, and it's incredible how much Huffy can help out in that situation. Filtering takes a long time to process, so in the case of using a multitude of filters in Vdub or in (my favorite) AviUtl, a serious amount of time can be saved. The conversion to a Huffy avi happens only once, and during that time you can use all your filters. So when the file gets to CCE, you no longer are getting .2RT due to your filter-implemantation, rather you are getting your normal-huffy speed of say 3RT. Huffy can help with everything when dealing with CCE, but it shows its TRUE colors when FTHOS (Filtering The Hell Outta Something). ;) As an example I purchased a DVD from overseas containing ~100+ episodes whose source was from a Laserdisc. Very poor source, conversion to the DVD, and compression. So essentially every episode had to be filtered. I implimented a noise reduce first then applied a sharpen. With a few days of tweaking, a dramatic improvement over the origional was obtained. The unfortunate thing is that this created a dramatic increase in processing time. But, it was about then that I found out about this Huffy codec and was able to reduce each episodes processing time from 5hours on a 4pass huffy encode to a little under 1.5hours (including the huffy conversion). Multiply that by 100 and you can easily see the benefit. OT (couple that with the autoit script I'm discussing in another forum here and you may be grinning too at the benefit of both.)

LB

[Arky]

lol - relax, I wasn't (and I'm still not, either) having a go at you. I was just responding to your points in, perhaps, too terse a manner. No arguement or offence intended. This is one of those cases where a person to person conversation would have been much more fruitful than text on a page.

Anyway, just to clear a few points up:



When I referred to "size" I meant frame size. If you are outputting to a small framesize (e.g. SVCD) then this will take less time than if you are encoding an output of full D-1 (720x576). I ddin't mean filesize.


I agree with you that where you are doing *upwards of 3 passes*, then the benefits of doing pre-processing would begin to make reasonable sense. Good point.

I brought VFAPI up purely to illustrate that there are slow and fast methods, which many Doom9 members are aware of - we don't all use the same codecs to frameserve with. It wasn't a crticism of your method - just an observation that decompression time of source (and sometimes frameserving) codec is a widely-known parameter.


On a humourous note, I can HONESTLY tell you that I have 3 80 gig drives, and I currently have too little space to even rip a DVD - and that's even if combined what little space I have on each drive!!! :eek: :(


One final thing - I don't know if you were being purely illustrative, or if you were being factual about your own rig, when you mentioned "dual 1800s". If you are usin an SMP rig, then of course there would benefits to using your method, because you'd be able to utilise both processors for everything after the first operation. CCE can use SMP, but this is bottlenecked, as I'm sure you are well aware, by, for example, AVIsynth, which isn't SMP aware. Reading directly from Huffy codec, however, CCE would be free to let rip on both processors, and this would account for such a dramatic speed increase.

Peace! :D


Arky ;o)

[HomerJ]

Guys,

A very stimulating dialogue.

Like most people, I'm always looking for speed improvements, so at first glance Huffy seems very inviting.

But Arky seems to be questioning some of the processes. Now I respect Arky a lot, and take notice of what he says and advises. (and I don't see him saying "go out and buy today !!!)

I also note that LB talks about Virtualdub. Now I don't use virtualdub at present. So does this mean I need to install it, as well as the Huffy code ?

Current I use the following :

Smartripper
DVD2AVI
AviSynth
Ifoedit
CCE
Spruce Up
Nero

Frame size is 720 X 576 (PAL of course)

Very simple question. Will Huffy be of benefit to me ?

At present I am very confused, but will follow this thread with interest.

HomerJ :confused:

[Arky]

Looking at your software setup, and the fact that you are apparently not doing any pre-processing, I would *personally* not bother with Huffy. Basically, as I conceded above, there are potential benefits if you are doing pre-processing/filtering etc., since LB's method would perhaps save you time with multiple pass VBR encoding because you would not need to run the filters every time you made an encoding pass. This does not appear to be the case with your setup, since you did not specify that you were using any AVIsynth filters. Neither are you resizing your frames. Therefore, IMHO, you would not benefit much unless you are doing upwards of 3 passes. I'm not talking about absolutes here - I'm just looking at it from the point of view of the amount of time saved versus the hassle taken to achieve this! The hassle is only worthwhile, I believe, under the circumstances discussed above. If you decide that you wish to introduce filtering into the equation for future jobs, then the case for using LB's method grows (broadly-speaking!) exponentially stronger the more filters you use. The main exception to this would be if you have a dual-processor machine, in which case, I would be quite keen, when doing multiple encoding passes, to use the Huffy method, regardless of whether or not I was pre-processing or re-sizing.


Arky ;o)

[HomerJ]

Arky,

Thanks for the prompt reply.

I guess you said it all.

I'll think I'll apply of of lifes sayings.

"If it aint broke, don't fix it.

HomerJ :D

[Arky]

Hey, LB,

as I've mentioned elsewhere on the forums, my next Motherboard+Processor upgrade will be next year, when I hope to get a dual-processor AMD Clawhammer compatible MoBo. In this instance, I will probably use your method virtually every time I encode. I am just curious, though, what is it that has been "hacked" in the Huffy codec you have attached?


Arky ;o)

[LB]

No idea what's been hacked. I was using the released version (2.2?) of it and my good friend was like "here try this, my 'other friend' made it". So I did and noticed a 60-80% improvement over the normal huffy codec and no quality loss. So, I don't know what has been hacked/removed/fixed, no idea. All I know is that its the fastest huffy codec out atm.

And yea. I got dual 1800s. Bought em last feb. Nothing to be fantasticly proud of anymore ;( with the new p4 3ghz out. My next rig is gonna be quad clawhammers whenever they are available and I get a lot of $$. Unfortunatly, the latter may take awhile, heh.

But yea, don't bother with this method unless you do a LOT of encoding. If you encode a movie a month then it's not going to be worth it.

LB

BTW arky, you may also want to look at a SCSI raid setup. For $800 more, which seems expensive but if you are going dual, go all out, get a raid U320 card and two cheetahs. The new cheetahs boast 83MB/s MINIMUM transfer speeds. Couple two together and you have the potential for 166MB/s MINIMUM availabe. As an example, I noticed over a 300% reduction in the time it took to DVD2AVI a file going from a 7200 IDE drive to my U160 raid setup. Plus, your overall system speed is much faster. But just remember, don't go for space unless you are a millionare. Save that for the IDE drives. Only get a few SCSI's so that you can use them to boot from and encode to/from. (That's the other benefit). Most SCSI drives can last 8 years no problems and they all have 5 year warrantys. So you should do all your editing on them and leave your other drives for storage. Higher reliability.

[Taxidermista]

A quick test with my rig (see signature) shows poor results:

6000 frames PAL movie segment (4 min.)
No filters in AviSynth
CCE Conf: VBR 3 pass 2000/4000/8000 (min/avg/max)

movie.m2v >> DVD2AVI 1.76 + AviSynth 2.06 + CCE 2.66 >> movie.mpv = 14 min.

Time: 14 min.


movie.m2v >> DVD2AVI 1.76 + AviSynth 2.06 + VDUB 1.4.10 (Huffy codec) >> movie.avi = 4 min. 55 sec.

movie.avi >> CCE 2.66 >> movie.mpv = 12 min. 18 sec.

Time: 4:55 + 12:18 = 17 min. 13 sec.

What's wrong here? Does it work only in dual systems? Only good with CCE 2.50?

------------

mmm, signature doesn't work:

Athlon 1800+
Asus A7V266-E Raid / 512 MB DDR266
2 x Seagate Barracuda ATA IV 60 GB

[sh0dan]

Your attachment does NOT include the source, only project file and an include file.

You are violating the GPL, and I'd advice you to remove the attachment until your friend has given you the complete source.

(FYI you are missing huffyuv.cpp, drvproc.cpp, tables.cpp at least).

This is not to annoy you, but we need to make sure that these things get back to the community.

[LB]

Taxidermista:
-------------
Did you remember to check both boxes during the SETUP of the huffy codec? If you didn't then it WILL be a heck of a lot slower.

sh0dan:
-------
I know that. I asked my friend and he said he hasn't spoken with whoever he got it from in over a year, and that person may not have even been the person that created it. What I gave you guys is what I recieved myself and also what my buddy recieved. So, I'm not personlly trying to violate anything. I'm just passing along what I recieved. Hope that doesn't offend anyone, but that's the fact of the matter.

LB

[Taxidermista]

Originally posted by LB
Taxidermista:
-------------
Did you remember to check both boxes during the SETUP of the huffy codec? If you didn't then it WILL be a heck of a lot slower.
LB

Yes, I remembered to check them. I did another test using CCE 2.50 and final time was aprox the same as 2.66, about 17 min. I think Arky has been very accurate when he pointed to your dual rig. That could explain your results.

[Arky]

SCSI RAID, huh?

Well I agree with your logic, and I like the idea, but *ouch!* - my back pocket is already hurting just THINKING about the cost! :(

I think my limited funds might be better directed towards dual CPUs (I like the idea of quad CPUs too, but I can't (unfortunately) see this becoming an economically-viable reality within the next year or so).

Thanks for the stimulating debate, anyway! :cool:

Regards,


Arky ;o)

[LB]

Well actually, that's probably accurate. Think about a few things...

He only did 4 mins. It's too late at night to figure out the equation but that's pretty darn good timewise what he saved. Think about it. 4Min clip. Most movies are 2hours. If on the 4Min clip he saved 13% in CCE, that % would grow almost exponentially. Basically, you could almost compare the 4min clip to a CBR segment. Also, he didn't use a IVTC filter while most encodes require it. I'd like, if you have the time, to test out this same time challange on a normal 2 hour movie with an IVTC filter enabled. Granted, you will spend time in the conversion to the HUFFY segmented avi, but I believe when coupled with the time saved in CCE you will see the difference.

Remember what I said previously. This method really isn't designed for CBR nor a SINGLE short clip. The whole benefit of the huffy avi is seen through the multiple passes in CCE and if the clip is too short, the results, as in this case, will appear as such. Try it again but this time on a clip that this method is designed for. ;)

LB

[bastioned]

I´ve just recently read this post and I think, at first, that what is said here is really interesting to most of us DVD to DVD burners. I have an idea on how you could post the stuff to us without compromising any ethical code of the forum: zip it all ( I mean ALL the files you where given by that mate of yours ), share it on Kazaa, Edonkey, whatever you please, and then tell us the name of the shared file. It´s been done before, and I think it´s worthwhile for all of us, it will save us a lot of encoding time!!!
Just my 2 c