HC encoder

[hank315]

Please post all HC issues here now.

HC can be found in the download section but also here:

HC016 latest release at: http://hcencoder.bitburners.com/

[hank315]

Some issues from the old thread:
Is it possible now to use one pass for DVD encoding (so HC will stay under a certain maximum bitrate, specified before encoding)?I'm with freelock - I'd love to see some sort of combination of constant Q and max bitrate where the encoder encodes at a certain quant unless the bitrate is exceeded and a lower quant is used for that section.This will be in the next release.

Bug:
1.) KVCD-NOTCH matrix didn't work.Yep, I know, will be fixed.

Might be nice to check for any hardware requirements up-front...It does.
To run HC needs MMX/ISSE, if SSE2 is supported it will use it, if SSE3 is supported it will use it.
Only changed the minimum check from MMX/SSE to MMX only otherwise some older AMD's wouldn't run it.

004722FF FF 15 F0 31 47 00 call dword ptr ds:[4731F0h]
00472305 8B 48 0C mov ecx,dword ptr [eax+0Ch] ; EAX == NULLDon't have a clue :) call is to a user32.dll routine, before that eax is pushed on the stack and probably comes out wrong, really don't know.

[dragongodz]

some more Q and A's moved here so they are not asked again. :)

Didn't save settings when quit. Everything is load to default.
yes hank315 knows this. we have discussed it so several times. i think,rightly, he was more interested in improving speed and rate control however for this release.

How fast is HCenc013 compared to CCE or to HCEnc012?
slower than CCE and faster than HC 0.12. speed increase seems to scale with the profile, that is BEST has the largest speed increase while FAST has the smallest. also hank315's P4 gained more increase than my Athlon XP 2400+ proportinatly. try it for yourself and see.

And, is that countdown really necessary? If I'm calling HCbatch from Rebuilder, and not looking at your GUI screen, it seems to be just a waste of time.
hank315 already added the *wait command in 0.12 so exiting could be faster if used.


and finally
i just noticed another little bonus. me mentioned in the thanks.txt. nice. thanks hank315. :D

[ljpp]

"Improved speed" is a huge understatement. On a P4 HT 2.66GHz the encoding speed has roughly doubled since the 0.12, and HC is now 4-5 times faster than QuEnc (profile BEST vs. QuEnc High Quality+Trellis).

I never expected such a speed bump! Awesome work.

[dragongodz]

profile BEST vs. QuEnc High Quality+Trellis
sorry but you are comparing things that 1 encoder does that the other doesnt, that is trellis. HC doesnt do it so if you want a more realistic comparison try QuEnc + HQ versus HC BEST. or you could try HC NORMAL or FAST to QuEnc with HQ and TRELLIS etc turned off.

also as i said P4's get a huge speed increase while Athlons increase ,while still big, is not as large.

[Yuri]

Beautiful Encoder!

Thank you for the great encoder

[onesoul]

@ Yuri

I've seen that same frame on a comparison done by you, it must be haunting you eheh ;).

Would be very useful if you provided all the settings used for each encoder such as max bitrate, GOP structure, number of passes, bias, quantizer precision, DC precision. Quantizer precision and bias only applies to CCE of course.

I did much more testing on interlaced source before, now I started on progressive tests such as an anime monsters_inc and unlike before, CCE is giving better results no matter what (I tried only against HC and amongst many different settings, average bitrate was between 3000 and 3500 and it envolved fading scenes).

I have of course no expertise on video encoding but what I guess is that HC handles fields in a better way then CCE when encoding as interlaced, but I'd like to hear someone else's opinion on that.

I'd like to hear your opinion on progressive natural film also, I will try that as soon as I can too.

[lamster]

Originally posted by hank315
004722FF FF 15 F0 31 47 00 call dword ptr ds:[4731F0h]
00472305 8B 48 0C mov ecx,dword ptr [eax+0Ch] ; EAX == NULLDon't have a clue :) call is to a user32.dll routine, before that eax is pushed on the stack and probably comes out wrong, really don't know.
More specifically, the call is to GetWindowLong(). It's pushing a value of 8 (representing the third integer from the extra window memory). A return value of 0 can indicate an error, but the routine that started at 004722f0 (in 0.13; 0046d510 in 0.12) doesn't check that the returned value is non-0; it just attempts to use it as a pointer.

[lamster]

Someone else reported a crash (in the Rebuilder thread).

Here's the call stack to that routine, in case you can use it to figure out what's going wrong.ChildEBP RetAddr Args to Child
WARNING: Stack unwind information not available. Following frames may be wrong.
0012fef4 0046d140 00598d64 00598d38 0012ff34 HCbatch_013+0x72305
0012ff04 004174aa 00598d38 00000000 08b81c40 HCbatch_013+0x6d140
0012ff34 0045a6b0 00400000 00000000 00132a57 HCbatch_013+0x174aa
0012ffc0 7c59893d 00000000 00138fb0 7ffdf000 HCbatch_013+0x5a6b0
0012fff0 00000000 0045a5e2 00000000 000000c8 KERNEL32!BaseProcessStart+0x3d

Note that 00598d64 is the hWnd being passed to GetWindowLong().

[hank315]

More specifically, the call is to GetWindowLong(). It's pushing a value of 8 (representing the third integer from the extra window memory). A return value of 0 can indicate an error, but the routine that started at 004722f0 (in 0.13; 0046d510 in 0.12) doesn't check that the returned value is non-0; it just attempts to use it as a pointer.Also looked at the disassembled code, see what you mean. Will try to see where this actually is in my code but because all the Windows calls are interfaced (Fortran --> Windows calls) it may be difficult to find it, this assembler code is probably from one of the interface routines.
Will check my own calls which ends the program.

BTW. if this happens the m2v file is already written and closed so it should be OK, seems it's easier to encode a movie then to end a Windows program :)

[lamster]

I tried putting a break-point on it, but it's called constantly - probably in a message loop.

I reproduced the crash a few times, but also had it end successfully a couple of times (all running against the same .ini file), which suggests it might be a timing problem. Maybe the order in which child windows are closed, or something like that?

Originally posted by hank315
BTW. if this happens the m2v file is already written and closed so it should be OK, seems it's easier to encode a movie then to end a Windows program :)
Well, I'm a developer, so I have the Windows exception handler set to pop into the debugger. If I change it to invoke Dr. Watson, and configure that to not prompt, I could just let Rebuilder chug away and the occasional exceptions wouldn't cause it to wait for a response to me.

[ljpp]

I encoded Disney's classic Robin Hood using HC 0.13 and DVD-RB. This old movie has a lot of noise/grain in the original. The average bitrate for the movie after encoding was around 5500.

The resulting video differed visibly from the original. While it wasn't too bad looking, the noise and grain of the original was clearly smoothed out. There was some artefacting and macroblocks, a bit similar to the cartoon example above.

How could I tune HC to preserve the noise, which creates the illusion of a sharper image, better? I used profile "BEST" and "HC" matrix - should I try another matrix?

[springl]

@hank315
If I use JAWOR1CD matrix, logfile reports "matrix: BACH1"
If I use BACH1 matrix, logfile reports "matrix: JAWOR1CD"
Here HC_013
Greetings,
Springl

[hank315]

I tried putting a break-point on it, but it's called constantly - probably in a message loop.Think I already found it, will PM you about it.


If I use JAWOR1CD matrix, logfile reports "matrix: BACH1"
If I use BACH1 matrix, logfile reports "matrix: JAWOR1CD"will correct that, thanks for reporting it.

[dragongodz]

The average bitrate for the movie after encoding was around 5500.
I used profile "BEST" and "HC" matrix - should I try another matrix?
YES. for a higher bitrate such as that try just useing the default MPEG matrix.

[Yuri]

@ onesoul
I've seen that same frame on a comparison done by you, it must be haunting you eheh .
---------------------
A year ago or so, I found out that the tricky frames (http://sch-yuri.by.ru/zips/tarzan_49_tricky_frames.m2v) were very hard to be encoded properly by CCE even at high bitrate. Now I use them as testing material to check an encoder's engine, this time the wonderous HC.

[fozzieb]

Hi, I'm using avi2dvd 0.2.5 to convert my avi's to dvd. I use the HC encoder but it only allows me to choose built in matrices and i would like to use the Bdvd one, Is there a wat to do this?


Cheers

[freelock7]

I like the picture quality of this new release (smooth without grain).
Faster (twice compared to old versions).
Prediction seems to be good now.
Very good bitrate control & quant_.

[hank315]

@fozzieb
With the batch version you can use the *CUSTOMMATRIX command to use any matrix you want, see the pdf document.

@freelock
Thanks :)

[ljpp]

Thanks dragongodz,

Indeed, the 'MPEG' matrix improved the result. It is now pretty damn close to the original (which isn't artefact free either).

[hoenes]

Hi,
I've used HCenc yesterday for the second time, first time with ver. 0.13 beta. Unfortunately my PC hang before the second pass was finished, and I now tried to use the *RESTART option but with no success. HCbatch reads the *Options frome the .ini file except the *RESTART parameter, .dbs file is allready there, also renamed/copied it to HC.dbs

HCbatch starts the encoding immediately (1st pass, with the parameters from the HC.ini)
HCGUI doesn't follow the HC.ini parameters.

So *RESTART didn't work and an other strange thing is, that HCbatch converts the bitrate parameter very confuse, so .ini file *MAXBITRATE 9000000 become 910065 and *BITRATE 5000000 become 705032 in HCbatch.

Best Regards

hoenes

[petermarina]

What setting should I use for HC 0.13, to get the best image quality. I am using it in conjunction with DVD-Rebuilder 0.85pro. So far I am using the best setting and I still cant get a result that beats DVD-Shrink.... What am I doing wrong?

[DK]

"Improved speed" is a huge understatement

true

while Athlons increase ,while still big, is not as large.

they are still very impressive

i have tested hc 012 against 013 with a 120 min title:

hc012 - "normal" => 228 min
hc013 - "normal" => 195 min

hc012 -"best" => 348 min
hc013 -"best" => 230 min


athlon64 - 3000 - 1gb ram

++++++++++++++++++++++++++++++++++++

cce 270 => 131 min

[dragongodz]

they are still very impressive
huh? who said it wasnt ? the fact is the speed up is smaller on Athlons than on P4s. so people should be aware of that so person A doesnt start asking why they dont get the same speed increase as person B.

just as they should be aware that the higher the profile the larger the speed increase aswell, so Best gets a higher speed increase comparitivly compared to FAST profiles speed increase.

does imparting such information scare people or something ????

I still cant get a result that beats DVD-Shrink.... What am I doing wrong?
first you have to understand that if the orginal bitrate is fairly high and/or the reduction is smallish then a lot of frames with dvdshrink may be the originals and ones changed will only be done so a little. how anyone can expect an encoder to look better than using a lot of the original frames and some little modified ones is beyond me. try a comparison where a large amount of reduction on an already low bitrate source(such as a long movie or one with lots of extras) and you should see the difference.

[scraper]

Great program, this!

Just one question: There is no way to specify if a frame should become an I-frame (like "chapter=..." in ecl-files)?


I'm sorry... I should have read before posting... I know now it may come some day, but has no priority...
Keep up this excellent work! :D


greetz JP

[DK]

huh? who said it wasnt ?

nobody wasn't - neither you nor me

i just wanted to post some results because so far i didn't see anything alike

[hoenes]

OK I got the *RESTART now, I must be blind!

[johnnyquid]

@hank315

I have been trying out the new batch13 version. It is working fine most of the time but it does occasionally goes into an infinte loop when trying to fix VBV buffer underflows during pass 2. The finding and fixing counters just keep on going up with no progress in the number of frames processed. Unfortunately the problem is not repeatable. Rerunning the program using the same input files does not duplicate the problem. Find at the bottom of this post an example of a log file from a job which did not work one time but then did another. What is very strange is that when the encoding does work no underflows are found or corrected but when it does not work the infinite loop of finding and fixing underflows occurs with the exact same input files. Being a programer myself, I realize that this post is not very useful for trying to locate the problem (if it is indeed a problem with HC and not some problem with my hardware/software enviroment) but am posting again since it has happened more than once on different encodings. (If the problem is not due to my setup, memory overwriting or lack of variable initialization would be my guess and those problems can be pains to debug.) I am running Windows XP SP2 on an Anthlon XP 2500 machine.

@all

Has anybody else seen this problem?


Logfile is below:


-------------------------------------------
| HCbatch - MPEG2 encoder - rel. 0.13 beta |
-------------------------------------------

input: g:\rbtemp\d2vavs\v01000000001001.avs
output: g:\rbtemp\d2vavs\v01000000001001.m2v

--------------------
| encoder settings |
--------------------

profile: GOOD
frames: 1 8989
framerate: 23.98
aspect ratio: 16:9
bitrate Kb/s: 3492
max. bitrate Kb/s: 9000
closed gops: no
VBV check: yes
scene change det.: yes
interlaced: no
goplen,B-pic: AUTO
dc_precision: 9
scan method: ZIGZAG
time code: 0 0 0 0
CPU: MMX/SSE
matrix: MPEG

--------------------
| source stats |
--------------------

nr. of frames in source: 8989
width*height: 720*480
fps: 23.98
nr. of frames to encode: 8989
frames to encode: 1 - 8989

---------------------
| encoding - pass 1 |
---------------------

pass 1 encoding time: 0:06:22 (382 s)
average fps: 23.5

--------------------------------
| encoding - intermediate pass |
--------------------------------

bitrate set to: 3492000 b/s
est. outfile length: 159816 kB
intermediate encoding time: 0.0 s

---------------------
| encoding - pass 2 |
---------------------

pass 2 encoding time: 0:04:04 (244 s)
average fps: 36.9

------------------
| encoding stats |
------------------

total encoding time: 0:10:26 (626 s)

intra matrix used
8 16 19 22 26 27 29 34
16 16 22 24 27 29 34 37
19 22 26 27 29 34 34 38
22 22 26 27 29 34 37 40
22 26 27 29 32 35 40 48
26 27 29 32 35 40 48 58
26 27 29 34 38 46 56 69
27 29 35 38 46 56 69 83

non-intra matrix used
16 17 18 19 20 21 22 23
17 18 19 20 21 22 23 24
18 19 20 21 22 23 24 25
19 20 21 22 23 24 26 27
20 21 22 23 25 26 27 28
21 22 23 24 26 27 28 30
22 23 24 26 27 28 30 31
23 24 25 27 28 30 31 33

nr. of gops: 661
nr. of frames: 8989
nr. of I-frames: 661
nr. of P-frames: 2804
nr. of B-frames: 5524
average quant (non linear): 6.713
VBV underflows detected: 0
VBV underflows fixed: 0
minimum bitrate: 146
maximum bitrate: 7866
average bitrate: 3492

[digitalman]

I get an error when selecting the KVCD-NOTCH matrix, that it is an invalid matrix. Otherwise this looks to be a promising alternative to QuEnc.

[DoctorRad]

Originally posted by ljpp
"Improved speed" is a huge understatement. On a P4 HT 2.66GHz the encoding speed has roughly doubled since the 0.12, and HC is now 4-5 times faster than QuEnc (profile BEST vs. QuEnc High Quality+Trellis).

I never expected such a speed bump! Awesome work.

We try our best... at least I assume I'm 'the dr.' mentioned in the help file... and there should be more to come. Hank is doing a superb job of coding on this one, I'm merely feeding him fast motion estimation ideas, the ones I never had any time to code up.

Cheers,

Dr. Matt...

[dragongodz]

hank315 asked me not to say the things we have and still do talk about, lets just say they are wide and varied including things like ME and RC etc etc etc. ;)

[lunario]

great encoder hank but everytime that i use hcenc it gives me a memory error at the end of the encode (just like in cce 2.70 ) i use avi2dvd with hcenc

[hank315]

@lunario
This should be solved in the next version which will be out very soon.

[hank315]

New release: HC014

- improvement of the encoding engine
- added *CQ_MAXBITRATE command, CQ encodes will now be DVD-compliant
- some other bugs fixed

HC014 will be in the download section soon but also here:

http://hank315.dyndns.org/HC_014.zip

[lunario]

great :) i will test it later

thanks again for this great encoder

[Frameserver]

Originally posted by hoenes
Hi,
I've used HCenc yesterday for the second time, first time with ver. 0.13 beta. Unfortunately my PC hang before the second pass was finished, and I now tried to use the *RESTART option but with no success. HCbatch reads the *Options frome the .ini file except the *RESTART parameter, .dbs file is allready there, also renamed/copied it to HC.dbs

HCbatch starts the encoding immediately (1st pass, with the parameters from the HC.ini)
HCGUI doesn't follow the HC.ini parameters.

So *RESTART didn't work and an other strange thing is, that HCbatch converts the bitrate parameter very confuse, so .ini file *MAXBITRATE 9000000 become 910065 and *BITRATE 5000000 become 705032 in HCbatch.

Best Regards

hoenes

hi hoenes,
hank315 changed the values for the bitrate (in hc_012 I guess) to Kbits and this caused the strange behaviour. So if You use 5000 instead of 5000000 anything should be ok.

@ hank315
impressive work!
The undersizing seemes to be fixed now.
Keep on coding!

Btw I remember Your "Fortran Compiler trouble" may this is interesting for You.

April 20, 2005
GCC 4.0.0 has been released.
GCC, the GNU Compiler Collection, includes front ends for C, C++, Objective-C, Fortran, Java, and Ada, as well as libraries for these languages (libstdc++, libgcj,...).

It is now equiped with a Fortran 95 compiler front end.

http://gcc.gnu.org/fortran/

http://gcc.gnu.org/

frameserver

[FredThompson]

Please allow larger GOPs.

There's work on improving TyTool (TiVo-centric MPEG2 editor) by changing the encoder from mpeg2enc to something else. QuEnc is a possibility. HC should also be a candidate, probably more so, givent he amount of activity in devloping it.

Some TiVo source has very long GOPs. Without getting into too much detail, there are technical reasons to keep long GOPs.

[lunario]

hank i´ve tested the new version and it gives same error at the end ..
same memory error the only encoders that give the error so far are hcenc end cce 2.70 (´ve tested with cce 2.67 and procoder and they don´t give the error.

the only bad thing on this is that the pc wont shutdown because of that error, when i click ok the rest of the job (muxing and authoring ) get´ts done

[gozo200x]

http://img.photobucket.com/albums/v694/gozo200x/hc.gif

I've been trying to run HC since version 0.12 without success. Also with the new 0.14 I get this error whenever I try to start encoding - be it with the standalone version or via DVD Rebuilder (0.86 pro).

Unfortunately I'm not a programmer and don't know what information could be useful for you.

My System:
AMD 64 3200+
1 GB RAM
WinXP Home SP2

Please tell me what other information I could provide, I'd love to give HC a try.

Thanks,
gozo

[lunario]

that is the same memory error that i have but in here only in the end of the encode

[hank315]

@lunario
Could you please post the exact error message.

[lamster]

Originally posted by gozo200x
http://img.photobucket.com/albums/v694/gozo200x/hc.gif

I've been trying to run HC since version 0.12 without success. Also with the new 0.14 I get this error whenever I try to start encoding - be it with the standalone version or via DVD Rebuilder (0.86 pro).

Unfortunately I'm not a programmer and don't know what information could be useful for you.
Curious - the error message says to press Cancel to debug; I only see that if there's a debugger installed (e.g., if you have Visual Studio).

One possible approach would be to select Cancel, see what loads, and post the assembler code around the point of the exception, along with the call stack.

Another approach would be to restore the default exception handler (DRWTSN32 -I) and provide the resulting Dr. Watson log.

[Malow]

dear hank315, i guess that the same problems of lunario is happening with me... this happens on all winxp sp1 machines where i work, but "apparently" no problems on winxp sp2 machines...

the screenshot:

http://www.alphapro.com.br/hcerror.jpg

[lunario]

Originally posted by Malow
dear hank315, i guess that the same problems of lunario is happening with me... this happens on all winxp sp1 machines where i work, but "apparently" no problems on winxp sp2 machines...

the screenshot:

http://www.alphapro.com.br/hcerror.jpg


yes that is the error but i have XP with SP2 (and last updates) so that is not only with machines with sp1

my rig

Pentium 4 2.6HT @3ghz
1GB Ram
XP media center 2005 with SP2

when i use the new cce gives me exactly the same error

[gozo200x]

Originally posted by lamster
[B]Curious - the error message says to press Cancel to debug; I only see that if there's a debugger installed (e.g., if you have Visual Studio).

I had Visual Basic installed a long time ago to recompile something I did even a far longer time ago. After uninstall the option to debug somehow remained on my system, but If I press OK to debug there's an error message that nothing to debug could be found. I really am NO programmer ;) What I did back then was kiddie work.

[Malow]

Originally posted by lunario
[B]yes that is the error but i have XP with SP2 (and last updates) so that is not only with machines with sp1

the sp2 in my home gives the same problems... but, in the machines that work, it have also these updates:

http://ryanvm.msfn.org/updatepack.html

[lamster]

Originally posted by gozo200x
If I press OK to debug there's an error message that nothing to debug could be found.
Try entering (from a command prompt window):
DrWtsn32 -I

This will install Dr. Watson as the default debugger.

[lunario]

Originally posted by lamster
Try entering (from a command prompt window):
DrWtsn32 -I

This will install Dr. Watson as the default debugger.

no debug option for me only the same error that mallow has

[lamster]

After executing "drwtsn32 -I" to install it, you can run it without the -I option to see / update the settings. In particular, you can see what the "Log File Path" is set to, and look for DrW*.log file there (or anywhere on your system if not set - the default location varies depending on which version of Windows you're running).

[gozo200x]

I found what causes the problem.
I got an AMD64, remember? and it has this NX extension. Code execution prevention or whatever it is called in English. Now if I add HC to the list of programs for which NOT to use this prevention thingy it doesn't crash anymore.
HC is in good company with WaveLab on this by the way ;)

now it only crashes upon exit as described above. but drwatason doesn't spring into action, no log or user.dmp is written.