HCenc 023 released

[Sharc]

@bt:
Not a solution, but perhaps you have seen my workaround in post #99.

[~bT~]

Ooops ... works perfectly when I
- Do the encoder settings in HCgui_023 and save the HCini
- Exit HCgui and fire up HCenc_023.exe separately
Looks as if the HCgui cannot start the encoder properly.
Scratching my head .... perhaps something is borked here ....

cool mate. will try it next time.

[Sharc]

@hank315:
When I convert a blu-ray clip (Film@23.976 fps) to PAL DVD format and press "make DVD compliant", HCgui inserts *pulldown in HC.ini, which results in a jerky playback.
This may be ok for NTSC, however for PAL I believe it should insert "*assumeFPS(25)" in order to do the PAL speedup.
(It works fine when I put assumeFPS(25) into the script.)

[um3k]

The speedup is your job, not the encoder's.

[~bT~]

Ooops ... works perfectly when I
- Do the encoder settings in HCgui_023 and save the HCini
- Exit HCgui and fire up HCenc_023.exe separately
Looks as if the HCgui cannot start the encoder properly.
Scratching my head .... perhaps something is borked here ....

just tried this with 022 & 023. no lolly. :confused:

[Chumbo]

...
btw, I happen to test this on an mpeg2 source and it works just fine. It only crashes when the source is an AVC. I normally use the .dga file as the source in the input avs script, but it crashes if I use either AVCSource or directshowsource as long as the source is AVC. Maybe that'll help narrow down things.

I tested this on my "working" system and sure enough it crashed there too. What I didn't remember is that when it worked, my sources were all mpeg2. Sigh...
@hank,
I guess the source being mpeg2 isn't really the issue here. I just ran into this yesterday and have narrowed it down to the script having the LanczosResize or Lanczos4Resize command. Without this it works fine with the lossless option, but once I add it, bang! HCenc crashes. I tried putting the resize before and after the convert... function with the same results.

Here's my script, it's very basic.LoadPlugin("c:\PROGRA~1\AutoGK\DGMPGDec\DGDecode.dll")

mpeg2source("H:\Media\movies\DVDs\SD\movie\movie.d2v")
LanczosResize(720,480)
converttoyv12()

[davexnet]

General HCenc questions thread?

I was wondering - are any of the built in matrix's suitable for animation ?

I tried the "custom matrix" and typed the numbers from Tmpgenc 2.5 animation/CG matrix, but the results were so-so.

TIA for any info.

[halsboss]

Just on matrices, I tend to default to FOX1 as I gather it retains the most detail... is that right ? I admit to not understanding matrices.

If it goes over 1 DVD, I then use Nero Recode to get it back down. Is that generally an OK thing to do ?

[Chumbo]

...If it goes over 1 DVD, I then use Nero Recode to get it back down. Is that generally an OK thing to do ?
Why don't you just use the target size and let HC estimate the bitrate for you? When you recode what you've already encoded, you'll lose more quality.

Depending on the audio track size, you can estimate the size with a bitrate calculator. I just multiply 3.8 x 1024 x 1024 and enter the rounded whole number into the file size. I always get exactly what I need. ;) Plus, you don't want to use every bit on the DVD as they tend to wear down on the outer edge first.

[AlanHK]

If it goes over 1 DVD, I then use Nero Recode to get it back down. Is that generally an OK thing to do ?
If you use two-pass encoding, you can set the exact file size in HC.
(In HCgui, under "bitrate", switch from "average bitrate" to "file size". Then you can switch back to see the corresponding bitrate.)

If I'm encoding several files for one disc, I make an omnibus AVS file that joins them all together, use HCgui to calculate the bitrate to make that 4000000 kB, then use that rate for each individual encode

eg:

Import("09.avs")
c=last
Import("10.avs")
c=c++last
Import("11.avs")
c=c++last
Import("12.avs")
c=c++last
Import("13.avs")
c=c++last
c

[~bT~]

If you use two-pass encoding, you can set the exact file size in HC.
(In HCgui, under "bitrate", switch from "average bitrate" to "file size". Then you can switch back to see the corresponding bitrate.)

does that take audio & subs into consideration? i doubt it.

[qyot27]

does that take audio & subs into consideration? i doubt it.
Why would it? HCenc only encodes video.

However, you can fudge it by making the size equal to the final video+audio size (subs take up virtually nil most times, just subtract the size from what you want to DVD itself to be, and then use the remaining number as the video+audio size), get the combined bitrate, subtract what bitrate you plan to encode the audio to from the combined bitrate, and voila.

[AlanHK]

does that take audio & subs into consideration? i doubt it.


4 GB for video
2-300 MB for audio (AC3 @ 128k)
subtitles negligible, 5-10 MB if I do them.
Menu, mux overhead about 20 MB (menu, no motion or sound).
Leaves usually 2-300 MB spare.

If you do high quality 5.1 sound or motion menus obviously you need to allow more space.

[~bT~]

^ well, some of the replies to halsboss were misleading..

obviously it is an encoder which can only work out the video bitrate via size.

what halsboss really needs is a calc: http://www.videohelp.com/calc.htm

[halsboss]

Ah, thankyou all. Wasn't aware of the filesize thing in HC , just wanted to max it out with the most detail as 1st choice. The calc will come in handy. Noone else uses FOX1 then ?

[~bT~]

Noone else uses FOX1 then ?

avamat6 or avamat7 is all i've needed.

[pdadi]

Great encoder . Thanks.

I have a NTSC DVD from PAL master. In my avs script I bring it back to 24.975fps (this looks best with all dupes gone).

After loading it in HC it is warning that it is not DVD complaint.

My plan is to do DGpulldown after HC encode is done to make it NTSC complaint.

Any solutions?

[qyot27]

Great encoder . Thanks.

I have a NTSC DVD from PAL master. In my avs script I bring it back to 24.975fps (this looks best with all dupes gone).

After loading it in HC it is warning that it is not DVD complaint.

My plan is to do DGpulldown after HC encode is done to make it NTSC complaint.

Any solutions?
24.975 isn't a proper MPEG-2 framerate, or at the very least it's not a PAL compliant fps. PAL actually is 25fps, it's not a situation like 24fps and 30fps often being used as aliases to 23.976fps and 29.97fps in NTSC land.

Use AssumeFPS(25,sync_audio=true).SSRC(48000,fast=false)* to correct for the audio difference (I'm assuming that your script has the audio correctly synced at 24.975), and then encode the 720x480, 25fps stream with HCenc. Afterward, you can use DGPulldown to set the right flags, and use mplex from the MJPEGTools project to mux the video and audio together. The .BAT file would look like this (I actually have a profile I use for 25fps material):

hcenc_023 -i "input.avs" -o "output.m2v" -ini "16-9_25fps-6000.ini"
dgpulldown "output.m2v" -inplace -srcfps 25 -destfps 29.97
mplex -f3 -V -o "finaloutput.mpg" "output.m2v" "audiofile.ac3"
I use mplex because it works. I'm not too up on MPEG-2 muxers, but I seem to remember that it was the only one I tried that handled 25->29.97 pulldown flags correctly.

*Sometimes that will fail, and can be fixed by sticking another SSRC(48000,fast=false) command before the AssumeFPS call. And sometimes even that will fail, requiring the before-AssumeFPS SSRC samplerate to be raised slightly: 49000, 50000, 51000, or 52000 usually work - although going over that tends to result in access violations (at least on my computer it does).

[pdadi]

thanks for the info qyot27. with 24.975 audio is always in in synch. just adding assumefps(25) at the end and encode it in HC and do a dgpulldown 24.975 to 29.97 might work

[QuadraQ]

Ooops ... works perfectly when I
- Do the encoder settings in HCgui_023 and save the HCini
- Exit HCgui and fire up HCenc_023.exe separately
Looks as if the HCgui cannot start the encoder properly.
Scratching my head .... perhaps something is borked here ....

Just wanted to add that I ran into the exact same issue. I create a dga from a film avc mkv file, and then create an avs file using meGUI. Open it up in HCgui 0.23 fine, but when I press encode it's stuck. Tried opening up the HCenc 0.23 directly as suggested and it runs through the first pass fine, but then freezes on the second pass. Suspend then resume doesn't help. :-(

Here's my AVS script:
AVCSource("C:\Users\Isaac\Videos\720p AVCHD\Test.dga")
#deinterlace
#crop
Spline36Resize(720,480) # Spline36 (Neutral)
#denoise

Here's my HC.ini file:
*INFILE c:\users\isaac\videos\720p avchd\Test.avs
*OUTFILE C:\Users\Isaac\Videos\Temp Files\Test.m2v
*LOGFILE C:\Users\Isaac\Videos\Temp Files\Test.log
*BITRATE 4000
*MAXBITRATE 8000
*FRAMES 0 71154
*AUTOGOP 12
*PROGRESSIVE
*PULLDOWN
*MATRIX mpeg

I've tried the 0.24 beta as well with the same results. I'm tearing my hear out here!

[Guest]

Hank never gave me a solution for this and without the source code I am powerless.

[QuadraQ]

I should have stated that this only seems to be a problem with the NV version of DGAVCDec. When I tried it with the "normal" DGAVCDec it works perfectly fine.

[hank315]

It will be solved in the final 0.24 release, I can replicate the issue using DGAVCDecNV.
It's a complete lack of time ATM to get a new release out :o

[G_M_C]

Finnaly found the time to do a BD -> DVD conversion, using HCEnc & DVD-lab. The encoder works perfectly, as usual; But is was kind of funny.

It's been some time that i've done such a project, and i've upgraded to a Core 2 Quad Xtreme 9650, wich runs at a conservative 3200 MHz (400 x 8). The funny thing is that the whole 2-pass encoding process of a 2 hrs movie took less than the 2 hours the movie is long. It ran at more than 150 fps during encode :eek:

I remember times that encoding went @ 5 or 6 fps. That goes to show how technoligy has evolved in the years, but also that HCEnc seems to have matured considerably during that time as well. And that not oly gos for the speed of the encoder, but the quality it produces as well; Thats very very good (lum_gain=1 and AQ=1 seems to be a very good combo). So thanx again for all the work and time you have invested in the years !

[halsboss]

create a dga from a film avc mkv file, and then create an avs file ... Open it up in HCgui 0.23 fine, but when I press encode it's stuck. Tried opening up the HCenc 0.23 directly as suggested and it runs through the first pass fine, but then freezes on the second pass.
Oh NO !
It will be solved in the final 0.24 release, I can replicate the issue using DGAVCDecNV.
It's a complete lack of time ATM to get a new release out :o
Oh Good. Thank goodness, this is significant, DGAVCDecNV is of no use to me otherwise. Is it still a problem when using the lossless intermediate file ?

Any timeframe on it as yet ?

[Guest]

You can use version 0.22 as I do.

Unfortunately it's an HCEnc issue. I tried to work around it but there is no way.

If Hank wants to lend me the source code I can try to fix it, otherwise we are totally at his mercy.

Don

[halsboss]

Thanks Neuron2. OK.

It it works for a 1st pass for QuadraQ, I wonder if it'll work when an intermediate lossless file is used for the 2nd pass to "work off". Will clutch at that straw tomorrow :)

Hank, you into sharing, matey ?

[hank315]

Here's a new HCenc 0.24 beta (http://www.mediafire.com/?kuzzgimg2um):

- added *INTRAVLC command
- added 4:2:2 input / output
- fixed some issues using AVCsource input

This will fix the problems using DGAVCDecNV.

[Boulder]

Thanks a lot, hank :) It feels weird that v023 was released a year ago..time flies so fast!

[Sharc]

- fixed some issues using AVCsource input
This will fix the problems using DGAVCDecNV.

Thanks, hank!

[halsboss]

Hank, you are a Legend, mate.

[manolito]

Thanks a lot, Hank!

I can confirm that the -1pass and -2pass command line parameters are now working...:thanks:

Cheers
manolito

[Fr4nz]

Thanks Hank for your efforts!

Excuse me for this question: did you see this (http://forum.doom9.org/showthread.php?t=146537) thread? Are you planning something in the upcoming 0.24, in order to improve encoding in "dark areas"?

[kumi]

Thanks for the new beta!

Can hank315 or someone else explain in layman's terms what INTRAVLC is, and what it's good for?

[hank315]

About IntraVLC...

MacroBlocks can be encoded in two different ways:

-------------------------------------
| Intra VLC | 0 | 1 |
|------------------------------------
| Intra blocks | B14 | B15 |
|------------------------------------
| Non-intra blocks | B14 | B14 |
-------------------------------------

B14 and B15 are two tables which are used to perform the VLC: Variable Length Coding.
The outcome of this will be written in the video bitstream.
Table B14 is more efficient when there are only a few values in the 8x8 block, table B15 is more efficient when there are lot of values in the 8x8 block.
So in general table B14 should be used for low bitrates, B15 for higher bitrates.
IntraVLC can be changed for every frame.
So *INTRAVLC 0 means: always use table B14, *INTRAVLC 1 means always use table B15, *INTRAVLC 2 will use the VLC method which gives the lowest nr of bits for that frame, *INTRAVLC 2 simply does both and picks the table which generates the lowest nr of bits to represent the encoded block.

[scharfis_brain]

How much gain in bitrate do you estimate by Automatic IntraVLC compared to fixed IntraVLC?

[hank315]

The table shows file sizes of a testclip for different INTRAVLC settings using different Quantizers.
All other settings are the same so it shows the efficiency of the INTRAVLC setting only.

---------------------------------------------
| CQ | intravlc 0 | intravlc 1 | intravlc 2 |
|----|------------|------------|------------|
| 2 | 101674 | 100606 | 100576 |
| 3 | 53550 | 53140 | 53086 |
| 4 | 45531 | 45247 | 45182 |
| 5 | 33787 | 33782 | 33629 |
| 6 | 28421 | 28541 | 28330 |
| 8 | 23530 | 23844 | 23507 |
| 10 | 16561 | 17036 | 16561 |
---------------------------------------------
It shows table 14 (intravlc 0) is better at low bitrates (high Q) and table 15 (intravlc 1) is better at high bitrates (low Q).
Intravlc 2 (auto mode) always produces the smallest file size and converges at low bitrates to table 14.

So the gain using auto mode is pretty small... but it comes at almost no extra cost, about 1% extra encoding time.

[doxville]

Hi hank315, many thanks for all the time you spend on this great encoder!

Since 0.24 is a beta, I wanted to ask you, if you had the time to look over this issue (http://forum.doom9.org/showpost.php?p=1161337&postcount=60), because I found the same behaviour as in 0.23. Would be great if that could be fixed in the final release (i am still sticking to 4:3 encodings), even if it's not a big thing to change AR yourself.

:thanks:

[cweb]

Just to thank you Hank for your new beta release..

[zack_ind]

Here's a new HCenc 0.24 beta (http://www.mediafire.com/?kuzzgimg2um):

- added *INTRAVLC command
- added 4:2:2 input / output
- fixed some issues using AVCsource input

This will fix the problems using DGAVCDecNV.

Adaptive Quantization even when set to 0 doesnt work. In the ini file its not there but once we encode the log file shows it as 2.
This is in 0.24 beta , 1 pass vbr is quite brilliant

[manolito]

Small Bug in the current 0.24 beta:

Contrary to the documentation the default INTRAVLC mode is 0 and not 2. To enable the INTRAVLC auto mode you have to add the "*INTRAVLC 2" command to the HC.ini file.

Cheers
manolito

[Guest]

This will fix the problems using DGAVCDecNV. I only just noticed this. :) Thanks!

I'm curious as to the nature of the bug that caused that. Can you give a quick explanation, please?

[hank315]

@neuron2
Previous versions did something like this:

avsDLL = LoadLibrary("avisynth.dll");
env = CreateScriptEnvironment(AVISYNTH_INTERFACE_VERSION);
...
//get video info
Video = new PClip();
*Video = env->Invoke("Import", AVSValue(arg,1)).AsClip();
get movie info...
...
//pass1
delete Video;
Video = new PClip();
*Video = env->Invoke("Import", AVSValue(arg,1)).AsClip();
get frames and encode...
...
//pass2
delete Video;
Video = new PClip();
*Video = env->Invoke("Import", AVSValue(arg,1)).AsClip();
get frames and encode...

HCEnc 024 does something like this:

avsDLL = LoadLibrary("avisynth.dll");
env = CreateScriptEnvironment(AVISYNTH_INTERFACE_VERSION);
...
//get video info
Video = new PClip();
*Video = env->Invoke("Import", AVSValue(arg,1)).AsClip();
get movie info...
...
//pass1
delete Video;
iret = FreeLibrary(avsDLL);
avsDLL = LoadLibrary("avisynth.dll");
env = CreateScriptEnvironment(AVISYNTH_INTERFACE_VERSION);
Video = new PClip();
*Video = env->Invoke("Import", AVSValue(arg,1)).AsClip();
get frames and encode...
...
//pass2
delete Video;
iret = FreeLibrary(avsDLL);
avsDLL = LoadLibrary("avisynth.dll");
env = CreateScriptEnvironment(AVISYNTH_INTERFACE_VERSION);
Video = new PClip();
*Video = env->Invoke("Import", AVSValue(arg,1)).AsClip();
get frames and encode...
Apparently CUDA doesn't like the old method, also the 024 GUI now releases all AVS stuff before starting the encoder.
Until now it passed all my tests using DGAVCDecNV but running multiple instances of HCEnc with CUDA won't work.

@manolito
About the intraVLC, you're right, default is now mode 0, should be 2 (auto), fixed... thanks for the report.

[Boulder]

Hank, I keep getting a Windows crash report message every time HC023 or 024 beta finishes encoding. This happens even with a simple script with only MPEG2Source in it (the latest DGIndex/DGDecode.dll used). I recall seeing the issue in DVD-RB some time ago, but jdobbs did something to fix it in his application. I uploaded the crash dump and a debugging log, here's the link: http://www.mediafire.com/?sharekey=e1621205ec925d0f4c17ca8801618ef7e04e75f6e8ebb871

I have not made any massive changes in my system, also different Avisynth versions do not help nor does emptying the plugins folder. It was not a long time ago that I was able to run multiple HC encodes in a row from a batch file so I'm quite puzzled.

[AlanHK]

I get random crashes with the message in the log:
*** ERROR, Avisynth message: invoke environment error - pass 2

AVS script:
DirectShowSource("17.m4v",fps=23.976,ConvertFPS=True)
EnsureVBRMP3sync()
ConvertToYV12()
AmplifydB(11.07)
LanczosResize(688,448)
AddBorders(16,16,16,16)

Source file is M4V, 720x544

HC invoked with this batch:
set rate=2164
set ar=4:3
set hc=P:\HCenc\HCenc_023
set mpgd=s:\mpg\
for %%A in (
16.avs
17.avs
18.avs
) do (
%hc% -i %%~fA -o %mpgd%%%~nA.m2v -b %rate% -aspectratio %ar% -pulldown -profile best -matrix qlb -frames all -noini -2pass -maxbitrate 8000 -log %%~dpAhenc.log -avsreload
)

File 16 encoded okay.
All files open in VDub, and I can extract audio with wavi with no issues.

ffdshow is being used by Avisynth.

I restarted the encode, and had Process Explorer open to observe when pass 1 was finished.
This time it crashed Windows (Win2k) and the the computer rebooted.

I tried again, and the third time it encoded perfectly ?!
When running on this HC is using about 360 MB RAM, out of 1 GB total. Nothing else running

I use the avsreload option hoping it would assist here, but obviously not.

Any point trying v 0.24? It isn't on the normal download page, so is it ready for general use?
I don't need any new features, but if this bug is fixed I'd obviously try.

EDIT:
With nothing to lose, I did try 0.24.
So far it's done the file that crashed and two others.
A few other nice enhancements too.
As I said, these crashes are erratic, too early to pronounce it solved, but it looks good.

[Sharc]

There still seems to be an issue with the interoperability of DG...NV tools (beta7) and HCenc 0.23 / 0.24 beta:
After running pass 1 successfully, pass 2 of HCenc just does not start. Any remedy for it?

[hank315]

A new beta of HCenc 024 is out.

Major changes:
- added *INTRAVLC command
- added *MINBRFAC command
- added 4:2:2 input/output
- fixed some issues using AVCsource input
- better fade detection/handling

and some other minor changes and bugfixes.

INTRAVLC 2 is now default, every frame uses the optimal intra VLC table.
The *MINBRFAC command shifts the lower part of the compression curve.



Link: HCenc024 beta (http://hank315.nl/files/HC024_beta_22-09-2009.zip)

[Boulder]

Thanks hank :)

[cweb]

thanks from myself too :)

[Dkruskie]

Thanks hank