HCenc 023 released

[rendez2k]

If you use reload Avisynth and you don't get an error everything should be fine.

The trouble is, in some apps that use HCEnc, you can't edit HC.ini to force reload. I'm wondering if using HC 0.22 would help?

[Boulder]

Search for HCWrap, it'll allow you to enter custom ini commands.

[halsboss]

Error "can't open dbs file, invalid path?" before 1st pass starts, from

*INFILE G:\DVD\testi\test1080iHD-CQ2.5.avs
*OUTFILE G:\DVD\testi\test1080iHD.mpv
*LOGFILE G:\DVD\testi\test1080iHD-HC-CQ2.5.log
*INTERLACED
*TFF
*PROFILE BEST
*ASPECT 16:9
*BIAS 50
*BITRATE 9300
*MAXBITRATE 9400
*LASTIFRAME
*AUTOGOP 15
*PULLDOWN
*CLOSEDGOPS
*DC_PREC 10
*MATRIX FOX3
*LOSSLESS
*DBPATH G:\DVD\testi
*LLPATH G:\DVD\testi

Any idea what I'm doing wrong ?

[Darksoul71]

Not really..just two questions:
1) Does G:\DVD\testi exist ?
2) Any special reason why you specify the DB path ?

[halsboss]

Thanks for looking; yes it's the same folder as the source, no reason other than I'm not sure what it defaults to and wanted to easily find and get rid of the files ...

edit removed the path lines and it works ... now where did those files get put ?

[Darksoul71]

My guess for the lossless file:
It goes to the same directory as the DB -> the directory where HCEnc resides when you call it.

[msdossys]

hank315, first let me say thank you for this very excellent program. Now, that being said, I have one feature request that seems to be in the GUI but not the encoder. When the output of the encode is set to a file that already exists, the GUI will ask you if you want to overwrite it. However in my haste after editing the source in my AVS, I double-clicked on the encoder instead of the GUI, and it just started encoding along on its merry way, trashing my 1hr encode that was just completed. Sooooo, is this a possibility to put in to save stupid people like me?

[Darksoul71]

@hank:
Just two short suggestions/feature requests:
1) Please add an option to set the DB filename. This would help to use HCEnc^n with multiple instances of HCEnc without generating a separate dir for every HC instance.
Currently *DBPATH doesn´t really help since I would have to create a separate DB dir for each HCEnc instance. This makes no big difference to my prior approach to copy the HCEnc.exe to a separate directory.

So something like *DBFILE c:\mypath\my_database.db would be extremely nice.

2) Please add an option to output a DB file during CQ encoding. As far as I´m aware of a DB file is only written in 2 pass mode, right ?
Nevertheless it would be cool to run a CQ encoding and have the option to do a second pass (non-CQ) if the resulting file is way off.
This is pretty similar to CCE which provided an option to write the VAF file during any encoding (be it CQ, CBR or VBR).

TIA,
D$

[Chumbo]

I wanted to report an issue I found yesterday. I was doing a reencode using a .d2v file. I normally just run the command line and feed it an .ini file. When I did this, the command line brought up the encoder status dialog which then reported that it cannot find an mpeg encoder. However, if I bring up the GUI, load the .ini file and run the encode, it works just fine. I searched this thread for this problem but didn't find anything as I didn't want to report a known issue.

This is with version 023.

[Nomolu]

I am having problems getting the aspect ratio choice to "stick". With 0.22 I can choose 4:3, hit save hc.ini, and the next time I start up hcenc the aspect ratio is set to 4:3. With 0.23 it seems to "reset" to 16:9 no matter what.
I'm seeing the same issue. Other options, such as BITRATE and MAXBITRATE, are loaded from the HC.INI file when HCGUI is launched. But "*ASPECT 4:3" is ignored. The ASPECT option only seems to work if I first load an AVS file, then manually load the HC.INI file.

[msdossys]

@hank:
Just two short suggestions/feature requests:
1) Please add an option to set the DB filename. This would help to use HCEnc^n with multiple instances of HCEnc without generating a separate dir for every HC instance.
Currently *DBPATH doesn´t really help since I would have to create a separate DB dir for each HCEnc instance. This makes no big difference to my prior approach to copy the HCEnc.exe to a separate directory.

So something like *DBFILE c:\mypath\my_database.db would be extremely nice.

You dont need HCEnc^n anymore... since 022 HCEnc has SMP built in :)

[mikeytown2]

Kinda merging 2 ideas here...


You dont need HCEnc^n anymore... since 022 HCEnc has SMP built in :)
HCenc^n splits up the video and then processes it.


Over in the AviSynth Dev Thread...
How about that idea: split a video into X parts, where X is the number of cores, let the parts overlap by the amount of frames, used by e.g. MVTools, process all parts simultaneously and rejoin them, removing the overlap?...

I could see HCenc splitting the avs into n different files since temp lossless is an option in 2 pass mode. For the second pass, HCenc would reference the multiple lossless files. This would only be useful if the script is a major CPU eater, thus it will not be beneficial at all times. I'm sure there might be filters out there that have potential issues when used simultaneously (GPU). But with more CPU's having more cores, and until a 2.6RC is done, this could be a big boost for people with heavy scripts and lots of cores. Don't know if it's worth the time and effort though...

[Gromozeka]

hank315
You may add trellis quantization and to increase length Gop up to 300 in HCencoder 0.24? :)
It is good for quality on low bitrate (and the autoring)

[Darksoul71]

Hi all,

@msdossys:
1st, I think it´s kinda unpolite to comment a feature request in this way.
No matter if the comment shows a smiley or not.

2nd, I agree that the MT implementation of HCenc is the best I´ve seen around compared to other freeware MPEG2 implementations (Mencoder, AQE) but I do not know any details in regard to approach Hank has choosen. In a lot of my tests HCEnc MT is as fast as HCEnc^n is. Sometimes HCEnc even beats HCEnc^n since HCEnc^n does the merging of the M2V segments and correction of the timecodes afterwards. Nevertheless I like HCEnc^n over HCEnc because I have the control over it. In HCenc^n I can e.g. limit HCEnc to two cores and leave to other cores free for AVISynth filtering. This could be a nice option for QuadCore owners.

All in all I do not see HCEnc^n and the MT implementation of HCenc as redundant. HCEnc MT is integrated in the encoder and more stable. HCEnc^n is external and allows the user more modifications. Esp. since I include the AutoIt sources with it.

@mikeytown2:
I really enjoy your idea. Of course this approach is limited to AVISynth filters which are not dependend to a single resource (GPU as you already named) but for complex scripts this could really help.
Just as my example above:
QuadCore CPU and only use 2 cores for HCEnc encoding and the other two CPUs free for AVS filtering.

Don't know if it's worth the time and effort though...
Hm, the effort is quite low. I simply had to hack in two more lines of AutoIt which add the *LOSSLESS and *LLPATH parameter.
I think I´ll make this controlable via a small "lossless" switch within the ini-file of HCEnc^n.

I´ll upload this version tomorrow after some testing but things are looking good.

@hank:
I agree with Gromozeka:
It would be nice to be able to set GOPs longer than 36. I used long gops on (k)SVCD and they always played back fine on my SAP.

Stay tuned,
D$

[Darksoul71]

Hi Hank,

could it be possible that HCEnc has a problem when parsing the ini-file and you´ve separated the command from the value by using a tab ?

I first added the *LLPATH and *DBPATH to the HCEnc.ini with a tab as separator in between.

Example
*LLPATH TAB C:\MyPath

This gave me the same problems as halsboss is describing here:
http://forum.doom9.org/showpost.php?p=1159595&postcount=53

HCEnc says ...error "can't open dbs file, invalid path?"

When I separate the command and the value by using blanks everything is working fine though.

Could you have a look at this and confirm it ?
:thanks:

I have no big problem to use blanks instead of tabs though :cool:

Later,
D$

[Darksoul71]

Yes, I know... it´s my third posting in this thread but merging them makes no sense to me since they are way to different.

I´ve (re)started to write a HCEnc prediction tool which acts as a wrapper (similar to HCWrap and HCEnc^n).
The idea is similar to CCEFront:
Make a simple commandline tool which does all the prediction and encoding itself. I´ll use the bitrate
specified and the movie length to calculate the target size. After this a simple prediction will take place (using
Inc´s old Ping Pong approach) and the final encoding will be done in CQ mode.

You could simply call the tool like HCEnc itself:
HCPred -ini "C:\my_movie.ini"
and the rest will be done automatically.

CQ encoding with HCEnc can often end up in mixed results. Sometimes a good 10 to 20% oversize are not that uncommon
(depending on your source).
CCEFront (as well as D2SRoba) had two nice features available for CCE:
1) Do a second pass in VBR mode if the CQ file is way too oversized
2) Shrink the resulting M2V with Rejig / Requant if the file is just a few percent too big.

I asked hank to output a DBS file even if you do only CQ mode. It would be even nicer if we could output a lossless file as well.

How it could work:
- Tool starts prediction
- HCEnc runs in CQ mode but outputs a lls file and a db file as well (specified by the tool)

If the resulting file overshoots the target size it could be either shrinked with Rejig / Requant (for up to 5% may be) or be transcoded in a "second pass" using the lossless file from the previous run as source.

What do you think ?

[45tripp]

bug:

from the log.

total CPU time: 0:07:07 (426.84 s)
total elapsed time: 0:04:12 (252.47 s)


total elapsed tme seems to be cpu time,
and cpu time is completely off.
hc 21 and 22 seemed fine.

request:

i'd like drag and drop for source input (of already valid inputs) into the gui.
and drag and drop wherever else it's valid, like ini input for example.


ty
tripp

[florinandrei]

HCenc 0.23 scales very well with the number of CPUs.
I'm transcoding a standard-def progressive MPEG2 source. I'm using the best profile. Previously, on a dual-core AMD (2600 MHz clock) I would get around 70 fps.
On my new Phenom quad-core (2500 MHz clock) I get around 140 fps. That's just awesome. :thanks:

[halsboss]

Fantastic product !!!

Couple of minor suggestions:

a) when trying to run 2 HC's with lossless set to the same path the 2nd crashes I guess since the filenames may be "fixed" ? Could the lossless files use say random names or the same as the .ini or output except with a different extension ?

b) instances all have the same name in task manager ... maybe the titles could reflect the file being processed ?

Just a thought. Thanks !!!

[Adub]

May I ask why you are running two instances of HC at the same time? This wonderful encoder was multithreaded ages ago, so there should be no reason to run to instances as the same time.

[halsboss]

Good question. 4 instances each with avisynth SetMTmode(mode=2,threads=2) and 256Mb setmemorymax. (q9450 4-core, 4gig memory, 750gb disk)

Consider the case of a PAL HDTV 1080i-> DVD 576i 4-quarter football match, with 4 per-quarter independent videos.

Each quarter edited independently, down-converted/HC'd independently, targetted to it's own DVD so as to use maximum DVD spec bitrate and hence highest possible quality, given it's 36 players in a very large grassed arena with very fast action.

Currently uses 2-pass profile=best 224k audio 9350k video fox1 matrix (happy to receive other suggestions as even then it doesn't max out the bitrate, hovering between 8800k and 9000k even though *AQ 0 *BIAS 70 *LUMGAIN 0 *BITRATE 9350 *MAXBITRATE 9350).

[blutach]

@halsboss,

Unless you're a Geelong supporter, don't worry :D :D :D

(Sorry for OT everybody).

Regards

[halsboss]

For some reason, parts of the grassed background seem to "blur/float" with the long-shot panning action ?? The source seems to have bits of it like that... would fox2 be slightly "gentler" with that effect do you think ?

@blutach :) Yes the Cats are a team of Big fast tough Men... Craigy's Crowbots are well drilled and never ever ever give up though :D Duracells, every lovely one of 'em (they keep on going).

[Boulder]

I'd say that MPEG2 at standard DVD bitrates is just not enough for such material :(

[halsboss]

Thanks, yes it seem I expect too much from the bitrate / resolution.

Oh well, my beloved footy team (the Crows, possibly to be renamed the Chooks) are now out of the finals, so I have about 6 months to fiddle and squeeze the most I can out of reasonable encoding times for PAL hdtv->dvd.

I did try *BIAS 100 and it still didn't max out the bitrate, under by 300k-700k, so I'm a bit unclear on what else to do. I'm using this resizing method http://forum.doom9.org/showthread.php?p=1177122#post1177122 rather than TDEINT&NNEDI deinterlacing/resizing/reinterlacing on the basis it looked the same or better visually and was faster ... open to suggestions on any front... since I have 6 months (it's a long long off season).

[AlanHK]

I was having the message :
ERROR, source mismatch in pass 2 starting at frame...
and found that setting "-avsreload" in the command line fixed this.

Is there any reason not to always use this, just as a precaution? There seems no noticeable delay.

[smok3]

May I ask when to use the *LASTIFRAME option?
Dose any editing/authoring software require the last frame to be an I-frame under some circumstances?

i wonder that too.

[pandy]

i wonder that too.

easier editing? - especially when joining few ES together or create some looped video - this make possibility to clean cut/join of ES - especially when few ES have different parameters (i.e. resolution, color space, interlaced vs progressive etc etc)

For me this give possibility to make seamless TS for playing in loop, can be useful for IP broadcast from playlists

[Chumbo]

I was having the message :
ERROR, source mismatch in pass 2 starting at frame...
and found that setting "-avsreload" in the command line fixed this.

Is there any reason not to always use this, just as a precaution? There seems no noticeable delay.
Reloading AVS (available in the UI too btw) will not always fix this issue. I encountered this issue regardless of this setting, so make sure to check your log after an encode to be sure.

What is 100% is using the lossless option and it's faster. However, you'll need a LOT of available hard drive space depending on what you're encoding.

[krosswindz]

I am also encountering the Error, source mismatch in pass 2 starting at frame... error. I have the option of reloading avisynth selected. I dont have a lot of space so dont have the option of running the lossless option. Any reason why this might be happening?

[mikeytown2]

What are you using to load your source? It needs to be frame accurate.

[krosswindz]

^ I am using HCEnc Gui to load the avs file.

[Chumbo]

I am also encountering the Error, source mismatch in pass 2 starting at frame... error. I have the option of reloading avisynth selected. I dont have a lot of space so dont have the option of running the lossless option. Any reason why this might be happening?
You should really consider getting an external drive just for this if you do a lot of encoding. It'll pay for itself in time spent doing this. Drives are inexpensive and all you probably need is a 250-300Gig drive. Good luck. :)

[mikeytown2]

^ I am using HCEnc Gui to load the avs file.

What are the files you are trying to load? AVI, FLV, ...? My guess is its a file that has a variable frame rate; which means you need to load it with the correct tools, not DirectShowSource().
http://avisynth.org/mediawiki/External_filters#Source_Filters

[krosswindz]

@chumbo yeah I have been thinking about getting an external drive. The problem being that I am running windows XP inside a VM in linux. I am not sure if I have to use fat32 on the external drive or if I can use ntfs.

@mikeytown2 I am trying to load DVD VOB files using DGDECODE_MPEG2source. I have pmed you the script I am using.

[Darksoul71]

@others: Sorry for OT

@krosswindz:
The problem being that I am running windows XP inside a VM in linux. I am not sure if I have to use fat32 on the external drive or if I can use ntfs.
Normally you should be able to use ntfs on the external drive. I never experienced any problems when accessing ntfs in read/write access from linux. Another approach might be using a linux FS on the external drive (ext3 for example) and publish it via Samba. From the Windows VMs perspective the underlying FS shouldn´t matter then.

Depending on your Virtualisation solution you might also be able to attach the USB drive to the VM directly. This is something which for example Sun´s VirtualBox supports. I haven´t been able to get this running on my Debian Host with a VBox WinXP Guest though.

[Chumbo]

@hank,
I've been trying to encode using the Lossless option on one of my computers but it has been crashing when checking this option. Seems to work fine with reload AVS, but I'd prefer to use the lossless file option.

The crash is just the standard application crash per the snapshot below. I tried all sorts of combinations of same and different paths and drives for the locations of both the lossless and db path, in case of some sort of clash, but nothing works. Please let me know if you want any other details. Thank you.
http://img360.imageshack.us/img360/7265/hclosslesscrashbe4.jpg

[halsboss]

Have you put 2 or more spaces between the option and the path/file in the .ini file ? :)

[Chumbo]

Have you put 2 or more spaces between the option and the path/file in the .ini file ? :)
On this PC, I happen to be using the GUI. I don't have this issue on my other computer either via the UI or if I run it from the CLI. But either way, I always create the ini file via the GUI.

[Boulder]

@hank,
I've been trying to encode using the Lossless option on one of my computers but it has been crashing when checking this option. Seems to work fine with reload AVS, but I'd prefer to use the lossless file option.

The crash is just the standard application crash per the snapshot below. I tried all sorts of combinations of same and different paths and drives for the locations of both the lossless and db path, in case of some sort of clash, but nothing works. Please let me know if you want any other details. Thank you.
http://img360.imageshack.us/img360/7265/hclosslesscrashbe4.jpgI wonder if that crash creates a dump file which you can analyse. The next time that happens, you could leave the crash report window open and see whether your Windows temp folder (I think it's usually c:\documents and settings\username\local settings\temp) contains a .dmp file. Copy that file somewhere safe and upload it somewhere so we can take a look at it. You have to leave the crash report window open because the dump files will be removed after the window is closed. Another place to look for the dump files is at c:\windows\minidump.

[Chumbo]

I wonder if that crash creates a dump file which you can analyse. The next time that happens, you could leave the crash report window open and see whether your Windows temp folder (I think it's usually c:\documents and settings\username\local settings\temp) contains a .dmp file. Copy that file somewhere safe and upload it somewhere so we can take a look at it. You have to leave the crash report window open because the dump files will be removed after the window is closed. Another place to look for the dump files is at c:\windows\minidump.
I sent the crash info via PM to you Boulder.

btw, I happen to test this on an mpeg2 source and it works just fine. It only crashes when the source is an AVC. I normally use the .dga file as the source in the input avs script, but it crashes if I use either AVCSource or directshowsource as long as the source is AVC. Maybe that'll help narrow down things.

I tested this on my "working" system and sure enough it crashed there too. What I didn't remember is that when it worked, my sources were all mpeg2. Sigh...

Let me know if you need any other info.

[moviemaven]

I have had the same exact problem here also. If I remove *LOSSLESS, it works okay, but crashes with it in.

MM

[Sharc]

I created an index file .dga of an .m2ts (VC-1 encoded) and an .avs using DGVC1IndexNV. I loaded the .avs successfuly in HCgui 0.23, and I can preview the movie in the "Preview/Chapter" Tab of HCgui, but when pressing "encode" nothing happens ...
Is HCenc not compatible with .dgv index files?

[Guest]

Works fine for me.

[Chumbo]

I have had the same exact problem here also. If I remove *LOSSLESS, it works okay, but crashes with it in.

MM
You can't just use this option by itself. You have to use it with a location. See the GUI for the option to set that. :)

I've moved my issue to the 024 beta thread, so it would be good to post any bugs you all have there.

[moviemaven]

You can't just use this option by itself. You have to use it with a location. See the GUI for the option to set that. :)

I've moved my issue to the 024 beta thread, so it would be good to post any bugs you all have there.

Actually, you do not need the location (LLPATH) so long as it is set in the program. I think LLPATH would override your default setting. Funny thing, I cannot reproduce the crash anymore, so it must have been only one kind of file.

Glenn

[Chumbo]

Actually, you do not need the location (LLPATH) so long as it is set in the program. I think LLPATH would override your default setting. Funny thing, I cannot reproduce the crash anymore, so it must have been only one kind of file.

Glenn
Oh yeah, I forgot. You're right. :) Is it working with your source being AVC?

[Sharc]

Works fine for me.
Ooops ... works perfectly when I
- Do the encoder settings in HCgui_023 and save the HCini
- Exit HCgui and fire up HCenc_023.exe separately
Looks as if the HCgui cannot start the encoder properly.
Scratching my head .... perhaps something is borked here ....

[moviemaven]

Oh yeah, I forgot. You're right. :) Is it working with your source being AVC?

Am I using AVCs?? I honestly don't know. I was under the impression that those are associated with h264 files. ALl my inputs have been AVIs.

[~bT~]

I created an index file .dga of an .m2ts (VC-1 encoded) and an .avs using DGVC1IndexNV. I loaded the .avs successfuly in HCgui 0.23, and I can preview the movie in the "Preview/Chapter" Tab of HCgui, but when pressing "encode" nothing happens ...
Is HCenc not compatible with .dgv index files?

exact same problem with same type of input file. hc 0.22 worked flawlessly.

pls let me know what info u need to rectify this if poss.