HCenc 022 released

[45tripp]

drag and drop for gui input.

[smok3]

ok, one more: dear santa, please bring me: fast, 1pass bitrate-managed vbr (compatible with dvd).

[buzzqw]

ok, one more: dear santa, please bring me: fast, 1pass bitrate-managed vbr (compatible with dvd).

i do 1 pass CQ with constraint max bitrate...

BHH

[smok3]

i do 1 pass CQ with constraint max bitrate...
buzzqw: that would be more like CBR?

[buzzqw]

no propely.. since is based on constant quality ,the bitrate will be heavily variable.. since every frame will be encododed in cq..

like crf in x264 .. but with the possibility of specify the maxbitrate

BHH

[smok3]

ok, so how would you do it if calculated avearge bitrate is 2500kbps? (just an example)

[dragongodz]

since is based on constant quality ,the bitrate will be heavily variable.. since every frame will be encododed in cq

please, will people get it through their heads. CQ does NOT mean Constant Quality, it means Constant Quant. they are NOT the same thing.

while CQ is VBR because of the difference in size different frames can be it is lacking in desired size targeting. a simpler possible way to add such would be to do ABR over something like 4 GOPs.

hank315 already knows this since he and i talked about it some time ago.

[manono]

Hi-

Anyone know what this message means when the .avs is opened in HCEnc:

"error loading Avisynth script,MPEG2Source:,Force Film mode not supported for frame repeats (C:\Documents and Settings\...\t2.avs, line2)."

I'm trying to help a guy do an NTSC2PAL conversion and he's run into a roadblock. Here's his full script (with a D2V made using Forced film):

LoadPlugin("C:\Programmi\dgmpgdec149\DGDecode.dll")
MPEG2Source("I:\T2_DGIndex\t2_movie.d2v")
Lanczos4Resize(720,576)

http://forum.videohelp.com/topic342775-30.html#1793453

Thanks.

[kumi]

The manual says:

Force Film - This option is intended for the special case of film content encoded at 23.976 fps but pulled down for display at 29.97 fps, i.e., 3:2 pulldown. This option will restore the 23.976 fps film rate while keeping audio and video in sync throughout the clip. This option works by ignoring pulldown flags and inserting/removing frames to maintain a constant and sync'ed output stream. Do not use this option on non-3:2 pulled-down material, and, therefore, never use it on PAL or MPEG1. Also do not use this option on streams containing frame repeats.


and also:

Fix D2V
[...]
Note that field order correction should not be applied to streams containing frame repeat RFFs.

[manono]

Thanks, kumi. Maybe I should have read the manual and figured it out myself. So, what's the answer? Would not fixing the D2V be enough, do you think? Or is he going to have to perform a full IVTC? I think this is basically a progressively encoded DVD, with a very little bit of video, perhaps at the very beginning.

[kumi]

I'm not sure, but aren't repeat flags a sign of soft telecine?

That is, maybe his source is 3:2 soft pulldown, and all he has to do to output a progressive 23.976 FPS stream is turn on "Honor Pulldown Flags". And not use the Fix D2V tool, of course.

[manono]

I'm not sure, but aren't repeat flags a sign of soft telecine?
When he ran the preview for a short time he got 98% Film. So I'm pretty sure there's only hard telecine (or true video) at the very beginning, some black frames or perhaps a studio logo or some such.

[buzzqw]

latest dgindex version will not prompt/flag d2v with this message if used in command line mode

BHH

[manono]

Thanks, buzzqw,

I'll have him upgrade his DGMPGDec package.

[manolito]

After many encodes with HCenc 0.22 which did not cause any problems today I finally could replicate Nomolu's findings.

Original post: http://forum.doom9.org/showthread.php?p=1066474#post1066474

My clip was an interlaced MPEG file (DVB-T capture). HC settings:
CQ_MAXBITRATE 2.13
CQ_PFACTOR 1.01
CQ_BFACTOR 1.45
LUMGAIN 3
Scene Change detection ON
DC Precision 10
Auto GOP and Auto Interlaced
PROFILE FAST
Max bitrate 8500
(TargetAVG bitrate around 7600)

Muxman had the following complaints:
Bytes remaining in buffer = 29784.
Bitrate - avg: 7577388, min: 354986 (lba 0), max: 8864914 (lba 508043).
Shortest GOP has 4 fields, longest GOP has 30 fields.
Fields: 249710, VOBU: 8494, Sectors: 2309753.
Video decoder buffer underflowed 2 times.
Check c:\muxman.log for details.
Resulting DVD will probably not play smoothly.

I should mention that muxing with Imago and authoring with DVDAuthor worked just fine, also no stuttering on my SA. But still it looks like the VBV buffer check does not always work reliably in 0.22.

Cheers
manolito

[Chumbo]

Following up again per some of my earlier posts. I just went back to 022 again to try it and got this again:
---------------------
| encoding - pass 2 |
---------------------

*** INFO, adjusting average bitrate: -0.16 %
*** INFO, adjusting average bitrate: -0.33 %
*** INFO, adjusting average bitrate: -0.38 %
*** INFO, adjusting average bitrate: -0.45 %
*** ERROR, source mismatch in pass 2 starting at frame: 25826

*** INFO, adjusting average bitrate: -1.01 %
*** INFO, adjusting average bitrate: -1.08 %
*** INFO, adjusting average bitrate: -1.35 %
*** INFO, adjusting average bitrate: -1.40 %
Am I the only one who consistently gets this error? I'm finding it difficult to move to 022 from 021 because I get these errors. I then have to redo the project in 021 or take my chance and run it through 022 again to see if it goes through cleanly.

Do we know why this keeps happening? Is there any info I can provide to help narrow down why I keep getting these? Let me know how I can help please. Thanks.

[Boulder]

There is something different between the sources in the two passes. Looking at your earlier posts, I think you need to find a different way to process your sources. I'm quite sure that the same error occurs in HC021 as well (because it belongs to the source decoding chain) but it's just not shown in the log.

[squid_80]

If you're using directshowsource, that's probably where the problem is. Another possible cause is using unpacked divx/xvid avi files as sources in your script.

[smok3]

Chumbo, try with *AVSRELOAD

[Chumbo]

Chumbo, try with *AVSRELOAD
I do use this. Per my earlier posts, I've gotten this with and without this setting. Currently I am using this setting.

I reran the encode last night and got it again, but early on in the 2nd pass: ---------------------
| encoding - pass 2 |
---------------------

*** ERROR, source mismatch in pass 2 starting at frame: 916

*** INFO, adjusting average bitrate: -0.16 %
*** INFO, adjusting average bitrate: -0.32 %
*** INFO, adjusting average bitrate: -0.37 %
*** INFO, adjusting average bitrate: -0.94 %
*** INFO, adjusting average bitrate: -1.00 %
*** INFO, adjusting average bitrate: -1.26 %
*** INFO, adjusting average bitrate: -1.30 %

@squid_80,
My sources are always avs scripts loading a grf file via directshowsource. My grf files are always either avc or vc1 sources. My scripts are pretty simple, for example:directshowsource("video.grf",fps=23.976,audio=false)
converttoyv12()

@Boulder,
Thanks for the info on 021. I wasn't aware that this wasn't logged. I thought it was logged which was a bad assumption on my part. When you say find a different way to process my sources, what exactly do you mean? Because I've tried moving streams into an MKV container and I still had the same problem. Is that what you mean?

Thanks for the ideas and info. :)

[DeeZiD]

@Hank:

Thank you for this incredible piece of software.
I'm filming in HDV with a Canon A1 and HV20.
Both are great cams and deliver HD Quality.


Converting to DVD was always a dilemma.
Before using avisynth plus hcenc I've used Virtualdubmod (for resizing, msharpen and neatvideo) and Canopus Procoder or squeeze (for mpeg2 encoding).

Canopus mpeg2s were artifact-free and had the right colours (after playing around with the colourshrink filter) but they were too soft.
Squeeze mpeg2s had many artifacts and too high contrast but they were crisp.


Avisynth (spline36, msharpen) + HCenc delivers the crispest picture, correct colours and an absolutely artifact-free picture !!! :thanks::thanks::thanks:


Opensource rulez.


regards Dennis

[halsboss]

Hi. HC is fantastic. :thanks:

Just a possible bug report for 022... An encode using 022 has started caused problems with multiple SONY DVD players "freezing" and "skipping" at certain points.
In the attached extracted clip, at about the point the red book opens. In the full encode at many points thereafter. Same encode with 021 works OK. (No longer have access to the source file.)
I'm back to 021 for the time being.

Download-Link: http://rapidshare.com/files/81896471/cut.vob.html
Delete-Link: http://rapidshare.com/files/81896471/cut.vob?killcode=12017906609541193058

*INFILE D:\HDTV\cut-CQ3.avs
*OUTFILE D:\HDTV\cut.mpv
*PROFILE BEST
*ASPECT 16:9
*BITRATE 9200
*MAXBITRATE 9400
*LASTIFRAME
*AUTOGOP
*PULLDOWN
*CLOSEDGOPS
*DC_PREC 10
*LUMGAIN 3
*PRIORITY NORMAL
*CQ_MAXBITRATE 3
*MATRIX MPEG

[Boulder]

Try authoring the HC022 encoded video in MuxMan and see what it reports.

[halsboss]

OK, thanks. Will demux the "bad" VOBs with the latest DGIndex and do that. Had used DVDlab to author this one, although usually use VideoReDo TV Suite. Am I looking for something particular like a log file or something or just to see how it plays ? 1st time I've had any problem with any of the s/w... :)

edit: PS the downloadable "cut" was created from VideoRedo's "Trim and Copy" from the 1st VOB, so I hope it hasn't mucked up the example.

[Boulder]

MuxMan should provide you with a warning if there are underflows, and it also outputs a log file (on my system it's c:\muxman.log). I suspect that you have underflowing when the skipping occurs but MuxMan will verify that.

[manolito]

@halsboss

There are several things you could try. A while ago Nomolu had the same issue (original post: http://forum.doom9.org/showthread.php?p=1066474#post1066474 ), and he solved it by switching to the "Normal" profile.

And your bitrate settings do seem a little high, especially considering that you are in CQ_MAXBITRATE mode. In this mode HC will always overshoot the max bitrate by about 500 kbps (much less than CCE in OPV mode or QuEnc in 1-pass VBR mod).

And please do what Boulder said and use Muxman. For me Muxman has become the ultimate compliance checker. If Muxman has no problems with the source streams then you can be 100% sure that every player will play it without stuttering.

Good luck
manolito

[Boulder]

And your bitrate settings do seem a little high, especially considering that you are in CQ_MAXBITRATE mode. In this mode HC will always overshoot the max bitrate by about 500 kbps (much less than CCE in OPV mode or QuEnc in 1-pass VBR mod).I've never seen this happen..or does it happen in HC022 only? I've done lots of CQ encodes with CQ_MAXBITRATE and the maximum bitrate according to the logfile has always stayed under the specified limit.

[halsboss]

Thankyou Boulder and manolito. MuxMan log below. Interesting about the 30 gopsize, I'd thought it was supposed to be PAL compliant so I'd understood that wrong. :) Should use "*AUTOGOP 15" I guess.

MuxMan version 0.15R
Accepted video D:\HDTV\c02\bad\bad.m2v size = 2393019636
no multichannel extension found.
Accepted audio D:\HDTV\c02\bad\bad.mpa

20:10:19 Begin multiplex VTS01.
Title Segment List
Segment_1
Maximum audio duration 167092 fields.
Starting scene Segment_1_scn1 at 00:00:00:00
End of video file
Bytes remaining in buffer = 0.
Bitrate - avg: 6030451, min: 1146880 (lba 1229973), max: 9803093 (lba 985404).
Shortest GOP has 14 fields, longest GOP has 30 fields.
Fields: 167096, VOBU: 5624, Sectors: 1230059.

20:16:30 Begin multiplex VMG.
20:16:30 End multiplex.


Let me re-do that test from a known bung DVD, just in case.

PS @halsboss
And your bitrate settings do seem a little high, especially considering that you are in CQ_MAXBITRATE mode. In this mode HC will always overshoot the max bitrate by about 500 kbps (much less than CCE in OPV mode or QuEnc in 1-pass VBR mod).

:) I figured high bitrate can't hurt and can only help. For a lot of my encodes the CQ in the HC window seems to spend a lot of time floating above the "3", which I thought it does when it needs to limit to the nominated bitrate, so I guess it needs it a fair size anyway ? I my arithmetic is right, with overshoot it's still within PAL spec assuming 192kbs audio.

[Boulder]

Thankyou Boulder and manolito. MuxMan log below. Interesting about the 30 gopsize, I'd thought it was supposed to be PAL compliant so I'd understood that wrong. :)15 frames == 30 fields ;)

[halsboss]

30 ... silly me. :)

Confirmed, that's the one with mpv and mpa extracted from DVDlab created DVD's VOBS using DGindex. Those DVDs (2 copies) stutter and whatnot on 3 independent standalone DVD players.

Next step I suppose, I'll burn the muxman produced files and see if the same result arises.

[halsboss]

I'll burn the muxman produced files and see if the same result arises.

Yes, same problem arose with the newly burned DVD from the muxman authoring. Stuttered at the same places - within a few secs and then again at about the red book opening.

Can't fault the discs as tried 2 different disc brands, also noting the 021 encode worked ... :confused: :o Oh well HC is still a fantastic product. Back to 021 I suppose as I'm stumped. PS VOBS play "fine" on PC with PowerDVD4...

[Boulder]

Could you please upload a sample of the same scene with HC021 as the encoder? I think it would be useful if hank (and us amateurs :)) could compare the two different encodes since you have proven that there are no underflows but it's something else.

[halsboss]

Sure ! :thanks: Searched and found the original input file too.
Below are the files demonstrating a potential issue with HC 022 vs HC 021. Both HC 022 outputs stutter etc on both auto and progressive whereas 021 work fine. I can't remember if *PROGRESSIVE is in 021, so I tried it as a variation anyway.

The input .AVI file chopped to 10secs zipped with the .INI files, the .BAT file, the .AVS file, the .LOG file, and the .MP2 file (note slight differences in the .INI files) - (1.5mb) -
Download-Link: http://rapidshare.com/files/82078014/test.zip.html

The .AVS

SetMemoryMax(256)
#
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins-dcw\mt_masktools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins-dcw\RemoveGrainSSE2.dll")
Import("C:\Program Files\AviSynth 2.5\LimitedSharpenFaster.avsi")
#
FRAMERATE=25
WIDTH=704
HEIGHT=576
#
global resizeWidth = 704
global resizeHeight = 576
global resizeBorderHalfHeight = 0
#
AviSource("D:\HDTV\c03\test.avi", audio=false)
AssumeFPS(25)
#
#Trim(0,-1500)
#
LimitedSharpenFaster(smode=4, dest_x=resizeWidth, dest_y=resizeHeight)
#
Converttoyv12()
#
SetPlanarLegacyAlignment(True)


The .BAT :-

echo on
"C:\SOFTWARE\ffmpeg\ffmpeg.exe" -i "D:\HDTV\c03\test.avi" -ab 192k -ar 48000 -ac 2 -acodec mp2 -y "D:\HDTV\c03\test.mp2"
"C:\SOFTWARE\HC\HCenc_021.exe" -ini "D:\HDTV\c03\test-021.ini"
"C:\SOFTWARE\HC\HCenc_021.exe" -ini "D:\HDTV\c03\test-021-progressive.ini"
"C:\SOFTWARE\HC\HCenc_022.exe" -ini "D:\HDTV\c03\test-022.ini"
"C:\SOFTWARE\HC\HCenc_022.exe" -ini "D:\HDTV\c03\test-022-progressive.ini"
pause


The output .MPVs range between 6.5Mb and 7.5Mb each -
Download-Link #2: http://rapidshare.com/files/82078015/test-021.mpv.html
Download-Link #3: http://rapidshare.com/files/82078016/test-021-progressive.mpv.html
Download-Link #4: http://rapidshare.com/files/82078017/test-022.mpv.html
Download-Link #5: http://rapidshare.com/files/82078018/test-022-progressive.mpv.html


Delete Links :-
Delete-Link #1: http://rapidshare.com/files/82078014/test.zip?killcode=16676761579340406147
Delete-Link #2: http://rapidshare.com/files/82078015/test-021.mpv?killcode=11572470511075943680
Delete-Link #3: http://rapidshare.com/files/82078016/test-021-progressive.mpv?killcode=7649425213280276128
Delete-Link #4: http://rapidshare.com/files/82078017/test-022.mpv?killcode=8225979848388166894
Delete-Link #5: http://rapidshare.com/files/82078018/test-022-progressive.mpv?killcode=5361686554984333942


edit: the 022 DVD worked on one standalone player (brand unknown), however it stuttered badly on 2 late-model Sonys and 3rd "other" too.

[ron spencer]

any update on this?

[Boulder]

You might want to try lowering the maximum and average bitrates and test that too. I'm sure hank will respond when he has the time :)

[cweb]

Does anyone have problems running hcenc 022 (gui) via a remote desktop? Encoding stops if hcenc has been started before the connection is started.

[halsboss]

You might want to try lowering the maximum and average bitrates and test that too. I'm sure hank will respond when he has the time :)

OK. :thanks: I authored a DVD with all of the clips and a menu, so I could compare the effect. Adding the clip created with the settings below resulted in the same stuttering at the same places for that clip ...

*INFILE D:\HDTV\c03\test.avs
*OUTFILE D:\HDTV\c03\test-022-6000.mpv
*LOGFILE D:\HDTV\c03\test.log
*PROFILE BEST
*ASPECT 16:9
*BIAS 30
*BITRATE 6000
*MAXBITRATE 6400
*LASTIFRAME
*AUTOGOP
*PULLDOWN
*CLOSEDGOPS
*DC_PREC 10
*LUMGAIN 3
*PRIORITY NORMAL
*CQ_MAXBITRATE 3
*MATRIX MPEG

[hank315]

@halsboss
Just looked at the clips, I couldn't find any errors in all four.
The main difference between the HC021 and HC022 encodes is the bitrate for HC022 is a bit higher but not too high.
The higher bitrate using HC022 is caused by the LUMGAIN command which adapts the matrices.
Some players don't like that, it's similar to the adaptive Q-matrix switching by CCE, in HC022 it's applied more frequently.
Please try the HC022 encode without the LUMGAIN command.


@manolito
About the VBV underflow issue Nomulu reported, I can replicate that one, got the source file from Nomulu.
It will be fixed in the next release, it's the last thing to do, always best to do the most difficult part last :(

[halsboss]

Please try the HC022 encode without the LUMGAIN command.

:thanks: hank.

Yes, removing *LUMGAIN 3 and leaving the rest at high bitrate etc did resolve the problem on the standalone players. I liked the *LUMGAIN too, but cannot be confident whether a result will play on standalone DVD players (previous encodes had worked !). This encode with *LUMGAIN 3 worked on only 1 of 4 standalone DVD player models tried, with the Aus n and n-1 model Sony's failing. Jolly, bother, and dash it all :). Oh well, I'll move to 022 and omit the problematic *LUMGAIN from now on even though HC is producing "the right thing" with it.

Thanks once again for a terrific software, hank.

[ron spencer]

I have never used LUMGAIN. Can someone tell me exactly what it is for and when to use it...perhaps a sticky is needed on this as it seems to be all the rage. It alters bitrate right? Is that what the issue is with standalones?

thanks!!!

[halsboss]

022 pdf manual says

*LUMGAIN
parameter sensitivity (0 - 4) type integer
Status not required
Default 0
Example *LUMGAIN 2
This command adapts the quantization matrices on dark scenes so the bitrate will be raised.
• 0 no change
• 1 mild change
• 2 moderate change
• 3 heavy change
• 4 hit dark scenes real hard

Not sure of the exact issue, hank said
caused by the LUMGAIN command which adapts the matrices

[ron spencer]

duh...I am an idiot.....never read the manual...sorry.

I guess one needs to know the bitrate gain for the options set, so that the stuttering problems some have faced are avoided.


thanks....

[halsboss]

I don't know how to find the bitrate gain :)

The mpv's are in this post http://forum.doom9.org/showthread.php?p=1084341#post1084341
and the latest mpv without the *LUMGAIN is at http://rapidshare.com/files/82567023/test-022-no-lumgain.mpv.html which I guess one could compare to http://rapidshare.com/files/82078017/test-022.mpv.html

Are you sure it's only bitrate gain and not matrices changing or something ? (I don't know what's in an mpv, so I don't know if that's a valid question). The reason I ask is that muxman accepted the mpv as valid.


The 1st stutter is within the first few seconds. The audio keeps going but the picture freezes for a second or two.

Delete-Link #1: http://rapidshare.com/files/82567023/test-022-no-lumgain.mpv?killcode=16058216441092167457

[ron spencer]

hank may need to document the gain....and then we can alter our encode.

[mashiwen]

"DirectShowSource ("main1.grf",fps=23.976,framecount=158725,audio=false)"
I think framecount is a new command in avis v 2.5.7,and HCenc 022 says that DirectShowSource doesn't have a named argument"framecount":thanks:

[cweb]

Did anyone else encounter a problem connecting via remote desktop to a running hc encoder, since this seems to just stop and not continue encoding when you do this?

[MrC]

Constant Quantization issue.

AVS Script:
DirectShowSource("E:\Movies\clip.avi", convertfps=true)
ConvertToYV12
Lanczos4Resize(720,576)
SSRC(48000)

Using HCenc thru CLI, output is very very flickering and blocky:
-i "C:\Documents and Settings\CORSIE\Desktop\prova.avs" -o "E:\Movies\VideoCQP.m2v" -maxbitrate 8000 -profile fast -aspectratio 4:3 -nointerlaced -dc_prec 8 -matrix MPEG -cq_maxbitrate 4.667 -wait 2

Using HCenc thru INI file, output is fine and encoding is faster:
______________________________________________
*infile C:\Documents and Settings\CORSIE\Desktop\prova.avs
*outfile E:\Movies\videocqp.m2v
*maxbitrate 8000
*profile fast
______________________________________________
*cq_maxbitrate 4.667
*matrix MPEG
______________________________________________
*dc_prec 8
*progressive
*aspect 4:3
*bias 30
______________________________________________

I think that encoding options are exactly the same... So why that difference?

:confused:

Bye

[Mr_Odwin]

Constant Quantization issue.

AVS Script:
DirectShowSource("E:\Movies\clip.avi", convertfps=true)
ConvertToYV12
Lanczos4Resize(720,576)
SSRC(48000)

Using HCenc thru CLI, output is very very flickering and blocky:
-i "C:\Documents and Settings\CORSIE\Desktop\prova.avs" -o "E:\Movies\VideoCQP.m2v" -maxbitrate 8000 -profile fast -aspectratio 4:3 -nointerlaced -dc_prec 8 -matrix MPEG -cq_maxbitrate 4.667 -wait 2

Using HCenc thru INI file, output is fine and encoding is faster:
______________________________________________
*infile C:\Documents and Settings\CORSIE\Desktop\prova.avs
*outfile E:\Movies\videocqp.m2v
*maxbitrate 8000
*profile fast
______________________________________________
*cq_maxbitrate 4.667
*matrix MPEG
______________________________________________
*dc_prec 8
*progressive
*aspect 4:3
*bias 30
______________________________________________

I think that encoding options are exactly the same... So why that difference?

:confused:

Bye

I had this problem too. I don't think I speak out of turn when I say that hank is now aware and has already implemented a fix for the next release.

[Graigddu]

Can i ask whether there would be a better matrix for xvid encoding than the default MPEG or BACH 1 which i have tried,
i have also changed autogop to 12 and set lumigain to 2 and bias to 30 in the HCGui and use Mr Odwin's FAVC for the main setting changes but would another matrix work better at smoothing macroblocking in darker scenes.

Sorry if this is posted in wrong area still a newbie on here

[foustapfe]

Just for the record, the problem I reported some time ago ( 1 (http://forum.doom9.org/showthread.php?p=1039444#post1039444) and 2 (http://forum.doom9.org/showthread.php?p=1041798#post1041798)) is fixed in 0.22.

I don't know it is has been fixed actively or as a side effect of some other changes. Anywas, thanks Hank for this encoder that gets better every day!