HC encoder, release 016

[hank315]

Thanks for the feedback, most issues are solved now:
- hangs while running minimized
- wrong GOP size for some fixed GOPs
- artifacts at the right side of some frames

The artifacts were caused because motion vectors could point out of the frame by half a pixel, some decoders don't like that...

Still one issue left, using Avisynth DirectShowSource with the WMV decoder sometimes won't read the stream correct during the second pass, it can skip frames or even jump back.

HC016 latest release at: http://hcencoder.bitburners.com/

[Amnon82]

It is a great work till now. When I find time I'll play a little with Your new version ;) Keep up the good work.

Mirror for HC016TR2_16-09-2005 (http://amnon82.cybton.com/files/HCbatch_016_testrelease2_16-09-2005.zip)

:thanks:

P.S.

If You need help with the prediction mode, tell me ;)

Take a look at AutoCQ (http://forum.doom9.org/showthread.php?t=89122)

Some news about HC016.

I wanted to release it also with an updated GUI version but the old GUI isn't flexible enough,
it's too big, just can't fit the new buttons in it anymore (and it's ugly also :))
So there will be a completely new GUI but that will take some time to finish...

But thanks to all comments/feedback the HC016batch version is now final:

http://hank315.dyndns.org/HCbatch_016.zip

- fixes the playback on some strict players (Pioneer)
- pulldown issue is fixed, streams can be pulldowned for PAL and NTSC GOP lengths
- updated 1 pass CQ_MAXBITRATE run, should result in better prediction
- autoq2 matrix replaced with avamat6 and avamat7 (both by Amnon82)

Some of this is already in the latest test releases but not all.
For those who use DBD-RB with HC, just replace and rename the exe file with this one.

[ljpp]

Another mirror (http://www.bitburners.com/remository/Download/Video_Codecs_and_Encoders/HC_Encoder_(HCEnc)_v0.16TR2/)

[Mr. Monte]

I take it jdobbs will put this into his RC6b downloads as of now, or will he wait for RC7 or 1.1?

[Boulder]

I've had no problems with the new test release:)

One small feature request for 016 though: would it be possible to set the path for the database file or possibly create it in the same directory as the encoded video? At the moment I've got all my video tools on a partition which doesn't have much free space left (only about 500MB), and sometimes the dbs file gets so large that the encoding doesn't proceed to the end of a long video.

[TheCreamCrackerBoy]

Is there a GUI version, like in HC 0.15?

Thank you,

TheCreamCrackerBoy

[Boulder]

I'm sure a GUI version will be available as soon as the official 016 is released. Besides, the commandline version isn't hard to use.

[Marius-the-Mad]

Hank, could you check whether HC writes correct information about colour matrix used ? I mentioned that problem to you and it's unclear whether the culprit lies within DGIndex' or HC's code. :confused:

<edit> Solved! Thank you very much for your assistance, Hank! :)

[hank315]

Is there a GUI version, like in HC 0.15?No, not yet.
But if this batch version runs pretty stable I will create a GUI version.
All development I do is always done on the batch version, it's faster for testing purposes, also it's easier to maintain because it needs less windows stuff.

[Prodater64]

@hank315: How the state of giving back to the stream the bitrate cutted by cq_maxbitrate option?

[rhayman]

Thanks for this release, hcbatch 15_a was crashing in the latest version of dvdrebuilder but this one works fine.

[feedback]

Thanks for this release, hcbatch 15_a was crashing in the latest version of dvdrebuilder but this one works fine.
I've been using HCbatch 15_A with DVD Rebuilder Pro. with no problem myself. But I always look forward to Hank315's latest and greatest version of his HC encoder.

Regards,:)

[Sounddude]

Is the GUI version ready yet?

[Sunhillow]

Is the GUI version ready yet?

If hank did not announce it, it isn't :rolleyes:

[DD51]

Hank,

Just FYI...

I've backed up a few Discs so far:
All of Season 1 of "LOST"
All of Season 2 and 3 of "24"
5 movies of "Movie only"

I have had absolutley "NO PROBLEMS" whatsoever yet
except that the final size are "ALL" 4.31Gigs!
Just letting you know my experience so far with test release 2.

Regards,

[writersblock29]

@DD51

You mean, using HC with DVD Rebuilder? 4.31 is Rebuilder's default target setting, so it's not HC's fault. If using Rebuilder isn't what you meant, sorry 'bout that! :)

[DD51]

@DD51

You mean, using HC with DVD Rebuilder? 4.31 is Rebuilder's default target setting, so it's not HC's fault. If using Rebuilder isn't what you meant, sorry 'bout that! :)

Correct...I'm using HC with Rebuilder. 4.31 is good for me...I thought it was 4.33 the default. Thanks for setting me straight.
I love HC!

Thanks Hank for all your hard work :thanks:

[writersblock29]

@DD51

Sure thing! Actually, Rebuilder's set for 4.31GB for safety. Certain projects will tend to run over that boundary + or - a few MBs. CCE will sometimes oversize a bit, too. That's probably where you've been seeing a few 4.33GB filesizes from. Jdobbs just wanted a safety net to keep people from having to totally re-encode because of oversizing. You can manually tweak this so that you're using more of the blank disk... but doing so only adds a very marginal amount of bitrate to your project and the difference isn't noticable just by watching it.

HC's becoming more impressive with each release. It's making me wish I'd have saved my money when I decided to buy CCE a few years back. Where where you then, Hank? ;)

[Pasqui]

Hi hank315,
as test release 2 seems to work fine, when do you expect to release final version ?
Regards,

[dragongodz]

when do you expect to release final version ?
it will be ready when its ready. hank315 has a lot going on in real life at the moment, seems we all do this time of year( :D ) plus there are all sorts of advanced things he is looking at for HC. since there is no way to be exact it is often better to not put a release date and then have people asking where is it.

[Boulder]

My last encode ended up with buffer underflow warnings in DVD Maestro.

Here's the last log entries:

intra matrix used
8 16 19 22 26 27 29 34
16 16 22 24 27 29 34 37
19 22 26 27 29 34 34 38
22 22 26 27 29 34 37 40
22 26 27 29 32 35 40 48
26 27 29 32 35 40 48 58
26 27 29 34 38 46 56 69
27 29 35 38 46 56 69 83

non-intra matrix used
16 17 18 19 20 21 22 23
17 18 19 20 21 22 23 24
18 19 20 21 22 23 24 25
19 20 21 22 23 24 26 27
20 21 22 23 25 26 27 28
21 22 23 24 26 27 28 30
22 23 24 26 27 28 30 31
23 24 25 27 28 30 31 33

nr. of gops: 10053
nr. of frames: 139500
nr. of I-frames: 10053
nr. of P-frames: 36609
nr. of B-frames: 92838
average quant (non linear): 4.612
VBV underflows detected: 6
VBV underflows fixed: 6
minimum bitrate: 193
maximum bitrate: 8492
average bitrate: 3364

I used a 256kbps audio track plus one subtitle track so the max bitrate definitely shouldn't have been exceeded.

[hank315]

@Boulder
Strange, the log values show an easy encode with just a few VBV fixes and a max. bitrate which isn't that high.
Have you also tried to author it with Muxman or IfoEdit?
How does it look when the m2v is loaded into BitrateViewer or MPEG Stream Eye?

[Boulder]

@Boulder
Strange, the log values show an easy encode with just a few VBV fixes and a max. bitrate which isn't that high.
Have you also tried to author it with Muxman or IfoEdit?
How does it look when the m2v is loaded into BitrateViewer or MPEG Stream Eye?
I'll try to author it with those programs as soon as time permits. BV showed max bitrate at ~8000kbps so the value in the log should be true.

This is the first time I've had such warnings and I've used 9000 as the maximum. Do you know any program that could tell me at what frame/GOP the underflows occur? I could check it on my picky Pioneer if it has a real effect.

[Mug Funky]

My last encode ended up with buffer underflow warnings in DVD Maestro.

were these green warnings or red warnings (well, a red warning would have stopped the compile dead...)?

green warnings from spruce are pretty safe to ignore - they don't translate to playback problems in my experience, though that fact is taken on faith because only the pedants like us will complain about occasionally stuttery playback... most people blame their equipment and forget about it (samsung owners are justified here...)

i suspect spruce gives a green bitrate warning when the bitrate exceeds the flagged max bitrate, and gives a red warning when the bitrate exceeds the 10.02 mbps limit of DVD. i've not tried this, but it's possible that if you were to change the max bitrate set in the m2v, the warning would go away.

also beware that BV free version reports too-low bitrates. you have to multiply them by 1.024 i believe.

[Boulder]

hank already found the problem with a little help from MuxMan's log file and a little util. It's not about exceeding the max bitrate, it's just a rare case of a couple of a bit too large frames not arriving on time. He said it'll be fixed in the next release.

I haven't tried whether my player will stutter at that point but it would have been a minor issue anyway.

[Moitah]

I am trying to frameserve from VirtualDub to AviSynth to encode with HC. The AVI loaded in VirtualDub is NTSC DV so it's 720x480 29.97fps. AviSynth script looks like this:

AVISource("enc.vdr")
ConvertToYV12(interlaced=true)

It works fine in HC 0.15A, but with the 0.16 test release it just exits as soon as I start it (no error message displayed, or if there is it exits so fast I can't see it). All settings were the same for both versions because I loaded them from the same INI. 0.16 works fine for normal AviSynth scripts where I'm not reading from a VirtualDub frameserver. Any ideas?

[hank315]

Some remarks here so the other thread will not be hijacked completely ;)
@Carpo & Pasqui
I think it's perfectly safe to use the latest HC016, IMHO it's the most stable version until now.
It's just called a test version to see if the crashes which some people experience with HC015 are now gone.
There will be an "official" release of it when it's finished, still working on 1 pass VBR.
Before the official release there will be another test version which can do 1 pass VBR.

Known bugs for HC016 (and all previous versions):
- GOP/VOBU/pulldown issue reported by Xeno86, will be solved in the next release
- VBV underflow reported by Boulder, working on that now

@Moitah
I never tested this but yes I can replicate it.
HC015 finishes normal, HC016 just disappears immediate, frameserver reports 1 frame served :confused:
Debugging time again...

[dragongodz]

still working on 1 pass VBR.
your as bad as Nic in keeping things for a surprise. ;)

[Carpo]

Known bugs for HC016 (and all previous versions):
- GOP/VOBU/pulldown issue reported by Xeno86, will be solved in the next release
- VBV underflow reported by Boulder, working on that now


@Hank315

thats why i never used it in the test (of course i didnt know of those issues) but
it was a case of just in case - and i never meant to knock your encoder as its a damn fine piece of kit to have - since my little test i been using HC15 non stop :D

[Boulder]

It appears that HC doesn't like ffdshow's mjpeg decoding.

I've got a TV capture, captured with ffdshow mjpeg. I open it via AVISource so the infamous DirectShowSource can't be the culprit. When the script is opened, HC quits almost immediately leaving a zero byte length m2v file behind.

EDIT: using DirectShowSource it works :confused:

[Sagittaire]

Possible to implement adaptative ratio for bframe ... ???

[LauriM]

Hank315, thank you for an excellent encoder!

I'm happy to report that the current testrelease works with Wine in Linux, hits the specified average and peak bitrates and does it with excellent image quality and speed. :cool: My Pioneer standalone (which is quite strict about standard compliance) has no problems with the resulting bitstream.

:thanks:

[hank315]

Possible to implement adaptative ratio for bframe ... ???Do you mean a manual setting for it?
Now the ratio for I,P,B is already done automatical depending on the bitrate.

@LauriM
Thanks...

[Sagittaire]

Do you mean a manual setting for it?
Now the ratio for I,P,B is already done automatical depending on the bitrate.
Thanks...

No ... as you know MPEG2 use no adaptative bframe ... always IBBPBBP for low or high motion. MPEG4 use adaptative bframe and generaly only Bframe in low motion with high ratio/offset because with exactly same size I(q3)b(q5)b(q5)p(q3) will be visually better than I(q4)b(q4)b(q4)p(q4). It's possible to use constant ratio/offset for MPEG2 but ratio/offset for high motion is bad in high motion. Use constant ratio/offset for MPEG4 is possible because bframe are adaptative but it's a problem with no adaptative bframe like for MPEG2 MP@ML DVD compliant stream.

Why not adaptative ratio for bframe and for example
I(q4)b(q4)b(q4)p(q4) in high motion part -> low ratio for high motion
I(q4)b(q6)b(q6)p(q4) in low motion part -> high ratio for low motion

You use low quant variability and for example
I(q4)b(q4)b(q4)p(q4) in high motion part -> low ratio for high motion
I(q3)b(q4)b(q4)p(q3) in low motion part -> high ratio for low motion

with that PSNR and visual quality could be very better ... ???

[FredThompson]

@hank315,

I've got some high-motion interlaced source which doesn't play well with any encoder but ProCoder. All other encoders yield blocky output. HC has the least blockiness. ProCoder shows no blocks. Would you be interested in some of this source? I could send you a minute or so of the original GOPs for testing along with Procoder and HC conversions.

My motivation? I want to switch to only using HC and provide you whatever support I can for your baby to be more capable.

[hank315]

Would you be interested in some of this source?Yes, just PM'ed you about it.

[hank315]

@Sagittaire
as you know MPEG2 use no adaptative bframe ...always IBBPBBP for low or high motionHC uses different GOP schemes for low/high motion (autogop option):
high motion: IBPBPBPB...
normal motion: IBBPBBPBB...
(very) low motion: IBBBPBBBPBBB...

In your blind test I noticed that for the HC encoding a fixed GOP was used.
A variable GOP scheme is DVD compliant, also Mencoder and CCE do it.
Mencoder uses very short GOPs on fades, CCE uses 2 or more consecutive P frames in low action scenes.

For HC the quant variability ATM is more based on bitrate than low/high motion, but maybe some tweaks with respect to motion could raise quality.

with that PSNR and visual quality could be very better ... ???Will test it to see if it does, if so it will be in the next release.

[dragongodz]

(very) low motion: IBBBPBBBPBBB...
this one is not dvd compliant though. maximum distance between reference frames is 3 for dvd ie. IBBPBBP
from http://www.mpeg2.de/doc/dvd/book_b/video.htm
MPEG-2 MP@ML (Main Profile @ Main Level) has no restriction to the number of consequitvely coded B frames. In DVD, it is limited to no more than two B frames (M<=3, the reference frame distance).

FredThompson - if you are making that sample available for download by hank315 then can you let me know where to get it aswell ? as i would like to test and see what it does with QuEnc.

[hank315]

@dragongodz
You're right, it's allowed for MPEG, not for DVD.
Will be fixed in the final HC016.

[Sagittaire]

adaptative Bframe number is compliant with DVD MP@ML ... ???

IPPPBPBBPBBPPP for example ... ???

In this case quality with Mencoder could be very better ... lol

FredThompson - if you are making that sample available for download by hank315 then can you let me know where to get it aswell ? as i would like to test and see what it does with QuEnc.

me too with mencoder ... ;)

[FredThompson]

FredThompson - if you are making that sample available for download by hank315 then can you let me know where to get it aswell ? as i would like to test and see what it does with QuEnc.Yes. I just sent a message to hank315 that I can't get to it until Friday due to a packed business schedule. I'll make it openly available. It's probably a good stress test for encoders.

[hank315]

@Sagittaire
adaptative Bframe number is compliant with DVD MP@ML ... ???
IPPPBPBBPBBPPP for example ... ???I think everybody agrees that CCE generates DVD compliant streams.
3 consecutive GOPs from the CCE encoding (from your blind test) look like:
IBPBPBPBPBPBPB-IBBPPBPBPBBPBB-IBBPBBPBBPPPBB

That doesn't look like the (default) IBBPBBPBBPBB... scheme you probably set in CCE :)

[hank315]

Some news about HC016.

I wanted to release it also with an updated GUI version but the old GUI isn't flexible enough,
it's too big, just can't fit the new buttons in it anymore (and it's ugly also :))
So there will be a completely new GUI but that will take some time to finish...

But thanks to all comments/feedback the HC016batch version is now final:

http://hank315.dyndns.org/HCbatch_016.zip

- fixes the playback on some strict players (Pioneer)
- pulldown issue is fixed, streams can be pulldowned for PAL and NTSC GOP lengths
- updated 1 pass CQ_MAXBITRATE run, should result in better prediction
- autoq2 matrix replaced with avamat6 and avamat7 (both by Amnon82)

Some of this is already in the latest test releases but not all.
For those who use DBD-RB with HC, just replace and rename the exe file with this one.

[bourtzovlakas]

:thanks:

[Lo_RaT_PeNaT]

many thank you for your work

[DK]

good news!

[Oldeman]

Great..... :o :o :o
I love HC... Thanks thanks thanks.... Hank315 :thanks: .

[kumi]

Thank you so much hank315 :)

[hartford]

Thankyou hank315.

I've used this on low bitrate MPEG2's and it is much better than CCE 2.50.

Can't say more than thankyou!

[buzzqw]

a big thanks isn't enought !

BHH