Experiences with HC Encoder

[DD51]

Originally posted by dragongodz
sorry but i find that amussing after you have said.
I'm glad I gave some people a chuckle. Laughing is very important for the state of mind.

quote from me:P4 3.0C GHz
OC'd to 3.554GHZ 948 FSB
Yes I love to overclock! It makes a big difference.

Originally posted by dragongodz
and you dont think you are stressing your cpu there ?
That's exactly the reason why it worries me. I should've mentioned that I love speed. I'm just trying to maximize the speed of my computer. This is the only program I use that stresses my CPU aside from gaming. My comp. was built with gaming in mind.

Originally posted by dragongodz
also did you turn off HT to compare the speed as thomasphoenix suggested ?

No I have not...I'd rather wait for the batch fix version and then test it out. To be honest I don't think I might. I don't like the fact that I would have to change my settings in the bios everytime:eek:

Regards,

[quantum]

Originally posted by jdobbs
Hey! quantum! Dude. Glad to see you're still alive and kickin' I'm alive. Don't know about kickin. Work.. work.. sleep.. back to work.. repeat. :-)

Originally posted by jdobbs
I'm trying to avoid adding more-and-more-and-more options that confuse people.
I agree, but you're too clever to let that happen. You already have things organized to prevent it with the advanced settings screen. Even better, if the option is truly obscure, put it in the rebuilder.ini which will totally hide it from the one clickers and still be accessible to the experts.

Then someone can write an add-on program to make a GUI for your INI file. Or has that been done alreay? :-)

[DK]

of course it is a good idea to keep rb as simple as possible in order not to confuse not too advanced users

on the other hand quite a few of those people who prefer encoder ala cce, procoder, quenc, hc_enc etc over transcoder might like to be able to *tweak* the settings every now and then

i for one would love to find a gop length option in rb some time ;)

[dragongodz]

No I have not...I'd rather wait for the batch fix version and then test it out. To be honest I don't think I might.
then HC couldnt even use 100% of your cpu if it wanted to. if the cpu usage is going up to 100% then its other things running using the other pseudo cpu(HT cpu cycles).

from what i understand CCE is HT aware so is using every bit of cpu it can so of course its way faster.

I don't like the fact that I would have to change my settings in the bios everytime
no i just meant change it the once to see the speed difference. i am sure hank315 will try to implament threading at some point in the future. its just not a high priority in the early stages of most programs.

[hank315]

About GOP length...

Unfortunately most users of DVD-RB don't know (or care) what a GOP is.I agree with jdobbs, and why should they, the only thing most people want is a good DVD backup.

But for those who are really interested:
A GOP is Group Of Pictures which starts with an I frame.
How long can a GOP be?
Simple, you can do a whole movie within one GOP.
Ofcourse there a lot of other limitations, chapters, scene changes, DVD-playback/skipping etc.
But there's also an ugly thing in MPEG called IDCT mismatch.
This may cause an error propagation between the encoder and the decoder.
That's one of the reasons the MPEG guys decided to limit the GOP length to 15/18 for PAL/NTSC. (frame based)
In fact this is a *very short* GOP length but a standard is a standard...
Doing a DVD backup, which means compressing the video, would benefit from a longer GOP but an encoder should *never* insert a GOP which is not DVD compliant.
So choosing a GOP of 12-2 is a good choice if the encoder sometimes "overshoots" the prescribed GOP length.

For HC this doesn't apply, the autogop or gop 15-2 settings will never insert a GOP longer than 15.
HC will produce best quality with the autogop option and it will always be DVD compliant for PAL and NTSC.

[DD51]

Originally posted by dragongodz

no i just meant change it the once to see the speed difference. i am sure hank315 will try to implament threading at some point in the future. its just not a high priority in the early stages of most programs.

I will try it and let you know the difference in speed. Thanks. BTW my cpu during encoding with HC never goes past 50%. I'm sure with HT turned off it should make a good difference in speed.
Regarding HT, I'm sure it wouldn't be a priority right now for Hank315. I totally understand.
Thanks for all your help.

Kind regards,

[DMagic1]

My experience with HC compared to CCE using RB are ok but could be better.

I've done two projects and get the purple line on both.
My first project was the very long Green Mile. Average bitrate was only about 2900.
CCE had very visible blocks in several places. HC also had blocks but they were blurred and therefore seem less visible.
The overall picture from HC seems softer and less sharp compared to CCE. Its hard to tell with I prefer on this project. The softer picture seems to take away from the whole movie.

The other project wasn't a complete movie. It was only a couple of cells from Taxi. I don't remember the bitrate but it was only about 92% compression. The softer picture was there again. One shot that stands out is how there where two razer bumps on a guys face. The CCE output allowed you to see them. The HC output blurred them out.

I'm planning a project tonight with average bitrate of 3800. CCE is going right now, HC will be later.

[dragongodz]

BTW my cpu during encoding with HC never goes past 50%.
which makes your fears that HC taking longer is stressing your cpu more incorrect then doesnt it ? ;)

I've done two projects and get the purple line on both.
i am sure you have already read the HC thread about this being fixed. so should not be an issue when hank315 releases a new version.

The overall picture from HC seems softer and less sharp compared to CCE.
this may be improved later. its probably to do with something hank315 and i have already discussed. cant say anymore, probably shouldnt have said that even. :D

[DMagic1]

Thats good to hear.

[Vanderlow]

Originally posted by dragongodz



this may be improved later. its probably to do with something hank315 and i have already discussed. cant say anymore, probably shouldnt have said that even. :D

Why? this is a free encoder, no?

[dragongodz]

Why? this is a free encoder, no?
and thats got what to do with it ?
we are talking about the way the encoder does and doesnt do things internally. most people wouldnt be interested to hear the fine details and/or discussions on how this should be changed. or possibly worse a heap of people could all have an opinion.
free does not mean you have some sort of right to know everything thats going on under the hood. :sly:

[Vanderlow]

Good answer!

[Pasqui]

hank315 released a new version:
Just put the new batch version online, new manual included.

get it at: http://hank315.dyndns.org/HCbatch_012.zip

Changelog:
- encoding engine same as GUI version
- runs in a small window, showing statistics
- can use ini file and parameters

It seems that the command line parameters were changed: DVD-RB 0.79 doesn't work. :(
I hope jdobbs will release a new version soon ! :D

[hank315]

It seems that the command line parameters were changed: DVD-RB 0.79 doesn't work.It's easy to make it work.
Set DVD-RB to Quenc mode and point it to the HCbatch exe.
Create a HC.ini file in the same directory as the HCbatch exe:

*profile best
*matrix qlb
*logfile <full pathname> --> not really necessary

That should work (for the time being), tested it last night...

[happycase]

I'm just curious, are there any plans to make HC Encoder support for DVD-RB available to the general public, or is it going to remain the kind of thing that only a donation will get you?

[Ebobtron]

Originally posted by happycase
I'm just curious, are there any plans to make HC Encoder support for DVD-RB available to the general public, or is it going to remain the kind of thing that only a donation will get you?

That was just done; I am very confused by your remark. Download HCBatch and then follow the instructions above your post. If you cannot make it work, ask for help. I believe that Hank's latest was written just for programs like RB.

[Vanderlow]

I just finish a movie (Seabiscuit) and I use 78 general release and created the HC.ini file as Hank suggest and repointed it using the QuEnc mode. HC has it own folder. Well it prepared it and it started encoding it with the current batch HC.exe, but in the middle of encoding the movie on segment 17 (out of 31) it gave me an error saying v**1017 shows completed but no .M2V file exists. I then clicked OK and it went on to encode the rest of the movie (5 hours in total). I then Rebuild it and at the end it gave another error saying "error in UpdateIFO:Shouldn't happen". The movie was created of sorts, 4.28GB, but was jerky and the progress bar on PowerDVD will not go to the end of the movie, so it's kind of screwed up. The HC executeable did seem to work though. What do you think I went wrong? TIA

My HC.ini is:

*matrix qlb
*profile best
*logfile C:\Program Files\DVD-RB\HC\test.log

[johnnyquid]

The hc.ini file must also include:

*maxbitrate 9000

RB does not specify the max bitrate when -novbr is specified.

[Vanderlow]

So add that line and that should do it then uh?

so my HC.ini looks like this now:

*maxbitrate 9000
*matrix qlb
*profile best
*logfile C:\Program Files\DVD-RB\HC\test.log

I want to make sure I get it right so I don't spend another 5 hours! :)


ADD ON to my post. Here's a picture of the HC executable on first try with the error and I think it does have max bitrate of 9000 (even with it not in the ini file).

http://img.photobucket.com/albums/v487/Vanderlow/hc1.jpg

Paste of the log after the errored 1017 segment, since it doesn't show up in the log:

==================================================
==================================================

--------------------------------------------
| HCbatch - MPEG2 encoder - rel. 0.12 beta |
--------------------------------------------

input: c:\rebuilder work folder\d2vavs\v01001700001018.avs
output: c:\rebuilder work folder\d2vavs\v01001700001018.m2v

--------------------
| encoder settings |
--------------------

profile: BEST
frames: 1 9029
framerate: 23.98
aspect ratio: 16:9
bitrate Kb/s: 4435
max. bitrate Kb/s: 9000
closed gops: no
VBV check: yes
scene change det.: yes
interlaced: no
goplen,B-pic: AUTO
dc_precision: 9
scan method: ZIGZAG
time code: 0 0 0 0
CPU: MMX/SSE
matrix: QLB

--------------------
| source stats |
--------------------

nr. of frames in source: 9029
width*height: 720*480
fps: 23.98
nr. of frames to encode: 9029
frames to encode: 1 - 9029

---------------------
| encoding - pass 1 |
---------------------

pass 1 encoding time: 0:14:10 (850 s)
average fps: 10.6

--------------------------------
| encoding - intermediate pass |
--------------------------------

bitrate set to: 4435000 b/s
est. outfile length: 203876 kB
intermediate encoding time: 0.0 s

---------------------
| encoding - pass 2 |
---------------------

pass 2 encoding time: 0:04:16 (256 s)
average fps: 35.3

------------------
| encoding stats |
------------------

total encoding time: 0:18:26 (1106 s)

intra matrix used
8 16 19 22 26 27 29 34
16 16 22 24 27 29 34 37
19 22 26 27 29 34 37 39
22 22 26 27 29 34 38 42
22 26 27 29 32 36 40 50
26 27 29 32 36 40 50 61
26 27 29 35 40 50 59 73
27 29 35 40 50 59 73 87

non-intra matrix used
16 17 18 19 20 21 22 23
17 18 19 20 21 22 23 25
18 19 20 21 22 23 24 26
19 20 21 22 23 24 26 28
20 21 22 23 25 26 28 29
21 22 23 24 26 28 29 31
22 23 24 26 28 29 31 32
23 24 25 28 29 31 32 34

nr. of gops: 678
nr. of frames: 9029
nr. of I-frames: 678
nr. of P-frames: 3177
nr. of B-frames: 5174
average quant (non linear): 4.522
VBV underflows detected: 0
VBV underflows fixed: 0
minimum bitrate: 920
maximum bitrate: 8979
average bitrate: 4434

[hank315]

@Vanderlow
Looking at your pic. it shows the right arguments were passed to the encoder, look at the upper left window.
The stats in the logfile also look good, max bitrate was 8979 (9000 set) average bitrate was 4434 (4435 set)
About the missing m2v file, seems DVD-RB never started the encode because there's no logging at all.
Try to create that missing segment with the GUI version.
I'm just curious, are there any plans to make HC Encoder support for DVD-RB available to the general public, or is it going to remain the kind of thing that only a donation will get you?There's support for HC in the latest Pro versions but if it becomes available to the general public is jdobbs decision.

[Vanderlow]

Where do I find the GUI version?

[hank315]

It's in the download section of Doom9, you can also use the batch version to create the missing part but with the GUI its easier, just set the input to the avs file in the d2vavs directory.

[quantum]

I got the same error "shows completed but no .M2V file exists". It was a 1 frame encode:

#------------------
# AVS File Created by DVD Rebuilder
# VOBID:05, CELLID:07
#------------------
mpeg2source("M:\MOVIE_OUT\WAW1_HC\D2VAVS\V01.D2V")
trim(94645,94645)
ConvertToYV12(interlaced=true)

[johnnyquid]

See my post above. The hc.ini file must also include:

*maxbitrate 9000

When there are not many frames, I believe RB tells Quenc to do a 1 pass encode CBR encode. When this happens, no -maxbitrate command is in the command line sent to HC. HC ignores the request to do a 1 pass CBR encode (it still wants to do a 2 pass VBR) but with the no maxbitrate specified in the hc.ini file or the command line, HC errors out thus creating the no .M2V message.

[hank315]

Will run a test with it later, also got that error on a 1 frame segment.
If DVD-RB not sets the maxbitrate HC will fail, it needs bitrate and maxbitrate set.
BTW. does HC starts encoding the 1 frame segment when you add the *maxbitrate 9000 command?

[johnnyquid]

I have had 1 frame cells encode successfully after adding "*maxbitrate 9000" in hc.ini file.

[happycase]

Ok, thanks Ebotron and hank315. I must have misread DVD-RBs changelog.

[jdobbs]

Hank315 was working on adding the QuEnc compatible command lines, so I knew it would work with the freeware version of DVD-RB when he finished. The reason I've created a special HC option in the Pro version is so it can take advantage of DVD-RB's Pro features like custom matrices for different bitrates, GOP settings, and DC Precision. In future versions DVD-RB will do more intelligent selection of settings for you.

[jdobbs]

Originally posted by hank315
Will run a test with it later, also got that error on a 1 frame segment.
If DVD-RB not sets the maxbitrate HC will fail, it needs bitrate and maxbitrate set.
BTW. does HC starts encoding the 1 frame segment when you add the *maxbitrate 9000 command? If you are going through the QuEnc interface, DVD-RB sets the maximum bitrate (via the command line) on every encode.

[jdobbs]

Whoops I have to correct myself. When there is a single frame, DVD-RB sets QuEnc to CBR, so there would be no "Maximum" bitrate. I'll add it to the command line in the next version (assuming it doesn't affect QuEnc). You really don't want to add it directly to the HC.INI file because the maximum bitrate changes depending upon the number of audio streams in use -- and when it makes it to the DVD you might get stutter from overshooting the DVD maximum. If you do set it in the meantime I'd recommend something lower than 9000 -- maybe 8000.

[onesoul]

jdobbs, I am guessing that you haven't seen this, but when encoding an interlaced source, the alternate scan setting doesn't get checked and bff is selected for some strange reason, which is weird because by default HC would automatically select alternate scan and tff flag when interlaced setting is provided at hc.ini file.

About hc support:
You mean hc.exe is to be specified at setup on quenc target with the next versions of dvd-rb?

edit: Another question: Where does this maximum bitrate is specified at rebuilder.ecl, is it this "vbr_brate_max=7464", if so i only get max bitrate of 9000 at hc and no other value.

[jdobbs]

If you are using HC.EXE with the Pro version, the parameters are all set in the INI file. I never set alternate encoding because the HC document says that it is automatically selected when you set *INTERLACED.

HC support will continue to be inherent in Pro versions of DVD-RB and you wouldn't need to use QUENC settings.

You can use HCBATCH_012.EXE with the freeware version by specifying pointing to it in the QuEnc path.

Good catch on the Maxbitrate. I just went back and looked and I found a bug that was causing "*MAXBITRATE" to always be 9000Kbs in both HC and QUENC... I'll fix that for v0.80.

[Vanderlow]

Yeah I just started over and changed my HC.ini file to read 8000 and when I manual call up the HC exe it reads 8000 for maxbitrate, but when called up thru Rebuilder it reads 9000. Guess it will be fixed.

[hank315]

If you are using HC.EXE with the Pro version, the parameters are all set in the INI file. I never set alternate encoding because the HC document says that it is automatically selected when you set *INTERLACED.And that should work :)
But just discovered if the interlaced is set using arguments the alt scan is not set, will correct that.

But still have my doubts, if the source is progressive and encoding is set to interlaced IMHO the scanning flag should be zigzag :confused:

[onesoul]

Originally posted by hank315
And that should work :)
But just discovered if the interlaced is set using arguments the alt scan is not set, will correct that.That's what I was trying to point out to jdobbs :)
But still have my doubts, if the source is progressive and encoding is set to interlaced IMHO the scanning flag should be zigzag :confused: Like it was said on other thread it is not going to do major difference doing that but if you disable interlaced with dvd-rb I believe it will encode as progressive and zigzag.

Also I'd like to point out that hc 0.012 batch undersized comparing to hc 0.01 batch, I'll post more details on hc thread.

[onesoul]

Originally posted by jdobbs
HC support will continue to be inherent in Pro versions of DVD-RB and you wouldn't need to use QUENC settings. I'm not sure you have seen a post before by someone else but just in case I point out again that dvd-rb 0.79 pro isn't working with latest hc .012 batch. Hc gives an error of some argument not specified. That's why I used the quenc path to try new hc, which wouldn't have the ini file you mentioned.

Sorry for the headaches :(

[quantum]

Originally posted by onesoul
I'm not sure you have seen a post before by someone else but just in case I point out again that dvd-rb 0.79 pro isn't working with latest hc .012 batch. Hc gives an error of some argument not I noticed this too. I'm glad you pointed this out since I was getting confused.

[jdobbs]

It doesn't work because a number of the HC parameters have changed:

1. Bitrates are specified in Kbs now... the old batch used full numbers.

2. The matrix inputs have changed.

3. The aspect ratio settings have changed (although it may take both).

4. The INI file used to read from the "current" directory... now the location has to be specified with the -ini argument.

There's more... but in short, you'll have to wait until v0.80 comes out in the next few days to use the new HCBATCH_012.EXE for encoding. The alternative, of course, is to use it with the QuEnc path as was already discussed.

HC is new software so you have to expect some changes. But it's good so I'll keep up.

[dragongodz]

a number of the HC parameters have changed:
yep and probably wont be the last time. :)
people have to remember the changes that QuEnc has gone through with its CLI commands aswell.
these are evolving programs in early stages so are prone to some change in things like commands etc. it will settle down eventually. :)

[jdobbs]

Originally posted by hank315
But still have my doubts, if the source is progressive and encoding is set to interlaced IMHO the scanning flag should be zigzag :confused: DVD-RB never sets a progressive source to interlaced. It does have the ability though to send sources that were previously flagged as interlaced to be encoded as progressive. That's to cover some reported PAL sources that are said to be improperly flagged.

[samuelal]

jdobbs, My sincere gratitude for your work on the entire DVD-RB project!

I was very happy to see on this particular thread that the latest HC batch version will be supported with the next v0.80 release and even more excited to see that you've embarked on a massive-scale bug-hunt in preparation for the big v1.00 release!

Thank you!!! :)

If this was a wrong place to post this 'thank you', my apologies, it only seemed right.

[jdobbs]

:cool:

[Vanderlow]

So I was able to encode with the latest batch HC and the freeware version of dvd rebuilder (.78). The movie looked good. The time was 5 1/2 hours. I had nothing to compare it to except the original, but they seem close. The image came to 3.28GB (on two encodes).

[jdobbs]

@vanderlow

I think something went wrong. In every encode I've done with HC I have gotten 4.32GB.

[Vanderlow]

Originally posted by jdobbs
@vanderlow

I think something went wrong. In every encode I've done with HC I have gotten 4.32GB.

But is that with the non-pro version setup per Hank's directions?

[dragongodz]

just so its clear, HC can undersize and it has been discussed in the HC thread. an easy example is to use a target bitrate of 2000 and a max bitrate of 3000 for a clip. HC will not(atleast in the tests we have done) hit the 2000 but be under it. the amount under may be small or may be larger depending on the clip. increase the max bitrate to 8000(for example) however and the target rate is hit or extremely close ,including the clips that were well undersized before.

the rate control is on the list of things hank315 has said he will be looking in to improving. just takes time. :)

[jdobbs]

Originally posted by Vanderlow
But is that with the non-pro version setup per Hank's directions? No, that's using the Pro version.

[TheSeeker]

Just so Im clear. All I have to do to make HC work is put the HC.exe (the batch enabled one that is 560kb) into the same folder as rebuilder.exe right? Because every time I try to use it I get a Run time error 5 invalid procedure call or argument, every time I hit Encode after the prepare phase.

[DMagic1]

You need the hc.ini file also.

[jdobbs]

@TheSeeker

If you're talking about the Pro version, yes. In the Pro version you don't need the HC.INI file -- DVD-RB creates it in the working directory.