HC encoder, release 016 [Archive] - Page 2

zsb

17th November 2005, 20:13

I must add my thank yous here also. I had a difficult time with certain anime sources. CCE produced a whole GOP of huge blocks in a particular vertical pan scene and lesser blocks in many other pans. HC does not seem to have any problems with these scenes :) Settings for bitrate min/ave/max is 500/2060/2450. Other settings were more or less default. In conclusion (pointed by some others already), for lower bitrates HC beats CCE easily.

EDIT:
I have another weird pan scene where 15A gives clearly better results than 16 (still major blocking, but better overall). If anyone's interested I'll put the m2v's etc up somewhere (around 1,8MB each).

doxville

17th November 2005, 23:35

It appears that HC doesn't like ffdshow's mjpeg decoding.

I've got a TV capture, captured with ffdshow mjpeg. I open it via AVISource so the infamous DirectShowSource can't be the culprit. When the script is opened, HC quits almost immediately leaving a zero byte length m2v file behind.

EDIT: using DirectShowSource it works :confused:
I just wanted to ask, if there is some kind of bugfix planned to the above problem, because I found out that I have the same crash with captures in ffdshows MJPEG. Using DirectshowSource instead of AviSource helps to encode anyway, but funny enough, with HCbatch_015 it is possible to open the capture-file via AviSource. It would be nice, if there was a solution to this (maybe in the next release... ;))...

Once more a BIG THANX for this great encoder, it is really my favourite! Keep up the good work!

Revgen

18th November 2005, 23:26

@Hank315

Thanks for the new version.

However, there is one problem that I ecountered.

I encoded my file at 2300KBPS with AVAMAT7 and it turned out looking great. But it also turned out slightly undersized. So I ran it through Bitrateviewer and it reported that the file was 2153KBPS. Thats about a 7% decrease in quality.

Could this be the fault of AVAMAT7? If so, what would be a better matrix to use.

Here are my settings:

HC16 -i mysource -o myvideo -b 2300 maxbitrate 9000 -profile best -aspectratio 4:3 -scene -nointerlace -matrix avamat7

The source is about 600-700 MB's, so I could upload it to www.yousendit.com and e-mail the link to you. PM your address if your interested.

Thanks.

EDIT
My Source is also B&W. Could the lack of chroma affect how HCEnc calculates bitrate?

pacohaas

19th November 2005, 00:53

Some news about HC016.
- updated 1 pass CQ_MAXBITRATE run, should result in better prediction

I've been testing HC, specifically interested in using the CQ mode to create constant quality SVCD encodes. The obvious shortcoming here is that with a max bitrate, CQ mode will tend to scale down the more difficult frames, while not scaling the easier low motion frames. I'd be down with a 2-pass CQ mode where Q stood for quality rather than Quantizer and all the quantizers appropriately scaled based on first pass info to achieve constant quality.

If you're up for it, I'm up for test encoding and reporting.

colinb

19th November 2005, 15:01

I have just tried the final HC016 and one of my encodings which is about 5 mins playing time in length has 9 VBV underflows in the log file.

What is a VBV underflow and should I be concerned about it?

I'm using AVAMAT6 matrix, max bitrate of 8000 and average bitrate of 4700.

Since posting the above, I've also got another approx 5 min encoding with the same settings except average bitrate is set to 4900 and there are 27 VBV underflows.

pacohaas

19th November 2005, 20:52

What is a VBV underflow and should I be concerned about it?
I was kinda wondering the same thing myself, here (http://www.bretl.com/mpeghtml/VBV.HTM)'s what a 20 second google found me. Mine always says zero, so i assume this has something to do with higher DVD bitrates since i'm only doing SVCD's.

Fishman0919

20th November 2005, 00:26

WOW Hank.... nice job :cool:

dragongodz

20th November 2005, 00:29

ok let me see if i can explain it really simply.

imagine a bucket with a hole in the bottom, this is the buffer. now imagine you have a lot of different size glasses filled with water, these are the video frames. now as you empty 1 glass at a time in to the bucket so much runs out the hole. the bucket wil be filled to different amount at different times because the glasses of water are different sizes. now if too many really big glasses are poured in before enough water drains from the hole water will run over the sides and you lose that water.this is a buffer underflow. its when the outflow of the buffer is under the amount needed to handle the amount of data being fed to it. a buffer overflow is of course the opposite. that is when the bucket gets empty and is waiting for a glass of water to be poured in.

what does a decoder do in these situations ? it can wait until you pour that glass if you dont want lose any or discard it as too big etc for an underflow. thats up to how the decoder was done. either way you lose totally smooth playback.

i hope i made that simple enough. :D

the only time you need to worry with HC is if it doesnt or can not fix an underflow. so if you see it say x amount of underflows fixed then you dont really need to worry it.

i assume this has something to do with higher DVD bitrates since i'm only doing SVCD's.
no its applicable for SVCD aswell.

rpboy

20th November 2005, 19:07

I encoded my file at 2300KBPS with AVAMAT7 and it turned out looking great. But it also turned out slightly undersized. So I ran it through Bitrateviewer and it reported that the file was 2153KBPS. Thats about a 7% decrease in quality.

Could this be the fault of AVAMAT7? If so, what would be a better matrix to use.

EDIT
My Source is also B&W. Could the lack of chroma affect how HCEnc calculates bitrate?
I also just a considerably undersized result as well.

I had processed a DVD with DVD-Rebuilder using the new 0.16 release. It came out perfect at 4.37 GB. But I wasn't happy with the quality of the output (It was like 5 hours of video, so I wasn't expecting miracles.) so I decided to try again, removing some of the video. So I deleted the contents of the working folder, and then changing nothing, I ran the Prepare phase again. I then used Preview/Edit feature to blank out some of the video. It ended up blanking out about 950MB. I saved my changes and let it go through overnight.

This morning I took a look at the output and it was 4.16 GB.

I also was using the AVAMAT7 matrix (well, I was using the AUTO-Q2 matrix that Matrix Editor had placed in my Matrix folder). My source was color, though, not black and white. I was also using HC running in Best mode.

Compared to CCE, I find that HC undersizes a little if I run the exact same DVD through DVD-Rebuilder, but I've never seen such a difference like the one I saw with this DVD.

FredThompson

20th November 2005, 19:48

You varied 2 parts of the same process.

Try it again using PGCEdit and VobBlanker to create a stable source then run again so the only variable is the new HC.

hank315 didn't write DVD-FB and jdobbs didn't write HC.

rpboy

20th November 2005, 22:29

You varied 2 parts of the same process.

Try it again using PGCEdit and VobBlanker to create a stable source then run again so the only variable is the new HC.
I didn't vary 2 parts of the same process.

First time was the entire disc, with HC 0.16. Second time was with several chapters blanked out by DVD-RB, with HC 0.16.

Everything was the same other than some of the video had been blanked out by DVD-RB.

hank315 didn't write DVD-FB and jdobbs didn't write HC.
So instead, you'd like me to throw in two other programs to the mix, that neither hank315 or jdobbs wrote?

hank315

21st November 2005, 00:07

@Revgen
Yes, I'm interested, will PM you.

@pacohaas
About constant quality, that's a hard one, in fact it doesn't really exist in MPEG encoding.
If an encoder claims it can encode with constant quality I've always wondered how they do that because quality isn't defined and hard to measure.

taken from the ISO MPEG document:
For constant bitrate (CBR) coding, the number of transmitted bits per unit time is constant on the channel.
Since the encoder output rate generally varies depending on the picture content,
it shall regulate the rate constant by buffering etc.
In CBR, picture quality may vary depending on its content.
The other mode is the variable bitrate (VBR) coding, in which case the number of transmitted bits per unit
time may vary on the channel under some constriction.
VBR is meant to provide constant quality coding. So I think a normal 2 pass (VBR) comes as close as possible to achieve constant quality.
Of course Constant Quantization also is VBR but is has some restrictions, it can raise bitrate very high so it must be limited to
the maximum bitrate but at low bitrate the Constant Quant may be too low.
Maybe in future versions there will be a setting for minimum bitrate, now that's set automatically to prevent bad results, if it's set too low results are pretty horrible...

@colinb
Dragongodz story explains the VBV stuff very well :)
With average bitrates 4700-4900 I think it's better to use another matrix like MPEG or QLB.

@rpboy
Just did a test with the avamat7 matrix using different bitrates (2000, 4000 and 6000) on a 3000 frame clip,
they all hit the target pretty accurate: 2000,4000 and 5982.
When I use DVD-RB I'm running a special version of HC which logs the bitrates for all segments to check the achieved bitrates,
in almost every case they match the values in the ecl file.
You could check the segments in the work folder using BRV or MPEG Stream Eye and compare the values set in the ecl file...

FredThompson

21st November 2005, 00:56

@rpboy, oops. I thought you'd used HC 15 for the whole disc then HC16 with different source. Sorry for the confusion.

dragongodz

21st November 2005, 02:12

With average bitrates 4700-4900 I think it's better to use another matrix like MPEG or QLB.
QLB was really designed for bitrates lower than that, unless its a VERY difficult source. MPEG martrix should be ok for that range IMHO.

rpboy - check the undersized encode to see what sort of quants you are getting. also try a different matrix and see if it also produces the same problem.

Revgen

21st November 2005, 03:24

@Revgen
Yes, I'm interested, will PM you.

I'm now uploading the source to yousendit's servers. It should take about 3-4 hrs. on my 512 kbps upload connection.

And yes they do handle files up to 1 GB. But they delete the file after it's been on the servers for 7 days and they set a certain limit on how many times the file can be downloaded. It's not the ideal solution for commercial use, but it's great for sending huge files for personal purposes such as what I'm doing.

EDIT

File has been uploaded. It took me 2 attempts to do it. :angry: The first attempt b0rked for some reason.

colinb

21st November 2005, 10:40

Thanks for the explanation.

now if too many really big glasses are poured in before enough water drains from the hole water will run over the sides and you lose that water.this is a buffer underflow. its when the outflow of the buffer is under the amount needed to handle the amount of data being fed to it. a buffer overflow is of course the opposite. that is when the bucket gets empty and is waiting for a glass of water to be poured in.

But isn't the terminology the wrong way round? My uderstanding of a "Buffer OVERflow" is that the buffer (or the bucket) is being fed with more data (from the glasses) than is being extracted (drains through the hole). E.g. one of the definitions I've found says "A buffer overflow will happen when you are trying to add too many bytes to a buffer, and underflow will happen when you are trying to read too many bytes from a buffer."

Also, if the encoder is deciding how much to compress each frame to keep to an average bitrate then surely avoiding a buffer over or underflow is a fundamental function of the encoder? If the buffer is emptying too quickly then surely the compression should be reduced and vice versa. So why is there a need to count under flows at all if the encoder's operation ensures that they should never happen?

(This if of course based on my poor knowledge of MPEG encoding).

dragongodz

21st November 2005, 12:46

the definitions I've found says "A buffer overflow will happen when you are trying to add too many bytes to a buffer, and underflow will happen when you are trying to read too many bytes from a buffer."
yes....i guess my explanation was too simple. :(

maybe this will help
from http://swpat.ffii.org/pikta/txt/ep/0742/674/
In the VBR mode, overflow can be avoided by only reading data from the
digital storage medium when the buffer can accept new data. In the VBR
mode, underflow is still possible and results from too much rather
than too little, data being used to represent video frames. When too
much data represents a frame, the video decoder cannot perform all the
necessary decoding operations in 1/30th of a second. An error results
because a frame needs to be displayed before it has been totally
decoded.
and from http://neuron2.net/library/mpeg1/MPGVIDEO.DOC
A fixed-rate channel is assumed to put bits at a constant rate into the Input Buffer. At regular intervals, set by the picture rate, the Picture Decoder instantaneously removes all the bits for the next picture from the Input Buffer. If there are too few bits in the Input Buffer, i.e. all the bits for the next picture have not been received, then the Input Buffer underflows and there is an underflow error. If, during the time between picture starts, the capacity of the Input Buffer is exceeded, then there is an overflow error.

why would the next picture not be entirely in the buffer ? because its too big and wont fit or can not be read and decoded at the required rate. thus the big glasses comparison. just think of underflows as caused by too big frames and overflows caused by too small frames unless you really want to learn all the ins and outs of both encoding and decoding, thats easiest.

Also, if the encoder is deciding how much to compress each frame to keep to an average bitrate then surely avoiding a buffer over or underflow is a fundamental function of the encoder? If the buffer is emptying too quickly then surely the compression should be reduced and vice versa. So why is there a need to count under flows at all if the encoder's operation ensures that they should never happen?
of course it is the encoders job to do these things. i have already suggested to hank315 that he should remove that information from the gui since people only need to know if the encoder fails to fix these and will just cause confusion with people wanting to know what it means. ;)

Amnon82

22nd November 2005, 00:36

Great release hank ;)

gilligan2

22nd November 2005, 01:41

I currently use DVD Rebuilder (the newest Pro version) I notice that everytime i encode it uses the mpeg standard matrix. How do i use and get the AVAMAT7 matrix or other matrices and use them or can't this be done with DVD-Rebuilder?

Thanks,

g:)

Amnon82

22nd November 2005, 01:57

It is simple:

-------------------------------
HC.ini template
These commands are required
Use full pathnames and just use the bitrate you want
-------------------------------
*infile c:\...\file.avs
*outfile c:\...\file.m2v
*bitrate 550
*maxbitrate 850
-------------------------------
Next commands are optional,
putting a space for it disables a command
Not all commands are shown here,
see the manual for a complete list and default values
-------------------------------
*logfile c:\...\file.log
*matrix AVAMAT7
*profile best
*aspect 4:3
*bias 30
*gop 12 2
*tff
*cq 4.0
*cq_maxbitrate 3.8
*dc_prec 10

AVAMAT7 is *matrix AVAMAT7, AVAMAT6 is *matrix AVAMAT6
The rest is like written in the HC015.Help.pdf. Only my matrices got changed.

@hank: I did a good 1CD encode with the INI I posted in this post. It has a good CSVCD potencial. I'll use it now for my new 1CD standard (a tool for it will come soon;)).

Maybe you can add a option to change the framerate
*framerate 25 (24,23.97,29.97,30) or
*fps 25 (24,23.97,29.97,30)

BTW: HCEnc is the fastest encoder in the resolution of 352x288 I've ever seen. AVG Frames on my AMD 64 3500+ machine are: 82 -102 (depends on the actual picture).

22nd November 2005, 02:00

if your question is where to load matrices in rb, then this might help:

http://img514.imageshack.us/img514/4521/rbmatrix9ng.png

if, however, you havent got avamat/autoq then take a look at RME/rebuilder's matrix editor

gilligan2

22nd November 2005, 02:04

Yes,this is more what i'm looking for but where do i get all the Matrices you have listed ? I only have the 5 original ones available to me.

And thanks to both of you for your help !!

TIA.

g:)

22nd November 2005, 02:28

like i said: download RME and consult the readme/help

rme will let you create/edit matrices for use within rebuilder

+++++++++++++

http://www.dvdrebuilder.com/modules.php?name=Downloads&d_op=viewdownload&cid=24

danpos

22nd November 2005, 02:37

@DK

OT: Nice skin !! :eek: I did like it. :) E-OT

Cheers,

22nd November 2005, 02:43

you DID?

why not anylonger?
^_~

and i must confess i still havent managed to update the skin in the setup department :-(

gilligan2

22nd November 2005, 03:12

rpboy

23rd November 2005, 00:33

rpboy - check the undersized encode to see what sort of quants you are getting. also try a different matrix and see if it also produces the same problem.
Well, I have no idea what the "quants" are so I can't provide any reasonable information there, but I did try an encode with a different matrix (I used QLB) and it undersized the same.

I am going to try an encode with CCE tonight and see if that undersizes.

dragongodz

23rd November 2005, 11:21

Well, I have no idea what the "quants" are so I can't provide any reasonable information there
you would load a vob in to something like bitrate viewer and see what the Q(for quant) values are.

alfixdvd

24th November 2005, 17:48

While HC016 is encoding sometimes I see at GOP:

1 - Closed

Is this good, bad or maybe a possible value ?

thanks in avance
alfixdvd

Boulder

24th November 2005, 18:03

The GOP consists of just one I-frame so it looks like the previous GOP ended just before a scene change which occurs right after the one-frame GOP. That means it's perfectly normal.

alfixdvd

24th November 2005, 18:38

Thanks

video_magic

1st December 2005, 12:14

Hello, HC batch v016 has been working fine up until this one time. Now it crashes without an error message, leaving just an 8kb dbs file.

The avs (avisynth script) that it gets stuck on does play fine in Media Player Classic v6487.
Here is the avs script:
Directshowsource("D:\atmp\Lemmi\lemmidvd\worldoftiers.wmv", fps=25, convertfps=true, audio=false)
Lanczos4Resize(352,288)
converttoyv12()

Here is the HC.INI:

video_magic

1st December 2005, 12:16

*ASPECT 4:3
*BITRATE 3800
*INTERLACED
*MAXBITRATE 7000
*CLOSEDGOPS
*INFILE D:\atmp\Lemmi\lemmidvd\worldoftiers.avs
*OUTFILE D:\atmp\Lemmi\lemmidvd\worldoftiers.m2v
*PROFILE BEST
*WAIT 30

Here is the dbs file attached

video_magic

1st December 2005, 15:56

Just got another crash on a different clip - this one a divx3.

Anyway, it crashes with an error message and when I close that error it gives a second error. It does this within seconds of starting the encode.

Here is the avs script:

AVISource("D:\atmp\Lemmi\lemmidvd\Solstice_At_Stonehenge_1984.avi", False, fourcc="DIV3")
crop(4, 0, -4, 0)
addborders(0, 2, 0, 16)
converttoyv12

Here is the HC.INI:

*ASPECT 4:3
*BITRATE 3800
*INTERLACED
*MAXBITRATE 7000
*CLOSEDGOPS
*INFILE D:\atmp\Lemmi\lemmidvd\solstice.avs
*OUTFILE D:\atmp\Lemmi\lemmidvd\solstice.m2v
*PROFILE BEST
*WAIT 30

Here is the error message:

video_magic

1st December 2005, 15:57

When I close that error, I then get this one pop up:

video_magic

1st December 2005, 16:49

I just installed the Divx3.11alpha codec and also restarted windows. That did not make any difference, HC still crashed with the same above message just as it had when I only had Divx6 installed. It also doesn't make any difference if I try using different fourcc="..." options. I can play in a player like MPC no problem at all with my avs script.

But if I use 'directshowsource' instead of avisource in my avs then HC works fine without crashing.

Anyway, I hope you are okay with my feedback - HCbatch 0.16 has worked just wonderfully on seveal other clips so far ;)

The ones I have had problems so far are wmv and Divx3 - I think I read that Divx3 was hacked WMV..... anyway

Prodater64

1st December 2005, 17:55

When I close that error, I then get this one pop up:

Could you upload your pics with imageshack.us or another similar site?

If you have a "memory could not be read error", you can search the forum.
A thread by me will give you the solution (if you have a non english OS, just tell me it).

Also you can try to close any path and filename between ' " '.

video_magic

2nd December 2005, 04:47

aye. (yes) that 'image2' of my previous post is the error pop-up that it gives - it has to be approved before it wil show as far as I can tell, I dont know anything else :(

video_magic

3rd December 2005, 14:02

Got another HC error - this time it is about half-way on pass 2 where it stalls at 1fps and reports underflows and fixing them.

This script works fine in Virtualdub and a media player...
Here follows my avs script, my HC.INI, and a screenshot attached.

c1 = ImageSource("D:\atmp\Lemmi\lemmidvd\4dvd\blackholecrop_ready2.bmp", end=987, fps=25)
c2 = ImageSource("D:\atmp\Lemmi\lemmidvd\4dvd\catacha_ready2.bmp", end=987, fps=25)
c3 = ImageSource("D:\atmp\Lemmi\lemmidvd\4dvd\mask__ready2.bmp", end=987, fps=25)
c4 = ImageSource("D:\atmp\Lemmi\lemmidvd\4dvd\angel_of_deception_ready2.bmp", end=987, fps=25)
c1+c2+c3+c4
ConvertToYV12()

*ASPECT 4:3
*BITRATE 3800
*INTERLACED
*MAXBITRATE 7000
*CLOSEDGOPS
*INFILE D:\atmp\Lemmi\lemmidvd\IMAGEschaintest.AVS
*OUTFILE D:\atmp\Lemmi\lemmidvd\menu4pics.m2v
*PROFILE BEST
*WAIT 30

Prodater64

3rd December 2005, 17:48

Got another HC error - this time it is about half-way on pass 2 where it stalls at 1fps and reports underflows and fixing them.

If the underflows are fixed, then not problem, not error!

video_magic

3rd December 2005, 21:11

No but it like gets stuck there.... I mean it slows right down to one frame a second and does hundreds of them!

But all my attachments are still 'pending approval' so you can't yet see the picture :/

video_magic

3rd December 2005, 21:13

p.s. the avs works fine in another encoder (Quenc) and it plays fine.

I think maybe HC has a few problems with avisynth I'm not sure (I understand I was using a beta of course :) )

neo squidward

3rd December 2005, 22:26

I also am having problems with HCbatch. My problem is HC just freezes when I use CQ_maxbitrate. Here is the avs:

avisource("c:\shows\nlcv.avi")
converttoyv12

klicker4546

4th December 2005, 00:13

Hello there, same problem for me here! I played around with Amnon82's GetCQ and experienced that the HC freezes all over. No matter what kind of source material I use. This occurs especially when I try to encode a SVCD with low bitrates. When I'm doing a prediction for a DVD there's no problem at all.

Think this definetly is not a problem of GetCQ. Any ideas?

I also am having problems with HCbatch. My problem is HC just freezes when I use CQ_maxbitrate. Here is the avs:

avisource("c:\shows\nlcv.avi")
converttoyv12

t_2

4th December 2005, 03:37

Ebobtron

4th December 2005, 06:56

@t_2

Look at page seven of HC’s user manual HC015.pdf. I do not think anything has changed for HC016.

A tip about command line strings; if your path statements include a space ( white space) enclose it with quotation marks.
Example:
C:\My Documents\myfile.avs like this “C:\My Documents\myfile.avs”.
Other wise HCBatch.exe will only see c:\my

Good luck,
Ebobtron
http://members.aol.com/avsfilmcutter

video_magic

4th December 2005, 09:29

Sorry about slipping in her with a "dumb" question. I have used HC a few times and really like the quality. Would like to use the newest version but ha.. hummm... can't figure out how to use the batch version....

Hello, you should have a folder somewhere which contains 2 files:

HCbatch_016.exe
HC.INI

Before you encode, you set-up the HC.INI file with all the commands you need for that session. Example, my HC.INI looks like this for the last time I ran it:

*ASPECT 4:3
*BITRATE 3800
*INTERLACED
*MAXBITRATE 7000
*CLOSEDGOPS
*INFILE D:\atmp\Lemmi\lemmidvd\IMAGEschaintest.AVS
*OUTFILE D:\atmp\Lemmi\lemmidvd\4picsmenutry3.M2V
*PROFILE BEST
*WAIT 30

So, when I click on Hcbatch_016.exe it using those settings for the encode.

Prodater64

4th December 2005, 13:37

Sorry about slipping in her with a "dumb" question. I have used HC a few times and really like the quality. Would like to use the newest version but ha.. hummm... can't figure out how to use the batch version....
Could someone direct me to a guide on this subject. I copied an ini file from the .15 release, and added the 4 necessary paramaters, but no luck. Sorry again for this question, I'm sure its a no brainer, but...
T_2

With HC Encoder 0.15 and further complete packages you will find out a pdf file.
There are all you need to know.
But, if you want a easier way try ProCalc ASPA (http://forum.doom9.org/showthread.php?t=96834) for 2 Pass VBR encoding (you still have to edit your hc.ini file and your avs script, but it will help you with the command line, and has a lot of utilities also) and for CQ encodings with prediction GetCQ and HIG (http://forum.doom9.org/showthread.php?t=103137).