HC018 released

[EpheMeroN]

@EpheMeroN
Have you tried the BDVD Matrix the QLB Matrix or the MainConcept Matrix? Give those a try...are you using HC in conjunction with DVD-RB? If you are using RB you can try the different Matrices on a short segment with Rebuilder's Matrix Editor. You may also want to drop your DC precision to 8.

Regards,
Nope. I've only tested the default MPEG Matrix. I'm using HC solely by itself. The source is an analog capture of a sports event. I am keeping it interlaced, and filtering it with DeFreq, FFT3DFilter or FRFun, and TDeint for very minor upsizing to NTSC DVD spec.

I asked about DC Precision because it says a precision of 10 is recommeded for high bitrate, and my avg bitrate is 4000, and max of 9000. I will try a DC Precision of 8 though. Will dropping it to 8 make it more blurry? Possibly add a sharpener to the script...

[dragongodz]

I've only tested the default MPEG Matrix. I'm using HC solely by itself.
HC comes with a range of matrices so you can test some out on short sections of your footage by useing Avisynths trim() command.

I asked about DC Precision because it says a precision of 10 is recommeded for high bitrate, and my avg bitrate is 4000, and max of 9000. I will try a DC Precision of 8 though. Will dropping it to 8 make it more blurry? Possibly add a sharpener to the script..
well i would call 4000 more mid bitrate rather than really high so using a DC of 8 or 9 is probably a good idea.
you lose a tiny bit of detail by lowering the DC precision but i would not be adding a sharpener because of it. that would only end up making life harder for the encoder. :)

[ricoman]

Well which is it v.17 or v.18 you are using? You obviously have your wires crossed somewhere.
You, also need to, early on, give more relevent info. like indicating that Munich encodes correctly with Quenc. from the start. If you had shared that little tidbit I wouldn't have suggested moving the thread.

People here will help you but, we aren't mind readers.

@EpheMeroN
Have you tried the BDVD Matrix the QLB Matrix or the MainConcept Matrix? Give those a try...are you using HC in conjunction with DVD-RB? If you are using RB you can try the different Matrices on a short segment with Rebuilder's Matrix Editor. You may also want to drop your DC precision to 8.

Regards,

Relax dude, no need to be so sharp. I hadn't tried Quenc at the time of the post because I prefer HC and it takes 5 hrs or more. I think RB grabbed HC17 (which had worked fine for me in the past) instead of HC18. I thought downloading 18 would just update 17 like some other software. I uninstalled 17 and am in the process of another try. As far as matrixes, this is only my 3rd or 4th use of RB, so I am not familiar with the more advanced techniques, so be gentle with me. I do appreciate the help though.

[ricoman]

I seem to have a new problem, originally I used the RB installer and it automatically installed HC17 with it. When I downloaded HC18 and placed it in RB's encoder file, RB can't find it. In the RB mode drop down menu, HC is grayed out and I can't select it. RB free does not let you choose HC in the setup gui. How can I get RB to "see" HC18? I know it must be something simple, but I can't figure it out. I tried dragging it to the encoder file, then uninstalled it, downloaded again and extracted it to the encoder file, but still no luck. Thanks.

[dialysis1]

I believe you have to rename the file to HCbatch.exe.

[ricoman]

I believe you have to rename the file to HCbatch.exe.
No go. :confused:

Hank, do you plan on adding HC18 to the installer? I'll uninstall everything and start from scratch. Thanks.

[Chainmax]

@revgen and Chainmax:

Honestly I´ve been already thinking about such a "middleware" for HCEnc. Esp. because of the results you´ve posted revgen. It shouldn´t take too much coding to adopt the approach from QuEnc to HCEnc.

Currently I still have to dish out some bugs on LAN_AQE-RB (network distributed encoding with DVD-RB and AutoMatQEnc) but once this beast is working, I could focus on HCEnc^n :)
Sounds great. Good luck with it.:thanks:

Sounds great indeed, thanks :).

[tom942]

Hi,

I continue doing tests, now with "The Matrix" movie. In the elevator scene, when she stars to walk on the wall in the next shot there is a police man shooting. The sound stutters again with the flashes of the gun, and in the same scene, when neo starts to walk along the corridor, there is the same problem when neo is shooting with both guns. I don´t know what can it be.

One thing that I notice is when the encoder works in these parts it gets weird, I mean, that if it is normally telling you that his encoding 30 fps, suddenly it says that he is encoding 300 or more, and in the part of gops, happens something similar, but in the other way, if you see that he says that there are gops of 12, 15, 2, 8..., it suddenly dissapears, there is no gops.

Then I go to the DVD player with that movie rebuilded, and just in this planes of the movie, it stutters.

I hope to have explain me well, and hope that it helps.

BTW, the image is really great, just hope this issue can be fixed to use it without any risk.

Greets :).

[Mug Funky]

sounds like buffer underflows. HC is trying to avoid them (with autogop on i see... not sure if that's a factor), and it's re-encoding the same gop several times (hence the encode fps going up/down/whatever. usually it'll go down in these bits, but a GUI issue could cause it to go up. not sure).

what is your max bitrate set to? stuttery playback on a standalone is almost always a too-high-bitrate thing. sometimes it's poor media, but you'd probably be seeing "glitching" too if that were the case.

try subtract the bitrate of your audio streams from 9800 and round to the nearest 100kbps, then use that as your max bitrate. (so if you have a 224kbps + a 448 audio track, your max will be 9800 - (448+224) = 9128 ~= 9100)

though it's also good practice to go a fair bit lower than that to allow for the muxer doing weird things, or subtitles, or poor media, etc. i do everything at max 8500 or lower (only lower if there's DTS or multiangle), and that generally gives your authoring program some headroom.

of course some DVD players are just weird and will bork at the strangest things.

[Boulder]

Sounds like either the maximum allowed total bitrate (video+audio) is exceeded or a problem with the VBV buffer.

Try lowering the max bitrate and see what happens.

EDIT: Mug was faster..

[tom942]

Hi again,

Before all I want to say that I'm totally new to digital video (to handle DVDShrink I think it's not to know about it ;) ), so there a lot of things or technical terms that I still don´t know, although I'm reading all the guides and posts that I can :).

Here it is a thread I started with the same problem in other two movies:

http://forum.doom9.org/showthread.php?t=110826

I use Rebuilder both free and pro, and while in CCE everything is okey, with HC, I've got that problem.

What I'm trying to do is to reencode DVD that are about 7 GB into 4,3GB and see what quality I achieve and I've got to say is really good for my taste, except that issue.

So in the case of matrix, it says this:

- Reduction Level for DVD-5: 53,6%
- Overall Bitrate : 2.348Kbs
- Space for Video : 3.429.874KB
- HIGH/LOW/TYPICAL Bitrates: 8.808/1.695/2.348 Kbs

It is a full backup, nothing removed.

With King Kong was something similar. Reencode main movie, menus and use 2 audios streams and same for sub's, and in Crash the same.

Well, with Matrix, I set Best profile, Bias=20, default options for GOP and DC precision and filters inactive. After see the problem (BTW, glitching is when apeears a lot of blocks in the image and all distorted? ), I changed GOP size to 15 and later 12 (I live in Spain R2, so I use PAL) and nothing, later DCprecision to 10 and nothing.

About what you say VBV overflow when it encodes that parts, it doesn´t say anything, I mean it remains 0. Ahm I forget to say that I'm not using Hank's gui but the encoder itself (the one that comes with rebuilder).

I've been reading what you say about bitrates (Mug), and with that info, do you recommended to set max bitrate to 8500 or less?.

Ahm the last, my player is a JVC XV-N33SL.

Thanks and greets :).

Edit: I have done tests with 8500, 7500 and 7000 as Max_bitrate, and nothing, the problem persists :(.

[EpheMeroN]

I am encoding a bunch of separate video files into DVD-Compliant video files that I will be authoring into a DVD.

My question is, if I use the chapter creator inside HC to make chapter points for each mpeg, and then later add all of them into MuxMan, will the chapters stay intact? Or am I just wasting my time with this, and should just make all the chapter points in MuxMan?

[rahzel]

using HC 0.18 with DVD-RB with the movie, "Gladiator" there are some low bitrate smokey/cloudy scenes that i notice some major blocking. this wasn't AS noticeable with 0.17, but its more clearly visible with 0.18. the source also had SOME blocking, but its VERY hard to notice it.

i've also tried CCE, and it was pretty close to the source.

i used my DVD player to display the bitrate, and HC seemed to have went lower than CCE did. i used a VBR bias of 20 for all encodes, btw.

by blocking, its not really pixelation, but i just see a bunch of blocks. best way to describe it would be that it looks like a brick wall.

if anyone else has the movie gladiator and wants to see if its just me, the scene im talking about, is when maximus just escapes being executed, and hes riding a horse through a cloudy and stormy background. the clouds in the background are what shows blockyness (specifically the bottom area). shortly after that, hes sitting by the fire, and the smoke from the fire also shows some blockyness.

[Boulder]

@EpheMeroN: the chapter points mean that HC puts an I frame there so that the authoring program can use that frame as the chapter point. Otherwise it's possible that the frame in question won't be an I frame and you cannot place a chapter point there when you author the video.

@rahzel: what are the settings you used? Smoke/fog is a very tough challenge for any encoder.

[rahzel]

i used vbr bias 20, highest quality setting and the Auto-Q2 matrix with all 3 encoders tested (HC 0.17/0.18 and CCE 2.70.2.9).

[Boulder]

At what average bitrate?

[rahzel]

around 3390 i think.

[smok3]

edit: nm

[techmule]

It has a better RD optimization for P-frames, also B-frames will benefit from that because most B-frames are predicted from the P-frames.
Also some changes to the quantization routines and a lot of other tweaks

On my PC best profile also runs a bit slower, normal and fast profile runs a bit faster than 017.

And about this:

I will never comment about visual quality, everybody has his own opinion about it, you can't measure it...

But the metric tests I did shows an improvement over HC017, especially with lower bitrates:
In the "Challenge Low Bitrate" by Sagittaire I got the next values:
HC017, OPSNR 46.71, SSIM2 81.38
HC018, OPSNR 46.97, SSIM2 83.04

using best profile and the standard MPEG matrix

Thanks hank, I appreciate your work, but IMHO, the quality factor is becoming saturated now with very little visible difference in ver17 and ver18 but ver18 has lost on speed (BEST profile), so probably adding multithreading to HC would be a big enhancement for the next release.

Just my 10 cents.

[Revgen]

@techmule

Darksoul71 is designing a multi-threading frontend for HCEnc called HCEnc^n. It's a version similar to his Quenc^n multithreading frontend for Quenc.

There's no need for Hank to do the work.

[Rumbah]

The design of Darksoul71's approach can never reach the quality a normal multithreaded encoder can reach as the encoder can distribute the bits over the whole video to encode. If you divide the video into n parts, you can only guess the best distribution or in the worst case, every part gets the same average bitrate. You can see where this leads if you divide the video with n frames into n parts. With equal bitrate for all parts you'll just get a CBR file.

[techmule]

@techmule

Darksoul71 is designing a multi-threading frontend for HCEnc called HCEnc^n. It's a version similar to his Quenc^n multithreading frontend for Quenc.

There's no need for Hank to do the work.

There is.....I personally like the multithreading implemented by SAPSTAR in AutoQmatEnc, it runs a single instance of encoder but using multiple threads as per the processing capacity of your CPU, giving you that speed gain.

Its an intelligent and efficient way to implement multi-threading, rest all are workarounds.

[smok3]

actually i would love to see an easy to use batch encoder, with workflow like;

1. construct the ini
2. select some video files (avisynth is generated automagically, based on selected template; you could have a plain converttoyv here or fully grown deinterlacer, or whatever...)
- video out dir is the same as in dir by default, log files are in the same dir as in files by default.
3. click 'do it'

:thanks:

[techmule]

actually i would love to see an easy to use batch encoder, with workflow like;

1. construct the ini
2. select some video files (avisynth is generated automagically, based on selected template; you could have a plain converttoyv here or fully grown deinterlacer, or whatever...)
- video out dir is the same as in dir by default, log files are in the same dir as in files by default.
3. click 'do it'

:thanks:

DVD rebuilder does this already, except for the avisynth part since that is limited to individual choice. The process might be different but batch functionality is available.

[Rumbah]

A user of the HCEnc provider for Cuttermaran noticed that when you encode small files (a few frames), they are always coded with max bitrate. I think it's because HC uses CQ in such cases. Is there a way to change this behavior and to encode even small samples at the given bitrate?

[hank315]

ATM encodings with 1 - 3 frames are done with Constant Quantization.
If there's no CQ command in the ini file a CQ of 4 is used for a 1 frame encoding, for a 2 or 3 frame encoding CQ = 5.
If the CQ command is present in the ini file the supplied value for CQ is used.

[Mug Funky]

@ smok3:

this can be handled quite comprehensively using regular windows batches.

i've already got ones that automatically convert your source to an avs (with PAL, top-first output), and others that turn .avs files (or folders full of them) into m2v's with optional avg rate, max rate, aspect ratio and interlaced/progressive.

it'll even select a CQM based on the above variables (15% penalty for interlaced, and max bitrate takes precedence over avg when choosing matrices).

[edit]

hint:

making an avs file in commandline is as easy as:

ECHO avisource("%~dpn1.avi")>"%~dpn1.avs"
ECHO converttopal()>>"%~dpn1.avs"
ECHO converttoyv12(interlaced=true)>>"%~dpn1.avs"

you'll have to switch out the functions called in there though... converttopal is a monster script i developed for work use, designed to mimic hardware standards conversion, and also add the option to IVTC+speedup, denoise, etc. you'll have to write your own :)

[foxyshadis]

Use DVD-RB, or read up on how to use the command line. DVD-RB will be a lot simpler.

[Audionut]

Or run the GUI. HCgui_18.exe. Set all the options nessecary and it will create the ini file and encode.

[Darksoul71]

Hi mates,

sorry, I didn´t visit this thread for quite some time. Otherwise I would have commented things earlier.

The design of Darksoul71's approach can never reach the quality a normal
multithreaded encoder can reach as the encoder can distribute the bits
over the whole video to encode.
That´s not quite correct. It depends on the encoding method you use wether the quality
of the segmented encoding can compete with the standard encoding. For example
CQ, Quality-based, CBR and One Pass VBR with Average bitrate methods would
show no difference in quality. Only multipass encoding is "problematic".

You can see where this leads if you divide the video with n frames
into n parts. With equal bitrate for all parts you'll just get a CBR file.
Yes, but hopefully no-one will push things that far ! The usage of segmented encoding
tools (such as HCEnc^n / QuEnc^n) requires some common sense and is not a no-brainer.

The initial version of QuEnc^n was developed by me with standard desktop PCs including
a dualcore CPU in mind. This would mean a segment for each core. Given the fact that
you use reasonable bitrate for a movie I still doubt that you would see a heavy impact in
quality. That´s something I still believe unless something proves me wrong. Of course
in theory there could be a quality impact but what if the user simply doesn´t see it ?

There is.....I personally like the multithreading implemented by SAPSTAR
in AutoQmatEnc, it runs a single instance of encoder but using multiple threads as
per the processing capacity of your CPU, giving you that speed gain.
The question which remains to me is if hank considers to do a MT implementation
of HCEnc after all. May be HCEnc is coded in a way that makes implementation of MT
very hard or only provides a very small speed increase. I can only guess....

Its an intelligent and efficient way to implement multi-threading, rest
all are workarounds.
Hm, I don´t want to sound offensive but you don´t have a clue how intelligent the
MT implementation of someone really is unless you understand enough of threaded
implementation (I do not :D) and look at his source code. Multithreaded implementations
are not "intelligent" per se. ;)

And in regard to efficency I´m keen on the numbers BR7 hopefully will report soon.
Why should one code a MT implementation when a simple external script written
in AutoIt can speed up the encoding process in a more efficent way ?
Even if you call it a workaround.

Note: I´m not encoder developer. Thus I can not comment on the possible
speed increase based on the usage of MT. I just can imagine that you are
not able to parallelize every part of the code and this would simply
mean a lower speedincrease compared to

Just some final thoughts:
As indicated by Rumbah (and LigH over at german doom9) my "segmented encoding"
approach could cause a drop in quality for multipass encoding (an no other
encoding method !).

This is caused by the simple fact that the multiple encoder instances are not
"aware of each other" -> They work independently for their segment with the
assigned bitrate not considering how complex the other segments might be.

Normally any encoder does at least two passes in multipass mode. In the
first run the encoder analyzes the movie (always writing some kind of
log file). In the 2nd (3rd, 4th, etC) run the encoder will distribute the "bits"
in the movie based on some algorithm. Scenes which are less complex get
less "bits". Scenes with higher complexity will get more bits. Thus the average
bitrate of a less complex part of a movie will be much lower than your target
average bitrate while a more complex part will have a much higher average
bitrate. Cutting the movie in "chunks" will lower the encoding quality since
you limit the encoder(s). The less complex part of the movie could get "too much"
bits available while the more complex part of the movie could need them.

All this is pure theory and (as already stated above) I doubt one would see
a serious impact on the majority of movies encoded in 2-4 segments at
a reasonable bitrate. If you push your encoder to the limit, things might
look different but I´m no-one of this "I wanna have 14 hrs of movie on
my DVD-5"-guys. YMMV.

Also I´m don´t believe in true quality problems I coded the approach
described in the HCEnc^n thread here in the forum as proof of concept
in QuEnc^n. For those of you who haven´t read this thread: I´ll do a
1% sample run in CQ for each segment and use the filesize deviation
as indicator for the complexity of each segment. After this I simply
scale the average bitrate for each segment based on the deviation factor.

I called this method ABD = Adaptive Bitrate Distribution

The results are quite intesting. The log posted below is generated by
QuEnc^n. It indicates the size deviation for each segment in % compared
to the average size over all samples. It also shows the correction
factor and the adjusted bitrate. I know..it´s a bit hard to read but
helps me to understand how well (or not) ADB works.

17:02:32 >>Starting Calculation for Adaptive Bitrate Distribution (ABD) !
17:02:32 >>Average Filesize for Samples: 5.22 MB
17:02:32 >>Average Bitrate specified in CLI: 3789 kBit/s
17:02:32 >>Filesize for Sample # 0: 5411385 Bytes
17:02:32 >>Filedeviation for Sample # 0: -1.22 %
17:02:32 >>Bitrate Correction Factor for Sample # 0: 1.01100454049247
17:02:32 >>Corrected Bitrate Factor for Sample # 0: 3831
17:02:32 >>Filesize for Sample # 1: 4472682 Bytes
17:02:32 >>Filedeviation for Sample # 1: -18.36 %
17:02:32 >>Bitrate Correction Factor for Sample # 1: 1.16521694726561
17:02:32 >>Corrected Bitrate Factor for Sample # 1: 4415
17:02:32 >>Filesize for Sample # 2: 5197223 Bytes
17:02:32 >>Filedeviation for Sample # 2: -5.13 %
17:02:32 >>Bitrate Correction Factor for Sample # 2: 1.04618759355542
17:02:32 >>Corrected Bitrate Factor for Sample # 2: 3964
17:02:32 >>Filesize for Sample # 3: 5603867 Bytes
17:02:32 >>Filedeviation for Sample # 3: 2.29 %
17:02:32 >>Bitrate Correction Factor for Sample # 3: 0.979383130439973
17:02:32 >>Corrected Bitrate Factor for Sample # 3: 3711
17:02:32 >>Filesize for Sample # 4: 6229628 Bytes
17:02:32 >>Filedeviation for Sample # 4: 13.71 %
17:02:32 >>Bitrate Correction Factor for Sample # 4: 0.87658159483737
17:02:32 >>Corrected Bitrate Factor for Sample # 4: 3321
17:02:32 >>Filesize for Sample # 5: 4563328 Bytes
17:02:32 >>Filedeviation for Sample # 5: -16.7 %
17:02:32 >>Bitrate Correction Factor for Sample # 5: 1.15032540241665
17:02:32 >>Corrected Bitrate Factor for Sample # 5: 4359
17:02:32 >>Filesize for Sample # 6: 4989777 Bytes
17:02:32 >>Filedeviation for Sample # 6: -8.92 %
17:02:32 >>Bitrate Correction Factor for Sample # 6: 1.08026732584078
17:02:32 >>Corrected Bitrate Factor for Sample # 6: 4093
17:02:32 >>Filesize for Sample # 7: 5721328 Bytes
17:02:32 >>Filedeviation for Sample # 7: 4.43 %
17:02:32 >>Bitrate Correction Factor for Sample # 7: 0.960086352319545
17:02:32 >>Corrected Bitrate Factor for Sample # 7: 3638
17:02:32 >>Filesize for Sample # 8: 6036680 Bytes
17:02:32 >>Filedeviation for Sample # 8: 10.19 %
17:02:32 >>Bitrate Correction Factor for Sample # 8: 0.9082795605007
17:02:32 >>Corrected Bitrate Factor for Sample # 8: 3441
17:02:32 >>Filesize for Sample # 9: 6557807 Bytes
17:02:32 >>Filedeviation for Sample # 9: 19.7 %
17:02:32 >>Bitrate Correction Factor for Sample # 9: 0.822667552331482
17:02:32 >>Corrected Bitrate Factor for Sample # 9: 3117


For some segments there is a variation up to 20%. This smallest bitrate
assigned were 3117 kBit/s, while the highest bitrate was 4415 kBit/s.
Doing such an estimation works quite fast. It only took something like
2 Minutes for a 100 min movie encoded on my A64 3200+ with QuEnc 0.60.

Although the dependency between filesize and complexity of each segment
might not be linear, this should still give a good guess. Based on the
values listed above I see my thoughts confirmed. If you use a reasonable
choose bitrate for your movie (e.g. >= 2.5 MBit for HalfD1 progressive
material) a variation of +/- 20% in bitrate should not cause such a
serious drop in quality.

While the current version of QuEnc^n is still limited to one segment
per core, the modified version which I will release shortly after some
more testing handles the number of segments independently from the
number of cores. In order to give the enduser more options to finetune,
he can determine the segment length in frames as well as some sort of
"weight factor" which controls the impact of the deviations described
above. Of course there could still be some drop in quality but with this
work around I´m pretty confident that it will be even less visible. All the
stuff described above is simply based on the fact that I don´t not the
complexity distribution within the movie and I do not know the algorithm
by which the encoder distributes.

All this changes if you use the "segmented encoding" approach within
an Encoder (Note: I do not say QuEnc, HCEnc, AQE :)). I don´t see
a big difference if an encoder does the first run parallel for "n" segments
or one run for complete movie. The encoder could "merge" the "n" logfiles
generated for each of the "n" segments. This should produce a logfile identical
to the standard "one instance" encoding process. After this the encoder could
start the 2nd pass and encode each segment based on the information stored
in the logfile using its "bitrate distribution" equation.

Benefits of this method which I can see:
- No big modifications required for the encoder source code as you "simply" do
the same things as QuEnc^n does but internally.

- No need to agonize which part of the algorithms used in your encoder are
are suitable for MT and which are not.

- May be a bigger speed increase over a "true" MT implementation.

Oh yes...there is also another "problem" with QuEnc^n (and of course HCEnc^n)
I´m aware of: They produce an I-frame for the begining of each segment as
I´m not doing any scene detection. I don´t think that this causes a tremendous
drop in quality since the amount of "unneccessary" I-frames is very small
in relation to the complete number of I-frames in the complete movie.

May be I´m able to work out a simple scene change detection script in AVISynth
and do a readjustment of the segment borders to match a scene change.

Best regards,
D$

[Boulder]

At command prompt: hcenc_018.exe -ini path\inifile.ini

You don't even need to save the ini, the GUI can launch the encoder as well when you've setup the parameters for the encode.

[techmule]

Hm, I don´t want to sound offensive but you don´t have a clue how intelligent the MT implementation of someone really is unless you understand enough of threaded implementation (I do not :D) and look at his source code. Multithreaded implementations are not "intelligent" per se. ;)

No offence taken, intelligence is only an expression of speech, we all know that these dumb machines run on our intelligence. Although the source for autoqmatenc is not available but the way it reads the free threads available and you CPU capacity and detemines the number of threads to be used for encoding, seems like some intelligence to me.


And in regard to efficency I´m keen on the numbers BR7 hopefully will report soon.
Why should one code a MT implementation when a simple external script written in AutoIt can speed up the encoding process in a more efficent way ?
Even if you call it a workaround.


Cos the processing speed of any hardware would be more than what any lines of code can achieve. Its to do with the CPU internals and how efficiently you can use the hardware power.

[mezzanine]

HCEnc does an amazing job with pal/progressive content. I've been using CCE for years but now i'm switching to HC, it gives higher detail and noise-free encodings.
Amazing work Hank, thanks.

[Darksoul71]

Hi techmule,

granted, I formulated my answer not clear enough. Intelligent was the wrong word. I guess smart would be more apropriate.

...but the way it reads the free threads available and you CPU capacity and detemines the number of threads to be used for encoding, seems like some intelligence to me.
Have you seen the number provided by BR7 over at VM ? While I don´t want to jump to conclusions it seems to me that the way AQE "judges" how many threads shall be used is not always the intelligent, er, smart one. There was a speed penalty from using MT by nearly 20% over the single thread encoding. I´m currently in the mood adapting my QuEnc^n code for AQE and wouldn´t be suprised if 2-4 running instances of AQE limited to a single thread run faster on a dualcore machine than one instance with MT enabled.

But as I said I don´t wanna jump to conclusions.

First "AQE^n" has to be coded and I need enough volunteers with dual core / dual CPU systems willing to do benchmarks for comparing AQE vs. AQE^n.
After this we can discuss further....

...but this thread was about HCEnc, wasn´t it ? ;)

Cos the processing speed of any hardware would be more than what any lines of code can achieve. Its to do with the CPU internals and how efficiently you can use the hardware power.
Hm, agreed, there might be better optimisation techniques than my simplified segmented encoding approach. Esp. when using CPU extensions such as 3DNow and ISSE. I would call something efficient if you are using the existing hardware power in a smart(er) way. The best example I know was the 3DNow enabled OpenGL miniport which was available for K6-2/3 CPUs a long time ago. You could play Quake2 on Voodoo2/3 graphic adapters in a speed that was identical to those on Intel PII (good FPU performance) although the K6 CPUs had crappy FPU performance.

So to come back to the point "external" versus "internal" implementation: I would simply call the implementation effictive which provides the biggest speed increase at identical quality on dual core / dual CPU machines. Unforunately I do not own such a machine (but hopefully will in future :D) and thus I´m depending on other people doing the testing to report "real numbers". To me it´s obvious that numbers don´t lie :rolleyes:

The implementation with the biggest encode FPS value is more effective. That´s it !

I wouldn´t write those things without the feeling that the "segmented" encoding approach can provide a tremendeous speed increase. May be even a lot better than a smart, efficient MT implementation. I really would like to hear on of our "encoder gods" commenting this "feeling". The number provided by Revgen when testing QuEnc^n on his AMD Multicore beast look really promising. Even I don´t have a multicore system myself, I have no doubt that 2-4 core CPUs will become standard over the next few years. Thus multicore / multi CPU enabled software will get more and more important.

For those who sing the "..but segmented encoding will not give me the same quality as non-segmented encoding"-song I´ll do some metrics (PSNR based on various combinations of bitrates and segments numbers). May be they are right or may be I am right although I still strongly doubt there is any visual impact on quality based on segmented encoding if you use reasonable parameters (e.g. segmentlength >= 5 mins, bitrate > 2.5MBit for progressive half D1 material, etc).

Best regards,
D$

[Revgen]

First "AQE^n" has to be coded and I need enough volunteers with dual core / dual CPU systems willing to do benchmarks for comparing AQE vs. AQE^n.
After this we can discuss further....

I'd rather see a Mencoder^n program before an AQE^n program is done. AQE is good but Mencoder is great quality and very slow when the advanced B-RDO options and others settings are turned on.


I wouldn´t write those things without the feeling that the "segmented" encoding approach can provide a tremendeous speed increase. May be even a lot better than a smart, efficient MT implementation. I really would like to hear on of our "encoder gods" commenting this "feeling".

MPEG-2 encoding isn't exactly slow anymore.

I use Quenc^n to encode DVD's faster. Plain and simple. If I really wanted to make a quality video, I can just use the plain vanilla program and it may take an hour or so longer. No big deal.

Quality/Speed tradeoffs shouldn't even be a concern with MPEG-2 anymore. We're not talking about codecs like AVC/H264 where speed/quality tradeoffs can affect the days rather than the hours a video takes to encode.



For those who sing the "..but segmented encoding will not give me the same quality as non-segmented encoding"-song I´ll do some metrics (PSNR based on various combinations of bitrates and segments numbers). May be they are right or may be I am right although I still strongly doubt there is any visual impact on quality based on segmented encoding if you use reasonable parameters (e.g. segmentlength >= 5 mins, bitrate > 2.5MBit for progressive half D1 material, etc).

Best regards,
D$

There is some visual difference between multi-threaded and single-threaded. But I have to squint at the vids in Vdub and browse frame-by-frame to see it. I wouldn't worry about it too much. It's absolutely unoticeable when viewed in full motion, even on my 24in. widescreen monitor.

[Ebobtron]

I can't seem to get command prompt to work...i need to like open it up in the location HC is or something...

and could you tell me how to get the gui to start the encoder...
For the GUI.
You click the long thin button on the bottom of the window, it is labeled encode.

If your going to tell me that it is grayed out, you do that.

First let me explain that a grayed out button normally means that you did not provided the program with the info it needs to do its job. With HC that normally means an input and output file name. Everything else will default to something.

Watch the info box, if your script is bad it will tell you.

[Darksoul71]

Sorry, I know it´s OT....

I'd rather see a Mencoder^n program before an AQE^n program is done. AQE is good but Mencoder is great quality and very slow when the advanced B-RDO options and others settings are turned on.
Hm, I just dl´ed MPlayer / MEncoder and I don´t have the urgent need to code a frontend for an encoder with a gazillion of parameters :D

So "MEncoder^n" might make sense for those who know to use MEncoder. To me it just looks too timeconsuming....

Take care,
D$

[cweb]

First let me explain that a grayed out button normally means that you did not provided the program with the info it needs to do its job. With HC that normally means an input and output file name. Everything else will default to something.

Watch the info box, if your script is bad it will tell you.

Regarding the command prompt, if you extracted HC to c:\apps\hc,
you will need to specify the full path to it in your batch file unless you add HC's path to the PATH environment variable (in Control Panel, System under XP).

[Rumbah]

I just had a crash with HC.
It was called very often in a short time by DVD-RB. You can see the error message:

http://img197.imageshack.us/img197/5698/hcerror1jx.th.png (http://img197.imageshack.us/my.php?image=hcerror1jx.png)

It seems that the file isn't released properly before HC is called again. I ran into this error with my Cuttermaran provider, too, as HC is called there very fast, too.

[hank315]

This is a strange error.
When a HC encoding is done it closes all files and signals the OS to delete the .dbs file.
I checked the error message in the Fortran manual but that isn't very helpful.
Maybe Windows isn't done deleting the file and the next HC session already tries to open it ?
In the next release HC will not finish until the .dbs file is really gone, maybe it will solve it.

[dragongodz]

Maybe Windows isn't done deleting the file and the next HC session already tries to open it ?
In the next release HC will not finish until the .dbs file is really gone, maybe it will solve it.
a possibly more elegant solution would be to have the .db named based on source file. so file1.avs would produce file1.db and even if it wasnt deleted in time file2.avs could still be started wih a file2.db.
the only danger there would be if people used the same names for multiple files in multiple places. then the check to make sure there is no existing .db would kick in and you could either change the name if so, such as if file1.db exists change to file11.db. or if it exist try to delete, wait for success or failure and then proceed to encode if deletion ok or change .db name if not.

lots of possible ways. :)

[Boulder]

I just got a huge spike above 10000kbps.

--------------------
| encoder settings |
--------------------

profile: BEST
frames: 0 14913
framerate: 25.00
aspect ratio: 4:3
bitrate: NA
max. bitrate Kb/s: 8000
pass: 1 (Constant Quant)
constant Q: 2.000
pulldown: no
closed gops: no
VBV check: yes
scene change det.: yes
interlaced: yes, TFF
goplen,B-pic: AUTO
dc_precision: 10
scan method: ALTERNATE
bias: 0
chapter frames: 0
time code: 0 0 0 0
CPU: SSE2
matrix: CUSTOM

--------------------
| source stats |
--------------------

nr. of frames in source: 14914
width*height: 720*576
fps: 25.00
nr. of frames to encode: 14914
frames to encode: 0 - 14913

---------------------
| encoding - pass 1 |
---------------------

pass 1 encoding time: 1:32:36 (5556 s)
average fps: 2.7

------------------
| encoding stats |
------------------

total encoding time: 1:32:36 (5556 s)

intra matrix used
8 8 9 9 10 10 11 11
8 9 9 10 10 11 11 12
9 9 10 10 11 11 12 12
9 10 10 11 11 12 13 13
10 10 11 11 12 13 13 14
10 11 11 12 13 13 14 15
11 11 12 13 13 14 15 15
11 12 12 13 14 15 15 16

non-intra matrix used
8 8 9 9 10 10 11 11
8 9 9 10 10 11 11 12
9 9 10 10 11 11 12 12
9 10 10 11 11 12 13 13
10 10 11 11 12 13 13 14
10 11 11 12 13 13 14 15
11 11 12 13 13 14 15 15
11 12 12 13 14 15 15 16

nr. of gops: 1022
nr. of frames: 14914
nr. of I-frames: 1022
nr. of P-frames: 4628
nr. of B-frames: 9264
average quant (non linear): 6.786
VBV underflows detected: 0
VBV underflows fixed: 0
minimum bitrate: 3670
maximum bitrate: 10891
average bitrate: 7104

I'll try to run the same encode with v0.17 tomorrow and see what happens.

EDIT: looks like v0.17 shows a spike as well, 10918kbps is the maximum at the moment. I opened the v0.18-created m2v file in MPEG Stream Eye and it appears that the spike occurs on an I-frame in which one field is from one scene and the other field is from another. Other similar I-frames also showed some large sizes but apparently didn't go above the limit.

[EpheMeroN]

How come when encoding NTSC material with the "autogop" option checked, it never goes higher than 15?

NTSC max per GOP is 18 yes?

[Mug Funky]

@ boulder:

maybe try mpeg default matrix. your average quant is 6.786 with your custom matrix, so with the default mpeg matrix it'll come out at rougly 3.3, which should be the same quality.

the (possible) advantage of that is that I-blocks will be smaller for the same quant, and so HC wont have to try so hard to prevent the underflow. your encode will most likely go faster too.

question: do you get black blocks on near flat white areas with the current matrix + "best" profile with 0.18? i got that once and haven't tried again (i should probably have sent a sample to hank...)

[Boulder]

I checked my encode for the blocks but couldn't see any. I'm mostly dealing with analog captures so white is not really white ;)

I'll try the MPEG standard and see how it comes out. Do you mean the one with flat-16 non-intra matrix or the one HC calls MPEG standard (which is actually CCE standard)?

[Boulder]

I tried changing the matrix but it still peaks above 10000kbps. However, I tried muxing with MuxMan and it doesn't complain so it's DVD compliant. Don't know if it plays smoothly on all standalones though.

[ronnylov]

Maybe a stupid qustion but I wonder how to make a single pass CBR encoding with HC encoder. I found the VBR bias option which I set to 100 and I set average and maximum bitrate to the same value but it still does 2 encoding passes. This results in a CBR file but the encoding time feels unnecessary long because of 2-pass encoding.

If I am encoding a DVD with less runtime than 60 minutes I want to use maximum bitrate of the entire clip so I want 8000 kbit/s CBR and I think it is enough to make this encoding in 1 pass.

[buzzqw]

@ronnylov

actually isn't possible to do a CBR encoding

BUT is possible to do a Costant Quant encoding (at Q 2) with max bitrate capped at 8000 :D


BHH

[Boulder]

You cannot do CBR with one pass at the moment. The fastest way to do something similar is what buzzqw said. With q2 you should get as much out of the encode as possible in one pass, provided that you choose an appropriate quantizer matrix.

[ronnylov]

@ronnylov

actually isn't possible to do a CBR encoding

BUT is possible to do a Costant Quant encoding (at Q 2) with max bitrate capped at 8000 :D


BHH

Oh, I did not think about that option! I guess I need to do some testing to see how it works. I want the best possible quality on DVD without exceeding maximum bitrate for DVD (I have found that 8000 is generally a good maximum bitrate for DVD because sometimes some encoders peaks above maximum). I use it for converting DV to DVD and then normally put one hour maximum per disc. 8000 kbit/s CBR have given me fine results with other encoders like Mainconcept and Procoder. I use the 'mb1 interlaced DV' matrix when it is possible to change the matrix in the encoder (I have not found that option in Procoder). I really like the many preinstalled matrices I can use in HC encoder.

Regarding choosing a good matrix, can I make any conclusion of for instance making a constant quality encode with q=2 (or any other optimized value) using different matrices and checking average and maximum bitrate compared to a target bitrate? I can think that choosing too high values in the matrix may saturate the encoder to not be optimal at a choosen target bitrate (the file size will be smaller than the available space on the disc and bits are waisted) or using too low values will perhaps make it too difficult for the encoder resulting in some kind of artifacts? Is there an optimal target Q-value I should aim for?