Who wants to test a new MPEG2 encoder

[hank315]

For some time I've been working on a MPEG2 encoder,
maybe there's somebody who want to run some tests with it.

Some information about the encoder (named HC):
(complete manual is included)

Input can be a d2v project or input using Avisynth.
GUI version and batch version.
The encoder should run on any Intel and AMD processor using Windows XP/2000.
At least MMX/ISSE must be present to run the encoder.
If SSE2 or SSE3 is present HC will also use it.
2 pass VBR encoding.
Variable GOP structure dependent of the video content or fixed GOP structure.
Scene change detection.
Pre-programmed matrices or use your own matrices.
Restart possibility to run the second pass again.
Bitrate control: average and max bitrate can be set.
Buffer underflows will not occur, the VBV is constantly checked so no bitrate spikes.

Complete package: http://hank315.dyndns.org/HC_013.zip 0.13 beta version, last update 17-04-2005

Please remember that it is a work in progress... there may be a few bugs :)

[Ebobtron]

I don't know what to say.

Configuration technique is unique and might even be elegant. Might be hard to do batch encoding, but I can write ini files from a batch file.

I still have testing to do but the stream looks compliant.

I test scene detection with music videos. The cuts are often quick and complex. A quick look at the first 15 cuts of my test video have me at a loss for words. In short, for now, HC appears to be dead accurate. Original video sometimes misses scene change by a frame or confuses large motion for a change. Your encoder hits the scenes a major motion picture release on DVD misses.

Quality looks very good.

And yes, bugs. I'll post logs and bug-list tomorrow.

Thanks for your help before. I do lots of NTSC.

Ebob

[video_magic]

;) I had downloaded last night but can't test yet because I have another big mpeg2 encode running at the moment with Nuenc, but I look forward to running a clip through yours after that finishes.

[Nic]

Is this written in Fortran?
If so. Very Hardcore. :)

Haven't tested it yet...I'll let you know.

-Nic

ps
May I ask what your future plans are for your encoder?
i.e. Going to go opensource, commercial, freeware, just doing it as a hobby etc...Either way, good luck :)

[hank315]

Is this written in Fortran?
If so. Very Hardcore.It is written in Fortran95
+ a little C code to communicate with DGdecode.dll and Avisynth
+ about 6000 lines of assembler, mostly MMX/SSE/SSE2 stuff

Hardcore, LOL.. that C stuff guys are writing here, that's real hardcore.
I've always programmed in Fortran, had to, when I started there was no C.
In the 80's I tried C but it was a great dissapointment, compilers were buggy and slow and the generated code was buggy and slow.
Nowadays a lot has changed, with C you can do everything and it is (almost) as fast as Fortran :), but for writing this kind of applications I'll just stick to Fortran.

May I ask what your future plans are for your encoder?
i.e. Going to go opensource, commercial, freeware, just doing it as a hobby etc...Either way, good luck I was just curious how the MPEG compression scheme works.
Also I was surprised to discover that all encoders are (very expensive) commercial codes or based on ffmpeg or mpeg2enc so I decided to start from scratch and not to look at the open source C code (which in my case wasn't so hard because I can hardly read the stuff :) )

So it started as a hobby...
open source - probably not, most people will not understand or can compile the Fortran sources
freeware - probably
commercial - unlikely, but come on guys, make me an offer I can't refuse:D

It was also a good exercise to start writing a H.264/AVC encoder.

Future plans for HC:
- Speed and quality improvement and getting all the bugs out
- Support for hyperthreading and multi-processing systems
- Writing a GUI but I really hate that!!

[Nic]

Very impressive.

If you really don't fancy making a GUI for it...Could you not make a DLL out of your Encoder, so that it could be fed the parameters and YV12 frames? That way I could plug it into QuEnc ;)

Good luck finding bugs, and you're right, I doubt many could follow the Fortran code. :)

Take Care,
-Nic

[mean]

... and building a .so for the nix guys :)

[Ebobtron]

Bug report:

------------------

Opening command window reports frame-rate incorrectly under encoder settings. Reports correctly under source stats. Output to log is correct.
-----------------------------------------------------------
--------------------> encoder settings <-------------------
-----------------------------------------------------------
input: faithv.avs
output: Faith_HC_T04.m2v
log file: Faith_HC_T04.log
profile: GOOD
frames: - end
framerate: 25.00 <---------------
aspect ratio: 9:16
bitrate: 8000000
max. bitrate: 8000000
restart: no
closed gops: no
VBV check: yes
scene change det.: yes
interlaced: yes
goplen,B-pic: 15 2
dc_precision: 9
scan method: ALTERNATE
CPU: SSE2
-------------------------------------------------

Sequence header reports bit-rate as 9800000 always.
Sequence header reports first GOP-timestamp as 1h 0m 0s 0f.

----------------------------------------

Flashing pink or sometimes green horizontal line in bottom right of displayed video. Bad lines are in odd fields only even fields look great..

------------------------------------------

Seems that CPU Auto doesn’t select SSE2 my max. But not sure.

----------------------

More later
Ebob

[hank315]

Opening command window reports frame-rate incorrectly under encoder settings. Reports correctly under source stats. Output to log is correct.
: 25.00 <---------------Will be corrected in the next version, just wrote the default value to the screen.
Sequence header reports bit-rate as 9800000 always.HC just writes the max. value for the VBV in the video file to be on the safe side. Should be OK.

Sequence header reports first GOP-timestamp as 1h 0m 0s 0f.You wonder why? I also did and still do;)
Just took this value because most encoders do but couldn't find any information why (is there somebody out there who knows). Can easily be changed or an extra command to set the timecode.

Seems that CPU Auto doesn’t select SSE2 my max. But not sure.The encoder was developed using a P4 3.2GHz (Prescot core) which can use all of the present extensions, testing on all other systems always shows the right CPU but there was only one AMD machine amongst them (which was recognized as an AuthenticAMD, only MMX/SSE support). I'm just checking bit 0, 23, 25 and 26 with the proper calls to cpuid. If you choose a wrong extension, one your CPU can't handle, the encoder will crash or it will produce absolute rubbish. BTW, what CPU do you have?

Flashing pink or sometimes green horizontal line in bottom right of displayed video. Bad lines are in odd fields only even fields look great..That's a hard one, I'll try and reproduce this error. Would you try to encode that scene again with *INTERLACED turned of and see if its still there? My experience is it is better to use a de-interlace filter in Avisynth and then just encode it as progressive.

If you set the average bitrate equal to the max. bitrate you will not get the desired filesize. HC handles the max. bitrate in the next way: max. bitrate is controlled per GOP, if the actual bitrate > max. bitrate the GOP is recycled with higher quantization values until the actual bitrate <= max. bitrate. For low action scenes bitrate will be lower but it will try to catch up to reach the target but will never reach it.

[Ebobtron]

HC just writes the max. value for the VBV in the video file to be on the safe side. Should be OK.

Thought that was the case.

--------------------------------------------
I have a Intel Celeron @ 2.80GHz. It looked faster when CPU set to SSE2 than Auto. I'll check again with same source.


If you choose a wrong extension, one your CPU can't handle, the encoder will crash or it will produce absolute rubbish. BTW, what CPU do you have?

Your right it crashes.
-----------------------------

I'll try some deinterlaced stuff.

------------------------

I was comparing other encoders with the 8000 8000. Some encoders spike over, yours doesn't.


Ebob

[hank315]

@NicIf you really don't fancy making a GUI for it...Could you not make a DLL out of your Encoder, so that it could be fed the parameters and YV12 frames? That way I could plug it into QuEnc ;)I think that could be done fairly easily, but call it masochistic behaviour, I just want to give it a try and write such a :devil:GUI-thing. And maybe after a while, when I'm totally frustrated banging my head against the Windows GUI wall, I will beg you on my knees to implement it in QuEnc:)
Anyway, thanks for the offer but let's not be too hasty, I'll wait for the tests people will do and first resolve the most bugs.

PS. I really like your latest (0.59) build, great quality, used it to do some comparisons with HC.

[Ebobtron]

Flashing pink or sometimes green horizontal line in bottom right of displayed video. Bad lines are in odd fields only even fields look great..

Originally posted by hank315


That's a hard one, I'll try and reproduce this error. Would you try to encode that scene again with *INTERLACED turned of and see if its still there? My experience is it is better to use a de-interlace filter in Avisynth and then just encode it as progressive.

If I encode a PAL clip no lines. If you change your source resolution to 720x480 I think you will see the lines. Non interlaced clips have the llne each frame if 720x480 used.

-------------
Let me show you how new to this hobby I am by asking this question.

Why would I want a DVD compliant video stream that is progressive?

I never encode film and all my stuff is intended to be burned as a DVD for viewing on my television or my mom's TV.
Am I not going to be able to tell the difference?

Off to see if I can break your new toy.
Thanks,
Ebob

[Amnon82]

@hank315: Your Encoder looks great! It has all features a MPEG2Encoder should have.

So You don't like Guis. If You want I can code a GUI for You. I'm coding in Delphi.

The Support of CQ-Values is great. I'm thinking to code AutoCQ for HC.
I did the program for CCE. It finds the right Q-Value for onepass-encoding.

I play a little with Your encoder and code a little GUI. I write You a PM.

... when do You release a non-limited test-version?

[hank315]

@Amnon82
Nice you like it, and I realize a GUI would be a good thing too.

But I'm also trying to program one (in Fortran).
I know when people hear about Fortran some of them still think we are carrying boxes of punchcards:)
The Fortran I'm using is Compaq Visual Fortran 6.6c which is totally integrated in Visual Studio 6.0.

The GUI I have in mind will look like: http://hank315.dyndns.org/GUI_template.jpg

It's not finished yet but some of the logic already works and it is also nice to cleanup the input part which has become a mess...

A full version will be posted soon, maybe even tonight.

And ofcourse feedback which might improve the encoder is always welcome.

@mean
I'm pretty much a complete Linux NOOB and don't have a Fortran95 compiler for Linux. There's an experimental compiler ( http://www.g95.org ) but the speed of it isn't optimal yet.
But if the GUI-Windows version is up and running I will give it a try.

[hank315]

Just put a version online without the 30000 frame limit.
No GUI yet, still working on it.
There are still some limitations, the max. nr. of GOPS is 64000 so you will run into a trouble if your movie is over 8 hours...

get it at http://hank315.dyndns.org/HC.zip

Have fun with it:)

[Amnon82]

I didn't looked at Your JPG.

Here is my Design-Sample #1. Hope You like it:

Go to the HC.GUI Thread (http://forum.doom9.org/showthread.php?s=&threadid=89101)

I started a new thread to keep this clear.

@Hank: did You add the AutoEncode and AutoClose Feature I asked for?
I'll need it to start my project AutoCQ for HC. It will get the right CQ-Value for the 1Pass VBR-Encode. I did some good results with the AutoQ for CCE Version.

Its good to see that You code it also in GUI. So Beginners will have it easier.

Keep the INI-Files. They are like ECL-Files for CCE.

I stay tuned for Your project. You have me 100%!
Lets play with Your last release.

... still missing the 2 functions ...

I can do the first <Enter>-Button request, but not the second. So the AutoEncode and AutoClose Functions are needed to code addons or external GUIs for Your encoder.

I'm working now on a little workaround. I call the little app 'BatchHC'. It do the 2 Enter-Commands.

[unixfs]

Hank315,
I just tried a 30 minutes encode of a full frame 720x576 dvb capture.
Simply put, it seems HC is absolutely the best mpeg2 encoder
I've seen so far!
Image quality (at 1600 kbps) is simply excellent, and you seem to
have found the way to avoid the most common artefact that all
other common encoders produce: blockyness in uniform or dark areas.

I really hope you will release your source code with an opensource license,
so that it can be used with other encoders like mencoder.

open source - probably not, most people will not understand or can compile the Fortran sources

There is a fortran compiler for *nix, released with gcc sources
(although it doesn't have a good fame).

Thanks a lot!

[Hemmo]

Hank315:
I just tried HC with one chapter from Matrix and I very happy to the result !
I hope HC will stay freeware ?

[Amnon82]

New Version Released: http://hank315.dyndns.org/HC_0.01.zip

Hank315 adds the *BATCH - Parameter.

How it works? Take a look into AutoCQ 0.2.0

[Mug Funky]

wow, now i know never to skip the mpeg-2 forum for more than a couple of days :)

open source - probably not, most people will not understand or can compile the Fortran sources
i don't know about that... my brother still uses fortran77 (he was born in 1980, go figure). he's never at a loss for a compiler in 'nix. does most of it for RISC processors though.

i can't code to save myself though.

but anyway, another opensource encoder can't possibly be a bad thing, especially if the quality is what people are saying it is (haven't had a chance to test yet).

of course it's most definitely your call, and freeware would be fine too.

k, now i'm going to test this baby :)

thanks heaps for your work.

[edit]

hmm.. what's a reasonable amount of time it should be on "intermediate pass" for an 823 frame clip? it's been on it for nearly 10 minutes now.

running a p3 733 (SSE1, MMX)

[freelock7]

Great results at the beginning of its development.
Tested 2pass.
Great quality up 3000kbs.
A little bit slow (7Fps).
Good job Hank315.

[unixfs]

wow, it's even working with avisynth under wine, and without slowdown

[hank315]

@Mug Funky
hmm.. what's a reasonable amount of time it should be on "intermediate pass" for an 823 frame clip? it's been on it for nearly 10 minutes now.That should be done in no time, with normal settings it should do the intermediate pass in just a few seconds for a 2 hour movie.
For very high or very low bitrates it might take longer but not that long.

Could you post your HC.ini file please.

[freelock7]

Sorry!
Here my script:
*PROFILE GOOD
*INFILE C:\Documents and Settings\Pierre\Mes documents\DVD.avs
*OUTFILE C:\Documents and Settings\Pierre\Mes documents\out.m2v
*LOGFILE C:\Documents and Settings\Pierre\Mes documents\out.log
*FRAMES 1 219715
*BITRATE 2500000
*MAXBITRATE 8500000
*ASPECT 16:9
*CQ 4
*MATRIX C:\Program Files\AutoCQ\matrices\AutoQ1.aqm
*RESTART
*NOSCD
*CLOSEDGOPS
*GOP 15 2
*CPU SSE
*SCANMETHOD ZZ
*PROGRESSIVE

First pass between 7-12 fps
second pass about 22 fps
The average prediction is correct with a global Quantizer Factor=11.05
Many blocs in picture- like a grainy movie.
QuEnc has a better texture quality at 2500 than HC because it uses for the same bitrate average a global Q factor of 6.5.
That makes the difference!
You should work in this way to decrease the quantization at low bitrate.

[Hemmo]

Originally posted by freelock7
...Many blocs in picture- like a grainy movie...

Have you try with:

*PROFILE BEST
*INFILE C:\Documents and Settings\Pierre\Mes documents\DVD.avs
*OUTFILE C:\Documents and Settings\Pierre\Mes documents\out.m2v
*LOGFILE C:\Documents and Settings\Pierre\Mes documents\out.log
*FRAMES 1 219715
*BITRATE 3500000
*MAXBITRATE 8500000
*ASPECT 9:16
*DC_PREC 10


so, replace HC.INI with that text and start HC.exe

[freelock7]

It doesn't change the quantization problem.
Too much compression at low bitrate with a quant=11 decreases quality.Test with 2Pass default HC encoding
--------------------------------
Bitrate Avg:3000kbs
Results:
max bitrate=6200
min bitrate=1200
Avg quant=9
----------------------
Bitrate Avg:4000kbs
Results:
max bitrate=8200
min bitrate=2100
Avg quant=7
----------------------
QuEnc0.59Beta2
Bitrate Avg:4000kbs
Results:
max bitrate=8500
min bitrate=2100
Avg quant=3.56

As you see, the global quantization remains high at 3000kbs even at 4000.
QuEnc have the best level balance between Bitrate and quantization.

The HC file is completely DVD compliant with Ifoedit0.976.

[Amnon82]

I coded a 1Pass Prediction-Tool for HC. So here is my first result of a 1Pass encode with HC Version 0.01:

HC.INI
*INFILE C:\Programme\AutoCQ\temp\AQFULLENC.avs
*OUTFILE F:\AutoQWork\OUT\AUTOCQ031_HC001.m2v
*LOGFILE F:\AutoQWork\OUT\AUTOCQ031_HC001.log
*CPU AUTO
*PROFILE GOOD
*SCANMETHOD ZIGZAG
*ASPECT 9:16
*BITRATE 8500000
*MAXBITRATE 9800000
*CQ 3
*DC_PREC 10
*GOP 12 2
*BATCH
*INTRAMATRIX
8 16 19 22 26 27 29 34
16 16 22 24 27 29 34 35
19 22 26 27 29 34 35 38
22 22 26 27 29 34 35 40
22 26 27 29 32 35 40 48
26 27 29 32 35 40 48 50
26 27 29 35 40 48 50 60
27 29 35 40 48 50 60 62
*NONINTRAMATRIX
16 20 24 28 32 36 40 44
20 24 28 32 36 40 44 48
24 28 32 36 40 44 48 52
28 32 36 40 44 48 52 56
32 36 40 44 48 52 56 58
36 40 44 48 52 56 58 60
40 44 48 52 56 58 60 62
44 48 52 56 58 60 62 62
HC.LOG
-----------------------------------------------------------
------------------> HC - MPEG2 encoder <-------------------
-----------------------------------------------------------
release: beta 0.01
-----------------------------------------------------------
---------------------> parsing stats <---------------------
-----------------------------------------------------------
input: C:\PROGRAMME\AUTOCQ\TEMP\AQFULLENC.AVS
output: F:\AutoQWork\OUT\AUTOCQ031_HC001.m2v
output: F:\AutoQWork\OUT\AUTOCQ031_HC001.log
frames: 1 - end
aspect ratio: 9:16
bitrate: 8500000
max. bitrate: 9500000
constant Q: 3
interlaced: no
goplen,B-pic: 12 2
dc_precision: 10
scan method: ZIGZAG
intramatrix: 64 values read
non-intramatrix: 64 values read
-----------------------------------------------------------
--------------------> encoder settings <-------------------
-----------------------------------------------------------
input: C:\PROGRAMME\AUTOCQ\TEMP\AQFULLENC.AVS
output: F:\AutoQWork\OUT\AUTOCQ031_HC001.m2v
output: F:\AutoQWork\OUT\AUTOCQ031_HC001.log
profile: NORMAL
frames: 1- end
framerate: 25.00
aspect ratio: 9:16
bitrate: 8500000
max. bitrate: 9500000
pass: 1 (Constant Quant)
constant Q: 3
restart: no
closed gops: no
VBV check: yes
scene change det.: yes
interlaced: no
goplen,B-pic: 12 2
dc_precision: 10
scan method: ZIGZAG
CPU: AUTO
intramatrix: 64 values read
non-intramatrix: 64 values read
-----------------------------------------------------------
---------------------> source stats <----------------------
-----------------------------------------------------------
nr. of frames in source: 135984
width*height: 720*576
fps: 25.00
nr. of frames to encode: 135984
frames to encode: 1 - 135984
movie length to encode: 5439.36 s
est. outfile length: ?????
-----------------------------------------------------------
-------------------> encoding - pass 1 <-------------------
-----------------------------------------------------------
pass 1 encoding time: 6566.7 s
average fps: 20.7
-----------------------------------------------------------
-------------------> encoding finished <-------------------
-----------------------------------------------------------
total encoding time: 6567.1 s
-----------------------------------------------------------
--------------------> encoding stats <---------------------
-----------------------------------------------------------
intra matrix used
8 16 19 22 26 27 29 34
16 16 22 24 27 29 34 35
19 22 26 27 29 34 35 38
22 22 26 27 29 34 35 40
22 26 27 29 32 35 40 48
26 27 29 32 35 40 48 50
26 27 29 35 40 48 50 60
27 29 35 40 48 50 60 62

non-intra matrix used
16 20 24 28 32 36 40 44
20 24 28 32 36 40 44 48
24 28 32 36 40 44 48 52
28 32 36 40 44 48 52 56
32 36 40 44 48 52 56 58
36 40 44 48 52 56 58 60
40 44 48 52 56 58 60 62
44 48 52 56 58 60 62 62

nr. of gops: 11884
nr. of frames: 135984
nr. of I-frames: 11884
nr. of P-frames: 35021
nr. of B-frames: 89079
average quant: 3.000 (non-linear scale)
VBV underflows detected: disabled
VBV underflows fixed: disabled
AUTOCQ 0.3.1 LOG:
AutoCQ-Log
___________________________________________________________________________
21:02:06 - 29.01.2005 >> AutoCQ Version 0.3.1 (Build 03100) started...
21:02:09 - 29.01.2005 >> AVS loaded from: G:\AutoCQWork\avs.avs
21:02:14 - 29.01.2005 >>
21:02:14 - 29.01.2005 >> _________________________________________
21:02:14 - 29.01.2005 >> Project-Info:
21:02:14 - 29.01.2005 >> _________________________________________
21:02:14 - 29.01.2005 >>
21:02:14 - 29.01.2005 >> Desired Target: DVD 720x576
21:02:14 - 29.01.2005 >> Entered Bitrate: min. 8500 max. 9500
21:02:14 - 29.01.2005 >> Entered wanted size in MB: 4476
21:02:14 - 29.01.2005 >> Entered Audio size(s) in MB: 1000
21:02:14 - 29.01.2005 >> Entered Extras size in MB: 0
21:02:14 - 29.01.2005 >> Authoring and ISO overhead in MB: 117
21:02:14 - 29.01.2005 >> % of Movie will be used: 4
21:02:14 - 29.01.2005 >> Your desired final encoded sample in MB: 134
21:02:14 - 29.01.2005 >> Your desired final encoded file in MB: 3350
21:02:14 - 29.01.2005 >> Safe Mode disabled
21:02:14 - 29.01.2005 >> Encode after Prediction enabled
21:02:14 - 29.01.2005 >> _________________________________________
21:02:14 - 29.01.2005 >>
21:02:14 - 29.01.2005 >> _________________________________________
21:02:14 - 29.01.2005 >> Start Prediction-Run #1
21:02:15 - 29.01.2005 >> First Q-Value: 6 [1|18]
21:02:15 - 29.01.2005 >> First Frameoffset: 00
21:07:57 - 29.01.2005 >> Results of Prediction-Run #1
21:07:57 - 29.01.2005 >> Encoded Sample in MB: 68
21:07:57 - 29.01.2005 >> Wanted Samplesize in MB: 134
21:07:57 - 29.01.2005 >> Precalculated Finalsize in MB: 1700
21:07:57 - 29.01.2005 >> Wanted Finalsize in MB: 3350
21:07:57 - 29.01.2005 >> _________________________________________
21:07:57 - 29.01.2005 >> Start Prediction-Run #2
21:07:57 - 29.01.2005 >> Q-Value of this pass: 4 [1|6]
21:07:57 - 29.01.2005 >> Frameoffset of this pass: 12
21:13:47 - 29.01.2005 >> Results of Prediction-Run #2
21:13:47 - 29.01.2005 >> Encoded Sample in MB: 99
21:13:47 - 29.01.2005 >> Wanted Samplesize in MB: 134
21:13:47 - 29.01.2005 >> Precalculated Finalsize in MB: 2475
21:13:47 - 29.01.2005 >> Wanted Finalsize in MB: 3350
21:13:48 - 29.01.2005 >> _________________________________________
21:13:48 - 29.01.2005 >> Start Prediction-Run #3
21:13:48 - 29.01.2005 >> Q-Value of this pass: 2 [1|4]
21:13:48 - 29.01.2005 >> Frameoffset of this pass: 00
21:19:46 - 29.01.2005 >> Results of Prediction-Run #3
21:19:46 - 29.01.2005 >> Encoded Sample in MB: 184
21:19:46 - 29.01.2005 >> Wanted Samplesize in MB: 134
21:19:46 - 29.01.2005 >> Precalculated Finalsize in MB: 4600
21:19:46 - 29.01.2005 >> Wanted Finalsize in MB: 3350
21:19:46 - 29.01.2005 >> _________________________________________
21:19:46 - 29.01.2005 >> Start Prediction-Run #4
21:19:46 - 29.01.2005 >> Q-Value of this pass: 3 [2|4]
21:19:46 - 29.01.2005 >> Frameoffset of this pass: 12
21:25:33 - 29.01.2005 >> Results of Prediction-Run #4
21:25:33 - 29.01.2005 >> Encoded Sample in MB: 125
21:25:33 - 29.01.2005 >> Wanted Samplesize in MB: 134
21:25:33 - 29.01.2005 >> Precalculated Finalsize in MB: 3125
21:25:33 - 29.01.2005 >> Wanted Finalsize in MB: 3350
21:25:33 - 29.01.2005 >> _________________________________________
21:25:33 - 29.01.2005 >> Encoding Final-Sample [E < W]
21:25:33 - 29.01.2005 >> Final Frameoffset: 00
21:25:33 - 29.01.2005 >> Final Q-Value: 3 [2|3]
21:31:15 - 29.01.2005 >> Encoded Final-Sample in MB: 121,97
21:31:15 - 29.01.2005 >> Wanted Samplesize in MB: 134
21:31:15 - 29.01.2005 >> Precalculated Finalsize in MB: 3049
21:31:15 - 29.01.2005 >> Wanted Finalsize in MB: 3350
21:31:15 - 29.01.2005 >> Calculated Finalsize in MB: 3087
21:31:15 - 29.01.2005 >> Renamed final Sample to: F:\AutoQWork\OUT\AUTOCQ031_HC001.sample
21:31:44 - 29.01.2005 >> _________________________________________
21:31:44 - 29.01.2005 >> Start Fullencode
21:31:44 - 29.01.2005 >> Final Q-Value: 3
23:30:32 - 29.01.2005 >> Results of Fullencode-Run:
23:30:32 - 29.01.2005 >> Your desired final encoded file: 3350
23:30:32 - 29.01.2005 >> Final encoded filesize: 3106
Now my Compiletests:

DVDLabPro: After 30% muxing the M2V with AC3(2Streams)DTS(1Stream) Error: Not DVD compliant.
IFOEdit0.96: After 2 Sec Error: Bufferunderruns
Rejig0.56: Compiled the full DVD.

Then I watched the whole movie in WinDVD6. It was sync but jumps on every VOB-Change (I mean the 1GB-Files. Maybe a matter of Rejigs DVD-Author.).

Now I'm encoding the same movie in 2Pass with this INI:
*INFILE G:\AutoCQWork\avs.avs
*OUTFILE F:\AutoQWork\OUT\2PASS_HC001.m2v
*LOGFILE F:\AutoQWork\OUT\2PASS_HC001.log
*CPU Auto
*PROFILE Good
*SCANMETHOD ZIGZAG
*ASPECT 9:16
*BITRATE 5045000
*MAXBITRATE 9800000
*DC_PREC 10
*GOP 12 2
*INTRAMATRIX
8 16 19 22 26 27 29 34
16 16 22 24 27 29 34 35
19 22 26 27 29 34 35 38
22 22 26 27 29 34 35 40
22 26 27 29 32 35 40 48
26 27 29 32 35 40 48 50
26 27 29 35 40 48 50 60
27 29 35 40 48 50 60 62
*NONINTRAMATRIX
16 20 24 28 32 36 40 44
20 24 28 32 36 40 44 48
24 28 32 36 40 44 48 52
28 32 36 40 44 48 52 56
32 36 40 44 48 52 56 58
36 40 44 48 52 56 58 60
40 44 48 52 56 58 60 62
44 48 52 56 58 60 62 62

[dragongodz]

Error: Not DVD compliant.
yep it isnt.

MAXBITRATE 9800000
muxing the M2V with AC3(2Streams)DTS(1Stream)
and thats why.

the 9.8Mb/s is max video bitrate for dvd specs but thats also if there is no audio or subs etc. add the total bitrates of the audio streams and minus them from the video max bitrate.

wait i hear you say, what about the 10.8Mb/s maximum ? that is the maximum of all streams muxxed together so includes muxxing overheads. easier to simply decrease the 9.8Mb/s by the extra streams bitrates. infact probably best to go slightly lower than that because the more streams the more overhead required aswell. :)

Rejig0.56: Compiled the full DVD.
yes Rejig isnt overly fussing compiling. mplex(which its based on) is generally over fussy and can complain about streams that are in spec so Rejig was made a little more forgiving. :)

[Mug Funky]

arch. never mind my post above - i had bitrate in kbps instead of bps. it's encoding now :)

[Amnon82]

@dragongodz: DVDLabPro show me the Bitrate as 4791 kbps of the 1pass CQ encode. and 5025 in the 2pass VBR encode.

The 2Pass VBR encode is DVD compliant.

[hank315]

Hi all,

ATM I'm very busy coding the GUI for HC which will be ready in about a week I think.

Maybe I should have waited to make the encoder public after writing a GUI but I wanted to know if the "community" liked the quality of the encoder.

After reading most messages some remarks:

- Please read the doc file and use only the commands in it.
- The *CQ command means a 1 pass encoding using Constant Quantization which may be not DVD compliant.
- Quantization used is nonlinear and can't be compared with QuEnc which uses linear q. scale code.

When the GUI is finished I think a lot of the problems will be solved so please be patient.

For the Linux guys:
With gcc I can compile Fortran77 stuff, it's converted into C and than compiled, not the best way to do it.
But HC is written in Fortran95 which gcc can't handle.
When the Windows GUI version is up and running I will try to create a version using the (experimental) g95 compiler, but I also read it runs using wine so all problems are already solved :D (nah, that's not the real thing).

About encoding speed:
I tweaked the settings in such a way that a (DVD backup) 2 hour movie should be done in 4 hours using the best settings and VBR 2 pass on my PC. I'm getting 15-20 fps in the first pass and about 50 fps in the second pass.
system: P4 3.2 Ghz Prescott which can use SSE3
But encoding interlaced DV stuff may take much longer especially in the first pass.

@Amnon82
I was wrong when I told you I hated GUI programming, I even start to like it, actually it's great fun;)
About the chapters, I tried that in previous versions but could not get it to work properly but it is certainly something worth to be implemented.

And this version will always be freeware (including the GUI version) but not open source.

[marcellus]

Hi hank,
I just 2 pass encoded a fragment and the result is very pleasant to watch.
Your encoder is pretty amazing. The quality is very high for a 0.01 release. For almost a year I use for my mpeg2 encodings only libavcodec based encoders because IMHO libavcodec have more quality than CCE especially at low bitrates (and my standalone is very forgiving with non compliant streams). Today I've seen a good competitor for libavcodec (with better and worse features). The most noticeable good feature for HC is, I don't know how to say, the "dithering", or "noising" of the areas in the picture where are blocks, making them less visible and less annoying. A good thing over libavcodec is, I think, the non-linear quantization.

On the other side why the HC encode has (acording to bitrate viewer) an average q of 8.87 with a peak of 11.92 and the one I made with ffdshow have 2.92 average q with a peak of 6.08? And I used the same q-matrix and of course, the same bitrate.
Still, HC is slower but not horribly slow.

Well done and I'm watching closely HC evolution.
Thanks for your hard work!

[hank315]

@Marcellus
Libavcodec encoders use a linear q scale, HC uses a non-linear scale.
That makes the difference in Bitrateviewer.

But ofcourse, a lot can be improved. At first the GUI should be completed which should it make easier for people to run HC.
After that I'll try to improve speed and quality.
It seems that HC runs pretty fast on CPU's which support SSE3 and have a large cache.

And about libavcodec, it is a very good encoder, I also used a lot but some older versions wouldn't run on my system or weren't stable. Also bitrate control (bitrate spikes) is still not optimal.
NuEnc does a good job but is IMHO very slow.

[Macanudo]

Does HC support High Definition resolutions?

As far as I know the only commercial mpeg2 encoder that supports HD is the newest version of MainConcept Mpeg Pro.

If HC doesn't already support HD and you are open to feature requests, this capability would be a great addition to your excellent encoder.

Thanks for the HC!!!

Macanudo

[dragongodz]

DVDLabPro show me the Bitrate as 4791 kbps of the 1pass CQ encode. and 5025 in the 2pass VBR encode.
Amnon82 - and i bet thats the average bitrate its reporting. this has nothing to do with what i was saying. its the peaks going over dvd spec thats causing underflows. when they go over the 10.08Mb/s (muxxed) they are out of dvd spec. some players(especially modern ones) will still play them by the way but thats not the point.

it is possible to do 1 pass VBV but its trickier. methods i have seen and read about include per MB rate control, per GOP(read and encode whole GOP in memory, if breaks VBV redo GOP in memory until it doesnt, then write that) etc. however doing a 1 pass constant quant doesnt do them.

And this version will always be freeware (including the GUI version) but not open source.
hank315 - it would still be nice to see a dll version though. ;)

Also bitrate control (bitrate spikes) is still not optimal.
understatement. its not not optimal, its plain bad. :)

NuEnc does a good job but is IMHO very slow.
and its a pity that Peter has found a home(it appears from comments made about his posting all the latest info there) on the kvcd forum. a closed forum most of us have no interest in paying to join.

[Amnon82]

I released a guied version of HC 0.01

Here is the screenshot:

http://www.brckomania.net/HCGUI/img/hcgui001.png

grap it here:

http://forum.doom9.org/showthread.php?s=&threadid=89101

I did some Logos:

Grap them here (http://www.brckomania.net/HCGUI/files/HC.Logos.rar)

[Mug Funky]

one question: is there a way to set field order?

[TEB]

@Macanudo: Procoder supports HD in mpeg2, all profiles.. even HDV.

[Ebobtron]

Originally posted by Mug Funky
one question: is there a way to set field order?
Why?

[The Edge]

I tried beta 0.01 on an AMD 1400 t-bird and it's says:

"No MMX/SSE detected"
"Enter to close program"

I'll give it a go later on my P4 laptop. Sounds promising hank315 :)

[Amnon82]

Did You enter MMX and SSE manually? Tryout my gui.

[The Edge]

Originally posted by Amnon82
Did You enter MMX and SSE manually? Tryout my gui.

I did...no joy.
This (http://members.boards.ie/the_edde/HC_Output.GIF) is the screen I get.


Originally posted by Amnon82
Tryout my gui.
Found it difficult to download....timeouts etc.
Anyway, exactly the same thing happens with GUI I'm afraid.

[hank315]

Add the next command:

*CPU MMX

This will force the CPU to use MMX and SSE, it will run (or crash if it is not supported;))

[The Edge]

I already tried that mate, like Amnon82 suggested.
It seems the default is already *CPU MMX when unzipped.
I've tried *CPU MMX & *CPU AUTO and the error still appears hank.
Also tried without an .ini too....same.
Is there anything else you would like me to try?

btw WinXP SP2 here.

[hank315]

@The Edge
Did some "googling" and found this:

http://www.hothardware.com/viewarticle.cfm?articleid=207&catid=1

It seems the Athlon Thunderbird has MMX support but no SSE support, SSE was available with the introduction of the Palomino core.
But I'm not a hardware expert, maybe there's someone who knows...

Just try it on your P4 laptop, should certainly run on that one.

[The Edge]

Your spot on Hank....T-Bird only has MMX & 3DNow!
Not to worry mate. Maybe it can be looked at some other time. ;)

*Fires up P4 lappy* :D

[Amnon82]

Maybe You must have MMX+SSE as minimum. Or I'm wrong, hank?

[dragongodz]

It seems the Athlon Thunderbird has MMX support but no SSE support, SSE was available with the introduction of the Palomino core.
Your spot on Hank....T-Bird only has MMX & 3DNow!
well close enough anyway. :)
all athlons and durons support ISSE but not full SSE until palomino(athlon) and morgan(duron) cores.

some ISSE information here
http://www.avisynth.org/IntegerSSE

[hank315]

Then it could work because the MMX routines only use MMX and integer SSE, only my CPU testing is clearly too severe.
I'm testing bit 23 (MMX) and bit 25 (SSE), is there somebody who knows how to test for the MMX / ISSE combination?

[ARDA]

Look in cpuaccel.cpp in avisynth sources.
But if I'm not wrong ISSE is an MMX extension of some Amd machines.

from:
http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/22466.pdf

"This chapter describes 19 new instructions added to the MMX
instruction set defined in the AMD-K6® MMX™ Enhanced
Processor Multimedia Technology Manual, order# 20726. Twelve
of the instructions improve multimedia-enhanced integer math
calculations used in such applications as speech recognition
and high-quality video processing. Seven instructions are
dedicated to efficiently moving multimedia data into and out of
the processor.
Programmers should check bit 22 of the Extended Feature
Flags in the EDX register after executing extended function
8000_0001h of the CPUID instruction. If bit 22 is set, the AMD
processor supports these 19 instructions. See the AMD Processor
Recognition Application Note, order# 20734 for more
information.
Instruction definitions are in alphabetical order according to
the instruction mnemonics."

http://www.amd.com/epd/processors/6.32bitproc/8.amdk6fami/x23913/23913a.pdf

If an AMD machine MMX-SSE Bit 22
If a Cyrix machine MMX-SSE Bit 24

What means that a SSE machine has also this extended feature.

I hope this can be usefull.
ARDA

[dragongodz]

how to test for the MMX / ISSE combination?
you cant test for ISSE itself. the SSE check fails since its not full SSE so the only way would be to check if the cpu is an athlon or duron since you already know they do support ISSE.
http://www.paradicesoftware.com/specs/cpuid/index.htm

But if I'm not wrong ISSE is an MMX extension of some Amd machines.
and what do you think SSE is ? it is an extension of MMX, an extra 70 instructions to be exact along with the original MMX instructions. infact it was originally called MMX-2. ISSE is just part of it that AMD include with 3dnow enhanced while 3dnow pro got full SSE.

[The Edge]

Originally posted by dragongodz
well close enough anyway. :)
all athlons and durons support ISSE but not full SSE until palomino(athlon) and morgan(duron) cores.

some ISSE information here
http://www.avisynth.org/IntegerSSE

Thanks for that dragongodz...so much to learn, only one lifetime!:)

[vhelp]

@ hank315

Three questions ...


May I ask a dumb question ? ...

Is part of your goal, to finaly fix the time-old issue with bitrate spikes.. ie, if
one set a MIN: 4000 and MAX: 8000 bitrate (any mode ie, CBR/VBR/CQ etc)
that you will *KEEP IT* with-in that bitrate range ??
.
I think that this is something that no one has ben able to accomplish. If you
could defeat this one, then you could (theoretically) be the ONE !!
.. Matrix stuff ..

Could you call your new encoder something like, HCenc or HCEnc.
if I'm not being too forward, that is.., cause it could be refered to as such.

.. For instance, I often refer to another pupular encoder, TMPG or TMPEenc.
.. over here and vcdhelp (I'm sure you know them) :!: I got my start in Video
.. productions from there.. and a little here, and other places too, to mention
.. a few.

Another question ...

What source(s) did you come across and read to learn all there is to know** about
and finally, resulting in the development of the MEAT (or building blocks) for
*your* MPEG engine ( I realize you are still learning** ) ???
I'm curious :rolleyes:

I'm with NIC on this. Could you code a DLL for us to use (Please) ??
I would love to code my own GUI shell (just for me, for when I'm debuggin certain
encoding scenarios, it would be easier to make the tool behave in such a way that
would be in favor of the coder - me. Hope that made sense) to play with..
(and test up against TMPG) - right away - thanks.

Can you keep your latest updates (Downloads) on the first page here ??
That would be great.

Anyways, I commend you with great success on your MPEG endeavor.., don't give
up.., - - look at TMPGenc 's success.
.
I encurage everyone in the things of developing :)

Thank you,
-vhelp

[freelock7]

Tested a movie encoding with success (2Pass, VBR=5000; max=8500).
Runs correctly with DVD player but the texture quality is poor compared to QuEnc0.59B2 (even at 5000kbs).
But i understand that this encoder is a "baby" and it has to learn more and more...
Don't blame me Hank. Your job is great.:)

[hank315]

@Freelock7
Strange, I used QuEnc 0.59 to do a lot of comparisons with my encoder with different (DVD) scenes.
Most of the time I had to go down in bitrate to 2000-3000 kb/s to see real difference doing a frame-frame comparison.

[video_magic]

Hello, I tried HC today but It stuck on frame 30 when trying to do pass 2

Here are the details:

Source video 35.3gb Huffyuv compressed AVI (1 hour 5 minutes) fed through AVS file as so:

OpenDMLSource("D:\capvid\matwed3.avi", FALSE, pixel_type="YUY2", fourCC="HFYU")
BilinearResize(720,576)
ConvertToYV12()

my HC.ini is:

*ASPECT 3:4
*BITRATE 6000000
*CLOSEDGOPS
SPACE *CQ
*DC_PREC 10
*GOP 4 0
*INFILE D:\capvid\matwedbilresconvyv12.avs
SPACE *INTERLACED
*MAXBITRATE 7200000
*NOSCD
*OUTFILE D:\capvid\hankwed.m2v
*PROFILE BEST

I have Windows XP pro , and more than enough free hard disk space formatted with NTFS. Pentium 3 @ 1.1 GHZ (not overclocked or anything).

Pass 1 goes seemingly okay, at around 3 FPS. It does the little quick scan afterwards and then begins pass 2 but gets to frame 30 and stops there with the frmae number flickering back and forth between frames 28 to 31 constantly. I leave it like that for over an hour but eventually did CTRL + C to abort :( here is a screen shot
http://www.geocities.com/amazed_leaflet/hcstuckpass2.gif

So, I checked the output folder and there is a little m2v of only a couple hundred KBs. In my HC folder there is HC.dbs of 921 MBs

(EDIT: is a space before CQ and INTERLACED but the space did not show up in my post :rolleyes: )

[video_magic]

Oh yeah, thought I should pot the contents of 'out.log' :)
(the formatting will probably be lost in my post though)

-----------------------------------------------------------
------------------> HC - MPEG2 encoder <-------------------
-----------------------------------------------------------
release: beta 0.00
-----------------------------------------------------------
---------------------> parsing stats <---------------------
-----------------------------------------------------------
input: D:\CAPVID\MATWEDBILRESCONVYV12.AVS
output: D:\capvid\hankwed.m2v
frames: 1 - end
aspect ratio: 3:4
bitrate: 6000000
max. bitrate: 7200000
profile: BEST
closed gops: yes
scene change det.: no
interlaced: no
goplen,B-pic: 4 0
dc_precision: 10
-----------------------------------------------------------
--------------------> encoder settings <-------------------
-----------------------------------------------------------
input: D:\CAPVID\MATWEDBILRESCONVYV12.AVS
output: D:\capvid\hankwed.m2v
output: out.log
profile: BEST
frames: 1- end
framerate: 25.00
aspect ratio: 3:4
bitrate: 6000000
max. bitrate: 7200000
restart: no
closed gops: yes
VBV check: yes
scene change det.: no
interlaced: no
goplen,B-pic: 4 0
dc_precision: 10
scan method: ZIGZAG
CPU: AUTO
-----------------------------------------------------------
---------------------> source stats <----------------------
-----------------------------------------------------------
nr. of frames in source: 98120
width*height: 720*576
fps: 25.00
nr. of frames to encode: 98120
frames to encode: 1 - 98120
movie length to encode: 3924.80 s
est. outfile length: 2874609 kB
-----------------------------------------------------------
-------------------> encoding - pass 1 <-------------------
-----------------------------------------------------------
pass 1 encoding time: 43464.2 s
average fps: 2.3
-----------------------------------------------------------
--------------> encoding - intermediate pass <-------------
-----------------------------------------------------------
bitrate set to: 6000000 b/s
est. outfile length: 2874609 kB
intermediate encoding time: 2.8 s
-----------------------------------------------------------
-------------------> encoding - pass 2 <-------------------
-----------------------------------------------------------


The version of HC.EXE which I am using is the one modified on the 28th which I thought is the latest available :) Thanks for any advice.

[hank315]

Is part of your goal, to finaly fix the time-old issue with bitrate spikes.. ie, if
one set a MIN: 4000 and MAX: 8000 bitrate (any mode ie, CBR/VBR/CQ etc)
that you will *KEEP IT* with-in that bitrate range ??It depends what your definition of a bitrate spike is, what is your time frame to look at?
The bitrate in HC is controlled per GOP and the average bitrate per GOP will never be larger than the MAXBITRATE value.
Keeping the bitrate at a minimum is a different thing, when there's nothing to quantize you can do some tricks to keep the bitrate as high as possible but even if that's not enough it can't be done (ofcourse padding can be used but that's really a waste of bits).

What source(s) did you come across and read to learn all there is to know** about
and finally, resulting in the development of the MEAT (or building blocks) for
*your* MPEG engine ( I realize you are still learning** ) ???Main source was ISO/IEC 13818-2 which I have read at least a dozen times and some parts even more...
Also got a lot of information here at Doom9 and then a lot of coding and trying and be very persistent and patient.
Soon I realized plain Fortran coding wasn't fast enough so i wrote an assembler library with all the MMX/SSE/SSE2/SSE3 stuff which handles almost everything: motion vector search, dct stuff (some parts of it are taken from skals FDCT/IDCT implementation), quantization and handling data blocks. And yes, I'm still learning...

I'm with NIC on this. Could you code a DLL for us to use (Please) ??
I would love to code my own GUI shell (just for me, for when I'm debuggin certain
encoding scenarios, it would be easier to make the tool behave in such a way that
would be in favor of the coder - me. Hope that made sense) to play with..That could (maybe) be done. Will think about it.
I also foresee some complications with the difference in data structure between Fortran and C...

Can you keep your latest updates (Downloads) on the first page here ??
That would be great.No problem, next update will be the GUI-release, when it is ready I will add it to my first post.

[hank315]

@Video Magic
I will try to reproduce this with some DV content and the settings you are using, let you know tomorrow.

BTW. using no B-frames and such a short GOP isn't the best setting IMHO.

Try to use the *RESTART command and set *MAXBITRATE to 9500000.

[vhelp]

@ hank315

FWIW, I'm under Windows 98 Gold, and I cannot open and read your
HC.DOC file. My wordpad crashes with a General Protection Fault
error, and warns me of the file in question:

mswrd832.cnv

My guess is this. You composed the document under Word in Office 2000 ??

I only have Office 97 and still can't open it.
-vhelp

[hank315]

I will create a PDF of it, stupid of me, should have thought about that. Will add it tomorrow to the first post.

(Would be nice if those MS products were a little bit more compatible :))

[dragongodz]

It depends what your definition of a bitrate spike is, what is your time frame to look at?
The bitrate in HC is controlled per GOP and the average bitrate per GOP will never be larger than the MAXBITRATE value.
the time frame is defined in the format/output, eg. for dvd it says 9.8Mbits per second maximum for video(that maximum as i have already said is only really usable if there is no audio etc aswell). so if using GOP then the target should be lower for the GOP and resultant bit amount remaining for the second should be carried forward to influence the next GOP. still you could probably ,most of the time, get away with just lowering the target per GOP to be the percentage of a second that the amount of frames in the GOP is. if you follow what i mean. :D

ofcourse padding can be used but that's really a waste of bits
a waste true but if the minimum bitrate has to have any meaning then it may be required. if you need to do CBR encoding for example, or if your hardware target doesnt play correctly without a set minimum(as some dvd players have been reported to do).

[Amnon82]

@vHelp: Here is a complete release of 0.01.

I created the manual in sxw, txt, rtf, pdf, doc.
... hope it helps You.

http://www.brckomania.net/HCGUI/files/HC001.zip

@hank: Great to hear that the next version will be guied.

[Guest]

Hank, you mentioned communication with DGDecode. Just out of curiosity, why would an encoder need to communicate with a decoder? :)

[freelock7]

@Hank
It's about the smoothness effect in a picture.
The global texture quality in a frame can be wrong if the encoder don't work correctly.
Dark area, flat background, sea, wall can be encoded with noise (macrobloc).
But it's hard to see on a TV screen.
Using a median filter may help to correct it-but I used the same avs script to do comparison.
HD seems to fail in this particular frames but strangely, it works very well in high motion scene (explosion,speed...).
I want to correct what I said before about quantization.
I encoded a movie with a VBR=5000kbs and an average quant=4.6 (0.01beta version).
The global quantization is very good here (my previous stats used the first limited encoder version ).
There is a difference with this one?
So, can you add with your updates an history log ?

[Amnon82]

@neuron2: HC needs DGDecode, cos it has no complette MPEG-Decoder build in. It will be used for the D2V-Files.

[hank315]

@neuron2
communication is maybe a big word for it, it's more one way traffic;)
It's just used to deliver the YUV planes.

still you could probably ,most of the time, get away with just lowering the target per GOP to be the percentage of a second that the amount of frames in the GOP is. if you follow what i mean.That's exactly the way it is done.

a waste true but if the minimum bitrate has to have any meaning then it may be required. if you need to do CBR encoding for example, or if your hardware target doesnt play correctly without a set minimum(as some dvd players have been reported to do).ATM I use a minimum value of 400 kb/s for correct DVD-playback and yes, for encoding a sequence of black frames it will use padding if all other things are not enough (lowering Q, no MB skipping in PB frames, coding all MB's intra). This is ofcourse also done per GOP so the bitrate per GOP will be between 400 kb/s and the value set for MAXBITRATE.

[TEB]

@Hank & Neuron2: Wouldnt this be a good oppertunity to make a perfect OFF-line transcoder (full renenc), where one could serve og send from DGindex the correct demulitplexed data + optional filters inbetween and then piping it into hanks encoder.. Couldnt this be more efficient as an alternative route than doing everything through avisynth?
Lets say, yv12 decode(dgindex) - yv12 encode(encode + multiplex) ?
Hank, are u thinking about doing multiplexing in the encoder? or do we need 3rd step = multiplexer with support for ES/PS/TS ?

keep up the excelent work guys! (im getting a worm woozie feeling of all this good teamwork these days!!)

[Mug Funky]

vhelp: try openoffice. works the same and will open it.

however, i don't think the .doc format has changed since 97. it _should_ be backward compatible (though with MS that's no guarantee it _is_ backward compatible).

[dragongodz]

hank315 - sorry you didnt make it clear the per GOP was a percentage of max bitrate target before. you just said a GOP would not exceed max bitrate. ;)

[video_magic]

Hank, I tried doing another encode with different settings but it got stuck once again around frame 30 at pass 2. :(
Here are the details: I changed the ini to let the encoder choose more how to distribute bitrate and use B-Frames

*ASPECT 3:4
*BITRATE 6200000
*CLOSEDGOPS
*DC_PREC 10
*INFILE D:\capvid\matwedbilresconvyv12.avs
<space>*INTERLACED
*MAXBITRATE 7400000
*OUTFILE D:\capvid\hankwed.m2v
*PROFILE BEST

Here's the 'Dos Window' contents:

C:\asblah\hc>hc.exe
-----------------------------------------------------------
------------------> HC - MPEG2 encoder <-------------------
-----------------------------------------------------------
release: beta 0.00
-----------------------------------------------------------
--------------------> CPU extensions <---------------------
-----------------------------------------------------------
CPU detected: GenuineIntel
MMX/SSE support: YES
SSE2 support: NO
SSE3 support: NO
-----------------------------------------------------------
--------------------> parsing HC.ini <---------------------
-----------------------------------------------------------
line 1 *ASPECT 3:4
line 2 *BITRATE 6200000
line 3 *CLOSEDGOPS
line 4 *DC_PREC 10
line 5 *INFILE D:\capvid\matwedbilresconvyv12.avs
line 7 *MAXBITRATE 7400000
line 8 *OUTFILE D:\capvid\hankwed.m2v
line 9 *PROFILE BEST
-----------------------------------------------------------
---------------------> parsing stats <---------------------
-----------------------------------------------------------
input: D:\capvid\matwedbilresconvyv12.avs
output: D:\capvid\hankwed.m2v
aspect ratio: 3:4
bitrate: 6200000
max. bitrate: 7400000
profile: BEST
closed gops: yes
dc_precision: 10
-----------------------------------------------------------
--------------------> encoder settings <-------------------
-----------------------------------------------------------
input: D:\capvid\matwedbilresconvyv12.avs
output: D:\capvid\hankwed.m2v
log file: out.log
profile: BEST
frames: 1 - end
aspect ratio: 3:4
bitrate: 6200000
max. bitrate: 7400000
restart: no
closed gops: yes
VBV check: yes
scene change det.: yes
interlaced: no
goplen,B-pic: AUTO
dc_precision: 10
scan method: ZIGZAG
CPU: AUTO
-----------------------------------------------------------
---------------------> source stats <----------------------
-----------------------------------------------------------
nr. of frames in source: 98120
width*height: 720*576
fps: 25.00
-----------------------------------------------------------
nr. of frames to encode: 98120
frames to encode: 1 - 98120
movie length to encode: 3924.80 s
est. outfile length: 2970430 kB
<ENTER> to encode or abort
-----------------------------------------------------------
-------------------> encoding - pass 1 <-------------------
-----------------------------------------------------------
pass 1 encoding time: 55718.3 s
average fps: 1.8
-----------------------------------------------------------
--------------> encoding - intermediate pass <-------------
-----------------------------------------------------------
optimizing picture lengths
bitrate set to: 6200000 b/s
est. outfile length: 2970430 kB
intermediate encoding time: 2.9 s
-----------------------------------------------------------
-------------------> encoding - pass 2 <-------------------
-----------------------------------------------------------
forrtl: error (200): program aborting due to control-C event

C:\asblah\hc>

[hank315]

@Video Magic
Did a test using the testclip from the BitrateViewer site with the next settings:

*PROFILE BEST
*INFILE test.avs
*OUTFILE out.m2v
*LOGFILE out.log
*BITRATE 6000000
*MAXBITRATE 7400000
*ASPECT 3:4
*CLOSEDGOPS
*DC_PREC 10
*GOP 4 0

This a real nasty clip, a part of it was used by Gleitz for his encoder test.
And this what came out:

-----------------------------------------------------------
---------------------> source stats <----------------------
-----------------------------------------------------------
nr. of frames in source: 567
width*height: 720*576
fps: 25.00
nr. of frames to encode: 567
frames to encode: 1 - 567
movie length to encode: 22.68 s
est. outfile length: 16611 kB
-----------------------------------------------------------
-------------------> encoding - pass 1 <-------------------
-----------------------------------------------------------
pass 1 encoding time: 66.9 s
average fps: 8.5
-----------------------------------------------------------
--------------> encoding - intermediate pass <-------------
-----------------------------------------------------------
bitrate set to: 6000000 b/s
est. outfile length: 16611 kB
intermediate encoding time: 0.0 s
-----------------------------------------------------------
-------------------> encoding - pass 2 <-------------------
-----------------------------------------------------------
pass 2 encoding time: 25.1 s
average fps: 22.6
-----------------------------------------------------------
-------------------> encoding finished <-------------------
-----------------------------------------------------------
total encoding time: 92.2 s
-----------------------------------------------------------
--------------------> encoding stats <---------------------
-----------------------------------------------------------
intra matrix used
8 16 19 22 26 27 29 34
16 16 22 24 27 29 34 37
19 22 26 27 29 34 34 38
22 22 26 27 29 34 37 40
22 26 27 29 32 35 40 48
26 27 29 32 35 40 48 58
26 27 29 34 38 46 56 69
27 29 35 38 46 56 69 83

non-intra matrix used
16 17 18 19 20 21 22 23
17 18 19 20 21 22 23 24
18 19 20 21 22 23 24 25
19 20 21 22 23 24 26 27
20 21 22 23 25 26 27 28
21 22 23 24 26 27 28 30
22 23 24 26 27 28 30 31
23 24 25 27 28 30 31 33

nr. of gops: 143
nr. of frames: 567
nr. of I-frames: 143
nr. of P-frames: 424
nr. of B-frames: 0
average quant: 10.674 (non-linear scale)
VBV underflows detected: 0
VBV underflows fixed: 0

GOPS looked like this: http://hank315.dyndns.org/video_magic.jpg

But something came to my mind, I also had one clip which stopped encoding after about 150 frames. It turned out Avisynth stopped serving frames, it didn't came out of the GetFrame function, CPU usage was 0%. I think your problem is a similar one.
Probably it will have something to do with the way I'm calling the Avisynth functions.
If I have the GUI version up and running I will ask the C-experts in the development forum if they have some ideas to solve this.

Would you do one thing for me please, test it once again with the next settings:

*PROFILE NORMAL
*FRAMES 1 1000
*INFILE yourfile.avs
*OUTFILE out.m2v
*LOGFILE out.log
*BITRATE 6000000
*MAXBITRATE 7400000
*ASPECT 3:4

This will run pretty fast (only 1000 frames).
If it stops encoding please look at your CPU usage and tell me.

[video_magic]

am doing this now :)

[video_magic]

Ok! Exactly the same thing - it has stuck on frame '30' (it flickers back and forth around frame 30)CPU usage constantly on 100%

I mean pass 1 went fine, but on pass 2 it sticks on frame 30. Pass 1 when encoding the CPU usage was around 90 - 94%, occasionally up to 100% but have other tasks I'm doing

[manolito]

I just wrote a small utility that gives HC a QuEnc compatible command line interface. This means that applications that can use QuEnc (like DVD-RB or DVD2SVCD) can now also use HC.

Download at
http://scifi.pages.at/manolito/HC_CLI/HC_CLI.zip

Cheers
manolito

[dragongodz]

video_magic - try using
DirectshowSource("D:\capvid\matwed3.avi")
instead of
OpenDMLSource("D:\capvid\matwed3.avi", FALSE, pixel_type="YUY2", fourCC="HFYU")
and see if it happens then aswell.

[video_magic]

Ok , tried that, now it gets stuck on pass 2, flicking between frames 24 & 25 with CPU showing at 100% usage :p

I will point out though that my computer doesn't feel 'locked up' or even very slow doing normal tasks, so that's a bit weird that it shows 100% usage.

[dragongodz]

hmm ok what happens if you feed the avs in to QuEnc ? does it get stuck aswell ?

if yes then that suggests its avisynth or the videos decoder thats the problem. if no then that suggests its HC thats having the problem.

if you can cut it to say 50 frames(stream both audio and video in virtualdub for example) and still get the same error then could you provide a link to that ? that way we could test the original footage and see whats wrong.

[video_magic]

I have previously used exactly the same avs in Nuenc 0.01b (opendmlsource) with 3 passes and it worked fine it did encode all the passes.

There is something that has crossed my mind about all these multi-pass encoders which I will throw out here.

You know Windows has a lot of flaws?
Well, sometimes I think Windows might try and open files before they have been written and closed properly, I know there is some kind of write-caching that takes place and there are cases of data loss like when Windows closes down very quickly. At the end of a pass I wonder if encoding applications should have a pause (of maybe one minute?) before beginning the next pass. Just to make sure the stats file has been written and closed and also perhaps the source video file is ready to be opened again, and maybe The Windows Page file would be emptied.

:o :scared: I don't know much about programming but I wondered about whether this would help?

[video_magic]

Okay, I just tried with a clip that is only the first 50 frames.

It did the same thing - gets stuck on pass 2 flickering around frame 30 and so not finishing the pass.

using avs script:

OpenDMLSource("D:\capvid\matwed1st50frames.avi", FALSE, pixel_type="YUY2", fourCC="HFYU")
BilinearResize(720,576)
ConvertToYV12()

and using hc.ini:

*ASPECT 3:4
*BITRATE 6200000
*CLOSEDGOPS
*DC_PREC 10
*INFILE D:\capvid\matwedbilresconvyv121st50frames.avs
<space>*INTERLACED
*MAXBITRATE 7400000
*OUTFILE D:\capvid\hankwed.m2v
*PROFILE BEST

Here is a screen shot
http://www.geocities.com/amazed_leaflet/1st50stuck.gif

And the cpu still showed 100% while it is stuck, but the computer is only a little slower than normal - not locked up or anything

[video_magic]

OK! , I then tried *RESTART with maxbitrate set to 9500000 (everything else the same), but it still doesn't progress beyond frame 30.

[dragongodz]

well 1 last thing you can try. use Bframes(i know i read back there somewhere you didnt :) ) and turn off scene change detection.

if that still fails then hank315 would have to do a debug version to try and catch exactly is going wrong.

[hank315]

if that still fails then hank315 would have to do a debug version to try and catch exactly is going wrong.Also think that's a good idea, will do that after I finished the GUI version (1 - 2 days). I'm planning to create a special version that will write the first 50 frames as bitmaps to see if they come OK out of Avisynth, and then output some parameters during the encoding session. I'm really curious what causes it, scanned the source but couldn't find anything...

@Video Magic
Would be nice if you want to try it, if so, it will be ready in a couple of days, please let me know, really want to have this bug solved.

Haven't looked here very often because I'm *very* busy coding the GUI version which is almost finished.

Vhelp couldn't open the .doc file, just put a .pdf online
get it at http://hank315.dyndns.org/HC.pdf

[video_magic]

Think my last attempt should be using B-Frames anyway

But now I have tried my '1st 50 frames' clip again with specifying B-frames and No Scene detection :) as follows with this HC.ini :

*ASPECT 3:4
*BITRATE 6200000
*GOP 12 2
*NOSCD
*DC_PREC 10
*INFILE D:\capvid\matwedbilresconvyv121st50frames.avs
<space>*INTERLACED
*MAXBITRATE 9500000
*OUTFILE D:\capvid\hankwed.m2v
*PROFILE BEST

Now, as usual first pass goes fine, 2nd pass it 'stops' around frame 30 (with it flickering on different frame numbers as always) and this time it looked like it's flickering back and forth over a wider range - from 27 - 36!

But suddenly it got past this! and I got this message - well here is the whole dos window


C:\asblah\hc>hc.exe
-----------------------------------------------------------
------------------> HC - MPEG2 encoder <-------------------
-----------------------------------------------------------
release: beta 0.00
-----------------------------------------------------------
--------------------> CPU extensions <---------------------
-----------------------------------------------------------
CPU detected: GenuineIntel
MMX/SSE support: YES
SSE2 support: NO
SSE3 support: NO
-----------------------------------------------------------
--------------------> parsing HC.ini <---------------------
-----------------------------------------------------------
line 1 *ASPECT 3:4
line 2 *BITRATE 6200000
line 3 *GOP 12 2
line 4 *NOSCD
line 5 *DC_PREC 10
line 6 *INFILE D:\capvid\matwedbilresconvyv121st50frames.avs
line 8 *MAXBITRATE 9500000
line 9 *OUTFILE D:\capvid\hankwed.m2v
line 10 *PROFILE BEST
-----------------------------------------------------------
---------------------> parsing stats <---------------------
-----------------------------------------------------------
input: D:\capvid\matwedbilresconvyv121st50frames.avs
output: D:\capvid\hankwed.m2v
aspect ratio: 3:4
bitrate: 6200000
max. bitrate: 9500000
profile: BEST
scene change det.: no
goplen,B-pic: 12 2
dc_precision: 10
-----------------------------------------------------------
--------------------> encoder settings <-------------------
-----------------------------------------------------------
input: D:\capvid\matwedbilresconvyv121st50frames.avs
output: D:\capvid\hankwed.m2v
log file: out.log
profile: BEST
frames: 1 - end
aspect ratio: 3:4
bitrate: 6200000
max. bitrate: 9500000
restart: no
closed gops: no
VBV check: yes
scene change det.: no
interlaced: no
goplen,B-pic: 12 2
dc_precision: 10
scan method: ZIGZAG
CPU: AUTO
-----------------------------------------------------------
---------------------> source stats <----------------------
-----------------------------------------------------------
nr. of frames in source: 50
width*height: 720*576
fps: 25.00
-----------------------------------------------------------
nr. of frames to encode: 50
frames to encode: 1 - 50
movie length to encode: 2.00 s
est. outfile length: 1514 kB
<ENTER> to encode or abort
-----------------------------------------------------------
-------------------> encoding - pass 1 <-------------------
-----------------------------------------------------------
pass 1 encoding time: 9.2 s
average fps: 5.4
-----------------------------------------------------------
--------------> encoding - intermediate pass <-------------
-----------------------------------------------------------
optimizing picture lengths
bitrate set to: 6199999 b/s
est. outfile length: 1514 kB
intermediate encoding time: 0.2 s
-----------------------------------------------------------
-------------------> encoding - pass 2 <-------------------
-----------------------------------------------------------
pass 2 encoding time: 18.1 s
average fps: 2.8
-----------------------------------------------------------
-------------------> encoding finished <-------------------
-----------------------------------------------------------
total encoding time: 27.7 s
VBV underflows detected: 24
VBV underflows fixed: 24
<ENTER> to close program...





:) So it looked like it actually finished and encoded the clip

[video_magic]

Oh yeah, just thought to tell you as it might help track it down. The beginning of my movie is almost 'pure blue' because it is the beginning of my camcorder tape before the picture kicks in, maybe this affects the ability to achive target bitrate?

Also, as pass 1 had always worked beofre, but pass two would use the database file, I thought maybe it was a problem with something that was written in the database file, which it couldn't process as maybe it didn't add up with what it encounters around the problem frames on pass 2?

Anyway, I am happy to give reports, and thankyou for your work on this.

Ooops, I forgot I should probably have copied out.log so here is the end of it:

nr. of gops: 5
nr. of frames: 50
nr. of I-frames: 5
nr. of P-frames: 13
nr. of B-frames: 32
average quant: 7.060 (non-linear scale)
VBV underflows detected: 24
VBV underflows fixed: 24

[dragongodz]

VBV underflows detected: 24
VBV underflows fixed: 24
thats what i suspected and why i said to make those changes. its the rate control trying to fix the underflows. :)

The beginning of my movie is almost 'pure blue' because it is the beginning of my camcorder tape before the picture kicks in, maybe this affects the ability to achive target bitrate?
strange. a pure single colour should be easier to compress not harder. is there lots of noise or any other disturbance in that footage ?

[video_magic]

For this '1st 50 frames' clip, Well you know 'tracking' that you get on video tapes. This can look like horizontal streaks and flecks of black and white. There is some of this, and the last couple of frames are pure 'snow' like a blank video has when you play it.

[video_magic]

Okay, well a series of reports here. I checked up on this 'noise' thing. I opened my 1st50frames clip in Virtualdub and I held down SHIFT so I can go frame by frame.

Indeed the frames look mostly almost 100% pure (same) blue as far as I can see with a few small horizontal lines of white.

But there is a 'snow' sequence from frames 23/24/25 - 30! This is where there is a sheet of white horizontal lines all over the blue covering most of it on these frames.

Now, I can seek every frame one at a time with SHIFT, except frames 23 & 24, 26, & 28 and 50.

I scanned video stream for errors and it reports: 0 frames masked (0 frames bad, 0 frames good but undecodable) , so apparantly no problems with the video, and indeed I then chose save image sequence and it outputted 50 bmps. However!, I am pretty sure after examining these, that although Virtualdub claims no problems with decoding, it looks like it duplicated the frames with the adjacent for those which it would not seek to when I had been holding SHIFT.


Maybe then this is what is causing the problem for your encoder somehow (or for avisynth?) . Hope that is helpful!

[dragongodz]

there is a 'snow' sequence from frames 23/24/25 - 30! This is where there is a sheet of white horizontal lines all over the blue covering most of it on these frames.
yes this would require a lot more work by the encoder to fix underflows.

also with scene change detection it could easily trigger practically every frame to be a scene change. so a limit to minimum I frame distance would help here(i dont know if HC does). so you wouldnt get IIII but something like IBBI as minimum, which would also help with lower bitrate encodes.

[video_magic]

I just tried to track the cause of it down! :devil:

Here is another one that also gets stuck on pass two at the around frame 30 point. This seems to indicate that only disabling scene detection doesn't fix it. Notice I went back to try without B Frames again, and lowered Maxbitrate to 8,000,000 (because I would have 48k stereo wave audio to mux with it). I also am trying with opendmlsource in the avs.

Avs is:

OpenDMLSource("D:\capvid\matwed1st50frames.avi", FALSE, pixel_type="YUY2", fourCC="HFYU")
BilinearResize(720,576)
ConvertToYV12()

-----

HC ini is:

*ASPECT 3:4
*BITRATE 7500000
*CLOSEDGOPS
*GOP 4 0
*NOSCD
*DC_PREC 10
*INFILE D:\capvid\matwedbilresconvyv121st50frames.avs
<space>*INTERLACED
*MAXBITRATE 8000000
*OUTFILE D:\capvid\hankwed.m2v
*PROFILE BEST

Then I tried exactly the same but enabling one B frame. It still gets stuck on frame 30, pass 2.

THEN I TRIED EXACTLY THE SAME BUT GOP 8 0 AND THIS WORKED!!! :)


C:\asblah\hc>hc.exe
-----------------------------------------------------------
------------------> HC - MPEG2 encoder <-------------------
-----------------------------------------------------------
release: beta 0.00
-----------------------------------------------------------
--------------------> CPU extensions <---------------------
-----------------------------------------------------------
CPU detected: GenuineIntel
MMX/SSE support: YES
SSE2 support: NO
SSE3 support: NO
-----------------------------------------------------------
--------------------> parsing HC.ini <---------------------
-----------------------------------------------------------
line 1 *ASPECT 3:4
line 2 *BITRATE 7500000
line 3 *CLOSEDGOPS
line 4 *GOP 8 0
line 5 *NOSCD
line 6 *DC_PREC 10
line 7 *INFILE D:\capvid\matwedbilresconvyv121st50frames.avs
line 9 *MAXBITRATE 8000000
line 10 *OUTFILE D:\capvid\hankwed.m2v
line 11 *PROFILE BEST
-----------------------------------------------------------
---------------------> parsing stats <---------------------
-----------------------------------------------------------
input: D:\capvid\matwedbilresconvyv121st50frames.avs
output: D:\capvid\hankwed.m2v
aspect ratio: 3:4
bitrate: 7500000
max. bitrate: 8000000
profile: BEST
closed gops: yes
scene change det.: no
goplen,B-pic: 8 0
dc_precision: 10
-----------------------------------------------------------
--------------------> encoder settings <-------------------
-----------------------------------------------------------
input: D:\capvid\matwedbilresconvyv121st50frames.avs
output: D:\capvid\hankwed.m2v
log file: out.log
profile: BEST
frames: 1 - end
aspect ratio: 3:4
bitrate: 7500000
max. bitrate: 8000000
restart: no
closed gops: yes
VBV check: yes
scene change det.: no
interlaced: no
goplen,B-pic: 8 0
dc_precision: 10
scan method: ZIGZAG
CPU: AUTO
-----------------------------------------------------------
---------------------> source stats <----------------------
-----------------------------------------------------------
nr. of frames in source: 50
width*height: 720*576
fps: 25.00
-----------------------------------------------------------
nr. of frames to encode: 50
frames to encode: 1 - 50
movie length to encode: 2.00 s
est. outfile length: 1831 kB
<ENTER> to encode or abort
-----------------------------------------------------------
-------------------> encoding - pass 1 <-------------------
-----------------------------------------------------------
pass 1 encoding time: 9.5 s
average fps: 5.3
-----------------------------------------------------------
--------------> encoding - intermediate pass <-------------
-----------------------------------------------------------
optimizing picture lengths
bitrate set to: 7500000 b/s
est. outfile length: 1831 kB
intermediate encoding time: 0.2 s
-----------------------------------------------------------
-------------------> encoding - pass 2 <-------------------
-----------------------------------------------------------
pass 2 encoding time: 7.8 s
average fps: 6.4
-----------------------------------------------------------
-------------------> encoding finished <-------------------
-----------------------------------------------------------
total encoding time: 17.7 s
VBV underflows detected: 0
VBV underflows fixed: 0
<ENTER> to close program...

[video_magic]

exactly the same but *GOP 5 0 also works!

-----------------------------------------------------------
-------------------> encoding finished <-------------------
-----------------------------------------------------------
total encoding time: 13.6 s
VBV underflows detected: 0
VBV underflows fixed: 0
<ENTER> to close program...

So it seems the trigger with *NOSCD is having a GOP length of 5 or more, with or without B-Frames, then it works :)

[dragongodz]

you have to realise that having a GOP length of 4(IPPPIPPP) is really putting pressure on the encoder aswell for VBV.

try turning off VBV checking (*NOVBV) and see if it passes that section with a GOP size of 4. i am betting it will. if it does then that will show again that it is the rate control and its getting stuck trying to reduce the last frame in the GOP low enough.

possible solutions hank315 can do if so.
1. harder/faster/earlier frame size reduction for VBV(i dont know when/how hank315 decides to start reducing frames or by how much etc, thats closed source for you ;) ). this can slightly reduce quality of course.
2. if last GOP cant be VBV corrected after a certain amount of attempts or the reduction limit is reached(making the whole frame skipped MB's is the smallest it can get) then accept it and continue encoding and just report a VBV failure at the end.

[hank315]

1. harder/faster/earlier frame size reduction for VBV(i dont know when/how hank315 decides to start reducing frames or by how much etc, thats closed source for you :). this can slightly reduce quality of course.It seems my source isn't so closed for you:), I also think that's exactly the problem, reducing the frames.
If a VBV underflow happens, HC raises Quants, first for B than P, at last for I frames etc. If the underflow is large, reducing frames is done more agressive but it seems that it should be much more agressive for these kind of sources.
2. if last GOP cant be VBV corrected after a certain amount of attempts or the reduction limit is reached(making the whole frame skipped MB's is the smallest it can get) then accept it and continue encoding and just report a VBV failure at the end.ATM this is not built in, never thought it would be necessary, but for this kind of sources it might. Skipping plain rubbish isn't so bad;)
Raising the Quants faster or using a very low bitrate matrix for these GOPS is also possible or forcing it into a IBBPBBPBB... scheme also could help.


This will be the first issue I will look at after finishing the GUI.
Just lost a complete day on searching for an error what turned out to be a compiler error:devil:

@ video_magic
thanks for all the feedback, this really is a big help.

@dragongodz
also thanks for your comments, useful as always;)

[video_magic]

:) Well I am now trying a FULL encode with the *GOP 5 0 tonight overnight, because I have heard good things about the quality if your encoder and now I hope I too can now get to see it! :p

My computer is quite slow on these kind of tasks of encoding video, I get like 1 - 2 fps on pass 1 of the full clip average, and with other encoders too quite slow - but I choose VHQ settings so I pay for it!

[Amnon82]

Here (http://forum.doom9.org/showthread.php?s=&threadid=89559) we go to a little quality-test ;)

[dragongodz]

ATM this is not built in, never thought it would be necessary, but for this kind of sources it might.
yep you have to think worst case scenario and try to cover it. 99.9% of the time it probably wont be needed but that 0.1% can rear its head and bite you on the arse. :D

Skipping plain rubbish isn't so bad ;)
well it may be more than rubbish on rare occasions such as low bitrate encodes with low max bitrate setting etc. but in those cases there really isnt much choice. :)

using a very low bitrate matrix for these GOPS is also possible or forcing it into a IBBPBBPBB... scheme also could help.
hmmm yes but they should probably be optional rather than forced and only used as worst case.

Just lost a complete day on searching for an error what turned out to be a compiler error :devil:
yep that sucks i know.

[video_magic]

I have just finished encoding, then muxing and creating a DVD iso, and then watching it mounted on Daemon Tools virtual drive on my PC :)

Here are the settings I used:

*ASPECT 3:4
*BITRATE 7400000
*CLOSEDGOPS
*GOP 5 0
*NOSCD
*DC_PREC 10
*INFILE D:\capvid\matwedbilresconvyv12.avs
<space>*INTERLACED
*MAXBITRATE 7900000
*OUTFILE D:\oz\hankgop5.m2v
*PROFILE BEST

(converting from 768x576) avs was:

OpenDMLSource("D:\capvid\matwed3.avi", FALSE, pixel_type="YUY2", fourCC="HFYU")
BilinearResize(720,576)
ConvertToYV12()


Well, my opinion is that HC is a great encoder! The quality was very nice, and definitely overall up to par with other mpeg2 results that I have seen.

I got the impression of slight 'darkening' of contrast but I'm not sure if this was some kind of sharpening? It was very difficult to tell from the original on a frame-by-frame basis and that's a good thing because I had used a high bitrate for this very purpose.

I would like to ask Hank, does any noise reduction take place with the settings I used? I would personally not like any Noise Reduction to take place because of wanting to preserve as much as possible the original detail.

Great encoder, thankyou very much for putting it out here for us to use :)


@Amnon82 - thanks for putting up that test, very interesting and it shows that HC is up there for quality results, at least at high bitrates, which is what I have just found too. A lower bitrate compare would be interesting also.

[Amnon82]

I did the 2nd test. HC won it!!!
Encodertest #2 is online.
I did the encode with 2000, 3000 and 4000 kbps.

[lithoc]

Hi,
Tested some trailers.

HC really produce great quality.
But I'm having problem with size prediction.
The final filesize is always under sized.

Right now, trying to compare QuEnc but need to reencode few times due to filesize different.

[Mug Funky]

could it be that VBV check is lowering the average significantly on very "spiky" sources? i'd imagine there'd be a convergence to target bitrate as the source clip gets longer.

perhaps it'd be useful for HC to keep track of VBV-induced bitrate reduction and allow the average to increase slightly to compensate (it probably does this already).

[Amnon82]

@Mug_funky: 'very "spiky" sources'? Manga or what?

@Hank: Did You downloaded the files of my 2nd test? I'm realy happy with the result.

[dragongodz]

Manga or what?
manga = comics
anime = animation

i would like to see a spiky manga. :D

[manolito]

@Hank315
It is probably too late for the upcoming GUI version, but for later versions I have two suggestions:

1. Could you add a parameter *PRIORITY LOW|NORMAL|HIGH to HC.ini? I use HC mainly together with DVD2SVCD and DVD Rebuilder, and especially with DVD Rebuilder (which starts the encoder many times) it would be very nice to not always have to set the priority to LOW manually.

2. It seems that for my "famous" 1 CD ultra low bitrate SVCD encodes (BR 800 and lower) HC's bitrate distribution favors high motion areas too much over simple scenes. In CCE there are two parameters (Bias and QC) that let the user control if relatively more bits are allocated to complicated scenes or to flat scenes.
Would it be possible to tweak this bitrate distribution strategy in HC by adding some parameter?

Cheers
manolito

[hank315]

@manolito
The GUI version sets the threads priority below normal, in the next version the user will be able to set it.
About your second point, I will do a complete workover of the bitrate distribution, there are more issues to solve ;)

@Mug Funky
perhaps it'd be useful for HC to keep track of VBV-induced bitrate reduction and allow the average to increase slightly to compensate (it probably does this already).It already does it but for short clips it needs a more agressive approach.

@Amnon
Didn't looked at your second test, no time, had to work today... Will do this evening.
BTW. read your last mail, think it's a good idea, will mail you.

@Lithoc
Also noticed that for short clips file size is too short, especially with a high average bitrate.
For a complete movie it will do a better job but after working on the bitrate distribution this can be solved for short clips also.

[Amnon82]

@dragongodz: Oh yeah I also want to see a spiky Manga.
Maybe he converts his manga-comics to an anime. Thats why it is spiky ;)

@Hank: Waiting for your mail :D My email is changed. I wrote You a new mail.

[hank315]

Just put the HC GUI version online.

Get it at http://hank315.dyndns.org/HC_0.1.zip

This is the first time I did such a GUI-thing, so please have mercy on me :)

[Guest]

It either hangs on file load or crashes when I start to encode.

EDIT: Could it be because my AVS already has DGDecode loaded to use MPEG2SOURCE()?

[hank315]

It either hangs on file load or crashes when I start to encode.Tested it on 3 different PC's with different sources :confused:

Basically the input method hasn't changed from the previous version.
EDIT: Used your dgindex/dgdecode version 1.0.12, are there incompatibilities with older/newer versions?

Is there anybody else who has this error?

[Amnon82]

Here my test-result of version 0.1

GUI: all functions working (preview of course not ;))
AVS: I had no problems to import my AVS-Scripts
Speed: On my machine the encode is now faster. In Version 0.01 it tooks 122.7 s and in 0.1 it tooks 88 s for the same source I used for the Encodertest#2.
Quality: The Quality of HC 0.1 is much better than the previous version. Great job hank :D!
INI-Files: I missed the INI-File and Batch-Option. I hope it come back in the next version.
1Pass: I didn't tested the 1 Pass CQ-Mode. I think this mode needs more time.
Bugs: The X-Button is not working. The only way to end HC is with the Exit-Button or the taskmanager. But for Your first GUI it is ok ;)

@Hank: I wrote you a email. We have work to do :p
@neuron2: Can You post Your AVS-Script? My Scripts are working. I'm using Your newest RC2 Version of DGIndex.

[Doom9]

Is it just me or is neither the X button nor Alt-F4 and the corresponding clicks (click on upper left icon, then close) active? I had to go to the task manager and kill the process to exit (I didn't do any encoding so far).

[Krismen]

Is it just me or is neither the X button nor Alt-F4 and the corresponding clicks (click on upper left icon, then close) active? I had to go to the task manager and kill the process to exit (I didn't do any encoding so far).
I have exactly the same problem.

[hank315]

Is it just me or is neither the X button nor Alt-F4 and the corresponding clicks (click on upper left icon, then close) active? I had to go to the task manager and kill the process to exit (I didn't do any encoding so far).No it's not just you, I also noticed it just before putting it online.

Decided to put it online with this flaw (the exit button really works :) ) just to see if the logic behind all the buttons is working properly.

But I already added this one to my ToDo list (hope it won't to get too long:) )

[Amnon82]

@Hank: I think You know this link already: Fortran-Tools (http://www.fortran.com/tools.html)
This might be interesting for You, also: Fortran Interface Wizard (http://forum.doom9.org/showthread.php?s=&threadid=89618)

[manolito]

Originally posted by Amnon82

INI-Files: I missed the INI-File and Batch-Option. I hope it come back in the next version.

I strongly second that!

Cheers
manolito

[Candock]

For Linux you could additionally try the free, unsupported, non-commercial Fortran90 Compiler from Intel.
http://www.intel.com/software/products/compilers/flin/noncom.htm

It compiled all my Fortran95 stuff with just minimal problems (my syntax was a little bit sluggish).

[Guest]

I must have had mismatched DGIndex and DGDecode. Imagine that!

Here's my newbie question. How do I set how many passes are made?

[Guest]

When I hit stop the encoded M2V file had size 0! Can't you close the file properly so I can do a quick test on the first part of a large file? All other encoders do that, even VDub does it for AVIs. Thank you.

[dragongodz]

When I hit stop the encoded M2V file had size 0!
just tested stopping part way through second pass and also CQ mode. both produced part files that are playable.

stopping in first pass(of 2 pass encoding) i would expect a 0 size file because its the stats pass and not producing any video.

hank315 - its a bit of a pain to have to close the gui and run it again just to do another encode etc. be nice when thats fixed. :)

[FredThompson]

This forum needs some kind of hit rating. Seems I can't read new posts before my inbox tells me there's more on this thread :)

[Guest]

Originally posted by dragongodz
stopping in first pass(of 2 pass encoding) i would expect a 0 size file because its the stats pass and not producing any video. OK, that was it. So, I ask again...how do you set how many passes are done?

[dragongodz]

how do you set how many passes are done?
CQ selected = 1 pass encoding
CQ unselected = 2 pass encoding.

thats all HC supports at the moment.

[dream-lx]

Really good mpeg2 encoder. If you could add support for HD resolutions and profiles, it would be perfect.

[buzzqw]

Could be possible to view in gui, during second pass, the low/average/high and current bitrate of encode ?

BHH

[freelock7]

@Hank
HC is working in Yuy2 color space or directly -like QuEnc- in Yv12
-You didn't mention it?

[Ebobtron]

Originally posted by freelock7
@Hank
HC is working in Yuy2 color space or directly -like QuEnc- in Yv12
-You didn't mention it?
I'm not Hank but from the documentation ver Beta 0.01

-----------------
Input can be a d2v project or input using Avisynth. HC expects as input YUV-planes (YV12 color space 4:2:0). During the encoding process there are no color plane conversions, the encoder keeps the color planes in YV12 color space.
-----------------

[gigaset2004]

@ hank
@Amnon82

I did some more tests with the new version in bitrate range of 1000.

The result is not so nice at the - best - encoding setting.

I have noticed that if there are a lot of movements in the pictures it becomes blocky around the movement.

I like to explain it a little more :

If there is a scene with one person an a static backround and the person move in the front so the area around the persons movement is blocky and the rest is less blocky.

Quenc - is more smooth in these scenes.

Question :

Could it possible that HC needs a more agressive Matrix like the `99 er from CCE or anything like that ?

Maybe Amnon82 could do more tests to get rid of these probs ?

[Amnon82]

@gigaset2004: I'm on it. 1000 I didn't test yet.
@hank: I wrote You a answer-mail. I hope we will communicate easier, soon ;)

Just finished my new GUI!

Download HCEnc 0.01 (http://www.brckomania.net/HCGUI/files/HCEnc001.rar)

Screenshots:

http://www.brckomania.net/HCGUI/img/HCENC1.png
http://www.brckomania.net/HCGUI/img/HCENC2.png
http://www.brckomania.net/HCGUI/img/HCENC3.png
http://www.brckomania.net/HCGUI/img/HCENC4.png

Features of the GUI:

*Very small and easy to use
*Reloads the settings of the last run
*Uses HCEnc 0.01 (Till now I don't have a newer commandline-version of HC)

I hope everybody likes it more than my last build.

[Mug Funky]

could possibly make a feature request? forced keyframes would be good for placing chapter points in precise places.

i've got a project ready to roll out of premiere (god, i hate that program, but it's all i've got for now) via pluginpac frameserver, but the problem is that if i make too many random accesses it'll just serve black frames, which is a huge waste of time as you could imagine. this is what's stopping me from making several HC.ini files with different begin and end points (or doing the same thing through avs).

a format like:

*FORCEIFRAME 2050 7800 9000 etc

or

*CHAPTER n1 n2 n3 etc

would be cool.

btw, i haven't tried the GUI yet... i probably should :)

[edit]

@ Amnon82:

by "spiky" i meant a clip that has moments of great complexity, hence large spikes in bitrate... doesn't matter, as hank's already answered my question :)

[dream-lx]

hank:

I have also noticed that CBR encoding is currently not supported, which would be very usefull.

So my personal whishlist:

1. CBR encoding
2. High Definition support (Main Profile at High level)
3. HDV compliant mpeg2 export (well I think this is not urgent at all)
4. A paypal account, so that we can donate you for your work (very urgent) :-)

[robot1]

@hank
I've tried your encoder for the first time today, and it did a very good job (4000 kbits/s)
I had already done the same encoding with CCE, and results were very similar.
Just a notice: I think in a couple of faster scenes HC had a bit of "grain", where CCE was cleaner (your default matrix used). In low motion scenes probably it was the opposite (less mosquito noise than CCE)
Anyway, for a 0.1 it's really really great.
Keep up the good work.

[Amnon82]

@Mug Funky: Chapterssupport will come in a version. When? When it's done ;)

Originally posted by hank315
@Amnon82: I was wrong when I told you I hated GUI programming, I even start to like it, actually it's great fun;)
About the chapters, I tried that in previous versions but could not get it to work properly but it is certainly something worth to be implemented.

[hank315]

Hi all,

Read all posts and saw all the options people wanted.
ATM I'm working on the GUI to resolve the reported errors, after that the bitrate distribution will be optimized which still has some flaws.
About the wanted options, some can be implemented easily, some will require a lot of work...


NOTE1: the encoding engine from this GUI version is still the same as the "old" 0.01 release.
NOTE2: I noticed that scanning the database for a restart takes a long time. This is caused by a call to a "sleep" function which I used to test the thing but forgot to remove. Scanning will now be 2-3 minutes longer for a 2 hour movie because of this. For the moment you have to live with it ;)

[FredThompson]

@Amnon82,

Good to see you have a well rounded avatar.

[Amnon82]

It's my girlfriend ... just kidding ...

[leviat]

I tried downloading the latest GUI version of HC but everytime I attempt to load an AVS script the program dies with a typical Windows death message. I tried using the HCenc GUI but again, HC appears to die almost immediately after the DOS window pops up. I was wondering if there was anything I could do to collect information?

My system:

AMB Athlon 64
Windows XP MCE (SP2)
Latest version of AviSynth2

Great work on the encoder,
Jason

PS - I'm a linux user so my Windows skills aren't the best, but I'll certainly try. :)

EDIT: Nevermind! I figured out my problem. After messing around with AviSynth for awhile I managed to discover I was using AviSynth2 instead of AviSynth25. After fixing that the problem went away. My guess is that HC was expecting a YV12 source and went it didn't find it, it got confused and crashed. You may want to try and add some error checking for that if possible :).

[dragongodz]

It's my girlfriend ..
you wish mate... you wish. :D

whoever she is she is much easier on the eye than your old avatar. ;)

hank315 - any news on what you are working on or problems etc would be interesting to some of us.

[yaz]

Originally posted by Amnon82
It's my girlfriend ... uhum ... address ... phonenumber .... whatever ? c'mon, don't lemme hangin' :-)))
the bests
y

[Amnon82]

@yaz: U got PM ;)

... but I think we are now a 'LITTLEBIT' offtopic ...

[unixfs]

Hank315,
is there any advantage at using a non-linear quantization function w.r.t to a linear one, that libavcodec uses?

Is it indicated for low-bitrate encodes or does it work better at high
bitrate?

Thanks.

[hank315]

hank315 - any news on what you are working on or problems etc would be interesting to some of us.ATM I'm working on problems with the GUI which are reported here and implementing the preview option. Also some cosmetic changes will be done in the encoding statistics window. Will be ready in 1 - 2 weeks (sometimes I even have to work and sleep :) )
is there any advantage at using a non-linear quantization function w.r.t to a linear one, that libavcodec uses?No not really, with a non-linear scale you have a little bit more control over the real quantisation values.
Relation between linear - non-linear Q:

quantiser_scale_code Q linear Q non-linear
0 forbidden
1 2 1
2 4 2
3 6 3
4 8 4
5 10 5
.. .. ..
.. .. ..
31 62 112
@Amnon
Your GUI really looks nice but your girlfriend even better :)
Will mail you tomorrow...

[FredThompson]

@Amnon,

Mail me your first girlfriend TODAY! You're obviously tired of her and I have a spare bedroom...

[Amnon82]

So this was it. I change the avatar ...

@Hank: Sorry ... ;)

[dragongodz]

Will be ready in 1 - 2 weeks (sometimes I even have to work and sleep ;) )
yep no probs. just curious what was next on your list of todo's. :)

[yaz]

@hank
all walking in the same (tight) shoes :)

@ammon
put that beauty back ! not 'asap' or sg but NOW ! or i will never ever ... (what a guy ... really) anyway, fred's chix are really hot too :)

the bests (and good dreaming :) )
y

[Amnon82]

@dragongodz: I sent You a PM. So know more about the project. Please send me a reply as soon as You readed it.

[dream-lx]

Amnon78: wouldn't it be possible to post this information on the project public ? As I think almost everyone reading this thread is interested about the future of HC ?

[dragongodz]

dream-lx - 2 things.

1. its Amnon82, not 78. :)
2. the stuff Amnon82 PM'd me about is programming stuff. not actually something most people will understand or be interested in. i am sure hank315 will post when there is actually more to say.

[Amnon82]

I posted dragongodz some codes. I only posted the last post here to get dragongodz attention. And yes I know a littlebit more than dragongodz about the project, cos I'm one of the betatester of this program. Till now there is no new news for the public. I can't post things from our internal talking, cos I wouldn't be fear.

Hank is here the boss and if he gives me a go I'll post it. Thats the only way I'll do it. So everybody can stop now the begging for new news.
If there are public news, everybody can read them here.

[Paazabel]

Newb question (maybe):

I am working on a wide variety of NTSC captured stuff. I love that HC takes the .avs file and I don't have to RGB and back through VirtualDub.

Anyway, some of the stuff is TFF (top field first) and some is BFF. If it's TFF, no problem, it encodes, muxes together with "muxman" just fine, and plays perfectly on the DVD. If the source is BFF, it seems to get field reversed (you know, video dyslexia).

Does HC expect TFF clips?

[Amnon82]

@Paazabel: I think till now no. Hank has PAL-Sources, so most of his DVDs are progressiv. In one post he told, that he will fix it soon.

@All: A little update. In maybe one week there will be the next release of HC.

[Mug Funky]

anyone with BFF videos, you can get around it somewhat with doubleweave().selectodd(), or if your source is RGB or yuy2, you can shift it up 1 pixel to make it TFF.

there's also reStream available to change the flag manually (you just need enough disk space for another copy of your encode).

[hank315]

Does HC expect TFF clips?It just encodes what Avisynth serves...

anyone with BFF videos, you can get around it somewhat with doubleweave().selectodd(), or if your source is RGB or yuy2, you can shift it up 1 pixel to make it TFF.

there's also reStream available to change the flag manually (you just need enough disk space for another copy of your encode).Will add some buttons so it can simply be set. Will be in the upcoming release.

[Paazabel]

Someone should package this with MuxMan ...

*acts casual*

It seems that it needs TFF to look right. Any time I sent BFF to it, it got all herky jerky. I used ...

DoubleWeave().SelectOdd()
DelayAudio(-0.0167) #Adjust audio one field NTSC

... and everything came out fine. Looks outstanding at 5 meg ...

[Amnon82]

Next version of HC will come soon. Maybe at the beginning of the coming week.

[Xeno86]

Hi hank315!

I'm new to this forum, although not quite a newbie in MPEG encoding.
I've tried your HC GUI 0.1 encoder and I think that it gives excellent output (comparing to libavcodec and tmpegenc) although some things could be improved:

1. When I run 1-pass encode based on quant value it doesn't preserve the maximum bitrate setting, so it's quite useless - even for DVD's, because there are spikes of 18.5 Mbit/s which are far too high :).

It would be excellent if you could add possibility to set maximum bitrate and vbv checking for 1-pass encoding.

2. Why is the nominal bitrate of each MPEG header always set to 9800000 bps? Output stream IS SVCD compatible but I have to use ReStream and change this setting to 2500000 bps in order to be able to read it on my DVD player. Could it be set according to "max. bitrate Kb/s" option as it is in most encoders?

3. Small bug: 2000 Kbit/s is not 2048000 bit/s. (Correct me if I'm wrong :) ).

4. In the docs you've written "Bit rate will never go below 0.4 Mb/s for DVD creation.". How does it know that I'm encoding a DVD? Anyway I'm encoding mainly SVCD so it would be better to let the user decide about the lowest bitrate - I prefer 64 Kbit/s :)

5. The rate control is not very accurate. I encoded my 50 minute material with this settings
--------------------
| encoder settings |
--------------------

profile: BEST
frames: 1 72299
framerate: 23.98
aspect ratio: 3:4
bitrate Kb/s: 2000
max. bitrate Kb/s: 2500
restart: no
closed gops: no
VBV check: yes
scene change det.: yes
interlaced: no
goplen,B-pic: 24 2
dc_precision: 9
scan method: ZIGZAG
time code: 0 0 0 0
CPU: MMX/SSE
matrix: NOTCH

but the size of the output is much lower than expected (Average bitrate = 1167 Kbit/s, Average Quant = 5.533).

Could this issues be fixed in the nearest version (especially 1., 2., 3. and 4.)?

Keep up the good work.

Btw. Sorry for my english, but it is not my native language.

[dragongodz]

1. When I run 1-pass encode based on quant value it doesn't preserve the maximum bitrate setting, so it's quite useless
then it wouldnt be constant quant would it. a 1 pass VBR would be good but consider this is version 0.1 and consider the difficulty in writing 1 pass VBR from scratch. you do know how hard that is right ?

2. Why is the nominal bitrate of each MPEG header always set to 9800000 bps?
because hank315 has just concentrated on dvd for it. i am sure he will change that in upcoming versions.

3. Small bug: 2000 Kbit/s is not 2048000 bit/s.
that is correct.

4. In the docs you've written "Bit rate will never go below 0.4 Mb/s for DVD creation.". How does it know that I'm encoding a DVD? Anyway I'm encoding mainly SVCD so it would be better to let the user decide about the lowest bitrate - I prefer 64 Kbit/s
nitpicking. yes changing the min bitrate would be nice aswell but prefering 64 Kbit/s ? ye thats going to make a huge difference. :rolleyes:

5. The rate control is not very accurate
goplen,B-pic: 24 2
matrix: NOTCH
ah, a follower of the kvcd hype. try using a normal matrix and GOP length and you will probably get closer to your target bitrate. and i have to add that hank315 is still working on rate control and for a version 0.1 it is looking pretty good.

[hank315]

Dragongodz, thanks for the reply :)

Just some short remarks:

1) Constant Quantization is just meant for high quality backup's and testing and is certainly not DVD compliant.

2) Will be set to Max. bitrate in the next version.

3) Just a matter of definition, using the old ones, 1Kbit = 1024 bits, 1 kbit = 1000 bits. A couple of years ago a new standard was set: 1 kBit = 1000 bits, 1 KiBit = 1024. If the majority thinks it should be 1000 bit I will set it.

4) Don't know if you want to create a DVD but ATM the settings are:
if width >= 704 and height >= 480 the minimum bitrate will be 400 otherwise it can go as low as possible...

5) I know, and especially it the difference between max and average is small it just means you want to do CBR and ATM it just can't do it. After the release of the next GUI version (which is almost finished) the bitrate is the first issue on a long list of wishes :D

[dragongodz]

3) Just a matter of definition, using the old ones, 1Kbit = 1024 bits, 1 kbit = 1000 bits. A couple of years ago a new standard was set: 1 kBit = 1000 bits, 1 KiBit = 1024. If the majority thinks it should be 1000 bit I will set it.
yes the Kibi is for binary definition. Kbit for video is 1000. you can confirm that in the mpeg specs by looking at the profile levels. ;)

[hank315]

You're right, think it's better to set it at 1000.

There are just 10 kinds of people, those who can do binary and those who can't, might even get used to it: 2^10 = 1000 :D

[Xeno86]

Thanks for quick replies.

@dragongodz
then it wouldnt be constant quant would it. a 1 pass VBR would be good but consider this is version 0.1 and consider the difficulty in writing 1 pass VBR from scratch. you do know how hard that is right ?
I know that HC is in a very early state and that it still needs lot of work. I really appreciate the work that hank315 does. I only wanted to share with my experience with the encoder :)

ah, a follower of the kvcd hype. try using a normal matrix and GOP length and you will probably get closer to your target bitrate.
I've tried some configurations and kvcd matrix gives me lower bitrate with same quality comparing to standard matrix. If you could recommend me a matrix that does better job at bitrates 1500-2000 I will give it a try (I've also tried QLB but with worse results).
I'll try 'normal' matrix and post the results.


@hank315
4) Don't know if you want to create a DVD but ATM the settings are:
if width >= 704 and height >= 480 the minimum bitrate will be 400 otherwise it can go as low as possible...
I use 528x480(576) so that's ok for me :D
5) I know, and especially it the difference between max and average is small it just means you want to do CBR and ATM it just can't do it.
I know that the difference is small but 2500 is the maximum value that I can use for SVCD and I want to put 50 minutes of movie with highest possible quality so that it would fill whole CD. CBR is not really what I want. I'll try to write it more clearly.

The situation looks as follows:
The output is 458884 kB, the desired is 700000-750000 kB.
Average quant is 5.533 BUT it hardly ever goes below 5.3: bitrate=400 => quant=5.3 (as you can see on the screenshots).
http://img150.exs.cx/img150/2954/14en.png
http://img230.exs.cx/img230/6838/26ln.png

I hope it will help you to find the problem.

[dragongodz]

I've tried some configurations and kvcd matrix gives me lower bitrate with same quality comparing to standard matrix. If you could recommend me a matrix that does better job at bitrates 1500-2000 I will give it a try (I've also tried QLB but with worse results).
what i want to know is if the size increases closer to the target.
as for prefering different matrrices...each to their own. ;)

actually lokking that you are doing svcd(except for out of spec GOP length and frame size) you shouldnt really need a very low bitrate matrix. a target of 2000 is practically equivilent to around 4000 for dvd so more mid range than low. so most matrices should look atleast ok. you are not doing any filtering or anything before the encoder are you ?

no i am not saying thats causing the problem of not hitting your target, as i said the rate control does need more work.

[dragongodz]

Xeno86 - i have come across an interesting anomoly. i did some tests with HC with a target bitrate of 2000. all were slightly under the target bitrate (closest was 1964) according to the HC gui information. now i loaded 1 in to bitrate viewer and it said the average was 1564. hmm thats strange i thought. so i loaded the same m2v in to mpeg stream eye and it said the real average was 1954. so we have 2 programs(1 being the encoder itself) saying the bitrate is only slightly under the target and bitrate viewer saying it well under. strange huh ?

[mean]

Bitrate viewer (last time i checked) takes 23.976 pulldowned stream as 29.96
Hence
1924*23.976/29.96=1539

[dragongodz]

except my test was done with PAL/25fps so pulldown not relevant.

[gizmau]

Originally posted by dragongodz
..real average was 1954. so we have 2 programs(1 being the encoder itself) saying the bitrate is only slightly under the target and bitrate viewer saying it well under. strange huh ?

could the reason be the old problem of Kbit (capital) and kbit (small), because 1954*1,024=2000,896 - which exactly hits the desired bitrate.

[Xeno86]

@dragongodz:
actually lokking that you are doing svcd(except for out of spec GOP length and frame size)
That's right! How did you know? ;)
you are not doing any filtering or anything before the encoder are you ?
I only use deblocking filter, lanczos4resize and blockbuster, but these only improve sharpness and shouldn't lower the bitrate...
under the target bitrate (closest was 1964) according to the HC gui information
I can't see where HC_GUI displays bitrate information. Are we using the same version (v0.1, size 634880 B)?
what i want to know is if the size increases closer to the target.
Yeah the size increased :)
Same source, all the options same (bitrate=2000, max=2500, 2pass), except:

kvcd matrix, gop 24 2:
avg.bitrate=1167
avg.quant=5.54
mpeg matrix, gop 15 2:
avg.bitrate=1250
avg.quant=5.73
Encode time ~4H
(info according to Bitrate Viewer)

So it is 7% bigger :D

Now let's see if it shows bitrate correctly:
469 897 135 B * 8 bit = 3 759 177 080 bit / 1000 bit = 3 759 177 kbit / 3015 sec = 1247 kbit
So it seems that Bitrate Viewer is right (3015 sec is real play time @ 23.976 fps, total num of frames 72299). I didn't do any pulldown.

It seems that min quant value is somehow locked. It hardly ever goes below 5.00. It looks as if it was hardcoded :confused:

[Amnon82]

A little update for the Guys reading this thread:

The Betatesters started to test the new version 0.11.

New Features of the new version:

Preview of the Source and the Encode
Interlaced fields are added now
Some changes to the encoderengine
Some fixes on the GUI

When Hank releases this version to the public I can't tell.

@Hank: Keep on the good work. ;) You got mail from me ...

[hank315]

It seems that min quant value is somehow locked. It hardly ever goes below 5.00. It looks as if it was hardcodedIt's not hardcoded but I know quants are not able to go low enough.
The bitrate control as it is implemented was meant to create DVD content with a normal DVD GOP structure and bitrate.
So doing KVCD stuff will not give optimal results ATM.

After finishing the new GUI version I will do a complete workover of the bitrate control, the next version might already improve things but these are just minor changes.

@Xeno86
Also did some testing with 2000/2500 average/maximum bitrate but couldn't reproduce the large difference you got.
Did a test with a 6 min. Matrix2 (PAL) source, settings: matrix: NOTCH, profile: BEST, scene change: ON
With GOP 24-2: bitrate HC: 1899, Bitrateviewer: 1899, MPEG Stream Eye: 1899
With AUTOGOP: bitrate HC: 1920, Bitrateviewer: 1910, MPEG Stream Eye: 1910
Both too low but not that low you got...

My goal was to create a thing what would do descent quality, runs stable and would give no problems with max bitrate and underflows, now the problem seems to be at the other end of the bitrate spectrum :( , think this one can easier be solved :)

The next version will be ready in a couple of days if all goes well, ATM I'm implementing the last changes from the beta test feedback I got (thanks for that guys!!).

[dragongodz]

I only use deblocking filter, lanczos4resize and blockbuster, but these only improve sharpness and shouldn't lower the bitrate...
actually deblocking can smooth aswell. ;)

I can't see where HC_GUI displays bitrate information. Are we using the same version (v0.1, size 634880 B)?
umm no, sorry, i was using 0.11 beta. :)

469 897 135 B * 8 bit = 3 759 177 080 bit / 1000 bit = 3 759 177 kbit / 3015 sec = 1247 kbit
i did the same calculation for my 25fps and it came out 1953 which tallies with HC and mpeg stream eye but not bitrate viewer. so atleast for this clip bitrate viewer is getting it wrong.

Did a test with a 6 min. Matrix2 (PAL) source, settings: matrix: NOTCH, profile: BEST, scene change: ON
i tested several different matrices, also normal and best settings with scene change turned off for all.
hank315 if you want a small sample clip that shows much lower in bitrate viewer but near target bitrate in HC and mpegh stream eye let me know.

[Ebobtron]

Originally posted by Amnon82
A little update for the Guys reading this thread:

The Betatesters started to test the new version 0.11.

New Features of the new version:

Interlaced fields are added now
"The Beta testers", guess I'll have to figure out what I have been doing with my spare time the last couple of weeks.
----------------
"Interlaced fields are added now",don't you mean a TFF or BFF check box to select the flag setting to match your source? Last time I used HC interlaced fields worked just fine. I'm not sure what the BFF guys do, but I use ReStream.
-----------------
I don't mean to be pissy. Sorry everyone, yes I do.
--------------
@Hank315
I have been away for the last week popping in from time to time to follow along. I'll post my findings tomorrow.

Sir, your public awaits you.

[dragongodz]

hmm i just did a test encoding with QuEnc 0.59 and bitrate viewer says thats well under the bitrate aswell while mpeg stream eys says its slightly over. what the ????

"Interlaced fields are added now",don't you mean a TFF or BFF check box to select the flag setting to match your source?
yes thats what he means.

"The Beta testers", guess I'll have to figure out what I have been doing with my spare time the last couple of weeks.
I don't mean to be pissy. Sorry everyone, yes I do.
dont be so ungreatful. so a couple of people got to check for last minute problems etc a matter of days before you are due to see it, get over it.

[Axed]

Originally posted by Ebobtron
"The Beta testers", guess I'll have to figure out what I have been doing with my spare time the last couple of weeks.

Theres a very good reason for other people to be checking it before its released to the major public. When the problems pop up, people more knowledgeable in how the encoders work, and also programers, are alot more useful to iron out major bugs then the public who cant properly describe their problems.

When the rest of us get our grubby fingers on it, im sure we'll find those annoying little random bugs that cant be found by a few people testing.

Thanks for the great work Hank.

[Ebobtron]

Originally posted by dragongodz
dont be so ungreatful. so a couple of people got to check for last minute problems etc a matter of days before you are due to see it, get over it. Didn't mean to pick a scab. I just thought that everyone working this thread was beta testing. I either stand corrected or not. Ungrateful is a little harsh, I could be wrong. But you have read much more into my statement than I meant.

For that I am truly sorry.

[Amnon82]

Hank recruted some of the HC user to betatesters. I'm one of them, for example. I know that some guys here tests HC for hours and weeks, but some are only user. They can only tell the programmer, that there is a problem when he did this and that.

I'm coding programs my self. So does dragongodz too, who is also one of the betatesters of HC. We're talking daily via email with hank and helping him so. We do a hard job to find all the bugs and codingmistakes hank did in the last build.

If we don't do this before hank releases it public, the most user here will post "man this program isn't finished yet. I can't encode my movies. etc ..." without knowing what the problem realy is.

In version 0.11 hank changed the core. But with this changes we lost quality. So hank need more time to change it to the quality we already know form HC 0.01.

The GUI has now more features. For example, hank added a preview. So You can switch between the source and the encode to see the quality. Via a zoom-function You can zoom into the picture during the preview.
Also the stats are now displayed in the right way.

Some reader here know that it is the first GUI for Hank. 0.01 was a commandline-tool. I coded the first GUI for it. Hank felt in love with it and started a GUI by him self. So 0.1 was the next release and the first with a GUI by Hank. So some features came and some are gone.

To get the features of my GUI, the commandline version of HC and the new guied version of HC into one program it will take a long time to get it done, but we're working hard to reach the goal soon.

When the next release will be released I only can tell You this:

When it's done! ;)
(a cool statement btw, so think I (thx goes to iD))

[Ebobtron]

Revisiting my bug report concerning the flashing pink and sometimes green lines at the bottom of the frame when using NTSC sources 720x480, 704x480, 352x480, 352x240, interlaced or progressive.

--------------------------------
++++++++++++++++++
++++++++++++++++++
++++++++++++++++++
++++++++++++++++++
+++++++++++xxxxxxxxx
------------------------------------ The xxxxx show where the lines appear in the video.

Roughly the last third of lines 476, 478 & 480 from pixels 464 to 720.

The following is a list of those encoder settings that have no noticeable effect on the aberrant lines.

Source --- avs or d2v: see source content below
Encoder profile:
Bit-rate:
Frames:
GOP Structure:
Matrix:
DC Precision:
CPU:
Interlaced:
Aspect Ratio:
Scan method:
Time Code:
Do I need to add VRB check:

That leaves us with source content, closed gops, scene detection, constant quantization.

Constant Quantization or CQ

“Using CQ”, @ 4,5,6 or 7 (other values not tested) the anomalous lines appear in every frame, are a solid green residing in lines 476, 478 & 480 from pixels 464 to 720. Feed “HC” PAL content of 576 or 288 vertical lines and the green lines do not appear. With NTSC content 480 and 240 vertical lines green lines are displayed. Horizontal size has no apparent effect on the unwanted lines. Interlaced or progressive encoding, no difference. Source content has no effect with the exception of vertical size.

“No CQ” -- “Open GOPs” -- “No Scene Detection”

Now is gets interesting. The lines are pink and they flash or they appear to flash. Our unwelcome guests show up only in the two frames displayed prior to an I frame. The pink lines are not solid but spread along the space between pixels 464 and 720 (as dots and dashes) again on lines 476,478 and 480. The little pink dots and dashes are never in the same position unless the frame lacks motion.

Source content has an effect if the scene is black or has no motion. Source content with large black borders at top and bottom of the frame produce no unwanted pink lines. So I ran a script to increase the borders around a frame in increments of one. As border size increased pink lines decreased until gone at eight pixels, border color had no effect. My conclusion is that if there is a change between frames in the lower 8 lines we get pink lines. But only in the two frames that proceed an I frame. So they appear to flash.

“No CQ” -- “Closed GOPs”
This one is easy, no pink lines. Remember that GOPs closed or open have no effect if CQ is used.
CQ and NTSC equals solid green lines in every frame.

“No CQ” -- “Open GOPs” -- “Scene Detection enabled”
The little pink flashes are here too but not in the two frames displayed prior to the inserted I frame for the detected scene of course that GOP is closed.


The following is meant not to show a GOP but to clarify the frames containing the misbehaved lines.
In displayed order viewing with VirtualDub-Mpeg2 1.5.10 (build 18294):

B B P B B P B B I <----This I frame must the beginning of an open GOP for the two B frames to flash pinkly.

Source effects
I use AviSynth 2.55 Build Sept. 1, 2004 (I have omitted the build time). DGIndex's d2v or AviSynth, the only thing that effects the behavior is vertical resolution. PAL source equals no green or pink lines however the encoder is configured. I have toyed with fps and field frames and TFF and BFF. Vertical frame size seems to trigger the behavior

My source doesn't have the unwanted lines, I tested with rips, camera clips, and still frames.

That’s all I have on this. I can’t even offer a guess.

I’m using XP Home SP1 on a Compact with 512 Mbytes of ram with a 2.8 GHz Celeron Processor.

I suppose it could be my set up here, stranger things have happened. I hope someone can duplicate my observations.

My time is up for today.

[Xeno86]

@hank315

The bitrate control as it is implemented was meant to create DVD content with a normal DVD GOP structure and bitrate.
But I have also tried it with standard gop and matrix and the differences were really small:
kvcd matrix, gop 24 2:
avg.bitrate=1167
avg.quant=5.54
mpeg matrix, gop 15 2:
avg.bitrate=1250
avg.quant=5.73

Anyway I've done few more tests with same settings but other source (Gladiator DVD PAL, 2 minutes) here are the results:
Avg.bitrate=1856
avg.quant=5.78

This time quantizers went much lower - the lowest is 3.20

It looks like the "too low bitrate problem" is related to my previous source. But I don't now why quant was locked to 5.3 and now it went much lower. Strange, isn't it? I'lldo more tests later.

[((( atom )))]

@hank315: your encoder looks very promising. some free tool like that with high quality was missing for a very long time now. thx for it!

there are a couple of questions i have. i read through the manual, not this entire thread, so i might ask questions that have been discussed, but every user might have these questions after having read the manual, so i dare.. ;)

- does your encoder support hd-resolutions?

- why can i only input .d2v or .avs files?
especially for dv, avi-input could easy things up for many people. others might want to reencode mpeg-files. is it because the app couldn't make use of other files atm or is it a restriction you wanted?

regards,

[Koepi]

Maybe because it's so simple to just write

AviSource("MyDVfile.avi")

into an avs file instead of dealing with missing codecs, framrates,...?

But if avs is supported, avi support shouldn't be that hard to do either. Btw., it's usual for the most recent encoders to just use avs input (QuEnc, avs2avi,...)

Just a wild guess though :)

Cheers
Koepi

[hank315]

Just finished a new version: HC 0.11 beta

Changes:
- GUI updated, many bugs fixed
- preview option added
- TFF/BFF flag added for interlaced encoding
- shows Avisynth script errors
- max. bitrate is written in sequence header instead of 9800
- minor changes in bitrate control for bitrate < 2000 kb/s
- AUTOGOP option didn't work if scene change detection was switched off, fixed
- bitrate now in kb/s, m2v file size in Kbytes (1 kbit = 1000 bit)
- MPEG matrix is set as default matrix

get it at: http://hank315.dyndns.org/HC_011.zip

Thanks to Amnon, dragongodz and scatha for testing and feedback.

Nothing really changed in the encoding engine except for bitrates < 2000 which should give a slightly better result.

[Amnon82]

@Hank: You're always welcome ;)

[hank315]

- does your encoder support hd-resolutions?No but maybe in the far future

- why can i only input .d2v or .avs files?
especially for dv, avi-input could easy things up for many people. others might want to reencode mpeg-files. is it because the app couldn't make use of other files atm or is it a restriction you wanted?Koepi already gave the answer, it's easier this way.
For Avisynth people can also use their favorite filters etc. and just the idea to write a descent avi reader... :scared:
The avs input is based on AVS2YUV by Loren Merritt and the mpeg input uses DGIndex/DGDecode by Donald Graft, so I just did it the easy way :)

[TEB]

Lovely encoder. Will u add some kind of SMP support in the encoder pipeline? and yuv2 (4:2:2) support?

keep up the good work.

teb

[Backwoods]

Originally posted by hank315
No but maybe in the far future

Concerning HD res, there is alot more to it besides being able to include different profiles? Obviously I have no idea, just curious.

[Ebobtron]

@Hank: Two questions.

1) I have been assuming that HC is self contained. And we know what happens when I assume:). So, is it? Is there anything HC needs from my system that could cause version conflicts?

2) Encoder profile: What is fast, normal, good, best? It is difficult to see any subjective quality difference between normal and good.

The output is as good as I can measure.

It’s great work just as it is. It’s hard to imagine the encoding being much better for high rate dvd source creation.

Congratulations,

Ebob

[Pasqui]

Hi hank315 !
First of all, thanks for your work !
Do you plan to reintegrate command line support ? It is sad that your encoder cannot be used by DVD-RB.

Regards,

Pasqui.

[dragongodz]

2) Encoder profile: What is fast, normal, good, best? It is difficult to see any subjective quality difference between normal and good.
i assume(:)) they use different motion search precision/range etc.

[hank315]

I have been assuming that HC is self contained. And we know what happens when I assume. So, is it? Is there anything HC needs from my system that could cause version conflicts?The exe file contains all the code but because it's now a "real" windows program it needs more windows DLL's but nothing special.
The first command prompt version only needed one windows DLL, now it needs more, so more fun :)
And ofcourse Avisynth and/or DGDecode.dll.


Encoder profile: What is fast, normal, good, best? It is difficult to see any subjective quality difference between normal and good.The way the motion search works is mostly influenced by the encoding profile. If the bitrate can be high enough the (visual) difference will be small but if you want to do a backup of a 2.5 - 3 hour movie you should use the best profile.

EDIT:i assume() they use different motion search precision/range etc.That was just a little bit faster :D

[manolito]

Originally posted by hank315
Nothing really changed in the encoding engine except for bitrates < 2000 which should give a slightly better result.
As long as the GUI version has no INI file / batch support, is there any chance you could update the non-GUI version with the updated encoding engine?

Cheers
manolito

[hank315]

As long as the GUI version has no INI file / batch support, is there any chance you could update the non-GUI version with the updated encoding engine?Will do that, it's just a small change, will let you know here when it's done...

[DK]

could please you take a look at this, hank?

http://forum.doom9.org/showthread.php?s=&threadid=89440

[hank315]

could please you take a look at this, hank?

http://forum.doom9.org/showthread.php?s=&threadid=89440Hey, that's nice, never saw this thread but I like DVD Rebuilder a lot, very nice program.
Will certainly read all of it and see if I can help but first I think I'm going to sleep for a week or so :D

[dragongodz]

hank315 - i see the gui is still in "monster" mode. :D

[hank315]

i see the gui is still in "monster" mode. :DYep, but the next one will be smaller :)
So people with EGA resolutions and 286's can also use it :D :D

[Amnon82]

Yeah. I'm running a 486 with a Rage128 and 20 GB harddrive. My monitor has 15" so I can't use Your guied version. So I coded one my self ;) Also I tried HC on my new PDA. It crashed. When do plan to write it for WinCE?

... just kidding :D

[dragongodz]

HAHAHA funny guys.

seriously though i do have freinds with 15" monitors who use 800x600 resolution. try the gui on that some time. ;)

[video_magic]

^That's what I'm using right now and I also know a couple of other people stuck with the same. Sometimes I do switch to 1024x768 but that's not very pleasant to use on a 15"

I do have a 19" at home though :)

That aside, congrats and thanks on the new HC Hank

[Amnon82]

@Dragongodz: Kost jetzt boarisch? Freinde! Des is ja da Wahnsinn!

Translation: Can You bavarian too? Friends! Wow this is amazing!

;) ;)

Yeah it should fit for 800x600 ...

[freelock7]

HC0.11 gives a really good picture.
Movie tested yesterday with great quality (VBR=5000-max bitrate=8500).
Global quant=4.5
bitrate peak=8920
Muxing with IfoEdit correct
Still needs a temporal filter to correct grain (convolution3D).
Deblocking is a good idea too (thanks Dragongodz).
So, for the next release, can you give us, Hank, some advanced settings to adjust?
Is it possible to do that with HC?

[zettai]

Everytime I try to load an avs file into HC it just hangs. No crash error message, just no response. The avs file works fine in vdub.

I have an old fashioed althon xp, using win2k. Suggestions?

[buzzqw]

@zettai

even with a simple avs as

colorbars(720,576)
converttoyv12()

BHH

[doxville]

Originally posted by zettai
Everytime I try to load an avs file into HC it just hangs. No crash error message, just no response. The avs file works fine in vdub.

I have an old fashioed althon xp, using win2k. Suggestions? Same with me, I am also using Win 2k and right after opening the source-avs, the programme hangs up with no response, no matter how simple the script is (colorbars, etc.). I am using the latest Avisynth (beta2), but that shouldn't be the problem, right? Both Versions of HC, 0.1 and 0.11 have the same problem... :(

[hank315]

Just installed the latest Avisynth beta but everything runs ok.
Using XP, SP1. Don't have a machine with 2000 to test it :(

[danpos]

Originally posted by hank315
Just installed the latest Avisynth beta but everything runs ok.
Using XP, SP1. Don't have a machine with 2000 to test it :(

Hi, hank! You could to try a win2k emulator like VMWare, QEmu, etc ... In this way, you can to test HC in win2k environment without the necessity of do partition on disk, to install the system, blabla ... It's just a suggestion!;)

Keep up the good work,

[hank315]

Will u add some kind of SMP support in the encoder pipeline? and yuv2 (4:2:2) support?SMP support is one of the things still to do but it is not easy to synchronize threads in an encoder, already tried it...

4:2:2 colorspace: probably not until I really need it.

Concerning HD res, there is alot more to it besides being able to include different profiles? Obviously I have no idea, just curious.Also don't know much about it but what I've seen so far Main profile @ High level could be implemented.
BTW, what exactly are the specs for HD?

[LigH]

On my AMD Duron 800, HC 0.11 does not detect that this processor supports MMX/ISSE, and creates a file "fort.24" with the content
No MMX/SSE detected
HC needs atleast MMX/SSE to run
aborting...
<ENTER> to close program

But afterwards, HC 0.11 GUI starts, and works well. (BTW: Great quality!)

[Amnon82]

I think it is GUI related. If there is a DLL version of the HC Encoder my DelphiGUI will work ;) Also the commandline version works with my gui.

:D

... or let us know which DLLs You're using for Your GUI ...

[hank315]

@LigH
That's strange because this message was removed, are you sure it's not an old message from the first version?
If MMX is present and SSE not, it should generate a warning but I couldn't test it, don't have a Duron or old Athlon around here...

[hank315]

... or let us know which DLLs You're using for Your GUI ...????
Just the normal DLL's are linked when you use API calls...
But if you want to know, it imports stuff from kernel32, user32, gdi32, comdlg32, comctl32, ole32 and oleaut32, so nothing special...

[zettai]

Solved my particular problem.

I was only quickly opening it up to have a look at it so instead of unpacking it, I ran the .exe by double clicking in winrar (which should just extract to a temp foler and run the executable). This was the version that would fail to open any scripts - if I extracted to a regular directory the normal way it would be fine.

No idea why, but there you have it.

[robot1]

Originally posted by danpos
Hi, hank! You could to try a win2k emulator like VMWare, QEmu, etc ... In this way, you can to test HC in win2k environment without the necessity of do partition on disk, to install the system, blabla ... HC works fine in win2k

[JetlagMk2]

Originally posted by robot1
HC works fine in win2k
Works fine on my w2k sp4. Thanks, hank!

[doxville]

Works fine on my W2k SP4, too... Now...;)
I found out that antivir (free-av.com) causes all the problems. After having deactivated the "AV-guard" the encoder opened my *.avs without problems, giving me opportunity for heavy testing! :D
Thanx, hank!

[rhayman]

First off i really like this encoder and think it has great potential.

Secondly i notice the program has a lot of built in martix's, i have been interested for a while to find out about these. However im useless at searching forums and couldent really find anything specific enough to help me.

My question is, what is the difference between the matix's, are some better quality than others, when should you use them etc. But mainly do they all create standard mpeg2 files for dvd authoring.

Sorry if this has been explained before and i've just missed.

Any help is appreciated.

[LigH]

Originally posted by hank315
@LigH
That's strange because this message was removed, are you sure it's not an old message from the first version?
Uh - probably; I tried all versions around the same time.

AMD Duron supports MMX and ISSE. I just wonder why it didn't detect it.
__

@ rhayman:

Different matrices are useful for different purposes.

For example, there are matrices which have the potential to encode the video very exactly - but at the cost of high bitrates. You will notice the rather low values even near the right bottom area of the matrices.

Other matrices will allow a quite fair quality even at low bitrates. But they will reduce details and provide a rather blurry result. You will notice quite high values especially at the right and bottom borders.

[Amnon82]

@rhayman: I and others explained it in this thread already:
http://forum.doom9.org/showthread.php?s=&threadid=89897

take a look ;):search:

[rhayman]

Sorry, i did a search before that was posted and could not find anything, i should of searched again before posting.

Thanks for the link :)

[Amnon82]

no problem

[hank315]

Seems that people who are running anti-virus programs have trouble running HC, it could take a long time to start because the exe file is packed with UPX, some anti-virus programs don't like that.

So I changed the download version to a non-packed executable.
Just download it again and see if it helps...

Still the same link: http://hank315.dyndns.org/HC_011.zip

[gigaset2004]

@ hank

I have tested your encoder with bitrates at 1000 with several files and i also tested bitrates at 2000 and rejig them.

I found the encoder pulses the B frames ?

The quality is not so nice at both ways. I know you said the encoder is normaly for DVD ranges.

Maybe it is possible to explain what happens wit the B / P Frames ?

Nice work anyway

[doxville]

Originally posted by hank315
Seems that people who are running anti-virus programs have trouble running HC, it could take a long time to start because the exe file is packed with UPX, some anti-virus programs don't like that.

So I changed the download version to a non-packed executable.
Just download it again and see if it helps...

Still the same link: http://hank315.dyndns.org/HC_011.zip
Sorry, but I'm afraid it doesn't help with my antivir-software. I still got the same problems as before with the unpacked version... :(

[Xeno86]

Hi everyone!

@ hank315 & dragongodz
I didn't have time recently, but I think that I've found why the quant doesn't go below 5.3. The problem is connected with mild noise. Anyway this is my (new :) ) test script, so you can duplicate my results:
BlankClip(length=1500,width=528,height=576,pixel_type="YV12",fps=25)
Blockbuster(method="noise",variance=1.5,cache=8192,detail_max=64,seed=5623
Encode it with default settings (using public ver. 0.11beta) with:
1) bitrate=2000 and max=2500
2) bitrate=2000 and max=6000
3) bitrate=2000 and max=8000

What you can see in bitrate viewer is that 1) and 2) looks almost exactly the same - only first 6-7 secs differ. The rest has quant=5.3 and bitrate=550-650. So it is not low max that spoils the result ;)
3) Looks totally different but very strange though - bitrate jump from 950 to 4000 kbps.
A quick test of my initial clip without this BlockBuster line has shown that bitrate raised. I'm doing a full encode now though. I'll post back later. It's strange that bitrate goes lower with this line, because it add more 'details' (noise) which should be more difficult to encode and bitrate should go higher, shouldn't it?

Btw. I like the new version :D

[hank315]

Posted by FreelockStill needs a temporal filter to correct grain (convolution3D).
Deblocking is a good idea too (thanks Dragongodz).
So, for the next release, can you give us, Hank, some advanced settings to adjust?
Is it possible to do that with HC?No, not ATM. It just does a straight forward encoding without filters etc.
Ofcourse it can be programmed but it will certainly not be in the next version.

@Gigaset2004
Bitrate is optimized for DVD creation, average bitrate > 2500-3000.
Pulsating B frames just means the nr. of bits for these frames is too low. In the next version I will try to get a better distribution between I, P and B frames at low bitrates.
But if you're doing 1000 Kb/s at full DVD resolution results will always be bad, IMHO the bitrate is just too low for MPEG2.

@Xeno86
Will try that noise stuff when I can find the time.
There's so much to do and so little time ;)

[dragongodz]

Xeno86 - i have had a look at that test. hmm interesting. i have emailed hank315 about what i see etc. so we will just have to wait and see what he makes of it aswell. :)

[freelock7]

HANK315-No matter. I found these settings for each encoder to produce the same great quality (VBR=2Pass/2Pass):

CCE2.70
cpu=4
Low pass filter: natural3* (32/10)-by default
Levels(0, 1, 255, 3, 255)
undot()
lanczos4resize(720,576)
converttoyuy2(interlaced=true/false)
-----------
QuEnc0.59B2
cpu=4
undot()
lanczos4resize(720,576)
-----------
HC0.11
cpu=4
Convolution3d (preset="movieHQ")
Blockbuster(method="noise",detail_min=1,detail_max=3,variance=0.1,seed=1)
undot()
lanczos4resize(720,576)

[Xeno86]

A quick test of my initial clip without this BlockBuster line has shown that bitrate raised.
Here are the final results of full encode:
All settings as before - best profile, notch matrix, gop 24 2, length 0:50:35, 528x480, 23.976 fps, max.bitrate=2500, bitrate=2000.
avg.bitrate=1691 (computed value; bv shows 1584, hc shows 1722, mpeg stream eye shows 1691)
avg.quant=4.16
min.quant=2.34
Filesize=637415901 BPrevious results (same source & settings but with Blockbuster(method="noise",variance=1.5,cache=8192,detail_max=64,seed=5623) ):
avg.bitrate=1247
avg.quant=5.54
min.quant~5.0
Filesize=469897135 BAs you can see there is a huge difference.

[dragongodz]

Xeno86 - dont get too caught up by quants rising. for harder to compress ,but bitrate limited, encodes i would expect to see the average bitrate rise noticably and the minimum bitrate to rise maybe slightly. obviously the minimum bitrate is rising more than a little and is a problem but we will have to wait for hank315 to have a look and see what he can do in code.

[hank315]

@Xeno86 and dragongodz
Just started some tests using noise with Blockbuster.
Using:
BlankClip(length=1500,width=528,height=576,pixel_type="YV12",fps=25)
Blockbuster(method="noise",variance=1.5,cache=8192,detail_max=64,seed=5623)gives bad results but the variance is just too small, the "energy density" of the planes is low, a small difference in Q will give a huge difference in bitrate.
Example: Q=2 --> all coefficients will be coded, Q=3 --> almost all coefficients will be zero.

Doing something likeBlankClip(length=1500,width=528,height=576,pixel_type="YV12",fps=25)
Blockbuster(method="noise",mean=80.,variance=200.,cache=8192,detail_max=64,seed=5623)would be better for testing but even then the bitrate should be better.

So there's still some work to do ;)

[fewtch]

I just tried this encoder for the first time. I'm a newbie to MPEG2 encoding, so bear with me...

First off I chose an average bitrate of 3700 and max of 7000 in order to fit on a DVDR along with some chosen audio. I used a D2V file (DGMPGDec v1.2.1) indexing the movie "Carlito's Way" from decrypted VOBs (NTSC, pure FILM) and changed aspect ratio to 1:1 in HC. I didn't change any other defaults except that and bitrate.

HC produced an M2V that TMPGEnc DVD Author rejected as being out of DVD spec (23.976 FPS instead of 29.97). This part I don't understand, but it's probably something very simple I missed... this is where I'm new to this sort of encoding (no big deal, it was just a test). Also, there were way too many artifacts in the final output for my taste, but clearly the bitrate is on the low side. The film itself is quite badly grainy as well and not very good quality for a DVD movie.

Not sure about the aspect ratio thing, I think the movie itself is 2.35:1 so dunno which I should have chosen. 1:1 seemed to make the most sense for re-encoding from VOBs, someone correct me if I'm wrong.

Anyway, just some output from a newbie encoder's experience with the program, hope it helps. Version is HC 0.11b with GUI.

[dragongodz]

out of DVD spec (23.976 FPS instead of 29.97). This part I don't understand, but it's probably something very simple I missed.
very simple indeed, 23.97fps is not a valid DVD framerate. to use 23.97fps footage you have to use pulldown which of course is equivilent to making a fake 29.97fps. lookup telecining for more information about that. ;)

Also, there were way too many artifacts in the final output for my taste, but clearly the bitrate is on the low side.
as you can see from the last few posts there is still work to be done on the rate control.

The film itself is quite grainy as well
and of course there are avisynth denoisers that can help clean up the source video if its a bit grainy to start with, which should look better if not over done.

Not sure about the aspect ratio thing, I think the movie itself is 2.35:1 so dunno which I should have chosen.
personally i have only ever seen 4:3 and 16:9 on dvds. even when they say another ratio they have had black bars added to make them either of those. 1:1 is definatly not right. :)

[fewtch]

Originally posted by dragongodz
very simple indeed, 23.97fps is not a valid DVD framerate. to use 23.97fps footage you have to use pulldown which of course is equivilent to making a fake 29.97fps. lookup telecining for more information about that. ;)
Isn't the majority of FILM material out there at 23.976fps (at least with NTSC)? If so, how about a warning from HC that the result won't be DVD compatible, since it seems to me re-encoding from VOBs to get something that would fit on DVD-5 wouldn't be an uncommon usage...

personally i have only ever seen 4:3 and 16:9 on dvds. even when they say another ratio they have had black bars added to make them either of those. 1:1 is definatly not right. :)
I'm still really lost when it comes to aspect ratios and such. When I used 1:1, black bars were there in the resulting .m2v... are you saying they would not if I used 16:9? Remember, this encoder has an option to use .d2v files, which if I'm not mistaken are always derived from DVD's.

P.S. sorry for any ignorance I'm displaying here. :confused: My experience is mostly with DVD->AVI and DVD->DVDShrink->DVD. Working directly with MPEG2 is still brand new stuff to me. Just thought a newcomer's perspective might add something to the discussion (then again, maybe not :rolleyes: :D).

[FredThompson]

Accounting for pulldown is a good idea. For the rest, you really should do some research on the basics.

[LigH]

MPEG-2 allows 4 different aspect ratios, in general. There are:

- 1:1 Pixel Aspect Ratio [= PAR] (not allowed on DVD!): The content shall not be stretched in any direction upon playback.

+ 4:3 Source Aspect Ratio [= SAR] (valid for DVD): The content shall be stretched slightly, so that the resulting image will have an aspect ratio of 4:3 (this would stretch lines with a width of 704 pixels to a width of 768 lines; if 720 pixels shall stretch to 768 lines width, too - or to rather 786 pixels instead, is a topic for philosophers for ages now...)

+ 16:9 Source aspect ratio (valid for DVD): Similar topic here, the content shall be stretched remarkably, so that the resulting image will have an aspect ratio of 16:9 (means, 704 pixels width to 1024, 720 to probably even more). This is known as the usual "anamorphic" encoding.

- 2.21:1 Source aspect ratio (not allowed on DVD!): Stretches even wider upon playback...
__

So what if your movie has "the interesting part of the image" with Cinemascope aspect ratio? Well, DVD allows only 4:3 SAR (704:768 PAR) or 16:9 SAR (704:1024 PAR). If the interesting part of the content would have a SAR of 2.35:1 (as often printed on the back side of the DVD case), it still can only be encoded at an anamorphic aspect ratio of 16:9 SAR. So it still needs some small black bars ("letterboxes").
__

And about d2v project files: They store the positions of the GOPs (or I frames) inside an MPEG file. But they don't store the aspect ratio - the safest way to determine the aspect ratio of a DVD, is to look at it and decide! DVD2AVI / DGIndex is only able to report bitstream flags. And it is possible that these are wrong (if the encoding studio had set up the encoder incorrectly - and this happens from time to time...).

Keep in mind, that DVD video may only have Source Aspect Ratios of "4:3" or "16:9". Use the thumb rule: If the actors have "egg heads" in the preview of DGIndex (press [F5] after loading the VOBs, and later [Esc]), then the SAR is "16:9" (well - except for the movie "The Coneheads", maybe); if their proportions look quite fine, it is "4:3". This is the best way to decide - software is not able to do so.

[fewtch]

If I recall correctly, with this DVD the view in DGIndex showed the video *squished* vertically (not stretched), i.e. heads too much rounded rather than egg-shaped, and with large horizontal black bars already there in the DGIndex display. If it matters, this is an old/OOP DVD from 1998. The back of the DVD does say "2.35:1 widescreen." Another weird thing is that the main movie, with no extras and few audio/subtitles takes up over 7.3GB on the disc. I'm thinking this may be an old, nonstandard encoding of some sort.

Anyway, thanks... that helps some, although some confusion remains (mainly on the framerate issue -- why pulling video directly from the VOBs using .d2v gave a DVD-incompatible framerate with HC?).

Will do some research, or if you know the specific answer maybe PM me (if you have time)... I'm done.

[Orion|69]

@ hank315 : respect.. i truly love your encoder already, altho it's ofcoz not finished :)

what i prolly love most is that all of this is done in fortran and an additional pure asm lib :)

since the company i worked for ditched fortran in the, lemme think, early nineties (i think around 93/94), i never saw such a great tool produced with the language :)

u made an old coder smile :D

respect also goes to the ongoing efforts of the betatesters and the input of the not-so-betatesters :) i'll try to do my share from now on :)

[dragongodz]

why pulling video directly from the VOBs using .d2v gave a DVD-incompatible framerate with HC?
thats what you havent understood. its NOT incompatible its just not a legal framerate used as is, so it has to be telecined(2:3 pulldown).

now different authoring programs will handle these diferent. from some checking i just did Tmpgenc it appears will just error and not accept it without pulldown already done. while DVD-Lab it appears will give you a warning and do the pulldown itself.

so yes maybe adding an option to output an already pulldown set stream is a good idea.

some reading on how telecining works can be found here
http://www.doom9.org/index.html?/synch.htm

The back of the DVD does say "2.35:1 widescreen.
and if you were to cut off the black bars thats what it no doubt would be. they still do that on dvds, quote the picture part(minus bars) aspect ratio. with the bars though you will find its either 4:3 or 16:9.

for those interested in dvd aspect ratios heres a little read aswell
http://www.doom9.org/index.html?/aspectratios.htm


anymore general dvd questions should be in a new post please.

[JetlagMk2]

hank, could the GOP structure be manually set? Maybe from right out of a d2v file? I'm sure you already have a lot to do it's just a humble feature request. Thanks for the great program!

[Trahald]

Did the .ini support go away in the gui versions.. i dont see anything specifically mentioning it but have not been able to get them to read the .ini

[hank315]

Did the .ini support go away in the gui versions.. i dont see anything specifically mentioning it but have not been able to get them to read the .iniYes it did...

But ATM i'm working on a batch version which will take command line arguments in the way Quenc does.
It will run in a command prompt window but the encoding engine is the same as the GUI version.
Not every option will be supported, some general options will be in the ini file but the next arguments will be supported:
-i ---> input file
-o ---> output file
-b ---> average bitrate
-maxbitrate
-aspectratio
-ini ---> HC.ini file for all other settings

Doing it this way it can also be used with DVD-Rebuilder.
Will be ready in a week or so...

[manolito]

Originally posted by hank315

Not every option will be supported, some general options will be in the ini file but the next arguments will be supported:
-i ---> input file
-o ---> output file
-b ---> average bitrate
-maxbitrate
-aspectratio
-ini ---> HC.ini file for all other settings


I think you should also support the "Interlaced" option as a command line argument. It is not important for DVD2SVCD, but DVD Rebuilder needs it because quite often some of the extras are interlaced while the main movie is progressive. Ask jdobbs...

Cheers
manolito

[FredThompson]

Originally posted by manolito
I think you should also support the "Interlaced" option as a command line argument. It is not important for DVD2SVCD...Huh?!?! The ONLY source DVD2SVCD accepts is progressive?!?! Specifying interlaced source most certainly is important.

[manolito]

@FredThompson

No no no, big misunderstanding! Of course you have to be able to specify interlaced or progressive when using any encoder with DVD2SVCD. It is just not absolutely necessary to support "Interlaced" as a command line option. In the case of HC there are many HC settings that are unique and do not have a corresponding option in QuEnc. So it will be necessary to also have a template for HC where these settings can be made independently of the command line options. Since DVD2SVCD only encodes the main movie, you can set up "Interlaced" or "Progressive" in the template.

For DVD Rebuilder it is a different story. It encodes the whole DVD, and it happens all the time that there are interlaced and progressive parts on one DVD. So Rebuilder must have a way to tell the encoder how to treat every part of this DVD independently.

Cheers
manolito

[Trahald]

@hank
excellent! being able to set tff/bff would be nice too (via ini is fine.) also start time of 00:00:00:00 as an option (i dunno if the gui versions changed since im still using .01)

[Mug Funky]

one thing that would be pretty cool is a "save settings" option in the GUI. possibly saving it to the old-style .ini file. i have to re-set everything every time i open the program.

i eagerly look forward to batch mode though :)

really great work on this encoder.

[FredThompson]

The graphic version hangs when I feed it this script with the intent of making an MPEG2 still:

ImageSource ("custommenu.bmp")
ConvertToYV12

Is the current release incapable of making stills?

[hank315]

Is the current release incapable of making stills?Just did a test with your bitmap, result at:
http://hank315.dyndns.org/stillmenu.m2v
one I frame, size 180 kB, Quant=1
One frame is a little bit difficult to play, loaded it in cuttermaran, looked good.
Turned the I frame into a bitmap again:
http://hank315.dyndns.org/still.bmp

[FredThompson]

Wow! That looks great! How did you make it? The GUI version freezes when I tried.

MPC plays the still just fine, btw.

[hank315]

Used the same avs file as you did, the first time got a "movie not progressive" error, second time was OK.
Think I'm going to dump this check...
Using the last Avisynth beta release.
Start HC_011 and change the next default values:

endframe 1
dc precision 8
activate Constant Q: value 1
aspect 4:3
matrix QLB

This should generate the same m2v file.

[FredThompson]

I've tried to set HC.ini with your settings but I'm seeing an error.

*INFILE custommenu.avs
*OUTFILE menu.m2v
*PROFILE BEST
*ASPECT 3:4
*CQ 1
*DC_PREC 8
*FRAMES 1 1
*MATRIX QLBcustommenu.avs, avisynth_c.dll and the entire HC set are in C:\Working\DTiVo\

If I launch HC.exe, AviSynth reports custommenu.avs cannot be found.

With this setting:

*INFILE C:\Working\DTiVo\custommenu.avs there is another error showing there is an attempt to read from:

C:\Working\DTiVo\ C:\WORKING\DTIVO\CUSTOMMENU.AVSWhy would this happen? Is it a parsing error in HC?

[hank315]

Used the GUI version to create it, and the avs file opened the second time I tried it :confused: think the command line version will just abort on that one.

Please try it again with the GUI version, that might help.
Will remove some avs checks in the next version that don't seem to work OK.

[FredThompson]

Ah, the command line version is a little behind the GUI version, huh? You'd almost think this was a one-man hobby project :)

The error is an AviSynth error, not HC. It looks like HC is screwing up the path when it calls AviSynth. I've tried the GUI version and it keeps hanging. Maybe it's got the same issue with opening the AVS file as the command line version. The installed AviSynth here is AviSynth_210205 and avisynth_c.dll from avisynth_c-0.15. Which are you using?

I was actually hoping for the command-line version. The root of this is mpeg2enc problems with TyTool. HC looks like a far better option and it has the command line interface which is a far easier to control than GUI automation, though it can be done. TyTool allows removing sections within an existing MPEG2 stream and only encodes the modified GOPs. Part of this process is calling mpeg2enc for each "new" GOP. That, and the single frames used for menus, are the parts which could benefit from HC. mpeg2enc is also lousy for HD content.

This is very impressive output, by the way, VERY impressive. I've got another "torture" background which looks like a red and green Christmas package decorated with gold bells. It makes just about everything else puke, especially if the menu text is white. Most encoders hate the high contrast of saturated red and green near each other and they smooth out the green too much. Here's that background image: http://home.mindspring.com/~fredthompson/Christmas Gift Wrap Title Page.zip The yellow edge of the ribbon was added to compensate for the corruption of most encoders. It could be replaced with the same red as the rest of the ribbon for an even harder test. The texture in the green was added in the hope it would reduce noise around the white menu text. It could be replaced with a uniform green without any objections. With the texture, it would provide an interesting test.

[hank315]

Even without adding text it's an ugly thing for every encoder to keep the small yellow lines OK.
It's almost as ugly as your home page :D

[FredThompson]

Yeah, the yellow lines weren't an original part of the design at all. They were added to try to reduce the corruption from mpeg2enc.

Don't know it really qualifies as a homepage. It's more a place to put stuff that's linked to various posts. I never thought anyone would actually look at the page itself. Nice "nose shot", don't you think? Bet that hurt.

The puppy's cute, though.

[dragongodz]

the first time got a "movie not progressive" error, second time was OK.
Think I'm going to dump this check...
this can be avoided by adding
AssumeFrameBased()
to the script. to quote the docs
AviSynth keeps track of whether a given clip is field-based or frame-based. If the clip is field-based is also keeps track of the parity of each field (that is, whether it's the top or the bottom field of a frame). If the clip is frame-based it keeps track of the dominant field in each frame (that is, which field in the frame comes first when they're separated).

However, this information isn't necessarily correct, because field information usually isn't stored in video files and Avisynth's source filters just guess at it. AssumeFrameBased and AssumeFieldBased let you tell AviSynth the correct type of a clip.

so yes i would drop checking that for now since its not exactly reliable.
you would maybe want to add it back in if/when you support interlaced input though. then people would have to be told to make sure avisynth is told the correct information about the source though. :D

[FredThompson]

Now I'm thoroughly confused. HC only supports progressive input?

@hank315,

I've found the untextured, un-yellowed original Christmas present background. Will put it on my page and add links for the "invisible" files.

[dragongodz]

HC only supports progressive input?
well i am assuming thats what a message such as
"movie not progressive" error
means. also the fact that uing AssumeFrameBased() fixes that message kind of suggests it no ? :)

remember though that HC is a very young encoder.

[FredThompson]

I'm not complaining, just making sure I understand. "Clean" still frames is a HUGE benefit as is the quality of progressive encoding.

At the same time, I'm not that adverse to using hot pokers and a rack to "encourage" hank315's development of HC. :devil:

[dragongodz]

ok heres a couple of quick encodes of the cusstommenu.bmp and christmas png(i am not going to type the whole name :D).

script used
ImageSource ("custommenu.bmp").trim(1,1)
ConvertToYV12()

trim used to make it 1 frame long, otherwise ImageSource makes it 41 seconds long of course. :)
same script used for the xmas pic aswell.

encoded constant quant 1, dc 8, no custom matrix, with both HC gui and QuEnc 0.59beta2.
resulting .m2v's here
http://www.pcpages.com/dragongodz2/
click the "menu tests" link to download.

to view with MediaPlayerClassic do NOT use the internal mpeg2 decoder or use another program.

[Ebobtron]

HC only supports progressive input?I can only get this message from HC when I feed it field based sources.
Any frame based source works fine, so far.

I'm using AviSynth 2.55 and ImageSource() spits out frame based. Is it safe to say that FredThompson's version spits out field based from ImageSource()?

@FredThompson
I encoded your menu pages with HC and got great results even great red lines.
So follow ImageSource with "AssumeFrameBased()" in your script.


so yes i would drop checking that for now since its not exactly reliable. you would maybe want to add it back in if/when you support interlaced input though. then people would have to be told to make sure avisynth is told the correct information about the source though.All the interlaced sources I fed to HC Play great on my old TV. I tested HC encoding progressive and interlaced. Progressive encoding of interlaced material shows up big-time on the boob-tube, kind of like the reverse of interlaced on a progressive display.

[FredThompson]

Try as I might, HC still freezes on me right after I select the AVS file. Contents of the custommenu.avs are:
ImageSource ("custommenu.bmp").trim(1,1)
ConvertToYV12()Here's a screenshot of the directory:

http://home.mindspring.com/~fredthompson/HCTest.gif

If I launch HC_011.exe and select the avs file, the file selection window disappears, the area it occupied is left grey and HC freezes.

If I configure HC.ini as defined above and launch HC.exe (the updated executable), I get this status:

http://home.mindspring.com/~fredthompson/HCCLTest.gif

Something must be incorrect but I'm stumped. Would someone please take a look at this and help me find out what's wrong?

@Ebobtron, the source is a bitmap. I probably confused things by mixing discussion of single-frame encoding and replacement of mpeg2enc with HC in TyTool. Sorry about that.

[dragongodz]

Any frame based source works fine, so far.
well i got the warning from HC with an avi which i guarentee is progressive. so avisynth got it wrong once. :)
and yes i am using 2.55 aswell.

Is it safe to say that FredThompson's version spits out field based from ImageSource()?
no because using my script caused HC to freeze and not throw the "not progressive" error.

FredThompson - Ebobtron suggested and add AssumeFrameBased() to it aswell and see if that makes difference. it shouldnt but still needs to be tested. ;)
also what have you got in your avisynth plugins folder ? try removing everything except directshowsource.dll and seeing if HC gui still freezes.

[FredThompson]

Just tried both suggestions; "empty" plugins folder and AssumeFrameBased()

Neither stopped the crashing. Darn it!

[dragongodz]

hmm that is strange. and the script loads up ok in other programs, no error messages etc ?

the only other things i could think to try is set CPU in HC to MMX before loading the script or changing your avisynth version to the 2.55 release.

[Ebobtron]

well i got the warning from HC with an avi which i guarentee is progressive. so avisynth got it wrong once.
@dragongodz

Sorry, I was trying to say any source I fed HC. Not the source inside of AviSynth. When AviSynth's output is framebased HC works for me. Interlaced or progressive.

[Ebobtron]

I had HC crash on me and stay resident causing second instance to act up.

[dragongodz]

Sorry, I was trying to say any source I fed HC. Not the source inside of AviSynth. When AviSynth's output is framebased HC works for me. Interlaced or progressive.
ahh. yes thats why i suggested AssumeFrameBased() before aswell. :D

[FredThompson]

Appreciate the help, gents. Still odd that HC crashes on me. I'm moving over to QuEnc as an mpeg2enc replacement because interlaced source is important. This is all really for TyTool which is TV-based, sort of, in a way.

[Ebobtron]

From hank

Seems that people who are running anti-virus programs have trouble running HC, it could take a long time to start because the exe file is packed with UPX, some anti-virus programs don't like that.

So I changed the download version to a non-packed executable.
Just download it again and see if it helps...

Still the same link: http://hank315.dyndns.org/HC_011.zip


@Fred, did you catch this from Hank?

I got very strange behavior.

[FredThompson]

Yeah, I saw that but the files I'm using were downloaded today so that shouldn't be the problem. Script loads fine into VirtualDub-MPEG, TMPGEnc, and others so it's probably not an issue with AviSynth. I still think it's a parsing issue with the path to the bitmap as I described above. The command line version can't find the bitmap with an implicit (same directory) definition or wiht an explicit path. Weird.

[hank315]

New version, bug fix release.

A while ago Ebobtron reported flashing colors in the lower lines.
This only occurred for some resolutions, especially NTSC stuff.
This is fixed, also the "movie not progressive" Avisynth check is deleted.

get it at: http://hank315.dyndns.org/HC_012.zip

Thanks to Ebobtron for reporting it and testing the new version.


Still working on a new batch version which will have all options of this latest GUI version.

[LigH]

Looks nice. Works well. (Even the preview, and catching AVS error messages.)

But I wonder: Is it intentional that the 1st-pass takes much longer than the 2nd-pass? On my Duron 800: 2.0 fps vs. 6.4 fps. Do you re-use any information beyond framesize statistics (e.g., motion vectors)?

[hank315]

Do you re-use any information beyond framesize statistics (e.g., motion vectors)?Yes, motion vectors are re-used in the second pass.

[LigH]

Ah, I see - that huge *.dbs file in the source file directory.

[hank315]

Yes but it also contains the information about the GOP structure, scene change etc.
In fact the first pass sets the structure for the whole movie.

[Doom9]

@hank: seems your server is experiencing some difficulties. I guess I could add it to the software archive but I first need to get it..

[GooglyBear]

during test runs like this, does anyone have a custom video to use so we can test better? I only use a real dvd to encode and I don't like waiting 3-4-5 hours just to see encoding quality..

does anyone have a custom zip of something that isn't GB big to test encoders out on? Custom sample clip Would help greatly

:confused:

[dragongodz]

cut a section of footage from the dvd and test with that. how ? try just ripping a couple of cells with dvddecrypter, load the produced vob in dgdecode to produce .d2v, create .avs to load and decode the .d2v.

[kitsaros2000]

where can i find the comand line options for HC 0.12 ? Can someone list them ? Thx ! :)

[dragongodz]

where can i find the comand line options for HC 0.12 ?
0.12 is a gui only version. you will have to wait for the next CLI version, hank315 has said he is working on it.

[kitsaros2000]

thx for the info !
i will wait !
byez :D

[archrival]

Once I noticed that the pink lines were fixed with NTSC film, I decided to test this encoder fully. I first prepared it (Heat) with DVD Rebuilder, then I checked the rebuilder.ecl file and manually encoded all 52 cells, it took a while, but DAMN was it worth it! There is no way any other encoder would give me the kind of quality that HC has put forth. Heat is 172 minutes long, I normally disappointed with CCE's quality on movies this long, not with HC combined with the dynamic cell bitrate assignment that DVD Rebuilder gave me. I'm very happy, and I'm definitely willing to test some more.

[Ishan]

I did some HDTV to DVD encode with HC and all I have to say is : "WAWWW!!"
For a free encoder it totaly kicks ass, I don't want to use CCE anymore :) I havn't noticed any bugs so far, I'm mostly encoding PAL material.

[Frameserver]

Originally posted by FredThompson
Try as I might, HC still freezes on me right after I select the AVS file.

@Hank315
Nearly the same problem happens on my machine.
But it is definitly not a Avisynth problem.
All scripts work fine with Quenc/CCE/V-Dub/MPC and (strange,isn't it?)
HCenc non GUI Version . If I feed the same scripts into HCenc
GUI it freezes immediatly.
The only thing that works with the GUI Version is a d2v project.
But this is not very useful.
On the other hand.
The quality of the test that I made using a d2v file was amazing.
So keep on going - nice soft ;-)


ps.: The test GUI from Amnon82 works fine with the same scripts.
Also this AutoCQ thing from Amnon82 (using HCenc) works fine.

[GooglyBear]

the real question is when will the next BATCH hc come out? as is the only one that really works with RB as is delivers those pink lines and they are a deal breaker :( so I'm guessing most everyone else is waiting for the next batch version to really start putting this baby to the test

[LigH]

From some debugging sessions done by 'katjarella', these details are suspicious:

- Do you write the preview video output directly into the video memory? If so: Instead, better use more compatible functions (because you don't do it so often, DIB functions shall be suitable, before trying e.g. DirectDraw).

- Do you really need WinSock DLL calls and "Network\SharingHandler" in an encoder? (Loading ws2_32.dll seems to provoke a crash.) Calling a website could as well be done using "ShellExecute". Don't act too complicated.

[Amnon82]

... maybe there will be another commandline-version of HCEnc ...

[LigH]

Several people would enjoy a new CLI version from hank315's HC encoder: Possible batch handling; neither preview nor networking code.

[DD51]

Originally posted by GooglyBear
the real question is when will the next BATCH hc come out? as is the only one that really works with RB as is delivers those pink lines and they are a deal breaker :( so I'm guessing most everyone else is waiting for the next batch version to really start putting this baby to the test

I'm definately one of those! I can't wait!
This encoders quality output is very impressive to me.

[hank315]

Coding of the batch version is finished, running tests now...

It's not a CLI version, it runs in a small window which shows input parameters, output stats etc.

It can use the old ini file, also takes arguments, arguments passed will have a higher priority than the ini file parameters.
Just create a .bat file and put in it what you want, additional options can be read from the ini file.

available arguments:
-i inputfile
-o outputfile
-log logfile
-ini inifile
-b average bitrate
-maxbitrate maximum bitrate
-profile fast-normal-good-best
-matrix --> as supported in GUI version
-aspectratio 1:1-4:3-16:9-2.21:1
-scene
-noscene
-interlaced
-nointerlaced
-tff
-bff


From some debugging sessions done by 'katjarella', these details are suspicious:

- Do you write the preview video output directly into the video memory? If so: Instead, better use more compatible functions (because you don't do it so often, DIB functions shall be suitable, before trying e.g. DirectDraw).

- Do you really need WinSock DLL calls and "Network\SharingHandler" in an encoder? (Loading ws2_32.dll seems to provoke a crash.) Calling a website could as well be done using "ShellExecute". Don't act too complicated.Would like to see some stats of this debugging session :)
The preview creates a bitmap in memory and uses a simple StretchBlt, all according to the "normal" windows rules.
Also there are no WinSock DLL calls or network stuff in the code.
It's all written in Fortran, the GUI also, data structures used and the dialog structure which it runs in can't be done in any other language, it also uses a different calling convention, maybe the debugger can't handle this :confused:

@Frameserver
Will have a look at the Avisynth freezing, could you be more specific please, which Avisynth version etc.

[DK]

that's good news

can't wait testing it

[DD51]

Originally posted by hank315
Coding of the batch version is finished, running tests now...

Aweswome news...I look forward to run some tests myself.

Thanks so much Hank315 and keep up the great work on this little puppy of yours.

[Shubin]

The program stops responding immediately after attempt
to load .avs file. The processor is P4, and it only
happens when CPU type is set to anything, except MMX/SSE.
This last setting works.

[Rickkins]

Am I missing something, or is there a way to use this with an .avi capture...???

And, I guess, if not...why not...???

[LigH]

HC requires YV12 input.

You may try an AviSynth script with the appropriate "Source" function (AviSource, SegmentedAviSource), and probably "ConvertToYV12(interlaced=true)" (because your capture shall be interlaced).

[Rickkins]

Oh. I grabbed avisynth...haven't got a clue.
Oh well...guess it's just not for me.:(

[LigH]

@ Rickkins:

If you don't even dare to read the documentation installed together with AviSynth (at least version 2.5.5, to not be outdated), then you probably won't be able to enjoy HC.
__

@ hank315:

All that katjarella can offer right now, is this file:

http://members.lycos.co.uk/verglade8kihr/IWatch_HC_012.zip

IWatch_HC_012_XP.log: under Windows XP
IWatch_HC_012_W2K.log: under Windows 2000 (worked 1 time)
IWatch_HC_012_HANG.log: under Windows 2000 (crashed)

Same script, same AviSynth version (one of the newest, probably newer than available on SourceForge, AFAIR). Crashes at: Process A18, Thread 384: Unload dll at 75950000

[katjarella]

Originally posted by LigH
All that katjarella can offer right now, is this file:
http://members.lycos.co.uk/verglade8kihr/IWatch_HC_012.zip

Referrer Error: (Sorry)
HC_012_XP (http://members.lycos.co.uk/verglade8kihr/IWatch_HC_012_XP.log)
HC_012_W2K OK (http://members.lycos.co.uk/verglade8kihr/IWatch_HC_012_W2K.log)
HC_012_W2K Freeze (http://members.lycos.co.uk/verglade8kihr/IWatch_HC_012_HANG.log)

[Rickkins]

Originally posted by LigH
@ Rickkins:

If you don't even dare to read the documentation installed together with AviSynth (at least version 2.5.5, to not be outdated), then you probably won't be able to enjoy HC.


Well thanks for playing.

Truth is,"First, you create a simple text document with special commands, called a script." is where it loses me. :eek:



I can do point and click, but creating scripts....just seems outta my league.

[Ebobtron]

@Hank315

From the beginning, up to and including version 0.10, I noticed that HC would always miss the target average bit-rate on the low side. I assumed that missing the target was a known problem and did not pay it much attention. While testing version 0.10 I had the opportunity to encode an off the air capture of a black and white film that ran 1:32:10 (165720 frames). With the encoder set at 6000 and 8000 I got a m2v file somewhere between 2.2 and 2.6 GB. Sorry I have no other details from that encode in my memory. I passed it off to the work in progress way of thinking and left the rates and target rates to the other more knowledgeable people working the thread.

I back checked the change logs and other than “many bug fixes” and “minor changes in bit-rate control for bit-rates < 2000 kb/s” I can find no reason for the change in behavior of your encoder. Not that I’m complaining.

I don’t do much backup and most of my encodes are fifteen to twenty minutes. After looking at the DVD RB thread I thought I’d run a long test again.

I ran my black and white movie thought HC version 0.12 over-night and again I have my mouth hanging open.


----------------encoder settings----------------

----------------profile: NORMAL
-----------------frames: 1 165720
--------------framerate: 29.97
-----------aspect ratio: 4:3
-----------bitrate Kb/s: 6370
------max. bitrate Kb/s: 8000
----------------restart: no
------------closed gops: no
--------------VBV check: yes
------scene change det.: yes
-------------interlaced: yes
-----------goplen,B-pic: AUTO
-----------dc_precision: 9
------------scan method: ALTERNATE
--------------time code: 0 0 0 0
--------------------CPU: AUTO
-----------------matrix: MPEG

-----------source stats-----------

nr. of frames in source: 165720
-----------width*height: 720*480
--------------------fps: 29.97
nr. of frames to encode: 165720
-------frames to encode: 1 - 165720
-movie length to encode: 1:32:10 (5529.53 s)
----est. outfile length: 4299695 kB

Actual out-file length 4,400,578,560 bytes / 1024 = 4,297,440 kB
Estimated = 4,299,695 kB, a 2255 kB miss, 0.0524% 5 hundreds of one percent. I asked for 4,300,000. How close could it get with profile set to Best?

Is it possible you fixed this and didn’t notice? Could it be related to the math problem causing the flashing lines? Wow!

Hard to think of this encoder as a “work in progress”.
Five hundreds of one percent is like 52 miles out of one hundred thousand. With accurate encoding and dead on scene detection.

This isn’t great work Hank, it’s wonderful work.

[Xesdeeni]

Originally posted by Rickkins
Truth is,"First, you create a simple text document with special commands, called a script." is where it loses me. :eek:

I can do point and click, but creating scripts....just seems outta my league. Come on! You can write e-mail right? It's just not that much more complicated.

First, run the AVISynth installer.

Now, run notepad. The easy way is to click Start|Run... and type in 'notepad'. You can also go to the directory where an AVI is stored, and right click, choose New|Text Document. Go ahead and rename the text file to 'Test.avs', and answer "Yes" to change the extension from TXT to AVS.

Now you type in something really complicated like:AVISource("test.avi")Now you save that file. If you used the Start|Run... method, give it the name 'test.avs' and put it with the AVI file.

Now you can pretend the AVS file is an AVI file.

I recommend opening it with VirtualDub, but MediaPlayer will open it as well. Use anything just to see that it works just like an AVI file.

Simple, huh?

Xesdeeni

[LigH]

Important: When you save the script, and enter the filename (e.g. "test.avs"), make sure that the file type is set to "Any file (*.*)"; else you would risk that the "Editor" adds another extension, and the file is indeed saved as "test.avs.txt" - this would not be intended!

Finally, you can open the *.avs script file in many applications and media players, as if it was a *.avi video file. Most applications ask Windows to read the video file (they don't really read AVIs on their own), and AviSynth - as registered filehandler - returns video, as if an uncompressed AVI would have been read.

More you can read in the AviSynth forum of this board, it is no longer necessary to discuss that topic in this thread!

[Frameserver]

Originally posted by hank315


@Frameserver
Will have a look at the Avisynth freezing, could you be more specific please, which Avisynth version etc.

Hi hank315,
AviSyth version is 2.54
OS Win 2000 SP4 + all Security Updates
Processor is an Athlon XP (Thoroughbred B)
so MMX+SSE is present.
Feel free to ask for additional infos You need to
solve this issue.

Got a look at the CPU Auto or MMX/SSE Set, makes no difference.
Still freezes immediatly after selecting the *.avs file.
This must be an GUI issue cause the ini driven HCenc works
fine on my machine.
Maybe the upcoming batch version is my favorite..........
don't like GUI's too ;-))

Best wishes
Frameserver

[hank315]

@LigH & katjarella
Think it's better a real Windows/Avisynth guru should look at this, which I'm certainly not...
The only thing I can think of is loading the TCPdeliver.dll which also references WS2_32.dll, TCPdeliver.dll is loaded by Avisynth from the Avisynth plugin directory.
Also did a debug session, even my Logitech mouse loaded WinSock stuff, that's simply the windows world, there were times when programming was easy :D


@Ebobtron
For a whole movie it's easier to keep the relative error small, there will always be an absolute difference which will be larger in a relative way for a short clip ofcourse.
No didn't change anything in the bitrate control yet, still has to be done.

[LigH]

TCPDeliver in the AviSynth AutoLoad directory... that's quite possible. Seems that IWatch doesn't distinguish well between DLLs which are loaded from the EXE directly, and loaded from called DLLs indirectly.

[Ebobtron]

Originally posted by hank315

@Ebobtron
For a whole movie it's easier to keep the relative error small, there will always be an absolute difference which will be larger in a relative way for a short clip ofcourse.
No didn't change anything in the bitrate control yet, still has to be done.

Ok, I will accept that to cover your skill as a programmer:rolleyes:. However, I did not miss the file size difference between the two encodes. Maybe I was drunk that night. Oh yea, I do not drink. Then something must have stopped the encoder. I know it did not crash. Maybe my wife shut it down. Yea, blame it on her.

I shall retest 0.01 just for grins.
I’m out of here.

[Ebobtron]

Ok, my bad, never mind, except the things I said about what a wonderful piece of work this encoder is.

I will go back and shine that star of yours some more.

[Frameserver]

Originally posted by hank315
@Frameserver
Will have a look at the Avisynth freezing, could you be more specific please, which Avisynth version etc.

Some additional things.
If HCenc GUI freezes the system is not harmed. You can work
in a normal way. Also there is no entry in the System event log
(error or warning) caused or depending by HCenc.
The only change is in the Task-Manager where the HCenc status
changed from "Wird ausgeführt" (running) to "Keine Rückmeldung"
(no response).
I post the german terms cause it's a german Win and if I translate it
picking the wrong words nobody's understanding what I'm talking about.

Another issue is the databasefile hc.dbs created from the first pass.
This file gets really huge (depending on the filmlength).
In my case (maybe in other cases too) the system drive (partition)
runs out of space (a 192minutes project creates a hc.dbs of 2.6Gig).
Don't know what kind of error this produces.
Maybe its better that You store this hc.dbs at In- or Outfile position
cause there will be enough space and name it unique (like the In- or
Outfile) so we can handle some project's parallel and not only once at a time.
Maybe it's possible in the future to save project settings as a file/job and batch this (maybe already done in the upcoming version).
Just a suggestion.

Solong
Frameserver

[LigH]

katjarella tried with all the current versions (at least between 2.5.4 and 2.5.6 Backout 2005-03-15).

[LigH]

Dear hank315,

please don't take this question as "urging you"... but: Are you at least able to guess the current state of the CLI development (something like "##% finished, testing this and that")? I just would like to tell some eagerly waiting members of the german board, how long it probably may take. (Please don't mind so many people asking so many times... and don't hesitate to ask for specific beta test systems, e.g. low-featured CPUs.)

[dragongodz]

to those having the freezing - do you have anything in your Avisynth plugins folder besides directshow.dll ? if yes can you please remove them all and try HC again ? ok may not make any difference but best to check all options. :)

LigH - there is no point hank315 trying to give a time for release since it will no doubt depend on testing time etc. :D

[LigH]

I just would like to know if we shall rather plan "this weekend", or "next month". ;)

[Doom9]

I think planning for "when it's done" should do it ;) It works even for Duke forever.. and that is somewhat of a record holder.

[LigH]

Duke Nukem Forever... don't remind me. :rolleyes: I start to doubt if anyone will be glad anymore, when that piece will finally be "done"... :sly: - especially since they stopped this "Duke Nukem: Source" fan mod! :(

Okay - so I'll continue to beg for "patience" in our board.

[Frameserver]

Originally posted by dragongodz
to those having the freezing - do you have anything in your Avisynth plugins folder besides directshow.dll ? if yes can you please remove them all and try HC again ? ok may not make any difference but best to check all options. :)


@dragongodz
Makes no sence cause only the GUI Version freezes not the .ini driven
one.
Maybe there are minor differences in the called dll's between XP and
W2k (diff.versions)and that's the prob

[hank315]

New batch version will be ready this weekend.

About the Avisynth freezing issue it's hard to solve for me because just don't get the error on every PC I tried.

Possible solutions
- Look at dragongodz post
- Make sure you are using the latest Avisynth version
- Avisynth.dll is packed, unpack it with UPX, might help, don't know

Question:
Those who have this error, can program in C (MS visual studio) and has some time available, please PM me, think it's the only way to solve this.

[dragongodz]

Frameserver - dont knock it until you have tried it please. i have already seen plenty of times where some plugins work fine with most programs but then stuff up with some others.

About the Avisynth freezing issue it's hard to solve for me because just don't get the error on every PC I tried.
and this is what people need to remember. its hard to fix a bug you simply do not get.

[fabrice]

Hi,

Originally posted by dragongodz
[B]to those having the freezing - do you have anything in your Avisynth plugins folder besides directshow.dll ? if yes can you please remove them all and try HC again ? ok may not make any difference but best to check all options. :)


I got freezing with hc0.12, and after cleaning avisynth plugin folder, it don't freeze anymore!

I'll be trying to find from which plugin comes the problems.

Thanks
Fabrice

[katjarella]

Originally posted by fabrice
I got freezing with hc0.12, and after cleaning avisynth plugin folder, it don't freeze anymore! I'll be trying to find from which plugin comes the problems.

Originally posted by hank315
- Make sure you are using the latest Avisynth version
- Avisynth.dll is packed, unpack it with UPX, might help, don't know


my Avisyth Folder is Clean (NO FILES INCLUDE)
Last Avisynth Version Test pack/unpack (inc MD5 Checksum)


.................................
avisynth_2005_02_10
\30e22ca6d620905fcae697e06e68385a *C:\\WINNT\\system32\\avisynth.dll
.................................
avisynth_2005_02_10_unpack
\6f1eb206807964872f383b816ecb6fa2 *C:\\WINNT\\system32\\avisynth.dll
.................................
avisynth_2005_02_28
\39976969bdb9668bc7826340aa71428b *C:\\WINNT\\system32\\avisynth.dll
.................................
avisynth_2005_02_28_unpack
\e0de5005fb4d7b8c8f299f7835c82e0d *C:\\WINNT\\system32\\avisynth.dll
.................................
avisynth_2005_03_10
\f7b9649bd4d27b48be383dc1473d1df7 *C:\\WINNT\\system32\\avisynth.dll
.................................
avisynth_2005_03_10_unpack
\056c1897bb59d252e066657308f3883f *C:\\WINNT\\system32\\avisynth.dll
.................................
avisynth_2005_03_15
\7ef5d8bc85ee15c80c5ef7da93b5d3c1 *C:\\WINNT\\system32\\avisynth.dll
.................................
avisynth_2005_03_15_unpack
\b683323246a6afee4005ac65ba3832f4 *C:\\WINNT\\system32\\avisynth.dll
.................................
avisynth_2005_03_15_backout
\79ddc3b28ffc069c0b7717ea68ebb6d4 *C:\\WINNT\\system32\\avisynth.dll
.................................
avisynth_2005_03_15_backout_unpack
\61724a00c17eaa046a83de56adb23a80 *C:\\WINNT\\system32\\avisynth.dll

always Freezing :( // the "old" NOGUI Version working

[dragongodz]

I got freezing with hc0.12, and after cleaning avisynth plugin folder, it don't freeze anymore!
nice to know it fixes it for atleast 1 person. :D

katjarella - can you post the script you use and versions of any plugins used in it etc ?

[katjarella]

my Script:

BlankClip(length=500,width=720,height=576,pixel_type="YV12",fps=25,audio_rate=48000,stereo=true,color=$000000)
ShowSMPTE(offset="00:00:00:00",x=64,y=28,font="Arial",size=24,text_color=$FFFFFF)

moment dragongodz... I have a workaround... LigH translate :)

[LigH]

Your wish is my command, master... :rolleyes: :D
__

It seems that HC 0.12 GUI freezes as soon as the AviSynth script is opened - when HC runs alone.

If an AviSynth script is being processed in the background (e.g. being encoded by CCE, or being played by VirtualDub), then HC is able to open another script without crashing. Now the "helper application" may be closed - HC can open the next script as well.

Looks to me as if HC would not be able to prepare everything that is necessary to open an AviSynth script; but what may it be? And why does it work well for me (Windows 2000 SP4, AMD Duron 800 - no SSE, AviSynth 2.55 and 2.56 alphas and betas)?

BTW: katjarella has a multi-monitor environment; but if that influences anything?!

[hank315]

If an AviSynth script is being processed in the background (e.g. being encoded by CCE, or being played by VirtualDub), then HC is able to open another script without crashing. Now the "helper application" may be closed - HC can open the next script as well.Really weird.
The strangest thing is the code used for avs input in the CLI version and the GUI version is exactly the same :confused:

[dragongodz]

yes very wierd. atleast it gives a starting point though. so it looks like its in the initialisation of the avisynth.dll i am guessing.

[hank315]

Katjarella just tested the new batch version, seems to work OK.
Will post it in a couple of hours.

[dragongodz]

cool. still strange though. :D

actually i am off to bed so will miss the release. :(
oh well something for me to look forward to in the morning. :)

[LigH]

Guys: Take your time! Better "finished" than "out of hell".

[hank315]

Just put the new batch version online, new manual included.

get it at: http://hank315.dyndns.org/HCbatch_012.zip

Changelog:
- encoding engine same as GUI version
- runs in a small window, showing statistics
- can use ini file and parameters

Have fun with it :)

[danpos]

@hank315

Great, mate!;)

Keep up the good work,

[doxville]

Thanks a lot for your great work!

There's one thing, I am afraid, that could lead to a crash: The *.dbs file is put into the same folder as the HCbatch.exe, and knowing that the *.dbs could get rather large, I would prefer putting it in another (maybe source-) folder. Is there a way to change this?
With my partition for program files being around 8 gig and filled to the half, i suppose it could easily be that it reaches the limit while converting.

Maybe a command in the ini-file could set another path for the *.dbs..??

Thanks again!

[Amnon82]

Great work hank!

As Doxville already posted, a path for *.dbs will be best ...

... maybe in version 0.12.1 ;)

[guada 2]

It is clear and very pleasant :)
Very, very good work.

Good continuation ;)

[johnnyquid]

Looks good hank. Thanks for all your work. One small comment for the new batch version. At the end of the encodes it seems to wait 5 seconds before quiting. In most cases this would be a trival amount of time, however when using it with dvd rebuilder, the encoder can be called hundreds of times. The disk I just encoded had close to 900 calls. If the waiting could be disabled it probably would speed up the whole process.

[dragongodz]

At the end of the encodes it seems to wait 5 seconds before quiting
yes this is so you can see the second pass information, time etc., otherwise that information in the window would be pointless. though i suggestsed more like a 3 second pause. :)

The disk I just encoded had close to 900 calls
are you sure about that number ? doesnt DVD-RB encode seperate cells ? that would be a hell of a lot of cells for just 1 dvd.

[Frameserver]

Originally posted by doxville
Thanks a lot for your great work!

There's one thing, I am afraid, that could lead to a crash: The *.dbs file is put into the same folder as the HCbatch.exe, and knowing that the *.dbs could get rather large, I would prefer putting it in another (maybe source-) folder. Is there a way to change this?
With my partition for program files being around 8 gig and filled to the half, i suppose it could easily be that it reaches the limit while converting.

Maybe a command in the ini-file could set another path for the *.dbs..??

Thanks again!

I just point on this "problem" some posts ago.
My partition is about 5gig's with 3.2gig's free but if I encode
a 192min movie the *dbs grows to 2.7gig's.
Works but is not really much space left.
Runs maybe into strange errors if the space gets empty.

[hank315]

@johnnyquid & frameserver
Just put the HC exe somewhere on you data partition, it doesn't has to be in the system partition.

[johnnyquid]

@dragongodz and hank315

Yep, It had around 880 cells. It was an extras disk that had a whole bunch of very short cells including many 1 frame stills. I have not verified what they actually were but I can image that a DVD might intentionally add a bunch of dummy cells as a copy protection measure to try to "overload" backup software. One frame cells used to cause RB+Quenc to fail. Since a logfile can be created with HC and batch mode typically means that you are not waiting "interactively" for it to end, I just think that the five second wait should be able to be disabled.

[LigH]

1-frame stills... obviously some kind of "photo gallery", or similar informational extras. They shall probably be excluded from further processing by ReBuilder (if not even excluded from the copy - depending on your needs). Except they are located in the menu, then it might be "menus switched via auto-activation buttons"...

[cmsoliveira]

Hi,

After hearing good things about things encvoder i decided to give it a try, and i must say, wonderful results where cce didn't behave that good.

However, i have a doubt. I'm encoding an interlaced source using hc12batch, with dvd-rb, and when doing the encoding the statistics window shows that it's using a method scan ZigZag instead of alternate (used for interlaced sources,i believe). I was able to change that by addind the statement "*methodscan alt" in the hc.ini (without quotes,of course)

The encoder shows that the source is interlaced, and according to the manual, it should use the alternate scan.

Is it ok to use any of the scan methods or is it a bug?

BTW, the source is Spiderman 2, PAL.
My pc is an AMD TBird 1,3 (Slow but get's the job done).

Hank315, great encoder, great job.

My sincere thank you.

[onesoul]

@cmsoliveira

This is not a bug! Of course you should use alternate scan when encoding interlaced content (you're talking about some extras, right? Spiderman movie is progressive).

The reason dvd-rb doesn't gives the appropriate commands is because it still doesn't support it (probably won't be long until jdobbs releases a new version supporting hc12_batch).

Cheers

[cmsoliveira]

Onesoul, thanx for the answer, but now i'm confused...

actually i'm encoding the movie, and analysing rebuilder.ecl i have the following:

progressive=0 --> (means that that cell is not progressive...)
alternate_scan=1 --> (...so use alternate_scan, i think)
intra_dc_prec=2

I'm considering the lack of support of rebuilder (for now)towards the HC, however, HC tells me that the source is interlaced...but uses ZigZag. So i'm confused.

Anyway, is it OK to use alternate scan in a progressive source, or vice-versa (i.e. could the output - image quality - suffer because of that choice)?

[Frameserver]

Originally posted by hank315
@johnnyquid & frameserver
Just put the HC exe somewhere on you data partition, it doesn't has to be in the system partition.

Yeah true...and no prob at all........only a suggestion
for the future.
Have tested hcbatch_012 runs flawless now.
You just change the bitrate from bits to kbits so it
confuses me at the start (showing some really strange bitrates ;-))
Is it right that hc deletes the *.dbs after encoding?
So no restart iss possible with that version?
Only problem for now is, it undersizes........
Just view the bottom of the log

--------------------------------------------
| HCbatch - MPEG2 encoder - rel. 0.12 beta |
--------------------------------------------

input: d:\pat.avs
output: d:\pat.m2v

--------------------
| encoder settings |
--------------------

profile: BEST
frames: 1 148157
framerate: 25.00
aspect ratio: 16:9
bitrate Kb/s: 5760
max. bitrate Kb/s: 9000
closed gops: yes
VBV check: yes
scene change det.: yes
interlaced: no
goplen,B-pic: AUTO
dc_precision: 10
scan method: ZIGZAG
time code: 0 0 0 0
CPU: MMX/SSE
matrix: CUSTOM

--------------------
| source stats |
--------------------

nr. of frames in source: 148157
width*height: 720*576
fps: 25.00
nr. of frames to encode: 148157
frames to encode: 1 - 148157

---------------------
| encoding - pass 1 |
---------------------

pass 1 encoding time: 4:31:35 (16295 s)
average fps: 9.1

--------------------------------
| encoding - intermediate pass |
--------------------------------

bitrate set to: 5760000 b/s
est. outfile length: 4166915 kB
intermediate encoding time: 3.3 s

---------------------
| encoding - pass 2 |
---------------------

pass 2 encoding time: 2:38:41 (9521 s)
average fps: 15.6

------------------
| encoding stats |
------------------

total encoding time: 7:10:21 (25821 s)

intra matrix used
deleted

non-intra matrix used
deleted

nr. of gops: 10723
nr. of frames: 148157
nr. of I-frames: 10723
nr. of P-frames: 60982
nr. of B-frames: 76452
average quant (non linear): 5.450
VBV underflows detected: 151
VBV underflows fixed: 151
minimum bitrate: 554
maximum bitrate: 8999
average bitrate: 5052

Any ideas/workarounds?
But great job hank315 just wonna point on this!

[onesoul]

cmsoliveira,

It is not HC encoder saying that the source is interlaced, it is what DVD-RB passed through. DVD-RB does nothing wrong, only analyzes the mpeg stream for flags and other information.

So you have the following choices:
- modify the hc.ini file for the correct setting
- use hc .011 batch (which is currently supported by dvd-rb pro)
- wait for next version of DVD-RB (pro version only maybe)

But, I am almost certain that this movie is progressive (i own that pal dvd also), so what I would do is:
Disable interlaced for that vts and encode it as progressive/zigzag, that option is available on dvd-rb.

For reference check this sticky at cce forum (http://forum.doom9.org/showthread.php?s=&threadid=70225&perpage=20&pagenumber=3).

edit: Like it is said on the previous link, you can check if the stream is interlaced or not by playing it on virtualdub(mod) or other media player and check for combed lines which would indicate an interlaced source.

Cheers

[hank315]

I was able to change that by addind the statement "*methodscan alt" in the hc.ini (without quotes,of course)The right syntax is *scanmethod alt ;)

For a progressive movie you can disable interlaced in DVD-RB with the avs/advanced options but this has to be done before the prepare phase.

But if HCbatch reports interlaced yes and scan zigzag that's a bug.
Will have a look at it.

In general if your movie is progressive zigzag is the best choice, for interlaced alternate CAN improve quality but not always.
It's just the order the coefficients are scanned so the VLC (Variable Length Coding) is optimal.

[cmsoliveira]

Hi,

once again, thnx :)

I've been searching in the forum about this subject, and you're right. Seems some dvd's have the flag interlaced on but analysing with virtualdub mod and dvd2avi the movie and some extras (for the movie in question) are progressive and not interlaced.

So the initial scan method ZigZag was correct.

Tnnx for all the help.

[LigH]

Originally posted by cmsoliveira
Seems some dvd's have the flag interlaced on but... the movie and some extras ... are progressive and not interlaced.
That's quite usual: Before DVD producers accidently set up the encoder to "progressive" and encode really interlaced material (which would result in horrible quality), they better lock the encoders to interlaced encoding, that doesn't hurt so much if the material is indeed progressive.

[hank315]

Just did a test with DVD-RB, without setting disable interlaced, came out as interlaced yes, scan zigzag.

In this case the best setting because the movie is progressive.
If the movie was really interlaced scan zigzag would be better.

But think I have to change this and set scan to alt if interlaced is flagged to be consistent with the ini file settings.

Think this is best:
movie progressive - encode progressive --> scan zigzag
movie progressive - encode interlaced --> scan zigzag
movie interlaced - encode interlaced --> scan alt

@LigH
Just read your post while typing this, think you're right but ofcourse it's best to encode progressive as progressive.
You might expect DVD producers know what they are doing, apparently not, saw a lot of DVD's which were progressive but were encoded interlaced but indeed that's not so bad as to encode interlaced source as progressive.

[LigH]

I once was a DVD authoring operator, a few years ago; I'm an "ex-insider"... ;)

Of course, it's best to encode depending on the material. But "assembly line" production is less expensive. :rolleyes:

[cmsoliveira]

Hi, again

@LigH
Yes, that's exactly what i've read from other posts...

@hank315
I don't want to make you chase bugs when, probably, they don't exist, so my procedure to backup spiderman 2, region 2, pal, unprocessed was:

->set the path do HCbatch.exe in the field "path to QuEnc" in rebuilder
-> set the mode as QuEnc
->Made an ini file with the following parameters
*maxbitrate 9000
*matrix qlb
*profile best
*logfile C:\Documents and Settings\XXXXXXXXXXXXXXX\My Documents\ReGet Downloads\DVD-RBv078PR\HCEnc\test.log
-> run prepare phase of rebuilder
-> Run encode phase

In the link below is an image with the arguments passed and encoder settings in a cell of the main movie

http://dadosrb.com.sapo.pt/Image2.jpg

If anything else is necessary, please say so.

Keep up the good work

@hank315
After finished writing this i saw the answer were you did a test with rb so this post might be unnecessary...

[onesoul]

@hank315

I've done a backup of interlaced source with dvd-rb using hc 0.01 batch and latest hc .012 batch. With this last version the result was in less ~400mb (now: 3917 mb, before: 4320 mb). I attach the log file from hc .012.

Regarding the settings used, the only differences were:
- now I used autogop. Previous version was set to 12 2.
- now i used mpeg default matrix. Previous version was set to hc matrix.

Cheers

[dragongodz]

onesoul - try again using 12 2 GOP setting and see if iut still undersizes.
i know rate control work is on the list of things to do though, it will take time though. :)

[Mug Funky]

That's quite usual: Before DVD producers accidently set up the encoder to "progressive" and encode really interlaced material (which would result in horrible quality), they better lock the encoders to interlaced encoding, that doesn't hurt so much if the material is indeed progressive.

yep. add to that the fact that often there's no previewing on a progressive screen - so even if something looks progressive on a TV monitor, it can still be interlaced (and probably is). you never know if there's field-shifts or mild field-blends until you look at things on a progressive screen. until then, it's much much safer to encode everything interlaced.

bear in mind that the people that send the master tapes are invariably going to bugger them up somehow (there's a long list of things that can go wrong... sometimes there's a tick in every box).

[buzzqw]

just another request :D

is possible to confine the bitrate by -b -maxbitrate in cq mode encoding ?

I try a cq 2 encoding and i had bitrate hight as 34000 (yes, 34000)

Thanks

BHH

[Mug Funky]

currently there's no way to do that (but it'd be cool if there were :))

[dragongodz]

is possible to confine the bitrate by -b -maxbitrate in cq mode encoding ?
it wouldnt be a constant quant mode then of course but yes it could be done, though you are for practical purposes talking about 1 pass VBR then. other modes such as CBR and 1 pass VBR would also be nice i know. remember though that hank315 does work so does this in his spare time. things will happen when and if he has time and the desire.

[buzzqw]

yes, sure i know i asked a "1 pass vbr"...

this is a request, that could be done or not...

But i hope Hank could have more spare time !

Thanks Hank for all your time !!!

BHH

[Rickkins]

@hank315...will this ever be made to work with a straight up avi file,
without having to go the avisynth route...??

[Frameserver]

Originally posted by Rickkins
@hank315...will this ever be made to work with a straight up avi file,
without having to go the avisynth route...??

For what?
You need at least two lines of "Code" in an AviSynth script to
allow avi input in hc.
For example type in:

AVISource("C:\temp\test.avi")
ConvertToYV12

->change "C" "temp" and "test" that it meets your path and filename.
Save this as test.avs (the proper extension is important) and run it
through HCbatch_012.
Of course You have to install AviSynth before.
Simple isn't it?

[hank315]

will this ever be made to work with a straight up avi file,
without having to go the avisynth route...??No, never.
Avisynth is just the optimal program to read and process input, using filters etc.

[Sounddude]

Every test I make with this encoder works fine. But when I do a whole movie and go to author with TMPGEnc DVD Author it complains that the GOP is too long.

What could be causing this? The samples I try work fine.

[onesoul]

Originally posted by dragongodz
onesoul - try again using 12 2 GOP setting and see if iut still undersizes.
i know rate control work is on the list of things to do though, it will take time though. :) I have been trying doing these encode again but surprisingly hc has been crashing (GPF) repeatedly on a couple of cells and it happens at the end of 2nd pass but the encoded file seems to be fine. This wasn't happening with autogop mode on, I did a fresh restart btw.

About the undersize, I'll update in a while about that (waiting for encoding the remaining cells=.

[hank315]

Every test I make with this encoder works fine. But when I do a whole movie and go to author with TMPGEnc DVD Author it complains that the GOP is too long.What GOP size was set?
Try to author it with IfoEdit or Muxman.

[dragongodz]

author with TMPGEnc DVD Author it complains that the GOP is too long.
Try to author it with IfoEdit or Muxman.
i agree 100%. i have seen the same problem reported for other encoders aswell. i really think this is a problem/bug with Tmpgend DVDAuthor. so yes please try some other authoring programs and report back.

oh and more details on what you are encoding, framerate, progressive/interlaced etc would not hurt. :)

[video]

Originally posted by hank315

@mean
I'm pretty much a complete Linux NOOB and don't have a Fortran95 compiler for Linux. There's an experimental compiler ( http://www.g95.org ) but the speed of it isn't optimal yet.


Great effort to see such a huge project as an mpeg2 encoder in fortran. However, I know very well, that in the HPC arena there nothing else, but fortran. regarding linux, there are intel's Fortran compile 8.1 for linux (ttp://www.intel.com/software/products/compilers/flin/) royalty free for non-commercial use. Even promising the support Compaq Visual Fortran.
regards and have fun.

[onesoul]

Originally posted by dragongodz
onesoul - try again using 12 2 GOP setting and see if iut still undersizes.
i know rate control work is on the list of things to do though, it will take time though. :) Hi again, with 12 2 GOP the output was 4026mb (autogop on delivered 3917) so it still undersizes comparing to hc 0.01 batch which did an ouput of 4320 mb.

[video]

hank315 okay it's my problem, but will it be possible to have a linux version? I'll happy with DV codec only (libdv.so) support. mor information on libdv can be found here: http://libdv.sourceforge.net/

[FredThompson]

Is there a hard-coded GOP length upper limit? If so, please consider making it very large. This would help for use with TyTool and other MPGE2 TV-oriented editors which work by re-encoding only modified GOPs.

[video]

Anyway hank315 a big respect for you. Considering 640x360@25fps at bitrate 1Mbps, with matrices hvs best, autogop and dc intra 8 your encoder produces output better visually than x264 does :D

[DoctorRad]

Just a couple of requests hank... could we have a way to load and save settings for the GUI version? And can we also have the command line back for those of us who are just as comfortable with that option?

Thanks,

Dr. Matt...
--

[Ebobtron]

Originally posted by hank315
Just put the new batch version online, new manual included.

get it at: http://hank315.dyndns.org/HCbatch_012.zip

Changelog:
- encoding engine same as GUI version
- runs in a small window, showing statistics
- can use ini file and parameters

Have fun with it :)

Where you been, see above Dr. Matt.

[ernstblaauw]

@Hank315

Your encoder is really great! I've got a question: Can you add the latest batch version to your first post in this topic, and also add a changelog to your first post?

[Xesdeeni]

Has anyone tried to encode HD resolutions? Will they encode at all? I still don't have an encoder that will allow me to create HD test streams to calibrate my HDTV. Will this encoder do it?

Xesdeeni

[Ebobtron]

@Xesdeeni

What resolutions

[hank315]

Can you add the latest batch version to your first post in this topic, and also add a changelog to your first post?Will do that, there will be an update of the batch version as well.
Some bugs are solved, in this case I was the first who saw them ;)
EDIT: not entirely correct, jdobbs also reported one...

Will make one complete zip-package of it, GUI version, batch version and manual.
Coming soon...

[DoctorRad]

Originally posted by Ebobtron
Where you been, see above Dr. Matt.

Heh, not wading through 18 pages of previous forum replies, that's where ;) Thanks for the pointer!

Matt...

[hank315]

Has anyone tried to encode HD resolutions? Will they encode at all? I still don't have an encoder that will allow me to create HD test streams to calibrate my HDTV. Will this encoder do it?No, max. resolution is 720 * 576.

[hank315]

HCbatch updated.

Changelog:
- fixed an error where the scan flag wasn't set properly for interlaced source using argument input
- added *WAIT command to speed up exit after encoding
- some other minor bugs fixed
- 1, 2 and 3 frame clips are now encoded using Constant Quant

Download contains GUI and batch version + updated manual.
Will be in the download section soon.

Also at: http://hank315.dyndns.org/HC_012_total.zip

[Nocturno]

thanks Hank, will try asap, used the previous batch version in my avi2dvd tool, seems to work great.will do more testing when i find the time.

Great work :D

[BugsBunny]

:) will give it a try have some movies comming in begining of week

hope it can take advantage od dual xeons..

looks decent BIG THANKS for the work

[Thor263]

I posted a message (http://forum.doom9.org/showthread.php?s=&postid=630375#post630375) over in the DVD-Rebuilder forum that I think might actually be a HCbatch problem w/ a possible loop. Because of the ongoing discussion in that thread, I just wanted to make sure it was noticed. :)

If you would like additional information, please let me know.

[hank315]

@Thor263
Could you post the avs file which was processed, might help solving it.

[gigaset2004]

@hank

what´s about low bitrate improvment ?

[Indignity]

hank, thanks for your work on this!!! Liking it so far.

I've got a question in regards to the batch encoding.. more like "how to do it"... I've read the manual about 20 times & have tried a few variables I've thought would work for this, but I'm stumped.

This is what's getting me from reading the manual & trying to apply it to run a batch encode:
HCbatch –i d:\movies\test1.avs –o d:\movies\test1.m2v –b 4000 –maxbitrate 9000 –ini d:\movies\hc.ini

I've tried placing this at the beginning of the hc.ini file, which didn't work(in series... test1, test2,..ect).. I tried placing this series in the *infile & *outfile with no luck either.

I've also tried creating an hcbatch.ini file as well.

Call me thick please!!! :D

Is there any way you could maybe post an example of how to use the batch feature? Dummies like me that have never used raw command lines for encoders, will probably get stuck trying to use it.

Great work thus far!!!!

Thanks for your time & look forward to your response!

[Xesdeeni]

Originally posted by Xesdeeni
Has anyone tried to encode HD resolutions? Will they encode at all? I still don't have an encoder that will allow me to create HD test streams to calibrate my HDTV. Will this encoder do it?Originally posted by Ebobtron
What resolutions ATSC HD resolutions are 1280x720 and 1920x1080 (http://support.gateway.com/s/CsmrEltrncs/DigitalTV/Shared/2517984faq42.shtml). Technically the 1080 is encoded at 1088, but I think the window is 1080. (I believe the PAL HD resolutions are the same, although the frame rates are obviously different.)Originally posted by hank315
No, max. resolution is 720 * 576. I can't remember from the MPEG-2 spec--do these higher resolution require the extension fields? Is that the reason for the hard-wired maximum resolution?

Xesdeeni

[cmsoliveira]

Hi,

I' doing my first full dvd backup using Dvd-RB and HC, but i have a question...

When using autogop does the encoder set a maximum length for the GOP, i.e, 15 frames for PAL and 18 frames for NTSC?

Or using a superior GOP could affect the quality, or playback of a movie?

[hank315]

When using autogop does the encoder set a maximum length for the GOP, i.e, 15 frames for PAL and 18 frames for NTSC?Using autogop sets the max. GOP length to 15.

I can't remember from the MPEG-2 spec--do these higher resolution require the extension fields? Is that the reason for the hard-wired maximum resolution?HC is meant to encode MPEG2 at Main profile @ Main level so 720*576 is the max. resolution.
The data structure used ATM is limited to 720*576.

[Thor263]

Originally posted by hank315
@Thor263
Could you post the avs file which was processed, might help solving it.

Ok, here it is. It's pretty basic.

#------------------
# AVS File Created by DVD Rebuilder
# VOBID:08, CELLID:01
#------------------
LoadPlugin("C:\Program Files\Video Tools\DVD Rebuilder\DGDecode.dll")
mpeg2source("D:\DVD TEMP\D2VAVS\V03.D2V",idct=7)
trim(71851,71864)
ConvertToYV12()

[video]

hank i feel you are definitely in the game :)

HC-12 (http://feliratok.rpotor.com/jadzia/enough.ac3.mpg) vs x264, build 180 (http://feliratok.rpotor.com/jadzia/enough.x264.mp4). All streams are 640x368@25fps, the bitrate is 1Mbps. Nothing else to say, but respect.
HC-12 setup: best quality, hsv best matrices and autogop.

[katjarella]

Source: PAL,704x576,1Frame

HC.ini: *INFILE F:\TEMP\MUXMAN\V03\TEST.AVS
*OUTFILE F:\TEMP\MUXMAN\V03\TESTX.M2V
*BITRATE 2500
*MAXBITRATE 9000
*LOGFILE F:\TEMP\MUXMAN\V03\TESTX.LOG
*MATRIX HVSBEST
*PROFILE BEST
*ASPECT 4:3
*CQ 1
*DC_PREC 10
*SCANMETHOD ZIGZAG
*CLOSEDGOPS
*NOSCD
*NOVBV
HCbatch rel. 0.12: profile: BEST
frames: 1 1
framerate: 25.00
aspect ratio: 4:3
bitrate Kb/s: 2500
max. bitrate Kb/s: 9000
pass: 1 (Constant Quant)
constant Q: 1
closed gops: yes
VBV check: no
scene change det.: no
interlaced: no
goplen,B-pic: AUTO
dc_precision: 10
scan method: ZIGZAG
time code: 0 0 0 0
CPU: MMX/SSE/SSE2
matrix: HVSBEST
*****
nr. of gops: 1
nr. of frames: 1
nr. of I-frames: 1
nr. of P-frames: 0
nr. of B-frames: 0
average quant (non linear): 1.000
VBV underflows detected: disabled
VBV underflows fixed: disabled
minimum bitrate: 999
maximum bitrate: 90403
average bitrate: 90403
HCbatch rel. 0.12a: profile: BEST
frames: 1 1
framerate: 25.00
aspect ratio: 4:3
bitrate Kb/s: 2500
max. bitrate Kb/s: 9000
pass: 1 (Constant Quant)
constant Q: 4
closed gops: yes
VBV check: no
scene change det.: no
interlaced: no
goplen,B-pic: AUTO
dc_precision: 10
scan method: ZIGZAG
time code: 0 0 0 0
CPU: MMX/SSE/SSE2
matrix: HVSBEST
*****
nr. of gops: 1
nr. of frames: 1
nr. of I-frames: 1
nr. of P-frames: 0
nr. of B-frames: 0
average quant (non linear): 4.000
VBV underflows detected: disabled
VBV underflows fixed: disabled
minimum bitrate: 999
maximum bitrate: 42018
average bitrate: 42018

Hello hank315!

Constant Q isn't working anymore in HCbatch v0.12a. Everything gets encoded with constant Q: 4. Is there a reason for it?

translate (c) Scharfis_brain :)

[hank315]

@Thor263
So a short clip...
Try to encode the avs file with the GUI version, bitrate 4000 and max bitrate 8000.

Constant Q isn't working anymore in HCbatch v0.12a. Everything gets encoded with constant Q: 4. Is there a reason for it?Reason is probably an error :D
Dragongodz also reported this but couldn't replicate the error.
Will do a similar test with your ini settings.

[Thor263]

Originally posted by hank315
@Thor263
So a short clip...
Try to encode the avs file with the GUI version, bitrate 4000 and max bitrate 8000.

The GUI version worked just fine. I'm going to have to do some more testing...

[freelock7]

HC0.12beta seems to encode with a wrong prediction.
Is it known or has to be fixed?
--------------------
| encoder settings |
--------------------

profile: GOOD
frames: 1 1001
framerate: 25.00
aspect ratio: 16:9
bitrate Kb/s: 6000
max. bitrate Kb/s: 8000
restart: no
closed gops: no
VBV check: yes
scene change det.: yes
interlaced: yes
goplen,B-pic: AUTO
dc_precision: 9
scan method: ALTERNATE
time code: 0 0 0 0
CPU: AUTO
matrix: MPEG

--------------------
| source stats |
--------------------

nr. of frames in source: 1001
width*height: 720*576
fps: 25.00
nr. of frames to encode: 1001
frames to encode: 1 - 1001
movie length to encode: 0:00:40 (40.04 s)
est. outfile length: 29326 kB

---------------------
| encoding - pass 1 |
---------------------

pass 1 encoding time: 0:01:49 (109 s)
average fps: 9.1

--------------------------------
| encoding - intermediate pass |
--------------------------------

bitrate set to: 6000000 b/s
est. outfile length: 29326 kB
intermediate encoding time: 0.0 s

---------------------
| encoding - pass 2 |
---------------------

pass 2 encoding time: 0:01:22 (82 s)
average fps: 12.2

total encoding time: 0:03:13 (193 s)

------------------
| encoding stats |
------------------

nr. of gops: 96
nr. of frames: 1001
nr. of I-frames: 96
nr. of P-frames: 397
nr. of B-frames: 508
average quant (non linear): 4.986
VBV underflows detected: 0
VBV underflows fixed: 0
minimum bitrate: 999
maximum bitrate: 7958
average bitrate: 4919

[Thor263]

Originally posted by Thor263
The GUI version worked just fine. I'm going to have to do some more testing...

I was able to replicate the problem in the GUI. When I matched the bitrates specified in the HC.INI file created by DVD-RB, the looping also occurred in the GUI. Both the batch and GUI state they are in the intermediate pass.

HC.INI file contents:

*BATCH
*PROFILE GOOD
*INFILE D:\DVD TEMP\D2VAVS\V03001300008001.avs
*OUTFILE D:\DVD TEMP\D2VAVS\V03001300008001.m2v
*BITRATE 6818
*MAXBITRATE 8360
*WAIT 0
*ASPECT 16:9

Let me know if you would like me to try anything else.

[manolito]

@Thor263

I noticed that you have the *BATCH parameter in your HC.INI. HCBatch does NOT support this parameter, only the old Non-GUI version of HC did that. Don't know if this causes the loop, but I think it's worth a try...

Cheers
manolito

[video]

hank minor bug only, but when i wanted enter manually the gop size, i got a nice runtime error and the app is terminated.

[hank315]

@video
What GOP size did you enter?
GUI or batch version?

[DoctorRad]

A fairly general problem I can't seem to find a way around: if the file referenced in the AVS script is on the network, i.e. not on the PC on which HC is running, HC hangs when I try to open the file. The AVS file plays fine in WMP over the network.

Can anyone else replicate this??

Thanks,

Matt...

[video]

Originally posted by hank315
@video
What GOP size did you enter?
GUI or batch version?

I intended entering 12. but just try delete the value from the combo and you will get 'visual fortran runtime error' - of course gui version, namely hc12_total rls. B frames input suffering from the same problem.

[video]

Originally posted by DoctorRad
A fairly general problem I can't seem to find a way around: if the file referenced in the AVS script is on the network, i.e. not on the PC on which HC is running, HC hangs when I try to open the file. The AVS file plays fine in WMP over the network.

Can anyone else replicate this??

Thanks,

Matt...

have you refrenced the file with unc name or the file is on an already mapped network drive? If not, try to map the remote share as a network drive. somtimes even M$ word having problems reading directly from unc references.

[DoctorRad]

Originally posted by video
have you refrenced the file with unc name or the file is on an already mapped network drive? If not, try to map the remote share as a network drive. somtimes even M$ word having problems reading directly from unc references.

Same problem with UNC refs and mapped drives.

Matt...

[video]

Originally posted by DoctorRad
Same problem with UNC refs and mapped drives.

Matt...

please consider that different levels of file service access are provided by the os for local and remote files. there is no problem with reading, opening, but maybe not possible to seek freely within the file. sequential forward read guaranteed to work on unc remote files, however that's might be not enough to dgindex/avisynth. for proper support, i feel dgindex might be prepared to handle files from an unc...

[Spyn]

hi hank315,

could you add an option to disable P-frames to HC ?

it seems that at very or extremly high bitrates enabling only I-frame give the best quality.

[dragongodz]

it seems that at very or extremly high bitrates enabling only I-frame give the best quality.
i assume you mean not for dvd bitrates. why not disable B frames and set a short GOP length (try 5) as that should be pretty damn good. or are you failing to reach a target bitrate with that ?

[Spyn]

Originally posted by dragongodz
i assume you mean not for dvd bitrates. why not disable B frames and set a short GOP length (try 5) as that should be pretty damn good. or are you failing to reach a target bitrate with that ?

i just want to test mpeg2 vs mpeg4 (xvid) at very high bitrates (higher than 10000 kbits), b-frames are already disabled.

i would like to test an encode with HC at Q1 with I-frame only, i think this is the best quality possible for mpeg2.

Tmpgenc allow to enable I-frame only but don't allow bitrate higher than 15000 kbits or something like this.

[dragongodz]

i just want to test mpeg2 vs mpeg4 (xvid) at very high bitrates (higher than 10000 kbits)
you shouldnt really be able to tell any difference if both are using just I frames and Q1.

Tmpgenc allow to enable I-frame only but don't allow bitrate higher than 15000 kbits or something like this.
change the profile from MP@ML to MP@HL to use higher bitrates.

[LigH]

Somehow, I prefer practical over academical tests... :sly: ;)

[dragongodz]

Somehow, I prefer practical over academical tests..
i agree. especially considering, as i said,
you shouldnt really be able to tell any difference if both are using just I frames and Q1. :D

[DoctorRad]

Originally posted by video
please consider that different levels of file service access are provided by the os for local and remote files. there is no problem with reading, opening, but maybe not possible to seek freely within the file. sequential forward read guaranteed to work on unc remote files, however that's might be not enough to dgindex/avisynth. for proper support, i feel dgindex might be prepared to handle files from an unc...
Errr... yes... my stupidly large capture source file would appear to be the cause... all 16.3Gb of it. A 10-frame test file works fine.

Sorry for the noise!

Matt...

[hank315]

Tmpgenc allow to enable I-frame only but don't allow bitrate higher than 15000 kbits or something like this.HC is meant for DVD creation, it can't create clips with only I-frames, why should it?

But it can create clips with a bitrate higher than 15000:
- GOP length 2, no B-frames --> IPIPIPIP scheme
- Constant Q run with Q=1
- no VBV check
- Profile best
- Disable scene change

Bitrate is unlimited this way, might go as high as 30000 or more...
Just don't see the need to do it this way :confused:

[LigH]

Because it is not configurable (yet?): Which luminance range does HC expect?

According to my tests, it seems to use "PC scale" (0..255).

Some MPEG encoders allow to set the input range. Other MPEG encoders seem to silently expect "TV scale" (16..235). I'll have to check that again before choss-comparing several different encoders, else comparison results will be rather useless, I'm afraid...

[Spyn]

Originally posted by hank315

Just don't see the need to do it this way :confused:

I want to know which codec between mpeg2 and mpeg4 (xvid) is the best at Q1, at 20000 kbits, at 10000 kbits and at 5000 kbits.

To get the better quality at Q1 i need to enable only I-frames.

[Xesdeeni]

Originally posted by Spyn
To get the better quality at Q1 i need to enable only I-frames. I don't know where you get that information, but I'm pretty sure it's wrong.

I-frames are simply encoded with no reference to other frames. P and B frames are deltas from other frames. But that doesn't make I-frames inherently higher quality. As long as the encoder is reasonable, the quality of P and B frames can be equal to I frames. And P and B frames use fewer bits! (on average, P-frames use about half the bits of I-frames for the same quality, while B-frames use 1/3-1/4 the bits of I-frames for the same quality.)

Also, a P or B frame can be composed of macroblocks that are I-like, if the encoder determines that this is more efficient (e.g. at scene changes that do not force an I-frame). So the least common denominator for P and B frames is an I-frame structure.

In reality (again, assuming a reasonable encoder algorithm), you will not get better quality at a given bit rate using I-frames only. And you are likely to have less quality, since you are not using the bits as efficiently.

Think of it this way:

Assume you feed an encoder the same frame over and over and encode a 30 fps video at 10 Mbps. For I-frame only, each frame is individually encoded. That means each frame could use 10 Mb / 30 fps = 333 Kb.

If you use P and B frames, a good encoder will see that there is no difference frame to frame. It will encode the delta frames as having no change, which is a very small amount of data, and all the bits can be concentrated on the occasional I-frame. Assuming an 18 frame GOP, that means one I-frame every 18 frames, or 1.667 I-frames every second. That equates to 10 Mb / 1.6667 fps = 6 Mb.

Certainly the 6 Mb I-frame will be better quality than the 333 Kb one.

Edit: BTW, this same holds true for MPEG-4. P and B frames are your friends :)

Xesdeeni

[Spyn]

Originally posted by Xesdeeni

But that doesn't make I-frames inherently higher quality. As long as the encoder is reasonable, the quality of P and B frames can be equal to I frames.

wrong

15000 kbits encode, source = very good mpeg2

SSIM results :

I-frame only : 96.30
1 I-frame every 5 P-frame : 95.46
1 I-frame every 14 P-frame : 94.51
default : BBIBBP : 95.20

[LigH]

Does is make any sense to use an already once DCT-based quantized material? Better always use original material which has not yet been encoded with the same technique, especially not with worse quality than your test options. Re-quantization does not provide valuable results.

For example, VQEG test samples are available as raw UYVY files, if you don't own a capturing source.

[video]

I think intra-only mpeg comes into the play if you have a hardware mpeg2 digitizer board and you would like to end up with a capture what is seamless key frame editable/cuttable. More recently these kind of setups appliing CBR encoding schemes as well.

[LigH]

Indeed, I-frame only MPEG is cuttable at any frame. But most MJPEG AVI codecs may provide similar, if not even better, quality.

[video]

Originally posted by LigH
Indeed, I-frame only MPEG is cuttable at any frame. But most MJPEG AVI codecs may provide similar, if not even better, quality.

in the last third of 90ies i had a damn good PCI mpeg-2 capture card(refurbished broadcast device). at that times the 1GByte avi barrirer was a huge problem, no opendml, no avisynth, no DV. But with mpeg i was able to have even 8G-16G captures on NT4. :)

[Xesdeeni]

Originally posted by Spyn
wrong

15000 kbits encode, source = very good mpeg2

SSIM results :

I-frame only : 96.30
1 I-frame every 5 P-frame : 95.46
1 I-frame every 14 P-frame : 94.51
default : BBIBBP : 95.20 You are basing your presumption on at least three assumptions:

1. The encoder is a representative encoder.

As I said, if the encoder wants, it can encode P and B frames in a way that is identical to I-frames. But the fact that it apparently does not is not indicative that I-frames are superior either. It's only indicative of the algorithms in the encoder. It may even be indicative of errors in your decoder. Many more calculations are necessary for P and B frames, so there is much more room for the decoder (and encoder) to make mistakes.

2. SSIM is accurate.

Keep in mind that P and B frames end up encoding data in a different way than I-frames. Different isn't necessarily worse. But SSIM's self admitted ("the SSIM metric still isn't perfect") bias may be on errors that occur less often in I-frames. Specifically, there may be motion errors introduced due to motion vector-based compression. There may in fact be less spatial error due to Discrete Cosine or quantization (and most probably will, given there are more bits to use). But if the SSIM algorithm is more sensitive to motion errors, it will erroneously report a lower quality.

To see an example of this. Use a computer generated (i.e. noiseless) source of some object moving across the frame at some non-integral number of pixels (e.g. 3.3 pixels per frame). I believe MPEG-2 has only a 1/2 pixel motion accuracy, so the location of the object can be off by as much as that. But fractions of pixels are not necessarily worse to the eye than DCT/Q artifacts. How SSIM reacts to these errors may be quite different.

3. MPEG source is a good source.

The MPEG source already has artifacts created by the first encoding. If the second encoder encodes similiarly to the first, you might find fewer additional artifacts in the result. But if the second encoder uses a different technique, even if only for scene change (forced I-frame) encoding, the artifacts could be multiplied. And using I-frames may very will avoid any additional motion errors at all. But this is not an indication of the validity of the second encoder.


Don't trust hype or arbitrary measurements. Trust your eyes. And be careful to understand what effects you are inadvertently introducing into the process.

Xesdeeni

[Spyn]

Originally posted by Xesdeeni

Don't trust hype or arbitrary measurements. Trust your eyes. And be careful to understand what effects you are inadvertently introducing into the process.

Xesdeeni [/B]

I prefer to trust SSIM cause i think no eyes are able to make the difference between each files at this bitrate, mine can't...

and i don't think the encoder can encode P and B frames as good as an I-frame, i got a partially proof of this by the test i made, you has no proof except you're words...

[Xesdeeni]

Originally posted by Spyn
i don't think the encoder can encode P and B frames as good as an I-frame, i got a partially proof of this by the test i made, you has no proof except you're words... Try reading the MPEG-2 specification. Note that each macroblock can be encoded as an Intra block. Note that all I-frame macroblocks are Intra blocks.

As for a specific encoder and judgement tools, please use whatever criteria you prefer. But please understand that results from one encoder are not universal.

Xesdeeni

[Spyn]

Originally posted by Xesdeeni
Try reading the MPEG-2 specification. Note that each macroblock can be encoded as an Intra block. Note that all I-frame macroblocks are Intra blocks.

As for a specific encoder and judgement tools, please use whatever criteria you prefer. But please understand that results from one encoder are not universal.

Xesdeeni

I don't know what is an intra or non-intra blocks, i just know that a P-frame use colors that are same to the previous frame to save bits.

I think it use the same colors even if they are not exactly the same, for example for a RGB pixel 13-13-13 of a frame, if the same pixel of the next P-frame is 13-14-13, it would use the pixel of the previous frame.

so colors are not exactly the same and that's why, quality is worse.

this is difficult for me to say all that i want in english, i hope you undestand me...

[video]

Originally posted by Spyn
I think it use the same colors even if they are not exactly the same, for example for a RGB pixel 13-13-13 of a frame, if the same pixel of the next P-frame is 13-14-13, it would use the pixel of the previous frame.
You sholud have a very good CRT to see any differences betwwen color #0C0C0C and #0C0D0C. speaking about a TFT, they can do 6bits/channel and trying to display the remained 2 bits via dithering. Even best EIZO S-IPS panels are 6bit depht per channel. The do a histogram recolorize on the fly to achive better look than cheap models.

keep in mind that mpeg is a publication format, and therefore not suited for intermediate mastering/editing even when there are mpeg capable video editors.
there are far more better 'intra only' codecs outside, such as morgan's MJPEG 2000, DV, and the EBU videocodec

@hank315
Would it hurt to extend the encoder's resolution to 1920x1080/1440x1080? Just because i've got a sony cinealta for a while, and it produces masters in size what is too much for even a 300gByte harddisk :)

[skal]

Hi,

Originally posted by video
Would it hurt to extend the encoder's resolution to 1920x1080/1440x1080? Just because i've got a sony cinealta for a while, and it produces masters in size what is too much for even a 300gByte harddisk :)

gee... i'd be delighted if you could shoot few samples with that cam, at such hi-res, and make them available somewhere (not 300gByte of course;). Raw, unprocessed, material of that size is hard to find for tests...

bye!
-Skal

[Koepi]

That would really be nice for testing, agreed :-)

Looking forward to some proper input material... (not just always decoded mpeg2 sources.)

Cheers
Koepi

[video]

Originally posted by skal
Hi,



gee... i'd be delighted if you could shoot few samples with that cam, at such hi-res, and make them available somewhere (not 300gByte of course;). Raw, unprocessed, material of that size is hard to find for tests...

bye!
-Skal

I appreciate your request, but I moved to the rural area, so I've got a gsm phone what can do a 9600bps. At work there is http, but it's just incoming. however cinealta really ROCKS :) I really love heavy cams, what can be put on your shoulder ;)

[Spyn]

someone know where i can find an uncompressed source ?

[konst]

Hi,
Sorry for my English, my native language is Russian and for my, maybe, stupid question. I'm a newbie to MPEG2 encoding and to this forum.
HCmanual has 10 intra- / non-intra matrices and somes a little bit similar to each other. Tell me, please, common practical recommendation for use these various matrices.
For examle:
*MATRIX NOTCH - appropriate to "slow" noiseless interlaced source at low bitrate (~2000kbps), produce 'soft' slightly blured MPEG with less details in black area ...and so on;
*MATRIX BACH1 - appropriate to "fast", dynamic, noisy interlaced or progressive source at average bitrate (~4000 - 6000 kbps), produce 'sharp' MPEG with more details in black area ... etc.

May be will a good think include such information to HCmanual?

Or advise any links about this, please.

I've read above posted explanations of LigH, posts from http://forum.doom9.org/showthread.php?s=&threadid=89897 and other Google's links, but, as for me, this almost 'pure' theory and mathematics :-) and don't practical recommendations.

I want to test HC and compare it with Procoder2 and I think: more correct matrices -> more correct result.

Once again sorry and Big Thanks.

[gigaset2004]

@ konst

I think tht´s the wrong threat for your questions.

So it woulb be best to read several postings for your need´s

Short expl.

You could get more compression with the matrix you choose. Bute every encoder needs normaly a specail matrix to get best results.

Means get a DVD movie on a CD with 700 MB

I coul recommend the Quenc 59 beta 3 with the QLB matrix.

So that´s it with the basic.

But you need the avisynth and so on.........

[konst]

@ gigaset2004

Thanks for quick replay.


I think tht's the wrong threat for your questions

The thread's name: "Who wants to test a new MPEG2 encoder".
I want :-) I want also to receive best result for my source.

OK.
My source is mainly a DV (PAL, interlaced) produced by Canon XM2 at daylight; the target is a ~60 - 90min DVD with IPB GOP 12/3 or IP 12/2, 2-pass VBR, 720x576.
My source is sometimes a DV (PAL, interlaced) produced by ADVC-300 from an old VHS; the target is a ~120 - 180min DVD with GOP IPB 12/3, 2-pass VBR, 480x576.

HCmanual has matrices:
* Default - 83; 33 (values from low right in the corner intra- ; non-intra)
*MATRIX HC - 87; 79
*MATRIX NOTCH - 79; 44
*MATRIX QLB - 87; 34
*MATRIX BACH1 - 83; 44

Their intra- parts are a little bit similar to each other. As far as I've understood, the lesser values in the low right area the more accuracy encoder can provide in high frequency range, the accuracy in small details is more correspondingly.

HCmanual has others matrices a little bit similar to each other:
*MATRIX HVSGOOD - 115; 144
*MATRIX HVSBETTER - 115; 137
*MATRIX HVSBEST - 115; 129

The detached matrices:
*MATRIX JAWOR1CD - 84; 124
*MATRIX AUTOQ2 - 94; 99


Bute every encoder needs normaly a specail matrix to get best results.

What is the matrices are more appropriate for me in HC-encoder (in both case)?
My 'ancient' PC very slow for ALL matrices testing :-(


Means get a DVD movie on a CD with 700 MB
I coul recommend the Quenc 59 beta 3 with the QLB matrix.
So that's it with the basic.
But you need the avisynth and so on.........

Thanks, but so far I use my friend's Procoder2.
Are avisynth really don't needed for HC-encoder?

P.S. A DVD movie on a CD with 700 MB - that's cool :-)

[gigaset2004]

@ konst

So far, but your source is interlaced thats the biggest problem.

Normally the matrices are optimised to produce a good quality on progressive material.

You have to choose :

encode interlace material - needs more space.

deinterlace the material - needs a avisynth sript and more know how.

I recommend you to read all the material in the forums and pick up best for you.

After reading the next 3- 5 days you know more what you do.

Your way is not a 1-2 klick programm..........

In the geramn forum there are a few members they have made test with Dv material so take a look to traslate.

look for scharffis brain or Kika or mb1

[Abond]

Normally the matrices are optimised to produce a good quality on progressive material.
Huh? This is new for me...

[guada 2]

For me too...

[skal]

Hi Video,

Originally posted by video
I appreciate your request, but I moved to the rural area, so I've got a gsm phone what can do a 9600bps.

too bad, thanks anyway!

Ahowever cinealta really ROCKS :) I really love heavy cams, what can be put on your shoulder ;)

Stop that, i'm drooling ;)

bye!
-Skal

[konst]

@ gigaset2004
Thanks for your philosophizing.
I watch my interlaced material by ordinary interlaced TV and deinterlace the material in this case is incorrectly, imho.
I'll have to must to spend many days on testing of all matrices.

@ ALL
Sorry, i did a search before that was posted and could not find any practical recommendation, i should of tested all matrices before posting.

[jeanl]

Guys,
I'm a bit of a newbie, so sorry if my question seems naive, or has been answered somewhere else (I did search though!). I've been trying HC on some home-videos (DV-NTSC, interlaced), and I've noticed that the quality of the output goes up tremendously if I de-interlace the input prior to encoding. To do my comparisons, I was using HC Best quality to encode into MPEG (using the default MPEG matrices - is that the problem?) at VBR with an average of 7.5Mb/s (pretty high).
- When encoding the interlaced input, I selected "interlaced" and chose Bottom Field First (as expected I think).
- To encode a de-interlaced stream, I ran either a simple Bob() or better, TomsMoComp(), and of course, deselected interlaced in HC.

My question is:
- Is it normal that the results should be so much better on the de-interlaced material? I watched the output with PowerDVD on my monitor. On a frame-by-frame basis, the MPEG version is very close to the DV original, with very little blocking, and no combing. This was not the case when encoding the original interlaced video: I could see a lot of combing on the output in that case.
- Am I doing something wrong when I encode the interlaced material? Why do I see so much combing on the output?
- Is there a better way to do what I'm doing? (like I said, I'm very much a noob).

Thanks for any input!
jeanl

[rhayman]

Jeanl: I think this is because interlaced material requires higher bitrates because the image is more complicated when its interlaced.
So if you can encode at a high bitrate i think your supposed to keep the video interlaced.

If im encoding at low bitrates (2000-2500) i usually deinterlace too because you get less blocking.

Hopefully someone can explain it better. I dont really know that much about it.

[jeanl]

Originally posted by rhayman
Jeanl: I think this is because interlaced material requires higher bitrates because the image is more complicated when its interlaced.
So if you can encode at a high bitrate i think your supposed to keep the video interlaced.

If im encoding at low bitrates (2000-2500) i usually deinterlace too because you get less blocking.

Hopefully someone can explain it better. I dont really know that much about it.
Thanks rhayman, that's useful input. I do want to keep my video interlaced, so I think de-interlacing to get better quality is the wrong way to go about it in this case. I'll do some more experiments tonight.
Jeanl

[onesoul]

@jeanl

What you describe i weird, HC does a pretty good job on interlaced content for what I have tested.
Could you please give the avisynth script when encoding as interlaced, did you use any filters?
And btw did you use autogop on? Try gop 12 2.

What I don't see very clear is how could you see combing on powerdvd as it deinterlaces unless you told it not to.

Cheers

[FredThompson]

Since when does HC support interlaced source?

@jeanl,
If your source is interlaced and not a candidate for decombing, you really should leave it interlaced if at all possible. Deinterlacing removes half the temporal information, introduces visual mud and creates artifacts, no matter how good the deinterlacing routine. You just can't smash the equivalent of 50 (PAL) or ~60 (NTSC) distinct images into 25 or 30 images and keep the quality. It is not uncommon for the fields of a single frame of interlaced source to have very different content.

Decombing is different. If the original source was progressive, it's almost always a good idea to decomb.

Early video compression didn't support interlaced source. MPEG1 doesn't, as an example. MPEG2 does support interlaced source. However, as I mentioned at the top of this post, make sure HC supports interlaced source before you try to use it for that.

[onesoul]

Originally posted by FredThompson
Since when does HC support interlaced source? What? Have you tried HC? It has supported interlaced encoding from the beginning...
You can check the test I made with pal DVD interlaced source.

[jeanl]

Yes, there's no doubt that HC supports interlaced source. And no, my source isn't originally progressive, so I should definitely leave it interlaced. This confirms that I'm on the wrong path with my de-interlacing, and that the higher encoding quality that I seem to observe is probably the result of faulty comparisons (on a progressive display, my PC monitor!).
jeanl

[jeanl]

Originally posted by onesoul
What? Have you tried HC? It has supported interlaced encoding from the beginning...
You can check the test I made with pal DVD interlaced source.
onesoul, do you have a link to that test? Couldn't find it... I'd be interested to compare to what I get (although I'm NTSC :( ).
jeanl
EDIT: Never mind! I found them, really interesting! Thanks!

[onesoul]

The test still appears on first page of this forum (other mpeg1/2 encoder) and it is here (http://forum.doom9.org/showthread.php?s=&threadid=91405),
Clicking on www under my signature will redirect you to the test also.

I don't recommend using any filtering when encoding interlaced source (except for reinterpolate411() when using mainconcept dvcodec), it's too painful to do it the right way (takes too long and it's not 100% accurate).

[jeanl]

Originally posted by onesoul

I don't recommend using any filtering when encoding interlaced source (except for reinterpolate411() when using mainconcept dvcodec), it's too painful to do it the right way (takes too long and it's not 100% accurate).
so according to you, I should encode straight from my interleaved DV NTSC video, no pre-processing, and the results should be pretty good (especially at 6-7Mb/s), right? I'll try again tonight, and i'll try to be more careful in the way I compare the output.
jeanl

[onesoul]

Originally posted by jeanl
so according to you, I should encode straight from my interleaved DV NTSC video, no pre-processing, and the results should be pretty good (especially at 6-7Mb/s), right? I'll try again tonight, and i'll try to be more careful in the way I compare the output.
jeanl Yes. Please check back to us then.

[FredThompson]

Originally posted by onesoul
What? Have you tried HC? It has supported interlaced encoding from the beginning...
You can check the test I made with pal DVD interlaced source. Look at pages 13 and 14 of this thread.

[Ebobtron]

@FredThompson
Sorry, but I would remind you, about page 13, that a number of us tried to explain to you that HC encodes interlaced frames.

The error as I remember it was caused by feeding HC field based material.

HC has from the start done as good or better on interlaced sources as any other encoder I have tried. The only progressive input I have, I have to make as all my sources are captured camera or broadcast.

[jeanl]

Originally posted by jeanl
so according to you, I should encode straight from my interleaved DV NTSC video, no pre-processing, and the results should be pretty good (especially at 6-7Mb/s), right? I'll try again tonight, and i'll try to be more careful in the way I compare the output.
jeanl
I did more tests tonight, and indeed, my (big) mistake was to compare the original DV and the output MPEG in PowerDVD on a frame-by-frame basis. Instead, I burned on a DVD using various options and bitrates, and watched on my TV. I must say I can't see any of the artifacts that I was seeing before (macroblocks etc) at 7.5Mb/s. In particular, deinterlacing before encoding did not improve things on my TV (but it sure did in PowerDVD, which makes sense).

So HC works really well for me. At the bitrates I was using (about 6Mb/s) I couldn't tell the difference with procoder (but my TV isn't the best so take all this with a grain of salt).
Now my question would be (but this is getting OT), how can you compare interlaced encoding on a frame-by-frame basis??? It's OK in steady shots, but in moving shots (the most interesting to compare) the combing effect you see in vdubmod for example makes any comparison impossible (it masks macroblocks etc). This makes it a bit hard for me to select the right bitrate.
Do you guys have a way you recommend to compare interlaced material?

jeanl
P.S. Thanks for the help!

[FredThompson]

I must have an HC mental block...

Ignore my babbling above...well, the babbling part, that is...

@jeanl,

PowerDVD has its own internal deinterlacing routines.

NTSC DV uses a different form of data reduction for chroma (color) than does MPEG2. NTSC DV is one chroma value for a 4-pixel rectangle. MPEG2 is one chroma value for a 2x2 pixel square. Long story short, converting from one to another involves some loss and visual artifacting.

If you poke around in the AviSynth and VirtualDub areas you'll find explanations of how to display the fields as side-by-side displays, not interleaved. That would be a way to compare the post-encoding results.

[jeanl]

Originally posted by FredThompson

If you poke around in the AviSynth and VirtualDub areas you'll find explanations of how to display the fields as side-by-side displays, not interleaved. That would be a way to compare the post-encoding results.
Yes, that's what I'm looking for, a way to look at a single field so I can see blocking artifacts and things like that. I'll look around. Can you open an avs script in vdubmod? That would be one way to do that...
jeanl

[FredThompson]

Go poke around the VirtualDub areas. You just open the AviSynth script.

[Calca_P]

I am backing up a 3 and a 1/4 hr movie which the quality is shocking (obviously taken from Anologue source and put onto DVD)
I am using HC 12 beta GUI and I have just completed the first pass of a backup (DVD) via Avisynth.

The filter script Im using is Kasandro's? RemoveDirt and Didee's iip, so if you are familiar with it, you would know it is a 1 frame/second process.

My questions are,
* Does the second pass encode use the filtering process again? (Avisynth script)

I know that in the readme file, it is written that a 2 hr movie takes approx 3 hrs for 1st pass and 1 hr for the second pass but my second pass is reading 1 frame/s again which is certainly no quicker than the first pass encode.
( I thought the second pass is created from the information from the first pass)

* I also have "Scene Change" and "VBV" ticked, should these be "Unticked"? as I frames would already be present at the appropriate frames for the scene change.

I did notice that when testing with other Avisynth scripts that HC threw up an error" Cant find DGDecode.dll " when loading the Avs, so I copied it ( DGDecode.dll ) into the HC exe folder without changing the call in the script and it worked.

I apreciate the work and time going into this project by Hank315 and others and am impressed with the outcome and the writings of others, Good on yers.

Thanks

[Koepi]

If you want to speedup encoding like you describe, you need to store your filtered output with a lossless codec and use that as source - else the filtering is _of course_ done on 2nd pass again.

During the first pass only statistical data gets collected (frame sizes, size of motion data, size of texture data, frame type and so on).

You could do a search for "two pass encoding" and go to posts back in i.e. 2002 or earlier, there the basics for 2pass encoding are explained.

Cheers
Koepi

[onesoul]

@ jeanl

I don't think there's nothing wrong comparing 2 interlaced frames, that's what I did on my test, using original and encoded ones. Like it is described, I used avscompare, very nice features including able to zoom and frame stepping (crashes sometimes but it's very good although).

[jeanl]

Originally posted by onesoul
@ jeanl

I don't think there's nothing wrong comparing 2 interlaced frames, that's what I did on my test, using original and encoded ones. Like it is described, I used avscompare, very nice features including able to zoom and frame stepping (crashes sometimes but it's very good although).
Have you tried to do that with moving frames? I noticed that in your tests, you seemed to have mostly steady shots (I couldn't detect any combing). If you compare 2 interlaced frames where the two fields have just been combined, the combing artifacts (the mouse teeth) are so obvious that it becomes really hard to see encoding artifacts... What de-interlacing technique does avscompare use to show you a full image from 2 consecutive interlaced fields?
jeanl

[onesoul]

@ jeanl

Th initial logo of HBO I agree that there isn't combing lines because it is originally progressive encoded as interlaced.

But the rest of the frames (from 375 forward in the test) all have combing lines and all are interlaced. And I repeat myself but if you wrongly use some spatial or temporal filter you can ruin the interlaced image. Currently I choose not to use any filtering on interlaced source unless I really need to.

Avscompare doesn't do any deinterlacing :), it displays just like it is or like you told it to do, no mistakes there. edit: it displays the full frame.

Here (http://forum.doom9.org/showthread.php?s=&threadid=40675) is the link for original thread and here (http://aquaplaning.20m.com/) is official page of avscompare, be sure to get the 2.5 avs version.

[jeanl]

Originally posted by onesoul

Avscompare doesn't do any deinterlacing :), it displays just like it is or like you told it to do, no mistakes there. edit: it displays the full frame.
So it simply combines the two fields. I will try to see if I can modify the script to make it show only 1 of the two fields (at half the vertical size). This of course won't show any combing/mouse-teeth, and should allow me to clearly see macroblocks caused by the encoding (at least I hope so!). I'll keep you posted! Thanks for the links.
jeanl

[onesoul]

separatefields() or verticalreduceby2() will do the trick, but imo it isn't necessary to look for macroblocks. And be sure to turn brightness really high to see more details, (don't forget to turn back brightness after testing or you will lose your eyesight ;))

[jeanl]

Originally posted by onesoul
separatefields() or verticalreduceby2() will do the trick, but imo it isn't necessary to look for macroblocks. And be sure to turn brightness really high to see more details, (don't forget to turn back brightness after testing or you will lose your eyesight ;))
I'll show you a couple frames where you can't see any detail because of the combing (I have to get back home for that). Thanks for the avisynth commands!
jeanl

[jeanl]

I found that using an avs script in vdubmod to separate the fiels was a very unreliable when the source is an mpeg file (lots of frames are dropped, etc). Instead, I'm inserting a de-interlacing filter (drop top field for example) which allows me to only visualize one of the two fields. This makes it much easier to compare the original and the encoded versions. I'm finding that HC gives very excellent results, even at bitrates down to 5Mb/s. The quality is as good (in my opinion) as CCE, although it appears that CCE does some spatial smoothing (even though I turned that off in the gui) so the outputs look slightly different in that respect.
So HC really rules. Thanks a million to the author.
jeanl

[onesoul]

Why not say better? eheh. It surely does rule :).
Originally posted by jeanl
I found that using an avs script in vdubmod to separate the fiels was a very unreliable when the source is an mpeg file (lots of frames are dropped, etc). Did you use DGIndex to create a d2v file of the mpeg file? I use that method to load it in avscompare, no dropped frames whatsoever, it can be used on virtualdub also.
Using the verticalreduceby2() avisynth command will drop a field also (unlike separatefields() which displays every field separated).

[jeanl]

Originally posted by onesoul
Why not say better? eheh. It surely does rule :).
Did you use DGIndex to create a d2v file of the mpeg file? I use that method to load it in avscompare, no dropped frames whatsoever, it can be used on virtualdub also.
Using the verticalreduceby2() avisynth command will drop a field also (unlike separatefields() which displays every field separated).
I want to use avscompare next, since having the two videos in a single app will simplify things for me. I'll keep you posted. But no I didn't use DGIndex. I used DShowSource() (or something like that) in my avs script. Maybe that was the problem.
jeanl

[onesoul]

Originally posted by jeanl
I want to use avscompare next, since having the two videos in a single app will simplify things for me. I'll keep you posted. But no I didn't use DGIndex. I used DShowSource() (or something like that) in my avs script. Maybe that was the problem.
jeanl Directshowsource, yes, that was the problem, but there's another one, you can't load multiple mpeg's with directshowsource, with the created .d2v files you can do it.

Maybe I didn't mention before, but the best feature of avscompare is that you can load up to 4 streams in different windows, which is awesome for you toggle bewtween them, zoom, etc, you can really see differences that you wouldn't be able before between encoded ones and original.

Have fun ;)

[hank315]

New release: HC013

- GUI and batch version updated
- faster encoding
- bitrate control updated
- encoding profiles updated: FAST - NORMAL - BEST
- QLB matrix updated
- added *NOSEQ_ENDCODE command to suppress output of the sequence endcode
- some other minor bugs fixed

This is a major update, alot of new code so there may be some (new) bugs :)
Will be in the download section soon.

But also at: http://hank315.dyndns.org/HC_013.zip

[dragongodz]

did a quick test using the gui version. not last frame missed like the last beta version did. will try CLI version to confirm shortly. :)

EDIT:
yep HC 0.13 CLI/batch working fine aswell. HC 0.12 is still getting stuck on the last frame however but hey 0.13 is here now so doesnt matter. :D

[onesoul]

@ dragongodz

See the quenc thread please.

@ hank315

From my ongoing tests, the new hc .13 is doing great, (even) better quality. I will try again the interlaced source.

[dragongodz]

See the quenc thread please.
onesoul - back at you. ;)

hank315 - may i suggest you actually start a new thread with this release since this one is 24 pages long already ? also a better title ,such as HC 0.13, would be an idea. :D

[ernstblaauw]

Originally posted by hank315
[B]New release: HC013
- bitrate control updated


Is it possible now to use one pass for DVD encoding (so HC will stay under a certain maximum bitrate, specified before encoding)?

[freelock7]

-Hank (about the 013 release)
The 'grain' effect in the picture is less perceptible while a lot of macroblocs are present now at low bitrate-like in QuEnc. :)
The constant Q could be changed as constant bitrate more useful-for me- and the quantization level is once again too high under 4000kbs. Damage because the bitrate control is perfect.

[vio_man]

Which should be the best matrix for XviD/DivX source? I've previously used the Mpeg matrix and I got undersized mpeg2.

[zettai]

I'm with freelock - I'd love to see some sort of combination of constant Q and max bitrate where the encoder encodes at a certain quant unless the bitrate is exceeded and a lower quant is used for that section.

[DoctorRad]

Originally posted by hank315
New release: HC013

Is this the same as the 0.13beta you sent me?

Thanks,

Matt...
--

[johnnyquid]

@hank315

Was just trying out the new batch13 version. It is working fine most of the time but it did go into an infinte loop when trying to fix a VBV buffer underflow. The finding and fixing counters just keep on going up with no progress in the number of frames processed. The same video encoded without any trouble using batch12a version.

[onesoul]

@ johnnyquid

Maybe posting your log would help.

Cheers

[johnnyquid]

I went back and tried to duplicate the problem but was not able to. It occurred during a DVD Rebuilder run so to try to duplicate the problem I edited the rebuilder.inf file to restart the encode on the file which I saw the problem. This time the encode completed without a problem. Go figure. If I can get the problem to repeat I will repost.

[lithoc]

Just a quick test. Really impress with speed of HC.
Thanks.

Bug:
1.) KVCD-NOTCH matrix didn't work.
2.) Didn't save settings when quit. Everything is load to default. (May be add extra button to load to dafult settings)

[ffreese]

[QUOTE]Originally posted by lithoc
[B]Just a quick test. Really impress with speed of HC.
Thanks.

How fast is HCenc013 compared to CCE or to HCEnc012?

[dragongodz]

Didn't save settings when quit. Everything is load to default.
yes hank315 knows this. we have discussed it so several times. i think,rightly, he was more interested in improving speed and rate control however for this release.

How fast is HCenc013 compared to CCE or to HCEnc012?
slower than CCE and faster than HC 0.12. speed increase seems to scale with the profile, that is BEST has the largest speed increase while FAST has the smallest. also hank315's P4 gained more increase than my Athlon XP 2400+ proportinatly. try it for yourself and see.

hey i just noticed another little bonus. me mentioned in the thanks.txt. nice. ;)

[alfixdvd]

Really impress with speed of HC 0.13 in the first pass : 25.6 fps against 12.6 of 0.12 version. The time cuts by half.

The size of m2v file is now more acurate with the bitrate specified.

Thanks hank315, good job

[lamster]

I tried running it on an older machine, and got a crash with an "illegal instruction" error when it tried to execute the following:

00402973 psadbw mm1,mm2

The computer has the following:
Processor Type: x86 Family 6 Model 6 Stepping 5

Might be nice to check for any hardware requirements up-front...

The other crash was running on a different computer, using Rebuilder. This one isn't an unsupported processor, since the crash occurred while processing the 3rd segment; the first 2 completed OK.004722FF FF 15 F0 31 47 00 call dword ptr ds:[4731F0h]
00472305 8B 48 0C mov ecx,dword ptr [eax+0Ch] ; EAX == NULL
It's reproducible; the first time I had Rebuilder set to its default of Quality = Normal; the second time I set it to "Best".

Just reproduced it with 0.12. I was watching it, and the exception occurred after the encoding was completed. Here's a screen shot:

[lamster]

@hank315 - A couple of questions...

Since the crash is occurring after the countdown showing "(n) encoding finished", is it safe to assume that the resulting M2V file is good? And, is that countdown really necessary? If I'm calling HCbatch from Rebuilder, and not looking at your GUI screen, it seems to be just a waste of time.

One more question - I'm letting it chug away (hoping that the answer to my first question is Yes ;) ) and I've seen only a few segments crash this way - it's on the eleventh, and only 3 have run into this. Is there any info I can provide to help you debug this? After it completes, I'll try seeing if there's any commonality amongst the ones that had a problem.