Who wants to test a new MPEG2 encoder

[Frameserver]

Originally posted by hank315


@Frameserver
Will have a look at the Avisynth freezing, could you be more specific please, which Avisynth version etc.

Hi hank315,
AviSyth version is 2.54
OS Win 2000 SP4 + all Security Updates
Processor is an Athlon XP (Thoroughbred B)
so MMX+SSE is present.
Feel free to ask for additional infos You need to
solve this issue.

Got a look at the CPU Auto or MMX/SSE Set, makes no difference.
Still freezes immediatly after selecting the *.avs file.
This must be an GUI issue cause the ini driven HCenc works
fine on my machine.
Maybe the upcoming batch version is my favorite..........
don't like GUI's too ;-))

Best wishes
Frameserver

[hank315]

@LigH & katjarella
Think it's better a real Windows/Avisynth guru should look at this, which I'm certainly not...
The only thing I can think of is loading the TCPdeliver.dll which also references WS2_32.dll, TCPdeliver.dll is loaded by Avisynth from the Avisynth plugin directory.
Also did a debug session, even my Logitech mouse loaded WinSock stuff, that's simply the windows world, there were times when programming was easy :D


@Ebobtron
For a whole movie it's easier to keep the relative error small, there will always be an absolute difference which will be larger in a relative way for a short clip ofcourse.
No didn't change anything in the bitrate control yet, still has to be done.

[LigH]

TCPDeliver in the AviSynth AutoLoad directory... that's quite possible. Seems that IWatch doesn't distinguish well between DLLs which are loaded from the EXE directly, and loaded from called DLLs indirectly.

[Ebobtron]

Originally posted by hank315

@Ebobtron
For a whole movie it's easier to keep the relative error small, there will always be an absolute difference which will be larger in a relative way for a short clip ofcourse.
No didn't change anything in the bitrate control yet, still has to be done.

Ok, I will accept that to cover your skill as a programmer:rolleyes:. However, I did not miss the file size difference between the two encodes. Maybe I was drunk that night. Oh yea, I do not drink. Then something must have stopped the encoder. I know it did not crash. Maybe my wife shut it down. Yea, blame it on her.

I shall retest 0.01 just for grins.
I’m out of here.

[Ebobtron]

Ok, my bad, never mind, except the things I said about what a wonderful piece of work this encoder is.

I will go back and shine that star of yours some more.

[Frameserver]

Originally posted by hank315
@Frameserver
Will have a look at the Avisynth freezing, could you be more specific please, which Avisynth version etc.

Some additional things.
If HCenc GUI freezes the system is not harmed. You can work
in a normal way. Also there is no entry in the System event log
(error or warning) caused or depending by HCenc.
The only change is in the Task-Manager where the HCenc status
changed from "Wird ausgeführt" (running) to "Keine Rückmeldung"
(no response).
I post the german terms cause it's a german Win and if I translate it
picking the wrong words nobody's understanding what I'm talking about.

Another issue is the databasefile hc.dbs created from the first pass.
This file gets really huge (depending on the filmlength).
In my case (maybe in other cases too) the system drive (partition)
runs out of space (a 192minutes project creates a hc.dbs of 2.6Gig).
Don't know what kind of error this produces.
Maybe its better that You store this hc.dbs at In- or Outfile position
cause there will be enough space and name it unique (like the In- or
Outfile) so we can handle some project's parallel and not only once at a time.
Maybe it's possible in the future to save project settings as a file/job and batch this (maybe already done in the upcoming version).
Just a suggestion.

Solong
Frameserver

[LigH]

katjarella tried with all the current versions (at least between 2.5.4 and 2.5.6 Backout 2005-03-15).

[LigH]

Dear hank315,

please don't take this question as "urging you"... but: Are you at least able to guess the current state of the CLI development (something like "##% finished, testing this and that")? I just would like to tell some eagerly waiting members of the german board, how long it probably may take. (Please don't mind so many people asking so many times... and don't hesitate to ask for specific beta test systems, e.g. low-featured CPUs.)

[dragongodz]

to those having the freezing - do you have anything in your Avisynth plugins folder besides directshow.dll ? if yes can you please remove them all and try HC again ? ok may not make any difference but best to check all options. :)

LigH - there is no point hank315 trying to give a time for release since it will no doubt depend on testing time etc. :D

[LigH]

I just would like to know if we shall rather plan "this weekend", or "next month". ;)

[Doom9]

I think planning for "when it's done" should do it ;) It works even for Duke forever.. and that is somewhat of a record holder.

[LigH]

Duke Nukem Forever... don't remind me. :rolleyes: I start to doubt if anyone will be glad anymore, when that piece will finally be "done"... :sly: - especially since they stopped this "Duke Nukem: Source" fan mod! :(

Okay - so I'll continue to beg for "patience" in our board.

[Frameserver]

Originally posted by dragongodz
to those having the freezing - do you have anything in your Avisynth plugins folder besides directshow.dll ? if yes can you please remove them all and try HC again ? ok may not make any difference but best to check all options. :)


@dragongodz
Makes no sence cause only the GUI Version freezes not the .ini driven
one.
Maybe there are minor differences in the called dll's between XP and
W2k (diff.versions)and that's the prob

[hank315]

New batch version will be ready this weekend.

About the Avisynth freezing issue it's hard to solve for me because just don't get the error on every PC I tried.

Possible solutions
- Look at dragongodz post
- Make sure you are using the latest Avisynth version
- Avisynth.dll is packed, unpack it with UPX, might help, don't know

Question:
Those who have this error, can program in C (MS visual studio) and has some time available, please PM me, think it's the only way to solve this.

[dragongodz]

Frameserver - dont knock it until you have tried it please. i have already seen plenty of times where some plugins work fine with most programs but then stuff up with some others.

About the Avisynth freezing issue it's hard to solve for me because just don't get the error on every PC I tried.
and this is what people need to remember. its hard to fix a bug you simply do not get.

[fabrice]

Hi,

Originally posted by dragongodz
[B]to those having the freezing - do you have anything in your Avisynth plugins folder besides directshow.dll ? if yes can you please remove them all and try HC again ? ok may not make any difference but best to check all options. :)


I got freezing with hc0.12, and after cleaning avisynth plugin folder, it don't freeze anymore!

I'll be trying to find from which plugin comes the problems.

Thanks
Fabrice

[katjarella]

Originally posted by fabrice
I got freezing with hc0.12, and after cleaning avisynth plugin folder, it don't freeze anymore! I'll be trying to find from which plugin comes the problems.

Originally posted by hank315
- Make sure you are using the latest Avisynth version
- Avisynth.dll is packed, unpack it with UPX, might help, don't know


my Avisyth Folder is Clean (NO FILES INCLUDE)
Last Avisynth Version Test pack/unpack (inc MD5 Checksum)


.................................
avisynth_2005_02_10
\30e22ca6d620905fcae697e06e68385a *C:\\WINNT\\system32\\avisynth.dll
.................................
avisynth_2005_02_10_unpack
\6f1eb206807964872f383b816ecb6fa2 *C:\\WINNT\\system32\\avisynth.dll
.................................
avisynth_2005_02_28
\39976969bdb9668bc7826340aa71428b *C:\\WINNT\\system32\\avisynth.dll
.................................
avisynth_2005_02_28_unpack
\e0de5005fb4d7b8c8f299f7835c82e0d *C:\\WINNT\\system32\\avisynth.dll
.................................
avisynth_2005_03_10
\f7b9649bd4d27b48be383dc1473d1df7 *C:\\WINNT\\system32\\avisynth.dll
.................................
avisynth_2005_03_10_unpack
\056c1897bb59d252e066657308f3883f *C:\\WINNT\\system32\\avisynth.dll
.................................
avisynth_2005_03_15
\7ef5d8bc85ee15c80c5ef7da93b5d3c1 *C:\\WINNT\\system32\\avisynth.dll
.................................
avisynth_2005_03_15_unpack
\b683323246a6afee4005ac65ba3832f4 *C:\\WINNT\\system32\\avisynth.dll
.................................
avisynth_2005_03_15_backout
\79ddc3b28ffc069c0b7717ea68ebb6d4 *C:\\WINNT\\system32\\avisynth.dll
.................................
avisynth_2005_03_15_backout_unpack
\61724a00c17eaa046a83de56adb23a80 *C:\\WINNT\\system32\\avisynth.dll

always Freezing :( // the "old" NOGUI Version working

[dragongodz]

I got freezing with hc0.12, and after cleaning avisynth plugin folder, it don't freeze anymore!
nice to know it fixes it for atleast 1 person. :D

katjarella - can you post the script you use and versions of any plugins used in it etc ?

[katjarella]

my Script:

BlankClip(length=500,width=720,height=576,pixel_type="YV12",fps=25,audio_rate=48000,stereo=true,color=$000000)
ShowSMPTE(offset="00:00:00:00",x=64,y=28,font="Arial",size=24,text_color=$FFFFFF)

moment dragongodz... I have a workaround... LigH translate :)

[LigH]

Your wish is my command, master... :rolleyes: :D
__

It seems that HC 0.12 GUI freezes as soon as the AviSynth script is opened - when HC runs alone.

If an AviSynth script is being processed in the background (e.g. being encoded by CCE, or being played by VirtualDub), then HC is able to open another script without crashing. Now the "helper application" may be closed - HC can open the next script as well.

Looks to me as if HC would not be able to prepare everything that is necessary to open an AviSynth script; but what may it be? And why does it work well for me (Windows 2000 SP4, AMD Duron 800 - no SSE, AviSynth 2.55 and 2.56 alphas and betas)?

BTW: katjarella has a multi-monitor environment; but if that influences anything?!

[hank315]

If an AviSynth script is being processed in the background (e.g. being encoded by CCE, or being played by VirtualDub), then HC is able to open another script without crashing. Now the "helper application" may be closed - HC can open the next script as well.Really weird.
The strangest thing is the code used for avs input in the CLI version and the GUI version is exactly the same :confused:

[dragongodz]

yes very wierd. atleast it gives a starting point though. so it looks like its in the initialisation of the avisynth.dll i am guessing.

[hank315]

Katjarella just tested the new batch version, seems to work OK.
Will post it in a couple of hours.

[dragongodz]

cool. still strange though. :D

actually i am off to bed so will miss the release. :(
oh well something for me to look forward to in the morning. :)

[LigH]

Guys: Take your time! Better "finished" than "out of hell".

[hank315]

Just put the new batch version online, new manual included.

get it at: http://hank315.dyndns.org/HCbatch_012.zip

Changelog:
- encoding engine same as GUI version
- runs in a small window, showing statistics
- can use ini file and parameters

Have fun with it :)

[danpos]

@hank315

Great, mate!;)

Keep up the good work,

[doxville]

Thanks a lot for your great work!

There's one thing, I am afraid, that could lead to a crash: The *.dbs file is put into the same folder as the HCbatch.exe, and knowing that the *.dbs could get rather large, I would prefer putting it in another (maybe source-) folder. Is there a way to change this?
With my partition for program files being around 8 gig and filled to the half, i suppose it could easily be that it reaches the limit while converting.

Maybe a command in the ini-file could set another path for the *.dbs..??

Thanks again!

[Amnon82]

Great work hank!

As Doxville already posted, a path for *.dbs will be best ...

... maybe in version 0.12.1 ;)

[guada 2]

It is clear and very pleasant :)
Very, very good work.

Good continuation ;)

[johnnyquid]

Looks good hank. Thanks for all your work. One small comment for the new batch version. At the end of the encodes it seems to wait 5 seconds before quiting. In most cases this would be a trival amount of time, however when using it with dvd rebuilder, the encoder can be called hundreds of times. The disk I just encoded had close to 900 calls. If the waiting could be disabled it probably would speed up the whole process.

[dragongodz]

At the end of the encodes it seems to wait 5 seconds before quiting
yes this is so you can see the second pass information, time etc., otherwise that information in the window would be pointless. though i suggestsed more like a 3 second pause. :)

The disk I just encoded had close to 900 calls
are you sure about that number ? doesnt DVD-RB encode seperate cells ? that would be a hell of a lot of cells for just 1 dvd.

[Frameserver]

Originally posted by doxville
Thanks a lot for your great work!

There's one thing, I am afraid, that could lead to a crash: The *.dbs file is put into the same folder as the HCbatch.exe, and knowing that the *.dbs could get rather large, I would prefer putting it in another (maybe source-) folder. Is there a way to change this?
With my partition for program files being around 8 gig and filled to the half, i suppose it could easily be that it reaches the limit while converting.

Maybe a command in the ini-file could set another path for the *.dbs..??

Thanks again!

I just point on this "problem" some posts ago.
My partition is about 5gig's with 3.2gig's free but if I encode
a 192min movie the *dbs grows to 2.7gig's.
Works but is not really much space left.
Runs maybe into strange errors if the space gets empty.

[hank315]

@johnnyquid & frameserver
Just put the HC exe somewhere on you data partition, it doesn't has to be in the system partition.

[johnnyquid]

@dragongodz and hank315

Yep, It had around 880 cells. It was an extras disk that had a whole bunch of very short cells including many 1 frame stills. I have not verified what they actually were but I can image that a DVD might intentionally add a bunch of dummy cells as a copy protection measure to try to "overload" backup software. One frame cells used to cause RB+Quenc to fail. Since a logfile can be created with HC and batch mode typically means that you are not waiting "interactively" for it to end, I just think that the five second wait should be able to be disabled.

[LigH]

1-frame stills... obviously some kind of "photo gallery", or similar informational extras. They shall probably be excluded from further processing by ReBuilder (if not even excluded from the copy - depending on your needs). Except they are located in the menu, then it might be "menus switched via auto-activation buttons"...

[cmsoliveira]

Hi,

After hearing good things about things encvoder i decided to give it a try, and i must say, wonderful results where cce didn't behave that good.

However, i have a doubt. I'm encoding an interlaced source using hc12batch, with dvd-rb, and when doing the encoding the statistics window shows that it's using a method scan ZigZag instead of alternate (used for interlaced sources,i believe). I was able to change that by addind the statement "*methodscan alt" in the hc.ini (without quotes,of course)

The encoder shows that the source is interlaced, and according to the manual, it should use the alternate scan.

Is it ok to use any of the scan methods or is it a bug?

BTW, the source is Spiderman 2, PAL.
My pc is an AMD TBird 1,3 (Slow but get's the job done).

Hank315, great encoder, great job.

My sincere thank you.

[onesoul]

@cmsoliveira

This is not a bug! Of course you should use alternate scan when encoding interlaced content (you're talking about some extras, right? Spiderman movie is progressive).

The reason dvd-rb doesn't gives the appropriate commands is because it still doesn't support it (probably won't be long until jdobbs releases a new version supporting hc12_batch).

Cheers

[cmsoliveira]

Onesoul, thanx for the answer, but now i'm confused...

actually i'm encoding the movie, and analysing rebuilder.ecl i have the following:

progressive=0 --> (means that that cell is not progressive...)
alternate_scan=1 --> (...so use alternate_scan, i think)
intra_dc_prec=2

I'm considering the lack of support of rebuilder (for now)towards the HC, however, HC tells me that the source is interlaced...but uses ZigZag. So i'm confused.

Anyway, is it OK to use alternate scan in a progressive source, or vice-versa (i.e. could the output - image quality - suffer because of that choice)?

[Frameserver]

Originally posted by hank315
@johnnyquid & frameserver
Just put the HC exe somewhere on you data partition, it doesn't has to be in the system partition.

Yeah true...and no prob at all........only a suggestion
for the future.
Have tested hcbatch_012 runs flawless now.
You just change the bitrate from bits to kbits so it
confuses me at the start (showing some really strange bitrates ;-))
Is it right that hc deletes the *.dbs after encoding?
So no restart iss possible with that version?
Only problem for now is, it undersizes........
Just view the bottom of the log

--------------------------------------------
| HCbatch - MPEG2 encoder - rel. 0.12 beta |
--------------------------------------------

input: d:\pat.avs
output: d:\pat.m2v

--------------------
| encoder settings |
--------------------

profile: BEST
frames: 1 148157
framerate: 25.00
aspect ratio: 16:9
bitrate Kb/s: 5760
max. bitrate Kb/s: 9000
closed gops: yes
VBV check: yes
scene change det.: yes
interlaced: no
goplen,B-pic: AUTO
dc_precision: 10
scan method: ZIGZAG
time code: 0 0 0 0
CPU: MMX/SSE
matrix: CUSTOM

--------------------
| source stats |
--------------------

nr. of frames in source: 148157
width*height: 720*576
fps: 25.00
nr. of frames to encode: 148157
frames to encode: 1 - 148157

---------------------
| encoding - pass 1 |
---------------------

pass 1 encoding time: 4:31:35 (16295 s)
average fps: 9.1

--------------------------------
| encoding - intermediate pass |
--------------------------------

bitrate set to: 5760000 b/s
est. outfile length: 4166915 kB
intermediate encoding time: 3.3 s

---------------------
| encoding - pass 2 |
---------------------

pass 2 encoding time: 2:38:41 (9521 s)
average fps: 15.6

------------------
| encoding stats |
------------------

total encoding time: 7:10:21 (25821 s)

intra matrix used
deleted

non-intra matrix used
deleted

nr. of gops: 10723
nr. of frames: 148157
nr. of I-frames: 10723
nr. of P-frames: 60982
nr. of B-frames: 76452
average quant (non linear): 5.450
VBV underflows detected: 151
VBV underflows fixed: 151
minimum bitrate: 554
maximum bitrate: 8999
average bitrate: 5052

Any ideas/workarounds?
But great job hank315 just wonna point on this!

[onesoul]

cmsoliveira,

It is not HC encoder saying that the source is interlaced, it is what DVD-RB passed through. DVD-RB does nothing wrong, only analyzes the mpeg stream for flags and other information.

So you have the following choices:
- modify the hc.ini file for the correct setting
- use hc .011 batch (which is currently supported by dvd-rb pro)
- wait for next version of DVD-RB (pro version only maybe)

But, I am almost certain that this movie is progressive (i own that pal dvd also), so what I would do is:
Disable interlaced for that vts and encode it as progressive/zigzag, that option is available on dvd-rb.

For reference check this sticky at cce forum (http://forum.doom9.org/showthread.php?s=&threadid=70225&perpage=20&pagenumber=3).

edit: Like it is said on the previous link, you can check if the stream is interlaced or not by playing it on virtualdub(mod) or other media player and check for combed lines which would indicate an interlaced source.

Cheers

[hank315]

I was able to change that by addind the statement "*methodscan alt" in the hc.ini (without quotes,of course)The right syntax is *scanmethod alt ;)

For a progressive movie you can disable interlaced in DVD-RB with the avs/advanced options but this has to be done before the prepare phase.

But if HCbatch reports interlaced yes and scan zigzag that's a bug.
Will have a look at it.

In general if your movie is progressive zigzag is the best choice, for interlaced alternate CAN improve quality but not always.
It's just the order the coefficients are scanned so the VLC (Variable Length Coding) is optimal.

[cmsoliveira]

Hi,

once again, thnx :)

I've been searching in the forum about this subject, and you're right. Seems some dvd's have the flag interlaced on but analysing with virtualdub mod and dvd2avi the movie and some extras (for the movie in question) are progressive and not interlaced.

So the initial scan method ZigZag was correct.

Tnnx for all the help.

[LigH]

Originally posted by cmsoliveira
Seems some dvd's have the flag interlaced on but... the movie and some extras ... are progressive and not interlaced.
That's quite usual: Before DVD producers accidently set up the encoder to "progressive" and encode really interlaced material (which would result in horrible quality), they better lock the encoders to interlaced encoding, that doesn't hurt so much if the material is indeed progressive.

[hank315]

Just did a test with DVD-RB, without setting disable interlaced, came out as interlaced yes, scan zigzag.

In this case the best setting because the movie is progressive.
If the movie was really interlaced scan zigzag would be better.

But think I have to change this and set scan to alt if interlaced is flagged to be consistent with the ini file settings.

Think this is best:
movie progressive - encode progressive --> scan zigzag
movie progressive - encode interlaced --> scan zigzag
movie interlaced - encode interlaced --> scan alt

@LigH
Just read your post while typing this, think you're right but ofcourse it's best to encode progressive as progressive.
You might expect DVD producers know what they are doing, apparently not, saw a lot of DVD's which were progressive but were encoded interlaced but indeed that's not so bad as to encode interlaced source as progressive.

[LigH]

I once was a DVD authoring operator, a few years ago; I'm an "ex-insider"... ;)

Of course, it's best to encode depending on the material. But "assembly line" production is less expensive. :rolleyes:

[cmsoliveira]

Hi, again

@LigH
Yes, that's exactly what i've read from other posts...

@hank315
I don't want to make you chase bugs when, probably, they don't exist, so my procedure to backup spiderman 2, region 2, pal, unprocessed was:

->set the path do HCbatch.exe in the field "path to QuEnc" in rebuilder
-> set the mode as QuEnc
->Made an ini file with the following parameters
*maxbitrate 9000
*matrix qlb
*profile best
*logfile C:\Documents and Settings\XXXXXXXXXXXXXXX\My Documents\ReGet Downloads\DVD-RBv078PR\HCEnc\test.log
-> run prepare phase of rebuilder
-> Run encode phase

In the link below is an image with the arguments passed and encoder settings in a cell of the main movie

http://dadosrb.com.sapo.pt/Image2.jpg

If anything else is necessary, please say so.

Keep up the good work

@hank315
After finished writing this i saw the answer were you did a test with rb so this post might be unnecessary...

[onesoul]

@hank315

I've done a backup of interlaced source with dvd-rb using hc 0.01 batch and latest hc .012 batch. With this last version the result was in less ~400mb (now: 3917 mb, before: 4320 mb). I attach the log file from hc .012.

Regarding the settings used, the only differences were:
- now I used autogop. Previous version was set to 12 2.
- now i used mpeg default matrix. Previous version was set to hc matrix.

Cheers

[dragongodz]

onesoul - try again using 12 2 GOP setting and see if iut still undersizes.
i know rate control work is on the list of things to do though, it will take time though. :)

[Mug Funky]

That's quite usual: Before DVD producers accidently set up the encoder to "progressive" and encode really interlaced material (which would result in horrible quality), they better lock the encoders to interlaced encoding, that doesn't hurt so much if the material is indeed progressive.

yep. add to that the fact that often there's no previewing on a progressive screen - so even if something looks progressive on a TV monitor, it can still be interlaced (and probably is). you never know if there's field-shifts or mild field-blends until you look at things on a progressive screen. until then, it's much much safer to encode everything interlaced.

bear in mind that the people that send the master tapes are invariably going to bugger them up somehow (there's a long list of things that can go wrong... sometimes there's a tick in every box).