Who wants to test a new MPEG2 encoder

[dragongodz]

I've tried some configurations and kvcd matrix gives me lower bitrate with same quality comparing to standard matrix. If you could recommend me a matrix that does better job at bitrates 1500-2000 I will give it a try (I've also tried QLB but with worse results).

what i want to know is if the size increases closer to the target.
as for prefering different matrrices...each to their own.

actually lokking that you are doing svcd(except for out of spec GOP length and frame size) you shouldnt really need a very low bitrate matrix. a target of 2000 is practically equivilent to around 4000 for dvd so more mid range than low. so most matrices should look atleast ok. you are not doing any filtering or anything before the encoder are you ?

no i am not saying thats causing the problem of not hitting your target, as i said the rate control does need more work.

[dragongodz]

Xeno86 - i have come across an interesting anomoly. i did some tests with HC with a target bitrate of 2000. all were slightly under the target bitrate (closest was 1964) according to the HC gui information. now i loaded 1 in to bitrate viewer and it said the average was 1564. hmm thats strange i thought. so i loaded the same m2v in to mpeg stream eye and it said the real average was 1954. so we have 2 programs(1 being the encoder itself) saying the bitrate is only slightly under the target and bitrate viewer saying it well under. strange huh ?

[mean]

Bitrate viewer (last time i checked) takes 23.976 pulldowned stream as 29.96
Hence
1924*23.976/29.96=1539

[dragongodz]

except my test was done with PAL/25fps so pulldown not relevant.

[gizmau]

Originally posted by dragongodz
..real average was 1954. so we have 2 programs(1 being the encoder itself) saying the bitrate is only slightly under the target and bitrate viewer saying it well under. strange huh ?

could the reason be the old problem of Kbit (capital) and kbit (small), because 1954*1,024=2000,896 - which exactly hits the desired bitrate.

[Xeno86]

@dragongodz:

actually lokking that you are doing svcd(except for out of spec GOP length and frame size)

That's right! How did you know?

you are not doing any filtering or anything before the encoder are you ?

I only use deblocking filter, lanczos4resize and blockbuster, but these only improve sharpness and shouldn't lower the bitrate...

under the target bitrate (closest was 1964) according to the HC gui information

I can't see where HC_GUI displays bitrate information. Are we using the same version (v0.1, size 634880 B)?

what i want to know is if the size increases closer to the target.

Yeah the size increased
Same source, all the options same (bitrate=2000, max=2500, 2pass), except:

kvcd matrix, gop 24 2:
avg.bitrate=1167
avg.quant=5.54
mpeg matrix, gop 15 2:
avg.bitrate=1250
avg.quant=5.73
Encode time ~4H
(info according to Bitrate Viewer)

So it is 7% bigger

Now let's see if it shows bitrate correctly:
469 897 135 B * 8 bit = 3 759 177 080 bit / 1000 bit = 3 759 177 kbit / 3015 sec = 1247 kbit
So it seems that Bitrate Viewer is right (3015 sec is real play time @ 23.976 fps, total num of frames 72299). I didn't do any pulldown.

It seems that min quant value is somehow locked. It hardly ever goes below 5.00. It looks as if it was hardcoded

[Amnon82]

A little update for the Guys reading this thread:

The Betatesters started to test the new version 0.11.

New Features of the new version:

Preview of the Source and the Encode
Interlaced fields are added now
Some changes to the encoderengine
Some fixes on the GUI

When Hank releases this version to the public I can't tell.

@Hank: Keep on the good work. You got mail from me ...

[hank315]

It seems that min quant value is somehow locked. It hardly ever goes below 5.00. It looks as if it was hardcoded

It's not hardcoded but I know quants are not able to go low enough.
The bitrate control as it is implemented was meant to create DVD content with a normal DVD GOP structure and bitrate.
So doing KVCD stuff will not give optimal results ATM.

After finishing the new GUI version I will do a complete workover of the bitrate control, the next version might already improve things but these are just minor changes.

@Xeno86
Also did some testing with 2000/2500 average/maximum bitrate but couldn't reproduce the large difference you got.
Did a test with a 6 min. Matrix2 (PAL) source, settings: matrix: NOTCH, profile: BEST, scene change: ON
With GOP 24-2: bitrate HC: 1899, Bitrateviewer: 1899, MPEG Stream Eye: 1899
With AUTOGOP: bitrate HC: 1920, Bitrateviewer: 1910, MPEG Stream Eye: 1910
Both too low but not that low you got...

My goal was to create a thing what would do descent quality, runs stable and would give no problems with max bitrate and underflows, now the problem seems to be at the other end of the bitrate spectrum , think this one can easier be solved

The next version will be ready in a couple of days if all goes well, ATM I'm implementing the last changes from the beta test feedback I got (thanks for that guys!!).

[dragongodz]

I only use deblocking filter, lanczos4resize and blockbuster, but these only improve sharpness and shouldn't lower the bitrate...

actually deblocking can smooth aswell.

I can't see where HC_GUI displays bitrate information. Are we using the same version (v0.1, size 634880 B)?

umm no, sorry, i was using 0.11 beta.

469 897 135 B * 8 bit = 3 759 177 080 bit / 1000 bit = 3 759 177 kbit / 3015 sec = 1247 kbit

i did the same calculation for my 25fps and it came out 1953 which tallies with HC and mpeg stream eye but not bitrate viewer. so atleast for this clip bitrate viewer is getting it wrong.

Did a test with a 6 min. Matrix2 (PAL) source, settings: matrix: NOTCH, profile: BEST, scene change: ON

i tested several different matrices, also normal and best settings with scene change turned off for all.
hank315 if you want a small sample clip that shows much lower in bitrate viewer but near target bitrate in HC and mpegh stream eye let me know.

[Ebobtron]

Originally posted by Amnon82
A little update for the Guys reading this thread:

The Betatesters started to test the new version 0.11.

New Features of the new version:

Interlaced fields are added now

"The Beta testers", guess I'll have to figure out what I have been doing with my spare time the last couple of weeks.
----------------
"Interlaced fields are added now",don't you mean a TFF or BFF check box to select the flag setting to match your source? Last time I used HC interlaced fields worked just fine. I'm not sure what the BFF guys do, but I use ReStream.
-----------------
I don't mean to be pissy. Sorry everyone, yes I do.
--------------
@Hank315
I have been away for the last week popping in from time to time to follow along. I'll post my findings tomorrow.

Sir, your public awaits you.

[dragongodz]

hmm i just did a test encoding with QuEnc 0.59 and bitrate viewer says thats well under the bitrate aswell while mpeg stream eys says its slightly over. what the ????

"Interlaced fields are added now",don't you mean a TFF or BFF check box to select the flag setting to match your source?

yes thats what he means.

"The Beta testers", guess I'll have to figure out what I have been doing with my spare time the last couple of weeks.

I don't mean to be pissy. Sorry everyone, yes I do.

dont be so ungreatful. so a couple of people got to check for last minute problems etc a matter of days before you are due to see it, get over it.

[Axed]

Originally posted by Ebobtron
"The Beta testers", guess I'll have to figure out what I have been doing with my spare time the last couple of weeks.

Theres a very good reason for other people to be checking it before its released to the major public. When the problems pop up, people more knowledgeable in how the encoders work, and also programers, are alot more useful to iron out major bugs then the public who cant properly describe their problems.

When the rest of us get our grubby fingers on it, im sure we'll find those annoying little random bugs that cant be found by a few people testing.

Thanks for the great work Hank.

[Ebobtron]

Originally posted by dragongodz
dont be so ungreatful. so a couple of people got to check for last minute problems etc a matter of days before you are due to see it, get over it.

Didn't mean to pick a scab. I just thought that everyone working this thread was beta testing. I either stand corrected or not. Ungrateful is a little harsh, I could be wrong. But you have read much more into my statement than I meant.

For that I am truly sorry.

[Amnon82]

Hank recruted some of the HC user to betatesters. I'm one of them, for example. I know that some guys here tests HC for hours and weeks, but some are only user. They can only tell the programmer, that there is a problem when he did this and that.

I'm coding programs my self. So does dragongodz too, who is also one of the betatesters of HC. We're talking daily via email with hank and helping him so. We do a hard job to find all the bugs and codingmistakes hank did in the last build.

If we don't do this before hank releases it public, the most user here will post "man this program isn't finished yet. I can't encode my movies. etc ..." without knowing what the problem realy is.

In version 0.11 hank changed the core. But with this changes we lost quality. So hank need more time to change it to the quality we already know form HC 0.01.

The GUI has now more features. For example, hank added a preview. So You can switch between the source and the encode to see the quality. Via a zoom-function You can zoom into the picture during the preview.
Also the stats are now displayed in the right way.

Some reader here know that it is the first GUI for Hank. 0.01 was a commandline-tool. I coded the first GUI for it. Hank felt in love with it and started a GUI by him self. So 0.1 was the next release and the first with a GUI by Hank. So some features came and some are gone.

To get the features of my GUI, the commandline version of HC and the new guied version of HC into one program it will take a long time to get it done, but we're working hard to reach the goal soon.

When the next release will be released I only can tell You this:

When it's done!
(a cool statement btw, so think I (thx goes to iD))

[Ebobtron]

Revisiting my bug report concerning the flashing pink and sometimes green lines at the bottom of the frame when using NTSC sources 720x480, 704x480, 352x480, 352x240, interlaced or progressive.

--------------------------------
++++++++++++++++++
++++++++++++++++++
++++++++++++++++++
++++++++++++++++++
+++++++++++xxxxxxxxx
------------------------------------ The xxxxx show where the lines appear in the video.

Roughly the last third of lines 476, 478 & 480 from pixels 464 to 720.

The following is a list of those encoder settings that have no noticeable effect on the aberrant lines.

Source --- avs or d2v: see source content below
Encoder profile:
Bit-rate:
Frames:
GOP Structure:
Matrix:
DC Precision:
CPU:
Interlaced:
Aspect Ratio:
Scan method:
Time Code:
Do I need to add VRB check:

That leaves us with source content, closed gops, scene detection, constant quantization.

Constant Quantization or CQ

“Using CQ”, @ 4,5,6 or 7 (other values not tested) the anomalous lines appear in every frame, are a solid green residing in lines 476, 478 & 480 from pixels 464 to 720. Feed “HC” PAL content of 576 or 288 vertical lines and the green lines do not appear. With NTSC content 480 and 240 vertical lines green lines are displayed. Horizontal size has no apparent effect on the unwanted lines. Interlaced or progressive encoding, no difference. Source content has no effect with the exception of vertical size.

“No CQ” -- “Open GOPs” -- “No Scene Detection”

Now is gets interesting. The lines are pink and they flash or they appear to flash. Our unwelcome guests show up only in the two frames displayed prior to an I frame. The pink lines are not solid but spread along the space between pixels 464 and 720 (as dots and dashes) again on lines 476,478 and 480. The little pink dots and dashes are never in the same position unless the frame lacks motion.

Source content has an effect if the scene is black or has no motion. Source content with large black borders at top and bottom of the frame produce no unwanted pink lines. So I ran a script to increase the borders around a frame in increments of one. As border size increased pink lines decreased until gone at eight pixels, border color had no effect. My conclusion is that if there is a change between frames in the lower 8 lines we get pink lines. But only in the two frames that proceed an I frame. So they appear to flash.

“No CQ” -- “Closed GOPs”
This one is easy, no pink lines. Remember that GOPs closed or open have no effect if CQ is used.
CQ and NTSC equals solid green lines in every frame.

“No CQ” -- “Open GOPs” -- “Scene Detection enabled”
The little pink flashes are here too but not in the two frames displayed prior to the inserted I frame for the detected scene of course that GOP is closed.


The following is meant not to show a GOP but to clarify the frames containing the misbehaved lines.
In displayed order viewing with VirtualDub-Mpeg2 1.5.10 (build 18294):

B B P B B P B B I <----This I frame must the beginning of an open GOP for the two B frames to flash pinkly.

Source effects
I use AviSynth 2.55 Build Sept. 1, 2004 (I have omitted the build time). DGIndex's d2v or AviSynth, the only thing that effects the behavior is vertical resolution. PAL source equals no green or pink lines however the encoder is configured. I have toyed with fps and field frames and TFF and BFF. Vertical frame size seems to trigger the behavior

My source doesn't have the unwanted lines, I tested with rips, camera clips, and still frames.

That’s all I have on this. I can’t even offer a guess.

I’m using XP Home SP1 on a Compact with 512 Mbytes of ram with a 2.8 GHz Celeron Processor.

I suppose it could be my set up here, stranger things have happened. I hope someone can duplicate my observations.

My time is up for today.

[Xeno86]

@hank315

The bitrate control as it is implemented was meant to create DVD content with a normal DVD GOP structure and bitrate.

But I have also tried it with standard gop and matrix and the differences were really small:

kvcd matrix, gop 24 2:
avg.bitrate=1167
avg.quant=5.54
mpeg matrix, gop 15 2:
avg.bitrate=1250
avg.quant=5.73

Anyway I've done few more tests with same settings but other source (Gladiator DVD PAL, 2 minutes) here are the results:
Avg.bitrate=1856
avg.quant=5.78

This time quantizers went much lower - the lowest is 3.20

It looks like the "too low bitrate problem" is related to my previous source. But I don't now why quant was locked to 5.3 and now it went much lower. Strange, isn't it? I'lldo more tests later.

[((( atom )))]

@hank315: your encoder looks very promising. some free tool like that with high quality was missing for a very long time now. thx for it!

there are a couple of questions i have. i read through the manual, not this entire thread, so i might ask questions that have been discussed, but every user might have these questions after having read the manual, so i dare..

- does your encoder support hd-resolutions?

- why can i only input .d2v or .avs files?
especially for dv, avi-input could easy things up for many people. others might want to reencode mpeg-files. is it because the app couldn't make use of other files atm or is it a restriction you wanted?

regards,

[Koepi]

Maybe because it's so simple to just write

AviSource("MyDVfile.avi")

into an avs file instead of dealing with missing codecs, framrates,...?

But if avs is supported, avi support shouldn't be that hard to do either. Btw., it's usual for the most recent encoders to just use avs input (QuEnc, avs2avi,...)

Just a wild guess though

Cheers
Koepi

[hank315]

Just finished a new version: HC 0.11 beta

Changes:
- GUI updated, many bugs fixed
- preview option added
- TFF/BFF flag added for interlaced encoding
- shows Avisynth script errors
- max. bitrate is written in sequence header instead of 9800
- minor changes in bitrate control for bitrate < 2000 kb/s
- AUTOGOP option didn't work if scene change detection was switched off, fixed
- bitrate now in kb/s, m2v file size in Kbytes (1 kbit = 1000 bit)
- MPEG matrix is set as default matrix

get it at: http://hank315.dyndns.org/HC_011.zip

Thanks to Amnon, dragongodz and scatha for testing and feedback.

Nothing really changed in the encoding engine except for bitrates < 2000 which should give a slightly better result.

[Amnon82]

@Hank: You're always welcome