Who wants to test a new MPEG2 encoder

[hank315]

For some time I've been working on a MPEG2 encoder,
maybe there's somebody who want to run some tests with it.

Some information about the encoder (named HC):
(complete manual is included)

Input can be a d2v project or input using Avisynth.
GUI version and batch version.
The encoder should run on any Intel and AMD processor using Windows XP/2000.
At least MMX/ISSE must be present to run the encoder.
If SSE2 or SSE3 is present HC will also use it.
2 pass VBR encoding.
Variable GOP structure dependent of the video content or fixed GOP structure.
Scene change detection.
Pre-programmed matrices or use your own matrices.
Restart possibility to run the second pass again.
Bitrate control: average and max bitrate can be set.
Buffer underflows will not occur, the VBV is constantly checked so no bitrate spikes.

Complete package: http://hank315.dyndns.org/HC_013.zip 0.13 beta version, last update 17-04-2005

Please remember that it is a work in progress... there may be a few bugs

[Ebobtron]

I don't know what to say.

Configuration technique is unique and might even be elegant. Might be hard to do batch encoding, but I can write ini files from a batch file.

I still have testing to do but the stream looks compliant.

I test scene detection with music videos. The cuts are often quick and complex. A quick look at the first 15 cuts of my test video have me at a loss for words. In short, for now, HC appears to be dead accurate. Original video sometimes misses scene change by a frame or confuses large motion for a change. Your encoder hits the scenes a major motion picture release on DVD misses.

Quality looks very good.

And yes, bugs. I'll post logs and bug-list tomorrow.

Thanks for your help before. I do lots of NTSC.

Ebob

[video_magic]

I had downloaded last night but can't test yet because I have another big mpeg2 encode running at the moment with Nuenc, but I look forward to running a clip through yours after that finishes.

[Nic]

Is this written in Fortran?
If so. Very Hardcore.

Haven't tested it yet...I'll let you know.

-Nic

ps
May I ask what your future plans are for your encoder?
i.e. Going to go opensource, commercial, freeware, just doing it as a hobby etc...Either way, good luck

[hank315]

Is this written in Fortran?
If so. Very Hardcore.

It is written in Fortran95
+ a little C code to communicate with DGdecode.dll and Avisynth
+ about 6000 lines of assembler, mostly MMX/SSE/SSE2 stuff

Hardcore, LOL.. that C stuff guys are writing here, that's real hardcore.
I've always programmed in Fortran, had to, when I started there was no C.
In the 80's I tried C but it was a great dissapointment, compilers were buggy and slow and the generated code was buggy and slow.
Nowadays a lot has changed, with C you can do everything and it is (almost) as fast as Fortran , but for writing this kind of applications I'll just stick to Fortran.

May I ask what your future plans are for your encoder?
i.e. Going to go opensource, commercial, freeware, just doing it as a hobby etc...Either way, good luck

I was just curious how the MPEG compression scheme works.
Also I was surprised to discover that all encoders are (very expensive) commercial codes or based on ffmpeg or mpeg2enc so I decided to start from scratch and not to look at the open source C code (which in my case wasn't so hard because I can hardly read the stuff )

So it started as a hobby...
open source - probably not, most people will not understand or can compile the Fortran sources
freeware - probably
commercial - unlikely, but come on guys, make me an offer I can't refuse

It was also a good exercise to start writing a H.264/AVC encoder.

Future plans for HC:
- Speed and quality improvement and getting all the bugs out
- Support for hyperthreading and multi-processing systems
- Writing a GUI but I really hate that!!

[Nic]

Very impressive.

If you really don't fancy making a GUI for it...Could you not make a DLL out of your Encoder, so that it could be fed the parameters and YV12 frames? That way I could plug it into QuEnc

Good luck finding bugs, and you're right, I doubt many could follow the Fortran code.

Take Care,
-Nic

[mean]

... and building a .so for the nix guys

[Ebobtron]

Bug report:

------------------

Opening command window reports frame-rate incorrectly under encoder settings. Reports correctly under source stats. Output to log is correct.
-----------------------------------------------------------
--------------------> encoder settings <-------------------
-----------------------------------------------------------
input: faithv.avs
output: Faith_HC_T04.m2v
log file: Faith_HC_T04.log
profile: GOOD
frames: - end
framerate: 25.00 <---------------
aspect ratio: 9:16
bitrate: 8000000
max. bitrate: 8000000
restart: no
closed gops: no
VBV check: yes
scene change det.: yes
interlaced: yes
goplen,B-pic: 15 2
dc_precision: 9
scan method: ALTERNATE
CPU: SSE2
-------------------------------------------------

Sequence header reports bit-rate as 9800000 always.
Sequence header reports first GOP-timestamp as 1h 0m 0s 0f.

----------------------------------------

Flashing pink or sometimes green horizontal line in bottom right of displayed video. Bad lines are in odd fields only even fields look great..

------------------------------------------

Seems that CPU Auto doesn’t select SSE2 my max. But not sure.

----------------------

More later
Ebob

[hank315]

Opening command window reports frame-rate incorrectly under encoder settings. Reports correctly under source stats. Output to log is correct.
: 25.00 <---------------

Will be corrected in the next version, just wrote the default value to the screen.

Sequence header reports bit-rate as 9800000 always.

HC just writes the max. value for the VBV in the video file to be on the safe side. Should be OK.

Sequence header reports first GOP-timestamp as 1h 0m 0s 0f.

You wonder why? I also did and still do
Just took this value because most encoders do but couldn't find any information why (is there somebody out there who knows). Can easily be changed or an extra command to set the timecode.

Seems that CPU Auto doesn’t select SSE2 my max. But not sure.

The encoder was developed using a P4 3.2GHz (Prescot core) which can use all of the present extensions, testing on all other systems always shows the right CPU but there was only one AMD machine amongst them (which was recognized as an AuthenticAMD, only MMX/SSE support). I'm just checking bit 0, 23, 25 and 26 with the proper calls to cpuid. If you choose a wrong extension, one your CPU can't handle, the encoder will crash or it will produce absolute rubbish. BTW, what CPU do you have?

Flashing pink or sometimes green horizontal line in bottom right of displayed video. Bad lines are in odd fields only even fields look great..

That's a hard one, I'll try and reproduce this error. Would you try to encode that scene again with *INTERLACED turned of and see if its still there? My experience is it is better to use a de-interlace filter in Avisynth and then just encode it as progressive.

If you set the average bitrate equal to the max. bitrate you will not get the desired filesize. HC handles the max. bitrate in the next way: max. bitrate is controlled per GOP, if the actual bitrate > max. bitrate the GOP is recycled with higher quantization values until the actual bitrate <= max. bitrate. For low action scenes bitrate will be lower but it will try to catch up to reach the target but will never reach it.

[Ebobtron]

HC just writes the max. value for the VBV in the video file to be on the safe side. Should be OK.

Thought that was the case.

--------------------------------------------
I have a Intel Celeron @ 2.80GHz. It looked faster when CPU set to SSE2 than Auto. I'll check again with same source.


If you choose a wrong extension, one your CPU can't handle, the encoder will crash or it will produce absolute rubbish. BTW, what CPU do you have?

Your right it crashes.
-----------------------------

I'll try some deinterlaced stuff.

------------------------

I was comparing other encoders with the 8000 8000. Some encoders spike over, yours doesn't.


Ebob

[hank315]

@Nic

If you really don't fancy making a GUI for it...Could you not make a DLL out of your Encoder, so that it could be fed the parameters and YV12 frames? That way I could plug it into QuEnc

I think that could be done fairly easily, but call it masochistic behaviour, I just want to give it a try and write such a GUI-thing. And maybe after a while, when I'm totally frustrated banging my head against the Windows GUI wall, I will beg you on my knees to implement it in QuEnc
Anyway, thanks for the offer but let's not be too hasty, I'll wait for the tests people will do and first resolve the most bugs.

PS. I really like your latest (0.59) build, great quality, used it to do some comparisons with HC.

[Ebobtron]

Flashing pink or sometimes green horizontal line in bottom right of displayed video. Bad lines are in odd fields only even fields look great..

Originally posted by hank315


That's a hard one, I'll try and reproduce this error. Would you try to encode that scene again with *INTERLACED turned of and see if its still there? My experience is it is better to use a de-interlace filter in Avisynth and then just encode it as progressive.

If I encode a PAL clip no lines. If you change your source resolution to 720x480 I think you will see the lines. Non interlaced clips have the llne each frame if 720x480 used.

-------------
Let me show you how new to this hobby I am by asking this question.

Why would I want a DVD compliant video stream that is progressive?

I never encode film and all my stuff is intended to be burned as a DVD for viewing on my television or my mom's TV.
Am I not going to be able to tell the difference?

Off to see if I can break your new toy.
Thanks,
Ebob

[Amnon82]

@hank315: Your Encoder looks great! It has all features a MPEG2Encoder should have.

So You don't like Guis. If You want I can code a GUI for You. I'm coding in Delphi.

The Support of CQ-Values is great. I'm thinking to code AutoCQ for HC.
I did the program for CCE. It finds the right Q-Value for onepass-encoding.

I play a little with Your encoder and code a little GUI. I write You a PM.

... when do You release a non-limited test-version?

[hank315]

@Amnon82
Nice you like it, and I realize a GUI would be a good thing too.

But I'm also trying to program one (in Fortran).
I know when people hear about Fortran some of them still think we are carrying boxes of punchcards
The Fortran I'm using is Compaq Visual Fortran 6.6c which is totally integrated in Visual Studio 6.0.

The GUI I have in mind will look like: http://hank315.dyndns.org/GUI_template.jpg

It's not finished yet but some of the logic already works and it is also nice to cleanup the input part which has become a mess...

A full version will be posted soon, maybe even tonight.

And ofcourse feedback which might improve the encoder is always welcome.

@mean
I'm pretty much a complete Linux NOOB and don't have a Fortran95 compiler for Linux. There's an experimental compiler ( http://www.g95.org ) but the speed of it isn't optimal yet.
But if the GUI-Windows version is up and running I will give it a try.

[hank315]

Just put a version online without the 30000 frame limit.
No GUI yet, still working on it.
There are still some limitations, the max. nr. of GOPS is 64000 so you will run into a trouble if your movie is over 8 hours...

get it at http://hank315.dyndns.org/HC.zip

Have fun with it

[Amnon82]

I didn't looked at Your JPG.

Here is my Design-Sample #1. Hope You like it:

Go to the HC.GUI Thread

I started a new thread to keep this clear.

@Hank: did You add the AutoEncode and AutoClose Feature I asked for?
I'll need it to start my project AutoCQ for HC. It will get the right CQ-Value for the 1Pass VBR-Encode. I did some good results with the AutoQ for CCE Version.

Its good to see that You code it also in GUI. So Beginners will have it easier.

Keep the INI-Files. They are like ECL-Files for CCE.

I stay tuned for Your project. You have me 100%!
Lets play with Your last release.

... still missing the 2 functions ...

I can do the first <Enter>-Button request, but not the second. So the AutoEncode and AutoClose Functions are needed to code addons or external GUIs for Your encoder.

I'm working now on a little workaround. I call the little app 'BatchHC'. It do the 2 Enter-Commands.

[unixfs]

Hank315,
I just tried a 30 minutes encode of a full frame 720x576 dvb capture.
Simply put, it seems HC is absolutely the best mpeg2 encoder
I've seen so far!
Image quality (at 1600 kbps) is simply excellent, and you seem to
have found the way to avoid the most common artefact that all
other common encoders produce: blockyness in uniform or dark areas.

I really hope you will release your source code with an opensource license,
so that it can be used with other encoders like mencoder.

open source - probably not, most people will not understand or can compile the Fortran sources

There is a fortran compiler for *nix, released with gcc sources
(although it doesn't have a good fame).

Thanks a lot!

[Hemmo]

Hank315:
I just tried HC with one chapter from Matrix and I very happy to the result !
I hope HC will stay freeware ?

[Amnon82]

New Version Released: http://hank315.dyndns.org/HC_0.01.zip

Hank315 adds the *BATCH - Parameter.

How it works? Take a look into AutoCQ 0.2.0

[Mug Funky]

wow, now i know never to skip the mpeg-2 forum for more than a couple of days

open source - probably not, most people will not understand or can compile the Fortran sources

i don't know about that... my brother still uses fortran77 (he was born in 1980, go figure). he's never at a loss for a compiler in 'nix. does most of it for RISC processors though.

i can't code to save myself though.

but anyway, another opensource encoder can't possibly be a bad thing, especially if the quality is what people are saying it is (haven't had a chance to test yet).

of course it's most definitely your call, and freeware would be fine too.

k, now i'm going to test this baby

thanks heaps for your work.

[edit]

hmm.. what's a reasonable amount of time it should be on "intermediate pass" for an 823 frame clip? it's been on it for nearly 10 minutes now.

running a p3 733 (SSE1, MMX)