Project Rémoulade Update: DivX H.264 Encoder Alpha 1 & Tutorial now available

[LoRd_MuldeR]

LigH, here is a simple Demo application I just hacked together:
http://www.mediafire.com/?mamwiyubawt

(Sources included, the EXE was compiled with Delphi 7.0 Professional)


EDIT: Re-uploaded an improved version. Link has been updated!

EDIT2: Re-uploaded an improved version - again. Link has been updated!

[LigH]

Great.

Not sure if I will use it already for DivX264 CLI alpha 1, but maybe some day...

[buzzqw]

btw i added support in AutoMKV, i just wait approval from DivX!

BHH

[LoRd_MuldeR]

Not sure if I will use it already for DivX264 CLI alpha 1, but maybe some day...

It already works with my simplistic Demo application and this commandline:
cmd.exe /c "C:\Program Files (x86)\DivX\DivX H.264 Codec CLI\bin\WhisX.bat"

(But the log filter would need some adjustment for divx264.exe -- done, see post above)


@ DivX developers:

It seems the DivX CLI encoder is missing to call fflush() after writing the progress line to STDOUT/STDERR.
This will cause a nasty delay when redirecting the console output...

x264 does do it the following way:
if( i_frame_total )
{
int eta = i_elapsed * (i_frame_total - i_frame) / ((int64_t)i_frame * 1000000);
i_progress = i_frame * 1000 / i_frame_total;
fprintf( stderr, "encoded frames: %d/%d (%.1f%%), %.2f fps, eta %d:%02d:%02d \r",
i_frame, i_frame_total, (float)i_progress / 10, fps,
eta/3600, (eta/60)%60, eta%60 );
}
else
fprintf( stderr, "encoded frames: %d, %.2f fps \r", i_frame, fps );
fflush( stderr ); // needed in windows

[DigitAl56K]

LoRd_MuldeR & LigH: Wow, you guys are on fire today!

LoRd_MuldeR: I'll log the fflush() issue. Side: I already have an issue logged where some of the -help is written to stderr instead of stdout.

LigH: I have a suggestion for your app!

You could support a wider range of inputs by doing this:

If the input is AVI or AVS, write an intermediary AVS script that loads the input file via AVISource
If the input is another type, write an intermediary AVS script that loads the input via DirectShowSource
You could use Don's command-line guide (http://neuron2.net/dgmpgdec/DGIndexManual.html#AppendixB) for DGMPGDec to handle MPEG formats if it's available on the system.


After writing these intermediary files you can open them with the windows AVIFile APIs and check the rate/scale of the video stream to make sure it's valid and if not automatically apply something like ConvertFPS with the nearest allowed rate/scale. You could also check the width and height and apply AddBorders and/or a resize if necessary, then finally pass this AVS file to the encoder. If you want to get really fancy you could add ConvertToYV12 to the AVS script and get the picture in a known format through the AVIFile interfaces, then apply a routine to determine a crop window to remove borders, checking various frames throughout the file. I think you can also get PCM samples easily this way so you could write a WAV file then pass it through an audio encoder and hook up mkvmerge to bring everything together.

buzzqw: I sent you an e-mail, I'd love to see AutoMKV support this!

LoRd_MuldeR & Dark Shikari: Thanks for the comparison vid, and for keeping the conversation on track! I haven't had time to thoroughly investigate it yet but x264's new psy does seem impressive :) I did notice that the clip is rather low motion and x264's settings were quite extreme - e.g. 16 refs, 15 consecutive b's, ~20s IDR interval, so I'm wondering if this is somewhat special case. Anyway, I'll look into it further :) I think we also have that clip so I can recreate it with more comparable settings.

[DeathTheSheep]

I'm disappointed that there is no constant quality mode, which is a necessity for my purposes. Did I just miss a commandline option or something?

[DigitAl56K]

DeathTheSheep:

Constant quality is not available because it doesn't allow management of rate spikes. Keep in mind that the encoder is designed to create CE-friendly streams. 1-Pass target-quality on the other hand is one of the most requested features at present.

[LoRd_MuldeR]

Constant quality is not available because it doesn't allow management of rate spikes.

Isn't that the entire point of "constant quality" mode? Use as many bitrate as needed and don't care about the final size or bitrate spikes?

Obviously 1-Pass makes it hard to implement VBV restrictions, but it should be possible to run a simple 1-Pass CQ encode without any VBV restrictions at least...

[DigitAl56K]

Isn't that the entire point of "constant quality" mode? Use as many bitrate as needed and don't care about the final size or bitrate spikes?

Obviously 1-Pass makes it hard to implement VBV restrictions, but it should be possible to run a simple 1-Pass CQ encode without any VBV restrictions at least...

Yes, that is the entire point of it and the entire problem! :) The encoder, at least in this early version, is designed to implement the draft profile. Modes that would typically be part of unconstrained operation aren't available because currently this isn't the intended purpose of the release.

However, we are taking feedback about all requested future options. How important to you is constant quality mode and what are the situation under which you'd want to use it?

[stax76]

Latest StaxRip beta has DivX H.264 support. Compressibility check is missing because there is no constant quality mode. I use crf 22 so not having a constant quality mode is a show stopper for me. Since the encoder is privat + alpha it will only show up if the encoder is installed. GUI looks as follows:

http://planetdvb.net/non_drupal/images/StaxRip_DivX_H.264_Options.png

For those who wonder why I often use the TreeView instead of real controls, reasons are: Every option requires a single line code only so it helps to fight bloat, it's quick and easy to code, easy to maintain, easy to change, easy to extend and automatically clean and polished.

[SeeMoreDigital]

For those who wonder why I often use the TreeView instead of real controls, reasons are: Every option requires a single line code only so it helps to fight bloat, it's quick and easy to code, easy to maintain, easy to change, easy to extend and automatically clean and polished.Yep.... I prefer this way too!

[DigitAl56K]

Latest StaxRip beta has DivX H.264 support. Compressibility check is missing because there is no constant quality mode. I use crf 22 so not having a constant quality mode is a show stopper for me.

Good point! :)

Off to download now!

[LoRd_MuldeR]

How important to you is constant quality mode and what are the situation under which you'd want to use it?

I personally don't have any "mobile" device, nor do I have a "standalone" player. So I don't care about profiles, I use MPlayer or MPC+ffdshow. I want to archive captured video clips on my HDD and hence I need some "constant quality" mode. My clips are usually between 3 and 10 minutes, mostly live music. Some clips have a LOT of motion, some are more static. I want to have the "same" quality (more or less) in all my clips and I don't care if some clips are MUCH bigger than others. At the moment x264 + Psy-RD does a great job with it's "CRF" mode. So I use CRF=22 for most of my encodes, I rarely use 2-Pass. If you ever want me to consider DivX H.264 as an serious alternative to x264, you'll have to provide some mode like CRF...

[DigitAl56K]

LoRd_MuldeR: Ah :) Yes! I think there has been some misunderstanding. A CRF-like mode is absolutely something we are considering! I read "constant quality" as fixed qp, although it sounds like Stax still needs a qp mode for compression check.

[stax76]

crf like is fine for comp. check.

[LoRd_MuldeR]

A CRF-like mode is absolutely something we are considering!

Good news :D

[DigitAl56K]

crf like is fine for comp. check.

Better news!

[LigH]

Sorry to disappoint you:

I won't work much further on my GUI as long as there is only an "Alpha version 1". ;)

Means: I may consider more efforts with a next encoder version... Patience. I have less life than ever before.

[Inventive Software]

Nice going with this BTW! It had to come whilst I was at Greenbelt! :D How's speed with the same settings as x264 comparable?

[Ajax_Undone]

Wow oss still kick butt Divx got to work a bit harder on that...

[Dark Shikari]

Nice going with this BTW! It had to come whilst I was at Greenbelt! :D How's speed with the same settings as x264 comparable?Speed can still use a lot of work--in my test DivX goes significantly slower in its fastest mode than x264 goes with psy RD and psy trellis (--subme 6 --trellis 1 --bframes 3 etc). This presents a serious problem, for obvious reasons. DivX has a huge amount of room for improvement here, so one can hope that they'll cover some ground in the next release.

[RickA]

Greets,

Really enjoying testing out the new codec. Quality looks great. Encode times seem reasonable.

@LigH, thanks for the GUI. Very handy.

Cheers,
Rick

[linyx]

You may want to update a shortcut. Under Start Menu > Programs > DivX > DivX H.264 Codec CLI > Learn about DivX H2.64 Codec CLI, I think it should be "Learn about DivX H.264 Codec CLI":)

[siva]

this is beta :devil:

[LigH]

this is beta :devil:

http://www.ligh.de/pics/this_is_beta.jpg

:o Sorry, couldn't resist.

[Kurtnoise]

found a typo at the end of the stdout...

AVI 720x480
AVI frame rate parameters:(30000/1001=29.970030)
DivX H.264/AVC encoder (cli/sdk 0.0.0.5/7.3.0)
Copyright (c) 2008 DivX

[time: 0:00:32] [left: 0:00:00] [speed: 7.4 fps] [frames: 241/241]

Summary information:
Number of coded frames 241
Total encoding time 32496 ms
Average time per frame 134.838 ms
Average speed achieved 7.4 fps
Average bitrate 1987.48 kbit/sec @ 29.970 Hz

[Sagekilla]

Technically that's not completely wrong, Hertz is used as a measure of frequency, in this case frames per second.

[Kurtnoise]

but quite unsual...

video frame != audio frame

[IgorC]

Hz is generic unit of frequency. It doesn't matter what. Frames per second, samples per second, apples per second, bananas per second - Hz.
Something per second.

[Kurtnoise]

hey, I'm not stupid...It was just a notice.

[CruNcher]

It's quiet common for a Video Engineer to use Hz instead of FPS especially if it's one from the Analog days :)

[IgorC]

hey, I'm not stupid...It was just a notice.
Then why do you ask? Is there somethig odd?
Admit that you didn't know that. It's simply to see. That's not shamy. Many people don't know what is Hz, Watt, Volt, Ampère, Faraday, Henry.....
For many people Hz is just for audio measurements.

Imagine situation:
John: Hey, how is your dog?
Paul: My pet is fine.

Is that talk odd for you?


The same way Hertz and fps.


but quite unsual...

video frame != audio frame

It means that if boys are weighted in kg then girls can't be weight in kg. Right?

[Kurtnoise]

wtf...

Then why do you ask? Is there somethig odd?
I wasn't asking (did I posted a question ?)...It was just a notice. I quite *understand* that we can use Hz for this. Period. Damn. :sly:

It means that if boys are weighted in kg then girls can't be weight in kg. Right?
mmh...you like to play, isn't it ? First, it's not weight but mass. And my point is : it's quite unusual to see in video tools fps measure expressed in Hz nowadays. That's all.

So, now back on topic. Did you try this encoder ?

[Sagittaire]

wtf...


I wasn't asking (did I posted a question ?)...It was just a notice. I quite *understand* that we can use Hz for this. Period. Damn. :sly:


mmh...you like to play, isn't it ? First, it's not weight but mass. And my point is : it's quite unusual to see in video tools fps measure expressed in Hz nowadays. That's all.

So, now back on topic. Did you try this encoder ?

1) Mainconcept use Hz since more than 2 years for internal CLI test.

2) You have exactly the same quality output than Mainconcept SDK without AQ and FGO (exactly same PSNR and same speed). DivX use Mainconcept SDK 7.3.0.4316 core codec. DivX7 CLI or Converter are really poor tools if you compare with Cinevision from Sonic or Reference from Mainconcept.

[Kurtnoise]

sure...it's in alpha stage. Hoping to have new features in the next builds.

[Sagittaire]

sure...it's in alpha stage. Hoping to have new features in the next builds.

No for me it's simply volontary limitation for Mainconcept SDK core from DivX Network. Mainconcept SDK is not in alpha and all the feature are well know since very long time (more than 2 years now). I have here really complete CLI from Mainconcept (2 years old) with all the fonctionality. DivX use internaly exactly the same tools. Here the command line that you can find in the h264vout.001 file from DivX Converter:

PrintProgress CalcQuality Buffering LiveMode NumThreads CpuOpt PicNumOffset FrameMBsMode Log2MaxFrameNum Log2MaxPOC WritePPS WriteSPS WriteSingleSEI WriteTS WriteEOS WriteAUD AnnexB StreamType NumUnitsInTick TimeScale PAR_Height PAR_Width SAR_Height SAR_Width SAR_IDC VUIFlags BlackLevel DenoiseStrengthC DenoiseStrengthY Lambda FGOStrength GrainOptimization QuantOptimization FastSBME FastMRME FastInterDecision FastIntraDecision HadamardTransform InsaneRDO EnableRDO OffsetsMatrixFile UseOffsetsMatrix ScalingMatrixFile UseScalingMatrix BetaOffset AlphaC0Offset DeblockMode EnableInter_PCM EnableInter_4x4 EnableInter_8x8 EnableInter_16x16 EnableIntra_PCM EnableIntra_4x4 EnableIntra_8x8 EnableIntra_16x16 ConstrainedME OutOfPicMVs SearchRange NumRefFrames WeightedBiPred WeightedPred SubBlockMode SubPelMode EnableME Pass StatFile MaskingStrength MaskingType CpbRemovalDelay CpbFullnessUnits CpbFullnessTarget CpbFullness CpbSize HssRate UseVclHRD UseNalHRD AvgBitrate CrOffset CbOffset QuantMax QuantMin QuantB QuantP QuantI RCMode IDRPeriod AdaptiveB EnableSCD PyramidCoding BFramesReference BFramesCount MinIntraPeriod MaxIntraPeriod SliceArgument SliceMode TopFieldFirst InterlaceMode LosslessCompression TransformType EntropyMode ChromaFormat Level Profile ColorSpace Height Width FramesCount StartFrame StatFileName ConfigFileName ReconstFileName TargetFileName SourceFileName


For me at this time DivX Converter is simply really useless and limited gui for Mainconcept SDK. Moreover the gui don't even use the lastest SDK build (7.5.xxxx from reference) or the more interessing build (7.4.xxxx from sonic with independant AQ strength for all mask MaskingStrengthCplx MaskingStrengthContr MaskingStrengthLuma).

Make MeGui support for this tools is at this time really useless because the codec from DivX is really crappy. x264 will always produce by far really better result.

[Sharktooth]

crappy or not, the divx encoder is alpha1 and we all know what alpha1 means...

[Sagittaire]

crappy or not, the divx encoder is alpha1 and we all know what alpha1 means...

Well problem is here ... Mainconcept SDK is not alpha at all ... since more than 2 years. And DivX encoder use the complete Mainconcept SDK binary with exactly the same command line. Really easy for DivX Network to compile executable with complete and native command line. Anyway DivX Network use really limited build ... and it's IMO completely volontary.

h264vout.001 from DivX Converter
7.3.0.4316 ... something like 08/2007 for this build

h264vout.001 from MC Converter
7.5.0.35746 ... 2008/10/02


You can if you want remplace the h264vout.001 file and DivX Converter or DivX Encoder Alpha 5 will work perfertly. DivX Converter or DivX Encoder Alpha 5 are not completely new encoder. It's just simple Mainconcept SDK core codec compilation.

[Sagittaire]

Here little sample with same speed for all AVC encoding.
I use simply insane speed setting for VC1 SDK.

http://jfl1974.free.fr/Sample/ParkRun_Ateme.mp4
http://jfl1974.free.fr/Sample/ParkRun_DivX7.mp4
http://jfl1974.free.fr/Sample/ParkRun_Mainconcept.mp4
http://jfl1974.free.fr/Sample/ParkRun_x264.mp4
http://jfl1974.free.fr/Sample/ParkRun_VC1MS.vc1

DivX7 AVC can potentialy produce exactly the same result than Mainconcept and certainely better with more recent build with more advanced HVS tools.

Little observation: Mainconcept SDK update stat file at each second pass. If you have problem with RC you can make Npass encoding.

[IgorC]

I'm downloading the videos. Can I ask you to upload video to another server in future? free.fr is very slow. Maybe mediafire.com