x262 Development Thread

[kieranrk]

ifb and I are writing x262, an mpeg-2 encoder building on the best-in-class features of x264

Git tree (more likely to be broken than working at any given time):

http://github.com/kierank/x262

Intra working. Inter in development.

[ifb]

Now I need to finish quant. :)

[shon3i]

I will send you DVD specs, as soon i remember to look at workplace ;)

[Mug Funky]

perfect! now all you need is to fill those blocks up with pixels!

:)

seriously looking forward to this one. who said mpeg-2 is dead? CCE is still selling and DTV is looking as crap as ever, so there's always going to be room to stir things up a bit.

best of luck!

[aegisofrime]

Interesting project! I don't think DVDs are going away anytime soon (The install base is too huge!), so somebody will find a use for this for a long time yet! All the best!

[Gromozeka]

:( But where compile program for testing?

[ifb]

:( But where compile program for testing?There's nothing to test yet, unless you want your encodes to look like the above. Follow one of the guides for compiling x264, but clone the x262 github repository instead of official x264. Patches welcome.

Quant is done, but mostly untested until AC coefficient VLCs are written to the bitstream. Then we can encode more than an 8x8 blockfest.

[Lyris]

who said mpeg-2 is dead?Hear hear. Honestly, I get a ton of satisfaction from producing DVDs with Cinema Craft, because the quality dumps all over what's seen on most commercial discs when it's used carefully. The satisfaction from doing MPEG-2 right is huge because it's not so easy.

So I look forward to seeing what this can do!

[turok]

so i notice there is no update on this topic... how is dev. going for this atm?

[kieranrk]

ifb finished off intra and we're moving on to inter.

[Mug Funky]

nice! so the blocks are filled with details now?

just to illuminate things here, how much needs to be done to squeeze the x264 elephant into the mpeg-2 morris mini?

[7ekno]

Is the VAQ framework also going to be featured ?!?

Tek

[jfcarbel]

Is this reason (http://forum.doom9.org/showpost.php?p=1462825&postcount=17) why you might be developing this project? :)

[kieranrk]

Is this reason (http://forum.doom9.org/showpost.php?p=1462825&postcount=17) why you might be developing this project? :)

Not at all. That reason doesn't even make much sense.

The Open Source Broadcast Encoder is the main reason x262 is being written.

[Biggiesized]

So you're targeting the broadcast space, not DVDs? Or both? Very interesting now.

[kolak]

Not at all. That reason doesn't even make much sense.

The Open Source Broadcast Encoder is the main reason x262 is being written.

And later introduce commercial license as with x264 :)


Andrew

[kieranrk]

So you're targeting the broadcast space, not DVDs?

Personally I don't care much for DVDs. I don't think ifb does either.

[hank315]

Personally I don't care much for DVDs. I don't think ifb does either.
But will it be possible to create DVD compliant encodes with x262?

[kieranrk]

But will it be possible to create DVD compliant encodes with x262?

Very likely yes.

[hank315]

OK, otherwise IMHO it will be useless.
Is there a time scheme when it will be finished?

[kieranrk]

OK, otherwise IMHO it will be useless.

Not for the people wanting to use it for ATSC MPEG-2 transmissions it won't :p

[nakTT]

Any info regarding the quality/filesize ratio compared to x264?

[Mug Funky]

you could look at the first post...

this is very much not ready for public consumption.

my guess, seeing the state of the art of current mpeg-2 and current h.264, this would be around same quality at 2-2.5x the bitrate of x264?

if we can get interlaced video to look good at less than 4mbps, i'd be screaming in the streets to crowds of uninterested and confused people. i suspect this may be the case.

[kieranrk]

ifb has gotten P-frames working.

[Biggiesized]

Any new updates?

[ifb]

Any new updates?I've been busy (plus a little lazy) and kierank has been working on OBE a lot, so there isn't that much x262 news.

I do have B frames nearly working in my local tree, however. Patches welcome if anyone wants to take on the TODO (http://github.com/kierank/x262/wiki/TODO) list.

[Mug Funky]

in case anyone's after an update of sorts...

first up, this is a fantastic project - just thought i'd add some encouragement to kierank and ifb's efforts on this.

i followed an x264 compiling guide and git'ed from x262's repo. i'm a complete newb at this, but it all worked smashingly in windows.

here's a rundown of what i can see so far:

- avs, raw and y4m support only (unless i'm doing it wrong).
- fullpel only for now (set subme to 0), as x264's qpel seems to get mapped to hpel and thus the blocks are moved twice as far as they should be. i imagine this is a largish undertaking on account of h.264 having different filtering etc for qpel than mpeg-2's hpel.

- crf values are comlpetely unrelated to those that x264 uses - try much lower values
- output streams are not compatible with DVD authoring programs (spruce gives me a "temporal references out of order" error).

- quality is surprisingly good, with the occasional blocky glitch on movement. the encoder seems to bias toward making grainy noise rather than flat blocks, which is good news for film lovers, especially when hpel gets implemented. by "surprisingly good", i was expecting something like the screenshot at the start of this thread. it's still not really watchable...

- no interlace as yet, but there is alternate scan.

tl;dr: this is not ready for general consumption yet.

i wont release my compile for this reason - i can't be sure i didn't do anything wrong in the compile (it's my first time), and if unofficial binaries start floating around at this early stage, the encoder might get an undeserved reputation before it gets a chance to be finished. i'm testing more out of curiosity than anything serious.

this is really promising...

[edit]

i just tried it with the mpeg matrix and it looks a lot better. about right for something that's crippled with fullpel. this has really come a long way in a short time!

[Sagittaire]

Really interessing project because x264 is really advanced codec (psy, RDO, RC ...) and portability for x262 will be potentially really promising ...

[CruNcher]

yup not only but if you look @ XviD i wonder what could have been reached their with x264 as its base (especially the asm and multithread optimization) :) Darks VAQ port is nice but when x262 is really successful and can show that the base of x264 works efficiently their then why shouldn't it be more efficient then the current base XviD code (RC,RD) and can we dream about a x263 then ? ;)

[Mr VacBob]

It wouldn't be difficult to make an x263 from a complete x262. That said, x264 --tune fastdecode recovers about half the decode time from normal settings and is still much more efficient than MPEG4 ASP.

[plonk420]

what switches are you all trying?

[iwod]

It seems x262 development has stopped? ( Although it is still included the x264 mailing list )

[kieranrk]

We have been busy on other things. Having been accepted to speak at IBC about open source broadcast I (and likely ifb) will have even less time.

[plonk420]

what switches are you guys using? all i've tried so far is

x262 --crf 8 --mpeg2 --subme 0 --fps 24 -o output.m2v input.avs

(adjusting the CRF)

anything else you guys are trying?


edit: thanks for this awesome project, kieranrk, ifb, and any anyone else contributing!

edit2: interestingly, kkrunchy and crinkler don't like it. upx does, and slightly amazingly "best compression" mode doesn't trigger any antiviruses

[kieranrk]

There is a tender out to add interlaced support to x262: http://mailman.videolan.org/pipermail/x264-devel/2011-October/008887.html

[nautilus7]

Where i can find x262 binaries?

[pandy]

There is a tender out to add interlaced support to x262: http://mailman.videolan.org/pipermail/x264-devel/2011-October/008887.html



But MPEG-2 is not able to have MBAF...

[MatLz]

Where i can find x262 binaries?

Here (http://hotfile.com/dl/133462733/16daece/x262.zip.html) is one.
However, crf seems to be broken ? Use -q instead.

[kieranrk]

But MPEG-2 is not able to have MBAF...

not true. MPEG-2 frame pictures can switch between field and frame modes.

[mp3dom]

Isn't it more similar to PAFF then?
Anyway, while it is supported by the MPEG-2 standard, it isn't well supported by a good amount of standalone players (they made visible judder when the switch happens). I'm referring to the picture structure (Field/Frame) that the encoder change on (I think) GOP basis. Don't know if you're referring to the same. GV ProCoder is one of the few encoder that support this feature.

[kieranrk]

No it is not like PAFF. PAFF is also allowed (field mode pictures).

[drmpeg]

Here's what MBAFF looks like in MPEG-2.

An interlaced frame (luma only).

http://www.w6rz.net/bluma.png

The macroblock types in a B-frame. blue = forward, yellow = backward, pink = bi-directional, dot = quant change, horizontal line = field DCT.

http://www.w6rz.net/bmtype.png

The motion vectors. Horizontal line = field vectors.

http://www.w6rz.net/bvectors.png

Ron

[ifb]

what switches are you guys using? all i've tried so far is

x262 --crf 8 --mpeg2 --subme 0 --fps 24 -o output.m2v input.avs

(adjusting the CRF)

anything else you guys are trying?

For NTSC DVD (untested in Scenarist, etc.):
x262 --mpeg2 --preset placebo --tune film --open-gop normal --fps 24000/1001 --pulldown 32 --sar 16:9 --keyint 14 --bframes 2 --vbv-maxrate 9800 --vbv-bufsize 1835 --crf 1.5 -o output.m2v input.avs
Actually DAR is broken. I'll take a look why later (it was working).
Here (http://hotfile.com/dl/133462733/16daece/x262.zip.html) is one.
However, crf seems to be broken ?Ratecontrol isn't optimal, but I don't think CRF is broken. The scale, however, is different from H.264 and you'll need a much lower value.
Use -q instead.Unless you're testing, I wouldn't recommend that. Constant quantizer is pretty useless.

[MatLz]

I meant this kind of thinghttp://data.imagup.com/11/1134819774.png (http://www.imagup.com/data/1134819775.html)


In case that can help, I did upload the files here (http://hotfile.com/dl/133939237/3b5b7d5/x262_all_files.7z.html).

But I maybe did it wrong...

[ifb]

I meant this kind of thing <image>

In case that can help, I did upload the files here (http://hotfile.com/dl/133939237/3b5b7d5/x262_all_files.7z.html).

But I maybe did it wrong...From the TODO (https://github.com/kierank/x262/wiki/TODO), certain subme levels are broken, so don't use them.

[ifb]

Actually DAR is broken. I'll take a look why later (it was working).Setting DAR is broken when using the internal resize filter (resize assumes SAR).

[ifb]

Just to say I've tried it, files can import in Encore CS5.

Parkjoy 25p (http://www.mediafire.com/?5auu918z3hzb47n)

That's a 10 second clip, so YMMV.
x264 park_joy_1080p.avs -o parkjoy_paldvd.m2v --mpeg2 --preset placebo --tune film --open-gop normal --keyint 12 --bframes 2
--vbv-maxrate 9400 --vbv-bufsize 1835 --crf 1 --fake-interlaced --sar 16:9 --colorprim smpte170m --colormatrix smpte170m --transfer smpte170m

ffvideosource("park_joy_1080p.mkv") #lossless, from .y4m
selecteven()
lanczosresize(720,576)
colormatrix(mode="Rec.709->Rec.601")

[pandy]

so picture can have combination of interlace and progressive macroblocks? ie MBAFF

[ifb]

so picture can have combination of interlace and progressive macroblocks? ie MBAFFCorrect. Each macroblock in a frame picture can use either field, frame, or dual-prime prediction. Macroblocks in field pictures can use either field, 16x8, or dual-prime prediction.

[pandy]

Ok - bit weird but all comparison between H.264 (and other codecs) assume superiority of H.264 due MBAFF which accordingly to various descriptions is unique for H.264

[Lyris]

Yes, the fact that MPEG-2 can do that is news to me!

[ifb]

I meant this kind of thing
http://data.imagup.com/11/1134819774.png (http://www.imagup.com/data/1134819775.html)


In case that can help, I did upload the files here (http://hotfile.com/dl/133939237/3b5b7d5/x262_all_files.7z.html).

But I maybe did it wrong...Your sample is fixed by this (https://github.com/kierank/x262/commit/a31f7c98af001b20f57621649d284c892960e600) commit.

There are still bugs out there, but that was probably the worst one (aside from --subme 1).

[Gromozeka]

There are still bugs out there, but that was probably the worst one (aside from --subme 1).

Sorry, but where I can give binaries x262?
:( You are not compile this.

[Gromozeka]

Thanks to Bugmaster
binaries there:
http://www.mediafire.com/?35clmyf3qu6h81y
The bugs is fixed - best mpeg2 encoder
I compressing with this option:
x262.exe --crf 6 --aq-mode 2 --mpeg2 --no-mbtree --subme 5 --qcomp 0.6 --keyint 15 --
bframes 2 --qpmin 3 --qpmax 31 --vbv-maxrate 9000 --vbv-bufsize 1835 --colorprim smpte170m --
colormatrix smpte170m --transfer smpte170m -o 1_out.m2v 1.avs

[ifb]

Thanks to Bugmaster
binaries there:
http://www.mediafire.com/?35clmyf3qu6h81y
The bugs is fixed - best mpeg2 encoder
I compressing with this option:
x262.exe --crf 6 --aq-mode 2 --mpeg2 --no-mbtree --subme 5 --qcomp 0.6 --keyint 15 --
bframes 2 --qpmin 3 --qpmax 31 --vbv-maxrate 9000 --vbv-bufsize 1835 --colorprim smpte170m --
colormatrix smpte170m --transfer smpte170m -o 1_out.m2v 1.avsBest? I don't believe that. Analysis hasn't really been touched. There are a bunch of things left to add (real interlacing, CQMs, trellis, slice-max-mbs). Speed hasn't been considered (though I do have a faster quant patch nearly finished). Any encoder that x262 may beat at this stage must be really bad. That or x264 is so awesome that even crippling it for MPEG-2, it still beats everything. I haven't done any comparisons myself.

As for your settings, QP min/max is 1 to 31 so your --qpmax 31 is unnecessary (and I wouldn't change qpmin). You also probably want --open-gop.

[Gromozeka]

Best? I don't believe that.
Sorry, I think that x262 is best, my tests already given surprising results, in particular, I was comparing x262 and CCE. x262 has won on all counts.
Here is a series of screenshots:
http://narod.ru/disk/31168326001/01.rar.html
Thank you very much for the x262.

Analysis hasn't really been touched. There are a bunch of things left to add (real interlacing, CQMs, trellis, slice-max-mbs). Speed hasn't been considered (though I do have a faster quant patch nearly finished). Any encoder that x262 may beat at this stage must be really bad. That or x264 is so awesome that even crippling it for MPEG-2, it still beats everything. I haven't done any comparisons myself.

x262 has a great potential for development

[MatLz]

Your sample is fixed by this (https://github.com/kierank/x262/commit/a31f7c98af001b20f57621649d284c892960e600) commit.

There are still bugs out there, but that was probably the worst one (aside from --subme 1). :thanks:

[mp3dom]

I was comparing x262 and CCE. x262 has won on all counts.

You have probably setup CCE (by the way, which version and settings?) in the worst way.

[Gromozeka]

You have probably setup CCE (by the way, which version and settings?) in the worst way.
I do not speak English
Please do yourself tests)))

[mp3dom]

My god... :rolleyes:

By the way the actual stage of x262 is far from 'good-enough' output (quality wise).

[Lyris]

How did you manage to get results that bad out of CCE?

[Gromozeka]

How did you manage to get results that bad out of CCE?
use bitrate 2000-3000 kbps 2 pass
or crf 6 with this option
x262.exe --crf 6 --aq-mode 2 --mpeg2 --no-mbtree --subme 5 --qcomp 0.6 --keyint 15 --
bframes 2 --qpmin 3 --qpmax 31 --vbv-maxrate 9000 --vbv-bufsize 1835 --colorprim smpte170m --
colormatrix smpte170m --transfer smpte170m -o 1_out.m2v 1.avs
Use the options CCE what you like

[TheSkiller]

For most types of video an average bitrate of just 2000-3000 kbit/s (assuming D1 or cropped D1 resolution) is pretty low and therefore will hardly ever give results that are "visually transparent" (while that doesn't mean it's impossible to have an encoded result that is still acceptable). This is MPEG2, often interlaced at full DVD resolution, not Xvid or x264. The required bitrates for DVD are much higher.

Are the file sizes the of the x262 and CCE output roughly the same? Doesn't --crf 6 produce a pretty good quality and therefore usually a higher average bitrate (sorry I don't know)? What I mean is you cannot compare a high bitrate encode to a low bitrate one, obviously the low bitrate one will always look inferior using any encoder.

[Gromozeka]

For most types of video an average bitrate of just 2000-3000 kbit/s (assuming D1 or cropped D1 resolution) is pretty low and therefore will hardly ever give results that are "visually transparent" (while that doesn't mean it's impossible to have an encoded result that is still acceptable). This is MPEG2, often interlaced at full DVD resolution, not Xvid or x264. The required bitrates for DVD are much higher.

Are the file sizes the of the x262 and CCE output roughly the same? Doesn't --crf 6 produce a pretty good quality and therefore usually a higher average bitrate (sorry I don't know)? What I mean is you cannot compare a high bitrate encode to a low bitrate one, obviously the low bitrate one will always look inferior using any encoder.

Sorry,I can hardly speak English, but I can tell - crf 6 gave me the bitrate in the range 2500-3000 kbps with those settings

[ifb]

Recents commits now use use the same deadzone quant algorithm as x264. I've only tested on foreman and parkjoy, but it seems better. Intra probably needs some tweaking, but encoding is about 20% faster overall (depending on preset).

x262 a7c540166a
Binary (http://dl.dropbox.com/u/53163598/x262.rar)

Source (lossless)
Download (http://dl.dropbox.com/u/53163598/parkjoy_paldvd-lossless.mkv)

Old Quant (x262 89f8657800 (http://github.com/kierank/x262/commit/89f8657800132ab8467e55dd0c00dbbc56039053))
Download (http://dl.dropbox.com/u/53163598/parkjoy_paldvd.m2v)

New Quant (x262 a7c540166a (http://github.com/kierank/x262/commit/a7c540166a7e16ca4ef61a6a48e52fcc6c2db157))
Download (http://dl.dropbox.com/u/53163598/parkjoy_paldvd-new.m2v)

x262 parkjoy_paldvd-lossless.mkv -o parkjoy_paldvd-new.m2v --mpeg2 --preset placebo --tune film --open-gop
--keyint 12 --bframes 2 --vbv-maxrate 9400 --vbv-bufsize 1835 --crf 1 --fake-interlaced --sar 16:9
--colorprim smpte170m --colormatrix smpte170m --transfer smpte170m

[Gromozeka]

hm, new patch size gives 20% more in the mode crf 6, but the quality is not better
AQ can not work?

[ifb]

hm, new patch size gives 20% more in the mode crf 6, but the quality is not better
AQ can not work?CRF probably changed, so use a higher value until you get the bitrate you want. Then you can compare.

I think the parkjoy clip is better at the same bitrate. If you don't agree, find better settings. The defaults for --deadzone-intra and --deadzone-inter may need to change. I haven't spent a lot of time on this.

[Gromozeka]

I build be testing over the weekend in mode 2 pass.
but my visual assessment shows that the new algorithm is worse. Maybe it has to be set up, but now the speed is not the most important thing, I'd love to coded old algorithm but with a working Trellis))
Thank you for your work

[Mug Funky]

i've been rolling my own compiles and following closely (i've got google reader pointed at github).

the quality is becoming quite amazing.

i've found one potential issue with crf and possibly other RC modes which i think relates to ratecontrol overcompensating when the max bitrate is hit.

i haven't the maths to fix this myself without making a fool of the bitrate of course. one day.

basically, i've set qcomp to 1.0 and crf to 2 and set the max quite high (25000). bitrate scans show the quantizers steady until the bitrate hits the roof, then the quants shoot up until the bitrate sits at about half of the max, rather than just sitting on the max. i figure it's something to do with the difference in MPEG-2's quants compared to h.264's. looking at the code i can only see one part that could be altered, but i'm not convinced the problem lies there (qp2qscale and qscale2qp).

[edit]

looks like setting qcomp to 0.0 or something low changes the behaviour somewhat, so that might be a better place to look.

[alexins]

x262 mpeg-2 Video Encoder rev. 2314 x86/x64 (git-af1fd71) (http://www.xvidvideo.ru/x262-video-encoder-mpeg-2/x262-mpeg-2-video-encoder-rev-2314-x86-x64.html)

[kieranrk]

x262 mpeg-2 Video Encoder rev. 2314 x86/x64 (git-af1fd71) (http://www.xvidvideo.ru/x262-video-encoder-mpeg-2/x262-mpeg-2-video-encoder-rev-2314-x86-x64.html)

Oh please don't make builds. That implies x262 is remotely usable.

[ifb]

Oh please don't make builds. That implies x262 is remotely usable.Agreed. At the very least it should be labeled as highly experimental.

[Cyber-Mav]

has this project reached the quality levels of CCE yet?

[Selur]

"highly experimental" -> no

[amtm]

has this project reached the quality levels of CCE yet?

Yes, in 15 days it went from the developers stating "That implies x262 is remotely usable" and "At the very least it should be labeled as highly experimental" to being as good as CCE.

[Gromozeka]

has this project reached the quality levels of CCE yet?
Yes, at low bit rates is particularly well seen, but on the test video (park joy) x262 is not better. Probably CCE has been tested on these videos.
On the DVD, and TV shows source x262 better result
P.S: Excuse me for my bad English

[Manaka]

Really? At 1700kbps x262 has worse PQ compare to HCE for me

[Cyber-Mav]

how well is the multithreadding on this comming along?

[marius87]

I downloaded the encoder from XvidVideo.RU and compressed a video but the mpchc (dowloaded from the same site) and mplayer2 not playing this video. Were fin a decoder?

[ifb]

There is experimental interlaced (MBAFF) support if anyone wants to test.

Download x264 0.125.0+2415M e5ae539 (https://dl.dropbox.com/u/53163598/x262-mbaff.zip)

Patch (https://dl.dropbox.com/u/53163598/x262_mbaff_v1.diff) against e5ae539 (https://github.com/kierank/x262/commit/e5ae5392bea99c2b6b565ceb9e6c66b547653e1e)

[ifb]

After all that excellent feedback, I thought I'd post another build. </sarc>

Download x264 0.125.0+2415M 814a5fe (https://dl.dropbox.com/u/53163598/x262-mbaff-v2.zip)

On VQEG sources (ftp://vqeg.its.bldrdoc.gov/SDTV/VQEG_PhaseI/TestSequences/Reference/) 13-22 concatenated:
--tune psnr
tff: x262 [info]: PSNR Mean Y:36.683 U:39.618 V:39.962 Avg:37.414 Global:36.178 kb/s:4429.03
fake-interlaced: x262 [info]: PSNR Mean Y:36.445 U:39.719 V:40.040 Avg:37.238 Global:36.016 kb/s:4415.31

--tune ssim
tff: x262 [info]: SSIM Mean Y:0.9505212 (13.056db)
fake-interlaced: x262 [info]: SSIM Mean Y:0.9477635 (12.820db)

4:2:2 is known to be broken with interlaced, but that's the only super major issue outstanding.

[pandy]

After all that excellent feedback, I thought I'd post another build. </sarc>

:p:thanks::thanks:

[Emulgator]

What would the recent commandline syntax be like ?

[ifb]

What would the recent commandline syntax be like ?
If you're trying out interlaced encoding, the only addition is --tff (or --bff). You will also probably want to use --altscan.
NTSC DVD:
x262 input.avs -o output.m2v --preset veryslow --tune film --open-gop --colorprim smpte170m --colormatrix smpte170m --transfer smpte170m --bframes 2 --vbv-maxrate 9800 --vbv-bufsize 1835 --keyint 15 --fps 30000/1001 --sar 16:9 --crf 1 --altscan --tff

PAL DVD:
x262 input.avs -o output.m2v --preset veryslow --tune film --open-gop --colorprim bt470bg --colormatrix bt470bg --transfer bt470bg --bframes 2 --vbv-maxrate 9800 --vbv-bufsize 1835 --keyint 12 --fps 25 --sar 16:9 --crf 1 --altscan --tff

Note that if the binary is named "x262", you no longer have to add --mpeg2 to you commandline. You can switch to H.264 by using --no-mpeg2 or renaming the binary.

[Emulgator]

Wow, that's versatile !
So just one compile does both, H.264 and H.262 ?
Picture looks good so far, just tried some 1000 frames of grainy interlaced SD and compared roughly to HC.
With a bitrate of 9200 kbit/s and the given vbv-bufsize of 1835 kbit ~ 229 kByte x262's bitrate did overshoot into 10300 Mbps
where HC stayed below 9144kbps (Konran's bitrate viewer, Mode: GOP enhanced).
Too much for DVD I guessed, I just lowered vbv-bufsize to the well known 224 kB = 1792 kbit.
The resulting 9900 kbit/s peak bitrate were more decent.
More to come...

[Emulgator]

Finally got --fullhelp to display completely,
my cmd.exe seems to cut off the beginning up to --ref, leaving only the last 300 lines to be read.
Mulders Simple launcher 2.05 will list --fullhelp of x262.exe completely,
when x262.exe is brought into \toolset and renamed to x264_8bit_x86.exe.

[kieranrk]

(Konran's bitrate viewer, Mode: GOP enhanced).

Use a proper VBV analyser. Elecard might work OK.

[Emulgator]

Hm, the $$ keep me off from Elecard a bit...

Now with a given bitrate of 8500 kbit/s, vbv_maxrate 9000 kbit/s and removed crf=1 parameter
peaks come out around 9500kbit/s.

This is ok to handle for me, I just keep a bit of distance to the limits for now.

Picture quality:
Well detailed.
The --tune film works beautifully with sources that already contain some DCT blocking (like DV).

Some behaviours indeed seem to be inherited from x264.
There are regions that are treated differently:

The "wet patches" (from some x264 revisions back then) still seem to be there:
Within black mattes containing a bit almost homogenous noise after encoding
there might be some wet patches occurring, these come out smoother
while the surrounding parts behave the other way around.

And around sharp borders (white Serif Letters on black) it does look a bit more mosquitoing.
Maybe related.


After scenecuts from film content to black (or colour) matte I found some residual DCT block patterns in the black
(or colour) for the first and a few, maybe 3 to 4 following frames.

This was one of my commandlines:

"C:\_PROG\x262\x262.exe" --demuxer avs "C:\_PROG\x262\x262.avs" --output "C:\_PROG\x262\PAL-DVD.m2v" --preset veryslow --tune film --open-gop --colorprim bt470bg --colormatrix bt470bg --transfer bt470bg --bframes 2 --bitrate 8500 --vbv-maxrate 9000 --vbv-bufsize 1792 --keyint 15 --dc 10 --sar 4:3 --altscan --bff

When aiming for DVD resolutions/bitrates:
With good bitrates (6000-8500kbps) for PAL-SD details are good,
some occasional blocks might come up in parts with strong motion.

With starving bitrates (2500kbps) for PAL-SD blocks may come up heavy,
other encoders seem to help by lowpassing there if they see bitrate starving.

Good work ! Many thanks to you.

[Emulgator]

Some two months later :
Is there a chance to get a more recent build of x262 ?

[the_weirdo]

Some two months later :
Is there a chance to get a more recent build of x262 ?

I think you can find one here:
http://www.xvidvideo.ru/x262-video-encoder-mpeg-2/

[Emulgator]

Many thanks !

[Emulgator]

A brief result:
Unfortunately x262 r2433 stalls after 2frames where r2421 encoded the whole 1140 frames testclip. Same commandline.
Source: 1140 frames SD PAL-DV-avi 720x576x25i BFF.

[ifb]

A brief result:
Unfortunately x262 r2433 stalls after 2frames where r2421 encoded the whole 1140 frames testclip. Same commandline.
Source: 1140 frames SD PAL-DV-avi 720x576x25i BFF.Link to source?

[Emulgator]

FTP upload coming.

Servername: (deleted)
Username: (deleted)
Password: (deleted)

Source file: Werbung 01.avi
Encode r2421: PAL-DVD_r2421.m2v
Encode r2433: PAL-DVD_r2433.m2v

Source Script x262r2433.avs:
AviSource("D:\_VID\Testfiles\Werbung\Werbung 01.avi")
Encode Batch Script
"C:\_PROG\x262\r2433\x262.exe" --demuxer avs "C:\_PROG\x262\x262r2433.avs" --output "C:\_PROG\x262\PAL-DVD_r2433nofkdb.m2v" --preset veryslow --tune film --open-gop --colorprim bt470bg --colormatrix bt470bg --transfer bt470bg --bframes 2 --bitrate 8500 --vbv-maxrate 9000 --vbv-bufsize 1792 --keyint 15 --dc 10 --sar 4:3 --altscan --bff

Should be completely up in 1 hour.

When you successfully downloaded these files, please let me know
so that I can free up the space again.

[ifb]

FTP upload coming.

Servername: ftp.hidrive.strato.com
Username: mulgi
Password: discman

Source file: Werbung 01.avi
Encode r2421: PAL-DVD_r2421.m2v
Encode r2433: PAL-DVD_r2433.m2v

Source Script x262r2433.avs:
AviSource("D:\_VID\Testfiles\Werbung\Werbung 01.avi")
Encode Batch Script
"C:\_PROG\x262\r2433\x262.exe" --demuxer avs "C:\_PROG\x262\x262r2433.avs" --output "C:\_PROG\x262\PAL-DVD_r2433nofkdb.m2v"
--preset veryslow --tune film --open-gop --colorprim bt470bg --colormatrix bt470bg --transfer bt470bg
--bframes 2 --bitrate 8500 --vbv-maxrate 9000
--vbv-bufsize 1792 --keyint 15 --dc 10 --sar 4:3 --altscan --bff

Should be completely up in 1 hour.

When you successfully downloaded these files, please let me know
so that I can free up the space again.
Works for me. It's probably a decoder problem. Why are you even using Avisynth? Just open it directly if you aren't going to process it further. lavf and ffms2 should open a DV AVI file just fine.

--demuxer avs foo.avi
That's the same as having a one-line AVS script with AviSource("foo.avi") in it.

[Emulgator]

Hm, leaving Avisynth out of the chain does not change the picture.
DV-AVI direct into x262: Same source, same commandline: r2421 works, r2433 fails.

WinXPPro32 SP3, CPU T7600G

P.S.
Ah, here it is:

--preset veryslow broke it. If I delete the --preset parameter, r2433 works.
--preset slow works.
--preset slower fails.
--preset veryslow fails.
--preset placebo fails.

[ifb]

Try this debug build (http://www.mediafire.com/download.php?8b17m0f9o069edz). Until I can reproduce or you at least get a backtrace, there isn't much I can do.

To rule out a decoder problem, dump the input to y4m with ffmpeg/avconv and encode that instead.

ffmpeg -i foo.avi out.y4m

[ifb]

Hm, leaving Avisynth out of the chain does not change the picture.So you used "--demuxer lavf" and/or "--demuxer ffms"?

[Emulgator]

Edit: The following is not about the debug build. Still about r2433

I followed your suggestion:
--demuxer avs <filename>.avi

Just tried lavf and ffms. Fails with --preset veryslow as well.

For a brief moment I could spot "MPEG2 + VFR is not allowed" in x262 cmd window before aborting.
Hope this helps tracing the bug.

ifb, if I may ask: Is there a way to read such error messages somewhere after cmd aborts ?

[Emulgator]

Now the x262 r2433_debug build: It fails the same way for any preset slower than "slow", so starting with --preset slower.
For lavf and ffms the x262 warning "MPEG2 + VFR is not allowed" is still in x262 cmd window before aborting.
But this does not seem to be significant, because the same warning is given when encode is successful.

There seems to be no dependency on demuxers.
The chain does work for faster presets than --preset slower.

1. With --demuxer lavf -> lavf frontend reports 720x576i 16:15 @ 25/1 fps <vfr> ...(?)
x262 warns: "MPEG2 + VFR is not allowed", also for the working presets
2. With --demuxer ffms, -> ffms frontend reports 720x576i 16:15 @ 25/1 fps <vfr> ...(?)
x262 warns: "MPEG2 + VFR is not allowed", also for the working presets
3. With --demuxer avs, -> avisynth frontend reports 720x576i 4:3 @ 25/1 fps <cfr>
x262: no warnings.

Now the ffmpeg way...

[Emulgator]

ffmpeg decoded to .y4m.
y4m input to x262 r2433_debug: The same behaviour.

Any --preset <faster than slower> succeeds.
Any --preset <slower than slow> fails.

y4m frontend reports 720x576i 16:15 @ 25/1 fps <cfr>
x262: no warnings.

What kicks in at --preset slower ?
--partitions all broken ? --trellis 2 broken ? --rc-lookahead 60 ? --ref 8 ? --subme 9 ?

[Emulgator]

Now commenting all the <slower> parameters in one by one.

Ah, found it: --subme 9 breaks it. --subme 10 and --subme 11 fail as well.

P.S. Could repeat the bug on a Win7U64 system, CPU i7 920XM.
Here in Win7 the cmd window stays open, but: Silent abort, no message

[ifb]

Now commenting all the <slower> parameters in one by one.

Ah, found it: --subme 9 breaks it. --subme 10 and --subme 11 fail as well.

P.S. Could repeat the bug on a Win7U64 system, CPU i7 920XM.
Here in Win7 the cmd window stays open, but: Silent abort, no message
I can reproduce a segfault using your original commandline with --threads 1, but the backtrace is pretty useless. I'll have to look at this later.

<edit>
OK. I see what's happening. I'll work on it tonight.
</edit>

[ifb]

Fixed (https://github.com/kierank/x262/commit/6fe55fd27ed39682a707d45f75749d4fb06c4f29). Both veryslow and placebo were tested, but somehow the usual test clips never triggered this (or at least didn't crash in it).

New build:
x262 0.128.0+2443 6fe55fd (http://www.mediafire.com/download.php?f5s9yrjltu1xln5)

There's still a known bug in interlaced chroma, but at least this crash is fixed.

[Emulgator]

Many thanks, this commit fixes the subme>8 bug on my side as well.

[Emulgator]

Now the residual block structures, especially visible at scenecuts (shown are pairs of consecutive frames):

Cut to Black.
Frame 413:
http://dvd-manufactur.de/files/x262r2443_Frame_000413.jpg

Frame 414:
http://dvd-manufactur.de/files/x262r2443_Frame_000414.jpg

Cut from Black.
Frame 417:
http://dvd-manufactur.de/files/x262r2443_Frame_000417.jpg

Frame 418:
http://dvd-manufactur.de/files/x262r2443_Frame_000418.jpg

Another Cut to Black.
Frame 542:
http://dvd-manufactur.de/files/x262r2443_Frame_000542.jpg

Frame 543:
http://dvd-manufactur.de/files/x262r2443_Frame_000543.jpg

The x262_r2443 encode: (http://dvd-manufactur.de/files/PAL-DVD2_r2443.m2v) (x262 r2421, r2433 through r2443 do exhibit the same)

[zmejce]

I'm using this commands on x264:

--crf 20 --tune film --b-adapt 2 --direct auto --demuxer raw --input-csp i420 --input-res %(width)x%(height) --fps %(fpsnum)/%(fpsden) -o "%(tempvideofile)" -

Which commands would be the x262's equivalent?

Would x262 produce same quality as x264 for dvd riping and 1080 hd?

[zmejce]

I'm using this commands on x264:

--crf 20 --tune film --b-adapt 2 --direct auto --demuxer raw --input-csp i420 --input-res %(width)x%(height) --fps %(fpsnum)/%(fpsden) -o "%(tempvideofile)" -

Which commands would be the x262's equivalent?

Would x262 produce same quality as x264 for dvd riping and 1080 hd?

I've tried it with the latest x262 build and the same commands dont produce same quality-x262 is worse, but makes double smaller size. Ive tried also with crf 15 and the result was same filesize as crf 20 and same or very similar quality.

Tried it on 720x576 resolution video.

[ifb]

I'm using this commands on x264:

--crf 20 --tune film --b-adapt 2 --direct auto --demuxer raw --input-csp i420 --input-res %(width)x%(height) --fps %(fpsnum)/%(fpsden) -o "%(tempvideofile)" -

Which commands would be the x262's equivalent?

Would x262 produce same quality as x264 for dvd riping and 1080 hd?
--direct auto doesn't apply to MPEG-2.

Also, note that MPEG-2 is two generations older than H.264. For most content, there's no way x262 could ever match x264 quality at the same bitrate.

I've tried it with the latest x262 build and the same commands dont produce same quality-x262 is worse, but makes double smaller size. Ive tried also with crf 15 and the result was same filesize as crf 20 and same or very similar quality.

Tried it on 720x576 resolution video.
Again, MPEG-2 is old. Also, the CRF scale is quite different between x262/x264 (likely a bug - ratecontrol patches welcome). x262 will need a much lower CRF value.

[qyot27]

@zmejce:
Refer to this post (http://forum.doom9.org/showpost.php?p=1282676&postcount=2), although the problem in that thread was precisely the opposite of what's being discussed here (experiencing file bloat with x264 output because they used the quant values they were used to from DivX rather than bad quality in x262 because you were using typical crf values from x264).

Point being, MPEG-2 and MPEG-4 ASP both used linear scales, not a logarithmic one like H.264. Of course, this is assuming x262 is expecting values from a linear scale and not trying to adapt a log scale to MPEG-2. It's been too long since I tested with x262 to remember if that's the case.

[ifb]

@zmejce:
Refer to this post (http://forum.doom9.org/showpost.php?p=1282676&postcount=2), although the problem in that thread was precisely the opposite of what's being discussed here (experiencing file bloat with x264 output because they used the quant values they were used to from DivX rather than bad quality in x262 because you were using typical crf values from x264).

Point being, MPEG-2 and MPEG-4 ASP both used linear scales, not a logarithmic one like H.264. Of course, this is assuming x262 is expecting values from a linear scale and not trying to adapt a log scale to MPEG-2. It's been too long since I tested with x262 to remember if that's the case.MPEG-2 has linear and non-linear quant. x262 can do either, but non-linear (the default) is better.

[Biggiesized]

Where can we download the latest compiled binary?

[djonline]

High level is buggy, bad broken picture.

[djonline]

Also, I want to make 720p/60p with 60fps. How can I do this with x262 ?

[Adub]

Write an Avisynth script to convert/feed your video in a 720p/60fps format and then pipe it into x262.

[ifb]

High level is buggy, bad broken picture.

And may remain so without steps to reproduce. Please post your full commandline.

[Boulder]

Is the encoder "safe to use" in everyday things? I occasionally need to author some DVDs and there's always nice to have extra items in the toolbox.

[kieranrk]

Please test with latest patch.

[Emulgator]

A build, a build, a kingdom for a build...

[LoRd_MuldeR]

http://www.mediafire.com/file/ec81p13yvb48esz/x262.2013-04-08.rar

[ifb]

A build, a build, a kingdom for a build...
fprofiled with lavf and ffms2:
x262 32-bit (http://www.mediafire.com/download.php?cx2d6dzjmsro6tp)
x262 64-bit (http://www.mediafire.com/download.php?amk2cgp85k84r6c)

[kieranrk]

There's a crash in interlaced mode (b16x8)

[Emulgator]

Many thanks, Mulder and ifb !
Soon crunching...

[Emulgator]

Yes, I can confirm the crash of ifb's x262 r2508 (32bit) after a few frames for a interlaced PAL-SD-Source
when called as x262 to encode H.262. If renamed and called as x264 r2508 delivers a proper H.264 encode.

For x262:
MediaInfo reports a duration of 160ms (4 PAL frames), MediaPlayerClassic with ffdshow can play 2 frames.

Output was specified as .m2v.
Once I additionally give the argument -mpeg2 as listed in -fullhelp, x262 crashes immediately without encoding anything.

From -fullhelp
MPEG-2 (H.262):

-mpeg2 Encode as MPEG-2 instead of H.264
-dc <integer> Specify intra DC precision to use (8 to 11) [8]
-altscan Use alternate MPEG-2 VLC scan order, not zigzag
-linear-quant Use MPEG-2 linear quantization table
-no-altintra Use MPEG-1 VLCs (Table B.14) for intra blocks

Outfile type is selected by filename:
.264 -> Raw bytestream
.mkv -> Matroska
.flv -> Flash Video
.mp4 -> MP4 if compiled with GPAC support (yes)
Output bit depth: 8 (configured at compile time)

Shouldn't the extension .m2v should be listed in fullhelp as one of the triggers for MPEG-2 (H.262) encoding ?...

.m2v -> MPEG-2 (H.262)

Edit: Nah. I was wrong.
The output extension .m2v does not trigger H.262.

It is indeed:

{Or is it the call as x262.exe alone ?}

Renaming the suspicious x262.exe to x264.exe and running it with the same commandline
containing the output extension .m2v delivers a successful Level 3.1 H.264 encode,
with the (specified from me) misleading extension .m2v.

[develX]

There's a crash in interlaced mode (b16x8)


Yes, it only happens in interlaced mode and multithreaded configurations. It seams that the stack is overwritten (i_width > 16!!). When i rewrite this to input == output (build 129) then it crashs in refine_subpel()/x264_pixel_satd_8x8_sse4 (), line 913 (build 133).
Is there a newer version without that problems?

[kieranrk]

It's a known bug but I haven't had time to fix it: https://github.com/kierank/x262/issues/2

[develX]

It's a known bug but I haven't had time to fix it: https://github.com/kierank/x262/issues/2

Ok, I'll try it.

[kieranrk]

You can come on #x264dev on freenode if you need help.

[kieranrk]

Can anyone reproduce the interlaced crashes with the latest git? As far as I know they are gone.

[Emulgator]

A build would be nice.

[kieranrk]

A build would be nice.

I am not permitted to do that for various reasons.

[zerowalker]

https://www.sendspace.com/file/fng0dj

Here is my build of it.

Is x262 DVD Compliant?

EDIT:

Link Fixed*

[Emulgator]

Many thanks zerowalker, but this link seems wrong somehow...
PM ?

[Emulgator]

Many thanks ! Crash on interlaced footage is gone for me with zerowalker's build (x262 core140 r2615),
tested on the same clip as in 2013.
Residual block artifacts 1 frame after cuts to black still there.

[zerowalker]

I did check the quality of the encoder.
I didn't do any thorough testing, but the from the looks of it the quality is Good, but not more than that.
It seems to have a hard time on Edges, probably retaining details.

Disturbing thing is that x262 is called x264 in the command tool (IIRC?), other than that, well it's x264 with MPEG-2;)

[Procrastinating]

Nowadays, how does the encoded quality of x262 compare with CCE? Ie, under the best-possible quality settings(not necessarily DVD-compliant), on say 720p test footage. That is to say, ignoring bugs and under ideal conditions, which is "best"?

[Emulgator]

CCE, in particular SP3 is not yet beaten by x262 for DVD encoding.
But if the fixes for residual artifacts go on, x262 may well sit on top one day.
I see potential for good use of bitrate and very good detail retention.

[Blue_MiSfit]

I wonder how it compares to Elemental. I was impressed by their encoder's balance of detail preservation and lack of blocking at low bitrates (i.e. 14 Mbps 1080i29.97 for CableLabs VOD files) enough to pick it over Canopus and Mainconcept (Carbon Coder and Digital Rapids respectively) for a pretty substantial workflow.

[poisondeathray]

Thanks zerowalker for recent compile

My general impressions on DVD compliant progressive testing on clean sources:


Overall OK, seems better on gradients that HCEnc

But:
1) Problems with some object edges, there tends to be artifacting, much like DCT ringing. I haven't played with psy settings yet

2) Seems to be some "keyframe" pumping issues, temporal stability issues, tested with open-gop settings. Perhaps I-B-P bitrate allocation settings isn't proportionally ideal yet

[Sparktank]

I'm pretty surprised by the quality for this.

Going from HCenc to x262 2pass+OpenCL, the quality difference is stunning.
I do notice some noise though, for detailed scenes.
Mostly when people are far away from camera (not medium shots or close ups).

I have to agree that gradients are much more better here.
Using cretindesalpes' Dither_package to resize, I'm just blown away by results.

Using parts of old commands I found earlier from this thread for NTSC. (-preset veryslow --tune film)
Added 2passes to target DVD5, changed fps to 24000/1001 with --pulldown 32 (took awhile to realize "32" didn't mean thirty-two).

I was worried at first that --opencl might break, but I guess Nvidia fixed some issues in recent updates.

I've only done 5min samples so far.
I have yet to do a full movie and burn to RW to check for compatibility in players. :devil:

I think I'm going to have some real fun with this and HCenc in the next few days. I quite like the idea of doing x264-like encoding for MPEG-2/DVD.

Thanks for the updated binary. :)

[Lyris]

Sparktank, any idea how it compares to CCE SP3?

[Sparktank]

Sparktank, any idea how it compares to CCE SP3?

I would have to guess with the noise generation, it still has some distance to go.

I do find that creating a lossless intermediate (UtVideo Codec) creates better quality.

I'm not entirely sure which settings have no effect on MPEG2 encoding. But I'm sure toning down some options would improve.

[Sagittaire]

1) Problems with some object edges, there tends to be artifacting, much like DCT ringing. I haven't played with psy settings yet

2) Seems to be some "keyframe" pumping issues, temporal stability issues, tested with open-gop settings. Perhaps I-B-P bitrate allocation settings isn't proportionally ideal yet

If you have the same Rate Control setting than x264, it's easy to solve that ... ;-)

[SuLyMaN]

Is this a good alternative to HCenc?

[Emulgator]

If development continues, yes.

[chiphoi]

Does this software continue development? Im looking forward for this :helpful:

[rwill]

Mine (https://code.google.com/p/y262/) is not ready either... I think mpeg2 is dead.

[Emulgator]

As long as DVDs are sold and have such a wide hardware player base, I wouldn't pronounce MPEG-2 as dead.

We shouldn't forget: every blu-ray player (next generation included) is required to play DVD,
so users will see no need to drop their DVD discs, and so will still buy DVDs as a habit.

I would gladly welcome x262 as an improved alternative, optimized to the max and even donate to have this achieved.

These days I am still asked to make DVDs for maximum compatibility's sake, no matter that I like blu-rays more...

[colinhunt]

It's not dead as long as DVDs are being authored and released commercially. I'm still being asked for more DVD than Blu-ray authoring services.

Like Emulgator, I'd be happy to donate to see x262 development continue.

[SuLyMaN]

Going from HCenc to x262 2pass+OpenCL, the quality difference is stunning.
I do notice some noise though, for detailed scenes.


Hi mate. You have any screenshots/video sample to compare to / as reference? Thanks.
Does it mean its much better than HCenc yet? Or is the noise too noticeable?

[Reel.Deel]

Can someone please build the latest x262 (https://github.com/kierank/x262/commits/master)? zerowalkers' link (http://forum.doom9.org/showthread.php?t=157675&page=7#post1688501) is dead and ifb's binaries (http://forum.doom9.org/showthread.php?t=157675&page=7#post1623674) are from 2013 :(.

[Sparktank]

Can someone please build the latest x262 (https://github.com/kierank/x262/commits/master)? zerowalkers' link (http://forum.doom9.org/showthread.php?t=157675&page=7#post1688501) is dead and ifb's binaries (http://forum.doom9.org/showthread.php?t=157675&page=7#post1623674) are from 2013 :(.

I would love to second this.

Would love to do some comparisons with recent builds of everything, especially since my downscaling has changed over the years.

[Boulder]

Can you try this one : https://drive.google.com/file/d/0BzeF_1syecQwQ0NyelFTSm92WTA/view?usp=sharing (Avisynth or raw input only)? I just did a quick compile to see how it goes.

[Reel.Deel]

Can you try this one : https://drive.google.com/file/d/0BzeF_1syecQwQ0NyelFTSm92WTA/view?usp=sharing (Avisynth or raw input only)? I just did a quick compile to see how it goes.

Thanks Boulder :). If it's not too much trouble can you also provide a 64-bit binary?

[Emulgator]

Many thanks, Boulder and kierank for r2633 !
CPU: i7-4960X, Win7U64
Avisource failed as expected (source was DV-AVI), fallback to ffmpegsource2 worked.

Residual block artifacts 1 frame after cuts to black still there.

[lithoc]

Just read about pied piper , https://github.com/danielrh/losslessh264

It would be fantastic if someone can port these to MPEG-2

[Boulder]

Thanks Boulder :). If it's not too much trouble can you also provide a 64-bit binary?Strange - the forum didn't send me an email until now.

I can make a 64-bit build probably on Sunday when I get back home.

[rwill]

Just read about pied piper , https://github.com/danielrh/losslessh264

It would be fantastic if someone can port these to MPEG-2

lithoc,

it appears pied piper applies to the recompression layer placed on top of the h.264 encoder in question. Given that this h.264 encoder is only supporting constrained baseline profile ( CAVLC ) a recompression layer with well designed context models will give at least 10% ( I am speculating here ) compression gain. This compression gain is also in the range someone would get when switching from CAVLC to CABAC ( I am not speculating here ) as the entropy coder.

A lossless recompression layer for older standards will almost always produce compression gain simply from implementing context models and arithmetic coding. For example there are lossless compression methods for JPEGs that are able to store a JPEG at a significantly reduced size. If this interests you reading up on the "PAQ" compressor or Context Mixing ( CM ) compressors in general is a good start.

Sadly the resulting files are not backwards compatible - so for example a DVD player will not know what to do with a recompressed MPEG-2 stream. So unless you control the whole ecosystem of generation, distribution and playback device such a thing cannot be just ported to MPEG-2.

And using an encoder of a more recent standard will most likely produce higher compression gain than switching out the entropy coder of some old standard.

[Boulder]

Here's an x64 build: https://drive.google.com/file/d/0BzeF_1syecQwSDZxZ25PQ3YtbEk/view?usp=sharing

I was unable to fprofile it, the binary kept on crashing at one profiling step (there were also some warnings shown during configure), lossless if I'm not mistaken. Hopefully the x262 portion works fine for you.

[Blue_MiSfit]

Mine (https://code.google.com/p/y262/) is not ready either... I think mpeg2 is dead.

MPEG-2 is FAR from dead :)

The vast majority of linear TV is originated from MPEG-2 masters - quite commonly XDCAM HD422 which is Long GOP 4:2:2 MPEG-2 at 50 Mbps.

The HD-SDI output from a playout chain is most commonly encoded and uplinked on satellite using H.264, but not always. There are lots of MPEG-2 channels still. More importantly, cable distribution still uses a ton of MPEG-2, even for VOD. At work we encode MANY 15 Mbps CableLabs MPEG-2 streams every month!

I think we'll be living with MPEG-2 for at least another 5-10 years!

[ifb]

Much work has been done over the last few months. The whole MPEG-2 tree has been rebased onto current x264 and I've added 82 commits on top of that rebase.

tl;dr - binaries (https://github.com/kierank/x262/releases)

Bitrate savings vs the last public x262

Negative BD-XPSNR-Y is good. It means x262 needs fewer bits for the same quality. Both encoders at --preset medium, --bframes 2, matched GOP length, six CRF points per curve.

clip format BD-XPSNR-Y
foreman_cif 352x288p -17.9%
VQEG src3 (PAL) 720x576i -29.8%
VQEG src13 (NTSC) 720x480i -14.4%
park_joy 1920x1080p -21.7%

Roughly 14-30% fewer bits for the same measured quality, averaging about -21%.

How much of that is the rebase and how much is from new work? Measuring a build at the rebase boundary on progressive content:

clip rebase only new work total
foreman_cif -2.3% -16.2% -17.9%
park_joy +1.2% -22.6% -21.7%

So the upstream rebase is close to neutral for MPEG-2 on its own. Essentially all of the gain is the MPEG-2-specific work this year.


Speed

At the same preset the current encoder is slower, because it is doing considerably more work per macroblock (trellis, per-MB field/frame RD, per-picture scan and VLC table decisions). Both at --preset medium:

clip old x262 current ratio
foreman_cif 1343.2 1213.2 0.90x
park_joy 1080p 69.3 53.2 0.77x
VQEG src3 576i 363.6 142.6 0.39x

But that is missing the point. The useful question is how fast it encodes at matched-or-better quality. Current x262 at --preset faster, against the old encoder at --preset medium:

clip speed vs old medium BD-XPSNR-Y
foreman_cif 2.03x -13.8%
park_joy 1080p 2.09x -17.3%
VQEG src3 576i 1.90x -23.5%

So roughly twice the speed and 14-24% fewer bits at the same time. --preset veryfast reaches about 2.3x and still comes in 11-20% ahead on BD-rate.

New MPEG-2 assembly added (checkasm on Zen4):

forward DCT, 4 blocks at once (AVX-512) 19.1x
half-pel filter (AVX2) 22.7x
lowres frame init (AVX2) 18.3x
coefficient clamp (AVX2) 45.6x
zigzag field scan (AVX-512) 16.1x
run/level extraction (AVX-512-ICL) 17.9x
dequant intra / inter (AVX2) 6.3x
IDCT (AVX2, 2 blocks) 7.0x
trellis dynamic programming (AVX-512) 1.9x

What is new since 2015

Interlaced coding

True field motion compensation, and field motion estimation now searches both reference field parities instead of only the top field.
Field/frame coding decided per macroblock by RD at subme >= 6, replacing a VSAD prepass that was a poor predictor. All 20 VQEG-422 clips improve, from -0.1% to -80% BD-XPSNR-Y, mean -29%. The extreme end is real: on src8 the VSAD prepass spent 18.8 Mbit to reach 37.2 dB where the RD decision gets 38.5 dB out of 7.4 Mbit.
Per-macroblock field DCT, independent of the motion decision, with its own RDO. On the 13 clips that pick field DCT for at least 15% of macroblocks it is worth -4.4% BD-XPSNR-Y on average and up to -12.6%. Where field DCT is rarely chosen it is neutral: -0.2% mean, worst case +0.8%.
Alternate (field) scan chosen adaptively per picture rather than forced for the whole stream. Forcing it interlaced loses on 13 of 20 VQEG clips.
Dual-prime prediction for interlaced P frames. It only applies to P frames with no B frame between them and their reference, so with the default --bframes 2 it rarely comes up in practice.
Interlaced 4:2:2 works.


Coding efficiency

Trellis quantisation. MPEG-2's run-level VLC has no adaptive state and its dequant is per-coefficient independent, so the trellis is exactly optimal rather than approximate. 2.6-5.8% BD-XPSNR across 12 clips.
DCT coefficient decimation, scored by real VLC bit cost.
Blocks whose coefficients IDCT to all-zero are no longer coded (spec Note 2).
Variable f_codes chosen per frame from actual motion instead of the level maximum. 1.9-4.3% BD-XPSNR.
Motion search constrained up front instead of clamped afterwards, and clamped to the MVD range rather than the absolute MV.
Adaptive intra VLC table (B.14 vs B.15) per picture. Beats the best single whole-clip table in either direction.
Per-picture CQM centering (--cqm-recenter, now on by default) so adaptive quantisation's per-MB decisions land where AQ intended. AQ delivery error 6.2% -> 4.0%.
Default inter matrix is now mildly shaped instead of flat-16.
Half-pel bidirectional refinement, four-way B_BI_BI RDO, accurate MB-type header costs in mode decision.
mb-tree qcompress recalibrated for MPEG-2's non-linear quantiser (0.6 -> 0.45), intra deadzone split into separate luma and chroma values.
b-adapt 2 fixes: an mb-tree boundary-anchor over-boost and an interlaced lowres distance-parity artifact that collapsed pan GOPs to IBPBP.
Fades: the lookahead no longer plans against a brightness-corrected reference it cannot actually use, which was starving fade frames of bits. Up to 11 dB recovered on the worst fade frames.


Rate control and VBV

Emergency VBV mode.
Real per-picture vbv_delay signalling with --nal-hrd cbr.
VBV timing fixed with --pulldown.
Better first-I-frame quantiser selection.
Filler written as zero stuffing rather than an H.264 filler NAL.


Formats and compliance

D-10 / IMX (SMPTE 356M) support via --imx-class 50|40|30.
--slice-max-mbs for configurable slice size (D-10 needs one MB per slice).
Periodic intra refresh.
Level validation: Main level forced when B-frames are used, horizontal MV range checked, asymmetric H/V MV range allowed.
CFR enforced (MPEG-2 has no VFR syntax); default keyint 125, default bframes 2.


Speed

Simple IDCT from ffmpeg replaces the MSSG W1-W7 IDCT. Lower error on every IEEE-1180 metric, 30% faster even in C, and reconstruction is now bit-exact with ffmpeg's decoder.
AP-922 forward DCT, batched two-block AVX2 and four-block AVX-512.
Pixel subtraction folded into the forward DCT; FDCT and IDCT batched across adjacent blocks.
Half-pel planes precomputed instead of interpolated on the fly (+17% median).
VLC tables precomputed with total bit sizes, and adaptive statistics counted inside the write loop instead of in extra passes. ~15% faster all-intra; VLC writing dropped from ~25% to ~9% of all-intra time.
New SIMD for dequant, half-pel filter, chroma MC, lowres init, field zigzag, run/level extraction, coefficient clamp, and the trellis.
Dedicated MPEG-2 preset and tune ladders, with the defaults retuned (psy-rd 1.0 -> 0.1, scenecut 40 -> 20, inter deadzone 21 -> 31).


Correctness

Fixed a stack buffer overflow in get_ref for interlaced B-frames uninitialised partition reads in bidir refinement
QP corruption across RDO runs
undefined behaviour in bitstream alignment and signed shifts
a memory leak in x264_encoder_parameters


Build and API

Github CI produces a self-contained x262.exe (ffmpeg, dav1d, ffms2, L-SMASH, AviSynth statically bundled).
X262_BUILD version macro, x264_mpeg2 runtime capability variable, and an x262.pc pkg-config file.

[Emulgator]

Wow. Now that IS impressive. Many thanks for your work, can not wait to test-fly that build.

[rwill]

Good to see that Mpeg-2 Video is still seeing some development. A popular encoder brand even.

Just a small hint, you might want to check out the error the AP-922 FDCT -> ffmpeg IDCT roundtrip gives. I personally ran into problems with AP-922 and subsequently dropped it for something slower but more precise.

[ifb]

You're right that the roundtrip matters (even though only IDCT precision is specified). A sloppy FDCT produces coefficients wrong relative to the true DCT, so even a perfect IDCT will faithfully reconstruct wrong coefficients. Trellis and DCT decimation both work in the coefficient domain, so FDCT error corrupts RD decisions, not just the residual. So I measured the roundtrip and I get the opposite result. :D

AP-922 emits in the 8x coefficient domain. y262_fdct_c emits 1x. The Simple IDCT wants 1x. So a test that feeds both to ff_simple_idct_int16_8bit has to shift the 8x output down by 3 first, but that shift isn't part of the transform. It throws away three bits of the FDCT's output before the IDCT ever sees them which biases toward a transform that is already 1x.

Same three transforms, same inputs, same Simple IDCT, the shift being the only variable:

Through the INTEGER Simple IDCT (naive >>3 to reach the 1x domain):
x262 AP-922 max_err=4 rmse=0.54179 (8x native, shifted >>3)
IJG-LLM islow max_err=4 rmse=0.53989 (8x native, shifted >>3)
y262 matmul max_err=1 rmse=0.28984 (1x native, no shift)

Through an EXACT IDCT, no integer shift anywhere:
x262 AP-922 max_err=0.2282 rmse=0.04662
IJG-LLM islow max_err=0.1840 rmse=0.03720
y262 matmul max_err=1.3661 rmse=0.28905

The ranking inverts. Under the shift, AP-922 looks 1.9x worse than your transform. Without it, it's 6x better.

The giveaway is the top block: AP-922 lands at 0.54179 and islow at 0.53989, a 0.35% difference. Measured without the shift, the same two transforms differ by 25% (0.04662 vs 0.03720). Under the shift that real gap collapses to almost nothing, because neither number is measuring the transform any more. Both are pinned to the >>3 truncation floor, which is larger than the quantity under test.

None of this is a claim that AP-922 is the most precise transform available. It isn't, and ffmpeg's own dct-test says so (DCT SSE2 is AP-922):

make libavcodec/tests/dct && ./libavcodec/tests/dct 0 8

DCT REF-DBL: max_err=0 omse=0.00000000
DCT IJG-LLM-INT: max_err=1 omse=0.07622812
DCT SSE2: max_err=2 omse=0.17118125
DCT IJG-AAN-INT: max_err=48 omse=26.39022891


Let's do a practical test (which I had done when switching transforms a few months back). Compare to x262 with a full double-precision DCT. This gives us an upper bound on what any FDCT can buy. Four clips encoded --trellis 2 and CQP. XPSNR-Y and SSIMULACRA2:

QP2 QP4 QP8 QP16
foreman ap922 41.4627/93.79 39.4058/92.29 33.8963/86.50 29.6242/77.22
exact 41.4717/93.77 39.4071/92.28 33.9017/86.53 29.6345/77.18
mobile ap922 47.6963/94.24 44.2696/92.24 35.7903/83.88 29.3435/67.92
exact 47.6919/94.25 44.2858/92.25 35.7908/83.88 29.3485/67.83
park_joy ap922 44.9198/91.12 42.2470/87.62 37.9421/78.77 33.9934/61.37
exact 44.9343/91.13 42.2490/87.63 37.9421/78.81 33.9923/61.36
src13 422i ap922 43.7065/93.48 41.4391/91.37 35.9449/85.75 31.0082/74.89
exact 43.7133/93.46 41.4533/91.34 35.9483/85.76 31.0100/74.87

Bitrates match within 0.1%. Largest XPSNR delta across 16 pairs is 0.016 dB and the largest SSIM2 delta is 0.090. Even the double transform doesn't win consistently. A transform that were actually limiting quality would win monotonically at every QP on both metrics.

If you've measured a real difference on real content, I'd love to see it. I simply can't find anything to backup the "AP-922 isn't good enough" claim, though a >>3 could make it seem that way.

[HQ-LQ]

Can any meaningful statements already be made about the quality compared to other MPEG-2 encoders—such as HCenc and CCE—or is the development at too early a stage for that?

[Z2697]

Maybe a bit weird but when I use --output-depth 10 (never gonna work of course) the CLI crashed instead of thorw error and exit.

CQP mode --ipratio and --pbratio seems broken.
Or is this a way to gain more precise control over quantization, like different quantization table or something?

With default parameter:

--qp 5
x262 [info]: frame I:15 Avg QP: 1.00 size:314784
x262 [info]: frame P:55 Avg QP: 9.00 size: 84954
x262 [info]: frame B:30 Avg QP: 9.00 size: 25639

--qp 6
x262 [info]: frame I:15 Avg QP: 9.00 size:148146
x262 [info]: frame P:55 Avg QP:10.00 size: 76315
x262 [info]: frame B:30 Avg QP:10.00 size: 21623

--qp 7
x262 [info]: frame I:15 Avg QP: 6.00 size:114118 (why is this smaller than 9?)
x262 [info]: frame P:55 Avg QP:11.00 size: 53507
x262 [info]: frame B:30 Avg QP:11.00 size: 15486

[rwill]

If you've measured a real difference on real content, I'd love to see it. I simply can't find anything to backup the "AP-922 isn't good enough" claim, though a >>3 could make it seem that way.

Ah, you went with more precision for the forward transform and less precision in the forward quant tables. I always had doubts when for higher quantizer entries the numbers got very small.

For me the Intel transforms were too much tuned for speed so I just did some precise idct and then a matching forward transform so y262 does not confuse itself. I tested precision with synthetic and clean CG content with a long GOP if I remember correctly, but I don't have any anymore as this was a decade ago...

[ifb]

Ah, you went with more precision for the forward transform and less precision in the forward quant tables. I always had doubts when for higher quantizer entries the numbers got very small.

The damage is actually at the other end of the table.

At high quantizers the reciprocals do get thin. At qscale 112 against the default intra matrix's 83, the multiplier is down to 14. That makes the effective step about 0.7% coarser than the dequant step. Trellis is mostly immune, though. The multiplier only picks the starting level n, and the candidates {0, n-1, n} are scored against the exact integer dequant, so a coarse reciprocal can mislocate the starting point but never the evaluation. I verified exhaustively (every coefficient magnitude against every table entry, full qscale grid, both default matrices) that the quantizer never lands above true round-to-nearest, so the residual failure mode is n starting one low and the true nearest level n+1 dropping out of the candidate set. Windows shrink as mf/65536, so it affects about 0.02% of magnitudes at the top of the grid.

Where precision actually went missing was the low end, where the multiplier is large and the rounding bias 2^15/mf is the small number. It was computed with a truncating divide on top of the already-truncated multiplier, so at qscale 1 an ideal bias of 4.75 became 4 (three quarters of a coefficient unit low) which pushed the round-to-nearest level out of the trellis candidate set for ~15% of coefficient magnitudes at that entry. Computing the bias with a rounding divide cuts that to ~4%. At QP1 this buys 1.5–2x the encoder's own QP2->QP1 rate-quality slope, though it only matters at near-lossless rates. Output is bit-identical from QP12 up.

Faster settings (trellis=0|1) don't get the exact-dequant protection. Against an idealized real-arithmetic deadzone quantizer, about 5% of magnitudes land one level off, fairly uniformly across the grid. But there is no round-to-nearest premise to violate. The quantizer is deliberately biased by a deadzone whose width was empirically tuned. So the systematic truncation is absorbed into the deadzone tuning rather than stacked on it.

As for fixing the trellis start "properly" (deriving it from a nearest-reconstruction search against the exact dequant), since the biased start provably never lands above true round-to-nearest, it could only ever raise the level by one, so its effect is to add n+1 as a fourth trellis candidate. That was tried when the trellis went in. On a 1080p CQP BD-XPSNR sweep it moved the result by 0.01% (and that was before this precision improvement). Not worth it.

I will push this rounding divide change (plus fixes for cqm-centering and the 10-bit crash) maybe today.

[ifb]

Build r1 (https://github.com/kierank/x262/releases#release-x262-r1-20260820031018) 72a1323e


Use rounding division for more accurate quantization tables (barely affected QP 1-4).
Don't crash on invalid bit depth or chroma formats.
Rework ip/pb ratio to be more accurate (especially at low qp).
Warn if a user CQM is invalid instead of silently falling back to flat16.
CQM centering: Price the cost of the CQM. Don't misfire when the scaled qp is clamped by qpmin/qpmax.


Note that updates will be rebased and force-pushed to GitHub. API will be tracked/bumped when needed (for library users) and downloadable releases are serialized by CI. Think of x262 as a bunch of WIP patches forever waiting to be reviewed, since the idea is still to merge it upstream. (You can still rename the binary to x264 or use '--no-mpeg2' and get identical output to x264.)

CQP mode --ipratio and --pbratio seems broken.
Or is this a way to gain more precise control over quantization, like different quantization table or something?

Exactly. --cqm-recenter was kicking in but there was a bug related to qpmin/qpmax clamping the scaled/effective QP. So let's talk about centering.

MPEG-2 quant only has codes 1-31. Its qscale has four linear regions, each with different slopes (compare to H.264 which is logarithmic and goes to 51). The low end of the scale is very coarse.

code qscale step rel.step
1 1 1 100% <- band start
2 2 1 50%
3 3 1 33%
4 4 1 25%
5 5 1 20%
6 6 1 17%
7 7 1 14%
8 8 2 25%
9 10 2 20% <- band start
10 12 2 17%
11 14 2 14%
12 16 2 12%
13 18 2 11%
14 20 2 10%
15 22 2 9%
16 24 4 17%
17 28 4 14% <- band start
18 32 4 12%
19 36 4 11%
20 40 4 10%
21 44 4 9%
22 48 4 8%
23 52 4 8%
24 56 8 14%
25 64 8 12% <- band start
26 72 8 11%
27 80 8 10%
28 88 8 9%
29 96 8 8%
30 104 8 8%

"Very coarse" = relative/ratio step size (Δqscale/qscale) is huge near the bottom because qscale itself is tiny there. Same absolute Δ=1 is a 100% jump at qscale=1, but only a 3% jump at qscale=31.

The idea behind --cqm-recenter is to use a CQM to move the effective QP into a region of the scale that is less coarse. This makes AQ and ratecontrol more accurate, but makes the reported QP especially meaningless. The CQM emitted is a scaled version of whatever matrix would otherwise be in effect (the default or a user CQM).

(I will take a quick moment to say that custom matrices in general are obsolete, as is using >8 DC precision or --linear-quant. Fight me.)

Fixing this reopened the ip/pbratio rabbit hole again. I made those ratios actually be ratios, which then lead to retuning --ipratio.

--tune animation/grain are not dialed in and --pbratio may still need tweaking (pbratio only matters for CQP or when mbtree is disabled).

[Blue_MiSfit]

This is so cool, I'm very glad to see open source MPEG-2 encoding get so much attention. One of the most relevant use cases I still see for MPEG-2 is producing XDCAM HD422 files for HD playout servers. There would be value to the community in a set of constraints like --imx-class for this use case. Has this been considered?

[ifb]

This is so cool, I'm very glad to see open source MPEG-2 encoding get so much attention. One of the most relevant use cases I still see for MPEG-2 is producing XDCAM HD422 files for HD playout servers. There would be value to the community in a set of constraints like --imx-class for this use case. Has this been considered?I will add XDCAM HD support if anything special is needed. For IMX, I have the SMPTE spec and samples I recorded with a PDW-HD1500, but I don't have anything for the HD operating points.

For now I'm taking it as a good sign that FranceBB hasn't made a new issue (https://github.com/kierank/x262/issues/14#issuecomment-5309214185) :D

[Blue_MiSfit]

https://docs.aws.amazon.com/mediaconvert/latest/ug/xdcam-rdd9.html

That's a starting point.

I thought the actual SMPTE RDD 9 document was free but it looks like it's behind a paywall. A lot of the hard stuff is in the MXF wrapping which is obviously out of scope.

Perhaps my back-and-forth with Frank will be useful :) https://forum.doom9.org/showthread.php?t=183932

[ifb]

RDD 9-2009 (https://pub.smpte.org/doc/rdd9/20090622-pub/rdd09-2009.pdf) Table 1 has some info about GOP structure for 1080i/720p. No mention of 1080p at all.

I'll take samples from a real Sony XDCAM HD recorder if anyone has some, but it looks like x262 already supports everything?

XDCAM could end up needing --xdcam-compat (analogous to --bluray-compat), but users would still need to specify other things (same as with bluray). I haven't seen anything yet that justifies adding such a format-hack, though.

[excellentswordfight]

RDD 9-2009 (https://pub.smpte.org/doc/rdd9/20090622-pub/rdd09-2009.pdf) Table 1 has some info about GOP structure for 1080i/720p. No mention of 1080p at all.

I'll take samples from a real Sony XDCAM HD recorder if anyone has some, but it looks like x262 already supports everything?

XDCAM could end up needing --xdcam-compat (analogous to --bluray-compat), but users would still need to specify other things (same as with bluray). I haven't seen anything yet that justifies adding such a format-hack, though.
As far as i know, at least for HD422 profile, its 12F for 50i/25p/23,976, and 15F for 59,95i/29,97p with 2 bframes, like this: IBBPBBPBBPBBI and both open and closed GOP are supported.

What flavor would you like a sample of?

Attached is the different MPEG-2-based XDCAM flavors.