[New MPEG2 (little) challenge] High complexity sample

[Sagittaire]

You think seriousely that:
1) My MPEG2 encoder is the best and by far ... !!!
2) DVD encoding can make praticaly lossless encoding

Try to encode with your favorite MPEG2 encoder and this little source (http://multimediacom.free.fr/Download/lossless.avi)

and see the result ... lol

[Mug Funky]

this isn't uncorellated noise is it?

or the gateway scenes from cowboy bebop after standards-converting?

there's a lot of stuff that'll kill MPEG. that's what pre-compression filtering is for - if you get something that looks bad all the way up to max bitrate, try remove detail in a less noticable way than the encoder does.

btw, i'm still downloading :)

[edit]

btw, i think someone boasting 1) will not be the kind who believes 2) in the slightest.

[Sagittaire]

this isn't uncorellated noise is it?
there's a lot of stuff that'll kill MPEG. that's what pre-compression filtering is for - if you get something that looks bad all the way up to max bitrate, try remove detail in a less noticable way than the encoder does.


don't work with this ... lol

[Amnon82]

I did a little test with CCE and HC. Both did the job great. CCE is better in the waves scene. HC did the rest better. The settings I used you can read out of the Last.run|HC.ini.

here we go (9-11 MB):

CCE 2.70|Q1|3500|9800|AVAMAT6 (http://www.filefactory.com/get/f.php?f=5dab24c749800890bd9e27f7)

CCE 2.70|Q1|9000|9800|AVAMAT6 (http://www.filefactory.com/get/f.php?f=f8895b0a02ab118963dd09bc)

HC0.16.0.1|CQ1|9800|AVAMAT6 (http://www.load.to/?d=MrPzyjIfgY)

[Sagittaire]

CCE is better in the waves scene

it's a blocking fest like for the other MPEG2 encoder ... MPEG2 is simply unable to make good DVD MPEG2 compliant stream encoding with this source ...

[sparskter]

You think seriousely that:
1) My MPEG2 encoder is the best and by far ... !!!
2) DVD encoding can make praticaly lossless encoding

Try to encode with your favorite MPEG2 encoder and this little source (http://multimediacom.free.fr/Download/lossless.avi)

and see the result ... lol

could you please give details on what you used for your tests ?
1] what(or which oneS) mpeg2 encoder did you used on YOUR own tests ?
2] which quantization matrix did you used ?
3] VBR ? CBR ? Q Factor ?
4] Passes how many ? 1,2,3 ,4 5 pass ?

could you give these details please ?

best regards!

happy new year!

[Sagittaire]

could you please give details on what you used for your tests ?
1] what(or which oneS) mpeg2 encoder did you used on YOUR own tests ?
2] which quantization matrix did you used ?
3] VBR ? CBR ? Q Factor ?
4] Passes how many ? 1,2,3 ,4 5 pass ?

could you give these details please ?

best regards!

happy new year!

you can use all the setting that you want ... but I try personnaly with these MPEG2 encoder:

- TMPGEnc with Notch matrix
- CCE with NPass and different matrix
- HCEnc with different matrix
- Libavcodec with different matrix

But it's useless to try yourself because MPEG2 must use very high quant to respect DVD vbv specifications -> high blocking level for water scene with this source ...

[Mug Funky]

i'm tempted to run this through a spruce encoder and see what comes out, but that's way too fiddly. those things perform very well at high quants though - they seem to fudge it somehow so that you still see detail even up to quant 11-15 (nonlinear). they do seem to do NTSC a shade better than PAL though, more's the pity.

[edit]

is there any set max bitrate? i'm doing 9250 at the moment, but i usually do 8500.

i'm trying out a precompression filter that i hope will find just the parts where SAD based motion estimation goes bad, and blur only these bits. it's looking pretty good so far - hardly any blocking at all, and looks just fine on a TV.

[CruNcher]

yeah chaotic movement the top ten all time video compression killer.
Sounds nice Mug Funky but i doubt you will get away under 11 Mbits with it tough
where someone would say subjectively it looks ok.

[dragongodz]

what mpeg2 at dvd rates introduces some blocking to high speed, high motion scenes ? when did this happen ? who knew ? ummm just about anybody who has been encoding for awhile. i can give you a bunch of clips that will all do the exact same thing, its absolutly nothing new.

you should note though that you can not see most of the blocking with regular playback with a reasonable high bitrate(i tested 8000 average, 9000 max) and generally have to resort to freezing on a frame to see how much there really is.

as for this little chestnut
2) DVD encoding can make praticaly lossless encoding
it all depends on the source and bitrate etc etc etc. with the right source and using a high bitrate etc an mpeg2(dvd) encode can be very hard to tell from the original source, especially when playing. of course its not lossless but if people can not tell the difference from the original to the encode playing without resorting to comparing file sizes or format or frame by frame analysis then its pretty damn good. :)

umm so what was the point of this thread exactly ???

[Audionut]

Encode the clip at constant quant 2, and see what the max bitrate is.

[john3voltas]

umm so what was the point of this thread exactly ???
None DG...
It's the same point that made him post this question at kvcd.net.
And since we're at it, would you watch his source using VDubMod?
Look how blooky the water scene is even in his source.
How are we supposed to find out an MPEG2 encoder that can treat the dancing blocks festival that appear in his source and make something that can be watched on TV?
The guy is just teasing us because he loves MPEG4 and derivates and it doesn't matter if you explain him crap-in-crap-out because he's so convinced he is right about anything that he won't "hear" anybody.
Cheers

[Amnon82]

The water scene is realy blooky in his source. It isn't even a lossless codec he used. It was DivX 503 with an 24867 kbps bitrate and 720 x 576 resolution in a OpenDML 2.0 AVI Container.

I never took his request seriously. I only encoded his sample with CCE and HC using almost the same settings.

If he wants to be taken seriously he should use "real" samples. Maybe he'll take a look at the Movie "Blue Crush (http://www.blue-crush.com/)".

[Sagittaire]

The water scene is realy blooky in his source. It isn't even a lossless codec he used. It was DivX 503 with an 24867 kbps bitrate and 720 x 576 resolution in a OpenDML 2.0 AVI Container.

Source is blocky ... lol
Try to find the 8*8 DCT block in source ... and make screenshoot

I use q1 MPEG4 ASP for lossless.avi. It's not really "mathematicaly" lossless but in practice it's lossless for eyes (MPEG4 ASP use only IFrame q1 for water high motion scene). But I can use real lossless codec if you want.

source MPEG4 q1 "lossless" at 30 Mbps
http://multimediacom.free.fr/Download/Source.jpg

MPEG2 encoding at 9 Mbps
http://multimediacom.free.fr/Download/MPEG2.jpg


And the challenge is simply to make good quality encoding with MPEG2 MP@ML DVD compliant stream ...



The guy is just teasing us because he loves MPEG4 and derivates and it doesn't matter if you explain him crap-in-crap-out because he's so convinced he is right about anything that he won't "hear" anybody.
Cheers
you should completely read the thread on KVCD Forum. It's not crap-in source. If you knew what you speaks about you would know it ... :rolleyes:


If he wants to be taken seriously he should use "real" samples. Maybe he'll take a look at the Movie "Blue Crush".
Source is real sample: http://www.apple.com/trailers/
Certainely the best available HD sources on the net ...

[Mug Funky]

question: is that blocking visible on a TV at a reasonable viewing distance? even my completely unfiltered encodes came out smooth on the TV. eyes just don't catch the blocking unless it's catastrophic, which this wasn't by any means.

[dragongodz]

Try to find the 8*8 DCT block in source ... and make screenshoot
i dont need to make a screenshot i just look at the 1 picture of source you posted and i can actually see some block edges. yes they are very faint and you do have to really look for them but they are there. also you should know 1 frame means absolutly nothing.
however the source is good enough to show how more blocks are introduced when encoding to dvd compliant mpeg2. as i said though i could have used a heap of other sources with the same properties to show the exact same thing. this is nothing new.

And the challenge is simply to make good quality encoding with MPEG2 MP@ML DVD compliant stream
please go reread your first post. you did not present it as something like "can anybody make a good mpeg2 encode of this source ?". your first post basically says "mpeg2 encoding is crap and i can prove it with this source".

eyes just don't catch the blocking unless it's catastrophic, which this wasn't by any means.
agree. infact will quote myself from earlier in this thread. :)
you should note though that you can not see most of the blocking with regular playback with a reasonable high bitrate(i tested 8000 average, 9000 max) and generally have to resort to freezing on a frame to see how much there really is.

[Sagittaire]

i dont need to make a screenshot i just look at the 1 picture of source you posted and i can actually see some block edges. yes they are very faint and you do have to really look for them but they are there. also you should know 1 frame means absolutly nothing.


It is not 8*8 DCT block simply because source is 1280*720 H264 resize to 720*576. There are perhabs little and very smooth block sometime but inloop work very well here and in this case there are absoluty no consequencies to DCT in MPEG2 encoding. There are incredible high level blocking in MPEG2 encoding simply because MPEG2 must use very high quantizer to respect vbv specification.


this is nothing new

Certainely for me or you but that does not seem to be the case for everyone.

[CruNcher]

@Sagittaire
why do you use this sample why don't do it with riverbed ?

[dragongodz]

There are perhabs little and very smooth block sometime but inloop work very well here and in this case there are absoluty no consequencies to DCT in MPEG2 encoding.
you are missing the point. you have encoded to 30mb/s with divx and yes it does have faint macroblocks. you are now challenging people to encode to a dvd compliant stream and not block more ? thats not reasonable or realistic.

There are incredible high level blocking in MPEG2 encoding simply because MPEG2 must use very high quantizer to respect vbv specification.
it is also because with such high speed motion and changing textures there is very few good motion hits that can be made to compensate the lack of bitrate.

[Pookie]

Here's the source file for your test:

Riverbed 1080p 250 Frames Riverbed seen through the water. Very hard to code.
Camera: Sony HDW-F900
Recorded on (Tape): HDCam
Stored on: DVS
Frame rate: 25 fps (progressive)
Resolution: 1920x1080
Color subsampling: 4:2:0
Filter Tabs for Subsampling: -0.0063 / 0 / 0.0299 / 0 / -0.0831 / 0 / 0.3098 / 0.4994 / 0.3098 / 0 / -0.0831 / 0 / 0.0299 / 0 / -0.0063
Color conversion: ITU Rec BT 709 (SMPTE 274M)
Restrictions of use: No restrictions

ftp://ftp.ldv.e-technik.tu-muenchen.de/pub/test_sequences/1080p/riverbed.yuv

Note: It's 741 MB, but there's no macroblocking...

[Sagittaire]

you are missing the point. you have encoded to 30mb/s with divx and yes it does have faint macroblocks. you are now challenging people to encode to a dvd compliant stream and not block more ? thats not reasonable or realistic.

1) q1 MPEG4 ASP (Iframe q1 for water) don't introduce blocking here. There are not 8*8 DCT block for my source because q1 MPEG4 ASP is in practice lossless for eyes (Perhaps more than 99 for SSIM and 60 dB for PSNR). But if you want real lossless encoding I can make that.

2) Smooth block are not really block because the source is 720p H264 with inloop strength at 0. Absolutly no consequencies for MPEG2 encoding. You can't compare this (IMO no blocking in practice but high bluring effect) with MPEG2 8*8 DCT block. My source is not perfect but perfect source is useless here.

http://multimediacom.free.fr/Download/source-zoom.jpg

http://multimediacom.free.fr/Download/MPEG2-zoom.jpg

As you can see on Screenshoot (I can make that praticaly for each frame in water source) the blocking for MPEG2 encoding are not introduce by the source but only by very high quant for each MPEG2 frame : more than q20 for each frame at 9 Mbps.

3) It's true : it's really easy to find very hard source for MPEG2. Another source here for example:
http://download.microsoft.com/download/e/a/d/eadb9b42-728b-42b0-bfdf-b472fa2a2464/Step_into_Liquid_720.exe

http://download.microsoft.com/download/9/7/9/97979408-9E08-4B85-9EFA-6E90897F4200/Dust_to_Glory_720.exe

[scharfis_brain]

@Sagittaire: Encode this scene with an IPPPPP... (GOP15) and you'll get much better results.

I tried it with Procoder 2. IBBPBBPBB... looked worse than without B-Frames.

[dragongodz]

ok i will have 1 last go at trying to understand what i mean before giving up.

My source is not perfect but perfect source is useless here.
and i already said it was fine to show the effects of high motion scenes such as this. so please dont react again as if i said it wasnt.

There are not 8*8 DCT block for my source because q1 MPEG4 ASP is in practice lossless for eyes
the blocking for MPEG2 encoding are not introduce by the source but only by very high quant for each MPEG2 frame
and you have several people here, including myself, who say we can see faint macroblocks in the source and that is the point. NOT that those macroblocks will make the mpeg2 encode block more or were the source of the blocking etc but that a 30mbps divx encode even has some faint macroblocks so ofcourse a sub 10mbps mpeg2 encode will have plenty of macroblocks.

Smooth block are not really block because the source is 720p H264 with inloop strength at 0.
not really blocks ? you must be kidding. the test file is encoded in divx NOT h264. at no point did i say the original h264(or whatever) had blocks, i never saw it. the file you made available to test with however does have blocks. it doesnt matter how faint they are if multiple people can see them they are real.

ok i give up on this. i think i have said what i wanted to pretty clearly. if anyone doesnt get what i mean then thats now their problem.

[Sagittaire]

Here sample for H264 720p source
http://jfl1974.free.fr/upload/Sample.mp4

Here encoding in LossLess huffYUV 720p@576p
http://jfl1974.free.fr/upload/LossLess.avi

Here encoding in MPEG4 ASP Q1 25 Mbps 720p@576p
http://jfl1974.free.fr/upload/MPEG4-Q1.avi

Here encoding in MPEG2 9 Mbps 720p@576p (no bframe like scharfis_brain)
http://jfl1974.free.fr/upload/MPEG2-Q100.mpg

1) As you can see it's very high quality source. Inloop work well for water scene. High bluring effect certainely because H264 use high quant for this extremely complexe scene. IMO Blocks are lacking by inloop and it's very difficult too see real and pure DCT block in H264 source.

2) If there are 8*8 DCT block in MPEG2 encoding it's not source problem simply because source is resized at 1280*720@720*576. No possible pertubations for 8*8 DCT MPEG2.

3) MPEG4 ASP Q1 encoding don't produce blocking because Q1 is extremely high quality. Moreover MPEG4 ASP use only Iframe for water scene. Compare LossLess.avi and MPEG4-Q1 and try to see visual differencies (good luck ... lol). IMO MPEG4 ASP Q1 is in practice "transparant" for eyes.

Source
http://jfl1974.free.fr/upload/Source-zoom.jpg

MPEG4 Quant 1
http://jfl1974.free.fr/upload/MPEG4-zoom.jpg

MPEG2 No Bframe 9 Mbps
http://jfl1974.free.fr/upload/MPEG2-zoom.jpg

4) Really good idea from scharfis_brain
IPPPP produce very better level blocking. It's is not very good but it's very better without bframe.
But In practice all the MPEG2 encoder use no adaptative bframe for this sequence ... ???


not really blocks ? you must be kidding. the test file is encoded in divx NOT h264. at no point did i say the original h264(or whatever) had blocks, i never saw it. the file you made available to test with however does have blocks. it doesnt matter how faint they are if multiple people can see them they are real.

hmm ... I can find block in SWIII MPEG2 DVD encoding, Matrix DVD or LOTR DVD but if my XviD 1CDR encoding is not good I would not say "It's a DVD problem, source is bad ... !!?". My source is not perfect but it's not important here.

[Audionut]

Mpeg2 is not the only codec that will fail to avoid blocking on this clip at 9Mbps.

Your Mp4 25Mbps encode aint perfect. Side by side comparision in photoshop, and I can see there clearly different clips.

Yes, the Mp4 clip is visualy lossless, especially when veiwing the clip in real time.

But so is Mpeg2 at 25Mbps.

So, in conclusion, your opening post should reflect, "All encoders suck at 9Mbps with this clip".

[freelock7]

Very interesting and...disturbing!
All Mpeg2 encoders I used (QuENC, Tmpgenc) failed (*).
Only Xvid with AS@L5 profile can do the job but it's not....MPEG2 (and not compliant for Divx DVD reader).
Yes, we can see here the great MPEG4 AVC potential and prepare us to meet some difficulties to encode in MPEG2 this kind of source!
______________________________
(*)Setting a max bitrate=16000 with mainconcept (Mpeg2 profile) reduce the quantization and encode correctly but it is not what we need! ;)

[Sagittaire]

Mpeg2 is not the only codec that will fail to avoid blocking on this clip at 9Mbps.

Your Mp4 25Mbps encode aint perfect. Side by side comparision in photoshop, and I can see there clearly different clips.

Yes, the Mp4 clip is visualy lossless, especially when veiwing the clip in real time.

But so is Mpeg2 at 25Mbps.

So, in conclusion, your opening post should reflect, "All encoders suck at 9Mbps with this clip".

It's not MPEG2 vs MPEG4 comparison (IMO there are no possible comparison between H264 and MPEG2) but I try to make the best possible encoding in MPEG2 MP@ML with this source.

No bframe (adaptative bframe) in very high motion part seem to be a good way ...


So, in conclusion, your opening post should reflect, "All encoders suck at 9Mbps with this clip

It is not codec problem : unresticted MPEG2 is able to encode correctly this scene. MPEG2 Q1 like MPEG4 Q1 produce extremely high quality "lossless" for eyes ...

Proleme here is a vbv saturation to respect the compliance for DVD MPEG2 profil.

[freelock7]

MPEG2 Q1 like MPEG4 Q1 produce extremely high quality "lossless" for eyes ...
So no issue with the standard DVD video profile...even without B frame.

[Mug Funky]

just for a little fun i decided to see how the blocks could be reduced with some (very slow) spatial filtering applied according to SAD scores with mvtools. anything with a high enough SAD gets a higher amount of fft3dfilter (this acts in a similar way to DCT quantization except it doesn't block and is not written with lossless coding in mind, so is probably not optimal. also a sigma that varies with SAD would be better).

conclusion - it can be made to look good with barely any visible blocking, but the clip will suffer in other ways (of course). i also conclude that though 9.8mbps may not be enough for critical viewing on a computer monitor, it is ample for a CRT TV which is after all what most DVDs are watched on.

btw, FWIW the source is okay for a test of this kind - even with heavier source blocking it's still quite a test for an mpeg-2 encoder (it's just much harder to judge the results :)).

it's also logical that a source of this kind will suffer with any motion + texture based codec. where motion compensation ceases to work, and there's not enough bandwidth to store adequate texture, the encoder will fail to achieve transparency. this is obvious, really.

[freelock7]

it can be made to look good with barely any visible blocking, but the clip will suffer in other ways
Yes, I tried the CCE 270 internal filters (low and high filter) which do the job too.
But it's not a"complete" solution.

[freelock7]

The only acceptable solution is to use the MPEG2 High profile which could be used by Sony according to this rumor:
"Sony Pictures has stated it will release all movies on Blu-ray using the old MPEG-2 standard--the same used for current DVDs.
The news comes as a surprise, since Blu-ray supports MPEG-4 AVC/H.264, which allows much better compression of video while producing better quality video. Blu-ray supporter Apple has also added H.264 support into its operating system.
According to Don Ecklund, Sony Pictures senior VP of advanced technologies, "Advanced formats don't necessarily improve picture quality ... Our goal is to present the best picture quality for Blu-ray. Right now, and for the foreseeable future, that's with MPEG-2." -Geek.com

[Mug Funky]

considering on a dual layer bluray you could fit an 80 minute SDTV movie in huffyuv, i don't see why mpeg-2 is supposed to be so bad.

remember h.264 is mainly intended for low bitrate, low noise sources. as bitrate increases you get diminishing returns from such a complex encoder

it could well be that at HDTV sizes with the kind of bitrates bluray will allow, MPEG-2 is in fact the winner. we have to see it first... now i'm as sceptical as the next person about sony's research (like how good they say ATRAC and it's derivatives are), but this decision can't have been made lightly.

when you consider the amount of work that doom9 and others put into codec comparisons, imagine how much testing the guys who choose the standards would do. now if it were microsoft doing this, it would probably be a different story, or if sony had their own compression format for video and magically chose that as the winner. but they chose mpeg-2, and there must be a good reason for it.

btw, does anyone know what the max bitrate is going to be for bluray? or at least a ballpark figure based on 1x bluray speed? hell, i don't even know what audio they're going to use (there'll be enough room for any format to sound good though, even DTS).

[freelock7]

does anyone know what the max bitrate is going to be for bluray?
Max bitrate= 40Mb/sec
http://www.blu-raydisc.com/assets/downloadablefile/2b_bdrom_audiovisualapplication_0305-12955.pdf
Audio format could be the DOLBY TRUE HD.
http://www.dolby.com/consumer/technology/trueHD.html

[Sagittaire]

I make actually Big Codec test:

MPEG2 vs H264

Very High Quant (low bitrate)
~q8 MPEG4 ASP for reference (quality/bitrate)
Ratio higher than 2.0 between MPEG2 and H264. I use MPEG2 MP@Unresticted Level for best possible comparison

Medium Quant (medium bitrate)
~q4 MPEG4 ASP for reference (quality/bitrate)
Ratio higher than 1.5 between MPEG2 and H264. I use always MPEG2 MP@Unresticted for best possible comparison

Low Quant (high bitrate)
~q2 MPEG4 ASP for reference (quality/bitrate)
Ratio higher than 1.2 between MPEG2 and H264. I use always MPEG2 MP@Unresticted for best possible comparison


More and more bitrate and difference between MPEG2 and H264 is less and less important. For Very High Bitrate (q2 encoding certainely like HD-DVD or BR encoding) MPEG2 seem always better than MPEG4 ASP in metric test. MPEG2 for very high bitrate is definitively a good choice.

[Mug Funky]

yeah, i agree on mpeg-2 at high bitrates. it's not deathly afraid of noise like a lot of ASP/AVC implemetations are (though i see no reason that it should be a fault of the spec).

[Teegedeck]

I'm rather taken lately with how x264 handles noise at low deblocking values like -1/-3. Not nearly as faithful as XviD with the SixOfNine CQM but still quite well and very efficient indeed. I should be surprised if development stops there... And how nice of Sony to use MPEG2 as a codec for BluRay - I should be very surprised if we'll have any problem whatsoever backing up the contents to DVD-R DL with AVC and AAC... ;) Well, if Sony had chosen a more modern codec they'd have run into trouble selling the stuff at all, wouldn't they? Who'd see the logic in buying expensive BluRay stuff if the discs were 8/10 empty? :D

[Sagittaire]

Well, if Sony had chosen a more modern codec they'd have run into trouble selling the stuff at all, wouldn't they? Who'd see the logic in buying expensive BluRay stuff if the discs were 8/10 empty?

In fact MPEG2 Main Profil Unrestriced Level (no restriction for GOP, VBV ...) seem produce the same or better quality than MPEG4 ASP q2 with equivalent bitrate. For example MPEG2 1280*720 6 Mbps (~q4 encoding) produce same or better quality than MPEG4 ASP 1280*720 6 Mbps (~q2 encoding). High efficiency codec are IMO useless for HDDVD or BR.

BluRay capacity is very high : too high for advanced codec like MPEG4 AVC and certainely too high for old codec like MPEG2 ... lol