GMC: observations... and now discussions!

[SeeMoreDigital]

Yesterday, a fellow Doom9 forum member going by the name of Skuto, made (what I think is) an interesting GMC observation, when playing back NeroDigital encodes via his NeroDigital certified stand-alone player - the Siemssen SCO 5000!

As some of you know NeroDigital generates Mpeg4 encodes with 3 warp-point GMC (the same as XviD) and as far as we know, there are currently no players supporting this implementation... Not even NeroDigital certified stand-alone players.

Anyway, when Skuto fed the player with such an encode, it played :eek: So during a moment of experimental madness I decided to see whether my Sigma Xcard would play such encodes too - And would you believe it, it did :)

So, this begs the question. How many other stand-alones can play NeroDigital encodes with 3 warp-point GMC?

Obviously, you'll need to de-mux Nero's Mpeg4 stream from out of the the .MP4 container to .AVI (and maybe even rename the 4CC code to XviD or NDIG). And, if you want audio, you'll have to convert Nero's AAC stream to MP3. But remember... it's just an experiment.


Now for all those who don't believe that Nero digital streams contain 3WP GMC. Here's what Mpeg4 Modifier had to say about it: -

http://img66.exs.cx/img66/8348/nd3wpgmc1wj.gif

And here is what Mpeg4 Modifier said about XviD streams encoded with 3WP GMC (which incidentally do not play with my Xcard): -

http://img66.exs.cx/img66/3618/xvid3wpgmc8kb.gif


Could XviD's GMC implementation be broken. Could NeroDigital's. What can anybody make of this?


Cheers

[Manao]

GMC in Nero's MPEG-4 is 3-Warppoints.

However, the player will play it correctly till GMC is indeed used, which may not often happen. Did you check with FFDShow that GMC was indeed used on at least a frame in the clip ?

[SeeMoreDigital]

Originally posted by Manao
Did you check with FFDShow that GMC was indeed used on at least a frame in the clip ? I'm curious, what relevance would this have for a "hardware player" user?

And how can XviD's 3WP implementation work so differently to NeroDigital's?


Cheers

[Doom9]

I'm curious, what relevance would this have for a "hardware player" user?GMC is not used throughout the movie, only on specific frames where it makes sense. I don't know how the GMC and warp point signalling works, but I suppose it could be signalled without the feature actually being used. And if GMC is not actually used in a clip, the clip should play on any device that cannot handle 3pt GMC since it never has to decode such frames.

[SeeMoreDigital]

Thanks Doom9,

This raise another question. If GMC does work in this way, then "might" it be possible to have a tool that can switch off or remove the GMC signalling all together?

Such an implementation might prove handy for people who've generated Mpeg4 encodes with (3WP) GMC but who have stand-alones that can't play them!


Cheers

[thana]

as i understand it, frames where GMC is used are called "S-VOPS". so if you trust MPEG4-Modifier, the get used a lot in both clips from SMD (47% and 19% of all frames). so if they can be played correctly (with no quality issues) it looks like 3WP-GMC is indeed supported.

@SMD: did you try to change the FourCC and the userdata string of the xvid clip to the same as the nero-digital clip?

[SeeMoreDigital]

Originally posted by thana
SMD: did you try to change the FourCC and the userdata string of the xvid clip to the same as the nero-digital clip? As a matter of fact I did... There was no change!

I'm glad you picked up on the fact that MPEG Modifier does indeed show more S-VOP's in the NeroDigital clip than in the XviD clip.


Cheers

[celtic_druid]

Could be because XviD doesn't use GMC if only 1 reference is found and Nero does. No idea if that is the case but it would explain why there are more S-VOP's.

For instance my MTK based player from what I remember plays GMC'ed XviD files fine until it (presumably) hits a 3 warp point S-VOP where decoding gets "stuck". Not sure what effect there is with 2 warp point S-VOP's.

[CruNcher]

jep what celtic_druid says is correct at least for the MediaTek Chip and Firmware it will bork on 3 Warppoint GMC frames (S-Vops) often they are used for a long duration, you'll have a slideshow then :P

[SeeMoreDigital]

Originally posted by CruNcher
jep what celtic_druid says is correct at least for the MediaTek Chip and Firmware it will bork on 3 Warppoint GMC frames (S-Vops) often they are used for a long duration, you'll have a slideshow then :P Yes... but have you or celtic_druid tried an "NeroDigital" Mpeg4 encode with 3 warp-point GMC?

EDIT: Like this one (http://homepage.ntlworld.com/seemoredigital/ND_with_3WP_GMC.zip)!


Cheers

[Sergei_Esenin]

But can you tell whether it's really using 3 warp points anywhere, or whether the stream reports 3 warp points while only 1 or 2 are actually used? I recall reading that NeroDigital only implemented 2 warp points, though that was quite a while ago.

[babayaga]

Originally posted by Sergei_Esenin
I recall reading that NeroDigital only implemented 2 warp points, though that was quite a while ago.
The NeroDigital ASP encoder has always encoded 3-warping points.

There is an optimization reducing the number of warpoing points if some are not necessary.

[SeeMoreDigital]

Anybody had chance to try my "NeroDigital" Mpeg4 3WP GMC .AVI encode, on their stand-alone player yet?

If not, it's still here (http://homepage.ntlworld.com/seemoredigital/ND_with_3WP_GMC.zip).


Cheers

[Teegedeck]

If it only was possible to find out how many warppoints actually are used in the bitstream! Anyone?

[niamh]

I've just tried SMD, and my standalone doesn't want to play it at all, so it's not much help to you :)

[SeeMoreDigital]

Originally posted by niamh
I've just tried SMD, and my standalone doesn't want to play it at all, so it's not much help to you :) What make/model of stand-alone do you have?


Cheers

[niamh]

It´s a non-brand... anyway, silly me forgot to change the 4cc to xvid :D.
It now plays flawlessly, motionwise.( Xvid gmc gives me the usual slideshow.)

[Laffer]

Originally posted by SeeMoreDigital
Thanks Doom9,

This raise another question. If GMC does work in this way, then "might" it be possible to have a tool that can switch off or remove the GMC signalling all together?

Cheers

That's what I am after a tool for removal. In the meatime can anyone point me to an FAQ or tutorial on GMC removal through re-encoding? For some reason my re-ecodes always end up 33% bigger than the source when I remove GMC?! Also whats the best tool for this? (only tried virtualdubmod)

Thanks,

Laffer.

[bond]

ok bobololo described what happened in the observation SMD described (3wp gmc playable on a player not handling 3wp gmc) here:
http://forum.doom9.org/showthread.php?s=&threadid=89861&perpage=20&pagenumber=2

to make it short: the clip only signals 3wps, but doesnt use them

[SeeMoreDigital]

Originally posted by bond
ok bobololo described what happened in the observation SMD described (3wp gmc playable on a player not handling 3wp gmc) here:
http://forum.doom9.org/showthread.php?s=&threadid=89861&perpage=20&pagenumber=2

to make it short: the clip only signals 3wps, but doesnt use them Jeez... me and my brain... I forgot I had started this thread :eek:

Yes, bobololo post (http://forum.doom9.org/showthread.php?s=&postid=610113#post610113) revealed some very interesting information about NeroDigital's approach toward their 3WP GMC implementation.

I wonder if the XviD guys could provide information about their approach too?


Cheers

[skal]

Originally posted by SeeMoreDigital

I wonder if the XviD guys could provide information about their approach too?


SMD: so far i recall, XviD will always use 3 warp points *if* it uses GMC at all. And rely on the internal GMC code to simplify these 3 warp points in case it's applicable.
Also: XviD never uses 1 warp point for S-VOP. Instead, if only 1 point is needed (pure translational motion), a P-VOP is used instead.

Hope i'm not mistaking...

-Skal

[SeeMoreDigital]

I conducted some more short GMC tests with XviD a few hours ago. With and without mattes. And sufficed to say, when the mattes are cropped away the implementation is much improved.

In fact, I would go as far to say that when the mattes are kept, an encode with GMC looks worse than an encode without GMC.

Also when I conducted the same experiment with the mattes cropped away, the encode with GMC looked slightly better than the encode without GMC.

All the above encodes were generated to identical file sizes using a "square pixel" frame size of 720x304 to (near as damn) match the 2.35:1 source.

Can anybody else confirm these findings? Or is this already an expected GMC fact?


Cheers

[Sharktooth]

i think the reason is mattes "disturb" the global motion detection.

[SeeMoreDigital]

Originally posted by Sharktooth
i think the reason is mattes "disturb" the global motion detection. If this is indeed the case, maybe the XviD guys (and DivX) should follow NeroDigital's example and automatically disable GMC if the encode contains mattes!

Who knows, maybe GMC's worse enemy has been the lack of definitive information about when it should... or should not be used... And maybe the same might be true when using Qpel :eek:

The XviD and DivX guides are - all we and good. But maybe a definitive "Mpeg4 - Do and Don't Guide" would be more useful.


Cheers

[Didée]

After thinking a second about what GMC actually does, it gets perfectly clear that in case mattes are present, GMC gets fully useless (in the best case), or even gets contraproductive, in worst case.
(I've pointed that out in the past, not only once.)

[akupenguin]

Disclaimer: I just read the relevant XviD source, and don't claim to fully grok it.

AFAIK, GMC isn't completely useless with mattes, but the encoder has to know to discard the mattes during the GMC calculation. (At which point, you might as well completely crop off the mattes. After all, if you give the encoder non-mod16 resolutions, it can put whatever it wants in the padding, including black if that were an efficient way to encode it.)
XviD discards the outer ring of MBs for GMC (always), but if your mattes are wider than 16 pixels, that's not sufficient. XviD also discards blocks with outlying MVs, but mattes may contribute enough zero MVs to not be considered outliers.

[SeeMoreDigital]

Given that MPEG-4 as we know it today is around three years old, surely the issues of when or when not to use it, and how to use it, should have been resolved ages ago!

So my interpretation of what have we learnt so far, is as follows : - 3WP GMC really does work. But...
Ideally "all" mattes (black bars) should be cropped away. Otherwise its use becomes counter productive. Although very simplified, would this be a fair assessment?


Cheers

[Manao]

Anyway, why bother with mattes ? They take space, they hurt compression, they create ringing if they aren't mod8, they fool the motion compensation, they fool the GMC...

[mp34ever]

I have a portable dvd player (Shinco)
So far i have tested these formats, and they work

xvid with gmc
nero with gmc (fourcc=Divx)

people say there is know player capable of playing gmc encodes "hello" mine playes it easy with my xvid encodes.

Just got to make sure you disable packed bitstream for smooth playbak

[SeeMoreDigital]

Originally posted by mp34ever
people say there is know player capable of playing gmc encodes "hello" mine playes it easy with my xvid encodes.

Just got to make sure you disable packed bitstream for smooth playbak If you are disabling "packed bit-stream", it sounds like you are generating encodes with b-frames (B-VOP) as well as GMC (S-VOP)...

Can you confirm?


Cheers

[bond]

Originally posted by mp34ever
I have a portable dvd player (Shinco)what chip does it use?

[celtic_druid]

Sounds like MTK of some sort.

[mp34ever]

yes i can confirm i thought gmc was no big deal cuz i played a xvid gmc file first day i got it.I tested it out but it was playing jerky
and it took me a while to figure out that packed bitsream was causing it on my port.

i don't know what chip it has ?

[SeeMoreDigital]

Originally posted by mp34ever
...and it took me a while to figure out that packed bitsream was causing it on my port. Can you confirm whether or not your Mpeg4 encodes contain B-VOP as well as GMC?


Cheers

[Skuto]

Originally posted by SeeMoreDigital
Anyway, when Skuto fed the player with such an encode, it played :eek:
Cheers

One of them did, yes. But it was a short clip and apparently it just didn't have any GMC frames. I did a test with a longer clip lather and that didn't work quite so well.

[mp34ever]

Sorry it took so long @ seemoredigital for some reason doom9 wasn't letting me post i could log in but not post i contacted admin and replied to you personally @ seemoredigital I have tried to reply about 15 times 2 days ago but looks like it is fixed now anyway

Anyway

Yes it did play fine and did have B-VOPs i think mostly had two some had one though.

[bond]

latest ffdshow now seems to be able to detect the warppoints of the gmc in a mpeg-4 stream :)

it shows two things:
- the first value seems to show the number of warppoints set in the VOL header
- the second ("used") seems to show the actual number thats really used

i tested this on the nero 3wp streams (kindly created and provided by seemoredigital here (http://homepage.ntlworld.com/seemoredigital/SMD_ND_3wp.zip) :) ) and it seems to detect correctly under used that it only uses 1wp (caused by the black borders)

another interesting thing i saw is that divx5 streams signal 2wp in the VOL (but actually only use 1wp) - might be a bug in divx!?
i verified this with the mpeg4vol tool from mpeg4ip which shows the same

[SeeMoreDigital]

Originally posted by bond
...i tested this on the nero 3wp streams (kindly created and provided by seemoredigital here (http://homepage.ntlworld.com/seemoredigital/SMD_ND_3wp.zip) :) ) and it seems to detect correctly under used that it only uses 1wp (caused by the black borders) Sorry bond, I actually changed the links to both NeroDigital 3WP GMC files :eek:

The NeroDigital encode "with mattes" can now be found here (http://homepage.ntlworld.com/seemoredigital/SMD_ND_3wp_(with_mattes).zip).

And the NeroDigital encode "without mattes" can now be found here (http://homepage.ntlworld.com/seemoredigital/SMD_ND_3wp_(without_mattes).zip).


Cheers

[bond]

thanks a lot :)

btw i now tested another clip (without black borders) encoded with nero and it shows also 1wps sometimes, but also mixed with 3wps.
in the beginning under "used" there is written that one warppoint gets used, after some time it changes to 3wp, than back to 1wp aso...

so it seems nero can also only use 1wp when the black borders are cropped away correctly

xvid always uses 3wp right from the beginning

[Manao]

Perhaps XviD has something in its code saying : "if only one WP, do not use GMC at all on this frame", since XviD devs are convinced of the uselessness of GMC 1-point.

[SeeMoreDigital]

Originally posted by bond
...btw i now tested another clip (without black borders) encoded with nero and it shows also 1wps sometimes, but also mixed with 3wps.
in the beginning under "used" there is written that one warppoint gets used, after some time it changes to 3wp, than back to 1wp aso...

xvid always uses 3wp right from the beginning Well I suppose it had to happen!

It looks like we now have "variable warp-point GMC". What ever will Mpeg4/ASP developers think of next?


Cheers

[Manao]

It isn't variable WP GMC at all.

A 3WP GMC transformation whose second and third WP are 0 will be reported by FFDShow as a 1WP GMC. But from the encoder point of view, it's still being computed as if it was a 3WP GMC ( which is that case happens to be only a translation, hence codable with 1 WP ).

[bond]

Originally posted by Manao
A 3WP GMC transformation whose second and third WP are 0 will be reported by FFDShow as a 1WP GMCin a stream encoded with a 1wp-only encoder (eg divx5), will the second and third wp also be 0?

[Manao]

It shouldn't be : the WP count is written in the VOL header ( one per video ), so if you only support 1-WP GMC, there's no need to write that the WP count is 3.

On the other hand, nothing prevents you from doing so...

[bond]

ok so whats the difference from the decoding side (!) between 1wp gmc from divx5 and 3wp gmc (with 2nd and 3rd wp being 0) from nero?

[Manao]

If the decoder is properly implemented ( decode all the WP, but take only the first into account ), it won't differ. However, the decoder can be implemented in a way that it crashes on streams with 3 WP, even with the two others null, if it considers that there must be no more than 1 WP ( hence the next two WP will offset the bitstream, which won't be decodable anymore ).

[Moitah]

The next version of MPEG4 Modifier will be able to give more detailed stats on warp points, thanks to Kurosu for sending a patch and motivating me :):
Packed bitstream: Yes
QPel: Yes
GMC: Yes (3 warp points max)
Interlaced: No
Aspect ratio: Square pixels
Quant type: H.263

I-VOPs: 3 (0.42%)
P-VOPs: 56 (7.87%)
B-VOPs: 461 (64.75%)
S-VOPs: 192 (26.97%)
N-VOPs: 0 (0.00%)

Max consecutive B-VOPs: 2
1 consec: 13.36%
2 consec: 86.64%

Warp points:
2: 19.27%
3: 80.73%
That's from a XviD encode. Does it make sense if I call it 2 warp point when the 3rd point has both values set to 0?

Also, for example, if the 2nd warp point has both values 0, but the 3rd is in use, I call this 3 WP because it wouldn't be possible to code without using 3 WPs, correct?

[Manao]

The encoder didn't try to make the distinction between 2WP and 3WP, so I wouldn't give per #WP statistics.

If 3 WP is written in the VOL header and 3WP are never used, i would however inform the user.

Edit :

Yes, if the third WP isn't null, it's definitely 3 WP.

[Moitah]

Originally posted by Manao
The encoder didn't try to make the distinction between 2WP and 3WP, so I wouldn't give per #WP statistics.
True. But is it correct that the VOPs where the 3rd WP is null would be equivalent if they were coded with just the first 2 WPs? I see what you're saying, I'll have to think about what I want the stats to show.

[SeeMoreDigital]

Originally posted by Moitah
The next version of MPEG4 Modifier will be able to give more detailed stats on warp points... Cool :cool:

Have you investigated whether or not XviD's and DivX's GMC offers different warp-point percentages when the black mattes have been cropped away?

Earlier today I generated two DivX (Build 503b1461p) encodes with GMC. One with mattes, the other without mattes. Both were identified as containing "2" warp-points: -

http://img147.exs.cx/img147/4118/divxgmc7dv.png

Both however played perfectly in hardware (ie: Sigma Xcard and Pioneer DV-575). Which are only designed to play encodes containing 1No warp-point


Cheers