Moonlight AVC/H.264 encoder released (OneClick Compressor 1.1)

[MuTeK]

http://www.moonlight.co.il/cons_oneclick_avc.php

[slavickas]

too fast :D dvb (mtv.de) to 384x288 avc @ 700k + 160kbps mp2 ~1.4 realtime on 1.913ghz AthlonXP,

btw what profile and features it supports?

edit:
seems mpg file i unplayable with vlc 0.8.1 :(
btw2: any chances of version with more options to configure

[MuTeK]

Originally posted by slavickas
btw what profile and features it supports?


MainProfile, 2Bframes, 2Refer, CABAC, RDO, Search - 64, Quarter pels, Inloop filter.

[temporance]

Why does no-one make a VfW codec for AVC??? Arghhhhh!

[bobololo]

Originally posted by temporance
Why does no-one make a VfW codec for AVC??? Arghhhhh!

Maybe because VfW doesn't provide suffisent support for proper operation without dealing with messing hack & tricks ? I'm thinking about bframes, dbp operation (cts/dts) handling, etc ...

[temporance]

Originally posted by bobololo
Maybe because VfW doesn't provide suffisent support for proper operation without dealing with messing hack & tricks ? I'm thinking about bframes, dbp operation (cts/dts) handling, etc ... That'a a matter of opinion. The problems you mention are all soluble with elegant engineering design (although AVI-phobes would still call it a hack). I won't go into details here as it's not appropriate in this thread or even this forum.

My reply was at least partly tongue-in-cheek. I'm just frustrated that more companies don't release a VfW codec that would hit a wider audience. Heck, even Microsoft did this with WMV9!

In any case, it's interesting to see companies jumping onto the H.264 bandwagon with closed encoding apps. Good luck to them :)

[SeeMoreDigital]

Originally posted by slavickas
...seems mpg file i unplayable with vlc 0.8.1 :( What a shame Moonlight are still outputting to this container!

Obviously .MP4 would have provided a more complete solution. Failing this, .AVI would have made a better compromise!

Personally speaking, I can't see myself testing it :(


Cheers

[slavickas]

Hi,
i have tried encode within graphedit (there i at least can change lottta options), but only with dump output into raw h264, which filter should i use for muxing into mpg?

2 more q:
what "adaptive quantization" in your encoder do?
are keyframes inserted at scene change, or is it fixed range?

[unixfs]

Originally posted by SeeMoreDigital
What a shame Moonlight are still outputting to this container!

Obviously .MP4 would have provided a more complete solution. Failing this, .AVI would have made a better compromise!

Personally speaking, I can't see myself testing it :(


Cheers

why? what more does mov/mp4 have to offer respect to mpg?
At least is the files is broken it's easier to read

[SeeMoreDigital]

Originally posted by unixfs
why? what more does mov/mp4 have to offer respect to mpg?
At least is the files is broken it's easier to read MP4 is the official/generic container for Mpeg4/AVC.

It can be argued that muxing Mpeg4/AVC or even Mpeg4/SP/ASP into any other type of container is hack ;)

We finally have developers creating working AVC codecs... So why confuse users by muxing such streams into loads of different containers!


Cheers

[unixfs]

Even muxing mpeg4 asp and avc in mpeg2-[pt]s was officially standardized
by ISO (IS-13818-2 AMD 7).
There's a very good reason to stick with "old" file formats:
years of experience and plenty of tools available.

Using mpeg2 format with its full set of features (including
PSM and PSD) is enough o store simple audio, video and subs,
although there are many aspects of it that suck (and the implementation
used by dvd-video sucks even more).

I really hope it will be used even for HD-DVD and Bluray

[SeeMoreDigital]

Originally posted by unixfs
I really hope it will be used even for HD-DVD and Bluray Yep, the prospect of having high-def Mpeg4/AVC on HD-DVD is indeed a tasty one!

... But does anybody know for sure, what container will be adopted?

One thing is for sure. High-def DVD's will have far more secure encryption protocols than current std-def DVD, so while the technology is very exciting, I doubt very much that we'll be able to back them up using our PC's!


Cheers

[temporance]

Originally posted by SeeMoreDigital
MP4 is the official/generic container for Mpeg4/AVC.

It can be argued that muxing Mpeg4/AVC or even Mpeg4/SP/ASP into any other type of container is hack ;)

We finally have developers creating working AVC codecs... So why confuse users by muxing such streams into loads of different containers!Hello SMD,

The people behind MPEG have deliberately designed their technology to be modular. They do not insist that you use the mp4 file format with MPEG-4 video/audio, in fact they do not require that you use a container at all! In fact if you write AVC bitstream onto a slice of cheese using a toothpick, it would be a reasonable use of MPEG technology.

If other containers, or transport layers, work better for particular applications then they are a good and valid solution. We are not talking about patent-encumbered technology here, in the whole: things like AVI and MPEG-2 TS are pretty benign in that respect.

[SeeMoreDigital]

Originally posted by temporance
The people behind MPEG have deliberately designed their technology to be modular. They do not insist that you use the mp4 file format with MPEG-4 video/audio.... Okay... before this thread turns into an yet another "he said, she said" type of threads... I would ask people to please find out the answer this question...

"What is official agreed container for Mpeg4/AVC?"


Cheers

[Doom9]

MP4 is the official/generic container for Mpeg4/AVC.

It can be argued that muxing Mpeg4/AVC or even Mpeg4/SP/ASP into any other type of container is hack Not correct. For starters, AVC is also supported as a broadcast format, and for broadcast we have TS. Moonlight does provide a TS muxer that comes with their tools. It's not like TS is a good option for end users, but it's a very valid choice.

And I presume with the advent of HD DVD and Blu-Ray we'll see another container similar to VOB - something that contains not only video and audio but also subs, menus and such.

[bond]

did anyone already conduce some tests regarding quality and speed?
it is simply a directshow encoder (funny that more and more companies recognize it as interface for encoding)

(note that lupus from moonlight said that he is currently working on a directshow .mp4 muxer, so we might see .mp4 support for it soon)

[SeeMoreDigital]

Nobody is denying that when broadcasting Mpeg4/AAC, different containers can be adopted. The same goes for Mpeg4/AVC as a high-def disc based format!

It's true to say, when Mpeg4/AVC makes it into a "broadcast" or "HD disc based" format, it's anybodies guess what type of container will be adopted.


Cheers

[Doom9]

I found (though that was present in prerelease software as well) a major AR bug. Starting with the 2nd test release I got all files written have the proper AR (1:1) when played in graphedit or WMP6.4 in windowed mode. I can stretch the window and things are still fine, until you go fullscreen, where the video is effectively a 1:1 rectangle. And in MPC and WMP the AR is always off by the same amount. I'm afraid with a completely screwed up AR I never did so much as watch a single movie I encoded, and since I encoded in the background, I didn't bother measuring speed.

[unixfs]

the speed seems to be good (roughly 20fps on my p4 2.4) and the format
used to store the video is mpeg-ts.
VLC crashes because it uses ffh264 to decode avc, but this codec
isn't fully working yet.

[stephanV]

Originally posted by SeeMoreDigital
We finally have developers creating working AVC codecs... So why confuse users by muxing such streams into loads of different containers!

I, as a user, do not want to be limited to use AVC only in MP4. In fact, that would be a serious consideration for me not to use AVC. The "confuse the user"-argument is so lame, that it is not even worth arguing about.

It can be argued that muxing Mpeg4/AVC or even Mpeg4/SP/ASP into any other type of container is hack
No it cannot. I think it is nowhere specified that AVC should be used in MP4. AVC in a container that is fully equiped to support it, is not to be considered a hack.

[MuTeK]

Originally posted by unixfs
the speed seems to be good (roughly 20fps on my p4 2.4) and the format
used to store the video is mpeg-ts.
VLC crashes because it uses ffh264 to decode avc, but this codec
isn't fully working yet.

It's not really fast, we have improved version that operates faster than real time on my computer (AMD Opteron 144 – 1.8Ghz) by encoding 720x480.

[MuTeK]

Originally posted by Doom9
I found (though that was present in prerelease software as well) a major AR bug. Starting with the 2nd test release I got all files written have the proper AR (1:1) when played in graphedit or WMP6.4 in windowed mode. I can stretch the window and things are still fine, until you go fullscreen, where the video is effectively a 1:1 rectangle. And in MPC and WMP the AR is always off by the same amount. I'm afraid with a completely screwed up AR I never did so much as watch a single movie I encoded, and since I encoded in the background, I didn't bother measuring speed.

Well, unfortunately I have to say that the problem realy exists...
The Demuxer was the source of this problem but as far as I know the problem has been fixed.
We are testing the new Demuxer version now.

ps I've sent you the fixed version of demuxer.

[unixfs]

will Moonlight add support for 2 pass encoding?

[SeeMoreDigital]

Originally posted by stephanV
I, as a user, do not want to be limited to use AVC only in MP4. In fact, that would be a serious consideration for me not to use AVC. The "confuse the user"-argument is so lame, that it is not even worth arguing about.Well that's alright.... For people like you, you can always de-mux the streams out of the MP4 container and re-mux them into a different container!

Which is exactly what us MP4 users have had to put up with... until Recode2 came along ;)


Cheers

[stephanV]

But thats no reason to make comments like this:

What a shame Moonlight are still outputting to this container!

It's their choice, and they probably have good reasons for doing so. There is no "shame" in putting H264 in mpg.

[Doom9]

ps I've sent you the fixed version of demuxer.Unfortunately, things remain unsolved. I'll try on a 2nd pc now just to make sure.

will Moonlight add support for 2 pass encoding?I've been told this is a low priority feature.

[bond]

i now had the time to analyse moonlight's avc encoder output more closely and it supports:

by default in oneclick compressor the following settings are used:
- 2 b-frames in a row (not adaptive)
- adaptive quantisation
- 2 ref frames signalled (as 1 is caused by b-frames propably no mref enabled)
- no weighted pred
- cabac
- 8x8 blocksize for p-frames (propably also 4x4)
- 16x16 blocksize for b-frames (propably also 8x8)
- no loop
- no pixel aspect ratio
- it outputs to mpeg-2 Transport streams

apart from that it offers (oneclick uses a dshow based encoder, thx cruncher!):
- Constant Quant, VBR and CBR (no 2pass)
- scenechange detection
- Loop
- Frame Stamps (?)
- RD Lossy and Lossless
- Search Range: Medium and Large
- Subq: Fullpel, Halfpel, and Qpel
- Aspect Ratio
- Interlaced encoding
- Number of slices per picture

[Sergey A. Sablin]

Just a note for options:

Originally posted by bond
i now had the time to analyse moonlight's avc encoder output more closely and it supports:

by default in oneclick compressor the following settings are used:
[skip]
- 8x8 blocksize for p-frames (propably also 4x4)
- 16x16 blocksize for b-frames (propably also 8x8)
- no loop
- no par
- data partitioning
- it outputs to mpeg-2 Transport streams


First of all I don't understand what do you mean under "no par" at all. Data partitioning is not supported and loop filter is disabled by my fault... sorry

Originally posted by bond
apart from that it offers (oneclick uses a dshow based encoder, thx cruncher!):
[skip]
- Frame Stamps (?)

Frame Stamps is a marking of each frame with decoder and stream numbers and frame types - just for debugging, comparing and so on.
2pass enconding will be available in next release (i think two month from now). Also it will include full amd64 optimization and may be something else ;)

If you want to get more about each option - me and MuTek can help you.

[bond]

Originally posted by Sergey A. Sablin
First of all I don't understand what do you mean under "no par" at all.i meant that the stream doesnt signal a Pixel Aspect Ratio

Data partitioning is not supportedhm, i remember cruncher mentioned that the dshow encoder offers an option for that?

Frame Stamps is a marking of each frame with decoder and stream numbers and frame types - just for debugging, comparing and so on.interesting, thx!

2pass enconding will be available in next release (i think two month from now). Also it will include full amd64 optimization and may be something else nice :)
i assume it will also include .mp4 output?

If you want to get more about each option - me and MuTek can help you.always nice to see representatives of companies around on doom9 :)

[Sergey A. Sablin]

Originally posted by bond
i meant that the stream doesnt signal a Pixel Aspect Ratio
stream contains AR, but now it is always the same as for input stream

Originally posted by bond
hm, i remember cruncher mentioned that the dshow encoder offers an option for that?[/B]
not yet

Originally posted by bond
nice :)
i assume it will also include .mp4 output? [/B]
yes, OneClick will include mp4 of course. But this question for Lupus ;) I'm just developer of encoder, not all these stuffs.

[dvdboy]

I've just downloaded this to give it a whirl, but it seems to be having 'issues' with my PAL footage. The software recognises the video as PAL and lists the aspect ration, but when it encoded in pillarboxes the video, so although the output is still the correct resolution, the video has black bars down the left and right side.

Am I missing something completely obvious?

DVD-Boy

[MuTeK]

Originally posted by bond
i assume it will also include .mp4 output?


Yes ... AVC/Mp4 (http://www.moonlightcordless.com/clips/mp4/Belukha512x384_535kbps.mp4)

[Lupus_aka_Den]

Originally posted by bond
i assume it will also include .mp4 output?

Yes 8) Now muxer support AVC, MPEG-4 and AAC; MP3 (i hope) very soon. Beta release will coming very soon

[bond]

cool stuff! :)

about the sample, it seems the nero parser/ffdshow decoder combination cant handle the file (crashes by starting), also videolan crashes at start of the playback

when i demux and remux to .mp4 with mp4creator, the file is played but with distorted video (both ffdshow/vlc):
http://www.8ung.at/bond/moonlight.jpg

actually thats an effect (half picture washed) i saw already with ffmpeg on that videosoft stream (http://www.videosoftinc.com/avi/vssh-ccir30_d1_2000.avi)

couldnt test moonlights splitter/decoder as the two dont want to connect :/


btw i hope the muxer/splitter will support mpeg-4 timed text subs too (can be decoded already with vsfilter on directshow) :)

[video]

Originally posted by temporance
Why does no-one make a VfW codec for AVC??? Arghhhhh!

Unfortunately x264 are working happily vith vfw and you can output AVC video into an avi container via virtualdub if you would like that.

But please note, that avi was developed mainly by intel/microsoft at the end of eighties. At that time there was no commercially available variable bitrate audio and video codec technology. However avi has some tolerance to handle that, it's a pain in the ass to keep good a-v sync during playback in case of vbr audio and video within the avi container. More sadly avi is fragile. mpeg like containers can survive bitstream errors, avi not so much. the original avi has a limit of 1 Gigs in size, however opendml removes this limit, there is not so much software/player what can handle correcly opendmls having a size more than 2Gigs. Other reasons:
- Microsoft wants avi to die
- vfw implementation in wxp is almost broken
- As a commercial product developer you may pay license fees to microsoft for the use of avi besides the mpegla.

So if you want to use vfw at all costs I recommend using matroska instead of avi

[MuTeK]

1. I failed to make up the decoding sequence consisting of the Nero parser and the ffdshow decoder. What version of ffdshow were you trying to decode with? What was the Nero parser version?

2. Videolan plays the stream with artifacts, but it does not crash.

3. The given stream (as well as the VideoSoft stream) has two slices, and it seems that VLC (and ffdshow) cannot correctly playback such streams. For example, here (http://www.moonlightcordless.com/clips/mp4/Belukha512x384_1slice.mp4) is the given stream, but it has one slice only.

4. Our decoder perfectly plays the stream from VideoSoft.

[temporance]

Originally posted by video
But please note, that avi was developed mainly by intel/microsoft at the end of eighties. At that time there was no commercially available variable bitrate audio and video codec technology. However avi has some tolerance to handle that, it's a pain in the ass to keep good a-v sync during playback in case of vbr audio and video within the avi container. More sadly avi is fragile. mpeg like containers can survive bitstream errors, avi not so much. the original avi has a limit of 1 Gigs in size, however opendml removes this limit, there is not so much software/player what can handle correcly opendmls having a size more than 2Gigs. Other reasons:
- Microsoft wants avi to die
- vfw implementation in wxp is almost broken
- As a commercial product developer you may pay license fees to microsoft for the use of avi besides the mpegla.The poster was asking if there was a VfW codec, not about the merits of AVI. At the risk of going way off topic I have to point out that:
- AVI is not going to die, it's now well rooted in digital cameras and DVD players
- mp4 files are more sensitive to bit errors than AVI files because the sample table atoms are not redundant unlike AVI's indexes
- Microsoft owns no patents in AVI so you do not have to pay them any license fees (edit: AFAIK IANAL)
So if you want to use vfw at all costs I recommend using matroska instead of avi Matroska is much more likely to die than AVI. Do any standalones support it?

[bond]

Originally posted by MuTeK
1. I failed to make up the decoding sequence consisting of the Nero parser and the ffdshow decoder. What version of ffdshow were you trying to decode with? What was the Nero parser version?hm, i used a ffdshow compile that uses the cvs sources from a few days ago
also the nero parser is from the december nero package i think

2. Videolan plays the stream with artifacts, but it does not crash.hm, i use the latest svn videolan and it crashes here (windows), maybe you use the official release package and something change in the .mp4 handling since then?
but as videolan also uses ffmpeg for decoding avc, like ffdshow, it makes sense that it doesnt handle that file (2 slices)

3. The given stream (as well as the VideoSoft stream) has two slices, and it seems that VLC (and ffdshow) cannot correctly playback such streams. For example, here (http://www.moonlightcordless.com/clips/mp4/Belukha512x384_1slice.mp4) is the given stream, but it has one slice only.interesting, may i ask you what you mean with two slices? do you refere to these SP/SI Frames of the extended profile?

4. Our decoder perfectly plays the stream from VideoSoft. nice :)

[MuTeK]

Originally posted by bond
interesting, may i ask you what you mean with two slices? do you refere to these SP/SI Frames of the extended profile?


Slice coding means that picture is divided into parts (for ex - lines of macroblocks for MPEG2, in h264 slice is not restricted to lines, but may be any combination of figures of macroblocks) and coding each part separatelly.
Non of Extended profile features (including SI/SP slices) are not implemented in our encoder now.
BTW: in h264 there is no such syntax element like frame or picture. Basic coding element in h264 is slice and all types I-, P-, B-, SI-, SP- means I-slice, P-slcie and so on... Cause picture may consist of slices of different types - there is no restrictions about that. But in most cases of course picture is coding with of type of slices for convinience.

So in case of this movie there are two slices - one is top half of picture, and another is bottom. Effect that bottom is washed caused by decoder or demuxer that can't understand slices. But by my mind this is demuxer bug.

[MuTeK]

Originally posted by bond
... maybe you use the official release package and something change in the .mp4 handling since then?


I used the version packages 0.8.1

[MuTeK]

Originally posted by bond
hm, i used a ffdshow compile that uses the cvs sources from a few days ago also the nero parser is from the december nero package i think

Works, but only with one slice.

[bond]

Originally posted by MuTeK
Slice coding means that picture is divided into parts (for ex - lines of macroblocks for MPEG2, in h264 slice is not restricted to lines, but may be any combination of figures of macroblocks) and coding each part separatelly.
Non of Extended profile features (including SI/SP slices) are not implemented in our encoder now.
BTW: in h264 there is no such syntax element like frame or picture. Basic coding element in h264 is slice and all types I-, P-, B-, SI-, SP- means I-slice, P-slcie and so on... Cause picture may consist of slices of different types - there is no restrictions about that. But in most cases of course picture is coding with of type of slices for convinience.
So in case of this movie there are two slices - one is top half of picture, and another is bottom. Effect that bottom is washed caused by decoder or demuxer that can't understand slices. very interesting :)
is this slice coding related to error resilience?

But by my mind this is demuxer bugic, so a demuxer also has to explicitely handle slice coding

[Sergey A. Sablin]

Originally posted by bond
very interesting :)
is this slice coding related to error resilience?

exactly!
another usefull side of slices that not even decoding, but also encoding can be performed independent one from another, so it is possible to encode frame as two slices more efficiently on multi CPU systems like HT or true MP systems ;)

[Lupus_aka_Den]

Originally posted by MuTeK
Effect that bottom is washed caused by decoder or demuxer that can't understand slices. But by my mind this is demuxer bug.
All MP4 demuxers work only with frames, not slices. And i think what this is decoder bug.

[MuTeK]

Originally posted by Sergey A. Sablin
... so it is possible to encode frame as two slices more efficiently on multi CPU systems like HT or true MP systems ;)

Mpeg2->AVC, 720x480@23.98fps, 900kbps
GOP 100/1, Ref2, CABAC, 1pass VBR, ME: 64, Partitioning, Inloop

Ateme
Extra - 6.5 fps, CPU load 55%
Fastest - 17.5 fps, CPU load 55%

Moonlight, RDO on
1 slice - 30.6 fps, CPU load 55%
2 slice - 50.6 fps, CPU load 85% ;)

ps on Dual AMD 1.8Ghz (Opteron 244) :rolleyes:

[bond]

hm i cant play your .mp4 files with your new player. i can also not play the graph using your filters in graphedit. i get the "this graph cant play" error message

are there any wierd restrictions in using the h.264 decoder (with the mpeg-4 decoder everything works fine) or?

[LordRPI]

Originally posted by Sergey A. Sablin
exactly!
another usefull side of slices that not even decoding, but also encoding can be performed independent one from another, so it is possible to encode frame as two slices more efficiently on multi CPU systems like HT or true MP systems ;)

And for all of you who have taken classes in computer architecture or work in the field, you really can get some evil thoughts on this :devil:

Sergey, are multiple slices per frame going to be supported on the HD-DVD standard and such?

[Sergey A. Sablin]

Originally posted by LordRPI
And for all of you who have taken classes in computer architecture or work in the field, you really can get some evil thoughts on this :devil:

I don't understand what do you mean... (may be because of my english :rolleyes: )

Originally posted by LordRPI

Sergey, are multiple slices per frame going to be supported on the HD-DVD standard and such?
I haven't seen HD-DVD standard for h.264 yet, but I don't see any reasons for restrictions about number of sices per picture for h.264. Do you?

[unixfs]

Can someone explain if/where in the video bitstream I can find
the necessary informations to determine the frame rate?
Is there any flag as in the case of mpeg1/2 or some timing
fields like timeinc_resolution and timeinc_incremenent in mpeg4-asp?

Thanks.

[Sergey A. Sablin]

Originally posted by unixfs
Can someone explain if/where in the video bitstream I can find
the necessary informations to determine the frame rate?
Is there any flag as in the case of mpeg1/2 or some timing
fields like timeinc_resolution and timeinc_incremenent in mpeg4-asp?

Thanks.

Take a look on VUI parameters set in h.264 standard draft (http://www.dspr.com/www/technology/h264.pdf) and also on SEI picture timing messages. I think for the first time VUI information about frame rate, time scale and fixed_frame_rate flag is all you need.

[unixfs]

found, thanks.

[LordRPI]

Originally posted by Sergey A. Sablin
I don't understand what do you mean... (may be because of my english :rolleyes: )


Hey sorry about the obscure English. It's just very exciting for people in the computer architecture field to want to exploit parallelism as much as possible. It seems to be a big design principle in modern chips.

[Sergey A. Sablin]

Originally posted by LordRPI
Hey sorry about the obscure English. It's just very exciting for people in the computer architecture field to want to exploit parallelism as much as possible.
Yes you right, but I don't know another way to encode HD streams on todays PC or another CPUs :( So we must use any possibility - parallelism is not the worth case ;)

[temporance]

Originally posted by LordRPI
Sergey, are multiple slices per frame going to be supported on the HD-DVD standard and such? An inevitable problem of multiple independently en/decodable slices is some loss of video compression efficiency. This is because a independent slice has no access to information in its peers - making intra slice prediction impossible. So parallelism comes with a penalty (guess 5%).

[Sergey A. Sablin]

Originally posted by temporance
An inevitable problem of multiple independently en/decodable slices is some loss of video compression efficiency. This is because a independent slice has no access to information in its peers - making intra slice prediction impossible. So parallelism comes with a penalty (guess 5%).
You right - there is quality loss with many slices. And it grows up with number of slices, you can estimate losses with our encoder in graphedit tool - Installing One Click compressor you will have DirectShow filter named moonlight H.264 encoder, in that tool you can set up all parameters encoder has (application gives to set up a small number of them).

By my test two slices loss 1-3% of whole efficiency, 30 slices for NTSC D1 (i.e. each row is a slice) loss is about 25-30% of bitrate with the same quality. The most part of losses for real encoding are not from unavailability of predicted pels for intra-prediction, but from unavailability of predicted motion vectors - that is much bigger problem that intra prediction.

In any way that is not a reason for excluding multislice coding from HD-DVD standard by my mind (at least one type of slices in a picture) - this is choice of user, if he wants faster but less efficient encoding that he must have such possibility, if not - ok, encode with one slice and wait for 2-8 times longer.
Minimal unit for decoding for h264 is slice and it doesn't matter for decoder how many slices in a picture - there is no any speed or memory penalties on decoder side at all.

[bond]

using multiple slices also means incompatiblity with decoders not supporting that error resilience feature of course. i think this can be called a downside, because support for ER isnt a standard feature people could expect from a decoder imho

[Sergey A. Sablin]

Originally posted by bond
using multiple slices also means incompatiblity with decoders not supporting that error resilience feature of course. i think this can be called a downside, because support for ER isnt a standard feature people could expect from a decoder imho
As I said, slice is a minimal unit for decoding, so any decoder support slice decoding. ER means in this case that multiple slices per picture can be used for ER, and decoder can run error concealment code for compensate error prone channel losses, and this is not a part of standard of course, but decoding slice by slice while picture is not fullfilled is part of standard. If some decoder doesn't support this, then it doesn't fully support standard.

[bond]

ok :p

and what about my bug report regarding your h.264 decoder?

[LordRPI]

Originally posted by Sergey A. Sablin
Yes you right, but I don't know another way to encode HD streams on todays PC or another CPUs :( So we must use any possibility - parallelism is not the worth case ;)

Sorry I may be misunderstanding your english as well :) You don't know another way in which you can encode HD streams on todays PC's without using multiple slices per frame? And parallelism is not the worth case?

[LordRPI]

Originally posted by bond
using multiple slices also means incompatiblity with decoders not supporting that error resilience feature of course. i think this can be called a downside, because support for ER isnt a standard feature people could expect from a decoder imho

Bond, are you thinking about the redundant slice feature / arbitrary slice ordering that is available in Baseline and Extented profiles but not in Main Profile? It's been a lont time since I've looked at the chart so I'm probably not right. I know that Main Profile excludes a handful of features offered by Baseline.

[Sergey A. Sablin]

Originally posted by LordRPI
Sorry I may be misunderstanding your english as well :) You don't know another way in which you can encode HD streams on todays PC's without using multiple slices per frame?
:)
I'm talking that I don't know the way of encoding HD streams without parallelism. Of course there are many ways of using parallelism and multiple slices per picture is only one of them. That is my choice now.

Originally posted by LordRPI
And parallelism is not the worth case?
I mean parallelism as multiprocessor systems (many CPUs on one board such as HT, dual CPUs, quadro CPUs...), but there are another ways such a DSPs, clusters and so on, that is located on different boards and is more difficult to control all this CPUs than just one system.

Is this enough for our understanding? ;)

[Sergey A. Sablin]

Originally posted by bond
and what about my bug report regarding your h.264 decoder?
Do you about mp4 splitter and h264 decoder connection? We will test this, thanks.

[bond]

Originally posted by Sergey A. Sablin
Do you about mp4 splitter and h264 decoder connection? We will test this, thanks. hm i dont know if i understood your question right, but yes the splitter and decoder coming with the player can connect

Originally posted by LordRPI
Bond, are you thinking about the redundant slice feature / arbitrary slice ordering that is available in Baseline and Extented profiles but not in Main Profile?nope :)

It's been a lont time since I've looked at the chart so I'm probably not right. I know that Main Profile excludes a handful of features offered by Baselinehm which ones for example?

[LordRPI]

Originally posted by Sergey A. Sablin


Is this enough for our understanding? ;)

Yes, that was a good explanation :)


Originally posted by bond
hm which ones for example?

I found a chart and as per my recollection, error resiliance was a part of baseline and extended profile but excluded from main profile as per the chart below: (You can find the original paper here: http://www.fastvdo.com/h264overview.pdf)

http://lordrpi.intercarve.net/profiles.JPG

[temporance]

There seem to be an implicit assumption in the discussion here that H.264 might have some advantage for real-time HD encoding because it has slices. I'd like to out these assumptions and debunk them ;)

First, let's distinguish slices and slice parallelism: an encoder using slice parallism could be designed for virtually any format (MPEG-2, xvid, etc). This could be achieved pretty effectively even if the format's syntax does not support slices. Some of the encoding processes which have unavoidable spatial dependencies (e.g. prediction for entropy coding of the first MB row of slices 2 and later) would have to be performed after the parallel thread have returned. There would be the same small quality loss as per Sergey's post above.

Second, slice parellism would require two or more processors both encoding video. Commodity PC hardware does not provide this. Yes, we have hyperthreading, but HT is not true SMP. And even SMP is not n*100% efficient parellism. With SMP, the processors share memory and often cache. With HT, the logical processors share even more: execution units etc. When running memory- and execution unit-intensive code like a video encoder, enabling HT may even reduce the available MIPS.

[LordRPI]

temporance, I wasn't thinking of parallelism just along the lines of SMP or hyperthreading. There are far more types of parallelism exhibited in modern designs, but I guess it's time to kill this thread within a thread.

Edit: I was also thinking more along the lines of decoding since I still have a machine sitting next to me that requires 135% CPU time to decode 24fps @ 720p (It's a dual processor). I would suspect that more people will be decoding AVC/H.264 than encoding this format if it achieves mass adoption.

[temporance]

Aha, I guess that some of my presumed assumptions were themselves assumed. Sorry to get the wrong end of the stick. Of course there are lots of ways of "going parallel": from SIMD aritmetic operations right out to network-level parallelism.

The first parallel encoders I remember were the realtime HDTV MPEG-2 encoders of 1997-8. This was when ATSC was new. Several vendors built real-time hardware solutions by running multiple SDTV encoders in parallel. The results were surprisingly good. Even today, I believe professional hardware HDTV encoders are built from multiple SDTV encoding chipsets, with data sharing to improve motion compensation across the boundaries.

Parallel decoding is perhaps more difficult than parallel encoding because you need to be able to decode any input, not just bitstreams that are well-suited to parallel decoding. A straightforward IPPPPPPPP with linear prediction (and no independent slices!) is probably the worse case as data dependencies make parallel decoding very difficult. Maybe you could do the entropy decoding out-of-loop. It should be reasonably easy to separate decoding proper and deblocking/deringing into parallel processes, and this might even work for AVC even though its filtering is a critical part of its decoding loop.

Btw, a lot of this stuff will have be patented in the late 1990's when hardware vendors of MPEG-2 equipment were figuring out how the hell they were going to encode 1.5Gbit/s uncompressed video.

[Sergey A. Sablin]

Originally posted by temporance
Parallel decoding is perhaps more difficult than parallel encoding because you need to be able to decode any input, not just bitstreams that are well-suited to parallel decoding. A straightforward IPPPPPPPP with linear prediction (and no independent slices!) is probably the worse case as data dependencies make parallel decoding very difficult. Maybe you could do the entropy decoding out-of-loop. It should be reasonably easy to separate decoding proper and deblocking/deringing into parallel processes, and this might even work for AVC even though its filtering is a critical part of its decoding loop.

temporance, you right, parallel decoding is more difficult. The way you told were proposed by JVT experts, I mean entropy decoding out-of-loop, but... speed-up is very little because of small cache size in current CPUs - decoded elements like transform coeffs, vectors and others needs much temporal memory to store, and because of that they are not stored in cache for all picture. For tradeoff cases like entropy-decoding several MBs and than decoding coeffs there is a little speed-up like 5-10% with CPU loads 1.5 times higher than with decoding in common way :(

[LordRPI]

The cache size would definately be the limit. A thought had occured to me though (be easy on me, I'm not a codec engineer.. just a recent college grad) that in AVC/H.264, the functional units being used for CABAC would be inherently different than the rest of the coding so there would be times on which some functional units on a chip would just go... unused. Although if we were able to make full use of the functional units, I could really see some problems with caching. As I seem to recall, a lot of parallelism at the instruction level does have a dependency of having adaquate memory bandwidth available.

[bond]

any news on the h.264 decoder issue?


actually i now analysed your 2slices sample more deeply and compared it to a 2slices sample of videosoft with mpeg4ip's h264_parse tool:

in vss streams i have 2 slices forming one frame correctly it seems:
- first p-type slice "nal is new picture" and frame_num = 1
- second p-slice "nal is part of last picture" and frame_num = 1
- third frame is b-slice with "nal is new pic" and frame_num = 2
so the first two p-slices form the frame number one. the next slice (b-vop type) is part of the next frame, aso...

but in moonlight i have:
- first p-slice "nal is new picture" and frame_num = 1
- second p-slice "nal is part of last picture" and frame_num = 1
but
- third frame is b-slice with "nal is new pic" and frame_num = 1

so altough the b-slice is part of a new pic, it still says frame number 1!? is that the correct behaviour?

[bobololo]

Originally posted by bond
in vss streams i have 2 slices forming one frame correctly it seems:
- first p-type slice "nal is new picture" and frame_num = 1
- second p-slice "nal is part of last picture" and frame_num = 1
- third frame is b-slice with "nal is new pic" and frame_num = 2
so the first two p-slices form the frame number one. the next slice (b-vop type) is part of the next frame, aso...

but in moonlight i have:
- first p-slice "nal is new picture" and frame_num = 1
- second p-slice "nal is part of last picture" and frame_num = 1
but
- third frame is b-slice with "nal is new pic" and frame_num = 1

so altough the b-slice is part of a new pic, it still says frame number 1!? is that the correct behaviour?

Yes, frame_num element syntax is only incremented for reference frames. In the case of moonlight, if their b-frame aren't used as reference, so the frame_num of 1 for the bframe is correct.

Regarding vss streams, it seems they use their bframe as reference or at least the frame is inserted in the reference list.

[Sergey A. Sablin]

Originally posted by bobololo
Yes, frame_num element syntax is only incremented for reference frames. In the case of moonlight, if their b-frame aren't used as reference, so the frame_num of 1 for the bframe is correct.

Regarding vss streams, it seems they use their bframe as reference or at least the frame is inserted in the reference list.
My experiments with vsoft shows that they use only one ref_frame.

If you want to analyze more deeply mpeg-2 or h.264 streams - just try to use our StreamEye Suite (http://www.moonlight.co.il/products/consumer/streameye/). It shows all sequence/picture parameter sets, slice headers, slice boundaries inside picture (that is what we talking about), MB info like mb_type, MVs with ref_idxs, quant, bits count and so on.
All values are named regarding mpeg-2 and h.264 standards, so it is easy to check what each paramater means.
I hope it helps you to save your time analyzing h.264 streams.

[bond]

Originally posted by bobololo
Yes, frame_num element syntax is only incremented for reference frames. In the case of moonlight, if their b-frame aren't used as reference, so the frame_num of 1 for the bframe is correct.

Regarding vss streams, it seems they use their bframe as reference or at least the frame is inserted in the reference list. thx! :)

[bobololo]

Originally posted by Sergey A. Sablin
My experiments with vsoft shows that they use only one ref_frame.

If you want to analyze more deeply mpeg-2 or h.264 streams - just try to use our StreamEye Suite (http://www.moonlight.co.il/products/consumer/streameye/). It shows all sequence/picture parameter sets, slice headers, slice boundaries inside picture (that is what we talking about), MB info like mb_type, MVs with ref_idxs, quant, bits count and so on.
All values are named regarding mpeg-2 and h.264 standards, so it is easy to check what each paramater means.
I hope it helps you to save your time analyzing h.264 streams.

Does it support MBAFF and High Profile ?

[bond]

btw ffmpeg now seems to be able to decode multiple slice frames :)
it still shows some artifacts (on some parts in the upper half of some frames on the moonlight sample discussed in this thread), dunno whats causing this :(

[Sergey A. Sablin]

Originally posted by bobololo
Does it support MBAFF and High Profile ?
High Profile is already supported, but it seems to me, that version on our site doesn't. MBAFF is still not supported - only frame/field picture modes.

[Sergey A. Sablin]

Originally posted by bond
btw ffmpeg now seems to be able to decode multiple slice frames :)
it still shows some artifacts (on some parts in the upper half of some frames on the moonlight sample discussed in this thread), dunno whats causing this :(

Can you post here some screen-shots and links to streams? May be I'll tell you what its causing by.
Also, do you check parts with artifacts with our stream eye app? I think it will be helpfull

[bond]

Originally posted by Sergey A. Sablin
Can you post here some screen-shots and links to streams? May be I'll tell you what its causing by.http://www.8ung.at/bond/moonlight2.jpg
this is only shown on some few frames

Also, do you check parts with artifacts with our stream eye app? I think it will be helpfullnope i try to avoid installing limited software

[Sergey A. Sablin]

Originally posted by bond
this is only shown on some few frames

can you give me this stream and link to decoder you use? it seems like this decoder hasn't proper support of multireference MC, may be it has improper handling of decoded picture buffer and when MB refers to some frame with some ref_idx decoder compensates it from another one. In this case errors should propagate to the next IDR picture. If there is no propagation, than error occurs on B-slice, so may be B-slices in decoder doesn't finished yet or has parts of incorrect code.
Originally posted by bond

nope i try to avoid installing limited software
As you wish, but comparing frame by frame output of yours decoder in some player and the same stream in our app you can quickly find the cause of error.
In any way if you encode streams with our encoder try to set Frame stamps flag at least to see on which frame number and frame type errors in decoder occurs.

[bond]

Originally posted by Sergey A. Sablin
can you give me this stream and link to decoder you use?the clip is the sample you posted some time agon in this thread unchanged
i use latest ffdshow (cvs), which has been updated with ffmpeg sources 5 days old

it seems like this decoder hasn't proper support of multireference MC, may be it has improper handling of decoded picture buffer and when MB refers to some frame with some ref_idx decoder compensates it from another one. In this case errors should propagate to the next IDR picture. If there is no propagation, than error occurs on B-slice, so may be B-slices in decoder doesn't finished yet or has parts of incorrect code.hm mref and b-frames should be working (of course i dunno if its 100% complete), also these effects dont occur only on b-frames, or on all b-frames
some scenes are completely artefact free, some scenes show it almost on any frame

[bond]

Originally posted by MuTeK
Yes ... AVC/Mp4 (http://www.moonlightcordless.com/clips/mp4/Belukha512x384_535kbps.mp4) i pointed the dev from mpeg4ip to this sample and he meant there is a problem he describes that way:
The problem with the moonlight file is that the last "frame" in the file appears
to have 3 NALs (1 SEI, 2 nals of 14 and 13 bytes), but in the mp4 file, they
have 2 NALs:
sample id: 2205
read offset 0 nallen 5 sample Size 44 1
read offset 9 nallen 31 sample Size 44 0

where it should look like:
sample id: 2205
read offset 0 nallen 5 sample Size 44 1
read offset 9 nallen 14 sample Size 44 0
read offset 27 nallen 13 sample Size 44 0
discussion can be found here:
https://sourceforge.net/forum/forum.php?thread_id=1233663&forum_id=59136

[Sergey A. Sablin]

Thank you Bond, I'll check the last frame of this clip.
Is there something new with decoding of our clips? Do you still see artifacts when decoding with ffdshow?

[bond]

some more findings on the sample:

There also seems to be a few problems with their file, as well - their avcC.profile_compatibility is wrong, as is their iods.audioProfileLevelId and iods.videoProfileLevelId (I will be removing the iods atom from files created by mp4creator unless specified by a command line parameter in a future release).

It looks like ftyp.majorBrand is incorrect - it should be mp42

Is there something new with decoding of our clips? Do you still see artifacts when decoding with ffdshow?still the same as ffmpeg wasnt updated till now

btw i noticed problems when seeking your files with the nero and ffdshow decoder
- the 1slice .mp4 takes very long when seeking but than plays
- the 2slice .mp4 also takes very long but than crashes with ffdshow / restarts playing from the beginning with nero

[bond]

heya moonlight guys (or now mainconcept guys?)

i noticed that the .mp4 files you linked to in this thread all only have one entry in the stss (sync sample atom)
of course thats not correct and should be fixed in the muxer