CoreCodec/H.264 Codec "CoreAVC"

[STaRGaZeR]

its not that simple to "just use the shaders" for the video processing. you need to interface with the streamprocessors/shaders on the graphics cards. Shader programming language is more complex to deal with compared to CUDA which uses the standard C language to communicate with the shaders and makes them do what the programmer wants to achieve.

What I mean is that, in both cases using the CUDA language to communicate with the shaders, it seems more logical to me writing the entire application in a way that it only uses the shaders and not a combination of the shaders and the PureVideo processor. Why? Using dedicated chips that do fixed functions have the avantage of being much faster than a general purpose processor. But the disavantage of not having any flexibility. I don't know the specs of the PV chip, but let's imagine it only supports AVC acceleration up to level 4.1 . What about if you want to add level 5.1 support? You can't if you use that dedicated PV unit. You'd have to use the shaders, that have no limitations, for that one. And doing this since the beginning is better IMO.

ati do have a method/api for interfacing with the shaders on thier graphics cards, i believe its called CTM and like OpenCL, its no where near as far developed and supported as CUDA is though.

one thing to remember, CUDA support is provided for free by Nvidia, even ATi are allowed to have their cards support CUDA with no royalty payments or catches. It has also been shown that 3rd party modified drivers bring working CUDA support to ATi graphics cards, so it is possible for ATi to adopt CUDA, but if they do or dont, well its up to them.
One thing to take into account is that CUDA is very large in its user base, many specialist companies are using it for thier specific applications, e.g the hospital in my city uses a CUDA accelerated system for tomographic scans (some sort of 3D x-rays).

You're right, the huge advantage of CUDA is that every one with a NV 8800 card already has a CUDA processor. That 3rd party software has never been proven to exist, so if it's not released it does not exist. Of course ATI cards can do CUDA, but they just don't want to support it officialy, and I understand them. Is a pain for a company to use the competitors technology (even being free) and become a puppet somehow. Do you have experience with both OpenCL and CUDA?

[Sagekilla]

Correct me if I'm wrong here, but also if you used CUDA based decoding, overclocking the video card would make decoding faster. Or am I wrong on that?

[Cyber-Mav]

Correct me if I'm wrong here, but also if you used CUDA based decoding, overclocking the video card would make decoding faster. Or am I wrong on that?

yes overclocking the video card can make things faster, but you need to understand that stress testing becomes more vital since an unstable overclock in games may just cause some artifacts to be displayed on screen or some dodgey textures to show up, but when using an app that wants precise calculation results a dodgey overclock can cause a whole lot of issues.


STaRGaZeR: i havent worked with OpenCL just have some CUDA and HLSL experience with a very very tiny bit of Cg experience.

[Inventive Software]

What I mean is that, in both cases using the CUDA language to communicate with the shaders, it seems more logical to me writing the entire application in a way that it only uses the shaders and not a combination of the shaders and the PureVideo processor. Why? Using dedicated chips that do fixed functions have the avantage of being much faster than a general purpose processor. But the disavantage of not having any flexibility. I don't know the specs of the PV chip, but let's imagine it only supports AVC acceleration up to level 4.1 . What about if you want to add level 5.1 support? You can't if you use that dedicated PV unit. You'd have to use the shaders, that have no limitations, for that one. And doing this since the beginning is better IMO.

The problem with that is that I'm almost certain ATI and NVIDIA use different methods to call the shaders, thus you can't do one solution fits all at the moment. NVIDIA's CUDA allows access to the VP2 decoder (which is not in G80 GPUs), ATI did have a solution but this seems to have slowly died. This is why OpenCL is around and being ratified.


You're right, the huge advantage of CUDA is that every one with a NV 8800 card already has a CUDA processor. That 3rd party software has never been proven to exist, so if it's not released it does not exist. Of course ATI cards can do CUDA, but they just don't want to support it officialy, and I understand them. Is a pain for a company to use the competitors technology (even being free) and become a puppet somehow. Do you have experience with both OpenCL and CUDA?

DGAVCIndexNV/DecodeNV are both work-in-progress programs that are proven to use the NVIDIA GPU effectively. See the thread in this sub-forum for more information.

[STaRGaZeR]

DGAVCIndexNV/DecodeNV are both work-in-progress programs that are proven to use the NVIDIA GPU effectively. See the thread in this sub-forum for more information.

3rd party software to enable CUDA on ATi cards, not 3rd party CUDA applications in general. To be more specific, the 'hack' shown was a few screenshots of 3DMark Vantage showing huge increases in the CPU score with an ATi card due to PhysX support through CUDA.

[BetaBoy]

CoreAVC v1.8.5 is in final QA now. If all goes we will release it tomorrow.

[TheShadowRunner]

Please BetaBoy, test to make sure that switching resolution while a video is playing doesn't produce artefacts. (happens after 1.6.5, up to 1.8)
See you,

TSR

[BetaBoy]

From the v1.8.5 changelog....
- Fix: Improved dynamic reconnection

[TheShadowRunner]

Sounds good to me ;)
Thanks,

TSR

[CruNcher]

BetaBoy very interesting read is the conversation between Donald Graft and a Nvidia Developer seems from his standpoint many Software Decoders/Encoders and he talks about a lot aren't fully Compliant yet with the H.264 Specification compared to Nvidias Hardware, what would you say to this statement.


It seems that the problem is the same for all 3, and I strongly suspect these are not compliant (The decode HW is very very strict on compliance, though there might be some fudging we can do in the driver).

The Cuvideosource will fail to detect mpeg4-hosed.264 as a valid elementary stream because of invalid nal_ref_idc (0) being specified for the PPS (I can modify that, but that's not a problem for actual decoding).

Looking at sample.264, it appears to be B-pyramid with 3-bframes, using B-frames as reference, and only one reference frame in the reference picture list.

I don't seem to see a problem except for the fact that it is truncated at a non-IDR point, so some of the reference frames will not be available for the first few B-frames that are then used as reference. It also seems a bit unusual to have multiple consecutive B-frames sharing the same frame_num value, even though these have nal_ref_idc!=0 (this can cause unpredictable results in the sorting of the reference picture list if they are used as a reference by a P-frame) -> I don't think this is compliant, but I'll have to double check.

Many encoders & decoders out there (including CoreAVC, libavcodec, MPC-HC, Cyberlink and x264) do not fully implement the H.264 specification, especially when it comes to DPB management and reference picture list reordering.

I guess with MPC-HC/Cyberlink he meant the Software parts

[nm]

BetaBoy very interesting read is the conversation between Donald Graft and a Nvidia Developer seems from his standpoint many Software Decoders/Encoders and he talks about a lot aren't fully Compliant yet with the H.264 Specification compared to Nvidias Hardware, what would you say to this statement.
Um, looks like he's just saying that most software decoders aren't as strict on H.264 spec compliancy as hardware implementations. This means that software decoders may be able to decode some broken streams that the hardware decoder fails on. A strict software decoder would be useful for testing encoders, but otherwise it doesn't really matter.

[Dark Shikari]

Um, looks like he's just saying that most software decoders aren't as strict on H.264 spec compliancy as hardware implementations. This means that software decoders may be able to decode some broken streams that the hardware decoder fails on. A strict software decoder would be useful for testing encoders, but otherwise it doesn't really matter.Moreso, "spec compliancy" says absolutely nothing about ability to decode broken streams; it only defines what a non-broken stream is.

[BetaBoy]

BetaBoy very interesting read is the conversation between Donald Graft and a Nvidia Developer seems from his standpoint many Software Decoders/Encoders and he talks about a lot aren't fully Compliant yet with the H.264 Specification compared to Nvidias Hardware, what would you say to this statement.



I guess with MPC-HC/Cyberlink he meant the Software parts

Define compliant? Since the H.264 spec is a moving target what was complaint last year may not be this year. As far as his notes... I'm not gonna feed the fire but say as Dark S has said in that once those features come around and are being used, does it really matter? IE; Cart before the horse?

Additionally if anyone is aware of and has a sample of any non-compliant h.264 stream that we don't support let me know (aside from 4:2:2 or 4:4:4 profiles).

[mlaviolette]

Additionally if anyone is aware of and has a sample of any non-compliant h.264 stream that we don't support let me know (aside from 4:2:2 or 4:4:4 profiles).I'm glad you asked! CoreAVC has some problems with Hauppauge-generated h.264 streams if you use the No IDR encoder default when creating the file. The file plays fine if you start from the beginning, but if you reposition (using the slider in Windows Media Player, for example), then playback is jerky as CoreAVC skips frames. Stopping and starting at the beginning corrects this but makes repositioning impossible.

If I use GraphEdit to load the Hauppauge encoder filter and choose one of the IDR options to create a new file, then I can reposition just fine. Unfortunately Hauppauge does not make these options easily available, and are only accessible via GraphEdit on a "development" tab of the filter.

The ArcSoft player handles No IDR just fine.

I opened a ticket with CoreAVC support but they just blew me off.

[BetaBoy]

The support center is not really the right place for more technical stuff as they answer more generic support issues and then they forward any technical issues our way (but they are becoming more technical every day and making it easier for the rest of the staff and our customers). For an expedited and direct 'technical' response is best handled to contact me directly at: betaboy AT corecodec DOT com and I'll review any potential issues/files with the devel staff.

mlaviolette please send a sample and we'll take a look at it. But if you look back a few pages the issue you talk about has been discussed and we are working on a fix.

[Cyber-Mav]

CoreAVC v1.8.5 is in final QA now. If all goes we will release it tomorrow.

is this the release you were mentioning that will be the last of the 1.x versions and the following releases after this one will be 2.0 based?

[Dark Shikari]

I can confirm the latest CoreAVC 1.8.5 demo supports Predictive Lossless.

Great job!

[Sharktooth]

on the website it's still 1.8.0

[BetaBoy]

I can confirm the latest CoreAVC 1.8.5 demo supports Predictive Lossless.

Great job!

Thx DS... we are trying to knock out the last of these features and issues before the v1.9.0 CUDA release as we know that CUDA will have its own issues we need to work through and that's why we wanted to focus on it and do it in v1.9.x and not post v2.0.

Also, the IDR problem is really the last big bug... but wouldn't you know that `md just pointed to a 59fps file that plays slower (we think that's CORE related and not filter).

[Malow]

hamz... gamma adjustment in the decoding options besides B/C/S is planned? :p

[me7]

Thx DS... we are trying to knock out the last of these features and issues before the v1.9.0 CUDA release...

Can you talk about the "when" yet? Next three months, half year, full year???

[BetaBoy]

When its ready.

[lilhobo]

how do we know which codec is used by what application?

[Dark Shikari]

More good news; I modified x264 to support lossless i8x8 blocks in Predictive profile and CoreAVC 1.8.5 beta supports those correctly too. I'm not 100% sure whether the standard implies that i8x8 predictive is supposed to use 8x8-filtered pixels or actual edge pixels; apparently CoreAVC and I agreed on the latter.

[BetaBoy]

BUG: coreavc.marvel.bug.png (http://x264.nl/coreavc.marvel.bug.png)
Sample: marvel.m2ts (http://x264.nl/marvel.m2ts)

Extra info:
[00:32] (CruNcher): jarod
[00:32] (CruNcher): i saw alot of these problems with CoreAVC and X264 streams
[00:32] (jarod): but this is not x264
[00:33] (CruNcher): oh

So look into it please!
People encoding bluray movies are still using CoreAVC instead of DGAVCDEC because CoreAVC is faster.

ok.... we looked at the sample some more.... and found that it has out of range motion vectors. Even more then what we added support for (for the older x264 builds). So no fix is planned for this as its out of spec. BTW... what was used to create this?

[Snowknight26]

I'm very much inclined to say that thats from the Iron Man Blu-ray, so most likely not x264.

[Dark Shikari]

Interesting, if I swap repeatedly between lossy and predictive lossless mode in the same frame, it almost cuts in half CoreAVC's performance :eek:

105fps lossless
115fps lossy (QP5)
56fps mix (QP8 / lossless mix)

(In other news, I'm not going to actually implement this swapping in x264 right now... :p )

[BetaBoy]

DS thx.... I'll pass it on but I think its also related to that current 'seek' bug as well. I'll let you know.

[Dark Shikari]

DS thx.... I'll pass it on but I think its also related to that current 'seek' bug as well. I'll let you know.It is? I was thinking what you were doing is something on the order of re-initting the decoder each time it switched between modes, the lazy way of dealing with function pointer overloading for replacing the various functions with what is necessary. This isn't a bad method, since its unlikely anyone is going to be that sadistic with the streams they generate... :p

[CruNcher]

ok.... we looked at the sample some more.... and found that it has out of range motion vectors. Even more then what we added support for (for the older x264 builds). So no fix is planned for this as its out of spec. BTW... what was used to create this?

BetaBoy but if that really is the case then it would mean

Nvidia (Hardware)
Mainconcept/Elecard/DivX
ffh264
Arcsoft
Cyberlink
Intervideo

All of them would do it wrong or better they all would try to make these wrong mastered BDs watchable, somehow it sounds crazy (unless for example these would have been mastered with Elecard/Mainconcept Solutions (Sonic Cinevision for example) and they never found this Bug and their SDK is also bugged (as many other Decoder are being based on it) and the others work around this spec non compliant streams (Nvidia) to prevent such viewing problems for their customers (and never ever came on the idea to contact the Studios about it) all of this would also apply if it's actually user damaged streams and error correction sets in in those Decoders fixing this small problems.

Not sure if all of them are so incompetent to see or realize that they doing it wrong (not for me to rate that), but i guess for the CoreAVC user it's better then to prevent such problematic renderings for already released BD's so this would mean accepting the Spec noncompliance and work around it (if it really is such and no other things play into this) :(
So in other words CoreCodec is standing currently against their Customer experience and all other Vendors and says "These BD's are mastered wrong we don't make this error all the others "accept" it so you have to accept that with CoreAVC you will have these problems with this kind of wrong Mastered BD's" not sure if that is doing the CoreAVC users any good here that try to experience such BD's with your Product currently.
At least then CoreCodec should take the next step and get in contact with the Studios (that BD's have been found wrongly mastered Spec non compliant,but first should really check such a BD themselves, maybe all these samples got damaged somehow at these frames and error correction sets in for the other Decoders @ that time) and tell them they master it wrong so that this can be prevented in future BD releases, the way it is currently it only hurts CoreCodec and their customers.

Of course i only say that because CoreCodec seems really 100% sure that this is actually a spec non compliant mastering behavior (or a damaged stream) (on the side of some Studios or users) and couldn't find any bug in the way they apply the Spec (Decoder does it 100% correct no bug found). Though I'm not sure why the other Decoder have no problems here but i realized that CoreAVC has no way of error correction (which would seem to be capable of fixing such bad 1 or 2 frame series) even not a very simple one as ffh264, maybe that plays into this problem and preventing it visually for the other Decoders.

[BetaBoy]

CruNcher.... we've all been through the Motion Vector issue before in this thread right? and nothing has changed. The AVC spec and reference is available for all to review... Dark Shikari had corrected the Motion Vector compliance in x264 post rev663. However, we totally understand the reasoning on why we should support it in CoreAVC.... but no matter what those reasons are, the bottom line is that it is _NOT_ spec compliant (I'm a broken record), period. Why follow the AVC Specifications? They are created for a reason and in this case it's very valid because a larger MV range comes at the cost CPU/GPU cycles, its as simple as that.

As I've already discussed with Jarod.... but yes I am going through the motions to reach out to each of the mastering houses for videos that are brought to our attention. However at the same time we will also address how we are going to deal with this in CoreAVC 2.0.

Thx everyone for the great feedback!

[CruNcher]

So these Motion Vector in compliant streams couldn't be actually bad streams caused in copy actions from or to the BD/HDD mastering media, that get fixed for the other Decoders because of error correction ? (You know i don't trust consumer HDD's these days in terms of reliability especially not with such extreme user behavior as BD backup with extreme I/O scenarios that result out of this)

[molitar]

Ok I wish to know why is it so hard just to add DXVA support in CoreAVC? After all it already works in Mediaplayer Classic Home Cinema but the only issue is if the format is even off by a little bit the file won't play :( ATI is the KING when it comes to using video card for media use because unlike Nvidia who removed secondary full screen display which is very crucial to a good media system they completely removed it so now I have myself an ATI Radeon HD 3870 video card and not going back to Nvida CRAP! I use my video card for gaming and watching video's on my HDTV and Nvidia can't do this properly anymore. But the one feature that I am wanting in CoreAVC still has not been implemented! GPU Support! If it wasn't that the DXVA will not work through ffdshow I wouldn't even bother using CoreAVC yet CoreAVC still does not support GPU the ONE and ONLY feature I have been wanting!

Every indication seems that this is still not being worked on as yet! Instead your going to spend your time working on something called CUDA for NVIDIA and not DXVA support for our GPU's! I'm getting so sick of hearing promises of GPU support comming soon!

[BetaBoy]

BTW Dark Shikari... congrats on: http://mailman.videolan.org/pipermail/x264-devel/2008-September/004993.html

[BetaBoy]

molitar Pls interject something of value and stop the DXVA ranting. I've been through this with you before and I'm not gonna do it again.

[nm]

Ok I wish to know why is it so hard just to add DXVA support in CoreAVC? After all it already works in Mediaplayer Classic Home Cinema but the only issue is if the format is even off by a little bit the file won't play :(
Having DXVA support in CoreAVC wouldn't change that at all compared to MPC HC. In DXVA, it's mostly the video driver and hardware that determine what can be decoded, not the filter that routes the video stream to the driver.

ATI is the KING when it comes to using video card for media use because unlike Nvidia who removed secondary full screen display which is very crucial to a good media system they completely removed it so now I have myself an ATI Radeon HD 3870 video card and not going back to Nvida CRAP!
Well, the latest driver version from NVIDIA now supports L5.1 H.264 decoding. Maybe ATI will follow at some point (if their hardware is up to it).

Every indication seems that this is still not being worked on as yet! Instead your going to spend your time working on something called CUDA for NVIDIA and not DXVA support for our GPU's! I'm getting so sick of hearing promises of GPU support comming soon!
If you want to use your ATI GPU for >L4.1 H.264 decoding with the current drivers, you need a custom GPU decoder implemented on the stream processors, not DXVA support.

[CruNcher]

Ok I wish to know why is it so hard just to add DXVA support in CoreAVC? After all it already works in Mediaplayer Classic Home Cinema but the only issue is if the format is even off by a little bit the file won't play :( ATI is the KING when it comes to using video card for media use because unlike Nvidia who removed secondary full screen display which is very crucial to a good media system they completely removed it so now I have myself an ATI Radeon HD 3870 video card and not going back to Nvida CRAP! I use my video card for gaming and watching video's on my HDTV and Nvidia can't do this properly anymore. But the one feature that I am wanting in CoreAVC still has not been implemented! GPU Support! If it wasn't that the DXVA will not work through ffdshow I wouldn't even bother using CoreAVC yet CoreAVC still does not support GPU the ONE and ONLY feature I have been wanting!

Every indication seems that this is still not being worked on as yet! Instead your going to spend your time working on something called CUDA for NVIDIA and not DXVA support for our GPU's! I'm getting so sick of hearing promises of GPU support comming soon!

Indeed it is strange that Nvidia removed it first on Vista now on XP (especially as it was flawless working for both), but the Problem here wasn't Nvidia Microsoft Demand it because of DRM protection reasons (see Vista Specs) someone could now get the idea that Nvidias Hardware design is weak in terms of useing this to circumvent DRM (and no Driver protection is possible or takes time to implement so the only solution was removing it for the time completely), i guess there will be some interesting things coming up in those regards soon when people begin to analyze this ATI yet seems to better protect it or is missing something so Microsoft isn't such harsh with them as they are currently with Nvidia ;)

https://nvidia.custhelp.com/cgi-bin/nvidia.cfg/php/enduser/std_adp.php?p_faqid=2011&p_created=1169076830

[Dark Shikari]

BTW Dark Shikari... congrats on: http://mailman.videolan.org/pipermail/x264-devel/2008-September/004993.htmlIndeed, now we'll see how long it takes everyone else to support it :p

[BetaBoy]

Interesting, if I swap repeatedly between lossy and predictive lossless mode in the same frame, it almost cuts in half CoreAVC's performance :eek:

105fps lossless
115fps lossy (QP5)
56fps mix (QP8 / lossless mix)

(In other news, I'm not going to actually implement this swapping in x264 right now... :p )


Dark Shikari... we looked at this more and looks like I was wrong about it being a bug and that lossy and predictive lossless switching _is_ slow atm in CoreAVC and that we will add it to the todo for 2.0.

BTW... do you have a sample(s) we can test?

[Dark Shikari]

Dark Shikari... we looked at this more and looks like I was wrong about it being a bug and that lossy and predictive lossless switching _is_ slow atm in CoreAVC and that we will add it to the todo for 2.0.

BTW... do you have a sample(s) we can test?Yeah, I figured it wasn't a bug; my guess was that you were re-inittiing the decoder on a switch between lossy and lossless to change function pointers or something like that. Which is a completely reasonable approach given that odds are nobody will do such a crazy thing. Of course, saying that is just asking for someone to do it for the purpose of making CoreAVC look slow ;)

I'll post a sample in a bit.

[qyqgpower]

@BetaBoy
CoreAVC 1.8 seems to always pass BFF flag to Enhanced Video Renderer and VMR9 (with Haali Renderer, everything is OK) even if the source is actually TFF, causing the pictrues to jump back and forth.
This issue can only be observed when the deinterlacer in EVR and VMR9 is activated.

And a little question, Will CoreAVC support those amazing quality deinterlacers in DXVA?

[nm]

CoreAVC 1.8 seems to always pass BFF flag to Enhanced Video Renderer and VMR9 (with Haali Renderer, everything is OK) even if the source is actually TFF, causing the pictrues to jump back and forth.
This issue can only be observed when the deinterlacer in EVR and VMR9 is activated.
Are these x264-encoded sources or something else? With x264, you need to use the nal-hrd+interlaced patch and correct parameters to insert field order SEI to the stream: http://forum.doom9.org/showthread.php?t=137432

And a little question, Will CoreAVC support those amazing quality deinterlacers in DXVA?
The same hardware deinterlacing, noise reduction etc. post-processing filters are in use with normal DirectShow video rendering too. Just set CoreAVC to use "Hardware deinterlacing", like you probably already have.

[STaRGaZeR]

Just to confirm, NV12 output will be available in 1.8.x or in 1.9?

Just set CoreAVC to use "Hardware deinterlacing", like you probably already have.

With ATI cards, you also need NV12, else it won't work.

[squid_80]

CoreAVC looks at the stream to determine if it's TFF or BFF and passes the appropriate flags to whatever renderer is connected; it's up to the renderer to interpret them correctly. VMR7 is almost always correct, Haali's is correct, VMR9 is a complete gamble based on your OS/video drivers (EVR I don't know about). Even when VMR9 does get it right seeking to a new position can make it flip the order (likewise when it gets the field order wrong seeking can sometimes correct it).

[qyqgpower]

@nm
The test clips are encoded by x264 and of course I set the --nal-hrd and interlaced parameters correctly, as they plays totally fine with all other decoders(mainconcept, cyberlink, ffdshow etc.) and renderers.
BFF video encoded same way (just change --tff to --bff) works fine with CoreAVC and all renderers.

@squid_80
If VMR9 is a complete gamble at field order, why other decoders could work with it (and EVR) correctly.

[squid_80]

I see the same symptoms when using VMR9 with hardware deinterlacing no matter which codec I use.

[BetaBoy]

Just to confirm, NV12 output will be available in 1.8.x or in 1.9?

Its already in 1.8.5... but this IDR/Seek bug is what's on task now and is holding up the release.

[STaRGaZeR]

Nice, looking forward to it :)

[Cyber-Mav]

would future version of coreavc using cuda be more tolerant of decoding h264 encoded video and does not follow the dxva specifications closely than current dxva implementations do, such as cyberlink etc?

seems like anything out of the norm breaks dxva acceleration very easily.

how is coreavc's gpu decoding assist going to be implemented? is it using cuda to access the video decode hardware? or will it be using cuda to communicate directly with the stream processors and have them do the general purpose processing that is required for the decoding? im guessing the latter method would eliminate a lot of the issues with encoded content variances that break hardware acceleration quite easily.

[BetaBoy]

Define tolerent as I thing if we handle it it should parse just fine..... Also CUDA only has one option atm... that's on or off.... We have not optimized it in this first phase for 1.9.x. but take a look at some numbers;
benchmarks on a 1440x1080 mbaff clip:
4 cores: coreavc = 136.6fps, cuda 69.1fps
2 cores: coreavc =79.5fps, cuda 69.2fps
1 core: coreavc 40fps, cuda 58fps

As you can see CoreAVC scales with the number of cpus, while CUDA is limited by the gfx card. But note that we are not showing it above but the offload from the CPU to the GPU in general makes this more then worth the effort.