Green diagonal line on screen during H264 playback, only in DShow apps and not in VLC

[christopherw]

Help!

For some reason (which I can't fathom) this ONE video file I'm trying to play back has a thin diagonal green line running down from the top left of the video area to almost the bottom left (the video file has a widescreen AR though so it doesn't touch the bottom right corner, it ends on the baseline before the corner). If I flip the image, the line flips along with it - but it's not actually in the video, it's being overlaid on top of it (and to prove it, I played it back in VLC and there's no line, so it's definitely something to do with how it's being decoded and played back).

The file in question is an h264 video stream, (detected as V_MPEG4/ISO/AVC) in an mkv along with an ac3 audio stream and an english subpicture stream (foreign movie). Its resolution is 1660x720. I have loads of these kinds of video files (they're HD video files) and this is the first mkv which exhibits this behaviour if I play it back.

VLC plays it fine, but my PC's not powerful enough to decode H264 stuff without at least some stuttering or framedrop, so I'm using CoreAVC along with Haali Media Splitter and AC3Filter. This combination has always worked great with everything I try to play back, except for this file.


Media Player Classic also shows the line, as well as wmp 6.4 which I normally use (force of habit, if it ain't broke etc). Does anybody have the slightest idea how to get rid of this? The only other reference to a diagonal green line (and which I also saw once on a mate's mac) was a VLC issue, but it's the other way round here - VLC plays it fine but my PC's not powerful to play back H264 stuff flawlessly in VLC, and directshow-based apps are the ones that are misbehaving. Anybody have the slightest idea as to how I can fix this?

OS is 2000 sp4 fwiw, radeon x1600xt.

[foxyshadis]

The obvious answer is your decoder, COreAVC. Is it the latest version? If not, update it; if so, or if it's the buggy old 0.4 and you don't want to pay, try ffdshow-tryout, Nero, or whatever else you might have. Since 1660 isn't mod16, brain-dead decoders will have problems - probably show 1080p with a green line across the bottom, as well. (Not-so-obvious answers include video driver and video acceleration settings.)

MPC gives you total control over the filter priorities; use it if you have to.

[christopherw]

The obvious answer is your decoder, COreAVC. Is it the latest version? If not, update it; if so, or if it's the buggy old 0.4 and you don't want to pay, try ffdshow-tryout, Nero, or whatever else you might have. Since 1660 isn't mod16, brain-dead decoders will have problems - probably show 1080p with a green line across the bottom, as well. (Not-so-obvious answers include video driver and video acceleration settings.)

MPC gives you total control over the filter priorities; use it if you have to.

I'm rolling with CoreAVC 1.2; I do use MPC on my laptop as my primary player but my associations are all delicately rigged on the desktop and I've just been too lazy to change em (not had a reason until today!)

Can't understand why it would do this though; any ideas as to why? Never had a situation where a video codec would place anything onto the overlay aside from the decoded video stream during playback, and there's bugger all on Google about this kinda problem.

Off to go play with MPC...

[foxyshadis]

Because 1660 isn't mod16, so there's a difference between the internal coded representation (which has to be mod16) and the external displayed (which can be anything); if the decoder doesn't account for that, you get exactly these types of bugs.

It's very possible that the real reason for the error is similar to an old masktools error, which expected the stride (internal buffering of the raw decoded picture, usually to mod-something) to be mod8 when it became mod16. Might be the same here, where it always expects the decoded size to be mod16, so it displays wrong, but either way it's the same result, a bug.

[christopherw]

Interesting, interesting...

Looks like CoreAVC is to blame as you suspected. I did a bit of housekeeping, rebooted, installed the latest ffdshow-tryout to test its x264 playback, and it played back ok - it just choked my CPU. I reverted back to CoreAVC 1.2 for playback, and now it plays back ok - but upside down (hah). Flipping the picture via the vobsub tray icon solves the problem, but that's only a workaround.

All other H264 video files play back fine, also as you predicted. You were right on the money there :)


No idea why the diagonal green line was there pre-reboot and isn't now (but similarly, no idea why the line should be replaced with an inverted image - it's never happened on video playback before, but no doubt it's due to a bug in CoreAVC as you said).

Anyway, thanks for your input - and I learnt something too, so you efforts weren't wasted ;)

[KoD]

You're making CoreAVC output RGB32 now, that's why you get an inverted image.