Haali Renderer

[leeperry]

there u go :

well as my PJ attached to DVI connectors. Since I have been upgrading my
video driver I notice crushed video VMR9 levels on my calibrated PJ. My
first thought was, ok a bad edited DVD. But, I double checked then with
well known flics, unfortunately, same result.

Nvidia mentions a work around for their very cool "new detection
feature"

http://download.nvidia.com/Windows/8...ease_Notes.pdf

--------------------------------------------------------------------------------
Video color-space range for DVI-only1 outputs is erroneously set to
standard mode (16-235) instead of extended mode (0-255).
A new detection feature to apply Standard CSC mode to TV outputs
(including NTSC, PAL, 480i, and 576i), included DVI-only outputs by
mistake.
Note: The driver correctly applies extended mode to analog outputs, and
standard mode to TV outputs (including NTSC, PAL, 480i, and 576i).
A future driver release will correct this and apply the extended-mode color
space to DVI-only outputs.
You can work around this issue by forcing either standard or extended mode
as follows:
1 Launch regedit and determine the current primary display card by
looking in
HKey_Local_Machine\Hardware\DeviceMap\Video
and note the GUID (global unique identifier assigned by Windows),
which is the long string in brackets { } at the end of the entry
"\device\video0".
2 Look in
HKey_Local_Machine\SYSTEM\CurrentControlSet\Control\Video\
{GUID}\0000
where {GUID} is the number derived from the previous step.
3 Open the "0000" directory and create a new DWORD called
VMRCCCSStatus and give it a value of
0x3 - to force use of the standard YUV range of 16-235
0x1 - to force use of the extended YUV range of 0-255
--------------------------------------------------------------------------------------------

So, I did apply VMRCCCSStatus=1 for both DVI heads and after a reboot all
went back to full PC-Level range!

[qyqgpower]

I think YUV range shoudn't be extended since renderer should do the same thing when finnally output the frame on PC monitor in RGB format

[wozio]

Originally Posted by leeperry

as far as I'm concerned I won't be throwing MPEG2 frames to HR....too cumbersome with the double TV>PC conversions(one from the MPEG2 decoder, and one from HR) and all...

and I'm not sure HR is supposed to cope with that.

Decoders doesn't and shouldn't do TV->PC conversion, this is done in renderer during yuv to rgb conversion or in filters like ffdshow. One decoder which doesn't some kind of tv->pc scale conversion is coreavc but I don't knowhow it is done there since it works only in direct connection to VMR9, if you throw some other filter between, there is no effect in this.

Originally Posted by leeperry

his job is too provide us with a playback smooth as butter and do the RGB32 conversion with the PS 2.0.......and it does it wonderfully

This applies only for progressive sources, Haali himself wrote me that he doesn't care about deinterlacing because he doesn't watch interlaced sources. What is the effect you can see on my snapshots.

And also this yuv to rgb conversion, as you can also see this is not that big difference comparing to VMR9.

I would rather say that HR is best in resizing area, but on my samples you can't see it since they are 1:1 without resizing in renderer.

[Seb.26]

Originally Posted by wozio

This applies only for progressive sources, Haali himself wrote me that he doesn't care about deinterlacing because he doesn't watch interlaced sources.

IMO : deinterlacing in HR is useless ... this must be done by FFDShow before first resizings/post processing ...

[Leak]

Originally Posted by Seb.26

IMO : deinterlacing in HR is useless ... this must be done by FFDShow before first resizings/post processing ...

Also, simply deinterlacing is not a great idea on telecined material (i.e. video with 3:2 pulldown applied) - and I never managed to get ATI's 3:2 pulldown removal/deinterlacing to work correctly on my Radeon X1900...

That's why I'm currently working on a patch for ffdshow to allow AviSynth and TIVTC to be used on the fly; it's not 100% finished, but it works already.

[wozio]

Originally Posted by Leak

Also, simply deinterlacing is not a great idea on telecined material (i.e. video with 3:2 pulldown applied) - and I never managed to get ATI's 3:2 pulldown removal/deinterlacing to work correctly on my Radeon X1900...

Thing is that not all mpeg2 decoders outputs frames in format which allows VMR9 to use dxva deinterlacing, I haven't had succes trying ffdshow. I use dscaler5 or cyberlink mpeg2 decoders and it works really good. Cyberlink only in dxva mode, dscaler5 when outputting nv12 colorspace and with deintelacing set to automatic.

Problem is that if you put ffdshow between decoder and vmr9 for postprocessing then deinterlacing will not work since as far as I know ffdshow doesn't preserve frames' interlacing flags (videoinfoheader2?). Also I don't know if ffdshow's filters and postprocessing work correctly on interlaced frames, I don't think so.

And also VMR9 must be in mixing mode for deinterlacing (yu can control this in MPC). And in ATI driver 3:2 pulldown is disabled by default, you must enable it via CCC or ati tray tools (much better to be honest and lightweight).

Originally Posted by Leak

That's why I'm currently working on a patch for ffdshow to allow AviSynth and TIVTC to be used on the fly; it's not 100% finished, but it works already.

I will try this and compare.

[Leak]

Originally Posted by wozio

I use dscaler5 or cyberlink mpeg2 decoders and it works really good. Cyberlink only in dxva mode, dscaler5 when outputting nv12 colorspace and with deintelacing set to automatic.

Yeah, I did that, but while it deinterlaced somewhat it also left some strange streaks in the chroma. Granted, it's been some time since I last tried this, but the result wasn't pretty.

Also, is 3:2 pulldown removal done by analyzing the video or just by relying on the interlace flags? If it's the latter, I have a few DVDs here where this plain won't work...

[wozio]

Originally Posted by Leak

Yeah, I did that, but while it deinterlaced somewhat it also left some strange streaks in the chroma. Granted, it's been some time since I last tried this, but the result wasn't pretty.

You mean something like that:
http://www.ixbt.com/video2/images/vi..._1_ati_ati.jpg
?

If yes then this is known issue with denoising in ati drivers and is possible to switch off:

http://www.avsforum.com/avs-vb/showt...3&page=1&pp=30

Originally Posted by Leak

Also, is 3:2 pulldown removal done by analyzing the video or just by relying on the interlace flags? If it's the latter, I have a few DVDs here where this plain won't work...

As fas as I know both:
http://www.anandtech.com/video/showdoc.aspx?i=2551&p=7

And as you can see this is very old article, there are some newer ones.

[Leak]

Originally Posted by wozio

You mean something like that:
http://www.ixbt.com/video2/images/vi..._1_ati_ati.jpg
?

If yes then this is known issue with denoising in ati drivers and is possible to switch off:

http://www.avsforum.com/avs-vb/showt...3&page=1&pp=30

Looks like that, yeah...

I'll give it another shot when I get time. Thanks for the info!

[leeperry]

Originally Posted by Seb.26

IMO : deinterlacing in HR is useless ... this must be done by FFDShow before first resizings/post processing ...

exactly

linear blending must be done prior to any other filters..otherwise you'll end up with jaggies

[wozio]

I did some checks and results are quite interesting. I compared yuv to rgb color conversion routines of three popular methods:

- ffdshow - hq yuv to rgb conversion checked and forced rgb output to vmr9
- haali renderer - hr doesn't want to connect to ffdshow on my machine god knows why, so I've used coreavc here but I checked on vmr9 and decoding results from coreavc and ffdshow were identical
- vmr9 - yuv mixing mode enabled, yv12 output from ffdshow

I used ati tray tools to make screenshots.

Here it is:
- ffdshow:


- haali:


- vmr9:


As you can see all methods are quite similar, differences are visible but not big. I won't choose any of them since I don't have reference to compare.

But thing are getting interesting when we feed VMR9 with HD source or at least with 720 lines or more (I used the same clip resized in ffdshow to 1920x1080).

- vmr9 hd:


It seems that ATI drivers apply other equations to convert in hd size than in sd size. Once more it is hard to say which method is better.

Regards
Piotr

[leeperry]

nice screenshots, but I'm on my beamer atm so it's hard to examinate.

OTOH, I can confirm that this goddamn nvidia card was outputting 16-235 to my SANYO Z3 beamer

I've forced 0-255, and now the quality is just jaw dropping with that registry hack and HR..

I'm stunned, perfect colors over HDMI in 720p, lanzcos 8 passes resize and the very natural "basic" sharpen filter of KMPlayer........which turns SD in HD with one click.........and HR makes it so freakin' smooth

[leeperry]

only last issue, the last horizontal line of the image is made of green artefacts....they don't show up when I make screenshots....and it's due to HR, not my other filters..

any clue ?!

Thanks,

[HeadBangeR77]

@ wozio:
Either I'm totally blind or the first screen grab using ffdshow's conversion is much more colourful in terms of intensity and saturation, while the other two remain a bit washed-out and dimmed, as it's typical for VMR9 (unless both nVidia and ATI finally fix their 2d drivers )? Excuse, but I do see a huge difference between those screen shots. If was to sort them in terms of intensity of colours, I would say the coveravc does the worst job here, since there's a difference between it and the VMR9 + YUV mixing mode (a bit better imho).

@ all
As to HD sources, we've had some interesting discussion here:
http://forum.doom9.org/showthread.ph...726#post972726

it really does depend on the resolution.

cheers,
HDBR77

PS. Ooops, I've just realized it's been OT.

[leeperry]

ooooooh....the HR screenshot looks washed out indeed..

[wozio]

ffdshow is very very close to vmr9 in sd mode, haali is slightly different it clearly uses other color matrix. CoreAVC has nothing to tell here since it only decodes the stream not colors. I compared coreavc with ffdshow both connected to vmr9 and frames were identical, I couldn't see any difference. So all changes we are seeing are because of haali renderer's color conversion.

Anyway there is possibility to pass to color converter (renderer in our case) color matrix used in stream but problem is that no decoder supports it.

One more time: VMR9's conversion is up to the drivers, on other driver version result could be different, the same on nvidia cards and drivers. These results are for catalyst 7.3. Here is real strenght of HR and ffdshow. It is not dependent on driver because it uses it's own method. On different cards results should be the same.

[leeperry]

but you got an ATI card.......colors don't look washed out on my 6800GT using the latest XP drivers.

and anyone's getting green artefacts on the last horizontal line when using HR ?

[wozio]

Originally Posted by leeperry

but you got an ATI card.......colors don't look washed out on my 6800GT using the latest XP drivers.

You mean using haali renderer? On my ATI they aren't also they are slightly different but not washed out.

[leeperry]

well on your screenshots, they obviously do ?!

[aydc]

Isn't this color business a bit off-topic for this thread?