madVR - high quality video renderer (GPU assisted) [Archive] - Page 53

Jong

11th May 2010, 11:22

Hmmmm... I've stopped playing Direct3D games a few years ago (due to lack of time), but I don't remember having ever seen games tear. Maybe newer games are different? Maybe they need ultra fast reaction times with 0 lag. In that situation maybe it's preferable to tear over having a small lag caused by proper VSyncing? Don't know... Yes, i think you are on to something. Lag would stop frames being generated too far in advance.

Most players of first person shooters turn off "vsync", so frames are written straight to the front buffer. You get as many frames, or part of frames, as are possible and minimum lag.

In other games you can turn on "vsync" (use of backbuffer), but that means, @60Hz, if the game is capable of generating >60fps you get 60fps. If the frame rate drops below 60fps you end up with 30fps. So this can be underdesirable not just because of lag but because 30fps is not fluid.

djsolidsnake86

11th May 2010, 11:29

i tried with ffdshow decoder and divx decoder h264

Jong

11th May 2010, 11:33

@madshi, the one interesting thing that all the commercial players seem to do that no open source renderers do is that for Blu-ray disc playback (and, for TMT only, DVD playback too) they render EVR onto an overlay surface.

This (according to some quite sophisticated tests done by others (not me!) over at AVS Forums) has the advantage, at least with the latest ATI cards that if you output YCbCr the video is rendered to a YCbCr surface and is output without any conversion to/from RGB at as close as possible to the original values off disc.

I realise one of the main purposes of madVR is to control this conversion to RGB. I'll be honest, we are not sure if setting YCbCr 4:2:2 output avoids chroma upsampling as well (that would be an interesting test to do) and even then I guess madVR probably does a better job than most if not all displays. But I wondered if you were aware of how they are able to use an overlay surface but also use EVR rather than the very old overlay mixer and what you think of this approach! In principle, it seems to come as close as possible on a PC to offering "bit perfect" video output into a high end display or scaler.

madshi

11th May 2010, 11:57

i tried with ffdshow decoder and divx decoder h264
And the crash occurs with both? Then please upload a small sample for me. Thanks.

@madshi, the one interesting thing that all the commercial players seem to do that no open source renderers do is that for Blu-ray disc playback (and, for TMT only, DVD playback too) they render EVR onto an overlay surface.

This (according to some quite sophisticated tests done by others (not me!) over at AVS Forums) has the advantage, at least with the latest ATI cards that if you output YCbCr the video is rendered to a YCbCr surface and is output without any conversion to/from RGB at as close as possible to the original values off disc.

I realise one of the main purposes of madVR is to control this conversion to RGB. I'll be honest, we are not sure if setting YCbCr 4:2:2 output avoids chroma upsampling as well (that would be an interesting test to do) and even then I guess madVR probably does a better job than most if not all displays.

But I wondered if you were aware of how they are able to use an overlay surface but also use EVR rather than the very old overlay mixer and what you think of this approach! In principle, it seems to come as close as possible on a PC to offering "bit perfect" video output into a high end display or scaler.
First of all: I'm not sure how the commercial players are doing that technically. But I'm not really interested in trying to find out, either.

As you say, I'm not sure if ATI's YCbCr 4:2:2 output is "lossless" in the same way e.g. an SDI modded DVD player would output YCbCr 4:2:2. Maybe it is, maybe not. But even if it is:

As you probably know, video is usually compressed in YCbCr 4:2:0. HDMI doesn't support this format (nor does DVI or SDI). Basically it's impossible to transport the original data unchanged from one device to the next with any digital transport that I know. So chroma *always* has to be upsampled to at least 4:2:2. It's possible (maybe even probable) that ATI is upsampling to 4:4:4 and then downsampling to 4:2:2 again.

The "final" aim of madVR is to beat everything else out there, including high end video processors. madVR might not be there yet in some aspects, but that is to be expected, because madVR is still very much a work in progress. I've also read that there are several projectors (even "new" models) which work better with RGB input than with YCbCr input. If madVR output YCbCr, someone would have to convert that back to RGB sooner or later, because all displays are RGB in the end. And the conversion from YCbCr to RGB can easily go wrong. E.g. imagine TMT plays back DVDs, upscales them to 1080p and then outputs them as YCbCr. The display doesn't know this was a DVD. The display will "guess" that this is a HD source and consequently it will likely use Rec.709 to do the YCbCr to RGB conversion. But the correct conversion would be Rec.601, because the source was SD. This is a real problem. Basically if the source device plays SD content and upscales it to HD resolution, you should not output YCbCr. Or if you do, you will get wrong colors.

I prefer to do this all in madVR and feed the display with "final" data, which doesn't need to be processed at all, anymore.

One problem I see with the EVR over Overlay solution is that it once again depends a lot on GPU drivers behaving correctly. As you probably know, being dependent on GPU drivers is every HTPC owner's worst nightmare, because things like that can break with any new driver revision. It may change from one GPU model to the next and from one OS to the next. One aim of madVR is to eliminate as much dependence on the GPU driver, OS, GPU model etc as possible.

djsolidsnake86

11th May 2010, 12:01

yes, both
i will send you a preview files soon

Jong

11th May 2010, 12:42

As you probably know, video is usually compressed in YCbCr 4:2:0. HDMI doesn't support this format (nor does DVI or SDI). Basically it's impossible to transport the original data unchanged from one device to the next with any digital transport that I know. So chroma *always* has to be upsampled to at least 4:2:2. It's possible (maybe even probable) that ATI is upsampling to 4:4:4 and then downsampling to 4:2:2 again.

The "final" aim of madVR is to beat everything else out there, including high end video processors. madVR might not be there yet in some aspects, but that is to be expected, because madVR is still very much a work in progress. I've also read that there are several projectors (even "new" models) which work better with RGB input than with YCbCr input. If madVR output YCbCr, someone would have to convert that back to RGB sooner or later, because all displays are RGB in the end. And the conversion from YCbCr to RGB can easily go wrong. E.g. imagine TMT plays back DVDs, upscales them to 1080p and then outputs them as YCbCr. The display doesn't know this was a DVD. The display will "guess" that this is a HD source and consequently it will likely use Rec.709 to do the YCbCr to RGB conversion. But the correct conversion would be Rec.601, because the source was SD. This is a real problem. Basically if the source device plays SD content and upscales it to HD resolution, you should not output YCbCr. Or if you do, you will get wrong colors.

I prefer to do this all in madVR and feed the display with "final" data, which doesn't need to be processed at all, anymore.

One problem I see with the EVR over Overlay solution is that it once again depends a lot on GPU drivers behaving correctly. As you probably know, being dependent on GPU drivers is every HTPC owner's worst nightmare, because things like that can break with any new driver revision. It may change from one GPU model to the next and from one OS to the next. One aim of madVR is to eliminate as much dependence on the GPU driver, OS, GPU model etc as possible.Yeah, I understand that. Just thought I'd throw it out there!

I do find it interesting they are doing something none of the opensource renderers do and it does have advantages. For example, in confirmation of your comment on GPU driver, the current 5xxx drivers (or maybe the hardware, we do not know) have a small error when they convert RGB to YCbCr (much smaller than a 601/709 conversion error). Because a YCbCr overlay never needs to go through RGB this error is not present when playing Blu-ray's in disc mode (or, for TMT, DVDs). Banding is also much improved, because video is not expanded and then re-compressed to video levels. Of course in theory you could use RGB output to bypass 'the bug', but currently that has other even more serious issues when using conventional renderers, not least it is always expanded at some point (although it can be recompressed if desired). :(

I am aware of the potential for colorspace error when upscaling. In fact there is no colourspace error with DVD in TMT. It must be doing a colorspace conversion along with upscaling. Of course that is not needed for Blu-ray, but I agree there is a good chance that chroma is being upsampled to 4:4:4 before being downsampled again.

madshi

11th May 2010, 13:35

Maybe for some reason this EVR + overlay solution is needed to satisfy those nasty Blu-Ray copy protection rules? Maybe that's why all the commercial software uses the same solution? Just a guess, though...

Jong

11th May 2010, 13:47

Yes, it does prevent screen copy. Although it is not the same as PVP. XP does not have PVP yet it also uses the same approach.

Anyway, I do like the idea of bypassing all Windows "desktop mixing"! Of course madVR (without Aero) will also do this. But, when we are searching for perfection I am nervous of passing RGB unless I know my display cleanly handles RGB input. I realise all displays end up with RGB, but as almost all consumer sources will normally supply YCbCr. RGB is normally there mainly for "old -fashioned" DVI compatibility. I fear some displays are optimised for this. For instance, nearly all displays are applying some post-processing. Do we know this is not done internally in YCbCr? I am sure there are displays that "passthrough" RGB cleanly, but all, or which?!

leeperry

11th May 2010, 13:54

ok, thanks for the detailed explanations Jong, it makes a lot more sense to me now! I agree that HR is not anything special but using its jitter figure, you can somehow "surf" on the VSYNC fliptime...even though it'll never be as smooth as mVR anyway :rolleyes:
Regarding Reclock though, it is not the cause of the problems. As Madshi said:ALL renderers are very tightly constrained in windowed mode.

Only an overlay surface, or exclusive mode or Aero frees up those constraints.

Furthermore, when those constraints are released, Reclock actually helps current renderers to be less time sensitive by positioning average start of presentation as early as possible in the vsync cycle to give it maximum time to complete.
ok! allow me to rephrase: Reclock's timings are too tight to be obeyed in windowed mode on XP...better? :D

but apparently Aero can/would/will help to overcome this problem? together w/ HPET's tightness of course.
Ignoring for a minute Madshi's plans to pre-render multiple frames in advance (which I agree would be awesome

Sometimes playback starts in, or very near, judder and Reclock cannot avoid that. A renderer that correctly controls vsync can hold onto the frame and present it at a safe spot, so judder can be avoided completely, even at the start or after a seek or pause. :)
I can discuss about audio or colorimetry until the end of days, but I'm glad you're so much into VSYNC because it's just very annoying to me :p

good news is: madshi understand where the issue lies, and could possibly offer a way to make Reclock work EVEN in windowed mode? or am I asking for the impossible here?
I've stopped playing Direct3D games a few years ago (due to lack of time), but I don't remember having ever seen games tear.
if you don't set Crysis in "ultra high" mode, I've also never seen a videogame tear or judder...it very much looks like a "hardware VSYNC interrupt" unstoppable train as you said, unlike Reclock in windowed mode on XP :o
So you dislike both Aero and exclusive mode? Don't worry, I'm not planning to drop non-Aero windowed mode. I want to have all modes work as good as possible.
well, one hotkey to go windowed<>FS and we'll be in business.

Jong

11th May 2010, 14:01

good news is: madshi understand where the issue lies, and could possibly offer a way to make Reclock work EVEN in windowed mode? or am I asking for the impossible here?

.........

if you don't set Crysis in "ultra high" mode, I've also never seen a videogame tear or judder...it very much looks like a "hardware VSYNC interrupt" unstoppable train as you said, unlike Reclock in windowed mode on XP.I think in windowed mode (without Aero) we will never be much better than Reclock. As I say, maybe madVR may have 3-4ms more time to play with, but that is unlikely to help when Windows scheduling goes haywire! With Aero, even that 3-4ms is erroded, so I'm honestly not sure whether madVR in Aero mode would be better or worse than Reclock in exclusive mode.

Games in ultra-high mode will be smooth as long as the PC is man enough! Even with vsync enabled, you will not get judder like the synchonised judder we have been discussing, but if the PC frame rate drops below the refresh rate at any time the frame rate drops to refresh rate/2, which is unlikely to be fluid. They do not have a magic solution to Windows scheduling, maybe, as Madshi suggests, because they cannot in practice render many frames in advance for fear of lag.

leeperry

11th May 2010, 14:11

I think in windowed mode we will never be much better than Reclock. As I say, maybe madVR may have 3-4ms more time to play with, but that is unlikely to help when Windows scheduling goes haywire!

Games in ultra-high mode will be smooth as long as the PC is man enough! Even with vsync enabled, you will not get judder like the synchonised judder we have been discussing, but if the PC frame rate drops below the refresh rate at any time the frame rate drops to refresh rate/2, which is unlikely to be fluid. They do not have a magic solution to Windows scheduling, maybe as Madshi suggest because they cannot in practice render many frames in advance for fear of lag.
I never spoke about getting rid of Reclock, to be perfecly clear...as I do need a way to playback audio over KS(KMixer on XP is :eek:) and I also occasionally need to play 25fps@48/96Hz

I'm not too worried about windows scheduling, as I run all the background processes in low prio on single cores and my media player in high prio on the 4 cores...I've also changed the NT affinity system to give 96X more prio to high over low...my issue lies more in the VSYNC fliptime problem as far as I can see :o

oh sure, but a fully capable computer on a non-too demanding game will never tear or judder IME...but again, I always run games in high prio, and this thing's supposed to help too: http://solariz.de/660/aoc-fps-boost-launcher-english.htm

it's a dirty hack, but I think it works :devil:

Mark_A_W

11th May 2010, 14:24

Hmmmm... I've stopped playing Direct3D games a few years ago (due to lack of time), but I don't remember having ever seen games tear.

Interesting,

I don't game anymore (I grew up), but I do stare at CAD all day long, spinning models around with my spaceball. Tearing is endemic in CAD systems. They don't even try to eliminate it.

The typical framerate on a big model may be around 10fps, but varying hugely dependent on load. Matching up to V-sync must be just too hard.

(BTW I've posted on the ZP development forum that an Exclusive mode interface is really, really needed.)

madshi

11th May 2010, 14:38

But, when we are searching for perfection I am nervous of passing RGB unless I know my display cleanly handles RGB input. I realise all displays end up with RGB, but as almost all consumer sources will normally supply YCbCr. RGB is normally there mainly for "old -fashioned" DVI compatibility.
Don't forget PC usage and games! Many displays and even projectors are also being optimized for game (e.g. PS3) usage. And I think Xbox and PS3 probably rather default to RGB output, I'd guess?

I fear some displays are optimised for this. For instance, nearly all displays are applying some post-processing. Do we know this is not done internally in YCbCr? I am sure there are displays that "passthrough" RGB cleanly, but all, or which?!
From what I've read in the projector forums, more projectors are having problems with YCbCr than with RGB. AFAIK, none of the current projectors has any problems with being fed with RGB, while I know of at least 2 projectors which have problems with YCbCr and where experts recommend to feed them with RGB instead.

Jong

11th May 2010, 14:47

I'm not too worried about windows scheduling, as I run all the background processes in low prio on single cores and my media player in high prio on the 4 cores...I've also changed the NT affinity system to give 96X more prio to high over low...my issue lies more in the VSYNC fliptime problem as far as I can see :oIn such a well managed system, Reclock or madVR, with working vsync correction, should be equally good at fixing judder, apart from the possibility with Reclock (1 in 5?) of 1-2 seconds judder at the start, or after a seek or pause.

Once MadVR is working as madshi intends, it will solve that initial judder problem and will either have slightly greater resilience (which you may not need), or massively greater, depending on whether he manages to pre-render several frames, as he intends.

Razoola

11th May 2010, 15:53

Games still tear if vsync is disabled (though triple buffering helps). Games never tear if vsync is enabled correctly.

The difference comparing games to video is your less likly to see the tearing in games due to the much higher framerates and panning that is hardly ever at a constant speed like with video. Your also more concentrated at looking at things to shoot inside the picture than the overall picture. Point being Tearing in games does not stand out as much as tearing in video your watching.

iSunrise

11th May 2010, 17:52

Bug report(s):

(1) When opening or re-opening videos ~5 times with 0.12 and KMP (it doesn´t matter how long you let it play, e.x. 10 seconds) I get the following error:

madVR reports:
- creating gpu 3dlut volume texture failed
- creating gpu 3dlut volume texture failed

madVR settings are:
PC levels
[x] use 3dlut
luma up - bicubic75
luma down - spline64
chroma resampling - bicubic75
(everything else unchecked)

This happens in e.x. windowed-mode _with 3dlut enabled_ and is reproducable every time, even with different files. I´ve replaced madVR.ax 0.12 with madVR.ax 0.11 and there´s absolutely no problem. No crashes anymore.

(2) Another strange behaviour that I came across with madVR (doesn´t happen with EVR or Haali):

When opening or re-opening videos with mpc-hc (tried 4 of the latest builds and one from 2009, just to be sure) the memory usage increases with every video that is loaded. I haven´t tested when or where this ends, but I´ve stopped at a memory usage of about 3.5GB (6GB RAM total). This really doesn´t look like intended behaviour to me. Maybe this is somehow connected to (1). Like with (1) when I did replaced madVR.ax 0.12 with madVR.ax 0.11 the problems are gone. The available system memory gets released after the next video is loaded, like with EVR or Haali.

Here´s also a debug log (madVRdebug.rar) which I stopped right after (1) occured, re-loading the same video exactly 5 times:
http://rapidshare.com/files/386112452/log.rar.html

And even if that sounds like a broken record, thanks again for giving us such a great tool to work with! If you need anything more to track the error(s) down, just ask.

djsolidsnake86

11th May 2010, 18:09

there is an incompatibility with madvr and directvob sub, that make the video crash at the start, anyone experienced this?

namaiki

11th May 2010, 18:12

I could produce the memory leak above, and MadVR eventually crashed, though the crashing might have had more to do with me using all MPC-HC's internal filters.

Windows 7 64-bit MPC-HC 32-bit svn1871
File was H.264

Problem signature:
Problem Event Name: APPCRASH
Application Name: mpc-hc.exe
Application Version: 1.3.1871.0
Application Timestamp: 4be774c9
Fault Module Name: madVR.ax
Fault Module Version: 0.12.0.0
Fault Module Timestamp: 4bde9186
Exception Code: c0000005
Exception Offset: 0000ac43
OS Version: 6.1.7600.2.0.0.256.1
Locale ID: 3081
Additional Information 1: 0a9e
Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
Additional Information 3: 0a9e
Additional Information 4: 0a9e372d3b4ad19135b953a78882e789

there is an incompatibility with madvr and directvob sub, that make the video crash at the start, anyone experienced this?
I had a crash with a 632x480 file. I think it's to do with the video's resolution. Didn't crash when set the option for resized to 640x480 by ffdshow video.

djsolidsnake86

11th May 2010, 18:52

my files that crash are : 712x304 pixels

cyberbeing

11th May 2010, 20:50

VSFilter Input Pin
Filter : DirectVobSub (auto-loading version) - CLSID : {9852A670-F845-491B-9BE6-EBD841B8A613}

- Connected to:

CLSID: {04FE9017-F873-410E-871E-AB91661A4EF7}
Filter: ffdshow Video Decoder
Pin: Out

- Connection media type:

Video: YV12 712x304 23.98fps

AM_MEDIA_TYPE:
majortype: MEDIATYPE_Video {73646976-0000-0010-8000-00AA00389B71}
subtype: MEDIASUBTYPE_YV12 {32315659-0000-0010-8000-00AA00389B71}
formattype: FORMAT_VideoInfo2 {F72A76A0-EB0A-11D0-ACE4-0000C0CC16BA}
bFixedSizeSamples: 1
bTemporalCompression: 0
lSampleSize: 324672
cbFormat: 112

VIDEOINFOHEADER:
rcSource: (0,0)-(712,304)
rcTarget: (0,0)-(712,304)
dwBitRate: 0
dwBitErrorRate: 0
AvgTimePerFrame: 417083

VIDEOINFOHEADER2:
dwInterlaceFlags: 0x00000000
dwCopyProtectFlags: 0x00000000
dwPictAspectRatioX: 89
dwPictAspectRatioY: 38
dwControlFlags: 0x00000000
dwReserved2: 0x00000000

BITMAPINFOHEADER:
biSize: 40
biWidth: 712
biHeight: 304
biPlanes: 3
biBitCount: 12
biCompression: YV12
biSizeImage: 324672
biXPelsPerMeter: 0
biYPelsPerMeter: 0
biClrUsed: 0
biClrImportant: 0
__________
VSFilter Output Pin
Filter : DirectVobSub (auto-loading version) - CLSID : {9852A670-F845-491B-9BE6-EBD841B8A613}

- Connected to:

CLSID: {51B4ABF3-748F-4E3B-A276-C828330E926A}
Filter: Video Mixing Render 9 (Renderless)
Pin: VMR Input0

- Connection media type:

Video: YV12 1024x304 (89:38) 23.98fps

AM_MEDIA_TYPE:
majortype: MEDIATYPE_Video {73646976-0000-0010-8000-00AA00389B71}
subtype: MEDIASUBTYPE_YV12 {32315659-0000-0010-8000-00AA00389B71}
formattype: FORMAT_VideoInfo2 {F72A76A0-EB0A-11D0-ACE4-0000C0CC16BA}
bFixedSizeSamples: 1
bTemporalCompression: 0
lSampleSize: 466944
cbFormat: 1152

VIDEOINFOHEADER:
rcSource: (0,0)-(712,304)
rcTarget: (0,0)-(712,304)
dwBitRate: 0
dwBitErrorRate: 0
AvgTimePerFrame: 417083

VIDEOINFOHEADER2:
dwInterlaceFlags: 0x00000000
dwCopyProtectFlags: 0x00000000
dwPictAspectRatioX: 89
dwPictAspectRatioY: 38
dwControlFlags: 0x00000000
dwReserved2: 0x00000000

BITMAPINFOHEADER:
biSize: 40
biWidth: 1024
biHeight: -304
biPlanes: 1
biBitCount: 12
biCompression: YV12
biSizeImage: 466944
biXPelsPerMeter: 0
biYPelsPerMeter: 0
biClrUsed: 0
biClrImportant: 0

I seem to have run into a problem similar to djsolidsnake86 with VSFilter and madVR in the past.

Just now I tested it again artificially by resizing a video to 712x304 in FFDshow. CRASH. The above are VSFilter's input/output pins when trying to connect a 712x304 video from FFDshow to VSFilter to VMR9. This of course could be different then VSFilter's Output Pin to madVR, but I thought I'd post it anyway.

madVR seems very finicky about how it is feed non-MOD16 width video. If the decoder pads the video to MOD16, it's happy, but if the decoder outputs non-MOD16 video directly and expects the video renderer to pad as necessary, it can sometimes cause madVR to crash. Odd anamorphic aspect ratios can also cause a crash. I suspect this is the same or similar problem as the Youtube FLV sample which caused a crash when using CoreAVC as the decoder.

Here is the debug log (10 seconds): http://www.mediafire.com/?wlumyjyzjmn

Mark_A_W

11th May 2010, 22:19

Directvobsub is working fine for me with madVR.

cyberbeing

12th May 2010, 00:32

Directvobsub is working fine for me with madVR.

Directvobsub is working fine for me as well 99.9% of the time, except for odd cases like above. Test a 712x304 video with VSFilter and you should also get a crash.

Mark_A_W

12th May 2010, 00:44

Ahhh, ok you were being specific, not a general comment about directvobsub.

I only watch 1920x1080, so haven't had that issue.

leeperry

12th May 2010, 02:09

Once MadVR is working as madshi intends, it will solve that initial judder problem and will either have slightly greater resilience (which you may not need), or massively greater, depending on whether he manages to pre-render several frames, as he intends.
well, I think there's also a container problem...because a 25fps MKV gives a dead stable jitter in HR, but a 23.976 MKV is one big jitter feast...getting 25fps@48Hz was dead easy, but 23.976@48 felt like rocket science :confused:

remux your 23.976 MKV to MP4 or AVI, and HR's jitter is dead stable....apparently MKV doesn't have a fixed framerate by design, so it seems to add even more jitter at the splitting stage....ah well watching 23.976fps MKV@48Hz is hopeless, I'm starting to lose faith.

FWIW, KMP's built-in MKV splitter seems far less jittery than Haali's.

djsolidsnake86

12th May 2010, 08:40

Directvobsub is working fine for me as well 99.9% of the time, except for odd cases like above. Test a 712x304 video with VSFilter and you should also get a crash.

but it's a madvr bug, because with other renders there isn't this problem

cyberbeing

12th May 2010, 10:25

but it's a madvr bug, because with other renders there isn't this problem

Correct, madVR is the only renderer which produces a crash when using VSFilter with a 712x304 video. I was implying the use of madVR in my previous posts, but I guess I should have been a bit clearer, sorry.

starla

12th May 2010, 10:31

So will I add a Reclock mode? I'd say: Let's wait and see how well madVR's exclusive mode / Aero solutions will work. And then we can decide whether adding a madVR Reclock rendering mode is worth it.

Just to add some info on this discussion:

1) Jong suggest that Reclock is able to move the ::Present() return point. This is wrong, what reclock is able to do is to move the point that frame is scheduled (reference clock adjustment affects this). Why in some renderers the end position might seem to move is caused by the fact that v-sync is disabled completely (if AERO is on v-sync is always enabled by the OS itself). So when reclock is able to move the end point of presentation it is just moving the start point of presentation on a system where no v-sync is done (tearing is tried to remove by scheduling a sample to be presented on a time when no screen update is done and the newly drawn frame wont update memory while the RAMDAC is "copying the data to screen").

2) Why Reclock (or any other audio renderer that is capable of resampling audio + modifying the reference clock) is required even when you would have perfect video renderer code? Audio and video chips have different time sources (even on latest GPUs that contain audio chip as well). If audio and video clocks drift away more than few ms during the two hour length movie video renderer is required to drop or duplicate a frame since you aren't able to present the video frame on a correct v-sync.

I'm personally currently experimenting with the MPC-HC audio render code base and turning it into open source reclock replacement (no guarantees that it will ever be ready :)). If that wont work we would need to ask James from Slysoft to allow video renreder to advice the reclock on few things to make the video renderers actually compatible with reclock.

--
tourettes / MediaPortal dev

Grmpf

12th May 2010, 10:49

2) Why Reclock (or any other audio renderer that is capable of resampling audio + modifying the reference clock) is required even when you would have perfect video renderer code? Audio and video chips have different time sources (even on latest GPUs that contain audio chip as well). If audio and video clocks drift away more than few ms during the two hour length movie video renderer is required to drop or duplicate a frame since you aren't able to present the video frame on a correct v-sync.

You can eleminate this by having very tight timings, sure thats not very convenient for 99% of the userbase, but there is for example a script out there (www.d65.de - be aware its german ! (http://www.d65.de/wiki/HTPC_Bildfrequenz_Optimierung) - from the author of eventghost), that lets you optimize your timings (ATi hardware + Powerstrip recommended), so you reduce the drift to get no drops/repeats anymore for many hours straight playing. I optimized my timings to get not a single drop/repeat for over 13h playing 23.976/24.000/50.000/59.940hz content.

madshi

12th May 2010, 11:03

1) Jong suggest that Reclock is able to move the ::Present() return point. This is wrong, what reclock is able to do is to move the point that frame is scheduled (reference clock adjustment affects this). Why in some renderers the end position might seem to move is caused by the fact that v-sync is disabled completely (if AERO is on v-sync is always enabled by the OS itself). So when reclock is able to move the end point of presentation it is just moving the start point of presentation on a system where no v-sync is done (tearing is tried to remove by scheduling a sample to be presented on a time when no screen update is done and the newly drawn frame wont update memory while the RAMDAC is "copying the data to screen").
Thanks, that makes sense.

2) Why Reclock (or any other audio renderer that is capable of resampling audio + modifying the reference clock) is required even when you would have perfect video renderer code? Audio and video chips have different time sources (even on latest GPUs that contain audio chip as well). If audio and video clocks drift away more than few ms during the two hour length movie video renderer is required to drop or duplicate a frame since you aren't able to present the video frame on a correct v-sync.
Yes, I'm aware of this problem. I do have some ideas on how to work around the problem in madVR alone, without needing audio manipulation. But I'm not sure yet how well it will work, and I also don't know how long it will take for me to find the time to actually implement my ideas.

I'm personally currently experimenting with the MPC-HC audio render code base and turning it into open source reclock replacement (no guarantees that it will ever be ready :)).
Well, that's *great* news! I was kind of wishing for someone to start investing time into creating a high quality audio renderer. So I'm happy to hear about your plans.

If you don't mind, I'd like to make a suggestion: Maybe you could start by making the MPC-HC audio renderer work as well or better than any other renderer, before starting to add Reclock like logic to it? As far as I've read, currently some audio renderers are better than others, depending on some specific circumstances. If you could make the MPC-HC audio renderer best in all circumstances, that would be a great start!

Let me point you to a problem with Reclock. This is a quote from an email yesgrey sent me. I think (hope) yesgrey won't mind me posting it here:

The problem with reclock, and the only thing that still bothers me and could make me switch for another audio renderer in the future, is that it cannot use the soundcard's clock as the reference clock.
Reclock creates its own reference clock from the System clock, and then the video is synchronized with it and the audio is resampled to keep the sync with video.
When we bypass the resampling, the audio clock is not synchronized with the video clock, so, it will only be in sync with video if the soundcard clock does not deviate too much from the video clock.
That's not the ideal solution, but there is not any better option available.
The Microsoft Audio Renderer, works exactly like I want, using the soundcard clock as the reference, but it doesn't use Kernel Streaming nor WASAPI Exclusive, like Reclock does, so it cannot bypass kmixer, and to have audio resampled, I prefer Reclock.;-)
With my current method, I might get one or two audio glitches while watching a movie, when reclock has to drop a few audio samples to resync the audio with the video, but it's not an issue.
I'm still waiting for the perfect Audio Renderer. If I knew how, I would write it myself exactly like I have described above...
Now MPC-HC also has its Audio Renderer and uses WASAPI, but it seems to not be working like I want to.
Would be great if you could make sure that the MPC-HC audio renderer properly uses the audio hardware's clock (in "non-Reclock mode"). Not sure which clock the MPC-HC audio renderer currently uses. Of course for a Reclock clone you'd need to use your own clock instead of the audio hardware clock, but for people who don't want audio to be resampled (I'm one of them), it would be great if "your" new audio renderer could be switched to a "non-Reclock" mode, which should then behave as normal renderers do (using the audio hardware clock).

yesgrey

12th May 2010, 11:50

Let me point you to a problem with Reclock. This is a quote from an email yesgrey sent me. I think (hope) yesgrey won't mind me posting it here:
No problem.:)
I just want to add that I still think reclock is a great audio renderer, and essential for people who wants smooth video.
There are two kind of users: those who prefer small glitches in audio and none in video (reclock users), and those who prefer small glitches in video and none in audio, and for this last group (where I'm included), there isn't any solution available, yet.

Jong

12th May 2010, 13:04

Just to add some info on this discussion:Hi tourettes, good to see you here.....Jong suggest that Reclock is able to move the ::Present() return point. This is wrong, what reclock is able to do is to move the point that frame is scheduled (reference clock adjustment affects this). Why in some renderers the end position might seem to move is caused by the fact that v-sync is disabled completely (if AERO is on v-sync is always enabled by the OS itself). So when reclock is able to move the end point of presentation it is just moving the start point of presentation on a system where no v-sync is done If this is what you took from my posts it is a misunderstanding. I believe what you and I are saying is the same. Specifically I said:By default all such renderers appear to do as I suggested a few days ago. The first frame appears ASAP after it is 'ready' and frames follow an appropriate period later, with no regard to vsync.

As a consequence of both the above "end present" also bears no regard to vsync and, without help, all such renderers are vulnerable to synchronised judder if using a front and back buffer.

By varying the reference clock ever so slightly (say 0.1%, actually the adjustment is greater the further from the target the current measured position is) Reclock is able to move the point frames are ready/due, hence "Present()", hence the point this returns. Once the measured position is inside the Reclock target correction is turned off and the frame rate returns to normal.I believe that is the same as what you are saying. If you can point to a post that says Reclock explicitly defines the point "Present()" returns, or implies it, I will happily change it. However, what is definitely true is the way it decides it needs to make a correction is by measuring "end of presentation" and if this point is pinned, eg. to vsync, as in windowed mode, Reclock vsync correction cannot work.
.....So when reclock is able to move the end point of presentation it is just moving the start point of presentation on a system where no v-sync is done (tearing is tried to remove by scheduling a sample to be presented on a time when no screen update is done and the newly drawn frame wont update memory while the RAMDAC is "copying the data to screen").If I am reading this right then this is wrong.

The confusion comes because there are two things people mean when they talk of "controlling vsync".

To some this means avoiding tearing e.g. by use of a back buffer and flipping front and back buffers during vsync. This is typically what is meant by the "vsync" setting in GPU drivers. Whilst, normally, eliminating tearing this directly causes the problem of synchronised frame rate/refresh rate judder for video (http://software.intel.com/en-us/articles/video-frame-display-synchronization/). When Reclock talks of controlling vsync, and what we are mostly discussing here, is ensuring presentation is not so close to this flip that synchronised judder can occur.

Reclock vsync correction is incompatible with renderers that either always return from"Present()" after vsync (i.e. normal windowed mode) or, if not, if they themselves attempt to position "Present()" and thus themselves pin its return a fairly fixed duration afterwards. However, if "Present()" returns immediately after it is complete as in exclusive or Aero modes it is perfectly fine that a back buffer and hardware flip is also used to avoid tearing.

So, using Reclock vsync correction does not require that presentation occurs during vBlank. In fact specifically it requires that it occurs a safe distance away from vBlank. This safe position is much more tolerant and carries a much lower risk of tearing. The "Present()" call can basically occur anywhere from about -18ms to -6ms @50Hz, or ~-14ms to -6ms @60Hz, with current compatible renderers and their workload (MadVR may need a little more margin at the lower end?)

bozydar

13th May 2010, 00:25

From what I've read in the projector forums, more projectors are having problems with YCbCr than with RGB.

I cannot comment on projectors, but all tv-sets that I tested (Pioneer’s Kuros, two Panasonic plasmas, couple of LG and Samsung LCDs) process pictures in YCbCr space. I am not sure about Panasonic, but rest of them are using YCbCr 4:2:2 or even 4:2:0 by default. There is much easier to do things like contrast, hue, saturation or basic CMS in YCbCr space. Also all the normal sources (DVD, Blu-ray, satellite tv, etc.) are using at best YCbCr 4:2:2, so there is no need to waste silicon on full processing in 4:4:4. Kuros and some Samsungs can be switched to 4:4:4 mode, but this disables almost all controls on tv (like CMS). I am not sure if this disables conversion from RGB to YCbCr…

RGB to YCbCr (4:2:2 or 4:2:0) conversion can be easily confirmed by playing FULL-HD full resolution color test patterns on any FULL-HD TV in 1:1 pixel mapping mode.

Hypernova

13th May 2010, 03:54

With 0.12 I can play 1080p on my monitor (2560x1600) using spline64 "smoothly" (not smooth as in what others are talking about, just no obvious skipping frames) using my 3870. This is super awesome. Thank you madshi!

Now I only have to wait for subtitle (pin) support and I'll be a happy panda.

leeperry

13th May 2010, 04:58

yes, 0.12 is really solid here...no freezing on seek, no more error msg in the event viewer...just pure bliss, the smoothness is really extraordinary...and did I mention the colorimetry http://forum-images.hardware.fr/images/perso/3/hide.gif

my CRT has almost a perfect SMPTE-C gamut, so I don't even bother w/ a LUT..96Hz in Reclock/KS and the PQ/SQ are just astounding http://forum-images.hardware.fr/images/perso/alhex.gif

I'm forced to use this on upscaled SD, but I guess that's fine?
ColorMatrix(mode="Rec.601->Rec.709",clamp=0,opt=0)

madshi

13th May 2010, 08:06

I cannot comment on projectors, but all tv-sets that I tested (Pioneer’s Kuros, two Panasonic plasmas, couple of LG and Samsung LCDs) process pictures in YCbCr space. I am not sure about Panasonic, but rest of them are using YCbCr 4:2:2 or even 4:2:0 by default. There is much easier to do things like contrast, hue, saturation or basic CMS in YCbCr space.
True. But do these displays convert RGB input back to YCbCr? Or do they disable brightness/contrast/CMS controls with RGB input? Even if they do convert RGB back to YCbCr and then back to RGB, at least they will probably use the same matrix for both conversions, so the colors should stay correct.

Also all the normal sources (DVD, Blu-ray, satellite tv, etc.) are using at best YCbCr 4:2:2, so there is no need to waste silicon on full processing in 4:4:4. Kuros and some Samsungs can be switched to 4:4:4 mode, but this disables almost all controls on tv (like CMS). I am not sure if this disables conversion from RGB to YCbCr…
If they are converting from RGB back to YCbCr 4:2:2, that means that some part of madVR's chroma upsampling quality is lost. That would be really sad. I do think that newer processing chips (future versions of Gennum VXP and HQV Realta chips etc) are likely to go 4:4:4, probably even 16bit. So I think in the long run downconverting to YCbCr 4:2:2 in the display should be a thing of the past. If you think about it, it should be considered a display "bug", if it converts 4:4:4 input to 4:2:2. After all, if a computer is connected to the display, the computer sends 4:4:4. If an XBox or PS3 is connected, they also usually send 4:4:4, too. So basically a display which internally downconverts all input data to 4:2:2 should be considered faulty, cause it definitely loses quality for computer and game input content.

Now I only have to wait for subtitle (pin) support and I'll be a happy panda.
Well, I've asked in this thread and in the MPC-HC thread about information about how a subtitle pin is supposed to work, but nobody bothered to provide me with *any* information. So right now I've no idea what madVR needs to do, technically, to provide that subtitle pin.

I'm forced to use this on upscaled SD, but I guess that's fine?
ColorMatrix(mode="Rec.601->Rec.709",clamp=0,opt=0)
I don't know what ColorMatrix does exactly, so I can't comment on that.

Hypernova

13th May 2010, 08:36

Well, I've asked in this thread and in the MPC-HC thread about information about how a subtitle pin is supposed to work, but nobody bothered to provide me with *any* information. So right now I've no idea what madVR needs to do, technically, to provide that subtitle pin.

Yes, I've seen them and worried about this getting lost in all other technicals discussiong, that's half the reason why I post my result with 0.12.

Hopefully someone can help you with that soon. (maybe tetsuo55?)

XhmikosR

13th May 2010, 12:37

Well, I've asked in this thread and in the MPC-HC thread about information about how a subtitle pin is supposed to work, but nobody bothered to provide me with *any* information. So right now I've no idea what madVR needs to do, technically, to provide that subtitle pin.
Sorry about that, but the people who can provide you this kind of info are very few. You can try contacting Casimir or ar-jar.

madshi

13th May 2010, 14:26

Sorry about that, but the people who can provide you this kind of info are very few. You can try contacting Casimir or ar-jar.
Thank you! I'll do that...

Keiyakusha

13th May 2010, 17:27

I'm forced to use this on upscaled SD, but I guess that's fine?
ColorMatrix(mode="Rec.601->Rec.709",clamp=0,opt=0)

Yes. The same thing with EVR. Renderer doesn't know if stream upscaled or not...

pirlouy

13th May 2010, 20:11

Sorry about that, but the people who can provide you this kind of info are very few. You can try contacting Casimir or ar-jar.

I don't think ar-jar deals with subtitles. He just uses default subtitles integration for EVR.

The truth is there is not really someone who works on subtitles. Tetsuo55 or clsid already explains that even the actual system is buggy, even if it works. Maybe MPC-HC changelog can help Madshi, but I'm not sure.

There's this project which can help: http://blog.aegisub.org/2009/02/kumaji-explained.html?showComment=1240844640000#c3378228634171132037

madshi

13th May 2010, 21:20

@pirlouy, I don't really want to write a full subtitle renderer. I just want to offer whatever is needed for MPC HC's internal subtitle renderer to work together with madVR. Currently that doesn't seem to work. Also, the MPC HC's OSD doesn't seem to work with madVR, either. I guess that the MPC HC OSD uses the same renderer input pin as the internal subtitle renderer, but I'm guessing here...

bozydar

13th May 2010, 23:38

If you think about it, it should be considered a display "bug", if it converts 4:4:4 input to 4:2:2. After all, if a computer is connected to the display, the computer sends 4:4:4. If an XBox or PS3 is connected, they also usually send 4:4:4, too. So basically a display which internally downconverts all input data to 4:2:2 should be considered faulty, cause it definitely loses quality for computer and game input content.

Well in reality you lose almost nothing. On Xbox and PS3 almost all “high end graphics” games render at 720p or even lower resolution and upconvert graphics to output resolution. This has much greater impact on quality than 4:2:2 processing. Even if some games use FULL-HD resolution they do not render enough information to show difference between 4:4:4 and 4:2:2 (low resolution textures, anti-aliased edges, even procedural texturing cannot run at full res, because of the aliasing).

Only PC-users using they TVs as main monitors will see difference between 4:4:4 and 4:2:2 and only in really bad cases (like red text on black background). And they are less than 0.1% of the market...

leeperry

14th May 2010, 01:33

I just want to offer whatever is needed for MPC HC's internal subtitle renderer to work together with madVR. Currently that doesn't seem to work. Also, the MPC HC's OSD doesn't seem to work with madVR, either. I guess that the MPC HC OSD uses the same renderer input pin as the internal subtitle renderer, but I'm guessing here...
same goes for KMP's OSD...Haali always seems to idle in the MPC-HC irc channel, maybe he could give you some hints? it's kinda weird that nothing on the MS dev. sites give info about this?!

KMP is able to use a plain D3D surface as VR, and subs do work...I guess its coder got the implementations infos from somewhere. Haali took part in VRM9's development, but kyh sure as hell didn't.

madshi

14th May 2010, 08:03

Well in reality you lose almost nothing. On Xbox and PS3 almost all “high end graphics” games render at 720p or even lower resolution and upconvert graphics to output resolution. This has much greater impact on quality than 4:2:2 processing. Even if some games use FULL-HD resolution they do not render enough information to show difference between 4:4:4 and 4:2:2 (low resolution textures, anti-aliased edges, even procedural texturing cannot run at full res, because of the aliasing).
There will surely be no big loss in the moving parts of the image. However, games often have static areas (score etc) with colored sharp edges, and there 4:4:4 vs. 4:2:2 should make a difference.

Only PC-users using they TVs as main monitors will see difference between 4:4:4 and 4:2:2 and only in really bad cases (like red text on black background).
I think you are underestimating the impact of downscaling chroma. Sure, in photos, video etc the difference is very small. But in computer type graphics (colored lines with 1 pixel width) chroma subsampling can have a noticeable blur effect. E.g. if I just look at my browser right now, the forum URL links at the top of the page are light brown on light grey with a 1 pixel wide font. I'm quite sure that these links would look blurred with 4:2:2, because the brown color would leak into the grey background.

Edit: I've just checked. See here:

http://www.beststuff.com/audiovideo/silicon-optix-announces-the-realta-chip-bringing-hollywood-quality-video-home.html
True to its professional heritage, Realta offers complete input to output 4:4:4 color processing and a full 10-bit or better internal data path

http://www.digitalconnection.com/products/video/vps3300.asp
True 10bit 4:4:4 broadcast grade processing delivering the worlds most advanced and flexible processor.

So both Gennum VXP and Realta are doing full 4:4:4 processing.