ffdshow tryouts project: Discussion & Development

[thewebchat]

Why does ffdshow increase CPU usage when accepting raw video, even when no colorspace conversions or filters are enabled?

[albain]

Originally Posted by G_M_C

Actually, it seems strange to ask here, but when i look at the graphs i can make and what i can deduce ... ffdshow doesnt need to be in the chain, or not ? There is no actual decoding going on here, its just bitstreaming. In passthrough mode it just does that, pass a signal though untouched. Isnt it just simpler (and more true to the idea of PAP) to connect to the demuxer directly ? This is what ArcSofts audio renderer does, it connects directly to the demuxer. Is it possible to connect reclock directly to AS MPEG demuxer ?

Well on theory you're right, no decoder is needed.

However, there are 2 reasons why a decoder and especially ffdshow will be useful :
1/ The splitters/source filters sometimes mess up with the detection of input format, they would detect AC3 instead of TrueHD or EAC3. FFDShow integrates an internal parser that will scan the stream to detect the right format
2/ Select what to bitstream or not : with FFDShow as for AC3/DTS, you can select what to bitstream and what to decode


@Rica : thank you for your post. I have seen the post which talks about the WAVEFORMATEXTENSIBLE_IEC61937 structure that I tried to implement a while ago. The thing is that this structure is not used at all today (I am sure about that) : because support has been brought by windows 7, so Arcsoft and Cyberlink did without it.
So the post is speculating about this, current hdmi 1.3 audio cards use another way and this is not sure that next radeon will do.
Another thing interesting : arcsoft brought support to radeon 5xxx whereas bitstream support was brought on powerdvd.
So this means that the way to implement bitstream is not very complicated (no hidden APIs)

[G_M_C]

Originally Posted by albain

Well on theory you're right, no decoder is needed.

[...]
@Rica : thank you for your post. I have seen the post which talks about the WAVEFORMATEXTENSIBLE_IEC61937 structure that I tried to implement a while ago. The thing is that this structure is not used at all today (I am sure about that) : because support has been brought by windows 7, so Arcsoft and Cyberlink did without it.
So the post is speculating about this, current hdmi 1.3 audio cards use another way and this is not sure that next radeon will do.
Another thing interesting : arcsoft brought support to radeon 5xxx whereas bitstream support was brought on powerdvd.
So this means that the way to implement bitstream is not very complicated (no hidden APIs)

Maybe it IS the WAVEFORMATEXTENSIBLE_IEC61937 that is used, and is that why it's reasonaby simple

[whurlston]

Originally Posted by G_M_C

Maybe it IS the WAVEFORMATEXTENSIBLE_IEC61937 that is used, and is that why it's reasonaby simple

I don't think so. I downloaded the 9.10 beta drivers and the HDMI audio driver is 9.8 version again. It does not have WAVEFORMATEXTENSIBLE_IEC61937 support. If it did, we could use albain's beta5 version.

I'm vladd by the way.

Edit: I was mistaken, the driver is newer so it might support WAVEFORMATEXTENSIBLE_IEC61937 on the 58xx series. It does not on the 46xx series.

[FoLLgoTT]

Originally Posted by Andy o

ReClock tweaks the reference clock instead.

Please read ReClock's readme file. Everything is explained very well in this document.

Instead of slaving the reference clock to audio (like the DirectShow mechanism) ReClock synchronizes the generated reference clock to video refresh rate to avoid judder. When video and audio will go out of sync for a specific amount of time (configurable in Reclock's options) audio packets get dropped/repeated or PCM get resampled. This is always necessary when video and sound card use different hardware clocks. This is the case with a dedicated sound card.
When using standard timings and HDMI output of the Radeon both chips may use the same hardware clock and may always stay in sync. But this is just a guess. I don't know if this is true.

If it is understood that ReClock synchronzies reference clock to the video card the next step is to tweak the refresh rate to give as few audio packet drops/repeats as possible! The perfect absolute refresh rate is different for every system, because the hardware clocks of video and sound cards differ slightly. For one HTPC 47,951Hz may be good, for others may be 47,953Hz etc.

Btw. the vsync correction of ReClock fights against a completely different problem in PC architecture. This important feature makes sure that the picture never arrives near the vertical sync signal. This also avoids judder, but of a different cause than described above.

Which video card do you have that gives you dropped/repeated frames?

Right now I use a Radeon 3850 with 47,952Hz. I cannot use standard timings, because the flickering would be to strong with my Sony G90. As I stated I have no dropped/repeated audio packets for many hours because of a tweaked refresh rate. And with tweaked vsync correction also no judder.

My point was just that ReClock treats AC3/DTS packets and PCM differently when trying to sync video and audio which is necessary on usual PC hardware. So a hd bitstream has to be treated as packets and not to be resampled.

Sorry for the offtopic here.

[Andy o]

Originally Posted by FoLLgoTT

Please read ReClock's readme file. Everything is explained very well in this document.

Instead of slaving the reference clock to audio (like the DirectShow mechanism) ReClock synchronizes the generated reference clock to video refresh rate to avoid judder. When video and audio will go out of sync for a specific amount of time (configurable in Reclock's options) audio packets get dropped/repeated or PCM get resampled. This is always necessary when video and sound card use different hardware clocks.

Does the "slave reference clock to audio" option not mean that? I think it does.

From the readme:

You also have the ability to disable completely the system and audio clocks adaptation using the checkbox “slave reference clock to audio”. Doing so, ReClock will work much like the default DirectSound audio renderer, but rate adaptation will still function and reference clock will be slaved to audio clock with a smooth algorithm.

Originally Posted by FoLLgoTT

This is the case with a dedicated sound card.
When using standard timings and HDMI output of the Radeon both chips may use the same hardware clock and may always stay in sync. But this is just a guess. I don't know if this is true.

This might or might not be the cause for my lack of problems with sync, but it's a different issue altogether.

Btw. the vsync correction of ReClock fights against a completely different problem in PC architecture. This important feature makes sure that the picture never arrives near the vertical sync signal. This also avoids judder, but of a different cause than described above.

I don't use VSync on ReClock, I haven't found that necessary. I was talking about VSync on EVR-custom in MPC-HC. Don't find it necessary either, and I don't experience tearing at all in Win 7 (used to in Vista though, but I don't know for sure if the OS is the culprit here).

[G_M_C]

On the ASIO drivers. I'ts correct that the Xonars have the abillity for ASIO 2.0. I have no idea how that corresponds to the actiual output, but in theory you could use the ASIO interface to stream out PCM @ 24 bits/192 KHz. While it's not exactly the same as streaming the losselss codecs, technically it's almost the same (decoding only happens somewhere else).

Does someone have a current version of Sony's Soundforge MT or Audition (or some other Multichannel audio-programm) installed ? See if the ASIO interface does indeed stream high bitrate/resolution PCM in 8 channels.

[FoLLgoTT]

Originally Posted by Andy o

Does the "slave reference clock to audio" option not mean that? I think it does.

The option "Slave reference clock to audio" disables ReClock's mechanism to sync the reference clock to video card's refresh rate. So when video and audio run out of sync you get the same judder as without ReClock. The "(not recommended)" behind this option is a hint that ReClock's purpose to avoid video judder will be disabled. Instead you get bit perfect sound, but no judder-free video.
Btw this judder is so small that most people even don't notice it. I use HTPC's over 10 years now and ReClock since the first released version. After a period of time you get very sensitive to micro judder.

I don't use VSync on ReClock, I haven't found that necessary. I was talking about VSync on EVR-custom in MPC-HC.

Ok, this was a misunderstanding.

Don't find it necessary either, and I don't experience tearing at all in Win 7 (used to in Vista though, but I don't know for sure if the OS is the culprit here).

Tearing is no problem with Aero activated under Vista/windows 7 using VMR9/EVR, because vertical synchronization is always turned on by the OS. But the vsync correction of ReClock has a different purpose. It completely eliminates micro judder if set up correctly as described in the readme. Even without tearing this feature is a must to completely get rid of judder.

[buletti]

Originally Posted by G_M_C

On the ASIO drivers. I'ts correct that the Xonars have the abillity for ASIO 2.0. I have no idea how that corresponds to the actiual output, but in theory you could use the ASIO interface to stream out PCM @ 24 bits/192 KHz. While it's not exactly the same as streaming the losselss codecs, technically it's almost the same (decoding only happens somewhere else).

Does someone have a current version of Sony's Soundforge MT or Audition (or some other Multichannel audio-programm) installed ? See if the ASIO interface does indeed stream high bitrate/resolution PCM in 8 channels.

I tested the ASIO device by playing DVD-Audio in foobar:
Multi channel MLP from DVD-A -> Foobar2000 DVD-Audio Decoder -> highres multi channel PCM -> Xonar ASIO device (2ch) via Foobar2000 -> Xonar HDAV Center (resampler) -> AV receiver via HDMI.
The result at the receiver was 2 channel PCM. Sampling rate and channel count was determined by the Xonar HDAV Center. It was possible to let the Xonar HDAV Center send 6 channel PCM to the receiver, but then all channels except FL and FR were silent.

So, although the ASIO device can provide hires PCM, it is limited to PCM 2.0 only. But hires multi channel PCM is not an issue anyway as it is working fine with WASAPI and the default Speakers device.

[albain]

Okay, next beta build to come : it will output TrueHD/DTSHD... as PCM to trick the audio renderer.

I hope that this will do it

[Emilot]

Originally Posted by albain

Okay, next beta build to come : it will output TrueHD/DTSHD... as PCM to trick the audio renderer.

I hope that this will do it

Cross fingers Albain!!!!!!!

[Andy o]

Originally Posted by FoLLgoTT

The option "Slave reference clock to audio" disables ReClock's mechanism to sync the reference clock to video card's refresh rate. So when video and audio run out of sync you get the same judder as without ReClock. The "(not recommended)" behind this option is a hint that ReClock's purpose to avoid video judder will be disabled. Instead you get bit perfect sound, but no judder-free video.
Btw this judder is so small that most people even don't notice it. I use HTPC's over 10 years now and ReClock since the first released version. After a period of time you get very sensitive to micro judder.
[...]
But the vsync correction of ReClock has a different purpose. It completely eliminates micro judder if set up correctly as described in the readme. Even without tearing this feature is a must to completely get rid of judder.

I'm just a little confused here, please bear with me. This micro judder you're referring to, is it because of dropped/repeated frames every once in a while, or is it because of tiny (ms) inconsistencies of timing of individual frames due to the video renderer dynamically adjusting frame rate to synchronize with the video card's refresh?

[nlnl]

Can ffdshow do real-time decoding + conversion 1080i59.94 --> 1080p23.976 for film source (H264, 3:2 pulldown inside)?
Is Intel Core 2 Duo powerfull enough for that?

[FoLLgoTT]

Originally Posted by Andy o

This micro judder you're referring to, is it because of dropped/repeated frames every once in a while, or is it because of tiny (ms) inconsistencies of timing of individual frames due to the video renderer dynamically adjusting frame rate to synchronize with the video card's refresh?

Yes, the micro judder I'm referring to is because frames are dropped or repeated. Timing inconsistencies (jitter) at the output of the video card are extremely low and should be invisible, because the refresh rate is fix and doesn't get changed. Only MPC-HC GothSyc's mode "Sync display to video" dynamically changes the refresh rate and produces this kind of jitter. But we don't talk about this right now. Let me explain what happens in a usual PC.

Imagine we want to play a 24p file.

Example 1)
With a refresh rate of 24Hz the cadence should look like this:

1, 1, 1, 1, 1, ...

Example 2)
With a higher refresh rate which is a multiple of 24Hz (in this case 48Hz) it should look like this:

2, 2, 2, 2, 2, ...


Resync judder
Without ReClock DirectShow always drops or repeats pictures when the asynchronicity of video and audio reaches a specific time span. In this example video is slower than audio so periodically a frame gets dropped:

Example 1)
1, 1, 0, 1, 1, ...

Example 2)
2, 2, 1, 2, 2, ...

In 1) you really loose pictures. In 2) you see all pictures but they are shown for different time spans. ReClock now slaves the reference clock to video refresh rate, leaves the cadence intact and resamples (drops/repeats) the audio instead. This is what ReClock is all about.


Judder because of vsync problem
Additionally the problem occurs that the picture arrives near the vertical sync signal (vertical blank). The graphics card then decides to show the last frame again which also causes a broken cadence.

Example 1)
1, 2, 0, 1, 1, ...

Example 2)
2, 3, 1, 2, 2, ...

As you can see the number of pictures stay the same. But again you loose a complete picture with 24Hz output. ReClock' vsync correction now makes sure that the picture always arrives with the necessary distance to the vertical blank. But this differs from refresh rate to refresh rate and even the player has an influence. So the tweaking of ReClock's vsync bar is a bit tricky. The method is described in the readme.


Btw. MPC-HC GothSyc's operating mode "Sync video display" is equal to ReClock's mechanism, but lacks the feature to drop/repeat AC3/DTS packets (though PCM gets resampled). Without that video and audio go out of sync after a while if the refresh rate does not match quite well. For more information follow this Link.

[Andy o]

Right, I know frame drop/repeat judder and what it looks like. I was just wondering because I'm not experiencing it with or without ReClock, and I did think jitter would be unnoticeable. The EVR Custom's OSD is pretty informative. I can set ReClock to slave to audio, and original speed (so audio is not touched) with VSync off, set my ATI on CCC to 23.976, and it's not dropping or repeating frames or going out of sync. With VSync on in MPC-HC, I can see the number for content frame rate stay steady, and the number for refresh rate and reference clock (only labeled "clock" here) change slightly and rapidly. With VSync off in MPC-HC, content rate and "clock" change dynamically, and refresh rate is not detected (just shows zero). I'm not 100% sure what all that means, but I think the content's frame rate at least is being slightly modified dynamically to keep sync (unnoticeable jitter).

[albain]

Hi,

here is the new beta build for HD bitstream : now the bitstream formats are sent over PCM media type so that Reclock will let them passthrough

I added a checkbox in the output section to activate the new WAVEFORMATEXTENSIBLE_IEC61937 structure, but it does not work with Reclock.
Do not enable this checkbox unless you have a radeon 5xxx to test with (and also maybe windows 7 is required).

FFDShow bitstream beta7

[THX-UltraII]

Originally Posted by albain

Hi,

here is the new beta build for HD bitstream : now the bitstream formats are sent over PCM media type so that Reclock will let them passthrough

I added a checkbox in the output section to activate the new WAVEFORMATEXTENSIBLE_IEC61937 structure, but it does not work with Reclock.
Do not enable this checkbox unless you have a radeon 5xxx to test with (and also maybe windows 7 is required).

FFDShow bitstream beta7

is bitstreaming with this new beta build only supported by the 5000-series of ATI or also with the ATI HD4350? I ask this because I heard some rumors on the Slysoft forum that the 4000-series from ATI would be able to bitstream.

[albain]

I see no reason why it shouldn't work on radeon 4xxx

I think that it has a bandwidth limit but it concerns LPCM, compressed formats even HD take less bandwidth

[THX-UltraII]

Originally Posted by albain

I see no reason why it shouldn't work on radeon 4xxx

I think that it has a bandwidth limit but it concerns LPCM, compressed formats even HD take less bandwidth

I dont understand what you mean, sorry
I thought only the new 5XXX cards support bitstream over HDMI

[THX-UltraII]

btw. will there also become a x64 version available?