Real-time comparison - two videos

[pandy]

Hi All,

Desperately need to find cheap and easily available way to compare two video inputs - one source, unknown latency in decoders (manually adjustable delay is OK), two decoders video outputs need to be compared. Preferred solution - software on PC with normal (VfW) capture devices.

Even simple PSNR calculation should be fine (i believe should be fine).

TIA

----------

Any chance to use ffmpeg for this?

capture one deocder output with ffmpeg, pipe to second ffmpeg (with capture from second decoder) as a reference, PSNR calculation - video resolution SD BT.656

[jmac698]

I can't believe no one answered this - perhaps you should ask in Avisynth Usage forum.
Comparing two videos in Avisynth is easy.
http://avisynth.org/mediawiki/Compare

Reading a live capture in real-time and running an Avisynth script on it is also easy. In windows at least.
Install ffdshow and ensure that you check Avisynth support (I believe that's necessary).
Then open the ffdsow video video decoder configuration.
Click Avisynth. You can write a script directly in the box to the right.

Now how to apply this real-time script to your capture source?
Basically you need to run ANY program that supports selecting a compression codec while capturing.
Here is an example of a DV capture program being used real-time with ffdshow:
http://www.scvi.net/ffdshow.htm

Here's an example of how to use kmplayer with analog capture devices:
http://www.kmplayer.com/forums/showthread.php?t=22604

At least one of those solutions should get you real-time Avisynth processing of live capture.

All scripts posted into the Avisynth script box should start with ffdshow_source()
More information is here:
http://ffdshow-tryout.sourceforge.net/wiki/video:avisynth

One problem with all of this; you are only allowed to enter one video source at a time.
I have an idea to work around this. Perhaps if you run your capture program, for one of the sources have it save to a file, then the other capture is running ffdshow. In that one, the script runs compare between the live capture and the other capture as a file. You're going to need some more help with this.

Your idea for ffmpeg sounds good too. Keep thinking about piping things...
I believe another option to explore is constructing a filter graph with GraphEdit. In this way you would add the two sources to your graph, put them through a codec to make use of ffdshow, then... I don't know. Perhaps there's some trick to combine two images as side-by-side, then you could crop and compare them in Avisynth?
Maybe some filter graph that is meant to support live stereo vision?

[pandy]

Thx jmac698 - i have to different hardware decoders - and i need to compare their outputs mostly searching for some errors in decoding in one of hardware decoders, so one issue is signal acquisition with aligning in time videos (different latency introduced by decoders) second is comparison problem - at the beginning i have idea about ffdshow and avisynth similar to Yours but based on fact that avisynth is file oriented this is very trick part now i shifted to ffmpeg and pipes but... i have no idea how to implement video delay with ffmpeg - seems that without dedicated app this can be not very easy task to do and what is worse even software designed to video comparison and quality analysis not allowed to use two live signals which seems to be very strange...

But anyway Big THX jmac698 - currently i'm stuck on different areas... and this one must be postponed.

[jmac698]

Then I have a completely different idea for this. Maybe you only need one live source, and one file, because the test signal is under your control. For example create the ideal decoded test video into a lossless codec, and use that as the reference, then use the ffdshow live method and compare to a file.

There's even live generated signals you could use. Obviously it's easy to compare the live to a generated colorbar or noise. It depends what kind of errors you are looking for.

compare(clip1, clip2) shows standard error measures like MSE.

There's also a way to write a script to time -align your signals. Just embed some image into the video that you can check, for example a timecode. I have written both a time code encoder and decoder just for this.
http://forum.doom9.org/showthread.php?p=1582574#post1582574
http://forum.doom9.org/showthread.php?p=1582700#post1582700

this is basically how it would work,
#global so that scriptclip can see it
global reference=directshowsource(...)
ffdshow_source()
scriptclip("""
code=slicer(...)
code>0 ? compare(last,reference.trim(code,1))
""")

The idea is that as soon as a timecode is recognized in the live stream, it trims the reference file to that same exact frame and then outputs the comparison statistics... I think that might actually work.

ps I'll be writing a live script like this myself sometime, to use my dropped frames analyzer.

[pandy]

Issue is that my intention is to use live network and i don't want use any stream due of architectural limitations - comparing two hardware - one is known and used as a reference, second is known to have problems and i need to verify that problem was fixed and no regression was made.
Issue is not even related to quality of decoding but errors like corrupted single macroblocks etc - this happens sometimes, it is very difficult to reproduce and usually long time is required to reproduce issue.

From my perspective - best is to have app that support two live video inputs, one of inputs can be delayed to second one to compensate different latency of the decoder, then simple delta between video - on a blocks 8x8 with some definable threshold - calculation of PSNR (or similar) is not required however it is nice to have.

Currently im involved in something different but in future i need to start think how to solve this issue.

Thank You jmac698!

[jmac698]

I understand, here's an idea then, you could use tcpdeliver, two machines to read the live video.
In any case, I'm afraid the two capture cards will get out of sync eventually.

[jmac698]

I understand you're working on other things, but for reference, I ran across this link:
http://www.shspvr.com/smf/index.php?topic=7680.0

It's a software that's able to record from multiple instances of cards.

I also found this, which uses a little regedit to make the two cards have different names in graphedit:
http://forum.videolan.org/viewtopic.php?f=2&t=5461

Though it's not directly related, it contains some hints for using two cards of the same type.

[pandy]

Thank You jmac698 - i really appreciate Your work! Thank You once more - i need to find some time i will try to test.