Buggy Simple-iDCT in MPEG2Dec3dg? Sporadic inverted macroblocks...

[LigH]

Dear AviSynth and XviD developers, dear Nic, dear Donald,

I recently tried to compare the output of MPEG2Dec3 using different iDCTs, just to find out how much difference really may occur.

Unfortunately, it was impossible to use two different MPEG2Source() calls with different iDCT parameters in the same script. Imagine the following code:
LoadPlugin("MPEG2Dec3dg.dll")
clip1=MPEG2Source("*.d2v", CPU=0, iDCT=4) # iDCT=4 is "Reference", right?!
clip2=MPEG2Source("*.d2v", CPU=0, iDCT=7) # iDCT=7 is "SimpleiDCT"
Subtract(clip1,clip2).Levels(112,1.0,144,0,255) # boost contrast
And the result (with AviSynth 2.5.5 latest alpha) is: Pure medium gray. No difference at all. :confused:

But this is obviously wrong, probably some kind of unwanted caching might deliver the same video for both clips: Instead, I converted video using different iDCTs to HuffYUV videos, and compared them afterwards.
clip1=AviSource("iDCT4.avi", false, "YUY2")
clip2=AviSource("iDCT7.avi", false, "YUY2")
Subtract(clip1,clip2).Levels(112,1.0,144,0,255)
The result: Indeed, a few differences. Probably usually invisible (like doom9 showed in his comparison), but nevertheless present...

Unfortunately, only iDCT=7 = SimpleiDCT showed a few frames where the difference between two frames of those AVIs contained some "inverted" blocks and comparable differences, as you can spot in the following archive (compressed with 7zip (http://www.7-zip.org/)):

http://www.ligh.de/software/Simple/Simple-iDCT_bug.7z

As you see: You won't spot the difference in the original movies, only in the calculated difference.

[Wilbert]

Donald is aware of this (this iDCT option is defined by some global variables, which implies that only the latter, ie the iDCT setting of clip2, is used for both clips). This should be corrected in mpeg2decdg3 v1.30. See dvd2avi forum for more info.

[Guest]

Originally posted by Wilbert
Donald is aware of this (this iDCT option is defined by some global variables, which implies that only the latter, ie the iDCT setting of clip2, is used for both clips). This should be corrected in mpeg2decdg3 v1.30. See dvd2avi forum for more info. Correct. Thank you, Wilbert. Version 1.3.0 RC5 is available, appears to be stable, and fixes this issue.

[Cyberia]

@neuron2 - Are you sure ALL the parameters are independent across Mpeg2source instances? I just tried this test
(because it's a damn good idea!) and got a completely grey screen on every test (compared with 64-bit FP iDCT)

I used this script (rotating the idct parameter on the "cpu_0" instance):
LoadPlugin("G:\DVD\AviSynth\plugins\MPEG2DEC3.DLL")
cpu_6 = mpeg2source("G:\My Programs\testvideo\vts_02_1.d2v", idct=3, moderate_h=20, moderate_v=40, cpu2="oooooo")
cpu_0 = mpeg2source("G:\My Programs\testvideo\vts_02_1.d2v", idct=1, moderate_h=20, moderate_v=40, cpu2="oooooo")
subtract(cpu_6, cpu_0).Levels(127,1,129,0,255)


This is with DVD2AVIdg 1.3.0

BTW, this thread should be moved to the DVD2AVI forum.

[Guest]

@Cyberia

Found in global.h:


// - Nic "The" iDCT function pointer
extern void (__fastcall *idctFunc)(short *block); I've fixed it and you will get it in DGMPGDec 1.0.0. The differences are small but visible.

Thank you for pointing it out. That appears to be the last global item.

BTW, this thread should be moved to the DVD2AVI forum.The last time I checked, mpeg2source() was still an Avisynth function. :)

[Guest]

LigH was pointing out a problem in the Simple IDCT. I doubt I'll have any time to look at that.

[Guest]

DGMPGDec 1.0.0 with the fix is now available.

[Cyberia]

@neuron2: Thanks, now they seem to work independently.

The last time I checked, mpeg2source() was still an Avisynth function.
Is it? I thought it was a MPEG2DEC3 (DGDECODE) function. I'm not being sarcastic, but neither problem was with AviSynth, they were both MPEG2DEC3 problems.... hence, it didn't seem to belong to the AviSynth Development forum.

Not a big deal though. :)


@all:

Using the above script I compared the various iDCT algorithms, and as I expected the 64-bit FP iDCT produces output closest to the IEEE-1180 Reference iDCT. The difference is quite noticeable with this script, but probably impossible to see otherwise.

Suspicion confirmed, I'm sticking with the 64-bit FP iDCT.

[LigH]

I'm sorry: The bug seems to be not in Simple-iDCT (iDCT=7), but in the "Reference" iDCT (iDCT=4). I just did a cross-comparison (each iDCT between 1 and 7 against each other) on a P4 this weekend, and all differences to iDCT=4 contained such inverted blocks. I'll test again on an AMD Duron, then I'll tell you the results (around tomorrow or the day after). If you like, just do the same with the following scripts in the meantime:

makeavis.bat@echo off
for /L %%a in (1,1,7) do call :loop %%a
goto end

:loop
echo LoadPlugin("MPEG2Dec3dg.dll") > !MPEG2Dec3dg-iDCT%%a.avs
echo MPEG2Source("(testclip).d2v",CPU=0,iDCT=%1).ConvertToYUY2() >> !MPEG2Dec3dg-iDCT%1.avs
avs2avi !MPEG2Dec3dg-iDCT%1.avs MPEG2Dec3dg-iDCT%1.avi -c hfyu

:end

makedifs.bat@echo off
for /L %%a in (1,1,6) do for /L %%b in (%%a+1,1,7) do if not %%a==%%b call :loop %%a %%b
goto end

:loop
echo clip1=AviSource("MPEG2Dec3dg-iDCT%1.avi",false,"YUY2") > d_%1-%2.avs
echo clip2=AviSource("MPEG2Dec3dg-iDCT%2.avi",false,"YUY2") >> d_%1-%2.avs
echo clip3=Compare(clip2,clip1,"","d_%1-%2.log",false) >> d_%1-%2.avs
echo clip4=Subtract(clip2,clip1).Levels(112,1,144,0,255) >> d_%1-%2.avs
echo Overlay(clip3,clip4) >> d_%1-%2.avs
avs2avi d_%1-%2.avs d_%1-%2.avi -c hfyu

:end

[Nic]

@neuron2: I knew it'd be my fault ;) lol

@LigH: Yup reference has always been reported bad:
http://arbor.ee.ntu.edu.tw/~jackei/dvd2avi/idctref/
Perhaps it should be taken out...

-Nic

[Guest]

Originally posted by Nic
@neuron2: I knew it'd be my fault ;) lol You've a great sense of humor, my friend. Hey, at least you put the big "The" in there, alerting people to it. Naturally, I missed the alert. :eek:

Yup reference has always been reported bad:
http://arbor.ee.ntu.edu.tw/~jackei/dvd2avi/idctref/
Perhaps it should be taken out...
OMG, Cyberia will pop a cerebral artery over this. :D

[Cyberia]

Originally posted by neuron2
OMG, Cyberia will pop a cerebral artery over this. :D

LOL! :) I have already mentioned it to Miha, but he's likely to need examples.

[LigH]

Okay - a little late, but finally my report.

At first: The updated batches:

makeavis.bat@echo off
for /L %%a in (1,1,7) do call :loop %%a
goto end

:loop
echo LoadPlugin("H:\Programme\GordianKnot\MPEG2Dec3dg.dll") > !MPEG2Dec3dg-iDCT%1.avs
echo MPEG2Source("N:\Movies\Trailer\Mortal_Kombat_2\trMK2.d2v",CPU=0,iDCT=%1) >> !MPEG2Dec3dg-iDCT%1.avs
rem echo ConvertToYUY2() >> !MPEG2Dec3dg-iDCT%1.avs
H:\Programme\avs2avi\bin\avs2avi !MPEG2Dec3dg-iDCT%1.avs MPEG2Dec3dg-iDCT%1.avi -c vble -w

:end
makediffs.bat@echo off
if exist dummy.avi del dummy.avi > nul
for /L %%a in (1,1,6) do for /L %%b in (%%a+1,1,7) do if not %%a==%%b call :loop %%a %%b
goto end

:loop
echo LoadPlugin("H:\Programme\GordianKnot\SSIM.dll") > d_%1-%2.avs
echo LoadPlugin("H:\Programme\GordianKnot\VQMCalc.dll") >> d_%1-%2.avs
echo clip1=AviSource("MPEG2Dec3dg-iDCT%1.avi",false,"YV12") >> d_%1-%2.avs
echo clip2=AviSource("MPEG2Dec3dg-iDCT%2.avi",false,"YV12") >> d_%1-%2.avs
echo Compare(clip2.ConvertToYUY2(),clip1.ConvertToYUY2(),"","d_%1-%2.log",false).ConvertToYV12() >> d_%1-%2.avs
echo Overlay(SSIM(clip2,clip1.ConvertToYV12(),"d_%1-%2.csv","d_%1-%2.ssim")) >> d_%1-%2.avs
echo Overlay(VQMCalc(clip2,clip1.ConvertToYV12(),"d_%1-%2.vqmc")) >> d_%1-%2.avs
echo Overlay(Subtract(clip2,clip1).Levels(96,1,160,0,255)) >> d_%1-%2.avs
H:\Programme\avs2avi\bin\avs2avi d_%1-%2.avs dummy.avi -c vble -w
del dummy.avi

:end
The content probably looks a little too complicated, but it was necessary to cheat AviSynth's internal optimization: I had to make it believe that all the comparing funktion would take part in the resulting video clip, therefore I used an opaque Overlay().

And now the results:

http://www.ligh.de/privhome/iDCT_Comparison.htm

I think it is clearly visible: Except for the "Reference" iDCT 4 with several incorrect blocks, in general the iDCTs do not differ too much, a few give exactly the same results.

Please note: This is no rating of the iDCTs (due to no real reference), but just a cross-comparison, to show the difference between each others.

So, my conclusion: Don't care too much, just take the one which is fastest for your machine.