weird problem with Lanczos4Resize

[Jan Marijniszoon]

Hi all,

I was busy converting a HDTV TS-file to XviD.

I used AviSynth 2.51 with both combinations XviD 1.1.0/VirtualDub 1.6.16 and XviD 1.2-127/xvid_encraw.

Further I used DGDecode/DGIndex 1.4.8

At frame number 130060 Virtualdub fully crashed and xvid_encraw just stopped without giving any errors.

I trimmed the clip so that I could do tests within the area of the troublesome frame and then eliminate each element from the script.

I started off with this script:

LoadPlugin("DGDecode.dll")
LoadPLugin("TIVTC.dll")
LoadPlugin("UnDot.dll")
MPEG2Source("neo.d2v")
tfm(d2v="neo.d2v")
Trim(130000,130100)
UnDot()
Lanczos4Resize(1280,720)

Then I eliminated all the plugins and parameters untill I got this script:

LoadPlugin("DGDecode.dll")
MPEG2Source("neo.d2v")
Trim(130000,130100)
Lanczos4Resize(1280,720)

The error was still there! So I tried to encode without Lanczos4Resize and then the error was gone!

I also tried to resize to a different resolution (1440,816) and then the error was gone too!

So, as weird as it may seem, the error seems to only occur when I resize to the specific resolution of 1280 x 720 with Lanczos4Resize.

Can this be a bug?

[foxyshadis]

Can you provide a cut of the mpeg stream around the 130000 range, if it still shows the bad behavior? It might be the mpeg, in which case dgdecode could be corrupting the environment.

Oh wait, avisynth 2.51? The curent releases are 2.56 and 2.57RC1. You might want to try those first.

[Jan Marijniszoon]

Thanks for the response!

I put the whole stream through mpeg2repair, which reported no errors in the stream.

A time ago the avisynth website was offline so I downloaded from the doom9 download section. I wasn't aware that I had an older version.

I will try the newer ones as soon as possible and then I will let you know!

[Jan Marijniszoon]

Oops sorry...I made a mistake..I do have 2.56 already.

Do you still want the stream sir?

[Fizick]

what if you replace Lanczos4Resize to LanczosResize or bicubicresize?

[Jan Marijniszoon]

Thanks for the suggestion, but the problem remains...no go with 1280 x 720 and it works with 1440 x 816...very weird.

What I noticed is that it only happens when I select "fast recompress" in Virtualdub. When I choose "full processing mode" it works. So maybe the problem is in conjunction with the XviD codec?

I did a small test with megui on the x264 codec and it worked also.

Anyway, I uploaded the stream. It can be found in this URL and it is called "stream_fragment.rar":

http://members.lycos.nl/ludosanders/video-images/

[Jan Marijniszoon]

I think I found the problem.

It's GMC in XviD. When I disable it, the problem is gone.

What should I do now? How can I report this bug properly?

And maybe someone can move this thread to the XviD section?

[unskinnyboy]

It's GMC in XviD. When I disable it, the problem is gone.
It doesn't crash for me, with or without GMC, on a constant quant encode I did. Avisynth 2.57 22042006 build & Koepi's XviD-1.2.-127-25022006 _Alpha Build_. I could try on more recent builds on other PCs, but I doubt GMC is the problem here.

[squid_80]

The GMC code in xvid was updated a few months ago (From the comments in the code Skal wrote it while watching the world cup) and it's not quite perfect, especially if compiled with MSVC. Either way I can't get it to crash here either, and I'm using a build from xvid's current cvs.

[Jan Marijniszoon]

Did you actually resize to 1280 x 720 as well?

I just tested with the lastest AviSynth 2.57 RC1 and the problem still remains while using XviD 1.1 stable from Koepi.

When I disable only GMC, the problem is gone.
Maybe it is GMC combined with other complex options? I enabled all the heavy options in XviD like qpel and VHQ 4 for b-frames too.

Maybe this is important too:
In DGIndex I chose the option IEEE-1180 Reference as iDCT Algorrithm. And my CPU is Inter Duo Core T2400

[squid_80]

Can you post the xvid_encraw command line which causes a crash?

[Jan Marijniszoon]

xvid_encraw.exe -i "D:\matrix\reloaded.avs" -single -cq 2 -max_key_interval 250 -nopacked -vhqmode 4 -qpel -gmc -lumimasking -bvhq -threads 2 -framerate 23.976 -w 1280 -h 720 -zones 0,q,4,KO/1928,q,2,KO/182827,q,4,KO -avi "D:\matrix\reloaded.avi"


But it didn't crash. Only Virtualdub does...xvid_encraw just stops at the specific frame as if like the job was done.

[squid_80]

It crashes and gets terminated by windows. I can reproduce it now using the sample you posted, the simplified script from your first post and the following command line:
xvid_encraw -i test.avs -vhqmode 4 -qpel -gmc -lumimasking -bvhq -zones 0,q,2,KO

The chroma optimizer setting was what I was missing. So the setting to crash it are:
Motion Search precision = 6
VHQ mode = 4
VHQ mode for b-frames
Quarterpel
Adaptive quantization (lumimasking)
GMC
Chroma optimizer
Constant Quant = 2
d2v made using the IEEE-1180 IDCT algorithm

Changing any of these seems to avoid crashing.

[Jan Marijniszoon]

I do not understand what you mean? What should I disable then?

Anyway, I tried it also with VirtualDubMod on my other PC (Pentium 4) and at least VirtualDubMod gives me some information about the crash...

An out-of-bounds memory access (access violation) occurred in module 'xvidcore'...
...while compressing frame 24 from 04b00000 to 07110020 (VideoSequenceCompressor.cpp:406)...
...while running thread "Processing" (thread.cpp:120).

What does this mean? XviD bug?

[squid_80]

Yes it's definitely an xvid bug. If you really want to process it with those options I'd suggest using a different IDCT algorithm in DGIndex. Tests have shown there's no real quality difference between them, 32-bit SSE2 will probably be fastest for your material (Skal's can be faster with low bitrate sources).

It just seems that using that IDCT, the lanczos4resize and those codec options triggers some obscure bug in xvid. An unlucky coincidence.

Hopefully a mod will move this thread to xvid's forum so the devs will notice it...

[Jan Marijniszoon]

Thanks for your help!

I tried 32-bit SSE2 MMX just now and it has the same error.

I hope indeed that the developers will get to see this.

Please move this thread to the XviD section!

[Jan Marijniszoon]

STOP PRESS!

Even weirder!

When I use 32-bit SSE2 MMX with DGIndex the error even occurs when GMC is disabled in XviD!!!

In VirtualDubMod it now says:

An out-of-bounds memory access (access violation) occurred in module 'xvidcore'...
...while compressing frame 24 from 04330000 to 072f0020 (VideoSequenceCompressor.cpp:406)...
...while running thread "Processing" (thread.cpp:120).

[Wilbert]

Yes it's definitely an xvid bug. If you really want to process it with those options I'd suggest using a different IDCT algorithm in DGIndex. Tests have shown there's no real quality difference between them, 32-bit SSE2 will probably be fastest for your material (Skal's can be faster with low bitrate sources).

It just seems that using that IDCT, the lanczos4resize and those codec options triggers some obscure bug in xvid. An unlucky coincidence.

Hopefully a mod will move this thread to xvid's forum so the devs will notice it...
on your request ... :)

[Blue_MiSfit]

What a strange bug.

I'm curious, have you found GMC to be really helpful in processing HD source? I've never found it to be terribly useful, only very slow. However, I've never tried it on HD stuff. What's your experience?

~MiSfit

[Jan Marijniszoon]

Well while watching the report of the frames being encoded I saw that it uses the S-frame (which is a GMC-ed frame right?) an awfull lot of times.

I am not a super expert, but isn't every advanced profile option usefull into getting the best visual quality with still good compression?

What does GMC really do then? What I do know for sure is that I want qpel badly. It makes the image alot more detailed to my opinion.

But GMC isn't the only problem anymore now. When I used 32-bit SSE2 MMX in DGIndex the crash also occcurs when GMC is disabled.

[Jan Marijniszoon]

When I use the resolution 1280 x 718 it works again. So I can manage with this script to get it to work:

LoadPlugin("DGDecode.dll")
MPEG2Source("neo.d2v")
Trim(130000,130100)
Lanczos4Resize(1280,718)
AddBorders(0,1,0,1)

Nontheless, it remains a weird kind of bug.