pixel errors with bicubic resize

[JensG.]

I am making a divx of the movie "Cube" with the help of GordianKnot 0.261. When checking the avi I find green pixels at some places. You can find three screenshots (avs in VirtualDub) at http://de.geocities.com/jens_giese/cube/ .They appear only when I am doing bicubic resizing. Most pixels appear with sharp, least with soft bicubic. With bilinear resizing everything is okay.
They are not divx artefacts, since they are already there when I watch the avs file in VirtualDub.
Resolution of the video is 688x384. With lower resolution there are probably less wrong pixels, but still they are there.

I found these green pixels around 01:18 h and 01:19 h. When checking other parts of the movie I found no other wrong pixels, but I did not search extensively, though. The ones I found are all in heavily red scenes.

Are these pixels a known problem of bicubic resizing? What can I do to avoid them?

[dividee]

This is a chroma wrap-around bug. It should be fixed in avisynth 2.05. Could you try this version (if you're not using it) and report back ?

[JensG.]

Thanks for your advice! Unfortunately replacing the old dll file in my system directory did not help.

Anything else I can do?

[Didée]

Did you only replace the DLL, instead of using the installation file, which writes the correct registry values?
(I didn´t even have a look into them - but to be on the safe side ...)

If nothing else helps, reduce the saturation by a little bit, then the overflow should disappear. [ Tweak( 0.0, 0.98, 0.0, 1.0) ]

BTW: starker Film!

[WarpEnterprises]

Are you sure you have only 1 avisynth.dll on your disk and rebooted after repacing? (normally reloading the calling application should be enough but for sure...)

[JensG.]

There is one avisynth.dll in \windows\system. One more is in my DVD2SVCD folder, none in Gordian Knot folder. I replaced both. And rebooted.

I will try to uninstall Gordian Knot this evening and do a fresh install.

[sh0dan]

Try to create "version.avs" with notepad and type version() as the only line. Open it (in Vdub/Nandub/zoomplayer/whatever) - then you'll be able to see which version is used. Reinstall shouldn't be necessary at all, if you're just sure you replace all avisynth.dll

[Belgabor]

@sh0dan: You have to be very careful with that. I once had the problem that mplayer and vdub used diffrent (!) dlls, but as you said, if you open it with the app you'll use it should be ok. (just not only check with mplayer for example)

Cheers
Belgabor

[WarpEnterprises]

What could it be for to have more than 1 DLL on the system?

[Belgabor]

Well, in my case i had one in my avisynth dir and one in the system dir. MPlayer used the one in the avisynth dir, and vdub the one in the system dir.

Belgabor

[JensG.]

Version check says 2.0.5 from August 22nd.

[sh0dan]

OK - could you upload a 2 frame un-resized huffyuv testclip, so we can test it out ourselves? Just zip it and attach it here - you should be able to get it down to 200k.

From your images it seems like an overrun (>255 being wrapped to 0(green)).

Good movie though :)

[JensG.]

2 frames means two consecutive pictures? Or two keyframes?

[sh0dan]

Just two frames - or one frame if you like. I'd prefer huffyuv, so keyframes doesn't matter.

[JensG.]

I put two zipped huffyuv frames into http://de.geocities.com/jens_giese/cube/

In one zip they are not resized, in the other they show the green pixels.
I found that when I do bicubic resize without actually changing horizontal resolution (with black bars at 720 or without black bars at 702 pixels) there are no green pixels. Changing horizontal resolution over or under original resolution introduces green pixels. Vertical resolution changes work fine.

Thanks a lot for your efforts!

[sh0dan]

I cannot reproduce the error with your files, but it may be because the huffyuv-file (no-resize) is RGB32, which means it has been converted. Be sure to have Virtual Dub in "Fast Recompress" mode, when you save the huffyuv.

Does this script cause any problems for you:


avisource("F118777-9.avi")
BicubicResize(688,384,0,0.75)

[JensG.]

Your script causes no problems with my original non-resized avi. But you are right, it is because of RGB. I put another "noresize.zip" file into http://de.geocities.com/jens_giese/cube/
With these fast recompressed frames the pixels appear again with your script.

[sh0dan]

Got it!

I'll see what I can do - perhaps dividee has an opinion about this - it does very much look like an overrun issue.

[JensG.]

I appreciate that very much! Great work!

And if I understood right, a workaround would be to switch to RGB mode?

[JensG.]

@ Didée:

Sorry, I have just found your post now for the first time!? Thanks for the workaround!
Ich finde den Film ... etwas verstörend :-)

[dividee]

@sh0dan:
The bug I corrected before was that accumulated chroma could go below 0 in AdjustFocusH; I guess this is the symmetric case where it goes above 255. I bet clipping at 255 will solve it, but it's a bit harder to do without speed penalty. I'll do it, but maybe you could read the diff and find a faster way (you're better than me at optimizations).

[sh0dan]

Fixed it.

Get a new test version from here (http://cultact-server.novi.dk/kpo/avisynth/avs_cvs.html).

[sh0dan]

Originally posted by dividee
@sh0dan:
I bet clipping at 255 will solve it, but it's a bit harder to do without speed penalty.

You're right! Just used an unsigned saturated word->byte pack, it eliminates both >255 and <0 cases at once, so the speed penalty is minimal (if any), because the old <0 could be removed.

[edit]Hmmm... thinking about it, I'll probably overflow when <0 ... damn ... back at work ;)

[JensG.]

Originally posted by sh0dan
Fixed it.

Get a new test version from here (http://cultact-server.novi.dk/kpo/avisynth/avs_cvs.html). I am speechless. Thank you very much!

[sh0dan]

You should probably redownload - there is a theoretical (but very likely) flaw in the version I did there. Check your avisynth.dll is from 19:47.

Now all these strange colors should be gone.

@dividee: Put your code back in. I could not find the clip that caused the underrun you fixed, but it was probably needed. Regarding speed, Integer SSE could make this part a bit faster using pminsw and pmaxsw to do these two checks - it could also eliminate the pand mm3, mm5 , so the code would be:

movq mm4, mm3 ; clip chroma at 0
psrad mm3, 31
pandn mm3, mm4

paddd mm1, mm6 ;Y1|Y1|Y0|Y0 (round)
paddd mm3, mm6 ; V| V| U| U (round)
pxor mm4, mm4
pslld mm3, 2 ; Shift up from 14 bits fraction

packsswb mm4,mm3 ; Clip >255 by using pack / unpack
punpckhbw mm3,mm4
psrld mm3,8
pand mm3, mm5 ;mm3 = v| 0|u| 0

psrld mm1, 14 ;mm1 = 0|y1|0|y0
por mm3, mm1
----------------------------------
could be done by:
----------------------------------
paddd mm3, mm6 ; V| V| U| U (round)
paddd mm1, mm6 ;Y1|Y1|Y0|Y0 (round)
pslld mm3, 2 ; Shift up from 14 bits fraction to 16 bit fraction
pxor mm4,mm4 ;Clear mm4 - utilize shifter stall
psrld mm1, 14 ;mm1 = 0|y1|0|y0
pmaxsw mm3,mm4 ;Clamp at 0
pminsw mm3,mm5 ;Clamp at 255
por mm3,mm1


(I'll just test it).

[edit] Constant was redundant - already had what I needed in mm5.
It works like a charm - Pairing is also much nicer (all but one instruction)... But I don't have the strength to do a seperate isse codepath now - anybody. ;)

[dividee]

Nice. I didn't think about pack...
But now I can see there was already a "pack" at the end. The problem came from the "pand mm3,mm5" that prevented the pack to saturate.
So, this should work:

movq mm5, xFFFF0000FFFF0000
...
movq mm4, mm3 ; clip chroma at 0
psrad mm3, 31
pandn mm3, mm4

paddd mm1, mm6 ;Y1|Y1|Y0|Y0 (round)
paddd mm3, mm6 ; V| V| U| U (round)
pslld mm3, 2 ; Shift up from 14 bits fraction
punpckhbw mm3,mm4
psrld mm3,8
pand mm3, mm5 ;mm3 = v| 0|u| 0
psrld mm1, 14 ;mm1 = 0|y1|0|y0
por mm3, mm1
packuswb mm3, mm3 ;mm3 = ...|v|y1|u|y0

Note that all that's needed is to change the constant in mm5 (was 0x00FF000000FF0000)

BTW, I was talking about FilteredResizeH, not AdjustFocusH in my previous post. Sorry for the confusion.

[sh0dan]

@dividee: Of course! That's gotta be the fastest way to do that in plain MMX. Brilliant! :)

Still there are nothing but dependency stalls all the way down, so except "paddd mm1, mm6" and "psrld mm1, 14" all instructions will take 2 cycles to complete, giving approx. 20 cycles per pixel, compared to approx. 10 cycles using i_sse.
Not much, but we should probably look if there are other thing we could speed up using i_sse. We could also use your modification so we can leave out the "pmin" instruction completely, saving us two more cycles ;)

[hakko504]

...and once again sh0dan has proven to be one hell of a bug fixer, even though he doesn't have a calendar. ;) :D ;) (020902 :confused: )

Nice work though.

[sh0dan]

@hakko: The most work was done by dividee on the last bugfix - and I'm using EU-notation (ddmmyy) :)

[edit]Ahhh!! You mean, it's the 3rd ;)
:stupid: