about GetPitch() and non "multiple of 8" resolutions

[jordanh]

It seems that a lot or most, or even all filters have problems when it comes to picture resolution widths that are not multiples of 8. I am not sure about this, thats why i made a topic out of it.

As i am currently developing a Source Filter that of course targets to be compatible to virtually "all" other AVisynth filters, i got to this problem:
720x576 = OK, 1920x1080 = OK, 852x480=FAIL

I needed to experiment more than i liked in order to find out, that all my problems are about about the Pitch. I always assumed that


width*height*bytesperpixel = buffersize //incorrect

but thats WRONG. The correct formula can only be done using the pich:


Height*Pitch = buffersize //correct


Which means for source plugins, they need to find out the pitch dynamically for each resolution and send avisynth the bytes it needs.

What i dont understand is, why video engines like AVISynth and also Carboncoder actually have something like a pitch. (in carbon they call it "stride") What is it good for? - i know that encoders need it that way, or maybe decoders decode it that way.. but is it really neccessary within a framework?
Everything seems to come down to either a multiple of 8 or a multiple of 16. AVISynth seems to keep a stride of multiple of 8.

When i do env->NewVideoFrame(InputVideoInfo), and then GetPitch() on this frame, it will return the next bigger value than the width that a multiple of 8.
e.g. 852 will have a stride of 856*2 (=1712) for yuy2 colorspace (2 bytes per pixel)
and one cannot change that.

Here is some picture visualizing the pitch (its the green bar on the right)

input video 852x480 from carbon where pitch is a multiple of 16, so the pitch is a 864x480 frame

http://img577.imageshack.us/img577/1216/852x480withstride.png (http://imageshack.us/photo/my-images/577/852x480withstride.png/)

...a frame dump of avisynth would have a smaller green bar because the pitch needs to be only 8 bytes less.

The question at this time is, if it can be that i am right and nearly all plugins have troubles with sources thats width isnt a multiple of 8. By now i only tested directshowsource and the Convertto functions. All had problems.
Can i write my source filter in a way that i provide a source that 852 in the "right way" so that the user does not need to do the resize in the avs script?

Harry

Some additional information:
___________________________________________________________________________________________________________________________________________________

Here is how i copy the frames: (unit->stride is actually the pitch)


for (int i=0; i<unit->height;i++){
unsigned char* _in = (unsigned char*) (in_ptr + (i * m_carboncode_inputpitch));//source
unsigned char* _out = (unsigned char*)(out_ptr + (i * m_avisynth_inputpitch));//target
memcpy(_out,_in,unit->width*2);//*2 = hardcodec aligned yuv422 support (2 bytes per pixel)
}





While i am developing my source filter, of course i always prove where things go wrong. I think i proved that the ConvertTo...() functions and the monitor filter have the problems, and i cannot really help them besides resizing the picture to an easy "by 8" resolution

1) "Hello World" just returns the same clip than the Source Plugin:
http://img713.imageshack.us/img713/814/cleanx.png (http://imageshack.us/photo/my-images/713/cleanx.png/)

Uploaded with ImageShack.us (http://imageshack.us)

2) When we use some filterint, the picture lines are shifted
http://img7.imageshack.us/img7/4336/1converttoyuyerror.png (http://imageshack.us/photo/my-images/7/1converttoyuyerror.png/)

Uploaded with ImageShack.us (http://imageshack.us)

3) We can workaround this by resizing the picture before doing other filtering
http://img46.imageshack.us/img46/4181/resizemakesitwork.png (http://imageshack.us/photo/my-images/46/resizemakesitwork.png/)

Uploaded with ImageShack.us (http://imageshack.us)

4) Just for fun, we test if tht monitor plugin is able to process the original resolution while Carboncoder on the right side shows the finally returned clip
http://img198.imageshack.us/img198/3375/monitorpluginhasalsopro.png (http://imageshack.us/photo/my-images/198/monitorpluginhasalsopro.png/)

Uploaded with ImageShack.us (http://imageshack.us)

5) When we use the monitor after the resize, it works as expected
http://img195.imageshack.us/img195/1215/monitorafterresconvert.png (http://imageshack.us/photo/my-images/195/monitorafterresconvert.png/)

Uploaded with ImageShack.us (http://imageshack.us)

[Reel.Deel]

Hello jordanh, I don't know much about programming but I had a similar situation when I was testing out (http://forum.doom9.org/showthread.php?p=1596178#post1596178) a plugin.
Addional link. (http://forum.doom9.org/showthread.php?p=1597694#post1597694) Hope it helps. :)

[jordanh]

Hmm... thanks, that reminds me of the correct word for "multiple of 8": mod 8...

Thanks reel.deel, it seems like your thread just hardens the assumption that most filters have a bug here.

Any other experiences to this?

cheers,
Harry

[Youka]

Image processing is expensive, so SIMD (http://de.wikipedia.org/wiki/Flynnsche_Klassifikation#SIMD_.28Single_Instruction.2C_Multiple_Data.29) instructions are often used to speed tasks up.
SSE uses 128 bit = 16 byte registers, so 8 yuv2 pixels can be processed the same time.
AVX uses 256 bit = 32 byte registers, so 16 yuv2 pixels can be processed the same time.
I guess it have to do something with this.

[IanB]

The current default memory alignment for Avisynth video structures is 16 bytes. This to provide SSE friendly memory addresses. This was 8 in earlier versions and may become 32 in future versions.

By default the address in memory of the 1st byte of each picture line will satisfy (int)Address & 15 == 0. This allows the use of fast MOVDQA instructions.

The Pitch or Stride is the distance from the start of one picture line to the start of the next and by default will be mod 16 in order to satisfy the condition above.

Various operations can effect this default behaviour, e.g a Crop() operation with a non mod 16 left value. The Crop() function has an optional Align=True argument that will blit unaligned picture contents into a new aligned picture structure. Of course doing this costs the memory copy time, but if a filter fails or runs slowly with unaligned memory it is necessary.

Also one must not assume any relationship between rowsize and pitch or that the pitch of different frames within a clip is consistent.

The optional align argument of the env-NewVideoFrame(vi, align); method allows overriding and increasing the default pitch. e.g you might specify 64 to be processor cache line aligned or 32 to be AVX instruction aligned. If you specify a negative align value you can force arbitrary pitch alignment, this option should be used with extreme caution, not all plugin authors correctly test input pointer alignment. The AVISource() filter use align=-4 to create a DIB compatible buffer to pass into the VFW API calls and avoid an extra input memory blit.

Filter code should always check input memory pointer alignment complies with the requirements of the code and fall back to an alternative unaligned code path. The unaligned code path must be functionally equivalent but need not be speed optimised. Bliting an unaligned input frame to an aligned copy is an option.


The IScriptEnvironment method :-

virtual void __stdcall BitBlt(BYTE* dstp, int dst_pitch, const BYTE* srcp, int src_pitch, int row_size, int height);

provides an ISSE optimised routine for copying rectangular blocks of memory (blitting).

[StainlessS]

Width & height can be changed in-situ by crop so using pitch is a must, also, pitch can change frame by frame in same clip (interleave might do this).

Oops, just beaten to it.

[Gavino]

It seems that a lot or most, or even all filters have problems when it comes to picture resolution widths that are not multiples of 8.
...
I think i proved that the ConvertTo...() functions and the monitor filter have the problems

Filter code should always check input memory pointer alignment complies with the requirements of the code and fall back to an alternative unaligned code path.

Ian, are you saying that ConvertToXXX() is deficient in this respect, or is something wrong with the frames generated by jordanh's source filter (which appears to take the pitch into account)?

[IanB]

The YV12 path of ConvertToYUY2() outputs mod 32 pitch so it can do 16 pixels per loop. Line 132 of convert/convert_yuy2.cpp
Also one must not assume any relationship between rowsize and pitch or that the pitch of different frames within a clip is consistent.
And must not assume any relationship between pitch of the luma planes and chroma planes for the planar formats.

You must use the GetPitch() method on each and every PVideoFrame you get.

Pitch torture test for filters....
A=SelectEvery(4, 0)
B=SelectEvery(4, 1).AddBorders(0,0,8,0).Crop(0,0,-8,0)
C=SelectEvery(4, 2).AddBorders(0,0,16,0).Crop(0,0,-16,0)
D=SelectEvery(4, 3).AddBorders(2,0,22,0).Crop(2,0,-22,0)
Interleave(A,B,C,D)
FilterUnderPitchStressTest(...)
...

[jordanh]

IanB is of course completely right. Thanks very much for explaining it so deeply.
My problem was that at initialisation of the transcode process, in the application that creates the avisynth env, i did this after initializing the avs script:


PVideoFrame analyseframe=env->NewVideoFrame(avisynthOutputVideoInfo);
m_outputstride = analyseframe->GetPitch();


Then i used this pitch also for frames that i pulled from the returned clip of the script.

I added this information to the cplusplus wiki.

Last question on this topic:

The following expression generates a frame with settings of the returned clip from a avs script:


*Video = env->Invoke("Import",AVSValue(args_userscript,1)).AsClip();
avisynthOutputVideoInfo = (*Video)->GetVideoInfo();



so why does this PVideoFrame have another pitch than a real frame thats created using



(*Video)->getFrame();



...

Also still there is the question why DirectshowSource and MonitorFilter also have problems with that resultion? (i only tested one file type, its hard to get something compatible to directshowsource)

[Gavino]

The following expression generates a frame with settings of the returned clip from a avs script:

*Video = env->Invoke("Import",AVSValue(args_userscript,1)).AsClip();
avisynthOutputVideoInfo = (*Video)->GetVideoInfo();
so why does this PVideoFrame have another pitch than a real frame thats created using
(*Video)->getFrame();
env->NewVideoFrame(avisynthOutputVideoInfo) will create a frame with the default pitch appropriate to the frame width.

(*Video)->getFrame() will deliver a frame that in general depends on the filters used in the script (and may be different for each frame). For example, Crop() may have been used to create a subframe of a wider frame - this will still have the same pitch as the wider source frame.