I found the problem. You're never supposed to create a format struct yourself, you query it from the core.
Code:
format->id = table[i].id;
strcpy(format->name, table[i].name);
format->colorFamily = table[i].color_family;
format->bitsPerSample = ah->bitdepth;
format->bytesPerSample = (ah->bitdepth + 7) / 8;
format->subSamplingW = table[i].subsample_w;
format->subSamplingH = table[i].subsample_h;
format->numPlanes = avs_is_y8(ah->avs_vi) ? 1 : 3;
return format;
should simply be:
Code:
return vsapi->getFormatPreset(table[i].id, core);
The whole function set_vs_format() function can also be simplified to one call to vsapi->registerFormat()
something like:
Code:
return registerFormat(cmYuv/cmRgb/cmGray, stInteger, bitsPerSample, subSamplingW, subSamplingH, core);
If you register a duplicate a pointer to the already registered format struct will be returned. => the same format always has the same pointer so the comparison I make is valid.