Yet another method to convert 120fps avi files to vfr matroska files

[Kurosu]

aka Un120: mostly harmless

Link for information:
http://forum.doom9.org/showthread.php?threadid=68562

To illustrate the example from another post (http://forum.doom9.org/showthread.php?threadid=68562#post428565) in a not so far away thread, here (http://kurosu.inforezo.org/avs/Un120.zip) is an avisynth filter that will parse AVI filestream to determine dropped frames and output the other as a video clip.

Nota: it is almost unusable as is
However, I'm sure people will be as interested as I, if not more, to fix them <g>. Here's the list of the problems:
- last group of frames might not have a proper frame number (ie "360, 360" may be emitted while "360, 361" was expected)
- if the avi file has audio, it will fail
- only the debug part really is likely to work
- maybe others, but I haven't tested this filter for that

Drawbacks:
- will parse on every pass (haven't tested with long files, so it might not be overkill)
- very, very, very incompatable: I guess it will only read AVI 1.0 files

How it works:
1) On load
- read at offset 140 number of frames, and verify it has the right number
- search for string "movi00dc", which should mark the start of the first frame, then rewind a bit
- verify if string 00dc is there (=a frame)
- skip beyong 00dc, read int32 size, skip 'size rounded up to nearest even number' bytes
- repeat until end of file or sufficient frames were read
It produces with that a list of dropped frames and a proper lookup table. It can also write the timecode file for mkvmerge.
2) Outputting frames
The lookup table make it returns immediately the right frame

What now?
Proof of concept: source code is provided under the LGPL license, it needs fixing. I have only checked quickly the patterns, so it might be much more buggy than HeadlessCow method. For the adventurous user, a quick readme is included.
Anyway, I'm waiting for any reply to see if it is worth supporting, fixing and releasing new versions of it. <g>
In addition, a tool like VirtualDub_Mod could have an option to parse the AVI file it's fed. In case the output file is detected as matroska:
- if direct streamcopy is used, only outputs to the mkv the non-dropped frames, and appropriately set the timestamps.
- otherwise, feed the coder with only the non-dropped frames, and act as described above
- for display, a request to a dropped frame would result in fact in displaying the previous non-dropped frame, bypassing any processing if the video is played by increments of 1 frame at a time.

Various corrections and ideas if anyone cares

[HeadlessCow]

Moo...can you repost your code please? I'd like to take a look at it.

Edit: link works today...nevermind :)

[Kurosu]

Originally posted by HeadlessCow
Edit: link works today...nevermind :)
Yup, system update...

Anyway, here is an updated and more efficient code:

/***************************************************************************************/
/* Function name: ScanStats()
/* Function: Scan AVI chunks for dropped video frames
/* How: - Reads important data from file (dimensions, length) then get to movi chunks
/* - parse bitstream for markers, avoiding non-video chunks and skipping to
/* appropriate portion.
/* - In case too many unknown chunks are found, processing fails
/***************************************************************************************/
uint32_t ScanStats()
{
FILE *in;
bool TryResynch = true;
uint32_t i, size, offset = 0, weird=0;
uint16_t width, height;

droplist = NULL;

/* Don't ask me why I use both FILE and new in the same program */
if ((in=fopen(Input, "rb")) == NULL)
{
fprintf(stderr, "Couldn't access %s for binary reading... Exiting", Input);
CleanUp();
exit(ERROR_IO);
}

/* Reading FPS stuff and abusing variable names*/
fseek(in, 0x80, SEEK_SET);
fread(&size, 1, 4, in);
fread(&offset, 1, 4, in);
FPS = (float)offset / (float)size;

/* Reading number of frames */
fseek(in, 0x8C, SEEK_SET);
fread(&NumFrames, 1, 4, in);

/* Reading video dimensiosn */
fseek(in, 0xA0, SEEK_SET);
fread(&width, 1, 2, in);
fread(&height, 1, 2, in);

printf("Scanning AVI file %s:\n", Input);
printf(" * Dimensions: %ux%u\n", width, height);
printf(" * Number of frames: %lu\n", NumFrames);
printf(" * Native framerate: %.4ffps\n", FPS);

/* Find this = find first frame */
if (SeekString(in, "movi", 4) == -1)
{
fprintf(stderr, "Because of my crappy code, I couldn't find AVI data");
CleanUp();
exit(ERROR_READ);
}

droplist = (uint8_t *)malloc(NumFrames);

i = offset = 0;
fseek(in, -1, SEEK_CUR);
/* Parsing frames */
do {
/* Check if we're on a video frame */
fread(&size, 1, 4, in);
if (size != 0x63643030) //Not video chunk
{
if (TryResynch)
{
/* Last chance */
if (size != 0x62773130) //Not audio chunk
{
fprintf(stderr, "Position %lu isn't the expected video frame %u (%lX)",
ftell(in), i, size);
CleanUp();
exit(ERROR_READ);
}
}
else
{
TryResynch = true;
weird++;
}
}
else TryResynch = false;

/* Get size and act accordingly */
fread(&size, 1, 4, in);
if (size == 0)
{
if (i == 0)
{
fprintf(stderr, "dropping frame 0? What the?");
CleanUp();
exit(ERROR_ABNORMAL);
}
droplist[i]=1;
}
else
{
droplist[i]=0;
offset++;
}
i++;

/* Now seek to the next chunk */
fseek(in, ((size+1)>>1)<<1, SEEK_CUR);
} while (i<NumFrames && !feof(in));
if (weird) printf(" * Skipped: %u non-video chuncks\n", weird);

if (offset>NumFrames)
{
fprintf(stderr, "Unvalid data: %lu frames dropped for %lu frames",
offset, NumFrames);
CleanUp();
exit(ERROR_ABNORMAL);
}
if (offset == NumFrames) fprintf(stderr, "No dropped frames!");
else printf("\nDropped frames: %lu/%lu video chuncks\n", NumFrames-offset, NumFrames);

fclose(in);
return NumFrames-offset;
}


(I could have just linked to a file but that's the only relevant part of the code)