Deinterlacing issues with a Sony m2ts (h264/avc interlaced) file

[Blimblim]

Hi everyone!

I'm a rather long time user of Avisynth, been using it for years now for all my video encoding needs. I can't say I totally understand everything I do with it, but still I'm totally happy.
Anyway, I bought a Sony HD camcorder that records H.264/AVC m2ts stream in 1080i. I'm relatively happy with the video quality, but I have a rather strange problem with the deinterlacing part.
I want to produce some 59.94 fps videos, and for some reason it's like the fields are totally out of order. Deinterlacing to a 29.97 fps file works totally normally with either tomsmocomp or smoothdeinterlace, but if I use the doublerate=true mode or this script :
Top=AssumeFrameBased().ComplementParity()
Bottom=Top.SeparateFields.Trim(1,0).weave
Interleave(Top.TomsMoComp(1,5,0),Bottom.TomsMoComp(0,5,0))
I get some very weird video where frames are duplicated, go back in time, you name it. Of course I try using tff or bff with the same result.
Here is a full script :

loadplugin("tomsmocomp.dll")
loadplugin("SmoothDeinterlacer.dll")
directshowsource("c:\capture\00036.MTS",fps=29.97)
tomsmocomp(1,15,1)
#SmoothDeinterlace(tff=true, doublerate=true)
lanczosresize(640,360)
This script works correctly and produces a 29.97 clip with no issues
Removing the fps=29.97 option, I get a video that's twice too fast. Using the smoothdeinterlace line, I get a this weird issue.
I'm using the CoreAVC Pro codec, with the option not to deinterlace anything. My guess is that the codec is somehow doing something wrong with the fields.
I can encode a small part of the video to another codec so you guys can have a sample if you want. Just tell me exactly what would be best for you to check the problem. I can also provide a sample m2ts file of course.

Thanks in advance for your help!

Jeff

[Pookie]

Would you mind posting an original Transport Stream sample? :)

[Blimblim]

Sure thing, here is a sample I just recorded, I tested it and it does the same thing as the other ones.
http://media.xboxyde.com/misc/cat.mts

[Pookie]

I couldn't get the file to open via directshowsource (although I know the problem is on my side). Converted the format and tried a couple of tests:

A regular Bob() created what you seem to describe - jumpy, jumbled frames in the resulting file.

This combo seems to work: (thanks Tritical, Chainmax, Foxyshadis)

Interp = SeparateFields().EEDI2(field=3)
TDeint(mode=1,order=1,edeint=Interp)


or if your source is BFF (I can't tell, since I had to convert your source to open it)

interp = SeparateFields().EEDI2(field=2)
TDeint(mode=1,order=0,edeint=Interp)

I think I recall something being strange about the interlacing in AVCHD. I'll look around for the article. Meanwhile, would you test the above and see if it helps your problem ?

[Blimblim]

Just gave it a try, same result.
http://media.xboxyde.com/misc/deint.avi
I guess it's a codec issue then?

[Blimblim]

I just noticed that the CoreAVC codec has a setting to double the framerate when deinterlacing, and lo and behold I get the exact same result. So I guess this definitely is a codec problem.
I'll post on their forum and see if they have a solution or not. I'll keep you guys informed if you want.

[Pookie]

It might be. Here are my results from the video you posted above.

http://www.mytempdir.com/1026059

Used these tools to convert the transport stream:

http://forum.doom9.org/showthread.php?p=891337#post891337

loadplugin("rawsource.dll")
rawsource("E:\decode.yuv",1440,1080,"I420")
Assumefps(29.97)

[foxyshadis]

Maybe it was just recorded badly. There are a lot of simple things you can do to create some pretty pathological video.

[Blimblim]

It might be. Here are my results from the video you posted above.

http://www.mytempdir.com/1026059

Used these tools to convert the transport stream:

http://forum.doom9.org/showthread.php?p=891337#post891337

loadplugin("rawsource.dll")
rawsource("E:\decode.yuv",1440,1080,"I420")
Assumefps(29.97)

Hmmm the bob result looks to be the same as what I get by using directshowsource, and the tdeint one is 29.97 so it's something that's already working correctly. Still, interesting to see that the "hard way" seems to be leading to same results (?)

foxyshadis: Well, this mts file is straight from the camcorder's hard drive. Of course it's possible that Sony did something wrong.

[Pookie]

The Tdeint was set to bob, so both clips on Kitty_xvid.avi are displaying 59.94fps

[Blimblim]

The Tdeint was set to bob, so both clips on Kitty_xvid.avi are displaying 59.94fps

I went through the movie frame by frame, and the one on the right has every frame duplicated :)

[ebc]

I'm not sure if you managed to work it out or not but I'll post what I have found.
Ever since I got my HDR-SR1 I've been trying to do the exact same thing as you have and through a lot of trial error I guess I worked something out.

The first thing I found was that for me, CoreAVC wouldn't play back my .mts files properly and it would do exactly what you found with the problem of jerky behaviour when you turn on the framerate doubler.

So I tried demux but I couldn't ever seem to get this (http://forum.doom9.org/showthread.php?p=891337#post891337) method to work.

Eventually I found Xmuxer (http://www.elecard.com/products/products-pc/consumer/xmuxer-pro/) and discovered that by using the remux feature and remuxing to mp4 as the output streamtype excluding the ac3 audio (I just demuxed that and left it to convert later), it would make a perfectly playable file using CoreAVC.

So then using this script:
directshowsource("f:\Temp\cat.mp4", fps=59.94, framecount=508)
lanczosresize(640,360)

I was able to encode your cat video:
http://www.megaupload.com/?d=URR0Y0GG

The script only works if CoreAVC is definately the decoder being used by directshowsource and also only if you have turned on the framerate doubler. I haven't tried leaving it off and using smoothdeinterlacer yet but I think that would be incredibly slow.
The reason for fps=59.94 is cause directshowsource is expecting 29.97 and without it in there the video will be slowmotion.
Then there's framecount=508 which you'll see is double the original number of frames of the sourse, I had to put that in because without it it would end the video exactly halfway through and not go right to the end. I guess that's directshowsource once again only expecting 29.97fps instead of 59.94.
So basically for that value just double the number of frames of the original video.

Hope that kinda helps. I guess if I could get the xport demuxing method to work on my .m2ts files then the process would be all free but I've just been using the Xmuxer trial cause it's the only thing I can get to work.

Here's one of my own clips: (note my camera is pal)
http://www.eyeballcancer.com/videos/flower_xvid.avi