Simpsons OTA .ts capture avs script help

[drtj]

Hello everyone,
Just was wondering if anyone could help me out with my avs scripts that I am using to convert .ts captures of The Simpsons and The Family Guy to avi. I see a number of posts of how to convert DVDs to avi, but I want to know what to use for OTA captures.

When I process the .ts through DGIndex, it states that the frame type is progressive, bottom field field order, frame rate 59.940060 fps, video type NTSC.

The avs script that I am using is the following:

SelectEven()
Telecide(guide=1,post=2,vthresh=15,dthresh=2)
Decimate(cycle=5)
crop(164,8,-164,0)
RemoveGrain(mode=1)
RemoveGrain(0,11)

As you can see I was trying to set Decomb to "hit a little harder", but still have some combing seen in the .bmps that I have attached.7527

7528

7529

7530


Basically I am really happy with my encodes except obviously for this combing effect, and just was wondering what I could use to better my results or the correct tweaking of Decomb.

Thanks for all the help that I have gotten from this awesome resource and for anyone who is kind enough to help me out with this specific request.

drj

[Guest]

Those resized BMPs are useless. Post a transport stream fragment that allows us to duplicate your issue.

[drtj]

Sorry about that.
I have posted a sample here:
http://www.bestsharing.com/files/JBXXim325645/2.ts.html

If there is anything please let me know

Thanks

[Guest]

If there is anything please let me know
Maybe choose an upload site that isn't limited to 9 KB/s! If the download completes, I'll have a look at it.

[drtj]

well this is link to rapidshare:
http://rapidshare.com/files/51087871/2.ts
I am trying to find a faster 1. Any suggestions?

If that is still too slow I will look into using a pay site. Again, any suggestions which ones are good?

Thanks again for taking the time to look into this and your help.

[Guest]

I have the stream. The stream is pathological. There are field blends but the pictures of the blend are combed, but not with a good spatial frequency of two. It may be a naive reduction from 1080i to 720p. Here is an example field:

http://neuron2.net/misc/badfield.png

I could probably write a filter to detect the bad fields and avoid them, but what could motivate me to do that? :)

I'd write it off if unless it was really important to me.

Did you capture that? Do all of your captures show that pathology?

[drtj]

I would really appreciate your help. Or anyone’s for that matter. I know there are captures and Divx encodes that eliminate these frames in the final product out there.

I know that writing a filter is a great deal of work, so any suggestions of filters that might help or any way I can avoid these pathological frames would be a great help.

It is important to me :) because I have numerous Simpsons and Family Guy captures and they ALL end up looking like that somewhere in the clip. So yes, ALL my captures of recent (this season’s episodes) from the station that broadcasts them (can I say which station?) have these pathological frames in them at some point in the show.

[Guest]

The basic idea is simple. Write a filter that detects the bad frames and writes an override file for Decimate() to force them to be decimated. I have made the override file by hand for your clip and the result is fine. The problem is that the combing does not have a spatial frequency of 2, which is what all the interlace detectors are designed for. I tried a couple deinterlacers and they aren't doing a good job of detecting those frames. So someone would have to write a special filter to look for that spatial frequency. But it may be unique to this stream.

Like I said it is an issue of motivation. Why would I sit down and spends hours, maybe days implementing this? Would you like to engage me as a consultant? If so, it's $125 per hour. :) Maybe there's someone out there with the skills and looking for something to do that will do it for you, but I wouldn't hold my breath.

If this is that important to you, you can laboriously create the Decimate() override file by hand. I hope you are a patient guy.

[drtj]

Well that price tag makes this just a little less important to me:eek:

Not to say that I don’t value your work neuron2, or that you shouldn’t be paid for your time and effort. I can say with certainty that your guides and filters are the definitive source for all those who start out capturing and editing video I have read and reread your guides so many times, using them as valuable tools for reference in regards to any questions or problems that I have encountered when it comes to frame serving and encoding my captures to Divx. Anyone who has ever asked me how I can do what I do with these files, I point them directly to your website.

It is just when you go from hours to days at 125 a pop, 2 days work then becomes $6000. And from the sounds of it it isn’t something you are too keen on doing. I am sure that if this was something that benefited the masses it would already have been done.

From what I gather reading your last response, the reason why writing this script is difficult is because that these pathological frames have no specific reproducible pattern? And so writing a filter that will go through and treat a video and remove these frames may be specific to each source .ts?

So when you say “that spatial frequency,” is there a spatial frequency in the clip that I sent you that would take care of the whole episode? And if so, what is it please?

Could I get a peak at the filter that you wrote for the clip that I sent you? I was hoping to learn from an example of what my script should look like and what settings you were using.

How can I create the Decimate() override file by hand?

I can be a patient guy, especially when it comes to figuring things out. There is nothing like the sense of accomplishment you get when you set out to do something on a computer, get to where you want to go, and get a working and beautiful end product. And the fact that there are codes and syntax makes it all the more difficult, but in the end, that much more rewarding.

Thanks again neuron2 for all your help, time, and expertise. As always it has been very appreciated and valued.

drj

[Guest]

That's right. You have a specific source stream that appears to have been crappily converted from 1080i to 720p. Yes, I could write a filter to detect those bad frames, but it would help only you. I'm very busy so how could I justify it? Do you have something I want? :) Maybe we could frame it in terms of "it would be nice to have a general purpose filter to detect a configurable vertical frequency above threshold" and yes, I agree, but I don't have time for it right now. Maybe someone will step up. Or you could learn to program Avisynth filters.

If you look at one of the bad frames you see a spatial frequency in the combed areas of ~4. Real interlacing is 2. That would take care of the whole episode if it is indeed a result of downsampling interlaced content.

I didn't make a filter. I just made a file that lists the bad frames, like this:

347
357
372
392
397
408
413
418
423
428
433
438
443
448
453

Then I passed that file to Decimate() as the override file (see manual). You would need to write a filter to detect frames with greater than a threshold amount of freqency 4 vertical content and save the frame numbers to a file. Then use the file to override Decimate.

[2Bdecided]

There's some kind of pattern to the location of bad frames. Depending on how consistent it is and if it breaks anywhere, that's probably the easiest approach - just generate the "bad frame" file using a simple program (or Excel!)

At $125 per hour, I think it might be worth tracking down this obscure content on DVD!

Cheers,
David.

[jmac698]

you need to use the cdeblend filter, there's several filters that detect blended frames. It comes up often when converting from PAL to NTSC.
http://forum.doom9.org/showthread.php?t=103940

[drtj]

Thanks jmac698 for the tip. I have tried the filter and it works great at eliminating the blended frames. This is getting me much closer to what I want.

The only thing that I am getting now is some slight interlacing /combing artifacts. Now it is not the whole frame being blended, but when the eyes blink or when their heads move or around their mouth/teeth I see random interlaced artifacts here and there, again the whole frame isn’t interlaced, only where the movement occurs. I have attached some pics to demonstrate. Cdeblend made it much better, thank you, but how to drive this one all the way home?


This is the script that I have so far:
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\DGDecode.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\TIVTC.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\RemoveGrainSSE3.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\Decomb.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\mt_masktools.dll")

mpeg2source("I:\1\1.d2v")

SelectEven()
Telecide(guide=1,post=4,vthresh=15,dthresh=1)
Decimate(cycle=5)
import("Cdeblend.avs")
Cdeblend()
crop(164,8,-164,0)
RemoveGrain(mode=1)
RemoveGrain(0,11)

Can I maybe tweak Cdeblend to catch those frames and delete them or use another filter that will take care of the partly interlaced frames?

Again, thank you so much for your help and input. I do so greatly appreciate it.

drj

[MOmonster]

@drtj
Mrestore produce a nice output for your short sample. I just use this on your clip:
Mrestore(numr=2002,denm=5005)
Maybe restore24 will also produce some nice results.
For tweaking the blenddetection you can higher the rr value a little bit (for example 3.0), but only if there are still some of these blends left.

[drtj]

Thank you very much MOmonster. Mrestore worked perfectly and that was exactly what I was looking for. It completely took care of the combed fields.

For anyone else that is wondering this is what my final script looks like and works to my satisfaction for the Simpsons:

Loadplugin("C:\Program Files\AviSynth 2.5\plugins\DGDecode.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\TIVTC.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\RemoveGrainSSE3.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\mt_masktools.dll")
mpeg2source("I:\1\1.d2v")

SelectEven()
Decimate(cycle=5)
import("Mrestore.avs")
Mrestore(quality=2,mlimit=0.1)
crop(164,8,-164,0)
RemoveGrain(mode=1)
RemoveGrain(0,11)


Thank you once again for your help.

[drtj]

@ jmac698

Thanks for the tip to you as well. I did try Cdeblend, but like MOmonster pointed out, Mrestore works much better and takes care of all the blends, big and small.

Thanks for taking the time out to read and respond to my post.

drj

[MOmonster]

mpeg2source("I:\1\1.d2v")

SelectEven()
Decimate(cycle=5)
import("Mrestore.avs")
Mrestore(quality=2,mlimit=0.1)

I can understand quality=2 and mlimit=0.1, but why selecteven + decimate?
With selecteven + decimate you already reduce the framerate to 23.976fps and Mrestore will reduce the framerate ones more.
The code I posted:
Mrestore(numr=2002,denm=5005)
gives a fluid output at 23.976fps. Mrestore is not more than a smart decimator with a blenddetection.

[drtj]

The script that I was actually using looks like this:
import("Mrestore.avs")
Mrestore(quality=2,mlimit=0.1)

Without the decimate and selecteven.
I realized that with the decimate and selecteven I was getting a frame rate of 11.496

So can I incorporate the Mrestore(numr=2002,denm=5005)

into what I have already as: Mrestore(quality=2,mlimit=0.1,numr=2002,denm=5005)

It is funny but my next question was going to be about frame rate and adding the numr=2002,denm=5005 gives me a perfect 23.976 fps which is much smoother, Without that my frame rate was 25.000.

So thanks again MOmonster. You have been a great help.

drj