Suggestions 1080i to x264 scripts?

[havix]

Just let me say first I'm a very long time lurker 5+ years. Usually I find what I want through search but this time I felt it necessary to post to get the answers I'm looking for. Recently I've started collecting many 1080i movies 7.5gb on up to 30gb. Well even though I'm at almost 5 terabytes of space I don't want to leave many of these movies at there original size. I'm looking for a some examples of what scripts other people are running on low-bitrate, high res material. I've done a few encodes of Men in Black that fit onto a single-sided dvd which turned out ok but I macroblocking is evident in certain scenes which I can alleviate somewhat by turning on the coreavc deblocking. The simple process I do is deinterlace with the analyze tool in MeGUI, overcrop to a mod-sixteen resolution with no resizing. Then a simple undot() and 2 pass with the hq-slow profile. I'm fine with a longer encode time since this runs on an extra machine but I would like to get the most that I can out of these TS rips. Perhaps a seesaw script, and something to help with macroblocking on low-bitrate hidef material???

[Blue_MiSfit]

sounds like you have your s*^( together :)

For better denoising, removegrain is much more up to date, and faster :)

mode=1 is undot
mode=2 is stronger
mode=5 is a happy medium (my standard).

Consider fft3dgpu (or its software cousin fft3dfilter if you don't have a badass dx9 card with 256+ mb) for tougher denoising. It can give you a really nice compressibility boost without hurting the source too bad.

As far as deblocking, try the different cpu modes in dgdecode. cpu=6 is the strongest.

Try also frameserving with ffdshow and use its postprocessing to get rid of blocks.

Alternatively downsize, this can kill blocks nicely...

MeGUI is a good choice for your deinterlacing needs, as long as you try force film in DGIndex first.

It may be simpler to check for 3:2 pulldown ahead of time (3 prog frames and 2 interlaced frames, etc....). If you see this pattern and it's consistent, just skip the MeGUI analysis and use (from tivtc)

tfm()
tdecimate()

It will do a nice job! Most movies are 3:2 pulldown from 24p to 60i. Lots of stuff is true 60i though, in which case you may be better off leaving it interlaced and using XviD which handles interlaced encoding.

~MiSfit

[Pookie]

I find macroblocks in 1080i material mainly on non film sources (Conan, Leno, The Office). The first season of 'Lost' in 720p had a habit of macroblocking on the first frame of every scene change.

Hint on SeeSaw - the lower you can keep the Nrlimit1 and Nrlimit2 settings, the faster it seems to run. The choice of denoiser also affects processing time, so there will be a big difference between FFT3dfilter and DegrainMedian or RemoveGrain.

Also, are you only deinterlacing your 1080i film, or are you bringing it back to 23.976 progressive? The quality difference is significant, and you can transcode at a higher bitrate. See BlueMisfit above :)

[sjchmura]

Can anyone recommend a good 1080i->720p script that is a source with some macroblocking....

I assume I shoud use the MEGUI recommend IVTC. But sounds like undot() is NOT what I should use??

Anyway, anyone have a nice script that gets 23.965 or whatever progressive FPS....??

[havix]

Pookie: To be honest I'm going to have to claim ignorance on this. I pretty much just run it through the automated d2v creator and run analyze. I'll look into it tomorrow though and see what dgindex is doing on the automated run. My whole goal of all of this is to create backups that are better then the original TS and within reach of a standard hd-dvd movie. I would like to wait until the smoke clears before I invest into a certain format. Hopefully these encodes will keep me happy until then.

[Blue_MiSfit]

@ sjchmura or havix:
You don't want to go down to 23.976 unless the content was originally 23.976 progressive (aka 24p) and then telecined to 29.97 interlaced (aka 60i). Most movies are, it just depends how they are.

Possibility 1: "soft telecine"
The video is actually 24p, but it's "flagged" as 60i. In other words when it gets decoded, the decoder outputs 60i. Not sure if this ever really applies to HDTV sources, but its totally the standard procedure for NTSC DVDs. Removing this is as simple as checking "Force Film" in DGIndex.

Possibility 2: "hard telecine"
The video is actually 24p, but has been converted to 60i. The original 24p is still there, but there's extra fields added when they interlace it in order to conform to 60i. Removing this requires an inverse telecine (IVTC) filter, like TIVTC. See my previous post for details about this. I think this is the most likely scenario if you are dealing with a hollywood film or high budget TV show. It's easy to spot by loading the video into virtual dub, and stepping through it one frame at a time. You will see 3 progressive frames followed by 2 interlaced frames, then 3 progressive, then 2 interlaced etc etc etc (3:2 pulldown, by far the most common telecine pattern).

Possibility 3: "really interlaced"
The video is 60i and was shot as 60i. If they shot the video on HDTV cameras this is very likely. You have two options. You can encode it at 60i, but this requires an encoder that supports interlaced encoding. x264 does not at the moment. There are some experimental patches that do IIRC, but I would stay away. XviD handles interlaced encoding, so that might be a good option.

The other option in this instance is to deinterlace. This converts 60i to 30p, so that you can encode with a progressive encoder like x264. That's all fine and well, but you can't keep ALL the information. It will either loose some detail or some fluidity of motion or have some artifacts. I'm no expert on deinterlacing, it's a hugely complicated topic requiring lots of experience and time to execute well. I would advise against it, but that seems to not be the consensus of this board. Oh well. I guess if you're showing it on a computer monitor you have to deinterlace sometime!

NOW... denoising:

removegrain(mode=1) is exactly the same as undot(), only faster. Start with this as a bare minimum. using mode=5 is stronger, and mode=2 is stronger still. I usually start with mode=5.

fft3dfilter (or its DirectX 9 hardware accelerated cousin fft3dgpu) are very powerful denoisers. They need some learning to tweak effectively though. Experiment.

Those are just some suggestions. There are scads of denoisers, each has a specific purpose. RemoveGrain has lots of modes.

If you want to downsize to 720p I think it's a good idea. Note that the p in 720p means progressive, so that means deinterlacing. Then you have to scale the output to 720p. In pseudocode it would be like this:


Mpeg2Source("X:\1080i.d2v")
MySuperCoolDeinterlacerOrIVTCer(...)
MySuperCoolDenoiser(...)
LanczosResize(1280,720)


Note that 720p generally refers to 720p60 - meaning 1280x720@ 60 progressive FPS. It doesn't always have to be though. If your movie is telecined from 24p you will have 720p24. If your movie is really 60i then you will have 720p30.

You're a lurker as you say, so you should be ok with doing lots of reading. Now you just have to try it all :)

I've been messing around for like 5 years and still know very little when it comes to the really nutzo stuff like deinterlacing and how the codecs really work under the hood.

Anyway, I hope I cleared up a few things without confusing you further or rambling excessively. Ask more questions!

~MiSfit

[tomos]

i've been converting 720/1080 broadcasts for over a year now. originally, i tried to make the scrip myself, but for a while now i let megui create it for me.

does the job very well.

i encode 1 mov to 1 dvd-r size unless its a larger film in which case it gets the dual layer treatment.

i always add

deblock and undot at the end.

end result looks very good.

the ROTS encode i did looks great on my 30" dell

[Blue_MiSfit]

I would imagine that deblocking would be essential for broadcast HD stuff, as the jackass companies try to cram as much content onto their existing bandwidth as possible...

What kind of deblocking do you use? DGDecode's, ffdshow's, or an avisynth filter?

~MiSfit

[tomos]

avisynths own i believe. someone on another forum advised me to try just adding those lines and it works a treat. much cleaner image in the output.

i dont add any extra dlls for deblock so i believe its avisynths own.

[sjchmura]

Yes, the deblocking is one area I wonder about. Should we use "CPU=5" in avisynth or let the deblocking in x264 handle it....

Also with MEGUI it uses "undot" which is MUCH MUCH slower (tested last ngiht) than removegrain(mode=1)

Has anyway gotten "removegrainSSE2" (the optimized one for AMD 64 etc) to work??? I keep getting an error?

[tomos]

i set x264s deblocking to -2 on both and that deblock() line

does that remove grain do the job as well as undot?

[Blue_MiSfit]

from what I understand, the x264 deblocker is NOT for deblocking source material. It's for compensating for blocks produced in the encoding process.

The SSE optimized versions of removegrain work perfectly fine for me. Perhaps you are missing a visual C runtime or something like that??

@tomos: if you read my previous posts, you will see that removegrain(mode=1) is the same as undot(), only faster.

[havix]

Sorry it's taken so long to respond back on this thread but I've been busy with a new server build and an OTA HDTV tuner. Alright so with my new Core 2 Duo clocked at 2.1Ghz I am able to complete a standard HDTV movie encode in a little over a day. This is by just overcropping to achieve mod-sixteen (no resize) ,color correction, dgdecode deblocking is set to "CPU=4", and removegrain(mode=1). All being encoded into X264 HQ-Slow with 2 threads. I can accept a day or so per movie. Thanks everyone for all the help, especially Blue_Misfit, I keep referencing your deinterlacing comments when I'm verifying the proper deinterlacing method for every new encode. Thanks again!