Reencode VC-1 to H.264

[magic144]

Hi,

I have recently bought Doctor Who (Complete 5th Series) on Blu-Ray and would like to do some h.264 encodes.

This one is a bit different.

a) It uses VC-1 instead of h.264.
b) It seems to be pulldown encoded - when viewing frame-by-frame it looks like 2:2:3:2:3

I have extracted one episode to .mkv with eac3to. eac3to reports 1080i60/1.001 for the VC-1 track.

Firstly, I used this script to verify that it is TFF and get the cadence...

DirectShowSource("video.GRF",audio=false)

AssumeFPS(30000,1001) # as per source

#USE Assume?FF().SeparateFields() separately to visually verify source
AssumeTFF().SeparateFields()

ConvertToYV12()

(fyi, the .grf is just a Haali source filter and ffdshow[wmv9] video decoder)


Q1. - how do I know the difference between hard and soft-pulldown? - presumably if the stream has "soft" pulldown, I should be able to extract the progressive frames somehow?


2nd, I tried the following to reconstitute progressive frames for x264, for a clip

LoadPlugin("C:\Program Files\MeGUI\tools\avisynth_plugin\TIVTC.dll")

DirectShowSource("video.GRF",audio=false)

AssumeFPS(30000,1001) # as per source

tfm(order=1).tdecimate(Mode=0,Cycle=6,CycleR=1)

ConvertToYV12()

TRIM(8000,9000)

The problem is, I still seem to get the occasional "blended" frame (where the 'lacing is still visible):-

http://i992.photobucket.com/albums/af43/magic144/Untitled.png


Q2. Am I using the right approach?? What can I do to improve my method??


and
Q3. If all else fails, can I simply reencode to h.264 PRESERVING the interlacing/cadence of the original (i.e. without trying to undo the telecine first)?


Thanks in advance of any and all help!

[Blue_MiSfit]

1) Correct me if I'm wrong, but if eac3to says it's 1080i60, it's actually encoded at 1080i60.

2) Any chance you can post a sample? It's hard to validate your script without having something to peek at.

3) If all else fails, yes just encode interlaced x264 by adding --tff to the command line (assuming your content is actually tff). I think you should be able to fix this source, unless it's field blended :devil:

Derek

[magic144]

Sure - how (and in what format) should I provide a sample? The .mkv eac3to produces for the video for this episode is ~14GB!!

I guess the other thing about recovering the progressive source is that I could do 720p resizes and such...

[Blue_MiSfit]

Use DGSplit or a similar app to cut ~100MB of the source. Upload this to mediafire or some other free file hosting service - preferably something OTHER than rapidshare.

Derek

[magic144]

Hi - I figured out how to do it with mkvmerge GUI just before I read your post
here's a 45s clip:-
*Link Removed - can reinstate link or clip on request*

I test-ran it through MeGUI and still got the interlacing using the script.

Thanks for the help.
m

ps - so if this is "interlaced" (hard-pulldown?) and full-frames are "progressive", how do you identify sources that have soft-pulldown? (maybe this only applies to SD stuff - I haven't dealt with these concepts since DVD days!)

[Blue_MiSfit]

Usually eac3to will remove any soft pulldown, so you usually don't have to deal with it :)

I'm taking a look at your sample though.

Derek

[magic144]

Cool - so that means this really is hard-pulldown... I wonder why they do this? Is it because it's a 25p source being sold in a North American market??

[Blue_MiSfit]

So this clip does not follow a standard 3:2 pulldown pattern.

It's something like 2i:4p:2i:4p:...3i:4p:2i:4p:2i:4p:...

So, it looks to me like special settings for TFM or Telecide will be necessary :) Forgive my laziness in determining these settings exactly!

Derek

[Sharc]

It looks like

yadif(mode=1,order=1) #or other bobber of your choice
srestore(frate=25)

would produce a perfect progressive judder-free playback.

[TinTime]

I've come across a few of these BBC HD sources in the past and did exactly what Sharc said with good results.

Try srestore(speed=-1) after bobbing. A negative value for speed is recommended for HD sources according to its readme.

[magic144]

Hmm, looking again it appears there is both
a) irregular cadence
b) field-blending (!)

thanks for the heads-up on srestore - I've never heard of that before - giving it a go right now!

so I guess straightforward IVTC is difficult/impossible for this source?!

*EDIT*
hmm, this is gonna take 11+ hours!!! (times 2!)

[InsulinJunkie]

so I guess straightforward IVTC is difficult/impossible for this source?!
For PAL->NTSC conversions, that's correct.

Could be worse. Could be a British program recorded off a local PBS affiliate, and having undergone all sorts of conversion nastiness at both the network and affiliate level :P

[magic144]

It's definitely already gone through some nastiness... They could have at least just done a soft-pulldown.
In some respects, the original TV caps from the UK at 25p are better!

I wonder what the UK Blu-ray release looks like?

Can they not just release 1080p25? (is that not in the Blu-ray standard?)

[Sharc]

I wonder what the UK Blu-ray release looks like?

Can they not just release 1080p25? (is that not in the Blu-ray standard?)
No, 1080p25 is not blu-ray compliant. 1080@25fps must be encoded interlaced

When you process your example with
yadif(mode=1,order=1)
srestore(frate=25)
you will obtain progressive frames at 25fps which you
could encode with x264 using --fake-interlaced to make it compliant for the playback devices.

[Blue_MiSfit]

Yeah, that was a substantial herp derp on the part of the BluRay spec ;)

[nm]

They could simply have slowed down to 24p if they want to target US audiences.

[rica]

EDITED: Sorry, i've missed that post.

[Sharc]

... Why don't you read before you post? :p

http://forum.doom9.org/showthread.php?p=1477502#post1477502

[TinTime]

I wonder what the UK Blu-ray release looks like?

Probably the same :(

[kieranrk]

Can they not just release 1080p25? (is that not in the Blu-ray standard?)

Some US HDTVs won't play 25fps material.

[magic144]

@TinTime - yeah, I guess so if 25p isn't in the standard! :-(

@kieranrk - sure sure, but couldn't they soft-pulldown encode it? that would seem a cleaner solution to me, without messing with the source (leaving it intact) - I remember the day I first found DGPulldown - that was a great day - finally I could reencode 25p source (PAL) material for smooth playback on my North American consumer devices.

[kieranrk]

@kieranrk - sure sure, but couldn't they soft-pulldown encode it? that would seem a cleaner solution to me, without messing with the source (leaving it intact) - I remember the day I first found DGPulldown - that was a great day - finally I could reencode 25p source (PAL) material for smooth playback on my North American consumer devices.

The way DGPulldown converts PAL to NTSC isn't a standard.

[magic144]

The way DGPulldown converts PAL to NTSC isn't a standard.

The way this disc has been hard-pulldown encoded (with seemingly random cadences) IS a standard??!

[Sharc]

The way this disc has been hard-pulldown encoded (with seemingly random cadences) IS a standard??!
I'd say the playback device doesn't actually care much how a blu-ray compatible stream has been produced (hard-pulldown, frame duplication, field blending, a mixture of all), as long as the resulting 1080 stream is presented to the player as being either
23.976 progressive
24.000 progressive
25.000 interlaced
29.970 interlaced
A standard-compliant stream does however not at all mean that it is nice to watch.

Soft pulldown at the other hand happens during playback. Not all playback devices/decoders are able to handle it properly.

[kieranrk]

The way this disc has been hard-pulldown encoded (with seemingly random cadences) IS a standard??!

Well sticking it through an alchemist or similar is sort of a standard.

[Sharc]

The way DGPulldown converts PAL to NTSC isn't a standard.
A bit off topic: Are the --pulldown .... options of x264 part of the h264 avc standard? Does the blu-ray standard include these?

[shon3i]

Pulldown is allowed by BD standard, and x264 will follow it.

[Sharc]

Thank you.

[magic144]

Hmm, I'm sure I had a successful encode the other day.
Now every time I try my script, it comes out with temporal glitches (video suddenly skips back a few frames from time to time).
Dunno if it's DirectShowSource - I tried FFVideoSource, but it kept failing using the .mkv as in input - maybe it doesn't handle interlaced VC-1?)

Only way I can think to do it now is to have an intermediate step to dump it to an .avi file (MASSIVE lossless, Huffyuv) - aarrgghhhh!!!

Here's how the script looks now:-
Import("Srestore.avsi")
LoadPlugin("mt_masktools-26.dll")
LoadCplugin("C:\Program Files\MeGUI\tools\yadif\yadif.dll")

DirectShowSource("video.GRF",audio=false)

AssumeFPS(30000,1001) # as per source

yadif(mode=1,order=1)
srestore(frate=25,speed=-1)

ConvertToYV12()

NB - when I skip through the script frame-by-frame in VirtualDub, it looks totally fine (output is frame-perfect)... but then I am only driving it forwards linearly, maybe that's different than how x264 is requesting frames??

[nm]

Now every time I try my script, it comes out with temporal glitches (video suddenly skips back a few frames from time to time).
Dunno if it's DirectShowSource - I tried FFVideoSource, but it kept failing using the .mkv as in input - maybe it doesn't handle interlaced VC-1?)

Yep. You could try Haali's DSS2 though.

[Didée]

You could try that script in panic mode:

(Maybe it's overzealous - but if it turns out to help ...)

Import("Srestore.avsi")
LoadPlugin("mt_masktools-26.dll") # you're using Avisynth 2.6? If not, you better load the ..-25.dll ...
LoadCplugin("C:\Program Files\MeGUI\tools\yadif\yadif.dll")
LoadPlugin("TIVTC.dll")

DirectShowSource("video.GRF",audio=false)
ChangeFPS(last,last,true) # small readahead buffer
AssumeFPS(30000,1001) # as per source

yadif(mode=1,order=1)
srestore(frate=25,speed=-1)
RequestLinear(clim=??) # default=10. Set it slightly larger than x264's actual RC-readahead (MB-tree)

# ConvertToYV12() # it already is, else Srestore wouldn't work

[magic144]

@nm - hmm, dss2 won't load the .mkv - keeps failing - works with AVC files, maybe, again, it's down to VC-1 interlaced - bah!

@Didee - thanks for the idea, never read about this before either - sounds like it should work but it still doesn't - even if I set rall=true

maybe the only surefire way to get this to work is to use this Huffyuv intermdiary (i.e. initially decode the whole file to LARGE .avi) - I tried this and it works OK - only problem is the source file is 14GB, the intermediary file is 188GB!!!

I guess if I had an NVidia card, I could try Donald Graft's DGDecNV - that's supposed to be about the only one that's frame-accurate!

Unless there's another VC-1 decoder I could try? I can't seem to use the WMVideo Decoder DMO itself - the graph keeps timing out... (Avisynth open failure: DirectShowSource: Timeout waiting for graph to start)

cheers,
m

[Blue_MiSfit]

Indeed, the testing I did was with DGDecNV. It's the only tool I've found that gives me reliable results with interlaced VC-1 all the time. WMVideo Decoder DMO is a pain sometimes, as you describe.

Derek

[Sharc]

Converting the interlaced VC-1 .m2ts to .mkv normally helps.

[magic144]

@Sharc - that's the first thing I do - eac3to extracts the video/episode I want into an mkv container with the interlaced VC-1 video inside.