Getting YATTA & srestore to play nice...

[Stereodude]

I have a 1080i60 source (recorded OTA ATSC HDTV) that contains progressive 1080p30 encoded as interlaced, true 1080i60, telecined 1080p24 to 1080i60, and field blended 1080p24 to 1080i60. I decided to give YATTA a try to automate going to 1080p VFR file.

Here's what I want to do:
1080p30 encoded as 1080i60 - pass as is
1080i60 - QTGMC and select even to 1080p30
telecined 1080p24 - IVTC to 1080p24
fieldblended 1080p24 - QTGMC + srestore to 1080p24

I'm pretty sure I've figured out how to handle the first 3 in YATTA. In the last, I can apply a preset consisting of QTGMC + srestore to the field blended sections, but it seems that YATTA doesn't have any way to deal with the fact that QTGMC + srestore returns a frame rate of 24000/1001 and not the 30000/1001 it's expecting. As a result the QTGMC + srestore sections are not put into the timecode file as 24000/1001 and it all goes wonky.

Short of doing a selectevery(4,0,1,2,3,3) after QTGMC + srestore to give YATTA a duplicated frame to decimate and then see it as 24000/1001 is there any way to handle this?

[osgZach]

I think your best bet might to be to break those sources up and process them as independent clips. As long as you are producing either 30 or 24 material you can merge them all back into an output file and its just a matter of manually writing your timecode ranges. At least the theory sounds good in my head, anyway.

Frankly this clip sounds like a nightmare to process, and just doing 30/24 FPS material in Yatta can be exhausting itself.

[Stereodude]

I thought about that. My reluctance is that the clip isn't a 5 segments each one a different type, but it would be more than 100 segments (I'm not entirely sure of the number since I haven't even sorted them all out let alone counted them up yet). Some are only a few seconds long. To make matters worse there's actually a 5th type of content I forgot to mention. 1080p24 with a duplicated frame to be 1080p30. This of course can be easily decimated, but it's just something else to deal with.

[osgZach]

An equally masochistic and potentially more difficult way might be to mark those frames as no-touch (don't PP, don't decimate, etc) and let it create an output timecodes, then manually feed those frames to a filter that can handle them in your encode script, merging them back into the clip afterwards, taking note to adjust your timecodes for the adjusted frame numbers?

You'd still be dealing with a bunch of split / trim / and merging operations but you could at least keep it confined to a single script maybe.

[Stereodude]

I decided to skip YATTA and go manual. I can actually process the whole clip the way I want with only two paths. Everything goes through TFM. I have an ovr file so that everything except the portions that need IVTC will pass through untouched. That's then fed into QTGMC. IMHO, QTGMC when set correctly doesn't negatively affect progressive content and it cleans up ugly interlaced MPEG-2 artifacts that would otherwise persist. The fieldblended portions see srestore & selectevery(4,0,1,2,3,3). All other portions get a selecteven(). This goes to a Lagarith compressed CFR intermediate file with no combing or interlacing that just needs decimation. Tdecimate + a little manipulation with an override file should get me the 30000/1001 & 24000/1001 FPS VFR output I'm after with an automatically generated timecode file (that will need some minor adjustments to correct TDecimate's forced 5 frame alignment).

The script to create the intermediate file looks like this:LoadPlugin("DGDecode-x86.dll")
LoadPlugin("TIVTC-x86.dll")
LoadPlugin("mvtools2.dll")
LoadPlugin("GRunT.dll")
Import("QTGMC-3.32.avsi")
import("Srestore_27g.avsi")

function QTGMCrestore(clip bobbed, int framestart, int frameend)
{
srestored = bobbed.trim((framestart*2),((frameend+1)*2)).srestore(frate=23.976023976023976023976023976024)
final = srestored.selectevery(4,0,1,2,3,3)
return final.trim(0,(frameend-framestart))
}


source=MPEG2Source("source.d2v", idct=5, moderate_h=40, moderate_v=40, cpu2="xxxxoo")
source.TFM(order=-1, mode=5, PP=0, slow=2, d2v="source.d2v", display=false, ovr="tfm_ovr.txt")
qtgmc2=QTGMC( Preset="Medium", InputType=0, Sharpness=0.2, ShowSettings=false )
qtgmc1=qtgmc2.selecteven()

qtgmc1.trim(0,7161) + QTGMCrestore(qtgmc2,7162,7526) + qtgmc1.trim(7527,7611) + QTGMCrestore(qtgmc2,7612,7881) + \
qtgmc1.trim(7882,7956) + QTGMCrestore(qtgmc2,7957,8168) + qtgmc1.trim(8169,8241) + QTGMCrestore(qtgmc2,8242,8419) + \
qtgmc1.trim(8420,8507) + QTGMCrestore(qtgmc2,8508,8644) + qtgmc1.trim(8645,8770) + QTGMCrestore(qtgmc2,8771,8927) + \
qtgmc1.trim(8928,8996) + QTGMCrestore(qtgmc2,8997,9028) + qtgmc1.trim(9029,9134) + QTGMCrestore(qtgmc2,9135,10032) + \
qtgmc1.trim(10033,10041) + QTGMCrestore(qtgmc2,10042,10380) + qtgmc1.trim(10381,11183) + QTGMCrestore(qtgmc2,11184,11216) + \
qtgmc1.trim(11217,-0)

[osgZach]

I would have suggested something like that too, but I thought it would be easier to just do it from YATTA since you'd have a GUI helping you step through and mark your overrides faster, etc. I mean, that's essentially what YATTA does anyway.

But either way, glad you found a solution that works for you

[Stereodude]

I would have suggested something like that too, but I thought it would be easier to just do it from YATTA since you'd have a GUI helping you step through and mark your overrides faster, etc. I mean, that's essentially what YATTA does anyway.

But either way, glad you found a solution that works for youI did not find YATTA the slightest bit intuitive to use. It's clear it wasn't designed around HD footage either. I found it more helpful to save the stats from TFM in a 1st pass, then load them back in a 2nd pass (for speed) with display=true and play the clip in VD at 60fps and watch the upper left corner. YATTA's use of the scene detection information from TFM to make segments wasn't particularly useful to me.

[osgZach]

Well, YATTA is pretty much infamous for being the masochists IVTC tool of choice for Anime. I don't think its ever claimed to be intuituve though :p HD footage can be a problem if you don't have a large large screen, I suppose. But generally most HD Anime tends to be 24p these days, thankfully.

[Stereodude]

But either way, glad you found a solution that works for youGrr... the solution doesn't work. The audio and video didn't stay in sync. My subfunction didn't correctly compensate against srestore's rounding behavior. Srestore uses the equivalent of rounddown which means it returns too few frames where there is a fractional component.

It's like this:
25 frame input = 20 frame output
26 frame input = 20 frame output
27 frame input = 21 frame output
28 frame input = 22 frame output
29 frame input = 23 frame output

Assuming you could have fractional video frames it should be:
25 frame input = 20 frame output
26 frame input = 20.8 frame output
27 frame input = 21.6 frame output
28 frame input = 22.4 frame output
29 frame input = 23.2 frame output

If you were to run an entire video through it doing 60i to 24fps you'd be short by a maximum of .8 frames (24fps frames) worth of video duration (33ms). It's not a big deal because the error is at the very end, but when you use srestore many times in small pieces through a video the rounding errors really add up. ~408ms across ~23800 frames with 24 calls to srestore per my calculations in my specific video.

I originally attempted to compensate by making sure the subfunction returned the same number of frames as the original video source, but that's not correct either as 5-> 4 frame decimation (via duplicated frame removal) doesn't get it back to the correct number in most case. This meant my "correction" more or less rounded up to the next integer which has the same problem but in the opposite direction. In my case the audio & video were off by ~300ms after ~23800 frames with 24 calls to srestore.

I changed the subfunction to this:
function QTGMCrestore(clip bobbed, int framestart, int frameend)
{
frames = ((frameend-framestart)+1)
extra = Round(Frac(frames/1.25))
srestored = bobbed.trim((framestart*2),((frameend+extra)*2)).srestore(frate=23.976023976023976023976023976024)
final = srestored.selectevery(4,0,1,2,3,3)
return final
}Which means it now does this:
25 frame input = 20 frame output
26 frame input = 21 frame output
27 frame input = 22 frame output
28 frame input = 22 frame output
29 frame input = 23 frame output

My calculations now show that my overall audio video synchronization error should only be 8ms after ~23800 frames with my 24 specific calls to srestore. Now to see if it works that way in reality. :)