Interlaced encoded as progressive and repeated fields

[FranceBB]

Hi,
I'm in troubles.
The service who shot and edited a beach soccer match delivered a mess: the file has been encoded (by them) in H.264 60fps progressive, but it's actually interlaced.
The problem is that fields are screwed up and I can't solve the problem by simply bobbing everything.
By applying plain and brute bobbing, I notice that chroma goes outside the luma boundaries and it looks simply awful. TDeint fails to deinterlace it properly and creates many artifacts with still noticeable fields in the picture. Libav Deinterlace with linear blending gives a somewhat better result, however I noticed that there are still repeated frames left.

Here's why I'm in trouble: air time is tomorrow and my target is 25i, so either 25fps progressive flagged as interlaced or 50fps progressive divided in fields to get a truly interlaced 25i. I've already contacted the service to send another file that is not so screwed up, however they won't be able to send it 'till tomorrow, therefore I gotta try to make something out of what I currently have.

I have uploaded a few sample: a lossless uncompressed one (but beware that the source they sent was already compressed) of 10 and 20 seconds and a long sample that I re-encoded in H.265 10bit leaving everything as it was (fields as they were) so that you can play with it.

Long Sample (lossy): https://we.tl/t-qaUrbdD8ns
Short Sample 1 (lossless): https://we.tl/t-yuNOTvgnzG
Short Sample 2 (lossless): https://we.tl/t-IUKdRvriLf
Short Sample 3 (lossless): https://we.tl/t-dQdrJm6Mq0
Short Sample 4 (lossless): https://we.tl/t-bocAED6sJb

Thank you in advance,
I have faith in you,
Frank.

[ChaosKing]

There is probably a better way. But this looks at least more or less ok. Or lets say it a start... :D

clip = core.lsmas.LWLibavSource(source=r"D:\Download\Sample_short.mkv")
clip0 = clip.std.SeparateFields(0).std.SelectEvery( cycle=2, offsets=0)
clip1 = clip.std.SeparateFields(1).std.SelectEvery( cycle=2, offsets=1)

clipi = core.std.Interleave([clip0, clip1]).std.SelectEvery( cycle=2, offsets=0)
clip = core.std.DoubleWeave(clipi, tff=True)

clip = haf.QTGMC(clip, TR2=1, TFF=False).std.SelectEvery( cycle=2, offsets=0)

[FranceBB]

There is probably a better way. But this looks at least more or less ok. Or lets say it a start... :D

clip = core.lsmas.LWLibavSource(source=r"D:\Download\Sample_short.mkv")
clip0 = clip.std.SeparateFields(0).std.SelectEvery( cycle=2, offsets=0)
clip1 = clip.std.SeparateFields(1).std.SelectEvery( cycle=2, offsets=1)

clipi = core.std.Interleave([clip0, clip1]).std.SelectEvery( cycle=2, offsets=0)
clip = core.std.DoubleWeave(clipi, tff=True)

clip = haf.QTGMC(clip, TR2=1, TFF=False).std.SelectEvery( cycle=2, offsets=0)


I'm not familiar with VapourSynth.
Translated in Avisynth language would be:

#Indexing
LWLibavVideoSource("Sample.mkv")

#Separate fields
clip0=SeparateFields(0).SelectEvery( cycle=2, offsets=0)
clip1=SeparateFields(1).SelectEvery( cycle=2, offsets=1)

#Interleaving both fields
Interleave(clip0, clip1)
SelectEvery(cycle=2, offsets=0)

#Weave
DoubleWeave(tff=True)

#Deinterlace
QTGMC(TR2=1, TFF=False)

SelectEvery(cycle=2, offsets=0)

Right? I'm gonna try it asap.

[ChaosKing]

I think this should be the correct translation:

ffms2("D:\Download\Sample_short.mkv")

clip0=AssumeBFF().SeparateFields().SelectEvery(2, 0)
clip1=AssumeTFF().SeparateFields().SelectEvery(2, 1)

#Interleaving both fields
Interleave(clip0, clip1)
SelectEvery(2, 0)

#Weave
AssumeTFF()
DoubleWeave()

#Deinterlace
QTGMC() ## use a preset here!
SelectEven()


I also noticed that the chroma has blends in SeparateFields(). Not sure what a good fix would look like. And I think every 6th frame is a double frame!? But it doesnt seems to be a fixed pattern.


EDIT:
I tested just qtgmc alone again and now it doesn't look "jumpy" like the first time. I don't understand why... the code above is unnecessary and not entirely correct anyway. I will post new code soon.

[wonkey_monkey]

What a mess. I can't see any pattern to the repeats.

Try this, though:


tdecimate(mode = 1, cycler = 7, cycle = 12)


You could also try mode = 0, or cycler = 35 and cycle = 60, although that will be a lot slower.

Unfortunately tdecimate only works in 8 bit, and there will always be remaining dupes which result in a stutter, but I think it's as good as you're going to get unless I can work out the exact dupe pattern (it's so random I think it was done analytically).

Also the chroma is completely blended so there's no hope of fixing that.

Also tell them they suck at compositing!

[wonkey_monkey]

ChaosKing:

clip0=AssumeBFF().SeparateFields().SelectEvery(2, 0)
clip1=AssumeTFF().SeparateFields().SelectEvery(2, 1)


I'm not sure what your intent was, but clip0 and clip1 are identical.

[StainlessS]

there will always be remaining dupes which result in a stutter

Duplicty2/DropDeadGorgeous could get rid of that [EDIT: at least with 8 bit].

from:- https://forum.doom9.org/showthread.php?p=1873808#post1873808

Perhaps this explains better

abcXdefgDDhi Src, Two dupe frames preceded (within a set range) by a jerk frame. Jerk frame X, has greater motion to its predecessor than do d, e, f, or g.
abc12Xdefghi DropDeadGorgeous dupe removal and Jerk interpolation

Lower case non dupe non Jerk
X = Frame of greatest motion from predecessor (Jerk)
D = Dupe Frames
Digit = Interpolated frames between X and its predecessor c (Same Count as D dupes)


EDIT:
Duplicity/DropDeadGorgeous only 8 bit, but could convert to 8 bit for duplicity DBase creation,
and hack DropDeadGorgeous script for 2nd pass with 10 bit and DropDeadGorgeous().

Function ZZ_AverageLuma(clip c, int "n") { ### Replace RT_AverageLuma() calls to ZZ_AverageLuma() EDIT: Simple Search/Replace in text editor.
c # EDIT: Oops, added
current_frame = Default(n,current_frame)
return AverageLuma
}

Untested, should work I think

[wonkey_monkey]

No interpolation is needed, just precise removal of exact (barring compression differences) duplicates. But there might be some scenes with static graphics, so there needs to be a limit on how many frames are removed per second, at least on average.

It seems that, at most, there are three duplicate frames in a row (original correct frame, plus three duplicates).

[StainlessS]

Perhaps, but given the alternative of [EDIT: left over dupes/] jerks, might be an option.
Duplicity has MaxDupeLen (where if greater then is presumed static scene).
EDIT: Duplicity does not have a 1 in n decimation mode [well, when creating Frame files, it does not actually decimate at all].

[poisondeathray]

You mentioned h.264 60fps progressive, but what was it? 4:2:2 8bit h264 ? 4:2:0 ?

The short sample is 8bit 4:2:2 ffv1, but long sample is 10bit 4:2:0

Why didn't you just upload the cut original ?
- smaller filesize compared to ffv1
- less chance of introducing more errors
a) maybe other issues like decoding, timecodes causing the problem, such as repeat fields. Maybe other decoding methods are better for the actual source ? Certain versions of ffms2 / lsmash/ ffmpeg are known to give problems , e.g. interlaced AVC transport streams
b)possibly introducing other interlaced/progressive up/downsampling errors exacerbating the chroma ghosting .

It probably doesn't matter here, but you should always upload the original , untouched as much as possible

[FranceBB]

You mentioned h.264 60fps progressive, but what was it? 4:2:2 8bit h264 ? 4:2:0 ?

The short sample is 8bit 4:2:2 ffv1, but long sample is 10bit 4:2:0


The original is H.264 4:2:0 yv12 8bit planar "fake" 60fps, encoded as progressive but with an interlacing mess inside. I encoded a few lossless samples to give you an idea of what it was and a lossy one 'cause I wouldn't have been able to upload the original one. Still, in this case, ffmpeg didn't introduce any new weird decoding issue and I left the fields exactly as they were.

For everyone: I've tried the code posted above and I encoded a few samples. I also tried various modes and parameters of tdecimate, chained tdecimate, single tdecimate, SRestore for dups only, ApparentFPS, Duplicity2 and I spent hours and hours on this flipping mess. I even tried to manually detect and trim out dups and then blend the missing frames to get a truly interlaced 25i (50fps) and although it took a lot of time, the result was promising. In order to try to get transition between missing frames smoother, I even tried linear interpolation but it was so inconstant that it was creating artifacts and it wasn't worth it. I called the producer and I showed them each and every result of the various encoding test, but he said that the "automatic solutions" were not good and although the manually decimated and blended one was promising, he recognised that it was too much effort to do it for a whole game of over 1h (I did it for several minutes but I was far from the end and my eyes were crossing already...).
Anyway, the producer informed the playout that we're gonna miss the scheduled deadline and that he's gonna talk to the service. (I don't know about what contract we have with them; it's not my field, I'm an encoder).

Alright, I would like to thank you all for the support as you tried to help me (well, "help us") and my boss is also thanking you as he knows about doom9 and he's totally in favour of open source and reciprocal help.

Thank you all anyway.

Alright, fellas, time to go home now and have dinner.
What a day at work today has been...

EDIT: They actually managed to send the real untouched masterfile in AppleProRes during the night and we encoded it straight away so we will be able to air in today, which also made me solve the "mystery" of what happened to the provided file: the original masterfile was an 25i truly interlaced (50fps) 4:2:2 yv16 10bit planar, however it has been edited by the guy who sent the file in a timeline that was set as 60fps progressive and that screwed up the fields. Then, he also brought it from 10bit to 8bit by truncation and it converted it from 4:2:2 planar to 4:2:0 planar while interlaced but treating it as progressive, which made the chroma expand from the fields to the frames... What a mess. Luckily the original untouched master has been provided just in time.