Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
16th June 2016, 04:52 | #1 | Link |
Guest
Posts: n/a
|
Choose the best frames from identical sources
Assuming I have two sources that are identical in timing and frames but were compressed differently, is there a plugin or script that can make a single video from the clearest bits of both sources?
A good example of this is Youtube in regards to videos that have both VP9 and H264 streams - each codec has strengths and weaknesses with the large amount of compression they're using, so would it be possible to pick the best frames from each to make an output video that's slightly better in quality than either of the originals are individually? Also note that this is only for cases when a better source can't be found. There's no substitute for quality input. Last edited by thecoreyburton; 16th June 2016 at 05:03. |
16th June 2016, 06:50 | #2 | Link |
Registered User
Join Date: Feb 2002
Location: California
Posts: 2,691
|
I think someone else asked for this a few months back, and StainlessS came up with a script. I didn't have time to do an extensive search, but I think this thread was inspired by the original request:
SpatialAlign v1.0 : Spatial Alignment of clips It doesn't actually analyze the two frames to figure out which is "better" (that would be really tough to do), but it does let you get them lined up. |
16th June 2016, 07:25 | #4 | Link |
Registered User
Join Date: May 2006
Posts: 3,997
|
One would need to have a clean reference to which the 2 clips can be compared, in order to select the "more similar" picture (assuming that this is the better one), e.g. based on PSNR or similar.
Obviously we normally don't have access to the clean reference clip. Also, the switching between the selected pictures originating from different encoders might create unexpected annoying effects. Last edited by Sharc; 16th June 2016 at 07:29. |
16th June 2016, 09:02 | #5 | Link |
Registered User
Join Date: Mar 2014
Posts: 308
|
Ditto the two posts above.
I actually do this regularly—such is the life of working with broadcast television and disgusting interlacing-related artifacts—but there are a lot of caveats and you have to carefully consider the tradeoffs you're making. If you have multiple sources that have more or less the same amount of detail, but lots of blocking and mosquito noise, you can do this: Code:
a = ffvideosource("foo.mkv") b = dgsource("bar.m2ts") c = mpeg2source("baz.d2v") ab = repair(a, b, 1) bc = repair(b, c, 1) ca = repair(c, a, 1) raveragew(ab, 22.0/64.0, bc, 21.0/64.0, ca, 21.0/64.0, lsb_out=true) ditherpost(mode=6) If you have two sources where one of them has more fine detail than the other, and they're both reasonably free of blocking and mosquito noise, then you can try this: Code:
function combine(clip a, clip b) { function blurl(clip c, bool lsb_inout) { lsb_inout ? c : c.dither_convert_8_to_16() dither_box_filter16(radius=3) dither_box_filter16(radius=3) lsb_inout ? last : ditherpost(mode=6) } ah = mt_makediff(a,a.blurl(false),u=3,v=3) al = mt_makediff(a,ah,u=3,v=3) bh = mt_makediff(b,b.blurl(false),u=3,v=3) bl = mt_makediff(b,bh,u=3,v=3) mt_lutxy(ah, bh, "x 128 - 3 ^ y 128 - 3 ^ + x 128 - 2 ^ y 128 - 2 ^ 0.0001 + + / 128 +", u=3, v=3) mt_adddiff(raveragew(al,33.0/64.0,bl,31.0/64.0), last, u=3, v=3) } For stuff like merging the VP9 and H.264 streams on YouTube, however, I don't think there's a good way of doing that (yet). Last time I tried, FFMS2 barfed on the VP9 input files, duplicating and discarding random frames everywhere, but I suppose that even if there were a source filter that worked with VP9 files, the artifacts between the two streams are too similar for either of the above approaches to be of any use.
__________________
Say no to AviSynth 2.5.8 and DirectShowSource! Last edited by colours; 16th June 2016 at 09:18. |
16th June 2016, 13:45 | #6 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
A couple of threads on similar
http://forum.doom9.org/showthread.ph...light=findcuts http://forum.doom9.org/showthread.ph...light=findcuts
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? |
17th June 2016, 16:08 | #7 | Link |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
There's another case where this is useful, digital TV streams.
One way, a 24fps source on a 60fps channel. I found some patterns, where the picture was always clearest (in mpeg2) on the P frame after an I frame. This goes in groups of 15, which is not always in sync with the 24fps of the source, but you can improve a lot of the frames. In fact you get the same source frame repeated 2 or 3 times and can select the best one. So if the new source begins on an I frame, the frame after that is clearest. Another case is where you merge two recordings of the same material. The source is exactly the same, the encodings (even with the same parameters) are different. Same idea as above, take the P after the I frame from each source. Or you can even average them. |
18th June 2016, 16:42 | #8 | Link | |
Soul Architect
Join Date: Apr 2014
Posts: 2,559
|
Quote:
If someone comes up with a script that generates a better video by taking both YouTube videos, that would be interesting. However, there will be quality loss by the re-encoding that may nullify whatever small quality gain you may get. |
|
19th June 2016, 02:32 | #9 | Link | |
Guest
Posts: n/a
|
MysteryX, you're right. I use LWLibAVVideoSource and it doesn't have the frame duplication problem that's present in FFMS2. The videos are out of sync on first load up of the script (in Virtualdub for instance), but if you close it and reopen it they're perfectly in sync for that load and all subsequent loads. If you're getting ghosting with the script then this is the problem and a reload fixes it.
I'm testing a bit of a variation of the script Colours wrote that works nicely. I managed to find two videos produced at the same time by the same channel that exhibit compression artifacts in both formats (one has a portion that looks okay in VP9 but not H264 and the other has a portion that looks okay in H264 but not VP9). The issue here is that neither of the clips themselves are guaranteed to be consistently superior to the other. I've managed to remove a lot of the compression artifacts, but there is some detail loss - it's not too drastic and it certainly outweights the initial problem (at least for me), but I'm experimenting with sharpening the source slightly at various points and changing the ditherpost mode in order to preserve any detail I can. Edit: Here's the script and the video files as promised, along with some more information. I acquired these videos using 4K Video Downloader (I compared the videos to the ones aquired through youtube-dl which grabs the streams directly and confirmed that there's no conversion or transcoding of the videos after the downloading is done). The MKV contains the VP9 video with a transcoded vorbis audio track. The MP4 contains the H264 video with the original AAC audio track, making it the better choice for audio source if it's required. In S05E02 at frame 722 the VP9/MKV looks significantly better than the H264/MP4. In S05E04 at frame 1568 the H264/MP4 looks significantly better than the VP9/MKV. Detail loss is very noticable in this clip, but I think that's related to how bad one of the sources are. Colors are also more vibrant at frame 700 of S05E02 in the VP9 clip, the red bar getting a more blue-ish hue in the H264 clip and in the output clip. Here's the WIP script, this seems to produce the cleanest result with relatively good detail retention, but it's still not perfect. A lot of it was trial an error there's a good chance I'm not using one of the plugins in the most efficient way. Anything after DitherPost is just a bit of personal post processing on my behalf and isn't necessary to repair the videos. Quote:
S05E02, frame 722, MKV/VP9: S05E02, frame 722, script output: S05E04, frame 1568, MP4/H264: S05E04, frame 1568, MKV/VP9: S05E04, frame 1568, script output: The result is definitely much more pleasant to watch without the problem sections, but there's definitely quite a bit of detail lost. I'm not sure how to go about increasing the detail much more without reintroducing the artifacts that were removed, but I'm happy with the result so far - so thank you all! Last edited by thecoreyburton; 22nd June 2016 at 04:07. |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|