Doom9's Forum - View Single Post - Issue with "morph" + questions about frame interpolation and other subjects

abolibibelot · 4th December 2018, 16:07

Hi,

I'm trying to improve a movie I made about my deceased grandmother; I want this to be finished by the end of December. I shot the AVCHD 1280x720 source footage with two Panasonic compact cameras : a ZS3 shooting in 25 FPS in dec. 2013, with a strong backlight issue (mostly shot in front of a large window) ; a ZS7 shooting in 29.97 FPS in dec. 2015, that camera had a malfunctioning optical stabilization system, causing spontaneous intermittent vertical jerkiness (with the picture shaking up and down by about 5 pixels, 5 times per second, at random intervals...).

I tried to improve the picture as much as possible, using :
– the stabilization feature from Magix Video Deluxe (NLE software used to edit this movie), followed by a combination of FrameSurgeon and Morph (Avisynth functions) to interpolate the remaining blurry frames caused by the jerkiness on the 2015 videos (see this thread – Sawbones was a huge help, considering that I had to manually check about 35min. of footage, amounting to about 50.000 frames) ;
– a combination of AutoAdjust and HDRAGC to get something watchable out of the dark shadows on the 2013 videos (see this thread).

The first thing I want to do is interpolate the bad frames on the source footage rather than the edited / rendered movie, since some of those bad / blurry frames are located at crossfade points and I couldn't correct them satisfyingly when I finalized the first version.
But when examining the scripts I made a few months ago, I found out that the Morph() function no longer works as expected : now if I use the command "Morph(137,139)" for instance, I get a blend of frame 137 and the last frame of the input movie, instead of a blend of 137 and 139. What could be the explanation ? I tried copying the Morph function inside a script, changing its name to Morph2 and calling Morph2(x,y), same result.
I haven't voluntarily changed anything in my Avisynth configuration since then, but I noticed that the modified date of most subdirectories in my Avisynth+ directory, 2018/10/26 22:52, is very close to the date of StaxRip subdirectories, so apparently something was changed in Avisynth when I first ran StaxRip. I couldn't say which changes were made though, but I could check the differences with a backup made before that date, if needed.

Another issue is that, previously, I ran the frame interpolation script on the already stabilized footage, at the final encoding stage. It would have been even more difficult and painstaking to spot damaged frames among the original jerky mess, and probably much less efficient too (potentially interpolating from misaligned frames). So now I have to stabilize the footage before interpolating. Are there Avisynth function(s)/plugin(s) that would be at least as good as the NLE feature, and produce a reliable result ? I have briefly tried Deshaker in VirtualDub, it seemed quite efficient, but quite complicated, I had already many complications to deal with and, since the Magix stabilizer was performing mostly fine (better in my experience than the third-party proDAD Mercalli plugin that it now also includes, and which is supposed to be a professional level tool), I figured that I wouldn't need anything else... The alternative would be to treat the whole source files with MVD's stabilizer, export them as lossless intermediates, then process them with Avisynth to interpolate the bad frames, then re-import them into MVD, but I'd prefer not to...

Then, would it be possible, and wise, to load the scripts (probably 5-6) to the NLE using AVFS (Avisynth Virtual File System), with my machine based on an Intel i7 6700K CPU with 16GB of RAM, or would it be better to first export the processed footage as intermediate lossless files ? The first option would be more convenient of course, but may be slower, and potentially less reliable (if there's an out-of-memory issue for instance the results can start to get wonky, and some Avisynth functions apparently don't produce the same output when processing footage in a linear way or when previewing a particular frame at a random spot). With the second option, I know what the processed footage will look like, there are less moving parts, but if I need to make a slight adjustement I have to start all over again and re-export the whole bunch of big files, instead of just reloading the virtual AVI files.

Apparently this depends partly on the source loading plugin used : for instance I've had very unreliable results in the past when using DirectShowSource. In this case the source files are .m2ts : which plugin is generally recommanded with that format, and why ? ffms2 / FFVideoSource, or LSMASH / LWLibavVideoSource ? or something else ?

On frame interpolation, again : with FrameSurgeon, I get ugly artifacts in case of fast motion (that's why I used Morph for some frames, as it usually performs better in those cases, even though it's much less convenient and gets unstable if called more than a few dozen times in the same script, whereas FrameSurgeon doesn't choke with more than 4800 frames to process). I have read (see here – it's in french but the screenshots alone illustrate the issue very well – and here) that this kind of artifacts were produced by MVTools2 when processing "HD" footage, and not at lower resolutions. Is it a known issue, and is it still the case ? (Those two threads are 5-6 years old...)

Then, with regards to the backlit footage, is there anything better than HDRAGC that I could try this time ? Apparently this plugin is quite old; yet this kind of function should be high in demand as this is a very common issue, but I found nothing equivalent for Avisynth. I have also tried the Gamma HDR function in Magix Video Deluxe, I have tried various processing methods in DaVinci Resolve (possibly not to the best of their abilities as I was just discovering this seemingly highly capable software), both of which produced worse results than HDRAGC.

And what is the wisest way of combining 29.97 FPS footage with 25 FPS footage in the same movie ? (In two distinct parts, not mixed together.) At first I tried exporting in 25 FPS, but it caused the Magix stabilizer to malfunction ; then (using the 2010 version of that software) I exported in 29.97 FPS and I had a desynchronization issue, solved by updating to the 2016 version. On VideoHelp I've been told that exporting in 59.94 FPS would result in a smoother playback, but Magix Video Deluxe doesn't allow that framerate, 29.97 or 30 are the maximum values available. Would it be worth anything to convert the framerate to 59.94 with an Avisynth function at the final x264 encoding stage ?

Thanks !

4th December 2018, 16:07	#1 \| Link
abolibibelot Registered User Join Date: Jun 2015 Location: France Posts: 46	Issue with "morph" + questions about frame interpolation and other subjects Hi, I'm trying to improve a movie I made about my deceased grandmother; I want this to be finished by the end of December. I shot the AVCHD 1280x720 source footage with two Panasonic compact cameras : a ZS3 shooting in 25 FPS in dec. 2013, with a strong backlight issue (mostly shot in front of a large window) ; a ZS7 shooting in 29.97 FPS in dec. 2015, that camera had a malfunctioning optical stabilization system, causing spontaneous intermittent vertical jerkiness (with the picture shaking up and down by about 5 pixels, 5 times per second, at random intervals...). I tried to improve the picture as much as possible, using : – the stabilization feature from Magix Video Deluxe (NLE software used to edit this movie), followed by a combination of FrameSurgeon and Morph (Avisynth functions) to interpolate the remaining blurry frames caused by the jerkiness on the 2015 videos (see this thread – Sawbones was a huge help, considering that I had to manually check about 35min. of footage, amounting to about 50.000 frames) ; – a combination of AutoAdjust and HDRAGC to get something watchable out of the dark shadows on the 2013 videos (see this thread). The first thing I want to do is interpolate the bad frames on the source footage rather than the edited / rendered movie, since some of those bad / blurry frames are located at crossfade points and I couldn't correct them satisfyingly when I finalized the first version. But when examining the scripts I made a few months ago, I found out that the Morph() function no longer works as expected : now if I use the command "Morph(137,139)" for instance, I get a blend of frame 137 and the last frame of the input movie, instead of a blend of 137 and 139. What could be the explanation ? I tried copying the Morph function inside a script, changing its name to Morph2 and calling Morph2(x,y), same result. I haven't voluntarily changed anything in my Avisynth configuration since then, but I noticed that the modified date of most subdirectories in my Avisynth+ directory, 2018/10/26 22:52, is very close to the date of StaxRip subdirectories, so apparently something was changed in Avisynth when I first ran StaxRip. I couldn't say which changes were made though, but I could check the differences with a backup made before that date, if needed. Another issue is that, previously, I ran the frame interpolation script on the already stabilized footage, at the final encoding stage. It would have been even more difficult and painstaking to spot damaged frames among the original jerky mess, and probably much less efficient too (potentially interpolating from misaligned frames). So now I have to stabilize the footage before interpolating. Are there Avisynth function(s)/plugin(s) that would be at least as good as the NLE feature, and produce a reliable result ? I have briefly tried Deshaker in VirtualDub, it seemed quite efficient, but quite complicated, I had already many complications to deal with and, since the Magix stabilizer was performing mostly fine (better in my experience than the third-party proDAD Mercalli plugin that it now also includes, and which is supposed to be a professional level tool), I figured that I wouldn't need anything else... The alternative would be to treat the whole source files with MVD's stabilizer, export them as lossless intermediates, then process them with Avisynth to interpolate the bad frames, then re-import them into MVD, but I'd prefer not to... Then, would it be possible, and wise, to load the scripts (probably 5-6) to the NLE using AVFS (Avisynth Virtual File System), with my machine based on an Intel i7 6700K CPU with 16GB of RAM, or would it be better to first export the processed footage as intermediate lossless files ? The first option would be more convenient of course, but may be slower, and potentially less reliable (if there's an out-of-memory issue for instance the results can start to get wonky, and some Avisynth functions apparently don't produce the same output when processing footage in a linear way or when previewing a particular frame at a random spot). With the second option, I know what the processed footage will look like, there are less moving parts, but if I need to make a slight adjustement I have to start all over again and re-export the whole bunch of big files, instead of just reloading the virtual AVI files. Apparently this depends partly on the source loading plugin used : for instance I've had very unreliable results in the past when using DirectShowSource. In this case the source files are .m2ts : which plugin is generally recommanded with that format, and why ? ffms2 / FFVideoSource, or LSMASH / LWLibavVideoSource ? or something else ? On frame interpolation, again : with FrameSurgeon, I get ugly artifacts in case of fast motion (that's why I used Morph for some frames, as it usually performs better in those cases, even though it's much less convenient and gets unstable if called more than a few dozen times in the same script, whereas FrameSurgeon doesn't choke with more than 4800 frames to process). I have read (see here – it's in french but the screenshots alone illustrate the issue very well – and here) that this kind of artifacts were produced by MVTools2 when processing "HD" footage, and not at lower resolutions. Is it a known issue, and is it still the case ? (Those two threads are 5-6 years old...) Then, with regards to the backlit footage, is there anything better than HDRAGC that I could try this time ? Apparently this plugin is quite old; yet this kind of function should be high in demand as this is a very common issue, but I found nothing equivalent for Avisynth. I have also tried the Gamma HDR function in Magix Video Deluxe, I have tried various processing methods in DaVinci Resolve (possibly not to the best of their abilities as I was just discovering this seemingly highly capable software), both of which produced worse results than HDRAGC. And what is the wisest way of combining 29.97 FPS footage with 25 FPS footage in the same movie ? (In two distinct parts, not mixed together.) At first I tried exporting in 25 FPS, but it caused the Magix stabilizer to malfunction ; then (using the 2010 version of that software) I exported in 29.97 FPS and I had a desynchronization issue, solved by updating to the 2016 version. On VideoHelp I've been told that exporting in 59.94 FPS would result in a smoother playback, but Magix Video Deluxe doesn't allow that framerate, 29.97 or 30 are the maximum values available. Would it be worth anything to convert the framerate to 59.94 with an Avisynth function at the final x264 encoding stage ? Thanks ! Last edited by abolibibelot; 5th December 2018 at 17:20.