24fps->60fps for Anime footage

[xyzdragon]

Hello,

I'm new here, but know of and was reading on this board for a long time.

I want to interpolate my Bluray-Animes, because I observe a nasty shaky movement especially in pure pans, since I got a newer and larger monitor. (I also noticed banding since then on many sources, but this is another matter)

Now I tried many Avisynth-Filters like InterFrame, SVPFlow and DePan, but they all are not satisfactory, because I definitely don't want to introduce artefacts.
DePan is momentarily my favorite, because it introduces the least artifacts on the interpolated frames.

The problem for DePan are scenes where a background pans with one velocity and a foreground (figures, faces, ...) pans/moves with another velocity. This results in jittery movement for one or another. DePan doesn't seem to be able to differentiate well enough between these mixed pans and normal ones. Heck sometimes pans with a perfectly still image has a trust value much lower than these mixed movements. (In the depan documentation it's also not mentionend, but it seems the dxmax, dymax affect the trust-value, even though 'stab' should be defaulted to 0)

Also a small problem seems, that for perfectly still scenes depan can actually introduce jitter! I would need a dxmin, dymin feature for that, because in the log the dx and dy always move around +0.0,-0.0,+0.1,-0.1 for perfectly still scenes.

Also there are two methods using depan. Using DePan or using ChangeFPS with following DePanStabilize. I noticed, that the latter method seems to perform worse when detecting motion, possibly because of the duplicated frames in-between.

SVPFlow is another matter. The parameters are not very well documented, so its really hard to tweak it.

I tried getting it to recognize only global motion, like hinted to in the svpproject wiki for the GUI-Version: http://www.svp-team.com/wiki/Watching_anime
My best settings were:

super   = SVSuper( "{ scale:{ up:0, down:4 }, gpu:1 }" )      
vectors = SVAnalyse( super, "{ block:{w:32, overlap:0 }, main:{ search:{ coarse:{type:2, distance:-6, satd:false}, penalty:{ lsad:0 }, distance:0, bad:{sad:0} } } }" )
SVSmoothFps( super, vectors, "{ rate: { num:60000 , den:1001, abs:true }, algo:2 , mask:{ cover:0 }, scene:{ blend:false, mode:0 } }", url="www.svp-team.com", mt=4 )

The problem with these settings, which I partially extracted from the InterFrame-Plugin, is, that it still tries to find individual movement on some scale in a pan or in a still background scene. This results in a really noticeable kind of 'aura' around moving objects, because part of the background also gets moved and distorted. Compare for example the two frames in the attachment and you will see it.


Does anyone know of some method to only interpolate perfect pans without moving objects or can tweak DePan a bit better than me or ... ?

[creaothceann]

Originally Posted by xyzdragon

I want to interpolate my Bluray-Animes, because I observe a nasty shaky movement especially in pure pans, since I got a newer and larger monitor.

TV or computer monitor? If it's the latter and if you can set it to 50Hz (you may need to create a custom resolution), try ReClock.

[xyzdragon]

Originally Posted by creaothceann

TV or computer monitor? If it's the latter and if you can set it to 50Hz (you may need to create a custom resolution), try ReClock.

Computer monitor "LG 24EA53VQ 24 Zoll IPS". I don't think this is the problem. (I tried setting it to 50Hz just now) I think I observe it only on this monitor, because it's larger and therefore the jump per frame is maybe 1cm instead of 0.5cm in real space.

The 60fps footage I got really looks awesome and is exactly what I want, if the pan recognition wouldn't be so random.

Is there maybe another way to prepare a clip which can get better results in DePanEstimate? Denoising for example helps extremely. Maybe it's possible that higher contrast or heavily overcleaned/denoised footage would lead to better results. But I don't know how DePan works, to know where to start with this.

[creaothceann]

Originally Posted by xyzdragon

I think I observe it only on this monitor, because it's larger and therefore the jump per frame is maybe 1cm instead of 0.5cm in real space.

Then wouldn't it be better to interpolate to 50 (or 50/1001) Hz instead of 60? I don't know about the interpolation glitches though.

[xyzdragon]

Originally Posted by creaothceann

Then wouldn't it be better to interpolate to 50 (or 50/1001) Hz instead of 60? I don't know about the interpolation glitches though.

50 or 60 (59.94), I don't really see the difference. 60Hz is standard maximum for many monitors, so I thought that would be best. That can be done with factor 5/2. But maybe your point is 24->48 would make less interpolation glitches, because there wouldn't be so much to interpolate... This is worth a try. Edit: Won't alleviate the problem.

[creaothceann]

Originally Posted by xyzdragon

50 or 60, I don't really see the difference.

Well, there should be completely smooth movement (though with large spatial increments) if everything is set up right: video (24/1.001Hz) → ReClock (25Hz) → monitor (50Hz)

[foxyshadis]

You should check out MadVR interpolation if you're only going to watch on a PC. MadVR has an option to blend to your monitor's framerate instead of juddering. (The artifacts of blending are usually less noticeable than the artifacts of flow or depan in motion. Flow tends to compress better, on the other hand.) That beats warping your backup to conform. Unfortunately, there's really no solution to flowing differing background/foreground framerates. clouded was writing the celforeground/celbackground plugins partly for that purpose, but the project got too difficult to finish.

[fvisagie]

Originally Posted by xyzdragon

I want to interpolate my Bluray-Animes, because I observe a asty shaky movement especially in pure pans

Just a question - are you throwing the right tool at the problem? More specifically, are you sure that what you observe isn't the result of incorrect field order, (poor) telecine handling, etc. earlier in the life of those videos? Just a thought .

[xyzdragon]

Originally Posted by fvisagie

Just a question - are you throwing the right tool at the problem? More specifically, are you sure that what you observe isn't the result of incorrect field order, (poor) telecine handling, etc. earlier in the life of those videos? Just a thought .

I'm pretty sure. The Blurays of Ichiban no Ushiro Daimaou are one of the better ones, unlike interlaced Ghost Hound or seeminlgy upscaled and noisy Jormungand (the cgi isn't even antialiased).
My avisynth script therefore:

function ´(clip) {
    data = DePanEstimate( clip, trust=5, dxmax=20, dymax=20, stab=-0.5, info=true, extlog="motiondata.log" )
    t0   = data
    t1   = DePan( clip, data, offset=0.5, mirror=15 ).DeleteFrame(0)
    Interleave( t0,t1 )
}

DirectShowSource("D:\Blurays\ichiban_bd1_006.mkv", audio=false, convertfps=true, fps=23.976)
Trim(100,-200)
ConvertToYV12()
Crop(2,2, -2,-2)    
fps1to2()
Crop(4,4, -4,-4)

I also checked every single frame by exporting to png using ffmpeg (mpc doesn't seem to work perfectly. It seems to jump 2 frames at once when using the arrow-right-key, also it's of course slow)

[creaothceann]

Some nitpicks...

Originally Posted by xyzdragon

DirectShowSource

DSS2 is better. DirectShowSource often had problems with frame-accurate seeking for me.

Originally Posted by xyzdragon

fps=23.976

Perfectly correct number would be 24.0/1.001.

Originally Posted by xyzdragon

ConvertToYV12()

Isn't it already?

Originally Posted by xyzdragon

mpc doesn't seem to work perfectly. It seems to jump 2 frames at once when using the arrow-right-key, also it's of course slow)

Tried AvsPmod / VirtualDub (while using DSS2)?

[avih]

Originally Posted by xyzdragon

Hello,
...
Does anyone know of some method to only interpolate perfect pans without moving objects or can tweak DePan a bit better than me or ... ?

I didn't approve those attachments because I don't see how these images contribute to understanding the issue or help in solving it.

Generally, try to be reasonable with the size and relevance of attachments. Thanks.