Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 10th April 2010, 04:20   #1  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,103
2 issues: Dering & Deshake

I came to this awful source inside a DVD, so some questions:

source 38Mb Sendspace

This source has a very strong ringing, I cant even remove it with Dehalo_alpha, some suggestions?

The other thing is deshake, the image jitters constantly so I tried Depan as described in Fred's film restoration script, playing mostly with cutoff and maxstabH variable values, I also increased range for accuracy and tweaked trust to look for improvements, but I really didnt see any. Does anyone know a better way or a better script to deshake? Ive heard of "deshaker by gunnar thalin" and "stab", but I dont know if they can perform better than Fizick's

Also, is this 8:9 or 4320:4739 as stated here?

Quote:
setmemorymax(768)
SetMTmode(5)
DGDecode_MPEG2Source("dvd.d2v")
SetMTmode(2)

AssumeFPS(30000,1001)
TFM(order=1,slow=2)
TDecimate()

source1=last
stab_reference= source1.crop(12,2,-8,0).greyscale().invert()
maxstabH=10
maxstabV=maxstabH
mdata=DePanEstimate(stab_reference,trust=1.0,dxmax=maxstabH,dymax=maxstabV,range=24,pixaspect=0.888)
DePanStabilize(source1,data=mdata,cutoff=0.2,dxmax=maxstabH,dymax=maxstabV,method=1,mirror=15,pixaspect=0.888)
crop(12,2,-8,0)
Dogway is offline   Reply With Quote
Old 11th April 2010, 01:01   #2  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 621
For the halos, try the following code after deinterlacing. Not perfect, but it gets rid of most of them. Of course, the picture will become more blured.

Code:
deconv_h = "-1 2 1 -8 20 -31 38 -3 -32 100 -32 -3 38 -31 20 -8 1 2 -1"
mt_convolution (deconv_h, "1", Y=3, U=2, V=2)

Last edited by cretindesalpes; 11th April 2010 at 01:07.
cretindesalpes is offline   Reply With Quote
Old 11th April 2010, 03:35   #3  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,103
Whoa! very good and natural results. It blurs a little bit, but I can compensate that with some sharpener, its worth it, I will have to indagate into that function.

Does it have any temporal component? if so it would need to be after the deshake right?

Does someone recommend me the Vdub gunnar thalin deshaker over Fizick's? Does it worth messing? or am I not going to get anything better than the Depan workaround?
Dogway is offline   Reply With Quote
Old 12th April 2010, 02:59   #4  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,103
Ok, since nobody knew to answer my questions I got to mess with gunnar thalin plugin, although in this way I will have to encode lossless before applying the real avisynth script for color correction, etc. thus dealing with Gygabites of video chunks. It looks to be better but it still jitters clearly, and I dont know if it has to be with my PAR setting in the script, which leads me to Q3, how do I know the PAR of my source?

you can see flaws in 0:39 and 0:44, check my settings.
The input is an avs with the deinterlacing and cropping


lossless result 190Mb: http://www.sendspace.com/file/gqyp3b

Deshaker settings:

Last edited by Dogway; 12th April 2010 at 03:01.
Dogway is offline   Reply With Quote
Old 12th April 2010, 16:18   #5  |  Link
videoFred
Registered User
 
videoFred's Avatar
 
Join Date: Dec 2004
Location: Gent, Flanders, Belgium, Europe, Earth, Milky Way,Universe
Posts: 672
Quote:
Originally Posted by Dogway View Post
It looks to be better but it still jitters clearly,
I do not see no jitter

I see moving parts, caused by noise reduction, or other filters.

Can you not upload an original source clip in Avi format?

Fred.
__________________
About 8mm film:
http://www.super-8.be
Film Transfer Tutorial and example clips:
https://www.youtube.com/watch?v=W4QBsWXKuV8
More Example clips:
http://www.vimeo.com/user678523/videos/sort:newest
videoFred is offline   Reply With Quote
Old 13th April 2010, 00:46   #6  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,103
thanks for the feedback

Source is in the first post, but is a .m2t.
Last post is a deinterlaced and deshaked lossles .avi
Do you want only the deinterlaced result in avi? divx do you mean? or lossless?

EDIT: anyway here you have a HQ xvid (avi) deinterlaced source-> 20Mb http://www.sendspace.com/file/mx2czq

Last edited by Dogway; 13th April 2010 at 11:32.
Dogway is offline   Reply With Quote
Old 22nd April 2010, 07:48   #7  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,103
Hello again, this is my first ever bump, but it is for a good cause: someone who notices me claiming help to restore this piece of art.

Here I go, I fixed the AR in the gunnar plugin, and as suggested in another thread I did the first pass without cropping, the second pass with cropping. To fix the heavy shaking introduced by Deshaker at 0:39 and 0.44, I zeroed the rotation smoothness, and yes, it doesnt rotate anymore, now it shakes horizontally.
Is there anything I can try? the image still shakes, although less than before, but it does (is this the limit for the plugin?). I dont know if it is shaking or denoisers waving artifacts (my script has no denoisers, only TFM(), TDecimate()) but whatever it is, I want to make it look good and stable, you know, near remastered DVD quality if possible.

New settings:


Result (in xvid 21Mb):
http://www.sendspace.com/file/q0q4mx


By the way, checking the log I noticed the difference on values at 0.39 and 0.44 in case it is relevant:
[HTML]@ 0.39s

1026 -0.00 -0.05 -0.004 1.00005
1027 0.17 -0.00 0.012 1.00030
1028 6.08 0.65 0.457 1.02236
1029 0.03 0.00 0.011 0.99993
1030 -0.10 -0.02 0.017 0.99950
1031 4.32 0.36 0.473 1.01440
1032 -0.13 -0.04 -0.003 1.00000


@ 0.44s

1053 0.04 0.11 0.009 1.00039
1054 0.00 0.00 0.000 1.00000 new_scene
1055 -0.03 0.00 -0.005 0.99995
1056 0.00 0.00 0.000 1.00000 new_scene
1057 0.01 -0.12 -0.012 1.00011
1058 -15.39 7.49 0.399 0.93799
1059 0.17 0.13 0.007 1.00013
1060 -0.01 -0.04 0.002 1.00005
1061 0.01 -0.01 0.001 1.00002
1062 0.00 0.00 0.000 1.00000 new_scene
1063 0.00 0.00 0.000 1.00000 new_scene
1064 0.16 -0.04 -0.009 1.00002
1065 0.00 0.00 0.000 1.00000 new_scene
1066 0.00 0.00 0.000 1.00000 new_scene
1067 -0.19 1.32 0.004 0.99772[/HTML]

Also how can I remove the first 30 frames of warning?

Last edited by Dogway; 22nd April 2010 at 08:01.
Dogway is offline   Reply With Quote
Old 22nd April 2010, 08:05   #8  |  Link
videoFred
Registered User
 
videoFred's Avatar
 
Join Date: Dec 2004
Location: Gent, Flanders, Belgium, Europe, Earth, Milky Way,Universe
Posts: 672
Quote:
Originally Posted by Dogway View Post
I dont know if it is shaking or denoisers waving artifacts (my script has no denoisers, only TFM(), TDecimate())
It looks like interpolation artefacts to me, caused by deinterlacing. But I'm not a deinterlacing expert.

PS: now that was a real fast download: over 500kbps

Fred.
__________________
About 8mm film:
http://www.super-8.be
Film Transfer Tutorial and example clips:
https://www.youtube.com/watch?v=W4QBsWXKuV8
More Example clips:
http://www.vimeo.com/user678523/videos/sort:newest
videoFred is offline   Reply With Quote
Old 22nd April 2010, 08:26   #9  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,103
I see your point, but no, the interlaced footage also has this wavy artifact, and now analyzing the second 10, I can see fixed areas (the upper side) and shaking areas (downer side), more noticeable on the chimney down-left over the roof, look how does it move related to the wall next to it. I dont know what caused it, but it was already on the source, so although Deshaker does some work, this craves for a more imaginative solution, something in the sense of masktools and Didee's comeout scripts. Pity he just runs away from animations :S

Last edited by Dogway; 22nd April 2010 at 08:30.
Dogway is offline   Reply With Quote
Old 22nd April 2010, 08:39   #10  |  Link
videoFred
Registered User
 
videoFred's Avatar
 
Join Date: Dec 2004
Location: Gent, Flanders, Belgium, Europe, Earth, Milky Way,Universe
Posts: 672
Quote:
Originally Posted by Dogway View Post
more noticeable on the chimney down-left over the roof, look how does it move related to the wall next to it.
This is going to be a difficult one! I do not know if there's a solution for this. Again, it's a typical artefact caused by denoising and/or interpolation. They must have applied this on the source.

Fred.
__________________
About 8mm film:
http://www.super-8.be
Film Transfer Tutorial and example clips:
https://www.youtube.com/watch?v=W4QBsWXKuV8
More Example clips:
http://www.vimeo.com/user678523/videos/sort:newest
videoFred is offline   Reply With Quote
Old 22nd April 2010, 08:43   #11  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,103
Sure, its not on my league, thats why I ask in The Best Restoration Forum in the World :P

Thanks Fred.
Dogway is offline   Reply With Quote
Old 22nd April 2010, 08:47   #12  |  Link
videoFred
Registered User
 
videoFred's Avatar
 
Join Date: Dec 2004
Location: Gent, Flanders, Belgium, Europe, Earth, Milky Way,Universe
Posts: 672
Quote:
Originally Posted by Dogway View Post
Thanks Fred.
You are most welcome!

I hope someone else can help you any further with this.

Fred.
__________________
About 8mm film:
http://www.super-8.be
Film Transfer Tutorial and example clips:
https://www.youtube.com/watch?v=W4QBsWXKuV8
More Example clips:
http://www.vimeo.com/user678523/videos/sort:newest
videoFred is offline   Reply With Quote
Old 3rd May 2010, 20:50   #13  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,103
Quote:
Originally Posted by cretindesalpes View Post
Code:
deconv_h = "-1 2 1 -8 20 -31 38 -3 -32 100 -32 -3 38 -31 20 -8 1 2 -1"
mt_convolution (deconv_h, "1", Y=3, U=2, V=2)
Im going to wrap so Im trying to fine tune everything. How did you get those values? I mean, I know the theory behind it, but did you use any software? You are sharpening also (which causes some ugly noise), I tried 1 1 4 1 1 and got also nice results but want to go further.
Dogway is offline   Reply With Quote
Old 5th May 2010, 20:34   #14  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 621
Quote:
Originally Posted by Dogway View Post
How did you get those values?
I used Matlab. If you haven't any access to Matlab, you can use GNU Octave. Its syntax is almost compatible with Matlab's. I wrote the following script, it's very basic but it gives decent results. To get rid of artifacts, you can reduce the kernel length with the maxlen variable.

Code:
%
% Dehalo calculator
%
% Fill "mes" and "fix" variables with measured luma values and desired values.
% Measured sample should be a thin line on a flat background.
% The line must be (almost) horizontal or vertical, and centered on a pixel.
% Write down the luma values into the "mes" array, starting from the center of
% the line, going to the flat background, moving orthogonally to the line.
% Fill the "fix" array with desired values for the dehaloed picture.
% Run the calculator and report the output data in the AviSynth script,
% adapting it for horizontal or vertical dehaloing :
%
% deconv_h = "1 -5 15 -19 100 -19 15 -5 1"
% mt_convolution (deconv_h, "1", Y=3, U=2, V=2)
%

clear;

% Input data
mes = [76 140 194 180 176];	% Line -> ringing -> (flat background)
fix = [76 140 181];		% Last value is the flat background value.
maxlen = 999;			% Maximum length for the convolution kernel. Should be odd.

% Preparation
bkg_val = fix (length (fix));
mes = mes - bkg_val;
fix = fix - bkg_val;

% Padding
lm = length (mes);
lf = length (fix);
c_len = lm*2-1 + lf*2-1 - 1;	% Padding required for the fft deconvolution.
mes = cat (2, cat (2, mes, zeros (1, c_len - lm*2 + 1)), mes (lm:-1:2));
fix = cat (2, cat (2, fix, zeros (1, c_len - lf*2 + 1)), fix (lf:-1:2));

% Resolution
af = fft (mes);
cf = fft (fix);
bf = cf ./ (af + 1e-30);	% Add a very small value to avoid divide-by-0 errors.
b = ifft (bf);
b = fftshift (real (b));

% Windowing and normalization
r_len = min (max (lm, lf) * 2 - 1, maxlen);
r_beg = (c_len - r_len) / 2 + 1;
r_end = r_beg + r_len - 1;
b = b (r_beg:r_end);
b = b .* transpose (hanning (r_len));
b = round (b * 100 / max (abs (b)));

% Display
b
plot (b);
grid on;
cretindesalpes is offline   Reply With Quote
Old 5th May 2010, 20:39   #15  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,103
It was indeed matlab, I was investigating that route a few days ago. Thank you a lot.
Dogway is offline   Reply With Quote
Old 11th November 2011, 02:00   #16  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,103
hello cretindesalpes, can you help me with this source? Ive tried a few times in the past with the code, but never got anything decent out of my hand. Now I have a source very very bad in terms of overshoot, I installed octave and loaded the code with the next input:

mes = [71 133 235 185 181]; % Line -> ringing -> (flat background)
fix = [71 181]; % Last value is the flat background value.
maxlen = 7;


In fact is: 71 75 70 53 18 133 235 185 181
but I ended up calibrating towards the bright, so when compensated the dark overshoot will become overlapped with the bright and hopefully corrected.

I got some values playing with the maxlength
-30 100 -30
14 -44 100 -44 14 (blurrier)
16 -29 51 100 51 -29 16 (much blurrier)
etc

but they don't work as they are, output is a mess, I need to make the values positive in order to see something expectable, although probably too blurry.
Another problem is that if I apply this on fields then QTGMC has a hard time stabilizing, so I need to recalculate the halo values after bob deinterlacing (my tests are on fields)
How would be your take on this? I also observed something else is wrong I'm not sure if aliasing...

36mb
http://www.mediafire.com/?q3cbzeloa6vg74a

Last edited by Dogway; 11th November 2011 at 02:25.
Dogway is offline   Reply With Quote
Old 11th November 2011, 03:02   #17  |  Link
markanini
Registered User
 
Join Date: Apr 2006
Posts: 297
Maye it swine before pearls but have you tried hqdering?
EDIT: Nevermind saw some examples and results were rather synthetic looking. I'm sure a convolution approach is better.

Last edited by markanini; 11th November 2011 at 03:09.
markanini is offline   Reply With Quote
Old 11th November 2011, 21:45   #18  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,103
Thanks for the input markanini. Yes dehalo_alpha and hqdering are more appropriate for animation, also they don't work with big ringing (halos?) like these. Anyways it served me to have a second look to HQDering and update it with a masktools2 version I found. I hope to solve this issue once for all.

if anyone's curious I ended up using stabilise for deshake, it requires lots of extra work, but it is the filter to be.
Dogway is offline   Reply With Quote
Old 14th November 2011, 09:32   #19  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 621
You could try this:
Code:
mt_convolution (horizontal="1", vertical="-1 -3 7 15 -4 -9 16 100 16 -9 -4 15 7 -3 -1", y=3, u=2, v=2)
Dehalo_alpha (rx=1.5, ry=3)
I can't do much better. There is still some haloing here and there, and lots of halo² and other halo^N marks. Dehalo_alpha does a good job as postprocessing, but high settings make the picture totally blurry and waxy. Maybe with some masking you could achieve a better result. The Didée's contrasharpened dehalo improves things nicely regarding the oil painting effect, but attenuates the halo removal.

Dehaloing also brings chroma bleeding in some areas, adding MergeChroma (aWarpSharp2 ()) could smooth them cheaply.

I deinterlaced first and haven't tried to dehalo on fields only.
__________________
dither 1.28.1 for AviSynth | avstp 1.0.4 for AviSynth development | fmtconv r22 for Vapoursynth | trimx264opt segmented encoding

Last edited by cretindesalpes; 14th November 2011 at 12:10. Reason: Added comment about Didée's contrasharpened dehalo
cretindesalpes is offline   Reply With Quote
Old 14th November 2011, 20:43   #20  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,103
Funny enough, now I tested with another profile and got similar results to yours, despite I made dozens of tests with other sources as well and never got convincing results, more sharpening instead of blurring or as you say many faint halo repetitions halo^n :S

For getting the samples I used in a stable scene:
temporalsoften(10,255,255)
blur(1.58,0) # since here halo is vertical

input:21,41,147,217,227,223,216,188,179,189

desired output:21,41,147,189

matrix: -3 10 4 -4 8 -6 100 -6 8 -4 4 10 -3

My thinking is its better to find a sample nearer to black as a base point, I think that's where I failed getting 71 as a base value.

I then used the contrasharpened dehalo you suggested (thanks a lot for this one)

and finally mcawarpsharp(4,4) for removing remaining halo and sharpening contours a bit.
In this case I will downscale everything a bit so look will be more compact.

http://screenshotcomparison.com/comparison/94353

From here I could do better with more fine tweaking, masking, etc sorry for disturbing again, now I guess I don't need to uninstall octave : D

Last edited by Dogway; 14th November 2011 at 20:46.
Dogway is offline   Reply With Quote
Reply

Tags
dejitter, dering, deshake, mt_convolution, stab

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 05:56.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.