Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 26th February 2007, 16:13   #1  |  Link
dexy
Piracy of the Caribbean
 
dexy's Avatar
 
Join Date: Jul 2005
Location: Serbia
Posts: 81
How to clean-up those blocky flv videos

Hi guys, you have, probably, downloaded many videos from YouTube etc. Video quality is really poor, so it needs to be deblocked and derringed. For deblocking I am using mpeg2dec deblock filter, and it does the job on static scenes, but on fades blocks are still there. For derringing, btw, I have no idea which filter to use. I wanna know which filters do you prefer for those kinds of video, and which settings can do the job great.

__________________
when there 's nothin' left to lose, there 's nothin' left to fear
dexy is offline   Reply With Quote
Old 27th February 2007, 00:53   #2  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,314
How much time are you willing to put into it? You can use DegrainMedian, MipSmooth, TTempSmooth, FFT3DFilter, all the way up to MVDegrain, in a rough order of speed and quality. However, in such badly damaged video there usually isn't a lot of real detail left hidden under all the noise, so you're going to have to accept some amount if you don't want it to look cartoony.

When I'm home I can post one of the scripts I use for recovering youtube video, if you like.
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. ~ Ed Howdershelt
foxyshadis is offline   Reply With Quote
Old 27th February 2007, 00:58   #3  |  Link
Terranigma
*Space Reserved*
 
Terranigma's Avatar
 
Join Date: May 2006
Posts: 953
Quote:
Originally Posted by foxyshadis View Post
When I'm home I can post one of the scripts I use for recovering youtube video, if you like.
I'd like to see the script. I'm thinking it's a script using mvdegrain
Terranigma is offline   Reply With Quote
Old 27th February 2007, 03:40   #4  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
I normally use something based on a recommendation Didée gave me a long time ago for some insanely crappy 160x120 digicam videos:

Quote:
- deblock_QED (remove blocking, but keep detail) (-> it's on kassandro's forum, search)
- soft upsampling (perhaps gaussian) to ~150%
- extreme LimitedSharpen (updated version), with Smode=3, strength=1000, wide=true, soft=75~100
- 2x Upsampling by EEDI2, or custom script with good old SangNom
- lanczos to final destination size

- temporal filtering somewhere inbetween, perhaps MC'ed
- eventually use SeeSaw instead of LimitedSharpen, or even both

The most extreme variation that I had to use was with the two anime versions of the 1992 X-Men series's intro, the script for both was this one:

Code:
DirectShowSource("C:\X-Men Anime Intro Nº1.flv",fps=25,convertfps=true,audio=false)

ConvertToYV12()

FixChromaBleeding()

FFT3DFilter(sigma=6,plane=3,bw=32,bh=32,bt=3,ow=16,oh=16)

Tweak(sat=1.4)

DeHalo_Alpha()

DeGrainMedian()

Deblock(quant=35)

Crop(2,2,318,236,align=true)

GaussResize(448,336)

EEDI2().TurnRight().EEDI2().TurnLeft()

Spline36Resize(656,448)

aWarpSharp(depth=16,cm=1)

LimitedSharpenFaster(Strength=200)

vmToon(strength=150,thinning=0)

gradfun2db()

AddGrain(5,0,0)

AddBorders(24,16,24,16)

Levels(0,1,255,16,235)
(I changed the location of the gradfun2db line in order to reflect further advice I've gotten about this filter)

Here's a before/after comparison with the latter script:



Yeah, the source was resized with PointResize to emphasize the difference, but I doubt using better resizers for the source picture would have changed the purpose of the comparison too much.

I figured the framerate thanks to tritical and foxyshadis's advice: using FLV Extractor to extract the timecodes, then the AVI_TC package to convert the timecodes to v1 and see what was the most common fps in the v1 list and/or if all framerates were close to being all submultiples of a single greater one.
__________________
Read Decomb's readmes and tutorials, the IVTC tutorial and the capture guide in order to learn about combing and how to deal with it.

Last edited by Chainmax; 27th February 2007 at 03:42.
Chainmax is offline   Reply With Quote
Old 27th February 2007, 23:25   #5  |  Link
dexy
Piracy of the Caribbean
 
dexy's Avatar
 
Join Date: Jul 2005
Location: Serbia
Posts: 81
@foxyshadis

Time is not metter, the only important thing to me is quality. I am planning to filter some videos which are actually clean off noise, the only thing I have to do is deblock/derring. But, maybe I will met some noisy video some time, so every suggestion about cleaning the noise is welcome too. I will agree with Terranigma, post your script when you have time to do that. Thanks...

@chainmax

Thanks... Hm, as I can see there are some artifacts still present on places where the color has been destroyed, but in comparasion with source, video looks really great. Which script do you most frequently use?
__________________
when there 's nothin' left to lose, there 's nothin' left to fear
dexy is offline   Reply With Quote
Old 28th February 2007, 00:01   #6  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
The script I most commonly use would be something along the lines of:

Code:
DirectShowSource("X:\wherever\myvideo.flv",fps=X,convertfps=true,audio=false)
ConvertToYV12()
Deblock_QED()
Crop(whatever,align=true)
GaussResize(cropped_width*sqrt[1,5],cropped height*sqrt[1.5])
dull=last
sharp=dull.LimitedSharpenFaster(SMode=4,Strength=1000,wide=true,soft=75)
Soothe(sharp,dull,25)
EEDI2().TurnRight().EEDI2().TurnLeft()
FFT3DFilter(sigma=1,bw=32,bh=32,bt=3,ow=16,oh=16,sharpen=0.7)
Spline36Resize(final_width,final_heigth)
Tweak(sat=1.2)
gradfun2db()
AddGrain(5,0,0)
Levels(0,1,255,16,235)
If more deblocking is needed, I raise DeBlock_QED's quant1 and quant2 parameters, readjusting aOff1 and aOff2 accordignly so that quant1+aOff1 = quant2+aOff2 = 51. If that's not enough, I add DeGrainMedian() right after DeBlock_QED. If that's still not enough, I use the original script but replace DeBlock_QED() by DeBlock(). Is most cases, the video will have some rainbowing, so I use FFT3DFilter on the chroma planes (plane=3) with a sigma of 3 right after the ConvertToYV12() line. If halo removal is needed, I use DeHalo_Alpha() somewhere in the script.
__________________
Read Decomb's readmes and tutorials, the IVTC tutorial and the capture guide in order to learn about combing and how to deal with it.

Last edited by Chainmax; 28th February 2007 at 00:03.
Chainmax is offline   Reply With Quote
Old 28th February 2007, 17:52   #7  |  Link
Terranigma
*Space Reserved*
 
Terranigma's Avatar
 
Join Date: May 2006
Posts: 953
Pardon my ignorance, but what benefit is there to using this line
Code:

EEDI2().TurnRight().EEDI2().TurnLeft()
? for extra deblocking, smoothing, something else?
Terranigma is offline   Reply With Quote
Old 28th February 2007, 19:11   #8  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
In addition to doubling the resolution (which is always preferable to the graphics card's bilinear stretch IMO) and some very slight extra smoothing, its main use was pointed out by foxyshadis some time ago, and it consists of reconstructing outlines and borders that were destroyed by the blocking.
__________________
Read Decomb's readmes and tutorials, the IVTC tutorial and the capture guide in order to learn about combing and how to deal with it.
Chainmax is offline   Reply With Quote
Old 2nd March 2007, 09:19   #9  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,314
Sorry, I lost my cool script somewhere and haven't had time to put it back together. I could have thrown together something based on random scharfi, Didée, Mug Funky, and Chainmax scripts scattered around, but I think you want something that'll finish this month.
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. ~ Ed Howdershelt
foxyshadis is offline   Reply With Quote
Old 2nd March 2007, 12:20   #10  |  Link
Pookie
Registered User
 
Join Date: Apr 2005
Posts: 1,339
Perhaps the EEDI2().TurnRight().EEDI2().TurnLeft() line would be more useful if it was earlier in the script. Once you've hit the video with your multitude of filters, resize back down to the original dimensions. It'll hide the filtering artifacts (pretty much).
Pookie is offline   Reply With Quote
Old 2nd March 2007, 13:29   #11  |  Link
*.mp4 guy
Registered User
 
*.mp4 guy's Avatar
 
Join Date: Feb 2004
Posts: 1,350
This is a pretty good all purpose script for tough sources, Its supposed to preserve whatever detail is in the source, though, so it won't be as artefact free as some of the more aggressive scripts.

Code:
source = last

w = source.width
h = source.height

backward_vec2 = source.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=2, idx = 1, truemotion=true)
backward_vec1 = source.MVAnalyse(isb = true, delta = 1, pel = 2, overlap=4, sharp=2, idx = 1, truemotion=true)
forward_vec1 = source.MVAnalyse(isb = false, delta = 1, pel = 2, overlap=4, sharp=2, idx = 1, truemotion=true)
forward_vec2 = source.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=2, idx = 1, truemotion=true)

maskp1 = mvmask(kind=1, vectors=forward_vec1, ysc=255).UtoY()
maskp2 = mvmask(kind=1, vectors=forward_vec2).UtoY()
maskp3 = mvmask(kind=1, vectors=backward_vec1, ysc=255).UtoY()
maskp4 = mvmask(kind=1, vectors=backward_vec2).UtoY()
maskf = average(maskp1, 0.25, maskp2, 0.25, maskp3, 0.25, maskp4, 0.25).spline36resize(w, h)

smooth = source.degrainmedian(mode=3, limity=16, limituv=8).fft3dgpu(bw=16, bh=16, bt=3, sigma=4, plane=0)
source2 = maskedmerge(source, smooth, maskf)

source3 = source2.MVDegrain2(backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400,idx=2)
source3
For stronger processing use higher values of thSAD and Sigma

Last edited by *.mp4 guy; 3rd March 2007 at 13:19.
*.mp4 guy is offline   Reply With Quote
Old 2nd March 2007, 17:26   #12  |  Link
dexy
Piracy of the Caribbean
 
dexy's Avatar
 
Join Date: Jul 2005
Location: Serbia
Posts: 81
This gonna be a long day. To load those scripts I need a fully reorganization of plugins. Sorry for offtopic, but I need to get a quick answer ; As I understood, I need to split old and new plugins, new I should put into the autoload directory, and for those old I must create function which will autoload them via loadpluginex. Am I right?
__________________
when there 's nothin' left to lose, there 's nothin' left to fear
dexy is offline   Reply With Quote
Old 2nd March 2007, 19:38   #13  |  Link
*.mp4 guy
Registered User
 
*.mp4 guy's Avatar
 
Join Date: Feb 2004
Posts: 1,350
Yes. <- This is apparently too short of a message, the message is now longer. Isn't that great.
*.mp4 guy is offline   Reply With Quote
Old 2nd March 2007, 20:00   #14  |  Link
dexy
Piracy of the Caribbean
 
dexy's Avatar
 
Join Date: Jul 2005
Location: Serbia
Posts: 81
Quote:
Isn't that great
Yes, yes it is. ;-) Thanks...
__________________
when there 's nothin' left to lose, there 's nothin' left to fear
dexy is offline   Reply With Quote
Old 3rd March 2007, 06:48   #15  |  Link
DSP8000
Doom9 Member
 
DSP8000's Avatar
 
Join Date: Sep 2003
Location: Australia
Posts: 210
Quote:
maskf = average(maskp1, 0.25, maskp2, 0.25, maskp3, 0.25, maskp4, 0.25).spline36resize(w, h)
I'm getting script error "there is no script function "average"
Am I missing something? Probably Yes

Any hints?
DSP8000 is offline   Reply With Quote
Old 3rd March 2007, 07:15   #16  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,314
http://forum.doom9.org/showthread.php?t=100626
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. ~ Ed Howdershelt
foxyshadis is offline   Reply With Quote
Old 3rd March 2007, 09:02   #17  |  Link
DSP8000
Doom9 Member
 
DSP8000's Avatar
 
Join Date: Sep 2003
Location: Australia
Posts: 210
Tnx. foxyshadis.

The script provided by .mp4guy didn't do it for me at all.
This is what I'm using for the "blocky FLV videos", borowed goods from Chainmax with modification:
Code:
DirectShowSource("D:\YOU_TUBE_VIDEO.flv")
ConvertToYV12()
LanczosResize(352,288)
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\fft3dfilter.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\degrainmedian.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\deblock.dll")
Import("C:\Program Files\AviSynth 2.5\plugins\LimitedSharpenFaster.avsi")
FFT3DFilter(sigma=3,plane=4)
DeGrainMedian(mode=3)
Deblock(quant=30)
LimitedSharpenFaster(Strength=160)
Levels(0,1,255,16,235)
Faster & better results.
Adjust strenght of deblocking/denoising accordingly

Last edited by DSP8000; 5th March 2007 at 03:55. Reason: updated script
DSP8000 is offline   Reply With Quote
Old 3rd March 2007, 10:21   #18  |  Link
*.mp4 guy
Registered User
 
*.mp4 guy's Avatar
 
Join Date: Feb 2004
Posts: 1,350
I didn't realise they were that bad, fft3d with sigma4 and deblock with quant 35, your videos must have been really mangled. Some youtube stuff is reasonably good, was your source very high motion?
*.mp4 guy is offline   Reply With Quote
Old 3rd March 2007, 10:46   #19  |  Link
DSP8000
Doom9 Member
 
DSP8000's Avatar
 
Join Date: Sep 2003
Location: Australia
Posts: 210
Yeah, they we're that bad, considering that these videos that I have are low quality+recompression from YouTube=strong deblocking,denoising...

But, then again, every script has to be adjusted accordingly to the current video source, coz, one size does not fit all.

I'm making a DVD for a friend of mine(he's into gypsy music) so we'll see how all comes out.
Thanks for sharing your script with us
DSP8000 is offline   Reply With Quote
Old 3rd March 2007, 13:09   #20  |  Link
Pookie
Registered User
 
Join Date: Apr 2005
Posts: 1,339
*.mp4 guy -

source2.MVDegrain2(backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400,idx=2)

(I'm surprised Didee hasn't pointed that one out yet )
There is no idx 2 (but I did play around with that script and you're right - it is pretty good with higher noise sources)

Chainmax - Fine job on the deblocking!

I'm wondering if that new Threshold script by mf might be useful in situations where blocking occurs in moving scenes - as in every flv file on gootube...
Pookie is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 07:50.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.