Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
|
Thread Tools | Search this Thread | Display Modes |
27th October 2007, 10:13 | #1 | Link |
MPC-HC Project Manager
Join Date: Mar 2007
Posts: 2,317
|
Need help to improve PQ and compressability of Stargate SG1
Hey everyone,
I recently got the original PAL boxed set of Stargate SG1. Starting with the first season i noticed some problems with the source(i bet most of you have already seen these problems) -The image is a bit on the soft side -The episodes where filmed with 16mm film, and the grain is painfully visable -I'm not sure if the grain is to blame or the low bitrate, but especially in the shadows there are visable flashing blocks -Colors seem a bit strange on monitor, but look okay(still not great) on a crt tv(i want it to look good on monitor) Now the grain and flashing blocks take up a lot of the bitrate, so i was hoping to do the following: -Clean up some of the grain and flashing blocks so the image looks better -Remove some of the grain to increase compressability -Sharpen the image a bit to get rid of the softness -Fix the colors -The image is anamorphic but has some dark lines on all sides, should these be removed? Currently i had the following in mind: Code:
FFT3DFILTER(sigma=0.6, bw=14, bh=14, ow=7, oh=7, bt=2, dehalo=1, hr=1, sharpen=0) grainoptimizer() SeeSaw(NRlimit=0, NRlimit2=5, Sstr=2.22, Szp=1, Slimit=9, Sdamplo=16, Spower=1, sootheT=95, SdampHi=39) ColorMatrix(Mode="Rec.601->Rec.709",clamp=false) I posted some screencaps (jpg compression makes it look a bit worse) Last edited by tetsuo55; 27th October 2007 at 10:21. |
27th October 2007, 11:07 | #2 | Link |
x264 developer
Join Date: Sep 2005
Posts: 8,666
|
FFT3DFILTER(sigma=0.6, bw=14, bh=14, ow=7, oh=7, bt=2, dehalo=1, hr=1, sharpen=0)
grainoptimizer() SeeSaw(NRlimit=0, NRlimit2=5, Sstr=2.22, Szp=1, Slimit=9, Sdamplo=16, Spower=1, sootheT=95, SdampHi=39) ColorMatrix(Mode="Rec.601->Rec.709",clamp=false) Looking at your script, it seems like you can't exactly decide what you're trying to do; sigma=0.6 won't be nearly enough to really remove all the grain. If you want to keep the grain as it is, use the latest version of grainoptimizer (which requires a heavily denoised input clip). Don't use the older versions; they've been found to artifact in some cases, which is probably something you want to avoid. If you want to remove the grain entirely, use a higher sigma value and higher block sizes. FFT3DGPU is good at this, IMO. Of course if you remove the grain entirely, the grainoptimizer is a bit pointless. Also, I've found the sharpen of FFT3D is quite good; you could try it instead of SeeSaw. Raising your BT parameter values might help also. If you want maximum denoising effectiveness, use MVTools to do motion-compensated FFT3DFilter. The documentation has an example of this using RemoveGrain, but you can put anything in its place and it'll still work. |
27th October 2007, 12:02 | #3 | Link |
MPC-HC Project Manager
Join Date: Mar 2007
Posts: 2,317
|
Won't such strong denoising destroy details?
Maybe i should post a little sample clip?? do these make more sense? Code:
FFT3DFilter(sigma=2, bw=48, bh=48, bt=5, ow=16, oh=16, dehalo=1, hr=1, sharpen=0.9) grainoptimizer() ColorMatrix(Mode="Rec.601->Rec.709",clamp=false, opt=0) Or when using FFT3DGPU FFT3DGPU(sigma=2, bw=48, bh=48, bt=5, precision=2, ow=16, oh=16, dehalo=1, hr=1, sharpen=0.9) grainoptimizer() ColorMatrix(Mode="Rec.601->Rec.709",clamp=false, opt=0) |
27th October 2007, 14:21 | #4 | Link |
Flying Skull
Join Date: Jan 2005
Posts: 397
|
A good denoiser will destroy a little real detail and a lot of grain/noise (which gives a false impression of detail). But yes, denoising will always remove some real detail with the noise and it's a judgement call. It's a matter of personal taste, and also your playback setup. Nobody can choose for you, you have to try a few methods and see what you like.
If you don't like noise, I'd denoise it then perhaps sharpen it (to get your false impression of detail by a different route). If you do like grain/noise, just use GrainOptimizer to help compression. As for denoisers, if you're going to motion compensate fft3d then you may as well go the whole hog and use mvdegrain2 but with blksize=16 and overlap=8 (three times the speed of the default settings for a very small quality drop). As for blocks, deblock_qed_mt2 is good for reasonable sources. My usual DVD cleanup looks like: Code:
# source # deinterlace if needed #TV (16..235) -> PC (0..255) levels. Much better blocking and banding in encode but your display must be adjusted for the different brightness/contrast to source DVD. Tweak(bright=-16, cont=1.164, coring=false) #deblock before any denoise/crop/resize to give deblocker a clean shot DeBlock_QED_mt2( quant1=20, quant2=24, aOff1=2, bOff1=4, aOff2=4, bOff2=8 ) #crop black borders crop(2,4,-6,-8) #or whatever #resize up to next multiple of 16 for encoder efficiency and compatibility with GradFunkMirror() later spline36resize ( 16*((width+15)/16), 16*((height+15)/16) ) #Denoise or grainoptimizer to taste #grainoptimizer() #FrFun7(1.1,2.5,1.5) #fractal spatial denoiser #TTempSmooth(maxr=5) #pure temporal #fft3dgpu(sigma=1,bh=32,bw=32,bt=4,precision=2,plane=4) #fast spatio-temporal, bw/bh=32 avoids occasional colour glitches mvdegrain2p (source=last,blksize=16,overlap=8,sharp=2,thSAD=400,idx=1) #mocomped temporal, the daddy #sharpen if you want (not my thing, ask somebody else) #Tackle banding in flat areas GradFunkMirror(1.2) function MVDegrain2p(clip "source", int "blksize", int "overlap", int "sharp", int "thSAD", int "idx") { blksize=default(blksize,8) # blksize value (4, 8 or 16) overlap=default(overlap,blksize/2) # overlap value (0 to half blksize) sharp=default(sharp,1) # thSAD=default(thSAD,400) # higher risks ghosting, lower risks "staggered denoising" idx=default(idx,1) # use various idx for different sources in same script backward_vec2 = source.MVAnalyse(isb = true, delta = 2, pel = 2, blksize=blksize, overlap=overlap, sharp=sharp, idx = idx) backward_vec1 = source.MVAnalyse(isb = true, delta = 1, pel = 2, blksize=blksize, overlap=overlap, sharp=sharp, idx = idx) forward_vec1 = source.MVAnalyse(isb = false, delta = 1, pel = 2, blksize=blksize, overlap=overlap, sharp=sharp, idx = idx) forward_vec2 = source.MVAnalyse(isb = false, delta = 2, pel = 2, blksize=blksize, overlap=overlap, sharp=sharp, idx = idx) source.MVDegrain2(backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=thSAD,idx=idx) } Function GradFunkMirror( clip c, float "strength" ) { #counters edge effects for gradfun2db() #gradfun2db() requires width mod8 and height mod2 strength = default( strength, 1.2 ) w = c.width() h = c.height() vflip = c.FlipVertical() hflip = c.FlipHorizontal() stackhorizontal( hflip.crop( w-16, 0, 16, h ).addborders( 0, 16, 0, 16 ), \ stackvertical( vflip.crop( 0, h-16, w, 16 ), c, vflip.crop(0, 0, w, 16 ) ), \ hflip.crop( 0, 0, 16, h ).addborders( 0, 16, 0, 16 ) ) gradfun2db( strength ) crop( 16, 16, -16, -16 ) Return last } Last edited by Morte66; 27th October 2007 at 20:19. |
30th October 2007, 11:52 | #5 | Link |
MPC-HC Project Manager
Join Date: Mar 2007
Posts: 2,317
|
thanks for the advice so far guys
My videocard has died so i cannot try anything yet However i did manage to upload a sample clip The clip has dark and light parts, imho the grain is even worse than i thought 50mb sample: http://www.sendspace.com/file/2sj4tm Edit: Looks like its interlaced too? Last edited by tetsuo55; 30th October 2007 at 12:06. |
30th October 2007, 17:39 | #6 | Link |
Flying Skull
Join Date: Jan 2005
Posts: 397
|
That's progressive.
I'd be happy with what I already posted to process that, but you need to try it for yourself once your computer is back in action. The only other thing I'd do is reduce it from 25 to 24fps and correct the audio pitch, but most people aren't as bothered about PAL pitch errors as me. |
30th October 2007, 19:42 | #8 | Link |
Flying Skull
Join Date: Jan 2005
Posts: 397
|
Film and a lot of TV (less in Europe) is shot at 24fps. PAL works at 25fps. One is usually converted to the other by just speeding it up, the video plays faster and the audio is higher pitched. Most people don't notice (e.g. you weren't aware of it), but some of us have a cow over it. So I use AviSynth to slow the video back to 24fps, drop the audio pitch to what it should be, and resample the audio for compatibility.
So my normal script starts like: Code:
video=DGDecode_mpeg2source("C:\meGUI\SG1\SG1.d2v") #video audio=nicac3source("SG1 T01 2_0ch 224Kbps DELAY -128ms.ac3",2) #audio audiodub(video,audio) # composite audio/video stream delayaudio(-0.128) # adjust sync like DGIndex told us to in filename assumefps(24,1,true) # redefine video as 24fps and audio as 46080Hz, slowing both to be correct ssrc(48000,false) # resample audio to 48KHz for hardware compatibility #rest of script does video stuff (denoise etc) Just try it a couple of times and see if you care. Last edited by Morte66; 30th October 2007 at 19:53. |
30th October 2007, 19:51 | #9 | Link |
MPC-HC Project Manager
Join Date: Mar 2007
Posts: 2,317
|
morte66,
Could you please post all the needed files for both those scripts? Megui and DVd rebuilder are giving me errors about everything I installed avisynth, megui, dvd rebuilder and masktools Last edited by tetsuo55; 30th October 2007 at 20:31. |
31st October 2007, 09:05 | #10 | Link | |
Flying Skull
Join Date: Jan 2005
Posts: 397
|
Quote:
A lot of it is internal or installed by meGUI, make sure you have AviSynth v2.57 (I don't think meGUI touches that). For Deblock_qed_mt2 see the wiki link in my earlier post. For FrFun7: http://soulhunter.chronocrossdev.com...rfun7_rev6.zip For TTempSmooth, fft3dgpu, MVDegrain, gradfun2db and lots more see here: http://avisynth.org/warpenterprises/ GrainOptimizer you know. If anything is still missing, Google. |
|
31st October 2007, 15:07 | #11 | Link |
MPC-HC Project Manager
Join Date: Mar 2007
Posts: 2,317
|
Deblock_qed_mt2 wiki link does not work, i cannot find the filter in google (they all point to the non working wiki)
I now have the following files fft3dgpu_20060614.zip frfun7_rev6.zip gradfun2db_25_dll_20060315.zip mvtools_25_dll_20060605.zip ttempsmooth_25_dll_20051117.zip tweakcolor_25_dll_20040412.zip Avisynth_257.exe megui-0.2.5.1007.exe GrainOptimizer 2.01.zip I found some newer versions though (not sure if this will break the script you posted?) fft3dgpu0.8.2.7z megui-0.2.6.1034.exe That audio fix part does not require any extra plugins right? Can i just cut and paste that to source like this: Code:
# source video=DGDecode_mpeg2source("C:\meGUI\SG1\SG1.d2v") #video audio=nicac3source("SG1 T01 2_0ch 224Kbps DELAY -128ms.ac3",2) #audio audiodub(video,audio) # composite audio/video stream delayaudio(-0.128) # adjust sync like DGIndex told us to in filename assumefps(24,1,true) # redefine video as 24fps and audio as 46080Hz, slowing both to be correct ssrc(48000,false) # resample audio to 48KHz for hardware compatibility # deinterlace if needed #TV (16..235) -> PC (0..255) levels. Much better blocking and banding in encode but your display must be adjusted for the different brightness/contrast to source DVD. Tweak(bright=-16, cont=1.164, coring=false) #deblock before any denoise/crop/resize to give deblocker a clean shot DeBlock_QED_mt2( quant1=20, quant2=24, aOff1=2, bOff1=4, aOff2=4, bOff2=8 ) #crop black borders crop(2,4,-6,-8) #or whatever #resize up to next multiple of 16 for encoder efficiency and compatibility with GradFunkMirror() later spline36resize ( 16*((width+15)/16), 16*((height+15)/16) ) #Denoise or grainoptimizer to taste #grainoptimizer() #FrFun7(1.1,2.5,1.5) #fractal spatial denoiser #TTempSmooth(maxr=5) #pure temporal #fft3dgpu(sigma=1,bh=32,bw=32,bt=4,precision=2,plane=4) #fast spatio-temporal, bw/bh=32 avoids occasional colour glitches mvdegrain2p (source=last,blksize=16,overlap=8,sharp=2,thSAD=400,idx=1) #mocomped temporal, the daddy #sharpen if you want (not my thing, ask somebody else) #Tackle banding in flat areas GradFunkMirror(1.2) function MVDegrain2p(clip "source", int "blksize", int "overlap", int "sharp", int "thSAD", int "idx") { blksize=default(blksize,8) # blksize value (4, 8 or 16) overlap=default(overlap,blksize/2) # overlap value (0 to half blksize) sharp=default(sharp,1) # thSAD=default(thSAD,400) # higher risks ghosting, lower risks "staggered denoising" idx=default(idx,1) # use various idx for different sources in same script backward_vec2 = source.MVAnalyse(isb = true, delta = 2, pel = 2, blksize=blksize, overlap=overlap, sharp=sharp, idx = idx) backward_vec1 = source.MVAnalyse(isb = true, delta = 1, pel = 2, blksize=blksize, overlap=overlap, sharp=sharp, idx = idx) forward_vec1 = source.MVAnalyse(isb = false, delta = 1, pel = 2, blksize=blksize, overlap=overlap, sharp=sharp, idx = idx) forward_vec2 = source.MVAnalyse(isb = false, delta = 2, pel = 2, blksize=blksize, overlap=overlap, sharp=sharp, idx = idx) source.MVDegrain2(backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=thSAD,idx=idx) } Function GradFunkMirror( clip c, float "strength" ) { #counters edge effects for gradfun2db() #gradfun2db() requires width mod8 and height mod2 strength = default( strength, 1.2 ) w = c.width() h = c.height() vflip = c.FlipVertical() hflip = c.FlipHorizontal() stackhorizontal( hflip.crop( w-16, 0, 16, h ).addborders( 0, 16, 0, 16 ), \ stackvertical( vflip.crop( 0, h-16, w, 16 ), c, vflip.crop(0, 0, w, 16 ) ), \ hflip.crop( 0, 0, 16, h ).addborders( 0, 16, 0, 16 ) ) gradfun2db( strength ) crop( 16, 16, -16, -16 ) Return last } Last edited by tetsuo55; 31st October 2007 at 15:10. |
31st October 2007, 17:25 | #12 | Link | ||||
Flying Skull
Join Date: Jan 2005
Posts: 397
|
Quote:
[When you start up with avisynth, it's a good idea to comment out the bits that are tricky to install then add them back in one line at a time.] Quote:
Quote:
Quote:
- You might get a different audio delay for the whole file from the cut section, so you need to check the filename and adjust the script. - You can put the two function definitions at the end in files called "GradFunkMirror.avsi" and "MVDegrain2p.avsi" in your avisynth plugins directory so you won't have to paste them into every script, you can just call them. - {edit} And make sure you pick the right audio track, I went for French! [I ran the encode from that script, and there's a lot of banding broken into crawling miniblocks. GradFunkMirror(1.2) on the encode side will help spread them evenly, and if you use x264 to encode then some adaptive quantization (e.g. --aq-strength 0.4 --aq-sensitivity 15) will stop the encoder clumping them into nasty macroblocks. Then if you use the ffdshow video decoder, you can run GradFunkMirror again in its AviSynth section to counter banding caused by encoder quantisation. I find it's better to use a small dose of GradFunkMirror before and after a lossy encode than a large dose at either stage. But worry about this once you have AviSynth working.] Last edited by Morte66; 31st October 2007 at 17:57. |
||||
1st November 2007, 10:32 | #14 | Link |
MPC-HC Project Manager
Join Date: Mar 2007
Posts: 2,317
|
So if i understand it correctly, the script causes "a lot of banding broken into crawling miniblocks"
My goals are 2fold: Firstly i want to create a DVD5 of each DVD9 disk. Secondly i want to make x264 encodes with subs, ac3 audio(and in later seasons commentary tracks) for my mediacenter I was hoping to have a cleanup that would not require any scripts afterward. |
1st November 2007, 22:27 | #15 | Link | |||
Flying Skull
Join Date: Jan 2005
Posts: 397
|
Quote:
You might prefer a lighter denoiser (e.g. fft3dgpu with sigma=1) to leave some noise to cover them. You might prefer GrainOptimizer to keep more noise, but store it efficiently. Or you might decide it's fine as it stands and just go with it. This is why you have to try it and see for yourself. Quote:
Quote:
Last edited by Morte66; 1st November 2007 at 22:30. |
|||
2nd November 2007, 00:48 | #16 | Link |
Registered User
Join Date: Jan 2007
Posts: 24
|
I find that using ffdshow's DeBand filter helps significantly, but I'm quite curious about hiding the banding with grain. How would, say, using gradfun2db effectively, be accomplished with an encoder like x264, which really just seems to remove the added grain or simply artifact at bit-rates that correspond to CRF 18?
|
2nd November 2007, 09:29 | #17 | Link | |
Flying Skull
Join Date: Jan 2005
Posts: 397
|
Quote:
Once you decode your x264, there will be some banding (and blocking at band edges). You can reduce it with appropriate encoder settings but it will be there. Ffdshow's Deband will sort it out pretty well, provided you gave it something it can work with. Or you can use noise to hide it, or a bit of DeBand and a bit of noise. As for noise... You've got two choices with x264: (a) don't denoise the original so much and adjust the x264 deadzone settings to make it retain more noise (which pushes the bitrate up for a given quality), or do a denoised encode and use the noise generator in ffdshow on playback to create "fake noise". |
|
5th November 2007, 15:47 | #18 | Link |
MPC-HC Project Manager
Join Date: Mar 2007
Posts: 2,317
|
Okay i must be doing something wrong
i formatted the pc with the new videocard, and its sortof working Anyway: I installed avisynth I installed megui I installed all the plugins to avisynth/plugins Now when i try to load the avs script like you posted the first time (without the audio changes), every function gives me a "invalid argument" error, rendereing every part of the script useless What am i doing wrong here? |
6th November 2007, 08:56 | #20 | Link |
MPC-HC Project Manager
Join Date: Mar 2007
Posts: 2,317
|
Well as i said i did not use that part.
Eventhough megui gives errors for every line of the script, dvd-rebuilder worked fine! However the result with dvd-rebuilder was not so good, the image was all washed out, way too bright, and almost all detail was lost into the blurryness. I have to admit that 80% of the grain was gone but the reslting image was too bad to watch. I'm going to play around with the settings a bit and see what i can make out of it |
|
|