PAL 1080p to PAL 576i with high quality motion compensated interpolation

[hydra3333]

I have a (legal) 1080p source of sports footage in an mp4 container. I only have a DVD player and an old simple media player. Sports footage looks better interlaced anyway (I once heard it called "motion fluidity").

I can use Mr Donald Graft's dgdecodeNV to make use of my Ti750i graphics card to resize the 1080p input to 576p apparently like
LoadPlugin("C:\DGindex\DGDecodeNV.dll")
DGSource("footy.dgi",resize_w=720,resize_h=576)

But then I'd need to
(i) double the framerate using the latest nice accurate form of motion compensated interpolation (something in mvtools2?)
(ii) and turn that into a DVD compatible TFF interlaced fields 576i (ie 50 fields/sec) output.

Suggestions very much welcomed on how to do (i) and (ii).


edit: for (ii) I think this turns 576p50 into 576i TFF interlaced
AssumeTFF()
SeparateFields()
SelectEvery(4,0,3) # top field first
Weave()

edit2: is it this using mvtools2 ?
#To double fps with MFlowFps for 'best' results (but slower processing):
super = MSuper(pel=2)
# Use block overlap, halfpixel accuracy and Exhaustive search
backward_vec = MAnalyse(super, overlap=4, isb = true, search=3)
forward_vec = MAnalyse(super, overlap=4, isb = false, search=3)
MFlowFps(super, backward_vec, forward_vec, num=2*FramerateNumerator(last), den=FramerateDenominator(last))

edit3: I just saw SVPflow ... and I'm a bit unsure if it works with avisynth 2.57
http://forum.doom9.org/showthread.php?t=164554

[Sharc]

What is the framerate of your progressive source? Isn't it already shot at "double rate" of 50 (or 60) fps?
If so, just resize it to DVD (720x576) and apply your 1st script to produce the interlaced DVD format.

[hydra3333]

What is the framerate of your progressive source? Isn't it already shot at "double rate" of 50 (or 60) fps?
mediainfo says the source is 1080p25
Frame rate : 25.000 fps
I paid 25 smackers to the local footy league for a prelim final game of local footy, and they emailed me a download link with this mp4 file as the one and only option.
It also is blocky as anything so I'm hoping this sorts it out (straight after the DGsource)
Deblock_QED(quant1=36)

[hydra3333]

Since I'm doing deblocking too, this is what I've got so far. Feel free to criticize.
SetMTmode(mode=5,threads=8) # start with mode=5 forAVIsource http://forum.doom9.org/showthread.php?p=1067216#post1067216
setmemorymax(1024)
LoadPlugin("C:\SOFTWARE\DGindex\DGDecodeNV.dll")
LoadPlugin("C:\SOFTWARE\DGindex\DGDecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\mt_masktools-25.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\mvtools2.dll")
#
Import("C:\Program Files (x86)\AviSynth 2.5\Deblock_QED_MT2.avs")
Import("C:\Program Files (x86)\AviSynth 2.5\LimitedSharpenFaster.avs")
DGSource("T:\HDTV\WDTVlive\MP4hd\progressive\2015.09.05-SANFL-QualifyingFinal-Westies-d-Port-Q1-HD.dgi")
SetMTmode(mode=2) #
Deblock_QED(quant1=36)
Spline36resize(720,576)
#--
#To double fps with MFlowFps for *best* results (but slower processing)
super = MSuper(pel=2)
# Use block overlap, halfpixel accuracy and Exhaustive search
backward_vec = MAnalyse(super, overlap=4, isb = true, search=3)
forward_vec = MAnalyse(super, overlap=4, isb = false, search=3)
MFlowFps(super, backward_vec, forward_vec, num=2*FramerateNumerator(last), den=FramerateDenominator(last))
#--
AssumeTFF() #choose the SAME field order like before deinterlacing
SeparateFields().SelectEvery(4,0,3).Weave() #reinterlace.
AssumeTFF() #choose the SAME field order like before deinterlacing
AssumeFPS(25)
#--
ColorMatrix(interlaced=true,mode="Rec.709->Rec.601")
SetMTmode(mode=5,threads=8)
Distributor() # use this when using HC and SetMTmode, per http://forum.doom9.org/showthread.php?p=1063622#post1063622

[colours]

Changing the colour matrix should be done prior to interlacing, because the interlacing step introduces aliasing, which makes the chroma resampling considerably less accurate. You could even do it before resizing to 576p to minimise the inaccuracy, but that's probably unnecessary.

[Sharc]

The SVP looks promising. Is it up and running?

If your 1080p25 source is a (poorly) deinterlaced version of a 1080i50 original you may want to pre-propcess (clean it up) it with QTGMC first.

[hydra3333]

If your 1080p25 source is a (poorly) deinterlaced version of a 1080i50 original you may want to pre-propcess (clean it up) it with QTGMC first.Thank you. I'd like to do that ... I only have the 1080p25 source :(
From my standard script template, I added SVP's Interframe and am using this :-
setmemorymax(1024)
LoadPlugin("C:\SOFTWARE\DGindex\DGDecodeNV.dll")
LoadPlugin("C:\SOFTWARE\DGindex\DGDecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\AddGrainC.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\avstp.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\dfttest.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\dither.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\EEDI2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\eedi3.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\FFT3DFilter.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\mt_masktools-25.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\mvtools2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\nnedi-from-tcritical.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\nnedi2-from-tcritical.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\nnedi3-from-tcritical.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\RemoveGrainSSE2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\RepairSSE2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\SSE2Tools.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\TDeint.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\VerticalCleanerSSE3.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\VariableBlur.dll")
LoadCPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\yadif.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\AGC.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\CNR2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\fft3dfilter.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\dctfilter.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\degrainmedian.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\Convolution3d.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\DePan.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\despot.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\WarpSharp.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\FFTQuiver.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\aWarpSharp.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\Unfilter.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\RemoveDirtSSE2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\Yadifmod.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\Colormatrix.dll")
#
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\InterFrame-2.8.2\Dependencies\svpflow1.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\InterFrame-2.8.2\Dependencies\svpflow2.dll")
#
Import("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\dither.avs")
Import("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\mt_xxpand_multi.avs")
Import("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\QTGMC-3.33.avs")
Import("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\SMDegrain-v.2.2d.avs")
Import("C:\Program Files (x86)\AviSynth 2.5\plugins-qtgmc-smdegrain\LSFmod.v1.9.avs")
Import("C:\Program Files (x86)\AviSynth 2.5\Deblock_QED_MT2.avs")
Import("C:\Program Files (x86)\AviSynth 2.5\LimitedSharpenFaster.avs")
#
Import("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\InterFrame-2.8.2\InterFrame2.avsi")
#
DGSource("T:\HDTV\WDTVlive\MP4hd\progressive\interframe\2015.09.05-SANFL-QualifyingFinal-Westies-d-Port-Q1-HD.dgi")
Deblock_QED() #default quant1=24
Deblock_QED() #default quant1=24
########
#--progressive-only MSUPER/MDegrain1 for use with SetMTMode(2)
#blksize=8 # 4, 8 or 16 ( default is 8 ). Larger blocks are less sensitive to noise, are faster, but also less accurate.
#overlap=4 # overlap value (0 to 4 for blksize=8) Must be even and less than block size
#dct=0 # use dct=1 for clip with light flicker
#super = MSuper(pel=2,chroma=true,sharp=2)
#backward_vec1 = MAnalyse(super, isb = true, delta = 1, blksize=blksize, overlap=overlap, dct=dct, chroma=true)
#forward_vec1 = MAnalyse(super, isb = false, delta = 1, blksize=blksize, overlap=overlap, dct=dct, chroma=true)
#MDegrain1(super, backward_vec1,forward_vec1,thSAD=400,plane=4)
########
LimitedSharpenFaster(smode=4,strength=100)
Spline36resize(720,576)
#Lanczos4resize(720,576)
#--
#InterFrame (GPU=true, Preset="Medium", Tuning="Smooth", Cores=8, FrameDouble=true)
#InterFrame (GPU=true, Preset="Medium", Tuning="Film", Cores=8, FrameDouble=true)
InterFrame (GPU=false, Preset="Medium", Tuning="Smooth", Cores=8, FrameDouble=true)
#InterFrame (GPU=false, Preset="Medium", Tuning="Film", Cores=8, FrameDouble=true)
#--
AssumeTFF() #choose the SAME field order like before deinterlacing
Blur(0,0.25).SeparateFields().SelectEvery(4,0,3).Weave() #reinterlace - ASSUMED TFF HERE # BLUR(0,1) per http://forum.doom9.org/showthread.php?p=1488308#post1488308
AssumeTFF() #choose the SAME field order like before deinterlacing
AssumeFPS(25)
ColorMatrix(interlaced=true,mode="Rec.709->Rec.601")
AssumeTFF()
AssumeFPS(25)


The "deblock_qed"s don't seem to be doing anything useful. I wonder if I should surround them with separatefields and weave (in case it's looking for a smaller size to deblock or something like that) ?
SeparateFields()
Deblock_QED() #default quant1=24
Deblock_QED() #default quant1=24
weave()


edit: uploaded a 5mb .mp4 sample of the source chopped out using ffmpeg with -c:v copy -c:a copy
https://drive.google.com/file/d/0B5RV2aJ2vdhSR3hocGJfbmhwVGs/view?usp=sharing
https://drive.google.com/file/d/0B5RV2aJ2vdhSVk55dm5KMWxaR3M/view?usp=sharing

[hydra3333]

OK, well now I am a bit stuck, nothing easy I try deblocks that source.
Can someone please provide advice on how to significantly deblock that source ?
https://drive.google.com/file/d/0B5RV2aJ2vdhSVk55dm5KMWxaR3M/view?usp=sharing

[Music Fan]

edit: uploaded a 5mb .mp4 sample of the source chopped out using ffmpeg with -c:v copy -c:a copy
https://drive.google.com/file/d/0B5RV2aJ2vdhSR3hocGJfbmhwVGs/view?usp=sharing
https://drive.google.com/file/d/0B5RV2aJ2vdhSVk55dm5KMWxaR3M/view?usp=sharing
The bitrate is too low for 1080p, there are a lot of artifacts that should be removed or reduced before doubling the framerate.


edit : I believe it's better to deblock before using SeparateFields() .

[hydra3333]

Thanks. I tried deblock with and without separatefields().weave() and it didn't make any difference that I could see.
Anyone ?

[vivan]

The "deblock_qed"s don't seem to be doing anything useful. I wonder if I should surround them with separatefields and weave (in case it's looking for a smaller size to deblock or something like that) ?No, if your video is progressive it's a bad idea. It will not make blocks smaller - it will make them bigger and just wrong.

I'm not sure how deblock_qed works but usually deblock deblocks until it's not satisfied (result have no blockiness from it's point of view that depends on it's settings), so chaining it is not effective (if not useless).
Instead it has options (all of them) that affect it's strength. The main one is quant1 (higher means stronger deblock). I've never used it so I don't know how to tune other options it has...

[Boulder]

I'd also try SmoothD2 for deblocking. I've found it quite good indeed :)

[hydra3333]

No, if your video is progressive it's a bad idea. It will not make blocks smaller - it will make them bigger and just wrong. Instead it has options (all of them) that affect it's strength. The main one is quant1 (higher means stronger deblock). I've never used it so I don't know how to tune other options it has...
OK. Neither do I, the doco is a tad underwhelming in explanation-for-dummies. I did see this
Deblock_QED(quant1=18,aOff1=16,quant2=28,aOff2=11) # (suggested by Didee for HD material somewhere).
Although this http://forum.doom9.org/showthread.php?p=1456387#post1456387 says
Wrong filter for this crappy mess. You don't use razorblades to fell a tree. Try ... Deblock(50)
which seems a tad strong given
quant: 0 to 51 (default: 25) Emulated Quantizer. Specifies strength of the deblocking process.
and this http://forum.doom9.org/showthread.php?p=944741#post944741 says
With increasing "severe-ness" of blocking artefacts, deblock_qed becomes more and more counter-indicated: Such stuff *needs* block-interior smoothing, and deblock_qed is mostly about *avoiding* block-interior smoothing.


So now I'm confused. I guess I'll look at increasing quant1, aOff1, bOff1 ... and then try DGIndex's Deblock/BlindPP at http://rationalqm.us/dgmpgdec/DGDecodeManual.html

http://avisynth.nl/index.php/Deblock_QED says
Syntax and Parameters
DeBlock_QED (clip clp, int "quant1", int "quant2", int "aOff1", int "bOff1", int "aOff2", int "bOff2", int "uv")

clp clip = Input clip.
quant1 int = 24 Strength of block edge deblocking.
quant2 int = 26 Strength of block internal deblocking.
aOff1 int = 1 Halfway "sensitivity" and halfway a strength modifier for borders. Setting it higher means than more edges will deblocked.
aOff2 int = 1 Halfway "sensitivity" and halfway a strength modifier for block interiors.
bOff1 int = 2 "Sensitivity to detect blocking" for borders. There again, the higher, the stronger.
bOff2 int = 2 "Sensitivity to detect blocking" for block interiors.
aOff and bOff controls block detection and removal strength, for quant1 and quant2 respectively. See DeBlock for more information.
uv int = 3 Deblock chroma:
2 : Copy chroma from source.
3 : Process chroma.
1/-1 : Process chroma with normal/strong Deblock().


And http://rationalqm.us/dgmpgdec/DGDecodeManual.html#BlindPP says
BlindPP(quant=2, cpu=6, iPP=false, moderate_h=20, moderate_v=40)
quant: 0 to 31 (default: 2) Emulated Quantizer. Specifies strength of the deblocking process.
cpu: 0 to 6 (default: 6) Same function as in MPEG2Source(), but with different default value.
iPP: true/false (default: false) Same function as in MPEG2Source(), but with different default value. Automatic-mode is NOT available.
moderate_h, moderate_v: 0 to 255 (default: moderate_h=20, moderate_v=40) Block Detection Sensitivity. (moderate_h=horizontal, moderate_v=vertical) Smaller values are stronger, use with care.

So I suppose could try starting with
BlindPP(quant=10, cpu=6, iPP=false, moderate_h=15, moderate_v=30)

[hydra3333]

Hmm neither of these seemed to do much :(
O=LAST
#P=Deblock_QED(quant1=48,aOff1=24,bOff1=24,aOff2=24,bOff2=24,uv=-1)
P=Deblock(quant=51,aOffset=24,bOffset=24)
dcwbox8(O,"original",P,"deblocked") # slice O and P into 8x8 and splice into O/P/O/P/O/P/O/P vert and horiz, label each splice, and present as single composite frames

[hydra3333]

I'd also try SmoothD2 for deblocking. I've found it quite good indeed :)

OK, a test for tomorrow.

http://forum.doom9.org/showthread.php?t=164800
http://avisynth.nl/index.php/External_filters#Deblocking
https://sites.google.com/site/jconklin754smoothd2/download

[Sharc]

When you cascade smoothd2(quant=31) say 4...8 times you will definitely see it's impact, but I'm not sure whether you like the "wax" picture any more ....

[hydra3333]

From a couple of quick tests this seemed to help.
SmoothD2(quant=5, qtype=4, num_shift=3, ncpu=4)
SmoothD2c(quant=5, qtype=4, num_shift=3, ncpu=4)

[Boulder]

The source looks like it really cannot be saved by any deblocker because there are no details to restore :D I've sometimes used quite unorthodox measures with such sources by replacing the extreme blocking with noise.

So maybe something like FFT3DFilter(sigma=5) + GrainFactory3(20,30,40) would help you? I would first use FFT3DFilter, then downsize to 720x576, do the framerate conversion, add noise and finally do whatever you need to do to get the final format you want (depending on whether you are going for case i or ii you mentioned).

[Sharc]

Good idea to add noise, Boulder!
How about this (probably a bit overdone)?
FFT3DFilter(sigma=5)
spline16resize(720,576)
AddGrainC(var=150.0, uvar=0.5, hcorr=0.0, vcorr=0.0, seed=-1, constant=false, sse2=true)
FFT3DFilter(sigma=5)
QTGMC(preset="medium", InputType=1)

[hydra3333]

Interesting! So is the below, based on yours. I must try full encodes and watch the variations on the telly.
LAST.HEIGHT==1080 ? AddBorders(0,0,0,8) : LAST
O = LAST
SmoothD2(quant=5, qtype=4, num_shift=3, ncpu=4)
SmoothD2c(quant=5, qtype=4, num_shift=3, ncpu=4)
FFT3DFilter(sigma=2)
#LAST.HEIGHT==1088 ? cropbottom(8) : LAST
#ColorMatrix(interlaced=true,mode="Rec.709->Rec.601")
# for test purposes, leave it at full size
#Spline36resize(720,576)
AddGrainC(var=2.0, uvar=0.5, hcorr=0.0, vcorr=0.0, seed=-1, constant=false, sse2=true)
#FFT3DFilter(sigma=1.5)
#QTGMC(preset="medium", InputType=1)
LimitedSharpenFaster(smode=4,strength=100)
P = LAST
boxify8x8(O,"original",P,"deblocked")

[Sharc]

@hydra3333
Out of curiosity, may I ask you whether you have been successful with the interpolation of your clip. If so, which method/tool did you choose?

[hydra3333]

Hello and not successful, no. It was too far gone for anything I could throw at it to make it better. I lived with this (it may be wrong, but it did something).
SeparateFields()
Deblock().deblock().deblock().deblock()
Deblock_QED() #default quant1=24
#SmoothD2(quant=4, qtype=4, num_shift=3, ncpu=4)
#SmoothD2c(quant=4, qtype=4, num_shift=3, ncpu=4)
Weave()
AssumeTFF()
# For RE-INTERLACING http://forum.doom9.org/showthread.php?p=1110741#post1110741
# AssumeTFF().SeparateFields().SelectEvery(4, 0, 3).Weave() -> TFF
# AssumeBFF().SeparateFields().SelectEvery(4, 0, 3).Weave() -> BFF
# AssumeTFF().SeparateFields().SelectEvery(4, 1, 2).Weave() -> BFF
# AssumeBFF().SeparateFields().SelectEvery(4, 1, 2).Weave() -> TFF
QTGMC(Preset="Slow",EdiThreads=8,Sharpness=1.2,SLMode=1) # result is double framerate progressive, so re-interlate it later
AddGrainC(var=2.0, uvar=0.5, hcorr=0.0, vcorr=0.0, seed=-1, constant=false, sse2=true)
Spline36resize(720,576)
LimitedSharpenFaster(smode=4,strength=100)
AssumeTFF()
Blur(0,0.25).SeparateFields().SelectEvery(4,0,3).Weave() #reinterlace - ASSUMED TFF HERE # BLUR(0,1) per http://forum.doom9.org/showthread.php?p=1488308#post1488308

edit: oh, did you mean Interframe ? Yes, that worked.

[Music Fan]

This script contains resize but no crop while you wanted to crop, are you sure you didn't forget it ?

[hydra3333]

Thank you. It's a snippet of the script.

[Sharc]

@hydra
I found it better to resize before adding the grain.

[hydra3333]

@hydra
I found it better to resize before adding the grain.

Ah. I should have thought of that. Thanks.