TDeint and TIVTC

[ChaosKing]

So all fansubber? :D

[FranceBB]

So all fansubber? :D

ROTFL
I've seen fansubber doing that in the 2006-2008 era.
Madness... and far too much free time those youngsters had back in the days...

(and I was "guilty" of being one of them back in the days :P )

[TheFluff]

Even back when some anime nerds did do "manual" IVTC, they never actually went through every frame one by one. The typical workflow in tools like YATTA and its successors was to use pattern guidance; that is find sequences of frames with a consistent 3:2 cadence and then generate an override file for those sequences that forced that pattern. When you were lucky you only had like 3-4 pattern breaks for an entire episode (e.g. it was produced as progressive and telecined very late in the production process); if you were unlucky you had hundreds, but that's still far short of looking at every frame.

[Katie Boundary]

Easy for humans. Quite a challenge for computer algorithms

LOL no. Interlacing is a mathematically very unique phenomenon that has nothing to do with image-recognition or any other human visual system bullshit. The key characteristic that makes it incredibly easy to distinguish from normal detail is that non-adjacent pixels are more similar to each other than adjacent pixels. In particular, for any given column of five pixels A, B, C, D, and E:

- pixels B and D will be more similar to each other than either one is to A, C, or E; and

- pixel C will be more similar to A and E than it is to B or D

This is literally all you need to distinguish interlacing from detail or from noise. If the problem seems "difficult", it's because certain people are ignoring differences between non-adjacent pixels and dicking around with multiplication and arbitrary user-defined threshold parameters, as if they were just copying and pasting from edge-detection algorithms. Hell, neither of TFM's combing metrics even bother to look at the differences between A and B, or between D and E, and Metric 1 doesn't bother looking at pixels A or E at all.

Katie, you think they're different 'cause you "see" them as being different, but a computer doesn't see, a computer works with numbers.

No, they are different because they are MATHEMATICALLY different. In fact, I've sometimes had to use sharpening or other filters to reveal interlacing that was too subtle for the human eye to notice, but which a competently designed algorithm would have picked up easily.

The whole frame is divided in blocks and macroblocks of a certain size, they can be 4x4, 8x8, 16x16 etc.

Macroblocks have nothing to do with deinterlacing because deinterlacing filters don't work on encoded video. They work on decoded video, which is a bunch of individual pixel values.

[FranceBB]

Macroblocks have nothing to do with deinterlacing because deinterlacing filters don't work on encoded video. They work on decoded video, which is a bunch of individual pixel values.

By motion vectors I mean newly created Vectors to find differences on an uncompressed AV Stream frameserved by Avisynth. Vectors are created from scratch trying to compare blocks temporarily and thus finding out repeating patterns (this is how TDecimate gets rid of telecine for instance) but sometimes this doesn't work well and I've already explained why...

[StainlessS]

If only we had some coders that were as brilliant as Katie, wouldn't that be something.

[Katie Boundary]

By motion vectors I mean newly created Vectors to find differences on an uncompressed AV Stream frameserved by Avisynth. Vectors are created from scratch trying to compare blocks temporarily

Again, nothing to do with interlacing.

repeating patterns (this is how TDecimate gets rid of telecine for instance) but sometimes this doesn't work well and I've already explained why...

YOU DON'T NEED TO LOOK FOR PATTERNS. YOU ONLY NEED TO COMPARE DIFFERENCES BETWEEN ADJACENT AND NON-ADJACENT PIXELS.

If only we had some coders that were as brilliant as Katie, wouldn't that be something.

Coding = taking existing ideas and translating them into computer code.

I'm not good at that. I'm better at correcting stupid ideas before they reach the "translated into code" stage.

[LigH]

So don't forget to consider all the possible kinds of motions that could appear in a video. There is not only "uniform linear horizontal", although it may be the most common. I am pretty sure one can find "academic examples" of motion which does not clearly calculate to lower differences for non-adjacent lines (like even pixel wide vertical).

I remember a more or less entertaining approach of (IIRC?) HDConvertToX: Let x264 encode a tiny crop one time with --tff and another time with --bff, and compare the resulting statistic outputs... :D

[FranceBB]

So don't forget to consider all the possible kinds of motions that could appear in a video. There is not only "uniform linear horizontal", although it may be the most common. I am pretty sure one can find "academic examples" of motion which does not clearly calculate to lower differences for non-adjacent lines (like even pixel wide vertical).


Yep, precisely. :)
That's exactly what I was trying to tell her...
It's not "that easy" nor "that simple" as she stated.
Anyway, I'm not gonna reply any further, I feel like I can't get my message through, but after all, I've always been terrible in talking with women even when it was weather-related chit-chat, let alone about encoding xD

[wonkey_monkey]

In particular, for any given column of five pixels A, B, C, D, and E:

- pixels B and D will be more similar to each other than either one is to A, C, or E; and

- pixel C will be more similar to A and E than it is to B or D

This is literally all you need to distinguish interlacing from detail or from noise.

Nonsense. Interlaced video doesn't always meet these criteria and non-interlaced video doesn't always fail to meet them.

Besides, there's more to it than simply deciding (imperfectly) whether or not an individual pixel is "interlaced".

There's no perfect solution to this problem. Any attempt will involve compromises.

If you haven't even implemented your solution, how do you know it's better than any other?

[Katie Boundary]

So don't forget to consider all the possible kinds of motions that could appear in a video.

Motion has nothing to do with interlacing.

[videoh]

Motion has nothing to do with interlacing. Of course it does. If there is no movement between the fields then there will be no combing and no need for deinterlacing.

[Katie Boundary]

Of course it does. If there is no movement between the fields then there will be no combing and no need for deinterlacing.

Unless it's around a scene change. Or it's a fade. Or there's literally any other change in the image that isn't caused by motion (someone turns a light on, for example).

[videoh]

Sure, those qualify as "motion between fields" too. But you made the false claim that motion has nothing to do with interlacing. What matters is whether the picture content changes across a field pair. Real motion is certainly a major instance thereof, and that explains why motion-adaptive methods can work better.

[LigH]

"Interlacing" means that two fields of a frame have been shot at different moments of time. If there is no motion, there is no difference to a progressive frame. "Combing" (while watching a whole frame) gets only visible for areas in motion because their content has different locations at different moments of time.

[StainlessS]

I've always been terrible in talking with women even when it was weather-related chit-chat, let alone about encoding xD

Me too, always seems like they would fail in a Turing Test :) :- https://en.wikipedia.org/wiki/Turing_test
EDIT: Or do I mean pass, or even Indeterminate.

[Katie Boundary]

Sure, those qualify as "motion between fields" too.

Linguists and physicists, and pretty much the entire english-speaking world, disagree with you.

But you made the false claim

There was nothing false about it. You can have motion without interlacing (it's called "progressive" video) and I already gave several examples of visible interlacing without motion.

What matters is whether the picture content changes across a field pair.

Agreed. And that's not the same thing as motion.

"Combing" (while watching a whole frame) gets only visible for areas in motion

Or if the picture changes for any other reason.

[Frank62]

Interlacing or even combing has (directly) nothing to do with motion.
The detection of interlacing and combing has.

[wonkey_monkey]

Linguists and physicists, and pretty much the entire english-speaking world, disagree with you.


This isn't a linguistic or physics forum. We have our own vernacular that doesn't always align with general usage, as does any field, and it's clear in this case that everyone except you is quite comfortable using the word "motion" to describe changes in pixel values. If you insist on being so pedantic (if that's even the right word when someone is incorrect), motion is at the very least one of the main causes of combing in interlaced video, so the claim that "motion has nothing to do with interlacing" is patently ridiculous in any case.

You might as well claim that there is no such thing as motion in a video at all, because it's just a collection of pixels changing over time.

[Katie Boundary]

The detection of interlacing and combing has.

Nope. Detection only requires that the two fields be from slightly different images.

[Frank62]

...the two fields be from slightly different images.

That exactly is how "motion" in connection with video is "defined", at least used: The two fields contain different images. Shot at a slightly different time. So what happened from the first to the second point in time is "motion" of some or all objects in the pictures.
If you can't accept this, w_m seems to be right with his last sentence. A question of terminology.

[StainlessS]

Shot at a slightly different time.
So progressive is temporally from the same position in time [excluding time taken for left to right, top to bottom, raster scan],
and interlaced fields are from temporally and also spatially different positions in time and space.
Katie should really teach herself to code, then she could better acquaint herself with the problem, that would be an education.
Miss Boundary is real exasperating, I bet its hell in their hourse deciding whats for supper.

[FranceBB]

Katie should really teach herself to code, then she could better acquaint herself with the problem, that would be an education.

Hear hear.
(Ok, I might have been watching the house of commons channel too much xD)

Jokes aside, she would grasp some of those things much much better if she was able to read and write code herself 'cause this way she would be able to actually test her ideas while right now she can only "trust" us with what we're telling her.


I bet its hell in their house deciding whats for supper.

ROTFL
Think about the chat with her husband, like:

Honey, how did your day go?

- Fine, I went to work, it was a normal day, I got stuck in traffic a bit on my way back home, but anyway... What about you?

Terrible, I came up with a revolutionary idea about duplicates detection, but everyone on Doom9 is saying it can't be used as it's not that simple... :(

[Katie Boundary]

That exactly is how "motion" in connection with video is "defined", at least used:

No. Motion means that a thing used to be in one place in space, and now it's in another.

Think about the chat with her husband,

I'm not likely to ever have a husband, given that I'm only attracted to women.

[FranceBB]

Motion means that a thing used to be in one place in space, and now it's in another.

Let's just stick with "difference" and we're good XD
No, jokes aside, we understand what you're saying, honestly, we do, but on the other hand it's just not as easy to detect differences (motion in your case) through an algorithm, but hey, if in the future someone will try to improve TDecimate and co. further he's gonna be more than welcome, but right now, if it doesn't fulfill your / our needs with any of the modes available, I guess we're just gonna have to live with it... :')

I'm not likely to ever have a husband, given that I'm only attracted to women.

Ah! Of course! Sorry, my fault, everything makes sense now eheheh.

(I'm joking of course :rolleyes:)

Ok, I should really get back to work now, I shouldn't check stuff on Doom9 while I'm at work, but I'm... like... addicted to this forum as I end up here most of the time anyway since I use Avisynth at work on a daily basis. :P

[Katie Boundary]

Let's just stick with "difference" and we're good XD
No, jokes aside, we understand what you're saying, honestly, we do, but on the other hand it's just not as easy to detect differences (motion in your case) through an algorithm,

It totally is. There's an unlimited number of algorithms that you can make based on the guidelines that I posted, but pretty much any algorithm based on those guidelines will be better than the options present in TFM.


Example algorithm 1:

Find average of A and E.
Find average of B and D.
If C is closer to the average of A and E than to the average of B and D, then treat C as interlaced.

Example algorithm 2:

m1 = |A-B|
m2 = |B-C|
m3 = |C-D|
m4 = |D-E|
n1 = |A-C|
n2 = |B-D|
n3 = |C-E|
If m1, m2, m3, and m4 are all bigger than n1, n2, and n3, then treat C as interlaced.

[Frank62]

No. Motion means that a thing used to be in one place in space, and now it's in another.

That's correct. But I wrote: "In connection with video", where your explanation/definition never happens. We are talking of pixels, bits, information, representations of pictures, not of "things".

[SaurusX]

Anyone ever run into this problem with TFM? The field matching fails to use the correct field combination and you end up with chunky aliased edges, particularly apparent in animation lines. It never happens if the source is completely stable, but if there's gate weave from a film source then hold on to your hat. You have to manually override with the correct field combo.

https://imgsli.com/NzA0MjU

This is a small-fry example. On occasion the entire frame will be huge chunky (but not combed) scanlines.

[VoodooFX]

This is a small-fry example.

Share a short snippet of original video and your script.

[SaurusX]

Share a short snippet of original video and your script.

Check out frame 32 and 33 in this clip using the following script.

MPEG2SOURCE("E:\Media\Video Editing Tools\MeGUI-2913-64\scripts\SwatKats\Tests\SwatKats Example.d2v")
TFM(slow=2,pp=4,mode=4)
TDecimate(mode=1)

https://mega.nz/file/qjpgXRqK#kS4NW-0oxPp_4qyearShAsfnTvHTuOPjYM7B-Y-mYmE

[VoodooFX]

Anyone ever run into this problem with TFM?
Many times, especially on animations like this.

Instead of TFM try this line (check if it fixes more than it destroys):
QTGMC(Preset="Slower").SelectOdd.TDecimate(mode=1).QTGMC( Preset="Medium", InputType=1, Sharpness=0.2)

[Katie Boundary]

I wrote: "In connection with video", where your explanation/definition never happens.

LOL, are you high? It happens all the time in video. It's the primary cause of differences between frames.

[Frank62]

Senseless. :rolleyes:

[markfilipak]

Katie Boundary! Hahaha... The joke's on you, folks. I visualize a T.Rex eating a guy who's sitting on a toilet as a comet streaks across the sky.

Edit: Oh, I see "Miss Boundary" has a T.Rex as an avatar -- that, after enlarging it to see it well. Well, "Miss Boundary", you need a better looking/better defined avatar. People are missing the joke.

[videoh]

I think you are the only guy that missed it.

[VoodooFX]

Unless it's around a scene change. Or it's a fade. Or there's literally any other change in the image that isn't caused by motion (someone turns a light on, for example).
But photons are moving from a bulb to a camera's lens...
A scene change and a fade... thermodynamics would tell that there is no motion, but quantum mechanics would disagree, I think. Anyway, a film editor moved his hand to make those effects, so literally they are caused by motion too. :p

[markfilipak]

I think you are the only guy that missed it.
I missed it? Hahaha... Missed what? I laughed my ass off. This whole thread segment with "Miss Boundary" is the silliest thing I've read in a long time. Thanks for the laughs.

[LigH]

Well, unfortunately it distracts from the topic. Much of the "discussion of basics of interlacing" deserves a separate thread. And a lot less personal insults. After all the time and number of involved people, it is now a bit hard to separate.

[kedautinh12]

Latest ver:
https://github.com/pinterf/TIVTC/releases

[Blankmedia]

From time to time I get this error with requestlinear:

"avs+ [FLAW]: RequestLinear: internal error (frame not cached)! occurred while reading frame 0"

And this is while running the same script, like I stopped the encode for whatever reason, start it back, get the error, flush intermediate files, get the error, reboot, get the error, type this message, no more error and encode is proceding (??).

Maybe I have trouble settings the parameters, so is there bigger documentation somewhere with requestlinear.

I run it because apparently there's a random bug in mvtools where it shoots a random frames from time to time.

This is the line I use :

RequestLinear(rlim=60, clim=100, elim=40)

it seem more stable than what I used before

RequestLinear(rlim=50, clim=100, elim=25)

Edit:

I run the script through x265 patman.

And even in avsmeter it started to fail. Avsmeter doesn't say anything, it just close quickly.

Edit 2:
it is not more stable.

I don't get it.


ClearAutoloadDirs()
#~ AddAutoloadDir("D:\A encoder\avs_simpsons_plugin")
AddAutoloadDir("D:\A encoder\simps_s02_60FPSv3\simps_s02_60FPSv3_plugin")
AddAutoloadDir("D:\A encoder\simps_s02_60FPSv3\simps_s02_60FPSv3_plugin\SMDegrain v3.5.3d")
SetMemoryMax(8192*3)
SetCacheMode(0)

chemin = "D:\A encoder\simpsons s02_enc\Simpsons_S02E01.d2v"

d2vsource(chemin, idct=5)


RequestLinear(rlim=50, clim=100, elim=25)
# RequestLinear(rlim=60, clim=100, elim=40)

RoboCrop(LeftAdd=2, TopAdd=0, RightAdd=0, BotAdd=0)
Convertbits(16)
QTGMCp( Preset="Very Slow", SourceMatch=3, Lossless=2, Tr2=3, Nnsize=3, NNeurons=3, EdiMode="BWDIF+NNEDI3").Selecteven()
TDecimate(Mode=1)

### Color Conversion ###
fmtc_bitdepth(bits=32, fulls=False, fulld=True)
fmtc_resample (css="444")
fmtc_matrix (mats="601", col_fam="RGB")
fmtc_transfer (transs="1886", transd="linear")
#~ fmtc_primaries (prims="470bg", primd="709")
fmtc_primaries (prims="601-525", primd="709")
fmtc_transfer (transs="linear", transd="1886")
fmtc_matrix (mat="709", col_fam="YUV")
fmtc_resample (css="420")
fmtc_bitdepth(bits=16, dmode=1, fulls=True, fulld=False)

### Adjust Color ###
MergeChroma(aWarpSharp2(Depth=14))


### Gibbs Noise Block ###
a=last
ex_boxblur(0,1,mode="mean")
turnright()
ex_median("verticalS")
turnleft()
ex_median("IQM")

# Valley Mask
edge=FlatMask(1, scale=10).removegrain(2)
Mask=ex_makediff(Edge.MT_Inflate().ex_Expand(4).ex_gaussianblur(6,UV=1),Edge.MT_Inflate(),dif=false).Blur(1.0) #.ex_gaussianblur(6,UV=2)

a.MT_Merge(a.ex_median("EMF").ex_median("EMF").ex_Minblur(1).ex_median("STWM"),Mask,Luma=True)

# ex_adddiff(mask.converttoyuv420(), dif=False)


### Derainbow ###

masklisse = ex_median("IQM").blur(0.25).Removegrain(17).Flatmask(lo=4,MSR=80,scale=8).ex_inflate(2).Removegrain(7).Removegrain(17) #.maa2().ex_median("STWM") #.ex_expand(1,"disk").ex_inpand(1,"disk")#.mt_binarize(threshold=25).ex_smooth(2,sharp=true)
masklisse = masklisse.ex_Expand(2, "disk").ex_median("STWM")
MT_Merge(last , ChubbyRain3().DFMDeRainbow(maskthresh=10, mc=False, use_Flux5framesT=False).ex_median("STWM"),masklisse,Luma=True)
#~ return ex_adddiff(masklisse.converttoyuv420(), dif=False)

### Overall Temporal Denoise ###
ConvertBits(bits=8)
#~ TemporalDegrain2(degrainTR=3, grainLevel=2, postFFT=5, postSigma=0.75, fftthreads=8,degrainPlane=0)
TemporalDegrain2(degrainTR=4, grainLevel=3, postFFT=5, postSigma=0.75, fftthreads=8,degrainPlane=0)
ConvertBits(bits=16)

### Resize ###
Crop(2,2,-2,-2)
deep_resize(960, 720,flat="nnedi3",grain=0,qual=1)

aWarpSharp2(type=1,Depth=4)
Sharpen(0.2)

### Darken-Thin Lines ###

#~ F=last
#~ S=F.FastLineDarkenMod(Strength=24,Prot=6).aWarpSharp2(Blur=4,Type=1,Depth=9,Chroma=2)
#~ D=ex_MakeDiff(S,F)
#~ ex_adddiff(Last,D,Dif=True,UV=2)

F=last
S=F.FastLineDarkenMod4(Strength=72,Prot=36,thinning=36).halobuster(thr=2, elast=2).aWarpSharp2(Blur=4,Type=1,Depth=4,Chroma=1)
D=ex_MakeDiff(S,F,UV=1)
ex_adddiff(Last,D,Dif=True,UV=2)

# F=last
# S=F.FastLineDarkenMod4(Strength=72,Prot=36,thinning=36).halobuster(thr=2, elast=2).aWarpSharp2(Blur=4,Type=1,Depth=4,Chroma=1).DRK(10)
# D=ex_MakeDiff(S,F,UV=1)
# ex_adddiff(Last,D,Dif=True,UV=2)

### Resize ###
deep_resize(1440, 1080,flat="nnedi3",grain=0,qual=1)

# aWarpSharp2(Blur=16,Type=1,Depth=3,Chroma=4)

# LSFplus(preset="slow", strength=50, smode=5, edgemode=0, preblur="""ex_median("IQM")""")
LSFplus(preset="slow", strength=150, smode=5, edgemode=0, preblur="""ex_median("IQM")""")

### Deband ###
# GradFun3plus(ThR=0.45,SMode=2) # 0.45 ep 1 à 6
# Merge(GrainFactory3mod(),0.225) # 0.225 ep 1 à 6

GradFun3plus(ThR=0.55,SMode=2) # 0.45 ep 1 à 6
Merge(GrainFactory3mod(),0.3) # 0.225 ep 1 à 6



# trim(9000,12000)
# trim(9000,9200)
trim(9000,10000)
# Prefetch(12,8) # 1.491
# Prefetch(4,8) # 1.096
# Prefetch(8,8) # 1.254
# Prefetch(12,12) # 1.557
Prefetch(2,8) #


Edit 3:

I have now split the code in 2 pass, just after the Gibbs noise block.

I can run 5 encode simultaneously without problems. At least in some tests. I will let it run over night to see what happens.

Yesterday I ran 3 encode at same time in one script and I had no problem. ¯\_(ツ)_/¯

I someone have input as to why it could happen I'm open.

Also I know the script seems intense, and it is. I'm open about it. But they get pretty clean. At least cleaner than Disney+

[StainlessS]

Sorry for late answer, I did not see your post, not been about much lately.

I think requestLinear would be best placed AFTER RoboCrop(),
you probably get the frame not cached thingy because RoboCrop() by default samples 40 frames, spread evenly throughout the source clip,
and requestLinear might need to cache entire clip for it to work correctly.
After its pre-scan, RoboCrop delivers cropped frames as per normal.

EDIT:
I did experience some problems with DGSource some months ago [dont know if same prob in current DGSource, updated several times since],
where RoboCrop failed to find good border, and I examined RoboCrop() Debug Info output to DebugView to view 1st frame which showed no border at all.
I then inserted "Return Last" immediately after DGSource [ie no RoboCrop], reloaded clip into VDub2, and jumped to that no-border frame.
Was a full-frame multi-colored mess due to DGS not decoding frame properley at that frame entry point. Repeat with jump to 10 or so frames before the offending frame,
then stepping forward, shows that there is no decode problem, only when decode from problem frame, ie not a source clip problem, is DGS x264 decode problem [was in MKV].

Maybe its fixed now in DGS [EDIT: DGS == DGSource, DGIndexNV source filter], I dont know.

Is it similar problem that you are having, I think I just did manual crop on clips exibiting that [fairly uncommon, maybe 1 in 20 or 30 clips] prob.

EDIT: You might be able to encourage frame decode failure [for experiment] by changing RoboCrop(SAMPLES=Src.FrameCount/50) [about 2%] on a clip of
200,000 frames or so. Maybe prob happens only on clips encoded by a certain encoder, or on all clips, dont know.

EDIT: eg

DGSource("D:\Source.dgi")
FC = FrameCount
PERCENT = 2.0 / 100.0 # 2.0 %
SAMPLES = Round(FC*PERCENT)
THRESH = -40 # -40 Default with AutoThresh Prescan, use eg THRESH=56 to set Explicit Threshold without AutoThresh prescan before border detect scan [ie faster but not as good if dodgy levels clip].
RT_DebugF("FRAMECOUNT=%d %.3f%% SAMPLES=%d THRESH=%f",FC,PERCENT,SAMPLES,THRESH)
Return RoboCrop(Samples=SAMPLES,Thresh=THRESH,Debug=True)


DebugVew

00006340 0.48317650 RT_DebugF: FRAMECOUNT=150401 0.020% SAMPLES=3008 THRESH=-40.000000

00006342 0.54735380 RoboCrop: v1.13 - 07 Feb 2020 - By StainlessS
00006343 0.54740667 Input: Width=1920 Height=1080 FrameCount=150401
00006345 0.54753429 Thresh set to user supplied -40.00
00006346 0.54755449 Thresh -40.00 recognised as DEFAULT AUTOTHRESH
00006347 0.54757363 Potential Auto Credits Skip Start@5%=7520 End @90%=135360 : Range=127841
00006348 0.54759288 Skip Start(7520) and End(135360) : Range=127841
00006350 0.54766637 Samples=3008 Thresh=-40.000 Laced=True Matrix=3
00006351 0.54774839 WMod=4 HMod=4 {Colorspace/Laced Restricted XMod=2 YMod=4}
00006352 0.54783821 RLBT=15 Ignore=0.400 Baffle=4
00006353 0.54789507 ScaleAutoThreshRGB=Yes ScaleAutoThreshYUV=No
00006354 0.54799873 CropMode=2(CropMore) Blank=No Align=Yes Show=No Log=No
00006355 0.54801899 Atm=4.000 SampStart=7520 SampEnd=135360 SampRange=127841
00006356 0.54803818 LeftAdd=0 TopAdd=0 RightAdd=0 BotAdd=0
00006357 0.54805768 LeftSkip=0 TopSkip=0 RightSkip=0 BotSkip=0
00006358 0.54807669 ScanPerc=49.000000
00006359 0.54809052 Prefix='ROBOCROP_'
00006361 0.62823510 AutoThresh 1) [ 7520] YPlaneMin = 16 (16)
00006362 0.69479620 AutoThresh 2) [ 7563] YPlaneMin = 16 (16)
00006363 0.75452262 AutoThresh 3) [ 7605] YPlaneMin = 16 (16)

# ...

00009366 126.16909790 AutoThresh 3006) [135275] YPlaneMin = 16 (14)
00009367 126.23580170 AutoThresh 3007) [135317] YPlaneMin = 16 (14)
00009368 126.29372406 AutoThresh 3008) [135360] YPlaneMin = 16 (14)

# ...

00009370 126.29382324 YPlaneMin=14 : Automatic set Thresh=54.000
00002917 122.82308197 Top 1) [ 7520] AveY= 92.22 Y1= 20
00009373 126.36152649 Bot 1) [ 7520] AveY= 81.78 Y2=1059
00009374 126.36154938 Lft 1) [ 7520] AveY= 90.43 X1= 1
00009375 126.36158752 Rgt 1) [ 7520] AveY= 70.37 X2=1919
00009376 130.46037292 Lft 100) [ 11729] AveY= 77.79 X1= 0
00009378 249.04609680 Active Frames (where image coords first found) = 2
00009379 249.04611206 Sampled ImageEdge: X1=0 Y1=20 X2=1919(W=1920,0) Y2=1059(H=1040,-20)
00009381 249.04624939 Crop(ModW/H): X = 0 Y = 20 W =1920(0) H =1040(-20)
00009382 249.04632568 Input: Width=1920 Height=1080
00009383 249.04640198 Crop(0,20,1920,1040)
00009384 249.04647827 Crop(0,20,0,-20)
00009385 249.04650879 Constructor Scan Total Time=248.555 secs

Border Edge
Sample Number
Frame Number
Scanline_H/V Luma Y
Coord
Above, Top, Bot, and Right borders found on first sample frame, but Left edge not found until 100'th sample,
prior to 100'th sample, best Left border was at X coord 1, at 100'th sample Left X coord found to be 0.
The test clip above did NOT reveal any bad decode in current DGS.

[Blankmedia]

Sorry for late answer, I did not see your post, not been about much lately.

I think requestLinear would be best placed AFTER RoboCrop(),
you probably get the frame not cached thingy because RoboCrop() by default samples 40 frames, spread evenly throughout the source clip,
and requestLinear might need to cache entire clip for it to work correctly.
After its pre-scan, RoboCrop delivers cropped frames as per normal.

EDIT:
I did experience some problems with DGSource some months ago [...]

Thanks, I took your advice, I even pushed robocrop in a second ,single threaded to be sure, script (now 3 part).

I swish d2vsource to lsmash to see if the problem repeated, and it did.

So Robocrop is alright! (I think)


ClearAutoloadDirs()
#~ AddAutoloadDir("D:\A encoder\avs_simpsons_plugin")
AddAutoloadDir("D:\A encoder\simpsons s02_enc\plugins")
AddAutoloadDir("D:\A encoder\simpsons s02_enc\plugins\SMDegrain v3.5.5d")
# SetMemoryMax(8192*2)
SetMemoryMax(12288)
SetCacheMode(0)

# chemin = "D:\A encoder\simps_s02_60FPSv2\e01\Simpsons_S02E01.d2v"
# chemin = "__CHEMIND2V__"

# d2vsource(chemin, idct=5)

chemin = "__CHEMIN__"
LSMASHSource_LWLibavVideoSource(source=chemin)

# RequestLinear(clim=100)
# RequestLinear(rlim=50, clim=100, elim=25)
# RequestLinear(rlim=60, clim=100, elim=40)
# RequestLinear(rlim=60, clim=200, elim=32)
RequestLinear(rlim=60, clim=30, elim=8)

Convertbits(16)
# QTGMCp(Preset="Very Slow", Nnsize=3, NNeurons=3, EdiMode="BWDIF+NNEDI3", TR2=3, FPSDivisor=2, ShutterBlur=2, ShutterAngleSrc=180, ShutterAngleOut=180, SBlurLimit=8, SourceMatch = 3, Lossless=2)
QTGMC(Preset="Very Slow", Nnsize=3, NNeurons=3, EdiMode="BWDIF+NNEDI3", TR2=3, FPSDivisor=2, ShutterBlur=2, ShutterAngleSrc=180, ShutterAngleOut=180, SBlurLimit=8, SourceMatch = 3, Lossless=2)

TDecimate(Mode=1)
santiag().yahr3()

### Derainbow ###
masklisse = ex_median("IQM").blur(0.25).Removegrain(17).Flatmask(lo=4,MSR=80,scale=8).ex_inflate(2).Removegrain(7).Removegrain(17) #.maa2().ex_median("STWM") #.ex_expand(1,"disk").ex_inpand(1,"disk")#.mt_binarize(threshold=25).ex_smooth(2,sharp=true)
masklisse = masklisse.ex_Expand(2, "disk").ex_median("STWM")
# ~ return ex_adddiff(masklisse.converttoyuv420(), dif=False)

MT_Merge(last , ChubbyRain3(),masklisse,Luma=True)
MT_Merge(last , DFMDeRainbow(maskthresh=10, mc=False, use_Flux5framesT=False),masklisse,Luma=True)


### Adjust Color ###
MergeChroma(aWarpSharp2(Depth=10,chroma=6))



# debut = round(59.940/23.976*9000)
# fin = round(59.940/23.976*12000)
# debut = round(29.970/23.976*9000)
# fin = round(29.970/23.976*12000)
# debut = round(9000)
# fin = round(12000)
# trim(debut, fin)
# trim(0,200)
# trim(0,1000)
# Prefetch(12,8) #
# Prefetch(4,8) # 2.4
# Prefetch(8,8) # 2.353
# Prefetch(4,12) # 2.197
Prefetch(2,8) # 2.146, 2.307 lsmash


The error showed up at somepoint, I think there's something in QTGMCp trying to suck memory and maybe accessing frames in some weird place. I don't know anything so people shouldn't rely on this. But switching to QTGMC removed the problem and memory usage is way lower.

Speed was better at first with QTGMCp, but it fills the memory and it gets lower speed over time.

I'll ask about this in Dogway's thread, but first I'll make some tests to be sure and better orient the discussion.

[StainlessS]

EDIT: Current Pinterf mod TIVTC, Both x64 & x86 versions.

Presume this is some kind of diff overflow bug in FieldDiff(),


#WhateverSource(...)
OPT=4 # (default) Auto
#OPT=0 # Use C routines
FieldDiff(debug=true,display=true,opt=OPT)
return last

Similar to both display and DebugView, thoughout video clip.


OPT=4 (Default)

00007315 3.68521953 FieldDiff: Frame = 14233 Diff = -207589436161 (sad)
00007316 3.73933315 FieldDiff: Frame = 14234 Diff = -207589439628 (sad)
00007317 3.76845598 FieldDiff: Frame = 14235 Diff = -207589438673 (sad)
00007318 3.81965804 FieldDiff: Frame = 14236 Diff = -207589375022 (sad)
00007319 3.85039353 FieldDiff: Frame = 14237 Diff = -207589400222 (sad)
00007320 3.88973689 FieldDiff: Frame = 14238 Diff = -207589383803 (sad)
00007321 3.93053317 FieldDiff: Frame = 14239 Diff = -207589353349 (sad)
00007322 3.97270870 FieldDiff: Frame = 14240 Diff = -207589428124 (sad)
00007323 4.05432177 FieldDiff: Frame = 14241 Diff = -207589397993 (sad)
00007324 4.05959177 FieldDiff: Frame = 14242 Diff = -207589429589 (sad)
00007325 4.08584976 FieldDiff: Frame = 14243 Diff = -207589433196 (sad)


OPT=0 # C

00000093 12.88996887 FieldDiff: Frame = 14233 Diff = -412316210604 (sad)
00000094 12.89195728 FieldDiff: Frame = 14234 Diff = -412316214071 (sad)
00000095 12.89433861 FieldDiff: Frame = 14235 Diff = -412316213116 (sad)
00000096 12.89671803 FieldDiff: Frame = 14236 Diff = -412316149465 (sad)
00000097 12.89924335 FieldDiff: Frame = 14237 Diff = -412316174664 (sad)
00000098 12.90135288 FieldDiff: Frame = 14238 Diff = -412316158245 (sad)
00000099 12.90339375 FieldDiff: Frame = 14239 Diff = -412316127792 (sad)
00000100 12.90548992 FieldDiff: Frame = 14240 Diff = -412316202567 (sad)
00000101 12.90757942 FieldDiff: Frame = 14241 Diff = -412316172435 (sad)
00000102 12.90983963 FieldDiff: Frame = 14242 Diff = -412316204032 (sad)
00000103 12.91174793 FieldDiff: Frame = 14243 Diff = -412316207639 (sad)

Above SAD.
[sse=true, = similar results (sum of squared errors, I think)].
Also different result for auto or C.

EDIT: PAL DVD source.
720 * 576 * 255 = only 105,753,600.

EDIT: Perhaps not limited to 0, after subtract of noise threshold. [avoid -ve addition to unsigned SAD]
Something like
Abs(a-b) - nt instead of Max(Abs(a-b) - nt, 0), just guessing.

FieldDiff(int nt, bool chroma, bool display, bool debug, bool sse, int opt)

[kedautinh12]

EDIT: Current Pinterf mod TIVTC, Both x64 & x86 versions.

Presume this is some kind of diff overflow bug in FieldDiff(),


#WhateverSource(...)
OPT=4 # (default) Auto
#OPT=0 # Use C routines
FieldDiff(debug=true,display=true,opt=OPT)
return last

Similar to both display and DebugView, thoughout video clip.


OPT=4 (Default)

00007315 3.68521953 FieldDiff: Frame = 14233 Diff = -207589436161 (sad)
00007316 3.73933315 FieldDiff: Frame = 14234 Diff = -207589439628 (sad)
00007317 3.76845598 FieldDiff: Frame = 14235 Diff = -207589438673 (sad)
00007318 3.81965804 FieldDiff: Frame = 14236 Diff = -207589375022 (sad)
00007319 3.85039353 FieldDiff: Frame = 14237 Diff = -207589400222 (sad)
00007320 3.88973689 FieldDiff: Frame = 14238 Diff = -207589383803 (sad)
00007321 3.93053317 FieldDiff: Frame = 14239 Diff = -207589353349 (sad)
00007322 3.97270870 FieldDiff: Frame = 14240 Diff = -207589428124 (sad)
00007323 4.05432177 FieldDiff: Frame = 14241 Diff = -207589397993 (sad)
00007324 4.05959177 FieldDiff: Frame = 14242 Diff = -207589429589 (sad)
00007325 4.08584976 FieldDiff: Frame = 14243 Diff = -207589433196 (sad)


OPT=0 # C

00000093 12.88996887 FieldDiff: Frame = 14233 Diff = -412316210604 (sad)
00000094 12.89195728 FieldDiff: Frame = 14234 Diff = -412316214071 (sad)
00000095 12.89433861 FieldDiff: Frame = 14235 Diff = -412316213116 (sad)
00000096 12.89671803 FieldDiff: Frame = 14236 Diff = -412316149465 (sad)
00000097 12.89924335 FieldDiff: Frame = 14237 Diff = -412316174664 (sad)
00000098 12.90135288 FieldDiff: Frame = 14238 Diff = -412316158245 (sad)
00000099 12.90339375 FieldDiff: Frame = 14239 Diff = -412316127792 (sad)
00000100 12.90548992 FieldDiff: Frame = 14240 Diff = -412316202567 (sad)
00000101 12.90757942 FieldDiff: Frame = 14241 Diff = -412316172435 (sad)
00000102 12.90983963 FieldDiff: Frame = 14242 Diff = -412316204032 (sad)
00000103 12.91174793 FieldDiff: Frame = 14243 Diff = -412316207639 (sad)

Above SAD.
[sse=true, = similar results (sum of squared errors, I think)].
Also different result for auto or C.

EDIT: PAL DVD source.
720 * 576 * 255 = only 105,753,600.

EDIT: Perhaps not limited to 0, after subtract of noise threshold. [avoid -ve addition to SAD]
Something like
Abs(a-b) - nt instead of Max(Abs(a-b) - nt, 0), just guessing.

FieldDiff(int nt, bool chroma, bool display, bool debug, bool sse, int opt)


Can you report to him here?? Cause he access to doom9 rarely

[flossy_cake]

Hello, I have some 30p streams which contain a mix of 1:1 (30fps) and 1:1:1:2 (24fps) cadences inside them. Is it possible to use TDecimate to convert these into a 60p stream containing 2:2 (30fps) and 3:2 (24fps)?

The closest I can get is 2:2 and 3:3:2:2 with this bodge:


ChangeFPS(120000,1001)
TDecimate(mode=1,hybrid=0,cycle=4,cycleR=2,vidThresh=1.1,display=true)


Test clip: https://drive.google.com/file/d/10pqePJ3oYjwDmKES233k7Il61sufzIy-/view

The first 8 seconds are 1:1:1:2 followed by 5 seconds of 1:1. If you are using MPC-HC you can step through each frame with ctrl+arrows to count the cadence manually.

[Katie Boundary]

Hello, I have some 30p streams which contain a mix of 1:1 (30fps) and 1:1:1:2 (24fps) cadences inside them. Is it possible to use TDecimate to convert these into a 60p stream

TDecimate is for reducing your frame rate, not increasing it.

[StainlessS]

not increasing it
I think he/she knows that.
Is increasing to 119.880 FPS [NTSC weird 120fps], then decimating to NTSC 60fps.
[intention to get smoother playing result without losing any source frames, I think, something like that.]

His/her other thread:- https://forum.doom9.org/showthread.php?t=184448

[hello_hello]

Hello, I have some 30p streams which contain a mix of 1:1 (30fps) and 1:1:1:2 (24fps) cadences inside them. Is it possible to use TDecimate to convert these into a 60p stream containing 2:2 (30fps) and 3:2 (24fps)?

Are there a lot of video sections? I ask, because the only way I could get the 3:2 cadence was to do it the hard way (samples downscaled to reduce the file size).

Trim(0,274).TDecimate().ChangeFPS(60000,1001) ++ \
Trim(275,0).ChangeFPS(60000,1001)

CFR.mkv (https://files.videohelp.com/u/210984/CFR.mkv)

Variable frame rate easier?

# Analysis pass (comment out any filters after TDecimate so they don't run)
# TFM(Output=FTFM()).TDecimate(Mode=4, Hybrid=2, Output=FTDec())

# Encoding pass (uncomment any filters after TDecimate)
TFM(Input=FTFM()).TDecimate(Mode=5, Hybrid=2, Input=FTDec(), TFMIn=FTFM(), mkvOut=FTC())

Sample x264 command line:
x264.exe --level 4.1 --preset slow --tune film --crf 18.0 --tcfile-in "D:\VFR_TIMES.txt" --output "D:\VFR.mkv" "D:\VFR.avs"

VFR.mkv (https://files.videohelp.com/u/210984/VFR.mkv)

Functions for automatic file name creation based on the script name.
Creates files in the script folder. If no script exists (ie before saving a script when using AVSPMod) they're configured to save to the root of drive D, but without using the script name (obviously).

# ===============================================================================

function FTFM() {

Try{
ScriptName = ScriptFile()
EndLength = FindStr(LCase(ScriptName), ".avs") - 1
FileName = ((EndLength > 0) ? LeftStr(ScriptName, EndLength) : ScriptName) + "_TFM.txt" }
catch(err){ FileName = "D:\_TFM.txt" }

return FileName }

# -------------------------------------------------------------------------------

function FTDec() {

Try{
ScriptName = ScriptFile()
EndLength = FindStr(LCase(ScriptName), ".avs") - 1
FileName = ((EndLength > 0) ? LeftStr(ScriptName, EndLength) : ScriptName) + "_TDecimate.txt" }
catch(err){ FileName = "D:\_TDecimate.txt" }

return FileName }

# -------------------------------------------------------------------------------

function FTC() {

Try{
ScriptName = ScriptFile()
EndLength = FindStr(LCase(ScriptName), ".avs") - 1
FileName = ((EndLength > 0) ? LeftStr(ScriptName, EndLength) : ScriptName) + "_TIMES.txt" }
catch(err){ FileName = "D:\_TIMES.txt" }

return FileName }

# ===============================================================================

[flossy_cake]

Thank you, the result of VFR.mkv is perfect :)

But is there a reason why TFM() needs to be used if it's not interlaced?

[hello_hello]

At one stage TDecimate required a TFM metrics file for mode=5, but the help file now says:

As of version 0.9.11.8 of TDecimate, an input and tfmIn file are no longer
required for mode 5. If an input file is not specified then a value of
ULLONG_MAX-1 (the maximum value tdecimate has) is assigned to all metrics.
Likewise, if a tfmIn file is not specified then all matches are assigned as 'c',
not combed, and not a d2v duplicate.

I'm not sure I fully understand that though. Including TFM shouldn't do any harm.