Moire Effect & TFM(pp=0) Solution

[Danette]

I've seen quite a few DVDs where the aspect ratio for the opening credits was wrong. Like these. I found them on the internet and don't know where they came from, but I've seen the poor MGM lion being tortured on DVDs in the past. These are both 1080p so it's likely they weren't resized.

Yes, Leo looks pretty bad in that one link. I uploaded a segment from the credits having the circle - in case you’re curious. Note, too, that the credits are sandwiched between an intro to the story and the main story. I can’t detect any adjustment vs. the main video.

circle segment: https://www.mediafire.com/file/w0p0dqdd8d3qtfz/Circle_Example.m2v/file

Not that I know of. To the best of my knowledge InputType=1 doesn't de-interlace, but QTGMC still does everything else it'd do in order to repair a de-interlaced picture. That's why it can be great at stabilising, but I guess this is one example of when that has a negative effect.

I used to think this, as well. However, I’ve now tested a few episodes of the next two TV series I intend to backup, and they have similar moire problems, while demonstrating no interlacing (IVTC only) according to the test I’m now using (mentioned in post #45). After processing with your suggestion of “TFM(y0=140,y1=480,pp=0,micmatching=0,slow=2)” they are good with no moire (many thanks for that). Unfortunately, in each case, when I add QTGMC(InputType=1) after TFM, the moire is reintroduced. It appears that QTGMC falls down where moire effects are involved and I suspect that moire problems will exist in all TV series at some point or another. It leaves me wondering if I have to abandon QTGMC, unless I want to analyze every single future show I intend backing up ...ugh!!

Maybe I'll post the issue on the QTGMC thread.

[Katie Boundary]

Wow. I never expected one of my crazy experiments to find a practical use. It probably wouldn't have even occurred to me to try using Ghetto IVTC on this content. I'm happy to be of help, I guess :)

and since it black and white film, it's better and faster to use Converttoy8/Converttoy

also I already made Katie method as function KIVTC() in AnimeIVTC.avsi if someone care

While I'm honored that you would include one of my experiments in your filter package, I must object to the name. KIVTC, which I assume stands for Katie's IVTC, implies that this is my preferred method of IVTC, which is definitely NOT the case, and I don't want anyone to get the impression that that's the case. A more appropriate name would be GIVTC (Ghetto IVTC).

[StainlessS]

implies that this is my preferred method of IVTC
No, just implies that it was your idea, no more, no less. [dont lump extra work onto Real.Finder]

[hello_hello]

Yes, Leo looks pretty bad in that one link. I uploaded a segment from the credits having the circle - in case you’re curious. Note, too, that the credits are sandwiched between an intro to the story and the main story. I can’t detect any adjustment vs. the main video.

circle segment: https://www.mediafire.com/file/w0p0dqdd8d3qtfz/Circle_Example.m2v/file

Edited to delete this part of my reply. See the next post for an updated version.

It appears that QTGMC falls down where moire effects are involved and I suspect that moire problems will exist in all TV series at some point or another. It leaves me wondering if I have to abandon QTGMC, unless I want to analyze every single future show I intend backing up ...ugh!!

Maybe I'll post the issue on the QTGMC thread.

That sort of thing is pretty rare. QTGMC tries to repair each scan-line (normally after de-interlacing) and I guess it's causing a problem because the lines on his jacket are so fine.

Even though progressive mode is intended for repairing bad de-interlacing, on progressive content it can sometimes affect straight lines in a way that looks like bad de-interlacing, if straight lines happen to be on just the right angle.

Here's an example of what I mean. It's not severe (unless you have a huge monitor you might need to run it fullscreen to see it), but I remember it from an encode a few days ago. It's a 1080p source resized to 720p. Watch the edge of the books on the far side of the desk. The problem already exists in the source to some extent, but you probably wouldn't notice unless you knew to look for it. QTGMC just made it worse. The slower speed preset tends to cause less of that sort of thing, although it doesn't happen all that often anyway. There's a re-encode without QTGMC included.
A similar thing can happen when straight edges are almost perfectly horizontal and there's a lot of noise dancing around the edges.

QTGMC.zip (https://www.udrop.com/1DE4/QTGMC.zip) (4.7MB)

If there's halos around objects in a source, especially if it's also noisy, QTGMC tends to enhance them a bit more than I'd like.

On rare occasions it can cause motion interpolation type artefacts when an object moves across an almost flat background. For example, a person walking on grass and all you can see past that person is more grass. As he moves, sometimes QTGMC can cause the grass immediately behind him to blur. That's quite rare though.

Despite those negatives, I still use QTGMC for denoising more than any other denoising method, especially for standard definition requiring some "stabilisation".

[hello_hello]

Well.... at the risk of it causing a brain haemorrhage, I might have to consider the possibility I was wrong. I'd prefer to live in denial and go with the credits being a different aspect ratio to the episodes themselves, but I tried drawing squares around circles as Irfanview displays the aspect ratio in the title bar. Based on that, it does seem to be 4:3. At least for the opening credits.... :)

4:3
https://i.postimg.cc/x8RfP05s/4-3-B.png

15:11
https://i.postimg.cc/Vv1sMSqr/15-11-B.png

For this one, I'll confess it's hard enough to guess where the edges are to be certain if 15:11 is the correct one because it's correct, or if it's the correct one because I want it to be.

4:3
https://i.postimg.cc/NGrh18bW/4-3-C.png

15:11
https://i.postimg.cc/JzpdwtPN/15-11-C.png

[Danette]

That sort of thing is pretty rare. QTGMC tries to repair each scan-line (normally after de-interlacing) and I guess it's causing a problem because the lines on his jacket are so fine.

Even though progressive mode is intended for repairing bad de-interlacing, on progressive content it can sometimes affect straight lines in a way that looks like bad de-interlacing, if straight lines happen to be on just the right angle.

Despite those negatives, I still use QTGMC for denoising more than any other denoising method, especially for standard definition requiring some "stabilisation".

I think that the rarity of the problem is a function of the type of videos we prefer to watch. I am nearly 100% shows from the 50’s through early 80’s, so my DVD’s are nearly all NTSC and 4:3 AR. IVTC is mainly all that is needed (rarely interlaced). I have tested 5 different shows now and found the moire in 4 of them. I think that B&W sees a jump in frequency of moire effects, although one was in color.

In the past, I have seen a noticeable difference when QTGMC(InputType=1) is employed on many of my backups. Although the difference in using QTGMC for it’s beneficial cleaning/stabilizing can be subtle, the moire effect is not, so I am going to have to start using QTGMC only when I am sure that there is little chance of it inducing a moire effect …and that’s unfortunate.

Well.... at the risk of it causing a brain haemorrhage, I might have to consider the possibility I was wrong.

For this one, I'll confess it's hard enough to guess where the edges are to be certain if 15:11 is the correct one because it's correct, or if it's the correct one because I want it to be.

Alas, I must advise you that you were right. I never thought to use my old image editor until I saw what you did with IrfanView. With my editor, I can create circles, squares, etc. and, as you can see below, the cropped version is the correct version. However, I'm going to take a little credit by saying that the left border should cropped by 6 and not 8. It may be hard to see, but the dashed blue lines are the drawn circle.

https://i.postimg.cc/dZMDyL2V/AR-Test-1.jpg (https://postimg.cc/dZMDyL2V)
https://i.postimg.cc/zHnvS87S/AR-Test-2.jpg (https://postimg.cc/zHnvS87S)

[hello_hello]

Now I'm confused. Is that the same episode as the sample you uploaded? Because I was happy to concede the intro is 4:3.

However, I'm going to take a little credit by saying that the left border should cropped by 6 and not 8. It may be hard to see, but the dashed blue lines are the drawn circle.

You'd probably hate the way I'd encode the episodes with the vertical lines on the left side. If I was to encode those lines I wouldn't be able to sleep at night. There's a plugin called EdgeFixer you might want to try. It should be able to make them less noticeable, usually at the expense of blurring the edge a little. Me.... I'd just remove them entirely, so instead of this:

Crop(6,0,-8,0)
Lanczos4Resize(640,480)

https://i.ibb.co/tLJVkQC/1e.png

I'd be doing something like:
CropResize(640,480, 18,0,-12,0, InDAR=15.0/11.0)
(the script will crop the appropriate amount from the top and bottom to prevent aspect error)

https://i.ibb.co/JBCFKm2/1f.png

Or maybe even this to keep the height resizing to a minimum:
CropResize(624,468, 18,0,-12,0, InDAR=15.0/11.0)

https://i.ibb.co/tbQg1CL/1g.png


Anyway...
I found a copy of the first episode and had a play. I think for me, the result was the same as before. It was already cropped, but the aspect ratio of the opening credits seems wider (the square is 1.027:1). On the other hand, I'm pretty sure the shot of the radar screen about 4 minutes in is correct, assuming it's supposed to be round.

https://i.ibb.co/51FnLrT/1b.png

https://i.ibb.co/C6c7kSx/1a.png

I couldn't find any other shots of circles that were even close to straight-on, but I let my brain have a go at the angled shot of the car wheels a little after 19 minutes, and it seems happier with 15:11.
I only added the borders to simulate no-crop resizing by reducing the width in MPC-HC, as a single resizing step is very close to the difference between a generic and mpeg4 aspect ratio. It's not exact, but it is very close, and good enough for a comparison.

https://i.ibb.co/Dzv02m5/1d.png

https://i.ibb.co/fXJQcjh/1c.png

If you can be bothered, try the first episode yourself to see if the result is the same, but for the first episode at least, the aspect ratio of the opening credits and the episode itself do seem a little different.

[Danette]

Now I'm confused. Is that the same episode as the sample you uploaded? Because I was happy to concede the intro is 4:3.

Yes, it was from the same episode and I did look at that first episode. Drawing a circle around many objects, in both episodes, leaves me believing that the cropping results in the correct AR after all. I didn’t use the radar screen because those were convex and the resultant shadow would throw it off. However, in that same scene the reel in the top right is one I used. There is also a globe in the episode that first appeared in this thread.

How would you compare the resize quality of CropResize to Lanczos4Resize?

You'd probably hate the way I'd encode the episodes with the vertical lines on the left side. If I was to encode those lines I wouldn't be able to sleep at night. There's a plugin called EdgeFixer you might want to try.

I would lose sleep if I cut the content at all, so I’m usually ok with minor lines such as that. However, I do like that EdgeFixer you suggested and have applied that.


Incidentally, I’ve found that modifying the TFM(y0=140,y1=480,pp=0,micmatching=0,slow=2) line to:
TFM(y0=160,y1=480,pp=5,micmatching=0,slow=2) gives better results and allows the use of QTGMC(InputType=1) without re-introducing moire effects.

Perhaps you can explain why that is, for educational purposes, and if you see any downside to it.

[poisondeathray]

Incidentally, I’ve found that modifying the TFM(y0=140,y1=480,pp=0,micmatching=0,slow=2) line to:
TFM(y0=160,y1=480,pp=5,micmatching=0,slow=2) gives better results and allows the use of QTGMC(InputType=1) without re-introducing moire effects.

Perhaps you can explain why that is, for educational purposes, and if you see any downside to it.

On that sample, you're blend deinterlacing almost every frame with those settings. It's blurry, that' s the downside. You can add display=true to TFM to see which frames are affected

It's ok to selectively low pass some signal if there are problems on certain parts of frames, but you're basically applying a blur() to every frame globally

[Danette]

On that sample, you're blend deinterlacing almost every frame with those settings. It's blurry, that' s the downside. You can add display=true to TFM to see which frames are affected

It's ok to selectively low pass some signal if there are problems on certain parts of frames, but you're basically applying a blur() to every frame globally

Yes, thanks, I can see that now on high zoom. It’s very subtle …almost a denoising effect. It doesn’t seem to be a bad thing. I can tone down denoising a little and, on my screens (TV and monitor), can’t detect it at a distance. Am I right that this slight blur is tricking QTGMC into not seeing what it thinks is combing?

I’ve been using ShowCombedTIVTC() to see where the nominal combing exists after processing and
A=BlankClip(Last,Color=color_red)
TFM(Clip2=A)
TDecimate()
ShowChannels()
to determine if I’m likely to need this moire suppression approach in the first place.

[poisondeathray]

Yes, thanks, I can see that now on high zoom. It’s very subtle …almost a denoising effect. It doesn’t seem to be a bad thing. I can tone down denoising a little and, on my screens (TV and monitor), can’t detect it at a distance. Am I right that this slight blur is tricking QTGMC into not seeing what it thinks is combing?


Yes, it's essentially a vertical blur. ie. blur(0,1) - and that makes it no longer detected as combing by QTGMC .

Pros/cons - can help with patterns prone to aliasing, but you soften everything else

If you use QTGMC in progressive mode, it smooths everything over, cleans it up, but at the expense of detail loss and possible motion artifacts .

Like everything, it's about personal taste.

[hello_hello]

How would you compare the resize quality of CropResize to Lanczos4Resize?

The default resizing method is Spline36Resize, but you can use any resizer with arguments for cropping. It doesn't have to be an Avisynth resizer.
CropResize(640,480, 8,0,-8,0, InDAR=15.0/11.0, Resizer="Lanczos4Resize")

There's a wrapper functions script included that lets you use another resizer as an alternative default. You have to modify the very last line in the CropResize script to specify the new default. That line looks like this:
function CR_Resizer() { return "Resize8" }
So you'd change it to:
function CR_Resizer() { return "Lanczos4Resize" }

Then you can use the alternative default resizer (Lanczos4Resize) by appending an X to the function name.
CropResizeX(640,480) or
CRX(640,480)

The default alternative is the Resize8 (http://avisynth.nl/index.php/Resize8) function. Resize8 corrects the very slight chroma shift (https://forum.doom9.org/showthread.php?p=1506374#post1506374) caused by the Avisynth resizers (not that it's anything to worry about unless you're resizing by a huge amount). The defaults for Resize8 are Lanczos4 for luma upscaling, Lanczos for chroma upscaling, and Spline36 for luma/chroma downscaling, however there's "Wrapper Functions" and "Resizer Functions" scripts supplied with CropResize that can be used to force a resizer for Resize8, but it's probably easier to use the AVSResize plugin (http://avisynth.nl/index.php/Avsresize). I don't think it's resizers have the chroma positioning bug.
CropResize(640,480, Resizer="z_Lanczos4")
Or just use AVSResize on it's own.

Are you glad you asked? :)

By the way, CropResize will always crop if necessary to prevent aspect error when resizing, so if you do this
CropResize(640,480, 6,0,-8,0, InDAR=15.0/11.0)
it'll crop an extra couple of pixels from the width to resize to 4:3 without aspect error. If you're really determined the video only needs 14 pixels cropped from the width for 4:3, you have to change the input display aspect ratio, or specify the appropriate sample/pixel aspect ratio instead.

640 / (720 - 14) = 0.90651558 - sample/pixel aspect ratio
720 * 0.90651558 = 652.691218
652.691218 / 480 = 1.35977 - display aspect ratio

CropResize(640,480, 6,0,-8,0, InDAR=1.35977) or
CropResize(640,480, 6,0,-8,0, InSAR=0.90651558)
Or if you want the cropping even, for an NTSC DVD this would crop 7 pixels each side. Six using Crop() and one with the resizer.
CropResize(640,480, InDAR=1.35977, Resizer="Lanczos4Resize")
It'd be the same as
Crop(6,0,-6,0)
Lanczos4Resize(640,480, 1,0,-1,0)

Or this would crop four from the left and ten from the right.
CropResize(640,480, 0,0,-3,0, InDAR=1.35977)
Once you've set the new InDAR or InSAR though, you can crop however you like and CropResize will still make sure there's no aspect error by cropping extra picture if necessary.

I would lose sleep if I cut the content at all, so I’m usually ok with minor lines such as that. However, I do like that EdgeFixer you suggested and have applied that.

DVDs are created to account for over-scanning (https://en.wikipedia.org/wiki/Overscan). That's why when you watch a DVD on a CRT, or even on a modern display set to 4:3 mode (which also over-scans) you don't see any of the crud. Even if it's all picture, you still don't see 5% to 10% of it (https://en.wikipedia.org/wiki/Safe_area_%28television%29#Action_safe_area), which is why cropping some of the the over-scan area doesn't bother me if it gives me clean edges. Even modern TVs probably mostly over-scan by default, although often you can disable it (except for analogue composite inputs and anything received over the airwaves).

[Danette]

Are you glad you asked? :)

YES!

By the way, CropResize will always crop if necessary to prevent aspect error when resizing

On this video, at least, when I set CropResize to (640,480,0,0,-0,0, InDAR=15.0/11.0) it performs an auto-cropping type function. Apparently, it agrees with us that the correct AR is delivered by cropping the borders, although it chewed into the content very slightly.

It may be that the best thing to do is to use it in this way and accept whatever borders may remain and/or whatever content may be cut.

Comments?

[hello_hello]

On this video, at least, when I set CropResize to (640,480,0,0,-0,0, InDAR=15.0/11.0) it performs an auto-cropping type function. Apparently, it agrees with us that the correct AR is delivered by cropping the borders, although it chewed into the content very slightly.

It may be that the best thing to do is to use it in this way and accept whatever borders may remain and/or whatever content may be cut.

Comments?

Edit: Sorry about the essay, but hopefully this will help makes things clearer. :)

The script simply uses the supplied InDAR to base the calculations on. If no InDAR/InSAR is specified, it assumes the source is non anamorphic (square pixels). If you were to do the following, it wouldn't crop at all as the InDAR and output DAR (resolution) are the same. Info=true will show you what it's doing.
CropResize(640,480, 0,0,0,0, InDAR=4.0/3.0, Info=true)
For the following it barely crops as the output dimensions almost match the InDAR.
CropResize(652,480, 0,0,0,0, InDAR=15.0/11.0, Info=true)
For the next example, the shape of objects in the picture will be exactly the same after resizing as for the previous one, but because the output width is narrower this time, it has to crop the sides.
CropResize(640,480, 0,0,0,0, InDAR=15.0/11.0, Info=true)
So it's not agreeing about the AR, or auto-cropping in the traditional sense, it's just preventing an aspect error based on the InDAR and output dimensions. It just happens that InDAR=15.0/11.0 requires 16 pixels to be cropped from the width if you want a 4:3 output. If you want to use that InDAR (which I think we decided is correct), but don't want to crop, you can increase the output width a bit.

Think of any specified cropping as the minimum cropping, but the script will increase it as required to prevent aspect error. To see what's being cropped, enable a cropping preview. The yellow lines are the specified cropping and any blue lines are the additional script cropping. The video isn't resized while the cropping preview is enabled.
CropResize(640,480, 10,4,-6,-4, InDAR=15.0/11.0, CPreview=1, Info=true)

For a 16:9 DVD the equivalent InDARs are (and specifying 16:9 output dimensions for resizing).
CropResize(768,432, 0,0,0,0, InDAR=16.0/9.0) - generic DAR
CropResize(768,432, 0,0,0,0, InDAR=20.0/11.0) - mpeg4 DAR

The main idea of CropResize is to crop and resize without having to calculate aspect error and adjust the cropping or resizing manually to prevent it. An example might be cropping a 1080p source and resizing to 720p. If you don't specify a height (use zero instead), the script will apply your cropping and then pick the appropriate height for you. Because it's free to pick the height, any additional cropping to prevent aspect error will be fairly minimal. You can do exactly the same thing with a DVD source as long as you use the correct InDAR
(cropping a 1080p source like this would mean you have a video with a 2.40:1 aspect ratio after cropping the black).
CropResize(1280,0, 4,140,-2,-142)
If you were to specify the same cropping as above with 4:3 output dimensions (still a 1080p source), it'd apply the specified cropping, then crop a huge amount from each side to give you a 4:3 output without distorting the picture.
CropResize(720,540, 4,140,-2,-142)

I find it very handy when working with video where the amount of black varies quite a bit (usually old DVDs), but I'm OCD about cropping it all. So I can do something like the following, specifying enough cropping to remove the black, without having to worry about distorting the picture, because CropResize won't let me.

Trim(0,1596).CropResize(640,480, 8,0,-8,0, InDAR=15.0/11.0) ++ \
Trim(1597,1884).CropResize(640,480, 16,2,-8,-4, InDAR=15.0/11.0) ++ \
Trim(1885,2597).CropResize(640,480, 24,4,-12,-2, InDAR=15.0/11.0) ++ \
Trim(2598,0).CropResize(640,480, 8,0,-8,0, InDAR=15.0/11.0)

But anyway... what I was trying to explain in my previous post, is if you were to decide the source is 4:3 after you only crop 14 pixels from the sides, you can do it by specifying the appropriate InDAR, and that's the DAR the script will base it's calculations on. Use Info=true and you'll see it's only cropping 7 pixels each side here instead of 8.
CropResize(640,480, InDAR=1.35977, Info=true)

CropResize does have a "traditional" auto-cropping option. It requires the AutoCrop plugin (autocrop.dll). When it's enabled, the source is auto-cropped with autocrop.dll, any specified cropping is applied to the auto-cropped video, if necessary the script applies additional cropping to prevent aspect error, then it resizes. Try the following. Assuming auto-crop crops some or all of the black from the sides, the script will then crop 8 more pixels each side, and to prevent aspect error it'll crop a few pixels top and bottom before resizing to 4:3 dimensions.
CropResize(640,480, 8,0,-8,0, AutoC=true, InDAR=15.0/11.0, Info=true)
If you don't specify a height, you probably won't end up with a 4:3 output, but at the same time the script probably won't have to crop much extra picture to prevent aspect error.
CropResize(640,0, 8,0,-8,0, AutoC=true, InDAR=15.0/11.0, Info=true)

[kuchikirukia]

I always suggest pp=0 because it's better to have combing which can be seen in spot checks and redone than artifacts which may be buried in one destroyed scene.

[real.finder]

so as proof of concept

bob(0,0.5).reduceflicker(2).Ablur(0,blurv=2).interlaced60or50(BFF=!(GetParity())).Blur(1.58,0)
tfm(pp=0, micmatching=0)


with this tfm seems dont matching the wrong fields, but we dont need the output to be blurry so https://github.com/pinterf/TIVTC/issues/27

[real.finder]

I think this fully fixed with
https://github.com/pinterf/TIVTC/issues/27#issuecomment-3532872619