Log in

View Full Version : Questions Regarding TFM False Detection, D2VSource/MPEG2Source, NTSC/PAL DVDs


StormMeows
19th April 2024, 23:44
Hi guys,

I am mainly working with NTSC/PAL DVDs and had some underlining questions that I hope you can answer.

Specifically, when I use TFM, I choose to use clip2=QTGMC() within it so that it uses the best deinterlacer possible. I also use DGIndex on most of the DVDs and put "d2v="source.dv2" inside of TFM so that way TFM uses the .d2v file created from DGindex to better field match.

With that being said, does it matter if I use D2VSource() or MPEG2Source() to index the .d2v file? I have read mixed reviews on using both. On one hand, D2VSource is newer and being updated constantly. I believe MPEG2Source is dead and no longer being updated/worked on. On both D2VSource's and MPEG2Sources's avisynth pages, they both support the use of using "d2v="source" within the TFM parameter. Is one of these plug-ins better to use in 2024 for MPEG DVDs specifically? I have seen a lot of you still using the old MPEG2Source and NOT the newer D2VSource. I have tried both and they don't seem any different for what I do. I use x64 for all of my avisynth plug-ins if that matters. Also, I will note, if I use MPEG2Source, and I try to do CPU2=, it says that CPU and CPU2 do not exist for MPEG2Source. This could be because I'm using all 64bit plug-ins/avisynth, I'm not quite sure.

The next big question, is there a basic way to quickly see when TFM is calling upon clip2= to deinterlace scenes throughout the entire movie? For instance, I currently use TFM(display=true) and then slowly scroll through the entire movie until I see "DE-INTERLACING" pop up on the display, but there is no way that I am going to be able to go slow enough to capture every frame that's being deinterlaced by TFM's clip2 this way. I figured there should be an easy way to analyze the video and say these specific frames have been deinterlaced, no? It would obviously be best to go frame by frame throughout the whole movie to see which scenes have been deinterlaced, but that would take several hours for one movie. I could also change the color of the clip2, but again I would have to find every frame with that specific color shown, in it. I have a lot of movies and shows to back up/encode, for my personal collection, so I need something quicker.

The reason I am going into so much depth about trying to find all frames that are being deinterlaced is because I have noticed, in a short time, that TFM really does not do a good job and it's constantly throwing false positives. For instance, it sees things like blinds, tube television presentations during a movie, jackets, etc as being combed, but they do not need deinterlaced at all. If I can find out each frame that it's detecting as combed, then I can adjust the thresholds accordingly, change the pp or MI value, etc. I have also noticed that DVDs that have a lot of artifacts/noise, TFM will falsely pick up as being combed, when it's certainly not. It also seems that on some movies, I have to use Vinverse instead of lowering the cthresh down because once you lower it, then other scenes are now getting detected since it's more sensitive.

My final question pertains to Field-blended NTSC and PAL movies. I have read that QTGMC() followed by srestore() is typically best for these. These seem to play choppy for me with that method. I have had the best playback experience with just simply QTGMC. This takes a NTSC fielded blended 29.97 FPS movie and turns it into a 59.97 FPS movie. I'm assuming this works better because when the studios do the field blending, there are now 60 frames, consisting of the ghosted frames and unique original frames? If I do that same NTSC movie as 23.976FPS (using srestore's frate=23.976) then the ghosted frames are much more apparent.

Sorry for the long rant. As I mentioned previously, I have a lot of movies to back up and encode. I typically leave them backed up into a .mkv container, but DVDs do not play too smoothly without being encoded. Since blurays and 4K UHDs are luckily 24p, I usually don't have this problem and I can just back them up into a .mkv with no extra encoding. DVDs are a whole different animal unfortunately and I'm hoping you can help make this painful process, a little easier on me.

Thank you in advance for your help.

SaurusX
20th April 2024, 01:04
I've always considered MPEG2SOURCE to be the gold standard. It is frame perfect. I've never seen the need to use the CPU2 function.

You can run a quick encode with just the TFM "output" feature to generate a text file showing how every frame was treated. Drop it into Excel and you can quickly find everything that was deinterlaced. Then go back to make any adjustments with override files.

Use animeivtc(mode=2) to get the best output from field blended anime. Yes, you may notice the blends a bit more, but I think that is due to individual frames remaining on-screen for longer intervals of time. A frame played back at 23.976 will stay on the screen longer than a full deinterlaced video at 59.96. The process will absolutely minimize the number of blends you get and get you as close as you can to the original frame sequence.

StormMeows
20th April 2024, 01:44
I've always considered MPEG2SOURCE to be the gold standard. It is frame perfect. I've never seen the need to use the CPU2 function.

You can run a quick encode with just the TFM "output" feature to generate a text file showing how every frame was treated. Drop it into Excel and you can quickly find everything that was deinterlaced. Then go back to make any adjustments with override files.

Use animeivtc(mode=2) to get the best output from field blended anime. Yes, you may notice the blends a bit more, but I think that is due to individual frames remaining on-screen for longer intervals of time. A frame played back at 23.976 will stay on the screen longer than a full deinterlaced video at 59.96. The process will absolutely minimize the number of blends you get and get you as close as you can to the original frame sequence.
Thank you for replying to my concerns. Based off of what you said, I will exchange the D2VSource that i have seen others mentioning to use for MPEG sources, and use MPEG2Source instead. Does it work fine with x64 plug-ins? As I mentioned previously, the CPU/CPU2 functions don't work with my avisynth x64, but MPEG2Source and info=3 works just fine, for instance. I do not need to use the CPU functions, but I was just curious if by it not working, it means that something is wrong with my MPEG2Source plug-in. I also saw that the DGindex manual specified an example using MPEG2Source, but the documentation seemed pretty old so I thought they might have just not updated it to use "D2VSource," for an example, yet.

Thanks for pointing out "output." That is VERY useful! When I had researched how to show a log of all deinterlaced frames prior, I just found really complex avisynth code that I did not comprehend. This output method seems to work well. I am now trying the "ovr" method of TFM so that I can specify which frames get de-interlaced. If the frames are false positives, is it better to do a pp=0 on them or just use the "-" to specify that these are indeed clean frames. I am going with the -, symbolizing that they are clean frames, instead of comb frames, for now.

I did the test encode of the entire movie in simple x264 launcher with --preset ultrafast --CRF 51. Is there an even faster way of encoding the whole movie so I can have it spit out the output file even quicker?

When I was talking about field blended NTSC and PAL DVD sources I have come across, I wasn't specifying anime. All of the DVDs that I have are live action movies. Would you handle these differently? I have been doing QTGMC() and keeping the NTSC at 59.97 FPS and have also tried them with QTGMC().srestore(frate=23.976). This movie played much smoother with just QTGMC at double the framerate. I have only tried the QTGMC+ srestore on the PAL field blended so far, but I was also thinking about trying double the QTGMC() double framerate on them. I'm not sure if on those I would want to do 50hz or 59.97 since most of those live action movies are NTSC original movies that were converted to PAL.

The crazy thing about the NTSC field blended movie I came across, is that it's a NTSC native movie (USA filmed) so it's weird that it was done field blended and not the standard NTSC 3:2 pulldown.

Thank you so much for your help! I am still learning a lot and have researched on here and other forums a lot before asking questions. I really would prefer not to encode and leave them as .mkv backups, but the live action DVDs really don't play well with Plex/Emby/Kodi, etc with just backing them up to .mkv.

StvG
20th April 2024, 05:37
With that being said, does it matter if I use D2VSource() or MPEG2Source() to index the .d2v file? I have read mixed reviews on using both. On one hand, D2VSource is newer and being updated constantly. I believe MPEG2Source is dead and no longer being updated/worked on. On both D2VSource's and MPEG2Sources's avisynth pages, they both support the use of using "d2v="source" within the TFM parameter. Is one of these plug-ins better to use in 2024 for MPEG DVDs specifically? I have seen a lot of you still using the old MPEG2Source and NOT the newer D2VSource. I have tried both and they don't seem any different for what I do. I use x64 for all of my avisynth plug-ins if that matters. Also, I will note, if I use MPEG2Source, and I try to do CPU2=, it says that CPU and CPU2 do not exist for MPEG2Source. This could be because I'm using all 64bit plug-ins/avisynth, I'm not quite sure.

A few years ago DGDecode (MPEG2Source) was kinda of abandoned. There was no x64 version.
MPEG2DecPlus (http://avisynth.nl/index.php?title=MPEG2DecPlus&oldid=11337) came up as "a project to modernize "DGDecode.dll" for AviSynth+" and included a x64 version.
After MPEG2DecPlus was was kinda of abandoned and this fork came up (https://github.com/Asd-g/MPEG2DecPlus). Some redundant functions was removed and the project was renamed to d2vsource (https://github.com/Asd-g/MPEG2DecPlus/issues/2).
After DGDecode (MPEG2Source) was updated and included a x64 plugin version - A 64-bit executable of DGDecode is now included (derived from fork by Asd-g). (https://www.rationalqm.us/dgmpgdec/FullChanges.html).
So both D2VSource() and MPEG2Source() are pretty much the same thing with some different parameters.

Edit:
I did the test encode of the entire movie in simple x264 launcher with --preset ultrafast --CRF 51. Is there an even faster way of encoding the whole movie so I can have it spit out the output file even quicker?

No need to encode. Just run the saved avs script with one of the following avsr64/avsmeter64 (https://forum.doom9.org/showthread.php?t=173259), ffmpeg (ffmpeg -i avs_script -f null -), avspmod (run analysys pass), maybe virtualdub2 also has option to run only the script.

StormMeows
20th April 2024, 06:02
A few years ago DGDecode (MPEG2Source) was kinda of abandoned. There was no x64 version.
MPEG2DecPlus (http://avisynth.nl/index.php?title=MPEG2DecPlus&oldid=11337) came up as "a project to modernize "DGDecode.dll" for AviSynth+" and included a x64 version.
After MPEG2DecPlus was was kinda of abandoned and this fork came up (https://github.com/Asd-g/MPEG2DecPlus). Some redundant functions was removed and the project was renamed to d2vsource (https://github.com/Asd-g/MPEG2DecPlus/issues/2).
After DGDecode (MPEG2Source) was updated and included a x64 plugin version - A 64-bit executable of DGDecode is now included (derived from fork by Asd-g). (https://www.rationalqm.us/dgmpgdec/FullChanges.html).
So both D2VSource() and MPEG2Source() are pretty much the same thing with some different parameters.

Edit:


No need to encode. Just run the saved avs script with one of the following avsr64/avsmeter64 (https://forum.doom9.org/showthread.php?t=173259), ffmpeg (ffmpeg -i avs_script -f null -), avspmod (run analysys pass), maybe virtualdub2 also has option to run only the script.

I really appreciate you breaking down the history of MPEG2Source and D2VSource! I kept searching keywords on this forum, other forums, and Google and could never get to the bottom of which one was the right one to use for this "DVD encode" project. Since most of the documentation for TFM, DGIndex, etc specifically mention using "MPEG2Source" plug-in, I guess I will just stick to that with x64, for this DVD project, since I want to ensure d2v= parameter inside of TFM works correctly. Although, the D2VSource avisynth documentation also specifies that I can also use d2v= with it, as well. It sounds like I'm good with either though, which is good news!

That's awesome that there's a way to just do a analysis pass without encoding the whole movie. I'll have to see which one is the quickest. I'm pretty happy so far with just using Simple x264 with the ultra fast preset and CRF 51 since it only takes around 4 minutes, but perhaps the other methods are even quicker. I use avspmod already so I will definitely have a look at the analysis pass option! Thanks SO much for the helpful information. It's members, like you guys that really remind me that there are so many amazing people in this world! I'm hopeful I can find out more about field blending soon. I have thoroughly researched it but it's definitely a head scratcher, indeed.

StvG
20th April 2024, 06:24
I really appreciate you breaking down the history of MPEG2Source and D2VSource! I kept searching keywords on this forum, other forums, and Google and could never get to the bottom of which one was the right one to use for this "DVD encode" project. Since most of the documentation for TFM, DGIndex, etc specifically mention using "MPEG2Source" plug-in, I guess I will just stick to that with x64, for this DVD project, since I want to ensure d2v= parameter inside of TFM works correctly. Although, the D2VSource avisynth documentation also specifies that I can also use d2v= with it, as well. It sounds like I'm good with either though, which is good news!

That's awesome that there's a way to just do a analysis pass without encoding the whole movie. I'll have to see which one is the quickest. I'm pretty happy so far with just using Simple x264 with the ultra fast preset and CRF 51 since it only takes around 4 minutes, but perhaps the other methods are even quicker. I use avspmod already so I will definitely have a look at the analysis pass option! Thanks SO much for the helpful information. It's members, like you guys that really remind me that there are so many amazing people in this world! I'm hopeful I can find out more about field blending soon. I have thoroughly researched it but it's definitely a head scratcher, indeed.

Since most of the documentation for TFM, DGIndex, etc specifically mention using "MPEG2Source" plug-in, - because they are written before MPEG2DecPlus/D2VSource came up and they are not completely updated with years.
d2v parameter performs identical job for both DGDecode and D2VSource.
The different parameters for DGDecode and D2VSource I mentioned - I had in mind mainly D2VSource rff parameter.

AvsPmod analysys pass - Video->Tools->Run analysis pass/Run FPS analysis.

StormMeows
20th April 2024, 07:00
Since most of the documentation for TFM, DGIndex, etc specifically mention using "MPEG2Source" plug-in, - because they are written before MPEG2DecPlus/D2VSource came up and they are not completely updated with years.
d2v parameter performs identical job for both DGDecode and D2VSource.
The different parameters for DGDecode and D2VSource I mentioned - I had in mind mainly D2VSource rff parameter.

AvsPmod analysys pass - Video->Tools->Run analysis pass/Run FPS analysis.

Thanks for clarifying that the documentation is not updated. I see now that the TFM avisyth nl page hasn't been updated since February of 2016! All in all, I'm glad to hear that whether I use MPEG2Source or D2VSource with x64 plugins/avisynth, it's all going to work the same, for my DVD encoding needs. :)

The "run analysis pass" works great and it gives me exactly what I need to see what frames are being detected as combed with TFM. Thanks again for your great insight!!