Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 1st March 2021, 22:58   #1  |  Link
pcroland
Registered User
 
Join Date: Mar 2014
Location: Hungary
Posts: 115
Properly deinterlace mostly telecined content

Hi!

I have videos where most of the content (like 99%) are telecined but have some animations and transitions that are regular interlaced frames. If I do a TFM().TDecimate() I'll have nice progressive frames but sometimes get interlaced frames. If I use QTGMC for the whole thing then I lose some resolution. Is there a way apply IVTC and then detect frames that are still interlaced and only deinterlace those frames?

Sample:
https://x0.at/hAf.mkv
https://mega.nz/file/DB5jCYiY#J89Tsm...H7EXL-d8nyNEhM

Thanks in advance.
pcroland is offline   Reply With Quote
Old 2nd March 2021, 01:37   #2  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
The PP settings in TFM are your friend.
johnmeyer is offline   Reply With Quote
Old 2nd March 2021, 01:41   #3  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
TFM will only return 29.97 max

A mix of 23.976p, 30p, 59.94p can be done with

TDeint(mode=1,tryweave=true, edeint=QTGMC())
#dedup 1st pass
#dedup 2nd pass

It will try to weave progressive content, but deinterlace interlaced content with QTGMC , all in a 59.94p base (23.976p sections will have 3:2 duplicates, 29.97p sections will have duplicates, if you had 12p sections they would have 5x duplicates...you get the idea)

Run 2pass dedup afterwards, this will decimate duplicates and output timecodes to make unique frames, and VFR timing to keep sync
poisondeathray is offline   Reply With Quote
Old 2nd March 2021, 03:02   #4  |  Link
pcroland
Registered User
 
Join Date: Mar 2014
Location: Hungary
Posts: 115
Quote:
Originally Posted by poisondeathray View Post
TFM will only return 29.97 max

A mix of 23.976p, 30p, 59.94p can be done with

TDeint(mode=1,tryweave=true, edeint=QTGMC())
#dedup 1st pass
#dedup 2nd pass

It will try to weave progressive content, but deinterlace interlaced content with QTGMC , all in a 59.94p base (23.976p sections will have 3:2 duplicates, 29.97p sections will have duplicates, if you had 12p sections they would have 5x duplicates...you get the idea)

Run 2pass dedup afterwards, this will decimate duplicates and output timecodes to make unique frames, and VFR timing to keep sync
Thank you very much! I thought it will be much harder. Now I need to decide whether I want encode it as 23.976 or 59.94fps or VFR. I really don't like VFR but since I want to upscale the whole series with waifu2x-ncnn-vulkan it would be much faster. Thanks again.
pcroland is offline   Reply With Quote
Old 2nd March 2021, 09:39   #5  |  Link
Overdrive80
Anime addict
 
Overdrive80's Avatar
 
Join Date: Feb 2009
Location: Spain
Posts: 673
As poison proposes, maybe after TDeint you can try Srestore instead of dedup.
__________________
Intel i7-6700K + Noctua NH-D15 + Z170A XPower G. Titanium + Kingston HyperX Savage DDR4 2x8GB + Radeon RX580 8GB DDR5 + ADATA SX8200 Pro 1 TB + Antec EDG750 80 Plus Gold Mod + Corsair 780T Graphite
Overdrive80 is offline   Reply With Quote
Old 2nd March 2021, 15:22   #6  |  Link
pcroland
Registered User
 
Join Date: Mar 2014
Location: Hungary
Posts: 115
I would lose frames at 59.94fps parts with Srestore but I would get alternating frame lengths at 23.976fps parts with DeDup (33.3 / 50 ms). From my tests the alternating frame lengths are not really noticable.
pcroland is offline   Reply With Quote
Old 2nd March 2021, 20:26   #7  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,823
According to TIVTC, just under 11% of that episode is video. Those interlaced transitions must add up. I encoded the series myself about 2 years ago, and tried picking out the interlaced sections manually while creating a timecodes file from scratch, but if memory serves I eventually realised there was too much I missed, so for most episodes, the second time around I just let TIVTC do it's thing for a variable frame rate. Those encodes have been sitting on my hard drive waiting for me to check them for 6 months now, so I had a look at the episode from which your sample came. TIVTC seems to have done a good job (using QTGMC to de-interlace). I'll have to encode it again though because there's banding I would've fixed if I'd noticed it. It's in the source so I'm not sure why I didn't. Anyway....

TFM glitched in one spot, for which I can probably blame myself because I was aware of the problem but apparently forgot to give TFM a hand before encoding. To save you a "gotcha", look out for frame #3434. It has a field that doesn't belong. The text hasn't started scrolling. Three frames later you can see it has.





I upscaled a little with nnedi3 myself, although some episodes contain such a plethora of encoding artefacts I almost didn't bother (the first episode was so bad I spent time replacing bad frames with good duplicates), but partly because I'm not overly crazy about more clever upscaling, at least not yet. I'd be interested to learn if the result is better, and if you've a clever way of dealing with the encoding artefacts.

Later versions of TDecimate have an argument called orgOut that seems to be designed to help with the creation of a timecodes file with the video sections at 59.94fps, but I've not played with it myself. I'll have to encode my effort again to fix the problems I mentioned, but here's a small VFR sample upscaled to 720p for anyone who cares (it's just the first minute or so). Personally, I prefer VFR encoding for hybrid sources.

Drawn.zip (25.6 MB)

First pass:

Code:
DClip = QTGMC(FPSDivisor=2)
TFM(Output="0215TFM.txt", Clip2=DClip)
TDecimate(Output="0215TD.txt", Mode=4, Hybrid=2, MKVOut="0215TimeCodes.txt")
2nd pass:

Code:
DClip = QTGMC(FPSDivisor=2)
TFM(Input="0215TFM.txt", Clip2=DClip)
TDecimate(Input="0215TD.txt", Mode=5, Hybrid=2, MKVOut="0215TimeCodes.txt", TFMIn="0215TFM.txt")
# Other filtering here.

Last edited by hello_hello; 2nd March 2021 at 21:48.
hello_hello is offline   Reply With Quote
Old 3rd March 2021, 15:34   #8  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by pcroland View Post
I would get alternating frame lengths at 23.976fps parts with DeDup (33.3 / 50 ms). From my tests the alternating frame lengths are not really noticable.
This is a form of "judder."

If you watched 23.976 CFR on a 60Hz display you'd get similar frames with different display times

I didn't test to verify this - but if you use ChangeFPS(120000,1001) before dedup, it will repeat frames in a "base" of 120000/1001. This should even out the timestamp deltas, and all typical content framerates are evenly divisible into it (integer result) . But if you watch that on a 60Hz display , you still will get judder from the display
poisondeathray is offline   Reply With Quote
Old 3rd March 2021, 18:23   #9  |  Link
pcroland
Registered User
 
Join Date: Mar 2014
Location: Hungary
Posts: 115
Quote:
Originally Posted by hello_hello View Post
TFM glitched in one spot, for which I can probably blame myself because I was aware of the problem but apparently forgot to give TFM a hand before encoding. To save you a "gotcha", look out for frame #3434. It has a field that doesn't belong. The text hasn't started scrolling. Three frames later you can see it has.
TDeint also has that small glitch. It doesn't really bother me. I guess it can't detect it because there are just a few pixels that the interlacing is visible on compared to the whole image.
Quote:
Originally Posted by hello_hello View Post
1st pass:
Code:
DClip = QTGMC(FPSDivisor=2)
TFM(Output="0215TFM.txt", Clip2=DClip)
TDecimate(Output="0215TD.txt", Mode=4, Hybrid=2, MKVOut="0215TimeCodes.txt")
2nd pass:

Code:
DClip = QTGMC(FPSDivisor=2)
TFM(Input="0215TFM.txt", Clip2=DClip)
TDecimate(Input="0215TD.txt", Mode=5, Hybrid=2, MKVOut="0215TimeCodes.txt", TFMIn="0215TFM.txt")
# Other filtering here.
I got this error for this:



With this script however:
Code:
FFMS2("source.mkv")
TDeint(mode=1, tryweave=true, edeint=QTGMC())
#DupMC(log="dupmc.txt")
DeDup(threshold=1.2, maxcopies=20, maxdrops=10, log="dupmc.txt", times="dedup.txt")
I extracted the frames with ffmpeg, upscaled them with waifu2x-ncnn-vulkan (-s 2 -n 3) and encoded them with x264 tMod:
https://0x0.st/-PXh.mkv
https://mega.nz/file/2R5yhJxa#QyZvBP...9PmFgoor7vivE4
It was just a quick test to see if everything works and it looks great in my opinion. I'll encode both the original and the waifu2x upscaled version (I know not everyone is a fan of this smoothed out upscaled image)
And I have another question:
If I encode with an mkv output both mediainfo and ffmpeg reports 59.94fps CFR and after a remux with mkvmerge they properly report VFR. What would be the correct workflow? Should I pass "--tcfile-in" for x264 and also specify the timecodes file when I remux it or should I only specify the timecodes file at encoding? Both will result in proper playback but I don't know if there is any difference at all.
pcroland is offline   Reply With Quote
Old 3rd March 2021, 19:00   #10  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by pcroland View Post
If I encode with an mkv output both mediainfo and ffmpeg reports 59.94fps CFR and after a remux with mkvmerge they properly report VFR. What would be the correct workflow? Should I pass "--tcfile-in" for x264 and also specify the timecodes file when I remux it or should I only specify the timecodes file at encoding? Both will result in proper playback but I don't know if there is any difference at all.

Either works. The difference is x264 is frame rate aware. If you use CRF encoding, it will adjust the quantizers according the the framerate with tcfile-in. Higher quantizers for higher fps sections. If you encoded it as CRF as CFR, it will be slightly different allocation
poisondeathray is offline   Reply With Quote
Old 3rd March 2021, 23:10   #11  |  Link
pcroland
Registered User
 
Join Date: Mar 2014
Location: Hungary
Posts: 115
Hmmm. Progressive frames that have interlaced fading will make the image jump up and down:

Can I combat this somehow? Or should I separate the fields and match the brightness before using TDeint so it doesn't detect the frame as interlaced in the first place??

Last edited by pcroland; 3rd March 2021 at 23:18.
pcroland is offline   Reply With Quote
Old 3rd March 2021, 23:29   #12  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Not sure, maybe use ovr , overrides
poisondeathray is offline   Reply With Quote
Old 3rd March 2021, 23:41   #13  |  Link
pcroland
Registered User
 
Join Date: Mar 2014
Location: Hungary
Posts: 115
What do you mean by ovr?
pcroland is offline   Reply With Quote
Old 3rd March 2021, 23:43   #14  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by pcroland View Post
What do you mean by ovr?
ovr is the overrides file path

Look in the documentation , there are examples

There might be other settings you could play with; ovr is usually "last resort" - if you do too many ovr, you might as well do it manually anyways (just specify sections that are ivtc'ed , double rate deinterlaced, and write the timestamp ranges in v2 format)
poisondeathray is offline   Reply With Quote
Old 3rd March 2021, 23:45   #15  |  Link
pcroland
Registered User
 
Join Date: Mar 2014
Location: Hungary
Posts: 115
I found it meanwhile. It would take many hours since this show has 36 episodes. I guess it's fine like this, I can live with this at least, already much better than what I was starting with. Thanks for all the help :-)
pcroland is offline   Reply With Quote
Old 4th March 2021, 00:17   #16  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,823
There's no need to use the timecodes file when muxing if x264 is writing it's output directly to MKV and you've used --tcfile-in, although if the output is a raw h264 stream, I'm not sure.

I'm short on internet time today, so a quick post to say I thought the quality of your upscaled encode looked quite good. The artefacts in the source I never found a way to remove completely without sacrificing picture detail.... did the upscaling take care of that in the process, or was there other filtering involved?

Sometime later today I'll try a 2 pass encode with TIVTC to see if I have the same problem and report back. I'm not sure I've bumped into that exact error myself. I did have a quick look at the old metrics files, and TFM seemed to be in agreement with TDecimate.

The last few lines of the TDecimate metrics file:

Quote:
38065 0 0
38066 0 0
38067 0 0
38068 0 0
38069 0 0
38070 0 0
38071 0 0
38072 512 0
From TFM:

Quote:
38068 c - [0]
38069 c - [0]
38070 c - [0]
38071 c - [0]
38072 c - [0]
#
#
# OVR HELP INFORMATION:
#
# [COMBED FRAMES]
#
One other quick thought. FFMS2 runs in VFR mode by default. Repeat flags are ignored and it decodes at the average frame rate. If there's no soft telecine it should be okay, but to tell FFMS2 to behave as DGIndex/DGDecode does and honour repeat flags....
FFMS2("source.mkv", rffmode=1)
Just in case you weren't aware.

Quote:
##### int rffmode = 0
Controls how RFF flags in the video stream are treated; in other words it's equivalent to the "field operation" mode switch in DVD2AVI/DGIndex.
Valid modes are:

- **0**: Ignore all flags (the default mode).
- **1**: Honor all pulldown flags.
- **2**: Equivalent to DVD2AVI's "force film" mode.

Note that using modes 1 or 2 will make `FFVideoSource` throw an error if the video stream has no RFF flags at all.
When using either of those modes, it will also make the output be assumed as CFR, disallow vertical scaling and disallow setting the output colorspace.
`FFPICT_TYPE` will also not be set as the output is a combination of several frames.
Other subtle behavior changes may also exist.
hello_hello is offline   Reply With Quote
Old 4th March 2021, 00:36   #17  |  Link
pcroland
Registered User
 
Join Date: Mar 2014
Location: Hungary
Posts: 115
"did the upscaling take care of that in the process, or was there other filtering involved?"
Yes, I didn't do anything yet besides TDeint and Dedup.
pcroland is offline   Reply With Quote
Old 4th March 2021, 02:36   #18  |  Link
pcroland
Registered User
 
Join Date: Mar 2014
Location: Hungary
Posts: 115
These are my scripts for the whole process if anyone is interested:

upscale.sh
Code:
#!/bin/bash

temp='temp'
output='output'
path_ffmpeg='ffmpeg.exe'
path_x264='x264.exe'
path_waifu2x_ncnn_vulkan='waifu2x-ncnn-vulkan.exe'
tg_token='asd'
tg_chat_id='asd'

current_frames() {
  sleep 3
  while true; do
    current_frames=$(ls "$temp"/frames_upscaled | wc -l)
	percent=$(bc <<< "scale=2; $current_frames*100/$frames")
	delta_upscale=$(($SECONDS - start_time))
	fps=$(bc <<< "scale=2; $current_frames/$delta_upscale")
    printf '\rupscaling frames \e[93m%s/%s [%s%%]\e[0m, FPS=\e[93m%s\e[0m' "$current_frames" "$frames" "$percent" "$fps"
    sleep 0.5
  done
}

print_separator() {
  printf '%.0s─' $(seq 1 "$(tput cols)")
}

notification() {
  printf 'sending telegram notification...\n'
  curl -s -X POST https://api.telegram.org/bot"$tg_token"/sendMessage -d chat_id="$tg_chat_id" -d parse_mode=html -d text="\
<code>${i}</code>%0A\
running dupmc with avsmeter took <code>${delta_dupmc}</code> seconds.%0A\
extracting frames took <code>${delta_extract}</code> seconds. (${size_extract})%0A\
upscaling <code>${frames}</code> frames with <code>${fps}</code> FPS took <code>${delta_upscale}</code> seconds. (${size_upscale})%0A\
merging upscaled frames took <code>${delta_merge}</code> seconds. (${size_merged}/${size_merged_all})" > /dev/null
}

for i in "$@"; do
  b=$(basename "$i")
  printf '\e[92m%s\e[0m\n' "$b"
  rm -rf "$temp"/frames_extracted "$temp"/frames_upscaled
  mkdir -p "$temp"/frames_extracted "$temp"/frames_upscaled "$output"

  start_time=$SECONDS
  printf 'running dupmc with avsmeter...'
  AVSMeter.exe -o ${i%.*}_dupmc.avs &> /dev/null
  end_time=$SECONDS
  delta_dupmc=$((end_time - start_time))
  printf ' took \e[93m%s\e[0m seconds.\n' "$delta_dupmc"

  start_time=$SECONDS
  printf 'extracting frames...'
  "$path_ffmpeg" -v quiet -i "$i" -pix_fmt rgb24 "$temp"/frames_extracted/%06d.png
  end_time=$SECONDS
  delta_extract=$((end_time - start_time))
  size_extract=$(du -s --block-size=M "$temp"/frames_extracted | sed -e 's/\s.*$/iB/')
  printf ' took \e[93m%s\e[0m seconds. (%s)\n' "$delta_extract" "$size_extract"

  start_time=$SECONDS
  frames=$(ls "$temp"/frames_extracted | wc -l)
  printf 'upscaling frames'
  current_frames &
  pid=$!
  "$path_waifu2x_ncnn_vulkan" -n 3 -i "$temp"/frames_extracted -o "$temp"/frames_upscaled &> /dev/null
  kill -PIPE "$pid"
  end_time=$SECONDS
  delta_upscale=$((end_time - start_time))
  size_upscale=$(du -s --block-size=M "$temp"/frames_upscaled | sed -e 's/\s.*$/iB/')
  fps=$(bc <<< "scale=2; $frames/$delta_upscale")
  printf '\rupscaling frames \e[93m%s/%s [100.00%%]\e[0m, FPS=\e[93m%s\e[0m took \e[93m%s\e[0m seconds. (%s)\n' "$frames" "$frames" "$fps" "$delta_upscale" "$size_upscale"

  start_time=$SECONDS
  printf 'merging upscaled frames...'
  "$path_x264" --qp 0 --preset veryfast --tcfile-in ${i%.*}.txt --output "$output"/${b%.*}.mkv source.avs &> /dev/null
  end_time=$SECONDS
  delta_merge=$((end_time - start_time))
  size_merged=$(du -s --block-size=M "$output"/${b%.*}.mkv | sed -e 's/\s.*$/iB/')
  size_merged_all=$(du -s --block-size=M "$output" | sed -e 's/\s.*$/iB/')
  printf ' took \e[93m%s\e[0m seconds. (%s/%s)\n' "$delta_merge" "$size_merged" "$size_merged_all"

  notification
  print_separator
done
xy_dupmc.avs:
Code:
FFMS2("path\upscale\source\01.mkv")
TDeint(mode=1, tryweave=true, edeint=QTGMC())
DupMC(log="01_dupmc.txt")
xy.avs:
Code:
FFMS2("path\source\01.mkv")
TDeint(mode=1, tryweave=true, edeint=QTGMC())
DeDup(threshold=1.2, maxcopies=20, maxdrops=10, log="01_dupmc.txt", times="01.txt")
source.avs:
Code:
RT_WriteFileList("path\temp\frames_upscaled\*.png", "temp.txt")
a=RT_FileQueryLines("temp.txt")
ImageSource("path\temp\frames_upscaled\%06d.png", start=1, end=Int(a))
ConvertToYV12()
For example if I run
Code:
./upscale.sh scripts/01.avs
It runs 01_dupmc.avs with AVSMeter, then extracts the frames from 01.avs with ffmpeg. Then frames inside temp/extracted_frames gets upscaled with waifu2x-ncnn-vulkan to temp/upscaled_frames.
Then x264 converts the frames to a video at the output folder with --qp 0 from source.avs. The source.avs file is static because the temp folders gets removed on every iteration.
(I use --qp 0 because I'll replace live action parts in the upscaled video with the original.)

It looks something like this in shell:

Last edited by pcroland; 4th March 2021 at 03:15.
pcroland is offline   Reply With Quote
Old 4th March 2021, 04:04   #19  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,823
Quote:
Originally Posted by pcroland View Post
Hmmm. Progressive frames that have interlaced fading will make the image jump up and down:
Can I combat this somehow? Or should I separate the fields and match the brightness before using TDeint so it doesn't detect the frame as interlaced in the first place??
I'd forgotten about that. I mentioned hacking into episode one the other day, but I forgot about the eradication of fading whites.

I've got to stay in the real world for most of today so it'll be tonight or sometime tomorrow, but I can dig out the old script so you don't have to hunt down frame ranges from scratch. The first minute of the episode is below. If you can live with the frames I dumped/duplicated and the whites that no longer fade etc, I'll post the Trims from the script, assuming you want them. If you don't like what I did for the first minute I won't be offended.

Drawn 1.zip

Last edited by hello_hello; 4th March 2021 at 04:14.
hello_hello is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 17:20.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.