IVTC causing weird vertical blocking

[np39]

I'm trying to convert a movie that is showing up as NTSC in DVD2AVI, but I've looked at it frame by frame and can tell it's telecined (3/2 frame interlacing pattern).

So I've set Gordian Knot to use IVTC, and though the preview looked okay, the end product has vertical rectangles all over the image whereever there is motion. The still parts look okay, but it looks like the frames are sort of temporally scrambled in vertical rectangles (about twice as tall as they are wide) in the regions where there is motion.

I'll attach my GKnot log file. I've read the various IVTC documentation, but it doesn't mention this issue. If this is the wrong place to ask this, please let me know if I should post it in the AVIsynth forum or what. The blocking doesn't appear when I don't use IVTC.

I -know- the content has been telecined, not only does it have the 3/2 interlacing pattern but it looks like crap when I do a straight deinterlace on it. :-(

---

2/23/2004 7:45:39 PM: Job " 2 - ginger snaps test" started.

o Transcode source -> MP3: C:\dvdrip\rips\Ginger Snaps\test\ivtc AC3 T01 2_0ch 192Kbps DELAY 18ms.ac3
o DivX5-Single, Quality 20 : C:\dvdrip\rips\Ginger Snaps\test\ivtc_Credits.avs
o DivX5-First Pass: C:\dvdrip\rips\Ginger Snaps\test\ivtc.avs
o DivX5 - Pass 2: C:\dvdrip\rips\Ginger Snaps\test\ivtc.avs
o Append Credits and Mux Audio.
x Delete Intermediate Files (disabled).
o Splitting final movie.
End of Job 2 ( 2 - ginger snaps test).
_______________________________________________________________________________________________________________________

Gordian Knot -
Encoding Job Data:

Type: EncD5_All_CreditsSep
Number: 2
Name: 2 - ginger snaps test
Platform: Win2000 (5.0.2195).2

Files:
VirtualDubModExe: C:\PROGRA~1\GORDIA~1\VirtualDubMod\VirtualDubMod.exe
NandubExe: C:\PROGRA~1\GORDIA~1\nandub\Nandub.exe
AviFinal: C:\dvdrip\rips\Ginger Snaps\test\ivtc.avi
AviMovie: C:\dvdrip\rips\Ginger Snaps\test\ivtc_Movie.avi
AviCredits: C:\dvdrip\rips\Ginger Snaps\test\ivtc_Credits.avi
MovieFS: C:\dvdrip\rips\Ginger Snaps\test\ivtc.avs
CreditsFS: C:\dvdrip\rips\Ginger Snaps\test\ivtc_Credits.avs
Log : C:\dvdrip\rips\Ginger Snaps\test\ivtc.log
Stats: C:\dvdrip\rips\Ginger Snaps\test\ivtc.stats
Ecf : C:\dvdrip\rips\Ginger Snaps\test\ivtc.ecf

Options:
Quality/DRF: 20
CreditsStart: 188972
UseEcf: 1
CreditsIVTC: 1
CreditsAppend: 1
DeleteInterm.: 0
EnforceBitrate: 0
AntiShit: 0
Split final file at CD size:700

Calculator:
VideoSize: 577724 kbyte.
Audio1Size: 126816 kbyte.
Audio2Size: 0 kbyte.
Overhead: 12259 kbyte.
AviSize: 716800 kbyte (700 mbyte).
FilesSize: 0 kbyte.
TotalSize: 716800 kbyte (700 mbyte).
Interleaving: vbrmp3
Bitrate: 729 k(=1000)Bits/s
Frames: 194584
FPS: 29.970
Duration: 148:13
Quality: 0.124 bits/(pixel*frame).
CompCheck3: 0.000 bits/(pixel*frame).
CompCheck5: 0.000 bits/(pixel*frame).
Resolution: 512 x 384

Audio 1:
Mode: ac3 -> mp3
SourceFile: C:\dvdrip\rips\Ginger Snaps\test\ivtc AC3 T01 2_0ch 192Kbps DELAY 18ms.ac3
WavFile: C:\dvdrip\rips\Ginger Snaps\test\ivtc AC3 T01 2_0ch 192Kbps DELAY 18ms.wav
Mp3File: C:\dvdrip\rips\Ginger Snaps\test\ivtc AC3 T01 2_0ch 192Kbps DELAY 18ms.mp3
Azid: -L -3db -s stereo -c normal
Lame: --alt-preset 160
Delay: 18
Interval: 1
Preload: 500
Mux: 1
Recalc: 1
DeleteWav: 0

Audio 2:
none.

7:45:39 PM: Re-Calculating Frame Count.
7:45:39 PM: FrameRate was 29.970 fps
7:45:39 PM: FrameRate now is 23.976 fps
7:45:39 PM: IVTC: 155676 Frames.

7:45:39 PM: Started Transcoding Audio.

7:45:39 PM: Using "BeSweet" to transcode C:\dvdrip\rips\Ginger Snaps\test\ivtc AC3 T01 2_0ch 192Kbps DELAY 18ms.ac3
CommandLine: C:\PROGRA~1\GORDIA~1\BeSweet.exe -core( -input "C:\dvdrip\rips\Ginger Snaps\test\ivtc AC3 T01 2_0ch 192Kbps DELAY 18ms.ac3" -output "C:\dvdrip\rips\Ginger Snaps\test\ivtc AC3 T01 2_0ch 192Kbps DELAY 18ms.mp3" -logfile "C:\dvdrip\rips\Ginger Snaps\test\ivtc AC3 T01 2_0ch 192Kbps DELAY 18ms.log" ) -ota( -hybridgain ) -azid( -L -3db -s stereo -c normal ) -shibatch( --rate 44100 ) -lame( --alt-preset 160 ) -profile( Gordian Knot 0.28 )
8:06:50 PM: Finished. Duration: 21 minutes, 11 seconds.

8:06:50 PM: Audio Done.
8:06:50 PM: MP3-File-Size is 128897 kb.
8:06:50 PM: Audio 1, determined: 128897 kb
8:06:50 PM: Audio 1, calc: 126816 kb
8:06:50 PM: New Bitrate for Movie = 729 k(=1000)Bits/s

Adding this script to C:\PROGRA~1\GORDIA~1\VirtualDubMod\LastJob.vcf :

// $job "DivX5 Single Pass"
// $input "C:\dvdrip\rips\Ginger Snaps\test\ivtc_Credits.avs"
// $output "C:\dvdrip\rips\Ginger Snaps\test\ivtc_Credits.avi"
// $state 0
// $start_time 0 0
// $end_time 0 0
// $script

VirtualDub.Open("C:\\dvdrip\\rips\\Ginger Snaps\\test\\ivtc_Credits.avs",0,0);
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(1);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x78766964,0,10000,0);
VirtualDub.video.filters.Clear();
VirtualDub.subset.Delete();
VirtualDub.SaveAVI("C:\\dvdrip\\rips\\Ginger Snaps\\test\\ivtc_Credits.avi");
VirtualDub.Close();

// $endjob
//
//--------------------------------------------------
// $done

8:06:50 PM: Started DivX5-Single, Quality 20: C:\dvdrip\rips\Ginger Snaps\test\ivtc_Credits.avs
8:09:41 PM: Finished DivX5-Single, Quality 20: Duration: 2 minutes, 50 seconds.
8:09:41 PM: Assumed: Main Movie has 151178 Frames.
8:09:41 PM: Assumed: Credits Part has 4498 Frames.
8:09:41 PM: Credits Size = 3736 Kb
8:09:41 PM: New Bitrate for Movie = 746 k(=1000)Bits/s

Adding this script to C:\PROGRA~1\GORDIA~1\VirtualDubMod\LastJob.vcf :

// $job "DivX5-First Pass"
// $input "C:\dvdrip\rips\Ginger Snaps\test\ivtc.avs"
// $output "C:\dvdrip\rips\Ginger Snaps\test\ivtc_Movie.avi"
// $state 0
// $start_time 0 0
// $end_time 0 0
// $script

VirtualDub.Open("C:\\dvdrip\\rips\\Ginger Snaps\\test\\ivtc.avs",0,0);
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(1);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x78766964,0,10000,0);
VirtualDub.video.filters.Clear();
VirtualDub.subset.Delete();
VirtualDub.SaveAVI("C:\\dvdrip\\rips\\Ginger Snaps\\test\\ivtc_Movie.avi");
VirtualDub.Close();

// $endjob
//
//--------------------------------------------------
// $done

8:09:41 PM: Started DivX5-First Pass: C:\dvdrip\rips\Ginger Snaps\test\ivtc.avs
5:50:35 AM: Finished DivX5-First Pass: Duration: 9 hours, 40 minutes, 54 seconds.
5:50:35 AM: Trying to open Log-file.
5:50:36 AM: Success: Log-file open.
5:50:36 AM: Encoded: 151177 Frames.
5:50:36 AM: Speed: 4.337 Frames per Second.

Adding this script to C:\PROGRA~1\GORDIA~1\VirtualDubMod\LastJob.vcf :

// $job "DivX5-Second Pass"
// $input "C:\dvdrip\rips\Ginger Snaps\test\ivtc.avs"
// $output "C:\dvdrip\rips\Ginger Snaps\test\ivtc_Movie.avi"
// $state 0
// $start_time 0 0
// $end_time 0 0
// $script

VirtualDub.Open("C:\\dvdrip\\rips\\Ginger Snaps\\test\\ivtc.avs",0,0);
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(1);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x78766964,0,10000,0);
VirtualDub.video.filters.Clear();
VirtualDub.subset.Delete();
VirtualDub.SaveAVI("C:\\dvdrip\\rips\\Ginger Snaps\\test\\ivtc_Movie.avi");
VirtualDub.Close();

// $endjob
//
//--------------------------------------------------
// $done

5:50:36 AM: Started DivX5 - Pass: 2: C:\dvdrip\rips\Ginger Snaps\test\ivtc.avs
3:10:08 PM: Finished DivX5 - Pass: 2: Duration: 9 hours, 19 minutes, 32 seconds.
3:10:08 PM: Speed: 4.503 Frames per Second.

Adding this script to C:\PROGRA~1\GORDIA~1\VirtualDubMod\LastJob.vcf :

// $job "DivX5 Append"
// $input "C:\dvdrip\rips\Ginger Snaps\test\ivtc_Movie.avi"
// $output "C:\dvdrip\rips\Ginger Snaps\test\ivtc.avi"
// $state 0
// $start_time 0 0
// $end_time 0 0
// $script

VirtualDub.Open("C:\\dvdrip\\rips\\Ginger Snaps\\test\\ivtc_Movie.avi",0,0);
VirtualDub.Append("C:\\dvdrip\\rips\\Ginger Snaps\\test\\ivtc_Credits.avi");
VirtualDub.RemoveInputStreams();
VirtualDub.stream[0].SetSource("C:\\dvdrip\\rips\\Ginger Snaps\\test\\ivtc AC3 T01 2_0ch 192Kbps DELAY 18ms.mp3",0x00000202,1);
VirtualDub.stream[0].SetMode(0);
VirtualDub.stream[0].SetInterleave(1,500,1,0,18);
VirtualDub.stream[0].SetClipMode(1,1);
VirtualDub.stream[0].SetConversion(0,0,0,0,0);
VirtualDub.stream[0].SetVolume();
VirtualDub.stream[0].SetCompression();
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(0);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression();
VirtualDub.video.filters.Clear();
VirtualDub.subset.Delete();
VirtualDub.SaveAVI("C:\\dvdrip\\rips\\Ginger Snaps\\test\\ivtc.avi");
VirtualDub.Close();

// $endjob
//
//--------------------------------------------------
// $done

3:10:08 PM: Started Appending Credits and Muxing Audio.: C:\dvdrip\rips\Ginger Snaps\test\ivtc_Movie.avi
3:24:59 PM: Finished Appending Credits and Muxing Audio.: Duration: 14 minutes, 50 seconds.
3:24:59 PM: Total File Size is: 700 Mb
3:24:59 PM: No splitting required

3:24:59 PM: Done.
3:24:59 PM: Movie = C:\dvdrip\rips\Ginger Snaps\test\ivtc.avi
Total Encoding Time: 19 hours, 39 minutes, 20 seconds.
2/24/2004 3:24:59 PM: Job " 2 - ginger snaps test" finished.
_______________________________________________________________________________________________________________________

[manono]

Hi and welcome to the forum-

Right off the top of my head, I haven't a clue. Can you post the .avs?

And have you tried to check for the blocks in VDubMod? Open the .avi in VDubMod and see if they're still there. If they are, that would rule out a decoder or playback issue.

[np39]

Yeah, I see them in VDubMod.

VDubMod also doesn't like my VBR audio, and complains about it and crashes because of it, but the blocks are also present in the avi that hasn't had audio muxed into it.

What's interesting is that the credits, which were encoded separately, don't seem to have the problem. This is with DivX 5, BTW.

Here's the avs:

#
# Created with Gordian Knot
#
# http://gknot.doom9.org
#
# PLUGINS
LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec3.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\undot.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\decomb.dll")
#LoadPlugin("C:\PROGRA~1\GORDIA~1\KernelDeInt.dll")
#LoadPlugin("C:\PROGRA~1\GORDIA~1\dgbob.dll")
#LoadPlugin("C:\PROGRA~1\GORDIA~1\Convolution3d.dll")
#LoadPlugin("C:\PROGRA~1\GORDIA~1\FluxSmooth.dll")
#LoadPlugin("C:\PROGRA~1\GORDIA~1\TomsMoComp.dll")
#LoadPlugin("C:\PROGRA~1\GORDIA~1\VSFilter.dll")
#LoadPlugin("C:\PROGRA~1\GORDIA~1\SimpleResize.dll")
#
# SOURCE
mpeg2source("C:\dvdrip\rips\Ginger Snaps\test\ivtc.d2v")
#
# TRIM
trim(0,188971)
#
# IVTC
Telecide(order=1,guide=1).Decimate()
# or use
#IVTC(44,11,95)
#GreedyHMA(1,0,4,0,0,0,0,0)
#
# CROPPING
crop(4,2,712,472)
#
# DEINTERLACING (1)
#FieldDeinterlace()
#FieldDeinterlace(blend=false)
#TomsMoComp(1,5,1)
#
# DEINTERLACING (2)
#KernelDeInt(order=1,sharp=true)
# or maybe
#DGBob(order=1,mode=0)
# DEINTERLACING (3) - special requests
#GreedyHMA(1,0,0,0,0,0,0,0)
#Telecide()
#SeparateFields()
#
#
# SUBTITLES
#VobSub("FileName")
#
# RESIZING
LanczosResize(512,384)
#
Undot()
#
# DENOISING: choose one combination (or none)
# 1) little noise
#Temporalsoften(2,3,3,mode=2,scenechange=6)
#mergechroma(blur(1.3))
#FluxSmooth(5,7)
#
# 2) medium noise
#Temporalsoften(3,5,5,mode=2,scenechange=10)
#Convolution3d("moviehq")
#FluxSmooth(7,7)
#
# 3) heavy noise
#Temporalsoften(4,8,8,mode=2,scenechange=10)
#Convolution3d("movielq")
#FluxSmooth(10,15)
#
# BORDERS
#AddBorders(left,top,right,bottom)
#
# COMPRESSIBILITY CHECK
# !!!!Snip Size now has to be 14 for use in GKnot!
#SelectRangeEvery(466,14)
#
# FOOL CCEnc
#empty = BlankClip()
#AudioDub(last,empty)

[jggimi]

Try playing this .avs within VdubMod or with any AVI player. Do you see the problem then?

If so, then one of the filters is causing it. But I would venture that you don't see it when "playing" the .avs script.

More likely, then, you are seeing macroblocks due to your relatively low bitrate for a 4:3 encoding. A compressibility check has not been performed, and I recommend one.

[np39]

I'll mess with it, but I saw the blocks at least once or twice in the GKnot preview, IIRC. I also didn't see them other times, when they appeared in the final result.

These do not look like compression artifacts. They look like blocks, twice as high as wide, that have been taken out of sync by a few frames pseudorandomly.

So if the camera pans right over a picket fence, chunks of the fence appear temporally out of place, and the edges of the rectangles are very clean, smooth, and well-defined with no ringing or any of that sort of thing. It's like somebody cut them out with scissors.

Perhaps I can try to cut a segment out of the result and put it up somewhere.

[np39]

Okay, I don't see it when playing the .AVS in media player. Maybe I have a misunderstanding of what macroblocks are. I'm trying a compressibility check now -- I'll go back and read the GKnot guide again, since I wasn't clear on precisely what it was for, but I do know that despite the low bitrate, the visual quality was decent. I've encoded longer movies at lower bitrates than this -- granted not 4:3, but I do resize it down pretty far -- and didn't get any phenomenon like this, ever. Usually I just got a bunch of ringing around edges, which I don't see here.. just the scrambled blocks.

Compressiblity check yields .363. .124 bits/pixel*frame is 34.1%, in the red, BUT due to IVTC, this goes up to .155 after the # of frames is reduced, which is 42.8%, not in the red (or yellow). Like I said, the visual quality of the output other than the temporal scrambling, for lack of a better term, was good. So I don't believe the content is being overcompressed.

I should mention I have framerate in GKnot set to 29.970, since with IVTC and the checkbox for IVTC in avs -> correct frame count checked, it automatically recalculates all this as you can see in the log.

It's either me doing something wrong in telling GKnot how to process the raw content, or DivX encoding improperly, I think. *confused*

[np39]

Okay, I just tried encoding a portion of the movie at a similar bitrate, and I didn't get the temporal scrambling. And due to the high motion of the portion I encoded, the bitrate WAS in the red, so it's not due to overcompression. As I suspected, I got more ringing and breakup and what I'd always thought were macroblocks (a sort of chunking in smoother regions of similar colors), but none of the scrambling I was describing.

I'm going to try re-encoding the whole movie from scratch, but being as the encode takes almost a day on my machine, I'm going to be very unhappy if the problem happens again. I'll be sure to post my results.

[manono]

Hi-

Is this still Ginger Snaps? It's a recent movie, so it should have been encoded decently for the DVD, but from your descriptions so far, it sounds like a mess.

So if the camera pans right over a picket fence, chunks of the fence appear temporally out of place

Sounds like the IVTC may be failing, assuming that part of the movie also has that 3/2 pattern you described earlier. You might have to tweak the Telecide settings, as described in the doc.

It's either me doing something wrong in telling GKnot how to process the raw content, or DivX encoding improperly, I think.

Assuming the IVTC works correctly, I think you're doing everything correctly. BUT, 42.8% is pretty low, and you should try and get it up. Maybe put on one of the GKnot temp smoothers and use a softer resizer (NeutralBicubic, maybe). 42.8% with a clean source will make blocks when there's movement/action.

And now you're saying an encode of a part of the movie turned out OK. You could probably get away with just rerunning the second pass, and save some time.

[np39]

I re-encoded from scratch and it came out fine. Go figure.

I admit Lanczos was a little optimistic for this bitrate, and there is some ringing, but it looks great when viewed from my bed about 8-10 feet away from a 19 inch monitor. :-)

Thanks for the help, all.

I'll note in case I didn't before that DVD2AVI detected this as NTSC (not even 98%, but full NTSC) but every spot I checked was telecined. Who knows.