Explain why one movie varies so greatly with constant quantizer

[jmrenrav]

Ok so i'm stumped and trying to figure out why A Knight's Tale is coming out extremely large when I run a Ultra Fast CQ 20 on it. Compared to other movies like Star Trek, Underworld, The Matrix, Bourne Series, ect.

I use this to determine a good target bitrate, but I don't see how this movie is so "complex"

The Matrix - Video File Size: 2Gig

# Set DAR in encoder to 145 : 59. The following line is for automatic signalling
global MeGUI_darx = 145
global MeGUI_dary = 59
LoadPlugin("C:\Documents and Settings\c31278\Desktop\Bored\MeGUI\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("C:\Documents and Settings\c31278\Desktop\Bored\THE MATRIX WS\VIDEO_TS\VTS_01_1.d2v", info=3)
LoadPlugin("C:\Documents and Settings\c31278\Desktop\Bored\MeGUI\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true, threads=0)
#deinterlace
crop( 0, 60, 0, -64)

#resize
#denoise


[NoImage] Standard error stream
-[NoImage] avs [info]: 720x356 @ 23.98 fps (196159 frames)
-[NoImage] x264 [info]: using SAR=2581/2124
-[NoImage] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
-[NoImage] x264 [info]: profile Baseline, level 3.0
-[NoImage]
-[NoImage] x264 [info]: frame I:785 Avg QP:17.00 size: 32525
-[NoImage] x264 [info]: frame P:195374 Avg QP:20.00 size: 10453
-[NoImage] x264 [info]: mb I I16..4: 100.0% 0.0% 0.0%
-[NoImage] x264 [info]: mb P I16..4: 18.9% 0.0% 0.0% P16..4: 55.4% 0.0% 0.0% 0.0% 0.0% skip:25.7%
-[NoImage] x264 [info]: coded y,uvDC,uvAC intra: 59.9% 47.3% 14.9% inter: 40.8% 22.7% 2.4%
-[NoImage] x264 [info]: i16 v,h,dc,p: 40% 16% 24% 19%
-[NoImage] x264 [info]: kb/s:2021.87
-[NoImage] encoded 196159 frames, 134.19 fps, 2021.87 kb/s





A Knight's Tale - Video File Size - 3.6Gig!!

# Set DAR in encoder to 47 : 20. The following line is for automatic signalling
global MeGUI_darx = 47
global MeGUI_dary = 20
LoadPlugin("C:\Documents and Settings\c31278\Desktop\Bored\MeGUI\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("C:\Documents and Settings\c31278\Desktop\Bored\A KNIGHTS TALE WS\VIDEO_TS\VTS_01_1.d2v", info=3)
LoadPlugin("C:\Documents and Settings\c31278\Desktop\Bored\MeGUI\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true, threads=0)
#deinterlace
crop( 6, 54, 0, -60)

#resize
#denoise


[NoImage] Standard error stream
-[NoImage] avs [info]: 714x366 @ 23.98 fps (190104 frames)
-[NoImage] x264 [info]: using SAR=2867/2380
-[NoImage] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
-[NoImage] x264 [info]: profile Baseline, level 3.0
-[NoImage]
-[NoImage] x264 [info]: frame I:761 Avg QP:17.00 size: 57129
-[NoImage] x264 [info]: frame P:189343 Avg QP:20.00 size: 19601
-[NoImage] x264 [info]: mb I I16..4: 100.0% 0.0% 0.0%
-[NoImage] x264 [info]: mb P I16..4: 6.7% 0.0% 0.0% P16..4: 78.7% 0.0% 0.0% 0.0% 0.0% skip:14.6%
-[NoImage] x264 [info]: coded y,uvDC,uvAC intra: 72.9% 70.3% 41.0% inter: 61.6% 33.2% 5.3%
-[NoImage] x264 [info]: i16 v,h,dc,p: 38% 17% 23% 23%
-[NoImage] x264 [info]: kb/s:3788.38
-[NoImage] encoded 190104 frames, 131.81 fps, 3788.38 kb/s


So you have a movie which is SMALLER, SHORTER, and has less AVG Bitrate, but comes out nearly 50% bigger...

[sneaker_ger]

Matrix is relatively easy to compress (which is known) and Knight's Tale not - what's the big deal? That's totally normal. I don't really see what's to discuss here. Although some people might start questioning your whole method of trying "to determine a good target bitrate" by encoding CQ 20 instead of letting x264 do the job for you by doing a single pass CRF encode pretty soon.

/edit: And internally it is not "smaller". Instead both videos have same resolution since h264 works with 16x16 blocks.

[jmrenrav]

Matrix is relatively easy to compress (which is known) and Knight's Tale not - what's the big deal? That's totally normal. I don't really see what's to discuss here. Although some people might start questioning your whole method of trying "to determine a good target bitrate" by encoding CQ 20 instead of letting x264 do the job for you by doing a single pass CRF encode pretty soon.

I see a pattern with other movies, but not this one, which is why I'm bringing it up. The source is around 3.5gig, so at CQ 20 you would think you would see some type of compression, but the final size is the same as the source. I've used CRF before but the results would be similar, around 50% bigger file size for a movie that is shorter and appears to be less "complex"

[jmrenrav]

I think it may be an issue with the source and how it is detected. I've only had one other issue like this and it was a movie that was clearly interlaced.

Here is the D2V for A Knight's Tale:

Stream Type: MPEG2 Program
Profile: main@main
Frame Size: 720x480
Display Size: [not specified]
Aspect Ratio: 16:9 [3]
Frame Rate: 29.970030 fps
Video Type: Film 98.03%
Frame Type: Progressive
Coding Type: B
Colorimetry: BT.470-2 B,G*
Frame Structure: Frame
Field Order: Top
Coded Number: 3311
Playback Number: 4122
Frame Repeats: 0
Field Repeats: 1622
VOB ID: 1
Cell ID: 1
Bitrate: 3.908 Mbps
Bitrate (Avg): 3.587 Mbps
Bitrate (Max): 7.314 Mbps
Audio Stream: 80: AC3 3/2 448
Timestamp: 0:02:17
Elapsed: 0:02:17
Remain: 2:13:35
FPS: 29.97
Info:


Here is the other example I've run into, Three Fugitives

Stream Type: MPEG2 Program
Profile: main@main
Frame Size: 720x480
Display Size: 720x480
Aspect Ratio: 16:9 [3]
Frame Rate: 29.970030 fps
Video Type: Film 98.08%
Frame Type: Interlaced
Coding Type: B
Colorimetry: BT.470-2 B,G
Frame Structure: Frame
Field Order:
Coded Number: 139034
Playback Number: 2
Frame Repeats: 0
Field Repeats: 0
VOB ID: 3
Cell ID: 1
Bitrate:
Bitrate (Avg):
Bitrate (Max):
Audio Stream: 80: AC3 2/0 192
Timestamp: 0:00:00
Elapsed: 0:01:19
Remain: FINISH
FPS:
Info:


If I just load up Avisynth and crop/resize and run it through x264, the bitrate will be nearly double, once I analyse the source correctly and deinterlace, then the compression appears to be normal, so do you think there is a problem with DGIndex or Avisynth detecting the source for A Knight's Tale?

[nurbs]

Does MeGUIs avisynth script creator detect it as interlaced? Normally it doesn't detect interlaced or telecined sources as progressive, at worst I've seen it not being able do correctly detect the proper deinterlacing method on a field blended source, which is not surprising. If you change the hight before encoding you should have visible artefacts if you didn't deinterlace first.
Anyway that you can't determine the complexity of the source (from the encoders perspective) just by looking at it isn't exactly news.

[sneaker_ger]

That would be total guessing on my side as I don't own those films and I'm no expert in ripping NTSC DVDs. (+ I assumed that you had your source filters set up correctly.) Maybe someone else can help out with a satisfying answer.

[jmrenrav]

Does MeGUIs avisynth script creator detect it as interlaced? Normally it doesn't detect interlaced or telecined sources as progressive, at worst I've seen it not being able do correctly detect the proper deinterlacing method on a field blended source, which is not surprising. If you change the hight before encoding you should have visible artefacts if you didn't deinterlace first.
Anyway that you can't determine the complexity of the source (from the encoders perspective) just by looking at it isn't exactly news.

It will detect it as straight progressive, so I'm really at a loss. haha I'm just trying to understand the details of things a little better. It very well may not compress as well as others, but I guess im trying to figure out "why"

[jmrenrav]

Ok I was about ready to scrap this as my ignorance, but now I feel like I'm onto something :)

I found another movie that displays odd behavior, so I decided to run it through at work and compare it to the above:

Movie is Cheaper by the Dozen 2

Here is D2V

Stream Type: MPEG2 Program
Profile: main@main
Frame Size: 720x480
Display Size: [not specified]
Aspect Ratio: 16:9 [3]
Frame Rate: 29.970030 fps
Video Type: Film 99.56%
Frame Type: Interlaced
Coding Type: B
Colorimetry: BT.470-2 B,G*
Frame Structure: Frame
Field Order:
Coded Number: 135158
Playback Number: 2
Frame Repeats: 0
Field Repeats: 0
VOB ID: 1
Cell ID: 29
Bitrate:
Bitrate (Avg):
Bitrate (Max):
Audio Stream: 80: AC3 3/2 384
Timestamp: 1:33:51
Elapsed: 0:03:39
Remain: FINISH


Once loaded into Avisynth I press analyze source and low and behold it comes up as progressive. When I run it through x264, I encounter the bloated file size.

So I have:
Three Fugitives that will bloat if I do not setup avisynth correctly
Cheaper by the Dozen 2 that is bloating because avisynth is not detecting it correctly
And A Knight’s Tale (still trying to figure out) but would it be safe to say it has something to do with detecting the source?

[nurbs]

"Frame Type: Interlaced" doesn't mean that it is actually interlaced. For more information see the manual that comes with DGIndex.

Why don't you make it easy for all of us and post a small sample (30 - 60 seconds) from the middle of the movie. A scene with movement sideways, like someone walking through the picture, would be good.