Transcoders and PSNR Values

[quantum]

REMOVED - Originally I had posted PSNR results here.
Further testing and visual frame comparisons
have demonstrated to me these numbers have little
relationship with obvious visual quality - results
removed since they seem useless and in fact can give
higher scores to bad frames and weaker transcoders.
I'll stick to my original suggestion for Comparing the Transcoders (http://forum.doom9.org/showthread.php?s=&threadid=58808)

[Richk50]

I'm confused. Isn't it that the larger the deviation, the worse the quality?

[dragongodz]

while PSNR is generally a reasonable test it is not absolute. yes a high average is good but the large variation can indeed produce extremely good frames but also extremely bad. in such cases only viewing yourself will really show how much effect the variation has had.

[chipvideo]

Well I guess that since for me IC always gives me a final output of 3.6 gigs I guess I go for the dvd20ne method then. With the new release of dvdshrink does the no compression method give original quality of the movie or does it still degrade the picture in reauthor mode? I was thinking of using dvdshrink to cut the end credits off and then use dvd20ne to compress.

[JFerguson]

chipvideo,

I have found (and reported) that, in full disc mode, DVD Shrink v3b5 will degrade the beginning of a video segment if it is set to Automatic w/ 100% video quality.

If set to No Compression, it doesn't touch it. Maybe this will work for Re-Author mode...

[quantum]

I'm beginning to think the PSNR values for Recode and Shrink are skewed. The PSNR is a score assigned to each frame where 103.5 is an exact duplicate of the original and lower numbers are more and more different from the original.

Good looking frames can be about 45, while a blocky frame may be in the 30's or high 20's. So a transcoder can make a bad frame, followed by a duplicate frame. The duplicate frame will get a 103.5. The bad frame may get a 30. So the average is in the 60's. This is getting a good score by taking advantage of a weakness in PSNR that gives a disproportionately high score to duplicate frames. I would like to think Recode and Shrink did not deliberately do this just to spike their PSNR scores.

I'm looking at redoing these tests with a different PSNR filter that may address this issue.

[JFerguson]

Interested in this technique...having some issues though. I posted these same questions/comments in the "Comparing the Transcoders" thread. I'll try here, too:


Need more information on this PSNR analysis...

You use DVD2AVI to create .D2V file?

Then, what does the .AVS script look like for VirtualDub??

A little confused here...

[quantum]

I no longer have any confidence these tests have any value, but..

Yes, you first create a DVD2Avi project from your Vobs. This is a sample AVS:

clip1=Mpeg2Source("new.d2v").trim(1000,50000)
clip2=Mpeg2Source("orig.d2v").trim(1000,50000)
Compare(clip1, clip2, "YUV", "out.log", show_graph=false)

Load this into Virtualdub and press F5. Out.log will be created.

[JFerguson]

Thanks, quantum.

My script looked similar to yours, but VirtualDub did not like (recognize) the Mpeg2Source function. Must be something wrong with my setup...

[deXtoRious]

Did you load the plugin?

[JFerguson]

Originally posted by deXtoRious
Did you load the plugin?

Probably not. I downloaded AVISynth and I think it dumped itself somewhere in Program Files\ but...something needs to sit in the plugins\ directory under the VirtualDub root, I imagine?

[DVDRFreak]

Originally posted by JFerguson
Probably not. I downloaded AVISynth and I think it dumped itself somewhere in Program Files\ but...something needs to sit in the plugins\ directory under the VirtualDub root, I imagine?

You need the MPEG2DEC3.DLL also you need to convert to YUY2 else virtual dub will complain it cannot find a decoder for YUV source.

Full script:
LoadPlugin("C:\Program Files\DoItFast4U\new.avs\mpeg2dec3.dll")
clip1=ConvertToYUY2(Mpeg2Source("lotr_nero.d2v").trim(1000,50000))
clip2=ConvertToYUY2(Mpeg2Source("lotr.d2v").trim(1000,50000))
Compare(clip1, clip2, "YUV", "out.log", show_graph=false)

[DVDRFreak]

While playing around with this comparison function I noticed that there is a pattern in the transcoded video (Recode/Shrink).

It looks like this
(Shrink/Recode):
Frame 1 103 (not compressed)
Frame 2 45 (compressed)
Frame 3 43 (compressed)
Frame 4 103 (not compressed)
Frame 5 44 (compressed)
Frame 6 47 (compressed)
Frame 7 103 (not compressed)

(CCE)
Frame 1 45 (compressed)
Frame 2 47 (compressed)
Frame 3 45 (compressed)
Frame 4 50 (compressed)
Frame 5 46 (compressed)
Frame 6 49 (compressed)
Frame 7 51 (compressed)

So every 4th frame is not compressed. Can someone explain why this is done. I could imagine that compressing all frames would result in a more even quality.

Is this a bug?

[quantum]

I can't think of any reason to sprinkle these uncompressed frames around other than to spike the PSNR scores. I didn't experience uncompressed frames with the frequency you're describing.

[DVDRFreak]

Originally posted by quantum
I can't think of any reason to sprinkle these uncompressed frames around other than to spike the PSNR scores. I didn't experience uncompressed frames with the frequency you're describing.

I had a movie with a low compression value (18% or so) maybe that is why I see them more often.
What I think is strange is that it is like a pattern 2 frames compressed one frame uncompressed. So it looks like it is done on purpose for some reason.

Edit:
This would maybe also explain the pumping pixels effect I see in some movies. It looks like the image is sharp then not sharp then not.

[luphy]

Quantum,

Since you seem to have done some very intensive comparisons of the
different transcoders, I'd love to hear your personal opinion on which one gives the better video quality for your eyes, using your system.
Recode vs Shrink3 (deep analysis) vs DVD2One vs IC. I think everyone agrees IC probably gives the best quality - I'd love to hear how close you think the other three come to IC.
Thanks.

[quantum]

In my view:

Recode and Shrink are about the same in quality, so I'll just mention Recode. I tend to use Recode because it's a bit faster and can use a custom bitmap for disabled titles, saving some space.

IC7/8 does not necessarily give a better quality than Recode, but it does seem to have better bitrate distribution. The average frames in Recode are usually better than IC7. The bad frames in Recode are usually worse than IC7's bad frames. So you may see compression artifacts on a Recode backup that you wouldn't see with IC7.

So which one is better? I'm not sure, but with the IC7 chronic resizing problem (not fixed in IC8) and the silly PDI file output, and the fact recode is much faster, I don't use IC7 for anything anymore. However with that said, I seriously wish Recode/Shrink had a more even bitrate distribution as IC7 does.

DVD2One is not in the running with the other tools unless you want movie only backups.

I plan on doing another attempt at a PSNR test on Two Towers. I'll post results when I get them.

[int 21h]

Study the idea of compressed domain transcoding and you'll understand better why some frames are left at their original compression values...

I can think of a few reasons...


Speed, 1/4 less frames to transcode.
Technically the quality could be interpeted as better since you've had to recompress less frames. Everytime you recompress you're losing a good portion more of the information you had when you started. (Study the idea of quantization, even a simple look at Uniform Quantization will give you the reason why)
This is how these programs do size calculations (through manipulation of frame quantization) By changing every 4th frame, they can lower size by X%, changing every 5th frame lowers size by X%. This is probably the correct answer, as I recall, the original algorithm of DVDShrink compressed only certain frame types in regards to the level settings. (The new algorithm is different though, but I would not be suprised if it implemented something similar). Try using different compression levels to find out.

[Tonio Roffo]

I believe in old docs that came with earlier versions of shrink, there was a mention that specific compression levels (back then) only affected one or two types of frames, not all MPEG frames. that also explains the pulsating.

Probably the uncompressed frames are from a certain type (I,B or P) and the compressed frames not.

Some frames are "calculated" based on former frames, so uncompressed frames could increase quality of following frames that are compressed.

Just my 2 cents...

[duartix]

DVDShrink started out by compressing just B frames.
After it reached the highest level of compression of B frames it would start compressing P frames too. Only after the highest level compression of P frames would it go to I frames. It guess it would be something like this:

Level - I P B
0 ----- 0 0 0
1 ----- 0 0 1
2 ----- 0 0 2
3 ----- 0 1 1
4 ----- 0 1 2
6 ----- 0 2 2
7 ----- 1 1 1
8 ----- 1 1 2
9 ----- 1 2 2
10 ---- 2 2 2


The pattern you show is tipically Level 1 or 2 (Just B-Frames).