IC filesizes again and what to check

[MackemX]

I reckon IC basically looks at the bitrates of the frames it analyzes and from this data collected creates a new average bitrate with a minimum and maximum bitrate. I don't think it does look at the actual image content of the frame, cos if it did then the sizes would be closer to being 100% accurate than the varied results people have shown at different reg analysis settings

it then goes through the movie frame by frame adjusting the frames to fit these new bitrates it has calculated just like CCE does

this is probably why even with 100% for the reg settings the resulting filesizes are also varied. It's just providing a more accurate original bitrate for IC to calculate new bitrates to work from

so a 10% analysis could in effect provide a higher/lower average bitrate than a 100% analysis depending on the frames selected during the 10% analysis and therefore produce the varying file sizes as people have experienced

when I look at the bitrates of both the original and IC's VOB's using bitrate viewer and check the sections of the lower bitrates of the two VOB's the patterns are very similar, nearly identical

It would probably be more accurate looking at the time of the title and size of the file and working out the average bitrate and reducing that to the slider % you choose and process the video to those new calculated figures. Why bother analyzing the video at all unless it does a 100% analysis of frame content

maybe it crashes on long dark scenes because the bitrate for that scene will be very low and maybe lower than what IC has set it's lowest bitrate to and it confuses itself, so maybe if IC set it's bitrate minimum to 0 it may fix the crashing

another point to note is that series DVD's are more accurate as they are mostly 4:3 formats throughout and have no black lines content to upset the resulting filesize

if you do a full analysis but also add 4% of the predicted size (Pinnacles buffer) you should get a better resulting filesize but this is for 4:3 formats, it's probably around 6% for anamorphic and even more for the letterbox format due to black bars within the image

IC sometimes comes across a mixture of screen formats during analysis so this will also upset the average bitrate and this is why you get such a variation of the resulting extra's titles yet the main movie size is always within 90-93% accurate for an average 1.85 anamorphic DVD and also why I have seen 94-96% accuracy for 4:3 series DVD's

so the screen format and black bars seem to have an effect on what IC will produce afterwards

If you select just one title and start IC it analysizes other titles to get the info if it cannot get enough from the one title you have selected!?!??! :p (though I never do that anyway, it's just to show what happens)

it does save individual title quality settings in the log file, yet they are always 0 :confused: after processing

IC needs to look at each title and keep that data collected for that individual title and not use it across the board or else they will never get accurate resulting file sizes :scared: (but even then titles can contain chapters that have black bars)

just a thought though and I'm probably talking crap like I normally do :D

[drmih]

I just wondered whether it is possible for a programme that re-encodes in a single pass to ever be able to produce an exact file size with a consistent picture quality. Certainly CCE can't and if you've ever used CCEGuesser you'll know that the results can vary by 200 to 300 Mb. I would think that Instantcopy always looks to be able to burn a disc and therefore errs (and often by quite a bit) on the safe side. Short of doing a check pass and the time penalty that goes with it, is there anything that can be done without ending up with a reduced picture quality like DVD2One or DVD95Copy. I must admit that I had a play for a few days with these programmes but have now stopped as the picture quality just doesn't compare to CCE. For speed but pretty good quality I've found that if you use the IfoUpdate Adjusted Cell Mode and one pass with CCE based upon CCEGuesser then I can do a disc quicker than Instant Copy (using CCE 2.5 with a dual Athalon motherboard I can achieve about 1.8 real-time). It does seem a pity that you can't get InstantCopy to do multi-passes as often you put a disc on to do overnight and you would be unaware of the extra time taken.

[MackemX]

LOL :D

the old CCE vs IC issue again, but I'll pass on that cos IC more than makes up for the quality difference in the cost/ease of use/options for me personally though not neccessarily for others so the choice is upto the user

It seems to do a full analysis if you set it to 100%, obviously if it displayed every frame during analysis this would further add to the time so it only shows every few secs worth. This does not improve the actual result by making it bigger as they have also included a 4% buffer but why don't they just do a full accurate analysis anyway and then they wouldn't need a buffer cos I've always found CCE quite accurate when after it's done the 1st pass and created the .vaf file (is that vaf as in video analysis file?)

I have posted all this on the Pinnacle board along with all the other issues asking for an explanation and also basically pleaded they do a full analysis of video content as some people ain't bothered about time taken processing

Pinnacle Support (http://webboard.pinnaclesys.com/read_messages.asp?WebboardID=1&ForumID=877&SectionID=170&ThreadID=135879&ThreadStart=0&Pos=0&cntThread=118&lng=1)

It would be nice if anyone is also interested in Pinnacle actually getting round to fixing this and other issues would also voice their opinion in the Pinnacle thread :)

[Rombaldi]

After doing a pile of DVD's I think I can (finally) understand what/how IC is doing it's analysis, and why it won't hit the file size exactly..

so forgive me if this is obvious...

it isn't analyzing every frame. It's only analyzing the 'I' Frames.
That's what's being displayed during the analysis process.

Analyze the 'I' Frame to calculate the new bitrates/compression.
Then when you transcode the 'I's would be dead on, the 'B'/'P'
will of course vary (depending on amount of repetative data).

The more 'static' the picture is (either in motion or actual picture)
the more compression in the 'B'/'P' frames and the farther off the
estimate is (this explains why very DARK or non-anamorphic LB streams are so far off, the same should be true for very bright streams or streams that do not have a lot of motion or scene changes).

The (obvious?) solution is a two-pass or two-stage analysis. It not only should get the size of the 'I' frames, and calculate the compression that will be obtained, but take into consideration the average 'B' and 'P' frame compression.

ie.

the way it's being done

for a GOP of 15 frames - (all numbers theortical)

it sees that an I frame takes 100 bytes.
it's set for a 60% reduction.
then the I frame should be reduced to 60 bytes.
it then figures that the other 14 frames will take 840 (14x60) bytes.

in reality, the 'B'/'P' frames may only be 60 bytes each (yes the 'B'/'P' frames are different sizes, but let's keep the math simple). So they would reduce to 36 bytes. Giveing a total for the
other 14 frames of 504 bytes.

The 'I' Frame only analysis is 900 (60+840) bytes
The actual 'I/B/P' compression is 564 bytes (60+504) bytes

in this example it would have missed it by 40%+

It need to take into consideration the (already) smaller size of the 'B/P' frames in it's calculation of the final size.

[mrbass]

DVDMaxAnalyzeRel set to 100% reads only each I frame? It must because that has no effect on final size outcome at least in my limited testing so far.

I still don't get why we need to do mutlipass with either IC or CCE because at these bitrates...let's say audio is 450MB (4.37GB minus .45GB = 3.92GB) so that leaves 3.92GB for video

3.92GB / 7200secs (2 hour movie) = 4.46Mbps which is pretty high for a quality CBR encode. CBR would enable accurate file prediction as opposed to single pass VBR.

[Rombaldi]

The reason is that CCE is (effectively) RECONSTUCTING EACH AND EVERY FRAME. EVERY frame is FULL SIZED and UNCOMPRESSED. When it does the re-encode EVERYTHING is re-encoded, what was an I-frame before could well be a B-Frame or P-Frame after CCE reprocesses it. Whereas the original stream could be

IBBBPBBBPBBBPIBBBPBBBPBBBPIBBBPBBBPBBPBBPBI

CCE would (effectively) see

IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII

and may well encode it..

IBBBPBPBIBBBPBBPBBPBBIBBBPBBPBBIBBBPBBBPBPI

notice how the I-Frames line up (and the GOPs)

I------------I------------I---------------I original
I-------I------------I---------I----------I CCE Encoded

every encoder will do things just a little different (this is why CCE streams can be a pain to remux with IFOEdit.

ReMpeg2 is (sorta) doing the same thing, but it takes and decomposes a single GOP back to it's component frames, re-compresses ONLY CONSIDERING THOSE FRAMES and puts it back. Since the I-FRAMES are in the same place in those GOPs, you can stick a M2V stream back in with IFO edit and everything will still work.

IBBBPBBBPBBBPIBBBPBBBPBBBPIBBBPBBBPBBPBBPBI original

IBBBBBBPBBBBPIBBBBBBPBPBBPIBPBBPBPBPBPBBBBI ReMpeg2

note the I frames are in the same place, but the distribution of the P/B frames are totally different.

IC/DVD2ONE/DVD95Copy is a totally different beast. It's like ReMpeg2 in that it maintains the original GOP structure, same number of frames per GOP, but unlike ReMpeg2, it keeps the same distribution and order of frames (IBBBPBBPBBPBB) and re-jiggers WITHIN the frame, no matter WHAT it is (I/B/P). It's just not taking into consideration the different sizes of an I frame to the B/P frames within the GOP (ie, it's assuming all the frames within a GOP are the same size) whereas (apparently) DVD2ONE and DVD95Copy DO.

[mrbass]

wow..awesome clarification..I think I understand about 95% or so.

anyway for those not versed in B frames, I frames, P frames here's a cut and paste from CCE manual.


In MPEG, a group of pictures is handled as a GOP (Group Of Pictures). The structure of GOP can be changed. Picture type --- In MPEG-2, three picture types, that is, an I picture, P picture and B picture, are defined.

I picture (Intra-coded picture)
An I picture holds all the picture information on one picture within itself. It is not necessary to refer to another picture to decode an I picture, but compression efficiency is not as good as for other types of pictures. Therefore, when the bitrate is the same, the picture quality of a stream of I pictures is lower than that of other types of streams. To edit encoded streams, however, it is more convenient to use numerous I pictures.

P picture (Predictive-coded picture)
A P picture consists of motion vectors when previous (just before) I pictures or P pictures are used for reference and differential information between a picture comprised of these motion vectors and an original picture. To decode a P picture, pictures used for reference are required, but compression can be more efficient than I pictures. In the case of a sequence where P pictures continue, however, picture quality may deteriorate as the P picture deviates from the reference I picture, since errors accumulate during decoding due to the difference in the IDCT (Inverse Discrete Cosine Transform) calculation method between the encoder and the decoder.

B picture (bi-directionally predictive-coded picture)
A B picture consists of motion vectors when previous (just before) I pictures or P pictures and/or future (just after) I pictures or P pictures are used for reference and differential information between a picture comprised of these motion vectors and an original picture. Therefore, to decode B pictures, two pictures used for reference are required, but compression efficiency is even better than P pictures. Since a B picture itself is not used for reference, errors do not accumulate even if B pictures continues, unlike the case of P pictures. However, if B pictures continue, the distance from a reference picture increases, and as a result, the motion compensation effect may decrease.

[Rombaldi]

thanks for the cut/paste.. I was too lazy..

the best way to test this theory is to use CCE (or something) to encode a MPEG stream of NOTHING but I-Frames. (the quality will suck, but that's not important) Then author a DVD-Structure and feed it to IC. If the resultant size is dead on... Q.E.D.

[red321]

Rombaldi,

Given that only the I frames matter for Ifoedit to remux te m2v, and CCE will take a file that sets the positions of chapters, forcing I frames, is there not scope to analyse any stream, as Rempeg does, and produce a file structure, and then use that to force CCE to produce a stream suitable for remuxing into the Vob ?

Most (?) video streams have a regular IBP pattern with occasional variations for scene changes, so perhaps this is worth a try ?

[Rombaldi]

red321..

you can give CCE a list of 'forced' I-Frames to perserve chapters, I thought that (somewhere down the line) something like that was planned for IFOedit, but since devlopment has (apparently) stopped for the forseeable future..

and you're operative word is 'most' streams... but most isn't all...

[MackemX]

great info Rombaldi, took me ages to get my head round it :) and this would definitely explain the variation in sizes

I'm under the impression it's down to the black content (dark scenes/bars) even more now and that's what's causing the major undersizing problem

I just done a 4:3 DVD series DVD with nothing but 4:3 formats throughout. It was Fawlty Towers which is not dark at all, and has no black bars anywhere

I had a 1% setting for save quality, and the 5% for MaxRel and 25MB for MinAbs

set it to 4.42Gb and it came out 4.38Gb or 4487MB :), only 2MB to spare!

1% of either 4.38Gb or 4.42Gb (not sure which it is tho!) is about 0.044Gb

takes this away from 4.42 and it's 4.38ish

yet I bet if it was a 1.85:1 format with the same type of content it would be affected by the black bars and probably have resulted in 4.16Gb which is what I have experienced just recently with 2 1.85:1 DVD's

most of the 1.85 DVD's I done since using the lateset version and 1% seem to be around 93-95% accurate

obviously having 1.85:1 movie but then having a mixture of both 4:3 & 1.85 menu/extras will affect the outcome by different amounts depending on content

this may explain why some extras titles are more accurate than others, but even then titles can be a mixture as the Star Wars AOTC disc 2 is, so cannot even be compensated for accurately

basically I'm gonna add 5% for the amount of predicted 1.85:1 anamorphic title's content and 1% for any 4:3 content and probably end up getting better results

letterbox is probably another another amount of error

I'm sure if I now do the Simpsons DVD's with settings of 4.42Gb'ish I will get around 4.37Gb due to the 4:3 content. It may need a little more due to black screen the black end credits as these will contribute to undersizing and I also see that it has two thin black lines either side from top to bottom that will also have an effect

[Arianos]

I just done a 4:3 DVD series DVD with nothing but 4:3 formats throughout. It was Fawlty Towers which is not dark at all, and has no black bars anywhere.
I had a 1% setting for save quality, and the 5% for MaxRel and 25MB for MinAbs
set it to 4.42Gb and it came out 4.38Gb or 4487MB , only 2MB to spare!

I followed the settings from the above posting religiously, for Snow Queen (Greek only edition), total 7.81Gb of 4:3 material.
I set the size to 4.40 (OK, so I was a little chicken...)
Came out 4,35 GB (4.681.433.088 bytes) and it's looking pretty darn good, even on dark scenes.
On the other hand, before that, DVD2ONE gave me the old "flickering effect" as the movie alone is over 6.5GBs, and shrinks down to about 3.5GB with both of the programs.
Now, Mister X. First: THANK YOU.
Second: that is a great rule of thumb, and I have saved the settings for future use. If only all your beautiful suggestions were that simple and fast to use :) :) :) for us common folks.
Now, don't get me wrong: What may seem simple to you, may not be that simple to others. Having spend some time teaching, I know how frustrating trying to get your point can sometimes be.
Keep it simple. Please.
Thanks again.